diff --git a/.gitattributes b/.gitattributes index c7e0c4779df108cca06ce19a3019c16992a5df0d..86a861a820f7108ce39f6eb66320bb5e8b9e3a06 100644 --- a/.gitattributes +++ b/.gitattributes @@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text *tfevents* filter=lfs diff=lfs merge=lfs -text git.diff filter=lfs diff=lfs merge=lfs -text replay.mp4 filter=lfs diff=lfs merge=lfs -text +sf_log.txt filter=lfs diff=lfs merge=lfs -text diff --git a/.summary/0/events.out.tfevents.1700548189.rhmmedcatt-proliant-ml350-gen10 b/.summary/0/events.out.tfevents.1700548189.rhmmedcatt-proliant-ml350-gen10 new file mode 100644 index 0000000000000000000000000000000000000000..625d6d7e6d62c6f2d59a6f9c650fbed85d04ef2e --- /dev/null +++ b/.summary/0/events.out.tfevents.1700548189.rhmmedcatt-proliant-ml350-gen10 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e72cab06f5d7c8398dc3e471abfb4f85262dbcb2bb67dbd6316a28a6b0f6f98d +size 94086055 diff --git a/.summary/1/events.out.tfevents.1700548189.rhmmedcatt-proliant-ml350-gen10 b/.summary/1/events.out.tfevents.1700548189.rhmmedcatt-proliant-ml350-gen10 new file mode 100644 index 0000000000000000000000000000000000000000..68816a481cbaa792d66341ec9978c6d820beb314 --- /dev/null +++ b/.summary/1/events.out.tfevents.1700548189.rhmmedcatt-proliant-ml350-gen10 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:463554fb7a493ddf0288f93aa36ca9c4de99841f2fb0b845a92bef14956a409e +size 49427605 diff --git a/README.md b/README.md index c7f7a59936de34609c980c40ec97cb106c02a176..f5069cff5d009b60826cc236b4f629e490ac1b55 100644 --- a/README.md +++ b/README.md @@ -15,35 +15,39 @@ model-index: type: atari_riverraid metrics: - type: mean_reward - value: 8855.00 +/- 577.95 + value: 52696.00 +/- 3378.58 name: mean_reward verified: false --- -A(n) **APPO** model trained on the **atari_riverraid** environment. +## About the Project -This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. -Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/ +This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). +In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this. -## Downloading the model +## Project Aims -After installing Sample-Factory, download the model with: -``` -python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_riverraid -``` +This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. - -## About the Model +I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. + +The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. -This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. +After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234') -The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. -The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his. -I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes: +## About the Model + +The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency: ``` hyperparameters = { + "help": false, + "algo": "APPO", + "env": "atari_asteroid", + "experiment": "atari_asteroid_APPO", + "train_dir": "./train_atari", + "restart_behavior": "restart", "device": "gpu", "seed": 1234, "num_policies": 2, @@ -141,12 +145,28 @@ hyperparameters = { "env_gpu_observations": true, "env_frameskip": 4, "env_framestack": 4, - } + "pixel_format": "CHW" +} ``` +A(n) **APPO** model trained on the **atari_riverraid** environment. + +This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a +high throughput on-policy RL framework. I have been using +Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/ + + +## Downloading the model + +After installing Sample-Factory, download the model with: +``` +python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_riverraid +``` + + ## Using the model To run the model after download, use the `enjoy` script corresponding to this environment: diff --git a/checkpoint_p0/best_001864192_477233152_reward_85.330.pth b/checkpoint_p0/best_001864192_477233152_reward_85.330.pth new file mode 100644 index 0000000000000000000000000000000000000000..836e14abf87adcc94043ebfc6c7c471f533be47a --- /dev/null +++ b/checkpoint_p0/best_001864192_477233152_reward_85.330.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4485448b1a5308f68a76e512ad60b1e5654a0c9b65c3fc7e4abdfb781ec6ea52 +size 20795763 diff --git a/checkpoint_p0/checkpoint_001958784_502906880.pth b/checkpoint_p0/checkpoint_001958784_502906880.pth new file mode 100644 index 0000000000000000000000000000000000000000..1054175ddea7ad91c605eb9aa8ac6eb82852c27a --- /dev/null +++ b/checkpoint_p0/checkpoint_001958784_502906880.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fff7997c09b6c75366cadd03249081f77c6c4c1e8da4bced153e1f1db5d5a634 +size 20796099 diff --git a/checkpoint_p0/checkpoint_001959392_503218176.pth b/checkpoint_p0/checkpoint_001959392_503218176.pth new file mode 100644 index 0000000000000000000000000000000000000000..f21eaec8ff71c3b7f60f21a91785eed8001e149a --- /dev/null +++ b/checkpoint_p0/checkpoint_001959392_503218176.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e76413845bbce6fadb76344fa9ad07635050ced44519afb00d15e19797d7f92a +size 20796099 diff --git a/checkpoint_p0/milestones/checkpoint_000012000_3072000.pth b/checkpoint_p0/milestones/checkpoint_000012000_3072000.pth new file mode 100644 index 0000000000000000000000000000000000000000..9a0ea16fb33ea9f8a9d191100dafb7ebe278fab7 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000012000_3072000.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1abe2a1afbfdfd09a2bb2904713a8c8e6f61ec6726005f2ce04a5a3022b6ad4 +size 20796955 diff --git a/checkpoint_p0/milestones/checkpoint_000024256_6209536.pth b/checkpoint_p0/milestones/checkpoint_000024256_6209536.pth new file mode 100644 index 0000000000000000000000000000000000000000..c2bd8449a6513834d5d758ec000b4ffeecc29fe1 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000024256_6209536.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3fbe252db6c521171dbadf6560c7e35b314a170d6b095cc6b327d1224c327b89 +size 20796955 diff --git a/checkpoint_p0/milestones/checkpoint_000036480_9338880.pth b/checkpoint_p0/milestones/checkpoint_000036480_9338880.pth new file mode 100644 index 0000000000000000000000000000000000000000..ab1cb8b5fa328f989dcb7a947665c757aba5854e --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000036480_9338880.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19ef78042aff31e395bbb94dcb078e7ede07755c2383894cd8a73df524b10a67 +size 20796955 diff --git a/checkpoint_p0/milestones/checkpoint_000048768_12484608.pth b/checkpoint_p0/milestones/checkpoint_000048768_12484608.pth new file mode 100644 index 0000000000000000000000000000000000000000..380af6dff236d9689a40d727697a34b2460347e1 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000048768_12484608.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:650198cd88da818d8ef0293848571689ad5bfc2856e099e39c9906b144e7351e +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000060864_15581184.pth b/checkpoint_p0/milestones/checkpoint_000060864_15581184.pth new file mode 100644 index 0000000000000000000000000000000000000000..1b46e458c1e5f06e2a71536aa7e4f600d21501f2 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000060864_15581184.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:50548337ac536b129360727a833cfff2cce81a00fd21ab7d6cef85ce457af983 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000073088_18710528.pth b/checkpoint_p0/milestones/checkpoint_000073088_18710528.pth new file mode 100644 index 0000000000000000000000000000000000000000..6aafd2f29b9d022b793f3e15babd867c84c7e057 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000073088_18710528.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6fd9d68767d9a67892bab7aa18b100782bd272d13dd0986bc9a48b6411e9644 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000085312_21839872.pth b/checkpoint_p0/milestones/checkpoint_000085312_21839872.pth new file mode 100644 index 0000000000000000000000000000000000000000..da243581ff0d6f52c42de27b4c5ede215cb6aed7 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000085312_21839872.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1e3acba8a8288ffa88146f4b54a700591fe1c3ba868d7453c21727a37354778d +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000097600_24985600.pth b/checkpoint_p0/milestones/checkpoint_000097600_24985600.pth new file mode 100644 index 0000000000000000000000000000000000000000..902389f52f3bf22bec0c69f3169346b620eac61b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000097600_24985600.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:202bc691ca41712795ebc6bba6e7a91930c48b70b04fea0d7b3b113f4ea0dbab +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000109824_28114944.pth b/checkpoint_p0/milestones/checkpoint_000109824_28114944.pth new file mode 100644 index 0000000000000000000000000000000000000000..299904abf1ac1b78174774e632768142099d9147 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000109824_28114944.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6857c9468540bc08e1cc06e5811d00052b4aa6373920e8c3de71bc9111e0bf82 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000122112_31260672.pth b/checkpoint_p0/milestones/checkpoint_000122112_31260672.pth new file mode 100644 index 0000000000000000000000000000000000000000..e4cf892c2b6774717fe00e2579711cafe63f8ae6 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000122112_31260672.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:084b9883bb0a35a67bac16a5a7a2297939d5a205ba2c22c492e4a972fedd6c0c +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000134400_34406400.pth b/checkpoint_p0/milestones/checkpoint_000134400_34406400.pth new file mode 100644 index 0000000000000000000000000000000000000000..725e2c4a1d14fa58848028916036c8497890ce40 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000134400_34406400.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b015636364ebc60ff3e77c11bb991d8402983bbca7e0a93c9f37fec15cdea715 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000146624_37535744.pth b/checkpoint_p0/milestones/checkpoint_000146624_37535744.pth new file mode 100644 index 0000000000000000000000000000000000000000..5acbab3eb1913cd597826d206f613c619bab7668 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000146624_37535744.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dbde70ce05ce40004b5696fbb7632b4bf4e33782858e4ba2b6648aaf61a3a9dd +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000158944_40689664.pth b/checkpoint_p0/milestones/checkpoint_000158944_40689664.pth new file mode 100644 index 0000000000000000000000000000000000000000..9c81f67262ef734ab00ecae09899f6690119bb9a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000158944_40689664.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:94dbc5af3139b16f42a1d8493c99eacbc25194270aad27eb12a5fc7836f91bf8 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000171264_43843584.pth b/checkpoint_p0/milestones/checkpoint_000171264_43843584.pth new file mode 100644 index 0000000000000000000000000000000000000000..a17952632ff0562bbc889aba1fd78d9aef3bcb54 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000171264_43843584.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:88d2109a2bd33e912aa2ef0a927fb1d65d32c79f8f8db38e5bd41e278fe633bf +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000183488_46972928.pth b/checkpoint_p0/milestones/checkpoint_000183488_46972928.pth new file mode 100644 index 0000000000000000000000000000000000000000..46b3d3a78c4f1ecc1c8f814cbe4394a4c3a709cf --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000183488_46972928.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1624dd5a55c4767b6ee35ddbe279dc4725d23e58bc157aed5d1184ee4f13ece3 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000195808_50126848.pth b/checkpoint_p0/milestones/checkpoint_000195808_50126848.pth new file mode 100644 index 0000000000000000000000000000000000000000..4d32234cc211a83b9824409c41720f5ed56c7973 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000195808_50126848.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:91ee8ddde9cf83d701b9ae77c073ee2c879947eec49cb770eaaa76a1be0981b0 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000208096_53272576.pth b/checkpoint_p0/milestones/checkpoint_000208096_53272576.pth new file mode 100644 index 0000000000000000000000000000000000000000..f38296237b0f3ae8a9e60c64e7c42a4334bca17a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000208096_53272576.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e83bdc85ac6428f123bd2d6c25ae6c6f73de0bc60e1a1cdb01073d1cdea211f +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000220384_56418304.pth b/checkpoint_p0/milestones/checkpoint_000220384_56418304.pth new file mode 100644 index 0000000000000000000000000000000000000000..a4977979bd38e9baa56ba5a63fc2c33036515dc0 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000220384_56418304.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:650077840031341b177e48b32b20fa6c5fa0facc848420fee729728e9583c081 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000232704_59572224.pth b/checkpoint_p0/milestones/checkpoint_000232704_59572224.pth new file mode 100644 index 0000000000000000000000000000000000000000..5c49ea17862a985900619f4746ef72b2c781571c --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000232704_59572224.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:24cd5414aae5583c11422de79bf41bf73aba87054b05af0538806435aaca5580 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000244928_62701568.pth b/checkpoint_p0/milestones/checkpoint_000244928_62701568.pth new file mode 100644 index 0000000000000000000000000000000000000000..d16b551c5d3a9df255e088d00000117527fc8130 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000244928_62701568.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce765e858f606267a302fd1bd6e7697dbac83f31fa66bf776c65e0d8babeb461 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000257280_65863680.pth b/checkpoint_p0/milestones/checkpoint_000257280_65863680.pth new file mode 100644 index 0000000000000000000000000000000000000000..476595b1cd0b95f371efc9d21d24db2766400eae --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000257280_65863680.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:556afe03e444dba03a2808f19f10baf93db2cd6d16c4c4f0214c4dcbad4b088c +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000269568_69009408.pth b/checkpoint_p0/milestones/checkpoint_000269568_69009408.pth new file mode 100644 index 0000000000000000000000000000000000000000..f5e11861cf1409dcba1de9a3271e36851edc5a68 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000269568_69009408.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cfb747082a13542653203b1f8a3c924a83ec4bd4b369ee4e1a1f118127c65624 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000281824_72146944.pth b/checkpoint_p0/milestones/checkpoint_000281824_72146944.pth new file mode 100644 index 0000000000000000000000000000000000000000..a359eeef1f33f71ec8feb247f81f164e077a2df3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000281824_72146944.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f6a8b8116e9bbe1b0dd21b370444b109b8dbdd1872b52a7ec051ac146282a5d +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000294144_75300864.pth b/checkpoint_p0/milestones/checkpoint_000294144_75300864.pth new file mode 100644 index 0000000000000000000000000000000000000000..69cf4ac2c84a7809fabefc651e49ff205bb6f416 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000294144_75300864.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b17a39dbfbd15ba7c4b68fc8d9d7b7e4de2185b977e70ad8ba103c3cdd83613 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000306432_78446592.pth b/checkpoint_p0/milestones/checkpoint_000306432_78446592.pth new file mode 100644 index 0000000000000000000000000000000000000000..620fc458578899e039f9832def53d80f7504da23 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000306432_78446592.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:33f9f92deb66b0dce7a2851c2a2eb1b843b136e4b4d9e8f84ff1107d47e05363 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000318720_81592320.pth b/checkpoint_p0/milestones/checkpoint_000318720_81592320.pth new file mode 100644 index 0000000000000000000000000000000000000000..fbe9a331d2c36eaa6e3a2fa3156a001adf105fbe --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000318720_81592320.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:db0dae8f7b4ae3c7b9b20a043b5605c5f62f9878ef1b8e890a7f2e6a704cd5f3 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000331008_84738048.pth b/checkpoint_p0/milestones/checkpoint_000331008_84738048.pth new file mode 100644 index 0000000000000000000000000000000000000000..bb05252d40ced4f99aff351b458e9ad95c582383 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000331008_84738048.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:abedf384a2d87f168d8cb4030f12cfe0c985cb63638e44a0bcbafe42ff2ba9ea +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000343264_87875584.pth b/checkpoint_p0/milestones/checkpoint_000343264_87875584.pth new file mode 100644 index 0000000000000000000000000000000000000000..c46e032a7d33e4da3d2d7ac84190104f4f2500bf --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000343264_87875584.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:06355f1b0b012d4dd4ff9dc187abc2c0216540770b66a1a361a2decb667d0136 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000355552_91021312.pth b/checkpoint_p0/milestones/checkpoint_000355552_91021312.pth new file mode 100644 index 0000000000000000000000000000000000000000..97272f1b1de9cf3fa9813c77659daf3e75f1eaba --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000355552_91021312.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1739124960fad362fd3a9d7d45a39fd6c0e2862d506243070702044747aafaf +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000367904_94183424.pth b/checkpoint_p0/milestones/checkpoint_000367904_94183424.pth new file mode 100644 index 0000000000000000000000000000000000000000..6065e5029409d9bb1e230f1821fa2a5937948c83 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000367904_94183424.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da6fcc98a6adc90ffb84ed0693c15391f6b17b63500b07d50760f47656fd341e +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000380192_97329152.pth b/checkpoint_p0/milestones/checkpoint_000380192_97329152.pth new file mode 100644 index 0000000000000000000000000000000000000000..5edd6b6d10f1562668cd9be1839f7a0e0e748a5e --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000380192_97329152.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b4225b53ec38fae9a63a9105f44a64c5d0d669bac0a688b1021c1e6e72e10e34 +size 20797011 diff --git a/checkpoint_p0/milestones/checkpoint_000392512_100483072.pth b/checkpoint_p0/milestones/checkpoint_000392512_100483072.pth new file mode 100644 index 0000000000000000000000000000000000000000..25f01a5fff9afe17a2568f0ce5216d80fd841d7d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000392512_100483072.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b0648e3c0e980ecdcce12d7e95a9f30a387daa2821f7ea362f5ee09d20e61816 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000404800_103628800.pth b/checkpoint_p0/milestones/checkpoint_000404800_103628800.pth new file mode 100644 index 0000000000000000000000000000000000000000..718b211b6d2ecf83ef330f8045adf5e943110dbd --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000404800_103628800.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:022e5bda879c420632acdfa8ffbc89067c446118f702d9f7f08b0e0574f4ccb3 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000417056_106766336.pth b/checkpoint_p0/milestones/checkpoint_000417056_106766336.pth new file mode 100644 index 0000000000000000000000000000000000000000..888c11496b5943553cc75a882b8756554a35982a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000417056_106766336.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:86803d83c83bfd5a70d51a2421759b925aa6b85c5a5278ce03efcf371e8bcc97 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000429408_109928448.pth b/checkpoint_p0/milestones/checkpoint_000429408_109928448.pth new file mode 100644 index 0000000000000000000000000000000000000000..2c1429d2c0bd744614108b13fd5c5176dbcd256c --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000429408_109928448.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6317c3ef8dd4abbf01cd51cb0c2d845b21986252cfa97558c9b34eede7eb98c7 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000441696_113074176.pth b/checkpoint_p0/milestones/checkpoint_000441696_113074176.pth new file mode 100644 index 0000000000000000000000000000000000000000..d7bf0b9f454a4b66b80512953f24c21ccf8af73b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000441696_113074176.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:108626ca04c5f3d0b2af603bb7049b770732827b1408ec606e5f954b3e14a1c8 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000453984_116219904.pth b/checkpoint_p0/milestones/checkpoint_000453984_116219904.pth new file mode 100644 index 0000000000000000000000000000000000000000..e0a740ce883f64c4c2d54fb1b093aa9008c93cb4 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000453984_116219904.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8562a0b3366155bc220c88c371c9c07acf7dca1fc697b2247a4eb665e06475f8 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000466240_119357440.pth b/checkpoint_p0/milestones/checkpoint_000466240_119357440.pth new file mode 100644 index 0000000000000000000000000000000000000000..d5a3e80d0c7079e34bf745712a54aa66c7ad9b5d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000466240_119357440.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7b41fe3769e3ce14804536ef17e31d75704257f49991bd2e014d1951755842e0 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000478592_122519552.pth b/checkpoint_p0/milestones/checkpoint_000478592_122519552.pth new file mode 100644 index 0000000000000000000000000000000000000000..3c8a51232626858b04e7c03cc3ff014605f8b8ba --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000478592_122519552.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c93cee50b0b2c222c12e7e4a8e7d3664458a1e61ebe63a0fef53bc7b506bdffb +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000490912_125673472.pth b/checkpoint_p0/milestones/checkpoint_000490912_125673472.pth new file mode 100644 index 0000000000000000000000000000000000000000..55da6054cac0cf75dc75b686989569ce762c80ab --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000490912_125673472.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a7a993ab358bb47b9e15db9afb84da980146c16449d97bc27342550fcfcf9395 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000503264_128835584.pth b/checkpoint_p0/milestones/checkpoint_000503264_128835584.pth new file mode 100644 index 0000000000000000000000000000000000000000..9788ec8a474ef208e2f9520a9e7deaaadfedaa4a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000503264_128835584.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ebb64401478897d6dfe06d89e6be3379987c55d843d15d1d7d3f1c3440b6af55 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000515552_131981312.pth b/checkpoint_p0/milestones/checkpoint_000515552_131981312.pth new file mode 100644 index 0000000000000000000000000000000000000000..134df8a4e31c4adb8dee2371d32e762233883b21 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000515552_131981312.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af867e058610134ce75faa4769c7c9368991f873e0c5a905bf91e10eb1c47f97 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000527840_135127040.pth b/checkpoint_p0/milestones/checkpoint_000527840_135127040.pth new file mode 100644 index 0000000000000000000000000000000000000000..e1ed68995bb1f9fee5503586d37a179546c35140 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000527840_135127040.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f38bfbde850cc2561740a7f036f5a8e145bab12315cc375f4055660505f820e3 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000540128_138272768.pth b/checkpoint_p0/milestones/checkpoint_000540128_138272768.pth new file mode 100644 index 0000000000000000000000000000000000000000..0d4c25ca94eabf86d99a36256753457e4e6bd551 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000540128_138272768.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:30a88f76702a4308a09cf0d64facc1f9927c185e191b830d321541eadf40ea40 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000552416_141418496.pth b/checkpoint_p0/milestones/checkpoint_000552416_141418496.pth new file mode 100644 index 0000000000000000000000000000000000000000..1e21e1e7dc0c3b37c050ef4ba10c541bd0d443be --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000552416_141418496.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce9b83b37b2f1b8fd03acbd92e648cb354781ce5c473e2d1a252c3560eddc6ff +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000564704_144564224.pth b/checkpoint_p0/milestones/checkpoint_000564704_144564224.pth new file mode 100644 index 0000000000000000000000000000000000000000..c0bb47ee826e8851ef8f6d8141ded8b06aecbbaf --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000564704_144564224.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4de3cd3d09e6ad043de02a8e53fc7523dd1c075a1913b6fd1a595d9871608c7 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000576992_147709952.pth b/checkpoint_p0/milestones/checkpoint_000576992_147709952.pth new file mode 100644 index 0000000000000000000000000000000000000000..4fd35f5bfe1037c5a2420b3c702c529e84fe2f51 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000576992_147709952.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aa593db1407c3f3c6550231440551d9c1ef33265e12eac883f1c14e63bbb614a +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000589280_150855680.pth b/checkpoint_p0/milestones/checkpoint_000589280_150855680.pth new file mode 100644 index 0000000000000000000000000000000000000000..c6ddae7df58547a723487fe3e7ce3505e05f9708 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000589280_150855680.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b8852c51b42b8dec5a6f010b7673696883237e3dbc9a41dd13d960873f6fc95e +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000601600_154009600.pth b/checkpoint_p0/milestones/checkpoint_000601600_154009600.pth new file mode 100644 index 0000000000000000000000000000000000000000..524924721be0af4650f0c203e89c5a7a75bbde99 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000601600_154009600.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8e970ea474852e64313510919b806e886223a3b091950d3c0864cccfb0658f95 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000613856_157147136.pth b/checkpoint_p0/milestones/checkpoint_000613856_157147136.pth new file mode 100644 index 0000000000000000000000000000000000000000..94ec60e0bfbd373f4ab9d9057fb07663226e90ad --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000613856_157147136.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9b72a25c72c69617affc1cd6780ec2ec130212b1d7c15f618f7236ce77342331 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000626144_160292864.pth b/checkpoint_p0/milestones/checkpoint_000626144_160292864.pth new file mode 100644 index 0000000000000000000000000000000000000000..4ab87cdf18b9e84c11a1486cb34ac4479408cefc --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000626144_160292864.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9b2967a3746950adc569306ff5bc2a6a88a43522aa2b9403a052e8e62b306136 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000638400_163430400.pth b/checkpoint_p0/milestones/checkpoint_000638400_163430400.pth new file mode 100644 index 0000000000000000000000000000000000000000..f0d83c344346cb8cb58283ced1bf0f748765e274 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000638400_163430400.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19d0f10469b3ba94b12e3718689f0794e30a91525371fab1440301c64a443d0d +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000650400_166502400.pth b/checkpoint_p0/milestones/checkpoint_000650400_166502400.pth new file mode 100644 index 0000000000000000000000000000000000000000..e6c640ff5dc0fae23d9e9b08f8e947e1dd77d332 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000650400_166502400.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:260347dc4a19eaa543bc9a6c66925073b7d0d7e691920262c4f9132a12da673a +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000662656_169639936.pth b/checkpoint_p0/milestones/checkpoint_000662656_169639936.pth new file mode 100644 index 0000000000000000000000000000000000000000..1a0c16589a6420d3cfb38eaca440757c1c7bfe70 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000662656_169639936.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b10dd0ac84e0735a739ad54de3996c55a696373b7a630b25915cbd9f4eb0b4c3 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000674880_172769280.pth b/checkpoint_p0/milestones/checkpoint_000674880_172769280.pth new file mode 100644 index 0000000000000000000000000000000000000000..4a7378ebbbd525da0a7e50f2cf9d0dec3c8d6e4f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000674880_172769280.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f0f42197676838b7e11f4af7070348e2bd0e6288e1c64f6d6cfc676a5472e91c +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000687136_175906816.pth b/checkpoint_p0/milestones/checkpoint_000687136_175906816.pth new file mode 100644 index 0000000000000000000000000000000000000000..7c46ea6eafb580b2937602174687484e3ad0a59d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000687136_175906816.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e44acd41baec668458c64adb4e2b1922b0660ecb6da8f1d630389b8662f665a6 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000699520_179077120.pth b/checkpoint_p0/milestones/checkpoint_000699520_179077120.pth new file mode 100644 index 0000000000000000000000000000000000000000..db73f03e8e140f5317631e0a603af502c04faf98 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000699520_179077120.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e8447f6fcb962e56a3a2566d010931bad71f1d2767a6ff2e1e12f94f2fab3c8a +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000711808_182222848.pth b/checkpoint_p0/milestones/checkpoint_000711808_182222848.pth new file mode 100644 index 0000000000000000000000000000000000000000..cff067d57578d2c436a3e9ee448839421f09d1b6 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000711808_182222848.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ab789f39264ace0531cd2c77f79cf12f34854a5fa8b91d09cad8966f77368123 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000724064_185360384.pth b/checkpoint_p0/milestones/checkpoint_000724064_185360384.pth new file mode 100644 index 0000000000000000000000000000000000000000..da2ac755101aa9f7cb86e1dacbb0bb652785d41a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000724064_185360384.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4b5e7a15564759aafd49da6a85940677e8e9deae24c10c2902a32b5d523af3cc +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000736320_188497920.pth b/checkpoint_p0/milestones/checkpoint_000736320_188497920.pth new file mode 100644 index 0000000000000000000000000000000000000000..3a6db7a3080651e4c9383ee2c2e8900caf21e2f0 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000736320_188497920.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:defecf47a8a5188c6bb4ac562285e4d050865110dd20ffa3db28445a7667e2e3 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000748608_191643648.pth b/checkpoint_p0/milestones/checkpoint_000748608_191643648.pth new file mode 100644 index 0000000000000000000000000000000000000000..0742cc0c22aa2bc2beba09c6f20504ba23e4d252 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000748608_191643648.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b2741692027ed87b3f039f2a0693f94d64dd8c573d70f4ba47f2da9facfe1449 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000760896_194789376.pth b/checkpoint_p0/milestones/checkpoint_000760896_194789376.pth new file mode 100644 index 0000000000000000000000000000000000000000..0cf06ec530f6118090605ee06231fbbf640d9f81 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000760896_194789376.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b9327230ce5f2d35ccf9257f4c6d0cd0606a216a1c63994a64c9b35a51af6b52 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000773152_197926912.pth b/checkpoint_p0/milestones/checkpoint_000773152_197926912.pth new file mode 100644 index 0000000000000000000000000000000000000000..6eabc81d4e30f0088fa971b3e96deefdd1650e0c --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000773152_197926912.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae04aec9185c961265992bf1da2fa51efbd44cad59e12b104996be5f6c4e694d +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000785440_201072640.pth b/checkpoint_p0/milestones/checkpoint_000785440_201072640.pth new file mode 100644 index 0000000000000000000000000000000000000000..313596481d388a7bc069d13d8aa38ba5066126a8 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000785440_201072640.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7882a3e691ac7046ac2baa5f098bb08b16ae078b3ece45b62951b31296f40b3e +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000797728_204218368.pth b/checkpoint_p0/milestones/checkpoint_000797728_204218368.pth new file mode 100644 index 0000000000000000000000000000000000000000..e348223e1ca96de3647fe6c7c812f3de299ee28e --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000797728_204218368.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:83364bc90c3d8e3e607964b657a61be195b537852e21c4472125329c9fc882f2 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000810080_207380480.pth b/checkpoint_p0/milestones/checkpoint_000810080_207380480.pth new file mode 100644 index 0000000000000000000000000000000000000000..7e4f3bf8e850a8e5d965f08a3ad4330d7a1e0cd7 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000810080_207380480.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:70d882b80e63fd1449802c0a9a973da7496a5f678fcd43a11a03af92d51237fe +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000822400_210534400.pth b/checkpoint_p0/milestones/checkpoint_000822400_210534400.pth new file mode 100644 index 0000000000000000000000000000000000000000..47137435ae785e4ad39b25811b2f04ce82040e20 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000822400_210534400.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:327f79b42f2af68d1f8bbccd3886e4b34d5805ab986762c3dc43b49cbc873790 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000834656_213671936.pth b/checkpoint_p0/milestones/checkpoint_000834656_213671936.pth new file mode 100644 index 0000000000000000000000000000000000000000..23acc711700c423a6c78abdbb43d7785cc68f7c9 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000834656_213671936.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af0c9accf2e23b99ffefa7e8e62d21f348c91d83c9a0f6433e0f8164c0f874fd +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000846880_216801280.pth b/checkpoint_p0/milestones/checkpoint_000846880_216801280.pth new file mode 100644 index 0000000000000000000000000000000000000000..d5a50b8f0583962c92c002c8d023143ceaca9306 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000846880_216801280.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc9d86c81bb19dbb58ec8e807591e6f6d69b56eaf278a4bdb94a4971002db3b6 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000859136_219938816.pth b/checkpoint_p0/milestones/checkpoint_000859136_219938816.pth new file mode 100644 index 0000000000000000000000000000000000000000..6f06fe6971301b2d8bb68ecfacf838d0f40bba15 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000859136_219938816.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f657d79f024c42e9074bb3eeca03513018136ed7fd82d40565ccef1cd712fd52 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000871424_223084544.pth b/checkpoint_p0/milestones/checkpoint_000871424_223084544.pth new file mode 100644 index 0000000000000000000000000000000000000000..4c0e0a7d745c656692a8e4dfa8d6783ff2865d92 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000871424_223084544.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e62c107fdc72d65088126e979b418f82c3158986f149149e58b30c8d0678afe8 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000883744_226238464.pth b/checkpoint_p0/milestones/checkpoint_000883744_226238464.pth new file mode 100644 index 0000000000000000000000000000000000000000..38eb52a254ab16873443c4813e484714546e04d3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000883744_226238464.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:227692ad6c4db7ae30e2084c7bd86b9cf40e75bc1c58278efac191a1e34601cb +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000896064_229392384.pth b/checkpoint_p0/milestones/checkpoint_000896064_229392384.pth new file mode 100644 index 0000000000000000000000000000000000000000..c31f89ff257a08d91054084dd0fd1e260d4dde8d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000896064_229392384.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d91862978441eccc3b25938f03e8792a12807d1edf149a2da98026da185cf8de +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000908096_232472576.pth b/checkpoint_p0/milestones/checkpoint_000908096_232472576.pth new file mode 100644 index 0000000000000000000000000000000000000000..9710233c62fd415ae78a4cd2df4c2a447f6f8f00 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000908096_232472576.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:72f0e21d18854d0514c716816a33117a26d6e4e6d0ed42a792c1235584ba4565 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000920352_235610112.pth b/checkpoint_p0/milestones/checkpoint_000920352_235610112.pth new file mode 100644 index 0000000000000000000000000000000000000000..2502d4a3b702fb31035ec5b44059863690861944 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000920352_235610112.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b500f7927bc84caafe33d0c218e0e7c6fbfd9e9b6b94592fcee0dd947974299e +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000932672_238764032.pth b/checkpoint_p0/milestones/checkpoint_000932672_238764032.pth new file mode 100644 index 0000000000000000000000000000000000000000..43cc638ac9843159bb6b8803eda1c84e130fef78 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000932672_238764032.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:75be98820cca9ac2ca4df6bdfdddfe8c8fcc2e331d8aba89e0e6b6f21a89f34f +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000944992_241917952.pth b/checkpoint_p0/milestones/checkpoint_000944992_241917952.pth new file mode 100644 index 0000000000000000000000000000000000000000..405115704154ddf6794af75dee7fe09b65c79c03 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000944992_241917952.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1f432a257c01b2f47451851ea3ee84228c713274d0905956548ff6e70631d609 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000957248_245055488.pth b/checkpoint_p0/milestones/checkpoint_000957248_245055488.pth new file mode 100644 index 0000000000000000000000000000000000000000..f5697bc7e940f87a1c0c7f18993aaa5617d5ecd7 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000957248_245055488.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9e88f91f908c16243b5c3cfc310891d62ad41d7fdfce5cc5ef3f303589a161e6 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000969504_248193024.pth b/checkpoint_p0/milestones/checkpoint_000969504_248193024.pth new file mode 100644 index 0000000000000000000000000000000000000000..1f062f22b210f9a7f935738e7fec931ad72c8549 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000969504_248193024.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9d8c7ad58046cd8b654fc9b14aeb89044cd46737dec0c05188b37fcbcefe07d0 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000981664_251305984.pth b/checkpoint_p0/milestones/checkpoint_000981664_251305984.pth new file mode 100644 index 0000000000000000000000000000000000000000..c5187b4d744e9c3d9b1d028c47194fbe3bf482e8 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000981664_251305984.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f8ca5ecfe1c72955205553f9aa8fa700118ed84448878d728127c069da239ada +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_000993952_254451712.pth b/checkpoint_p0/milestones/checkpoint_000993952_254451712.pth new file mode 100644 index 0000000000000000000000000000000000000000..a4f8d53696ac5718ce12528765c254e90c5ced48 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_000993952_254451712.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e86814849dc78614b9ce884af1295465d54c55e5e99496923f2474e17f429675 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001006272_257605632.pth b/checkpoint_p0/milestones/checkpoint_001006272_257605632.pth new file mode 100644 index 0000000000000000000000000000000000000000..813f16266e37962a5caaa1c8aef7bf02d1bc455d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001006272_257605632.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af9bc258a1fff6c9767fc52ab7a839e614b0e70e2597f1ae1ecd00d3c85071d6 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001018592_260759552.pth b/checkpoint_p0/milestones/checkpoint_001018592_260759552.pth new file mode 100644 index 0000000000000000000000000000000000000000..3d1107671584b5441f3357a6723c5c78a8ef78e2 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001018592_260759552.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6897c26f9c2b42767ce40e7dbd5fc6fe438e1042f824aca6d57b793441ce3415 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001030912_263913472.pth b/checkpoint_p0/milestones/checkpoint_001030912_263913472.pth new file mode 100644 index 0000000000000000000000000000000000000000..1c5b3261d125c2d511900ab25fc67324ca6bc6c8 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001030912_263913472.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af66e6a4195ca5a101d01d4339a39d7ffe8b066d49829c66b9367bd2be053a65 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001043232_267067392.pth b/checkpoint_p0/milestones/checkpoint_001043232_267067392.pth new file mode 100644 index 0000000000000000000000000000000000000000..f64fb2cff05da2d4d14773c612f988be9c7b254f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001043232_267067392.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9cc29f9613f9a2dbdf3a442202a1f796f2a7f1b186a683e2b76d07f3599ab8f3 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001055552_270221312.pth b/checkpoint_p0/milestones/checkpoint_001055552_270221312.pth new file mode 100644 index 0000000000000000000000000000000000000000..f6c75d62d690bddff5d566cdd09de81a9f9753b7 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001055552_270221312.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:729416c48a5d92af5eb9465fce84dd394a4ba2f7eceed11f0e0a6e452153e504 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001067872_273375232.pth b/checkpoint_p0/milestones/checkpoint_001067872_273375232.pth new file mode 100644 index 0000000000000000000000000000000000000000..7e9e3bc76b9c34aacc33ef310ca7e535eaf99eca --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001067872_273375232.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8223651cad09d711113d248c8db7693aca796200f4ef502d4b7020ebfdf14aa9 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001080128_276512768.pth b/checkpoint_p0/milestones/checkpoint_001080128_276512768.pth new file mode 100644 index 0000000000000000000000000000000000000000..7bb0aa5de91db27cfa8958c654d383ecde92450f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001080128_276512768.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4a07380076df6dfa28c60c9faa21970f122287940807796bd4c763f2e3250d36 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001092352_279642112.pth b/checkpoint_p0/milestones/checkpoint_001092352_279642112.pth new file mode 100644 index 0000000000000000000000000000000000000000..0cce0272a83e6d8fa056eeb251cddc806be828f7 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001092352_279642112.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b622b0adb3ae48a0958ba80290159c8da8a35604653b792c1eba65d35baf2879 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001104640_282787840.pth b/checkpoint_p0/milestones/checkpoint_001104640_282787840.pth new file mode 100644 index 0000000000000000000000000000000000000000..5238f34386ac5c80eda4b327a8a7ea7fa419febb --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001104640_282787840.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3a9101a26e74fb47d1e064ed5f328487b33584241dfa14e17766f3e34e7c0f9e +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001116992_285949952.pth b/checkpoint_p0/milestones/checkpoint_001116992_285949952.pth new file mode 100644 index 0000000000000000000000000000000000000000..fab54a75506d0658a571684f3ad3c44b6aaa8ae8 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001116992_285949952.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1e8ad9c2916a08bd5c3fdc5db124ea483cca77b1f5e561064c8d26c94712453a +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001129248_289087488.pth b/checkpoint_p0/milestones/checkpoint_001129248_289087488.pth new file mode 100644 index 0000000000000000000000000000000000000000..c96516ad9d05a0155b593a2f1db0a72a375beace --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001129248_289087488.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd1ad5c622482a5a50bda812b82d353d8fe291a56b000b44f4979bfcbd01ccc9 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001141600_292249600.pth b/checkpoint_p0/milestones/checkpoint_001141600_292249600.pth new file mode 100644 index 0000000000000000000000000000000000000000..4e9093e9e46118c487514b96cc46d579228852dd --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001141600_292249600.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:330f735b79475e1f9635b8f15503b28742b2abe2c8aa00620830f2c3e15327c8 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001153920_295403520.pth b/checkpoint_p0/milestones/checkpoint_001153920_295403520.pth new file mode 100644 index 0000000000000000000000000000000000000000..c59f752b09f542a50bb5bad47ffad27d515d71e3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001153920_295403520.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ccc53d2f812a57c16447d84005923f5df9599b2dbaed41676829003b3a154676 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001166240_298557440.pth b/checkpoint_p0/milestones/checkpoint_001166240_298557440.pth new file mode 100644 index 0000000000000000000000000000000000000000..906dfe023aaaa2df5d8bbe9c791b4133df25556e --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001166240_298557440.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a575ab7da795513e76e0f9249e3d99d72e4cec4d972e8099cbe2094f0561df6 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001178496_301694976.pth b/checkpoint_p0/milestones/checkpoint_001178496_301694976.pth new file mode 100644 index 0000000000000000000000000000000000000000..6c70650b25d36621d693adefdc57a172784d8281 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001178496_301694976.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9936ebdeb041d5044d1f44993e27d3216627db2e86e44ecebd89ebae5517a4ba +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001190880_304865280.pth b/checkpoint_p0/milestones/checkpoint_001190880_304865280.pth new file mode 100644 index 0000000000000000000000000000000000000000..b38ea55e4b8fffbe89648a335fb2af3567e98e2f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001190880_304865280.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cdbd925666b87096f35fd066eea7abab2b390c26ceaa94070ab52e0539305d1a +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001203200_308019200.pth b/checkpoint_p0/milestones/checkpoint_001203200_308019200.pth new file mode 100644 index 0000000000000000000000000000000000000000..695a7028c94c1ff4d8cd1b0214b7c56fcc4cd96b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001203200_308019200.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4187ba144bc7b93613fef39ed18ff7e2ffac03e7f27da76b2ef7002ea67ed251 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001215552_311181312.pth b/checkpoint_p0/milestones/checkpoint_001215552_311181312.pth new file mode 100644 index 0000000000000000000000000000000000000000..70651779cbb29210532a421fb1049bdae060172a --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001215552_311181312.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89b6a4e5b6422a25ec107e9222cced7cc71e06a21484a4f4345134d4dbab1941 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001227936_314351616.pth b/checkpoint_p0/milestones/checkpoint_001227936_314351616.pth new file mode 100644 index 0000000000000000000000000000000000000000..eec26217d095e7e8044bda077afb6b262557524f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001227936_314351616.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e60609b8aac131148d66ca3bfcc24e1133c06a5ed37c4c46837535459458c5a +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001240224_317497344.pth b/checkpoint_p0/milestones/checkpoint_001240224_317497344.pth new file mode 100644 index 0000000000000000000000000000000000000000..e707173cf6841dbc4f29a309f111dc652dd2b173 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001240224_317497344.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a252aeb4cca923cc08d8d221ef900775858c9266eda06c475f0a65c19574b32c +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001252512_320643072.pth b/checkpoint_p0/milestones/checkpoint_001252512_320643072.pth new file mode 100644 index 0000000000000000000000000000000000000000..4e7084dd86eb91c1905509e40b431cbf5fe6b4bf --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001252512_320643072.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1beade1311752f39ddb42319ce3d302e3fc0138dcb1bbd41f434bebdc6fc2651 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001264768_323780608.pth b/checkpoint_p0/milestones/checkpoint_001264768_323780608.pth new file mode 100644 index 0000000000000000000000000000000000000000..f38acc787243ac2029354f6659493f9a13aecc93 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001264768_323780608.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8d2b09d9d1e2e9542626ac0bda84363c22f76d97e1f544d8e3807f61f0842b0c +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001277120_326942720.pth b/checkpoint_p0/milestones/checkpoint_001277120_326942720.pth new file mode 100644 index 0000000000000000000000000000000000000000..6ce06df2a6a4e9e482dc3f3fa31155cffaba19ce --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001277120_326942720.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f6f34a47605a9887bfbe8185282629471aabf46b839918ee213897b722124796 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001289472_330104832.pth b/checkpoint_p0/milestones/checkpoint_001289472_330104832.pth new file mode 100644 index 0000000000000000000000000000000000000000..1acf26a8acf066ce71aa0d0bf0ea8c53a211920b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001289472_330104832.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8742ad62ad332ed7059fab101d947135c3e51d5d63b19ce5560b2ee95768a6fb +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001301792_333258752.pth b/checkpoint_p0/milestones/checkpoint_001301792_333258752.pth new file mode 100644 index 0000000000000000000000000000000000000000..0217998862e3edd26d06e13d15bac379acc55a84 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001301792_333258752.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b107a3a4543d2024e8a25b672107a039b4b18017dfd8d2969647c2962feb1916 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001314112_336412672.pth b/checkpoint_p0/milestones/checkpoint_001314112_336412672.pth new file mode 100644 index 0000000000000000000000000000000000000000..2daeee4a0b0ab72d055c1e982d2dce862325e94b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001314112_336412672.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0a15ec92d4f9c980a80b38422467fd449f2a25fb77e32a1908a8c742076bfb4d +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001326432_339566592.pth b/checkpoint_p0/milestones/checkpoint_001326432_339566592.pth new file mode 100644 index 0000000000000000000000000000000000000000..264f2413dd6147041b5aa99168c25062af44b719 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001326432_339566592.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:262b7bc3139c883459ad1b3aecdc912dcfb076873311679f4f391bc731eb766c +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001338720_342712320.pth b/checkpoint_p0/milestones/checkpoint_001338720_342712320.pth new file mode 100644 index 0000000000000000000000000000000000000000..cd10883213ce346da71eaea075cfec2fac79f718 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001338720_342712320.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7ad9c80ecaf131374140c9e7c8cafd62ecd8d7bee243809398e7ff56c6bbf923 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001351008_345858048.pth b/checkpoint_p0/milestones/checkpoint_001351008_345858048.pth new file mode 100644 index 0000000000000000000000000000000000000000..9a761748be9bbd53daac992f67bdd20920ac1a21 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001351008_345858048.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e0337a3eef07d473040700fe45471a47d8639c45b990cb89c79c3770b81d6b35 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001363264_348995584.pth b/checkpoint_p0/milestones/checkpoint_001363264_348995584.pth new file mode 100644 index 0000000000000000000000000000000000000000..d9ec3d429467eb11dedc9b2578639824612358a2 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001363264_348995584.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:68a3baec4963f49a229649dd739e85ef76299ec48cd0fe30db9f59864f54be30 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001375680_352174080.pth b/checkpoint_p0/milestones/checkpoint_001375680_352174080.pth new file mode 100644 index 0000000000000000000000000000000000000000..15ad160808754d6365eaa01fff228d73bd09aaa9 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001375680_352174080.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:872a9ac9f978c246591aa57259aabe6de4e1ef3611f4e158d4e23d7a12d7da71 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001388032_355336192.pth b/checkpoint_p0/milestones/checkpoint_001388032_355336192.pth new file mode 100644 index 0000000000000000000000000000000000000000..7dd79ec6b5f84441f4912cd9e4eea94882774ea0 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001388032_355336192.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:61587434c78f1f0a50070f132e564c559a407012ceee26c5bb8baec79e7e266c +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001400352_358490112.pth b/checkpoint_p0/milestones/checkpoint_001400352_358490112.pth new file mode 100644 index 0000000000000000000000000000000000000000..8569a51a0d83e60d330d496c43b87789d50d2ed9 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001400352_358490112.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8aa0f743696ae5472bee94d440c26f9e219052c594f4a372d688f7233e9166fe +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001412704_361652224.pth b/checkpoint_p0/milestones/checkpoint_001412704_361652224.pth new file mode 100644 index 0000000000000000000000000000000000000000..647f2520b7c16dfc51ca44cf73fb9ca8941785b0 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001412704_361652224.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:357268059626971f468160ddd55c344506347d1d0e616aa9eb1e9e60c50cd09f +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001424992_364797952.pth b/checkpoint_p0/milestones/checkpoint_001424992_364797952.pth new file mode 100644 index 0000000000000000000000000000000000000000..dd09f3593362eb61eddb6545991a6bb3470a50c3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001424992_364797952.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5b1f1bfcfb96c8156849cac248cd01bb6e35946758bbd1f25a3a142ade10f929 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001437312_367951872.pth b/checkpoint_p0/milestones/checkpoint_001437312_367951872.pth new file mode 100644 index 0000000000000000000000000000000000000000..c805006887088c88280246d3f46fb8b301c914cb --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001437312_367951872.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6169accf8b41a0cd3946510d57501c97ff0ff39610546a222fd06f551c79f134 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001449600_371097600.pth b/checkpoint_p0/milestones/checkpoint_001449600_371097600.pth new file mode 100644 index 0000000000000000000000000000000000000000..1bade2e87dc147e2e249f0405d6ed7476f65d68d --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001449600_371097600.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5b1481f8f06f6dcb4d8b8521c04331027b88c66d16fd6f82e6b0c35af6cb11bb +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001461888_374243328.pth b/checkpoint_p0/milestones/checkpoint_001461888_374243328.pth new file mode 100644 index 0000000000000000000000000000000000000000..36037ec337678107fce10d303e55f136cec90bee --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001461888_374243328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f2035fe371e42e00089dd87e61eafef16a6c677dd77823a3eab89d2d0f5cd7c4 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001474144_377380864.pth b/checkpoint_p0/milestones/checkpoint_001474144_377380864.pth new file mode 100644 index 0000000000000000000000000000000000000000..a5c0698f64c860c0c242c4b429a88e3e9cbc0dce --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001474144_377380864.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ab8cce621a9fc346d73a48ff33c8eaf239e86f91f47bb49a72fc52589829ff8 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001486400_380518400.pth b/checkpoint_p0/milestones/checkpoint_001486400_380518400.pth new file mode 100644 index 0000000000000000000000000000000000000000..9ff23c764defb03cb4e65a0dd3a453182847fc82 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001486400_380518400.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a689df3058dd0b198be0cc4dd4a8d56d6c447fe7390c1f1f4dc1cc31ade7787b +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001498688_383664128.pth b/checkpoint_p0/milestones/checkpoint_001498688_383664128.pth new file mode 100644 index 0000000000000000000000000000000000000000..ecbe0f26e43f1867f702f65acf97b738499dc0d3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001498688_383664128.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19d37c4423574a25bf944b3b2f2393706fb90845c558e89d3623bc579ea7b724 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001511008_386818048.pth b/checkpoint_p0/milestones/checkpoint_001511008_386818048.pth new file mode 100644 index 0000000000000000000000000000000000000000..728677641a6f99ea0bb3a7320d6d1ea0c569a90f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001511008_386818048.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4c16ff883aa1a6d615949178f68b49627c1facd6db6eecd30d687cd5aa9b583e +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001523296_389963776.pth b/checkpoint_p0/milestones/checkpoint_001523296_389963776.pth new file mode 100644 index 0000000000000000000000000000000000000000..96ad73d9e0af26e40355d0f37d2dfb844569ca90 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001523296_389963776.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8048f37728217a1f6287c55edfe64108975573d32201ad68e2db9769ffac202 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001535264_393027584.pth b/checkpoint_p0/milestones/checkpoint_001535264_393027584.pth new file mode 100644 index 0000000000000000000000000000000000000000..6f7e8ec295d89faa0e2f4f8f0dd96fa8e621ffd2 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001535264_393027584.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:addde8263eb4e18971e9703d9a9d96bba13bd69f5e64cd32f4417d19d0e62fe3 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001547584_396181504.pth b/checkpoint_p0/milestones/checkpoint_001547584_396181504.pth new file mode 100644 index 0000000000000000000000000000000000000000..56cb4ab2bf358820f85027ee8e18125b6c34a071 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001547584_396181504.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cf6bfe39f16ade01302cf4bc764095bbcac1a79cd592d4c6bb9c53ad3170e9f5 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001559936_399343616.pth b/checkpoint_p0/milestones/checkpoint_001559936_399343616.pth new file mode 100644 index 0000000000000000000000000000000000000000..a599e1d889f7848b5599af03d8204fb36c4d6469 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001559936_399343616.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:efd26b507afe955e2c2e952c65ec303ee25453b7033a02aec7c669e33f0464a7 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001572288_402505728.pth b/checkpoint_p0/milestones/checkpoint_001572288_402505728.pth new file mode 100644 index 0000000000000000000000000000000000000000..5b7670798daf1ba4db5d58a9fd2c6c1618dd4de7 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001572288_402505728.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c85d63adbadc2787f4c8d8fdf8a58f09796c64d6c4a44f4b58e51822d4f34cd +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001584608_405659648.pth b/checkpoint_p0/milestones/checkpoint_001584608_405659648.pth new file mode 100644 index 0000000000000000000000000000000000000000..9a0687795f00fd1ace7e02305a9121bca71f2321 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001584608_405659648.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ceb54e40f2f276be6996bcc8214d890e70fce4d524500d972d48cc4b32339497 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001596928_408813568.pth b/checkpoint_p0/milestones/checkpoint_001596928_408813568.pth new file mode 100644 index 0000000000000000000000000000000000000000..fa374eb60e9b2aeb910037c63c86aebb3af0a593 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001596928_408813568.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:62d58a473f75b4e182006cff49c3e0a249d73ae2f2566619065e8de175bfff0b +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001609184_411951104.pth b/checkpoint_p0/milestones/checkpoint_001609184_411951104.pth new file mode 100644 index 0000000000000000000000000000000000000000..8f7b4861c40982dff8c3b9f66756c6b465063c1b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001609184_411951104.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1e678024e17cbf15163ec42988fd545db464f79daa36f8357a24795055fac737 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001621472_415096832.pth b/checkpoint_p0/milestones/checkpoint_001621472_415096832.pth new file mode 100644 index 0000000000000000000000000000000000000000..0cdba2d5fa31d12ac30ada639f71c98510028c6c --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001621472_415096832.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:473ce95f65b7321ac7fe73cc3baa53faa3e1028dac7781f2babfa62fd1a0c026 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001633792_418250752.pth b/checkpoint_p0/milestones/checkpoint_001633792_418250752.pth new file mode 100644 index 0000000000000000000000000000000000000000..0ab4f4c6c529b700f902fa598b3cf5833c93d276 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001633792_418250752.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:db32d0bb322a7c383e3c8e44a67e2c76e4d0557f3658b64fc6d06b8efee7de18 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001646080_421396480.pth b/checkpoint_p0/milestones/checkpoint_001646080_421396480.pth new file mode 100644 index 0000000000000000000000000000000000000000..0e43d551a409ca315a2dabbf77a06baf66888921 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001646080_421396480.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:69bab2f5bfbcea6babbcd427e9c3d4903d7f2d25c3b9160e0b800c96d0c0e265 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001658368_424542208.pth b/checkpoint_p0/milestones/checkpoint_001658368_424542208.pth new file mode 100644 index 0000000000000000000000000000000000000000..5aa3954b53b79035b519b3404b3aa7c0a2f696ec --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001658368_424542208.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4902bd2f998dc39bd944b3908cd50b168f6b72a8f800427b9c0a0a7bf4ba742b +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001670656_427687936.pth b/checkpoint_p0/milestones/checkpoint_001670656_427687936.pth new file mode 100644 index 0000000000000000000000000000000000000000..ad27fca738db817eadc300ab150910ce2a60520f --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001670656_427687936.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8ad3ba1fd4ada8f1f78a4acc9b7ce1f3f0351ad7477c1024702aae0623d02750 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001683008_430850048.pth b/checkpoint_p0/milestones/checkpoint_001683008_430850048.pth new file mode 100644 index 0000000000000000000000000000000000000000..e0b8f93524ca6a0d9d5a386d8c67edc9a30721e4 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001683008_430850048.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:281cce7d1f9a7ff37ae2d93a58f42ec810be51913872719363fd04cdf827aea8 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001695328_434003968.pth b/checkpoint_p0/milestones/checkpoint_001695328_434003968.pth new file mode 100644 index 0000000000000000000000000000000000000000..f6683826f16eb62e60457e483532723d85f7a29b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001695328_434003968.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7864e820b856f9ab8019b39469f04d7324ff39f3e791a2a22ee192b4156e11ee +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001707616_437149696.pth b/checkpoint_p0/milestones/checkpoint_001707616_437149696.pth new file mode 100644 index 0000000000000000000000000000000000000000..d5f07378f224c65db71b9d1e7733268ac15f042b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001707616_437149696.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3acabb4fc9fa2dae6b7581e39dbffcd3e4c26bdea306774a57fdea0185057cac +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001719936_440303616.pth b/checkpoint_p0/milestones/checkpoint_001719936_440303616.pth new file mode 100644 index 0000000000000000000000000000000000000000..39b8ec5b58a0d0ff88eb16c0c424c56b861a5a1b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001719936_440303616.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0fc6cb4b9a6f2d3aae04db9f8ddd03f8ba2c659acba5ff686fe98d77738cc7f3 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001732224_443449344.pth b/checkpoint_p0/milestones/checkpoint_001732224_443449344.pth new file mode 100644 index 0000000000000000000000000000000000000000..4e7c72981fe1bcf676e3726b081ccfff602756d0 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001732224_443449344.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:866b424f7a4b25cff7c44d0bf98a404d17c4e66247d6168d28623e9d0fe13397 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001744576_446611456.pth b/checkpoint_p0/milestones/checkpoint_001744576_446611456.pth new file mode 100644 index 0000000000000000000000000000000000000000..53ce1b737f0f8ee6f0dddff58fbbd565c4910eaa --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001744576_446611456.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2a170b18a2fe397f57b97f865d8f65a03cb3214b556dc7ff6694fa224eb45081 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001756896_449765376.pth b/checkpoint_p0/milestones/checkpoint_001756896_449765376.pth new file mode 100644 index 0000000000000000000000000000000000000000..02d09712fc9e7df51b0baa81bbcc94c82c88225b --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001756896_449765376.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de65b8db91d0d46cb4a90d5905e71e0d0208f9af02102272c67c9f66e8e5b1d2 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001769216_452919296.pth b/checkpoint_p0/milestones/checkpoint_001769216_452919296.pth new file mode 100644 index 0000000000000000000000000000000000000000..e35a7c0e0e1594af26488acf1cc32ce0e4bb8313 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001769216_452919296.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:518879c3f72a42449a69a7c4d31b7d0f4d96c341030267c6c3ed7d63a44a49d1 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001781536_456073216.pth b/checkpoint_p0/milestones/checkpoint_001781536_456073216.pth new file mode 100644 index 0000000000000000000000000000000000000000..2de32ff4392f2e3e707ea06b3a714f8c096d3ed6 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001781536_456073216.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:995b815ae06364f2c58c1cc79516f382d10c11fa541947445f90224c6868e80d +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001793792_459210752.pth b/checkpoint_p0/milestones/checkpoint_001793792_459210752.pth new file mode 100644 index 0000000000000000000000000000000000000000..2e2b3dcb413f080d3741b87d6692a9238799f874 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001793792_459210752.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:74032d6eb70aede65a1eaa83db2f78fa86f7bb73bd306aeecab608228d9f4ef4 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001806144_462372864.pth b/checkpoint_p0/milestones/checkpoint_001806144_462372864.pth new file mode 100644 index 0000000000000000000000000000000000000000..eef41017cc9863ba166027723f9ace204f03cb69 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001806144_462372864.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f346648baa7e02badc78c00db7dbe1a105ddba82a1ced26573315956f11c7734 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001818400_465510400.pth b/checkpoint_p0/milestones/checkpoint_001818400_465510400.pth new file mode 100644 index 0000000000000000000000000000000000000000..7b2b438bbfa13b65d81eacd38a239762ccbca9c2 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001818400_465510400.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:31276ecc37b9ac4a55e1c815f1a9a1d1890d9ed41ef3a89388e937db7eb4425e +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001830720_468664320.pth b/checkpoint_p0/milestones/checkpoint_001830720_468664320.pth new file mode 100644 index 0000000000000000000000000000000000000000..83e3744d093a882774addc567e4b456476d476e9 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001830720_468664320.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:849b44bae84c1d394b6707d54213b2ee05153d5b0bf84c808179724dd5da1137 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001842880_471777280.pth b/checkpoint_p0/milestones/checkpoint_001842880_471777280.pth new file mode 100644 index 0000000000000000000000000000000000000000..906fbf65a7d0a45281dc6373c7af200871356770 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001842880_471777280.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8fba3255a5ed8f51947f8849326f1932cb4ddf1e7f43c1f378fbad464d1a2309 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001855200_474931200.pth b/checkpoint_p0/milestones/checkpoint_001855200_474931200.pth new file mode 100644 index 0000000000000000000000000000000000000000..89229109db216d1b99578654407dfa85d37763e4 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001855200_474931200.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:21853358148be847092ebfcc6a570774358a0891eaa69fc6c3b137f720a03900 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001867456_478068736.pth b/checkpoint_p0/milestones/checkpoint_001867456_478068736.pth new file mode 100644 index 0000000000000000000000000000000000000000..ff975f4a71c8d3a80ebcc5c2179705f20600fc15 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001867456_478068736.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19f9dd698f5b6c5c4dd464aea9260b7e1bb5d2e6d6c00b05fda6aefb6cade885 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001879776_481222656.pth b/checkpoint_p0/milestones/checkpoint_001879776_481222656.pth new file mode 100644 index 0000000000000000000000000000000000000000..db4ea217f9f2d132a63e0a8b73e7c5d999d13064 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001879776_481222656.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f1157b880afa2a39fb256a102348936aa0e2cbfdd31b4edef000dd60d251ea3f +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001891968_484343808.pth b/checkpoint_p0/milestones/checkpoint_001891968_484343808.pth new file mode 100644 index 0000000000000000000000000000000000000000..91355166c5bd64cefc17fef53ef0c562cda91dfb --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001891968_484343808.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89996324a73a42855815707c0490e47b9c87c7bc16cf0f57e8c416fe485a71aa +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001904224_487481344.pth b/checkpoint_p0/milestones/checkpoint_001904224_487481344.pth new file mode 100644 index 0000000000000000000000000000000000000000..c83c4789c65fc6dad378af2ee88614cc0294b6c3 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001904224_487481344.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:619f1550b5fd939ae154214d37477f222471c5e58f7d77960023822e23f748e1 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001916512_490627072.pth b/checkpoint_p0/milestones/checkpoint_001916512_490627072.pth new file mode 100644 index 0000000000000000000000000000000000000000..cd5ef8ac1a416cf1d6a652f95f7fd1cc6371f587 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001916512_490627072.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0a035ed707da26808a3165a6d1c9ad968883d776ea4560ff1abc82edfb05b820 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001928736_493756416.pth b/checkpoint_p0/milestones/checkpoint_001928736_493756416.pth new file mode 100644 index 0000000000000000000000000000000000000000..f7a11a34543de7b81f284ca4d6a3194b256cef34 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001928736_493756416.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:65e10947d00fe8440acb06d1a7001b284a35f70490e4413dd9f84f6848500d86 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001940768_496836608.pth b/checkpoint_p0/milestones/checkpoint_001940768_496836608.pth new file mode 100644 index 0000000000000000000000000000000000000000..71887110598365b18ce15ca453c67cdffa856a79 --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001940768_496836608.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ed9f91464a79bf453ca6b30eb8de2c80053829764a4ad3d0afcd92cf36fea376 +size 20797067 diff --git a/checkpoint_p0/milestones/checkpoint_001952992_499965952.pth b/checkpoint_p0/milestones/checkpoint_001952992_499965952.pth new file mode 100644 index 0000000000000000000000000000000000000000..3b1a6e5ee8da9c4dc374bd2604f8bd301490ecfe --- /dev/null +++ b/checkpoint_p0/milestones/checkpoint_001952992_499965952.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3249be3d980808da1c12f9e0a01f4af8c7f3918e522218a2f1c81ebfcab041aa +size 20797067 diff --git a/checkpoint_p1/best_001933184_494895104_reward_83.710.pth b/checkpoint_p1/best_001933184_494895104_reward_83.710.pth new file mode 100644 index 0000000000000000000000000000000000000000..3af3227242de711939001fa453d7ea881216a28d --- /dev/null +++ b/checkpoint_p1/best_001933184_494895104_reward_83.710.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:79d2d2b9f0986c856102c1f2bdcb60f9d1aeacd07d5e1d60b52445240fac54b9 +size 20795763 diff --git a/checkpoint_p1/checkpoint_001952000_499712000.pth b/checkpoint_p1/checkpoint_001952000_499712000.pth new file mode 100644 index 0000000000000000000000000000000000000000..7bf8a5cf03746b376a39b1d95cc7b00abc8d9663 --- /dev/null +++ b/checkpoint_p1/checkpoint_001952000_499712000.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:718ae5d02a8127ed6c26cfeb262a46c25aef64d7cc97b29674af0e6653c3938b +size 20796099 diff --git a/checkpoint_p1/checkpoint_001953120_500006912.pth b/checkpoint_p1/checkpoint_001953120_500006912.pth new file mode 100644 index 0000000000000000000000000000000000000000..835ea48fb129926700dd15b5477c81c37bac6e34 --- /dev/null +++ b/checkpoint_p1/checkpoint_001953120_500006912.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8092cebb6c0a3580dba6b8131fa199d112b6c0ee55b9462675fce2c1bfc9aa5 +size 20796099 diff --git a/checkpoint_p1/milestones/checkpoint_000011872_3039232.pth b/checkpoint_p1/milestones/checkpoint_000011872_3039232.pth new file mode 100644 index 0000000000000000000000000000000000000000..680591032856120f58bc3a2e175b5fb254cce8bf --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000011872_3039232.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7278699400f96731e0103cbd41a9277f51a85cca8877576c53da16497095b01c +size 20796955 diff --git a/checkpoint_p1/milestones/checkpoint_000023968_6135808.pth b/checkpoint_p1/milestones/checkpoint_000023968_6135808.pth new file mode 100644 index 0000000000000000000000000000000000000000..00b516251711a21ae260beffb167c76532193010 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000023968_6135808.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d2b74106bc1ce4ddbfdaf1f31761a939f8171cec58222146585f895094bf126 +size 20796955 diff --git a/checkpoint_p1/milestones/checkpoint_000036128_9248768.pth b/checkpoint_p1/milestones/checkpoint_000036128_9248768.pth new file mode 100644 index 0000000000000000000000000000000000000000..e7c73f51ec73d8da0ff29a3cce64e77c03cd45d8 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000036128_9248768.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b08701e0032187480a8182e624b1eeff41d1c22fb44450dbc72ec109fe808577 +size 20796955 diff --git a/checkpoint_p1/milestones/checkpoint_000048288_12361728.pth b/checkpoint_p1/milestones/checkpoint_000048288_12361728.pth new file mode 100644 index 0000000000000000000000000000000000000000..e2c1a2d6ab832c217e0bb38f8713e2bca5729960 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000048288_12361728.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:93b5f07363ee67e36a2f5f6be499770a627eec7789b4b0bf03bdb43a9ed95485 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000060320_15441920.pth b/checkpoint_p1/milestones/checkpoint_000060320_15441920.pth new file mode 100644 index 0000000000000000000000000000000000000000..833455e941087a739ccad5edc85fd3afedf2197f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000060320_15441920.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:061283797d59d3cb838165ff29c246c056e57f5f9c59778b84a1cc37ec155588 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000072544_18571264.pth b/checkpoint_p1/milestones/checkpoint_000072544_18571264.pth new file mode 100644 index 0000000000000000000000000000000000000000..d3e85de11d8b39f8578daae27484acb6c8759f5e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000072544_18571264.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:097f02a33f7496bea867b207702dfc2c840038fa4db88ed610a08369c79db244 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000084736_21692416.pth b/checkpoint_p1/milestones/checkpoint_000084736_21692416.pth new file mode 100644 index 0000000000000000000000000000000000000000..d216ae8e99c3ea0078d0aaa1e3bd541cd02a31ed --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000084736_21692416.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5e12517bc53488fd4805f859e9ee0f6e012ba3b2264ab7cd2b30742ae07d8b93 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000096960_24821760.pth b/checkpoint_p1/milestones/checkpoint_000096960_24821760.pth new file mode 100644 index 0000000000000000000000000000000000000000..0eb4b050675590f3a745fbca9180c654032c1ec4 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000096960_24821760.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3bd0eecc7dc9780718c4f99d99e42d966de7c70dad9c95efcbf60c6cfbe8129 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000109184_27951104.pth b/checkpoint_p1/milestones/checkpoint_000109184_27951104.pth new file mode 100644 index 0000000000000000000000000000000000000000..1af575b167abf2786232a930aac80a7665e7411d --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000109184_27951104.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:843fcebf0c9281eb9ad6af74f1e9b060d2c041aacdabc8f18eafbb9463af51e6 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000121376_31072256.pth b/checkpoint_p1/milestones/checkpoint_000121376_31072256.pth new file mode 100644 index 0000000000000000000000000000000000000000..4c649e10e60f245f790f976bc6b023fd9dc940a9 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000121376_31072256.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa0b5c294649e62923891a3bbfa8e105dce55be31f795a095f3f52759cf18dc0 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000133504_34177024.pth b/checkpoint_p1/milestones/checkpoint_000133504_34177024.pth new file mode 100644 index 0000000000000000000000000000000000000000..348d0e32dcca0dcd705dcad9150bda46fca937e8 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000133504_34177024.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:29dd9f808824f9ffec3115ee900bff4515b88143e8d0a814a9c21c016256e95e +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000145728_37306368.pth b/checkpoint_p1/milestones/checkpoint_000145728_37306368.pth new file mode 100644 index 0000000000000000000000000000000000000000..b36fcabfcc7b0bb1e8b5f366e9afe6e4f5e1f7c9 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000145728_37306368.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5518b8a3b65579e3086b04b5d44ffe98cac64390fcae48b562d64591926e0aa +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000157952_40435712.pth b/checkpoint_p1/milestones/checkpoint_000157952_40435712.pth new file mode 100644 index 0000000000000000000000000000000000000000..a67d41d1df488ada0c5737b8ccd34381bee33964 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000157952_40435712.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:27f922fad988552eccd66e9a47b48876a3c32c005af3490a249d6f4e57809105 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000170112_43548672.pth b/checkpoint_p1/milestones/checkpoint_000170112_43548672.pth new file mode 100644 index 0000000000000000000000000000000000000000..5ce50e6fe3778eae9c2a00076d6b5e8862f21297 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000170112_43548672.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b3045b017286afbdb4f2a96e914ba0086c7b80f49707f2cbccd77f494b56340c +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000182368_46686208.pth b/checkpoint_p1/milestones/checkpoint_000182368_46686208.pth new file mode 100644 index 0000000000000000000000000000000000000000..c82cb77360c6fd321add703a48d4c6f0b58c4f98 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000182368_46686208.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa29db7fed5a7e16274a1b506182100352ad6a8a7a7a7c278dbac2c8ad3fdebe +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000194592_49815552.pth b/checkpoint_p1/milestones/checkpoint_000194592_49815552.pth new file mode 100644 index 0000000000000000000000000000000000000000..68aa8c89c699bebbc7d3f41d8040a62a5cd3d4bf --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000194592_49815552.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e54f5cea464ea11fdbb3f76e251c990e58f15c1a79a606f29e5b6b4851b59ed5 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000206848_52953088.pth b/checkpoint_p1/milestones/checkpoint_000206848_52953088.pth new file mode 100644 index 0000000000000000000000000000000000000000..be17fc8f032e27758ac94c8e266af8e330d9bdab --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000206848_52953088.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:822f2d835a766813c1debe5ebe057d6c4169ba6a18c1f70067c6bd333a071742 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000219104_56090624.pth b/checkpoint_p1/milestones/checkpoint_000219104_56090624.pth new file mode 100644 index 0000000000000000000000000000000000000000..e02dfd70fa31a43bdbdad23254198695d730db19 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000219104_56090624.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5623de9c6a66dbbda7391cd2b190ab9e896951f794ecd6c4469dbdbe698fd56c +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000231392_59236352.pth b/checkpoint_p1/milestones/checkpoint_000231392_59236352.pth new file mode 100644 index 0000000000000000000000000000000000000000..21c1b0c4b75d6f80f11b02e8db1363fdf15a9115 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000231392_59236352.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:87059413d4865ffa0235aeb61f829e406cc6806848c5dae866b9134f1b9dc88f +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000243648_62373888.pth b/checkpoint_p1/milestones/checkpoint_000243648_62373888.pth new file mode 100644 index 0000000000000000000000000000000000000000..00efcdbe89a188d432f843bac2bf1a51e735934e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000243648_62373888.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1acf131f18150ea2fb7b338921a7e16399be41d052b4108a112c57505def16ec +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000255808_65486848.pth b/checkpoint_p1/milestones/checkpoint_000255808_65486848.pth new file mode 100644 index 0000000000000000000000000000000000000000..c0895ee1af65bd397e61afb79fd116b95d5dae64 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000255808_65486848.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d06ed3ea663fcb4c2d94301794530561b457d7a215cdb51ad8fa16d3c66f911a +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000268064_68624384.pth b/checkpoint_p1/milestones/checkpoint_000268064_68624384.pth new file mode 100644 index 0000000000000000000000000000000000000000..764f7254cade21691bb98005a33cbfd77aa32b70 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000268064_68624384.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ab1fe9fb4491047f50a154c7edde6307c56b8871aff3e34a98a89afc0782304c +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000280224_71737344.pth b/checkpoint_p1/milestones/checkpoint_000280224_71737344.pth new file mode 100644 index 0000000000000000000000000000000000000000..5858dda59fb0d7041d58bdff1cbbad2e32ff41c1 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000280224_71737344.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:378f6f3352bf18f73431f20101c098663d58859f9a4c4113c4b2dd66a843a5bb +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000292480_74874880.pth b/checkpoint_p1/milestones/checkpoint_000292480_74874880.pth new file mode 100644 index 0000000000000000000000000000000000000000..c1107d3d141f16be5df2dbb22e598daed7d89183 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000292480_74874880.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3e03c58923ef9e7de43c4ff39b9a3655b3d6baaf3613a5533940f5fa5d6e1374 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000304736_78012416.pth b/checkpoint_p1/milestones/checkpoint_000304736_78012416.pth new file mode 100644 index 0000000000000000000000000000000000000000..b7c0639bc263cb389098815f473814a254c463bb --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000304736_78012416.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6d78a29b261648c54fe808bd84f7820f0d129ed81055c1b6548940f20c7fc07 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000316896_81125376.pth b/checkpoint_p1/milestones/checkpoint_000316896_81125376.pth new file mode 100644 index 0000000000000000000000000000000000000000..fb44188a845da7b8f363a6bbc76aec3d31410f67 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000316896_81125376.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ee98a3ea115b30e6ae66bef60b18403d1a668bdd975e341b40f274ed11bc3bb9 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000329056_84238336.pth b/checkpoint_p1/milestones/checkpoint_000329056_84238336.pth new file mode 100644 index 0000000000000000000000000000000000000000..c318f3e4af7331c90bee8d910dc8f16d55f19d6e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000329056_84238336.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a71686f0eec7e5b198642df36a53a7fbeb65bea49695838fa8530d4141df039e +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000341280_87367680.pth b/checkpoint_p1/milestones/checkpoint_000341280_87367680.pth new file mode 100644 index 0000000000000000000000000000000000000000..15f3f9eb0a85056f982703e063636038b0f9248c --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000341280_87367680.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7070fa34092f9b5e7c1ed7f7cd447355a64256979c27f9264c2387f377f4936a +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000353472_90488832.pth b/checkpoint_p1/milestones/checkpoint_000353472_90488832.pth new file mode 100644 index 0000000000000000000000000000000000000000..17443712f61b2e5ba6b87f090e4c6f00722d8434 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000353472_90488832.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2a5c022f1859f148598a5892dbdd78f4d419e03150b0d7653f4068d7c9e9fe06 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000365664_93609984.pth b/checkpoint_p1/milestones/checkpoint_000365664_93609984.pth new file mode 100644 index 0000000000000000000000000000000000000000..4c23ffc724bf1abdb8a6066fe0106b022235f260 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000365664_93609984.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:402e43921ee6946a55a718ed6d1caeaa7d51ec51b572a18db41c74d842e1efbf +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000377952_96755712.pth b/checkpoint_p1/milestones/checkpoint_000377952_96755712.pth new file mode 100644 index 0000000000000000000000000000000000000000..f5d61f51f400b936773801da8971af699d09a407 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000377952_96755712.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae09f34c27599a695fb226708838ac5c51aaa77dc31a472d573ba99830f235c0 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000390144_99876864.pth b/checkpoint_p1/milestones/checkpoint_000390144_99876864.pth new file mode 100644 index 0000000000000000000000000000000000000000..8aabe54e21f2b6d8f4b875c89db832befd8919d6 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000390144_99876864.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5340b7a8c5ecb3abb2187f211d01fcd6ad6242bdd8d81b97cf1a926793bc347 +size 20797011 diff --git a/checkpoint_p1/milestones/checkpoint_000402368_103006208.pth b/checkpoint_p1/milestones/checkpoint_000402368_103006208.pth new file mode 100644 index 0000000000000000000000000000000000000000..4ae48b3d3d90527207b574df378fd8c50b5ab91b --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000402368_103006208.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a08cccb3ca4d3ac8dfc6644dd940d5d41f20abac81b892fdebc921e6dae13e4 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000414592_106135552.pth b/checkpoint_p1/milestones/checkpoint_000414592_106135552.pth new file mode 100644 index 0000000000000000000000000000000000000000..73f2f6dc28969cc211bbc84e8b8b3f3e55040141 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000414592_106135552.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89fda01736f910a2b9a258759be40ff2287c5c08984aa0dd831bd83636ba0a88 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000426816_109264896.pth b/checkpoint_p1/milestones/checkpoint_000426816_109264896.pth new file mode 100644 index 0000000000000000000000000000000000000000..415382ad2cd25c8114e3be53b84779e2e235cfd9 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000426816_109264896.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:20b7213fe17d3e93d30ef5809af44ebc0e2d05ef84bdfdba3a49c48442de2124 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000439040_112394240.pth b/checkpoint_p1/milestones/checkpoint_000439040_112394240.pth new file mode 100644 index 0000000000000000000000000000000000000000..71d2d183004f0ce65dc1dd537f76304ba7a09656 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000439040_112394240.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce5e313c4eea0904ead3994cfa77d77f0976e6efb4b4c1c5eb5ac0bd5be6975b +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000451264_115523584.pth b/checkpoint_p1/milestones/checkpoint_000451264_115523584.pth new file mode 100644 index 0000000000000000000000000000000000000000..f42a4cda31f5bcf2cd75b424be993c75089d4285 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000451264_115523584.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:626f12823c8a4ab39cf57801a7ee0e228f99830565255580b70477c047912cb1 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000463488_118652928.pth b/checkpoint_p1/milestones/checkpoint_000463488_118652928.pth new file mode 100644 index 0000000000000000000000000000000000000000..8dd99f0f1bc8bc7d9208a26fc7bf772ca62e3e8f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000463488_118652928.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0feef303766e289e824ded648ba08793bd161a47b0be01f5d2b255c00d1a697d +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000475776_121798656.pth b/checkpoint_p1/milestones/checkpoint_000475776_121798656.pth new file mode 100644 index 0000000000000000000000000000000000000000..358683ca3f1fc8a27297f8256c02eaf9f937cba1 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000475776_121798656.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cd2f8254d546297352172d0ab21ec590ee736335ecda5d8a0aa4e72ae2b6a949 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000488032_124936192.pth b/checkpoint_p1/milestones/checkpoint_000488032_124936192.pth new file mode 100644 index 0000000000000000000000000000000000000000..b0ecfcc4a3e524cb81730d29a549f0080a76d8bd --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000488032_124936192.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4763875219d0d50febee48113b1173fa02d0a0a5fefb753b606d96a31ac09574 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000500288_128073728.pth b/checkpoint_p1/milestones/checkpoint_000500288_128073728.pth new file mode 100644 index 0000000000000000000000000000000000000000..b0fc84f000f1a717ace23681d482e81b2755976f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000500288_128073728.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ebf9f92d7b941e164d58a94d4ed9584e75608f4a7a0f1b8f180c368e4ad2f659 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000512576_131219456.pth b/checkpoint_p1/milestones/checkpoint_000512576_131219456.pth new file mode 100644 index 0000000000000000000000000000000000000000..7757137a2df6018af404c28e789bce644a0665bb --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000512576_131219456.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bf50f0d2c11691d5aeaf046a6e7da8506804c2ff243ef59378c98f401d80d87d +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000524704_134324224.pth b/checkpoint_p1/milestones/checkpoint_000524704_134324224.pth new file mode 100644 index 0000000000000000000000000000000000000000..a32c92b6c3d2a84f4edcae5581745f2e742c83b2 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000524704_134324224.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:74ba1eb781361aa5ed52441411c7568bb56442fca484dcd10a55f109a8d60e7a +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000536928_137453568.pth b/checkpoint_p1/milestones/checkpoint_000536928_137453568.pth new file mode 100644 index 0000000000000000000000000000000000000000..fc03147368393ee4c5ed8927cc54a0274515b0ce --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000536928_137453568.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c9bcf17f1469944bfce4a01e3e8ed82cbd9525fc3ddaa5b85914c37b3db84e2 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000549152_140582912.pth b/checkpoint_p1/milestones/checkpoint_000549152_140582912.pth new file mode 100644 index 0000000000000000000000000000000000000000..52a4c9820dea81e565772c984947e383b3f7b946 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000549152_140582912.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3946829d828ad3c86579f8b9ccac3574cf0e5e543e99051d9f832d386e8a18f +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000561440_143728640.pth b/checkpoint_p1/milestones/checkpoint_000561440_143728640.pth new file mode 100644 index 0000000000000000000000000000000000000000..9a5a1e30f76810cef614bf4463691a4a8f18f6fe --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000561440_143728640.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e288e7680e501e212ea92a2c08bb641a32af8c52050a970932a1b427748f0988 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000573664_146857984.pth b/checkpoint_p1/milestones/checkpoint_000573664_146857984.pth new file mode 100644 index 0000000000000000000000000000000000000000..bfb50a375c2aafab22dcb04201b493ec1cf192df --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000573664_146857984.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c20a20876e1cf72c15db9da03ce02aaadc0b386336fa2700e14c50c50dba39ff +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000585888_149987328.pth b/checkpoint_p1/milestones/checkpoint_000585888_149987328.pth new file mode 100644 index 0000000000000000000000000000000000000000..12969e5bd5a9ab0da6d9b6ce8e4c7ad7d3308851 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000585888_149987328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d32e3594b49dda3eddff3559f79b6057cb2be28316abf961bd6e7f40b6a87a3d +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000598112_153116672.pth b/checkpoint_p1/milestones/checkpoint_000598112_153116672.pth new file mode 100644 index 0000000000000000000000000000000000000000..d469c6a66576e8d2a2f4b620bdc2a205c5f893e7 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000598112_153116672.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19bc0a7b9132241e11b38ed77d367b937d2b557286cbc6e815415cd47e7da331 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000610336_156246016.pth b/checkpoint_p1/milestones/checkpoint_000610336_156246016.pth new file mode 100644 index 0000000000000000000000000000000000000000..facec5156a65067c3aa788213430676ad843933a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000610336_156246016.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e00c174c89c5cba308b1c161132b712523b907c993862f73db8786be78f0c0c8 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000622592_159383552.pth b/checkpoint_p1/milestones/checkpoint_000622592_159383552.pth new file mode 100644 index 0000000000000000000000000000000000000000..8732c7479472013267298e9396507990a776a8cd --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000622592_159383552.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:72dc4cf2b022353af6059c43c02777322f1c97caaa96c04b90c93fb5c36e795b +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000634816_162512896.pth b/checkpoint_p1/milestones/checkpoint_000634816_162512896.pth new file mode 100644 index 0000000000000000000000000000000000000000..b7c87f3045ce76be768d575c4cc10240af1e1fcf --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000634816_162512896.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:18baa13cd1087d88ef77208e4bbad98b5f29ed4280316ae069748f292d29e37f +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000646656_165543936.pth b/checkpoint_p1/milestones/checkpoint_000646656_165543936.pth new file mode 100644 index 0000000000000000000000000000000000000000..b36568e8a660deb56c36a10c1e4e5594877dcb50 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000646656_165543936.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a4b7024e4cccc588fc67d78509f0801c019a051bfd0dc58df04da8cf2d43fd97 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000658944_168689664.pth b/checkpoint_p1/milestones/checkpoint_000658944_168689664.pth new file mode 100644 index 0000000000000000000000000000000000000000..e349d56484265ccfeb7d720dca47db105864334e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000658944_168689664.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3f9c875494039450728c1ade4445e0bc7c109ec3184c6a55814cda6085ea4cc5 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000671136_171810816.pth b/checkpoint_p1/milestones/checkpoint_000671136_171810816.pth new file mode 100644 index 0000000000000000000000000000000000000000..d2ebf52d6060a12e6e830834cf400a939d47aaa0 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000671136_171810816.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc1bf6edc570f989495f42a6302b50f520e4446a35859941efa95051f1e31f4a +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000683296_174923776.pth b/checkpoint_p1/milestones/checkpoint_000683296_174923776.pth new file mode 100644 index 0000000000000000000000000000000000000000..24d575697e52e5387741a35e5711258c0e9cbe6a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000683296_174923776.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7036ba71c8091e8ccf789e9c47bb9cfbc38eecdedcb904ad3190d5d7b6994239 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000695520_178053120.pth b/checkpoint_p1/milestones/checkpoint_000695520_178053120.pth new file mode 100644 index 0000000000000000000000000000000000000000..a57ff653010cc422f614f5f7a950bcc0fef0deb8 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000695520_178053120.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:83c55af640b5289b6bf9bd6764c46f1bd097de58d23d66d2e9873737edee3375 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000707744_181182464.pth b/checkpoint_p1/milestones/checkpoint_000707744_181182464.pth new file mode 100644 index 0000000000000000000000000000000000000000..2fcf8ac0dc270c28d094369d2b93fc5c3bab87fb --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000707744_181182464.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5b2b049d92ff4efd7f73e0af20656163feb5b769a398d84b0d3c52f39f6bf952 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000719968_184311808.pth b/checkpoint_p1/milestones/checkpoint_000719968_184311808.pth new file mode 100644 index 0000000000000000000000000000000000000000..e612cf22535b5c7f76d14d2fe1d1ccb17b583012 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000719968_184311808.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:136b0e8355de3bc57ca3ad8b6c45c70d915e72c53dccf9b1941508ea4290bbdc +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000732128_187424768.pth b/checkpoint_p1/milestones/checkpoint_000732128_187424768.pth new file mode 100644 index 0000000000000000000000000000000000000000..3face19ea9b30a88a6c8b5c29c67d9bc53d48395 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000732128_187424768.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:968286229d439015d6150c035fb7f83093489ef4dfccc5ccac487c44b17a4df5 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000744320_190545920.pth b/checkpoint_p1/milestones/checkpoint_000744320_190545920.pth new file mode 100644 index 0000000000000000000000000000000000000000..cb6921f267330d7f4a44ecc91d61f059d7ad2557 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000744320_190545920.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4292db7177eb8ba82d1d70bc2493adab4a875b2d8909e2fea054aa79a62c7be +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000756576_193683456.pth b/checkpoint_p1/milestones/checkpoint_000756576_193683456.pth new file mode 100644 index 0000000000000000000000000000000000000000..cdcd54863dc6a5acea9075a5ca5c7e7d316a4911 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000756576_193683456.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f48172e0bdead949cec4c596a1234a088a0b40225e4e6753879f2cad9cecc22b +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000768800_196812800.pth b/checkpoint_p1/milestones/checkpoint_000768800_196812800.pth new file mode 100644 index 0000000000000000000000000000000000000000..fb290a1d89d885711bf1a33ad950a452dd191a7e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000768800_196812800.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a3b3af6cfcc4c56d0fc2839c1516bc7f2d89387348545dc8586aacc0a9701109 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000780992_199933952.pth b/checkpoint_p1/milestones/checkpoint_000780992_199933952.pth new file mode 100644 index 0000000000000000000000000000000000000000..1579de8ccee53701e6acaf3adc8d7e85beef9db5 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000780992_199933952.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:657c17199cf3f9f2fff21fb87dd12c742ec8f1ec9f113dc012b14c355f5142a6 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000793248_203071488.pth b/checkpoint_p1/milestones/checkpoint_000793248_203071488.pth new file mode 100644 index 0000000000000000000000000000000000000000..c76964fea5e92b0c50fd61546cb7366d0c32708f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000793248_203071488.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa045c544b94bb9f19bd728f263b73fc465e57e471375e0d537c930f402b3857 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000805408_206184448.pth b/checkpoint_p1/milestones/checkpoint_000805408_206184448.pth new file mode 100644 index 0000000000000000000000000000000000000000..cb5a3998c3e1e68bb94fd8c79714fc92dd22e1c0 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000805408_206184448.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2d3ea65619f0bf2d439333caa5301109cb7a63aabc90d8d544b088defd12a281 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000817664_209321984.pth b/checkpoint_p1/milestones/checkpoint_000817664_209321984.pth new file mode 100644 index 0000000000000000000000000000000000000000..b52f92d10bca4413d0bd760d83e49a0be4483a39 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000817664_209321984.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:82e8efcc6df50328f23c94c9e54d6b49ecae2f9283618e5d45bc82bfee402910 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000829888_212451328.pth b/checkpoint_p1/milestones/checkpoint_000829888_212451328.pth new file mode 100644 index 0000000000000000000000000000000000000000..5aeab381debb67ddd0cce77e65466a5d8bb8794d --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000829888_212451328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a44c02c6bee17b826285fb5b6dfce9fbcf6169c984c1b3a06def8e6266655e1b +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000842176_215597056.pth b/checkpoint_p1/milestones/checkpoint_000842176_215597056.pth new file mode 100644 index 0000000000000000000000000000000000000000..63c49c36ede720b36772cb69fe5bd806efb7e738 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000842176_215597056.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f358086964a355717545fcbf16d7a7b53e8eb2059926819c1225459c04ed1858 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000854400_218726400.pth b/checkpoint_p1/milestones/checkpoint_000854400_218726400.pth new file mode 100644 index 0000000000000000000000000000000000000000..60c8b380c50a240003ab9a05125c958c9930501b --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000854400_218726400.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:94a85fc0dd45960fa9685dd963a7121ba51d6c32d8305a1d2518b927c1b7ee9b +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000866656_221863936.pth b/checkpoint_p1/milestones/checkpoint_000866656_221863936.pth new file mode 100644 index 0000000000000000000000000000000000000000..bfb654e6cc7a51359f4d10ca817a846850f1bd53 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000866656_221863936.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26f3bafbb73b0294ac4a35388a92326a67001d1488110207158e300a0bb7c750 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000878880_224993280.pth b/checkpoint_p1/milestones/checkpoint_000878880_224993280.pth new file mode 100644 index 0000000000000000000000000000000000000000..b0c11c4a3afee178b927b58295d4762b5af4a5d2 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000878880_224993280.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5a2994001905af4a629a4197f15ccafaf4936b0c415caa018aba9817ed230225 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000891104_228122624.pth b/checkpoint_p1/milestones/checkpoint_000891104_228122624.pth new file mode 100644 index 0000000000000000000000000000000000000000..3709624c80ebf48bc4ba437ee493947ebcd1d7b2 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000891104_228122624.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ef5ee26b01f27d8cbc9189598054c9a100f459efa76278231f2dd3394a756c9 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000903072_231186432.pth b/checkpoint_p1/milestones/checkpoint_000903072_231186432.pth new file mode 100644 index 0000000000000000000000000000000000000000..e90eaea0a3042092206057df028ff04e9e7aeb21 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000903072_231186432.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:646c1c3a1e9ef4e320e933a7cdaa823a2a45418a8ada7d6297fe03ea4c74289c +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000915296_234315776.pth b/checkpoint_p1/milestones/checkpoint_000915296_234315776.pth new file mode 100644 index 0000000000000000000000000000000000000000..e354a0f6428f458f233dc24e092c944e9bac935d --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000915296_234315776.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7a3523fa3229cf9eccda47b073ce9029259f72edf78e52f239822b5d7e1552d7 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000927552_237453312.pth b/checkpoint_p1/milestones/checkpoint_000927552_237453312.pth new file mode 100644 index 0000000000000000000000000000000000000000..71466fe0926c09f468d9ef0b9a37cfe6f13fe876 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000927552_237453312.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c641945d63e1996dce9f2cbe75cf87e1916606449ca6b295880405e261595ace +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000939840_240599040.pth b/checkpoint_p1/milestones/checkpoint_000939840_240599040.pth new file mode 100644 index 0000000000000000000000000000000000000000..4f97157e84f9a2eb847eb90cbf85e5c577d9cf5b --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000939840_240599040.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c4b62f805226268c6548778408acb23e6aed7a4cacbd1f32ea893d6abdff24a1 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000952096_243736576.pth b/checkpoint_p1/milestones/checkpoint_000952096_243736576.pth new file mode 100644 index 0000000000000000000000000000000000000000..c5d6eb8b99a38fca062d578147457df5aa54294e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000952096_243736576.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f67ede8c10a4160b82bff739df43536e57285a913798b94fb02f1c669367184f +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000964288_246857728.pth b/checkpoint_p1/milestones/checkpoint_000964288_246857728.pth new file mode 100644 index 0000000000000000000000000000000000000000..c9bb9a0a3383241231d55922128792696cdd1ee0 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000964288_246857728.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:32b38728cb3fd902ab5f9341edf9e0205055fc575afbd86d9ac0ec6d574dae19 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000976352_249946112.pth b/checkpoint_p1/milestones/checkpoint_000976352_249946112.pth new file mode 100644 index 0000000000000000000000000000000000000000..98f33ee28b000bd2b155b69a1a7556edc1d866bd --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000976352_249946112.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2a0fd9fbb6b8c9cb19768834271c417c867304ce588038e6f6684b4e8f617dc +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_000988608_253083648.pth b/checkpoint_p1/milestones/checkpoint_000988608_253083648.pth new file mode 100644 index 0000000000000000000000000000000000000000..4e4ab3305a006ca34bafc70806991d797e785887 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_000988608_253083648.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f43bee1b1b20fdc35b9a39facb77eb93eb1f806219be7b502268bc312673527 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001000832_256212992.pth b/checkpoint_p1/milestones/checkpoint_001000832_256212992.pth new file mode 100644 index 0000000000000000000000000000000000000000..51b915747248c3afc509aee775416f2302e49276 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001000832_256212992.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:83bafbc9d900ee36e2bed55fc4df15849d734db82dbde0dc537c92874280bb4a +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001013056_259342336.pth b/checkpoint_p1/milestones/checkpoint_001013056_259342336.pth new file mode 100644 index 0000000000000000000000000000000000000000..43a7bdc3efb9eb9938bb71f4ba4e019be8a8acaa --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001013056_259342336.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:85e4948644a13fbe23ceedcb6972b7f335900300900a86c6d644587024a5a9ce +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001025312_262479872.pth b/checkpoint_p1/milestones/checkpoint_001025312_262479872.pth new file mode 100644 index 0000000000000000000000000000000000000000..0eaf4e10b6b41f9ca5547f02fe050f4a9fe020e5 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001025312_262479872.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a812c6639d8a99c6f38478dacc6a34b0a0deb94c0ceb5ff71e9d01a245ac5d20 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001037504_265601024.pth b/checkpoint_p1/milestones/checkpoint_001037504_265601024.pth new file mode 100644 index 0000000000000000000000000000000000000000..f1be2faa18ab606fe8119ee08fe6d2356fab1cf6 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001037504_265601024.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d8c660035c2de09096ec09495ce5b6e7bcf1f39ad4448cce1e7ed01138ad40f9 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001049760_268738560.pth b/checkpoint_p1/milestones/checkpoint_001049760_268738560.pth new file mode 100644 index 0000000000000000000000000000000000000000..8049aa4400ad2e7018a3ee6c10ba317b0a85b948 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001049760_268738560.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6791df6cdb8dd648c3dcd0179e3c3b0721af33447f26f1c60f0e42c4997002c4 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001061920_271851520.pth b/checkpoint_p1/milestones/checkpoint_001061920_271851520.pth new file mode 100644 index 0000000000000000000000000000000000000000..51e0a641147789f63c2fd3737f27384a9f310ea4 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001061920_271851520.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37c8c8d8e01354aa6cfd896afa8843d80267edf7d1bbf929bf264729b38b7e88 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001074176_274989056.pth b/checkpoint_p1/milestones/checkpoint_001074176_274989056.pth new file mode 100644 index 0000000000000000000000000000000000000000..ee85dca79e1ae09e2cb52d056fbf85c77544a5e0 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001074176_274989056.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4ef65ec6b3fc4edb5d2fa4408f877b229361951f2f16a05a3aaa35833fc2aea2 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001086400_278118400.pth b/checkpoint_p1/milestones/checkpoint_001086400_278118400.pth new file mode 100644 index 0000000000000000000000000000000000000000..8f20c571ea9733e9064541c0e0f4912e2bff6f48 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001086400_278118400.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:153ed9f41acb01996cef747b442be8444f0611624e9eef150d3aa15bbbc7948e +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001098560_281231360.pth b/checkpoint_p1/milestones/checkpoint_001098560_281231360.pth new file mode 100644 index 0000000000000000000000000000000000000000..62e2b3d675de5e143b620733b6a797665af2dbb5 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001098560_281231360.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:152c14af7a2281a1c77c26dd0c36d8220091fdee266453a88101dbb4b1c3c04d +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001110720_284344320.pth b/checkpoint_p1/milestones/checkpoint_001110720_284344320.pth new file mode 100644 index 0000000000000000000000000000000000000000..44e78f36401d72348bb366f767d41724fbcec206 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001110720_284344320.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d83ba8a2bb9c0aaf15ea8442b1d355262ceccd28be345116991f7484d04caa51 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001122944_287473664.pth b/checkpoint_p1/milestones/checkpoint_001122944_287473664.pth new file mode 100644 index 0000000000000000000000000000000000000000..6114980a9c50f648ccd1b0c40e3a9793953226e2 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001122944_287473664.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:60948fd7a3e3a8580f2869c0f48973ba6196a50d67199cf8f0d52ac9401af312 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001135168_290603008.pth b/checkpoint_p1/milestones/checkpoint_001135168_290603008.pth new file mode 100644 index 0000000000000000000000000000000000000000..74dcdcd62cfebee6f111714919e56ea949b9a54e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001135168_290603008.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e281be457bac0de830a6284ee02ecf5d796e7152a3fdd6259394c0f5fa439312 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001147488_293756928.pth b/checkpoint_p1/milestones/checkpoint_001147488_293756928.pth new file mode 100644 index 0000000000000000000000000000000000000000..2cf58801458d8ecb0ea6105b2659e7ebb50783b5 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001147488_293756928.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:29774108b74e4b1b758b2b0c6d8cdc6a695f7286c4682d975b689428d244e084 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001159680_296878080.pth b/checkpoint_p1/milestones/checkpoint_001159680_296878080.pth new file mode 100644 index 0000000000000000000000000000000000000000..e91fd6daede65e6cff40ea45765043cbf6f84918 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001159680_296878080.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:287e97f94adf0de1a01a67e7453cb7cb89dd27448b38c52359d48d9da50ca14f +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001171904_300007424.pth b/checkpoint_p1/milestones/checkpoint_001171904_300007424.pth new file mode 100644 index 0000000000000000000000000000000000000000..aef2d37e29146ffc2ff7f618aeec2d95bebca7c4 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001171904_300007424.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d6cd49be40e358a326acb6e12acccebbb4e7036b3b554da27b8c1e8eb667b123 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001184128_303136768.pth b/checkpoint_p1/milestones/checkpoint_001184128_303136768.pth new file mode 100644 index 0000000000000000000000000000000000000000..ba264831c7964f437669d7a1d8083af04c3e7888 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001184128_303136768.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:08e71a7728c5ab8a937d0413e2784438cb1f0b570a9f37186714d26dcf01e1e5 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001196320_306257920.pth b/checkpoint_p1/milestones/checkpoint_001196320_306257920.pth new file mode 100644 index 0000000000000000000000000000000000000000..cf6fa898101cf383207ce4478b4adaf88df25e54 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001196320_306257920.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f607871aff83b9e853c3af753ca0cb90ee8f044d23cf2bf7fa1cacd17a5339a2 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001208576_309395456.pth b/checkpoint_p1/milestones/checkpoint_001208576_309395456.pth new file mode 100644 index 0000000000000000000000000000000000000000..f4ea67231050b76c55d82a79c8dd78479319fe68 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001208576_309395456.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7b16a3d07765ce7e4d4b3cf7585b4801f27e612b656c051201208d7877630f30 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001220864_312541184.pth b/checkpoint_p1/milestones/checkpoint_001220864_312541184.pth new file mode 100644 index 0000000000000000000000000000000000000000..080274e46213ce7115e868a1a4a75fff577b71e0 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001220864_312541184.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3bfbb6a360f492516ffc33c131be787c690647a1f782f14d36ffd2aa8487b006 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001233056_315662336.pth b/checkpoint_p1/milestones/checkpoint_001233056_315662336.pth new file mode 100644 index 0000000000000000000000000000000000000000..2051263d5ee16bbdce899efe80ae7bab8fcc4d19 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001233056_315662336.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:06388455f9335ddb8c71e450f5ed4be3649b5c41bb5cc33016f379e3af99c3ce +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001245312_318799872.pth b/checkpoint_p1/milestones/checkpoint_001245312_318799872.pth new file mode 100644 index 0000000000000000000000000000000000000000..dc007535b71c8da0eab4152c40281d9aeffbce99 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001245312_318799872.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:88804ad62092f81fff36f5ec49a04400cba33b622811e8604000fb111d7ccb35 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001257568_321937408.pth b/checkpoint_p1/milestones/checkpoint_001257568_321937408.pth new file mode 100644 index 0000000000000000000000000000000000000000..e67dcdbc92d88eb625871a84cbbf3791ec965f91 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001257568_321937408.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0d28e4ffc2ad71a8314a3ace0cb9fb870b72d95f3166e7e45adf12c56dd2f6a6 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001269792_325066752.pth b/checkpoint_p1/milestones/checkpoint_001269792_325066752.pth new file mode 100644 index 0000000000000000000000000000000000000000..d48f273996d286f193ac59166766e1ec7bd2e1f1 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001269792_325066752.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:234db23c98a1db3d57b58130ce396649db59c1403a408d0972e262f88c5835de +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001282016_328196096.pth b/checkpoint_p1/milestones/checkpoint_001282016_328196096.pth new file mode 100644 index 0000000000000000000000000000000000000000..b4201ac4b43910463c1c2aadb514df6b0ea0bb78 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001282016_328196096.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f1bac21e7500c273e8c1c0400d1d5b4977a93aa48227d4ec91d40e7812511671 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001294336_331350016.pth b/checkpoint_p1/milestones/checkpoint_001294336_331350016.pth new file mode 100644 index 0000000000000000000000000000000000000000..891c5828f69085f9fbf46613b8c2cca396e6edd1 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001294336_331350016.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2eb294d9ff9e0708af7c0b3fa358f23e115c83470ade2adddcf1e20ee32ed3d8 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001306528_334471168.pth b/checkpoint_p1/milestones/checkpoint_001306528_334471168.pth new file mode 100644 index 0000000000000000000000000000000000000000..42e1a102d3ca804729a5d1982ee81210b7ab2154 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001306528_334471168.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6ecf7544c10b0bb7ea11c5809f474bef7ad949accdc4edd143cf9d5922be2123 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001318752_337600512.pth b/checkpoint_p1/milestones/checkpoint_001318752_337600512.pth new file mode 100644 index 0000000000000000000000000000000000000000..8d9d6b167e0ff4890c2d1bb03ad8cd6cb7aa21ff --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001318752_337600512.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:226ecd5dc326d27d084842a585094f8cb93c36a06d4e8528ee1db934fccad07b +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001331008_340738048.pth b/checkpoint_p1/milestones/checkpoint_001331008_340738048.pth new file mode 100644 index 0000000000000000000000000000000000000000..0f616940d19ac6a7dd966f98f870b05322087321 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001331008_340738048.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b4734e8e9759cb86495fedc86c87fc2c21e69f69cee125baaa4b159ea0b473f +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001343264_343875584.pth b/checkpoint_p1/milestones/checkpoint_001343264_343875584.pth new file mode 100644 index 0000000000000000000000000000000000000000..f01502dc7f53a4342537df57f2f66a0e35189037 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001343264_343875584.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e51ef855dd9f94ec8f7d6d4f636739532a4fee0a169078114a13ef036ecaf30b +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001355520_347013120.pth b/checkpoint_p1/milestones/checkpoint_001355520_347013120.pth new file mode 100644 index 0000000000000000000000000000000000000000..62d146ab3bc2ea70b0cf14559be00b3664009154 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001355520_347013120.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:392451d0685c03f96d39891b859e84c9f2ff2e9c5073726c15902fa3e960f357 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001367776_350150656.pth b/checkpoint_p1/milestones/checkpoint_001367776_350150656.pth new file mode 100644 index 0000000000000000000000000000000000000000..601395165ede01df4c0c0c34026027f3ec4dcfc8 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001367776_350150656.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4b4f25ac720c89075c7246f33f5945be035f832fd1f5b30126fdd553d8f5d5f5 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001379968_353271808.pth b/checkpoint_p1/milestones/checkpoint_001379968_353271808.pth new file mode 100644 index 0000000000000000000000000000000000000000..7cf238d83cb21a5eebb5f9d9492476d87571bf84 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001379968_353271808.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a7095466beaaf9c9e9ecb216e28762038a6eb83b9ab869943a6dc647027f703f +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001392192_356401152.pth b/checkpoint_p1/milestones/checkpoint_001392192_356401152.pth new file mode 100644 index 0000000000000000000000000000000000000000..68f327bb06f136c0db6c4a3d2adc0df8ce0edbcd --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001392192_356401152.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b076aeed655760d29d975bb1555147a33feb4b9eabb04452fb320f78b5ebe110 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001404512_359555072.pth b/checkpoint_p1/milestones/checkpoint_001404512_359555072.pth new file mode 100644 index 0000000000000000000000000000000000000000..ba4ea4684a2ba7d5830923905e45f6de1d1374eb --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001404512_359555072.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1163110956d00bf4ffc99bec3dd8ff41025da78c2909de7b65e9d234b52dd3a +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001416704_362676224.pth b/checkpoint_p1/milestones/checkpoint_001416704_362676224.pth new file mode 100644 index 0000000000000000000000000000000000000000..892f739bba67e53251656bb500dc76575116844a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001416704_362676224.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ee73892c47b85be577996e4b0ee69d600b427545f2ae9dfe8e1988edd2584233 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001428928_365805568.pth b/checkpoint_p1/milestones/checkpoint_001428928_365805568.pth new file mode 100644 index 0000000000000000000000000000000000000000..8222cd8abd91f7099eac61a113fb42fa98386883 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001428928_365805568.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e6e12bea7769a65622dfe7e6fbf25aeddb2b89e38ae51868fdc0eacaadee9626 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001441152_368934912.pth b/checkpoint_p1/milestones/checkpoint_001441152_368934912.pth new file mode 100644 index 0000000000000000000000000000000000000000..5fb0848570496ec0c834d6dca3f400d99e8efea3 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001441152_368934912.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:deec3f9772032d83193ef4ba6e3b16d980cfb86d30002d24b39e7129c7169b8a +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001453376_372064256.pth b/checkpoint_p1/milestones/checkpoint_001453376_372064256.pth new file mode 100644 index 0000000000000000000000000000000000000000..3ad4138316fa7aa7795989a7056e4a0193ec3ea9 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001453376_372064256.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:22294de0cd7859321c9977a0759085082ed324f2c4ccfb0025b7cb9a50f40a4e +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001465536_375177216.pth b/checkpoint_p1/milestones/checkpoint_001465536_375177216.pth new file mode 100644 index 0000000000000000000000000000000000000000..1df8b22f939465d139c99768fc33e6da106a308f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001465536_375177216.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c6d3e81c75327f11b8ef53aa333fffcdc1ecac07d1e8a3b3db930ac859ed50b4 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001477728_378298368.pth b/checkpoint_p1/milestones/checkpoint_001477728_378298368.pth new file mode 100644 index 0000000000000000000000000000000000000000..1d0d47982e409f659bc6d5ff24f65905eb0f3c1a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001477728_378298368.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:074cf1a7d361c73a88627027918d5ee125ade6ab6a8d342481b3b00af452abbd +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001489984_381435904.pth b/checkpoint_p1/milestones/checkpoint_001489984_381435904.pth new file mode 100644 index 0000000000000000000000000000000000000000..d7ae10ae5d2fe522deb7db06e572d5604a772e26 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001489984_381435904.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6176dcd379b99efdc3ddd341b2f86b31beda576db72cd02b1c8669f4041b5b33 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001502144_384548864.pth b/checkpoint_p1/milestones/checkpoint_001502144_384548864.pth new file mode 100644 index 0000000000000000000000000000000000000000..b167a6ab9d34b84e64108c517360b2afc8f398db --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001502144_384548864.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aa8e4b1c0d987b23464229ffb610e982309dabf41b07f778ef416bf2cd0b0951 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001514368_387678208.pth b/checkpoint_p1/milestones/checkpoint_001514368_387678208.pth new file mode 100644 index 0000000000000000000000000000000000000000..b588be78bd8fce8ee876d3cc280e8e4ffc9ac985 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001514368_387678208.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:04333301b116216fa4f2dd479e17207cbc91deab5804b25968d2ce9a67452d10 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001526304_390733824.pth b/checkpoint_p1/milestones/checkpoint_001526304_390733824.pth new file mode 100644 index 0000000000000000000000000000000000000000..e46fd0d7c69d8f793fb9fb5b6f97a34bada6aecf --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001526304_390733824.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce3e92f060ba9a21dba14eff8c874858f264b61c1b86457ee16b1801e0c39727 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001538496_393854976.pth b/checkpoint_p1/milestones/checkpoint_001538496_393854976.pth new file mode 100644 index 0000000000000000000000000000000000000000..e800bf664977d637c276895b595b0685e2e29ee6 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001538496_393854976.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b642528b767d805c7bd9213df55f6d76d12e065309215618ec67c82dee86077e +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001550688_396976128.pth b/checkpoint_p1/milestones/checkpoint_001550688_396976128.pth new file mode 100644 index 0000000000000000000000000000000000000000..9ef72d1803df65253a51a010c9cc0e2ce12b79a9 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001550688_396976128.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9f55ea71ce566b1d160b822a6b3fa520c1884adc94908762d3cf382eb50918df +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001562880_400097280.pth b/checkpoint_p1/milestones/checkpoint_001562880_400097280.pth new file mode 100644 index 0000000000000000000000000000000000000000..d62ff9fee4ed0a9183c9875cd3bf6e0146b2b353 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001562880_400097280.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:30aa45aeb4c3924c09077501321351ea316a136d0a3c6ea4e6302ba10f29f125 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001575104_403226624.pth b/checkpoint_p1/milestones/checkpoint_001575104_403226624.pth new file mode 100644 index 0000000000000000000000000000000000000000..15f5d68b986a8303ed59ff2a1a5c41622baaf905 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001575104_403226624.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b8c40d6e00c01b92c29c41f369f6f9b2322534835c62ad98ff42546024accf4b +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001587328_406355968.pth b/checkpoint_p1/milestones/checkpoint_001587328_406355968.pth new file mode 100644 index 0000000000000000000000000000000000000000..0b67c8359ec47fb7e06fae591dbf92e195fabbb7 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001587328_406355968.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dbdec077c88f5e628707990e8ceb003250736817de33664ab27acc709826a8aa +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001599552_409485312.pth b/checkpoint_p1/milestones/checkpoint_001599552_409485312.pth new file mode 100644 index 0000000000000000000000000000000000000000..d06ffdc7491f11a9446fffd1cd530b86a54c4d36 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001599552_409485312.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44d049f7cfe50554170975ac653bcc82dbbdfab03a2ec8bb7025592e2f544ed5 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001611744_412606464.pth b/checkpoint_p1/milestones/checkpoint_001611744_412606464.pth new file mode 100644 index 0000000000000000000000000000000000000000..d83d3c44bf10231b3f47f27cdfefc7b313046ee4 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001611744_412606464.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f33e4f68a3f56d4d15e010c487b187fd064c85a1bdd30ee7969719d1dd1e5f8b +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001623968_415735808.pth b/checkpoint_p1/milestones/checkpoint_001623968_415735808.pth new file mode 100644 index 0000000000000000000000000000000000000000..7c2be6f21a783c5cb367d29a69fc4c8eb1f71c13 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001623968_415735808.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9c4361980be414b6d8125180e85546de053f9cde903a06f246b2141f0458bf0 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001636160_418856960.pth b/checkpoint_p1/milestones/checkpoint_001636160_418856960.pth new file mode 100644 index 0000000000000000000000000000000000000000..2508ec2006acaa66a3d0926249c93fb2a2bbc163 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001636160_418856960.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:31b55aa640dd3e73eef0f3de1756e13daf322d64742d017d70e9bb21497194f1 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001648416_421994496.pth b/checkpoint_p1/milestones/checkpoint_001648416_421994496.pth new file mode 100644 index 0000000000000000000000000000000000000000..b33d853909152af24aa3659fcce29d8390863eb6 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001648416_421994496.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:babcbcae91597931767d3cf5807a47fc21f738d052662d300fb99d9ac4d66e20 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001660640_425123840.pth b/checkpoint_p1/milestones/checkpoint_001660640_425123840.pth new file mode 100644 index 0000000000000000000000000000000000000000..61da03948e77c7559927b7b697f3ba67d8c9074e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001660640_425123840.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d52683a85d0e9121e2b6dbebd2ad44f3720a057b69691ec6a483db44573ad247 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001672800_428236800.pth b/checkpoint_p1/milestones/checkpoint_001672800_428236800.pth new file mode 100644 index 0000000000000000000000000000000000000000..acae761afbd732f804407dc12806b16e78d28937 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001672800_428236800.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:666aadb71f0489cfd3fa5ce97e083a88d118ca4b990ffe0dba2a04bf072f69e1 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001685088_431382528.pth b/checkpoint_p1/milestones/checkpoint_001685088_431382528.pth new file mode 100644 index 0000000000000000000000000000000000000000..e8da692a2c6eca37d19dd1ccec5656003b26d8f7 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001685088_431382528.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1e92140db9aae0ac3d0033ee9434e30d8d22afdf27f43ce54ee124669fa388fe +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001697280_434503680.pth b/checkpoint_p1/milestones/checkpoint_001697280_434503680.pth new file mode 100644 index 0000000000000000000000000000000000000000..b3bf1cee1c93ae61924f83cd14dd88e9b76202a4 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001697280_434503680.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c05b3641481c97f10a3b00ebf93dd6b52c8cfd23195d8fb41a60ee9cf213e34 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001709568_437649408.pth b/checkpoint_p1/milestones/checkpoint_001709568_437649408.pth new file mode 100644 index 0000000000000000000000000000000000000000..3a80873fdc712306c338ec38fd63bb0af77793db --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001709568_437649408.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:74f346f69b367369283b21509e62cd96fd1e435d3872af94749ef7fb3c81f3d3 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001721888_440803328.pth b/checkpoint_p1/milestones/checkpoint_001721888_440803328.pth new file mode 100644 index 0000000000000000000000000000000000000000..cdc37d65e04b7563c1803122d7f46d99cc0c64ec --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001721888_440803328.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dbc52fa9f0375b3eea081b8263bf3c4fe050095abb590669263c34f934a91baa +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001734112_443932672.pth b/checkpoint_p1/milestones/checkpoint_001734112_443932672.pth new file mode 100644 index 0000000000000000000000000000000000000000..2e04b44dbf2bbc2d35b3b18935ead09aa595e58f --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001734112_443932672.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a77e0b16c4722e39e0095349982e3f63b6f635b6207d0f223b88bf555ec9b471 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001746272_447045632.pth b/checkpoint_p1/milestones/checkpoint_001746272_447045632.pth new file mode 100644 index 0000000000000000000000000000000000000000..5d7b706bc56f0c7659107ff173645aec2978e87d --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001746272_447045632.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a58aecadae8d69035807b0ad6fc23394da0aa1cd4278b8353793d6b369bf99c0 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001758592_450199552.pth b/checkpoint_p1/milestones/checkpoint_001758592_450199552.pth new file mode 100644 index 0000000000000000000000000000000000000000..996ee85a6658af34bb3eb2b29e22e3beeac5d8fb --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001758592_450199552.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:247d1c2244d4b5ea03169c935dbc7fa5d11453030252e909ac09efac3a975e8f +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001770816_453328896.pth b/checkpoint_p1/milestones/checkpoint_001770816_453328896.pth new file mode 100644 index 0000000000000000000000000000000000000000..b12e93e1dac93ee41aaa789e13e71f35667b4364 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001770816_453328896.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:850c7e7f72951547a7e6e27e7197b22550699684b099f21b9702e1a8f469bf17 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001783040_456458240.pth b/checkpoint_p1/milestones/checkpoint_001783040_456458240.pth new file mode 100644 index 0000000000000000000000000000000000000000..06fcc11b1abf14db88585aba7b41177901eafb00 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001783040_456458240.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:50da5dcf58bdcc97a8a74229c653f89423eec7ed9f5c2de6a468392be97d414e +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001795232_459579392.pth b/checkpoint_p1/milestones/checkpoint_001795232_459579392.pth new file mode 100644 index 0000000000000000000000000000000000000000..4fcc16bbe71e8a2c9365744a0cf595e3b2513cb1 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001795232_459579392.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc518998e729be3b8db1ab7127d5ad915a40823bfafe38914580ba207b6d5e78 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001807456_462708736.pth b/checkpoint_p1/milestones/checkpoint_001807456_462708736.pth new file mode 100644 index 0000000000000000000000000000000000000000..069e8afd8a46568f3ba54f5a1270048dc65f699b --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001807456_462708736.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3f36aa9632bdbdab8b3163209e780f508521422f231dfefca2125527d0cb7672 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001819744_465854464.pth b/checkpoint_p1/milestones/checkpoint_001819744_465854464.pth new file mode 100644 index 0000000000000000000000000000000000000000..f4710c923b105532a7ad85daced7b24e161c0385 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001819744_465854464.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a2d8d89e0242748713beb9d568362ccd5738a2b86f2612a95a076bdf11056a5b +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001831744_468926464.pth b/checkpoint_p1/milestones/checkpoint_001831744_468926464.pth new file mode 100644 index 0000000000000000000000000000000000000000..56c87e7791a9b4a36a69ae81adccdff650a1768a --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001831744_468926464.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a744b03906aa95674f27425168220aedb7602243af0a91d4bce2a23cd1e0e666 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001843936_472047616.pth b/checkpoint_p1/milestones/checkpoint_001843936_472047616.pth new file mode 100644 index 0000000000000000000000000000000000000000..4f866b65dbdbb21b84c6c6368e1292fa215932fc --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001843936_472047616.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4eeaf715c590b03368afc34d2e98be7e2d2d4076964852f7dc75ed9de256b580 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001856192_475185152.pth b/checkpoint_p1/milestones/checkpoint_001856192_475185152.pth new file mode 100644 index 0000000000000000000000000000000000000000..8611094aa2669115bba92642c858024f8f23377e --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001856192_475185152.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:529141bda3213c8ebec29028d36f46c7419304fabcccf0b3fa9c573e27997a3d +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001868448_478322688.pth b/checkpoint_p1/milestones/checkpoint_001868448_478322688.pth new file mode 100644 index 0000000000000000000000000000000000000000..b361541b8aff7e974cf3ae719a194b2424e3475b --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001868448_478322688.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ace6b9355dfa6008a50d9fe7c9af8bc09f4d6cb50cfedf3819b7a27e75527c2d +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001880608_481435648.pth b/checkpoint_p1/milestones/checkpoint_001880608_481435648.pth new file mode 100644 index 0000000000000000000000000000000000000000..8e2183abecbf1ffabecf7fa26f1f96c4725e7067 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001880608_481435648.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7fa275a533a05f6a81c7f02688c67da96e2d4c14fe815c5e83cb21f10e151aa0 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001892736_484540416.pth b/checkpoint_p1/milestones/checkpoint_001892736_484540416.pth new file mode 100644 index 0000000000000000000000000000000000000000..daa8ba8c2a2372ebc781825990363bab35a83b05 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001892736_484540416.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:faf81e97272e88ec855b4c058fa9cc15563b67745a014f00f6da18f52d3c5958 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001904896_487653376.pth b/checkpoint_p1/milestones/checkpoint_001904896_487653376.pth new file mode 100644 index 0000000000000000000000000000000000000000..f913d042838d890fee7f9e650effaa788e985f56 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001904896_487653376.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:49c94ec34ebd34ea8aeb5453abdb870fb1c130f6f34fc2bc651ba1465c80ff84 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001917056_490766336.pth b/checkpoint_p1/milestones/checkpoint_001917056_490766336.pth new file mode 100644 index 0000000000000000000000000000000000000000..4376a92b5a12863879df75099f8a44505abecda3 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001917056_490766336.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4c4fdf79f3c0f2ab25f318b60768ef11496980c0d35ce998d01951b1a507368a +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001929024_493830144.pth b/checkpoint_p1/milestones/checkpoint_001929024_493830144.pth new file mode 100644 index 0000000000000000000000000000000000000000..f4364d83fd6fa07d5f6f6593ea99ce4b8af29a16 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001929024_493830144.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:85c319ad29168e96f2934e319ee6725370317e14aafdeb3f2d38c367cb9199f0 +size 20797067 diff --git a/checkpoint_p1/milestones/checkpoint_001941120_496926720.pth b/checkpoint_p1/milestones/checkpoint_001941120_496926720.pth new file mode 100644 index 0000000000000000000000000000000000000000..e3bfe058bb14213d625b5c24db131f856ddf55a8 --- /dev/null +++ b/checkpoint_p1/milestones/checkpoint_001941120_496926720.pth @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6fd25c3b1279ff52fa6251f7539ca0739359daf023235ed656de07a189a0f545 +size 20797067 diff --git a/config.json b/config.json index 7d086928572f1fdf39848ce60c0df93918d89766..bd02d227adee74c96d0c5dc8ef95a344c3b57c6e 100644 --- a/config.json +++ b/config.json @@ -4,7 +4,7 @@ "env": "atari_riverraid", "experiment": "atari_riverraid_APPO", "train_dir": "./train_atari", - "restart_behavior": "restart", + "restart_behavior": "resume", "device": "gpu", "seed": 1234, "num_policies": 2, @@ -12,11 +12,11 @@ "serial_mode": false, "batched_sampling": true, "num_batches_to_accumulate": 2, - "worker_num_splits": 1, + "worker_num_splits": 2, "policy_workers_per_policy": 1, "max_policy_lag": 1000, "num_workers": 16, - "num_envs_per_worker": 2, + "num_envs_per_worker": 8, "batch_size": 1024, "num_batches_per_epoch": 8, "num_epochs": 4, @@ -64,10 +64,10 @@ "experiment_summaries_interval": 3, "flush_summaries_interval": 30, "stats_avg": 100, - "summaries_use_frameskip": true, + "summaries_use_frameskip": false, "heartbeat_interval": 10, "heartbeat_reporting_interval": 60, - "train_for_env_steps": 100000000, + "train_for_env_steps": 500000000, "train_for_seconds": 10000000000, "save_every_sec": 120, "keep_checkpoints": 2, @@ -124,28 +124,30 @@ "pbt_target_objective": "true_objective", "pbt_perturb_min": 1.1, "pbt_perturb_max": 1.5, - "command_line": "--algo=APPO --env=atari_riverraid --experiment=atari_riverraid_APPO --num_policies=2 --restart_behavior=restart --train_dir=./train_atari --train_for_env_steps=100000000 --seed=1234 --num_workers=16 --num_envs_per_worker=2 --num_batches_per_epoch=8 --async_rl=true --batched_sampling=true --batch_size=1024 --max_grad_norm=0 --learning_rate=0.0003033891184 --heartbeat_interval=10 --heartbeat_reporting_interval=60 --save_milestones_sec=1200 --num_epochs=4 --exploration_loss_coeff=0.0004677351413 --with_wandb=true --wandb_user=matt-stammers --wandb_project=atari_APPO --wandb_group=atari_riverraid --wandb_job_type=SF --wandb_tags=atari", + "command_line": "--algo=APPO --env=atari_riverraid --experiment=atari_riverraid_APPO --num_policies=2 --restart_behavior=resume --train_dir=./train_atari --train_for_env_steps=500000000 --seed=1234 --num_workers=16 --num_envs_per_worker=8 --num_batches_per_epoch=8 --worker_num_splits=2 --async_rl=true --batched_sampling=true --batch_size=1024 --max_grad_norm=0 --learning_rate=0.0003033891184 --heartbeat_interval=10 --heartbeat_reporting_interval=60 --save_milestones_sec=1200 --num_epochs=4 --exploration_loss_coeff=0.0004677351413 --summaries_use_frameskip=False --with_wandb=true --wandb_user=matt-stammers --wandb_project=atari_APPO --wandb_group=atari_riverraid --wandb_job_type=SF --wandb_tags=atari", "cli_args": { "algo": "APPO", "env": "atari_riverraid", "experiment": "atari_riverraid_APPO", "train_dir": "./train_atari", - "restart_behavior": "restart", + "restart_behavior": "resume", "seed": 1234, "num_policies": 2, "async_rl": true, "batched_sampling": true, + "worker_num_splits": 2, "num_workers": 16, - "num_envs_per_worker": 2, + "num_envs_per_worker": 8, "batch_size": 1024, "num_batches_per_epoch": 8, "num_epochs": 4, "exploration_loss_coeff": 0.0004677351413, "max_grad_norm": 0.0, "learning_rate": 0.0003033891184, + "summaries_use_frameskip": false, "heartbeat_interval": 10, "heartbeat_reporting_interval": 60, - "train_for_env_steps": 100000000, + "train_for_env_steps": 500000000, "save_milestones_sec": 1200, "with_wandb": true, "wandb_user": "matt-stammers", @@ -158,5 +160,5 @@ }, "git_hash": "5fff97c2f535da5987d358cdbe6927cccd43621e", "git_repo_name": "not a git repository", - "wandb_unique_id": "atari_riverraid_APPO_20231014_132240_388669" + "wandb_unique_id": "atari_riverraid_APPO_20231121_062947_031688" } \ No newline at end of file diff --git a/git.diff b/git.diff index 960bf7b013feefe7b56842bffdcf222f0bdf7dbd..f2014ff0d08b4ad19d4c267f4668e0df6f312c93 100644 --- a/git.diff +++ b/git.diff @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:3357904f421d3f4924836316b1741bf64d5dd0e807d5e80ac07059b4c52a7008 -size 14426734 +oid sha256:de4fecb91705490b8f6f89418f0c59ae52b7bc523a512f22d64b0d2006864d31 +size 380928 diff --git a/replay.mp4 b/replay.mp4 index 93ae97f675ecb183b105a82c07a3a59439b63d9c..4d4663cdda5b2e75aae4a7414f3b36991847551d 100644 --- a/replay.mp4 +++ b/replay.mp4 @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:65401be6a06fe8952a9d5417b17751be9f8879860beb4ca1bdff96da4f0ac9b6 -size 3294333 +oid sha256:a6e96792dc7f94f34bda1b3449302f8f79f8c21682b33a1a766047c9bcc04881 +size 16611995 diff --git a/sf_log.txt b/sf_log.txt index d276c26a10b6d68e13b32edbbc0b1b924c7f3f1f..940ed6688df0afedb079d696422c08e09c672a00 100644 --- a/sf_log.txt +++ b/sf_log.txt @@ -1,26615 +1,3 @@ -[2023-10-14 13:22:47,072][74987] Saving configuration to ./train_atari/atari_riverraid_APPO/config.json... -[2023-10-14 13:22:47,389][74987] Rollout worker 0 uses device cpu -[2023-10-14 13:22:47,390][74987] Rollout worker 1 uses device cpu -[2023-10-14 13:22:47,391][74987] Rollout worker 2 uses device cpu -[2023-10-14 13:22:47,391][74987] Rollout worker 3 uses device cpu -[2023-10-14 13:22:47,392][74987] Rollout worker 4 uses device cpu -[2023-10-14 13:22:47,392][74987] Rollout worker 5 uses device cpu -[2023-10-14 13:22:47,392][74987] Rollout worker 6 uses device cpu -[2023-10-14 13:22:47,393][74987] Rollout worker 7 uses device cpu -[2023-10-14 13:22:47,393][74987] Rollout worker 8 uses device cpu -[2023-10-14 13:22:47,394][74987] Rollout worker 9 uses device cpu -[2023-10-14 13:22:47,394][74987] Rollout worker 10 uses device cpu -[2023-10-14 13:22:47,395][74987] Rollout worker 11 uses device cpu -[2023-10-14 13:22:47,395][74987] Rollout worker 12 uses device cpu -[2023-10-14 13:22:47,395][74987] Rollout worker 13 uses device cpu -[2023-10-14 13:22:47,396][74987] Rollout worker 14 uses device cpu -[2023-10-14 13:22:47,396][74987] Rollout worker 15 uses device cpu -[2023-10-14 13:22:47,687][74987] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-10-14 13:22:47,688][74987] InferenceWorker_p0-w0: min num requests: 2 -[2023-10-14 13:22:47,691][74987] Using GPUs [1] for process 1 (actually maps to GPUs [1]) -[2023-10-14 13:22:47,691][74987] InferenceWorker_p1-w0: min num requests: 2 -[2023-10-14 13:22:47,742][74987] Starting all processes... -[2023-10-14 13:22:47,742][74987] Starting process learner_proc0 -[2023-10-14 13:22:49,452][74987] Starting process learner_proc1 -[2023-10-14 13:22:49,455][75615] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-10-14 13:22:49,455][75615] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 -[2023-10-14 13:22:49,473][75615] Num visible devices: 1 -[2023-10-14 13:22:49,489][75615] Setting fixed seed 1234 -[2023-10-14 13:22:49,490][75615] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-10-14 13:22:49,490][75615] Initializing actor-critic model on device cuda:0 -[2023-10-14 13:22:49,491][75615] RunningMeanStd input shape: (4, 84, 84) -[2023-10-14 13:22:49,491][75615] RunningMeanStd input shape: (1,) -[2023-10-14 13:22:49,508][75615] ConvEncoder: input_channels=4 -[2023-10-14 13:22:49,683][75615] Conv encoder output size: 512 -[2023-10-14 13:22:49,685][75615] Created Actor Critic model with architecture: -[2023-10-14 13:22:49,686][75615] ActorCriticSharedWeights( - (obs_normalizer): ObservationNormalizer( - (running_mean_std): RunningMeanStdDictInPlace( - (running_mean_std): ModuleDict( - (obs): RunningMeanStdInPlace() - ) - ) - ) - (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) - (encoder): MultiInputEncoder( - (encoders): ModuleDict( - (obs): ConvEncoder( - (enc): RecursiveScriptModule( - original_name=ConvEncoderImpl - (conv_head): RecursiveScriptModule( - original_name=Sequential - (0): RecursiveScriptModule(original_name=Conv2d) - (1): RecursiveScriptModule(original_name=ReLU) - (2): RecursiveScriptModule(original_name=Conv2d) - (3): RecursiveScriptModule(original_name=ReLU) - (4): RecursiveScriptModule(original_name=Conv2d) - (5): RecursiveScriptModule(original_name=ReLU) - ) - (mlp_layers): RecursiveScriptModule( - original_name=Sequential - (0): RecursiveScriptModule(original_name=Linear) - (1): RecursiveScriptModule(original_name=ReLU) - ) - ) - ) - ) - ) - (core): ModelCoreIdentity() - (decoder): MlpDecoder( - (mlp): Identity() - ) - (critic_linear): Linear(in_features=512, out_features=1, bias=True) - (action_parameterization): ActionParameterizationDefault( - (distribution_linear): Linear(in_features=512, out_features=18, bias=True) - ) -) -[2023-10-14 13:22:50,276][75615] Using optimizer -[2023-10-14 13:22:50,277][75615] No checkpoints found -[2023-10-14 13:22:50,277][75615] Did not load from checkpoint, starting from scratch! -[2023-10-14 13:22:50,277][75615] Initialized policy 0 weights for model version 0 -[2023-10-14 13:22:50,279][75615] LearnerWorker_p0 finished initialization! -[2023-10-14 13:22:50,279][75615] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-10-14 13:22:51,185][74987] Starting all processes... -[2023-10-14 13:22:51,188][75801] Using GPUs [1] for process 1 (actually maps to GPUs [1]) -[2023-10-14 13:22:51,188][75801] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 -[2023-10-14 13:22:51,193][74987] Starting process inference_proc0-0 -[2023-10-14 13:22:51,193][74987] Starting process inference_proc1-0 -[2023-10-14 13:22:51,194][74987] Starting process rollout_proc0 -[2023-10-14 13:22:51,207][75801] Num visible devices: 1 -[2023-10-14 13:22:51,194][74987] Starting process rollout_proc1 -[2023-10-14 13:22:51,194][74987] Starting process rollout_proc2 -[2023-10-14 13:22:51,194][74987] Starting process rollout_proc3 -[2023-10-14 13:22:51,194][74987] Starting process rollout_proc4 -[2023-10-14 13:22:51,225][75801] Setting fixed seed 1234 -[2023-10-14 13:22:51,226][75801] Using GPUs [0] for process 1 (actually maps to GPUs [1]) -[2023-10-14 13:22:51,226][75801] Initializing actor-critic model on device cuda:0 -[2023-10-14 13:22:51,227][75801] RunningMeanStd input shape: (4, 84, 84) -[2023-10-14 13:22:51,195][74987] Starting process rollout_proc5 -[2023-10-14 13:22:51,228][75801] RunningMeanStd input shape: (1,) -[2023-10-14 13:22:51,197][74987] Starting process rollout_proc6 -[2023-10-14 13:22:51,200][74987] Starting process rollout_proc7 -[2023-10-14 13:22:51,204][74987] Starting process rollout_proc8 -[2023-10-14 13:22:51,241][75801] ConvEncoder: input_channels=4 -[2023-10-14 13:22:51,210][74987] Starting process rollout_proc9 -[2023-10-14 13:22:51,211][74987] Starting process rollout_proc10 -[2023-10-14 13:22:51,211][74987] Starting process rollout_proc11 -[2023-10-14 13:22:51,211][74987] Starting process rollout_proc12 -[2023-10-14 13:22:51,212][74987] Starting process rollout_proc13 -[2023-10-14 13:22:51,475][75801] Conv encoder output size: 512 -[2023-10-14 13:22:51,490][75801] Created Actor Critic model with architecture: -[2023-10-14 13:22:51,492][75801] ActorCriticSharedWeights( - (obs_normalizer): ObservationNormalizer( - (running_mean_std): RunningMeanStdDictInPlace( - (running_mean_std): ModuleDict( - (obs): RunningMeanStdInPlace() - ) - ) - ) - (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) - (encoder): MultiInputEncoder( - (encoders): ModuleDict( - (obs): ConvEncoder( - (enc): RecursiveScriptModule( - original_name=ConvEncoderImpl - (conv_head): RecursiveScriptModule( - original_name=Sequential - (0): RecursiveScriptModule(original_name=Conv2d) - (1): RecursiveScriptModule(original_name=ReLU) - (2): RecursiveScriptModule(original_name=Conv2d) - (3): RecursiveScriptModule(original_name=ReLU) - (4): RecursiveScriptModule(original_name=Conv2d) - (5): RecursiveScriptModule(original_name=ReLU) - ) - (mlp_layers): RecursiveScriptModule( - original_name=Sequential - (0): RecursiveScriptModule(original_name=Linear) - (1): RecursiveScriptModule(original_name=ReLU) - ) - ) - ) - ) - ) - (core): ModelCoreIdentity() - (decoder): MlpDecoder( - (mlp): Identity() - ) - (critic_linear): Linear(in_features=512, out_features=1, bias=True) - (action_parameterization): ActionParameterizationDefault( - (distribution_linear): Linear(in_features=512, out_features=18, bias=True) - ) -) -[2023-10-14 13:22:52,278][75801] Using optimizer -[2023-10-14 13:22:52,279][75801] No checkpoints found -[2023-10-14 13:22:52,279][75801] Did not load from checkpoint, starting from scratch! -[2023-10-14 13:22:52,279][75801] Initialized policy 1 weights for model version 0 -[2023-10-14 13:22:52,281][75801] LearnerWorker_p1 finished initialization! -[2023-10-14 13:22:52,281][75801] Using GPUs [0] for process 1 (actually maps to GPUs [1]) -[2023-10-14 13:22:53,396][74987] Starting process rollout_proc14 -[2023-10-14 13:22:53,405][76012] Worker 12 uses CPU cores [24, 25] -[2023-10-14 13:22:53,408][74987] Starting process rollout_proc15 -[2023-10-14 13:22:53,413][76011] Worker 11 uses CPU cores [22, 23] -[2023-10-14 13:22:53,445][76005] Worker 6 uses CPU cores [12, 13] -[2023-10-14 13:22:53,609][76010] Worker 10 uses CPU cores [20, 21] -[2023-10-14 13:22:53,613][75984] Worker 2 uses CPU cores [4, 5] -[2023-10-14 13:22:53,624][75983] Worker 0 uses CPU cores [0, 1] -[2023-10-14 13:22:53,643][75987] Worker 1 uses CPU cores [2, 3] -[2023-10-14 13:22:53,701][75993] Worker 5 uses CPU cores [10, 11] -[2023-10-14 13:22:53,711][76006] Worker 7 uses CPU cores [14, 15] -[2023-10-14 13:22:53,720][75950] Using GPUs [1] for process 1 (actually maps to GPUs [1]) -[2023-10-14 13:22:53,720][75950] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 -[2023-10-14 13:22:53,740][75950] Num visible devices: 1 -[2023-10-14 13:22:53,824][75949] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-10-14 13:22:53,824][75949] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 -[2023-10-14 13:22:53,849][75949] Num visible devices: 1 -[2023-10-14 13:22:53,913][76008] Worker 9 uses CPU cores [18, 19] -[2023-10-14 13:22:53,953][76013] Worker 13 uses CPU cores [26, 27] -[2023-10-14 13:22:54,004][75996] Worker 3 uses CPU cores [6, 7] -[2023-10-14 13:22:54,088][75994] Worker 4 uses CPU cores [8, 9] -[2023-10-14 13:22:54,205][76007] Worker 8 uses CPU cores [16, 17] -[2023-10-14 13:22:54,444][75950] RunningMeanStd input shape: (4, 84, 84) -[2023-10-14 13:22:54,445][75950] RunningMeanStd input shape: (1,) -[2023-10-14 13:22:54,457][75950] ConvEncoder: input_channels=4 -[2023-10-14 13:22:54,489][75949] RunningMeanStd input shape: (4, 84, 84) -[2023-10-14 13:22:54,489][75949] RunningMeanStd input shape: (1,) -[2023-10-14 13:22:54,501][75949] ConvEncoder: input_channels=4 -[2023-10-14 13:22:54,559][75950] Conv encoder output size: 512 -[2023-10-14 13:22:54,603][75949] Conv encoder output size: 512 -[2023-10-14 13:22:55,302][76659] Worker 15 uses CPU cores [30, 31] -[2023-10-14 13:22:55,318][74987] Inference worker 1-0 is ready! -[2023-10-14 13:22:55,319][74987] Inference worker 0-0 is ready! -[2023-10-14 13:22:55,319][74987] All inference workers are ready! Signal rollout workers to start! -[2023-10-14 13:22:55,320][76006] EnvRunner 7-0 uses policy 1 -[2023-10-14 13:22:55,320][74987] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-10-14 13:22:55,321][75984] EnvRunner 2-0 uses policy 0 -[2023-10-14 13:22:55,321][75983] EnvRunner 0-0 uses policy 0 -[2023-10-14 13:22:55,321][76007] EnvRunner 8-0 uses policy 0 -[2023-10-14 13:22:55,321][76005] EnvRunner 6-0 uses policy 0 -[2023-10-14 13:22:55,321][75987] EnvRunner 1-0 uses policy 1 -[2023-10-14 13:22:55,321][75996] EnvRunner 3-0 uses policy 1 -[2023-10-14 13:22:55,321][76010] EnvRunner 10-0 uses policy 0 -[2023-10-14 13:22:55,321][76013] EnvRunner 13-0 uses policy 1 -[2023-10-14 13:22:55,321][76011] EnvRunner 11-0 uses policy 1 -[2023-10-14 13:22:55,321][76008] EnvRunner 9-0 uses policy 1 -[2023-10-14 13:22:55,321][75993] EnvRunner 5-0 uses policy 1 -[2023-10-14 13:22:55,321][75994] EnvRunner 4-0 uses policy 0 -[2023-10-14 13:22:55,321][76012] EnvRunner 12-0 uses policy 0 -[2023-10-14 13:22:55,321][76627] Worker 14 uses CPU cores [28, 29] -[2023-10-14 13:22:55,430][76627] EnvRunner 14-0 uses policy 0 -[2023-10-14 13:22:55,494][76659] EnvRunner 15-0 uses policy 1 -[2023-10-14 13:22:57,675][74987] Heartbeat connected on Batcher_0 -[2023-10-14 13:22:57,678][74987] Heartbeat connected on LearnerWorker_p0 -[2023-10-14 13:22:57,681][74987] Heartbeat connected on Batcher_1 -[2023-10-14 13:22:57,684][74987] Heartbeat connected on LearnerWorker_p1 -[2023-10-14 13:22:57,693][74987] Heartbeat connected on InferenceWorker_p1-w0 -[2023-10-14 13:22:57,694][74987] Heartbeat connected on InferenceWorker_p0-w0 -[2023-10-14 13:22:57,699][74987] Heartbeat connected on RolloutWorker_w0 -[2023-10-14 13:22:57,701][74987] Heartbeat connected on RolloutWorker_w2 -[2023-10-14 13:22:57,701][74987] Heartbeat connected on RolloutWorker_w1 -[2023-10-14 13:22:57,706][74987] Heartbeat connected on RolloutWorker_w4 -[2023-10-14 13:22:57,707][74987] Heartbeat connected on RolloutWorker_w3 -[2023-10-14 13:22:57,710][74987] Heartbeat connected on RolloutWorker_w5 -[2023-10-14 13:22:57,713][74987] Heartbeat connected on RolloutWorker_w6 -[2023-10-14 13:22:57,716][74987] Heartbeat connected on RolloutWorker_w7 -[2023-10-14 13:22:57,723][74987] Heartbeat connected on RolloutWorker_w9 -[2023-10-14 13:22:57,723][74987] Heartbeat connected on RolloutWorker_w8 -[2023-10-14 13:22:57,728][74987] Heartbeat connected on RolloutWorker_w10 -[2023-10-14 13:22:57,732][74987] Heartbeat connected on RolloutWorker_w11 -[2023-10-14 13:22:57,733][74987] Heartbeat connected on RolloutWorker_w12 -[2023-10-14 13:22:57,734][74987] Heartbeat connected on RolloutWorker_w13 -[2023-10-14 13:22:57,741][74987] Heartbeat connected on RolloutWorker_w14 -[2023-10-14 13:22:57,744][74987] Heartbeat connected on RolloutWorker_w15 -[2023-10-14 13:22:58,163][74987] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 519.9, 1: 435.5. Samples: 2716. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-10-14 13:22:58,164][74987] Avg episode reward: [(0, '2.000'), (1, '3.000')] -[2023-10-14 13:23:03,163][74987] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 897.1, 1: 912.9. Samples: 14196. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-10-14 13:23:03,164][74987] Avg episode reward: [(0, '5.385'), (1, '5.140')] -[2023-10-14 13:23:05,559][75950] Updated weights for policy 1, policy_version 10 (0.0010) -[2023-10-14 13:23:05,859][75949] Updated weights for policy 0, policy_version 10 (0.0008) -[2023-10-14 13:23:05,913][75950] Updated weights for policy 1, policy_version 20 (0.0008) -[2023-10-14 13:23:06,233][75949] Updated weights for policy 0, policy_version 20 (0.0007) -[2023-10-14 13:23:06,273][75950] Updated weights for policy 1, policy_version 30 (0.0009) -[2023-10-14 13:23:06,605][75949] Updated weights for policy 0, policy_version 30 (0.0009) -[2023-10-14 13:23:08,164][74987] Fps is (10 sec: 6553.5, 60 sec: 5102.8, 300 sec: 5102.8). Total num frames: 65536. Throughput: 0: 1189.1, 1: 1193.6. Samples: 30602. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 13:23:08,164][74987] Avg episode reward: [(0, '5.458'), (1, '5.263')] -[2023-10-14 13:23:08,860][75949] Updated weights for policy 0, policy_version 40 (0.0009) -[2023-10-14 13:23:08,957][75950] Updated weights for policy 1, policy_version 40 (0.0008) -[2023-10-14 13:23:09,228][75949] Updated weights for policy 0, policy_version 50 (0.0009) -[2023-10-14 13:23:09,333][75950] Updated weights for policy 1, policy_version 50 (0.0009) -[2023-10-14 13:23:09,594][75949] Updated weights for policy 0, policy_version 60 (0.0009) -[2023-10-14 13:23:09,692][75950] Updated weights for policy 1, policy_version 60 (0.0008) -[2023-10-14 13:23:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 7345.9, 300 sec: 7345.9). Total num frames: 131072. Throughput: 0: 1420.2, 1: 1433.5. Samples: 50918. Policy #0 lag: (min: 33.0, avg: 33.0, max: 33.0) -[2023-10-14 13:23:13,164][74987] Avg episode reward: [(0, '5.620'), (1, '5.400')] -[2023-10-14 13:23:13,219][75949] Updated weights for policy 0, policy_version 70 (0.0009) -[2023-10-14 13:23:13,290][75950] Updated weights for policy 1, policy_version 70 (0.0008) -[2023-10-14 13:23:13,586][75949] Updated weights for policy 0, policy_version 80 (0.0007) -[2023-10-14 13:23:13,661][75950] Updated weights for policy 1, policy_version 80 (0.0007) -[2023-10-14 13:23:13,946][75949] Updated weights for policy 0, policy_version 90 (0.0007) -[2023-10-14 13:23:14,023][75950] Updated weights for policy 1, policy_version 90 (0.0008) -[2023-10-14 13:23:17,565][75949] Updated weights for policy 0, policy_version 100 (0.0007) -[2023-10-14 13:23:17,635][75950] Updated weights for policy 1, policy_version 100 (0.0007) -[2023-10-14 13:23:17,938][75949] Updated weights for policy 0, policy_version 110 (0.0010) -[2023-10-14 13:23:17,993][75950] Updated weights for policy 1, policy_version 110 (0.0007) -[2023-10-14 13:23:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 8606.9, 300 sec: 8606.9). Total num frames: 196608. Throughput: 0: 1316.8, 1: 1322.4. Samples: 60288. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 13:23:18,164][74987] Avg episode reward: [(0, '5.390'), (1, '5.230')] -[2023-10-14 13:23:18,310][75949] Updated weights for policy 0, policy_version 120 (0.0008) -[2023-10-14 13:23:18,359][75950] Updated weights for policy 1, policy_version 120 (0.0007) -[2023-10-14 13:23:22,276][75949] Updated weights for policy 0, policy_version 130 (0.0008) -[2023-10-14 13:23:22,576][75950] Updated weights for policy 1, policy_version 130 (0.0007) -[2023-10-14 13:23:22,646][75949] Updated weights for policy 0, policy_version 140 (0.0009) -[2023-10-14 13:23:22,937][75950] Updated weights for policy 1, policy_version 140 (0.0007) -[2023-10-14 13:23:23,025][75949] Updated weights for policy 0, policy_version 150 (0.0008) -[2023-10-14 13:23:23,163][74987] Fps is (10 sec: 13107.2, 60 sec: 9415.1, 300 sec: 9415.1). Total num frames: 262144. Throughput: 0: 1449.6, 1: 1455.5. Samples: 80888. Policy #0 lag: (min: 22.0, avg: 29.1, max: 54.0) -[2023-10-14 13:23:23,164][74987] Avg episode reward: [(0, '5.750'), (1, '5.180')] -[2023-10-14 13:23:23,295][75950] Updated weights for policy 1, policy_version 150 (0.0007) -[2023-10-14 13:23:23,396][75615] Saving new best policy, reward=5.750! -[2023-10-14 13:23:23,397][75949] Updated weights for policy 0, policy_version 160 (0.0008) -[2023-10-14 13:23:23,658][75801] Saving new best policy, reward=5.180! -[2023-10-14 13:23:23,658][75950] Updated weights for policy 1, policy_version 160 (0.0009) -[2023-10-14 13:23:27,573][75949] Updated weights for policy 0, policy_version 170 (0.0009) -[2023-10-14 13:23:27,807][75950] Updated weights for policy 1, policy_version 170 (0.0008) -[2023-10-14 13:23:27,946][75949] Updated weights for policy 0, policy_version 180 (0.0008) -[2023-10-14 13:23:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 9977.2, 300 sec: 9977.2). Total num frames: 327680. Throughput: 0: 1528.0, 1: 1538.0. Samples: 100698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:23:28,164][74987] Avg episode reward: [(0, '6.110'), (1, '5.150')] -[2023-10-14 13:23:28,169][75950] Updated weights for policy 1, policy_version 180 (0.0008) -[2023-10-14 13:23:28,307][75949] Updated weights for policy 0, policy_version 190 (0.0007) -[2023-10-14 13:23:28,384][75615] Saving new best policy, reward=6.110! -[2023-10-14 13:23:28,529][75950] Updated weights for policy 1, policy_version 190 (0.0010) -[2023-10-14 13:23:32,552][75949] Updated weights for policy 0, policy_version 200 (0.0010) -[2023-10-14 13:23:32,708][75950] Updated weights for policy 1, policy_version 200 (0.0007) -[2023-10-14 13:23:32,926][75949] Updated weights for policy 0, policy_version 210 (0.0008) -[2023-10-14 13:23:33,076][75950] Updated weights for policy 1, policy_version 210 (0.0008) -[2023-10-14 13:23:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 10390.7, 300 sec: 10390.7). Total num frames: 393216. Throughput: 0: 1455.9, 1: 1458.2. Samples: 110276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:23:33,164][74987] Avg episode reward: [(0, '6.500'), (1, '5.430')] -[2023-10-14 13:23:33,299][75949] Updated weights for policy 0, policy_version 220 (0.0008) -[2023-10-14 13:23:33,442][75950] Updated weights for policy 1, policy_version 220 (0.0008) -[2023-10-14 13:23:33,446][75615] Saving new best policy, reward=6.500! -[2023-10-14 13:23:33,588][75801] Saving new best policy, reward=5.430! -[2023-10-14 13:23:37,266][75949] Updated weights for policy 0, policy_version 230 (0.0008) -[2023-10-14 13:23:37,553][75950] Updated weights for policy 1, policy_version 230 (0.0009) -[2023-10-14 13:23:37,624][75949] Updated weights for policy 0, policy_version 240 (0.0007) -[2023-10-14 13:23:37,910][75950] Updated weights for policy 1, policy_version 240 (0.0009) -[2023-10-14 13:23:37,997][75949] Updated weights for policy 0, policy_version 250 (0.0007) -[2023-10-14 13:23:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 10707.7, 300 sec: 10707.7). Total num frames: 458752. Throughput: 0: 1529.7, 1: 1526.0. Samples: 130916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:23:38,164][74987] Avg episode reward: [(0, '6.710'), (1, '5.770')] -[2023-10-14 13:23:38,218][75615] Saving new best policy, reward=6.710! -[2023-10-14 13:23:38,277][75950] Updated weights for policy 1, policy_version 250 (0.0008) -[2023-10-14 13:23:38,497][75801] Saving new best policy, reward=5.770! -[2023-10-14 13:23:42,092][75949] Updated weights for policy 0, policy_version 260 (0.0008) -[2023-10-14 13:23:42,395][75950] Updated weights for policy 1, policy_version 260 (0.0009) -[2023-10-14 13:23:42,458][75949] Updated weights for policy 0, policy_version 270 (0.0009) -[2023-10-14 13:23:42,756][75950] Updated weights for policy 1, policy_version 270 (0.0010) -[2023-10-14 13:23:42,834][75949] Updated weights for policy 0, policy_version 280 (0.0007) -[2023-10-14 13:23:43,129][75950] Updated weights for policy 1, policy_version 280 (0.0009) -[2023-10-14 13:23:43,164][74987] Fps is (10 sec: 16383.7, 60 sec: 11643.4, 300 sec: 11643.4). Total num frames: 557056. Throughput: 0: 1636.8, 1: 1642.6. Samples: 150288. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-14 13:23:43,165][74987] Avg episode reward: [(0, '6.770'), (1, '5.970')] -[2023-10-14 13:23:43,174][75615] Saving new best policy, reward=6.770! -[2023-10-14 13:23:43,415][75801] Saving new best policy, reward=5.970! -[2023-10-14 13:23:46,921][75949] Updated weights for policy 0, policy_version 290 (0.0008) -[2023-10-14 13:23:47,273][75950] Updated weights for policy 1, policy_version 290 (0.0010) -[2023-10-14 13:23:47,332][75949] Updated weights for policy 0, policy_version 300 (0.0008) -[2023-10-14 13:23:47,681][75950] Updated weights for policy 1, policy_version 300 (0.0008) -[2023-10-14 13:23:47,697][75949] Updated weights for policy 0, policy_version 310 (0.0008) -[2023-10-14 13:23:48,037][75950] Updated weights for policy 1, policy_version 310 (0.0008) -[2023-10-14 13:23:48,065][75949] Updated weights for policy 0, policy_version 320 (0.0007) -[2023-10-14 13:23:48,164][74987] Fps is (10 sec: 16383.6, 60 sec: 11781.9, 300 sec: 11781.9). Total num frames: 622592. Throughput: 0: 1634.0, 1: 1620.3. Samples: 160638. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 13:23:48,165][74987] Avg episode reward: [(0, '7.000'), (1, '6.770')] -[2023-10-14 13:23:48,166][75615] Saving new best policy, reward=7.000! -[2023-10-14 13:23:48,407][75801] Saving new best policy, reward=6.770! -[2023-10-14 13:23:48,411][75950] Updated weights for policy 1, policy_version 320 (0.0010) -[2023-10-14 13:23:52,202][75949] Updated weights for policy 0, policy_version 330 (0.0009) -[2023-10-14 13:23:52,529][75950] Updated weights for policy 1, policy_version 330 (0.0010) -[2023-10-14 13:23:52,575][75949] Updated weights for policy 0, policy_version 340 (0.0008) -[2023-10-14 13:23:52,897][75950] Updated weights for policy 1, policy_version 340 (0.0008) -[2023-10-14 13:23:52,937][75949] Updated weights for policy 0, policy_version 350 (0.0008) -[2023-10-14 13:23:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 11896.5, 300 sec: 11896.5). Total num frames: 688128. Throughput: 0: 1676.1, 1: 1665.7. Samples: 180982. Policy #0 lag: (min: 26.0, avg: 26.0, max: 28.0) -[2023-10-14 13:23:53,164][74987] Avg episode reward: [(0, '7.240'), (1, '7.260')] -[2023-10-14 13:23:53,165][75615] Saving new best policy, reward=7.240! -[2023-10-14 13:23:53,257][75950] Updated weights for policy 1, policy_version 350 (0.0010) -[2023-10-14 13:23:53,331][75801] Saving new best policy, reward=7.260! -[2023-10-14 13:23:57,133][75949] Updated weights for policy 0, policy_version 360 (0.0009) -[2023-10-14 13:23:57,336][75950] Updated weights for policy 1, policy_version 360 (0.0008) -[2023-10-14 13:23:57,494][75949] Updated weights for policy 0, policy_version 370 (0.0008) -[2023-10-14 13:23:57,698][75950] Updated weights for policy 1, policy_version 370 (0.0008) -[2023-10-14 13:23:57,863][75949] Updated weights for policy 0, policy_version 380 (0.0009) -[2023-10-14 13:23:58,059][75950] Updated weights for policy 1, policy_version 380 (0.0008) -[2023-10-14 13:23:58,163][74987] Fps is (10 sec: 13107.6, 60 sec: 12561.1, 300 sec: 11992.8). Total num frames: 753664. Throughput: 0: 1661.2, 1: 1655.2. Samples: 200152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:23:58,164][74987] Avg episode reward: [(0, '6.770'), (1, '7.120')] -[2023-10-14 13:24:02,108][75950] Updated weights for policy 1, policy_version 390 (0.0007) -[2023-10-14 13:24:02,153][75949] Updated weights for policy 0, policy_version 390 (0.0008) -[2023-10-14 13:24:02,469][75950] Updated weights for policy 1, policy_version 400 (0.0008) -[2023-10-14 13:24:02,516][75949] Updated weights for policy 0, policy_version 400 (0.0007) -[2023-10-14 13:24:02,843][75950] Updated weights for policy 1, policy_version 410 (0.0007) -[2023-10-14 13:24:02,887][75949] Updated weights for policy 0, policy_version 410 (0.0009) -[2023-10-14 13:24:03,164][74987] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 12557.9). Total num frames: 851968. Throughput: 0: 1674.1, 1: 1666.7. Samples: 210628. Policy #0 lag: (min: 22.0, avg: 23.9, max: 52.0) -[2023-10-14 13:24:03,165][74987] Avg episode reward: [(0, '6.210'), (1, '7.220')] -[2023-10-14 13:24:06,868][75949] Updated weights for policy 0, policy_version 420 (0.0008) -[2023-10-14 13:24:07,032][75950] Updated weights for policy 1, policy_version 420 (0.0007) -[2023-10-14 13:24:07,228][75949] Updated weights for policy 0, policy_version 430 (0.0008) -[2023-10-14 13:24:07,392][75950] Updated weights for policy 1, policy_version 430 (0.0008) -[2023-10-14 13:24:07,599][75949] Updated weights for policy 0, policy_version 440 (0.0007) -[2023-10-14 13:24:07,749][75950] Updated weights for policy 1, policy_version 440 (0.0008) -[2023-10-14 13:24:08,164][74987] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 12595.6). Total num frames: 917504. Throughput: 0: 1673.2, 1: 1663.7. Samples: 231048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:24:08,165][74987] Avg episode reward: [(0, '7.230'), (1, '7.000')] -[2023-10-14 13:24:11,844][75949] Updated weights for policy 0, policy_version 450 (0.0009) -[2023-10-14 13:24:11,848][75950] Updated weights for policy 1, policy_version 450 (0.0007) -[2023-10-14 13:24:12,217][75949] Updated weights for policy 0, policy_version 460 (0.0007) -[2023-10-14 13:24:12,219][75950] Updated weights for policy 1, policy_version 460 (0.0010) -[2023-10-14 13:24:12,588][75949] Updated weights for policy 0, policy_version 470 (0.0008) -[2023-10-14 13:24:12,589][75950] Updated weights for policy 1, policy_version 470 (0.0009) -[2023-10-14 13:24:12,959][75950] Updated weights for policy 1, policy_version 480 (0.0008) -[2023-10-14 13:24:12,961][75949] Updated weights for policy 0, policy_version 480 (0.0008) -[2023-10-14 13:24:13,163][74987] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 12628.5). Total num frames: 983040. Throughput: 0: 1660.7, 1: 1650.0. Samples: 249678. Policy #0 lag: (min: 1.0, avg: 1.2, max: 10.0) -[2023-10-14 13:24:13,164][74987] Avg episode reward: [(0, '7.380'), (1, '6.530')] -[2023-10-14 13:24:13,174][75615] Saving new best policy, reward=7.380! -[2023-10-14 13:24:17,031][75950] Updated weights for policy 1, policy_version 490 (0.0010) -[2023-10-14 13:24:17,133][75949] Updated weights for policy 0, policy_version 490 (0.0007) -[2023-10-14 13:24:17,401][75950] Updated weights for policy 1, policy_version 500 (0.0007) -[2023-10-14 13:24:17,505][75949] Updated weights for policy 0, policy_version 500 (0.0008) -[2023-10-14 13:24:17,758][75950] Updated weights for policy 1, policy_version 510 (0.0008) -[2023-10-14 13:24:17,876][75949] Updated weights for policy 0, policy_version 510 (0.0008) -[2023-10-14 13:24:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 12657.4). Total num frames: 1048576. Throughput: 0: 1673.9, 1: 1662.7. Samples: 260422. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-14 13:24:18,165][74987] Avg episode reward: [(0, '6.850'), (1, '6.550')] -[2023-10-14 13:24:21,949][75950] Updated weights for policy 1, policy_version 520 (0.0008) -[2023-10-14 13:24:21,966][75949] Updated weights for policy 0, policy_version 520 (0.0009) -[2023-10-14 13:24:22,308][75950] Updated weights for policy 1, policy_version 530 (0.0007) -[2023-10-14 13:24:22,331][75949] Updated weights for policy 0, policy_version 530 (0.0008) -[2023-10-14 13:24:22,671][75950] Updated weights for policy 1, policy_version 540 (0.0008) -[2023-10-14 13:24:22,706][75949] Updated weights for policy 0, policy_version 540 (0.0007) -[2023-10-14 13:24:23,163][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 12683.0). Total num frames: 1114112. Throughput: 0: 1668.2, 1: 1660.8. Samples: 280724. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) -[2023-10-14 13:24:23,164][74987] Avg episode reward: [(0, '6.590'), (1, '7.240')] -[2023-10-14 13:24:26,704][75949] Updated weights for policy 0, policy_version 550 (0.0007) -[2023-10-14 13:24:26,800][75950] Updated weights for policy 1, policy_version 550 (0.0009) -[2023-10-14 13:24:27,068][75949] Updated weights for policy 0, policy_version 560 (0.0007) -[2023-10-14 13:24:27,175][75950] Updated weights for policy 1, policy_version 560 (0.0007) -[2023-10-14 13:24:27,436][75949] Updated weights for policy 0, policy_version 570 (0.0007) -[2023-10-14 13:24:27,534][75950] Updated weights for policy 1, policy_version 570 (0.0008) -[2023-10-14 13:24:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 12705.8). Total num frames: 1179648. Throughput: 0: 1655.7, 1: 1646.9. Samples: 298904. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 13:24:28,165][74987] Avg episode reward: [(0, '7.050'), (1, '7.400')] -[2023-10-14 13:24:28,178][75801] Saving new best policy, reward=7.400! -[2023-10-14 13:24:31,541][75949] Updated weights for policy 0, policy_version 580 (0.0009) -[2023-10-14 13:24:31,711][75950] Updated weights for policy 1, policy_version 580 (0.0009) -[2023-10-14 13:24:31,912][75949] Updated weights for policy 0, policy_version 590 (0.0007) -[2023-10-14 13:24:32,077][75950] Updated weights for policy 1, policy_version 590 (0.0009) -[2023-10-14 13:24:32,287][75949] Updated weights for policy 0, policy_version 600 (0.0009) -[2023-10-14 13:24:32,441][75950] Updated weights for policy 1, policy_version 600 (0.0009) -[2023-10-14 13:24:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 12726.3). Total num frames: 1245184. Throughput: 0: 1662.9, 1: 1659.6. Samples: 310150. Policy #0 lag: (min: 17.0, avg: 22.9, max: 49.0) -[2023-10-14 13:24:33,165][74987] Avg episode reward: [(0, '7.440'), (1, '7.520')] -[2023-10-14 13:24:33,166][75615] Saving new best policy, reward=7.440! -[2023-10-14 13:24:33,166][75801] Saving new best policy, reward=7.520! -[2023-10-14 13:24:36,294][75949] Updated weights for policy 0, policy_version 610 (0.0008) -[2023-10-14 13:24:36,485][75950] Updated weights for policy 1, policy_version 610 (0.0009) -[2023-10-14 13:24:36,661][75949] Updated weights for policy 0, policy_version 620 (0.0009) -[2023-10-14 13:24:36,898][75950] Updated weights for policy 1, policy_version 620 (0.0008) -[2023-10-14 13:24:37,033][75949] Updated weights for policy 0, policy_version 630 (0.0009) -[2023-10-14 13:24:37,261][75950] Updated weights for policy 1, policy_version 630 (0.0007) -[2023-10-14 13:24:37,402][75949] Updated weights for policy 0, policy_version 640 (0.0009) -[2023-10-14 13:24:37,628][75950] Updated weights for policy 1, policy_version 640 (0.0008) -[2023-10-14 13:24:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 12744.8). Total num frames: 1310720. Throughput: 0: 1656.9, 1: 1654.0. Samples: 329976. Policy #0 lag: (min: 28.0, avg: 29.0, max: 50.0) -[2023-10-14 13:24:38,165][74987] Avg episode reward: [(0, '7.750'), (1, '7.270')] -[2023-10-14 13:24:38,166][75615] Saving new best policy, reward=7.750! -[2023-10-14 13:24:41,394][75949] Updated weights for policy 0, policy_version 650 (0.0009) -[2023-10-14 13:24:41,764][75949] Updated weights for policy 0, policy_version 660 (0.0008) -[2023-10-14 13:24:41,837][75950] Updated weights for policy 1, policy_version 650 (0.0008) -[2023-10-14 13:24:42,129][75949] Updated weights for policy 0, policy_version 670 (0.0008) -[2023-10-14 13:24:42,210][75950] Updated weights for policy 1, policy_version 660 (0.0008) -[2023-10-14 13:24:42,582][75950] Updated weights for policy 1, policy_version 670 (0.0007) -[2023-10-14 13:24:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 12761.6). Total num frames: 1376256. Throughput: 0: 1656.9, 1: 1646.7. Samples: 348814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:24:43,165][74987] Avg episode reward: [(0, '7.270'), (1, '7.210')] -[2023-10-14 13:24:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000000672_688128.pth... -[2023-10-14 13:24:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000000672_688128.pth... -[2023-10-14 13:24:46,236][75949] Updated weights for policy 0, policy_version 680 (0.0011) -[2023-10-14 13:24:46,603][75949] Updated weights for policy 0, policy_version 690 (0.0008) -[2023-10-14 13:24:46,639][75950] Updated weights for policy 1, policy_version 680 (0.0008) -[2023-10-14 13:24:46,973][75949] Updated weights for policy 0, policy_version 700 (0.0010) -[2023-10-14 13:24:47,004][75950] Updated weights for policy 1, policy_version 690 (0.0008) -[2023-10-14 13:24:47,379][75950] Updated weights for policy 1, policy_version 700 (0.0009) -[2023-10-14 13:24:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 12777.0). Total num frames: 1441792. Throughput: 0: 1668.5, 1: 1658.2. Samples: 360330. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 13:24:48,165][74987] Avg episode reward: [(0, '6.650'), (1, '7.600')] -[2023-10-14 13:24:48,166][75801] Saving new best policy, reward=7.600! -[2023-10-14 13:24:51,225][75949] Updated weights for policy 0, policy_version 710 (0.0008) -[2023-10-14 13:24:51,549][75950] Updated weights for policy 1, policy_version 710 (0.0009) -[2023-10-14 13:24:51,594][75949] Updated weights for policy 0, policy_version 720 (0.0009) -[2023-10-14 13:24:51,920][75950] Updated weights for policy 1, policy_version 720 (0.0010) -[2023-10-14 13:24:51,968][75949] Updated weights for policy 0, policy_version 730 (0.0007) -[2023-10-14 13:24:52,284][75950] Updated weights for policy 1, policy_version 730 (0.0009) -[2023-10-14 13:24:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 12791.0). Total num frames: 1507328. Throughput: 0: 1648.9, 1: 1650.4. Samples: 379514. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-10-14 13:24:53,164][74987] Avg episode reward: [(0, '7.780'), (1, '7.880')] -[2023-10-14 13:24:53,165][75615] Saving new best policy, reward=7.780! -[2023-10-14 13:24:53,165][75801] Saving new best policy, reward=7.880! -[2023-10-14 13:24:56,048][75949] Updated weights for policy 0, policy_version 740 (0.0009) -[2023-10-14 13:24:56,422][75949] Updated weights for policy 0, policy_version 750 (0.0009) -[2023-10-14 13:24:56,532][75950] Updated weights for policy 1, policy_version 740 (0.0008) -[2023-10-14 13:24:56,787][75949] Updated weights for policy 0, policy_version 760 (0.0009) -[2023-10-14 13:24:56,894][75950] Updated weights for policy 1, policy_version 750 (0.0008) -[2023-10-14 13:24:57,262][75950] Updated weights for policy 1, policy_version 760 (0.0010) -[2023-10-14 13:24:58,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 12803.9). Total num frames: 1572864. Throughput: 0: 1657.6, 1: 1646.8. Samples: 398378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:24:58,164][74987] Avg episode reward: [(0, '7.530'), (1, '7.280')] -[2023-10-14 13:25:00,999][75949] Updated weights for policy 0, policy_version 770 (0.0009) -[2023-10-14 13:25:01,362][75949] Updated weights for policy 0, policy_version 780 (0.0008) -[2023-10-14 13:25:01,434][75950] Updated weights for policy 1, policy_version 770 (0.0010) -[2023-10-14 13:25:01,731][75949] Updated weights for policy 0, policy_version 790 (0.0007) -[2023-10-14 13:25:01,804][75950] Updated weights for policy 1, policy_version 780 (0.0009) -[2023-10-14 13:25:02,108][75949] Updated weights for policy 0, policy_version 800 (0.0009) -[2023-10-14 13:25:02,176][75950] Updated weights for policy 1, policy_version 790 (0.0007) -[2023-10-14 13:25:02,542][75950] Updated weights for policy 1, policy_version 800 (0.0007) -[2023-10-14 13:25:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12815.7). Total num frames: 1638400. Throughput: 0: 1663.6, 1: 1660.2. Samples: 409992. Policy #0 lag: (min: 15.0, avg: 19.3, max: 47.0) -[2023-10-14 13:25:03,165][74987] Avg episode reward: [(0, '8.290'), (1, '7.210')] -[2023-10-14 13:25:03,166][75615] Saving new best policy, reward=8.290! -[2023-10-14 13:25:06,288][75949] Updated weights for policy 0, policy_version 810 (0.0008) -[2023-10-14 13:25:06,600][75950] Updated weights for policy 1, policy_version 810 (0.0008) -[2023-10-14 13:25:06,655][75949] Updated weights for policy 0, policy_version 820 (0.0008) -[2023-10-14 13:25:06,971][75950] Updated weights for policy 1, policy_version 820 (0.0008) -[2023-10-14 13:25:07,026][75949] Updated weights for policy 0, policy_version 830 (0.0009) -[2023-10-14 13:25:07,334][75950] Updated weights for policy 1, policy_version 830 (0.0011) -[2023-10-14 13:25:08,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 12826.7). Total num frames: 1703936. Throughput: 0: 1649.4, 1: 1656.5. Samples: 429490. Policy #0 lag: (min: 12.0, avg: 12.5, max: 27.0) -[2023-10-14 13:25:08,165][74987] Avg episode reward: [(0, '7.780'), (1, '8.050')] -[2023-10-14 13:25:08,166][75801] Saving new best policy, reward=8.050! -[2023-10-14 13:25:11,030][75949] Updated weights for policy 0, policy_version 840 (0.0008) -[2023-10-14 13:25:11,387][75950] Updated weights for policy 1, policy_version 840 (0.0009) -[2023-10-14 13:25:11,400][75949] Updated weights for policy 0, policy_version 850 (0.0009) -[2023-10-14 13:25:11,758][75950] Updated weights for policy 1, policy_version 850 (0.0008) -[2023-10-14 13:25:11,772][75949] Updated weights for policy 0, policy_version 860 (0.0009) -[2023-10-14 13:25:12,130][75950] Updated weights for policy 1, policy_version 860 (0.0008) -[2023-10-14 13:25:13,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12836.9). Total num frames: 1769472. Throughput: 0: 1668.3, 1: 1662.7. Samples: 448800. Policy #0 lag: (min: 15.0, avg: 38.6, max: 40.0) -[2023-10-14 13:25:13,164][74987] Avg episode reward: [(0, '7.460'), (1, '7.910')] -[2023-10-14 13:25:15,752][75949] Updated weights for policy 0, policy_version 870 (0.0008) -[2023-10-14 13:25:16,122][75949] Updated weights for policy 0, policy_version 880 (0.0010) -[2023-10-14 13:25:16,162][75950] Updated weights for policy 1, policy_version 870 (0.0009) -[2023-10-14 13:25:16,491][75949] Updated weights for policy 0, policy_version 890 (0.0009) -[2023-10-14 13:25:16,533][75950] Updated weights for policy 1, policy_version 880 (0.0008) -[2023-10-14 13:25:16,896][75950] Updated weights for policy 1, policy_version 890 (0.0008) -[2023-10-14 13:25:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12846.3). Total num frames: 1835008. Throughput: 0: 1668.1, 1: 1672.6. Samples: 460480. Policy #0 lag: (min: 16.0, avg: 44.8, max: 48.0) -[2023-10-14 13:25:18,165][74987] Avg episode reward: [(0, '7.570'), (1, '7.230')] -[2023-10-14 13:25:20,541][75949] Updated weights for policy 0, policy_version 900 (0.0010) -[2023-10-14 13:25:20,833][75950] Updated weights for policy 1, policy_version 900 (0.0008) -[2023-10-14 13:25:20,911][75949] Updated weights for policy 0, policy_version 910 (0.0011) -[2023-10-14 13:25:21,216][75950] Updated weights for policy 1, policy_version 910 (0.0008) -[2023-10-14 13:25:21,277][75949] Updated weights for policy 0, policy_version 920 (0.0009) -[2023-10-14 13:25:21,586][75950] Updated weights for policy 1, policy_version 920 (0.0007) -[2023-10-14 13:25:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12855.1). Total num frames: 1900544. Throughput: 0: 1652.0, 1: 1657.2. Samples: 478890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:25:23,164][74987] Avg episode reward: [(0, '7.670'), (1, '7.970')] -[2023-10-14 13:25:25,464][75949] Updated weights for policy 0, policy_version 930 (0.0007) -[2023-10-14 13:25:25,571][75950] Updated weights for policy 1, policy_version 930 (0.0009) -[2023-10-14 13:25:25,838][75949] Updated weights for policy 0, policy_version 940 (0.0009) -[2023-10-14 13:25:25,937][75950] Updated weights for policy 1, policy_version 940 (0.0008) -[2023-10-14 13:25:26,218][75949] Updated weights for policy 0, policy_version 950 (0.0010) -[2023-10-14 13:25:26,304][75950] Updated weights for policy 1, policy_version 950 (0.0009) -[2023-10-14 13:25:26,581][75949] Updated weights for policy 0, policy_version 960 (0.0011) -[2023-10-14 13:25:26,673][75950] Updated weights for policy 1, policy_version 960 (0.0009) -[2023-10-14 13:25:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12863.4). Total num frames: 1966080. Throughput: 0: 1673.3, 1: 1670.9. Samples: 499304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:25:28,165][74987] Avg episode reward: [(0, '8.030'), (1, '7.800')] -[2023-10-14 13:25:30,524][75949] Updated weights for policy 0, policy_version 970 (0.0008) -[2023-10-14 13:25:30,896][75949] Updated weights for policy 0, policy_version 980 (0.0007) -[2023-10-14 13:25:31,030][75950] Updated weights for policy 1, policy_version 970 (0.0008) -[2023-10-14 13:25:31,262][75949] Updated weights for policy 0, policy_version 990 (0.0010) -[2023-10-14 13:25:31,398][75950] Updated weights for policy 1, policy_version 980 (0.0008) -[2023-10-14 13:25:31,764][75950] Updated weights for policy 1, policy_version 990 (0.0009) -[2023-10-14 13:25:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12871.1). Total num frames: 2031616. Throughput: 0: 1662.3, 1: 1667.7. Samples: 510180. Policy #0 lag: (min: 21.0, avg: 24.6, max: 53.0) -[2023-10-14 13:25:33,164][74987] Avg episode reward: [(0, '7.930'), (1, '7.860')] -[2023-10-14 13:25:35,335][75949] Updated weights for policy 0, policy_version 1000 (0.0008) -[2023-10-14 13:25:35,708][75949] Updated weights for policy 0, policy_version 1010 (0.0009) -[2023-10-14 13:25:35,905][75950] Updated weights for policy 1, policy_version 1000 (0.0008) -[2023-10-14 13:25:36,090][75949] Updated weights for policy 0, policy_version 1020 (0.0009) -[2023-10-14 13:25:36,280][75950] Updated weights for policy 1, policy_version 1010 (0.0007) -[2023-10-14 13:25:36,643][75950] Updated weights for policy 1, policy_version 1020 (0.0009) -[2023-10-14 13:25:38,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 12878.4). Total num frames: 2097152. Throughput: 0: 1668.7, 1: 1649.7. Samples: 528842. Policy #0 lag: (min: 13.0, avg: 13.0, max: 14.0) -[2023-10-14 13:25:38,164][74987] Avg episode reward: [(0, '8.480'), (1, '7.380')] -[2023-10-14 13:25:38,165][75615] Saving new best policy, reward=8.480! -[2023-10-14 13:25:40,150][75949] Updated weights for policy 0, policy_version 1030 (0.0008) -[2023-10-14 13:25:40,522][75949] Updated weights for policy 0, policy_version 1040 (0.0008) -[2023-10-14 13:25:40,784][75950] Updated weights for policy 1, policy_version 1030 (0.0009) -[2023-10-14 13:25:40,890][75949] Updated weights for policy 0, policy_version 1050 (0.0007) -[2023-10-14 13:25:41,143][75950] Updated weights for policy 1, policy_version 1040 (0.0008) -[2023-10-14 13:25:41,509][75950] Updated weights for policy 1, policy_version 1050 (0.0010) -[2023-10-14 13:25:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 12885.2). Total num frames: 2162688. Throughput: 0: 1681.4, 1: 1670.4. Samples: 549210. Policy #0 lag: (min: 3.0, avg: 5.8, max: 35.0) -[2023-10-14 13:25:43,164][74987] Avg episode reward: [(0, '8.290'), (1, '7.440')] -[2023-10-14 13:25:45,118][75949] Updated weights for policy 0, policy_version 1060 (0.0010) -[2023-10-14 13:25:45,491][75949] Updated weights for policy 0, policy_version 1070 (0.0008) -[2023-10-14 13:25:45,497][75950] Updated weights for policy 1, policy_version 1060 (0.0009) -[2023-10-14 13:25:45,863][75950] Updated weights for policy 1, policy_version 1070 (0.0007) -[2023-10-14 13:25:45,863][75949] Updated weights for policy 0, policy_version 1080 (0.0008) -[2023-10-14 13:25:46,235][75950] Updated weights for policy 1, policy_version 1080 (0.0009) -[2023-10-14 13:25:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12891.6). Total num frames: 2228224. Throughput: 0: 1664.4, 1: 1666.0. Samples: 559858. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) -[2023-10-14 13:25:48,164][74987] Avg episode reward: [(0, '8.390'), (1, '7.740')] -[2023-10-14 13:25:50,065][75949] Updated weights for policy 0, policy_version 1090 (0.0008) -[2023-10-14 13:25:50,359][75950] Updated weights for policy 1, policy_version 1090 (0.0009) -[2023-10-14 13:25:50,436][75949] Updated weights for policy 0, policy_version 1100 (0.0008) -[2023-10-14 13:25:50,726][75950] Updated weights for policy 1, policy_version 1100 (0.0010) -[2023-10-14 13:25:50,803][75949] Updated weights for policy 0, policy_version 1110 (0.0007) -[2023-10-14 13:25:51,097][75950] Updated weights for policy 1, policy_version 1110 (0.0009) -[2023-10-14 13:25:51,170][75949] Updated weights for policy 0, policy_version 1120 (0.0009) -[2023-10-14 13:25:51,461][75950] Updated weights for policy 1, policy_version 1120 (0.0010) -[2023-10-14 13:25:53,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12897.7). Total num frames: 2293760. Throughput: 0: 1664.5, 1: 1652.5. Samples: 578752. Policy #0 lag: (min: 6.0, avg: 15.4, max: 38.0) -[2023-10-14 13:25:53,164][74987] Avg episode reward: [(0, '8.720'), (1, '8.640')] -[2023-10-14 13:25:53,165][75801] Saving new best policy, reward=8.640! -[2023-10-14 13:25:53,165][75615] Saving new best policy, reward=8.720! -[2023-10-14 13:25:55,237][75949] Updated weights for policy 0, policy_version 1130 (0.0010) -[2023-10-14 13:25:55,603][75949] Updated weights for policy 0, policy_version 1140 (0.0009) -[2023-10-14 13:25:55,688][75950] Updated weights for policy 1, policy_version 1130 (0.0008) -[2023-10-14 13:25:55,979][75949] Updated weights for policy 0, policy_version 1150 (0.0009) -[2023-10-14 13:25:56,050][75950] Updated weights for policy 1, policy_version 1140 (0.0010) -[2023-10-14 13:25:56,423][75950] Updated weights for policy 1, policy_version 1150 (0.0009) -[2023-10-14 13:25:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12903.4). Total num frames: 2359296. Throughput: 0: 1673.8, 1: 1669.3. Samples: 599238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:25:58,164][74987] Avg episode reward: [(0, '7.820'), (1, '8.150')] -[2023-10-14 13:26:00,194][75949] Updated weights for policy 0, policy_version 1160 (0.0009) -[2023-10-14 13:26:00,557][75949] Updated weights for policy 0, policy_version 1170 (0.0009) -[2023-10-14 13:26:00,633][75950] Updated weights for policy 1, policy_version 1160 (0.0008) -[2023-10-14 13:26:00,931][75949] Updated weights for policy 0, policy_version 1180 (0.0009) -[2023-10-14 13:26:01,006][75950] Updated weights for policy 1, policy_version 1170 (0.0009) -[2023-10-14 13:26:01,368][75950] Updated weights for policy 1, policy_version 1180 (0.0009) -[2023-10-14 13:26:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12908.8). Total num frames: 2424832. Throughput: 0: 1654.1, 1: 1658.6. Samples: 609550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:26:03,164][74987] Avg episode reward: [(0, '8.050'), (1, '8.280')] -[2023-10-14 13:26:05,083][75949] Updated weights for policy 0, policy_version 1190 (0.0009) -[2023-10-14 13:26:05,319][75950] Updated weights for policy 1, policy_version 1190 (0.0008) -[2023-10-14 13:26:05,442][75949] Updated weights for policy 0, policy_version 1200 (0.0008) -[2023-10-14 13:26:05,688][75950] Updated weights for policy 1, policy_version 1200 (0.0008) -[2023-10-14 13:26:05,811][75949] Updated weights for policy 0, policy_version 1210 (0.0008) -[2023-10-14 13:26:06,061][75950] Updated weights for policy 1, policy_version 1210 (0.0008) -[2023-10-14 13:26:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 12914.0). Total num frames: 2490368. Throughput: 0: 1665.8, 1: 1659.2. Samples: 628516. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 13:26:08,164][74987] Avg episode reward: [(0, '7.930'), (1, '8.430')] -[2023-10-14 13:26:10,032][75950] Updated weights for policy 1, policy_version 1220 (0.0009) -[2023-10-14 13:26:10,127][75949] Updated weights for policy 0, policy_version 1220 (0.0008) -[2023-10-14 13:26:10,424][75950] Updated weights for policy 1, policy_version 1230 (0.0009) -[2023-10-14 13:26:10,516][75949] Updated weights for policy 0, policy_version 1230 (0.0009) -[2023-10-14 13:26:10,789][75950] Updated weights for policy 1, policy_version 1240 (0.0009) -[2023-10-14 13:26:10,880][75949] Updated weights for policy 0, policy_version 1240 (0.0008) -[2023-10-14 13:26:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12918.9). Total num frames: 2555904. Throughput: 0: 1655.6, 1: 1664.2. Samples: 648692. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) -[2023-10-14 13:26:13,164][74987] Avg episode reward: [(0, '8.560'), (1, '8.380')] -[2023-10-14 13:26:14,824][75950] Updated weights for policy 1, policy_version 1250 (0.0009) -[2023-10-14 13:26:14,942][75949] Updated weights for policy 0, policy_version 1250 (0.0008) -[2023-10-14 13:26:15,199][75950] Updated weights for policy 1, policy_version 1260 (0.0009) -[2023-10-14 13:26:15,310][75949] Updated weights for policy 0, policy_version 1260 (0.0008) -[2023-10-14 13:26:15,561][75950] Updated weights for policy 1, policy_version 1270 (0.0008) -[2023-10-14 13:26:15,688][75949] Updated weights for policy 0, policy_version 1270 (0.0008) -[2023-10-14 13:26:15,924][75950] Updated weights for policy 1, policy_version 1280 (0.0007) -[2023-10-14 13:26:16,054][75949] Updated weights for policy 0, policy_version 1280 (0.0009) -[2023-10-14 13:26:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12923.5). Total num frames: 2621440. Throughput: 0: 1646.4, 1: 1653.0. Samples: 658650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:26:18,165][74987] Avg episode reward: [(0, '8.340'), (1, '8.710')] -[2023-10-14 13:26:18,166][75801] Saving new best policy, reward=8.710! -[2023-10-14 13:26:20,100][75949] Updated weights for policy 0, policy_version 1290 (0.0010) -[2023-10-14 13:26:20,120][75950] Updated weights for policy 1, policy_version 1290 (0.0008) -[2023-10-14 13:26:20,477][75949] Updated weights for policy 0, policy_version 1300 (0.0009) -[2023-10-14 13:26:20,478][75950] Updated weights for policy 1, policy_version 1300 (0.0009) -[2023-10-14 13:26:20,855][75950] Updated weights for policy 1, policy_version 1310 (0.0008) -[2023-10-14 13:26:20,856][75949] Updated weights for policy 0, policy_version 1310 (0.0009) -[2023-10-14 13:26:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12927.9). Total num frames: 2686976. Throughput: 0: 1652.4, 1: 1671.0. Samples: 678396. Policy #0 lag: (min: 26.0, avg: 32.5, max: 58.0) -[2023-10-14 13:26:23,165][74987] Avg episode reward: [(0, '9.020'), (1, '8.680')] -[2023-10-14 13:26:23,166][75615] Saving new best policy, reward=9.020! -[2023-10-14 13:26:25,033][75949] Updated weights for policy 0, policy_version 1320 (0.0009) -[2023-10-14 13:26:25,047][75950] Updated weights for policy 1, policy_version 1320 (0.0008) -[2023-10-14 13:26:25,414][75949] Updated weights for policy 0, policy_version 1330 (0.0009) -[2023-10-14 13:26:25,415][75950] Updated weights for policy 1, policy_version 1330 (0.0007) -[2023-10-14 13:26:25,777][75949] Updated weights for policy 0, policy_version 1340 (0.0008) -[2023-10-14 13:26:25,781][75950] Updated weights for policy 1, policy_version 1340 (0.0009) -[2023-10-14 13:26:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12932.1). Total num frames: 2752512. Throughput: 0: 1657.5, 1: 1670.7. Samples: 698978. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 13:26:28,165][74987] Avg episode reward: [(0, '9.010'), (1, '8.710')] -[2023-10-14 13:26:29,693][75949] Updated weights for policy 0, policy_version 1350 (0.0010) -[2023-10-14 13:26:29,946][75950] Updated weights for policy 1, policy_version 1350 (0.0007) -[2023-10-14 13:26:30,061][75949] Updated weights for policy 0, policy_version 1360 (0.0008) -[2023-10-14 13:26:30,327][75950] Updated weights for policy 1, policy_version 1360 (0.0008) -[2023-10-14 13:26:30,440][75949] Updated weights for policy 0, policy_version 1370 (0.0010) -[2023-10-14 13:26:30,697][75950] Updated weights for policy 1, policy_version 1370 (0.0009) -[2023-10-14 13:26:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12936.1). Total num frames: 2818048. Throughput: 0: 1649.8, 1: 1655.5. Samples: 708596. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-14 13:26:33,165][74987] Avg episode reward: [(0, '9.130'), (1, '9.200')] -[2023-10-14 13:26:33,167][75615] Saving new best policy, reward=9.130! -[2023-10-14 13:26:33,167][75801] Saving new best policy, reward=9.200! -[2023-10-14 13:26:34,493][75949] Updated weights for policy 0, policy_version 1380 (0.0008) -[2023-10-14 13:26:34,865][75950] Updated weights for policy 1, policy_version 1380 (0.0008) -[2023-10-14 13:26:34,873][75949] Updated weights for policy 0, policy_version 1390 (0.0007) -[2023-10-14 13:26:35,238][75950] Updated weights for policy 1, policy_version 1390 (0.0009) -[2023-10-14 13:26:35,240][75949] Updated weights for policy 0, policy_version 1400 (0.0007) -[2023-10-14 13:26:35,596][75950] Updated weights for policy 1, policy_version 1400 (0.0007) -[2023-10-14 13:26:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12940.0). Total num frames: 2883584. Throughput: 0: 1665.6, 1: 1667.7. Samples: 728752. Policy #0 lag: (min: 10.0, avg: 11.5, max: 36.0) -[2023-10-14 13:26:38,165][74987] Avg episode reward: [(0, '8.270'), (1, '8.990')] -[2023-10-14 13:26:39,375][75949] Updated weights for policy 0, policy_version 1410 (0.0007) -[2023-10-14 13:26:39,660][75950] Updated weights for policy 1, policy_version 1410 (0.0009) -[2023-10-14 13:26:39,741][75949] Updated weights for policy 0, policy_version 1420 (0.0009) -[2023-10-14 13:26:40,030][75950] Updated weights for policy 1, policy_version 1420 (0.0007) -[2023-10-14 13:26:40,106][75949] Updated weights for policy 0, policy_version 1430 (0.0008) -[2023-10-14 13:26:40,396][75950] Updated weights for policy 1, policy_version 1430 (0.0008) -[2023-10-14 13:26:40,478][75949] Updated weights for policy 0, policy_version 1440 (0.0009) -[2023-10-14 13:26:40,761][75950] Updated weights for policy 1, policy_version 1440 (0.0009) -[2023-10-14 13:26:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 12943.6). Total num frames: 2949120. Throughput: 0: 1664.7, 1: 1665.3. Samples: 749090. Policy #0 lag: (min: 15.0, avg: 20.2, max: 47.0) -[2023-10-14 13:26:43,165][74987] Avg episode reward: [(0, '8.770'), (1, '8.900')] -[2023-10-14 13:26:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000001440_1474560.pth... -[2023-10-14 13:26:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000001440_1474560.pth... -[2023-10-14 13:26:44,540][75949] Updated weights for policy 0, policy_version 1450 (0.0008) -[2023-10-14 13:26:44,916][75949] Updated weights for policy 0, policy_version 1460 (0.0009) -[2023-10-14 13:26:45,083][75950] Updated weights for policy 1, policy_version 1450 (0.0007) -[2023-10-14 13:26:45,277][75949] Updated weights for policy 0, policy_version 1470 (0.0010) -[2023-10-14 13:26:45,456][75950] Updated weights for policy 1, policy_version 1460 (0.0007) -[2023-10-14 13:26:45,821][75950] Updated weights for policy 1, policy_version 1470 (0.0007) -[2023-10-14 13:26:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12947.2). Total num frames: 3014656. Throughput: 0: 1656.8, 1: 1652.0. Samples: 758444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:26:48,165][74987] Avg episode reward: [(0, '8.270'), (1, '9.420')] -[2023-10-14 13:26:48,166][75801] Saving new best policy, reward=9.420! -[2023-10-14 13:26:49,430][75949] Updated weights for policy 0, policy_version 1480 (0.0011) -[2023-10-14 13:26:49,800][75949] Updated weights for policy 0, policy_version 1490 (0.0009) -[2023-10-14 13:26:49,942][75950] Updated weights for policy 1, policy_version 1480 (0.0007) -[2023-10-14 13:26:50,182][75949] Updated weights for policy 0, policy_version 1500 (0.0009) -[2023-10-14 13:26:50,308][75950] Updated weights for policy 1, policy_version 1490 (0.0007) -[2023-10-14 13:26:50,674][75950] Updated weights for policy 1, policy_version 1500 (0.0008) -[2023-10-14 13:26:53,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 12950.5). Total num frames: 3080192. Throughput: 0: 1668.4, 1: 1662.4. Samples: 778402. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) -[2023-10-14 13:26:53,164][74987] Avg episode reward: [(0, '8.790'), (1, '8.700')] -[2023-10-14 13:26:54,306][75949] Updated weights for policy 0, policy_version 1510 (0.0008) -[2023-10-14 13:26:54,683][75949] Updated weights for policy 0, policy_version 1520 (0.0009) -[2023-10-14 13:26:55,000][75950] Updated weights for policy 1, policy_version 1510 (0.0010) -[2023-10-14 13:26:55,052][75949] Updated weights for policy 0, policy_version 1530 (0.0007) -[2023-10-14 13:26:55,381][75950] Updated weights for policy 1, policy_version 1520 (0.0008) -[2023-10-14 13:26:55,745][75950] Updated weights for policy 1, policy_version 1530 (0.0008) -[2023-10-14 13:26:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12953.8). Total num frames: 3145728. Throughput: 0: 1682.1, 1: 1658.9. Samples: 799040. Policy #0 lag: (min: 5.0, avg: 5.5, max: 21.0) -[2023-10-14 13:26:58,164][74987] Avg episode reward: [(0, '9.130'), (1, '8.480')] -[2023-10-14 13:26:59,023][75949] Updated weights for policy 0, policy_version 1540 (0.0008) -[2023-10-14 13:26:59,397][75949] Updated weights for policy 0, policy_version 1550 (0.0007) -[2023-10-14 13:26:59,765][75949] Updated weights for policy 0, policy_version 1560 (0.0010) -[2023-10-14 13:26:59,896][75950] Updated weights for policy 1, policy_version 1540 (0.0008) -[2023-10-14 13:27:00,264][75950] Updated weights for policy 1, policy_version 1550 (0.0008) -[2023-10-14 13:27:00,630][75950] Updated weights for policy 1, policy_version 1560 (0.0009) -[2023-10-14 13:27:03,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 12956.8). Total num frames: 3211264. Throughput: 0: 1674.2, 1: 1654.7. Samples: 808448. Policy #0 lag: (min: 17.0, avg: 33.5, max: 49.0) -[2023-10-14 13:27:03,165][74987] Avg episode reward: [(0, '8.320'), (1, '8.530')] -[2023-10-14 13:27:03,881][75949] Updated weights for policy 0, policy_version 1570 (0.0008) -[2023-10-14 13:27:04,255][75949] Updated weights for policy 0, policy_version 1580 (0.0008) -[2023-10-14 13:27:04,641][75949] Updated weights for policy 0, policy_version 1590 (0.0008) -[2023-10-14 13:27:04,785][75950] Updated weights for policy 1, policy_version 1570 (0.0008) -[2023-10-14 13:27:05,008][75949] Updated weights for policy 0, policy_version 1600 (0.0009) -[2023-10-14 13:27:05,156][75950] Updated weights for policy 1, policy_version 1580 (0.0008) -[2023-10-14 13:27:05,523][75950] Updated weights for policy 1, policy_version 1590 (0.0009) -[2023-10-14 13:27:05,902][75950] Updated weights for policy 1, policy_version 1600 (0.0010) -[2023-10-14 13:27:08,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12959.8). Total num frames: 3276800. Throughput: 0: 1681.5, 1: 1656.6. Samples: 828610. Policy #0 lag: (min: 30.0, avg: 36.4, max: 62.0) -[2023-10-14 13:27:08,164][74987] Avg episode reward: [(0, '9.510'), (1, '8.130')] -[2023-10-14 13:27:08,165][75615] Saving new best policy, reward=9.510! -[2023-10-14 13:27:09,108][75949] Updated weights for policy 0, policy_version 1610 (0.0008) -[2023-10-14 13:27:09,471][75949] Updated weights for policy 0, policy_version 1620 (0.0008) -[2023-10-14 13:27:09,781][75950] Updated weights for policy 1, policy_version 1610 (0.0007) -[2023-10-14 13:27:09,849][75949] Updated weights for policy 0, policy_version 1630 (0.0010) -[2023-10-14 13:27:10,155][75950] Updated weights for policy 1, policy_version 1620 (0.0007) -[2023-10-14 13:27:10,528][75950] Updated weights for policy 1, policy_version 1630 (0.0007) -[2023-10-14 13:27:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.1, 300 sec: 12962.7). Total num frames: 3342336. Throughput: 0: 1676.3, 1: 1665.6. Samples: 849362. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 13:27:13,165][74987] Avg episode reward: [(0, '9.590'), (1, '8.210')] -[2023-10-14 13:27:13,175][75615] Saving new best policy, reward=9.590! -[2023-10-14 13:27:13,856][75949] Updated weights for policy 0, policy_version 1640 (0.0009) -[2023-10-14 13:27:14,223][75949] Updated weights for policy 0, policy_version 1650 (0.0010) -[2023-10-14 13:27:14,447][75950] Updated weights for policy 1, policy_version 1640 (0.0009) -[2023-10-14 13:27:14,598][75949] Updated weights for policy 0, policy_version 1660 (0.0008) -[2023-10-14 13:27:14,824][75950] Updated weights for policy 1, policy_version 1650 (0.0009) -[2023-10-14 13:27:15,190][75950] Updated weights for policy 1, policy_version 1660 (0.0008) -[2023-10-14 13:27:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12965.4). Total num frames: 3407872. Throughput: 0: 1677.7, 1: 1656.5. Samples: 858634. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-10-14 13:27:18,164][74987] Avg episode reward: [(0, '9.120'), (1, '8.440')] -[2023-10-14 13:27:18,672][75949] Updated weights for policy 0, policy_version 1670 (0.0010) -[2023-10-14 13:27:19,048][75949] Updated weights for policy 0, policy_version 1680 (0.0011) -[2023-10-14 13:27:19,326][75950] Updated weights for policy 1, policy_version 1670 (0.0008) -[2023-10-14 13:27:19,415][75949] Updated weights for policy 0, policy_version 1690 (0.0008) -[2023-10-14 13:27:19,691][75950] Updated weights for policy 1, policy_version 1680 (0.0009) -[2023-10-14 13:27:20,059][75950] Updated weights for policy 1, policy_version 1690 (0.0009) -[2023-10-14 13:27:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12968.1). Total num frames: 3473408. Throughput: 0: 1679.7, 1: 1668.2. Samples: 879410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:27:23,164][74987] Avg episode reward: [(0, '9.770'), (1, '8.680')] -[2023-10-14 13:27:23,165][75615] Saving new best policy, reward=9.770! -[2023-10-14 13:27:23,499][75949] Updated weights for policy 0, policy_version 1700 (0.0008) -[2023-10-14 13:27:23,876][75949] Updated weights for policy 0, policy_version 1710 (0.0009) -[2023-10-14 13:27:23,947][75950] Updated weights for policy 1, policy_version 1700 (0.0007) -[2023-10-14 13:27:24,240][75949] Updated weights for policy 0, policy_version 1720 (0.0007) -[2023-10-14 13:27:24,320][75950] Updated weights for policy 1, policy_version 1710 (0.0007) -[2023-10-14 13:27:24,684][75950] Updated weights for policy 1, policy_version 1720 (0.0008) -[2023-10-14 13:27:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 12970.6). Total num frames: 3538944. Throughput: 0: 1677.3, 1: 1677.7. Samples: 900062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:27:28,164][74987] Avg episode reward: [(0, '9.560'), (1, '8.190')] -[2023-10-14 13:27:28,472][75949] Updated weights for policy 0, policy_version 1730 (0.0008) -[2023-10-14 13:27:28,804][75950] Updated weights for policy 1, policy_version 1730 (0.0010) -[2023-10-14 13:27:28,854][75949] Updated weights for policy 0, policy_version 1740 (0.0008) -[2023-10-14 13:27:29,176][75950] Updated weights for policy 1, policy_version 1740 (0.0009) -[2023-10-14 13:27:29,229][75949] Updated weights for policy 0, policy_version 1750 (0.0009) -[2023-10-14 13:27:29,546][75950] Updated weights for policy 1, policy_version 1750 (0.0008) -[2023-10-14 13:27:29,603][75949] Updated weights for policy 0, policy_version 1760 (0.0007) -[2023-10-14 13:27:29,913][75950] Updated weights for policy 1, policy_version 1760 (0.0008) -[2023-10-14 13:27:33,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12973.1). Total num frames: 3604480. Throughput: 0: 1675.8, 1: 1670.0. Samples: 909004. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-14 13:27:33,164][74987] Avg episode reward: [(0, '9.340'), (1, '8.540')] -[2023-10-14 13:27:33,833][75949] Updated weights for policy 0, policy_version 1770 (0.0008) -[2023-10-14 13:27:33,859][75950] Updated weights for policy 1, policy_version 1770 (0.0008) -[2023-10-14 13:27:34,200][75949] Updated weights for policy 0, policy_version 1780 (0.0007) -[2023-10-14 13:27:34,227][75950] Updated weights for policy 1, policy_version 1780 (0.0008) -[2023-10-14 13:27:34,563][75949] Updated weights for policy 0, policy_version 1790 (0.0007) -[2023-10-14 13:27:34,598][75950] Updated weights for policy 1, policy_version 1790 (0.0009) -[2023-10-14 13:27:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12975.4). Total num frames: 3670016. Throughput: 0: 1678.3, 1: 1683.5. Samples: 929682. Policy #0 lag: (min: 27.0, avg: 39.6, max: 40.0) -[2023-10-14 13:27:38,165][74987] Avg episode reward: [(0, '9.670'), (1, '8.830')] -[2023-10-14 13:27:38,542][75949] Updated weights for policy 0, policy_version 1800 (0.0008) -[2023-10-14 13:27:38,733][75950] Updated weights for policy 1, policy_version 1800 (0.0007) -[2023-10-14 13:27:38,907][75949] Updated weights for policy 0, policy_version 1810 (0.0007) -[2023-10-14 13:27:39,093][75950] Updated weights for policy 1, policy_version 1810 (0.0007) -[2023-10-14 13:27:39,280][75949] Updated weights for policy 0, policy_version 1820 (0.0009) -[2023-10-14 13:27:39,468][75950] Updated weights for policy 1, policy_version 1820 (0.0008) -[2023-10-14 13:27:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 12977.7). Total num frames: 3735552. Throughput: 0: 1675.1, 1: 1683.1. Samples: 950160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:27:43,165][74987] Avg episode reward: [(0, '9.740'), (1, '8.780')] -[2023-10-14 13:27:43,259][75949] Updated weights for policy 0, policy_version 1830 (0.0008) -[2023-10-14 13:27:43,653][75949] Updated weights for policy 0, policy_version 1840 (0.0007) -[2023-10-14 13:27:43,690][75950] Updated weights for policy 1, policy_version 1830 (0.0009) -[2023-10-14 13:27:44,027][75949] Updated weights for policy 0, policy_version 1850 (0.0008) -[2023-10-14 13:27:44,067][75950] Updated weights for policy 1, policy_version 1840 (0.0007) -[2023-10-14 13:27:44,436][75950] Updated weights for policy 1, policy_version 1850 (0.0009) -[2023-10-14 13:27:48,125][75949] Updated weights for policy 0, policy_version 1860 (0.0010) -[2023-10-14 13:27:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12979.9). Total num frames: 3801088. Throughput: 0: 1672.5, 1: 1672.9. Samples: 958994. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) -[2023-10-14 13:27:48,164][74987] Avg episode reward: [(0, '9.970'), (1, '8.550')] -[2023-10-14 13:27:48,480][75950] Updated weights for policy 1, policy_version 1860 (0.0007) -[2023-10-14 13:27:48,488][75949] Updated weights for policy 0, policy_version 1870 (0.0010) -[2023-10-14 13:27:48,857][75950] Updated weights for policy 1, policy_version 1870 (0.0007) -[2023-10-14 13:27:48,869][75949] Updated weights for policy 0, policy_version 1880 (0.0009) -[2023-10-14 13:27:49,166][75615] Saving new best policy, reward=9.970! -[2023-10-14 13:27:49,221][75950] Updated weights for policy 1, policy_version 1880 (0.0008) -[2023-10-14 13:27:52,900][75949] Updated weights for policy 0, policy_version 1890 (0.0007) -[2023-10-14 13:27:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 3866624. Throughput: 0: 1670.5, 1: 1680.6. Samples: 979410. Policy #0 lag: (min: 26.0, avg: 27.3, max: 46.0) -[2023-10-14 13:27:53,164][74987] Avg episode reward: [(0, '9.150'), (1, '8.450')] -[2023-10-14 13:27:53,269][75949] Updated weights for policy 0, policy_version 1900 (0.0007) -[2023-10-14 13:27:53,373][75950] Updated weights for policy 1, policy_version 1890 (0.0008) -[2023-10-14 13:27:53,635][75949] Updated weights for policy 0, policy_version 1910 (0.0007) -[2023-10-14 13:27:53,744][75950] Updated weights for policy 1, policy_version 1900 (0.0010) -[2023-10-14 13:27:54,013][75949] Updated weights for policy 0, policy_version 1920 (0.0009) -[2023-10-14 13:27:54,108][75950] Updated weights for policy 1, policy_version 1910 (0.0007) -[2023-10-14 13:27:54,469][75950] Updated weights for policy 1, policy_version 1920 (0.0007) -[2023-10-14 13:27:58,144][75949] Updated weights for policy 0, policy_version 1930 (0.0008) -[2023-10-14 13:27:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 3932160. Throughput: 0: 1673.7, 1: 1677.5. Samples: 1000166. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 13:27:58,165][74987] Avg episode reward: [(0, '9.690'), (1, '9.060')] -[2023-10-14 13:27:58,516][75949] Updated weights for policy 0, policy_version 1940 (0.0008) -[2023-10-14 13:27:58,679][75950] Updated weights for policy 1, policy_version 1930 (0.0008) -[2023-10-14 13:27:58,887][75949] Updated weights for policy 0, policy_version 1950 (0.0007) -[2023-10-14 13:27:59,049][75950] Updated weights for policy 1, policy_version 1940 (0.0008) -[2023-10-14 13:27:59,410][75950] Updated weights for policy 1, policy_version 1950 (0.0008) -[2023-10-14 13:28:02,898][75949] Updated weights for policy 0, policy_version 1960 (0.0009) -[2023-10-14 13:28:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 3997696. Throughput: 0: 1671.8, 1: 1675.3. Samples: 1009254. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) -[2023-10-14 13:28:03,164][74987] Avg episode reward: [(0, '9.740'), (1, '9.440')] -[2023-10-14 13:28:03,165][75801] Saving new best policy, reward=9.440! -[2023-10-14 13:28:03,275][75949] Updated weights for policy 0, policy_version 1970 (0.0008) -[2023-10-14 13:28:03,638][75949] Updated weights for policy 0, policy_version 1980 (0.0009) -[2023-10-14 13:28:03,715][75950] Updated weights for policy 1, policy_version 1960 (0.0009) -[2023-10-14 13:28:04,079][75950] Updated weights for policy 1, policy_version 1970 (0.0011) -[2023-10-14 13:28:04,448][75950] Updated weights for policy 1, policy_version 1980 (0.0010) -[2023-10-14 13:28:07,800][75949] Updated weights for policy 0, policy_version 1990 (0.0010) -[2023-10-14 13:28:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4063232. Throughput: 0: 1670.0, 1: 1667.6. Samples: 1029598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:28:08,164][74987] Avg episode reward: [(0, '9.360'), (1, '8.790')] -[2023-10-14 13:28:08,173][75949] Updated weights for policy 0, policy_version 2000 (0.0007) -[2023-10-14 13:28:08,550][75949] Updated weights for policy 0, policy_version 2010 (0.0008) -[2023-10-14 13:28:08,601][75950] Updated weights for policy 1, policy_version 1990 (0.0009) -[2023-10-14 13:28:08,968][75950] Updated weights for policy 1, policy_version 2000 (0.0010) -[2023-10-14 13:28:09,343][75950] Updated weights for policy 1, policy_version 2010 (0.0008) -[2023-10-14 13:28:12,661][75949] Updated weights for policy 0, policy_version 2020 (0.0008) -[2023-10-14 13:28:13,035][75949] Updated weights for policy 0, policy_version 2030 (0.0007) -[2023-10-14 13:28:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4128768. Throughput: 0: 1669.1, 1: 1658.5. Samples: 1049804. Policy #0 lag: (min: 1.0, avg: 1.5, max: 14.0) -[2023-10-14 13:28:13,164][74987] Avg episode reward: [(0, '11.020'), (1, '8.980')] -[2023-10-14 13:28:13,414][75949] Updated weights for policy 0, policy_version 2040 (0.0007) -[2023-10-14 13:28:13,453][75950] Updated weights for policy 1, policy_version 2020 (0.0009) -[2023-10-14 13:28:13,711][75615] Saving new best policy, reward=11.020! -[2023-10-14 13:28:13,822][75950] Updated weights for policy 1, policy_version 2030 (0.0009) -[2023-10-14 13:28:14,200][75950] Updated weights for policy 1, policy_version 2040 (0.0010) -[2023-10-14 13:28:17,487][75949] Updated weights for policy 0, policy_version 2050 (0.0011) -[2023-10-14 13:28:17,852][75949] Updated weights for policy 0, policy_version 2060 (0.0007) -[2023-10-14 13:28:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4194304. Throughput: 0: 1676.7, 1: 1658.8. Samples: 1059104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:28:18,164][74987] Avg episode reward: [(0, '10.230'), (1, '9.050')] -[2023-10-14 13:28:18,215][75949] Updated weights for policy 0, policy_version 2070 (0.0008) -[2023-10-14 13:28:18,340][75950] Updated weights for policy 1, policy_version 2050 (0.0007) -[2023-10-14 13:28:18,589][75949] Updated weights for policy 0, policy_version 2080 (0.0007) -[2023-10-14 13:28:18,708][75950] Updated weights for policy 1, policy_version 2060 (0.0007) -[2023-10-14 13:28:19,073][75950] Updated weights for policy 1, policy_version 2070 (0.0008) -[2023-10-14 13:28:19,450][75950] Updated weights for policy 1, policy_version 2080 (0.0009) -[2023-10-14 13:28:22,585][75949] Updated weights for policy 0, policy_version 2090 (0.0010) -[2023-10-14 13:28:22,952][75949] Updated weights for policy 0, policy_version 2100 (0.0010) -[2023-10-14 13:28:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4259840. Throughput: 0: 1677.8, 1: 1655.4. Samples: 1079676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:28:23,164][74987] Avg episode reward: [(0, '11.210'), (1, '8.560')] -[2023-10-14 13:28:23,331][75949] Updated weights for policy 0, policy_version 2110 (0.0007) -[2023-10-14 13:28:23,399][75615] Saving new best policy, reward=11.210! -[2023-10-14 13:28:23,484][75950] Updated weights for policy 1, policy_version 2090 (0.0009) -[2023-10-14 13:28:23,859][75950] Updated weights for policy 1, policy_version 2100 (0.0009) -[2023-10-14 13:28:24,232][75950] Updated weights for policy 1, policy_version 2110 (0.0008) -[2023-10-14 13:28:27,388][75949] Updated weights for policy 0, policy_version 2120 (0.0007) -[2023-10-14 13:28:27,756][75949] Updated weights for policy 0, policy_version 2130 (0.0010) -[2023-10-14 13:28:28,128][75949] Updated weights for policy 0, policy_version 2140 (0.0008) -[2023-10-14 13:28:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4325376. Throughput: 0: 1662.6, 1: 1662.2. Samples: 1099776. Policy #0 lag: (min: 26.0, avg: 26.2, max: 35.0) -[2023-10-14 13:28:28,164][74987] Avg episode reward: [(0, '10.350'), (1, '9.350')] -[2023-10-14 13:28:28,343][75950] Updated weights for policy 1, policy_version 2120 (0.0008) -[2023-10-14 13:28:28,719][75950] Updated weights for policy 1, policy_version 2130 (0.0008) -[2023-10-14 13:28:29,093][75950] Updated weights for policy 1, policy_version 2140 (0.0009) -[2023-10-14 13:28:32,369][75949] Updated weights for policy 0, policy_version 2150 (0.0008) -[2023-10-14 13:28:32,734][75949] Updated weights for policy 0, policy_version 2160 (0.0007) -[2023-10-14 13:28:33,099][75950] Updated weights for policy 1, policy_version 2150 (0.0008) -[2023-10-14 13:28:33,114][75949] Updated weights for policy 0, policy_version 2170 (0.0007) -[2023-10-14 13:28:33,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 4390912. Throughput: 0: 1676.6, 1: 1663.4. Samples: 1109292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:28:33,164][74987] Avg episode reward: [(0, '9.650'), (1, '8.940')] -[2023-10-14 13:28:33,460][75950] Updated weights for policy 1, policy_version 2160 (0.0008) -[2023-10-14 13:28:33,832][75950] Updated weights for policy 1, policy_version 2170 (0.0009) -[2023-10-14 13:28:37,170][75949] Updated weights for policy 0, policy_version 2180 (0.0008) -[2023-10-14 13:28:37,548][75949] Updated weights for policy 0, policy_version 2190 (0.0009) -[2023-10-14 13:28:37,845][75950] Updated weights for policy 1, policy_version 2180 (0.0007) -[2023-10-14 13:28:37,915][75949] Updated weights for policy 0, policy_version 2200 (0.0009) -[2023-10-14 13:28:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 4456448. Throughput: 0: 1676.4, 1: 1665.6. Samples: 1129804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:28:38,164][74987] Avg episode reward: [(0, '10.460'), (1, '8.800')] -[2023-10-14 13:28:38,213][75950] Updated weights for policy 1, policy_version 2190 (0.0007) -[2023-10-14 13:28:38,578][75950] Updated weights for policy 1, policy_version 2200 (0.0009) -[2023-10-14 13:28:42,027][75949] Updated weights for policy 0, policy_version 2210 (0.0009) -[2023-10-14 13:28:42,403][75949] Updated weights for policy 0, policy_version 2220 (0.0008) -[2023-10-14 13:28:42,567][75950] Updated weights for policy 1, policy_version 2210 (0.0010) -[2023-10-14 13:28:42,780][75949] Updated weights for policy 0, policy_version 2230 (0.0007) -[2023-10-14 13:28:42,937][75950] Updated weights for policy 1, policy_version 2220 (0.0007) -[2023-10-14 13:28:43,154][75949] Updated weights for policy 0, policy_version 2240 (0.0007) -[2023-10-14 13:28:43,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 4554752. Throughput: 0: 1658.9, 1: 1660.9. Samples: 1149556. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 13:28:43,164][74987] Avg episode reward: [(0, '10.030'), (1, '8.980')] -[2023-10-14 13:28:43,170][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000002240_2293760.pth... -[2023-10-14 13:28:43,201][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000000672_688128.pth -[2023-10-14 13:28:43,297][75950] Updated weights for policy 1, policy_version 2230 (0.0010) -[2023-10-14 13:28:43,671][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000002240_2293760.pth... -[2023-10-14 13:28:43,671][75950] Updated weights for policy 1, policy_version 2240 (0.0009) -[2023-10-14 13:28:43,710][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000000672_688128.pth -[2023-10-14 13:28:47,357][75949] Updated weights for policy 0, policy_version 2250 (0.0007) -[2023-10-14 13:28:47,727][75949] Updated weights for policy 0, policy_version 2260 (0.0007) -[2023-10-14 13:28:47,909][75950] Updated weights for policy 1, policy_version 2250 (0.0009) -[2023-10-14 13:28:48,094][75949] Updated weights for policy 0, policy_version 2270 (0.0007) -[2023-10-14 13:28:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 4587520. Throughput: 0: 1675.2, 1: 1665.3. Samples: 1159578. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 13:28:48,165][74987] Avg episode reward: [(0, '9.430'), (1, '8.900')] -[2023-10-14 13:28:48,285][75950] Updated weights for policy 1, policy_version 2260 (0.0009) -[2023-10-14 13:28:48,656][75950] Updated weights for policy 1, policy_version 2270 (0.0009) -[2023-10-14 13:28:52,316][75949] Updated weights for policy 0, policy_version 2280 (0.0008) -[2023-10-14 13:28:52,696][75949] Updated weights for policy 0, policy_version 2290 (0.0008) -[2023-10-14 13:28:52,781][75950] Updated weights for policy 1, policy_version 2280 (0.0007) -[2023-10-14 13:28:53,058][75949] Updated weights for policy 0, policy_version 2300 (0.0010) -[2023-10-14 13:28:53,154][75950] Updated weights for policy 1, policy_version 2290 (0.0007) -[2023-10-14 13:28:53,164][74987] Fps is (10 sec: 9830.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 4653056. Throughput: 0: 1672.1, 1: 1665.5. Samples: 1179794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:28:53,165][74987] Avg episode reward: [(0, '9.750'), (1, '9.510')] -[2023-10-14 13:28:53,517][75950] Updated weights for policy 1, policy_version 2300 (0.0007) -[2023-10-14 13:28:53,662][75801] Saving new best policy, reward=9.510! -[2023-10-14 13:28:57,230][75949] Updated weights for policy 0, policy_version 2310 (0.0008) -[2023-10-14 13:28:57,605][75949] Updated weights for policy 0, policy_version 2320 (0.0009) -[2023-10-14 13:28:57,819][75950] Updated weights for policy 1, policy_version 2310 (0.0008) -[2023-10-14 13:28:57,982][75949] Updated weights for policy 0, policy_version 2330 (0.0009) -[2023-10-14 13:28:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 4718592. Throughput: 0: 1656.3, 1: 1666.5. Samples: 1199332. Policy #0 lag: (min: 9.0, avg: 30.1, max: 41.0) -[2023-10-14 13:28:58,164][74987] Avg episode reward: [(0, '9.900'), (1, '9.520')] -[2023-10-14 13:28:58,183][75950] Updated weights for policy 1, policy_version 2320 (0.0008) -[2023-10-14 13:28:58,556][75950] Updated weights for policy 1, policy_version 2330 (0.0008) -[2023-10-14 13:28:58,782][75801] Saving new best policy, reward=9.520! -[2023-10-14 13:29:02,063][75949] Updated weights for policy 0, policy_version 2340 (0.0010) -[2023-10-14 13:29:02,440][75949] Updated weights for policy 0, policy_version 2350 (0.0009) -[2023-10-14 13:29:02,741][75950] Updated weights for policy 1, policy_version 2340 (0.0008) -[2023-10-14 13:29:02,814][75949] Updated weights for policy 0, policy_version 2360 (0.0009) -[2023-10-14 13:29:03,101][75950] Updated weights for policy 1, policy_version 2350 (0.0007) -[2023-10-14 13:29:03,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 4816896. Throughput: 0: 1665.4, 1: 1667.0. Samples: 1209064. Policy #0 lag: (min: 14.0, avg: 22.0, max: 46.0) -[2023-10-14 13:29:03,165][74987] Avg episode reward: [(0, '10.260'), (1, '9.460')] -[2023-10-14 13:29:03,466][75950] Updated weights for policy 1, policy_version 2360 (0.0007) -[2023-10-14 13:29:06,919][75949] Updated weights for policy 0, policy_version 2370 (0.0008) -[2023-10-14 13:29:07,288][75949] Updated weights for policy 0, policy_version 2380 (0.0010) -[2023-10-14 13:29:07,599][75950] Updated weights for policy 1, policy_version 2370 (0.0008) -[2023-10-14 13:29:07,664][75949] Updated weights for policy 0, policy_version 2390 (0.0010) -[2023-10-14 13:29:07,974][75950] Updated weights for policy 1, policy_version 2380 (0.0010) -[2023-10-14 13:29:08,031][75949] Updated weights for policy 0, policy_version 2400 (0.0008) -[2023-10-14 13:29:08,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 4882432. Throughput: 0: 1660.8, 1: 1666.2. Samples: 1229390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:29:08,164][74987] Avg episode reward: [(0, '10.400'), (1, '9.480')] -[2023-10-14 13:29:08,345][75950] Updated weights for policy 1, policy_version 2390 (0.0008) -[2023-10-14 13:29:08,716][75950] Updated weights for policy 1, policy_version 2400 (0.0008) -[2023-10-14 13:29:12,083][75949] Updated weights for policy 0, policy_version 2410 (0.0010) -[2023-10-14 13:29:12,455][75949] Updated weights for policy 0, policy_version 2420 (0.0010) -[2023-10-14 13:29:12,818][75950] Updated weights for policy 1, policy_version 2410 (0.0007) -[2023-10-14 13:29:12,825][75949] Updated weights for policy 0, policy_version 2430 (0.0008) -[2023-10-14 13:29:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 4947968. Throughput: 0: 1651.6, 1: 1658.8. Samples: 1248748. Policy #0 lag: (min: 4.0, avg: 26.8, max: 36.0) -[2023-10-14 13:29:13,165][74987] Avg episode reward: [(0, '10.130'), (1, '9.090')] -[2023-10-14 13:29:13,193][75950] Updated weights for policy 1, policy_version 2420 (0.0007) -[2023-10-14 13:29:13,570][75950] Updated weights for policy 1, policy_version 2430 (0.0009) -[2023-10-14 13:29:17,018][75949] Updated weights for policy 0, policy_version 2440 (0.0007) -[2023-10-14 13:29:17,391][75949] Updated weights for policy 0, policy_version 2450 (0.0009) -[2023-10-14 13:29:17,582][75950] Updated weights for policy 1, policy_version 2440 (0.0008) -[2023-10-14 13:29:17,750][75949] Updated weights for policy 0, policy_version 2460 (0.0009) -[2023-10-14 13:29:17,945][75950] Updated weights for policy 1, policy_version 2450 (0.0007) -[2023-10-14 13:29:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 5013504. Throughput: 0: 1660.2, 1: 1667.2. Samples: 1259022. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 13:29:18,164][74987] Avg episode reward: [(0, '9.680'), (1, '9.140')] -[2023-10-14 13:29:18,315][75950] Updated weights for policy 1, policy_version 2460 (0.0011) -[2023-10-14 13:29:21,861][75949] Updated weights for policy 0, policy_version 2470 (0.0010) -[2023-10-14 13:29:22,245][75949] Updated weights for policy 0, policy_version 2480 (0.0008) -[2023-10-14 13:29:22,459][75950] Updated weights for policy 1, policy_version 2470 (0.0008) -[2023-10-14 13:29:22,615][75949] Updated weights for policy 0, policy_version 2490 (0.0008) -[2023-10-14 13:29:22,824][75950] Updated weights for policy 1, policy_version 2480 (0.0008) -[2023-10-14 13:29:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 5079040. Throughput: 0: 1660.1, 1: 1662.5. Samples: 1279324. Policy #0 lag: (min: 21.0, avg: 21.1, max: 29.0) -[2023-10-14 13:29:23,168][74987] Avg episode reward: [(0, '10.100'), (1, '9.230')] -[2023-10-14 13:29:23,201][75950] Updated weights for policy 1, policy_version 2490 (0.0008) -[2023-10-14 13:29:26,771][75949] Updated weights for policy 0, policy_version 2500 (0.0009) -[2023-10-14 13:29:27,138][75949] Updated weights for policy 0, policy_version 2510 (0.0008) -[2023-10-14 13:29:27,319][75950] Updated weights for policy 1, policy_version 2500 (0.0007) -[2023-10-14 13:29:27,506][75949] Updated weights for policy 0, policy_version 2520 (0.0009) -[2023-10-14 13:29:27,690][75950] Updated weights for policy 1, policy_version 2510 (0.0007) -[2023-10-14 13:29:28,060][75950] Updated weights for policy 1, policy_version 2520 (0.0008) -[2023-10-14 13:29:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 5144576. Throughput: 0: 1649.6, 1: 1651.0. Samples: 1298086. Policy #0 lag: (min: 18.0, avg: 18.1, max: 25.0) -[2023-10-14 13:29:28,164][74987] Avg episode reward: [(0, '9.950'), (1, '9.110')] -[2023-10-14 13:29:31,572][75949] Updated weights for policy 0, policy_version 2530 (0.0009) -[2023-10-14 13:29:31,946][75949] Updated weights for policy 0, policy_version 2540 (0.0008) -[2023-10-14 13:29:32,030][75950] Updated weights for policy 1, policy_version 2530 (0.0008) -[2023-10-14 13:29:32,321][75949] Updated weights for policy 0, policy_version 2550 (0.0008) -[2023-10-14 13:29:32,401][75950] Updated weights for policy 1, policy_version 2540 (0.0008) -[2023-10-14 13:29:32,698][75949] Updated weights for policy 0, policy_version 2560 (0.0010) -[2023-10-14 13:29:32,778][75950] Updated weights for policy 1, policy_version 2550 (0.0008) -[2023-10-14 13:29:33,154][75950] Updated weights for policy 1, policy_version 2560 (0.0008) -[2023-10-14 13:29:33,163][74987] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 5242880. Throughput: 0: 1656.7, 1: 1661.2. Samples: 1308882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:29:33,164][74987] Avg episode reward: [(0, '10.180'), (1, '9.720')] -[2023-10-14 13:29:33,165][75801] Saving new best policy, reward=9.720! -[2023-10-14 13:29:36,721][75949] Updated weights for policy 0, policy_version 2570 (0.0007) -[2023-10-14 13:29:37,090][75949] Updated weights for policy 0, policy_version 2580 (0.0008) -[2023-10-14 13:29:37,355][75950] Updated weights for policy 1, policy_version 2570 (0.0008) -[2023-10-14 13:29:37,458][75949] Updated weights for policy 0, policy_version 2590 (0.0007) -[2023-10-14 13:29:37,727][75950] Updated weights for policy 1, policy_version 2580 (0.0008) -[2023-10-14 13:29:38,092][75950] Updated weights for policy 1, policy_version 2590 (0.0008) -[2023-10-14 13:29:38,168][74987] Fps is (10 sec: 16376.0, 60 sec: 14198.3, 300 sec: 13329.2). Total num frames: 5308416. Throughput: 0: 1649.7, 1: 1664.9. Samples: 1328964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:29:38,169][74987] Avg episode reward: [(0, '10.150'), (1, '9.570')] -[2023-10-14 13:29:41,449][75949] Updated weights for policy 0, policy_version 2600 (0.0009) -[2023-10-14 13:29:41,837][75949] Updated weights for policy 0, policy_version 2610 (0.0008) -[2023-10-14 13:29:42,084][75950] Updated weights for policy 1, policy_version 2600 (0.0010) -[2023-10-14 13:29:42,202][75949] Updated weights for policy 0, policy_version 2620 (0.0009) -[2023-10-14 13:29:42,453][75950] Updated weights for policy 1, policy_version 2610 (0.0008) -[2023-10-14 13:29:42,818][75950] Updated weights for policy 1, policy_version 2620 (0.0008) -[2023-10-14 13:29:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 5373952. Throughput: 0: 1651.3, 1: 1649.3. Samples: 1347860. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 13:29:43,164][74987] Avg episode reward: [(0, '10.260'), (1, '9.150')] -[2023-10-14 13:29:46,294][75949] Updated weights for policy 0, policy_version 2630 (0.0009) -[2023-10-14 13:29:46,668][75949] Updated weights for policy 0, policy_version 2640 (0.0010) -[2023-10-14 13:29:46,950][75950] Updated weights for policy 1, policy_version 2630 (0.0008) -[2023-10-14 13:29:47,032][75949] Updated weights for policy 0, policy_version 2650 (0.0008) -[2023-10-14 13:29:47,308][75950] Updated weights for policy 1, policy_version 2640 (0.0008) -[2023-10-14 13:29:47,680][75950] Updated weights for policy 1, policy_version 2650 (0.0009) -[2023-10-14 13:29:48,163][74987] Fps is (10 sec: 13113.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 5439488. Throughput: 0: 1667.1, 1: 1669.9. Samples: 1359226. Policy #0 lag: (min: 31.0, avg: 31.6, max: 47.0) -[2023-10-14 13:29:48,164][74987] Avg episode reward: [(0, '9.960'), (1, '9.110')] -[2023-10-14 13:29:51,198][75949] Updated weights for policy 0, policy_version 2660 (0.0007) -[2023-10-14 13:29:51,563][75949] Updated weights for policy 0, policy_version 2670 (0.0008) -[2023-10-14 13:29:51,865][75950] Updated weights for policy 1, policy_version 2660 (0.0008) -[2023-10-14 13:29:51,938][75949] Updated weights for policy 0, policy_version 2680 (0.0009) -[2023-10-14 13:29:52,225][75950] Updated weights for policy 1, policy_version 2670 (0.0007) -[2023-10-14 13:29:52,590][75950] Updated weights for policy 1, policy_version 2680 (0.0009) -[2023-10-14 13:29:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13329.3). Total num frames: 5505024. Throughput: 0: 1656.9, 1: 1673.1. Samples: 1379240. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) -[2023-10-14 13:29:53,165][74987] Avg episode reward: [(0, '10.080'), (1, '9.430')] -[2023-10-14 13:29:55,891][75949] Updated weights for policy 0, policy_version 2690 (0.0007) -[2023-10-14 13:29:56,264][75949] Updated weights for policy 0, policy_version 2700 (0.0009) -[2023-10-14 13:29:56,589][75950] Updated weights for policy 1, policy_version 2690 (0.0010) -[2023-10-14 13:29:56,652][75949] Updated weights for policy 0, policy_version 2710 (0.0009) -[2023-10-14 13:29:56,965][75950] Updated weights for policy 1, policy_version 2700 (0.0008) -[2023-10-14 13:29:57,023][75949] Updated weights for policy 0, policy_version 2720 (0.0008) -[2023-10-14 13:29:57,334][75950] Updated weights for policy 1, policy_version 2710 (0.0008) -[2023-10-14 13:29:57,702][75950] Updated weights for policy 1, policy_version 2720 (0.0008) -[2023-10-14 13:29:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 5570560. Throughput: 0: 1669.7, 1: 1656.1. Samples: 1398412. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 13:29:58,164][74987] Avg episode reward: [(0, '10.660'), (1, '9.470')] -[2023-10-14 13:30:01,056][75949] Updated weights for policy 0, policy_version 2730 (0.0008) -[2023-10-14 13:30:01,434][75949] Updated weights for policy 0, policy_version 2740 (0.0008) -[2023-10-14 13:30:01,725][75950] Updated weights for policy 1, policy_version 2730 (0.0008) -[2023-10-14 13:30:01,805][75949] Updated weights for policy 0, policy_version 2750 (0.0007) -[2023-10-14 13:30:02,093][75950] Updated weights for policy 1, policy_version 2740 (0.0011) -[2023-10-14 13:30:02,456][75950] Updated weights for policy 1, policy_version 2750 (0.0009) -[2023-10-14 13:30:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 5636096. Throughput: 0: 1678.4, 1: 1679.7. Samples: 1410134. Policy #0 lag: (min: 15.0, avg: 20.2, max: 47.0) -[2023-10-14 13:30:03,164][74987] Avg episode reward: [(0, '10.270'), (1, '9.760')] -[2023-10-14 13:30:03,165][75801] Saving new best policy, reward=9.760! -[2023-10-14 13:30:05,798][75949] Updated weights for policy 0, policy_version 2760 (0.0009) -[2023-10-14 13:30:06,165][75949] Updated weights for policy 0, policy_version 2770 (0.0008) -[2023-10-14 13:30:06,528][75950] Updated weights for policy 1, policy_version 2760 (0.0010) -[2023-10-14 13:30:06,534][75949] Updated weights for policy 0, policy_version 2780 (0.0009) -[2023-10-14 13:30:06,893][75950] Updated weights for policy 1, policy_version 2770 (0.0007) -[2023-10-14 13:30:07,264][75950] Updated weights for policy 1, policy_version 2780 (0.0009) -[2023-10-14 13:30:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 5701632. Throughput: 0: 1658.1, 1: 1673.9. Samples: 1429266. Policy #0 lag: (min: 25.0, avg: 42.3, max: 57.0) -[2023-10-14 13:30:08,164][74987] Avg episode reward: [(0, '10.560'), (1, '9.710')] -[2023-10-14 13:30:10,799][75949] Updated weights for policy 0, policy_version 2790 (0.0008) -[2023-10-14 13:30:11,187][75949] Updated weights for policy 0, policy_version 2800 (0.0009) -[2023-10-14 13:30:11,311][75950] Updated weights for policy 1, policy_version 2790 (0.0009) -[2023-10-14 13:30:11,546][75949] Updated weights for policy 0, policy_version 2810 (0.0008) -[2023-10-14 13:30:11,675][75950] Updated weights for policy 1, policy_version 2800 (0.0009) -[2023-10-14 13:30:12,058][75950] Updated weights for policy 1, policy_version 2810 (0.0010) -[2023-10-14 13:30:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 5767168. Throughput: 0: 1680.2, 1: 1665.8. Samples: 1448654. Policy #0 lag: (min: 19.0, avg: 19.4, max: 32.0) -[2023-10-14 13:30:13,165][74987] Avg episode reward: [(0, '10.550'), (1, '9.700')] -[2023-10-14 13:30:15,567][75949] Updated weights for policy 0, policy_version 2820 (0.0007) -[2023-10-14 13:30:15,952][75949] Updated weights for policy 0, policy_version 2830 (0.0008) -[2023-10-14 13:30:16,151][75950] Updated weights for policy 1, policy_version 2820 (0.0008) -[2023-10-14 13:30:16,326][75949] Updated weights for policy 0, policy_version 2840 (0.0008) -[2023-10-14 13:30:16,514][75950] Updated weights for policy 1, policy_version 2830 (0.0010) -[2023-10-14 13:30:16,886][75950] Updated weights for policy 1, policy_version 2840 (0.0008) -[2023-10-14 13:30:18,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 5832704. Throughput: 0: 1677.2, 1: 1683.3. Samples: 1460106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) -[2023-10-14 13:30:18,164][74987] Avg episode reward: [(0, '10.190'), (1, '9.600')] -[2023-10-14 13:30:20,403][75949] Updated weights for policy 0, policy_version 2850 (0.0010) -[2023-10-14 13:30:20,783][75949] Updated weights for policy 0, policy_version 2860 (0.0011) -[2023-10-14 13:30:21,146][75950] Updated weights for policy 1, policy_version 2850 (0.0009) -[2023-10-14 13:30:21,154][75949] Updated weights for policy 0, policy_version 2870 (0.0009) -[2023-10-14 13:30:21,513][75950] Updated weights for policy 1, policy_version 2860 (0.0007) -[2023-10-14 13:30:21,527][75949] Updated weights for policy 0, policy_version 2880 (0.0008) -[2023-10-14 13:30:21,875][75950] Updated weights for policy 1, policy_version 2870 (0.0009) -[2023-10-14 13:30:22,239][75950] Updated weights for policy 1, policy_version 2880 (0.0009) -[2023-10-14 13:30:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 5898240. Throughput: 0: 1664.2, 1: 1672.6. Samples: 1479106. Policy #0 lag: (min: 27.0, avg: 46.1, max: 48.0) -[2023-10-14 13:30:23,165][74987] Avg episode reward: [(0, '10.320'), (1, '9.540')] -[2023-10-14 13:30:25,621][75949] Updated weights for policy 0, policy_version 2890 (0.0007) -[2023-10-14 13:30:25,995][75949] Updated weights for policy 0, policy_version 2900 (0.0007) -[2023-10-14 13:30:26,249][75950] Updated weights for policy 1, policy_version 2890 (0.0008) -[2023-10-14 13:30:26,368][75949] Updated weights for policy 0, policy_version 2910 (0.0007) -[2023-10-14 13:30:26,626][75950] Updated weights for policy 1, policy_version 2900 (0.0007) -[2023-10-14 13:30:26,985][75950] Updated weights for policy 1, policy_version 2910 (0.0007) -[2023-10-14 13:30:28,164][74987] Fps is (10 sec: 13106.6, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 5963776. Throughput: 0: 1683.4, 1: 1678.3. Samples: 1499136. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 13:30:28,165][74987] Avg episode reward: [(0, '10.890'), (1, '10.060')] -[2023-10-14 13:30:28,175][75801] Saving new best policy, reward=10.060! -[2023-10-14 13:30:30,450][75949] Updated weights for policy 0, policy_version 2920 (0.0008) -[2023-10-14 13:30:30,815][75949] Updated weights for policy 0, policy_version 2930 (0.0008) -[2023-10-14 13:30:30,920][75950] Updated weights for policy 1, policy_version 2920 (0.0009) -[2023-10-14 13:30:31,186][75949] Updated weights for policy 0, policy_version 2940 (0.0008) -[2023-10-14 13:30:31,283][75950] Updated weights for policy 1, policy_version 2930 (0.0010) -[2023-10-14 13:30:31,655][75950] Updated weights for policy 1, policy_version 2940 (0.0010) -[2023-10-14 13:30:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6029312. Throughput: 0: 1671.9, 1: 1686.6. Samples: 1510360. Policy #0 lag: (min: 3.0, avg: 10.3, max: 35.0) -[2023-10-14 13:30:33,164][74987] Avg episode reward: [(0, '10.260'), (1, '8.840')] -[2023-10-14 13:30:35,366][75949] Updated weights for policy 0, policy_version 2950 (0.0008) -[2023-10-14 13:30:35,747][75949] Updated weights for policy 0, policy_version 2960 (0.0010) -[2023-10-14 13:30:35,815][75950] Updated weights for policy 1, policy_version 2950 (0.0008) -[2023-10-14 13:30:36,115][75949] Updated weights for policy 0, policy_version 2970 (0.0009) -[2023-10-14 13:30:36,192][75950] Updated weights for policy 1, policy_version 2960 (0.0009) -[2023-10-14 13:30:36,555][75950] Updated weights for policy 1, policy_version 2970 (0.0009) -[2023-10-14 13:30:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13108.2, 300 sec: 13329.4). Total num frames: 6094848. Throughput: 0: 1670.3, 1: 1662.9. Samples: 1529234. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 13:30:38,165][74987] Avg episode reward: [(0, '10.590'), (1, '9.260')] -[2023-10-14 13:30:40,001][75949] Updated weights for policy 0, policy_version 2980 (0.0009) -[2023-10-14 13:30:40,376][75949] Updated weights for policy 0, policy_version 2990 (0.0008) -[2023-10-14 13:30:40,635][75950] Updated weights for policy 1, policy_version 2980 (0.0009) -[2023-10-14 13:30:40,745][75949] Updated weights for policy 0, policy_version 3000 (0.0008) -[2023-10-14 13:30:41,007][75950] Updated weights for policy 1, policy_version 2990 (0.0007) -[2023-10-14 13:30:41,367][75950] Updated weights for policy 1, policy_version 3000 (0.0009) -[2023-10-14 13:30:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 6160384. Throughput: 0: 1681.5, 1: 1678.0. Samples: 1549594. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 13:30:43,165][74987] Avg episode reward: [(0, '10.340'), (1, '9.830')] -[2023-10-14 13:30:43,178][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000003008_3080192.pth... -[2023-10-14 13:30:43,178][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000003008_3080192.pth... -[2023-10-14 13:30:43,213][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000001440_1474560.pth -[2023-10-14 13:30:43,220][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000001440_1474560.pth -[2023-10-14 13:30:44,885][75949] Updated weights for policy 0, policy_version 3010 (0.0009) -[2023-10-14 13:30:45,262][75949] Updated weights for policy 0, policy_version 3020 (0.0008) -[2023-10-14 13:30:45,494][75950] Updated weights for policy 1, policy_version 3010 (0.0009) -[2023-10-14 13:30:45,646][75949] Updated weights for policy 0, policy_version 3030 (0.0009) -[2023-10-14 13:30:45,856][75950] Updated weights for policy 1, policy_version 3020 (0.0008) -[2023-10-14 13:30:46,015][75949] Updated weights for policy 0, policy_version 3040 (0.0008) -[2023-10-14 13:30:46,226][75950] Updated weights for policy 1, policy_version 3030 (0.0008) -[2023-10-14 13:30:46,590][75950] Updated weights for policy 1, policy_version 3040 (0.0008) -[2023-10-14 13:30:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6225920. Throughput: 0: 1659.0, 1: 1671.3. Samples: 1560000. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 13:30:48,165][74987] Avg episode reward: [(0, '9.900'), (1, '9.430')] -[2023-10-14 13:30:49,901][75949] Updated weights for policy 0, policy_version 3050 (0.0007) -[2023-10-14 13:30:50,270][75949] Updated weights for policy 0, policy_version 3060 (0.0007) -[2023-10-14 13:30:50,614][75950] Updated weights for policy 1, policy_version 3050 (0.0008) -[2023-10-14 13:30:50,642][75949] Updated weights for policy 0, policy_version 3070 (0.0009) -[2023-10-14 13:30:50,981][75950] Updated weights for policy 1, policy_version 3060 (0.0008) -[2023-10-14 13:30:51,345][75950] Updated weights for policy 1, policy_version 3070 (0.0008) -[2023-10-14 13:30:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6291456. Throughput: 0: 1677.2, 1: 1656.3. Samples: 1579270. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 13:30:53,165][74987] Avg episode reward: [(0, '10.280'), (1, '10.060')] -[2023-10-14 13:30:54,684][75949] Updated weights for policy 0, policy_version 3080 (0.0009) -[2023-10-14 13:30:55,052][75949] Updated weights for policy 0, policy_version 3090 (0.0008) -[2023-10-14 13:30:55,420][75949] Updated weights for policy 0, policy_version 3100 (0.0008) -[2023-10-14 13:30:55,483][75950] Updated weights for policy 1, policy_version 3080 (0.0008) -[2023-10-14 13:30:55,868][75950] Updated weights for policy 1, policy_version 3090 (0.0010) -[2023-10-14 13:30:56,231][75950] Updated weights for policy 1, policy_version 3100 (0.0010) -[2023-10-14 13:30:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6356992. Throughput: 0: 1683.5, 1: 1678.0. Samples: 1599924. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 13:30:58,164][74987] Avg episode reward: [(0, '10.820'), (1, '9.210')] -[2023-10-14 13:30:59,448][75949] Updated weights for policy 0, policy_version 3110 (0.0009) -[2023-10-14 13:30:59,809][75949] Updated weights for policy 0, policy_version 3120 (0.0009) -[2023-10-14 13:31:00,185][75949] Updated weights for policy 0, policy_version 3130 (0.0007) -[2023-10-14 13:31:00,234][75950] Updated weights for policy 1, policy_version 3110 (0.0008) -[2023-10-14 13:31:00,599][75950] Updated weights for policy 1, policy_version 3120 (0.0007) -[2023-10-14 13:31:00,953][75950] Updated weights for policy 1, policy_version 3130 (0.0008) -[2023-10-14 13:31:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6422528. Throughput: 0: 1663.3, 1: 1665.0. Samples: 1609880. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) -[2023-10-14 13:31:03,165][74987] Avg episode reward: [(0, '10.580'), (1, '9.610')] -[2023-10-14 13:31:04,317][75949] Updated weights for policy 0, policy_version 3140 (0.0008) -[2023-10-14 13:31:04,686][75949] Updated weights for policy 0, policy_version 3150 (0.0008) -[2023-10-14 13:31:05,058][75950] Updated weights for policy 1, policy_version 3140 (0.0009) -[2023-10-14 13:31:05,065][75949] Updated weights for policy 0, policy_version 3160 (0.0007) -[2023-10-14 13:31:05,419][75950] Updated weights for policy 1, policy_version 3150 (0.0007) -[2023-10-14 13:31:05,780][75950] Updated weights for policy 1, policy_version 3160 (0.0010) -[2023-10-14 13:31:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6488064. Throughput: 0: 1693.1, 1: 1660.8. Samples: 1630030. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) -[2023-10-14 13:31:08,165][74987] Avg episode reward: [(0, '10.230'), (1, '10.280')] -[2023-10-14 13:31:08,166][75801] Saving new best policy, reward=10.280! -[2023-10-14 13:31:09,190][75949] Updated weights for policy 0, policy_version 3170 (0.0009) -[2023-10-14 13:31:09,561][75949] Updated weights for policy 0, policy_version 3180 (0.0008) -[2023-10-14 13:31:09,841][75950] Updated weights for policy 1, policy_version 3170 (0.0010) -[2023-10-14 13:31:09,938][75949] Updated weights for policy 0, policy_version 3190 (0.0010) -[2023-10-14 13:31:10,211][75950] Updated weights for policy 1, policy_version 3180 (0.0007) -[2023-10-14 13:31:10,299][75949] Updated weights for policy 0, policy_version 3200 (0.0007) -[2023-10-14 13:31:10,577][75950] Updated weights for policy 1, policy_version 3190 (0.0008) -[2023-10-14 13:31:10,957][75950] Updated weights for policy 1, policy_version 3200 (0.0007) -[2023-10-14 13:31:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6553600. Throughput: 0: 1689.2, 1: 1674.3. Samples: 1650494. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 13:31:13,164][74987] Avg episode reward: [(0, '10.510'), (1, '9.780')] -[2023-10-14 13:31:14,336][75949] Updated weights for policy 0, policy_version 3210 (0.0009) -[2023-10-14 13:31:14,704][75949] Updated weights for policy 0, policy_version 3220 (0.0008) -[2023-10-14 13:31:15,071][75950] Updated weights for policy 1, policy_version 3210 (0.0008) -[2023-10-14 13:31:15,074][75949] Updated weights for policy 0, policy_version 3230 (0.0007) -[2023-10-14 13:31:15,438][75950] Updated weights for policy 1, policy_version 3220 (0.0008) -[2023-10-14 13:31:15,803][75950] Updated weights for policy 1, policy_version 3230 (0.0007) -[2023-10-14 13:31:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6619136. Throughput: 0: 1671.9, 1: 1654.8. Samples: 1660064. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 13:31:18,164][74987] Avg episode reward: [(0, '9.950'), (1, '9.870')] -[2023-10-14 13:31:19,142][75949] Updated weights for policy 0, policy_version 3240 (0.0008) -[2023-10-14 13:31:19,518][75949] Updated weights for policy 0, policy_version 3250 (0.0007) -[2023-10-14 13:31:19,764][75950] Updated weights for policy 1, policy_version 3240 (0.0007) -[2023-10-14 13:31:19,883][75949] Updated weights for policy 0, policy_version 3260 (0.0007) -[2023-10-14 13:31:20,128][75950] Updated weights for policy 1, policy_version 3250 (0.0008) -[2023-10-14 13:31:20,505][75950] Updated weights for policy 1, policy_version 3260 (0.0008) -[2023-10-14 13:31:23,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6684672. Throughput: 0: 1688.3, 1: 1671.7. Samples: 1680430. Policy #0 lag: (min: 19.0, avg: 19.3, max: 30.0) -[2023-10-14 13:31:23,164][74987] Avg episode reward: [(0, '10.850'), (1, '10.480')] -[2023-10-14 13:31:23,165][75801] Saving new best policy, reward=10.480! -[2023-10-14 13:31:23,741][75949] Updated weights for policy 0, policy_version 3270 (0.0007) -[2023-10-14 13:31:24,107][75949] Updated weights for policy 0, policy_version 3280 (0.0008) -[2023-10-14 13:31:24,481][75949] Updated weights for policy 0, policy_version 3290 (0.0011) -[2023-10-14 13:31:24,692][75950] Updated weights for policy 1, policy_version 3270 (0.0009) -[2023-10-14 13:31:25,062][75950] Updated weights for policy 1, policy_version 3280 (0.0010) -[2023-10-14 13:31:25,436][75950] Updated weights for policy 1, policy_version 3290 (0.0008) -[2023-10-14 13:31:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 6750208. Throughput: 0: 1690.5, 1: 1675.7. Samples: 1701074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:31:28,164][74987] Avg episode reward: [(0, '10.230'), (1, '9.860')] -[2023-10-14 13:31:28,409][75949] Updated weights for policy 0, policy_version 3300 (0.0007) -[2023-10-14 13:31:28,782][75949] Updated weights for policy 0, policy_version 3310 (0.0009) -[2023-10-14 13:31:29,158][75949] Updated weights for policy 0, policy_version 3320 (0.0007) -[2023-10-14 13:31:29,631][75950] Updated weights for policy 1, policy_version 3300 (0.0011) -[2023-10-14 13:31:29,997][75950] Updated weights for policy 1, policy_version 3310 (0.0009) -[2023-10-14 13:31:30,364][75950] Updated weights for policy 1, policy_version 3320 (0.0009) -[2023-10-14 13:31:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6815744. Throughput: 0: 1685.3, 1: 1655.0. Samples: 1710316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:31:33,165][74987] Avg episode reward: [(0, '11.190'), (1, '10.940')] -[2023-10-14 13:31:33,166][75801] Saving new best policy, reward=10.940! -[2023-10-14 13:31:33,220][75949] Updated weights for policy 0, policy_version 3330 (0.0008) -[2023-10-14 13:31:33,601][75949] Updated weights for policy 0, policy_version 3340 (0.0008) -[2023-10-14 13:31:33,964][75949] Updated weights for policy 0, policy_version 3350 (0.0008) -[2023-10-14 13:31:34,330][75950] Updated weights for policy 1, policy_version 3330 (0.0007) -[2023-10-14 13:31:34,346][75949] Updated weights for policy 0, policy_version 3360 (0.0008) -[2023-10-14 13:31:34,705][75950] Updated weights for policy 1, policy_version 3340 (0.0009) -[2023-10-14 13:31:35,066][75950] Updated weights for policy 1, policy_version 3350 (0.0009) -[2023-10-14 13:31:35,444][75950] Updated weights for policy 1, policy_version 3360 (0.0009) -[2023-10-14 13:31:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6881280. Throughput: 0: 1689.0, 1: 1675.2. Samples: 1730660. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-14 13:31:38,164][74987] Avg episode reward: [(0, '10.530'), (1, '9.690')] -[2023-10-14 13:31:38,464][75949] Updated weights for policy 0, policy_version 3370 (0.0009) -[2023-10-14 13:31:38,847][75949] Updated weights for policy 0, policy_version 3380 (0.0009) -[2023-10-14 13:31:39,217][75949] Updated weights for policy 0, policy_version 3390 (0.0009) -[2023-10-14 13:31:39,566][75950] Updated weights for policy 1, policy_version 3370 (0.0011) -[2023-10-14 13:31:39,949][75950] Updated weights for policy 1, policy_version 3380 (0.0011) -[2023-10-14 13:31:40,311][75950] Updated weights for policy 1, policy_version 3390 (0.0011) -[2023-10-14 13:31:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6946816. Throughput: 0: 1687.3, 1: 1674.1. Samples: 1751188. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) -[2023-10-14 13:31:43,165][74987] Avg episode reward: [(0, '10.480'), (1, '9.920')] -[2023-10-14 13:31:43,333][75949] Updated weights for policy 0, policy_version 3400 (0.0008) -[2023-10-14 13:31:43,706][75949] Updated weights for policy 0, policy_version 3410 (0.0009) -[2023-10-14 13:31:44,078][75949] Updated weights for policy 0, policy_version 3420 (0.0011) -[2023-10-14 13:31:44,635][75950] Updated weights for policy 1, policy_version 3400 (0.0008) -[2023-10-14 13:31:45,008][75950] Updated weights for policy 1, policy_version 3410 (0.0009) -[2023-10-14 13:31:45,381][75950] Updated weights for policy 1, policy_version 3420 (0.0010) -[2023-10-14 13:31:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 7012352. Throughput: 0: 1685.5, 1: 1655.8. Samples: 1760236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:31:48,165][74987] Avg episode reward: [(0, '10.560'), (1, '10.260')] -[2023-10-14 13:31:48,178][75949] Updated weights for policy 0, policy_version 3430 (0.0009) -[2023-10-14 13:31:48,553][75949] Updated weights for policy 0, policy_version 3440 (0.0011) -[2023-10-14 13:31:48,919][75949] Updated weights for policy 0, policy_version 3450 (0.0010) -[2023-10-14 13:31:49,633][75950] Updated weights for policy 1, policy_version 3430 (0.0009) -[2023-10-14 13:31:50,002][75950] Updated weights for policy 1, policy_version 3440 (0.0007) -[2023-10-14 13:31:50,369][75950] Updated weights for policy 1, policy_version 3450 (0.0007) -[2023-10-14 13:31:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 7077888. Throughput: 0: 1682.4, 1: 1665.9. Samples: 1780702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:31:53,165][74987] Avg episode reward: [(0, '10.640'), (1, '9.630')] -[2023-10-14 13:31:53,173][75949] Updated weights for policy 0, policy_version 3460 (0.0008) -[2023-10-14 13:31:53,543][75949] Updated weights for policy 0, policy_version 3470 (0.0007) -[2023-10-14 13:31:53,917][75949] Updated weights for policy 0, policy_version 3480 (0.0008) -[2023-10-14 13:31:54,464][75950] Updated weights for policy 1, policy_version 3460 (0.0008) -[2023-10-14 13:31:54,834][75950] Updated weights for policy 1, policy_version 3470 (0.0008) -[2023-10-14 13:31:55,204][75950] Updated weights for policy 1, policy_version 3480 (0.0010) -[2023-10-14 13:31:57,970][75949] Updated weights for policy 0, policy_version 3490 (0.0011) -[2023-10-14 13:31:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 7143424. Throughput: 0: 1682.2, 1: 1666.9. Samples: 1801206. Policy #0 lag: (min: 7.0, avg: 7.1, max: 15.0) -[2023-10-14 13:31:58,166][74987] Avg episode reward: [(0, '10.720'), (1, '10.410')] -[2023-10-14 13:31:58,347][75949] Updated weights for policy 0, policy_version 3500 (0.0008) -[2023-10-14 13:31:58,713][75949] Updated weights for policy 0, policy_version 3510 (0.0008) -[2023-10-14 13:31:59,089][75949] Updated weights for policy 0, policy_version 3520 (0.0007) -[2023-10-14 13:31:59,343][75950] Updated weights for policy 1, policy_version 3490 (0.0009) -[2023-10-14 13:31:59,719][75950] Updated weights for policy 1, policy_version 3500 (0.0007) -[2023-10-14 13:32:00,078][75950] Updated weights for policy 1, policy_version 3510 (0.0008) -[2023-10-14 13:32:00,455][75950] Updated weights for policy 1, policy_version 3520 (0.0008) -[2023-10-14 13:32:03,059][75949] Updated weights for policy 0, policy_version 3530 (0.0010) -[2023-10-14 13:32:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7208960. Throughput: 0: 1680.8, 1: 1660.5. Samples: 1810424. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 13:32:03,165][74987] Avg episode reward: [(0, '10.430'), (1, '10.070')] -[2023-10-14 13:32:03,432][75949] Updated weights for policy 0, policy_version 3540 (0.0009) -[2023-10-14 13:32:03,802][75949] Updated weights for policy 0, policy_version 3550 (0.0009) -[2023-10-14 13:32:04,351][75950] Updated weights for policy 1, policy_version 3530 (0.0008) -[2023-10-14 13:32:04,714][75950] Updated weights for policy 1, policy_version 3540 (0.0008) -[2023-10-14 13:32:05,079][75950] Updated weights for policy 1, policy_version 3550 (0.0008) -[2023-10-14 13:32:08,023][75949] Updated weights for policy 0, policy_version 3560 (0.0007) -[2023-10-14 13:32:08,163][74987] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7274496. Throughput: 0: 1679.0, 1: 1666.4. Samples: 1830976. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 13:32:08,164][74987] Avg episode reward: [(0, '10.640'), (1, '11.170')] -[2023-10-14 13:32:08,165][75801] Saving new best policy, reward=11.170! -[2023-10-14 13:32:08,399][75949] Updated weights for policy 0, policy_version 3570 (0.0007) -[2023-10-14 13:32:08,771][75949] Updated weights for policy 0, policy_version 3580 (0.0007) -[2023-10-14 13:32:09,014][75950] Updated weights for policy 1, policy_version 3560 (0.0007) -[2023-10-14 13:32:09,382][75950] Updated weights for policy 1, policy_version 3570 (0.0007) -[2023-10-14 13:32:09,756][75950] Updated weights for policy 1, policy_version 3580 (0.0008) -[2023-10-14 13:32:13,041][75949] Updated weights for policy 0, policy_version 3590 (0.0009) -[2023-10-14 13:32:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7340032. Throughput: 0: 1673.5, 1: 1667.3. Samples: 1851410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:32:13,164][74987] Avg episode reward: [(0, '10.230'), (1, '10.890')] -[2023-10-14 13:32:13,414][75949] Updated weights for policy 0, policy_version 3600 (0.0011) -[2023-10-14 13:32:13,788][75949] Updated weights for policy 0, policy_version 3610 (0.0010) -[2023-10-14 13:32:13,912][75950] Updated weights for policy 1, policy_version 3590 (0.0008) -[2023-10-14 13:32:14,284][75950] Updated weights for policy 1, policy_version 3600 (0.0008) -[2023-10-14 13:32:14,654][75950] Updated weights for policy 1, policy_version 3610 (0.0008) -[2023-10-14 13:32:18,011][75949] Updated weights for policy 0, policy_version 3620 (0.0009) -[2023-10-14 13:32:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7405568. Throughput: 0: 1668.4, 1: 1668.8. Samples: 1860492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:32:18,165][74987] Avg episode reward: [(0, '10.440'), (1, '10.360')] -[2023-10-14 13:32:18,379][75949] Updated weights for policy 0, policy_version 3630 (0.0010) -[2023-10-14 13:32:18,752][75949] Updated weights for policy 0, policy_version 3640 (0.0010) -[2023-10-14 13:32:18,771][75950] Updated weights for policy 1, policy_version 3620 (0.0008) -[2023-10-14 13:32:19,134][75950] Updated weights for policy 1, policy_version 3630 (0.0009) -[2023-10-14 13:32:19,504][75950] Updated weights for policy 1, policy_version 3640 (0.0008) -[2023-10-14 13:32:22,858][75949] Updated weights for policy 0, policy_version 3650 (0.0008) -[2023-10-14 13:32:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7471104. Throughput: 0: 1665.2, 1: 1670.8. Samples: 1880778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:32:23,164][74987] Avg episode reward: [(0, '10.030'), (1, '11.280')] -[2023-10-14 13:32:23,165][75801] Saving new best policy, reward=11.280! -[2023-10-14 13:32:23,227][75949] Updated weights for policy 0, policy_version 3660 (0.0009) -[2023-10-14 13:32:23,605][75949] Updated weights for policy 0, policy_version 3670 (0.0007) -[2023-10-14 13:32:23,700][75950] Updated weights for policy 1, policy_version 3650 (0.0007) -[2023-10-14 13:32:23,966][75949] Updated weights for policy 0, policy_version 3680 (0.0009) -[2023-10-14 13:32:24,071][75950] Updated weights for policy 1, policy_version 3660 (0.0009) -[2023-10-14 13:32:24,437][75950] Updated weights for policy 1, policy_version 3670 (0.0010) -[2023-10-14 13:32:24,807][75950] Updated weights for policy 1, policy_version 3680 (0.0009) -[2023-10-14 13:32:28,011][75949] Updated weights for policy 0, policy_version 3690 (0.0008) -[2023-10-14 13:32:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 7536640. Throughput: 0: 1660.0, 1: 1670.1. Samples: 1901044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:32:28,165][74987] Avg episode reward: [(0, '10.290'), (1, '9.980')] -[2023-10-14 13:32:28,375][75949] Updated weights for policy 0, policy_version 3700 (0.0007) -[2023-10-14 13:32:28,748][75949] Updated weights for policy 0, policy_version 3710 (0.0007) -[2023-10-14 13:32:29,026][75950] Updated weights for policy 1, policy_version 3690 (0.0009) -[2023-10-14 13:32:29,405][75950] Updated weights for policy 1, policy_version 3700 (0.0010) -[2023-10-14 13:32:29,771][75950] Updated weights for policy 1, policy_version 3710 (0.0010) -[2023-10-14 13:32:32,799][75949] Updated weights for policy 0, policy_version 3720 (0.0007) -[2023-10-14 13:32:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 7602176. Throughput: 0: 1662.5, 1: 1665.7. Samples: 1910002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:32:33,164][74987] Avg episode reward: [(0, '10.960'), (1, '11.040')] -[2023-10-14 13:32:33,179][75949] Updated weights for policy 0, policy_version 3730 (0.0007) -[2023-10-14 13:32:33,552][75949] Updated weights for policy 0, policy_version 3740 (0.0009) -[2023-10-14 13:32:33,725][75950] Updated weights for policy 1, policy_version 3720 (0.0008) -[2023-10-14 13:32:34,096][75950] Updated weights for policy 1, policy_version 3730 (0.0007) -[2023-10-14 13:32:34,456][75950] Updated weights for policy 1, policy_version 3740 (0.0008) -[2023-10-14 13:32:37,433][75949] Updated weights for policy 0, policy_version 3750 (0.0008) -[2023-10-14 13:32:37,809][75949] Updated weights for policy 0, policy_version 3760 (0.0008) -[2023-10-14 13:32:38,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7667712. Throughput: 0: 1660.6, 1: 1670.5. Samples: 1930604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:32:38,164][74987] Avg episode reward: [(0, '10.650'), (1, '10.200')] -[2023-10-14 13:32:38,177][75949] Updated weights for policy 0, policy_version 3770 (0.0009) -[2023-10-14 13:32:38,634][75950] Updated weights for policy 1, policy_version 3750 (0.0008) -[2023-10-14 13:32:39,004][75950] Updated weights for policy 1, policy_version 3760 (0.0008) -[2023-10-14 13:32:39,375][75950] Updated weights for policy 1, policy_version 3770 (0.0009) -[2023-10-14 13:32:42,305][75949] Updated weights for policy 0, policy_version 3780 (0.0009) -[2023-10-14 13:32:42,673][75949] Updated weights for policy 0, policy_version 3790 (0.0009) -[2023-10-14 13:32:43,037][75949] Updated weights for policy 0, policy_version 3800 (0.0009) -[2023-10-14 13:32:43,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 7733248. Throughput: 0: 1655.7, 1: 1670.4. Samples: 1950878. Policy #0 lag: (min: 16.0, avg: 44.1, max: 48.0) -[2023-10-14 13:32:43,164][74987] Avg episode reward: [(0, '10.560'), (1, '10.230')] -[2023-10-14 13:32:43,171][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000003776_3866624.pth... -[2023-10-14 13:32:43,211][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000002240_2293760.pth -[2023-10-14 13:32:43,334][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000003808_3899392.pth... -[2023-10-14 13:32:43,372][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000002240_2293760.pth -[2023-10-14 13:32:43,496][75950] Updated weights for policy 1, policy_version 3780 (0.0009) -[2023-10-14 13:32:43,858][75950] Updated weights for policy 1, policy_version 3790 (0.0008) -[2023-10-14 13:32:44,229][75950] Updated weights for policy 1, policy_version 3800 (0.0007) -[2023-10-14 13:32:47,117][75949] Updated weights for policy 0, policy_version 3810 (0.0009) -[2023-10-14 13:32:47,489][75949] Updated weights for policy 0, policy_version 3820 (0.0009) -[2023-10-14 13:32:47,864][75949] Updated weights for policy 0, policy_version 3830 (0.0009) -[2023-10-14 13:32:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7798784. Throughput: 0: 1667.9, 1: 1668.6. Samples: 1960568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:32:48,165][74987] Avg episode reward: [(0, '10.550'), (1, '10.260')] -[2023-10-14 13:32:48,226][75949] Updated weights for policy 0, policy_version 3840 (0.0010) -[2023-10-14 13:32:48,298][75950] Updated weights for policy 1, policy_version 3810 (0.0007) -[2023-10-14 13:32:48,670][75950] Updated weights for policy 1, policy_version 3820 (0.0008) -[2023-10-14 13:32:49,035][75950] Updated weights for policy 1, policy_version 3830 (0.0007) -[2023-10-14 13:32:49,406][75950] Updated weights for policy 1, policy_version 3840 (0.0009) -[2023-10-14 13:32:52,333][75949] Updated weights for policy 0, policy_version 3850 (0.0010) -[2023-10-14 13:32:52,704][75949] Updated weights for policy 0, policy_version 3860 (0.0007) -[2023-10-14 13:32:53,076][75949] Updated weights for policy 0, policy_version 3870 (0.0009) -[2023-10-14 13:32:53,163][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 7897088. Throughput: 0: 1670.9, 1: 1665.0. Samples: 1981090. Policy #0 lag: (min: 17.0, avg: 36.5, max: 49.0) -[2023-10-14 13:32:53,164][74987] Avg episode reward: [(0, '9.990'), (1, '10.330')] -[2023-10-14 13:32:53,576][75950] Updated weights for policy 1, policy_version 3850 (0.0010) -[2023-10-14 13:32:53,945][75950] Updated weights for policy 1, policy_version 3860 (0.0011) -[2023-10-14 13:32:54,308][75950] Updated weights for policy 1, policy_version 3870 (0.0011) -[2023-10-14 13:32:57,091][75949] Updated weights for policy 0, policy_version 3880 (0.0009) -[2023-10-14 13:32:57,469][75949] Updated weights for policy 0, policy_version 3890 (0.0008) -[2023-10-14 13:32:57,847][75949] Updated weights for policy 0, policy_version 3900 (0.0008) -[2023-10-14 13:32:58,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 7962624. Throughput: 0: 1652.9, 1: 1668.9. Samples: 2000892. Policy #0 lag: (min: 17.0, avg: 36.5, max: 49.0) -[2023-10-14 13:32:58,165][74987] Avg episode reward: [(0, '10.350'), (1, '10.340')] -[2023-10-14 13:32:58,366][75950] Updated weights for policy 1, policy_version 3880 (0.0009) -[2023-10-14 13:32:58,736][75950] Updated weights for policy 1, policy_version 3890 (0.0009) -[2023-10-14 13:32:59,103][75950] Updated weights for policy 1, policy_version 3900 (0.0007) -[2023-10-14 13:33:01,984][75949] Updated weights for policy 0, policy_version 3910 (0.0009) -[2023-10-14 13:33:02,364][75949] Updated weights for policy 0, policy_version 3920 (0.0007) -[2023-10-14 13:33:02,740][75949] Updated weights for policy 0, policy_version 3930 (0.0009) -[2023-10-14 13:33:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 8028160. Throughput: 0: 1678.8, 1: 1665.4. Samples: 2010980. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) -[2023-10-14 13:33:03,164][74987] Avg episode reward: [(0, '9.990'), (1, '10.310')] -[2023-10-14 13:33:03,342][75950] Updated weights for policy 1, policy_version 3910 (0.0007) -[2023-10-14 13:33:03,707][75950] Updated weights for policy 1, policy_version 3920 (0.0009) -[2023-10-14 13:33:04,072][75950] Updated weights for policy 1, policy_version 3930 (0.0011) -[2023-10-14 13:33:06,938][75949] Updated weights for policy 0, policy_version 3940 (0.0008) -[2023-10-14 13:33:07,311][75949] Updated weights for policy 0, policy_version 3950 (0.0007) -[2023-10-14 13:33:07,690][75949] Updated weights for policy 0, policy_version 3960 (0.0007) -[2023-10-14 13:33:08,114][75950] Updated weights for policy 1, policy_version 3940 (0.0008) -[2023-10-14 13:33:08,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 8093696. Throughput: 0: 1679.6, 1: 1671.3. Samples: 2031568. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) -[2023-10-14 13:33:08,164][74987] Avg episode reward: [(0, '10.130'), (1, '10.360')] -[2023-10-14 13:33:08,477][75950] Updated weights for policy 1, policy_version 3950 (0.0008) -[2023-10-14 13:33:08,843][75950] Updated weights for policy 1, policy_version 3960 (0.0008) -[2023-10-14 13:33:11,693][75949] Updated weights for policy 0, policy_version 3970 (0.0008) -[2023-10-14 13:33:12,062][75949] Updated weights for policy 0, policy_version 3980 (0.0009) -[2023-10-14 13:33:12,436][75949] Updated weights for policy 0, policy_version 3990 (0.0011) -[2023-10-14 13:33:12,812][75949] Updated weights for policy 0, policy_version 4000 (0.0008) -[2023-10-14 13:33:12,814][75950] Updated weights for policy 1, policy_version 3970 (0.0008) -[2023-10-14 13:33:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 8159232. Throughput: 0: 1661.7, 1: 1676.3. Samples: 2051250. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) -[2023-10-14 13:33:13,164][74987] Avg episode reward: [(0, '10.780'), (1, '10.930')] -[2023-10-14 13:33:13,178][75950] Updated weights for policy 1, policy_version 3980 (0.0008) -[2023-10-14 13:33:13,541][75950] Updated weights for policy 1, policy_version 3990 (0.0009) -[2023-10-14 13:33:13,910][75950] Updated weights for policy 1, policy_version 4000 (0.0008) -[2023-10-14 13:33:17,042][75949] Updated weights for policy 0, policy_version 4010 (0.0010) -[2023-10-14 13:33:17,419][75949] Updated weights for policy 0, policy_version 4020 (0.0010) -[2023-10-14 13:33:17,786][75949] Updated weights for policy 0, policy_version 4030 (0.0008) -[2023-10-14 13:33:18,035][75950] Updated weights for policy 1, policy_version 4010 (0.0009) -[2023-10-14 13:33:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 8224768. Throughput: 0: 1680.1, 1: 1681.7. Samples: 2061284. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) -[2023-10-14 13:33:18,164][74987] Avg episode reward: [(0, '10.280'), (1, '10.810')] -[2023-10-14 13:33:18,419][75950] Updated weights for policy 1, policy_version 4020 (0.0009) -[2023-10-14 13:33:18,776][75950] Updated weights for policy 1, policy_version 4030 (0.0007) -[2023-10-14 13:33:22,013][75949] Updated weights for policy 0, policy_version 4040 (0.0007) -[2023-10-14 13:33:22,392][75949] Updated weights for policy 0, policy_version 4050 (0.0007) -[2023-10-14 13:33:22,766][75949] Updated weights for policy 0, policy_version 4060 (0.0007) -[2023-10-14 13:33:22,783][75950] Updated weights for policy 1, policy_version 4040 (0.0009) -[2023-10-14 13:33:23,157][75950] Updated weights for policy 1, policy_version 4050 (0.0007) -[2023-10-14 13:33:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 8290304. Throughput: 0: 1672.1, 1: 1680.4. Samples: 2081468. Policy #0 lag: (min: 9.0, avg: 31.2, max: 41.0) -[2023-10-14 13:33:23,165][74987] Avg episode reward: [(0, '10.270'), (1, '10.520')] -[2023-10-14 13:33:23,520][75950] Updated weights for policy 1, policy_version 4060 (0.0007) -[2023-10-14 13:33:26,873][75949] Updated weights for policy 0, policy_version 4070 (0.0009) -[2023-10-14 13:33:27,250][75949] Updated weights for policy 0, policy_version 4080 (0.0008) -[2023-10-14 13:33:27,621][75949] Updated weights for policy 0, policy_version 4090 (0.0010) -[2023-10-14 13:33:27,671][75950] Updated weights for policy 1, policy_version 4070 (0.0008) -[2023-10-14 13:33:28,042][75950] Updated weights for policy 1, policy_version 4080 (0.0010) -[2023-10-14 13:33:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 8355840. Throughput: 0: 1654.2, 1: 1673.1. Samples: 2100610. Policy #0 lag: (min: 2.0, avg: 2.8, max: 20.0) -[2023-10-14 13:33:28,165][74987] Avg episode reward: [(0, '10.020'), (1, '11.190')] -[2023-10-14 13:33:28,412][75950] Updated weights for policy 1, policy_version 4090 (0.0008) -[2023-10-14 13:33:31,623][75949] Updated weights for policy 0, policy_version 4100 (0.0008) -[2023-10-14 13:33:31,999][75949] Updated weights for policy 0, policy_version 4110 (0.0008) -[2023-10-14 13:33:32,374][75949] Updated weights for policy 0, policy_version 4120 (0.0008) -[2023-10-14 13:33:32,577][75950] Updated weights for policy 1, policy_version 4100 (0.0008) -[2023-10-14 13:33:32,943][75950] Updated weights for policy 1, policy_version 4110 (0.0009) -[2023-10-14 13:33:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 8421376. Throughput: 0: 1667.5, 1: 1677.2. Samples: 2111080. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-14 13:33:33,164][74987] Avg episode reward: [(0, '9.880'), (1, '10.370')] -[2023-10-14 13:33:33,310][75950] Updated weights for policy 1, policy_version 4120 (0.0008) -[2023-10-14 13:33:36,434][75949] Updated weights for policy 0, policy_version 4130 (0.0008) -[2023-10-14 13:33:36,807][75949] Updated weights for policy 0, policy_version 4140 (0.0011) -[2023-10-14 13:33:37,191][75949] Updated weights for policy 0, policy_version 4150 (0.0007) -[2023-10-14 13:33:37,257][75950] Updated weights for policy 1, policy_version 4130 (0.0010) -[2023-10-14 13:33:37,564][75949] Updated weights for policy 0, policy_version 4160 (0.0008) -[2023-10-14 13:33:37,616][75950] Updated weights for policy 1, policy_version 4140 (0.0008) -[2023-10-14 13:33:37,985][75950] Updated weights for policy 1, policy_version 4150 (0.0010) -[2023-10-14 13:33:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 8486912. Throughput: 0: 1656.6, 1: 1679.2. Samples: 2131200. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) -[2023-10-14 13:33:38,164][74987] Avg episode reward: [(0, '10.570'), (1, '10.810')] -[2023-10-14 13:33:38,356][75950] Updated weights for policy 1, policy_version 4160 (0.0009) -[2023-10-14 13:33:41,508][75949] Updated weights for policy 0, policy_version 4170 (0.0010) -[2023-10-14 13:33:41,885][75949] Updated weights for policy 0, policy_version 4180 (0.0008) -[2023-10-14 13:33:42,249][75949] Updated weights for policy 0, policy_version 4190 (0.0009) -[2023-10-14 13:33:42,591][75950] Updated weights for policy 1, policy_version 4170 (0.0009) -[2023-10-14 13:33:42,953][75950] Updated weights for policy 1, policy_version 4180 (0.0007) -[2023-10-14 13:33:43,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 8552448. Throughput: 0: 1659.4, 1: 1661.7. Samples: 2150344. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) -[2023-10-14 13:33:43,164][74987] Avg episode reward: [(0, '10.560'), (1, '10.640')] -[2023-10-14 13:33:43,322][75950] Updated weights for policy 1, policy_version 4190 (0.0008) -[2023-10-14 13:33:46,309][75949] Updated weights for policy 0, policy_version 4200 (0.0008) -[2023-10-14 13:33:46,683][75949] Updated weights for policy 0, policy_version 4210 (0.0008) -[2023-10-14 13:33:47,060][75949] Updated weights for policy 0, policy_version 4220 (0.0007) -[2023-10-14 13:33:47,405][75950] Updated weights for policy 1, policy_version 4200 (0.0008) -[2023-10-14 13:33:47,772][75950] Updated weights for policy 1, policy_version 4210 (0.0008) -[2023-10-14 13:33:48,142][75950] Updated weights for policy 1, policy_version 4220 (0.0010) -[2023-10-14 13:33:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 8617984. Throughput: 0: 1664.7, 1: 1672.3. Samples: 2161148. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 13:33:48,165][74987] Avg episode reward: [(0, '10.110'), (1, '11.020')] -[2023-10-14 13:33:51,056][75949] Updated weights for policy 0, policy_version 4230 (0.0009) -[2023-10-14 13:33:51,424][75949] Updated weights for policy 0, policy_version 4240 (0.0009) -[2023-10-14 13:33:51,796][75949] Updated weights for policy 0, policy_version 4250 (0.0010) -[2023-10-14 13:33:52,098][75950] Updated weights for policy 1, policy_version 4230 (0.0008) -[2023-10-14 13:33:52,474][75950] Updated weights for policy 1, policy_version 4240 (0.0007) -[2023-10-14 13:33:52,853][75950] Updated weights for policy 1, policy_version 4250 (0.0008) -[2023-10-14 13:33:53,163][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 8716288. Throughput: 0: 1648.7, 1: 1670.7. Samples: 2180942. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 13:33:53,164][74987] Avg episode reward: [(0, '10.560'), (1, '11.350')] -[2023-10-14 13:33:53,165][75801] Saving new best policy, reward=11.350! -[2023-10-14 13:33:55,912][75949] Updated weights for policy 0, policy_version 4260 (0.0010) -[2023-10-14 13:33:56,290][75949] Updated weights for policy 0, policy_version 4270 (0.0009) -[2023-10-14 13:33:56,659][75949] Updated weights for policy 0, policy_version 4280 (0.0008) -[2023-10-14 13:33:56,832][75950] Updated weights for policy 1, policy_version 4260 (0.0008) -[2023-10-14 13:33:57,189][75950] Updated weights for policy 1, policy_version 4270 (0.0009) -[2023-10-14 13:33:57,555][75950] Updated weights for policy 1, policy_version 4280 (0.0010) -[2023-10-14 13:33:58,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 8781824. Throughput: 0: 1662.4, 1: 1646.7. Samples: 2200160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:33:58,165][74987] Avg episode reward: [(0, '10.940'), (1, '10.730')] -[2023-10-14 13:34:00,854][75949] Updated weights for policy 0, policy_version 4290 (0.0007) -[2023-10-14 13:34:01,225][75949] Updated weights for policy 0, policy_version 4300 (0.0008) -[2023-10-14 13:34:01,596][75949] Updated weights for policy 0, policy_version 4310 (0.0009) -[2023-10-14 13:34:01,632][75950] Updated weights for policy 1, policy_version 4290 (0.0010) -[2023-10-14 13:34:01,963][75949] Updated weights for policy 0, policy_version 4320 (0.0007) -[2023-10-14 13:34:02,007][75950] Updated weights for policy 1, policy_version 4300 (0.0009) -[2023-10-14 13:34:02,360][75950] Updated weights for policy 1, policy_version 4310 (0.0007) -[2023-10-14 13:34:02,735][75950] Updated weights for policy 1, policy_version 4320 (0.0009) -[2023-10-14 13:34:03,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 8847360. Throughput: 0: 1671.0, 1: 1672.1. Samples: 2211722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:34:03,165][74987] Avg episode reward: [(0, '10.610'), (1, '11.020')] -[2023-10-14 13:34:06,079][75949] Updated weights for policy 0, policy_version 4330 (0.0008) -[2023-10-14 13:34:06,449][75949] Updated weights for policy 0, policy_version 4340 (0.0008) -[2023-10-14 13:34:06,814][75949] Updated weights for policy 0, policy_version 4350 (0.0008) -[2023-10-14 13:34:07,121][75950] Updated weights for policy 1, policy_version 4330 (0.0009) -[2023-10-14 13:34:07,492][75950] Updated weights for policy 1, policy_version 4340 (0.0009) -[2023-10-14 13:34:07,870][75950] Updated weights for policy 1, policy_version 4350 (0.0009) -[2023-10-14 13:34:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 8912896. Throughput: 0: 1653.1, 1: 1672.5. Samples: 2231122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:34:08,165][74987] Avg episode reward: [(0, '10.650'), (1, '10.120')] -[2023-10-14 13:34:11,110][75949] Updated weights for policy 0, policy_version 4360 (0.0007) -[2023-10-14 13:34:11,488][75949] Updated weights for policy 0, policy_version 4370 (0.0008) -[2023-10-14 13:34:11,866][75949] Updated weights for policy 0, policy_version 4380 (0.0008) -[2023-10-14 13:34:11,900][75950] Updated weights for policy 1, policy_version 4360 (0.0009) -[2023-10-14 13:34:12,275][75950] Updated weights for policy 1, policy_version 4370 (0.0008) -[2023-10-14 13:34:12,650][75950] Updated weights for policy 1, policy_version 4380 (0.0008) -[2023-10-14 13:34:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 8978432. Throughput: 0: 1673.2, 1: 1650.0. Samples: 2250154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:34:13,165][74987] Avg episode reward: [(0, '10.390'), (1, '9.960')] -[2023-10-14 13:34:15,724][75949] Updated weights for policy 0, policy_version 4390 (0.0008) -[2023-10-14 13:34:16,090][75949] Updated weights for policy 0, policy_version 4400 (0.0007) -[2023-10-14 13:34:16,462][75949] Updated weights for policy 0, policy_version 4410 (0.0011) -[2023-10-14 13:34:16,841][75950] Updated weights for policy 1, policy_version 4390 (0.0007) -[2023-10-14 13:34:17,198][75950] Updated weights for policy 1, policy_version 4400 (0.0008) -[2023-10-14 13:34:17,565][75950] Updated weights for policy 1, policy_version 4410 (0.0008) -[2023-10-14 13:34:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 9043968. Throughput: 0: 1669.4, 1: 1663.3. Samples: 2261050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:34:18,164][74987] Avg episode reward: [(0, '11.000'), (1, '11.120')] -[2023-10-14 13:34:20,725][75949] Updated weights for policy 0, policy_version 4420 (0.0009) -[2023-10-14 13:34:21,089][75949] Updated weights for policy 0, policy_version 4430 (0.0007) -[2023-10-14 13:34:21,456][75949] Updated weights for policy 0, policy_version 4440 (0.0011) -[2023-10-14 13:34:21,673][75950] Updated weights for policy 1, policy_version 4420 (0.0008) -[2023-10-14 13:34:22,032][75950] Updated weights for policy 1, policy_version 4430 (0.0008) -[2023-10-14 13:34:22,405][75950] Updated weights for policy 1, policy_version 4440 (0.0009) -[2023-10-14 13:34:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 9109504. Throughput: 0: 1657.2, 1: 1661.2. Samples: 2280528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:34:23,164][74987] Avg episode reward: [(0, '10.470'), (1, '10.140')] -[2023-10-14 13:34:25,542][75949] Updated weights for policy 0, policy_version 4450 (0.0010) -[2023-10-14 13:34:25,923][75949] Updated weights for policy 0, policy_version 4460 (0.0008) -[2023-10-14 13:34:26,283][75949] Updated weights for policy 0, policy_version 4470 (0.0009) -[2023-10-14 13:34:26,502][75950] Updated weights for policy 1, policy_version 4450 (0.0007) -[2023-10-14 13:34:26,658][75949] Updated weights for policy 0, policy_version 4480 (0.0007) -[2023-10-14 13:34:26,869][75950] Updated weights for policy 1, policy_version 4460 (0.0010) -[2023-10-14 13:34:27,240][75950] Updated weights for policy 1, policy_version 4470 (0.0010) -[2023-10-14 13:34:27,609][75950] Updated weights for policy 1, policy_version 4480 (0.0008) -[2023-10-14 13:34:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 9175040. Throughput: 0: 1672.9, 1: 1654.2. Samples: 2300064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:34:28,165][74987] Avg episode reward: [(0, '10.830'), (1, '10.700')] -[2023-10-14 13:34:30,679][75949] Updated weights for policy 0, policy_version 4490 (0.0009) -[2023-10-14 13:34:31,053][75949] Updated weights for policy 0, policy_version 4500 (0.0010) -[2023-10-14 13:34:31,422][75949] Updated weights for policy 0, policy_version 4510 (0.0009) -[2023-10-14 13:34:31,661][75950] Updated weights for policy 1, policy_version 4490 (0.0008) -[2023-10-14 13:34:32,030][75950] Updated weights for policy 1, policy_version 4500 (0.0007) -[2023-10-14 13:34:32,397][75950] Updated weights for policy 1, policy_version 4510 (0.0008) -[2023-10-14 13:34:33,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.6). Total num frames: 9240576. Throughput: 0: 1663.8, 1: 1673.0. Samples: 2311304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:34:33,164][74987] Avg episode reward: [(0, '10.600'), (1, '11.590')] -[2023-10-14 13:34:33,165][75801] Saving new best policy, reward=11.590! -[2023-10-14 13:34:35,569][75949] Updated weights for policy 0, policy_version 4520 (0.0009) -[2023-10-14 13:34:35,934][75949] Updated weights for policy 0, policy_version 4530 (0.0010) -[2023-10-14 13:34:36,310][75949] Updated weights for policy 0, policy_version 4540 (0.0010) -[2023-10-14 13:34:36,454][75950] Updated weights for policy 1, policy_version 4520 (0.0009) -[2023-10-14 13:34:36,825][75950] Updated weights for policy 1, policy_version 4530 (0.0009) -[2023-10-14 13:34:37,197][75950] Updated weights for policy 1, policy_version 4540 (0.0010) -[2023-10-14 13:34:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 9306112. Throughput: 0: 1663.1, 1: 1661.5. Samples: 2330548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-14 13:34:38,165][74987] Avg episode reward: [(0, '10.610'), (1, '11.900')] -[2023-10-14 13:34:38,166][75801] Saving new best policy, reward=11.900! -[2023-10-14 13:34:40,390][75949] Updated weights for policy 0, policy_version 4550 (0.0007) -[2023-10-14 13:34:40,762][75949] Updated weights for policy 0, policy_version 4560 (0.0007) -[2023-10-14 13:34:41,128][75949] Updated weights for policy 0, policy_version 4570 (0.0009) -[2023-10-14 13:34:41,281][75950] Updated weights for policy 1, policy_version 4550 (0.0009) -[2023-10-14 13:34:41,654][75950] Updated weights for policy 1, policy_version 4560 (0.0009) -[2023-10-14 13:34:42,024][75950] Updated weights for policy 1, policy_version 4570 (0.0009) -[2023-10-14 13:34:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 9371648. Throughput: 0: 1673.9, 1: 1667.8. Samples: 2350534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-14 13:34:43,164][74987] Avg episode reward: [(0, '10.670'), (1, '11.770')] -[2023-10-14 13:34:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000004576_4685824.pth... -[2023-10-14 13:34:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000004576_4685824.pth... -[2023-10-14 13:34:43,213][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000003008_3080192.pth -[2023-10-14 13:34:43,217][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000003008_3080192.pth -[2023-10-14 13:34:45,191][75949] Updated weights for policy 0, policy_version 4580 (0.0008) -[2023-10-14 13:34:45,563][75949] Updated weights for policy 0, policy_version 4590 (0.0010) -[2023-10-14 13:34:45,930][75949] Updated weights for policy 0, policy_version 4600 (0.0011) -[2023-10-14 13:34:46,078][75950] Updated weights for policy 1, policy_version 4580 (0.0008) -[2023-10-14 13:34:46,443][75950] Updated weights for policy 1, policy_version 4590 (0.0010) -[2023-10-14 13:34:46,813][75950] Updated weights for policy 1, policy_version 4600 (0.0009) -[2023-10-14 13:34:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 9437184. Throughput: 0: 1656.8, 1: 1672.6. Samples: 2361546. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-14 13:34:48,165][74987] Avg episode reward: [(0, '10.230'), (1, '11.530')] -[2023-10-14 13:34:49,954][75949] Updated weights for policy 0, policy_version 4610 (0.0009) -[2023-10-14 13:34:50,327][75949] Updated weights for policy 0, policy_version 4620 (0.0008) -[2023-10-14 13:34:50,701][75949] Updated weights for policy 0, policy_version 4630 (0.0008) -[2023-10-14 13:34:50,988][75950] Updated weights for policy 1, policy_version 4610 (0.0009) -[2023-10-14 13:34:51,069][75949] Updated weights for policy 0, policy_version 4640 (0.0009) -[2023-10-14 13:34:51,399][75950] Updated weights for policy 1, policy_version 4620 (0.0009) -[2023-10-14 13:34:51,769][75950] Updated weights for policy 1, policy_version 4630 (0.0009) -[2023-10-14 13:34:52,138][75950] Updated weights for policy 1, policy_version 4640 (0.0009) -[2023-10-14 13:34:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 9502720. Throughput: 0: 1666.3, 1: 1653.2. Samples: 2380502. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-10-14 13:34:53,165][74987] Avg episode reward: [(0, '10.590'), (1, '11.370')] -[2023-10-14 13:34:55,066][75949] Updated weights for policy 0, policy_version 4650 (0.0009) -[2023-10-14 13:34:55,427][75949] Updated weights for policy 0, policy_version 4660 (0.0008) -[2023-10-14 13:34:55,801][75949] Updated weights for policy 0, policy_version 4670 (0.0009) -[2023-10-14 13:34:56,189][75950] Updated weights for policy 1, policy_version 4650 (0.0010) -[2023-10-14 13:34:56,556][75950] Updated weights for policy 1, policy_version 4660 (0.0010) -[2023-10-14 13:34:56,926][75950] Updated weights for policy 1, policy_version 4670 (0.0009) -[2023-10-14 13:34:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 9568256. Throughput: 0: 1673.3, 1: 1668.8. Samples: 2400552. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) -[2023-10-14 13:34:58,165][74987] Avg episode reward: [(0, '10.580'), (1, '12.000')] -[2023-10-14 13:34:58,175][75801] Saving new best policy, reward=12.000! -[2023-10-14 13:34:59,964][75949] Updated weights for policy 0, policy_version 4680 (0.0008) -[2023-10-14 13:35:00,344][75949] Updated weights for policy 0, policy_version 4690 (0.0009) -[2023-10-14 13:35:00,720][75949] Updated weights for policy 0, policy_version 4700 (0.0008) -[2023-10-14 13:35:00,974][75950] Updated weights for policy 1, policy_version 4680 (0.0008) -[2023-10-14 13:35:01,351][75950] Updated weights for policy 1, policy_version 4690 (0.0009) -[2023-10-14 13:35:01,712][75950] Updated weights for policy 1, policy_version 4700 (0.0008) -[2023-10-14 13:35:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 9633792. Throughput: 0: 1655.1, 1: 1678.3. Samples: 2411054. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) -[2023-10-14 13:35:03,165][74987] Avg episode reward: [(0, '10.640'), (1, '11.320')] -[2023-10-14 13:35:04,637][75949] Updated weights for policy 0, policy_version 4710 (0.0010) -[2023-10-14 13:35:05,014][75949] Updated weights for policy 0, policy_version 4720 (0.0010) -[2023-10-14 13:35:05,390][75949] Updated weights for policy 0, policy_version 4730 (0.0009) -[2023-10-14 13:35:05,805][75950] Updated weights for policy 1, policy_version 4710 (0.0007) -[2023-10-14 13:35:06,168][75950] Updated weights for policy 1, policy_version 4720 (0.0009) -[2023-10-14 13:35:06,537][75950] Updated weights for policy 1, policy_version 4730 (0.0010) -[2023-10-14 13:35:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 9699328. Throughput: 0: 1670.3, 1: 1659.6. Samples: 2430370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:35:08,165][74987] Avg episode reward: [(0, '10.500'), (1, '12.320')] -[2023-10-14 13:35:08,166][75801] Saving new best policy, reward=12.320! -[2023-10-14 13:35:09,638][75949] Updated weights for policy 0, policy_version 4740 (0.0008) -[2023-10-14 13:35:10,005][75949] Updated weights for policy 0, policy_version 4750 (0.0009) -[2023-10-14 13:35:10,367][75949] Updated weights for policy 0, policy_version 4760 (0.0008) -[2023-10-14 13:35:10,627][75950] Updated weights for policy 1, policy_version 4740 (0.0010) -[2023-10-14 13:35:11,002][75950] Updated weights for policy 1, policy_version 4750 (0.0009) -[2023-10-14 13:35:11,375][75950] Updated weights for policy 1, policy_version 4760 (0.0008) -[2023-10-14 13:35:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 9764864. Throughput: 0: 1670.1, 1: 1676.5. Samples: 2450660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:35:13,165][74987] Avg episode reward: [(0, '10.890'), (1, '11.930')] -[2023-10-14 13:35:14,357][75949] Updated weights for policy 0, policy_version 4770 (0.0009) -[2023-10-14 13:35:14,734][75949] Updated weights for policy 0, policy_version 4780 (0.0008) -[2023-10-14 13:35:15,107][75949] Updated weights for policy 0, policy_version 4790 (0.0009) -[2023-10-14 13:35:15,476][75949] Updated weights for policy 0, policy_version 4800 (0.0007) -[2023-10-14 13:35:15,647][75950] Updated weights for policy 1, policy_version 4770 (0.0008) -[2023-10-14 13:35:16,022][75950] Updated weights for policy 1, policy_version 4780 (0.0009) -[2023-10-14 13:35:16,379][75950] Updated weights for policy 1, policy_version 4790 (0.0007) -[2023-10-14 13:35:16,746][75950] Updated weights for policy 1, policy_version 4800 (0.0011) -[2023-10-14 13:35:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 9830400. Throughput: 0: 1652.4, 1: 1670.2. Samples: 2460822. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-14 13:35:18,164][74987] Avg episode reward: [(0, '10.720'), (1, '12.930')] -[2023-10-14 13:35:18,165][75801] Saving new best policy, reward=12.930! -[2023-10-14 13:35:19,620][75949] Updated weights for policy 0, policy_version 4810 (0.0010) -[2023-10-14 13:35:20,000][75949] Updated weights for policy 0, policy_version 4820 (0.0010) -[2023-10-14 13:35:20,368][75949] Updated weights for policy 0, policy_version 4830 (0.0008) -[2023-10-14 13:35:20,912][75950] Updated weights for policy 1, policy_version 4810 (0.0009) -[2023-10-14 13:35:21,283][75950] Updated weights for policy 1, policy_version 4820 (0.0011) -[2023-10-14 13:35:21,646][75950] Updated weights for policy 1, policy_version 4830 (0.0009) -[2023-10-14 13:35:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 9895936. Throughput: 0: 1674.1, 1: 1656.6. Samples: 2480432. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-14 13:35:23,164][74987] Avg episode reward: [(0, '10.310'), (1, '11.750')] -[2023-10-14 13:35:24,363][75949] Updated weights for policy 0, policy_version 4840 (0.0010) -[2023-10-14 13:35:24,734][75949] Updated weights for policy 0, policy_version 4850 (0.0008) -[2023-10-14 13:35:25,108][75949] Updated weights for policy 0, policy_version 4860 (0.0009) -[2023-10-14 13:35:25,737][75950] Updated weights for policy 1, policy_version 4840 (0.0009) -[2023-10-14 13:35:26,098][75950] Updated weights for policy 1, policy_version 4850 (0.0009) -[2023-10-14 13:35:26,466][75950] Updated weights for policy 1, policy_version 4860 (0.0007) -[2023-10-14 13:35:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 9961472. Throughput: 0: 1671.7, 1: 1669.8. Samples: 2500902. Policy #0 lag: (min: 12.0, avg: 12.3, max: 24.0) -[2023-10-14 13:35:28,164][74987] Avg episode reward: [(0, '10.780'), (1, '11.530')] -[2023-10-14 13:35:29,126][75949] Updated weights for policy 0, policy_version 4870 (0.0008) -[2023-10-14 13:35:29,491][75949] Updated weights for policy 0, policy_version 4880 (0.0007) -[2023-10-14 13:35:29,862][75949] Updated weights for policy 0, policy_version 4890 (0.0007) -[2023-10-14 13:35:30,463][75950] Updated weights for policy 1, policy_version 4870 (0.0009) -[2023-10-14 13:35:30,843][75950] Updated weights for policy 1, policy_version 4880 (0.0009) -[2023-10-14 13:35:31,209][75950] Updated weights for policy 1, policy_version 4890 (0.0010) -[2023-10-14 13:35:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10027008. Throughput: 0: 1656.4, 1: 1661.6. Samples: 2510854. Policy #0 lag: (min: 12.0, avg: 12.3, max: 24.0) -[2023-10-14 13:35:33,165][74987] Avg episode reward: [(0, '10.570'), (1, '11.490')] -[2023-10-14 13:35:34,116][75949] Updated weights for policy 0, policy_version 4900 (0.0009) -[2023-10-14 13:35:34,493][75949] Updated weights for policy 0, policy_version 4910 (0.0008) -[2023-10-14 13:35:34,854][75949] Updated weights for policy 0, policy_version 4920 (0.0008) -[2023-10-14 13:35:35,330][75950] Updated weights for policy 1, policy_version 4900 (0.0010) -[2023-10-14 13:35:35,702][75950] Updated weights for policy 1, policy_version 4910 (0.0008) -[2023-10-14 13:35:36,069][75950] Updated weights for policy 1, policy_version 4920 (0.0009) -[2023-10-14 13:35:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10092544. Throughput: 0: 1672.6, 1: 1658.5. Samples: 2530402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:35:38,164][74987] Avg episode reward: [(0, '10.910'), (1, '11.070')] -[2023-10-14 13:35:38,919][75949] Updated weights for policy 0, policy_version 4930 (0.0008) -[2023-10-14 13:35:39,288][75949] Updated weights for policy 0, policy_version 4940 (0.0009) -[2023-10-14 13:35:39,670][75949] Updated weights for policy 0, policy_version 4950 (0.0008) -[2023-10-14 13:35:40,044][75949] Updated weights for policy 0, policy_version 4960 (0.0009) -[2023-10-14 13:35:40,262][75950] Updated weights for policy 1, policy_version 4930 (0.0009) -[2023-10-14 13:35:40,669][75950] Updated weights for policy 1, policy_version 4940 (0.0010) -[2023-10-14 13:35:41,043][75950] Updated weights for policy 1, policy_version 4950 (0.0008) -[2023-10-14 13:35:41,406][75950] Updated weights for policy 1, policy_version 4960 (0.0007) -[2023-10-14 13:35:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10158080. Throughput: 0: 1671.9, 1: 1666.7. Samples: 2550790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:35:43,164][74987] Avg episode reward: [(0, '10.470'), (1, '11.780')] -[2023-10-14 13:35:44,127][75949] Updated weights for policy 0, policy_version 4970 (0.0007) -[2023-10-14 13:35:44,501][75949] Updated weights for policy 0, policy_version 4980 (0.0009) -[2023-10-14 13:35:44,874][75949] Updated weights for policy 0, policy_version 4990 (0.0007) -[2023-10-14 13:35:45,347][75950] Updated weights for policy 1, policy_version 4970 (0.0007) -[2023-10-14 13:35:45,712][75950] Updated weights for policy 1, policy_version 4980 (0.0007) -[2023-10-14 13:35:46,089][75950] Updated weights for policy 1, policy_version 4990 (0.0008) -[2023-10-14 13:35:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 10223616. Throughput: 0: 1671.3, 1: 1651.3. Samples: 2560574. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) -[2023-10-14 13:35:48,164][74987] Avg episode reward: [(0, '10.510'), (1, '11.760')] -[2023-10-14 13:35:49,151][75949] Updated weights for policy 0, policy_version 5000 (0.0008) -[2023-10-14 13:35:49,515][75949] Updated weights for policy 0, policy_version 5010 (0.0010) -[2023-10-14 13:35:49,896][75949] Updated weights for policy 0, policy_version 5020 (0.0008) -[2023-10-14 13:35:50,337][75950] Updated weights for policy 1, policy_version 5000 (0.0008) -[2023-10-14 13:35:50,710][75950] Updated weights for policy 1, policy_version 5010 (0.0009) -[2023-10-14 13:35:51,086][75950] Updated weights for policy 1, policy_version 5020 (0.0009) -[2023-10-14 13:35:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10289152. Throughput: 0: 1674.1, 1: 1660.4. Samples: 2580424. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) -[2023-10-14 13:35:53,165][74987] Avg episode reward: [(0, '10.580'), (1, '11.550')] -[2023-10-14 13:35:53,942][75949] Updated weights for policy 0, policy_version 5030 (0.0009) -[2023-10-14 13:35:54,307][75949] Updated weights for policy 0, policy_version 5040 (0.0010) -[2023-10-14 13:35:54,683][75949] Updated weights for policy 0, policy_version 5050 (0.0008) -[2023-10-14 13:35:55,258][75950] Updated weights for policy 1, policy_version 5030 (0.0009) -[2023-10-14 13:35:55,622][75950] Updated weights for policy 1, policy_version 5040 (0.0010) -[2023-10-14 13:35:55,993][75950] Updated weights for policy 1, policy_version 5050 (0.0009) -[2023-10-14 13:35:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10354688. Throughput: 0: 1678.0, 1: 1660.4. Samples: 2600890. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-14 13:35:58,165][74987] Avg episode reward: [(0, '10.610'), (1, '12.310')] -[2023-10-14 13:35:58,699][75949] Updated weights for policy 0, policy_version 5060 (0.0010) -[2023-10-14 13:35:59,060][75949] Updated weights for policy 0, policy_version 5070 (0.0007) -[2023-10-14 13:35:59,430][75949] Updated weights for policy 0, policy_version 5080 (0.0008) -[2023-10-14 13:36:00,075][75950] Updated weights for policy 1, policy_version 5060 (0.0010) -[2023-10-14 13:36:00,455][75950] Updated weights for policy 1, policy_version 5070 (0.0008) -[2023-10-14 13:36:00,827][75950] Updated weights for policy 1, policy_version 5080 (0.0009) -[2023-10-14 13:36:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10420224. Throughput: 0: 1677.8, 1: 1652.5. Samples: 2610686. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) -[2023-10-14 13:36:03,164][74987] Avg episode reward: [(0, '10.590'), (1, '11.270')] -[2023-10-14 13:36:03,409][75949] Updated weights for policy 0, policy_version 5090 (0.0009) -[2023-10-14 13:36:03,778][75949] Updated weights for policy 0, policy_version 5100 (0.0008) -[2023-10-14 13:36:04,152][75949] Updated weights for policy 0, policy_version 5110 (0.0011) -[2023-10-14 13:36:04,528][75949] Updated weights for policy 0, policy_version 5120 (0.0008) -[2023-10-14 13:36:04,896][75950] Updated weights for policy 1, policy_version 5090 (0.0010) -[2023-10-14 13:36:05,263][75950] Updated weights for policy 1, policy_version 5100 (0.0011) -[2023-10-14 13:36:05,641][75950] Updated weights for policy 1, policy_version 5110 (0.0008) -[2023-10-14 13:36:06,011][75950] Updated weights for policy 1, policy_version 5120 (0.0008) -[2023-10-14 13:36:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 10485760. Throughput: 0: 1683.1, 1: 1665.4. Samples: 2631116. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) -[2023-10-14 13:36:08,165][74987] Avg episode reward: [(0, '10.980'), (1, '11.680')] -[2023-10-14 13:36:08,649][75949] Updated weights for policy 0, policy_version 5130 (0.0009) -[2023-10-14 13:36:09,016][75949] Updated weights for policy 0, policy_version 5140 (0.0009) -[2023-10-14 13:36:09,384][75949] Updated weights for policy 0, policy_version 5150 (0.0010) -[2023-10-14 13:36:10,068][75950] Updated weights for policy 1, policy_version 5130 (0.0009) -[2023-10-14 13:36:10,440][75950] Updated weights for policy 1, policy_version 5140 (0.0008) -[2023-10-14 13:36:10,814][75950] Updated weights for policy 1, policy_version 5150 (0.0007) -[2023-10-14 13:36:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 10551296. Throughput: 0: 1687.6, 1: 1667.6. Samples: 2651886. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) -[2023-10-14 13:36:13,165][74987] Avg episode reward: [(0, '10.020'), (1, '11.170')] -[2023-10-14 13:36:13,361][75949] Updated weights for policy 0, policy_version 5160 (0.0009) -[2023-10-14 13:36:13,738][75949] Updated weights for policy 0, policy_version 5170 (0.0010) -[2023-10-14 13:36:14,105][75949] Updated weights for policy 0, policy_version 5180 (0.0008) -[2023-10-14 13:36:14,894][75950] Updated weights for policy 1, policy_version 5160 (0.0008) -[2023-10-14 13:36:15,260][75950] Updated weights for policy 1, policy_version 5170 (0.0008) -[2023-10-14 13:36:15,630][75950] Updated weights for policy 1, policy_version 5180 (0.0007) -[2023-10-14 13:36:18,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10616832. Throughput: 0: 1690.8, 1: 1652.0. Samples: 2661278. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-14 13:36:18,164][74987] Avg episode reward: [(0, '11.160'), (1, '11.700')] -[2023-10-14 13:36:18,416][75949] Updated weights for policy 0, policy_version 5190 (0.0007) -[2023-10-14 13:36:18,772][75949] Updated weights for policy 0, policy_version 5200 (0.0008) -[2023-10-14 13:36:19,151][75949] Updated weights for policy 0, policy_version 5210 (0.0008) -[2023-10-14 13:36:19,619][75950] Updated weights for policy 1, policy_version 5190 (0.0011) -[2023-10-14 13:36:19,985][75950] Updated weights for policy 1, policy_version 5200 (0.0012) -[2023-10-14 13:36:20,356][75950] Updated weights for policy 1, policy_version 5210 (0.0010) -[2023-10-14 13:36:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10682368. Throughput: 0: 1686.6, 1: 1668.8. Samples: 2681394. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-14 13:36:23,164][74987] Avg episode reward: [(0, '10.180'), (1, '11.740')] -[2023-10-14 13:36:23,178][75949] Updated weights for policy 0, policy_version 5220 (0.0008) -[2023-10-14 13:36:23,546][75949] Updated weights for policy 0, policy_version 5230 (0.0009) -[2023-10-14 13:36:23,916][75949] Updated weights for policy 0, policy_version 5240 (0.0010) -[2023-10-14 13:36:24,451][75950] Updated weights for policy 1, policy_version 5220 (0.0009) -[2023-10-14 13:36:24,818][75950] Updated weights for policy 1, policy_version 5230 (0.0008) -[2023-10-14 13:36:25,188][75950] Updated weights for policy 1, policy_version 5240 (0.0009) -[2023-10-14 13:36:27,881][75949] Updated weights for policy 0, policy_version 5250 (0.0009) -[2023-10-14 13:36:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10747904. Throughput: 0: 1689.2, 1: 1676.0. Samples: 2702220. Policy #0 lag: (min: 1.0, avg: 20.5, max: 33.0) -[2023-10-14 13:36:28,164][74987] Avg episode reward: [(0, '10.810'), (1, '12.260')] -[2023-10-14 13:36:28,262][75949] Updated weights for policy 0, policy_version 5260 (0.0009) -[2023-10-14 13:36:28,625][75949] Updated weights for policy 0, policy_version 5270 (0.0009) -[2023-10-14 13:36:29,004][75949] Updated weights for policy 0, policy_version 5280 (0.0009) -[2023-10-14 13:36:29,447][75950] Updated weights for policy 1, policy_version 5250 (0.0008) -[2023-10-14 13:36:29,819][75950] Updated weights for policy 1, policy_version 5260 (0.0009) -[2023-10-14 13:36:30,193][75950] Updated weights for policy 1, policy_version 5270 (0.0008) -[2023-10-14 13:36:30,560][75950] Updated weights for policy 1, policy_version 5280 (0.0009) -[2023-10-14 13:36:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10813440. Throughput: 0: 1686.6, 1: 1660.7. Samples: 2711200. Policy #0 lag: (min: 1.0, avg: 20.5, max: 33.0) -[2023-10-14 13:36:33,165][74987] Avg episode reward: [(0, '10.640'), (1, '11.020')] -[2023-10-14 13:36:33,191][75949] Updated weights for policy 0, policy_version 5290 (0.0007) -[2023-10-14 13:36:33,553][75949] Updated weights for policy 0, policy_version 5300 (0.0007) -[2023-10-14 13:36:33,929][75949] Updated weights for policy 0, policy_version 5310 (0.0007) -[2023-10-14 13:36:34,664][75950] Updated weights for policy 1, policy_version 5290 (0.0008) -[2023-10-14 13:36:35,030][75950] Updated weights for policy 1, policy_version 5300 (0.0009) -[2023-10-14 13:36:35,399][75950] Updated weights for policy 1, policy_version 5310 (0.0009) -[2023-10-14 13:36:37,961][75949] Updated weights for policy 0, policy_version 5320 (0.0008) -[2023-10-14 13:36:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 10878976. Throughput: 0: 1689.3, 1: 1676.0. Samples: 2731864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:36:38,164][74987] Avg episode reward: [(0, '10.630'), (1, '11.180')] -[2023-10-14 13:36:38,343][75949] Updated weights for policy 0, policy_version 5330 (0.0008) -[2023-10-14 13:36:38,715][75949] Updated weights for policy 0, policy_version 5340 (0.0007) -[2023-10-14 13:36:39,329][75950] Updated weights for policy 1, policy_version 5320 (0.0010) -[2023-10-14 13:36:39,700][75950] Updated weights for policy 1, policy_version 5330 (0.0007) -[2023-10-14 13:36:40,065][75950] Updated weights for policy 1, policy_version 5340 (0.0008) -[2023-10-14 13:36:42,671][75949] Updated weights for policy 0, policy_version 5350 (0.0007) -[2023-10-14 13:36:43,041][75949] Updated weights for policy 0, policy_version 5360 (0.0008) -[2023-10-14 13:36:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 10944512. Throughput: 0: 1677.9, 1: 1684.6. Samples: 2752200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:36:43,165][74987] Avg episode reward: [(0, '10.520'), (1, '11.540')] -[2023-10-14 13:36:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000005344_5472256.pth... -[2023-10-14 13:36:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000003776_3866624.pth -[2023-10-14 13:36:43,409][75949] Updated weights for policy 0, policy_version 5370 (0.0008) -[2023-10-14 13:36:43,626][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000005376_5505024.pth... -[2023-10-14 13:36:43,655][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000003808_3899392.pth -[2023-10-14 13:36:44,076][75950] Updated weights for policy 1, policy_version 5350 (0.0009) -[2023-10-14 13:36:44,451][75950] Updated weights for policy 1, policy_version 5360 (0.0007) -[2023-10-14 13:36:44,810][75950] Updated weights for policy 1, policy_version 5370 (0.0008) -[2023-10-14 13:36:47,479][75949] Updated weights for policy 0, policy_version 5380 (0.0008) -[2023-10-14 13:36:47,844][75949] Updated weights for policy 0, policy_version 5390 (0.0009) -[2023-10-14 13:36:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11010048. Throughput: 0: 1681.8, 1: 1667.4. Samples: 2761400. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-10-14 13:36:48,164][74987] Avg episode reward: [(0, '10.430'), (1, '11.810')] -[2023-10-14 13:36:48,221][75949] Updated weights for policy 0, policy_version 5400 (0.0008) -[2023-10-14 13:36:49,119][75950] Updated weights for policy 1, policy_version 5380 (0.0009) -[2023-10-14 13:36:49,493][75950] Updated weights for policy 1, policy_version 5390 (0.0007) -[2023-10-14 13:36:49,863][75950] Updated weights for policy 1, policy_version 5400 (0.0007) -[2023-10-14 13:36:52,381][75949] Updated weights for policy 0, policy_version 5410 (0.0011) -[2023-10-14 13:36:52,768][75949] Updated weights for policy 0, policy_version 5420 (0.0007) -[2023-10-14 13:36:53,141][75949] Updated weights for policy 0, policy_version 5430 (0.0009) -[2023-10-14 13:36:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11075584. Throughput: 0: 1671.8, 1: 1678.6. Samples: 2781882. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-10-14 13:36:53,164][74987] Avg episode reward: [(0, '10.770'), (1, '11.570')] -[2023-10-14 13:36:53,506][75949] Updated weights for policy 0, policy_version 5440 (0.0008) -[2023-10-14 13:36:53,821][75950] Updated weights for policy 1, policy_version 5410 (0.0009) -[2023-10-14 13:36:54,199][75950] Updated weights for policy 1, policy_version 5420 (0.0009) -[2023-10-14 13:36:54,571][75950] Updated weights for policy 1, policy_version 5430 (0.0010) -[2023-10-14 13:36:54,930][75950] Updated weights for policy 1, policy_version 5440 (0.0009) -[2023-10-14 13:36:57,501][75949] Updated weights for policy 0, policy_version 5450 (0.0008) -[2023-10-14 13:36:57,870][75949] Updated weights for policy 0, policy_version 5460 (0.0008) -[2023-10-14 13:36:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 11141120. Throughput: 0: 1657.7, 1: 1680.1. Samples: 2802088. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 13:36:58,164][74987] Avg episode reward: [(0, '10.740'), (1, '12.630')] -[2023-10-14 13:36:58,240][75949] Updated weights for policy 0, policy_version 5470 (0.0008) -[2023-10-14 13:36:58,825][75950] Updated weights for policy 1, policy_version 5450 (0.0008) -[2023-10-14 13:36:59,199][75950] Updated weights for policy 1, policy_version 5460 (0.0008) -[2023-10-14 13:36:59,559][75950] Updated weights for policy 1, policy_version 5470 (0.0008) -[2023-10-14 13:37:02,263][75949] Updated weights for policy 0, policy_version 5480 (0.0009) -[2023-10-14 13:37:02,631][75949] Updated weights for policy 0, policy_version 5490 (0.0010) -[2023-10-14 13:37:03,004][75949] Updated weights for policy 0, policy_version 5500 (0.0009) -[2023-10-14 13:37:03,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11239424. Throughput: 0: 1673.9, 1: 1676.3. Samples: 2812036. Policy #0 lag: (min: 17.0, avg: 24.1, max: 49.0) -[2023-10-14 13:37:03,165][74987] Avg episode reward: [(0, '10.510'), (1, '12.680')] -[2023-10-14 13:37:03,515][75950] Updated weights for policy 1, policy_version 5480 (0.0009) -[2023-10-14 13:37:03,886][75950] Updated weights for policy 1, policy_version 5490 (0.0008) -[2023-10-14 13:37:04,260][75950] Updated weights for policy 1, policy_version 5500 (0.0009) -[2023-10-14 13:37:07,111][75949] Updated weights for policy 0, policy_version 5510 (0.0008) -[2023-10-14 13:37:07,479][75949] Updated weights for policy 0, policy_version 5520 (0.0007) -[2023-10-14 13:37:07,848][75949] Updated weights for policy 0, policy_version 5530 (0.0007) -[2023-10-14 13:37:08,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11304960. Throughput: 0: 1676.2, 1: 1682.7. Samples: 2832544. Policy #0 lag: (min: 17.0, avg: 24.1, max: 49.0) -[2023-10-14 13:37:08,164][74987] Avg episode reward: [(0, '10.650'), (1, '13.440')] -[2023-10-14 13:37:08,406][75950] Updated weights for policy 1, policy_version 5510 (0.0010) -[2023-10-14 13:37:08,778][75950] Updated weights for policy 1, policy_version 5520 (0.0009) -[2023-10-14 13:37:09,146][75950] Updated weights for policy 1, policy_version 5530 (0.0008) -[2023-10-14 13:37:09,362][75801] Saving new best policy, reward=13.440! -[2023-10-14 13:37:11,926][75949] Updated weights for policy 0, policy_version 5540 (0.0009) -[2023-10-14 13:37:12,295][75949] Updated weights for policy 0, policy_version 5550 (0.0010) -[2023-10-14 13:37:12,674][75949] Updated weights for policy 0, policy_version 5560 (0.0007) -[2023-10-14 13:37:13,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 11370496. Throughput: 0: 1655.9, 1: 1683.9. Samples: 2852508. Policy #0 lag: (min: 17.0, avg: 23.7, max: 49.0) -[2023-10-14 13:37:13,164][74987] Avg episode reward: [(0, '10.840'), (1, '11.820')] -[2023-10-14 13:37:13,242][75950] Updated weights for policy 1, policy_version 5540 (0.0007) -[2023-10-14 13:37:13,613][75950] Updated weights for policy 1, policy_version 5550 (0.0009) -[2023-10-14 13:37:13,983][75950] Updated weights for policy 1, policy_version 5560 (0.0008) -[2023-10-14 13:37:16,701][75949] Updated weights for policy 0, policy_version 5570 (0.0008) -[2023-10-14 13:37:17,070][75949] Updated weights for policy 0, policy_version 5580 (0.0009) -[2023-10-14 13:37:17,441][75949] Updated weights for policy 0, policy_version 5590 (0.0009) -[2023-10-14 13:37:17,807][75949] Updated weights for policy 0, policy_version 5600 (0.0009) -[2023-10-14 13:37:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11436032. Throughput: 0: 1678.0, 1: 1684.3. Samples: 2862506. Policy #0 lag: (min: 17.0, avg: 23.7, max: 49.0) -[2023-10-14 13:37:18,164][74987] Avg episode reward: [(0, '10.900'), (1, '11.890')] -[2023-10-14 13:37:18,255][75950] Updated weights for policy 1, policy_version 5570 (0.0008) -[2023-10-14 13:37:18,638][75950] Updated weights for policy 1, policy_version 5580 (0.0010) -[2023-10-14 13:37:19,007][75950] Updated weights for policy 1, policy_version 5590 (0.0009) -[2023-10-14 13:37:19,370][75950] Updated weights for policy 1, policy_version 5600 (0.0009) -[2023-10-14 13:37:21,946][75949] Updated weights for policy 0, policy_version 5610 (0.0008) -[2023-10-14 13:37:22,318][75949] Updated weights for policy 0, policy_version 5620 (0.0011) -[2023-10-14 13:37:22,683][75949] Updated weights for policy 0, policy_version 5630 (0.0010) -[2023-10-14 13:37:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11501568. Throughput: 0: 1677.6, 1: 1678.5. Samples: 2882888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:37:23,164][74987] Avg episode reward: [(0, '10.890'), (1, '12.430')] -[2023-10-14 13:37:23,419][75950] Updated weights for policy 1, policy_version 5610 (0.0007) -[2023-10-14 13:37:23,786][75950] Updated weights for policy 1, policy_version 5620 (0.0007) -[2023-10-14 13:37:24,159][75950] Updated weights for policy 1, policy_version 5630 (0.0007) -[2023-10-14 13:37:26,809][75949] Updated weights for policy 0, policy_version 5640 (0.0009) -[2023-10-14 13:37:27,186][75949] Updated weights for policy 0, policy_version 5650 (0.0009) -[2023-10-14 13:37:27,558][75949] Updated weights for policy 0, policy_version 5660 (0.0008) -[2023-10-14 13:37:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11567104. Throughput: 0: 1661.8, 1: 1678.3. Samples: 2902506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:37:28,164][74987] Avg episode reward: [(0, '10.190'), (1, '12.780')] -[2023-10-14 13:37:28,324][75950] Updated weights for policy 1, policy_version 5640 (0.0008) -[2023-10-14 13:37:28,696][75950] Updated weights for policy 1, policy_version 5650 (0.0009) -[2023-10-14 13:37:29,062][75950] Updated weights for policy 1, policy_version 5660 (0.0010) -[2023-10-14 13:37:31,543][75949] Updated weights for policy 0, policy_version 5670 (0.0009) -[2023-10-14 13:37:31,918][75949] Updated weights for policy 0, policy_version 5680 (0.0010) -[2023-10-14 13:37:32,294][75949] Updated weights for policy 0, policy_version 5690 (0.0007) -[2023-10-14 13:37:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11632640. Throughput: 0: 1687.5, 1: 1678.5. Samples: 2912872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:37:33,165][74987] Avg episode reward: [(0, '11.190'), (1, '12.670')] -[2023-10-14 13:37:33,256][75950] Updated weights for policy 1, policy_version 5670 (0.0009) -[2023-10-14 13:37:33,629][75950] Updated weights for policy 1, policy_version 5680 (0.0008) -[2023-10-14 13:37:33,996][75950] Updated weights for policy 1, policy_version 5690 (0.0008) -[2023-10-14 13:37:36,409][75949] Updated weights for policy 0, policy_version 5700 (0.0008) -[2023-10-14 13:37:36,777][75949] Updated weights for policy 0, policy_version 5710 (0.0009) -[2023-10-14 13:37:37,153][75949] Updated weights for policy 0, policy_version 5720 (0.0009) -[2023-10-14 13:37:38,139][75950] Updated weights for policy 1, policy_version 5700 (0.0007) -[2023-10-14 13:37:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 11698176. Throughput: 0: 1684.7, 1: 1675.4. Samples: 2933086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:37:38,164][74987] Avg episode reward: [(0, '10.670'), (1, '12.850')] -[2023-10-14 13:37:38,502][75950] Updated weights for policy 1, policy_version 5710 (0.0009) -[2023-10-14 13:37:38,862][75950] Updated weights for policy 1, policy_version 5720 (0.0008) -[2023-10-14 13:37:41,051][75949] Updated weights for policy 0, policy_version 5730 (0.0010) -[2023-10-14 13:37:41,413][75949] Updated weights for policy 0, policy_version 5740 (0.0008) -[2023-10-14 13:37:41,779][75949] Updated weights for policy 0, policy_version 5750 (0.0009) -[2023-10-14 13:37:42,151][75949] Updated weights for policy 0, policy_version 5760 (0.0010) -[2023-10-14 13:37:43,013][75950] Updated weights for policy 1, policy_version 5730 (0.0008) -[2023-10-14 13:37:43,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 11763712. Throughput: 0: 1681.1, 1: 1671.2. Samples: 2952940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:37:43,164][74987] Avg episode reward: [(0, '10.570'), (1, '12.170')] -[2023-10-14 13:37:43,383][75950] Updated weights for policy 1, policy_version 5740 (0.0010) -[2023-10-14 13:37:43,764][75950] Updated weights for policy 1, policy_version 5750 (0.0010) -[2023-10-14 13:37:44,126][75950] Updated weights for policy 1, policy_version 5760 (0.0008) -[2023-10-14 13:37:46,140][75949] Updated weights for policy 0, policy_version 5770 (0.0010) -[2023-10-14 13:37:46,512][75949] Updated weights for policy 0, policy_version 5780 (0.0008) -[2023-10-14 13:37:46,885][75949] Updated weights for policy 0, policy_version 5790 (0.0007) -[2023-10-14 13:37:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 11829248. Throughput: 0: 1696.4, 1: 1667.3. Samples: 2963404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:37:48,165][74987] Avg episode reward: [(0, '10.910'), (1, '12.310')] -[2023-10-14 13:37:48,230][75950] Updated weights for policy 1, policy_version 5770 (0.0009) -[2023-10-14 13:37:48,600][75950] Updated weights for policy 1, policy_version 5780 (0.0009) -[2023-10-14 13:37:48,972][75950] Updated weights for policy 1, policy_version 5790 (0.0009) -[2023-10-14 13:37:50,882][75949] Updated weights for policy 0, policy_version 5800 (0.0009) -[2023-10-14 13:37:51,243][75949] Updated weights for policy 0, policy_version 5810 (0.0010) -[2023-10-14 13:37:51,621][75949] Updated weights for policy 0, policy_version 5820 (0.0008) -[2023-10-14 13:37:52,974][75950] Updated weights for policy 1, policy_version 5800 (0.0008) -[2023-10-14 13:37:53,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 11894784. Throughput: 0: 1672.5, 1: 1669.5. Samples: 2982934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:37:53,165][74987] Avg episode reward: [(0, '10.590'), (1, '11.550')] -[2023-10-14 13:37:53,347][75950] Updated weights for policy 1, policy_version 5810 (0.0008) -[2023-10-14 13:37:53,720][75950] Updated weights for policy 1, policy_version 5820 (0.0008) -[2023-10-14 13:37:55,588][75949] Updated weights for policy 0, policy_version 5830 (0.0007) -[2023-10-14 13:37:55,968][75949] Updated weights for policy 0, policy_version 5840 (0.0007) -[2023-10-14 13:37:56,334][75949] Updated weights for policy 0, policy_version 5850 (0.0011) -[2023-10-14 13:37:57,719][75950] Updated weights for policy 1, policy_version 5830 (0.0009) -[2023-10-14 13:37:58,085][75950] Updated weights for policy 1, policy_version 5840 (0.0009) -[2023-10-14 13:37:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 11960320. Throughput: 0: 1685.2, 1: 1665.9. Samples: 3003310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:37:58,165][74987] Avg episode reward: [(0, '10.510'), (1, '11.380')] -[2023-10-14 13:37:58,463][75950] Updated weights for policy 1, policy_version 5850 (0.0008) -[2023-10-14 13:38:00,496][75949] Updated weights for policy 0, policy_version 5860 (0.0008) -[2023-10-14 13:38:00,862][75949] Updated weights for policy 0, policy_version 5870 (0.0009) -[2023-10-14 13:38:01,236][75949] Updated weights for policy 0, policy_version 5880 (0.0009) -[2023-10-14 13:38:02,551][75950] Updated weights for policy 1, policy_version 5860 (0.0008) -[2023-10-14 13:38:02,915][75950] Updated weights for policy 1, policy_version 5870 (0.0010) -[2023-10-14 13:38:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 12025856. Throughput: 0: 1682.0, 1: 1672.4. Samples: 3013452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:38:03,165][74987] Avg episode reward: [(0, '10.790'), (1, '12.390')] -[2023-10-14 13:38:03,289][75950] Updated weights for policy 1, policy_version 5880 (0.0010) -[2023-10-14 13:38:05,279][75949] Updated weights for policy 0, policy_version 5890 (0.0008) -[2023-10-14 13:38:05,649][75949] Updated weights for policy 0, policy_version 5900 (0.0008) -[2023-10-14 13:38:06,014][75949] Updated weights for policy 0, policy_version 5910 (0.0007) -[2023-10-14 13:38:06,385][75949] Updated weights for policy 0, policy_version 5920 (0.0010) -[2023-10-14 13:38:07,390][75950] Updated weights for policy 1, policy_version 5890 (0.0008) -[2023-10-14 13:38:07,776][75950] Updated weights for policy 1, policy_version 5900 (0.0008) -[2023-10-14 13:38:08,150][75950] Updated weights for policy 1, policy_version 5910 (0.0008) -[2023-10-14 13:38:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12091392. Throughput: 0: 1661.7, 1: 1679.6. Samples: 3033246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:38:08,164][74987] Avg episode reward: [(0, '9.740'), (1, '12.750')] -[2023-10-14 13:38:08,517][75950] Updated weights for policy 1, policy_version 5920 (0.0008) -[2023-10-14 13:38:10,422][75949] Updated weights for policy 0, policy_version 5930 (0.0007) -[2023-10-14 13:38:10,798][75949] Updated weights for policy 0, policy_version 5940 (0.0007) -[2023-10-14 13:38:11,166][75949] Updated weights for policy 0, policy_version 5950 (0.0007) -[2023-10-14 13:38:12,648][75950] Updated weights for policy 1, policy_version 5930 (0.0009) -[2023-10-14 13:38:13,005][75950] Updated weights for policy 1, policy_version 5940 (0.0009) -[2023-10-14 13:38:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 12156928. Throughput: 0: 1696.7, 1: 1664.0. Samples: 3053740. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) -[2023-10-14 13:38:13,165][74987] Avg episode reward: [(0, '10.920'), (1, '14.040')] -[2023-10-14 13:38:13,374][75950] Updated weights for policy 1, policy_version 5950 (0.0010) -[2023-10-14 13:38:13,449][75801] Saving new best policy, reward=14.040! -[2023-10-14 13:38:15,251][75949] Updated weights for policy 0, policy_version 5960 (0.0010) -[2023-10-14 13:38:15,626][75949] Updated weights for policy 0, policy_version 5970 (0.0009) -[2023-10-14 13:38:16,005][75949] Updated weights for policy 0, policy_version 5980 (0.0009) -[2023-10-14 13:38:17,419][75950] Updated weights for policy 1, policy_version 5960 (0.0010) -[2023-10-14 13:38:17,787][75950] Updated weights for policy 1, policy_version 5970 (0.0009) -[2023-10-14 13:38:18,151][75950] Updated weights for policy 1, policy_version 5980 (0.0007) -[2023-10-14 13:38:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12222464. Throughput: 0: 1675.1, 1: 1676.3. Samples: 3063684. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) -[2023-10-14 13:38:18,165][74987] Avg episode reward: [(0, '10.880'), (1, '11.640')] -[2023-10-14 13:38:20,213][75949] Updated weights for policy 0, policy_version 5990 (0.0010) -[2023-10-14 13:38:20,575][75949] Updated weights for policy 0, policy_version 6000 (0.0009) -[2023-10-14 13:38:20,950][75949] Updated weights for policy 0, policy_version 6010 (0.0012) -[2023-10-14 13:38:22,157][75950] Updated weights for policy 1, policy_version 5990 (0.0008) -[2023-10-14 13:38:22,518][75950] Updated weights for policy 1, policy_version 6000 (0.0010) -[2023-10-14 13:38:22,889][75950] Updated weights for policy 1, policy_version 6010 (0.0011) -[2023-10-14 13:38:23,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 12320768. Throughput: 0: 1669.0, 1: 1679.2. Samples: 3083752. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) -[2023-10-14 13:38:23,164][74987] Avg episode reward: [(0, '10.530'), (1, '12.680')] -[2023-10-14 13:38:25,099][75949] Updated weights for policy 0, policy_version 6020 (0.0008) -[2023-10-14 13:38:25,468][75949] Updated weights for policy 0, policy_version 6030 (0.0010) -[2023-10-14 13:38:25,844][75949] Updated weights for policy 0, policy_version 6040 (0.0009) -[2023-10-14 13:38:26,959][75950] Updated weights for policy 1, policy_version 6020 (0.0010) -[2023-10-14 13:38:27,342][75950] Updated weights for policy 1, policy_version 6030 (0.0007) -[2023-10-14 13:38:27,706][75950] Updated weights for policy 1, policy_version 6040 (0.0008) -[2023-10-14 13:38:28,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 12386304. Throughput: 0: 1691.0, 1: 1662.5. Samples: 3103850. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) -[2023-10-14 13:38:28,164][74987] Avg episode reward: [(0, '10.830'), (1, '12.140')] -[2023-10-14 13:38:29,909][75949] Updated weights for policy 0, policy_version 6050 (0.0011) -[2023-10-14 13:38:30,288][75949] Updated weights for policy 0, policy_version 6060 (0.0010) -[2023-10-14 13:38:30,665][75949] Updated weights for policy 0, policy_version 6070 (0.0008) -[2023-10-14 13:38:31,034][75949] Updated weights for policy 0, policy_version 6080 (0.0007) -[2023-10-14 13:38:31,678][75950] Updated weights for policy 1, policy_version 6050 (0.0010) -[2023-10-14 13:38:32,044][75950] Updated weights for policy 1, policy_version 6060 (0.0008) -[2023-10-14 13:38:32,418][75950] Updated weights for policy 1, policy_version 6070 (0.0007) -[2023-10-14 13:38:32,787][75950] Updated weights for policy 1, policy_version 6080 (0.0008) -[2023-10-14 13:38:33,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 12451840. Throughput: 0: 1666.4, 1: 1688.1. Samples: 3114358. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-14 13:38:33,164][74987] Avg episode reward: [(0, '10.790'), (1, '13.200')] -[2023-10-14 13:38:35,120][75949] Updated weights for policy 0, policy_version 6090 (0.0011) -[2023-10-14 13:38:35,497][75949] Updated weights for policy 0, policy_version 6100 (0.0008) -[2023-10-14 13:38:35,866][75949] Updated weights for policy 0, policy_version 6110 (0.0010) -[2023-10-14 13:38:36,905][75950] Updated weights for policy 1, policy_version 6090 (0.0008) -[2023-10-14 13:38:37,270][75950] Updated weights for policy 1, policy_version 6100 (0.0008) -[2023-10-14 13:38:37,638][75950] Updated weights for policy 1, policy_version 6110 (0.0007) -[2023-10-14 13:38:38,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 12517376. Throughput: 0: 1678.4, 1: 1683.3. Samples: 3134206. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) -[2023-10-14 13:38:38,164][74987] Avg episode reward: [(0, '10.890'), (1, '13.200')] -[2023-10-14 13:38:39,863][75949] Updated weights for policy 0, policy_version 6120 (0.0008) -[2023-10-14 13:38:40,232][75949] Updated weights for policy 0, policy_version 6130 (0.0008) -[2023-10-14 13:38:40,606][75949] Updated weights for policy 0, policy_version 6140 (0.0009) -[2023-10-14 13:38:41,661][75950] Updated weights for policy 1, policy_version 6120 (0.0008) -[2023-10-14 13:38:42,037][75950] Updated weights for policy 1, policy_version 6130 (0.0007) -[2023-10-14 13:38:42,406][75950] Updated weights for policy 1, policy_version 6140 (0.0007) -[2023-10-14 13:38:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 12582912. Throughput: 0: 1682.4, 1: 1655.8. Samples: 3153532. Policy #0 lag: (min: 10.0, avg: 13.7, max: 42.0) -[2023-10-14 13:38:43,164][74987] Avg episode reward: [(0, '11.160'), (1, '12.190')] -[2023-10-14 13:38:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000006144_6291456.pth... -[2023-10-14 13:38:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000006144_6291456.pth... -[2023-10-14 13:38:43,214][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000004576_4685824.pth -[2023-10-14 13:38:43,215][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000004576_4685824.pth -[2023-10-14 13:38:44,837][75949] Updated weights for policy 0, policy_version 6150 (0.0011) -[2023-10-14 13:38:45,205][75949] Updated weights for policy 0, policy_version 6160 (0.0007) -[2023-10-14 13:38:45,574][75949] Updated weights for policy 0, policy_version 6170 (0.0009) -[2023-10-14 13:38:46,569][75950] Updated weights for policy 1, policy_version 6150 (0.0009) -[2023-10-14 13:38:46,928][75950] Updated weights for policy 1, policy_version 6160 (0.0009) -[2023-10-14 13:38:47,297][75950] Updated weights for policy 1, policy_version 6170 (0.0007) -[2023-10-14 13:38:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 12648448. Throughput: 0: 1665.9, 1: 1677.8. Samples: 3163916. Policy #0 lag: (min: 10.0, avg: 13.7, max: 42.0) -[2023-10-14 13:38:48,164][74987] Avg episode reward: [(0, '10.730'), (1, '13.680')] -[2023-10-14 13:38:49,811][75949] Updated weights for policy 0, policy_version 6180 (0.0007) -[2023-10-14 13:38:50,188][75949] Updated weights for policy 0, policy_version 6190 (0.0010) -[2023-10-14 13:38:50,553][75949] Updated weights for policy 0, policy_version 6200 (0.0008) -[2023-10-14 13:38:51,289][75950] Updated weights for policy 1, policy_version 6180 (0.0008) -[2023-10-14 13:38:51,655][75950] Updated weights for policy 1, policy_version 6190 (0.0007) -[2023-10-14 13:38:52,029][75950] Updated weights for policy 1, policy_version 6200 (0.0007) -[2023-10-14 13:38:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 12713984. Throughput: 0: 1675.9, 1: 1665.5. Samples: 3183606. Policy #0 lag: (min: 30.0, avg: 30.4, max: 44.0) -[2023-10-14 13:38:53,164][74987] Avg episode reward: [(0, '10.950'), (1, '12.980')] -[2023-10-14 13:38:54,616][75949] Updated weights for policy 0, policy_version 6210 (0.0008) -[2023-10-14 13:38:54,990][75949] Updated weights for policy 0, policy_version 6220 (0.0008) -[2023-10-14 13:38:55,358][75949] Updated weights for policy 0, policy_version 6230 (0.0007) -[2023-10-14 13:38:55,729][75949] Updated weights for policy 0, policy_version 6240 (0.0007) -[2023-10-14 13:38:56,173][75950] Updated weights for policy 1, policy_version 6210 (0.0008) -[2023-10-14 13:38:56,594][75950] Updated weights for policy 1, policy_version 6220 (0.0008) -[2023-10-14 13:38:56,968][75950] Updated weights for policy 1, policy_version 6230 (0.0009) -[2023-10-14 13:38:57,334][75950] Updated weights for policy 1, policy_version 6240 (0.0008) -[2023-10-14 13:38:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 12779520. Throughput: 0: 1668.0, 1: 1661.7. Samples: 3203580. Policy #0 lag: (min: 30.0, avg: 30.4, max: 44.0) -[2023-10-14 13:38:58,165][74987] Avg episode reward: [(0, '10.770'), (1, '14.360')] -[2023-10-14 13:38:58,175][75801] Saving new best policy, reward=14.360! -[2023-10-14 13:38:59,856][75949] Updated weights for policy 0, policy_version 6250 (0.0010) -[2023-10-14 13:39:00,229][75949] Updated weights for policy 0, policy_version 6260 (0.0009) -[2023-10-14 13:39:00,597][75949] Updated weights for policy 0, policy_version 6270 (0.0007) -[2023-10-14 13:39:01,354][75950] Updated weights for policy 1, policy_version 6250 (0.0009) -[2023-10-14 13:39:01,712][75950] Updated weights for policy 1, policy_version 6260 (0.0007) -[2023-10-14 13:39:02,080][75950] Updated weights for policy 1, policy_version 6270 (0.0009) -[2023-10-14 13:39:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 12845056. Throughput: 0: 1655.5, 1: 1678.7. Samples: 3213722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:39:03,165][74987] Avg episode reward: [(0, '10.800'), (1, '13.080')] -[2023-10-14 13:39:04,500][75949] Updated weights for policy 0, policy_version 6280 (0.0007) -[2023-10-14 13:39:04,875][75949] Updated weights for policy 0, policy_version 6290 (0.0008) -[2023-10-14 13:39:05,245][75949] Updated weights for policy 0, policy_version 6300 (0.0009) -[2023-10-14 13:39:06,222][75950] Updated weights for policy 1, policy_version 6280 (0.0011) -[2023-10-14 13:39:06,596][75950] Updated weights for policy 1, policy_version 6290 (0.0010) -[2023-10-14 13:39:06,967][75950] Updated weights for policy 1, policy_version 6300 (0.0010) -[2023-10-14 13:39:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 12910592. Throughput: 0: 1674.4, 1: 1661.9. Samples: 3233882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:39:08,164][74987] Avg episode reward: [(0, '11.360'), (1, '12.940')] -[2023-10-14 13:39:08,165][75615] Saving new best policy, reward=11.360! -[2023-10-14 13:39:09,283][75949] Updated weights for policy 0, policy_version 6310 (0.0009) -[2023-10-14 13:39:09,651][75949] Updated weights for policy 0, policy_version 6320 (0.0007) -[2023-10-14 13:39:10,022][75949] Updated weights for policy 0, policy_version 6330 (0.0008) -[2023-10-14 13:39:10,981][75950] Updated weights for policy 1, policy_version 6310 (0.0008) -[2023-10-14 13:39:11,340][75950] Updated weights for policy 1, policy_version 6320 (0.0008) -[2023-10-14 13:39:11,720][75950] Updated weights for policy 1, policy_version 6330 (0.0008) -[2023-10-14 13:39:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 12976128. Throughput: 0: 1671.1, 1: 1669.9. Samples: 3254194. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) -[2023-10-14 13:39:13,164][74987] Avg episode reward: [(0, '10.730'), (1, '13.600')] -[2023-10-14 13:39:13,971][75949] Updated weights for policy 0, policy_version 6340 (0.0008) -[2023-10-14 13:39:14,341][75949] Updated weights for policy 0, policy_version 6350 (0.0009) -[2023-10-14 13:39:14,709][75949] Updated weights for policy 0, policy_version 6360 (0.0011) -[2023-10-14 13:39:15,722][75950] Updated weights for policy 1, policy_version 6340 (0.0010) -[2023-10-14 13:39:16,082][75950] Updated weights for policy 1, policy_version 6350 (0.0009) -[2023-10-14 13:39:16,446][75950] Updated weights for policy 1, policy_version 6360 (0.0009) -[2023-10-14 13:39:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 13041664. Throughput: 0: 1664.3, 1: 1675.8. Samples: 3264662. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) -[2023-10-14 13:39:18,165][74987] Avg episode reward: [(0, '11.100'), (1, '13.570')] -[2023-10-14 13:39:18,701][75949] Updated weights for policy 0, policy_version 6370 (0.0010) -[2023-10-14 13:39:19,069][75949] Updated weights for policy 0, policy_version 6380 (0.0007) -[2023-10-14 13:39:19,436][75949] Updated weights for policy 0, policy_version 6390 (0.0009) -[2023-10-14 13:39:19,813][75949] Updated weights for policy 0, policy_version 6400 (0.0008) -[2023-10-14 13:39:20,637][75950] Updated weights for policy 1, policy_version 6370 (0.0011) -[2023-10-14 13:39:21,006][75950] Updated weights for policy 1, policy_version 6380 (0.0008) -[2023-10-14 13:39:21,378][75950] Updated weights for policy 1, policy_version 6390 (0.0008) -[2023-10-14 13:39:21,743][75950] Updated weights for policy 1, policy_version 6400 (0.0009) -[2023-10-14 13:39:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13107200. Throughput: 0: 1681.3, 1: 1656.0. Samples: 3284386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:39:23,164][74987] Avg episode reward: [(0, '11.030'), (1, '13.710')] -[2023-10-14 13:39:23,907][75949] Updated weights for policy 0, policy_version 6410 (0.0009) -[2023-10-14 13:39:24,278][75949] Updated weights for policy 0, policy_version 6420 (0.0008) -[2023-10-14 13:39:24,653][75949] Updated weights for policy 0, policy_version 6430 (0.0008) -[2023-10-14 13:39:25,871][75950] Updated weights for policy 1, policy_version 6410 (0.0009) -[2023-10-14 13:39:26,245][75950] Updated weights for policy 1, policy_version 6420 (0.0008) -[2023-10-14 13:39:26,603][75950] Updated weights for policy 1, policy_version 6430 (0.0010) -[2023-10-14 13:39:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13172736. Throughput: 0: 1682.1, 1: 1678.3. Samples: 3304750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:39:28,164][74987] Avg episode reward: [(0, '10.300'), (1, '13.780')] -[2023-10-14 13:39:28,717][75949] Updated weights for policy 0, policy_version 6440 (0.0009) -[2023-10-14 13:39:29,100][75949] Updated weights for policy 0, policy_version 6450 (0.0008) -[2023-10-14 13:39:29,482][75949] Updated weights for policy 0, policy_version 6460 (0.0008) -[2023-10-14 13:39:30,676][75950] Updated weights for policy 1, policy_version 6440 (0.0010) -[2023-10-14 13:39:31,033][75950] Updated weights for policy 1, policy_version 6450 (0.0008) -[2023-10-14 13:39:31,397][75950] Updated weights for policy 1, policy_version 6460 (0.0007) -[2023-10-14 13:39:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13238272. Throughput: 0: 1677.2, 1: 1674.5. Samples: 3314742. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-14 13:39:33,165][74987] Avg episode reward: [(0, '11.230'), (1, '13.980')] -[2023-10-14 13:39:33,661][75949] Updated weights for policy 0, policy_version 6470 (0.0010) -[2023-10-14 13:39:34,031][75949] Updated weights for policy 0, policy_version 6480 (0.0010) -[2023-10-14 13:39:34,402][75949] Updated weights for policy 0, policy_version 6490 (0.0010) -[2023-10-14 13:39:35,481][75950] Updated weights for policy 1, policy_version 6470 (0.0008) -[2023-10-14 13:39:35,851][75950] Updated weights for policy 1, policy_version 6480 (0.0007) -[2023-10-14 13:39:36,227][75950] Updated weights for policy 1, policy_version 6490 (0.0007) -[2023-10-14 13:39:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13303808. Throughput: 0: 1684.9, 1: 1665.5. Samples: 3334376. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-14 13:39:38,164][74987] Avg episode reward: [(0, '10.770'), (1, '13.990')] -[2023-10-14 13:39:38,591][75949] Updated weights for policy 0, policy_version 6500 (0.0007) -[2023-10-14 13:39:38,965][75949] Updated weights for policy 0, policy_version 6510 (0.0009) -[2023-10-14 13:39:39,335][75949] Updated weights for policy 0, policy_version 6520 (0.0008) -[2023-10-14 13:39:40,366][75950] Updated weights for policy 1, policy_version 6500 (0.0008) -[2023-10-14 13:39:40,740][75950] Updated weights for policy 1, policy_version 6510 (0.0008) -[2023-10-14 13:39:41,117][75950] Updated weights for policy 1, policy_version 6520 (0.0008) -[2023-10-14 13:39:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13369344. Throughput: 0: 1681.3, 1: 1684.0. Samples: 3355018. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) -[2023-10-14 13:39:43,165][74987] Avg episode reward: [(0, '10.470'), (1, '13.580')] -[2023-10-14 13:39:43,480][75949] Updated weights for policy 0, policy_version 6530 (0.0009) -[2023-10-14 13:39:43,847][75949] Updated weights for policy 0, policy_version 6540 (0.0007) -[2023-10-14 13:39:44,211][75949] Updated weights for policy 0, policy_version 6550 (0.0009) -[2023-10-14 13:39:44,581][75949] Updated weights for policy 0, policy_version 6560 (0.0010) -[2023-10-14 13:39:45,271][75950] Updated weights for policy 1, policy_version 6530 (0.0008) -[2023-10-14 13:39:45,679][75950] Updated weights for policy 1, policy_version 6540 (0.0010) -[2023-10-14 13:39:46,063][75950] Updated weights for policy 1, policy_version 6550 (0.0009) -[2023-10-14 13:39:46,422][75950] Updated weights for policy 1, policy_version 6560 (0.0008) -[2023-10-14 13:39:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13434880. Throughput: 0: 1686.9, 1: 1674.1. Samples: 3364964. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) -[2023-10-14 13:39:48,164][74987] Avg episode reward: [(0, '11.060'), (1, '13.240')] -[2023-10-14 13:39:48,533][75949] Updated weights for policy 0, policy_version 6570 (0.0009) -[2023-10-14 13:39:48,909][75949] Updated weights for policy 0, policy_version 6580 (0.0009) -[2023-10-14 13:39:49,280][75949] Updated weights for policy 0, policy_version 6590 (0.0008) -[2023-10-14 13:39:50,459][75950] Updated weights for policy 1, policy_version 6570 (0.0008) -[2023-10-14 13:39:50,835][75950] Updated weights for policy 1, policy_version 6580 (0.0007) -[2023-10-14 13:39:51,200][75950] Updated weights for policy 1, policy_version 6590 (0.0007) -[2023-10-14 13:39:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13500416. Throughput: 0: 1679.7, 1: 1670.9. Samples: 3384658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:39:53,165][74987] Avg episode reward: [(0, '10.670'), (1, '12.960')] -[2023-10-14 13:39:53,359][75949] Updated weights for policy 0, policy_version 6600 (0.0007) -[2023-10-14 13:39:53,730][75949] Updated weights for policy 0, policy_version 6610 (0.0010) -[2023-10-14 13:39:54,090][75949] Updated weights for policy 0, policy_version 6620 (0.0008) -[2023-10-14 13:39:55,173][75950] Updated weights for policy 1, policy_version 6600 (0.0008) -[2023-10-14 13:39:55,546][75950] Updated weights for policy 1, policy_version 6610 (0.0009) -[2023-10-14 13:39:55,906][75950] Updated weights for policy 1, policy_version 6620 (0.0008) -[2023-10-14 13:39:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13565952. Throughput: 0: 1676.8, 1: 1683.7. Samples: 3405418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:39:58,164][74987] Avg episode reward: [(0, '11.030'), (1, '13.690')] -[2023-10-14 13:39:58,169][75949] Updated weights for policy 0, policy_version 6630 (0.0011) -[2023-10-14 13:39:58,542][75949] Updated weights for policy 0, policy_version 6640 (0.0008) -[2023-10-14 13:39:58,915][75949] Updated weights for policy 0, policy_version 6650 (0.0008) -[2023-10-14 13:39:59,987][75950] Updated weights for policy 1, policy_version 6630 (0.0008) -[2023-10-14 13:40:00,348][75950] Updated weights for policy 1, policy_version 6640 (0.0008) -[2023-10-14 13:40:00,727][75950] Updated weights for policy 1, policy_version 6650 (0.0008) -[2023-10-14 13:40:02,928][75949] Updated weights for policy 0, policy_version 6660 (0.0008) -[2023-10-14 13:40:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13631488. Throughput: 0: 1674.7, 1: 1661.6. Samples: 3414796. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-14 13:40:03,164][74987] Avg episode reward: [(0, '11.100'), (1, '14.140')] -[2023-10-14 13:40:03,302][75949] Updated weights for policy 0, policy_version 6670 (0.0007) -[2023-10-14 13:40:03,681][75949] Updated weights for policy 0, policy_version 6680 (0.0010) -[2023-10-14 13:40:05,007][75950] Updated weights for policy 1, policy_version 6660 (0.0009) -[2023-10-14 13:40:05,366][75950] Updated weights for policy 1, policy_version 6670 (0.0009) -[2023-10-14 13:40:05,733][75950] Updated weights for policy 1, policy_version 6680 (0.0007) -[2023-10-14 13:40:07,721][75949] Updated weights for policy 0, policy_version 6690 (0.0010) -[2023-10-14 13:40:08,098][75949] Updated weights for policy 0, policy_version 6700 (0.0008) -[2023-10-14 13:40:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13697024. Throughput: 0: 1670.5, 1: 1674.0. Samples: 3434890. Policy #0 lag: (min: 31.0, avg: 43.8, max: 63.0) -[2023-10-14 13:40:08,164][74987] Avg episode reward: [(0, '10.240'), (1, '14.080')] -[2023-10-14 13:40:08,464][75949] Updated weights for policy 0, policy_version 6710 (0.0009) -[2023-10-14 13:40:08,838][75949] Updated weights for policy 0, policy_version 6720 (0.0010) -[2023-10-14 13:40:09,809][75950] Updated weights for policy 1, policy_version 6690 (0.0007) -[2023-10-14 13:40:10,179][75950] Updated weights for policy 1, policy_version 6700 (0.0007) -[2023-10-14 13:40:10,545][75950] Updated weights for policy 1, policy_version 6710 (0.0007) -[2023-10-14 13:40:10,924][75950] Updated weights for policy 1, policy_version 6720 (0.0010) -[2023-10-14 13:40:12,897][75949] Updated weights for policy 0, policy_version 6730 (0.0010) -[2023-10-14 13:40:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 13762560. Throughput: 0: 1668.4, 1: 1677.7. Samples: 3455324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:40:13,165][74987] Avg episode reward: [(0, '11.340'), (1, '14.820')] -[2023-10-14 13:40:13,175][75801] Saving new best policy, reward=14.820! -[2023-10-14 13:40:13,266][75949] Updated weights for policy 0, policy_version 6740 (0.0010) -[2023-10-14 13:40:13,641][75949] Updated weights for policy 0, policy_version 6750 (0.0010) -[2023-10-14 13:40:14,931][75950] Updated weights for policy 1, policy_version 6730 (0.0009) -[2023-10-14 13:40:15,307][75950] Updated weights for policy 1, policy_version 6740 (0.0009) -[2023-10-14 13:40:15,672][75950] Updated weights for policy 1, policy_version 6750 (0.0007) -[2023-10-14 13:40:17,790][75949] Updated weights for policy 0, policy_version 6760 (0.0009) -[2023-10-14 13:40:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 13828096. Throughput: 0: 1671.1, 1: 1661.7. Samples: 3464716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:40:18,165][74987] Avg episode reward: [(0, '10.690'), (1, '14.320')] -[2023-10-14 13:40:18,165][75949] Updated weights for policy 0, policy_version 6770 (0.0009) -[2023-10-14 13:40:18,536][75949] Updated weights for policy 0, policy_version 6780 (0.0008) -[2023-10-14 13:40:19,777][75950] Updated weights for policy 1, policy_version 6760 (0.0008) -[2023-10-14 13:40:20,143][75950] Updated weights for policy 1, policy_version 6770 (0.0007) -[2023-10-14 13:40:20,514][75950] Updated weights for policy 1, policy_version 6780 (0.0007) -[2023-10-14 13:40:22,797][75949] Updated weights for policy 0, policy_version 6790 (0.0010) -[2023-10-14 13:40:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13893632. Throughput: 0: 1671.0, 1: 1675.4. Samples: 3484964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:40:23,165][74987] Avg episode reward: [(0, '10.430'), (1, '14.490')] -[2023-10-14 13:40:23,166][75949] Updated weights for policy 0, policy_version 6800 (0.0008) -[2023-10-14 13:40:23,553][75949] Updated weights for policy 0, policy_version 6810 (0.0009) -[2023-10-14 13:40:24,497][75950] Updated weights for policy 1, policy_version 6790 (0.0009) -[2023-10-14 13:40:24,860][75950] Updated weights for policy 1, policy_version 6800 (0.0008) -[2023-10-14 13:40:25,227][75950] Updated weights for policy 1, policy_version 6810 (0.0008) -[2023-10-14 13:40:27,563][75949] Updated weights for policy 0, policy_version 6820 (0.0010) -[2023-10-14 13:40:27,930][75949] Updated weights for policy 0, policy_version 6830 (0.0008) -[2023-10-14 13:40:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13959168. Throughput: 0: 1666.6, 1: 1672.4. Samples: 3505272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:40:28,164][74987] Avg episode reward: [(0, '11.490'), (1, '14.400')] -[2023-10-14 13:40:28,310][75949] Updated weights for policy 0, policy_version 6840 (0.0009) -[2023-10-14 13:40:28,605][75615] Saving new best policy, reward=11.490! -[2023-10-14 13:40:29,280][75950] Updated weights for policy 1, policy_version 6820 (0.0008) -[2023-10-14 13:40:29,645][75950] Updated weights for policy 1, policy_version 6830 (0.0007) -[2023-10-14 13:40:30,018][75950] Updated weights for policy 1, policy_version 6840 (0.0007) -[2023-10-14 13:40:32,490][75949] Updated weights for policy 0, policy_version 6850 (0.0008) -[2023-10-14 13:40:32,860][75949] Updated weights for policy 0, policy_version 6860 (0.0007) -[2023-10-14 13:40:33,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 14024704. Throughput: 0: 1673.1, 1: 1653.4. Samples: 3514656. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 13:40:33,164][74987] Avg episode reward: [(0, '10.570'), (1, '15.130')] -[2023-10-14 13:40:33,165][75801] Saving new best policy, reward=15.130! -[2023-10-14 13:40:33,229][75949] Updated weights for policy 0, policy_version 6870 (0.0010) -[2023-10-14 13:40:33,605][75949] Updated weights for policy 0, policy_version 6880 (0.0009) -[2023-10-14 13:40:34,110][75950] Updated weights for policy 1, policy_version 6850 (0.0010) -[2023-10-14 13:40:34,525][75950] Updated weights for policy 1, policy_version 6860 (0.0010) -[2023-10-14 13:40:34,894][75950] Updated weights for policy 1, policy_version 6870 (0.0009) -[2023-10-14 13:40:35,261][75950] Updated weights for policy 1, policy_version 6880 (0.0010) -[2023-10-14 13:40:37,872][75949] Updated weights for policy 0, policy_version 6890 (0.0007) -[2023-10-14 13:40:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 14090240. Throughput: 0: 1676.7, 1: 1671.6. Samples: 3535332. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 13:40:38,164][74987] Avg episode reward: [(0, '10.950'), (1, '13.850')] -[2023-10-14 13:40:38,244][75949] Updated weights for policy 0, policy_version 6900 (0.0009) -[2023-10-14 13:40:38,614][75949] Updated weights for policy 0, policy_version 6910 (0.0007) -[2023-10-14 13:40:39,376][75950] Updated weights for policy 1, policy_version 6890 (0.0008) -[2023-10-14 13:40:39,746][75950] Updated weights for policy 1, policy_version 6900 (0.0008) -[2023-10-14 13:40:40,113][75950] Updated weights for policy 1, policy_version 6910 (0.0009) -[2023-10-14 13:40:42,765][75949] Updated weights for policy 0, policy_version 6920 (0.0011) -[2023-10-14 13:40:43,127][75949] Updated weights for policy 0, policy_version 6930 (0.0011) -[2023-10-14 13:40:43,164][74987] Fps is (10 sec: 13106.6, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 14155776. Throughput: 0: 1664.1, 1: 1671.0. Samples: 3555498. Policy #0 lag: (min: 24.0, avg: 50.1, max: 56.0) -[2023-10-14 13:40:43,165][74987] Avg episode reward: [(0, '10.670'), (1, '14.190')] -[2023-10-14 13:40:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000006912_7077888.pth... -[2023-10-14 13:40:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000005344_5472256.pth -[2023-10-14 13:40:43,494][75949] Updated weights for policy 0, policy_version 6940 (0.0011) -[2023-10-14 13:40:43,641][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000006944_7110656.pth... -[2023-10-14 13:40:43,680][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000005376_5505024.pth -[2023-10-14 13:40:44,082][75950] Updated weights for policy 1, policy_version 6920 (0.0007) -[2023-10-14 13:40:44,449][75950] Updated weights for policy 1, policy_version 6930 (0.0007) -[2023-10-14 13:40:44,816][75950] Updated weights for policy 1, policy_version 6940 (0.0009) -[2023-10-14 13:40:47,670][75949] Updated weights for policy 0, policy_version 6950 (0.0009) -[2023-10-14 13:40:48,043][75949] Updated weights for policy 0, policy_version 6960 (0.0008) -[2023-10-14 13:40:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 14221312. Throughput: 0: 1667.1, 1: 1665.3. Samples: 3564756. Policy #0 lag: (min: 24.0, avg: 50.1, max: 56.0) -[2023-10-14 13:40:48,165][74987] Avg episode reward: [(0, '9.970'), (1, '14.030')] -[2023-10-14 13:40:48,413][75949] Updated weights for policy 0, policy_version 6970 (0.0008) -[2023-10-14 13:40:49,020][75950] Updated weights for policy 1, policy_version 6950 (0.0011) -[2023-10-14 13:40:49,383][75950] Updated weights for policy 1, policy_version 6960 (0.0009) -[2023-10-14 13:40:49,753][75950] Updated weights for policy 1, policy_version 6970 (0.0007) -[2023-10-14 13:40:52,430][75949] Updated weights for policy 0, policy_version 6980 (0.0008) -[2023-10-14 13:40:52,799][75949] Updated weights for policy 0, policy_version 6990 (0.0010) -[2023-10-14 13:40:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 14286848. Throughput: 0: 1665.0, 1: 1674.6. Samples: 3585172. Policy #0 lag: (min: 31.0, avg: 45.5, max: 63.0) -[2023-10-14 13:40:53,165][74987] Avg episode reward: [(0, '11.140'), (1, '14.220')] -[2023-10-14 13:40:53,171][75949] Updated weights for policy 0, policy_version 7000 (0.0009) -[2023-10-14 13:40:53,930][75950] Updated weights for policy 1, policy_version 6980 (0.0008) -[2023-10-14 13:40:54,306][75950] Updated weights for policy 1, policy_version 6990 (0.0009) -[2023-10-14 13:40:54,672][75950] Updated weights for policy 1, policy_version 7000 (0.0009) -[2023-10-14 13:40:57,260][75949] Updated weights for policy 0, policy_version 7010 (0.0007) -[2023-10-14 13:40:57,627][75949] Updated weights for policy 0, policy_version 7020 (0.0008) -[2023-10-14 13:40:58,008][75949] Updated weights for policy 0, policy_version 7030 (0.0009) -[2023-10-14 13:40:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 14352384. Throughput: 0: 1657.4, 1: 1680.4. Samples: 3605528. Policy #0 lag: (min: 31.0, avg: 45.5, max: 63.0) -[2023-10-14 13:40:58,165][74987] Avg episode reward: [(0, '10.940'), (1, '14.870')] -[2023-10-14 13:40:58,379][75949] Updated weights for policy 0, policy_version 7040 (0.0009) -[2023-10-14 13:40:58,740][75950] Updated weights for policy 1, policy_version 7010 (0.0009) -[2023-10-14 13:40:59,112][75950] Updated weights for policy 1, policy_version 7020 (0.0011) -[2023-10-14 13:40:59,475][75950] Updated weights for policy 1, policy_version 7030 (0.0011) -[2023-10-14 13:40:59,846][75950] Updated weights for policy 1, policy_version 7040 (0.0010) -[2023-10-14 13:41:02,361][75949] Updated weights for policy 0, policy_version 7050 (0.0009) -[2023-10-14 13:41:02,739][75949] Updated weights for policy 0, policy_version 7060 (0.0009) -[2023-10-14 13:41:03,103][75949] Updated weights for policy 0, policy_version 7070 (0.0008) -[2023-10-14 13:41:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 14417920. Throughput: 0: 1669.8, 1: 1674.7. Samples: 3615216. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 13:41:03,165][74987] Avg episode reward: [(0, '10.610'), (1, '14.260')] -[2023-10-14 13:41:03,841][75950] Updated weights for policy 1, policy_version 7050 (0.0008) -[2023-10-14 13:41:04,201][75950] Updated weights for policy 1, policy_version 7060 (0.0009) -[2023-10-14 13:41:04,571][75950] Updated weights for policy 1, policy_version 7070 (0.0007) -[2023-10-14 13:41:07,172][75949] Updated weights for policy 0, policy_version 7080 (0.0008) -[2023-10-14 13:41:07,539][75949] Updated weights for policy 0, policy_version 7090 (0.0010) -[2023-10-14 13:41:07,904][75949] Updated weights for policy 0, policy_version 7100 (0.0010) -[2023-10-14 13:41:08,164][74987] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14516224. Throughput: 0: 1672.3, 1: 1680.6. Samples: 3635844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:41:08,164][74987] Avg episode reward: [(0, '11.130'), (1, '15.130')] -[2023-10-14 13:41:08,858][75950] Updated weights for policy 1, policy_version 7080 (0.0009) -[2023-10-14 13:41:09,226][75950] Updated weights for policy 1, policy_version 7090 (0.0011) -[2023-10-14 13:41:09,593][75950] Updated weights for policy 1, policy_version 7100 (0.0009) -[2023-10-14 13:41:11,865][75949] Updated weights for policy 0, policy_version 7110 (0.0009) -[2023-10-14 13:41:12,232][75949] Updated weights for policy 0, policy_version 7120 (0.0007) -[2023-10-14 13:41:12,607][75949] Updated weights for policy 0, policy_version 7130 (0.0010) -[2023-10-14 13:41:13,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14581760. Throughput: 0: 1653.8, 1: 1684.7. Samples: 3655504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:41:13,165][74987] Avg episode reward: [(0, '10.580'), (1, '15.060')] -[2023-10-14 13:41:13,483][75950] Updated weights for policy 1, policy_version 7110 (0.0009) -[2023-10-14 13:41:13,853][75950] Updated weights for policy 1, policy_version 7120 (0.0007) -[2023-10-14 13:41:14,227][75950] Updated weights for policy 1, policy_version 7130 (0.0009) -[2023-10-14 13:41:16,605][75949] Updated weights for policy 0, policy_version 7140 (0.0011) -[2023-10-14 13:41:16,967][75949] Updated weights for policy 0, policy_version 7150 (0.0010) -[2023-10-14 13:41:17,332][75949] Updated weights for policy 0, policy_version 7160 (0.0008) -[2023-10-14 13:41:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14647296. Throughput: 0: 1674.0, 1: 1684.3. Samples: 3665780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:41:18,165][74987] Avg episode reward: [(0, '10.470'), (1, '15.290')] -[2023-10-14 13:41:18,166][75801] Saving new best policy, reward=15.290! -[2023-10-14 13:41:18,445][75950] Updated weights for policy 1, policy_version 7140 (0.0008) -[2023-10-14 13:41:18,810][75950] Updated weights for policy 1, policy_version 7150 (0.0007) -[2023-10-14 13:41:19,171][75950] Updated weights for policy 1, policy_version 7160 (0.0007) -[2023-10-14 13:41:21,631][75949] Updated weights for policy 0, policy_version 7170 (0.0008) -[2023-10-14 13:41:22,004][75949] Updated weights for policy 0, policy_version 7180 (0.0007) -[2023-10-14 13:41:22,379][75949] Updated weights for policy 0, policy_version 7190 (0.0007) -[2023-10-14 13:41:22,755][75949] Updated weights for policy 0, policy_version 7200 (0.0007) -[2023-10-14 13:41:23,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 14712832. Throughput: 0: 1668.4, 1: 1684.1. Samples: 3686194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:41:23,164][74987] Avg episode reward: [(0, '10.860'), (1, '15.100')] -[2023-10-14 13:41:23,233][75950] Updated weights for policy 1, policy_version 7170 (0.0008) -[2023-10-14 13:41:23,612][75950] Updated weights for policy 1, policy_version 7180 (0.0009) -[2023-10-14 13:41:23,985][75950] Updated weights for policy 1, policy_version 7190 (0.0009) -[2023-10-14 13:41:24,355][75950] Updated weights for policy 1, policy_version 7200 (0.0010) -[2023-10-14 13:41:26,930][75949] Updated weights for policy 0, policy_version 7210 (0.0009) -[2023-10-14 13:41:27,311][75949] Updated weights for policy 0, policy_version 7220 (0.0008) -[2023-10-14 13:41:27,687][75949] Updated weights for policy 0, policy_version 7230 (0.0008) -[2023-10-14 13:41:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14778368. Throughput: 0: 1655.5, 1: 1680.9. Samples: 3705636. Policy #0 lag: (min: 29.0, avg: 29.8, max: 47.0) -[2023-10-14 13:41:28,164][74987] Avg episode reward: [(0, '10.380'), (1, '14.370')] -[2023-10-14 13:41:28,512][75950] Updated weights for policy 1, policy_version 7210 (0.0008) -[2023-10-14 13:41:28,887][75950] Updated weights for policy 1, policy_version 7220 (0.0007) -[2023-10-14 13:41:29,250][75950] Updated weights for policy 1, policy_version 7230 (0.0007) -[2023-10-14 13:41:31,531][75949] Updated weights for policy 0, policy_version 7240 (0.0010) -[2023-10-14 13:41:31,899][75949] Updated weights for policy 0, policy_version 7250 (0.0011) -[2023-10-14 13:41:32,259][75949] Updated weights for policy 0, policy_version 7260 (0.0010) -[2023-10-14 13:41:33,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14843904. Throughput: 0: 1683.1, 1: 1679.8. Samples: 3716086. Policy #0 lag: (min: 29.0, avg: 29.8, max: 47.0) -[2023-10-14 13:41:33,164][74987] Avg episode reward: [(0, '10.610'), (1, '14.260')] -[2023-10-14 13:41:33,281][75950] Updated weights for policy 1, policy_version 7240 (0.0007) -[2023-10-14 13:41:33,651][75950] Updated weights for policy 1, policy_version 7250 (0.0008) -[2023-10-14 13:41:34,019][75950] Updated weights for policy 1, policy_version 7260 (0.0008) -[2023-10-14 13:41:36,435][75949] Updated weights for policy 0, policy_version 7270 (0.0009) -[2023-10-14 13:41:36,796][75949] Updated weights for policy 0, policy_version 7280 (0.0011) -[2023-10-14 13:41:37,164][75949] Updated weights for policy 0, policy_version 7290 (0.0008) -[2023-10-14 13:41:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14909440. Throughput: 0: 1675.1, 1: 1684.3. Samples: 3736344. Policy #0 lag: (min: 1.0, avg: 18.1, max: 33.0) -[2023-10-14 13:41:38,164][74987] Avg episode reward: [(0, '11.170'), (1, '14.660')] -[2023-10-14 13:41:38,210][75950] Updated weights for policy 1, policy_version 7270 (0.0011) -[2023-10-14 13:41:38,575][75950] Updated weights for policy 1, policy_version 7280 (0.0008) -[2023-10-14 13:41:38,933][75950] Updated weights for policy 1, policy_version 7290 (0.0008) -[2023-10-14 13:41:41,038][75949] Updated weights for policy 0, policy_version 7300 (0.0008) -[2023-10-14 13:41:41,405][75949] Updated weights for policy 0, policy_version 7310 (0.0011) -[2023-10-14 13:41:41,773][75949] Updated weights for policy 0, policy_version 7320 (0.0008) -[2023-10-14 13:41:42,916][75950] Updated weights for policy 1, policy_version 7300 (0.0007) -[2023-10-14 13:41:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 14974976. Throughput: 0: 1669.3, 1: 1682.3. Samples: 3756352. Policy #0 lag: (min: 1.0, avg: 18.1, max: 33.0) -[2023-10-14 13:41:43,164][74987] Avg episode reward: [(0, '9.990'), (1, '15.530')] -[2023-10-14 13:41:43,290][75950] Updated weights for policy 1, policy_version 7310 (0.0007) -[2023-10-14 13:41:43,655][75950] Updated weights for policy 1, policy_version 7320 (0.0009) -[2023-10-14 13:41:43,939][75801] Saving new best policy, reward=15.530! -[2023-10-14 13:41:45,715][75949] Updated weights for policy 0, policy_version 7330 (0.0007) -[2023-10-14 13:41:46,085][75949] Updated weights for policy 0, policy_version 7340 (0.0009) -[2023-10-14 13:41:46,455][75949] Updated weights for policy 0, policy_version 7350 (0.0010) -[2023-10-14 13:41:46,829][75949] Updated weights for policy 0, policy_version 7360 (0.0010) -[2023-10-14 13:41:47,649][75950] Updated weights for policy 1, policy_version 7330 (0.0008) -[2023-10-14 13:41:48,018][75950] Updated weights for policy 1, policy_version 7340 (0.0008) -[2023-10-14 13:41:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 15040512. Throughput: 0: 1689.6, 1: 1681.9. Samples: 3766932. Policy #0 lag: (min: 13.0, avg: 13.9, max: 33.0) -[2023-10-14 13:41:48,165][74987] Avg episode reward: [(0, '11.350'), (1, '15.210')] -[2023-10-14 13:41:48,383][75950] Updated weights for policy 1, policy_version 7350 (0.0008) -[2023-10-14 13:41:48,749][75950] Updated weights for policy 1, policy_version 7360 (0.0008) -[2023-10-14 13:41:50,921][75949] Updated weights for policy 0, policy_version 7370 (0.0010) -[2023-10-14 13:41:51,301][75949] Updated weights for policy 0, policy_version 7380 (0.0009) -[2023-10-14 13:41:51,672][75949] Updated weights for policy 0, policy_version 7390 (0.0008) -[2023-10-14 13:41:52,791][75950] Updated weights for policy 1, policy_version 7370 (0.0009) -[2023-10-14 13:41:53,160][75950] Updated weights for policy 1, policy_version 7380 (0.0008) -[2023-10-14 13:41:53,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 15106048. Throughput: 0: 1660.8, 1: 1678.7. Samples: 3786126. Policy #0 lag: (min: 13.0, avg: 13.9, max: 33.0) -[2023-10-14 13:41:53,165][74987] Avg episode reward: [(0, '10.580'), (1, '14.990')] -[2023-10-14 13:41:53,524][75950] Updated weights for policy 1, policy_version 7390 (0.0008) -[2023-10-14 13:41:55,645][75949] Updated weights for policy 0, policy_version 7400 (0.0011) -[2023-10-14 13:41:56,016][75949] Updated weights for policy 0, policy_version 7410 (0.0009) -[2023-10-14 13:41:56,390][75949] Updated weights for policy 0, policy_version 7420 (0.0008) -[2023-10-14 13:41:57,738][75950] Updated weights for policy 1, policy_version 7400 (0.0009) -[2023-10-14 13:41:58,105][75950] Updated weights for policy 1, policy_version 7410 (0.0008) -[2023-10-14 13:41:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 15171584. Throughput: 0: 1680.9, 1: 1668.0. Samples: 3806204. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 13:41:58,165][74987] Avg episode reward: [(0, '10.590'), (1, '15.160')] -[2023-10-14 13:41:58,485][75950] Updated weights for policy 1, policy_version 7420 (0.0009) -[2023-10-14 13:42:00,599][75949] Updated weights for policy 0, policy_version 7430 (0.0008) -[2023-10-14 13:42:00,971][75949] Updated weights for policy 0, policy_version 7440 (0.0007) -[2023-10-14 13:42:01,335][75949] Updated weights for policy 0, policy_version 7450 (0.0009) -[2023-10-14 13:42:02,676][75950] Updated weights for policy 1, policy_version 7430 (0.0010) -[2023-10-14 13:42:03,049][75950] Updated weights for policy 1, policy_version 7440 (0.0010) -[2023-10-14 13:42:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 15237120. Throughput: 0: 1675.6, 1: 1672.8. Samples: 3816460. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 13:42:03,165][74987] Avg episode reward: [(0, '11.450'), (1, '15.630')] -[2023-10-14 13:42:03,420][75950] Updated weights for policy 1, policy_version 7450 (0.0010) -[2023-10-14 13:42:03,642][75801] Saving new best policy, reward=15.630! -[2023-10-14 13:42:05,394][75949] Updated weights for policy 0, policy_version 7460 (0.0009) -[2023-10-14 13:42:05,760][75949] Updated weights for policy 0, policy_version 7470 (0.0008) -[2023-10-14 13:42:06,126][75949] Updated weights for policy 0, policy_version 7480 (0.0009) -[2023-10-14 13:42:07,538][75950] Updated weights for policy 1, policy_version 7460 (0.0011) -[2023-10-14 13:42:07,910][75950] Updated weights for policy 1, policy_version 7470 (0.0009) -[2023-10-14 13:42:08,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 15302656. Throughput: 0: 1659.0, 1: 1673.4. Samples: 3836154. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-14 13:42:08,164][74987] Avg episode reward: [(0, '10.080'), (1, '15.350')] -[2023-10-14 13:42:08,272][75950] Updated weights for policy 1, policy_version 7480 (0.0009) -[2023-10-14 13:42:10,256][75949] Updated weights for policy 0, policy_version 7490 (0.0009) -[2023-10-14 13:42:10,636][75949] Updated weights for policy 0, policy_version 7500 (0.0009) -[2023-10-14 13:42:11,015][75949] Updated weights for policy 0, policy_version 7510 (0.0009) -[2023-10-14 13:42:11,378][75949] Updated weights for policy 0, policy_version 7520 (0.0009) -[2023-10-14 13:42:12,346][75950] Updated weights for policy 1, policy_version 7490 (0.0009) -[2023-10-14 13:42:12,767][75950] Updated weights for policy 1, policy_version 7500 (0.0009) -[2023-10-14 13:42:13,145][75950] Updated weights for policy 1, policy_version 7510 (0.0008) -[2023-10-14 13:42:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 15368192. Throughput: 0: 1681.5, 1: 1661.9. Samples: 3856086. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-14 13:42:13,164][74987] Avg episode reward: [(0, '10.270'), (1, '15.160')] -[2023-10-14 13:42:13,502][75950] Updated weights for policy 1, policy_version 7520 (0.0008) -[2023-10-14 13:42:15,692][75949] Updated weights for policy 0, policy_version 7530 (0.0009) -[2023-10-14 13:42:16,071][75949] Updated weights for policy 0, policy_version 7540 (0.0008) -[2023-10-14 13:42:16,443][75949] Updated weights for policy 0, policy_version 7550 (0.0010) -[2023-10-14 13:42:17,588][75950] Updated weights for policy 1, policy_version 7530 (0.0008) -[2023-10-14 13:42:17,963][75950] Updated weights for policy 1, policy_version 7540 (0.0007) -[2023-10-14 13:42:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 15433728. Throughput: 0: 1668.1, 1: 1666.1. Samples: 3866128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:42:18,165][74987] Avg episode reward: [(0, '11.420'), (1, '15.370')] -[2023-10-14 13:42:18,322][75950] Updated weights for policy 1, policy_version 7550 (0.0007) -[2023-10-14 13:42:20,546][75949] Updated weights for policy 0, policy_version 7560 (0.0010) -[2023-10-14 13:42:20,931][75949] Updated weights for policy 0, policy_version 7570 (0.0008) -[2023-10-14 13:42:21,302][75949] Updated weights for policy 0, policy_version 7580 (0.0009) -[2023-10-14 13:42:22,337][75950] Updated weights for policy 1, policy_version 7560 (0.0007) -[2023-10-14 13:42:22,703][75950] Updated weights for policy 1, policy_version 7570 (0.0008) -[2023-10-14 13:42:23,066][75950] Updated weights for policy 1, policy_version 7580 (0.0007) -[2023-10-14 13:42:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 15499264. Throughput: 0: 1653.8, 1: 1662.9. Samples: 3885596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:42:23,164][74987] Avg episode reward: [(0, '10.290'), (1, '16.070')] -[2023-10-14 13:42:23,213][75801] Saving new best policy, reward=16.070! -[2023-10-14 13:42:25,406][75949] Updated weights for policy 0, policy_version 7590 (0.0010) -[2023-10-14 13:42:25,774][75949] Updated weights for policy 0, policy_version 7600 (0.0011) -[2023-10-14 13:42:26,153][75949] Updated weights for policy 0, policy_version 7610 (0.0009) -[2023-10-14 13:42:27,003][75950] Updated weights for policy 1, policy_version 7590 (0.0008) -[2023-10-14 13:42:27,363][75950] Updated weights for policy 1, policy_version 7600 (0.0009) -[2023-10-14 13:42:27,724][75950] Updated weights for policy 1, policy_version 7610 (0.0012) -[2023-10-14 13:42:28,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 15597568. Throughput: 0: 1673.2, 1: 1643.6. Samples: 3905612. Policy #0 lag: (min: 28.0, avg: 33.1, max: 60.0) -[2023-10-14 13:42:28,165][74987] Avg episode reward: [(0, '10.540'), (1, '16.500')] -[2023-10-14 13:42:28,176][75801] Saving new best policy, reward=16.500! -[2023-10-14 13:42:30,224][75949] Updated weights for policy 0, policy_version 7620 (0.0008) -[2023-10-14 13:42:30,603][75949] Updated weights for policy 0, policy_version 7630 (0.0007) -[2023-10-14 13:42:30,978][75949] Updated weights for policy 0, policy_version 7640 (0.0007) -[2023-10-14 13:42:31,836][75950] Updated weights for policy 1, policy_version 7620 (0.0009) -[2023-10-14 13:42:32,214][75950] Updated weights for policy 1, policy_version 7630 (0.0008) -[2023-10-14 13:42:32,584][75950] Updated weights for policy 1, policy_version 7640 (0.0008) -[2023-10-14 13:42:33,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 15663104. Throughput: 0: 1655.3, 1: 1661.8. Samples: 3916198. Policy #0 lag: (min: 28.0, avg: 33.1, max: 60.0) -[2023-10-14 13:42:33,164][74987] Avg episode reward: [(0, '11.280'), (1, '15.910')] -[2023-10-14 13:42:34,896][75949] Updated weights for policy 0, policy_version 7650 (0.0009) -[2023-10-14 13:42:35,274][75949] Updated weights for policy 0, policy_version 7660 (0.0008) -[2023-10-14 13:42:35,643][75949] Updated weights for policy 0, policy_version 7670 (0.0009) -[2023-10-14 13:42:36,021][75949] Updated weights for policy 0, policy_version 7680 (0.0010) -[2023-10-14 13:42:36,645][75950] Updated weights for policy 1, policy_version 7650 (0.0011) -[2023-10-14 13:42:37,011][75950] Updated weights for policy 1, policy_version 7660 (0.0008) -[2023-10-14 13:42:37,382][75950] Updated weights for policy 1, policy_version 7670 (0.0011) -[2023-10-14 13:42:37,748][75950] Updated weights for policy 1, policy_version 7680 (0.0009) -[2023-10-14 13:42:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 15728640. Throughput: 0: 1669.9, 1: 1662.4. Samples: 3936078. Policy #0 lag: (min: 25.0, avg: 42.6, max: 57.0) -[2023-10-14 13:42:38,165][74987] Avg episode reward: [(0, '10.260'), (1, '15.610')] -[2023-10-14 13:42:40,146][75949] Updated weights for policy 0, policy_version 7690 (0.0008) -[2023-10-14 13:42:40,527][75949] Updated weights for policy 0, policy_version 7700 (0.0007) -[2023-10-14 13:42:40,893][75949] Updated weights for policy 0, policy_version 7710 (0.0009) -[2023-10-14 13:42:41,975][75950] Updated weights for policy 1, policy_version 7690 (0.0008) -[2023-10-14 13:42:42,347][75950] Updated weights for policy 1, policy_version 7700 (0.0008) -[2023-10-14 13:42:42,716][75950] Updated weights for policy 1, policy_version 7710 (0.0007) -[2023-10-14 13:42:43,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 15794176. Throughput: 0: 1671.5, 1: 1647.1. Samples: 3955542. Policy #0 lag: (min: 25.0, avg: 42.6, max: 57.0) -[2023-10-14 13:42:43,165][74987] Avg episode reward: [(0, '11.290'), (1, '15.160')] -[2023-10-14 13:42:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000007712_7897088.pth... -[2023-10-14 13:42:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000007712_7897088.pth... -[2023-10-14 13:42:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000006144_6291456.pth -[2023-10-14 13:42:43,213][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000006144_6291456.pth -[2023-10-14 13:42:43,214][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000007712_7897088.pth -[2023-10-14 13:42:43,217][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000007712_7897088.pth -[2023-10-14 13:42:44,956][75949] Updated weights for policy 0, policy_version 7720 (0.0008) -[2023-10-14 13:42:45,329][75949] Updated weights for policy 0, policy_version 7730 (0.0011) -[2023-10-14 13:42:45,701][75949] Updated weights for policy 0, policy_version 7740 (0.0010) -[2023-10-14 13:42:46,906][75950] Updated weights for policy 1, policy_version 7720 (0.0009) -[2023-10-14 13:42:47,275][75950] Updated weights for policy 1, policy_version 7730 (0.0008) -[2023-10-14 13:42:47,640][75950] Updated weights for policy 1, policy_version 7740 (0.0010) -[2023-10-14 13:42:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 15859712. Throughput: 0: 1651.4, 1: 1670.7. Samples: 3965952. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 13:42:48,165][74987] Avg episode reward: [(0, '10.930'), (1, '15.520')] -[2023-10-14 13:42:49,617][75949] Updated weights for policy 0, policy_version 7750 (0.0009) -[2023-10-14 13:42:49,993][75949] Updated weights for policy 0, policy_version 7760 (0.0010) -[2023-10-14 13:42:50,365][75949] Updated weights for policy 0, policy_version 7770 (0.0009) -[2023-10-14 13:42:51,823][75950] Updated weights for policy 1, policy_version 7750 (0.0008) -[2023-10-14 13:42:52,190][75950] Updated weights for policy 1, policy_version 7760 (0.0009) -[2023-10-14 13:42:52,563][75950] Updated weights for policy 1, policy_version 7770 (0.0007) -[2023-10-14 13:42:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 15925248. Throughput: 0: 1668.7, 1: 1665.4. Samples: 3986186. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 13:42:53,164][74987] Avg episode reward: [(0, '10.320'), (1, '14.580')] -[2023-10-14 13:42:54,582][75949] Updated weights for policy 0, policy_version 7780 (0.0008) -[2023-10-14 13:42:54,955][75949] Updated weights for policy 0, policy_version 7790 (0.0010) -[2023-10-14 13:42:55,331][75949] Updated weights for policy 0, policy_version 7800 (0.0009) -[2023-10-14 13:42:56,740][75950] Updated weights for policy 1, policy_version 7780 (0.0008) -[2023-10-14 13:42:57,105][75950] Updated weights for policy 1, policy_version 7790 (0.0008) -[2023-10-14 13:42:57,473][75950] Updated weights for policy 1, policy_version 7800 (0.0009) -[2023-10-14 13:42:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 15990784. Throughput: 0: 1675.1, 1: 1651.0. Samples: 4005760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:42:58,164][74987] Avg episode reward: [(0, '11.170'), (1, '16.030')] -[2023-10-14 13:42:59,358][75949] Updated weights for policy 0, policy_version 7810 (0.0008) -[2023-10-14 13:42:59,736][75949] Updated weights for policy 0, policy_version 7820 (0.0011) -[2023-10-14 13:43:00,106][75949] Updated weights for policy 0, policy_version 7830 (0.0011) -[2023-10-14 13:43:00,483][75949] Updated weights for policy 0, policy_version 7840 (0.0010) -[2023-10-14 13:43:01,656][75950] Updated weights for policy 1, policy_version 7810 (0.0009) -[2023-10-14 13:43:02,043][75950] Updated weights for policy 1, policy_version 7820 (0.0008) -[2023-10-14 13:43:02,405][75950] Updated weights for policy 1, policy_version 7830 (0.0010) -[2023-10-14 13:43:02,777][75950] Updated weights for policy 1, policy_version 7840 (0.0008) -[2023-10-14 13:43:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 16056320. Throughput: 0: 1655.5, 1: 1667.4. Samples: 4015658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:43:03,164][74987] Avg episode reward: [(0, '11.080'), (1, '15.950')] -[2023-10-14 13:43:04,527][75949] Updated weights for policy 0, policy_version 7850 (0.0007) -[2023-10-14 13:43:04,887][75949] Updated weights for policy 0, policy_version 7860 (0.0010) -[2023-10-14 13:43:05,255][75949] Updated weights for policy 0, policy_version 7870 (0.0009) -[2023-10-14 13:43:06,676][75950] Updated weights for policy 1, policy_version 7850 (0.0010) -[2023-10-14 13:43:07,043][75950] Updated weights for policy 1, policy_version 7860 (0.0011) -[2023-10-14 13:43:07,416][75950] Updated weights for policy 1, policy_version 7870 (0.0007) -[2023-10-14 13:43:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 16121856. Throughput: 0: 1681.1, 1: 1661.3. Samples: 4036002. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 13:43:08,165][74987] Avg episode reward: [(0, '10.730'), (1, '16.860')] -[2023-10-14 13:43:08,166][75801] Saving new best policy, reward=16.860! -[2023-10-14 13:43:09,328][75949] Updated weights for policy 0, policy_version 7880 (0.0009) -[2023-10-14 13:43:09,700][75949] Updated weights for policy 0, policy_version 7890 (0.0009) -[2023-10-14 13:43:10,085][75949] Updated weights for policy 0, policy_version 7900 (0.0009) -[2023-10-14 13:43:11,469][75950] Updated weights for policy 1, policy_version 7880 (0.0010) -[2023-10-14 13:43:11,833][75950] Updated weights for policy 1, policy_version 7890 (0.0010) -[2023-10-14 13:43:12,202][75950] Updated weights for policy 1, policy_version 7900 (0.0010) -[2023-10-14 13:43:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 16187392. Throughput: 0: 1680.0, 1: 1657.1. Samples: 4055782. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 13:43:13,165][74987] Avg episode reward: [(0, '11.200'), (1, '15.730')] -[2023-10-14 13:43:14,167][75949] Updated weights for policy 0, policy_version 7910 (0.0009) -[2023-10-14 13:43:14,539][75949] Updated weights for policy 0, policy_version 7920 (0.0008) -[2023-10-14 13:43:14,902][75949] Updated weights for policy 0, policy_version 7930 (0.0008) -[2023-10-14 13:43:16,432][75950] Updated weights for policy 1, policy_version 7910 (0.0009) -[2023-10-14 13:43:16,789][75950] Updated weights for policy 1, policy_version 7920 (0.0007) -[2023-10-14 13:43:17,158][75950] Updated weights for policy 1, policy_version 7930 (0.0009) -[2023-10-14 13:43:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 16252928. Throughput: 0: 1664.5, 1: 1666.1. Samples: 4066076. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 13:43:18,164][74987] Avg episode reward: [(0, '10.460'), (1, '16.410')] -[2023-10-14 13:43:18,957][75949] Updated weights for policy 0, policy_version 7940 (0.0008) -[2023-10-14 13:43:19,333][75949] Updated weights for policy 0, policy_version 7950 (0.0009) -[2023-10-14 13:43:19,715][75949] Updated weights for policy 0, policy_version 7960 (0.0009) -[2023-10-14 13:43:21,117][75950] Updated weights for policy 1, policy_version 7940 (0.0010) -[2023-10-14 13:43:21,482][75950] Updated weights for policy 1, policy_version 7950 (0.0008) -[2023-10-14 13:43:21,852][75950] Updated weights for policy 1, policy_version 7960 (0.0010) -[2023-10-14 13:43:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 16318464. Throughput: 0: 1680.2, 1: 1652.9. Samples: 4086066. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 13:43:23,164][74987] Avg episode reward: [(0, '10.230'), (1, '16.240')] -[2023-10-14 13:43:23,960][75949] Updated weights for policy 0, policy_version 7970 (0.0008) -[2023-10-14 13:43:24,336][75949] Updated weights for policy 0, policy_version 7980 (0.0009) -[2023-10-14 13:43:24,710][75949] Updated weights for policy 0, policy_version 7990 (0.0008) -[2023-10-14 13:43:25,081][75949] Updated weights for policy 0, policy_version 8000 (0.0009) -[2023-10-14 13:43:25,998][75950] Updated weights for policy 1, policy_version 7970 (0.0008) -[2023-10-14 13:43:26,366][75950] Updated weights for policy 1, policy_version 7980 (0.0012) -[2023-10-14 13:43:26,738][75950] Updated weights for policy 1, policy_version 7990 (0.0011) -[2023-10-14 13:43:27,094][75950] Updated weights for policy 1, policy_version 8000 (0.0010) -[2023-10-14 13:43:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 16384000. Throughput: 0: 1682.0, 1: 1664.4. Samples: 4106130. Policy #0 lag: (min: 1.0, avg: 1.6, max: 17.0) -[2023-10-14 13:43:28,164][74987] Avg episode reward: [(0, '11.270'), (1, '17.110')] -[2023-10-14 13:43:28,175][75801] Saving new best policy, reward=17.110! -[2023-10-14 13:43:29,207][75949] Updated weights for policy 0, policy_version 8010 (0.0008) -[2023-10-14 13:43:29,570][75949] Updated weights for policy 0, policy_version 8020 (0.0008) -[2023-10-14 13:43:29,947][75949] Updated weights for policy 0, policy_version 8030 (0.0010) -[2023-10-14 13:43:31,078][75950] Updated weights for policy 1, policy_version 8010 (0.0011) -[2023-10-14 13:43:31,450][75950] Updated weights for policy 1, policy_version 8020 (0.0008) -[2023-10-14 13:43:31,827][75950] Updated weights for policy 1, policy_version 8030 (0.0009) -[2023-10-14 13:43:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16449536. Throughput: 0: 1678.9, 1: 1668.3. Samples: 4116576. Policy #0 lag: (min: 1.0, avg: 1.6, max: 17.0) -[2023-10-14 13:43:33,164][74987] Avg episode reward: [(0, '10.690'), (1, '16.560')] -[2023-10-14 13:43:34,032][75949] Updated weights for policy 0, policy_version 8040 (0.0011) -[2023-10-14 13:43:34,396][75949] Updated weights for policy 0, policy_version 8050 (0.0010) -[2023-10-14 13:43:34,778][75949] Updated weights for policy 0, policy_version 8060 (0.0010) -[2023-10-14 13:43:35,912][75950] Updated weights for policy 1, policy_version 8040 (0.0008) -[2023-10-14 13:43:36,273][75950] Updated weights for policy 1, policy_version 8050 (0.0008) -[2023-10-14 13:43:36,648][75950] Updated weights for policy 1, policy_version 8060 (0.0008) -[2023-10-14 13:43:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16515072. Throughput: 0: 1684.8, 1: 1654.2. Samples: 4136442. Policy #0 lag: (min: 21.0, avg: 22.1, max: 39.0) -[2023-10-14 13:43:38,164][74987] Avg episode reward: [(0, '10.710'), (1, '17.190')] -[2023-10-14 13:43:38,165][75801] Saving new best policy, reward=17.190! -[2023-10-14 13:43:38,938][75949] Updated weights for policy 0, policy_version 8070 (0.0009) -[2023-10-14 13:43:39,311][75949] Updated weights for policy 0, policy_version 8080 (0.0010) -[2023-10-14 13:43:39,673][75949] Updated weights for policy 0, policy_version 8090 (0.0010) -[2023-10-14 13:43:40,689][75950] Updated weights for policy 1, policy_version 8070 (0.0009) -[2023-10-14 13:43:41,062][75950] Updated weights for policy 1, policy_version 8080 (0.0009) -[2023-10-14 13:43:41,424][75950] Updated weights for policy 1, policy_version 8090 (0.0007) -[2023-10-14 13:43:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 16580608. Throughput: 0: 1681.9, 1: 1680.2. Samples: 4157056. Policy #0 lag: (min: 21.0, avg: 22.1, max: 39.0) -[2023-10-14 13:43:43,165][74987] Avg episode reward: [(0, '11.210'), (1, '16.510')] -[2023-10-14 13:43:43,644][75949] Updated weights for policy 0, policy_version 8100 (0.0007) -[2023-10-14 13:43:44,024][75949] Updated weights for policy 0, policy_version 8110 (0.0009) -[2023-10-14 13:43:44,396][75949] Updated weights for policy 0, policy_version 8120 (0.0007) -[2023-10-14 13:43:45,446][75950] Updated weights for policy 1, policy_version 8100 (0.0007) -[2023-10-14 13:43:45,807][75950] Updated weights for policy 1, policy_version 8110 (0.0009) -[2023-10-14 13:43:46,178][75950] Updated weights for policy 1, policy_version 8120 (0.0008) -[2023-10-14 13:43:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16646144. Throughput: 0: 1685.2, 1: 1681.6. Samples: 4167164. Policy #0 lag: (min: 2.0, avg: 11.1, max: 34.0) -[2023-10-14 13:43:48,164][74987] Avg episode reward: [(0, '10.650'), (1, '17.490')] -[2023-10-14 13:43:48,165][75801] Saving new best policy, reward=17.490! -[2023-10-14 13:43:48,449][75949] Updated weights for policy 0, policy_version 8130 (0.0009) -[2023-10-14 13:43:48,841][75949] Updated weights for policy 0, policy_version 8140 (0.0010) -[2023-10-14 13:43:49,214][75949] Updated weights for policy 0, policy_version 8150 (0.0011) -[2023-10-14 13:43:49,582][75949] Updated weights for policy 0, policy_version 8160 (0.0009) -[2023-10-14 13:43:50,575][75950] Updated weights for policy 1, policy_version 8130 (0.0008) -[2023-10-14 13:43:50,950][75950] Updated weights for policy 1, policy_version 8140 (0.0008) -[2023-10-14 13:43:51,312][75950] Updated weights for policy 1, policy_version 8150 (0.0008) -[2023-10-14 13:43:51,682][75950] Updated weights for policy 1, policy_version 8160 (0.0008) -[2023-10-14 13:43:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16711680. Throughput: 0: 1684.5, 1: 1668.2. Samples: 4186872. Policy #0 lag: (min: 2.0, avg: 11.1, max: 34.0) -[2023-10-14 13:43:53,164][74987] Avg episode reward: [(0, '10.780'), (1, '15.990')] -[2023-10-14 13:43:53,651][75949] Updated weights for policy 0, policy_version 8170 (0.0010) -[2023-10-14 13:43:54,023][75949] Updated weights for policy 0, policy_version 8180 (0.0009) -[2023-10-14 13:43:54,393][75949] Updated weights for policy 0, policy_version 8190 (0.0007) -[2023-10-14 13:43:55,769][75950] Updated weights for policy 1, policy_version 8170 (0.0009) -[2023-10-14 13:43:56,150][75950] Updated weights for policy 1, policy_version 8180 (0.0009) -[2023-10-14 13:43:56,517][75950] Updated weights for policy 1, policy_version 8190 (0.0009) -[2023-10-14 13:43:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16777216. Throughput: 0: 1684.1, 1: 1687.9. Samples: 4207522. Policy #0 lag: (min: 15.0, avg: 16.6, max: 41.0) -[2023-10-14 13:43:58,165][74987] Avg episode reward: [(0, '10.980'), (1, '16.590')] -[2023-10-14 13:43:58,226][75949] Updated weights for policy 0, policy_version 8200 (0.0008) -[2023-10-14 13:43:58,596][75949] Updated weights for policy 0, policy_version 8210 (0.0010) -[2023-10-14 13:43:58,964][75949] Updated weights for policy 0, policy_version 8220 (0.0008) -[2023-10-14 13:44:00,385][75950] Updated weights for policy 1, policy_version 8200 (0.0009) -[2023-10-14 13:44:00,754][75950] Updated weights for policy 1, policy_version 8210 (0.0008) -[2023-10-14 13:44:01,126][75950] Updated weights for policy 1, policy_version 8220 (0.0009) -[2023-10-14 13:44:03,084][75949] Updated weights for policy 0, policy_version 8230 (0.0009) -[2023-10-14 13:44:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16842752. Throughput: 0: 1683.3, 1: 1674.1. Samples: 4217158. Policy #0 lag: (min: 15.0, avg: 16.6, max: 41.0) -[2023-10-14 13:44:03,164][74987] Avg episode reward: [(0, '10.940'), (1, '16.120')] -[2023-10-14 13:44:03,453][75949] Updated weights for policy 0, policy_version 8240 (0.0010) -[2023-10-14 13:44:03,827][75949] Updated weights for policy 0, policy_version 8250 (0.0010) -[2023-10-14 13:44:05,108][75950] Updated weights for policy 1, policy_version 8230 (0.0009) -[2023-10-14 13:44:05,477][75950] Updated weights for policy 1, policy_version 8240 (0.0008) -[2023-10-14 13:44:05,847][75950] Updated weights for policy 1, policy_version 8250 (0.0009) -[2023-10-14 13:44:07,948][75949] Updated weights for policy 0, policy_version 8260 (0.0008) -[2023-10-14 13:44:08,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 16908288. Throughput: 0: 1685.4, 1: 1674.7. Samples: 4237268. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 13:44:08,164][74987] Avg episode reward: [(0, '11.240'), (1, '15.800')] -[2023-10-14 13:44:08,326][75949] Updated weights for policy 0, policy_version 8270 (0.0010) -[2023-10-14 13:44:08,689][75949] Updated weights for policy 0, policy_version 8280 (0.0011) -[2023-10-14 13:44:10,136][75950] Updated weights for policy 1, policy_version 8260 (0.0008) -[2023-10-14 13:44:10,513][75950] Updated weights for policy 1, policy_version 8270 (0.0009) -[2023-10-14 13:44:10,874][75950] Updated weights for policy 1, policy_version 8280 (0.0011) -[2023-10-14 13:44:12,534][75949] Updated weights for policy 0, policy_version 8290 (0.0009) -[2023-10-14 13:44:12,913][75949] Updated weights for policy 0, policy_version 8300 (0.0011) -[2023-10-14 13:44:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 16973824. Throughput: 0: 1682.4, 1: 1686.3. Samples: 4257722. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 13:44:13,164][74987] Avg episode reward: [(0, '10.840'), (1, '15.800')] -[2023-10-14 13:44:13,285][75949] Updated weights for policy 0, policy_version 8310 (0.0010) -[2023-10-14 13:44:13,655][75949] Updated weights for policy 0, policy_version 8320 (0.0007) -[2023-10-14 13:44:15,127][75950] Updated weights for policy 1, policy_version 8290 (0.0009) -[2023-10-14 13:44:15,495][75950] Updated weights for policy 1, policy_version 8300 (0.0009) -[2023-10-14 13:44:15,870][75950] Updated weights for policy 1, policy_version 8310 (0.0011) -[2023-10-14 13:44:16,232][75950] Updated weights for policy 1, policy_version 8320 (0.0010) -[2023-10-14 13:44:17,629][75949] Updated weights for policy 0, policy_version 8330 (0.0010) -[2023-10-14 13:44:18,006][75949] Updated weights for policy 0, policy_version 8340 (0.0007) -[2023-10-14 13:44:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17039360. Throughput: 0: 1690.2, 1: 1669.9. Samples: 4267780. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 13:44:18,164][74987] Avg episode reward: [(0, '10.930'), (1, '15.340')] -[2023-10-14 13:44:18,376][75949] Updated weights for policy 0, policy_version 8350 (0.0007) -[2023-10-14 13:44:20,133][75950] Updated weights for policy 1, policy_version 8330 (0.0011) -[2023-10-14 13:44:20,506][75950] Updated weights for policy 1, policy_version 8340 (0.0009) -[2023-10-14 13:44:20,881][75950] Updated weights for policy 1, policy_version 8350 (0.0009) -[2023-10-14 13:44:22,403][75949] Updated weights for policy 0, policy_version 8360 (0.0007) -[2023-10-14 13:44:22,782][75949] Updated weights for policy 0, policy_version 8370 (0.0007) -[2023-10-14 13:44:23,140][75949] Updated weights for policy 0, policy_version 8380 (0.0007) -[2023-10-14 13:44:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17104896. Throughput: 0: 1686.8, 1: 1678.1. Samples: 4287864. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 13:44:23,164][74987] Avg episode reward: [(0, '10.900'), (1, '15.390')] -[2023-10-14 13:44:25,030][75950] Updated weights for policy 1, policy_version 8360 (0.0007) -[2023-10-14 13:44:25,398][75950] Updated weights for policy 1, policy_version 8370 (0.0008) -[2023-10-14 13:44:25,762][75950] Updated weights for policy 1, policy_version 8380 (0.0009) -[2023-10-14 13:44:27,230][75949] Updated weights for policy 0, policy_version 8390 (0.0009) -[2023-10-14 13:44:27,597][75949] Updated weights for policy 0, policy_version 8400 (0.0009) -[2023-10-14 13:44:27,984][75949] Updated weights for policy 0, policy_version 8410 (0.0010) -[2023-10-14 13:44:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17170432. Throughput: 0: 1670.8, 1: 1681.0. Samples: 4307886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:44:28,165][74987] Avg episode reward: [(0, '10.270'), (1, '15.940')] -[2023-10-14 13:44:29,786][75950] Updated weights for policy 1, policy_version 8390 (0.0009) -[2023-10-14 13:44:30,145][75950] Updated weights for policy 1, policy_version 8400 (0.0011) -[2023-10-14 13:44:30,520][75950] Updated weights for policy 1, policy_version 8410 (0.0008) -[2023-10-14 13:44:32,194][75949] Updated weights for policy 0, policy_version 8420 (0.0009) -[2023-10-14 13:44:32,568][75949] Updated weights for policy 0, policy_version 8430 (0.0010) -[2023-10-14 13:44:32,946][75949] Updated weights for policy 0, policy_version 8440 (0.0010) -[2023-10-14 13:44:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17235968. Throughput: 0: 1683.1, 1: 1660.8. Samples: 4317636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:44:33,164][74987] Avg episode reward: [(0, '10.810'), (1, '15.520')] -[2023-10-14 13:44:34,471][75950] Updated weights for policy 1, policy_version 8420 (0.0009) -[2023-10-14 13:44:34,833][75950] Updated weights for policy 1, policy_version 8430 (0.0009) -[2023-10-14 13:44:35,201][75950] Updated weights for policy 1, policy_version 8440 (0.0011) -[2023-10-14 13:44:37,031][75949] Updated weights for policy 0, policy_version 8450 (0.0010) -[2023-10-14 13:44:37,427][75949] Updated weights for policy 0, policy_version 8460 (0.0008) -[2023-10-14 13:44:37,795][75949] Updated weights for policy 0, policy_version 8470 (0.0007) -[2023-10-14 13:44:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17301504. Throughput: 0: 1688.2, 1: 1680.4. Samples: 4338458. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 13:44:38,165][74987] Avg episode reward: [(0, '10.760'), (1, '16.910')] -[2023-10-14 13:44:38,171][75949] Updated weights for policy 0, policy_version 8480 (0.0010) -[2023-10-14 13:44:39,388][75950] Updated weights for policy 1, policy_version 8450 (0.0008) -[2023-10-14 13:44:39,809][75950] Updated weights for policy 1, policy_version 8460 (0.0007) -[2023-10-14 13:44:40,175][75950] Updated weights for policy 1, policy_version 8470 (0.0009) -[2023-10-14 13:44:40,544][75950] Updated weights for policy 1, policy_version 8480 (0.0008) -[2023-10-14 13:44:42,308][75949] Updated weights for policy 0, policy_version 8490 (0.0007) -[2023-10-14 13:44:42,686][75949] Updated weights for policy 0, policy_version 8500 (0.0007) -[2023-10-14 13:44:43,051][75949] Updated weights for policy 0, policy_version 8510 (0.0007) -[2023-10-14 13:44:43,163][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 17399808. Throughput: 0: 1665.4, 1: 1676.9. Samples: 4357924. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-14 13:44:43,164][74987] Avg episode reward: [(0, '10.450'), (1, '15.700')] -[2023-10-14 13:44:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000008512_8716288.pth... -[2023-10-14 13:44:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000008480_8683520.pth... -[2023-10-14 13:44:43,212][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000006912_7077888.pth -[2023-10-14 13:44:43,217][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000006944_7110656.pth -[2023-10-14 13:44:44,685][75950] Updated weights for policy 1, policy_version 8490 (0.0008) -[2023-10-14 13:44:45,052][75950] Updated weights for policy 1, policy_version 8500 (0.0008) -[2023-10-14 13:44:45,429][75950] Updated weights for policy 1, policy_version 8510 (0.0007) -[2023-10-14 13:44:47,158][75949] Updated weights for policy 0, policy_version 8520 (0.0007) -[2023-10-14 13:44:47,526][75949] Updated weights for policy 0, policy_version 8530 (0.0007) -[2023-10-14 13:44:47,904][75949] Updated weights for policy 0, policy_version 8540 (0.0011) -[2023-10-14 13:44:48,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17465344. Throughput: 0: 1683.1, 1: 1663.6. Samples: 4367760. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) -[2023-10-14 13:44:48,165][74987] Avg episode reward: [(0, '11.320'), (1, '17.790')] -[2023-10-14 13:44:48,166][75801] Saving new best policy, reward=17.790! -[2023-10-14 13:44:49,371][75950] Updated weights for policy 1, policy_version 8520 (0.0009) -[2023-10-14 13:44:49,737][75950] Updated weights for policy 1, policy_version 8530 (0.0011) -[2023-10-14 13:44:50,101][75950] Updated weights for policy 1, policy_version 8540 (0.0008) -[2023-10-14 13:44:52,041][75949] Updated weights for policy 0, policy_version 8550 (0.0009) -[2023-10-14 13:44:52,414][75949] Updated weights for policy 0, policy_version 8560 (0.0008) -[2023-10-14 13:44:52,793][75949] Updated weights for policy 0, policy_version 8570 (0.0008) -[2023-10-14 13:44:53,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 17530880. Throughput: 0: 1679.3, 1: 1678.0. Samples: 4388344. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-14 13:44:53,164][74987] Avg episode reward: [(0, '10.380'), (1, '15.820')] -[2023-10-14 13:44:54,106][75950] Updated weights for policy 1, policy_version 8550 (0.0009) -[2023-10-14 13:44:54,471][75950] Updated weights for policy 1, policy_version 8560 (0.0008) -[2023-10-14 13:44:54,842][75950] Updated weights for policy 1, policy_version 8570 (0.0008) -[2023-10-14 13:44:56,861][75949] Updated weights for policy 0, policy_version 8580 (0.0010) -[2023-10-14 13:44:57,235][75949] Updated weights for policy 0, policy_version 8590 (0.0007) -[2023-10-14 13:44:57,605][75949] Updated weights for policy 0, policy_version 8600 (0.0007) -[2023-10-14 13:44:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17596416. Throughput: 0: 1659.1, 1: 1682.1. Samples: 4408076. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-14 13:44:58,165][74987] Avg episode reward: [(0, '10.710'), (1, '16.680')] -[2023-10-14 13:44:58,953][75950] Updated weights for policy 1, policy_version 8580 (0.0008) -[2023-10-14 13:44:59,324][75950] Updated weights for policy 1, policy_version 8590 (0.0009) -[2023-10-14 13:44:59,693][75950] Updated weights for policy 1, policy_version 8600 (0.0008) -[2023-10-14 13:45:01,745][75949] Updated weights for policy 0, policy_version 8610 (0.0009) -[2023-10-14 13:45:02,126][75949] Updated weights for policy 0, policy_version 8620 (0.0008) -[2023-10-14 13:45:02,489][75949] Updated weights for policy 0, policy_version 8630 (0.0010) -[2023-10-14 13:45:02,862][75949] Updated weights for policy 0, policy_version 8640 (0.0010) -[2023-10-14 13:45:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17661952. Throughput: 0: 1673.5, 1: 1665.6. Samples: 4418038. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) -[2023-10-14 13:45:03,164][74987] Avg episode reward: [(0, '10.750'), (1, '15.540')] -[2023-10-14 13:45:03,846][75950] Updated weights for policy 1, policy_version 8610 (0.0009) -[2023-10-14 13:45:04,222][75950] Updated weights for policy 1, policy_version 8620 (0.0008) -[2023-10-14 13:45:04,583][75950] Updated weights for policy 1, policy_version 8630 (0.0009) -[2023-10-14 13:45:04,952][75950] Updated weights for policy 1, policy_version 8640 (0.0010) -[2023-10-14 13:45:06,818][75949] Updated weights for policy 0, policy_version 8650 (0.0009) -[2023-10-14 13:45:07,186][75949] Updated weights for policy 0, policy_version 8660 (0.0007) -[2023-10-14 13:45:07,564][75949] Updated weights for policy 0, policy_version 8670 (0.0007) -[2023-10-14 13:45:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17727488. Throughput: 0: 1668.5, 1: 1678.9. Samples: 4438498. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) -[2023-10-14 13:45:08,165][74987] Avg episode reward: [(0, '10.630'), (1, '16.780')] -[2023-10-14 13:45:08,939][75950] Updated weights for policy 1, policy_version 8650 (0.0007) -[2023-10-14 13:45:09,307][75950] Updated weights for policy 1, policy_version 8660 (0.0007) -[2023-10-14 13:45:09,685][75950] Updated weights for policy 1, policy_version 8670 (0.0009) -[2023-10-14 13:45:11,731][75949] Updated weights for policy 0, policy_version 8680 (0.0009) -[2023-10-14 13:45:12,104][75949] Updated weights for policy 0, policy_version 8690 (0.0008) -[2023-10-14 13:45:12,483][75949] Updated weights for policy 0, policy_version 8700 (0.0009) -[2023-10-14 13:45:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17793024. Throughput: 0: 1659.5, 1: 1683.7. Samples: 4458330. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 13:45:13,164][74987] Avg episode reward: [(0, '10.740'), (1, '15.040')] -[2023-10-14 13:45:13,542][75950] Updated weights for policy 1, policy_version 8680 (0.0009) -[2023-10-14 13:45:13,909][75950] Updated weights for policy 1, policy_version 8690 (0.0009) -[2023-10-14 13:45:14,271][75950] Updated weights for policy 1, policy_version 8700 (0.0008) -[2023-10-14 13:45:16,608][75949] Updated weights for policy 0, policy_version 8710 (0.0007) -[2023-10-14 13:45:16,985][75949] Updated weights for policy 0, policy_version 8720 (0.0007) -[2023-10-14 13:45:17,343][75949] Updated weights for policy 0, policy_version 8730 (0.0011) -[2023-10-14 13:45:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17858560. Throughput: 0: 1674.8, 1: 1679.6. Samples: 4468586. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 13:45:18,165][74987] Avg episode reward: [(0, '11.180'), (1, '16.460')] -[2023-10-14 13:45:18,289][75950] Updated weights for policy 1, policy_version 8710 (0.0007) -[2023-10-14 13:45:18,656][75950] Updated weights for policy 1, policy_version 8720 (0.0007) -[2023-10-14 13:45:19,024][75950] Updated weights for policy 1, policy_version 8730 (0.0009) -[2023-10-14 13:45:21,507][75949] Updated weights for policy 0, policy_version 8740 (0.0009) -[2023-10-14 13:45:21,898][75949] Updated weights for policy 0, policy_version 8750 (0.0007) -[2023-10-14 13:45:22,268][75949] Updated weights for policy 0, policy_version 8760 (0.0008) -[2023-10-14 13:45:23,038][75950] Updated weights for policy 1, policy_version 8740 (0.0008) -[2023-10-14 13:45:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17924096. Throughput: 0: 1662.1, 1: 1680.9. Samples: 4488894. Policy #0 lag: (min: 9.0, avg: 26.8, max: 41.0) -[2023-10-14 13:45:23,165][74987] Avg episode reward: [(0, '10.250'), (1, '16.060')] -[2023-10-14 13:45:23,399][75950] Updated weights for policy 1, policy_version 8750 (0.0009) -[2023-10-14 13:45:23,773][75950] Updated weights for policy 1, policy_version 8760 (0.0007) -[2023-10-14 13:45:26,304][75949] Updated weights for policy 0, policy_version 8770 (0.0009) -[2023-10-14 13:45:26,673][75949] Updated weights for policy 0, policy_version 8780 (0.0009) -[2023-10-14 13:45:27,051][75949] Updated weights for policy 0, policy_version 8790 (0.0008) -[2023-10-14 13:45:27,415][75949] Updated weights for policy 0, policy_version 8800 (0.0008) -[2023-10-14 13:45:27,991][75950] Updated weights for policy 1, policy_version 8770 (0.0007) -[2023-10-14 13:45:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 17989632. Throughput: 0: 1661.9, 1: 1690.6. Samples: 4508786. Policy #0 lag: (min: 9.0, avg: 26.8, max: 41.0) -[2023-10-14 13:45:28,164][74987] Avg episode reward: [(0, '10.180'), (1, '17.690')] -[2023-10-14 13:45:28,387][75950] Updated weights for policy 1, policy_version 8780 (0.0007) -[2023-10-14 13:45:28,755][75950] Updated weights for policy 1, policy_version 8790 (0.0007) -[2023-10-14 13:45:29,126][75950] Updated weights for policy 1, policy_version 8800 (0.0010) -[2023-10-14 13:45:31,266][75949] Updated weights for policy 0, policy_version 8810 (0.0011) -[2023-10-14 13:45:31,630][75949] Updated weights for policy 0, policy_version 8820 (0.0010) -[2023-10-14 13:45:32,009][75949] Updated weights for policy 0, policy_version 8830 (0.0011) -[2023-10-14 13:45:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 18055168. Throughput: 0: 1675.5, 1: 1686.9. Samples: 4519068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:45:33,165][74987] Avg episode reward: [(0, '11.170'), (1, '16.500')] -[2023-10-14 13:45:33,248][75950] Updated weights for policy 1, policy_version 8810 (0.0007) -[2023-10-14 13:45:33,617][75950] Updated weights for policy 1, policy_version 8820 (0.0007) -[2023-10-14 13:45:33,979][75950] Updated weights for policy 1, policy_version 8830 (0.0008) -[2023-10-14 13:45:36,249][75949] Updated weights for policy 0, policy_version 8840 (0.0008) -[2023-10-14 13:45:36,619][75949] Updated weights for policy 0, policy_version 8850 (0.0008) -[2023-10-14 13:45:36,988][75949] Updated weights for policy 0, policy_version 8860 (0.0008) -[2023-10-14 13:45:38,082][75950] Updated weights for policy 1, policy_version 8840 (0.0008) -[2023-10-14 13:45:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.5). Total num frames: 18120704. Throughput: 0: 1654.2, 1: 1687.4. Samples: 4538714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:45:38,164][74987] Avg episode reward: [(0, '9.880'), (1, '17.400')] -[2023-10-14 13:45:38,451][75950] Updated weights for policy 1, policy_version 8850 (0.0010) -[2023-10-14 13:45:38,826][75950] Updated weights for policy 1, policy_version 8860 (0.0011) -[2023-10-14 13:45:41,080][75949] Updated weights for policy 0, policy_version 8870 (0.0008) -[2023-10-14 13:45:41,441][75949] Updated weights for policy 0, policy_version 8880 (0.0009) -[2023-10-14 13:45:41,815][75949] Updated weights for policy 0, policy_version 8890 (0.0011) -[2023-10-14 13:45:42,954][75950] Updated weights for policy 1, policy_version 8870 (0.0007) -[2023-10-14 13:45:43,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 18186240. Throughput: 0: 1664.6, 1: 1681.1. Samples: 4558630. Policy #0 lag: (min: 17.0, avg: 21.9, max: 49.0) -[2023-10-14 13:45:43,164][74987] Avg episode reward: [(0, '10.720'), (1, '16.160')] -[2023-10-14 13:45:43,324][75950] Updated weights for policy 1, policy_version 8880 (0.0009) -[2023-10-14 13:45:43,690][75950] Updated weights for policy 1, policy_version 8890 (0.0009) -[2023-10-14 13:45:46,064][75949] Updated weights for policy 0, policy_version 8900 (0.0011) -[2023-10-14 13:45:46,434][75949] Updated weights for policy 0, policy_version 8910 (0.0007) -[2023-10-14 13:45:46,810][75949] Updated weights for policy 0, policy_version 8920 (0.0008) -[2023-10-14 13:45:47,721][75950] Updated weights for policy 1, policy_version 8900 (0.0009) -[2023-10-14 13:45:48,088][75950] Updated weights for policy 1, policy_version 8910 (0.0009) -[2023-10-14 13:45:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 18251776. Throughput: 0: 1669.5, 1: 1685.1. Samples: 4568996. Policy #0 lag: (min: 17.0, avg: 21.9, max: 49.0) -[2023-10-14 13:45:48,165][74987] Avg episode reward: [(0, '11.070'), (1, '16.980')] -[2023-10-14 13:45:48,457][75950] Updated weights for policy 1, policy_version 8920 (0.0010) -[2023-10-14 13:45:50,896][75949] Updated weights for policy 0, policy_version 8930 (0.0008) -[2023-10-14 13:45:51,265][75949] Updated weights for policy 0, policy_version 8940 (0.0009) -[2023-10-14 13:45:51,633][75949] Updated weights for policy 0, policy_version 8950 (0.0007) -[2023-10-14 13:45:52,006][75949] Updated weights for policy 0, policy_version 8960 (0.0007) -[2023-10-14 13:45:52,457][75950] Updated weights for policy 1, policy_version 8930 (0.0007) -[2023-10-14 13:45:52,825][75950] Updated weights for policy 1, policy_version 8940 (0.0007) -[2023-10-14 13:45:53,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 18317312. Throughput: 0: 1657.2, 1: 1683.8. Samples: 4588846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:45:53,165][74987] Avg episode reward: [(0, '10.200'), (1, '16.510')] -[2023-10-14 13:45:53,206][75950] Updated weights for policy 1, policy_version 8950 (0.0007) -[2023-10-14 13:45:53,576][75950] Updated weights for policy 1, policy_version 8960 (0.0008) -[2023-10-14 13:45:56,002][75949] Updated weights for policy 0, policy_version 8970 (0.0009) -[2023-10-14 13:45:56,381][75949] Updated weights for policy 0, policy_version 8980 (0.0011) -[2023-10-14 13:45:56,748][75949] Updated weights for policy 0, policy_version 8990 (0.0009) -[2023-10-14 13:45:57,735][75950] Updated weights for policy 1, policy_version 8970 (0.0008) -[2023-10-14 13:45:58,106][75950] Updated weights for policy 1, policy_version 8980 (0.0008) -[2023-10-14 13:45:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 18382848. Throughput: 0: 1672.5, 1: 1675.6. Samples: 4608996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:45:58,165][74987] Avg episode reward: [(0, '11.590'), (1, '17.670')] -[2023-10-14 13:45:58,174][75615] Saving new best policy, reward=11.590! -[2023-10-14 13:45:58,476][75950] Updated weights for policy 1, policy_version 8990 (0.0008) -[2023-10-14 13:46:00,805][75949] Updated weights for policy 0, policy_version 9000 (0.0009) -[2023-10-14 13:46:01,180][75949] Updated weights for policy 0, policy_version 9010 (0.0009) -[2023-10-14 13:46:01,557][75949] Updated weights for policy 0, policy_version 9020 (0.0008) -[2023-10-14 13:46:02,553][75950] Updated weights for policy 1, policy_version 9000 (0.0009) -[2023-10-14 13:46:02,921][75950] Updated weights for policy 1, policy_version 9010 (0.0009) -[2023-10-14 13:46:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18448384. Throughput: 0: 1672.5, 1: 1681.3. Samples: 4619506. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) -[2023-10-14 13:46:03,164][74987] Avg episode reward: [(0, '10.520'), (1, '17.650')] -[2023-10-14 13:46:03,292][75950] Updated weights for policy 1, policy_version 9020 (0.0007) -[2023-10-14 13:46:05,583][75949] Updated weights for policy 0, policy_version 9030 (0.0011) -[2023-10-14 13:46:05,962][75949] Updated weights for policy 0, policy_version 9040 (0.0009) -[2023-10-14 13:46:06,340][75949] Updated weights for policy 0, policy_version 9050 (0.0010) -[2023-10-14 13:46:07,532][75950] Updated weights for policy 1, policy_version 9030 (0.0009) -[2023-10-14 13:46:07,899][75950] Updated weights for policy 1, policy_version 9040 (0.0009) -[2023-10-14 13:46:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 18513920. Throughput: 0: 1654.4, 1: 1681.4. Samples: 4639004. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) -[2023-10-14 13:46:08,164][74987] Avg episode reward: [(0, '10.400'), (1, '18.190')] -[2023-10-14 13:46:08,274][75950] Updated weights for policy 1, policy_version 9050 (0.0008) -[2023-10-14 13:46:08,496][75801] Saving new best policy, reward=18.190! -[2023-10-14 13:46:10,378][75949] Updated weights for policy 0, policy_version 9060 (0.0008) -[2023-10-14 13:46:10,771][75949] Updated weights for policy 0, policy_version 9070 (0.0008) -[2023-10-14 13:46:11,147][75949] Updated weights for policy 0, policy_version 9080 (0.0008) -[2023-10-14 13:46:12,261][75950] Updated weights for policy 1, policy_version 9060 (0.0009) -[2023-10-14 13:46:12,626][75950] Updated weights for policy 1, policy_version 9070 (0.0008) -[2023-10-14 13:46:13,001][75950] Updated weights for policy 1, policy_version 9080 (0.0011) -[2023-10-14 13:46:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18579456. Throughput: 0: 1671.0, 1: 1662.7. Samples: 4658804. Policy #0 lag: (min: 26.0, avg: 29.5, max: 56.0) -[2023-10-14 13:46:13,165][74987] Avg episode reward: [(0, '11.150'), (1, '18.070')] -[2023-10-14 13:46:15,259][75949] Updated weights for policy 0, policy_version 9090 (0.0007) -[2023-10-14 13:46:15,623][75949] Updated weights for policy 0, policy_version 9100 (0.0007) -[2023-10-14 13:46:16,006][75949] Updated weights for policy 0, policy_version 9110 (0.0007) -[2023-10-14 13:46:16,372][75949] Updated weights for policy 0, policy_version 9120 (0.0009) -[2023-10-14 13:46:17,148][75950] Updated weights for policy 1, policy_version 9090 (0.0009) -[2023-10-14 13:46:17,568][75950] Updated weights for policy 1, policy_version 9100 (0.0008) -[2023-10-14 13:46:17,933][75950] Updated weights for policy 1, policy_version 9110 (0.0008) -[2023-10-14 13:46:18,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 18644992. Throughput: 0: 1657.8, 1: 1679.2. Samples: 4669234. Policy #0 lag: (min: 26.0, avg: 29.5, max: 56.0) -[2023-10-14 13:46:18,165][74987] Avg episode reward: [(0, '9.930'), (1, '16.570')] -[2023-10-14 13:46:18,307][75950] Updated weights for policy 1, policy_version 9120 (0.0008) -[2023-10-14 13:46:20,365][75949] Updated weights for policy 0, policy_version 9130 (0.0010) -[2023-10-14 13:46:20,729][75949] Updated weights for policy 0, policy_version 9140 (0.0010) -[2023-10-14 13:46:21,102][75949] Updated weights for policy 0, policy_version 9150 (0.0010) -[2023-10-14 13:46:22,347][75950] Updated weights for policy 1, policy_version 9130 (0.0007) -[2023-10-14 13:46:22,704][75950] Updated weights for policy 1, policy_version 9140 (0.0007) -[2023-10-14 13:46:23,078][75950] Updated weights for policy 1, policy_version 9150 (0.0007) -[2023-10-14 13:46:23,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 18743296. Throughput: 0: 1663.9, 1: 1677.9. Samples: 4689096. Policy #0 lag: (min: 31.0, avg: 32.3, max: 57.0) -[2023-10-14 13:46:23,165][74987] Avg episode reward: [(0, '10.370'), (1, '16.660')] -[2023-10-14 13:46:25,060][75949] Updated weights for policy 0, policy_version 9160 (0.0008) -[2023-10-14 13:46:25,429][75949] Updated weights for policy 0, policy_version 9170 (0.0008) -[2023-10-14 13:46:25,798][75949] Updated weights for policy 0, policy_version 9180 (0.0007) -[2023-10-14 13:46:26,910][75950] Updated weights for policy 1, policy_version 9160 (0.0008) -[2023-10-14 13:46:27,276][75950] Updated weights for policy 1, policy_version 9170 (0.0008) -[2023-10-14 13:46:27,641][75950] Updated weights for policy 1, policy_version 9180 (0.0007) -[2023-10-14 13:46:28,163][74987] Fps is (10 sec: 16384.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 18808832. Throughput: 0: 1682.4, 1: 1660.0. Samples: 4709034. Policy #0 lag: (min: 31.0, avg: 32.3, max: 57.0) -[2023-10-14 13:46:28,164][74987] Avg episode reward: [(0, '10.650'), (1, '15.450')] -[2023-10-14 13:46:29,928][75949] Updated weights for policy 0, policy_version 9190 (0.0009) -[2023-10-14 13:46:30,300][75949] Updated weights for policy 0, policy_version 9200 (0.0008) -[2023-10-14 13:46:30,668][75949] Updated weights for policy 0, policy_version 9210 (0.0007) -[2023-10-14 13:46:31,821][75950] Updated weights for policy 1, policy_version 9190 (0.0009) -[2023-10-14 13:46:32,174][75950] Updated weights for policy 1, policy_version 9200 (0.0008) -[2023-10-14 13:46:32,552][75950] Updated weights for policy 1, policy_version 9210 (0.0007) -[2023-10-14 13:46:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 18874368. Throughput: 0: 1662.7, 1: 1681.9. Samples: 4719502. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 13:46:33,164][74987] Avg episode reward: [(0, '10.450'), (1, '17.650')] -[2023-10-14 13:46:34,926][75949] Updated weights for policy 0, policy_version 9220 (0.0007) -[2023-10-14 13:46:35,291][75949] Updated weights for policy 0, policy_version 9230 (0.0008) -[2023-10-14 13:46:35,657][75949] Updated weights for policy 0, policy_version 9240 (0.0008) -[2023-10-14 13:46:36,598][75950] Updated weights for policy 1, policy_version 9220 (0.0008) -[2023-10-14 13:46:36,976][75950] Updated weights for policy 1, policy_version 9230 (0.0010) -[2023-10-14 13:46:37,347][75950] Updated weights for policy 1, policy_version 9240 (0.0008) -[2023-10-14 13:46:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 18939904. Throughput: 0: 1668.7, 1: 1673.0. Samples: 4739222. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 13:46:38,165][74987] Avg episode reward: [(0, '11.070'), (1, '15.420')] -[2023-10-14 13:46:39,662][75949] Updated weights for policy 0, policy_version 9250 (0.0007) -[2023-10-14 13:46:40,034][75949] Updated weights for policy 0, policy_version 9260 (0.0009) -[2023-10-14 13:46:40,414][75949] Updated weights for policy 0, policy_version 9270 (0.0009) -[2023-10-14 13:46:40,782][75949] Updated weights for policy 0, policy_version 9280 (0.0009) -[2023-10-14 13:46:41,625][75950] Updated weights for policy 1, policy_version 9250 (0.0008) -[2023-10-14 13:46:41,987][75950] Updated weights for policy 1, policy_version 9260 (0.0010) -[2023-10-14 13:46:42,352][75950] Updated weights for policy 1, policy_version 9270 (0.0010) -[2023-10-14 13:46:42,721][75950] Updated weights for policy 1, policy_version 9280 (0.0009) -[2023-10-14 13:46:43,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 19005440. Throughput: 0: 1680.2, 1: 1653.3. Samples: 4759004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:46:43,165][74987] Avg episode reward: [(0, '10.610'), (1, '18.320')] -[2023-10-14 13:46:43,178][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000009280_9502720.pth... -[2023-10-14 13:46:43,178][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000009280_9502720.pth... -[2023-10-14 13:46:43,215][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000007712_7897088.pth -[2023-10-14 13:46:43,219][75801] Saving new best policy, reward=18.320! -[2023-10-14 13:46:43,220][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000007712_7897088.pth -[2023-10-14 13:46:44,890][75949] Updated weights for policy 0, policy_version 9290 (0.0008) -[2023-10-14 13:46:45,265][75949] Updated weights for policy 0, policy_version 9300 (0.0007) -[2023-10-14 13:46:45,643][75949] Updated weights for policy 0, policy_version 9310 (0.0008) -[2023-10-14 13:46:46,705][75950] Updated weights for policy 1, policy_version 9290 (0.0009) -[2023-10-14 13:46:47,077][75950] Updated weights for policy 1, policy_version 9300 (0.0009) -[2023-10-14 13:46:47,451][75950] Updated weights for policy 1, policy_version 9310 (0.0008) -[2023-10-14 13:46:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 19070976. Throughput: 0: 1654.4, 1: 1676.9. Samples: 4769412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:46:48,164][74987] Avg episode reward: [(0, '10.870'), (1, '17.170')] -[2023-10-14 13:46:49,830][75949] Updated weights for policy 0, policy_version 9320 (0.0008) -[2023-10-14 13:46:50,194][75949] Updated weights for policy 0, policy_version 9330 (0.0007) -[2023-10-14 13:46:50,572][75949] Updated weights for policy 0, policy_version 9340 (0.0008) -[2023-10-14 13:46:51,659][75950] Updated weights for policy 1, policy_version 9320 (0.0008) -[2023-10-14 13:46:52,025][75950] Updated weights for policy 1, policy_version 9330 (0.0010) -[2023-10-14 13:46:52,386][75950] Updated weights for policy 1, policy_version 9340 (0.0009) -[2023-10-14 13:46:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19136512. Throughput: 0: 1676.5, 1: 1667.9. Samples: 4789502. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) -[2023-10-14 13:46:53,165][74987] Avg episode reward: [(0, '11.400'), (1, '17.950')] -[2023-10-14 13:46:54,621][75949] Updated weights for policy 0, policy_version 9350 (0.0010) -[2023-10-14 13:46:54,990][75949] Updated weights for policy 0, policy_version 9360 (0.0009) -[2023-10-14 13:46:55,371][75949] Updated weights for policy 0, policy_version 9370 (0.0011) -[2023-10-14 13:46:56,553][75950] Updated weights for policy 1, policy_version 9350 (0.0010) -[2023-10-14 13:46:56,922][75950] Updated weights for policy 1, policy_version 9360 (0.0008) -[2023-10-14 13:46:57,293][75950] Updated weights for policy 1, policy_version 9370 (0.0007) -[2023-10-14 13:46:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 19202048. Throughput: 0: 1682.4, 1: 1660.1. Samples: 4809212. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) -[2023-10-14 13:46:58,164][74987] Avg episode reward: [(0, '10.550'), (1, '16.530')] -[2023-10-14 13:46:59,461][75949] Updated weights for policy 0, policy_version 9380 (0.0010) -[2023-10-14 13:46:59,847][75949] Updated weights for policy 0, policy_version 9390 (0.0009) -[2023-10-14 13:47:00,219][75949] Updated weights for policy 0, policy_version 9400 (0.0009) -[2023-10-14 13:47:01,180][75950] Updated weights for policy 1, policy_version 9380 (0.0008) -[2023-10-14 13:47:01,555][75950] Updated weights for policy 1, policy_version 9390 (0.0011) -[2023-10-14 13:47:01,911][75950] Updated weights for policy 1, policy_version 9400 (0.0007) -[2023-10-14 13:47:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19267584. Throughput: 0: 1663.6, 1: 1678.9. Samples: 4819646. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) -[2023-10-14 13:47:03,165][74987] Avg episode reward: [(0, '11.360'), (1, '16.530')] -[2023-10-14 13:47:04,223][75949] Updated weights for policy 0, policy_version 9410 (0.0010) -[2023-10-14 13:47:04,592][75949] Updated weights for policy 0, policy_version 9420 (0.0008) -[2023-10-14 13:47:04,962][75949] Updated weights for policy 0, policy_version 9430 (0.0008) -[2023-10-14 13:47:05,335][75949] Updated weights for policy 0, policy_version 9440 (0.0008) -[2023-10-14 13:47:06,155][75950] Updated weights for policy 1, policy_version 9410 (0.0009) -[2023-10-14 13:47:06,578][75950] Updated weights for policy 1, policy_version 9420 (0.0007) -[2023-10-14 13:47:06,946][75950] Updated weights for policy 1, policy_version 9430 (0.0007) -[2023-10-14 13:47:07,316][75950] Updated weights for policy 1, policy_version 9440 (0.0008) -[2023-10-14 13:47:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19333120. Throughput: 0: 1679.6, 1: 1666.7. Samples: 4839678. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) -[2023-10-14 13:47:08,164][74987] Avg episode reward: [(0, '10.030'), (1, '16.540')] -[2023-10-14 13:47:09,491][75949] Updated weights for policy 0, policy_version 9450 (0.0007) -[2023-10-14 13:47:09,865][75949] Updated weights for policy 0, policy_version 9460 (0.0008) -[2023-10-14 13:47:10,231][75949] Updated weights for policy 0, policy_version 9470 (0.0009) -[2023-10-14 13:47:11,336][75950] Updated weights for policy 1, policy_version 9450 (0.0010) -[2023-10-14 13:47:11,709][75950] Updated weights for policy 1, policy_version 9460 (0.0008) -[2023-10-14 13:47:12,071][75950] Updated weights for policy 1, policy_version 9470 (0.0010) -[2023-10-14 13:47:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 19398656. Throughput: 0: 1680.7, 1: 1669.5. Samples: 4859796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:47:13,165][74987] Avg episode reward: [(0, '10.270'), (1, '16.700')] -[2023-10-14 13:47:14,133][75949] Updated weights for policy 0, policy_version 9480 (0.0008) -[2023-10-14 13:47:14,503][75949] Updated weights for policy 0, policy_version 9490 (0.0009) -[2023-10-14 13:47:14,877][75949] Updated weights for policy 0, policy_version 9500 (0.0007) -[2023-10-14 13:47:16,070][75950] Updated weights for policy 1, policy_version 9480 (0.0009) -[2023-10-14 13:47:16,431][75950] Updated weights for policy 1, policy_version 9490 (0.0008) -[2023-10-14 13:47:16,800][75950] Updated weights for policy 1, policy_version 9500 (0.0008) -[2023-10-14 13:47:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 19464192. Throughput: 0: 1671.9, 1: 1678.0. Samples: 4870244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:47:18,164][74987] Avg episode reward: [(0, '11.370'), (1, '16.610')] -[2023-10-14 13:47:19,070][75949] Updated weights for policy 0, policy_version 9510 (0.0009) -[2023-10-14 13:47:19,449][75949] Updated weights for policy 0, policy_version 9520 (0.0010) -[2023-10-14 13:47:19,819][75949] Updated weights for policy 0, policy_version 9530 (0.0010) -[2023-10-14 13:47:20,834][75950] Updated weights for policy 1, policy_version 9510 (0.0009) -[2023-10-14 13:47:21,209][75950] Updated weights for policy 1, policy_version 9520 (0.0010) -[2023-10-14 13:47:21,567][75950] Updated weights for policy 1, policy_version 9530 (0.0008) -[2023-10-14 13:47:23,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 19529728. Throughput: 0: 1684.8, 1: 1671.6. Samples: 4890258. Policy #0 lag: (min: 9.0, avg: 17.6, max: 41.0) -[2023-10-14 13:47:23,164][74987] Avg episode reward: [(0, '10.480'), (1, '16.940')] -[2023-10-14 13:47:23,905][75949] Updated weights for policy 0, policy_version 9540 (0.0010) -[2023-10-14 13:47:24,273][75949] Updated weights for policy 0, policy_version 9550 (0.0008) -[2023-10-14 13:47:24,653][75949] Updated weights for policy 0, policy_version 9560 (0.0007) -[2023-10-14 13:47:25,499][75950] Updated weights for policy 1, policy_version 9540 (0.0010) -[2023-10-14 13:47:25,874][75950] Updated weights for policy 1, policy_version 9550 (0.0008) -[2023-10-14 13:47:26,234][75950] Updated weights for policy 1, policy_version 9560 (0.0009) -[2023-10-14 13:47:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 19595264. Throughput: 0: 1681.7, 1: 1694.6. Samples: 4910934. Policy #0 lag: (min: 9.0, avg: 17.6, max: 41.0) -[2023-10-14 13:47:28,165][74987] Avg episode reward: [(0, '11.060'), (1, '17.320')] -[2023-10-14 13:47:28,726][75949] Updated weights for policy 0, policy_version 9570 (0.0008) -[2023-10-14 13:47:29,105][75949] Updated weights for policy 0, policy_version 9580 (0.0009) -[2023-10-14 13:47:29,482][75949] Updated weights for policy 0, policy_version 9590 (0.0009) -[2023-10-14 13:47:29,854][75949] Updated weights for policy 0, policy_version 9600 (0.0008) -[2023-10-14 13:47:30,308][75950] Updated weights for policy 1, policy_version 9570 (0.0010) -[2023-10-14 13:47:30,674][75950] Updated weights for policy 1, policy_version 9580 (0.0007) -[2023-10-14 13:47:31,026][75950] Updated weights for policy 1, policy_version 9590 (0.0008) -[2023-10-14 13:47:31,394][75950] Updated weights for policy 1, policy_version 9600 (0.0008) -[2023-10-14 13:47:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 19660800. Throughput: 0: 1681.9, 1: 1687.5. Samples: 4921034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:47:33,165][74987] Avg episode reward: [(0, '11.200'), (1, '17.490')] -[2023-10-14 13:47:33,870][75949] Updated weights for policy 0, policy_version 9610 (0.0008) -[2023-10-14 13:47:34,246][75949] Updated weights for policy 0, policy_version 9620 (0.0009) -[2023-10-14 13:47:34,613][75949] Updated weights for policy 0, policy_version 9630 (0.0009) -[2023-10-14 13:47:35,316][75950] Updated weights for policy 1, policy_version 9610 (0.0008) -[2023-10-14 13:47:35,685][75950] Updated weights for policy 1, policy_version 9620 (0.0010) -[2023-10-14 13:47:36,044][75950] Updated weights for policy 1, policy_version 9630 (0.0010) -[2023-10-14 13:47:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 19726336. Throughput: 0: 1686.0, 1: 1676.3. Samples: 4940806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:47:38,164][74987] Avg episode reward: [(0, '10.670'), (1, '17.310')] -[2023-10-14 13:47:38,607][75949] Updated weights for policy 0, policy_version 9640 (0.0008) -[2023-10-14 13:47:38,976][75949] Updated weights for policy 0, policy_version 9650 (0.0007) -[2023-10-14 13:47:39,354][75949] Updated weights for policy 0, policy_version 9660 (0.0009) -[2023-10-14 13:47:40,038][75950] Updated weights for policy 1, policy_version 9640 (0.0010) -[2023-10-14 13:47:40,408][75950] Updated weights for policy 1, policy_version 9650 (0.0007) -[2023-10-14 13:47:40,782][75950] Updated weights for policy 1, policy_version 9660 (0.0008) -[2023-10-14 13:47:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 19791872. Throughput: 0: 1686.1, 1: 1697.9. Samples: 4961494. Policy #0 lag: (min: 25.0, avg: 42.5, max: 57.0) -[2023-10-14 13:47:43,165][74987] Avg episode reward: [(0, '10.430'), (1, '15.770')] -[2023-10-14 13:47:43,466][75949] Updated weights for policy 0, policy_version 9670 (0.0007) -[2023-10-14 13:47:43,842][75949] Updated weights for policy 0, policy_version 9680 (0.0007) -[2023-10-14 13:47:44,208][75949] Updated weights for policy 0, policy_version 9690 (0.0007) -[2023-10-14 13:47:44,870][75950] Updated weights for policy 1, policy_version 9670 (0.0009) -[2023-10-14 13:47:45,238][75950] Updated weights for policy 1, policy_version 9680 (0.0008) -[2023-10-14 13:47:45,608][75950] Updated weights for policy 1, policy_version 9690 (0.0010) -[2023-10-14 13:47:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 19857408. Throughput: 0: 1690.8, 1: 1671.3. Samples: 4970942. Policy #0 lag: (min: 25.0, avg: 42.5, max: 57.0) -[2023-10-14 13:47:48,164][74987] Avg episode reward: [(0, '11.110'), (1, '17.780')] -[2023-10-14 13:47:48,239][75949] Updated weights for policy 0, policy_version 9700 (0.0008) -[2023-10-14 13:47:48,617][75949] Updated weights for policy 0, policy_version 9710 (0.0010) -[2023-10-14 13:47:48,994][75949] Updated weights for policy 0, policy_version 9720 (0.0009) -[2023-10-14 13:47:49,751][75950] Updated weights for policy 1, policy_version 9700 (0.0008) -[2023-10-14 13:47:50,119][75950] Updated weights for policy 1, policy_version 9710 (0.0007) -[2023-10-14 13:47:50,495][75950] Updated weights for policy 1, policy_version 9720 (0.0011) -[2023-10-14 13:47:52,984][75949] Updated weights for policy 0, policy_version 9730 (0.0009) -[2023-10-14 13:47:53,163][74987] Fps is (10 sec: 13107.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 19922944. Throughput: 0: 1692.0, 1: 1678.9. Samples: 4991368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:47:53,164][74987] Avg episode reward: [(0, '10.860'), (1, '16.860')] -[2023-10-14 13:47:53,359][75949] Updated weights for policy 0, policy_version 9740 (0.0008) -[2023-10-14 13:47:53,730][75949] Updated weights for policy 0, policy_version 9750 (0.0009) -[2023-10-14 13:47:54,095][75949] Updated weights for policy 0, policy_version 9760 (0.0007) -[2023-10-14 13:47:54,635][75950] Updated weights for policy 1, policy_version 9730 (0.0009) -[2023-10-14 13:47:55,031][75950] Updated weights for policy 1, policy_version 9740 (0.0011) -[2023-10-14 13:47:55,399][75950] Updated weights for policy 1, policy_version 9750 (0.0008) -[2023-10-14 13:47:55,765][75950] Updated weights for policy 1, policy_version 9760 (0.0008) -[2023-10-14 13:47:58,152][75949] Updated weights for policy 0, policy_version 9770 (0.0009) -[2023-10-14 13:47:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 19988480. Throughput: 0: 1689.1, 1: 1695.9. Samples: 5012118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:47:58,164][74987] Avg episode reward: [(0, '10.650'), (1, '17.210')] -[2023-10-14 13:47:58,536][75949] Updated weights for policy 0, policy_version 9780 (0.0010) -[2023-10-14 13:47:58,909][75949] Updated weights for policy 0, policy_version 9790 (0.0010) -[2023-10-14 13:47:59,824][75950] Updated weights for policy 1, policy_version 9770 (0.0008) -[2023-10-14 13:48:00,190][75950] Updated weights for policy 1, policy_version 9780 (0.0007) -[2023-10-14 13:48:00,562][75950] Updated weights for policy 1, policy_version 9790 (0.0007) -[2023-10-14 13:48:03,030][75949] Updated weights for policy 0, policy_version 9800 (0.0010) -[2023-10-14 13:48:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 20054016. Throughput: 0: 1689.5, 1: 1667.7. Samples: 5021320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:48:03,165][74987] Avg episode reward: [(0, '10.930'), (1, '17.320')] -[2023-10-14 13:48:03,393][75949] Updated weights for policy 0, policy_version 9810 (0.0008) -[2023-10-14 13:48:03,766][75949] Updated weights for policy 0, policy_version 9820 (0.0008) -[2023-10-14 13:48:04,527][75950] Updated weights for policy 1, policy_version 9800 (0.0010) -[2023-10-14 13:48:04,897][75950] Updated weights for policy 1, policy_version 9810 (0.0009) -[2023-10-14 13:48:05,264][75950] Updated weights for policy 1, policy_version 9820 (0.0008) -[2023-10-14 13:48:07,997][75949] Updated weights for policy 0, policy_version 9830 (0.0011) -[2023-10-14 13:48:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 20119552. Throughput: 0: 1681.4, 1: 1683.7. Samples: 5041686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:48:08,164][74987] Avg episode reward: [(0, '10.790'), (1, '18.050')] -[2023-10-14 13:48:08,376][75949] Updated weights for policy 0, policy_version 9840 (0.0011) -[2023-10-14 13:48:08,758][75949] Updated weights for policy 0, policy_version 9850 (0.0010) -[2023-10-14 13:48:09,236][75950] Updated weights for policy 1, policy_version 9830 (0.0008) -[2023-10-14 13:48:09,597][75950] Updated weights for policy 1, policy_version 9840 (0.0009) -[2023-10-14 13:48:09,966][75950] Updated weights for policy 1, policy_version 9850 (0.0007) -[2023-10-14 13:48:12,766][75949] Updated weights for policy 0, policy_version 9860 (0.0007) -[2023-10-14 13:48:13,133][75949] Updated weights for policy 0, policy_version 9870 (0.0008) -[2023-10-14 13:48:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 20185088. Throughput: 0: 1675.6, 1: 1682.2. Samples: 5062034. Policy #0 lag: (min: 3.0, avg: 4.8, max: 32.0) -[2023-10-14 13:48:13,164][74987] Avg episode reward: [(0, '10.590'), (1, '16.320')] -[2023-10-14 13:48:13,514][75949] Updated weights for policy 0, policy_version 9880 (0.0009) -[2023-10-14 13:48:14,029][75950] Updated weights for policy 1, policy_version 9860 (0.0009) -[2023-10-14 13:48:14,396][75950] Updated weights for policy 1, policy_version 9870 (0.0008) -[2023-10-14 13:48:14,768][75950] Updated weights for policy 1, policy_version 9880 (0.0012) -[2023-10-14 13:48:17,526][75949] Updated weights for policy 0, policy_version 9890 (0.0009) -[2023-10-14 13:48:17,895][75949] Updated weights for policy 0, policy_version 9900 (0.0009) -[2023-10-14 13:48:18,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 20250624. Throughput: 0: 1676.4, 1: 1666.3. Samples: 5071456. Policy #0 lag: (min: 3.0, avg: 4.8, max: 32.0) -[2023-10-14 13:48:18,164][74987] Avg episode reward: [(0, '10.910'), (1, '17.460')] -[2023-10-14 13:48:18,263][75949] Updated weights for policy 0, policy_version 9910 (0.0009) -[2023-10-14 13:48:18,634][75949] Updated weights for policy 0, policy_version 9920 (0.0011) -[2023-10-14 13:48:18,767][75950] Updated weights for policy 1, policy_version 9890 (0.0008) -[2023-10-14 13:48:19,143][75950] Updated weights for policy 1, policy_version 9900 (0.0008) -[2023-10-14 13:48:19,505][75950] Updated weights for policy 1, policy_version 9910 (0.0008) -[2023-10-14 13:48:19,871][75950] Updated weights for policy 1, policy_version 9920 (0.0008) -[2023-10-14 13:48:22,825][75949] Updated weights for policy 0, policy_version 9930 (0.0007) -[2023-10-14 13:48:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 20316160. Throughput: 0: 1676.8, 1: 1688.8. Samples: 5092256. Policy #0 lag: (min: 24.0, avg: 35.6, max: 56.0) -[2023-10-14 13:48:23,164][74987] Avg episode reward: [(0, '10.850'), (1, '17.700')] -[2023-10-14 13:48:23,187][75949] Updated weights for policy 0, policy_version 9940 (0.0008) -[2023-10-14 13:48:23,564][75949] Updated weights for policy 0, policy_version 9950 (0.0010) -[2023-10-14 13:48:23,949][75950] Updated weights for policy 1, policy_version 9930 (0.0010) -[2023-10-14 13:48:24,304][75950] Updated weights for policy 1, policy_version 9940 (0.0010) -[2023-10-14 13:48:24,674][75950] Updated weights for policy 1, policy_version 9950 (0.0009) -[2023-10-14 13:48:27,594][75949] Updated weights for policy 0, policy_version 9960 (0.0007) -[2023-10-14 13:48:27,957][75949] Updated weights for policy 0, policy_version 9970 (0.0007) -[2023-10-14 13:48:28,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 20381696. Throughput: 0: 1662.9, 1: 1689.5. Samples: 5112352. Policy #0 lag: (min: 24.0, avg: 35.6, max: 56.0) -[2023-10-14 13:48:28,165][74987] Avg episode reward: [(0, '11.020'), (1, '16.440')] -[2023-10-14 13:48:28,320][75949] Updated weights for policy 0, policy_version 9980 (0.0007) -[2023-10-14 13:48:28,688][75950] Updated weights for policy 1, policy_version 9960 (0.0008) -[2023-10-14 13:48:29,052][75950] Updated weights for policy 1, policy_version 9970 (0.0008) -[2023-10-14 13:48:29,427][75950] Updated weights for policy 1, policy_version 9980 (0.0007) -[2023-10-14 13:48:32,385][75949] Updated weights for policy 0, policy_version 9990 (0.0007) -[2023-10-14 13:48:32,753][75949] Updated weights for policy 0, policy_version 10000 (0.0008) -[2023-10-14 13:48:33,123][75949] Updated weights for policy 0, policy_version 10010 (0.0010) -[2023-10-14 13:48:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 20447232. Throughput: 0: 1670.1, 1: 1685.5. Samples: 5121946. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 13:48:33,165][74987] Avg episode reward: [(0, '10.850'), (1, '17.100')] -[2023-10-14 13:48:33,601][75950] Updated weights for policy 1, policy_version 9990 (0.0009) -[2023-10-14 13:48:33,974][75950] Updated weights for policy 1, policy_version 10000 (0.0009) -[2023-10-14 13:48:34,344][75950] Updated weights for policy 1, policy_version 10010 (0.0007) -[2023-10-14 13:48:37,215][75949] Updated weights for policy 0, policy_version 10020 (0.0008) -[2023-10-14 13:48:37,607][75949] Updated weights for policy 0, policy_version 10030 (0.0009) -[2023-10-14 13:48:37,981][75949] Updated weights for policy 0, policy_version 10040 (0.0011) -[2023-10-14 13:48:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 20512768. Throughput: 0: 1674.2, 1: 1692.4. Samples: 5142864. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 13:48:38,165][74987] Avg episode reward: [(0, '10.490'), (1, '18.070')] -[2023-10-14 13:48:38,437][75950] Updated weights for policy 1, policy_version 10020 (0.0010) -[2023-10-14 13:48:38,806][75950] Updated weights for policy 1, policy_version 10030 (0.0009) -[2023-10-14 13:48:39,171][75950] Updated weights for policy 1, policy_version 10040 (0.0007) -[2023-10-14 13:48:42,076][75949] Updated weights for policy 0, policy_version 10050 (0.0009) -[2023-10-14 13:48:42,447][75949] Updated weights for policy 0, policy_version 10060 (0.0009) -[2023-10-14 13:48:42,821][75949] Updated weights for policy 0, policy_version 10070 (0.0007) -[2023-10-14 13:48:43,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 20578304. Throughput: 0: 1654.9, 1: 1687.4. Samples: 5162522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:48:43,164][74987] Avg episode reward: [(0, '10.920'), (1, '19.110')] -[2023-10-14 13:48:43,184][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000010080_10321920.pth... -[2023-10-14 13:48:43,184][75949] Updated weights for policy 0, policy_version 10080 (0.0010) -[2023-10-14 13:48:43,225][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000008512_8716288.pth -[2023-10-14 13:48:43,323][75950] Updated weights for policy 1, policy_version 10050 (0.0007) -[2023-10-14 13:48:43,738][75950] Updated weights for policy 1, policy_version 10060 (0.0009) -[2023-10-14 13:48:44,105][75950] Updated weights for policy 1, policy_version 10070 (0.0009) -[2023-10-14 13:48:44,478][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000010080_10321920.pth... -[2023-10-14 13:48:44,482][75950] Updated weights for policy 1, policy_version 10080 (0.0009) -[2023-10-14 13:48:44,517][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000008480_8683520.pth -[2023-10-14 13:48:44,520][75801] Saving new best policy, reward=19.110! -[2023-10-14 13:48:47,320][75949] Updated weights for policy 0, policy_version 10090 (0.0007) -[2023-10-14 13:48:47,681][75949] Updated weights for policy 0, policy_version 10100 (0.0007) -[2023-10-14 13:48:48,052][75949] Updated weights for policy 0, policy_version 10110 (0.0008) -[2023-10-14 13:48:48,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 20676608. Throughput: 0: 1672.4, 1: 1680.6. Samples: 5172206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:48:48,164][74987] Avg episode reward: [(0, '10.980'), (1, '17.830')] -[2023-10-14 13:48:48,587][75950] Updated weights for policy 1, policy_version 10090 (0.0009) -[2023-10-14 13:48:48,950][75950] Updated weights for policy 1, policy_version 10100 (0.0009) -[2023-10-14 13:48:49,321][75950] Updated weights for policy 1, policy_version 10110 (0.0009) -[2023-10-14 13:48:52,011][75949] Updated weights for policy 0, policy_version 10120 (0.0007) -[2023-10-14 13:48:52,395][75949] Updated weights for policy 0, policy_version 10130 (0.0009) -[2023-10-14 13:48:52,761][75949] Updated weights for policy 0, policy_version 10140 (0.0007) -[2023-10-14 13:48:53,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 20742144. Throughput: 0: 1678.7, 1: 1681.2. Samples: 5192882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:48:53,164][74987] Avg episode reward: [(0, '10.660'), (1, '18.700')] -[2023-10-14 13:48:53,336][75950] Updated weights for policy 1, policy_version 10120 (0.0008) -[2023-10-14 13:48:53,701][75950] Updated weights for policy 1, policy_version 10130 (0.0008) -[2023-10-14 13:48:54,076][75950] Updated weights for policy 1, policy_version 10140 (0.0010) -[2023-10-14 13:48:56,940][75949] Updated weights for policy 0, policy_version 10150 (0.0007) -[2023-10-14 13:48:57,316][75949] Updated weights for policy 0, policy_version 10160 (0.0007) -[2023-10-14 13:48:57,688][75949] Updated weights for policy 0, policy_version 10170 (0.0010) -[2023-10-14 13:48:58,129][75950] Updated weights for policy 1, policy_version 10150 (0.0008) -[2023-10-14 13:48:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 20807680. Throughput: 0: 1662.3, 1: 1683.6. Samples: 5212598. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 13:48:58,164][74987] Avg episode reward: [(0, '11.600'), (1, '15.690')] -[2023-10-14 13:48:58,171][75615] Saving new best policy, reward=11.600! -[2023-10-14 13:48:58,497][75950] Updated weights for policy 1, policy_version 10160 (0.0008) -[2023-10-14 13:48:58,863][75950] Updated weights for policy 1, policy_version 10170 (0.0007) -[2023-10-14 13:49:01,703][75949] Updated weights for policy 0, policy_version 10180 (0.0009) -[2023-10-14 13:49:02,079][75949] Updated weights for policy 0, policy_version 10190 (0.0009) -[2023-10-14 13:49:02,450][75949] Updated weights for policy 0, policy_version 10200 (0.0009) -[2023-10-14 13:49:02,806][75950] Updated weights for policy 1, policy_version 10180 (0.0008) -[2023-10-14 13:49:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 20873216. Throughput: 0: 1678.0, 1: 1681.3. Samples: 5222626. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 13:49:03,164][74987] Avg episode reward: [(0, '10.730'), (1, '17.900')] -[2023-10-14 13:49:03,177][75950] Updated weights for policy 1, policy_version 10190 (0.0010) -[2023-10-14 13:49:03,545][75950] Updated weights for policy 1, policy_version 10200 (0.0010) -[2023-10-14 13:49:06,369][75949] Updated weights for policy 0, policy_version 10210 (0.0008) -[2023-10-14 13:49:06,747][75949] Updated weights for policy 0, policy_version 10220 (0.0010) -[2023-10-14 13:49:07,117][75949] Updated weights for policy 0, policy_version 10230 (0.0008) -[2023-10-14 13:49:07,482][75949] Updated weights for policy 0, policy_version 10240 (0.0008) -[2023-10-14 13:49:07,720][75950] Updated weights for policy 1, policy_version 10210 (0.0010) -[2023-10-14 13:49:08,099][75950] Updated weights for policy 1, policy_version 10220 (0.0008) -[2023-10-14 13:49:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 20938752. Throughput: 0: 1673.0, 1: 1676.6. Samples: 5242986. Policy #0 lag: (min: 5.0, avg: 6.6, max: 31.0) -[2023-10-14 13:49:08,165][74987] Avg episode reward: [(0, '11.040'), (1, '17.400')] -[2023-10-14 13:49:08,468][75950] Updated weights for policy 1, policy_version 10230 (0.0009) -[2023-10-14 13:49:08,836][75950] Updated weights for policy 1, policy_version 10240 (0.0009) -[2023-10-14 13:49:11,723][75949] Updated weights for policy 0, policy_version 10250 (0.0008) -[2023-10-14 13:49:12,097][75949] Updated weights for policy 0, policy_version 10260 (0.0010) -[2023-10-14 13:49:12,475][75949] Updated weights for policy 0, policy_version 10270 (0.0008) -[2023-10-14 13:49:12,854][75950] Updated weights for policy 1, policy_version 10250 (0.0008) -[2023-10-14 13:49:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21004288. Throughput: 0: 1658.9, 1: 1676.1. Samples: 5262426. Policy #0 lag: (min: 5.0, avg: 6.6, max: 31.0) -[2023-10-14 13:49:13,164][74987] Avg episode reward: [(0, '11.230'), (1, '18.520')] -[2023-10-14 13:49:13,229][75950] Updated weights for policy 1, policy_version 10260 (0.0009) -[2023-10-14 13:49:13,596][75950] Updated weights for policy 1, policy_version 10270 (0.0007) -[2023-10-14 13:49:16,557][75949] Updated weights for policy 0, policy_version 10280 (0.0008) -[2023-10-14 13:49:16,917][75949] Updated weights for policy 0, policy_version 10290 (0.0009) -[2023-10-14 13:49:17,297][75949] Updated weights for policy 0, policy_version 10300 (0.0010) -[2023-10-14 13:49:17,926][75950] Updated weights for policy 1, policy_version 10280 (0.0010) -[2023-10-14 13:49:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21069824. Throughput: 0: 1678.1, 1: 1676.0. Samples: 5272878. Policy #0 lag: (min: 19.0, avg: 20.8, max: 47.0) -[2023-10-14 13:49:18,165][74987] Avg episode reward: [(0, '10.940'), (1, '18.210')] -[2023-10-14 13:49:18,301][75950] Updated weights for policy 1, policy_version 10290 (0.0011) -[2023-10-14 13:49:18,671][75950] Updated weights for policy 1, policy_version 10300 (0.0008) -[2023-10-14 13:49:21,474][75949] Updated weights for policy 0, policy_version 10310 (0.0009) -[2023-10-14 13:49:21,847][75949] Updated weights for policy 0, policy_version 10320 (0.0010) -[2023-10-14 13:49:22,221][75949] Updated weights for policy 0, policy_version 10330 (0.0008) -[2023-10-14 13:49:22,765][75950] Updated weights for policy 1, policy_version 10310 (0.0008) -[2023-10-14 13:49:23,131][75950] Updated weights for policy 1, policy_version 10320 (0.0007) -[2023-10-14 13:49:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21135360. Throughput: 0: 1660.4, 1: 1675.2. Samples: 5292970. Policy #0 lag: (min: 19.0, avg: 20.8, max: 47.0) -[2023-10-14 13:49:23,165][74987] Avg episode reward: [(0, '11.250'), (1, '17.700')] -[2023-10-14 13:49:23,507][75950] Updated weights for policy 1, policy_version 10330 (0.0010) -[2023-10-14 13:49:26,336][75949] Updated weights for policy 0, policy_version 10340 (0.0010) -[2023-10-14 13:49:26,702][75949] Updated weights for policy 0, policy_version 10350 (0.0010) -[2023-10-14 13:49:27,072][75949] Updated weights for policy 0, policy_version 10360 (0.0010) -[2023-10-14 13:49:27,531][75950] Updated weights for policy 1, policy_version 10340 (0.0010) -[2023-10-14 13:49:27,899][75950] Updated weights for policy 1, policy_version 10350 (0.0010) -[2023-10-14 13:49:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21200896. Throughput: 0: 1656.9, 1: 1677.9. Samples: 5312588. Policy #0 lag: (min: 22.0, avg: 31.5, max: 54.0) -[2023-10-14 13:49:28,165][74987] Avg episode reward: [(0, '11.220'), (1, '17.970')] -[2023-10-14 13:49:28,264][75950] Updated weights for policy 1, policy_version 10360 (0.0007) -[2023-10-14 13:49:31,092][75949] Updated weights for policy 0, policy_version 10370 (0.0010) -[2023-10-14 13:49:31,460][75949] Updated weights for policy 0, policy_version 10380 (0.0010) -[2023-10-14 13:49:31,825][75949] Updated weights for policy 0, policy_version 10390 (0.0007) -[2023-10-14 13:49:32,198][75949] Updated weights for policy 0, policy_version 10400 (0.0008) -[2023-10-14 13:49:32,345][75950] Updated weights for policy 1, policy_version 10370 (0.0007) -[2023-10-14 13:49:32,755][75950] Updated weights for policy 1, policy_version 10380 (0.0007) -[2023-10-14 13:49:33,121][75950] Updated weights for policy 1, policy_version 10390 (0.0009) -[2023-10-14 13:49:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 21266432. Throughput: 0: 1672.4, 1: 1686.6. Samples: 5323358. Policy #0 lag: (min: 22.0, avg: 31.5, max: 54.0) -[2023-10-14 13:49:33,164][74987] Avg episode reward: [(0, '11.210'), (1, '17.140')] -[2023-10-14 13:49:33,491][75950] Updated weights for policy 1, policy_version 10400 (0.0008) -[2023-10-14 13:49:36,177][75949] Updated weights for policy 0, policy_version 10410 (0.0009) -[2023-10-14 13:49:36,550][75949] Updated weights for policy 0, policy_version 10420 (0.0009) -[2023-10-14 13:49:36,920][75949] Updated weights for policy 0, policy_version 10430 (0.0010) -[2023-10-14 13:49:37,623][75950] Updated weights for policy 1, policy_version 10410 (0.0007) -[2023-10-14 13:49:37,991][75950] Updated weights for policy 1, policy_version 10420 (0.0007) -[2023-10-14 13:49:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 21331968. Throughput: 0: 1655.1, 1: 1679.5. Samples: 5342940. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) -[2023-10-14 13:49:38,165][74987] Avg episode reward: [(0, '11.210'), (1, '18.610')] -[2023-10-14 13:49:38,363][75950] Updated weights for policy 1, policy_version 10430 (0.0010) -[2023-10-14 13:49:40,984][75949] Updated weights for policy 0, policy_version 10440 (0.0010) -[2023-10-14 13:49:41,362][75949] Updated weights for policy 0, policy_version 10450 (0.0010) -[2023-10-14 13:49:41,730][75949] Updated weights for policy 0, policy_version 10460 (0.0008) -[2023-10-14 13:49:42,482][75950] Updated weights for policy 1, policy_version 10440 (0.0008) -[2023-10-14 13:49:42,850][75950] Updated weights for policy 1, policy_version 10450 (0.0008) -[2023-10-14 13:49:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 21397504. Throughput: 0: 1672.0, 1: 1664.8. Samples: 5362754. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) -[2023-10-14 13:49:43,164][74987] Avg episode reward: [(0, '10.980'), (1, '18.330')] -[2023-10-14 13:49:43,216][75950] Updated weights for policy 1, policy_version 10460 (0.0010) -[2023-10-14 13:49:45,927][75949] Updated weights for policy 0, policy_version 10470 (0.0009) -[2023-10-14 13:49:46,310][75949] Updated weights for policy 0, policy_version 10480 (0.0010) -[2023-10-14 13:49:46,666][75949] Updated weights for policy 0, policy_version 10490 (0.0009) -[2023-10-14 13:49:47,423][75950] Updated weights for policy 1, policy_version 10470 (0.0009) -[2023-10-14 13:49:47,790][75950] Updated weights for policy 1, policy_version 10480 (0.0009) -[2023-10-14 13:49:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 21463040. Throughput: 0: 1680.3, 1: 1669.3. Samples: 5373358. Policy #0 lag: (min: 28.0, avg: 42.9, max: 60.0) -[2023-10-14 13:49:48,165][74987] Avg episode reward: [(0, '10.990'), (1, '18.260')] -[2023-10-14 13:49:48,167][75950] Updated weights for policy 1, policy_version 10490 (0.0008) -[2023-10-14 13:49:50,713][75949] Updated weights for policy 0, policy_version 10500 (0.0008) -[2023-10-14 13:49:51,088][75949] Updated weights for policy 0, policy_version 10510 (0.0009) -[2023-10-14 13:49:51,448][75949] Updated weights for policy 0, policy_version 10520 (0.0007) -[2023-10-14 13:49:52,214][75950] Updated weights for policy 1, policy_version 10500 (0.0008) -[2023-10-14 13:49:52,583][75950] Updated weights for policy 1, policy_version 10510 (0.0007) -[2023-10-14 13:49:52,952][75950] Updated weights for policy 1, policy_version 10520 (0.0007) -[2023-10-14 13:49:53,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 21528576. Throughput: 0: 1656.2, 1: 1672.8. Samples: 5392788. Policy #0 lag: (min: 28.0, avg: 42.9, max: 60.0) -[2023-10-14 13:49:53,164][74987] Avg episode reward: [(0, '11.180'), (1, '18.520')] -[2023-10-14 13:49:55,450][75949] Updated weights for policy 0, policy_version 10530 (0.0007) -[2023-10-14 13:49:55,823][75949] Updated weights for policy 0, policy_version 10540 (0.0008) -[2023-10-14 13:49:56,195][75949] Updated weights for policy 0, policy_version 10550 (0.0008) -[2023-10-14 13:49:56,572][75949] Updated weights for policy 0, policy_version 10560 (0.0008) -[2023-10-14 13:49:56,901][75950] Updated weights for policy 1, policy_version 10530 (0.0008) -[2023-10-14 13:49:57,273][75950] Updated weights for policy 1, policy_version 10540 (0.0008) -[2023-10-14 13:49:57,639][75950] Updated weights for policy 1, policy_version 10550 (0.0008) -[2023-10-14 13:49:58,020][75950] Updated weights for policy 1, policy_version 10560 (0.0007) -[2023-10-14 13:49:58,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21626880. Throughput: 0: 1678.0, 1: 1656.3. Samples: 5412468. Policy #0 lag: (min: 25.0, avg: 45.2, max: 57.0) -[2023-10-14 13:49:58,165][74987] Avg episode reward: [(0, '11.020'), (1, '15.920')] -[2023-10-14 13:50:00,809][75949] Updated weights for policy 0, policy_version 10570 (0.0007) -[2023-10-14 13:50:01,195][75949] Updated weights for policy 0, policy_version 10580 (0.0009) -[2023-10-14 13:50:01,557][75949] Updated weights for policy 0, policy_version 10590 (0.0010) -[2023-10-14 13:50:01,998][75950] Updated weights for policy 1, policy_version 10570 (0.0008) -[2023-10-14 13:50:02,372][75950] Updated weights for policy 1, policy_version 10580 (0.0009) -[2023-10-14 13:50:02,733][75950] Updated weights for policy 1, policy_version 10590 (0.0008) -[2023-10-14 13:50:03,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21692416. Throughput: 0: 1667.3, 1: 1676.2. Samples: 5423336. Policy #0 lag: (min: 25.0, avg: 45.2, max: 57.0) -[2023-10-14 13:50:03,165][74987] Avg episode reward: [(0, '10.980'), (1, '18.480')] -[2023-10-14 13:50:05,756][75949] Updated weights for policy 0, policy_version 10600 (0.0008) -[2023-10-14 13:50:06,129][75949] Updated weights for policy 0, policy_version 10610 (0.0007) -[2023-10-14 13:50:06,508][75949] Updated weights for policy 0, policy_version 10620 (0.0010) -[2023-10-14 13:50:06,612][75950] Updated weights for policy 1, policy_version 10600 (0.0008) -[2023-10-14 13:50:06,991][75950] Updated weights for policy 1, policy_version 10610 (0.0007) -[2023-10-14 13:50:07,358][75950] Updated weights for policy 1, policy_version 10620 (0.0009) -[2023-10-14 13:50:08,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 21757952. Throughput: 0: 1658.4, 1: 1670.8. Samples: 5442786. Policy #0 lag: (min: 30.0, avg: 31.8, max: 60.0) -[2023-10-14 13:50:08,164][74987] Avg episode reward: [(0, '10.940'), (1, '17.100')] -[2023-10-14 13:50:10,507][75949] Updated weights for policy 0, policy_version 10630 (0.0010) -[2023-10-14 13:50:10,887][75949] Updated weights for policy 0, policy_version 10640 (0.0009) -[2023-10-14 13:50:11,252][75949] Updated weights for policy 0, policy_version 10650 (0.0008) -[2023-10-14 13:50:11,733][75950] Updated weights for policy 1, policy_version 10630 (0.0010) -[2023-10-14 13:50:12,111][75950] Updated weights for policy 1, policy_version 10640 (0.0009) -[2023-10-14 13:50:12,477][75950] Updated weights for policy 1, policy_version 10650 (0.0008) -[2023-10-14 13:50:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21823488. Throughput: 0: 1678.3, 1: 1649.2. Samples: 5462322. Policy #0 lag: (min: 30.0, avg: 31.8, max: 60.0) -[2023-10-14 13:50:13,164][74987] Avg episode reward: [(0, '10.730'), (1, '19.680')] -[2023-10-14 13:50:13,172][75801] Saving new best policy, reward=19.680! -[2023-10-14 13:50:15,181][75949] Updated weights for policy 0, policy_version 10660 (0.0010) -[2023-10-14 13:50:15,540][75949] Updated weights for policy 0, policy_version 10670 (0.0010) -[2023-10-14 13:50:15,911][75949] Updated weights for policy 0, policy_version 10680 (0.0007) -[2023-10-14 13:50:16,754][75950] Updated weights for policy 1, policy_version 10660 (0.0008) -[2023-10-14 13:50:17,116][75950] Updated weights for policy 1, policy_version 10670 (0.0009) -[2023-10-14 13:50:17,480][75950] Updated weights for policy 1, policy_version 10680 (0.0009) -[2023-10-14 13:50:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 21889024. Throughput: 0: 1661.2, 1: 1666.1. Samples: 5473088. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) -[2023-10-14 13:50:18,164][74987] Avg episode reward: [(0, '10.650'), (1, '15.690')] -[2023-10-14 13:50:20,172][75949] Updated weights for policy 0, policy_version 10690 (0.0007) -[2023-10-14 13:50:20,543][75949] Updated weights for policy 0, policy_version 10700 (0.0009) -[2023-10-14 13:50:20,923][75949] Updated weights for policy 0, policy_version 10710 (0.0008) -[2023-10-14 13:50:21,293][75949] Updated weights for policy 0, policy_version 10720 (0.0010) -[2023-10-14 13:50:21,722][75950] Updated weights for policy 1, policy_version 10690 (0.0009) -[2023-10-14 13:50:22,146][75950] Updated weights for policy 1, policy_version 10700 (0.0008) -[2023-10-14 13:50:22,504][75950] Updated weights for policy 1, policy_version 10710 (0.0007) -[2023-10-14 13:50:22,874][75950] Updated weights for policy 1, policy_version 10720 (0.0008) -[2023-10-14 13:50:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21954560. Throughput: 0: 1661.9, 1: 1669.9. Samples: 5492872. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) -[2023-10-14 13:50:23,164][74987] Avg episode reward: [(0, '11.160'), (1, '18.780')] -[2023-10-14 13:50:25,280][75949] Updated weights for policy 0, policy_version 10730 (0.0010) -[2023-10-14 13:50:25,656][75949] Updated weights for policy 0, policy_version 10740 (0.0010) -[2023-10-14 13:50:26,020][75949] Updated weights for policy 0, policy_version 10750 (0.0010) -[2023-10-14 13:50:26,719][75950] Updated weights for policy 1, policy_version 10730 (0.0008) -[2023-10-14 13:50:27,087][75950] Updated weights for policy 1, policy_version 10740 (0.0009) -[2023-10-14 13:50:27,452][75950] Updated weights for policy 1, policy_version 10750 (0.0008) -[2023-10-14 13:50:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 22020096. Throughput: 0: 1667.5, 1: 1656.1. Samples: 5512316. Policy #0 lag: (min: 2.0, avg: 2.3, max: 14.0) -[2023-10-14 13:50:28,164][74987] Avg episode reward: [(0, '10.760'), (1, '15.480')] -[2023-10-14 13:50:30,000][75949] Updated weights for policy 0, policy_version 10760 (0.0009) -[2023-10-14 13:50:30,365][75949] Updated weights for policy 0, policy_version 10770 (0.0010) -[2023-10-14 13:50:30,748][75949] Updated weights for policy 0, policy_version 10780 (0.0011) -[2023-10-14 13:50:31,461][75950] Updated weights for policy 1, policy_version 10760 (0.0009) -[2023-10-14 13:50:31,823][75950] Updated weights for policy 1, policy_version 10770 (0.0007) -[2023-10-14 13:50:32,196][75950] Updated weights for policy 1, policy_version 10780 (0.0007) -[2023-10-14 13:50:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 22085632. Throughput: 0: 1648.6, 1: 1676.8. Samples: 5523002. Policy #0 lag: (min: 2.0, avg: 2.3, max: 14.0) -[2023-10-14 13:50:33,165][74987] Avg episode reward: [(0, '10.570'), (1, '17.580')] -[2023-10-14 13:50:35,171][75949] Updated weights for policy 0, policy_version 10790 (0.0009) -[2023-10-14 13:50:35,553][75949] Updated weights for policy 0, policy_version 10800 (0.0010) -[2023-10-14 13:50:35,928][75949] Updated weights for policy 0, policy_version 10810 (0.0009) -[2023-10-14 13:50:36,220][75950] Updated weights for policy 1, policy_version 10790 (0.0008) -[2023-10-14 13:50:36,589][75950] Updated weights for policy 1, policy_version 10800 (0.0010) -[2023-10-14 13:50:36,956][75950] Updated weights for policy 1, policy_version 10810 (0.0011) -[2023-10-14 13:50:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 22151168. Throughput: 0: 1666.4, 1: 1662.6. Samples: 5542592. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 13:50:38,164][74987] Avg episode reward: [(0, '11.420'), (1, '15.740')] -[2023-10-14 13:50:39,874][75949] Updated weights for policy 0, policy_version 10820 (0.0010) -[2023-10-14 13:50:40,249][75949] Updated weights for policy 0, policy_version 10830 (0.0010) -[2023-10-14 13:50:40,616][75949] Updated weights for policy 0, policy_version 10840 (0.0009) -[2023-10-14 13:50:41,100][75950] Updated weights for policy 1, policy_version 10820 (0.0008) -[2023-10-14 13:50:41,465][75950] Updated weights for policy 1, policy_version 10830 (0.0010) -[2023-10-14 13:50:41,830][75950] Updated weights for policy 1, policy_version 10840 (0.0009) -[2023-10-14 13:50:43,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 22216704. Throughput: 0: 1676.0, 1: 1666.5. Samples: 5562878. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 13:50:43,164][74987] Avg episode reward: [(0, '10.830'), (1, '17.210')] -[2023-10-14 13:50:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000010848_11108352.pth... -[2023-10-14 13:50:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000010848_11108352.pth... -[2023-10-14 13:50:43,201][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000009280_9502720.pth -[2023-10-14 13:50:43,207][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000009280_9502720.pth -[2023-10-14 13:50:44,659][75949] Updated weights for policy 0, policy_version 10850 (0.0009) -[2023-10-14 13:50:45,035][75949] Updated weights for policy 0, policy_version 10860 (0.0009) -[2023-10-14 13:50:45,407][75949] Updated weights for policy 0, policy_version 10870 (0.0007) -[2023-10-14 13:50:45,775][75949] Updated weights for policy 0, policy_version 10880 (0.0007) -[2023-10-14 13:50:45,984][75950] Updated weights for policy 1, policy_version 10850 (0.0007) -[2023-10-14 13:50:46,351][75950] Updated weights for policy 1, policy_version 10860 (0.0008) -[2023-10-14 13:50:46,713][75950] Updated weights for policy 1, policy_version 10870 (0.0008) -[2023-10-14 13:50:47,085][75950] Updated weights for policy 1, policy_version 10880 (0.0008) -[2023-10-14 13:50:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 22282240. Throughput: 0: 1659.4, 1: 1669.0. Samples: 5573114. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) -[2023-10-14 13:50:48,164][74987] Avg episode reward: [(0, '9.900'), (1, '17.120')] -[2023-10-14 13:50:49,961][75949] Updated weights for policy 0, policy_version 10890 (0.0010) -[2023-10-14 13:50:50,337][75949] Updated weights for policy 0, policy_version 10900 (0.0010) -[2023-10-14 13:50:50,705][75949] Updated weights for policy 0, policy_version 10910 (0.0008) -[2023-10-14 13:50:51,314][75950] Updated weights for policy 1, policy_version 10890 (0.0009) -[2023-10-14 13:50:51,674][75950] Updated weights for policy 1, policy_version 10900 (0.0009) -[2023-10-14 13:50:52,048][75950] Updated weights for policy 1, policy_version 10910 (0.0007) -[2023-10-14 13:50:53,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 22347776. Throughput: 0: 1673.8, 1: 1655.0. Samples: 5592580. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) -[2023-10-14 13:50:53,164][74987] Avg episode reward: [(0, '11.530'), (1, '17.140')] -[2023-10-14 13:50:54,613][75949] Updated weights for policy 0, policy_version 10920 (0.0007) -[2023-10-14 13:50:54,983][75949] Updated weights for policy 0, policy_version 10930 (0.0011) -[2023-10-14 13:50:55,354][75949] Updated weights for policy 0, policy_version 10940 (0.0011) -[2023-10-14 13:50:55,986][75950] Updated weights for policy 1, policy_version 10920 (0.0009) -[2023-10-14 13:50:56,356][75950] Updated weights for policy 1, policy_version 10930 (0.0008) -[2023-10-14 13:50:56,729][75950] Updated weights for policy 1, policy_version 10940 (0.0008) -[2023-10-14 13:50:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22413312. Throughput: 0: 1674.4, 1: 1670.1. Samples: 5612828. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) -[2023-10-14 13:50:58,165][74987] Avg episode reward: [(0, '10.120'), (1, '17.690')] -[2023-10-14 13:50:59,582][75949] Updated weights for policy 0, policy_version 10950 (0.0010) -[2023-10-14 13:50:59,958][75949] Updated weights for policy 0, policy_version 10960 (0.0007) -[2023-10-14 13:51:00,331][75949] Updated weights for policy 0, policy_version 10970 (0.0007) -[2023-10-14 13:51:00,792][75950] Updated weights for policy 1, policy_version 10950 (0.0008) -[2023-10-14 13:51:01,156][75950] Updated weights for policy 1, policy_version 10960 (0.0009) -[2023-10-14 13:51:01,530][75950] Updated weights for policy 1, policy_version 10970 (0.0008) -[2023-10-14 13:51:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22478848. Throughput: 0: 1657.5, 1: 1677.3. Samples: 5623156. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) -[2023-10-14 13:51:03,165][74987] Avg episode reward: [(0, '10.300'), (1, '18.410')] -[2023-10-14 13:51:04,591][75949] Updated weights for policy 0, policy_version 10980 (0.0009) -[2023-10-14 13:51:04,959][75949] Updated weights for policy 0, policy_version 10990 (0.0009) -[2023-10-14 13:51:05,325][75949] Updated weights for policy 0, policy_version 11000 (0.0007) -[2023-10-14 13:51:05,632][75950] Updated weights for policy 1, policy_version 10980 (0.0010) -[2023-10-14 13:51:06,006][75950] Updated weights for policy 1, policy_version 10990 (0.0008) -[2023-10-14 13:51:06,384][75950] Updated weights for policy 1, policy_version 11000 (0.0008) -[2023-10-14 13:51:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 22544384. Throughput: 0: 1678.3, 1: 1654.8. Samples: 5642862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:51:08,164][74987] Avg episode reward: [(0, '11.360'), (1, '19.630')] -[2023-10-14 13:51:09,448][75949] Updated weights for policy 0, policy_version 11010 (0.0007) -[2023-10-14 13:51:09,814][75949] Updated weights for policy 0, policy_version 11020 (0.0009) -[2023-10-14 13:51:10,187][75949] Updated weights for policy 0, policy_version 11030 (0.0008) -[2023-10-14 13:51:10,338][75950] Updated weights for policy 1, policy_version 11010 (0.0008) -[2023-10-14 13:51:10,547][75949] Updated weights for policy 0, policy_version 11040 (0.0009) -[2023-10-14 13:51:10,737][75950] Updated weights for policy 1, policy_version 11020 (0.0007) -[2023-10-14 13:51:11,105][75950] Updated weights for policy 1, policy_version 11030 (0.0010) -[2023-10-14 13:51:11,470][75950] Updated weights for policy 1, policy_version 11040 (0.0008) -[2023-10-14 13:51:13,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.5). Total num frames: 22609920. Throughput: 0: 1676.4, 1: 1681.6. Samples: 5663426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:51:13,164][74987] Avg episode reward: [(0, '10.590'), (1, '16.980')] -[2023-10-14 13:51:14,646][75949] Updated weights for policy 0, policy_version 11050 (0.0008) -[2023-10-14 13:51:15,029][75949] Updated weights for policy 0, policy_version 11060 (0.0008) -[2023-10-14 13:51:15,398][75949] Updated weights for policy 0, policy_version 11070 (0.0009) -[2023-10-14 13:51:15,544][75950] Updated weights for policy 1, policy_version 11050 (0.0008) -[2023-10-14 13:51:15,905][75950] Updated weights for policy 1, policy_version 11060 (0.0008) -[2023-10-14 13:51:16,273][75950] Updated weights for policy 1, policy_version 11070 (0.0010) -[2023-10-14 13:51:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22675456. Throughput: 0: 1668.3, 1: 1672.4. Samples: 5673332. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 13:51:18,165][74987] Avg episode reward: [(0, '10.700'), (1, '17.960')] -[2023-10-14 13:51:19,424][75949] Updated weights for policy 0, policy_version 11080 (0.0008) -[2023-10-14 13:51:19,786][75949] Updated weights for policy 0, policy_version 11090 (0.0008) -[2023-10-14 13:51:20,161][75949] Updated weights for policy 0, policy_version 11100 (0.0010) -[2023-10-14 13:51:20,428][75950] Updated weights for policy 1, policy_version 11080 (0.0007) -[2023-10-14 13:51:20,803][75950] Updated weights for policy 1, policy_version 11090 (0.0007) -[2023-10-14 13:51:21,168][75950] Updated weights for policy 1, policy_version 11100 (0.0007) -[2023-10-14 13:51:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 22740992. Throughput: 0: 1677.1, 1: 1662.0. Samples: 5692850. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 13:51:23,165][74987] Avg episode reward: [(0, '11.080'), (1, '16.870')] -[2023-10-14 13:51:24,172][75949] Updated weights for policy 0, policy_version 11110 (0.0008) -[2023-10-14 13:51:24,548][75949] Updated weights for policy 0, policy_version 11120 (0.0009) -[2023-10-14 13:51:24,919][75949] Updated weights for policy 0, policy_version 11130 (0.0009) -[2023-10-14 13:51:25,263][75950] Updated weights for policy 1, policy_version 11110 (0.0010) -[2023-10-14 13:51:25,627][75950] Updated weights for policy 1, policy_version 11120 (0.0007) -[2023-10-14 13:51:25,990][75950] Updated weights for policy 1, policy_version 11130 (0.0008) -[2023-10-14 13:51:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22806528. Throughput: 0: 1673.6, 1: 1670.4. Samples: 5713358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:51:28,164][74987] Avg episode reward: [(0, '10.960'), (1, '18.060')] -[2023-10-14 13:51:28,928][75949] Updated weights for policy 0, policy_version 11140 (0.0010) -[2023-10-14 13:51:29,301][75949] Updated weights for policy 0, policy_version 11150 (0.0009) -[2023-10-14 13:51:29,673][75949] Updated weights for policy 0, policy_version 11160 (0.0008) -[2023-10-14 13:51:30,115][75950] Updated weights for policy 1, policy_version 11140 (0.0008) -[2023-10-14 13:51:30,477][75950] Updated weights for policy 1, policy_version 11150 (0.0010) -[2023-10-14 13:51:30,850][75950] Updated weights for policy 1, policy_version 11160 (0.0010) -[2023-10-14 13:51:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22872064. Throughput: 0: 1669.2, 1: 1662.3. Samples: 5723032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:51:33,165][74987] Avg episode reward: [(0, '11.220'), (1, '18.030')] -[2023-10-14 13:51:33,859][75949] Updated weights for policy 0, policy_version 11170 (0.0007) -[2023-10-14 13:51:34,239][75949] Updated weights for policy 0, policy_version 11180 (0.0008) -[2023-10-14 13:51:34,612][75949] Updated weights for policy 0, policy_version 11190 (0.0009) -[2023-10-14 13:51:34,975][75950] Updated weights for policy 1, policy_version 11170 (0.0009) -[2023-10-14 13:51:34,983][75949] Updated weights for policy 0, policy_version 11200 (0.0007) -[2023-10-14 13:51:35,351][75950] Updated weights for policy 1, policy_version 11180 (0.0010) -[2023-10-14 13:51:35,708][75950] Updated weights for policy 1, policy_version 11190 (0.0008) -[2023-10-14 13:51:36,077][75950] Updated weights for policy 1, policy_version 11200 (0.0008) -[2023-10-14 13:51:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22937600. Throughput: 0: 1677.1, 1: 1668.8. Samples: 5743148. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 13:51:38,164][74987] Avg episode reward: [(0, '11.190'), (1, '18.860')] -[2023-10-14 13:51:39,020][75949] Updated weights for policy 0, policy_version 11210 (0.0010) -[2023-10-14 13:51:39,392][75949] Updated weights for policy 0, policy_version 11220 (0.0011) -[2023-10-14 13:51:39,768][75949] Updated weights for policy 0, policy_version 11230 (0.0009) -[2023-10-14 13:51:40,113][75950] Updated weights for policy 1, policy_version 11210 (0.0008) -[2023-10-14 13:51:40,476][75950] Updated weights for policy 1, policy_version 11220 (0.0009) -[2023-10-14 13:51:40,856][75950] Updated weights for policy 1, policy_version 11230 (0.0008) -[2023-10-14 13:51:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23003136. Throughput: 0: 1675.4, 1: 1675.8. Samples: 5763632. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 13:51:43,164][74987] Avg episode reward: [(0, '10.730'), (1, '17.850')] -[2023-10-14 13:51:43,950][75949] Updated weights for policy 0, policy_version 11240 (0.0009) -[2023-10-14 13:51:44,320][75949] Updated weights for policy 0, policy_version 11250 (0.0008) -[2023-10-14 13:51:44,696][75949] Updated weights for policy 0, policy_version 11260 (0.0008) -[2023-10-14 13:51:44,954][75950] Updated weights for policy 1, policy_version 11240 (0.0008) -[2023-10-14 13:51:45,327][75950] Updated weights for policy 1, policy_version 11250 (0.0010) -[2023-10-14 13:51:45,704][75950] Updated weights for policy 1, policy_version 11260 (0.0010) -[2023-10-14 13:51:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23068672. Throughput: 0: 1676.4, 1: 1651.5. Samples: 5772908. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) -[2023-10-14 13:51:48,164][74987] Avg episode reward: [(0, '10.780'), (1, '16.740')] -[2023-10-14 13:51:48,704][75949] Updated weights for policy 0, policy_version 11270 (0.0007) -[2023-10-14 13:51:49,073][75949] Updated weights for policy 0, policy_version 11280 (0.0011) -[2023-10-14 13:51:49,452][75949] Updated weights for policy 0, policy_version 11290 (0.0010) -[2023-10-14 13:51:49,976][75950] Updated weights for policy 1, policy_version 11270 (0.0009) -[2023-10-14 13:51:50,347][75950] Updated weights for policy 1, policy_version 11280 (0.0009) -[2023-10-14 13:51:50,726][75950] Updated weights for policy 1, policy_version 11290 (0.0009) -[2023-10-14 13:51:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23134208. Throughput: 0: 1670.5, 1: 1663.1. Samples: 5792874. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) -[2023-10-14 13:51:53,165][74987] Avg episode reward: [(0, '10.660'), (1, '17.480')] -[2023-10-14 13:51:53,526][75949] Updated weights for policy 0, policy_version 11300 (0.0009) -[2023-10-14 13:51:53,901][75949] Updated weights for policy 0, policy_version 11310 (0.0009) -[2023-10-14 13:51:54,260][75949] Updated weights for policy 0, policy_version 11320 (0.0010) -[2023-10-14 13:51:54,924][75950] Updated weights for policy 1, policy_version 11300 (0.0009) -[2023-10-14 13:51:55,321][75950] Updated weights for policy 1, policy_version 11310 (0.0008) -[2023-10-14 13:51:55,687][75950] Updated weights for policy 1, policy_version 11320 (0.0009) -[2023-10-14 13:51:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23199744. Throughput: 0: 1670.8, 1: 1660.5. Samples: 5813334. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-14 13:51:58,164][74987] Avg episode reward: [(0, '10.650'), (1, '17.140')] -[2023-10-14 13:51:58,298][75949] Updated weights for policy 0, policy_version 11330 (0.0009) -[2023-10-14 13:51:58,668][75949] Updated weights for policy 0, policy_version 11340 (0.0007) -[2023-10-14 13:51:59,042][75949] Updated weights for policy 0, policy_version 11350 (0.0007) -[2023-10-14 13:51:59,412][75949] Updated weights for policy 0, policy_version 11360 (0.0007) -[2023-10-14 13:51:59,727][75950] Updated weights for policy 1, policy_version 11330 (0.0009) -[2023-10-14 13:52:00,101][75950] Updated weights for policy 1, policy_version 11340 (0.0010) -[2023-10-14 13:52:00,464][75950] Updated weights for policy 1, policy_version 11350 (0.0008) -[2023-10-14 13:52:00,827][75950] Updated weights for policy 1, policy_version 11360 (0.0007) -[2023-10-14 13:52:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 23265280. Throughput: 0: 1669.6, 1: 1648.6. Samples: 5822650. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) -[2023-10-14 13:52:03,165][74987] Avg episode reward: [(0, '11.210'), (1, '19.720')] -[2023-10-14 13:52:03,166][75801] Saving new best policy, reward=19.720! -[2023-10-14 13:52:03,607][75949] Updated weights for policy 0, policy_version 11370 (0.0007) -[2023-10-14 13:52:03,983][75949] Updated weights for policy 0, policy_version 11380 (0.0010) -[2023-10-14 13:52:04,354][75949] Updated weights for policy 0, policy_version 11390 (0.0007) -[2023-10-14 13:52:05,147][75950] Updated weights for policy 1, policy_version 11370 (0.0007) -[2023-10-14 13:52:05,518][75950] Updated weights for policy 1, policy_version 11380 (0.0009) -[2023-10-14 13:52:05,892][75950] Updated weights for policy 1, policy_version 11390 (0.0010) -[2023-10-14 13:52:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23330816. Throughput: 0: 1665.8, 1: 1663.3. Samples: 5842658. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) -[2023-10-14 13:52:08,164][74987] Avg episode reward: [(0, '10.790'), (1, '18.230')] -[2023-10-14 13:52:08,533][75949] Updated weights for policy 0, policy_version 11400 (0.0007) -[2023-10-14 13:52:08,898][75949] Updated weights for policy 0, policy_version 11410 (0.0008) -[2023-10-14 13:52:09,273][75949] Updated weights for policy 0, policy_version 11420 (0.0010) -[2023-10-14 13:52:09,828][75950] Updated weights for policy 1, policy_version 11400 (0.0009) -[2023-10-14 13:52:10,193][75950] Updated weights for policy 1, policy_version 11410 (0.0010) -[2023-10-14 13:52:10,563][75950] Updated weights for policy 1, policy_version 11420 (0.0012) -[2023-10-14 13:52:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23396352. Throughput: 0: 1664.8, 1: 1669.1. Samples: 5863384. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) -[2023-10-14 13:52:13,164][74987] Avg episode reward: [(0, '10.800'), (1, '18.350')] -[2023-10-14 13:52:13,297][75949] Updated weights for policy 0, policy_version 11430 (0.0007) -[2023-10-14 13:52:13,669][75949] Updated weights for policy 0, policy_version 11440 (0.0007) -[2023-10-14 13:52:14,033][75949] Updated weights for policy 0, policy_version 11450 (0.0009) -[2023-10-14 13:52:14,698][75950] Updated weights for policy 1, policy_version 11430 (0.0010) -[2023-10-14 13:52:15,059][75950] Updated weights for policy 1, policy_version 11440 (0.0008) -[2023-10-14 13:52:15,421][75950] Updated weights for policy 1, policy_version 11450 (0.0009) -[2023-10-14 13:52:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 23461888. Throughput: 0: 1668.7, 1: 1654.4. Samples: 5872568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:52:18,164][74987] Avg episode reward: [(0, '10.910'), (1, '17.000')] -[2023-10-14 13:52:18,248][75949] Updated weights for policy 0, policy_version 11460 (0.0009) -[2023-10-14 13:52:18,617][75949] Updated weights for policy 0, policy_version 11470 (0.0007) -[2023-10-14 13:52:18,984][75949] Updated weights for policy 0, policy_version 11480 (0.0007) -[2023-10-14 13:52:19,476][75950] Updated weights for policy 1, policy_version 11460 (0.0009) -[2023-10-14 13:52:19,842][75950] Updated weights for policy 1, policy_version 11470 (0.0012) -[2023-10-14 13:52:20,213][75950] Updated weights for policy 1, policy_version 11480 (0.0011) -[2023-10-14 13:52:22,870][75949] Updated weights for policy 0, policy_version 11490 (0.0009) -[2023-10-14 13:52:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23527424. Throughput: 0: 1667.9, 1: 1660.4. Samples: 5892922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:52:23,164][74987] Avg episode reward: [(0, '11.030'), (1, '18.890')] -[2023-10-14 13:52:23,236][75949] Updated weights for policy 0, policy_version 11500 (0.0007) -[2023-10-14 13:52:23,603][75949] Updated weights for policy 0, policy_version 11510 (0.0007) -[2023-10-14 13:52:23,973][75949] Updated weights for policy 0, policy_version 11520 (0.0009) -[2023-10-14 13:52:24,455][75950] Updated weights for policy 1, policy_version 11490 (0.0009) -[2023-10-14 13:52:24,825][75950] Updated weights for policy 1, policy_version 11500 (0.0009) -[2023-10-14 13:52:25,183][75950] Updated weights for policy 1, policy_version 11510 (0.0007) -[2023-10-14 13:52:25,547][75950] Updated weights for policy 1, policy_version 11520 (0.0007) -[2023-10-14 13:52:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23592960. Throughput: 0: 1666.8, 1: 1659.2. Samples: 5913302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:52:28,164][74987] Avg episode reward: [(0, '10.610'), (1, '18.050')] -[2023-10-14 13:52:28,204][75949] Updated weights for policy 0, policy_version 11530 (0.0009) -[2023-10-14 13:52:28,582][75949] Updated weights for policy 0, policy_version 11540 (0.0009) -[2023-10-14 13:52:28,956][75949] Updated weights for policy 0, policy_version 11550 (0.0011) -[2023-10-14 13:52:29,704][75950] Updated weights for policy 1, policy_version 11530 (0.0008) -[2023-10-14 13:52:30,078][75950] Updated weights for policy 1, policy_version 11540 (0.0007) -[2023-10-14 13:52:30,433][75950] Updated weights for policy 1, policy_version 11550 (0.0009) -[2023-10-14 13:52:33,112][75949] Updated weights for policy 0, policy_version 11560 (0.0009) -[2023-10-14 13:52:33,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23658496. Throughput: 0: 1664.8, 1: 1657.3. Samples: 5922404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:52:33,164][74987] Avg episode reward: [(0, '10.760'), (1, '18.340')] -[2023-10-14 13:52:33,486][75949] Updated weights for policy 0, policy_version 11570 (0.0008) -[2023-10-14 13:52:33,856][75949] Updated weights for policy 0, policy_version 11580 (0.0009) -[2023-10-14 13:52:34,388][75950] Updated weights for policy 1, policy_version 11560 (0.0008) -[2023-10-14 13:52:34,751][75950] Updated weights for policy 1, policy_version 11570 (0.0010) -[2023-10-14 13:52:35,116][75950] Updated weights for policy 1, policy_version 11580 (0.0011) -[2023-10-14 13:52:37,967][75949] Updated weights for policy 0, policy_version 11590 (0.0010) -[2023-10-14 13:52:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23724032. Throughput: 0: 1666.7, 1: 1671.8. Samples: 5943106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:52:38,165][74987] Avg episode reward: [(0, '11.070'), (1, '16.940')] -[2023-10-14 13:52:38,352][75949] Updated weights for policy 0, policy_version 11600 (0.0009) -[2023-10-14 13:52:38,741][75949] Updated weights for policy 0, policy_version 11610 (0.0009) -[2023-10-14 13:52:39,088][75950] Updated weights for policy 1, policy_version 11590 (0.0009) -[2023-10-14 13:52:39,456][75950] Updated weights for policy 1, policy_version 11600 (0.0008) -[2023-10-14 13:52:39,813][75950] Updated weights for policy 1, policy_version 11610 (0.0007) -[2023-10-14 13:52:42,879][75949] Updated weights for policy 0, policy_version 11620 (0.0009) -[2023-10-14 13:52:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 23789568. Throughput: 0: 1663.7, 1: 1676.0. Samples: 5963622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:52:43,164][74987] Avg episode reward: [(0, '10.900'), (1, '18.100')] -[2023-10-14 13:52:43,172][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000011616_11894784.pth... -[2023-10-14 13:52:43,211][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000010080_10321920.pth -[2023-10-14 13:52:43,241][75949] Updated weights for policy 0, policy_version 11630 (0.0009) -[2023-10-14 13:52:43,618][75949] Updated weights for policy 0, policy_version 11640 (0.0008) -[2023-10-14 13:52:43,913][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000011648_11927552.pth... -[2023-10-14 13:52:43,953][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000010080_10321920.pth -[2023-10-14 13:52:44,128][75950] Updated weights for policy 1, policy_version 11620 (0.0009) -[2023-10-14 13:52:44,532][75950] Updated weights for policy 1, policy_version 11630 (0.0008) -[2023-10-14 13:52:44,897][75950] Updated weights for policy 1, policy_version 11640 (0.0009) -[2023-10-14 13:52:47,946][75949] Updated weights for policy 0, policy_version 11650 (0.0008) -[2023-10-14 13:52:48,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23855104. Throughput: 0: 1664.8, 1: 1665.8. Samples: 5972526. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 13:52:48,164][74987] Avg episode reward: [(0, '11.260'), (1, '18.120')] -[2023-10-14 13:52:48,315][75949] Updated weights for policy 0, policy_version 11660 (0.0008) -[2023-10-14 13:52:48,694][75949] Updated weights for policy 0, policy_version 11670 (0.0009) -[2023-10-14 13:52:48,904][75950] Updated weights for policy 1, policy_version 11650 (0.0008) -[2023-10-14 13:52:49,062][75949] Updated weights for policy 0, policy_version 11680 (0.0008) -[2023-10-14 13:52:49,264][75950] Updated weights for policy 1, policy_version 11660 (0.0009) -[2023-10-14 13:52:49,642][75950] Updated weights for policy 1, policy_version 11670 (0.0010) -[2023-10-14 13:52:50,002][75950] Updated weights for policy 1, policy_version 11680 (0.0008) -[2023-10-14 13:52:53,092][75949] Updated weights for policy 0, policy_version 11690 (0.0008) -[2023-10-14 13:52:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23920640. Throughput: 0: 1667.0, 1: 1672.4. Samples: 5992930. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 13:52:53,164][74987] Avg episode reward: [(0, '10.920'), (1, '18.750')] -[2023-10-14 13:52:53,460][75949] Updated weights for policy 0, policy_version 11700 (0.0007) -[2023-10-14 13:52:53,833][75949] Updated weights for policy 0, policy_version 11710 (0.0008) -[2023-10-14 13:52:54,158][75950] Updated weights for policy 1, policy_version 11690 (0.0007) -[2023-10-14 13:52:54,526][75950] Updated weights for policy 1, policy_version 11700 (0.0007) -[2023-10-14 13:52:54,892][75950] Updated weights for policy 1, policy_version 11710 (0.0008) -[2023-10-14 13:52:57,878][75949] Updated weights for policy 0, policy_version 11720 (0.0008) -[2023-10-14 13:52:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23986176. Throughput: 0: 1663.9, 1: 1672.9. Samples: 6013542. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 13:52:58,164][74987] Avg episode reward: [(0, '10.750'), (1, '18.590')] -[2023-10-14 13:52:58,257][75949] Updated weights for policy 0, policy_version 11730 (0.0008) -[2023-10-14 13:52:58,627][75949] Updated weights for policy 0, policy_version 11740 (0.0007) -[2023-10-14 13:52:58,919][75950] Updated weights for policy 1, policy_version 11720 (0.0008) -[2023-10-14 13:52:59,290][75950] Updated weights for policy 1, policy_version 11730 (0.0007) -[2023-10-14 13:52:59,665][75950] Updated weights for policy 1, policy_version 11740 (0.0007) -[2023-10-14 13:53:02,516][75949] Updated weights for policy 0, policy_version 11750 (0.0010) -[2023-10-14 13:53:02,885][75949] Updated weights for policy 0, policy_version 11760 (0.0007) -[2023-10-14 13:53:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 24051712. Throughput: 0: 1665.9, 1: 1671.7. Samples: 6022760. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 13:53:03,164][74987] Avg episode reward: [(0, '11.080'), (1, '18.180')] -[2023-10-14 13:53:03,257][75949] Updated weights for policy 0, policy_version 11770 (0.0009) -[2023-10-14 13:53:03,499][75950] Updated weights for policy 1, policy_version 11750 (0.0008) -[2023-10-14 13:53:03,875][75950] Updated weights for policy 1, policy_version 11760 (0.0008) -[2023-10-14 13:53:04,244][75950] Updated weights for policy 1, policy_version 11770 (0.0007) -[2023-10-14 13:53:07,519][75949] Updated weights for policy 0, policy_version 11780 (0.0009) -[2023-10-14 13:53:07,886][75949] Updated weights for policy 0, policy_version 11790 (0.0009) -[2023-10-14 13:53:08,145][75950] Updated weights for policy 1, policy_version 11780 (0.0009) -[2023-10-14 13:53:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 24117248. Throughput: 0: 1664.4, 1: 1684.0. Samples: 6043602. Policy #0 lag: (min: 9.0, avg: 23.8, max: 41.0) -[2023-10-14 13:53:08,165][74987] Avg episode reward: [(0, '11.330'), (1, '19.070')] -[2023-10-14 13:53:08,252][75949] Updated weights for policy 0, policy_version 11800 (0.0008) -[2023-10-14 13:53:08,509][75950] Updated weights for policy 1, policy_version 11790 (0.0009) -[2023-10-14 13:53:08,888][75950] Updated weights for policy 1, policy_version 11800 (0.0009) -[2023-10-14 13:53:12,330][75949] Updated weights for policy 0, policy_version 11810 (0.0008) -[2023-10-14 13:53:12,696][75949] Updated weights for policy 0, policy_version 11820 (0.0007) -[2023-10-14 13:53:12,867][75950] Updated weights for policy 1, policy_version 11810 (0.0009) -[2023-10-14 13:53:13,073][75949] Updated weights for policy 0, policy_version 11830 (0.0007) -[2023-10-14 13:53:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 24182784. Throughput: 0: 1653.8, 1: 1689.5. Samples: 6063752. Policy #0 lag: (min: 9.0, avg: 23.8, max: 41.0) -[2023-10-14 13:53:13,164][74987] Avg episode reward: [(0, '10.490'), (1, '17.770')] -[2023-10-14 13:53:13,232][75950] Updated weights for policy 1, policy_version 11820 (0.0010) -[2023-10-14 13:53:13,438][75949] Updated weights for policy 0, policy_version 11840 (0.0008) -[2023-10-14 13:53:13,613][75950] Updated weights for policy 1, policy_version 11830 (0.0009) -[2023-10-14 13:53:13,977][75950] Updated weights for policy 1, policy_version 11840 (0.0009) -[2023-10-14 13:53:17,579][75949] Updated weights for policy 0, policy_version 11850 (0.0009) -[2023-10-14 13:53:17,943][75949] Updated weights for policy 0, policy_version 11860 (0.0007) -[2023-10-14 13:53:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 24248320. Throughput: 0: 1664.0, 1: 1686.0. Samples: 6073158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:53:18,164][74987] Avg episode reward: [(0, '10.770'), (1, '19.470')] -[2023-10-14 13:53:18,240][75950] Updated weights for policy 1, policy_version 11850 (0.0009) -[2023-10-14 13:53:18,321][75949] Updated weights for policy 0, policy_version 11870 (0.0008) -[2023-10-14 13:53:18,600][75950] Updated weights for policy 1, policy_version 11860 (0.0007) -[2023-10-14 13:53:18,968][75950] Updated weights for policy 1, policy_version 11870 (0.0009) -[2023-10-14 13:53:22,301][75949] Updated weights for policy 0, policy_version 11880 (0.0007) -[2023-10-14 13:53:22,669][75949] Updated weights for policy 0, policy_version 11890 (0.0008) -[2023-10-14 13:53:23,048][75949] Updated weights for policy 0, policy_version 11900 (0.0008) -[2023-10-14 13:53:23,120][75950] Updated weights for policy 1, policy_version 11880 (0.0009) -[2023-10-14 13:53:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 24313856. Throughput: 0: 1663.8, 1: 1680.4. Samples: 6093594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:53:23,165][74987] Avg episode reward: [(0, '11.240'), (1, '17.860')] -[2023-10-14 13:53:23,480][75950] Updated weights for policy 1, policy_version 11890 (0.0008) -[2023-10-14 13:53:23,852][75950] Updated weights for policy 1, policy_version 11900 (0.0008) -[2023-10-14 13:53:27,217][75949] Updated weights for policy 0, policy_version 11910 (0.0007) -[2023-10-14 13:53:27,593][75949] Updated weights for policy 0, policy_version 11920 (0.0008) -[2023-10-14 13:53:27,891][75950] Updated weights for policy 1, policy_version 11910 (0.0008) -[2023-10-14 13:53:27,960][75949] Updated weights for policy 0, policy_version 11930 (0.0008) -[2023-10-14 13:53:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 24379392. Throughput: 0: 1647.0, 1: 1682.7. Samples: 6113460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:53:28,164][74987] Avg episode reward: [(0, '10.330'), (1, '18.550')] -[2023-10-14 13:53:28,257][75950] Updated weights for policy 1, policy_version 11920 (0.0009) -[2023-10-14 13:53:28,633][75950] Updated weights for policy 1, policy_version 11930 (0.0008) -[2023-10-14 13:53:32,153][75949] Updated weights for policy 0, policy_version 11940 (0.0008) -[2023-10-14 13:53:32,519][75949] Updated weights for policy 0, policy_version 11950 (0.0008) -[2023-10-14 13:53:32,838][75950] Updated weights for policy 1, policy_version 11940 (0.0008) -[2023-10-14 13:53:32,895][75949] Updated weights for policy 0, policy_version 11960 (0.0007) -[2023-10-14 13:53:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 24444928. Throughput: 0: 1663.2, 1: 1684.4. Samples: 6123168. Policy #0 lag: (min: 1.0, avg: 4.1, max: 33.0) -[2023-10-14 13:53:33,164][74987] Avg episode reward: [(0, '10.780'), (1, '16.880')] -[2023-10-14 13:53:33,237][75950] Updated weights for policy 1, policy_version 11950 (0.0007) -[2023-10-14 13:53:33,601][75950] Updated weights for policy 1, policy_version 11960 (0.0009) -[2023-10-14 13:53:36,999][75949] Updated weights for policy 0, policy_version 11970 (0.0008) -[2023-10-14 13:53:37,379][75949] Updated weights for policy 0, policy_version 11980 (0.0009) -[2023-10-14 13:53:37,734][75950] Updated weights for policy 1, policy_version 11970 (0.0008) -[2023-10-14 13:53:37,760][75949] Updated weights for policy 0, policy_version 11990 (0.0009) -[2023-10-14 13:53:38,103][75950] Updated weights for policy 1, policy_version 11980 (0.0008) -[2023-10-14 13:53:38,122][75949] Updated weights for policy 0, policy_version 12000 (0.0009) -[2023-10-14 13:53:38,163][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 24543232. Throughput: 0: 1664.3, 1: 1685.5. Samples: 6143668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:53:38,164][74987] Avg episode reward: [(0, '11.310'), (1, '18.640')] -[2023-10-14 13:53:38,468][75950] Updated weights for policy 1, policy_version 11990 (0.0007) -[2023-10-14 13:53:38,847][75950] Updated weights for policy 1, policy_version 12000 (0.0007) -[2023-10-14 13:53:42,073][75949] Updated weights for policy 0, policy_version 12010 (0.0010) -[2023-10-14 13:53:42,445][75949] Updated weights for policy 0, policy_version 12020 (0.0010) -[2023-10-14 13:53:42,812][75949] Updated weights for policy 0, policy_version 12030 (0.0009) -[2023-10-14 13:53:42,914][75950] Updated weights for policy 1, policy_version 12010 (0.0008) -[2023-10-14 13:53:43,164][74987] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 24608768. Throughput: 0: 1642.7, 1: 1684.0. Samples: 6163244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:53:43,165][74987] Avg episode reward: [(0, '10.710'), (1, '18.430')] -[2023-10-14 13:53:43,290][75950] Updated weights for policy 1, policy_version 12020 (0.0009) -[2023-10-14 13:53:43,655][75950] Updated weights for policy 1, policy_version 12030 (0.0009) -[2023-10-14 13:53:46,995][75949] Updated weights for policy 0, policy_version 12040 (0.0010) -[2023-10-14 13:53:47,362][75949] Updated weights for policy 0, policy_version 12050 (0.0008) -[2023-10-14 13:53:47,734][75949] Updated weights for policy 0, policy_version 12060 (0.0008) -[2023-10-14 13:53:47,807][75950] Updated weights for policy 1, policy_version 12040 (0.0009) -[2023-10-14 13:53:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 24674304. Throughput: 0: 1659.6, 1: 1684.0. Samples: 6173226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:53:48,164][74987] Avg episode reward: [(0, '11.040'), (1, '18.320')] -[2023-10-14 13:53:48,167][75950] Updated weights for policy 1, policy_version 12050 (0.0010) -[2023-10-14 13:53:48,543][75950] Updated weights for policy 1, policy_version 12060 (0.0009) -[2023-10-14 13:53:51,819][75949] Updated weights for policy 0, policy_version 12070 (0.0009) -[2023-10-14 13:53:52,186][75949] Updated weights for policy 0, policy_version 12080 (0.0007) -[2023-10-14 13:53:52,561][75949] Updated weights for policy 0, policy_version 12090 (0.0008) -[2023-10-14 13:53:52,669][75950] Updated weights for policy 1, policy_version 12070 (0.0008) -[2023-10-14 13:53:53,037][75950] Updated weights for policy 1, policy_version 12080 (0.0009) -[2023-10-14 13:53:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 24739840. Throughput: 0: 1658.3, 1: 1674.0. Samples: 6193556. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) -[2023-10-14 13:53:53,164][74987] Avg episode reward: [(0, '11.420'), (1, '18.800')] -[2023-10-14 13:53:53,405][75950] Updated weights for policy 1, policy_version 12090 (0.0008) -[2023-10-14 13:53:56,636][75949] Updated weights for policy 0, policy_version 12100 (0.0008) -[2023-10-14 13:53:57,007][75949] Updated weights for policy 0, policy_version 12110 (0.0009) -[2023-10-14 13:53:57,333][75950] Updated weights for policy 1, policy_version 12100 (0.0007) -[2023-10-14 13:53:57,380][75949] Updated weights for policy 0, policy_version 12120 (0.0008) -[2023-10-14 13:53:57,695][75950] Updated weights for policy 1, policy_version 12110 (0.0008) -[2023-10-14 13:53:58,058][75950] Updated weights for policy 1, policy_version 12120 (0.0009) -[2023-10-14 13:53:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 24805376. Throughput: 0: 1646.1, 1: 1663.2. Samples: 6212668. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) -[2023-10-14 13:53:58,164][74987] Avg episode reward: [(0, '10.280'), (1, '18.660')] -[2023-10-14 13:54:01,603][75949] Updated weights for policy 0, policy_version 12130 (0.0008) -[2023-10-14 13:54:01,972][75949] Updated weights for policy 0, policy_version 12140 (0.0009) -[2023-10-14 13:54:02,205][75950] Updated weights for policy 1, policy_version 12130 (0.0007) -[2023-10-14 13:54:02,354][75949] Updated weights for policy 0, policy_version 12150 (0.0008) -[2023-10-14 13:54:02,573][75950] Updated weights for policy 1, policy_version 12140 (0.0008) -[2023-10-14 13:54:02,728][75949] Updated weights for policy 0, policy_version 12160 (0.0009) -[2023-10-14 13:54:02,937][75950] Updated weights for policy 1, policy_version 12150 (0.0007) -[2023-10-14 13:54:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 24870912. Throughput: 0: 1664.1, 1: 1673.6. Samples: 6223354. Policy #0 lag: (min: 22.0, avg: 22.0, max: 25.0) -[2023-10-14 13:54:03,165][74987] Avg episode reward: [(0, '10.630'), (1, '18.330')] -[2023-10-14 13:54:03,305][75950] Updated weights for policy 1, policy_version 12160 (0.0010) -[2023-10-14 13:54:06,856][75949] Updated weights for policy 0, policy_version 12170 (0.0011) -[2023-10-14 13:54:07,222][75949] Updated weights for policy 0, policy_version 12180 (0.0010) -[2023-10-14 13:54:07,373][75950] Updated weights for policy 1, policy_version 12170 (0.0008) -[2023-10-14 13:54:07,589][75949] Updated weights for policy 0, policy_version 12190 (0.0009) -[2023-10-14 13:54:07,744][75950] Updated weights for policy 1, policy_version 12180 (0.0008) -[2023-10-14 13:54:08,104][75950] Updated weights for policy 1, policy_version 12190 (0.0007) -[2023-10-14 13:54:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 24936448. Throughput: 0: 1657.1, 1: 1675.5. Samples: 6243562. Policy #0 lag: (min: 22.0, avg: 22.0, max: 25.0) -[2023-10-14 13:54:08,164][74987] Avg episode reward: [(0, '11.510'), (1, '19.750')] -[2023-10-14 13:54:08,178][75801] Saving new best policy, reward=19.750! -[2023-10-14 13:54:11,849][75949] Updated weights for policy 0, policy_version 12200 (0.0007) -[2023-10-14 13:54:12,176][75950] Updated weights for policy 1, policy_version 12200 (0.0008) -[2023-10-14 13:54:12,222][75949] Updated weights for policy 0, policy_version 12210 (0.0008) -[2023-10-14 13:54:12,545][75950] Updated weights for policy 1, policy_version 12210 (0.0007) -[2023-10-14 13:54:12,591][75949] Updated weights for policy 0, policy_version 12220 (0.0009) -[2023-10-14 13:54:12,910][75950] Updated weights for policy 1, policy_version 12220 (0.0009) -[2023-10-14 13:54:13,163][74987] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 25034752. Throughput: 0: 1652.1, 1: 1655.2. Samples: 6262288. Policy #0 lag: (min: 5.0, avg: 11.0, max: 37.0) -[2023-10-14 13:54:13,164][74987] Avg episode reward: [(0, '10.490'), (1, '18.560')] -[2023-10-14 13:54:16,562][75949] Updated weights for policy 0, policy_version 12230 (0.0008) -[2023-10-14 13:54:16,930][75949] Updated weights for policy 0, policy_version 12240 (0.0009) -[2023-10-14 13:54:16,981][75950] Updated weights for policy 1, policy_version 12230 (0.0009) -[2023-10-14 13:54:17,296][75949] Updated weights for policy 0, policy_version 12250 (0.0008) -[2023-10-14 13:54:17,345][75950] Updated weights for policy 1, policy_version 12240 (0.0008) -[2023-10-14 13:54:17,713][75950] Updated weights for policy 1, policy_version 12250 (0.0008) -[2023-10-14 13:54:18,164][74987] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 25100288. Throughput: 0: 1660.7, 1: 1673.8. Samples: 6273218. Policy #0 lag: (min: 5.0, avg: 11.0, max: 37.0) -[2023-10-14 13:54:18,164][74987] Avg episode reward: [(0, '10.980'), (1, '19.020')] -[2023-10-14 13:54:21,301][75949] Updated weights for policy 0, policy_version 12260 (0.0009) -[2023-10-14 13:54:21,659][75949] Updated weights for policy 0, policy_version 12270 (0.0009) -[2023-10-14 13:54:21,954][75950] Updated weights for policy 1, policy_version 12260 (0.0007) -[2023-10-14 13:54:22,031][75949] Updated weights for policy 0, policy_version 12280 (0.0009) -[2023-10-14 13:54:22,343][75950] Updated weights for policy 1, policy_version 12270 (0.0009) -[2023-10-14 13:54:22,713][75950] Updated weights for policy 1, policy_version 12280 (0.0010) -[2023-10-14 13:54:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 25165824. Throughput: 0: 1652.9, 1: 1672.6. Samples: 6293318. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) -[2023-10-14 13:54:23,164][74987] Avg episode reward: [(0, '11.150'), (1, '17.030')] -[2023-10-14 13:54:25,976][75949] Updated weights for policy 0, policy_version 12290 (0.0007) -[2023-10-14 13:54:26,347][75949] Updated weights for policy 0, policy_version 12300 (0.0007) -[2023-10-14 13:54:26,727][75949] Updated weights for policy 0, policy_version 12310 (0.0008) -[2023-10-14 13:54:26,885][75950] Updated weights for policy 1, policy_version 12290 (0.0009) -[2023-10-14 13:54:27,089][75949] Updated weights for policy 0, policy_version 12320 (0.0007) -[2023-10-14 13:54:27,248][75950] Updated weights for policy 1, policy_version 12300 (0.0011) -[2023-10-14 13:54:27,614][75950] Updated weights for policy 1, policy_version 12310 (0.0008) -[2023-10-14 13:54:27,983][75950] Updated weights for policy 1, policy_version 12320 (0.0008) -[2023-10-14 13:54:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 25231360. Throughput: 0: 1663.9, 1: 1647.0. Samples: 6312232. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) -[2023-10-14 13:54:28,164][74987] Avg episode reward: [(0, '10.630'), (1, '18.250')] -[2023-10-14 13:54:31,194][75949] Updated weights for policy 0, policy_version 12330 (0.0008) -[2023-10-14 13:54:31,570][75949] Updated weights for policy 0, policy_version 12340 (0.0007) -[2023-10-14 13:54:31,933][75949] Updated weights for policy 0, policy_version 12350 (0.0007) -[2023-10-14 13:54:32,107][75950] Updated weights for policy 1, policy_version 12330 (0.0007) -[2023-10-14 13:54:32,480][75950] Updated weights for policy 1, policy_version 12340 (0.0008) -[2023-10-14 13:54:32,848][75950] Updated weights for policy 1, policy_version 12350 (0.0011) -[2023-10-14 13:54:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 25296896. Throughput: 0: 1674.1, 1: 1667.2. Samples: 6323586. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) -[2023-10-14 13:54:33,165][74987] Avg episode reward: [(0, '10.580'), (1, '19.180')] -[2023-10-14 13:54:36,098][75949] Updated weights for policy 0, policy_version 12360 (0.0009) -[2023-10-14 13:54:36,474][75949] Updated weights for policy 0, policy_version 12370 (0.0009) -[2023-10-14 13:54:36,853][75949] Updated weights for policy 0, policy_version 12380 (0.0009) -[2023-10-14 13:54:36,911][75950] Updated weights for policy 1, policy_version 12360 (0.0007) -[2023-10-14 13:54:37,272][75950] Updated weights for policy 1, policy_version 12370 (0.0007) -[2023-10-14 13:54:37,640][75950] Updated weights for policy 1, policy_version 12380 (0.0008) -[2023-10-14 13:54:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 25362432. Throughput: 0: 1656.0, 1: 1671.1. Samples: 6343274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:54:38,165][74987] Avg episode reward: [(0, '11.420'), (1, '18.570')] -[2023-10-14 13:54:40,966][75949] Updated weights for policy 0, policy_version 12390 (0.0009) -[2023-10-14 13:54:41,332][75949] Updated weights for policy 0, policy_version 12400 (0.0011) -[2023-10-14 13:54:41,693][75949] Updated weights for policy 0, policy_version 12410 (0.0009) -[2023-10-14 13:54:41,711][75950] Updated weights for policy 1, policy_version 12390 (0.0009) -[2023-10-14 13:54:42,074][75950] Updated weights for policy 1, policy_version 12400 (0.0008) -[2023-10-14 13:54:42,445][75950] Updated weights for policy 1, policy_version 12410 (0.0008) -[2023-10-14 13:54:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 25427968. Throughput: 0: 1672.5, 1: 1654.3. Samples: 6362378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:54:43,165][74987] Avg episode reward: [(0, '11.000'), (1, '18.710')] -[2023-10-14 13:54:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000012416_12713984.pth... -[2023-10-14 13:54:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000012416_12713984.pth... -[2023-10-14 13:54:43,213][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000010848_11108352.pth -[2023-10-14 13:54:43,216][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000010848_11108352.pth -[2023-10-14 13:54:45,688][75949] Updated weights for policy 0, policy_version 12420 (0.0009) -[2023-10-14 13:54:46,066][75949] Updated weights for policy 0, policy_version 12430 (0.0008) -[2023-10-14 13:54:46,443][75949] Updated weights for policy 0, policy_version 12440 (0.0008) -[2023-10-14 13:54:46,471][75950] Updated weights for policy 1, policy_version 12420 (0.0008) -[2023-10-14 13:54:46,839][75950] Updated weights for policy 1, policy_version 12430 (0.0011) -[2023-10-14 13:54:47,211][75950] Updated weights for policy 1, policy_version 12440 (0.0009) -[2023-10-14 13:54:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 25493504. Throughput: 0: 1677.3, 1: 1672.0. Samples: 6374072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:54:48,165][74987] Avg episode reward: [(0, '10.440'), (1, '18.320')] -[2023-10-14 13:54:50,765][75949] Updated weights for policy 0, policy_version 12450 (0.0010) -[2023-10-14 13:54:51,135][75949] Updated weights for policy 0, policy_version 12460 (0.0010) -[2023-10-14 13:54:51,397][75950] Updated weights for policy 1, policy_version 12450 (0.0009) -[2023-10-14 13:54:51,517][75949] Updated weights for policy 0, policy_version 12470 (0.0008) -[2023-10-14 13:54:51,771][75950] Updated weights for policy 1, policy_version 12460 (0.0008) -[2023-10-14 13:54:51,890][75949] Updated weights for policy 0, policy_version 12480 (0.0008) -[2023-10-14 13:54:52,140][75950] Updated weights for policy 1, policy_version 12470 (0.0007) -[2023-10-14 13:54:52,515][75950] Updated weights for policy 1, policy_version 12480 (0.0011) -[2023-10-14 13:54:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 25559040. Throughput: 0: 1661.6, 1: 1664.2. Samples: 6393224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:54:53,165][74987] Avg episode reward: [(0, '11.270'), (1, '18.620')] -[2023-10-14 13:54:55,991][75949] Updated weights for policy 0, policy_version 12490 (0.0011) -[2023-10-14 13:54:56,367][75949] Updated weights for policy 0, policy_version 12500 (0.0009) -[2023-10-14 13:54:56,581][75950] Updated weights for policy 1, policy_version 12490 (0.0008) -[2023-10-14 13:54:56,736][75949] Updated weights for policy 0, policy_version 12510 (0.0010) -[2023-10-14 13:54:56,950][75950] Updated weights for policy 1, policy_version 12500 (0.0007) -[2023-10-14 13:54:57,320][75950] Updated weights for policy 1, policy_version 12510 (0.0007) -[2023-10-14 13:54:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 25624576. Throughput: 0: 1679.5, 1: 1661.5. Samples: 6412638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:54:58,165][74987] Avg episode reward: [(0, '10.760'), (1, '20.510')] -[2023-10-14 13:54:58,178][75801] Saving new best policy, reward=20.510! -[2023-10-14 13:55:00,752][75949] Updated weights for policy 0, policy_version 12520 (0.0009) -[2023-10-14 13:55:01,123][75949] Updated weights for policy 0, policy_version 12530 (0.0007) -[2023-10-14 13:55:01,251][75950] Updated weights for policy 1, policy_version 12520 (0.0008) -[2023-10-14 13:55:01,506][75949] Updated weights for policy 0, policy_version 12540 (0.0007) -[2023-10-14 13:55:01,621][75950] Updated weights for policy 1, policy_version 12530 (0.0008) -[2023-10-14 13:55:01,989][75950] Updated weights for policy 1, policy_version 12540 (0.0009) -[2023-10-14 13:55:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 25690112. Throughput: 0: 1680.5, 1: 1675.6. Samples: 6424242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:55:03,164][74987] Avg episode reward: [(0, '10.270'), (1, '18.600')] -[2023-10-14 13:55:05,451][75949] Updated weights for policy 0, policy_version 12550 (0.0009) -[2023-10-14 13:55:05,820][75949] Updated weights for policy 0, policy_version 12560 (0.0007) -[2023-10-14 13:55:05,850][75950] Updated weights for policy 1, policy_version 12550 (0.0009) -[2023-10-14 13:55:06,200][75949] Updated weights for policy 0, policy_version 12570 (0.0009) -[2023-10-14 13:55:06,216][75950] Updated weights for policy 1, policy_version 12560 (0.0007) -[2023-10-14 13:55:06,587][75950] Updated weights for policy 1, policy_version 12570 (0.0008) -[2023-10-14 13:55:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 25755648. Throughput: 0: 1668.5, 1: 1657.7. Samples: 6443000. Policy #0 lag: (min: 12.0, avg: 14.5, max: 44.0) -[2023-10-14 13:55:08,164][74987] Avg episode reward: [(0, '10.940'), (1, '19.840')] -[2023-10-14 13:55:10,085][75949] Updated weights for policy 0, policy_version 12580 (0.0011) -[2023-10-14 13:55:10,466][75949] Updated weights for policy 0, policy_version 12590 (0.0010) -[2023-10-14 13:55:10,795][75950] Updated weights for policy 1, policy_version 12580 (0.0007) -[2023-10-14 13:55:10,834][75949] Updated weights for policy 0, policy_version 12600 (0.0010) -[2023-10-14 13:55:11,191][75950] Updated weights for policy 1, policy_version 12590 (0.0008) -[2023-10-14 13:55:11,558][75950] Updated weights for policy 1, policy_version 12600 (0.0010) -[2023-10-14 13:55:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25821184. Throughput: 0: 1682.4, 1: 1674.2. Samples: 6463280. Policy #0 lag: (min: 12.0, avg: 14.5, max: 44.0) -[2023-10-14 13:55:13,164][74987] Avg episode reward: [(0, '11.400'), (1, '17.180')] -[2023-10-14 13:55:14,872][75949] Updated weights for policy 0, policy_version 12610 (0.0008) -[2023-10-14 13:55:15,240][75949] Updated weights for policy 0, policy_version 12620 (0.0007) -[2023-10-14 13:55:15,615][75949] Updated weights for policy 0, policy_version 12630 (0.0007) -[2023-10-14 13:55:15,632][75950] Updated weights for policy 1, policy_version 12610 (0.0008) -[2023-10-14 13:55:15,993][75949] Updated weights for policy 0, policy_version 12640 (0.0009) -[2023-10-14 13:55:16,004][75950] Updated weights for policy 1, policy_version 12620 (0.0007) -[2023-10-14 13:55:16,370][75950] Updated weights for policy 1, policy_version 12630 (0.0009) -[2023-10-14 13:55:16,730][75950] Updated weights for policy 1, policy_version 12640 (0.0010) -[2023-10-14 13:55:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25886720. Throughput: 0: 1657.5, 1: 1678.0. Samples: 6473680. Policy #0 lag: (min: 12.0, avg: 14.5, max: 44.0) -[2023-10-14 13:55:18,164][74987] Avg episode reward: [(0, '10.680'), (1, '18.720')] -[2023-10-14 13:55:20,115][75949] Updated weights for policy 0, policy_version 12650 (0.0008) -[2023-10-14 13:55:20,477][75949] Updated weights for policy 0, policy_version 12660 (0.0009) -[2023-10-14 13:55:20,845][75949] Updated weights for policy 0, policy_version 12670 (0.0007) -[2023-10-14 13:55:21,015][75950] Updated weights for policy 1, policy_version 12650 (0.0008) -[2023-10-14 13:55:21,381][75950] Updated weights for policy 1, policy_version 12660 (0.0010) -[2023-10-14 13:55:21,747][75950] Updated weights for policy 1, policy_version 12670 (0.0008) -[2023-10-14 13:55:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 25952256. Throughput: 0: 1673.0, 1: 1655.1. Samples: 6493040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:55:23,165][74987] Avg episode reward: [(0, '10.610'), (1, '18.510')] -[2023-10-14 13:55:25,011][75949] Updated weights for policy 0, policy_version 12680 (0.0008) -[2023-10-14 13:55:25,384][75949] Updated weights for policy 0, policy_version 12690 (0.0009) -[2023-10-14 13:55:25,763][75949] Updated weights for policy 0, policy_version 12700 (0.0008) -[2023-10-14 13:55:25,797][75950] Updated weights for policy 1, policy_version 12680 (0.0008) -[2023-10-14 13:55:26,173][75950] Updated weights for policy 1, policy_version 12690 (0.0007) -[2023-10-14 13:55:26,532][75950] Updated weights for policy 1, policy_version 12700 (0.0007) -[2023-10-14 13:55:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26017792. Throughput: 0: 1681.7, 1: 1673.9. Samples: 6513376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:55:28,164][74987] Avg episode reward: [(0, '11.610'), (1, '18.850')] -[2023-10-14 13:55:28,176][75615] Saving new best policy, reward=11.610! -[2023-10-14 13:55:29,809][75949] Updated weights for policy 0, policy_version 12710 (0.0009) -[2023-10-14 13:55:30,174][75949] Updated weights for policy 0, policy_version 12720 (0.0009) -[2023-10-14 13:55:30,538][75949] Updated weights for policy 0, policy_version 12730 (0.0007) -[2023-10-14 13:55:30,722][75950] Updated weights for policy 1, policy_version 12710 (0.0010) -[2023-10-14 13:55:31,091][75950] Updated weights for policy 1, policy_version 12720 (0.0009) -[2023-10-14 13:55:31,456][75950] Updated weights for policy 1, policy_version 12730 (0.0008) -[2023-10-14 13:55:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26083328. Throughput: 0: 1655.8, 1: 1669.2. Samples: 6523696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:55:33,164][74987] Avg episode reward: [(0, '10.760'), (1, '20.130')] -[2023-10-14 13:55:34,612][75949] Updated weights for policy 0, policy_version 12740 (0.0008) -[2023-10-14 13:55:34,982][75949] Updated weights for policy 0, policy_version 12750 (0.0009) -[2023-10-14 13:55:35,354][75949] Updated weights for policy 0, policy_version 12760 (0.0007) -[2023-10-14 13:55:35,549][75950] Updated weights for policy 1, policy_version 12740 (0.0009) -[2023-10-14 13:55:35,903][75950] Updated weights for policy 1, policy_version 12750 (0.0011) -[2023-10-14 13:55:36,271][75950] Updated weights for policy 1, policy_version 12760 (0.0010) -[2023-10-14 13:55:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 26148864. Throughput: 0: 1675.5, 1: 1649.6. Samples: 6542852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:55:38,165][74987] Avg episode reward: [(0, '10.760'), (1, '18.360')] -[2023-10-14 13:55:39,452][75949] Updated weights for policy 0, policy_version 12770 (0.0009) -[2023-10-14 13:55:39,820][75949] Updated weights for policy 0, policy_version 12780 (0.0008) -[2023-10-14 13:55:40,198][75949] Updated weights for policy 0, policy_version 12790 (0.0009) -[2023-10-14 13:55:40,520][75950] Updated weights for policy 1, policy_version 12770 (0.0009) -[2023-10-14 13:55:40,569][75949] Updated weights for policy 0, policy_version 12800 (0.0008) -[2023-10-14 13:55:40,889][75950] Updated weights for policy 1, policy_version 12780 (0.0009) -[2023-10-14 13:55:41,252][75950] Updated weights for policy 1, policy_version 12790 (0.0009) -[2023-10-14 13:55:41,616][75950] Updated weights for policy 1, policy_version 12800 (0.0009) -[2023-10-14 13:55:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 26214400. Throughput: 0: 1682.3, 1: 1668.0. Samples: 6563402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:55:43,165][74987] Avg episode reward: [(0, '10.940'), (1, '20.520')] -[2023-10-14 13:55:43,174][75801] Saving new best policy, reward=20.520! -[2023-10-14 13:55:44,589][75949] Updated weights for policy 0, policy_version 12810 (0.0008) -[2023-10-14 13:55:44,956][75949] Updated weights for policy 0, policy_version 12820 (0.0007) -[2023-10-14 13:55:45,335][75949] Updated weights for policy 0, policy_version 12830 (0.0008) -[2023-10-14 13:55:45,758][75950] Updated weights for policy 1, policy_version 12810 (0.0009) -[2023-10-14 13:55:46,128][75950] Updated weights for policy 1, policy_version 12820 (0.0009) -[2023-10-14 13:55:46,493][75950] Updated weights for policy 1, policy_version 12830 (0.0009) -[2023-10-14 13:55:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 26279936. Throughput: 0: 1655.2, 1: 1658.6. Samples: 6573366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:55:48,165][74987] Avg episode reward: [(0, '11.170'), (1, '17.990')] -[2023-10-14 13:55:49,496][75949] Updated weights for policy 0, policy_version 12840 (0.0007) -[2023-10-14 13:55:49,876][75949] Updated weights for policy 0, policy_version 12850 (0.0010) -[2023-10-14 13:55:50,246][75949] Updated weights for policy 0, policy_version 12860 (0.0008) -[2023-10-14 13:55:50,786][75950] Updated weights for policy 1, policy_version 12840 (0.0010) -[2023-10-14 13:55:51,156][75950] Updated weights for policy 1, policy_version 12850 (0.0009) -[2023-10-14 13:55:51,529][75950] Updated weights for policy 1, policy_version 12860 (0.0007) -[2023-10-14 13:55:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26345472. Throughput: 0: 1678.2, 1: 1655.4. Samples: 6593010. Policy #0 lag: (min: 24.0, avg: 43.3, max: 56.0) -[2023-10-14 13:55:53,164][74987] Avg episode reward: [(0, '10.670'), (1, '19.320')] -[2023-10-14 13:55:54,436][75949] Updated weights for policy 0, policy_version 12870 (0.0008) -[2023-10-14 13:55:54,819][75949] Updated weights for policy 0, policy_version 12880 (0.0009) -[2023-10-14 13:55:55,194][75949] Updated weights for policy 0, policy_version 12890 (0.0008) -[2023-10-14 13:55:55,399][75950] Updated weights for policy 1, policy_version 12870 (0.0007) -[2023-10-14 13:55:55,769][75950] Updated weights for policy 1, policy_version 12880 (0.0009) -[2023-10-14 13:55:56,141][75950] Updated weights for policy 1, policy_version 12890 (0.0009) -[2023-10-14 13:55:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 26411008. Throughput: 0: 1674.6, 1: 1667.0. Samples: 6613652. Policy #0 lag: (min: 24.0, avg: 43.3, max: 56.0) -[2023-10-14 13:55:58,165][74987] Avg episode reward: [(0, '10.600'), (1, '18.380')] -[2023-10-14 13:55:59,145][75949] Updated weights for policy 0, policy_version 12900 (0.0008) -[2023-10-14 13:55:59,511][75949] Updated weights for policy 0, policy_version 12910 (0.0008) -[2023-10-14 13:55:59,885][75949] Updated weights for policy 0, policy_version 12920 (0.0008) -[2023-10-14 13:56:00,349][75950] Updated weights for policy 1, policy_version 12900 (0.0009) -[2023-10-14 13:56:00,748][75950] Updated weights for policy 1, policy_version 12910 (0.0009) -[2023-10-14 13:56:01,113][75950] Updated weights for policy 1, policy_version 12920 (0.0010) -[2023-10-14 13:56:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 26476544. Throughput: 0: 1670.2, 1: 1658.3. Samples: 6623460. Policy #0 lag: (min: 24.0, avg: 43.3, max: 56.0) -[2023-10-14 13:56:03,165][74987] Avg episode reward: [(0, '11.150'), (1, '20.400')] -[2023-10-14 13:56:04,016][75949] Updated weights for policy 0, policy_version 12930 (0.0009) -[2023-10-14 13:56:04,387][75949] Updated weights for policy 0, policy_version 12940 (0.0011) -[2023-10-14 13:56:04,756][75949] Updated weights for policy 0, policy_version 12950 (0.0009) -[2023-10-14 13:56:05,118][75949] Updated weights for policy 0, policy_version 12960 (0.0009) -[2023-10-14 13:56:05,182][75950] Updated weights for policy 1, policy_version 12930 (0.0009) -[2023-10-14 13:56:05,545][75950] Updated weights for policy 1, policy_version 12940 (0.0010) -[2023-10-14 13:56:05,915][75950] Updated weights for policy 1, policy_version 12950 (0.0010) -[2023-10-14 13:56:06,272][75950] Updated weights for policy 1, policy_version 12960 (0.0010) -[2023-10-14 13:56:08,163][74987] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26542080. Throughput: 0: 1678.7, 1: 1659.5. Samples: 6643258. Policy #0 lag: (min: 1.0, avg: 12.2, max: 33.0) -[2023-10-14 13:56:08,164][74987] Avg episode reward: [(0, '11.300'), (1, '18.540')] -[2023-10-14 13:56:09,064][75949] Updated weights for policy 0, policy_version 12970 (0.0008) -[2023-10-14 13:56:09,434][75949] Updated weights for policy 0, policy_version 12980 (0.0011) -[2023-10-14 13:56:09,798][75949] Updated weights for policy 0, policy_version 12990 (0.0009) -[2023-10-14 13:56:10,438][75950] Updated weights for policy 1, policy_version 12970 (0.0011) -[2023-10-14 13:56:10,801][75950] Updated weights for policy 1, policy_version 12980 (0.0009) -[2023-10-14 13:56:11,175][75950] Updated weights for policy 1, policy_version 12990 (0.0009) -[2023-10-14 13:56:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26607616. Throughput: 0: 1679.0, 1: 1665.6. Samples: 6663884. Policy #0 lag: (min: 1.0, avg: 12.2, max: 33.0) -[2023-10-14 13:56:13,164][74987] Avg episode reward: [(0, '11.000'), (1, '18.360')] -[2023-10-14 13:56:13,793][75949] Updated weights for policy 0, policy_version 13000 (0.0009) -[2023-10-14 13:56:14,172][75949] Updated weights for policy 0, policy_version 13010 (0.0009) -[2023-10-14 13:56:14,542][75949] Updated weights for policy 0, policy_version 13020 (0.0008) -[2023-10-14 13:56:15,046][75950] Updated weights for policy 1, policy_version 13000 (0.0010) -[2023-10-14 13:56:15,405][75950] Updated weights for policy 1, policy_version 13010 (0.0009) -[2023-10-14 13:56:15,782][75950] Updated weights for policy 1, policy_version 13020 (0.0011) -[2023-10-14 13:56:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26673152. Throughput: 0: 1676.4, 1: 1650.0. Samples: 6673382. Policy #0 lag: (min: 1.0, avg: 12.2, max: 33.0) -[2023-10-14 13:56:18,165][74987] Avg episode reward: [(0, '10.840'), (1, '17.470')] -[2023-10-14 13:56:18,785][75949] Updated weights for policy 0, policy_version 13030 (0.0009) -[2023-10-14 13:56:19,156][75949] Updated weights for policy 0, policy_version 13040 (0.0011) -[2023-10-14 13:56:19,520][75949] Updated weights for policy 0, policy_version 13050 (0.0010) -[2023-10-14 13:56:20,034][75950] Updated weights for policy 1, policy_version 13030 (0.0009) -[2023-10-14 13:56:20,408][75950] Updated weights for policy 1, policy_version 13040 (0.0009) -[2023-10-14 13:56:20,769][75950] Updated weights for policy 1, policy_version 13050 (0.0010) -[2023-10-14 13:56:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 26738688. Throughput: 0: 1677.1, 1: 1668.3. Samples: 6693396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:56:23,165][74987] Avg episode reward: [(0, '11.380'), (1, '18.340')] -[2023-10-14 13:56:23,520][75949] Updated weights for policy 0, policy_version 13060 (0.0010) -[2023-10-14 13:56:23,891][75949] Updated weights for policy 0, policy_version 13070 (0.0010) -[2023-10-14 13:56:24,269][75949] Updated weights for policy 0, policy_version 13080 (0.0011) -[2023-10-14 13:56:24,991][75950] Updated weights for policy 1, policy_version 13060 (0.0009) -[2023-10-14 13:56:25,365][75950] Updated weights for policy 1, policy_version 13070 (0.0008) -[2023-10-14 13:56:25,731][75950] Updated weights for policy 1, policy_version 13080 (0.0008) -[2023-10-14 13:56:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26804224. Throughput: 0: 1680.2, 1: 1666.8. Samples: 6714018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:56:28,165][74987] Avg episode reward: [(0, '11.190'), (1, '18.920')] -[2023-10-14 13:56:28,405][75949] Updated weights for policy 0, policy_version 13090 (0.0010) -[2023-10-14 13:56:28,768][75949] Updated weights for policy 0, policy_version 13100 (0.0010) -[2023-10-14 13:56:29,141][75949] Updated weights for policy 0, policy_version 13110 (0.0008) -[2023-10-14 13:56:29,510][75949] Updated weights for policy 0, policy_version 13120 (0.0009) -[2023-10-14 13:56:29,757][75950] Updated weights for policy 1, policy_version 13090 (0.0011) -[2023-10-14 13:56:30,127][75950] Updated weights for policy 1, policy_version 13100 (0.0010) -[2023-10-14 13:56:30,502][75950] Updated weights for policy 1, policy_version 13110 (0.0009) -[2023-10-14 13:56:30,874][75950] Updated weights for policy 1, policy_version 13120 (0.0009) -[2023-10-14 13:56:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26869760. Throughput: 0: 1681.9, 1: 1653.2. Samples: 6723446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:56:33,164][74987] Avg episode reward: [(0, '10.550'), (1, '17.760')] -[2023-10-14 13:56:33,664][75949] Updated weights for policy 0, policy_version 13130 (0.0009) -[2023-10-14 13:56:34,034][75949] Updated weights for policy 0, policy_version 13140 (0.0009) -[2023-10-14 13:56:34,404][75949] Updated weights for policy 0, policy_version 13150 (0.0008) -[2023-10-14 13:56:34,982][75950] Updated weights for policy 1, policy_version 13130 (0.0008) -[2023-10-14 13:56:35,355][75950] Updated weights for policy 1, policy_version 13140 (0.0008) -[2023-10-14 13:56:35,715][75950] Updated weights for policy 1, policy_version 13150 (0.0010) -[2023-10-14 13:56:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26935296. Throughput: 0: 1679.0, 1: 1670.9. Samples: 6743758. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 13:56:38,164][74987] Avg episode reward: [(0, '11.010'), (1, '18.480')] -[2023-10-14 13:56:38,567][75949] Updated weights for policy 0, policy_version 13160 (0.0009) -[2023-10-14 13:56:38,935][75949] Updated weights for policy 0, policy_version 13170 (0.0009) -[2023-10-14 13:56:39,311][75949] Updated weights for policy 0, policy_version 13180 (0.0007) -[2023-10-14 13:56:39,554][75950] Updated weights for policy 1, policy_version 13160 (0.0009) -[2023-10-14 13:56:39,924][75950] Updated weights for policy 1, policy_version 13170 (0.0009) -[2023-10-14 13:56:40,300][75950] Updated weights for policy 1, policy_version 13180 (0.0010) -[2023-10-14 13:56:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 27000832. Throughput: 0: 1682.2, 1: 1675.1. Samples: 6764730. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 13:56:43,164][74987] Avg episode reward: [(0, '10.890'), (1, '17.440')] -[2023-10-14 13:56:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000013184_13500416.pth... -[2023-10-14 13:56:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000013184_13500416.pth... -[2023-10-14 13:56:43,212][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000011648_11927552.pth -[2023-10-14 13:56:43,212][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000011616_11894784.pth -[2023-10-14 13:56:43,641][75949] Updated weights for policy 0, policy_version 13190 (0.0008) -[2023-10-14 13:56:44,028][75949] Updated weights for policy 0, policy_version 13200 (0.0009) -[2023-10-14 13:56:44,376][75950] Updated weights for policy 1, policy_version 13190 (0.0008) -[2023-10-14 13:56:44,397][75949] Updated weights for policy 0, policy_version 13210 (0.0010) -[2023-10-14 13:56:44,744][75950] Updated weights for policy 1, policy_version 13200 (0.0007) -[2023-10-14 13:56:45,113][75950] Updated weights for policy 1, policy_version 13210 (0.0008) -[2023-10-14 13:56:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 27066368. Throughput: 0: 1678.4, 1: 1660.3. Samples: 6773702. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 13:56:48,165][74987] Avg episode reward: [(0, '10.600'), (1, '18.730')] -[2023-10-14 13:56:48,317][75949] Updated weights for policy 0, policy_version 13220 (0.0008) -[2023-10-14 13:56:48,680][75949] Updated weights for policy 0, policy_version 13230 (0.0010) -[2023-10-14 13:56:49,050][75949] Updated weights for policy 0, policy_version 13240 (0.0010) -[2023-10-14 13:56:49,294][75950] Updated weights for policy 1, policy_version 13220 (0.0007) -[2023-10-14 13:56:49,661][75950] Updated weights for policy 1, policy_version 13230 (0.0009) -[2023-10-14 13:56:50,028][75950] Updated weights for policy 1, policy_version 13240 (0.0007) -[2023-10-14 13:56:53,041][75949] Updated weights for policy 0, policy_version 13250 (0.0007) -[2023-10-14 13:56:53,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 27131904. Throughput: 0: 1679.4, 1: 1678.5. Samples: 6794362. Policy #0 lag: (min: 14.0, avg: 38.4, max: 40.0) -[2023-10-14 13:56:53,164][74987] Avg episode reward: [(0, '10.860'), (1, '19.460')] -[2023-10-14 13:56:53,411][75949] Updated weights for policy 0, policy_version 13260 (0.0007) -[2023-10-14 13:56:53,789][75949] Updated weights for policy 0, policy_version 13270 (0.0008) -[2023-10-14 13:56:54,157][75949] Updated weights for policy 0, policy_version 13280 (0.0009) -[2023-10-14 13:56:54,284][75950] Updated weights for policy 1, policy_version 13250 (0.0008) -[2023-10-14 13:56:54,656][75950] Updated weights for policy 1, policy_version 13260 (0.0008) -[2023-10-14 13:56:55,021][75950] Updated weights for policy 1, policy_version 13270 (0.0008) -[2023-10-14 13:56:55,393][75950] Updated weights for policy 1, policy_version 13280 (0.0007) -[2023-10-14 13:56:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 27197440. Throughput: 0: 1677.6, 1: 1679.3. Samples: 6814944. Policy #0 lag: (min: 14.0, avg: 38.4, max: 40.0) -[2023-10-14 13:56:58,164][74987] Avg episode reward: [(0, '11.040'), (1, '18.090')] -[2023-10-14 13:56:58,314][75949] Updated weights for policy 0, policy_version 13290 (0.0009) -[2023-10-14 13:56:58,683][75949] Updated weights for policy 0, policy_version 13300 (0.0009) -[2023-10-14 13:56:59,071][75949] Updated weights for policy 0, policy_version 13310 (0.0009) -[2023-10-14 13:56:59,415][75950] Updated weights for policy 1, policy_version 13290 (0.0008) -[2023-10-14 13:56:59,777][75950] Updated weights for policy 1, policy_version 13300 (0.0009) -[2023-10-14 13:57:00,149][75950] Updated weights for policy 1, policy_version 13310 (0.0009) -[2023-10-14 13:57:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 27262976. Throughput: 0: 1675.4, 1: 1671.7. Samples: 6824000. Policy #0 lag: (min: 14.0, avg: 38.4, max: 40.0) -[2023-10-14 13:57:03,165][74987] Avg episode reward: [(0, '11.020'), (1, '18.690')] -[2023-10-14 13:57:03,207][75949] Updated weights for policy 0, policy_version 13320 (0.0010) -[2023-10-14 13:57:03,585][75949] Updated weights for policy 0, policy_version 13330 (0.0009) -[2023-10-14 13:57:03,951][75949] Updated weights for policy 0, policy_version 13340 (0.0009) -[2023-10-14 13:57:04,252][75950] Updated weights for policy 1, policy_version 13320 (0.0010) -[2023-10-14 13:57:04,632][75950] Updated weights for policy 1, policy_version 13330 (0.0010) -[2023-10-14 13:57:04,998][75950] Updated weights for policy 1, policy_version 13340 (0.0009) -[2023-10-14 13:57:07,955][75949] Updated weights for policy 0, policy_version 13350 (0.0008) -[2023-10-14 13:57:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 27328512. Throughput: 0: 1678.8, 1: 1680.8. Samples: 6844578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:57:08,165][74987] Avg episode reward: [(0, '10.810'), (1, '18.690')] -[2023-10-14 13:57:08,334][75949] Updated weights for policy 0, policy_version 13360 (0.0007) -[2023-10-14 13:57:08,713][75949] Updated weights for policy 0, policy_version 13370 (0.0008) -[2023-10-14 13:57:09,062][75950] Updated weights for policy 1, policy_version 13350 (0.0008) -[2023-10-14 13:57:09,428][75950] Updated weights for policy 1, policy_version 13360 (0.0008) -[2023-10-14 13:57:09,798][75950] Updated weights for policy 1, policy_version 13370 (0.0008) -[2023-10-14 13:57:12,690][75949] Updated weights for policy 0, policy_version 13380 (0.0009) -[2023-10-14 13:57:13,074][75949] Updated weights for policy 0, policy_version 13390 (0.0009) -[2023-10-14 13:57:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 27394048. Throughput: 0: 1675.7, 1: 1682.4. Samples: 6865134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:57:13,164][74987] Avg episode reward: [(0, '11.160'), (1, '19.460')] -[2023-10-14 13:57:13,440][75949] Updated weights for policy 0, policy_version 13400 (0.0009) -[2023-10-14 13:57:13,785][75950] Updated weights for policy 1, policy_version 13380 (0.0008) -[2023-10-14 13:57:14,157][75950] Updated weights for policy 1, policy_version 13390 (0.0007) -[2023-10-14 13:57:14,527][75950] Updated weights for policy 1, policy_version 13400 (0.0008) -[2023-10-14 13:57:17,503][75949] Updated weights for policy 0, policy_version 13410 (0.0008) -[2023-10-14 13:57:17,879][75949] Updated weights for policy 0, policy_version 13420 (0.0010) -[2023-10-14 13:57:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 27459584. Throughput: 0: 1678.3, 1: 1675.0. Samples: 6874346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:57:18,165][74987] Avg episode reward: [(0, '11.200'), (1, '19.320')] -[2023-10-14 13:57:18,260][75949] Updated weights for policy 0, policy_version 13430 (0.0011) -[2023-10-14 13:57:18,623][75950] Updated weights for policy 1, policy_version 13410 (0.0008) -[2023-10-14 13:57:18,624][75949] Updated weights for policy 0, policy_version 13440 (0.0008) -[2023-10-14 13:57:18,997][75950] Updated weights for policy 1, policy_version 13420 (0.0008) -[2023-10-14 13:57:19,359][75950] Updated weights for policy 1, policy_version 13430 (0.0008) -[2023-10-14 13:57:19,728][75950] Updated weights for policy 1, policy_version 13440 (0.0008) -[2023-10-14 13:57:22,830][75949] Updated weights for policy 0, policy_version 13450 (0.0009) -[2023-10-14 13:57:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 27525120. Throughput: 0: 1678.6, 1: 1673.3. Samples: 6894594. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 13:57:23,165][74987] Avg episode reward: [(0, '10.990'), (1, '18.900')] -[2023-10-14 13:57:23,211][75949] Updated weights for policy 0, policy_version 13460 (0.0007) -[2023-10-14 13:57:23,582][75949] Updated weights for policy 0, policy_version 13470 (0.0008) -[2023-10-14 13:57:23,853][75950] Updated weights for policy 1, policy_version 13450 (0.0010) -[2023-10-14 13:57:24,225][75950] Updated weights for policy 1, policy_version 13460 (0.0009) -[2023-10-14 13:57:24,597][75950] Updated weights for policy 1, policy_version 13470 (0.0011) -[2023-10-14 13:57:27,838][75949] Updated weights for policy 0, policy_version 13480 (0.0009) -[2023-10-14 13:57:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 27590656. Throughput: 0: 1670.1, 1: 1662.6. Samples: 6914700. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 13:57:28,164][74987] Avg episode reward: [(0, '10.920'), (1, '18.200')] -[2023-10-14 13:57:28,218][75949] Updated weights for policy 0, policy_version 13490 (0.0010) -[2023-10-14 13:57:28,586][75949] Updated weights for policy 0, policy_version 13500 (0.0009) -[2023-10-14 13:57:28,787][75950] Updated weights for policy 1, policy_version 13480 (0.0008) -[2023-10-14 13:57:29,156][75950] Updated weights for policy 1, policy_version 13490 (0.0008) -[2023-10-14 13:57:29,519][75950] Updated weights for policy 1, policy_version 13500 (0.0010) -[2023-10-14 13:57:32,618][75949] Updated weights for policy 0, policy_version 13510 (0.0007) -[2023-10-14 13:57:32,997][75949] Updated weights for policy 0, policy_version 13520 (0.0011) -[2023-10-14 13:57:33,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 27656192. Throughput: 0: 1673.1, 1: 1664.1. Samples: 6923872. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 13:57:33,164][74987] Avg episode reward: [(0, '11.310'), (1, '19.570')] -[2023-10-14 13:57:33,380][75949] Updated weights for policy 0, policy_version 13530 (0.0008) -[2023-10-14 13:57:33,699][75950] Updated weights for policy 1, policy_version 13510 (0.0009) -[2023-10-14 13:57:34,071][75950] Updated weights for policy 1, policy_version 13520 (0.0008) -[2023-10-14 13:57:34,435][75950] Updated weights for policy 1, policy_version 13530 (0.0009) -[2023-10-14 13:57:37,348][75949] Updated weights for policy 0, policy_version 13540 (0.0009) -[2023-10-14 13:57:37,723][75949] Updated weights for policy 0, policy_version 13550 (0.0008) -[2023-10-14 13:57:38,090][75949] Updated weights for policy 0, policy_version 13560 (0.0008) -[2023-10-14 13:57:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 27721728. Throughput: 0: 1670.7, 1: 1666.3. Samples: 6944524. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 13:57:38,164][74987] Avg episode reward: [(0, '10.750'), (1, '18.270')] -[2023-10-14 13:57:38,487][75950] Updated weights for policy 1, policy_version 13540 (0.0008) -[2023-10-14 13:57:38,875][75950] Updated weights for policy 1, policy_version 13550 (0.0009) -[2023-10-14 13:57:39,242][75950] Updated weights for policy 1, policy_version 13560 (0.0009) -[2023-10-14 13:57:42,142][75949] Updated weights for policy 0, policy_version 13570 (0.0007) -[2023-10-14 13:57:42,508][75949] Updated weights for policy 0, policy_version 13580 (0.0007) -[2023-10-14 13:57:42,886][75949] Updated weights for policy 0, policy_version 13590 (0.0009) -[2023-10-14 13:57:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 27787264. Throughput: 0: 1659.5, 1: 1664.2. Samples: 6964508. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 13:57:43,165][74987] Avg episode reward: [(0, '11.000'), (1, '20.170')] -[2023-10-14 13:57:43,247][75949] Updated weights for policy 0, policy_version 13600 (0.0010) -[2023-10-14 13:57:43,257][75950] Updated weights for policy 1, policy_version 13570 (0.0010) -[2023-10-14 13:57:43,622][75950] Updated weights for policy 1, policy_version 13580 (0.0009) -[2023-10-14 13:57:43,987][75950] Updated weights for policy 1, policy_version 13590 (0.0008) -[2023-10-14 13:57:44,359][75950] Updated weights for policy 1, policy_version 13600 (0.0010) -[2023-10-14 13:57:47,220][75949] Updated weights for policy 0, policy_version 13610 (0.0008) -[2023-10-14 13:57:47,598][75949] Updated weights for policy 0, policy_version 13620 (0.0009) -[2023-10-14 13:57:47,973][75949] Updated weights for policy 0, policy_version 13630 (0.0008) -[2023-10-14 13:57:48,164][74987] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27885568. Throughput: 0: 1676.8, 1: 1661.7. Samples: 6974234. Policy #0 lag: (min: 13.0, avg: 27.3, max: 45.0) -[2023-10-14 13:57:48,165][74987] Avg episode reward: [(0, '11.520'), (1, '17.850')] -[2023-10-14 13:57:48,689][75950] Updated weights for policy 1, policy_version 13610 (0.0009) -[2023-10-14 13:57:49,049][75950] Updated weights for policy 1, policy_version 13620 (0.0009) -[2023-10-14 13:57:49,422][75950] Updated weights for policy 1, policy_version 13630 (0.0008) -[2023-10-14 13:57:52,006][75949] Updated weights for policy 0, policy_version 13640 (0.0008) -[2023-10-14 13:57:52,376][75949] Updated weights for policy 0, policy_version 13650 (0.0008) -[2023-10-14 13:57:52,758][75949] Updated weights for policy 0, policy_version 13660 (0.0010) -[2023-10-14 13:57:53,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 27951104. Throughput: 0: 1675.7, 1: 1659.4. Samples: 6994656. Policy #0 lag: (min: 13.0, avg: 27.3, max: 45.0) -[2023-10-14 13:57:53,164][74987] Avg episode reward: [(0, '10.600'), (1, '19.340')] -[2023-10-14 13:57:53,347][75950] Updated weights for policy 1, policy_version 13640 (0.0007) -[2023-10-14 13:57:53,723][75950] Updated weights for policy 1, policy_version 13650 (0.0007) -[2023-10-14 13:57:54,083][75950] Updated weights for policy 1, policy_version 13660 (0.0009) -[2023-10-14 13:57:56,877][75949] Updated weights for policy 0, policy_version 13670 (0.0009) -[2023-10-14 13:57:57,254][75949] Updated weights for policy 0, policy_version 13680 (0.0010) -[2023-10-14 13:57:57,613][75949] Updated weights for policy 0, policy_version 13690 (0.0010) -[2023-10-14 13:57:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 28016640. Throughput: 0: 1650.5, 1: 1658.9. Samples: 7014060. Policy #0 lag: (min: 13.0, avg: 27.3, max: 45.0) -[2023-10-14 13:57:58,164][74987] Avg episode reward: [(0, '10.910'), (1, '20.940')] -[2023-10-14 13:57:58,249][75950] Updated weights for policy 1, policy_version 13670 (0.0007) -[2023-10-14 13:57:58,621][75950] Updated weights for policy 1, policy_version 13680 (0.0010) -[2023-10-14 13:57:58,989][75950] Updated weights for policy 1, policy_version 13690 (0.0010) -[2023-10-14 13:57:59,210][75801] Saving new best policy, reward=20.940! -[2023-10-14 13:58:01,804][75949] Updated weights for policy 0, policy_version 13700 (0.0011) -[2023-10-14 13:58:02,175][75949] Updated weights for policy 0, policy_version 13710 (0.0010) -[2023-10-14 13:58:02,544][75949] Updated weights for policy 0, policy_version 13720 (0.0011) -[2023-10-14 13:58:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 28082176. Throughput: 0: 1676.0, 1: 1654.6. Samples: 7024220. Policy #0 lag: (min: 2.0, avg: 2.1, max: 9.0) -[2023-10-14 13:58:03,165][74987] Avg episode reward: [(0, '10.890'), (1, '17.910')] -[2023-10-14 13:58:03,190][75950] Updated weights for policy 1, policy_version 13700 (0.0009) -[2023-10-14 13:58:03,561][75950] Updated weights for policy 1, policy_version 13710 (0.0009) -[2023-10-14 13:58:03,929][75950] Updated weights for policy 1, policy_version 13720 (0.0008) -[2023-10-14 13:58:06,615][75949] Updated weights for policy 0, policy_version 13730 (0.0012) -[2023-10-14 13:58:06,988][75949] Updated weights for policy 0, policy_version 13740 (0.0010) -[2023-10-14 13:58:07,359][75949] Updated weights for policy 0, policy_version 13750 (0.0007) -[2023-10-14 13:58:07,730][75949] Updated weights for policy 0, policy_version 13760 (0.0007) -[2023-10-14 13:58:08,065][75950] Updated weights for policy 1, policy_version 13730 (0.0007) -[2023-10-14 13:58:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 28147712. Throughput: 0: 1672.4, 1: 1661.9. Samples: 7044636. Policy #0 lag: (min: 2.0, avg: 2.1, max: 9.0) -[2023-10-14 13:58:08,165][74987] Avg episode reward: [(0, '10.820'), (1, '18.810')] -[2023-10-14 13:58:08,445][75950] Updated weights for policy 1, policy_version 13740 (0.0008) -[2023-10-14 13:58:08,814][75950] Updated weights for policy 1, policy_version 13750 (0.0007) -[2023-10-14 13:58:09,177][75950] Updated weights for policy 1, policy_version 13760 (0.0007) -[2023-10-14 13:58:11,868][75949] Updated weights for policy 0, policy_version 13770 (0.0011) -[2023-10-14 13:58:12,244][75949] Updated weights for policy 0, policy_version 13780 (0.0009) -[2023-10-14 13:58:12,624][75949] Updated weights for policy 0, policy_version 13790 (0.0011) -[2023-10-14 13:58:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 28213248. Throughput: 0: 1654.8, 1: 1663.4. Samples: 7064020. Policy #0 lag: (min: 2.0, avg: 2.1, max: 9.0) -[2023-10-14 13:58:13,165][74987] Avg episode reward: [(0, '11.000'), (1, '17.200')] -[2023-10-14 13:58:13,265][75950] Updated weights for policy 1, policy_version 13770 (0.0010) -[2023-10-14 13:58:13,642][75950] Updated weights for policy 1, policy_version 13780 (0.0009) -[2023-10-14 13:58:14,021][75950] Updated weights for policy 1, policy_version 13790 (0.0007) -[2023-10-14 13:58:16,792][75949] Updated weights for policy 0, policy_version 13800 (0.0009) -[2023-10-14 13:58:17,168][75949] Updated weights for policy 0, policy_version 13810 (0.0010) -[2023-10-14 13:58:17,526][75949] Updated weights for policy 0, policy_version 13820 (0.0008) -[2023-10-14 13:58:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 28278784. Throughput: 0: 1680.8, 1: 1663.2. Samples: 7074350. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-14 13:58:18,164][74987] Avg episode reward: [(0, '10.680'), (1, '19.700')] -[2023-10-14 13:58:18,231][75950] Updated weights for policy 1, policy_version 13800 (0.0007) -[2023-10-14 13:58:18,596][75950] Updated weights for policy 1, policy_version 13810 (0.0008) -[2023-10-14 13:58:18,971][75950] Updated weights for policy 1, policy_version 13820 (0.0007) -[2023-10-14 13:58:21,757][75949] Updated weights for policy 0, policy_version 13830 (0.0008) -[2023-10-14 13:58:22,133][75949] Updated weights for policy 0, policy_version 13840 (0.0009) -[2023-10-14 13:58:22,505][75949] Updated weights for policy 0, policy_version 13850 (0.0007) -[2023-10-14 13:58:23,039][75950] Updated weights for policy 1, policy_version 13830 (0.0008) -[2023-10-14 13:58:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 28344320. Throughput: 0: 1670.5, 1: 1670.3. Samples: 7094860. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-14 13:58:23,164][74987] Avg episode reward: [(0, '10.660'), (1, '18.740')] -[2023-10-14 13:58:23,423][75950] Updated weights for policy 1, policy_version 13840 (0.0009) -[2023-10-14 13:58:23,793][75950] Updated weights for policy 1, policy_version 13850 (0.0010) -[2023-10-14 13:58:26,484][75949] Updated weights for policy 0, policy_version 13860 (0.0008) -[2023-10-14 13:58:26,862][75949] Updated weights for policy 0, policy_version 13870 (0.0010) -[2023-10-14 13:58:27,232][75949] Updated weights for policy 0, policy_version 13880 (0.0010) -[2023-10-14 13:58:27,736][75950] Updated weights for policy 1, policy_version 13860 (0.0011) -[2023-10-14 13:58:28,102][75950] Updated weights for policy 1, policy_version 13870 (0.0007) -[2023-10-14 13:58:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 28409856. Throughput: 0: 1660.3, 1: 1666.1. Samples: 7114192. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-10-14 13:58:28,164][74987] Avg episode reward: [(0, '11.060'), (1, '19.660')] -[2023-10-14 13:58:28,476][75950] Updated weights for policy 1, policy_version 13880 (0.0008) -[2023-10-14 13:58:31,225][75949] Updated weights for policy 0, policy_version 13890 (0.0010) -[2023-10-14 13:58:31,592][75949] Updated weights for policy 0, policy_version 13900 (0.0007) -[2023-10-14 13:58:31,957][75949] Updated weights for policy 0, policy_version 13910 (0.0008) -[2023-10-14 13:58:32,331][75949] Updated weights for policy 0, policy_version 13920 (0.0007) -[2023-10-14 13:58:32,392][75950] Updated weights for policy 1, policy_version 13890 (0.0008) -[2023-10-14 13:58:32,763][75950] Updated weights for policy 1, policy_version 13900 (0.0009) -[2023-10-14 13:58:33,127][75950] Updated weights for policy 1, policy_version 13910 (0.0010) -[2023-10-14 13:58:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 28475392. Throughput: 0: 1672.7, 1: 1673.5. Samples: 7124814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:58:33,165][74987] Avg episode reward: [(0, '11.190'), (1, '18.160')] -[2023-10-14 13:58:33,489][75950] Updated weights for policy 1, policy_version 13920 (0.0011) -[2023-10-14 13:58:36,254][75949] Updated weights for policy 0, policy_version 13930 (0.0009) -[2023-10-14 13:58:36,622][75949] Updated weights for policy 0, policy_version 13940 (0.0009) -[2023-10-14 13:58:36,992][75949] Updated weights for policy 0, policy_version 13950 (0.0007) -[2023-10-14 13:58:37,799][75950] Updated weights for policy 1, policy_version 13930 (0.0009) -[2023-10-14 13:58:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 28540928. Throughput: 0: 1655.6, 1: 1675.9. Samples: 7144574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:58:38,165][74987] Avg episode reward: [(0, '10.520'), (1, '18.930')] -[2023-10-14 13:58:38,170][75950] Updated weights for policy 1, policy_version 13940 (0.0008) -[2023-10-14 13:58:38,533][75950] Updated weights for policy 1, policy_version 13950 (0.0009) -[2023-10-14 13:58:41,077][75949] Updated weights for policy 0, policy_version 13960 (0.0008) -[2023-10-14 13:58:41,450][75949] Updated weights for policy 0, policy_version 13970 (0.0011) -[2023-10-14 13:58:41,818][75949] Updated weights for policy 0, policy_version 13980 (0.0010) -[2023-10-14 13:58:42,651][75950] Updated weights for policy 1, policy_version 13960 (0.0008) -[2023-10-14 13:58:43,022][75950] Updated weights for policy 1, policy_version 13970 (0.0010) -[2023-10-14 13:58:43,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 28606464. Throughput: 0: 1670.3, 1: 1671.4. Samples: 7164438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:58:43,164][74987] Avg episode reward: [(0, '10.750'), (1, '18.710')] -[2023-10-14 13:58:43,174][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000013984_14319616.pth... -[2023-10-14 13:58:43,207][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000012416_12713984.pth -[2023-10-14 13:58:43,391][75950] Updated weights for policy 1, policy_version 13980 (0.0010) -[2023-10-14 13:58:43,538][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000013984_14319616.pth... -[2023-10-14 13:58:43,577][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000012416_12713984.pth -[2023-10-14 13:58:45,770][75949] Updated weights for policy 0, policy_version 13990 (0.0009) -[2023-10-14 13:58:46,146][75949] Updated weights for policy 0, policy_version 14000 (0.0007) -[2023-10-14 13:58:46,515][75949] Updated weights for policy 0, policy_version 14010 (0.0011) -[2023-10-14 13:58:47,498][75950] Updated weights for policy 1, policy_version 13990 (0.0008) -[2023-10-14 13:58:47,865][75950] Updated weights for policy 1, policy_version 14000 (0.0008) -[2023-10-14 13:58:48,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 28672000. Throughput: 0: 1671.2, 1: 1679.3. Samples: 7174992. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-14 13:58:48,164][74987] Avg episode reward: [(0, '10.960'), (1, '19.630')] -[2023-10-14 13:58:48,245][75950] Updated weights for policy 1, policy_version 14010 (0.0008) -[2023-10-14 13:58:50,633][75949] Updated weights for policy 0, policy_version 14020 (0.0008) -[2023-10-14 13:58:51,012][75949] Updated weights for policy 0, policy_version 14030 (0.0009) -[2023-10-14 13:58:51,383][75949] Updated weights for policy 0, policy_version 14040 (0.0008) -[2023-10-14 13:58:52,287][75950] Updated weights for policy 1, policy_version 14020 (0.0008) -[2023-10-14 13:58:52,643][75950] Updated weights for policy 1, policy_version 14030 (0.0007) -[2023-10-14 13:58:53,008][75950] Updated weights for policy 1, policy_version 14040 (0.0008) -[2023-10-14 13:58:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 28737536. Throughput: 0: 1652.8, 1: 1680.9. Samples: 7194654. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-14 13:58:53,164][74987] Avg episode reward: [(0, '10.580'), (1, '19.460')] -[2023-10-14 13:58:55,400][75949] Updated weights for policy 0, policy_version 14050 (0.0008) -[2023-10-14 13:58:55,768][75949] Updated weights for policy 0, policy_version 14060 (0.0008) -[2023-10-14 13:58:56,139][75949] Updated weights for policy 0, policy_version 14070 (0.0010) -[2023-10-14 13:58:56,504][75949] Updated weights for policy 0, policy_version 14080 (0.0007) -[2023-10-14 13:58:57,080][75950] Updated weights for policy 1, policy_version 14050 (0.0008) -[2023-10-14 13:58:57,440][75950] Updated weights for policy 1, policy_version 14060 (0.0009) -[2023-10-14 13:58:57,807][75950] Updated weights for policy 1, policy_version 14070 (0.0008) -[2023-10-14 13:58:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 28803072. Throughput: 0: 1680.1, 1: 1672.2. Samples: 7214872. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) -[2023-10-14 13:58:58,164][74987] Avg episode reward: [(0, '11.090'), (1, '19.110')] -[2023-10-14 13:58:58,174][75950] Updated weights for policy 1, policy_version 14080 (0.0010) -[2023-10-14 13:59:00,531][75949] Updated weights for policy 0, policy_version 14090 (0.0008) -[2023-10-14 13:59:00,891][75949] Updated weights for policy 0, policy_version 14100 (0.0007) -[2023-10-14 13:59:01,264][75949] Updated weights for policy 0, policy_version 14110 (0.0010) -[2023-10-14 13:59:02,103][75950] Updated weights for policy 1, policy_version 14090 (0.0009) -[2023-10-14 13:59:02,477][75950] Updated weights for policy 1, policy_version 14100 (0.0009) -[2023-10-14 13:59:02,848][75950] Updated weights for policy 1, policy_version 14110 (0.0009) -[2023-10-14 13:59:03,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 28901376. Throughput: 0: 1671.2, 1: 1688.4. Samples: 7225530. Policy #0 lag: (min: 17.0, avg: 26.5, max: 49.0) -[2023-10-14 13:59:03,165][74987] Avg episode reward: [(0, '10.870'), (1, '19.320')] -[2023-10-14 13:59:05,584][75949] Updated weights for policy 0, policy_version 14120 (0.0009) -[2023-10-14 13:59:05,967][75949] Updated weights for policy 0, policy_version 14130 (0.0007) -[2023-10-14 13:59:06,343][75949] Updated weights for policy 0, policy_version 14140 (0.0008) -[2023-10-14 13:59:06,844][75950] Updated weights for policy 1, policy_version 14120 (0.0007) -[2023-10-14 13:59:07,215][75950] Updated weights for policy 1, policy_version 14130 (0.0008) -[2023-10-14 13:59:07,587][75950] Updated weights for policy 1, policy_version 14140 (0.0009) -[2023-10-14 13:59:08,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 28966912. Throughput: 0: 1656.9, 1: 1683.7. Samples: 7245188. Policy #0 lag: (min: 17.0, avg: 26.5, max: 49.0) -[2023-10-14 13:59:08,165][74987] Avg episode reward: [(0, '11.290'), (1, '18.340')] -[2023-10-14 13:59:10,315][75949] Updated weights for policy 0, policy_version 14150 (0.0009) -[2023-10-14 13:59:10,674][75949] Updated weights for policy 0, policy_version 14160 (0.0012) -[2023-10-14 13:59:11,047][75949] Updated weights for policy 0, policy_version 14170 (0.0009) -[2023-10-14 13:59:11,638][75950] Updated weights for policy 1, policy_version 14150 (0.0008) -[2023-10-14 13:59:12,016][75950] Updated weights for policy 1, policy_version 14160 (0.0007) -[2023-10-14 13:59:12,387][75950] Updated weights for policy 1, policy_version 14170 (0.0008) -[2023-10-14 13:59:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 29032448. Throughput: 0: 1684.7, 1: 1663.2. Samples: 7264844. Policy #0 lag: (min: 17.0, avg: 26.5, max: 49.0) -[2023-10-14 13:59:13,164][74987] Avg episode reward: [(0, '11.330'), (1, '17.720')] -[2023-10-14 13:59:15,184][75949] Updated weights for policy 0, policy_version 14180 (0.0010) -[2023-10-14 13:59:15,550][75949] Updated weights for policy 0, policy_version 14190 (0.0009) -[2023-10-14 13:59:15,922][75949] Updated weights for policy 0, policy_version 14200 (0.0010) -[2023-10-14 13:59:16,639][75950] Updated weights for policy 1, policy_version 14180 (0.0007) -[2023-10-14 13:59:17,014][75950] Updated weights for policy 1, policy_version 14190 (0.0009) -[2023-10-14 13:59:17,388][75950] Updated weights for policy 1, policy_version 14200 (0.0009) -[2023-10-14 13:59:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 29097984. Throughput: 0: 1669.8, 1: 1684.7. Samples: 7275766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:59:18,165][74987] Avg episode reward: [(0, '11.030'), (1, '17.820')] -[2023-10-14 13:59:19,972][75949] Updated weights for policy 0, policy_version 14210 (0.0012) -[2023-10-14 13:59:20,344][75949] Updated weights for policy 0, policy_version 14220 (0.0008) -[2023-10-14 13:59:20,716][75949] Updated weights for policy 0, policy_version 14230 (0.0010) -[2023-10-14 13:59:21,082][75949] Updated weights for policy 0, policy_version 14240 (0.0008) -[2023-10-14 13:59:21,287][75950] Updated weights for policy 1, policy_version 14210 (0.0008) -[2023-10-14 13:59:21,657][75950] Updated weights for policy 1, policy_version 14220 (0.0009) -[2023-10-14 13:59:22,035][75950] Updated weights for policy 1, policy_version 14230 (0.0008) -[2023-10-14 13:59:22,405][75950] Updated weights for policy 1, policy_version 14240 (0.0008) -[2023-10-14 13:59:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 29163520. Throughput: 0: 1673.5, 1: 1681.1. Samples: 7295530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:59:23,164][74987] Avg episode reward: [(0, '11.280'), (1, '18.540')] -[2023-10-14 13:59:25,235][75949] Updated weights for policy 0, policy_version 14250 (0.0008) -[2023-10-14 13:59:25,595][75949] Updated weights for policy 0, policy_version 14260 (0.0007) -[2023-10-14 13:59:25,970][75949] Updated weights for policy 0, policy_version 14270 (0.0008) -[2023-10-14 13:59:26,481][75950] Updated weights for policy 1, policy_version 14250 (0.0009) -[2023-10-14 13:59:26,841][75950] Updated weights for policy 1, policy_version 14260 (0.0010) -[2023-10-14 13:59:27,210][75950] Updated weights for policy 1, policy_version 14270 (0.0008) -[2023-10-14 13:59:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 29229056. Throughput: 0: 1686.4, 1: 1670.0. Samples: 7315478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:59:28,164][74987] Avg episode reward: [(0, '11.550'), (1, '17.930')] -[2023-10-14 13:59:29,869][75949] Updated weights for policy 0, policy_version 14280 (0.0008) -[2023-10-14 13:59:30,242][75949] Updated weights for policy 0, policy_version 14290 (0.0009) -[2023-10-14 13:59:30,617][75949] Updated weights for policy 0, policy_version 14300 (0.0008) -[2023-10-14 13:59:31,248][75950] Updated weights for policy 1, policy_version 14280 (0.0009) -[2023-10-14 13:59:31,624][75950] Updated weights for policy 1, policy_version 14290 (0.0009) -[2023-10-14 13:59:31,986][75950] Updated weights for policy 1, policy_version 14300 (0.0009) -[2023-10-14 13:59:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 29294592. Throughput: 0: 1666.8, 1: 1692.9. Samples: 7326178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:59:33,165][74987] Avg episode reward: [(0, '11.400'), (1, '19.500')] -[2023-10-14 13:59:34,803][75949] Updated weights for policy 0, policy_version 14310 (0.0010) -[2023-10-14 13:59:35,175][75949] Updated weights for policy 0, policy_version 14320 (0.0010) -[2023-10-14 13:59:35,554][75949] Updated weights for policy 0, policy_version 14330 (0.0008) -[2023-10-14 13:59:36,141][75950] Updated weights for policy 1, policy_version 14310 (0.0009) -[2023-10-14 13:59:36,513][75950] Updated weights for policy 1, policy_version 14320 (0.0010) -[2023-10-14 13:59:36,884][75950] Updated weights for policy 1, policy_version 14330 (0.0007) -[2023-10-14 13:59:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 29360128. Throughput: 0: 1684.8, 1: 1676.9. Samples: 7345930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:59:38,164][74987] Avg episode reward: [(0, '11.440'), (1, '18.740')] -[2023-10-14 13:59:39,530][75949] Updated weights for policy 0, policy_version 14340 (0.0009) -[2023-10-14 13:59:39,907][75949] Updated weights for policy 0, policy_version 14350 (0.0007) -[2023-10-14 13:59:40,284][75949] Updated weights for policy 0, policy_version 14360 (0.0008) -[2023-10-14 13:59:40,961][75950] Updated weights for policy 1, policy_version 14340 (0.0007) -[2023-10-14 13:59:41,329][75950] Updated weights for policy 1, policy_version 14350 (0.0010) -[2023-10-14 13:59:41,697][75950] Updated weights for policy 1, policy_version 14360 (0.0011) -[2023-10-14 13:59:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 29425664. Throughput: 0: 1689.8, 1: 1676.5. Samples: 7366358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 13:59:43,165][74987] Avg episode reward: [(0, '10.900'), (1, '20.200')] -[2023-10-14 13:59:44,217][75949] Updated weights for policy 0, policy_version 14370 (0.0008) -[2023-10-14 13:59:44,596][75949] Updated weights for policy 0, policy_version 14380 (0.0010) -[2023-10-14 13:59:44,967][75949] Updated weights for policy 0, policy_version 14390 (0.0008) -[2023-10-14 13:59:45,336][75949] Updated weights for policy 0, policy_version 14400 (0.0011) -[2023-10-14 13:59:45,787][75950] Updated weights for policy 1, policy_version 14370 (0.0009) -[2023-10-14 13:59:46,151][75950] Updated weights for policy 1, policy_version 14380 (0.0008) -[2023-10-14 13:59:46,519][75950] Updated weights for policy 1, policy_version 14390 (0.0008) -[2023-10-14 13:59:46,891][75950] Updated weights for policy 1, policy_version 14400 (0.0009) -[2023-10-14 13:59:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 29491200. Throughput: 0: 1674.4, 1: 1687.5. Samples: 7376814. Policy #0 lag: (min: 31.0, avg: 39.7, max: 40.0) -[2023-10-14 13:59:48,165][74987] Avg episode reward: [(0, '11.440'), (1, '18.920')] -[2023-10-14 13:59:49,486][75949] Updated weights for policy 0, policy_version 14410 (0.0010) -[2023-10-14 13:59:49,862][75949] Updated weights for policy 0, policy_version 14420 (0.0010) -[2023-10-14 13:59:50,230][75949] Updated weights for policy 0, policy_version 14430 (0.0010) -[2023-10-14 13:59:50,908][75950] Updated weights for policy 1, policy_version 14410 (0.0009) -[2023-10-14 13:59:51,279][75950] Updated weights for policy 1, policy_version 14420 (0.0009) -[2023-10-14 13:59:51,647][75950] Updated weights for policy 1, policy_version 14430 (0.0010) -[2023-10-14 13:59:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 29556736. Throughput: 0: 1692.5, 1: 1662.0. Samples: 7396142. Policy #0 lag: (min: 31.0, avg: 39.7, max: 40.0) -[2023-10-14 13:59:53,165][74987] Avg episode reward: [(0, '11.870'), (1, '19.800')] -[2023-10-14 13:59:53,166][75615] Saving new best policy, reward=11.870! -[2023-10-14 13:59:54,366][75949] Updated weights for policy 0, policy_version 14440 (0.0008) -[2023-10-14 13:59:54,752][75949] Updated weights for policy 0, policy_version 14450 (0.0008) -[2023-10-14 13:59:55,126][75949] Updated weights for policy 0, policy_version 14460 (0.0008) -[2023-10-14 13:59:55,628][75950] Updated weights for policy 1, policy_version 14440 (0.0008) -[2023-10-14 13:59:55,992][75950] Updated weights for policy 1, policy_version 14450 (0.0008) -[2023-10-14 13:59:56,370][75950] Updated weights for policy 1, policy_version 14460 (0.0009) -[2023-10-14 13:59:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 29622272. Throughput: 0: 1683.1, 1: 1686.9. Samples: 7416496. Policy #0 lag: (min: 31.0, avg: 39.7, max: 40.0) -[2023-10-14 13:59:58,165][74987] Avg episode reward: [(0, '11.090'), (1, '18.180')] -[2023-10-14 13:59:59,203][75949] Updated weights for policy 0, policy_version 14470 (0.0007) -[2023-10-14 13:59:59,572][75949] Updated weights for policy 0, policy_version 14480 (0.0008) -[2023-10-14 13:59:59,949][75949] Updated weights for policy 0, policy_version 14490 (0.0009) -[2023-10-14 14:00:00,446][75950] Updated weights for policy 1, policy_version 14470 (0.0010) -[2023-10-14 14:00:00,831][75950] Updated weights for policy 1, policy_version 14480 (0.0007) -[2023-10-14 14:00:01,198][75950] Updated weights for policy 1, policy_version 14490 (0.0010) -[2023-10-14 14:00:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 29687808. Throughput: 0: 1667.7, 1: 1677.3. Samples: 7426290. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:00:03,164][74987] Avg episode reward: [(0, '12.890'), (1, '19.550')] -[2023-10-14 14:00:03,165][75615] Saving new best policy, reward=12.890! -[2023-10-14 14:00:04,142][75949] Updated weights for policy 0, policy_version 14500 (0.0010) -[2023-10-14 14:00:04,512][75949] Updated weights for policy 0, policy_version 14510 (0.0008) -[2023-10-14 14:00:04,879][75949] Updated weights for policy 0, policy_version 14520 (0.0010) -[2023-10-14 14:00:05,312][75950] Updated weights for policy 1, policy_version 14500 (0.0009) -[2023-10-14 14:00:05,688][75950] Updated weights for policy 1, policy_version 14510 (0.0008) -[2023-10-14 14:00:06,058][75950] Updated weights for policy 1, policy_version 14520 (0.0007) -[2023-10-14 14:00:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 29753344. Throughput: 0: 1677.2, 1: 1666.9. Samples: 7446018. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:00:08,165][74987] Avg episode reward: [(0, '12.020'), (1, '19.210')] -[2023-10-14 14:00:08,998][75949] Updated weights for policy 0, policy_version 14530 (0.0008) -[2023-10-14 14:00:09,367][75949] Updated weights for policy 0, policy_version 14540 (0.0007) -[2023-10-14 14:00:09,736][75949] Updated weights for policy 0, policy_version 14550 (0.0009) -[2023-10-14 14:00:10,019][75950] Updated weights for policy 1, policy_version 14530 (0.0007) -[2023-10-14 14:00:10,108][75949] Updated weights for policy 0, policy_version 14560 (0.0009) -[2023-10-14 14:00:10,382][75950] Updated weights for policy 1, policy_version 14540 (0.0008) -[2023-10-14 14:00:10,759][75950] Updated weights for policy 1, policy_version 14550 (0.0009) -[2023-10-14 14:00:11,127][75950] Updated weights for policy 1, policy_version 14560 (0.0010) -[2023-10-14 14:00:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 29818880. Throughput: 0: 1672.0, 1: 1679.2. Samples: 7466278. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:00:13,164][74987] Avg episode reward: [(0, '11.840'), (1, '18.990')] -[2023-10-14 14:00:14,417][75949] Updated weights for policy 0, policy_version 14570 (0.0007) -[2023-10-14 14:00:14,820][75949] Updated weights for policy 0, policy_version 14581 (0.0008) -[2023-10-14 14:00:15,200][75949] Updated weights for policy 0, policy_version 14591 (0.0009) -[2023-10-14 14:00:15,434][75950] Updated weights for policy 1, policy_version 14570 (0.0009) -[2023-10-14 14:00:15,795][75950] Updated weights for policy 1, policy_version 14580 (0.0008) -[2023-10-14 14:00:16,156][75950] Updated weights for policy 1, policy_version 14590 (0.0007) -[2023-10-14 14:00:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 29884416. Throughput: 0: 1655.6, 1: 1665.2. Samples: 7475614. Policy #0 lag: (min: 23.0, avg: 26.0, max: 55.0) -[2023-10-14 14:00:18,165][74987] Avg episode reward: [(0, '12.310'), (1, '18.590')] -[2023-10-14 14:00:19,405][75949] Updated weights for policy 0, policy_version 14601 (0.0010) -[2023-10-14 14:00:19,764][75949] Updated weights for policy 0, policy_version 14611 (0.0008) -[2023-10-14 14:00:20,139][75949] Updated weights for policy 0, policy_version 14621 (0.0007) -[2023-10-14 14:00:20,283][75950] Updated weights for policy 1, policy_version 14600 (0.0008) -[2023-10-14 14:00:20,648][75950] Updated weights for policy 1, policy_version 14610 (0.0007) -[2023-10-14 14:00:21,004][75950] Updated weights for policy 1, policy_version 14620 (0.0008) -[2023-10-14 14:00:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 29949952. Throughput: 0: 1655.7, 1: 1658.5. Samples: 7495070. Policy #0 lag: (min: 23.0, avg: 26.0, max: 55.0) -[2023-10-14 14:00:23,165][74987] Avg episode reward: [(0, '12.300'), (1, '18.450')] -[2023-10-14 14:00:24,250][75949] Updated weights for policy 0, policy_version 14631 (0.0010) -[2023-10-14 14:00:24,627][75949] Updated weights for policy 0, policy_version 14641 (0.0008) -[2023-10-14 14:00:24,987][75949] Updated weights for policy 0, policy_version 14651 (0.0010) -[2023-10-14 14:00:25,029][75950] Updated weights for policy 1, policy_version 14630 (0.0008) -[2023-10-14 14:00:25,389][75950] Updated weights for policy 1, policy_version 14640 (0.0008) -[2023-10-14 14:00:25,752][75950] Updated weights for policy 1, policy_version 14650 (0.0010) -[2023-10-14 14:00:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30015488. Throughput: 0: 1649.4, 1: 1674.7. Samples: 7515942. Policy #0 lag: (min: 23.0, avg: 26.0, max: 55.0) -[2023-10-14 14:00:28,164][74987] Avg episode reward: [(0, '11.740'), (1, '18.520')] -[2023-10-14 14:00:29,141][75949] Updated weights for policy 0, policy_version 14661 (0.0008) -[2023-10-14 14:00:29,505][75949] Updated weights for policy 0, policy_version 14671 (0.0010) -[2023-10-14 14:00:29,875][75949] Updated weights for policy 0, policy_version 14681 (0.0008) -[2023-10-14 14:00:29,967][75950] Updated weights for policy 1, policy_version 14660 (0.0009) -[2023-10-14 14:00:30,336][75950] Updated weights for policy 1, policy_version 14670 (0.0007) -[2023-10-14 14:00:30,699][75950] Updated weights for policy 1, policy_version 14680 (0.0007) -[2023-10-14 14:00:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30081024. Throughput: 0: 1649.2, 1: 1657.2. Samples: 7525604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:00:33,165][74987] Avg episode reward: [(0, '12.360'), (1, '18.330')] -[2023-10-14 14:00:33,960][75949] Updated weights for policy 0, policy_version 14691 (0.0009) -[2023-10-14 14:00:34,342][75949] Updated weights for policy 0, policy_version 14701 (0.0008) -[2023-10-14 14:00:34,714][75949] Updated weights for policy 0, policy_version 14711 (0.0008) -[2023-10-14 14:00:34,769][75950] Updated weights for policy 1, policy_version 14690 (0.0008) -[2023-10-14 14:00:35,139][75950] Updated weights for policy 1, policy_version 14700 (0.0008) -[2023-10-14 14:00:35,515][75950] Updated weights for policy 1, policy_version 14710 (0.0010) -[2023-10-14 14:00:35,881][75950] Updated weights for policy 1, policy_version 14720 (0.0009) -[2023-10-14 14:00:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30146560. Throughput: 0: 1651.5, 1: 1668.6. Samples: 7545546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:00:38,165][74987] Avg episode reward: [(0, '12.060'), (1, '20.230')] -[2023-10-14 14:00:38,885][75949] Updated weights for policy 0, policy_version 14721 (0.0007) -[2023-10-14 14:00:39,310][75949] Updated weights for policy 0, policy_version 14731 (0.0009) -[2023-10-14 14:00:39,674][75949] Updated weights for policy 0, policy_version 14741 (0.0009) -[2023-10-14 14:00:39,875][75950] Updated weights for policy 1, policy_version 14730 (0.0008) -[2023-10-14 14:00:40,041][75949] Updated weights for policy 0, policy_version 14751 (0.0007) -[2023-10-14 14:00:40,247][75950] Updated weights for policy 1, policy_version 14740 (0.0008) -[2023-10-14 14:00:40,607][75950] Updated weights for policy 1, policy_version 14750 (0.0009) -[2023-10-14 14:00:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 30212096. Throughput: 0: 1657.6, 1: 1668.5. Samples: 7566172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:00:43,165][74987] Avg episode reward: [(0, '11.720'), (1, '18.420')] -[2023-10-14 14:00:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000014752_15106048.pth... -[2023-10-14 14:00:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000014752_15106048.pth... -[2023-10-14 14:00:43,215][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000013184_13500416.pth -[2023-10-14 14:00:43,222][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000013184_13500416.pth -[2023-10-14 14:00:43,951][75949] Updated weights for policy 0, policy_version 14761 (0.0008) -[2023-10-14 14:00:44,328][75949] Updated weights for policy 0, policy_version 14771 (0.0008) -[2023-10-14 14:00:44,692][75949] Updated weights for policy 0, policy_version 14781 (0.0007) -[2023-10-14 14:00:44,799][75950] Updated weights for policy 1, policy_version 14760 (0.0009) -[2023-10-14 14:00:45,160][75950] Updated weights for policy 1, policy_version 14770 (0.0010) -[2023-10-14 14:00:45,544][75950] Updated weights for policy 1, policy_version 14780 (0.0010) -[2023-10-14 14:00:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30277632. Throughput: 0: 1660.1, 1: 1653.1. Samples: 7575384. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 14:00:48,164][74987] Avg episode reward: [(0, '12.980'), (1, '17.710')] -[2023-10-14 14:00:48,165][75615] Saving new best policy, reward=12.980! -[2023-10-14 14:00:48,934][75949] Updated weights for policy 0, policy_version 14791 (0.0008) -[2023-10-14 14:00:49,306][75949] Updated weights for policy 0, policy_version 14801 (0.0008) -[2023-10-14 14:00:49,678][75949] Updated weights for policy 0, policy_version 14811 (0.0008) -[2023-10-14 14:00:49,770][75950] Updated weights for policy 1, policy_version 14790 (0.0008) -[2023-10-14 14:00:50,143][75950] Updated weights for policy 1, policy_version 14800 (0.0010) -[2023-10-14 14:00:50,513][75950] Updated weights for policy 1, policy_version 14810 (0.0010) -[2023-10-14 14:00:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30343168. Throughput: 0: 1659.3, 1: 1665.5. Samples: 7595636. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 14:00:53,164][74987] Avg episode reward: [(0, '12.510'), (1, '18.140')] -[2023-10-14 14:00:53,703][75949] Updated weights for policy 0, policy_version 14821 (0.0008) -[2023-10-14 14:00:54,077][75949] Updated weights for policy 0, policy_version 14831 (0.0009) -[2023-10-14 14:00:54,454][75949] Updated weights for policy 0, policy_version 14841 (0.0008) -[2023-10-14 14:00:54,798][75950] Updated weights for policy 1, policy_version 14820 (0.0008) -[2023-10-14 14:00:55,194][75950] Updated weights for policy 1, policy_version 14830 (0.0010) -[2023-10-14 14:00:55,562][75950] Updated weights for policy 1, policy_version 14840 (0.0008) -[2023-10-14 14:00:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30408704. Throughput: 0: 1663.4, 1: 1661.3. Samples: 7615890. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 14:00:58,165][74987] Avg episode reward: [(0, '12.240'), (1, '18.970')] -[2023-10-14 14:00:58,613][75949] Updated weights for policy 0, policy_version 14851 (0.0009) -[2023-10-14 14:00:58,988][75949] Updated weights for policy 0, policy_version 14861 (0.0008) -[2023-10-14 14:00:59,365][75949] Updated weights for policy 0, policy_version 14871 (0.0007) -[2023-10-14 14:00:59,603][75950] Updated weights for policy 1, policy_version 14850 (0.0009) -[2023-10-14 14:00:59,972][75950] Updated weights for policy 1, policy_version 14860 (0.0007) -[2023-10-14 14:01:00,344][75950] Updated weights for policy 1, policy_version 14870 (0.0009) -[2023-10-14 14:01:00,710][75950] Updated weights for policy 1, policy_version 14880 (0.0009) -[2023-10-14 14:01:03,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 30474240. Throughput: 0: 1673.3, 1: 1653.0. Samples: 7625300. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 14:01:03,165][74987] Avg episode reward: [(0, '12.140'), (1, '19.530')] -[2023-10-14 14:01:03,432][75949] Updated weights for policy 0, policy_version 14881 (0.0007) -[2023-10-14 14:01:03,803][75949] Updated weights for policy 0, policy_version 14891 (0.0009) -[2023-10-14 14:01:04,187][75949] Updated weights for policy 0, policy_version 14901 (0.0010) -[2023-10-14 14:01:04,560][75949] Updated weights for policy 0, policy_version 14911 (0.0009) -[2023-10-14 14:01:04,815][75950] Updated weights for policy 1, policy_version 14890 (0.0009) -[2023-10-14 14:01:05,191][75950] Updated weights for policy 1, policy_version 14900 (0.0008) -[2023-10-14 14:01:05,567][75950] Updated weights for policy 1, policy_version 14910 (0.0007) -[2023-10-14 14:01:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 30539776. Throughput: 0: 1674.6, 1: 1670.8. Samples: 7645614. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 14:01:08,165][74987] Avg episode reward: [(0, '12.120'), (1, '19.310')] -[2023-10-14 14:01:08,527][75949] Updated weights for policy 0, policy_version 14921 (0.0010) -[2023-10-14 14:01:08,889][75949] Updated weights for policy 0, policy_version 14931 (0.0009) -[2023-10-14 14:01:09,264][75949] Updated weights for policy 0, policy_version 14941 (0.0009) -[2023-10-14 14:01:09,491][75950] Updated weights for policy 1, policy_version 14920 (0.0007) -[2023-10-14 14:01:09,864][75950] Updated weights for policy 1, policy_version 14930 (0.0007) -[2023-10-14 14:01:10,225][75950] Updated weights for policy 1, policy_version 14940 (0.0009) -[2023-10-14 14:01:13,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30605312. Throughput: 0: 1674.4, 1: 1665.2. Samples: 7666224. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 14:01:13,164][74987] Avg episode reward: [(0, '13.350'), (1, '19.640')] -[2023-10-14 14:01:13,363][75949] Updated weights for policy 0, policy_version 14951 (0.0010) -[2023-10-14 14:01:13,738][75949] Updated weights for policy 0, policy_version 14961 (0.0009) -[2023-10-14 14:01:14,112][75949] Updated weights for policy 0, policy_version 14971 (0.0010) -[2023-10-14 14:01:14,289][75615] Saving new best policy, reward=13.350! -[2023-10-14 14:01:14,401][75950] Updated weights for policy 1, policy_version 14950 (0.0008) -[2023-10-14 14:01:14,770][75950] Updated weights for policy 1, policy_version 14960 (0.0008) -[2023-10-14 14:01:15,134][75950] Updated weights for policy 1, policy_version 14970 (0.0008) -[2023-10-14 14:01:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30670848. Throughput: 0: 1673.4, 1: 1658.0. Samples: 7675520. Policy #0 lag: (min: 5.0, avg: 6.0, max: 23.0) -[2023-10-14 14:01:18,165][74987] Avg episode reward: [(0, '12.940'), (1, '18.820')] -[2023-10-14 14:01:18,224][75949] Updated weights for policy 0, policy_version 14981 (0.0007) -[2023-10-14 14:01:18,608][75949] Updated weights for policy 0, policy_version 14991 (0.0009) -[2023-10-14 14:01:18,983][75949] Updated weights for policy 0, policy_version 15001 (0.0010) -[2023-10-14 14:01:19,184][75950] Updated weights for policy 1, policy_version 14980 (0.0008) -[2023-10-14 14:01:19,555][75950] Updated weights for policy 1, policy_version 14990 (0.0008) -[2023-10-14 14:01:19,919][75950] Updated weights for policy 1, policy_version 15000 (0.0009) -[2023-10-14 14:01:23,111][75949] Updated weights for policy 0, policy_version 15011 (0.0010) -[2023-10-14 14:01:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30736384. Throughput: 0: 1672.3, 1: 1671.8. Samples: 7696028. Policy #0 lag: (min: 5.0, avg: 6.0, max: 23.0) -[2023-10-14 14:01:23,164][74987] Avg episode reward: [(0, '12.690'), (1, '18.440')] -[2023-10-14 14:01:23,494][75949] Updated weights for policy 0, policy_version 15021 (0.0008) -[2023-10-14 14:01:23,816][75950] Updated weights for policy 1, policy_version 15010 (0.0009) -[2023-10-14 14:01:23,868][75949] Updated weights for policy 0, policy_version 15031 (0.0009) -[2023-10-14 14:01:24,184][75950] Updated weights for policy 1, policy_version 15020 (0.0008) -[2023-10-14 14:01:24,554][75950] Updated weights for policy 1, policy_version 15030 (0.0010) -[2023-10-14 14:01:24,921][75950] Updated weights for policy 1, policy_version 15040 (0.0009) -[2023-10-14 14:01:28,008][75949] Updated weights for policy 0, policy_version 15041 (0.0007) -[2023-10-14 14:01:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30801920. Throughput: 0: 1668.4, 1: 1677.3. Samples: 7716726. Policy #0 lag: (min: 5.0, avg: 6.0, max: 23.0) -[2023-10-14 14:01:28,164][74987] Avg episode reward: [(0, '11.800'), (1, '18.500')] -[2023-10-14 14:01:28,396][75949] Updated weights for policy 0, policy_version 15051 (0.0007) -[2023-10-14 14:01:28,765][75949] Updated weights for policy 0, policy_version 15061 (0.0008) -[2023-10-14 14:01:28,970][75950] Updated weights for policy 1, policy_version 15050 (0.0007) -[2023-10-14 14:01:29,135][75949] Updated weights for policy 0, policy_version 15071 (0.0007) -[2023-10-14 14:01:29,330][75950] Updated weights for policy 1, policy_version 15060 (0.0007) -[2023-10-14 14:01:29,697][75950] Updated weights for policy 1, policy_version 15070 (0.0010) -[2023-10-14 14:01:32,974][75949] Updated weights for policy 0, policy_version 15081 (0.0008) -[2023-10-14 14:01:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30867456. Throughput: 0: 1668.0, 1: 1675.6. Samples: 7725850. Policy #0 lag: (min: 23.0, avg: 23.0, max: 24.0) -[2023-10-14 14:01:33,164][74987] Avg episode reward: [(0, '12.790'), (1, '17.490')] -[2023-10-14 14:01:33,349][75949] Updated weights for policy 0, policy_version 15091 (0.0009) -[2023-10-14 14:01:33,721][75949] Updated weights for policy 0, policy_version 15101 (0.0009) -[2023-10-14 14:01:33,808][75950] Updated weights for policy 1, policy_version 15080 (0.0009) -[2023-10-14 14:01:34,173][75950] Updated weights for policy 1, policy_version 15090 (0.0008) -[2023-10-14 14:01:34,533][75950] Updated weights for policy 1, policy_version 15100 (0.0010) -[2023-10-14 14:01:37,885][75949] Updated weights for policy 0, policy_version 15111 (0.0008) -[2023-10-14 14:01:38,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 30932992. Throughput: 0: 1673.0, 1: 1676.4. Samples: 7746360. Policy #0 lag: (min: 23.0, avg: 23.0, max: 24.0) -[2023-10-14 14:01:38,165][74987] Avg episode reward: [(0, '12.840'), (1, '18.620')] -[2023-10-14 14:01:38,257][75949] Updated weights for policy 0, policy_version 15121 (0.0008) -[2023-10-14 14:01:38,624][75949] Updated weights for policy 0, policy_version 15131 (0.0007) -[2023-10-14 14:01:38,764][75950] Updated weights for policy 1, policy_version 15110 (0.0008) -[2023-10-14 14:01:39,128][75950] Updated weights for policy 1, policy_version 15120 (0.0009) -[2023-10-14 14:01:39,500][75950] Updated weights for policy 1, policy_version 15130 (0.0007) -[2023-10-14 14:01:42,647][75949] Updated weights for policy 0, policy_version 15141 (0.0008) -[2023-10-14 14:01:43,021][75949] Updated weights for policy 0, policy_version 15151 (0.0009) -[2023-10-14 14:01:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30998528. Throughput: 0: 1667.0, 1: 1685.6. Samples: 7766756. Policy #0 lag: (min: 23.0, avg: 23.0, max: 24.0) -[2023-10-14 14:01:43,164][74987] Avg episode reward: [(0, '12.330'), (1, '18.720')] -[2023-10-14 14:01:43,396][75949] Updated weights for policy 0, policy_version 15161 (0.0010) -[2023-10-14 14:01:43,702][75950] Updated weights for policy 1, policy_version 15140 (0.0010) -[2023-10-14 14:01:44,095][75950] Updated weights for policy 1, policy_version 15150 (0.0008) -[2023-10-14 14:01:44,456][75950] Updated weights for policy 1, policy_version 15160 (0.0008) -[2023-10-14 14:01:47,546][75949] Updated weights for policy 0, policy_version 15171 (0.0008) -[2023-10-14 14:01:47,924][75949] Updated weights for policy 0, policy_version 15181 (0.0010) -[2023-10-14 14:01:48,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31064064. Throughput: 0: 1670.4, 1: 1677.0. Samples: 7775932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:01:48,164][74987] Avg episode reward: [(0, '12.390'), (1, '18.600')] -[2023-10-14 14:01:48,293][75949] Updated weights for policy 0, policy_version 15191 (0.0009) -[2023-10-14 14:01:48,635][75950] Updated weights for policy 1, policy_version 15170 (0.0009) -[2023-10-14 14:01:48,997][75950] Updated weights for policy 1, policy_version 15180 (0.0008) -[2023-10-14 14:01:49,365][75950] Updated weights for policy 1, policy_version 15190 (0.0010) -[2023-10-14 14:01:49,739][75950] Updated weights for policy 1, policy_version 15200 (0.0009) -[2023-10-14 14:01:52,391][75949] Updated weights for policy 0, policy_version 15201 (0.0009) -[2023-10-14 14:01:52,752][75949] Updated weights for policy 0, policy_version 15211 (0.0009) -[2023-10-14 14:01:53,120][75949] Updated weights for policy 0, policy_version 15221 (0.0007) -[2023-10-14 14:01:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31129600. Throughput: 0: 1673.5, 1: 1680.6. Samples: 7796550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:01:53,164][74987] Avg episode reward: [(0, '11.390'), (1, '19.750')] -[2023-10-14 14:01:53,486][75949] Updated weights for policy 0, policy_version 15231 (0.0008) -[2023-10-14 14:01:53,582][75950] Updated weights for policy 1, policy_version 15210 (0.0008) -[2023-10-14 14:01:53,946][75950] Updated weights for policy 1, policy_version 15220 (0.0009) -[2023-10-14 14:01:54,317][75950] Updated weights for policy 1, policy_version 15230 (0.0010) -[2023-10-14 14:01:57,780][75949] Updated weights for policy 0, policy_version 15241 (0.0007) -[2023-10-14 14:01:58,156][75949] Updated weights for policy 0, policy_version 15251 (0.0007) -[2023-10-14 14:01:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 31195136. Throughput: 0: 1661.8, 1: 1678.8. Samples: 7816550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:01:58,164][74987] Avg episode reward: [(0, '12.620'), (1, '18.470')] -[2023-10-14 14:01:58,409][75950] Updated weights for policy 1, policy_version 15240 (0.0007) -[2023-10-14 14:01:58,530][75949] Updated weights for policy 0, policy_version 15261 (0.0008) -[2023-10-14 14:01:58,783][75950] Updated weights for policy 1, policy_version 15250 (0.0007) -[2023-10-14 14:01:59,141][75950] Updated weights for policy 1, policy_version 15260 (0.0009) -[2023-10-14 14:02:02,603][75949] Updated weights for policy 0, policy_version 15271 (0.0008) -[2023-10-14 14:02:02,965][75949] Updated weights for policy 0, policy_version 15281 (0.0008) -[2023-10-14 14:02:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 31260672. Throughput: 0: 1664.5, 1: 1675.4. Samples: 7825814. Policy #0 lag: (min: 31.0, avg: 32.6, max: 54.0) -[2023-10-14 14:02:03,164][75950] Updated weights for policy 1, policy_version 15270 (0.0007) -[2023-10-14 14:02:03,164][74987] Avg episode reward: [(0, '13.050'), (1, '20.710')] -[2023-10-14 14:02:03,348][75949] Updated weights for policy 0, policy_version 15291 (0.0007) -[2023-10-14 14:02:03,527][75950] Updated weights for policy 1, policy_version 15280 (0.0009) -[2023-10-14 14:02:03,902][75950] Updated weights for policy 1, policy_version 15290 (0.0008) -[2023-10-14 14:02:07,476][75949] Updated weights for policy 0, policy_version 15301 (0.0009) -[2023-10-14 14:02:07,857][75949] Updated weights for policy 0, policy_version 15311 (0.0009) -[2023-10-14 14:02:08,041][75950] Updated weights for policy 1, policy_version 15300 (0.0008) -[2023-10-14 14:02:08,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31326208. Throughput: 0: 1667.6, 1: 1672.7. Samples: 7846342. Policy #0 lag: (min: 31.0, avg: 32.6, max: 54.0) -[2023-10-14 14:02:08,164][74987] Avg episode reward: [(0, '13.030'), (1, '19.820')] -[2023-10-14 14:02:08,214][75949] Updated weights for policy 0, policy_version 15321 (0.0008) -[2023-10-14 14:02:08,406][75950] Updated weights for policy 1, policy_version 15310 (0.0008) -[2023-10-14 14:02:08,771][75950] Updated weights for policy 1, policy_version 15320 (0.0007) -[2023-10-14 14:02:12,437][75949] Updated weights for policy 0, policy_version 15331 (0.0008) -[2023-10-14 14:02:12,789][75950] Updated weights for policy 1, policy_version 15330 (0.0010) -[2023-10-14 14:02:12,831][75949] Updated weights for policy 0, policy_version 15341 (0.0008) -[2023-10-14 14:02:13,149][75950] Updated weights for policy 1, policy_version 15340 (0.0007) -[2023-10-14 14:02:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31391744. Throughput: 0: 1656.0, 1: 1671.4. Samples: 7866456. Policy #0 lag: (min: 31.0, avg: 32.6, max: 54.0) -[2023-10-14 14:02:13,164][74987] Avg episode reward: [(0, '13.120'), (1, '20.170')] -[2023-10-14 14:02:13,200][75949] Updated weights for policy 0, policy_version 15351 (0.0008) -[2023-10-14 14:02:13,515][75950] Updated weights for policy 1, policy_version 15350 (0.0007) -[2023-10-14 14:02:13,882][75950] Updated weights for policy 1, policy_version 15360 (0.0008) -[2023-10-14 14:02:17,264][75949] Updated weights for policy 0, policy_version 15361 (0.0007) -[2023-10-14 14:02:17,628][75949] Updated weights for policy 0, policy_version 15371 (0.0007) -[2023-10-14 14:02:17,998][75949] Updated weights for policy 0, policy_version 15381 (0.0009) -[2023-10-14 14:02:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31457280. Throughput: 0: 1660.1, 1: 1667.8. Samples: 7875606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:02:18,164][74987] Avg episode reward: [(0, '13.940'), (1, '19.190')] -[2023-10-14 14:02:18,215][75950] Updated weights for policy 1, policy_version 15370 (0.0007) -[2023-10-14 14:02:18,373][75949] Updated weights for policy 0, policy_version 15391 (0.0008) -[2023-10-14 14:02:18,403][75615] Saving new best policy, reward=13.940! -[2023-10-14 14:02:18,578][75950] Updated weights for policy 1, policy_version 15380 (0.0008) -[2023-10-14 14:02:18,949][75950] Updated weights for policy 1, policy_version 15390 (0.0008) -[2023-10-14 14:02:22,492][75949] Updated weights for policy 0, policy_version 15401 (0.0011) -[2023-10-14 14:02:22,875][75949] Updated weights for policy 0, policy_version 15411 (0.0009) -[2023-10-14 14:02:23,053][75950] Updated weights for policy 1, policy_version 15400 (0.0008) -[2023-10-14 14:02:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 31522816. Throughput: 0: 1657.6, 1: 1667.2. Samples: 7895978. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:02:23,165][74987] Avg episode reward: [(0, '12.650'), (1, '19.160')] -[2023-10-14 14:02:23,247][75949] Updated weights for policy 0, policy_version 15421 (0.0008) -[2023-10-14 14:02:23,424][75950] Updated weights for policy 1, policy_version 15410 (0.0007) -[2023-10-14 14:02:23,782][75950] Updated weights for policy 1, policy_version 15420 (0.0008) -[2023-10-14 14:02:27,318][75949] Updated weights for policy 0, policy_version 15431 (0.0009) -[2023-10-14 14:02:27,689][75949] Updated weights for policy 0, policy_version 15441 (0.0009) -[2023-10-14 14:02:27,792][75950] Updated weights for policy 1, policy_version 15430 (0.0008) -[2023-10-14 14:02:28,062][75949] Updated weights for policy 0, policy_version 15451 (0.0008) -[2023-10-14 14:02:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31588352. Throughput: 0: 1649.4, 1: 1665.5. Samples: 7915924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:02:28,164][74987] Avg episode reward: [(0, '12.860'), (1, '19.270')] -[2023-10-14 14:02:28,183][75950] Updated weights for policy 1, policy_version 15440 (0.0009) -[2023-10-14 14:02:28,558][75950] Updated weights for policy 1, policy_version 15450 (0.0008) -[2023-10-14 14:02:32,146][75949] Updated weights for policy 0, policy_version 15461 (0.0008) -[2023-10-14 14:02:32,515][75949] Updated weights for policy 0, policy_version 15471 (0.0010) -[2023-10-14 14:02:32,639][75950] Updated weights for policy 1, policy_version 15460 (0.0007) -[2023-10-14 14:02:32,891][75949] Updated weights for policy 0, policy_version 15481 (0.0007) -[2023-10-14 14:02:33,010][75950] Updated weights for policy 1, policy_version 15470 (0.0007) -[2023-10-14 14:02:33,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31686656. Throughput: 0: 1655.6, 1: 1669.3. Samples: 7925554. Policy #0 lag: (min: 30.0, avg: 33.8, max: 62.0) -[2023-10-14 14:02:33,165][74987] Avg episode reward: [(0, '12.900'), (1, '17.560')] -[2023-10-14 14:02:33,386][75950] Updated weights for policy 1, policy_version 15480 (0.0009) -[2023-10-14 14:02:37,020][75949] Updated weights for policy 0, policy_version 15491 (0.0008) -[2023-10-14 14:02:37,256][75950] Updated weights for policy 1, policy_version 15490 (0.0010) -[2023-10-14 14:02:37,390][75949] Updated weights for policy 0, policy_version 15501 (0.0009) -[2023-10-14 14:02:37,622][75950] Updated weights for policy 1, policy_version 15500 (0.0009) -[2023-10-14 14:02:37,769][75949] Updated weights for policy 0, policy_version 15511 (0.0008) -[2023-10-14 14:02:37,987][75950] Updated weights for policy 1, policy_version 15510 (0.0008) -[2023-10-14 14:02:38,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 31752192. Throughput: 0: 1652.1, 1: 1674.4. Samples: 7946244. Policy #0 lag: (min: 30.0, avg: 33.8, max: 62.0) -[2023-10-14 14:02:38,164][74987] Avg episode reward: [(0, '13.460'), (1, '19.930')] -[2023-10-14 14:02:38,363][75950] Updated weights for policy 1, policy_version 15520 (0.0008) -[2023-10-14 14:02:42,022][75949] Updated weights for policy 0, policy_version 15521 (0.0008) -[2023-10-14 14:02:42,372][75950] Updated weights for policy 1, policy_version 15530 (0.0007) -[2023-10-14 14:02:42,394][75949] Updated weights for policy 0, policy_version 15531 (0.0008) -[2023-10-14 14:02:42,745][75950] Updated weights for policy 1, policy_version 15540 (0.0008) -[2023-10-14 14:02:42,757][75949] Updated weights for policy 0, policy_version 15541 (0.0009) -[2023-10-14 14:02:43,113][75950] Updated weights for policy 1, policy_version 15550 (0.0008) -[2023-10-14 14:02:43,132][75949] Updated weights for policy 0, policy_version 15551 (0.0008) -[2023-10-14 14:02:43,166][74987] Fps is (10 sec: 13103.4, 60 sec: 13652.7, 300 sec: 13329.2). Total num frames: 31817728. Throughput: 0: 1640.8, 1: 1666.6. Samples: 7965394. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 14:02:43,167][74987] Avg episode reward: [(0, '13.540'), (1, '19.190')] -[2023-10-14 14:02:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000015552_15925248.pth... -[2023-10-14 14:02:43,177][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000015552_15925248.pth... -[2023-10-14 14:02:43,204][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000013984_14319616.pth -[2023-10-14 14:02:43,206][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000013984_14319616.pth -[2023-10-14 14:02:43,208][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000015552_15925248.pth -[2023-10-14 14:02:43,210][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000015552_15925248.pth -[2023-10-14 14:02:47,198][75949] Updated weights for policy 0, policy_version 15561 (0.0008) -[2023-10-14 14:02:47,237][75950] Updated weights for policy 1, policy_version 15560 (0.0007) -[2023-10-14 14:02:47,558][75949] Updated weights for policy 0, policy_version 15571 (0.0008) -[2023-10-14 14:02:47,602][75950] Updated weights for policy 1, policy_version 15570 (0.0010) -[2023-10-14 14:02:47,939][75949] Updated weights for policy 0, policy_version 15581 (0.0010) -[2023-10-14 14:02:47,972][75950] Updated weights for policy 1, policy_version 15580 (0.0009) -[2023-10-14 14:02:48,164][74987] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 31916032. Throughput: 0: 1652.2, 1: 1681.8. Samples: 7975846. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 14:02:48,165][74987] Avg episode reward: [(0, '12.990'), (1, '19.830')] -[2023-10-14 14:02:52,036][75949] Updated weights for policy 0, policy_version 15591 (0.0009) -[2023-10-14 14:02:52,127][75950] Updated weights for policy 1, policy_version 15590 (0.0009) -[2023-10-14 14:02:52,395][75949] Updated weights for policy 0, policy_version 15601 (0.0010) -[2023-10-14 14:02:52,492][75950] Updated weights for policy 1, policy_version 15600 (0.0009) -[2023-10-14 14:02:52,755][75949] Updated weights for policy 0, policy_version 15611 (0.0007) -[2023-10-14 14:02:52,852][75950] Updated weights for policy 1, policy_version 15610 (0.0009) -[2023-10-14 14:02:53,164][74987] Fps is (10 sec: 16388.5, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 31981568. Throughput: 0: 1653.6, 1: 1680.3. Samples: 7996372. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 14:02:53,165][74987] Avg episode reward: [(0, '12.140'), (1, '18.390')] -[2023-10-14 14:02:56,861][75949] Updated weights for policy 0, policy_version 15621 (0.0007) -[2023-10-14 14:02:57,030][75950] Updated weights for policy 1, policy_version 15620 (0.0008) -[2023-10-14 14:02:57,237][75949] Updated weights for policy 0, policy_version 15631 (0.0009) -[2023-10-14 14:02:57,403][75950] Updated weights for policy 1, policy_version 15630 (0.0008) -[2023-10-14 14:02:57,606][75949] Updated weights for policy 0, policy_version 15641 (0.0008) -[2023-10-14 14:02:57,766][75950] Updated weights for policy 1, policy_version 15640 (0.0008) -[2023-10-14 14:02:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 32047104. Throughput: 0: 1642.2, 1: 1657.3. Samples: 8014936. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 14:02:58,165][74987] Avg episode reward: [(0, '13.220'), (1, '19.300')] -[2023-10-14 14:03:01,861][75950] Updated weights for policy 1, policy_version 15650 (0.0009) -[2023-10-14 14:03:01,963][75949] Updated weights for policy 0, policy_version 15651 (0.0009) -[2023-10-14 14:03:02,236][75950] Updated weights for policy 1, policy_version 15660 (0.0008) -[2023-10-14 14:03:02,355][75949] Updated weights for policy 0, policy_version 15661 (0.0008) -[2023-10-14 14:03:02,603][75950] Updated weights for policy 1, policy_version 15670 (0.0007) -[2023-10-14 14:03:02,727][75949] Updated weights for policy 0, policy_version 15671 (0.0007) -[2023-10-14 14:03:02,968][75950] Updated weights for policy 1, policy_version 15680 (0.0009) -[2023-10-14 14:03:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 32112640. Throughput: 0: 1658.4, 1: 1678.1. Samples: 8025746. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 14:03:03,165][74987] Avg episode reward: [(0, '13.590'), (1, '18.130')] -[2023-10-14 14:03:06,672][75949] Updated weights for policy 0, policy_version 15681 (0.0009) -[2023-10-14 14:03:07,039][75949] Updated weights for policy 0, policy_version 15691 (0.0009) -[2023-10-14 14:03:07,206][75950] Updated weights for policy 1, policy_version 15690 (0.0008) -[2023-10-14 14:03:07,404][75949] Updated weights for policy 0, policy_version 15701 (0.0009) -[2023-10-14 14:03:07,576][75950] Updated weights for policy 1, policy_version 15700 (0.0009) -[2023-10-14 14:03:07,786][75949] Updated weights for policy 0, policy_version 15711 (0.0008) -[2023-10-14 14:03:07,934][75950] Updated weights for policy 1, policy_version 15710 (0.0010) -[2023-10-14 14:03:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 32178176. Throughput: 0: 1659.2, 1: 1679.3. Samples: 8046210. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 14:03:08,164][74987] Avg episode reward: [(0, '13.960'), (1, '19.680')] -[2023-10-14 14:03:08,165][75615] Saving new best policy, reward=13.960! -[2023-10-14 14:03:11,776][75949] Updated weights for policy 0, policy_version 15721 (0.0009) -[2023-10-14 14:03:11,919][75950] Updated weights for policy 1, policy_version 15720 (0.0007) -[2023-10-14 14:03:12,156][75949] Updated weights for policy 0, policy_version 15731 (0.0008) -[2023-10-14 14:03:12,288][75950] Updated weights for policy 1, policy_version 15730 (0.0008) -[2023-10-14 14:03:12,535][75949] Updated weights for policy 0, policy_version 15741 (0.0008) -[2023-10-14 14:03:12,655][75950] Updated weights for policy 1, policy_version 15740 (0.0007) -[2023-10-14 14:03:13,163][74987] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 32243712. Throughput: 0: 1647.8, 1: 1656.8. Samples: 8064632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:03:13,164][74987] Avg episode reward: [(0, '14.190'), (1, '19.390')] -[2023-10-14 14:03:13,171][75615] Saving new best policy, reward=14.190! -[2023-10-14 14:03:16,877][75949] Updated weights for policy 0, policy_version 15751 (0.0008) -[2023-10-14 14:03:16,884][75950] Updated weights for policy 1, policy_version 15750 (0.0008) -[2023-10-14 14:03:17,244][75949] Updated weights for policy 0, policy_version 15761 (0.0008) -[2023-10-14 14:03:17,261][75950] Updated weights for policy 1, policy_version 15760 (0.0008) -[2023-10-14 14:03:17,609][75949] Updated weights for policy 0, policy_version 15771 (0.0009) -[2023-10-14 14:03:17,627][75950] Updated weights for policy 1, policy_version 15770 (0.0008) -[2023-10-14 14:03:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 32309248. Throughput: 0: 1661.2, 1: 1678.8. Samples: 8075856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:03:18,164][74987] Avg episode reward: [(0, '14.040'), (1, '19.690')] -[2023-10-14 14:03:21,765][75949] Updated weights for policy 0, policy_version 15781 (0.0008) -[2023-10-14 14:03:21,833][75950] Updated weights for policy 1, policy_version 15780 (0.0007) -[2023-10-14 14:03:22,131][75949] Updated weights for policy 0, policy_version 15791 (0.0008) -[2023-10-14 14:03:22,193][75950] Updated weights for policy 1, policy_version 15790 (0.0007) -[2023-10-14 14:03:22,514][75949] Updated weights for policy 0, policy_version 15801 (0.0007) -[2023-10-14 14:03:22,558][75950] Updated weights for policy 1, policy_version 15800 (0.0008) -[2023-10-14 14:03:23,163][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 32374784. Throughput: 0: 1660.5, 1: 1669.2. Samples: 8096084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:03:23,164][74987] Avg episode reward: [(0, '14.150'), (1, '19.120')] -[2023-10-14 14:03:26,376][75949] Updated weights for policy 0, policy_version 15811 (0.0007) -[2023-10-14 14:03:26,749][75949] Updated weights for policy 0, policy_version 15821 (0.0008) -[2023-10-14 14:03:26,803][75950] Updated weights for policy 1, policy_version 15810 (0.0010) -[2023-10-14 14:03:27,128][75949] Updated weights for policy 0, policy_version 15831 (0.0009) -[2023-10-14 14:03:27,160][75950] Updated weights for policy 1, policy_version 15820 (0.0007) -[2023-10-14 14:03:27,537][75950] Updated weights for policy 1, policy_version 15830 (0.0007) -[2023-10-14 14:03:27,901][75950] Updated weights for policy 1, policy_version 15840 (0.0009) -[2023-10-14 14:03:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 32440320. Throughput: 0: 1659.0, 1: 1655.5. Samples: 8114540. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) -[2023-10-14 14:03:28,165][74987] Avg episode reward: [(0, '13.480'), (1, '19.950')] -[2023-10-14 14:03:31,231][75949] Updated weights for policy 0, policy_version 15841 (0.0009) -[2023-10-14 14:03:31,600][75949] Updated weights for policy 0, policy_version 15851 (0.0008) -[2023-10-14 14:03:31,968][75949] Updated weights for policy 0, policy_version 15861 (0.0008) -[2023-10-14 14:03:32,043][75950] Updated weights for policy 1, policy_version 15850 (0.0007) -[2023-10-14 14:03:32,331][75949] Updated weights for policy 0, policy_version 15871 (0.0007) -[2023-10-14 14:03:32,416][75950] Updated weights for policy 1, policy_version 15860 (0.0007) -[2023-10-14 14:03:32,780][75950] Updated weights for policy 1, policy_version 15870 (0.0007) -[2023-10-14 14:03:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 32505856. Throughput: 0: 1673.4, 1: 1657.6. Samples: 8125744. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) -[2023-10-14 14:03:33,164][74987] Avg episode reward: [(0, '14.530'), (1, '19.400')] -[2023-10-14 14:03:33,165][75615] Saving new best policy, reward=14.530! -[2023-10-14 14:03:36,371][75949] Updated weights for policy 0, policy_version 15881 (0.0008) -[2023-10-14 14:03:36,745][75949] Updated weights for policy 0, policy_version 15891 (0.0010) -[2023-10-14 14:03:37,040][75950] Updated weights for policy 1, policy_version 15880 (0.0007) -[2023-10-14 14:03:37,114][75949] Updated weights for policy 0, policy_version 15901 (0.0008) -[2023-10-14 14:03:37,399][75950] Updated weights for policy 1, policy_version 15890 (0.0007) -[2023-10-14 14:03:37,770][75950] Updated weights for policy 1, policy_version 15900 (0.0010) -[2023-10-14 14:03:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 32571392. Throughput: 0: 1660.7, 1: 1658.3. Samples: 8145728. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) -[2023-10-14 14:03:38,165][74987] Avg episode reward: [(0, '14.650'), (1, '20.580')] -[2023-10-14 14:03:38,166][75615] Saving new best policy, reward=14.650! -[2023-10-14 14:03:41,095][75949] Updated weights for policy 0, policy_version 15911 (0.0009) -[2023-10-14 14:03:41,467][75949] Updated weights for policy 0, policy_version 15921 (0.0009) -[2023-10-14 14:03:41,828][75949] Updated weights for policy 0, policy_version 15931 (0.0008) -[2023-10-14 14:03:41,883][75950] Updated weights for policy 1, policy_version 15910 (0.0008) -[2023-10-14 14:03:42,251][75950] Updated weights for policy 1, policy_version 15920 (0.0007) -[2023-10-14 14:03:42,614][75950] Updated weights for policy 1, policy_version 15930 (0.0008) -[2023-10-14 14:03:43,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13654.0, 300 sec: 13440.4). Total num frames: 32636928. Throughput: 0: 1675.8, 1: 1652.2. Samples: 8164698. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:03:43,164][74987] Avg episode reward: [(0, '14.790'), (1, '19.930')] -[2023-10-14 14:03:43,170][75615] Saving new best policy, reward=14.790! -[2023-10-14 14:03:45,719][75949] Updated weights for policy 0, policy_version 15941 (0.0008) -[2023-10-14 14:03:46,088][75949] Updated weights for policy 0, policy_version 15951 (0.0009) -[2023-10-14 14:03:46,469][75949] Updated weights for policy 0, policy_version 15961 (0.0007) -[2023-10-14 14:03:46,789][75950] Updated weights for policy 1, policy_version 15940 (0.0009) -[2023-10-14 14:03:47,163][75950] Updated weights for policy 1, policy_version 15950 (0.0010) -[2023-10-14 14:03:47,534][75950] Updated weights for policy 1, policy_version 15960 (0.0008) -[2023-10-14 14:03:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 32702464. Throughput: 0: 1680.8, 1: 1656.3. Samples: 8175918. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:03:48,165][74987] Avg episode reward: [(0, '15.180'), (1, '19.530')] -[2023-10-14 14:03:48,166][75615] Saving new best policy, reward=15.180! -[2023-10-14 14:03:50,529][75949] Updated weights for policy 0, policy_version 15971 (0.0009) -[2023-10-14 14:03:50,903][75949] Updated weights for policy 0, policy_version 15981 (0.0008) -[2023-10-14 14:03:51,279][75949] Updated weights for policy 0, policy_version 15991 (0.0010) -[2023-10-14 14:03:51,633][75950] Updated weights for policy 1, policy_version 15970 (0.0010) -[2023-10-14 14:03:52,000][75950] Updated weights for policy 1, policy_version 15980 (0.0008) -[2023-10-14 14:03:52,362][75950] Updated weights for policy 1, policy_version 15990 (0.0008) -[2023-10-14 14:03:52,738][75950] Updated weights for policy 1, policy_version 16000 (0.0008) -[2023-10-14 14:03:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 32768000. Throughput: 0: 1657.9, 1: 1656.8. Samples: 8195374. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:03:53,164][74987] Avg episode reward: [(0, '14.490'), (1, '18.860')] -[2023-10-14 14:03:55,367][75949] Updated weights for policy 0, policy_version 16001 (0.0009) -[2023-10-14 14:03:55,789][75949] Updated weights for policy 0, policy_version 16011 (0.0007) -[2023-10-14 14:03:56,159][75949] Updated weights for policy 0, policy_version 16021 (0.0008) -[2023-10-14 14:03:56,528][75949] Updated weights for policy 0, policy_version 16031 (0.0008) -[2023-10-14 14:03:56,547][75950] Updated weights for policy 1, policy_version 16010 (0.0008) -[2023-10-14 14:03:56,917][75950] Updated weights for policy 1, policy_version 16020 (0.0008) -[2023-10-14 14:03:57,285][75950] Updated weights for policy 1, policy_version 16030 (0.0009) -[2023-10-14 14:03:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 32833536. Throughput: 0: 1685.5, 1: 1656.1. Samples: 8215004. Policy #0 lag: (min: 23.0, avg: 27.5, max: 55.0) -[2023-10-14 14:03:58,164][74987] Avg episode reward: [(0, '15.530'), (1, '19.460')] -[2023-10-14 14:03:58,173][75615] Saving new best policy, reward=15.530! -[2023-10-14 14:04:00,440][75949] Updated weights for policy 0, policy_version 16041 (0.0010) -[2023-10-14 14:04:00,818][75949] Updated weights for policy 0, policy_version 16051 (0.0010) -[2023-10-14 14:04:01,189][75949] Updated weights for policy 0, policy_version 16061 (0.0009) -[2023-10-14 14:04:01,475][75950] Updated weights for policy 1, policy_version 16040 (0.0008) -[2023-10-14 14:04:01,854][75950] Updated weights for policy 1, policy_version 16050 (0.0009) -[2023-10-14 14:04:02,228][75950] Updated weights for policy 1, policy_version 16060 (0.0010) -[2023-10-14 14:04:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32899072. Throughput: 0: 1675.2, 1: 1659.5. Samples: 8225914. Policy #0 lag: (min: 23.0, avg: 27.5, max: 55.0) -[2023-10-14 14:04:03,164][74987] Avg episode reward: [(0, '13.730'), (1, '19.480')] -[2023-10-14 14:04:05,392][75949] Updated weights for policy 0, policy_version 16071 (0.0007) -[2023-10-14 14:04:05,757][75949] Updated weights for policy 0, policy_version 16081 (0.0007) -[2023-10-14 14:04:06,132][75949] Updated weights for policy 0, policy_version 16091 (0.0007) -[2023-10-14 14:04:06,341][75950] Updated weights for policy 1, policy_version 16070 (0.0009) -[2023-10-14 14:04:06,709][75950] Updated weights for policy 1, policy_version 16080 (0.0009) -[2023-10-14 14:04:07,076][75950] Updated weights for policy 1, policy_version 16090 (0.0009) -[2023-10-14 14:04:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32964608. Throughput: 0: 1666.8, 1: 1646.7. Samples: 8245194. Policy #0 lag: (min: 23.0, avg: 27.5, max: 55.0) -[2023-10-14 14:04:08,165][74987] Avg episode reward: [(0, '14.450'), (1, '21.070')] -[2023-10-14 14:04:08,166][75801] Saving new best policy, reward=21.070! -[2023-10-14 14:04:10,240][75949] Updated weights for policy 0, policy_version 16101 (0.0009) -[2023-10-14 14:04:10,605][75949] Updated weights for policy 0, policy_version 16111 (0.0009) -[2023-10-14 14:04:10,973][75949] Updated weights for policy 0, policy_version 16121 (0.0008) -[2023-10-14 14:04:11,118][75950] Updated weights for policy 1, policy_version 16100 (0.0009) -[2023-10-14 14:04:11,485][75950] Updated weights for policy 1, policy_version 16110 (0.0007) -[2023-10-14 14:04:11,850][75950] Updated weights for policy 1, policy_version 16120 (0.0007) -[2023-10-14 14:04:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 33030144. Throughput: 0: 1688.6, 1: 1662.7. Samples: 8265350. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 14:04:13,165][74987] Avg episode reward: [(0, '14.740'), (1, '20.950')] -[2023-10-14 14:04:15,021][75949] Updated weights for policy 0, policy_version 16131 (0.0009) -[2023-10-14 14:04:15,381][75949] Updated weights for policy 0, policy_version 16141 (0.0007) -[2023-10-14 14:04:15,750][75949] Updated weights for policy 0, policy_version 16151 (0.0007) -[2023-10-14 14:04:16,009][75950] Updated weights for policy 1, policy_version 16130 (0.0008) -[2023-10-14 14:04:16,375][75950] Updated weights for policy 1, policy_version 16140 (0.0009) -[2023-10-14 14:04:16,749][75950] Updated weights for policy 1, policy_version 16150 (0.0010) -[2023-10-14 14:04:17,117][75950] Updated weights for policy 1, policy_version 16160 (0.0008) -[2023-10-14 14:04:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 33095680. Throughput: 0: 1670.3, 1: 1670.6. Samples: 8276086. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 14:04:18,165][74987] Avg episode reward: [(0, '14.310'), (1, '19.560')] -[2023-10-14 14:04:20,001][75949] Updated weights for policy 0, policy_version 16161 (0.0008) -[2023-10-14 14:04:20,381][75949] Updated weights for policy 0, policy_version 16171 (0.0007) -[2023-10-14 14:04:20,749][75949] Updated weights for policy 0, policy_version 16181 (0.0008) -[2023-10-14 14:04:21,127][75949] Updated weights for policy 0, policy_version 16191 (0.0008) -[2023-10-14 14:04:21,255][75950] Updated weights for policy 1, policy_version 16170 (0.0008) -[2023-10-14 14:04:21,624][75950] Updated weights for policy 1, policy_version 16180 (0.0008) -[2023-10-14 14:04:21,988][75950] Updated weights for policy 1, policy_version 16190 (0.0010) -[2023-10-14 14:04:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 33161216. Throughput: 0: 1669.9, 1: 1653.6. Samples: 8295286. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 14:04:23,165][74987] Avg episode reward: [(0, '14.940'), (1, '18.910')] -[2023-10-14 14:04:25,185][75949] Updated weights for policy 0, policy_version 16201 (0.0010) -[2023-10-14 14:04:25,562][75949] Updated weights for policy 0, policy_version 16211 (0.0008) -[2023-10-14 14:04:25,936][75949] Updated weights for policy 0, policy_version 16221 (0.0010) -[2023-10-14 14:04:26,104][75950] Updated weights for policy 1, policy_version 16200 (0.0008) -[2023-10-14 14:04:26,474][75950] Updated weights for policy 1, policy_version 16210 (0.0008) -[2023-10-14 14:04:26,835][75950] Updated weights for policy 1, policy_version 16220 (0.0010) -[2023-10-14 14:04:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 33226752. Throughput: 0: 1680.6, 1: 1671.2. Samples: 8315528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:04:28,165][74987] Avg episode reward: [(0, '14.100'), (1, '18.210')] -[2023-10-14 14:04:29,988][75949] Updated weights for policy 0, policy_version 16231 (0.0010) -[2023-10-14 14:04:30,346][75949] Updated weights for policy 0, policy_version 16241 (0.0008) -[2023-10-14 14:04:30,717][75949] Updated weights for policy 0, policy_version 16251 (0.0007) -[2023-10-14 14:04:30,733][75950] Updated weights for policy 1, policy_version 16230 (0.0009) -[2023-10-14 14:04:31,102][75950] Updated weights for policy 1, policy_version 16240 (0.0008) -[2023-10-14 14:04:31,469][75950] Updated weights for policy 1, policy_version 16250 (0.0008) -[2023-10-14 14:04:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 33292288. Throughput: 0: 1656.7, 1: 1678.3. Samples: 8325992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:04:33,165][74987] Avg episode reward: [(0, '15.120'), (1, '19.810')] -[2023-10-14 14:04:34,822][75949] Updated weights for policy 0, policy_version 16261 (0.0007) -[2023-10-14 14:04:35,196][75949] Updated weights for policy 0, policy_version 16271 (0.0011) -[2023-10-14 14:04:35,566][75949] Updated weights for policy 0, policy_version 16281 (0.0008) -[2023-10-14 14:04:35,574][75950] Updated weights for policy 1, policy_version 16260 (0.0008) -[2023-10-14 14:04:35,948][75950] Updated weights for policy 1, policy_version 16270 (0.0008) -[2023-10-14 14:04:36,324][75950] Updated weights for policy 1, policy_version 16280 (0.0010) -[2023-10-14 14:04:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33357824. Throughput: 0: 1672.4, 1: 1655.3. Samples: 8345122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:04:38,164][74987] Avg episode reward: [(0, '14.030'), (1, '19.630')] -[2023-10-14 14:04:39,701][75949] Updated weights for policy 0, policy_version 16291 (0.0008) -[2023-10-14 14:04:40,068][75949] Updated weights for policy 0, policy_version 16301 (0.0008) -[2023-10-14 14:04:40,380][75950] Updated weights for policy 1, policy_version 16290 (0.0008) -[2023-10-14 14:04:40,427][75949] Updated weights for policy 0, policy_version 16311 (0.0009) -[2023-10-14 14:04:40,747][75950] Updated weights for policy 1, policy_version 16300 (0.0007) -[2023-10-14 14:04:41,117][75950] Updated weights for policy 1, policy_version 16310 (0.0008) -[2023-10-14 14:04:41,483][75950] Updated weights for policy 1, policy_version 16320 (0.0009) -[2023-10-14 14:04:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 33423360. Throughput: 0: 1675.1, 1: 1678.4. Samples: 8365912. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 14:04:43,165][74987] Avg episode reward: [(0, '15.300'), (1, '18.410')] -[2023-10-14 14:04:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000016320_16711680.pth... -[2023-10-14 14:04:43,177][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000016320_16711680.pth... -[2023-10-14 14:04:43,214][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000014752_15106048.pth -[2023-10-14 14:04:43,217][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000014752_15106048.pth -[2023-10-14 14:04:44,544][75949] Updated weights for policy 0, policy_version 16321 (0.0008) -[2023-10-14 14:04:44,963][75949] Updated weights for policy 0, policy_version 16331 (0.0011) -[2023-10-14 14:04:45,332][75949] Updated weights for policy 0, policy_version 16341 (0.0009) -[2023-10-14 14:04:45,440][75950] Updated weights for policy 1, policy_version 16330 (0.0007) -[2023-10-14 14:04:45,706][75949] Updated weights for policy 0, policy_version 16351 (0.0008) -[2023-10-14 14:04:45,807][75950] Updated weights for policy 1, policy_version 16340 (0.0008) -[2023-10-14 14:04:46,179][75950] Updated weights for policy 1, policy_version 16350 (0.0008) -[2023-10-14 14:04:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 33488896. Throughput: 0: 1660.4, 1: 1669.8. Samples: 8375774. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 14:04:48,164][74987] Avg episode reward: [(0, '15.690'), (1, '18.620')] -[2023-10-14 14:04:48,165][75615] Saving new best policy, reward=15.690! -[2023-10-14 14:04:49,783][75949] Updated weights for policy 0, policy_version 16361 (0.0008) -[2023-10-14 14:04:50,154][75949] Updated weights for policy 0, policy_version 16371 (0.0009) -[2023-10-14 14:04:50,275][75950] Updated weights for policy 1, policy_version 16360 (0.0007) -[2023-10-14 14:04:50,525][75949] Updated weights for policy 0, policy_version 16381 (0.0007) -[2023-10-14 14:04:50,651][75950] Updated weights for policy 1, policy_version 16370 (0.0008) -[2023-10-14 14:04:51,014][75950] Updated weights for policy 1, policy_version 16380 (0.0007) -[2023-10-14 14:04:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33554432. Throughput: 0: 1668.2, 1: 1671.5. Samples: 8395480. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 14:04:53,164][74987] Avg episode reward: [(0, '15.470'), (1, '19.070')] -[2023-10-14 14:04:54,609][75949] Updated weights for policy 0, policy_version 16391 (0.0008) -[2023-10-14 14:04:54,983][75949] Updated weights for policy 0, policy_version 16401 (0.0010) -[2023-10-14 14:04:55,288][75950] Updated weights for policy 1, policy_version 16390 (0.0007) -[2023-10-14 14:04:55,343][75949] Updated weights for policy 0, policy_version 16411 (0.0008) -[2023-10-14 14:04:55,666][75950] Updated weights for policy 1, policy_version 16400 (0.0007) -[2023-10-14 14:04:56,035][75950] Updated weights for policy 1, policy_version 16410 (0.0007) -[2023-10-14 14:04:58,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 33619968. Throughput: 0: 1679.2, 1: 1675.7. Samples: 8416324. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 14:04:58,165][74987] Avg episode reward: [(0, '15.250'), (1, '18.690')] -[2023-10-14 14:04:59,400][75949] Updated weights for policy 0, policy_version 16421 (0.0009) -[2023-10-14 14:04:59,769][75949] Updated weights for policy 0, policy_version 16431 (0.0009) -[2023-10-14 14:04:59,934][75950] Updated weights for policy 1, policy_version 16420 (0.0009) -[2023-10-14 14:05:00,146][75949] Updated weights for policy 0, policy_version 16441 (0.0008) -[2023-10-14 14:05:00,306][75950] Updated weights for policy 1, policy_version 16430 (0.0008) -[2023-10-14 14:05:00,682][75950] Updated weights for policy 1, policy_version 16440 (0.0009) -[2023-10-14 14:05:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33685504. Throughput: 0: 1664.1, 1: 1664.9. Samples: 8425892. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 14:05:03,164][74987] Avg episode reward: [(0, '13.960'), (1, '20.150')] -[2023-10-14 14:05:04,355][75949] Updated weights for policy 0, policy_version 16451 (0.0007) -[2023-10-14 14:05:04,725][75949] Updated weights for policy 0, policy_version 16461 (0.0008) -[2023-10-14 14:05:04,732][75950] Updated weights for policy 1, policy_version 16450 (0.0009) -[2023-10-14 14:05:05,099][75950] Updated weights for policy 1, policy_version 16460 (0.0009) -[2023-10-14 14:05:05,104][75949] Updated weights for policy 0, policy_version 16471 (0.0008) -[2023-10-14 14:05:05,472][75950] Updated weights for policy 1, policy_version 16470 (0.0007) -[2023-10-14 14:05:05,833][75950] Updated weights for policy 1, policy_version 16480 (0.0007) -[2023-10-14 14:05:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33751040. Throughput: 0: 1675.3, 1: 1676.0. Samples: 8446094. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 14:05:08,165][74987] Avg episode reward: [(0, '15.820'), (1, '18.440')] -[2023-10-14 14:05:08,166][75615] Saving new best policy, reward=15.820! -[2023-10-14 14:05:09,195][75949] Updated weights for policy 0, policy_version 16481 (0.0008) -[2023-10-14 14:05:09,557][75949] Updated weights for policy 0, policy_version 16491 (0.0009) -[2023-10-14 14:05:09,857][75950] Updated weights for policy 1, policy_version 16490 (0.0007) -[2023-10-14 14:05:09,937][75949] Updated weights for policy 0, policy_version 16501 (0.0009) -[2023-10-14 14:05:10,219][75950] Updated weights for policy 1, policy_version 16500 (0.0009) -[2023-10-14 14:05:10,313][75949] Updated weights for policy 0, policy_version 16511 (0.0009) -[2023-10-14 14:05:10,586][75950] Updated weights for policy 1, policy_version 16510 (0.0010) -[2023-10-14 14:05:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33816576. Throughput: 0: 1673.6, 1: 1692.4. Samples: 8466998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:05:13,165][74987] Avg episode reward: [(0, '14.950'), (1, '19.550')] -[2023-10-14 14:05:14,344][75949] Updated weights for policy 0, policy_version 16521 (0.0007) -[2023-10-14 14:05:14,503][75950] Updated weights for policy 1, policy_version 16520 (0.0007) -[2023-10-14 14:05:14,712][75949] Updated weights for policy 0, policy_version 16531 (0.0008) -[2023-10-14 14:05:14,878][75950] Updated weights for policy 1, policy_version 16530 (0.0009) -[2023-10-14 14:05:15,083][75949] Updated weights for policy 0, policy_version 16541 (0.0009) -[2023-10-14 14:05:15,254][75950] Updated weights for policy 1, policy_version 16540 (0.0009) -[2023-10-14 14:05:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33882112. Throughput: 0: 1671.3, 1: 1665.9. Samples: 8476166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:05:18,164][74987] Avg episode reward: [(0, '16.470'), (1, '18.470')] -[2023-10-14 14:05:18,166][75615] Saving new best policy, reward=16.470! -[2023-10-14 14:05:19,142][75949] Updated weights for policy 0, policy_version 16551 (0.0010) -[2023-10-14 14:05:19,341][75950] Updated weights for policy 1, policy_version 16550 (0.0009) -[2023-10-14 14:05:19,509][75949] Updated weights for policy 0, policy_version 16561 (0.0009) -[2023-10-14 14:05:19,715][75950] Updated weights for policy 1, policy_version 16560 (0.0009) -[2023-10-14 14:05:19,882][75949] Updated weights for policy 0, policy_version 16571 (0.0008) -[2023-10-14 14:05:20,080][75950] Updated weights for policy 1, policy_version 16570 (0.0008) -[2023-10-14 14:05:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33947648. Throughput: 0: 1681.6, 1: 1688.5. Samples: 8496780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:05:23,165][74987] Avg episode reward: [(0, '15.340'), (1, '19.810')] -[2023-10-14 14:05:23,918][75949] Updated weights for policy 0, policy_version 16581 (0.0009) -[2023-10-14 14:05:24,285][75949] Updated weights for policy 0, policy_version 16591 (0.0009) -[2023-10-14 14:05:24,391][75950] Updated weights for policy 1, policy_version 16580 (0.0008) -[2023-10-14 14:05:24,655][75949] Updated weights for policy 0, policy_version 16601 (0.0007) -[2023-10-14 14:05:24,753][75950] Updated weights for policy 1, policy_version 16590 (0.0008) -[2023-10-14 14:05:25,115][75950] Updated weights for policy 1, policy_version 16600 (0.0008) -[2023-10-14 14:05:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34013184. Throughput: 0: 1682.5, 1: 1683.7. Samples: 8517392. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 14:05:28,165][74987] Avg episode reward: [(0, '15.250'), (1, '19.380')] -[2023-10-14 14:05:28,525][75949] Updated weights for policy 0, policy_version 16611 (0.0007) -[2023-10-14 14:05:28,895][75949] Updated weights for policy 0, policy_version 16621 (0.0008) -[2023-10-14 14:05:29,192][75950] Updated weights for policy 1, policy_version 16610 (0.0010) -[2023-10-14 14:05:29,263][75949] Updated weights for policy 0, policy_version 16631 (0.0009) -[2023-10-14 14:05:29,561][75950] Updated weights for policy 1, policy_version 16620 (0.0010) -[2023-10-14 14:05:29,932][75950] Updated weights for policy 1, policy_version 16630 (0.0008) -[2023-10-14 14:05:30,307][75950] Updated weights for policy 1, policy_version 16640 (0.0009) -[2023-10-14 14:05:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34078720. Throughput: 0: 1682.7, 1: 1666.6. Samples: 8526494. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 14:05:33,164][74987] Avg episode reward: [(0, '14.050'), (1, '18.880')] -[2023-10-14 14:05:33,302][75949] Updated weights for policy 0, policy_version 16641 (0.0008) -[2023-10-14 14:05:33,718][75949] Updated weights for policy 0, policy_version 16651 (0.0009) -[2023-10-14 14:05:34,092][75949] Updated weights for policy 0, policy_version 16661 (0.0010) -[2023-10-14 14:05:34,366][75950] Updated weights for policy 1, policy_version 16650 (0.0009) -[2023-10-14 14:05:34,472][75949] Updated weights for policy 0, policy_version 16671 (0.0008) -[2023-10-14 14:05:34,739][75950] Updated weights for policy 1, policy_version 16660 (0.0010) -[2023-10-14 14:05:35,104][75950] Updated weights for policy 1, policy_version 16670 (0.0007) -[2023-10-14 14:05:38,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34144256. Throughput: 0: 1684.1, 1: 1685.2. Samples: 8547098. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 14:05:38,164][74987] Avg episode reward: [(0, '15.920'), (1, '18.150')] -[2023-10-14 14:05:38,478][75949] Updated weights for policy 0, policy_version 16681 (0.0008) -[2023-10-14 14:05:38,858][75949] Updated weights for policy 0, policy_version 16691 (0.0007) -[2023-10-14 14:05:39,227][75950] Updated weights for policy 1, policy_version 16680 (0.0009) -[2023-10-14 14:05:39,232][75949] Updated weights for policy 0, policy_version 16701 (0.0007) -[2023-10-14 14:05:39,598][75950] Updated weights for policy 1, policy_version 16690 (0.0007) -[2023-10-14 14:05:39,957][75950] Updated weights for policy 1, policy_version 16700 (0.0007) -[2023-10-14 14:05:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 34209792. Throughput: 0: 1679.4, 1: 1688.3. Samples: 8567872. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 14:05:43,165][74987] Avg episode reward: [(0, '15.460'), (1, '19.030')] -[2023-10-14 14:05:43,199][75949] Updated weights for policy 0, policy_version 16711 (0.0008) -[2023-10-14 14:05:43,581][75949] Updated weights for policy 0, policy_version 16721 (0.0008) -[2023-10-14 14:05:43,952][75949] Updated weights for policy 0, policy_version 16731 (0.0008) -[2023-10-14 14:05:43,990][75950] Updated weights for policy 1, policy_version 16710 (0.0007) -[2023-10-14 14:05:44,379][75950] Updated weights for policy 1, policy_version 16720 (0.0008) -[2023-10-14 14:05:44,747][75950] Updated weights for policy 1, policy_version 16730 (0.0008) -[2023-10-14 14:05:48,052][75949] Updated weights for policy 0, policy_version 16741 (0.0009) -[2023-10-14 14:05:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34275328. Throughput: 0: 1683.6, 1: 1673.3. Samples: 8576954. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 14:05:48,164][74987] Avg episode reward: [(0, '16.670'), (1, '18.820')] -[2023-10-14 14:05:48,427][75949] Updated weights for policy 0, policy_version 16751 (0.0009) -[2023-10-14 14:05:48,807][75949] Updated weights for policy 0, policy_version 16761 (0.0011) -[2023-10-14 14:05:48,903][75950] Updated weights for policy 1, policy_version 16740 (0.0008) -[2023-10-14 14:05:49,056][75615] Saving new best policy, reward=16.670! -[2023-10-14 14:05:49,267][75950] Updated weights for policy 1, policy_version 16750 (0.0008) -[2023-10-14 14:05:49,645][75950] Updated weights for policy 1, policy_version 16760 (0.0008) -[2023-10-14 14:05:52,916][75949] Updated weights for policy 0, policy_version 16771 (0.0007) -[2023-10-14 14:05:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34340864. Throughput: 0: 1682.2, 1: 1676.6. Samples: 8597240. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 14:05:53,164][74987] Avg episode reward: [(0, '15.970'), (1, '18.650')] -[2023-10-14 14:05:53,283][75949] Updated weights for policy 0, policy_version 16781 (0.0009) -[2023-10-14 14:05:53,655][75949] Updated weights for policy 0, policy_version 16791 (0.0009) -[2023-10-14 14:05:53,768][75950] Updated weights for policy 1, policy_version 16770 (0.0008) -[2023-10-14 14:05:54,133][75950] Updated weights for policy 1, policy_version 16780 (0.0009) -[2023-10-14 14:05:54,504][75950] Updated weights for policy 1, policy_version 16790 (0.0010) -[2023-10-14 14:05:54,866][75950] Updated weights for policy 1, policy_version 16800 (0.0008) -[2023-10-14 14:05:57,757][75949] Updated weights for policy 0, policy_version 16801 (0.0008) -[2023-10-14 14:05:58,117][75949] Updated weights for policy 0, policy_version 16811 (0.0008) -[2023-10-14 14:05:58,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 34406400. Throughput: 0: 1684.5, 1: 1670.1. Samples: 8617952. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:05:58,164][74987] Avg episode reward: [(0, '16.300'), (1, '19.980')] -[2023-10-14 14:05:58,489][75949] Updated weights for policy 0, policy_version 16821 (0.0007) -[2023-10-14 14:05:58,813][75950] Updated weights for policy 1, policy_version 16810 (0.0007) -[2023-10-14 14:05:58,860][75949] Updated weights for policy 0, policy_version 16831 (0.0008) -[2023-10-14 14:05:59,176][75950] Updated weights for policy 1, policy_version 16820 (0.0007) -[2023-10-14 14:05:59,551][75950] Updated weights for policy 1, policy_version 16830 (0.0008) -[2023-10-14 14:06:02,986][75949] Updated weights for policy 0, policy_version 16841 (0.0010) -[2023-10-14 14:06:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34471936. Throughput: 0: 1684.5, 1: 1668.3. Samples: 8627040. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:06:03,164][74987] Avg episode reward: [(0, '16.140'), (1, '19.180')] -[2023-10-14 14:06:03,359][75949] Updated weights for policy 0, policy_version 16851 (0.0011) -[2023-10-14 14:06:03,700][75950] Updated weights for policy 1, policy_version 16840 (0.0008) -[2023-10-14 14:06:03,738][75949] Updated weights for policy 0, policy_version 16861 (0.0008) -[2023-10-14 14:06:04,069][75950] Updated weights for policy 1, policy_version 16850 (0.0008) -[2023-10-14 14:06:04,432][75950] Updated weights for policy 1, policy_version 16860 (0.0008) -[2023-10-14 14:06:07,859][75949] Updated weights for policy 0, policy_version 16871 (0.0009) -[2023-10-14 14:06:08,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 34537472. Throughput: 0: 1684.4, 1: 1667.4. Samples: 8647612. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:06:08,165][74987] Avg episode reward: [(0, '15.450'), (1, '19.770')] -[2023-10-14 14:06:08,236][75949] Updated weights for policy 0, policy_version 16881 (0.0010) -[2023-10-14 14:06:08,600][75949] Updated weights for policy 0, policy_version 16891 (0.0010) -[2023-10-14 14:06:08,625][75950] Updated weights for policy 1, policy_version 16870 (0.0009) -[2023-10-14 14:06:08,996][75950] Updated weights for policy 1, policy_version 16880 (0.0008) -[2023-10-14 14:06:09,373][75950] Updated weights for policy 1, policy_version 16890 (0.0007) -[2023-10-14 14:06:12,639][75949] Updated weights for policy 0, policy_version 16901 (0.0008) -[2023-10-14 14:06:13,009][75949] Updated weights for policy 0, policy_version 16911 (0.0008) -[2023-10-14 14:06:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34603008. Throughput: 0: 1676.9, 1: 1670.5. Samples: 8668026. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 14:06:13,165][74987] Avg episode reward: [(0, '15.150'), (1, '19.330')] -[2023-10-14 14:06:13,374][75949] Updated weights for policy 0, policy_version 16921 (0.0008) -[2023-10-14 14:06:13,471][75950] Updated weights for policy 1, policy_version 16900 (0.0007) -[2023-10-14 14:06:13,843][75950] Updated weights for policy 1, policy_version 16910 (0.0009) -[2023-10-14 14:06:14,216][75950] Updated weights for policy 1, policy_version 16920 (0.0008) -[2023-10-14 14:06:17,428][75949] Updated weights for policy 0, policy_version 16931 (0.0008) -[2023-10-14 14:06:17,799][75949] Updated weights for policy 0, policy_version 16941 (0.0007) -[2023-10-14 14:06:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 34668544. Throughput: 0: 1682.5, 1: 1666.3. Samples: 8677190. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 14:06:18,165][74987] Avg episode reward: [(0, '16.100'), (1, '20.760')] -[2023-10-14 14:06:18,169][75949] Updated weights for policy 0, policy_version 16951 (0.0007) -[2023-10-14 14:06:18,400][75950] Updated weights for policy 1, policy_version 16930 (0.0010) -[2023-10-14 14:06:18,767][75950] Updated weights for policy 1, policy_version 16940 (0.0008) -[2023-10-14 14:06:19,144][75950] Updated weights for policy 1, policy_version 16950 (0.0008) -[2023-10-14 14:06:19,509][75950] Updated weights for policy 1, policy_version 16960 (0.0007) -[2023-10-14 14:06:22,177][75949] Updated weights for policy 0, policy_version 16961 (0.0008) -[2023-10-14 14:06:22,592][75949] Updated weights for policy 0, policy_version 16971 (0.0008) -[2023-10-14 14:06:22,972][75949] Updated weights for policy 0, policy_version 16981 (0.0007) -[2023-10-14 14:06:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 34734080. Throughput: 0: 1687.9, 1: 1663.7. Samples: 8697924. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 14:06:23,165][74987] Avg episode reward: [(0, '15.870'), (1, '19.210')] -[2023-10-14 14:06:23,338][75949] Updated weights for policy 0, policy_version 16991 (0.0008) -[2023-10-14 14:06:23,448][75950] Updated weights for policy 1, policy_version 16970 (0.0007) -[2023-10-14 14:06:23,822][75950] Updated weights for policy 1, policy_version 16980 (0.0007) -[2023-10-14 14:06:24,190][75950] Updated weights for policy 1, policy_version 16990 (0.0007) -[2023-10-14 14:06:27,371][75949] Updated weights for policy 0, policy_version 17001 (0.0008) -[2023-10-14 14:06:27,738][75949] Updated weights for policy 0, policy_version 17011 (0.0009) -[2023-10-14 14:06:28,115][75949] Updated weights for policy 0, policy_version 17021 (0.0009) -[2023-10-14 14:06:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 34799616. Throughput: 0: 1666.9, 1: 1662.6. Samples: 8717700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:06:28,165][74987] Avg episode reward: [(0, '15.750'), (1, '21.460')] -[2023-10-14 14:06:28,342][75950] Updated weights for policy 1, policy_version 17000 (0.0008) -[2023-10-14 14:06:28,712][75950] Updated weights for policy 1, policy_version 17010 (0.0009) -[2023-10-14 14:06:29,086][75950] Updated weights for policy 1, policy_version 17020 (0.0007) -[2023-10-14 14:06:29,224][75801] Saving new best policy, reward=21.460! -[2023-10-14 14:06:32,041][75949] Updated weights for policy 0, policy_version 17031 (0.0009) -[2023-10-14 14:06:32,411][75949] Updated weights for policy 0, policy_version 17041 (0.0007) -[2023-10-14 14:06:32,784][75949] Updated weights for policy 0, policy_version 17051 (0.0007) -[2023-10-14 14:06:33,164][74987] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 34897920. Throughput: 0: 1682.6, 1: 1661.7. Samples: 8727448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:06:33,164][74987] Avg episode reward: [(0, '15.890'), (1, '16.960')] -[2023-10-14 14:06:33,288][75950] Updated weights for policy 1, policy_version 17030 (0.0008) -[2023-10-14 14:06:33,663][75950] Updated weights for policy 1, policy_version 17040 (0.0007) -[2023-10-14 14:06:34,025][75950] Updated weights for policy 1, policy_version 17050 (0.0011) -[2023-10-14 14:06:37,020][75949] Updated weights for policy 0, policy_version 17061 (0.0009) -[2023-10-14 14:06:37,386][75949] Updated weights for policy 0, policy_version 17071 (0.0008) -[2023-10-14 14:06:37,763][75949] Updated weights for policy 0, policy_version 17081 (0.0009) -[2023-10-14 14:06:38,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 34963456. Throughput: 0: 1684.5, 1: 1667.1. Samples: 8748062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:06:38,164][74987] Avg episode reward: [(0, '14.910'), (1, '19.190')] -[2023-10-14 14:06:38,176][75950] Updated weights for policy 1, policy_version 17060 (0.0008) -[2023-10-14 14:06:38,550][75950] Updated weights for policy 1, policy_version 17070 (0.0008) -[2023-10-14 14:06:38,912][75950] Updated weights for policy 1, policy_version 17080 (0.0007) -[2023-10-14 14:06:41,878][75949] Updated weights for policy 0, policy_version 17091 (0.0008) -[2023-10-14 14:06:42,245][75949] Updated weights for policy 0, policy_version 17101 (0.0007) -[2023-10-14 14:06:42,614][75949] Updated weights for policy 0, policy_version 17111 (0.0008) -[2023-10-14 14:06:43,043][75950] Updated weights for policy 1, policy_version 17090 (0.0008) -[2023-10-14 14:06:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 35028992. Throughput: 0: 1661.4, 1: 1666.5. Samples: 8767708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:06:43,164][74987] Avg episode reward: [(0, '15.990'), (1, '17.600')] -[2023-10-14 14:06:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000017120_17530880.pth... -[2023-10-14 14:06:43,205][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000015552_15925248.pth -[2023-10-14 14:06:43,418][75950] Updated weights for policy 1, policy_version 17100 (0.0009) -[2023-10-14 14:06:43,777][75950] Updated weights for policy 1, policy_version 17110 (0.0008) -[2023-10-14 14:06:44,146][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000017120_17530880.pth... -[2023-10-14 14:06:44,148][75950] Updated weights for policy 1, policy_version 17120 (0.0008) -[2023-10-14 14:06:44,185][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000015552_15925248.pth -[2023-10-14 14:06:46,608][75949] Updated weights for policy 0, policy_version 17121 (0.0009) -[2023-10-14 14:06:46,977][75949] Updated weights for policy 0, policy_version 17131 (0.0008) -[2023-10-14 14:06:47,345][75949] Updated weights for policy 0, policy_version 17141 (0.0009) -[2023-10-14 14:06:47,713][75949] Updated weights for policy 0, policy_version 17151 (0.0008) -[2023-10-14 14:06:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 35094528. Throughput: 0: 1684.7, 1: 1665.2. Samples: 8777782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:06:48,165][74987] Avg episode reward: [(0, '16.480'), (1, '20.050')] -[2023-10-14 14:06:48,362][75950] Updated weights for policy 1, policy_version 17130 (0.0010) -[2023-10-14 14:06:48,739][75950] Updated weights for policy 1, policy_version 17140 (0.0008) -[2023-10-14 14:06:49,110][75950] Updated weights for policy 1, policy_version 17150 (0.0009) -[2023-10-14 14:06:51,806][75949] Updated weights for policy 0, policy_version 17161 (0.0008) -[2023-10-14 14:06:52,173][75949] Updated weights for policy 0, policy_version 17171 (0.0007) -[2023-10-14 14:06:52,548][75949] Updated weights for policy 0, policy_version 17181 (0.0009) -[2023-10-14 14:06:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 35160064. Throughput: 0: 1678.9, 1: 1665.1. Samples: 8798094. Policy #0 lag: (min: 3.0, avg: 10.5, max: 35.0) -[2023-10-14 14:06:53,164][74987] Avg episode reward: [(0, '16.320'), (1, '19.550')] -[2023-10-14 14:06:53,310][75950] Updated weights for policy 1, policy_version 17160 (0.0010) -[2023-10-14 14:06:53,679][75950] Updated weights for policy 1, policy_version 17170 (0.0009) -[2023-10-14 14:06:54,049][75950] Updated weights for policy 1, policy_version 17180 (0.0009) -[2023-10-14 14:06:56,555][75949] Updated weights for policy 0, policy_version 17191 (0.0009) -[2023-10-14 14:06:56,925][75949] Updated weights for policy 0, policy_version 17201 (0.0007) -[2023-10-14 14:06:57,308][75949] Updated weights for policy 0, policy_version 17211 (0.0007) -[2023-10-14 14:06:58,130][75950] Updated weights for policy 1, policy_version 17190 (0.0008) -[2023-10-14 14:06:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 35225600. Throughput: 0: 1659.7, 1: 1667.2. Samples: 8817736. Policy #0 lag: (min: 3.0, avg: 10.5, max: 35.0) -[2023-10-14 14:06:58,165][74987] Avg episode reward: [(0, '16.650'), (1, '20.320')] -[2023-10-14 14:06:58,489][75950] Updated weights for policy 1, policy_version 17200 (0.0008) -[2023-10-14 14:06:58,863][75950] Updated weights for policy 1, policy_version 17210 (0.0007) -[2023-10-14 14:07:01,368][75949] Updated weights for policy 0, policy_version 17221 (0.0009) -[2023-10-14 14:07:01,744][75949] Updated weights for policy 0, policy_version 17231 (0.0008) -[2023-10-14 14:07:02,113][75949] Updated weights for policy 0, policy_version 17241 (0.0007) -[2023-10-14 14:07:02,775][75950] Updated weights for policy 1, policy_version 17220 (0.0007) -[2023-10-14 14:07:03,147][75950] Updated weights for policy 1, policy_version 17230 (0.0008) -[2023-10-14 14:07:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 35291136. Throughput: 0: 1686.6, 1: 1669.6. Samples: 8828220. Policy #0 lag: (min: 3.0, avg: 10.5, max: 35.0) -[2023-10-14 14:07:03,164][74987] Avg episode reward: [(0, '15.860'), (1, '19.270')] -[2023-10-14 14:07:03,517][75950] Updated weights for policy 1, policy_version 17240 (0.0008) -[2023-10-14 14:07:06,245][75949] Updated weights for policy 0, policy_version 17251 (0.0009) -[2023-10-14 14:07:06,631][75949] Updated weights for policy 0, policy_version 17261 (0.0010) -[2023-10-14 14:07:07,000][75949] Updated weights for policy 0, policy_version 17271 (0.0011) -[2023-10-14 14:07:07,637][75950] Updated weights for policy 1, policy_version 17250 (0.0008) -[2023-10-14 14:07:08,006][75950] Updated weights for policy 1, policy_version 17260 (0.0007) -[2023-10-14 14:07:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 35356672. Throughput: 0: 1674.8, 1: 1664.0. Samples: 8848172. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) -[2023-10-14 14:07:08,165][74987] Avg episode reward: [(0, '16.090'), (1, '18.980')] -[2023-10-14 14:07:08,383][75950] Updated weights for policy 1, policy_version 17270 (0.0007) -[2023-10-14 14:07:08,749][75950] Updated weights for policy 1, policy_version 17280 (0.0008) -[2023-10-14 14:07:11,013][75949] Updated weights for policy 0, policy_version 17281 (0.0009) -[2023-10-14 14:07:11,437][75949] Updated weights for policy 0, policy_version 17291 (0.0009) -[2023-10-14 14:07:11,802][75949] Updated weights for policy 0, policy_version 17301 (0.0007) -[2023-10-14 14:07:12,172][75949] Updated weights for policy 0, policy_version 17311 (0.0010) -[2023-10-14 14:07:12,785][75950] Updated weights for policy 1, policy_version 17290 (0.0010) -[2023-10-14 14:07:13,158][75950] Updated weights for policy 1, policy_version 17300 (0.0008) -[2023-10-14 14:07:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 35422208. Throughput: 0: 1674.6, 1: 1666.4. Samples: 8868044. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) -[2023-10-14 14:07:13,165][74987] Avg episode reward: [(0, '16.480'), (1, '20.780')] -[2023-10-14 14:07:13,521][75950] Updated weights for policy 1, policy_version 17310 (0.0009) -[2023-10-14 14:07:16,101][75949] Updated weights for policy 0, policy_version 17321 (0.0009) -[2023-10-14 14:07:16,465][75949] Updated weights for policy 0, policy_version 17331 (0.0008) -[2023-10-14 14:07:16,846][75949] Updated weights for policy 0, policy_version 17341 (0.0009) -[2023-10-14 14:07:17,462][75950] Updated weights for policy 1, policy_version 17320 (0.0008) -[2023-10-14 14:07:17,822][75950] Updated weights for policy 1, policy_version 17330 (0.0008) -[2023-10-14 14:07:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 35487744. Throughput: 0: 1689.1, 1: 1672.7. Samples: 8878728. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) -[2023-10-14 14:07:18,164][74987] Avg episode reward: [(0, '16.420'), (1, '19.500')] -[2023-10-14 14:07:18,195][75950] Updated weights for policy 1, policy_version 17340 (0.0007) -[2023-10-14 14:07:21,022][75949] Updated weights for policy 0, policy_version 17351 (0.0009) -[2023-10-14 14:07:21,390][75949] Updated weights for policy 0, policy_version 17361 (0.0009) -[2023-10-14 14:07:21,753][75949] Updated weights for policy 0, policy_version 17371 (0.0008) -[2023-10-14 14:07:22,384][75950] Updated weights for policy 1, policy_version 17350 (0.0008) -[2023-10-14 14:07:22,762][75950] Updated weights for policy 1, policy_version 17360 (0.0009) -[2023-10-14 14:07:23,139][75950] Updated weights for policy 1, policy_version 17370 (0.0009) -[2023-10-14 14:07:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 35553280. Throughput: 0: 1664.7, 1: 1672.0. Samples: 8898216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:07:23,164][74987] Avg episode reward: [(0, '17.810'), (1, '20.510')] -[2023-10-14 14:07:23,165][75615] Saving new best policy, reward=17.810! -[2023-10-14 14:07:25,800][75949] Updated weights for policy 0, policy_version 17381 (0.0008) -[2023-10-14 14:07:26,176][75949] Updated weights for policy 0, policy_version 17391 (0.0009) -[2023-10-14 14:07:26,541][75949] Updated weights for policy 0, policy_version 17401 (0.0009) -[2023-10-14 14:07:27,160][75950] Updated weights for policy 1, policy_version 17380 (0.0010) -[2023-10-14 14:07:27,523][75950] Updated weights for policy 1, policy_version 17390 (0.0010) -[2023-10-14 14:07:27,894][75950] Updated weights for policy 1, policy_version 17400 (0.0007) -[2023-10-14 14:07:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 35618816. Throughput: 0: 1682.4, 1: 1657.7. Samples: 8918012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:07:28,164][74987] Avg episode reward: [(0, '15.850'), (1, '19.100')] -[2023-10-14 14:07:30,641][75949] Updated weights for policy 0, policy_version 17411 (0.0008) -[2023-10-14 14:07:31,012][75949] Updated weights for policy 0, policy_version 17421 (0.0008) -[2023-10-14 14:07:31,388][75949] Updated weights for policy 0, policy_version 17431 (0.0009) -[2023-10-14 14:07:32,050][75950] Updated weights for policy 1, policy_version 17410 (0.0007) -[2023-10-14 14:07:32,421][75950] Updated weights for policy 1, policy_version 17420 (0.0008) -[2023-10-14 14:07:32,794][75950] Updated weights for policy 1, policy_version 17430 (0.0009) -[2023-10-14 14:07:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 35684352. Throughput: 0: 1682.9, 1: 1673.9. Samples: 8928836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:07:33,165][75950] Updated weights for policy 1, policy_version 17440 (0.0010) -[2023-10-14 14:07:33,165][74987] Avg episode reward: [(0, '17.050'), (1, '19.510')] -[2023-10-14 14:07:35,329][75949] Updated weights for policy 0, policy_version 17441 (0.0009) -[2023-10-14 14:07:35,694][75949] Updated weights for policy 0, policy_version 17451 (0.0007) -[2023-10-14 14:07:36,072][75949] Updated weights for policy 0, policy_version 17461 (0.0007) -[2023-10-14 14:07:36,445][75949] Updated weights for policy 0, policy_version 17471 (0.0009) -[2023-10-14 14:07:37,309][75950] Updated weights for policy 1, policy_version 17450 (0.0010) -[2023-10-14 14:07:37,674][75950] Updated weights for policy 1, policy_version 17460 (0.0007) -[2023-10-14 14:07:38,042][75950] Updated weights for policy 1, policy_version 17470 (0.0007) -[2023-10-14 14:07:38,163][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.6). Total num frames: 35782656. Throughput: 0: 1666.4, 1: 1674.4. Samples: 8948430. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 14:07:38,164][74987] Avg episode reward: [(0, '16.190'), (1, '16.780')] -[2023-10-14 14:07:40,665][75949] Updated weights for policy 0, policy_version 17481 (0.0011) -[2023-10-14 14:07:41,036][75949] Updated weights for policy 0, policy_version 17491 (0.0010) -[2023-10-14 14:07:41,407][75949] Updated weights for policy 0, policy_version 17501 (0.0010) -[2023-10-14 14:07:42,151][75950] Updated weights for policy 1, policy_version 17480 (0.0008) -[2023-10-14 14:07:42,505][75950] Updated weights for policy 1, policy_version 17490 (0.0007) -[2023-10-14 14:07:42,876][75950] Updated weights for policy 1, policy_version 17500 (0.0007) -[2023-10-14 14:07:43,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 35848192. Throughput: 0: 1689.6, 1: 1655.2. Samples: 8968250. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 14:07:43,164][74987] Avg episode reward: [(0, '16.550'), (1, '19.740')] -[2023-10-14 14:07:45,361][75949] Updated weights for policy 0, policy_version 17511 (0.0008) -[2023-10-14 14:07:45,728][75949] Updated weights for policy 0, policy_version 17521 (0.0008) -[2023-10-14 14:07:46,106][75949] Updated weights for policy 0, policy_version 17531 (0.0007) -[2023-10-14 14:07:47,016][75950] Updated weights for policy 1, policy_version 17510 (0.0007) -[2023-10-14 14:07:47,395][75950] Updated weights for policy 1, policy_version 17520 (0.0007) -[2023-10-14 14:07:47,755][75950] Updated weights for policy 1, policy_version 17530 (0.0010) -[2023-10-14 14:07:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 35913728. Throughput: 0: 1668.2, 1: 1674.8. Samples: 8978654. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 14:07:48,164][74987] Avg episode reward: [(0, '16.770'), (1, '17.300')] -[2023-10-14 14:07:50,246][75949] Updated weights for policy 0, policy_version 17541 (0.0009) -[2023-10-14 14:07:50,614][75949] Updated weights for policy 0, policy_version 17551 (0.0010) -[2023-10-14 14:07:50,980][75949] Updated weights for policy 0, policy_version 17561 (0.0009) -[2023-10-14 14:07:51,747][75950] Updated weights for policy 1, policy_version 17540 (0.0010) -[2023-10-14 14:07:52,115][75950] Updated weights for policy 1, policy_version 17550 (0.0007) -[2023-10-14 14:07:52,481][75950] Updated weights for policy 1, policy_version 17560 (0.0007) -[2023-10-14 14:07:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 35979264. Throughput: 0: 1658.7, 1: 1678.6. Samples: 8998350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:07:53,164][74987] Avg episode reward: [(0, '16.580'), (1, '20.220')] -[2023-10-14 14:07:54,991][75949] Updated weights for policy 0, policy_version 17571 (0.0009) -[2023-10-14 14:07:55,368][75949] Updated weights for policy 0, policy_version 17581 (0.0008) -[2023-10-14 14:07:55,742][75949] Updated weights for policy 0, policy_version 17591 (0.0008) -[2023-10-14 14:07:56,589][75950] Updated weights for policy 1, policy_version 17570 (0.0008) -[2023-10-14 14:07:56,947][75950] Updated weights for policy 1, policy_version 17580 (0.0008) -[2023-10-14 14:07:57,308][75950] Updated weights for policy 1, policy_version 17590 (0.0010) -[2023-10-14 14:07:57,672][75950] Updated weights for policy 1, policy_version 17600 (0.0009) -[2023-10-14 14:07:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 36044800. Throughput: 0: 1677.3, 1: 1654.7. Samples: 9017984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:07:58,165][74987] Avg episode reward: [(0, '18.190'), (1, '18.940')] -[2023-10-14 14:07:58,174][75615] Saving new best policy, reward=18.190! -[2023-10-14 14:07:59,786][75949] Updated weights for policy 0, policy_version 17601 (0.0011) -[2023-10-14 14:08:00,215][75949] Updated weights for policy 0, policy_version 17611 (0.0011) -[2023-10-14 14:08:00,582][75949] Updated weights for policy 0, policy_version 17621 (0.0011) -[2023-10-14 14:08:00,963][75949] Updated weights for policy 0, policy_version 17631 (0.0008) -[2023-10-14 14:08:01,866][75950] Updated weights for policy 1, policy_version 17610 (0.0007) -[2023-10-14 14:08:02,232][75950] Updated weights for policy 1, policy_version 17620 (0.0009) -[2023-10-14 14:08:02,608][75950] Updated weights for policy 1, policy_version 17630 (0.0009) -[2023-10-14 14:08:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 36110336. Throughput: 0: 1652.1, 1: 1675.2. Samples: 9028456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:08:03,165][74987] Avg episode reward: [(0, '17.110'), (1, '20.030')] -[2023-10-14 14:08:05,061][75949] Updated weights for policy 0, policy_version 17641 (0.0008) -[2023-10-14 14:08:05,433][75949] Updated weights for policy 0, policy_version 17651 (0.0009) -[2023-10-14 14:08:05,814][75949] Updated weights for policy 0, policy_version 17661 (0.0010) -[2023-10-14 14:08:06,515][75950] Updated weights for policy 1, policy_version 17640 (0.0010) -[2023-10-14 14:08:06,879][75950] Updated weights for policy 1, policy_version 17650 (0.0007) -[2023-10-14 14:08:07,246][75950] Updated weights for policy 1, policy_version 17660 (0.0008) -[2023-10-14 14:08:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 36175872. Throughput: 0: 1667.6, 1: 1668.9. Samples: 9048358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) -[2023-10-14 14:08:08,165][74987] Avg episode reward: [(0, '18.010'), (1, '18.570')] -[2023-10-14 14:08:09,835][75949] Updated weights for policy 0, policy_version 17671 (0.0007) -[2023-10-14 14:08:10,216][75949] Updated weights for policy 0, policy_version 17681 (0.0007) -[2023-10-14 14:08:10,586][75949] Updated weights for policy 0, policy_version 17691 (0.0009) -[2023-10-14 14:08:11,657][75950] Updated weights for policy 1, policy_version 17670 (0.0009) -[2023-10-14 14:08:12,036][75950] Updated weights for policy 1, policy_version 17680 (0.0010) -[2023-10-14 14:08:12,405][75950] Updated weights for policy 1, policy_version 17690 (0.0010) -[2023-10-14 14:08:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 36241408. Throughput: 0: 1673.2, 1: 1656.9. Samples: 9067870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) -[2023-10-14 14:08:13,164][74987] Avg episode reward: [(0, '16.740'), (1, '19.790')] -[2023-10-14 14:08:14,775][75949] Updated weights for policy 0, policy_version 17701 (0.0011) -[2023-10-14 14:08:15,149][75949] Updated weights for policy 0, policy_version 17711 (0.0009) -[2023-10-14 14:08:15,520][75949] Updated weights for policy 0, policy_version 17721 (0.0009) -[2023-10-14 14:08:16,423][75950] Updated weights for policy 1, policy_version 17700 (0.0008) -[2023-10-14 14:08:16,780][75950] Updated weights for policy 1, policy_version 17710 (0.0008) -[2023-10-14 14:08:17,144][75950] Updated weights for policy 1, policy_version 17720 (0.0008) -[2023-10-14 14:08:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 36306944. Throughput: 0: 1654.7, 1: 1671.8. Samples: 9078526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) -[2023-10-14 14:08:18,165][74987] Avg episode reward: [(0, '18.410'), (1, '18.910')] -[2023-10-14 14:08:18,166][75615] Saving new best policy, reward=18.410! -[2023-10-14 14:08:19,644][75949] Updated weights for policy 0, policy_version 17731 (0.0011) -[2023-10-14 14:08:20,026][75949] Updated weights for policy 0, policy_version 17741 (0.0007) -[2023-10-14 14:08:20,393][75949] Updated weights for policy 0, policy_version 17751 (0.0009) -[2023-10-14 14:08:21,110][75950] Updated weights for policy 1, policy_version 17730 (0.0010) -[2023-10-14 14:08:21,479][75950] Updated weights for policy 1, policy_version 17740 (0.0008) -[2023-10-14 14:08:21,847][75950] Updated weights for policy 1, policy_version 17750 (0.0009) -[2023-10-14 14:08:22,206][75950] Updated weights for policy 1, policy_version 17760 (0.0008) -[2023-10-14 14:08:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 36372480. Throughput: 0: 1673.9, 1: 1666.1. Samples: 9098728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:08:23,165][74987] Avg episode reward: [(0, '17.970'), (1, '19.230')] -[2023-10-14 14:08:24,373][75949] Updated weights for policy 0, policy_version 17761 (0.0010) -[2023-10-14 14:08:24,752][75949] Updated weights for policy 0, policy_version 17771 (0.0010) -[2023-10-14 14:08:25,118][75949] Updated weights for policy 0, policy_version 17781 (0.0008) -[2023-10-14 14:08:25,494][75949] Updated weights for policy 0, policy_version 17791 (0.0009) -[2023-10-14 14:08:26,277][75950] Updated weights for policy 1, policy_version 17770 (0.0010) -[2023-10-14 14:08:26,643][75950] Updated weights for policy 1, policy_version 17780 (0.0010) -[2023-10-14 14:08:27,008][75950] Updated weights for policy 1, policy_version 17790 (0.0009) -[2023-10-14 14:08:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 36438016. Throughput: 0: 1674.7, 1: 1674.3. Samples: 9118954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:08:28,164][74987] Avg episode reward: [(0, '17.300'), (1, '19.260')] -[2023-10-14 14:08:29,455][75949] Updated weights for policy 0, policy_version 17801 (0.0007) -[2023-10-14 14:08:29,819][75949] Updated weights for policy 0, policy_version 17811 (0.0010) -[2023-10-14 14:08:30,196][75949] Updated weights for policy 0, policy_version 17821 (0.0010) -[2023-10-14 14:08:30,896][75950] Updated weights for policy 1, policy_version 17800 (0.0007) -[2023-10-14 14:08:31,257][75950] Updated weights for policy 1, policy_version 17810 (0.0007) -[2023-10-14 14:08:31,631][75950] Updated weights for policy 1, policy_version 17820 (0.0009) -[2023-10-14 14:08:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 36503552. Throughput: 0: 1666.6, 1: 1686.4. Samples: 9129538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:08:33,164][74987] Avg episode reward: [(0, '18.750'), (1, '19.880')] -[2023-10-14 14:08:33,165][75615] Saving new best policy, reward=18.750! -[2023-10-14 14:08:34,146][75949] Updated weights for policy 0, policy_version 17831 (0.0009) -[2023-10-14 14:08:34,522][75949] Updated weights for policy 0, policy_version 17841 (0.0007) -[2023-10-14 14:08:34,892][75949] Updated weights for policy 0, policy_version 17851 (0.0009) -[2023-10-14 14:08:35,695][75950] Updated weights for policy 1, policy_version 17830 (0.0009) -[2023-10-14 14:08:36,056][75950] Updated weights for policy 1, policy_version 17840 (0.0009) -[2023-10-14 14:08:36,423][75950] Updated weights for policy 1, policy_version 17850 (0.0009) -[2023-10-14 14:08:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36569088. Throughput: 0: 1689.6, 1: 1664.2. Samples: 9149272. Policy #0 lag: (min: 1.0, avg: 13.7, max: 33.0) -[2023-10-14 14:08:38,164][74987] Avg episode reward: [(0, '17.590'), (1, '19.700')] -[2023-10-14 14:08:39,050][75949] Updated weights for policy 0, policy_version 17861 (0.0008) -[2023-10-14 14:08:39,420][75949] Updated weights for policy 0, policy_version 17871 (0.0009) -[2023-10-14 14:08:39,799][75949] Updated weights for policy 0, policy_version 17881 (0.0010) -[2023-10-14 14:08:40,514][75950] Updated weights for policy 1, policy_version 17860 (0.0008) -[2023-10-14 14:08:40,878][75950] Updated weights for policy 1, policy_version 17870 (0.0008) -[2023-10-14 14:08:41,240][75950] Updated weights for policy 1, policy_version 17880 (0.0008) -[2023-10-14 14:08:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 36634624. Throughput: 0: 1688.0, 1: 1683.3. Samples: 9169694. Policy #0 lag: (min: 1.0, avg: 13.7, max: 33.0) -[2023-10-14 14:08:43,164][74987] Avg episode reward: [(0, '18.500'), (1, '19.030')] -[2023-10-14 14:08:43,172][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000017888_18317312.pth... -[2023-10-14 14:08:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000017888_18317312.pth... -[2023-10-14 14:08:43,209][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000016320_16711680.pth -[2023-10-14 14:08:43,212][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000016320_16711680.pth -[2023-10-14 14:08:43,830][75949] Updated weights for policy 0, policy_version 17891 (0.0009) -[2023-10-14 14:08:44,200][75949] Updated weights for policy 0, policy_version 17901 (0.0009) -[2023-10-14 14:08:44,568][75949] Updated weights for policy 0, policy_version 17911 (0.0010) -[2023-10-14 14:08:45,364][75950] Updated weights for policy 1, policy_version 17890 (0.0007) -[2023-10-14 14:08:45,734][75950] Updated weights for policy 1, policy_version 17900 (0.0007) -[2023-10-14 14:08:46,106][75950] Updated weights for policy 1, policy_version 17910 (0.0008) -[2023-10-14 14:08:46,478][75950] Updated weights for policy 1, policy_version 17920 (0.0007) -[2023-10-14 14:08:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36700160. Throughput: 0: 1685.8, 1: 1675.8. Samples: 9179728. Policy #0 lag: (min: 1.0, avg: 13.7, max: 33.0) -[2023-10-14 14:08:48,164][74987] Avg episode reward: [(0, '18.260'), (1, '19.000')] -[2023-10-14 14:08:48,577][75949] Updated weights for policy 0, policy_version 17921 (0.0010) -[2023-10-14 14:08:48,949][75949] Updated weights for policy 0, policy_version 17931 (0.0009) -[2023-10-14 14:08:49,327][75949] Updated weights for policy 0, policy_version 17941 (0.0009) -[2023-10-14 14:08:49,686][75949] Updated weights for policy 0, policy_version 17951 (0.0009) -[2023-10-14 14:08:50,685][75950] Updated weights for policy 1, policy_version 17930 (0.0009) -[2023-10-14 14:08:51,054][75950] Updated weights for policy 1, policy_version 17940 (0.0009) -[2023-10-14 14:08:51,419][75950] Updated weights for policy 1, policy_version 17950 (0.0008) -[2023-10-14 14:08:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 36765696. Throughput: 0: 1696.7, 1: 1658.5. Samples: 9199342. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:08:53,165][74987] Avg episode reward: [(0, '18.660'), (1, '18.590')] -[2023-10-14 14:08:53,755][75949] Updated weights for policy 0, policy_version 17961 (0.0010) -[2023-10-14 14:08:54,117][75949] Updated weights for policy 0, policy_version 17971 (0.0008) -[2023-10-14 14:08:54,485][75949] Updated weights for policy 0, policy_version 17981 (0.0009) -[2023-10-14 14:08:55,420][75950] Updated weights for policy 1, policy_version 17960 (0.0008) -[2023-10-14 14:08:55,780][75950] Updated weights for policy 1, policy_version 17970 (0.0008) -[2023-10-14 14:08:56,145][75950] Updated weights for policy 1, policy_version 17980 (0.0008) -[2023-10-14 14:08:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 36831232. Throughput: 0: 1696.3, 1: 1684.3. Samples: 9220000. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:08:58,165][74987] Avg episode reward: [(0, '17.830'), (1, '19.460')] -[2023-10-14 14:08:58,643][75949] Updated weights for policy 0, policy_version 17991 (0.0009) -[2023-10-14 14:08:59,015][75949] Updated weights for policy 0, policy_version 18001 (0.0009) -[2023-10-14 14:08:59,390][75949] Updated weights for policy 0, policy_version 18011 (0.0008) -[2023-10-14 14:09:00,436][75950] Updated weights for policy 1, policy_version 17990 (0.0010) -[2023-10-14 14:09:00,814][75950] Updated weights for policy 1, policy_version 18000 (0.0008) -[2023-10-14 14:09:01,184][75950] Updated weights for policy 1, policy_version 18010 (0.0009) -[2023-10-14 14:09:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 36896768. Throughput: 0: 1689.7, 1: 1672.9. Samples: 9229842. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:09:03,165][74987] Avg episode reward: [(0, '18.370'), (1, '18.910')] -[2023-10-14 14:09:03,490][75949] Updated weights for policy 0, policy_version 18021 (0.0009) -[2023-10-14 14:09:03,869][75949] Updated weights for policy 0, policy_version 18031 (0.0009) -[2023-10-14 14:09:04,237][75949] Updated weights for policy 0, policy_version 18041 (0.0011) -[2023-10-14 14:09:05,279][75950] Updated weights for policy 1, policy_version 18020 (0.0011) -[2023-10-14 14:09:05,643][75950] Updated weights for policy 1, policy_version 18030 (0.0011) -[2023-10-14 14:09:06,013][75950] Updated weights for policy 1, policy_version 18040 (0.0009) -[2023-10-14 14:09:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36962304. Throughput: 0: 1695.2, 1: 1662.5. Samples: 9249824. Policy #0 lag: (min: 9.0, avg: 20.9, max: 41.0) -[2023-10-14 14:09:08,164][74987] Avg episode reward: [(0, '17.690'), (1, '20.010')] -[2023-10-14 14:09:08,273][75949] Updated weights for policy 0, policy_version 18051 (0.0009) -[2023-10-14 14:09:08,650][75949] Updated weights for policy 0, policy_version 18061 (0.0009) -[2023-10-14 14:09:09,008][75949] Updated weights for policy 0, policy_version 18071 (0.0010) -[2023-10-14 14:09:10,142][75950] Updated weights for policy 1, policy_version 18050 (0.0008) -[2023-10-14 14:09:10,511][75950] Updated weights for policy 1, policy_version 18060 (0.0007) -[2023-10-14 14:09:10,878][75950] Updated weights for policy 1, policy_version 18070 (0.0008) -[2023-10-14 14:09:11,245][75950] Updated weights for policy 1, policy_version 18080 (0.0009) -[2023-10-14 14:09:12,985][75949] Updated weights for policy 0, policy_version 18081 (0.0009) -[2023-10-14 14:09:13,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37027840. Throughput: 0: 1691.2, 1: 1675.1. Samples: 9270438. Policy #0 lag: (min: 9.0, avg: 20.9, max: 41.0) -[2023-10-14 14:09:13,164][74987] Avg episode reward: [(0, '18.360'), (1, '19.220')] -[2023-10-14 14:09:13,358][75949] Updated weights for policy 0, policy_version 18091 (0.0008) -[2023-10-14 14:09:13,716][75949] Updated weights for policy 0, policy_version 18101 (0.0010) -[2023-10-14 14:09:14,090][75949] Updated weights for policy 0, policy_version 18111 (0.0010) -[2023-10-14 14:09:15,482][75950] Updated weights for policy 1, policy_version 18090 (0.0008) -[2023-10-14 14:09:15,854][75950] Updated weights for policy 1, policy_version 18100 (0.0008) -[2023-10-14 14:09:16,218][75950] Updated weights for policy 1, policy_version 18110 (0.0009) -[2023-10-14 14:09:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37093376. Throughput: 0: 1685.8, 1: 1658.9. Samples: 9280048. Policy #0 lag: (min: 9.0, avg: 20.9, max: 41.0) -[2023-10-14 14:09:18,165][74987] Avg episode reward: [(0, '18.490'), (1, '19.880')] -[2023-10-14 14:09:18,219][75949] Updated weights for policy 0, policy_version 18121 (0.0009) -[2023-10-14 14:09:18,587][75949] Updated weights for policy 0, policy_version 18131 (0.0008) -[2023-10-14 14:09:18,959][75949] Updated weights for policy 0, policy_version 18141 (0.0007) -[2023-10-14 14:09:20,285][75950] Updated weights for policy 1, policy_version 18120 (0.0007) -[2023-10-14 14:09:20,652][75950] Updated weights for policy 1, policy_version 18130 (0.0009) -[2023-10-14 14:09:21,025][75950] Updated weights for policy 1, policy_version 18140 (0.0010) -[2023-10-14 14:09:23,141][75949] Updated weights for policy 0, policy_version 18151 (0.0009) -[2023-10-14 14:09:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37158912. Throughput: 0: 1681.2, 1: 1667.8. Samples: 9299976. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-14 14:09:23,165][74987] Avg episode reward: [(0, '18.600'), (1, '19.290')] -[2023-10-14 14:09:23,514][75949] Updated weights for policy 0, policy_version 18161 (0.0010) -[2023-10-14 14:09:23,875][75949] Updated weights for policy 0, policy_version 18171 (0.0007) -[2023-10-14 14:09:24,809][75950] Updated weights for policy 1, policy_version 18150 (0.0008) -[2023-10-14 14:09:25,179][75950] Updated weights for policy 1, policy_version 18160 (0.0010) -[2023-10-14 14:09:25,549][75950] Updated weights for policy 1, policy_version 18170 (0.0007) -[2023-10-14 14:09:27,935][75949] Updated weights for policy 0, policy_version 18181 (0.0009) -[2023-10-14 14:09:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 37224448. Throughput: 0: 1683.5, 1: 1678.1. Samples: 9320968. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-14 14:09:28,165][74987] Avg episode reward: [(0, '18.830'), (1, '19.220')] -[2023-10-14 14:09:28,298][75949] Updated weights for policy 0, policy_version 18191 (0.0007) -[2023-10-14 14:09:28,670][75949] Updated weights for policy 0, policy_version 18201 (0.0008) -[2023-10-14 14:09:28,919][75615] Saving new best policy, reward=18.830! -[2023-10-14 14:09:29,450][75950] Updated weights for policy 1, policy_version 18180 (0.0008) -[2023-10-14 14:09:29,825][75950] Updated weights for policy 1, policy_version 18190 (0.0009) -[2023-10-14 14:09:30,191][75950] Updated weights for policy 1, policy_version 18200 (0.0007) -[2023-10-14 14:09:32,560][75949] Updated weights for policy 0, policy_version 18211 (0.0008) -[2023-10-14 14:09:32,940][75949] Updated weights for policy 0, policy_version 18221 (0.0011) -[2023-10-14 14:09:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37289984. Throughput: 0: 1684.0, 1: 1660.8. Samples: 9330248. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) -[2023-10-14 14:09:33,164][74987] Avg episode reward: [(0, '19.200'), (1, '20.230')] -[2023-10-14 14:09:33,300][75949] Updated weights for policy 0, policy_version 18231 (0.0011) -[2023-10-14 14:09:33,632][75615] Saving new best policy, reward=19.200! -[2023-10-14 14:09:34,503][75950] Updated weights for policy 1, policy_version 18210 (0.0008) -[2023-10-14 14:09:34,865][75950] Updated weights for policy 1, policy_version 18220 (0.0009) -[2023-10-14 14:09:35,235][75950] Updated weights for policy 1, policy_version 18230 (0.0008) -[2023-10-14 14:09:35,605][75950] Updated weights for policy 1, policy_version 18240 (0.0008) -[2023-10-14 14:09:37,299][75949] Updated weights for policy 0, policy_version 18241 (0.0010) -[2023-10-14 14:09:37,702][75949] Updated weights for policy 0, policy_version 18251 (0.0009) -[2023-10-14 14:09:38,079][75949] Updated weights for policy 0, policy_version 18261 (0.0009) -[2023-10-14 14:09:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37355520. Throughput: 0: 1683.8, 1: 1683.2. Samples: 9350860. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) -[2023-10-14 14:09:38,165][74987] Avg episode reward: [(0, '18.910'), (1, '20.090')] -[2023-10-14 14:09:38,449][75949] Updated weights for policy 0, policy_version 18271 (0.0010) -[2023-10-14 14:09:39,729][75950] Updated weights for policy 1, policy_version 18250 (0.0009) -[2023-10-14 14:09:40,100][75950] Updated weights for policy 1, policy_version 18260 (0.0011) -[2023-10-14 14:09:40,471][75950] Updated weights for policy 1, policy_version 18270 (0.0012) -[2023-10-14 14:09:42,599][75949] Updated weights for policy 0, policy_version 18281 (0.0008) -[2023-10-14 14:09:42,969][75949] Updated weights for policy 0, policy_version 18291 (0.0009) -[2023-10-14 14:09:43,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 37421056. Throughput: 0: 1673.9, 1: 1677.3. Samples: 9370802. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) -[2023-10-14 14:09:43,164][74987] Avg episode reward: [(0, '19.650'), (1, '20.220')] -[2023-10-14 14:09:43,345][75949] Updated weights for policy 0, policy_version 18301 (0.0009) -[2023-10-14 14:09:43,454][75615] Saving new best policy, reward=19.650! -[2023-10-14 14:09:44,724][75950] Updated weights for policy 1, policy_version 18280 (0.0010) -[2023-10-14 14:09:45,095][75950] Updated weights for policy 1, policy_version 18290 (0.0010) -[2023-10-14 14:09:45,460][75950] Updated weights for policy 1, policy_version 18300 (0.0010) -[2023-10-14 14:09:47,389][75949] Updated weights for policy 0, policy_version 18311 (0.0009) -[2023-10-14 14:09:47,758][75949] Updated weights for policy 0, policy_version 18321 (0.0008) -[2023-10-14 14:09:48,123][75949] Updated weights for policy 0, policy_version 18331 (0.0008) -[2023-10-14 14:09:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 37486592. Throughput: 0: 1688.1, 1: 1656.7. Samples: 9380354. Policy #0 lag: (min: 0.0, avg: 23.2, max: 32.0) -[2023-10-14 14:09:48,164][74987] Avg episode reward: [(0, '18.610'), (1, '19.220')] -[2023-10-14 14:09:49,628][75950] Updated weights for policy 1, policy_version 18310 (0.0008) -[2023-10-14 14:09:49,996][75950] Updated weights for policy 1, policy_version 18320 (0.0007) -[2023-10-14 14:09:50,352][75950] Updated weights for policy 1, policy_version 18330 (0.0007) -[2023-10-14 14:09:52,132][75949] Updated weights for policy 0, policy_version 18341 (0.0009) -[2023-10-14 14:09:52,508][75949] Updated weights for policy 0, policy_version 18351 (0.0008) -[2023-10-14 14:09:52,876][75949] Updated weights for policy 0, policy_version 18361 (0.0009) -[2023-10-14 14:09:53,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 37584896. Throughput: 0: 1686.0, 1: 1670.6. Samples: 9400868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:09:53,165][74987] Avg episode reward: [(0, '18.240'), (1, '19.530')] -[2023-10-14 14:09:54,466][75950] Updated weights for policy 1, policy_version 18340 (0.0007) -[2023-10-14 14:09:54,872][75950] Updated weights for policy 1, policy_version 18350 (0.0008) -[2023-10-14 14:09:55,244][75950] Updated weights for policy 1, policy_version 18360 (0.0007) -[2023-10-14 14:09:56,997][75949] Updated weights for policy 0, policy_version 18371 (0.0009) -[2023-10-14 14:09:57,374][75949] Updated weights for policy 0, policy_version 18381 (0.0007) -[2023-10-14 14:09:57,740][75949] Updated weights for policy 0, policy_version 18391 (0.0007) -[2023-10-14 14:09:58,164][74987] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 37650432. Throughput: 0: 1675.7, 1: 1668.1. Samples: 9420910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:09:58,165][74987] Avg episode reward: [(0, '17.950'), (1, '18.820')] -[2023-10-14 14:09:59,351][75950] Updated weights for policy 1, policy_version 18370 (0.0008) -[2023-10-14 14:09:59,707][75950] Updated weights for policy 1, policy_version 18380 (0.0011) -[2023-10-14 14:10:00,076][75950] Updated weights for policy 1, policy_version 18390 (0.0011) -[2023-10-14 14:10:00,447][75950] Updated weights for policy 1, policy_version 18400 (0.0011) -[2023-10-14 14:10:01,821][75949] Updated weights for policy 0, policy_version 18401 (0.0009) -[2023-10-14 14:10:02,199][75949] Updated weights for policy 0, policy_version 18411 (0.0009) -[2023-10-14 14:10:02,566][75949] Updated weights for policy 0, policy_version 18421 (0.0007) -[2023-10-14 14:10:02,939][75949] Updated weights for policy 0, policy_version 18431 (0.0007) -[2023-10-14 14:10:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 37715968. Throughput: 0: 1693.4, 1: 1652.7. Samples: 9430622. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) -[2023-10-14 14:10:03,165][74987] Avg episode reward: [(0, '17.620'), (1, '20.550')] -[2023-10-14 14:10:04,570][75950] Updated weights for policy 1, policy_version 18410 (0.0009) -[2023-10-14 14:10:04,934][75950] Updated weights for policy 1, policy_version 18420 (0.0010) -[2023-10-14 14:10:05,306][75950] Updated weights for policy 1, policy_version 18430 (0.0009) -[2023-10-14 14:10:07,108][75949] Updated weights for policy 0, policy_version 18441 (0.0009) -[2023-10-14 14:10:07,479][75949] Updated weights for policy 0, policy_version 18451 (0.0009) -[2023-10-14 14:10:07,853][75949] Updated weights for policy 0, policy_version 18461 (0.0009) -[2023-10-14 14:10:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 37781504. Throughput: 0: 1693.6, 1: 1664.7. Samples: 9451098. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) -[2023-10-14 14:10:08,165][74987] Avg episode reward: [(0, '17.500'), (1, '18.210')] -[2023-10-14 14:10:09,554][75950] Updated weights for policy 1, policy_version 18440 (0.0011) -[2023-10-14 14:10:09,929][75950] Updated weights for policy 1, policy_version 18450 (0.0010) -[2023-10-14 14:10:10,286][75950] Updated weights for policy 1, policy_version 18460 (0.0008) -[2023-10-14 14:10:11,755][75949] Updated weights for policy 0, policy_version 18471 (0.0010) -[2023-10-14 14:10:12,128][75949] Updated weights for policy 0, policy_version 18481 (0.0008) -[2023-10-14 14:10:12,503][75949] Updated weights for policy 0, policy_version 18491 (0.0007) -[2023-10-14 14:10:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 37847040. Throughput: 0: 1666.4, 1: 1659.6. Samples: 9470640. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) -[2023-10-14 14:10:13,165][74987] Avg episode reward: [(0, '18.030'), (1, '19.310')] -[2023-10-14 14:10:14,261][75950] Updated weights for policy 1, policy_version 18470 (0.0008) -[2023-10-14 14:10:14,625][75950] Updated weights for policy 1, policy_version 18480 (0.0008) -[2023-10-14 14:10:15,000][75950] Updated weights for policy 1, policy_version 18490 (0.0007) -[2023-10-14 14:10:16,409][75949] Updated weights for policy 0, policy_version 18501 (0.0008) -[2023-10-14 14:10:16,776][75949] Updated weights for policy 0, policy_version 18511 (0.0008) -[2023-10-14 14:10:17,149][75949] Updated weights for policy 0, policy_version 18521 (0.0011) -[2023-10-14 14:10:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 37912576. Throughput: 0: 1689.0, 1: 1658.0. Samples: 9480862. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) -[2023-10-14 14:10:18,164][74987] Avg episode reward: [(0, '19.230'), (1, '17.940')] -[2023-10-14 14:10:19,005][75950] Updated weights for policy 1, policy_version 18500 (0.0008) -[2023-10-14 14:10:19,360][75950] Updated weights for policy 1, policy_version 18510 (0.0009) -[2023-10-14 14:10:19,734][75950] Updated weights for policy 1, policy_version 18520 (0.0009) -[2023-10-14 14:10:21,140][75949] Updated weights for policy 0, policy_version 18531 (0.0009) -[2023-10-14 14:10:21,510][75949] Updated weights for policy 0, policy_version 18541 (0.0009) -[2023-10-14 14:10:21,888][75949] Updated weights for policy 0, policy_version 18551 (0.0008) -[2023-10-14 14:10:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 37978112. Throughput: 0: 1675.5, 1: 1659.7. Samples: 9500944. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) -[2023-10-14 14:10:23,165][74987] Avg episode reward: [(0, '20.260'), (1, '20.000')] -[2023-10-14 14:10:23,166][75615] Saving new best policy, reward=20.260! -[2023-10-14 14:10:23,887][75950] Updated weights for policy 1, policy_version 18530 (0.0009) -[2023-10-14 14:10:24,253][75950] Updated weights for policy 1, policy_version 18540 (0.0009) -[2023-10-14 14:10:24,633][75950] Updated weights for policy 1, policy_version 18550 (0.0009) -[2023-10-14 14:10:25,002][75950] Updated weights for policy 1, policy_version 18560 (0.0008) -[2023-10-14 14:10:25,838][75949] Updated weights for policy 0, policy_version 18561 (0.0008) -[2023-10-14 14:10:26,220][75949] Updated weights for policy 0, policy_version 18571 (0.0010) -[2023-10-14 14:10:26,595][75949] Updated weights for policy 0, policy_version 18581 (0.0009) -[2023-10-14 14:10:26,968][75949] Updated weights for policy 0, policy_version 18591 (0.0010) -[2023-10-14 14:10:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 38043648. Throughput: 0: 1672.3, 1: 1665.2. Samples: 9520992. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) -[2023-10-14 14:10:28,164][74987] Avg episode reward: [(0, '17.830'), (1, '18.810')] -[2023-10-14 14:10:29,207][75950] Updated weights for policy 1, policy_version 18570 (0.0011) -[2023-10-14 14:10:29,585][75950] Updated weights for policy 1, policy_version 18580 (0.0008) -[2023-10-14 14:10:29,944][75950] Updated weights for policy 1, policy_version 18590 (0.0007) -[2023-10-14 14:10:31,083][75949] Updated weights for policy 0, policy_version 18601 (0.0009) -[2023-10-14 14:10:31,444][75949] Updated weights for policy 0, policy_version 18611 (0.0011) -[2023-10-14 14:10:31,823][75949] Updated weights for policy 0, policy_version 18621 (0.0008) -[2023-10-14 14:10:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 38109184. Throughput: 0: 1691.7, 1: 1666.2. Samples: 9531460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:10:33,165][74987] Avg episode reward: [(0, '16.820'), (1, '19.170')] -[2023-10-14 14:10:33,846][75950] Updated weights for policy 1, policy_version 18600 (0.0008) -[2023-10-14 14:10:34,199][75950] Updated weights for policy 1, policy_version 18610 (0.0009) -[2023-10-14 14:10:34,569][75950] Updated weights for policy 1, policy_version 18620 (0.0009) -[2023-10-14 14:10:35,897][75949] Updated weights for policy 0, policy_version 18631 (0.0008) -[2023-10-14 14:10:36,271][75949] Updated weights for policy 0, policy_version 18641 (0.0010) -[2023-10-14 14:10:36,643][75949] Updated weights for policy 0, policy_version 18651 (0.0008) -[2023-10-14 14:10:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 38174720. Throughput: 0: 1668.6, 1: 1674.9. Samples: 9551326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:10:38,165][74987] Avg episode reward: [(0, '17.880'), (1, '18.880')] -[2023-10-14 14:10:38,723][75950] Updated weights for policy 1, policy_version 18630 (0.0009) -[2023-10-14 14:10:39,077][75950] Updated weights for policy 1, policy_version 18640 (0.0011) -[2023-10-14 14:10:39,451][75950] Updated weights for policy 1, policy_version 18650 (0.0010) -[2023-10-14 14:10:40,767][75949] Updated weights for policy 0, policy_version 18661 (0.0009) -[2023-10-14 14:10:41,143][75949] Updated weights for policy 0, policy_version 18671 (0.0012) -[2023-10-14 14:10:41,521][75949] Updated weights for policy 0, policy_version 18681 (0.0011) -[2023-10-14 14:10:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 38240256. Throughput: 0: 1673.0, 1: 1679.1. Samples: 9571752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:10:43,164][74987] Avg episode reward: [(0, '19.110'), (1, '20.160')] -[2023-10-14 14:10:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000018656_19103744.pth... -[2023-10-14 14:10:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000018688_19136512.pth... -[2023-10-14 14:10:43,204][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000017120_17530880.pth -[2023-10-14 14:10:43,211][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000017120_17530880.pth -[2023-10-14 14:10:43,535][75950] Updated weights for policy 1, policy_version 18660 (0.0009) -[2023-10-14 14:10:43,930][75950] Updated weights for policy 1, policy_version 18670 (0.0009) -[2023-10-14 14:10:44,302][75950] Updated weights for policy 1, policy_version 18680 (0.0009) -[2023-10-14 14:10:45,602][75949] Updated weights for policy 0, policy_version 18691 (0.0010) -[2023-10-14 14:10:45,977][75949] Updated weights for policy 0, policy_version 18701 (0.0010) -[2023-10-14 14:10:46,351][75949] Updated weights for policy 0, policy_version 18711 (0.0010) -[2023-10-14 14:10:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 38305792. Throughput: 0: 1680.9, 1: 1678.3. Samples: 9581784. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 14:10:48,164][74987] Avg episode reward: [(0, '19.080'), (1, '20.640')] -[2023-10-14 14:10:48,442][75950] Updated weights for policy 1, policy_version 18690 (0.0010) -[2023-10-14 14:10:48,809][75950] Updated weights for policy 1, policy_version 18700 (0.0008) -[2023-10-14 14:10:49,178][75950] Updated weights for policy 1, policy_version 18710 (0.0008) -[2023-10-14 14:10:49,546][75950] Updated weights for policy 1, policy_version 18720 (0.0008) -[2023-10-14 14:10:50,401][75949] Updated weights for policy 0, policy_version 18721 (0.0009) -[2023-10-14 14:10:50,773][75949] Updated weights for policy 0, policy_version 18731 (0.0007) -[2023-10-14 14:10:51,136][75949] Updated weights for policy 0, policy_version 18741 (0.0009) -[2023-10-14 14:10:51,507][75949] Updated weights for policy 0, policy_version 18751 (0.0009) -[2023-10-14 14:10:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 38371328. Throughput: 0: 1660.1, 1: 1676.0. Samples: 9601222. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 14:10:53,164][74987] Avg episode reward: [(0, '18.710'), (1, '20.420')] -[2023-10-14 14:10:53,573][75950] Updated weights for policy 1, policy_version 18730 (0.0008) -[2023-10-14 14:10:53,944][75950] Updated weights for policy 1, policy_version 18740 (0.0010) -[2023-10-14 14:10:54,307][75950] Updated weights for policy 1, policy_version 18750 (0.0008) -[2023-10-14 14:10:55,433][75949] Updated weights for policy 0, policy_version 18761 (0.0007) -[2023-10-14 14:10:55,810][75949] Updated weights for policy 0, policy_version 18771 (0.0008) -[2023-10-14 14:10:56,186][75949] Updated weights for policy 0, policy_version 18781 (0.0009) -[2023-10-14 14:10:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 38436864. Throughput: 0: 1689.3, 1: 1676.5. Samples: 9622100. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 14:10:58,164][74987] Avg episode reward: [(0, '18.900'), (1, '19.360')] -[2023-10-14 14:10:58,419][75950] Updated weights for policy 1, policy_version 18760 (0.0008) -[2023-10-14 14:10:58,780][75950] Updated weights for policy 1, policy_version 18770 (0.0007) -[2023-10-14 14:10:59,148][75950] Updated weights for policy 1, policy_version 18780 (0.0009) -[2023-10-14 14:11:00,090][75949] Updated weights for policy 0, policy_version 18791 (0.0009) -[2023-10-14 14:11:00,461][75949] Updated weights for policy 0, policy_version 18801 (0.0009) -[2023-10-14 14:11:00,824][75949] Updated weights for policy 0, policy_version 18811 (0.0008) -[2023-10-14 14:11:03,007][75950] Updated weights for policy 1, policy_version 18790 (0.0011) -[2023-10-14 14:11:03,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 38502400. Throughput: 0: 1676.4, 1: 1679.7. Samples: 9631888. Policy #0 lag: (min: 28.0, avg: 30.8, max: 60.0) -[2023-10-14 14:11:03,165][74987] Avg episode reward: [(0, '17.770'), (1, '19.020')] -[2023-10-14 14:11:03,372][75950] Updated weights for policy 1, policy_version 18800 (0.0007) -[2023-10-14 14:11:03,743][75950] Updated weights for policy 1, policy_version 18810 (0.0010) -[2023-10-14 14:11:04,793][75949] Updated weights for policy 0, policy_version 18821 (0.0009) -[2023-10-14 14:11:05,152][75949] Updated weights for policy 0, policy_version 18831 (0.0012) -[2023-10-14 14:11:05,520][75949] Updated weights for policy 0, policy_version 18841 (0.0011) -[2023-10-14 14:11:07,931][75950] Updated weights for policy 1, policy_version 18820 (0.0008) -[2023-10-14 14:11:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 38567936. Throughput: 0: 1682.4, 1: 1681.3. Samples: 9652314. Policy #0 lag: (min: 28.0, avg: 30.8, max: 60.0) -[2023-10-14 14:11:08,164][74987] Avg episode reward: [(0, '18.000'), (1, '19.150')] -[2023-10-14 14:11:08,292][75950] Updated weights for policy 1, policy_version 18830 (0.0008) -[2023-10-14 14:11:08,660][75950] Updated weights for policy 1, policy_version 18840 (0.0010) -[2023-10-14 14:11:09,612][75949] Updated weights for policy 0, policy_version 18851 (0.0007) -[2023-10-14 14:11:09,979][75949] Updated weights for policy 0, policy_version 18861 (0.0009) -[2023-10-14 14:11:10,343][75949] Updated weights for policy 0, policy_version 18871 (0.0008) -[2023-10-14 14:11:12,771][75950] Updated weights for policy 1, policy_version 18850 (0.0008) -[2023-10-14 14:11:13,139][75950] Updated weights for policy 1, policy_version 18860 (0.0007) -[2023-10-14 14:11:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 38633472. Throughput: 0: 1696.4, 1: 1679.2. Samples: 9672894. Policy #0 lag: (min: 28.0, avg: 30.8, max: 60.0) -[2023-10-14 14:11:13,165][74987] Avg episode reward: [(0, '17.020'), (1, '19.080')] -[2023-10-14 14:11:13,508][75950] Updated weights for policy 1, policy_version 18870 (0.0008) -[2023-10-14 14:11:13,882][75950] Updated weights for policy 1, policy_version 18880 (0.0009) -[2023-10-14 14:11:14,453][75949] Updated weights for policy 0, policy_version 18881 (0.0008) -[2023-10-14 14:11:14,873][75949] Updated weights for policy 0, policy_version 18891 (0.0008) -[2023-10-14 14:11:15,241][75949] Updated weights for policy 0, policy_version 18901 (0.0010) -[2023-10-14 14:11:15,612][75949] Updated weights for policy 0, policy_version 18911 (0.0007) -[2023-10-14 14:11:17,991][75950] Updated weights for policy 1, policy_version 18890 (0.0010) -[2023-10-14 14:11:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 38699008. Throughput: 0: 1664.0, 1: 1679.2. Samples: 9681904. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 14:11:18,165][74987] Avg episode reward: [(0, '19.430'), (1, '20.150')] -[2023-10-14 14:11:18,354][75950] Updated weights for policy 1, policy_version 18900 (0.0011) -[2023-10-14 14:11:18,727][75950] Updated weights for policy 1, policy_version 18910 (0.0011) -[2023-10-14 14:11:19,531][75949] Updated weights for policy 0, policy_version 18921 (0.0008) -[2023-10-14 14:11:19,900][75949] Updated weights for policy 0, policy_version 18931 (0.0007) -[2023-10-14 14:11:20,272][75949] Updated weights for policy 0, policy_version 18941 (0.0008) -[2023-10-14 14:11:22,971][75950] Updated weights for policy 1, policy_version 18920 (0.0007) -[2023-10-14 14:11:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 38764544. Throughput: 0: 1688.6, 1: 1671.6. Samples: 9702536. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 14:11:23,165][74987] Avg episode reward: [(0, '17.880'), (1, '19.180')] -[2023-10-14 14:11:23,338][75950] Updated weights for policy 1, policy_version 18930 (0.0007) -[2023-10-14 14:11:23,709][75950] Updated weights for policy 1, policy_version 18940 (0.0007) -[2023-10-14 14:11:24,464][75949] Updated weights for policy 0, policy_version 18951 (0.0007) -[2023-10-14 14:11:24,838][75949] Updated weights for policy 0, policy_version 18961 (0.0010) -[2023-10-14 14:11:25,208][75949] Updated weights for policy 0, policy_version 18971 (0.0009) -[2023-10-14 14:11:27,798][75950] Updated weights for policy 1, policy_version 18950 (0.0010) -[2023-10-14 14:11:28,164][75950] Updated weights for policy 1, policy_version 18960 (0.0010) -[2023-10-14 14:11:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 38830080. Throughput: 0: 1696.0, 1: 1668.3. Samples: 9723142. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 14:11:28,164][74987] Avg episode reward: [(0, '19.710'), (1, '20.510')] -[2023-10-14 14:11:28,533][75950] Updated weights for policy 1, policy_version 18970 (0.0008) -[2023-10-14 14:11:29,202][75949] Updated weights for policy 0, policy_version 18981 (0.0009) -[2023-10-14 14:11:29,569][75949] Updated weights for policy 0, policy_version 18991 (0.0012) -[2023-10-14 14:11:29,943][75949] Updated weights for policy 0, policy_version 19001 (0.0011) -[2023-10-14 14:11:32,590][75950] Updated weights for policy 1, policy_version 18980 (0.0008) -[2023-10-14 14:11:32,976][75950] Updated weights for policy 1, policy_version 18990 (0.0007) -[2023-10-14 14:11:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 38895616. Throughput: 0: 1671.9, 1: 1671.6. Samples: 9732240. Policy #0 lag: (min: 27.0, avg: 27.1, max: 35.0) -[2023-10-14 14:11:33,164][74987] Avg episode reward: [(0, '18.130'), (1, '19.010')] -[2023-10-14 14:11:33,333][75950] Updated weights for policy 1, policy_version 19000 (0.0007) -[2023-10-14 14:11:34,138][75949] Updated weights for policy 0, policy_version 19011 (0.0010) -[2023-10-14 14:11:34,500][75949] Updated weights for policy 0, policy_version 19021 (0.0010) -[2023-10-14 14:11:34,865][75949] Updated weights for policy 0, policy_version 19031 (0.0012) -[2023-10-14 14:11:37,415][75950] Updated weights for policy 1, policy_version 19010 (0.0008) -[2023-10-14 14:11:37,777][75950] Updated weights for policy 1, policy_version 19020 (0.0008) -[2023-10-14 14:11:38,130][75950] Updated weights for policy 1, policy_version 19030 (0.0007) -[2023-10-14 14:11:38,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 38961152. Throughput: 0: 1694.1, 1: 1673.2. Samples: 9752750. Policy #0 lag: (min: 27.0, avg: 27.1, max: 35.0) -[2023-10-14 14:11:38,164][74987] Avg episode reward: [(0, '19.700'), (1, '20.030')] -[2023-10-14 14:11:38,495][75950] Updated weights for policy 1, policy_version 19040 (0.0009) -[2023-10-14 14:11:39,079][75949] Updated weights for policy 0, policy_version 19041 (0.0009) -[2023-10-14 14:11:39,450][75949] Updated weights for policy 0, policy_version 19051 (0.0008) -[2023-10-14 14:11:39,810][75949] Updated weights for policy 0, policy_version 19061 (0.0009) -[2023-10-14 14:11:40,180][75949] Updated weights for policy 0, policy_version 19071 (0.0012) -[2023-10-14 14:11:42,618][75950] Updated weights for policy 1, policy_version 19050 (0.0007) -[2023-10-14 14:11:42,984][75950] Updated weights for policy 1, policy_version 19060 (0.0007) -[2023-10-14 14:11:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39026688. Throughput: 0: 1694.4, 1: 1663.2. Samples: 9773194. Policy #0 lag: (min: 27.0, avg: 27.1, max: 35.0) -[2023-10-14 14:11:43,165][74987] Avg episode reward: [(0, '19.260'), (1, '18.290')] -[2023-10-14 14:11:43,348][75950] Updated weights for policy 1, policy_version 19070 (0.0007) -[2023-10-14 14:11:44,279][75949] Updated weights for policy 0, policy_version 19081 (0.0010) -[2023-10-14 14:11:44,642][75949] Updated weights for policy 0, policy_version 19091 (0.0009) -[2023-10-14 14:11:45,015][75949] Updated weights for policy 0, policy_version 19101 (0.0008) -[2023-10-14 14:11:47,244][75950] Updated weights for policy 1, policy_version 19080 (0.0009) -[2023-10-14 14:11:47,618][75950] Updated weights for policy 1, policy_version 19090 (0.0009) -[2023-10-14 14:11:47,994][75950] Updated weights for policy 1, policy_version 19100 (0.0010) -[2023-10-14 14:11:48,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 39124992. Throughput: 0: 1679.3, 1: 1671.5. Samples: 9782670. Policy #0 lag: (min: 26.0, avg: 26.8, max: 40.0) -[2023-10-14 14:11:48,164][74987] Avg episode reward: [(0, '19.000'), (1, '19.790')] -[2023-10-14 14:11:49,107][75949] Updated weights for policy 0, policy_version 19111 (0.0011) -[2023-10-14 14:11:49,469][75949] Updated weights for policy 0, policy_version 19121 (0.0011) -[2023-10-14 14:11:49,841][75949] Updated weights for policy 0, policy_version 19131 (0.0010) -[2023-10-14 14:11:52,005][75950] Updated weights for policy 1, policy_version 19110 (0.0010) -[2023-10-14 14:11:52,374][75950] Updated weights for policy 1, policy_version 19120 (0.0011) -[2023-10-14 14:11:52,746][75950] Updated weights for policy 1, policy_version 19130 (0.0009) -[2023-10-14 14:11:53,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 39190528. Throughput: 0: 1679.7, 1: 1671.6. Samples: 9803124. Policy #0 lag: (min: 26.0, avg: 26.8, max: 40.0) -[2023-10-14 14:11:53,165][74987] Avg episode reward: [(0, '18.330'), (1, '19.480')] -[2023-10-14 14:11:53,939][75949] Updated weights for policy 0, policy_version 19141 (0.0009) -[2023-10-14 14:11:54,305][75949] Updated weights for policy 0, policy_version 19151 (0.0007) -[2023-10-14 14:11:54,672][75949] Updated weights for policy 0, policy_version 19161 (0.0007) -[2023-10-14 14:11:56,860][75950] Updated weights for policy 1, policy_version 19140 (0.0010) -[2023-10-14 14:11:57,233][75950] Updated weights for policy 1, policy_version 19150 (0.0007) -[2023-10-14 14:11:57,600][75950] Updated weights for policy 1, policy_version 19160 (0.0007) -[2023-10-14 14:11:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 39256064. Throughput: 0: 1682.4, 1: 1654.9. Samples: 9823072. Policy #0 lag: (min: 26.0, avg: 26.8, max: 40.0) -[2023-10-14 14:11:58,164][74987] Avg episode reward: [(0, '18.320'), (1, '20.230')] -[2023-10-14 14:11:58,719][75949] Updated weights for policy 0, policy_version 19171 (0.0009) -[2023-10-14 14:11:59,087][75949] Updated weights for policy 0, policy_version 19181 (0.0009) -[2023-10-14 14:11:59,462][75949] Updated weights for policy 0, policy_version 19191 (0.0009) -[2023-10-14 14:12:01,809][75950] Updated weights for policy 1, policy_version 19170 (0.0008) -[2023-10-14 14:12:02,177][75950] Updated weights for policy 1, policy_version 19180 (0.0012) -[2023-10-14 14:12:02,544][75950] Updated weights for policy 1, policy_version 19190 (0.0010) -[2023-10-14 14:12:02,902][75950] Updated weights for policy 1, policy_version 19200 (0.0009) -[2023-10-14 14:12:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 39321600. Throughput: 0: 1679.6, 1: 1678.7. Samples: 9833024. Policy #0 lag: (min: 22.0, avg: 22.3, max: 35.0) -[2023-10-14 14:12:03,164][74987] Avg episode reward: [(0, '18.100'), (1, '20.680')] -[2023-10-14 14:12:03,612][75949] Updated weights for policy 0, policy_version 19201 (0.0011) -[2023-10-14 14:12:04,007][75949] Updated weights for policy 0, policy_version 19211 (0.0009) -[2023-10-14 14:12:04,371][75949] Updated weights for policy 0, policy_version 19221 (0.0010) -[2023-10-14 14:12:04,745][75949] Updated weights for policy 0, policy_version 19231 (0.0009) -[2023-10-14 14:12:07,135][75950] Updated weights for policy 1, policy_version 19210 (0.0009) -[2023-10-14 14:12:07,512][75950] Updated weights for policy 1, policy_version 19220 (0.0009) -[2023-10-14 14:12:07,881][75950] Updated weights for policy 1, policy_version 19230 (0.0010) -[2023-10-14 14:12:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 39387136. Throughput: 0: 1674.4, 1: 1678.6. Samples: 9853422. Policy #0 lag: (min: 22.0, avg: 22.3, max: 35.0) -[2023-10-14 14:12:08,165][74987] Avg episode reward: [(0, '19.870'), (1, '19.570')] -[2023-10-14 14:12:08,850][75949] Updated weights for policy 0, policy_version 19241 (0.0010) -[2023-10-14 14:12:09,224][75949] Updated weights for policy 0, policy_version 19251 (0.0007) -[2023-10-14 14:12:09,598][75949] Updated weights for policy 0, policy_version 19261 (0.0009) -[2023-10-14 14:12:12,055][75950] Updated weights for policy 1, policy_version 19240 (0.0009) -[2023-10-14 14:12:12,418][75950] Updated weights for policy 1, policy_version 19250 (0.0007) -[2023-10-14 14:12:12,785][75950] Updated weights for policy 1, policy_version 19260 (0.0007) -[2023-10-14 14:12:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 39452672. Throughput: 0: 1683.7, 1: 1654.7. Samples: 9873370. Policy #0 lag: (min: 22.0, avg: 22.3, max: 35.0) -[2023-10-14 14:12:13,165][74987] Avg episode reward: [(0, '19.780'), (1, '19.600')] -[2023-10-14 14:12:13,417][75949] Updated weights for policy 0, policy_version 19271 (0.0007) -[2023-10-14 14:12:13,795][75949] Updated weights for policy 0, policy_version 19281 (0.0008) -[2023-10-14 14:12:14,167][75949] Updated weights for policy 0, policy_version 19291 (0.0009) -[2023-10-14 14:12:16,888][75950] Updated weights for policy 1, policy_version 19270 (0.0009) -[2023-10-14 14:12:17,264][75950] Updated weights for policy 1, policy_version 19280 (0.0010) -[2023-10-14 14:12:17,639][75950] Updated weights for policy 1, policy_version 19290 (0.0009) -[2023-10-14 14:12:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 39518208. Throughput: 0: 1683.9, 1: 1671.5. Samples: 9883234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:12:18,164][74987] Avg episode reward: [(0, '19.920'), (1, '20.740')] -[2023-10-14 14:12:18,189][75949] Updated weights for policy 0, policy_version 19301 (0.0009) -[2023-10-14 14:12:18,567][75949] Updated weights for policy 0, policy_version 19311 (0.0011) -[2023-10-14 14:12:18,939][75949] Updated weights for policy 0, policy_version 19321 (0.0010) -[2023-10-14 14:12:21,783][75950] Updated weights for policy 1, policy_version 19300 (0.0010) -[2023-10-14 14:12:22,184][75950] Updated weights for policy 1, policy_version 19310 (0.0009) -[2023-10-14 14:12:22,563][75950] Updated weights for policy 1, policy_version 19320 (0.0010) -[2023-10-14 14:12:22,919][75949] Updated weights for policy 0, policy_version 19331 (0.0008) -[2023-10-14 14:12:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 39583744. Throughput: 0: 1686.1, 1: 1675.9. Samples: 9904042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:12:23,164][74987] Avg episode reward: [(0, '20.600'), (1, '21.440')] -[2023-10-14 14:12:23,292][75949] Updated weights for policy 0, policy_version 19341 (0.0010) -[2023-10-14 14:12:23,664][75949] Updated weights for policy 0, policy_version 19351 (0.0008) -[2023-10-14 14:12:23,995][75615] Saving new best policy, reward=20.600! -[2023-10-14 14:12:26,722][75950] Updated weights for policy 1, policy_version 19330 (0.0009) -[2023-10-14 14:12:27,096][75950] Updated weights for policy 1, policy_version 19340 (0.0008) -[2023-10-14 14:12:27,467][75950] Updated weights for policy 1, policy_version 19350 (0.0009) -[2023-10-14 14:12:27,499][75949] Updated weights for policy 0, policy_version 19361 (0.0009) -[2023-10-14 14:12:27,829][75950] Updated weights for policy 1, policy_version 19360 (0.0007) -[2023-10-14 14:12:27,879][75949] Updated weights for policy 0, policy_version 19371 (0.0008) -[2023-10-14 14:12:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 39649280. Throughput: 0: 1687.0, 1: 1661.1. Samples: 9923858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:12:28,164][74987] Avg episode reward: [(0, '19.400'), (1, '21.990')] -[2023-10-14 14:12:28,173][75801] Saving new best policy, reward=21.990! -[2023-10-14 14:12:28,249][75949] Updated weights for policy 0, policy_version 19381 (0.0008) -[2023-10-14 14:12:28,615][75949] Updated weights for policy 0, policy_version 19391 (0.0012) -[2023-10-14 14:12:31,714][75950] Updated weights for policy 1, policy_version 19370 (0.0010) -[2023-10-14 14:12:32,077][75950] Updated weights for policy 1, policy_version 19380 (0.0007) -[2023-10-14 14:12:32,443][75950] Updated weights for policy 1, policy_version 19390 (0.0008) -[2023-10-14 14:12:32,682][75949] Updated weights for policy 0, policy_version 19401 (0.0009) -[2023-10-14 14:12:33,055][75949] Updated weights for policy 0, policy_version 19411 (0.0007) -[2023-10-14 14:12:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 39714816. Throughput: 0: 1695.6, 1: 1674.8. Samples: 9934338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:12:33,164][74987] Avg episode reward: [(0, '19.030'), (1, '21.260')] -[2023-10-14 14:12:33,424][75949] Updated weights for policy 0, policy_version 19421 (0.0008) -[2023-10-14 14:12:36,664][75950] Updated weights for policy 1, policy_version 19400 (0.0009) -[2023-10-14 14:12:37,029][75950] Updated weights for policy 1, policy_version 19410 (0.0008) -[2023-10-14 14:12:37,399][75950] Updated weights for policy 1, policy_version 19420 (0.0008) -[2023-10-14 14:12:37,440][75949] Updated weights for policy 0, policy_version 19431 (0.0009) -[2023-10-14 14:12:37,811][75949] Updated weights for policy 0, policy_version 19441 (0.0008) -[2023-10-14 14:12:38,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 39780352. Throughput: 0: 1706.5, 1: 1665.7. Samples: 9954874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:12:38,164][74987] Avg episode reward: [(0, '18.840'), (1, '20.410')] -[2023-10-14 14:12:38,191][75949] Updated weights for policy 0, policy_version 19451 (0.0009) -[2023-10-14 14:12:41,338][75950] Updated weights for policy 1, policy_version 19430 (0.0010) -[2023-10-14 14:12:41,713][75950] Updated weights for policy 1, policy_version 19440 (0.0010) -[2023-10-14 14:12:42,073][75950] Updated weights for policy 1, policy_version 19450 (0.0009) -[2023-10-14 14:12:42,154][75949] Updated weights for policy 0, policy_version 19461 (0.0007) -[2023-10-14 14:12:42,515][75949] Updated weights for policy 0, policy_version 19471 (0.0008) -[2023-10-14 14:12:42,890][75949] Updated weights for policy 0, policy_version 19481 (0.0008) -[2023-10-14 14:12:43,164][74987] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 39878656. Throughput: 0: 1690.2, 1: 1665.3. Samples: 9974072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:12:43,164][74987] Avg episode reward: [(0, '18.130'), (1, '20.600')] -[2023-10-14 14:12:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000019488_19955712.pth... -[2023-10-14 14:12:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000019456_19922944.pth... -[2023-10-14 14:12:43,213][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000017888_18317312.pth -[2023-10-14 14:12:43,215][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000017888_18317312.pth -[2023-10-14 14:12:46,162][75950] Updated weights for policy 1, policy_version 19460 (0.0009) -[2023-10-14 14:12:46,529][75950] Updated weights for policy 1, policy_version 19470 (0.0010) -[2023-10-14 14:12:46,906][75950] Updated weights for policy 1, policy_version 19480 (0.0010) -[2023-10-14 14:12:46,925][75949] Updated weights for policy 0, policy_version 19491 (0.0007) -[2023-10-14 14:12:47,293][75949] Updated weights for policy 0, policy_version 19501 (0.0007) -[2023-10-14 14:12:47,669][75949] Updated weights for policy 0, policy_version 19511 (0.0007) -[2023-10-14 14:12:48,163][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 39944192. Throughput: 0: 1709.1, 1: 1668.8. Samples: 9985030. Policy #0 lag: (min: 26.0, avg: 31.2, max: 58.0) -[2023-10-14 14:12:48,164][74987] Avg episode reward: [(0, '19.630'), (1, '19.750')] -[2023-10-14 14:12:50,894][75950] Updated weights for policy 1, policy_version 19490 (0.0008) -[2023-10-14 14:12:51,254][75950] Updated weights for policy 1, policy_version 19500 (0.0010) -[2023-10-14 14:12:51,615][75950] Updated weights for policy 1, policy_version 19510 (0.0010) -[2023-10-14 14:12:51,706][75949] Updated weights for policy 0, policy_version 19521 (0.0010) -[2023-10-14 14:12:51,987][75950] Updated weights for policy 1, policy_version 19520 (0.0008) -[2023-10-14 14:12:52,082][75949] Updated weights for policy 0, policy_version 19531 (0.0007) -[2023-10-14 14:12:52,447][75949] Updated weights for policy 0, policy_version 19541 (0.0009) -[2023-10-14 14:12:52,812][75949] Updated weights for policy 0, policy_version 19551 (0.0007) -[2023-10-14 14:12:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40009728. Throughput: 0: 1713.9, 1: 1654.1. Samples: 10004984. Policy #0 lag: (min: 26.0, avg: 31.2, max: 58.0) -[2023-10-14 14:12:53,165][74987] Avg episode reward: [(0, '19.360'), (1, '19.930')] -[2023-10-14 14:12:56,054][75950] Updated weights for policy 1, policy_version 19530 (0.0010) -[2023-10-14 14:12:56,411][75950] Updated weights for policy 1, policy_version 19540 (0.0009) -[2023-10-14 14:12:56,780][75950] Updated weights for policy 1, policy_version 19550 (0.0009) -[2023-10-14 14:12:56,859][75949] Updated weights for policy 0, policy_version 19561 (0.0008) -[2023-10-14 14:12:57,240][75949] Updated weights for policy 0, policy_version 19571 (0.0008) -[2023-10-14 14:12:57,606][75949] Updated weights for policy 0, policy_version 19581 (0.0010) -[2023-10-14 14:12:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40075264. Throughput: 0: 1679.7, 1: 1672.5. Samples: 10024220. Policy #0 lag: (min: 26.0, avg: 31.2, max: 58.0) -[2023-10-14 14:12:58,165][74987] Avg episode reward: [(0, '20.590'), (1, '20.700')] -[2023-10-14 14:13:01,001][75950] Updated weights for policy 1, policy_version 19560 (0.0009) -[2023-10-14 14:13:01,377][75950] Updated weights for policy 1, policy_version 19570 (0.0010) -[2023-10-14 14:13:01,683][75949] Updated weights for policy 0, policy_version 19591 (0.0008) -[2023-10-14 14:13:01,744][75950] Updated weights for policy 1, policy_version 19580 (0.0008) -[2023-10-14 14:13:02,055][75949] Updated weights for policy 0, policy_version 19601 (0.0008) -[2023-10-14 14:13:02,423][75949] Updated weights for policy 0, policy_version 19611 (0.0009) -[2023-10-14 14:13:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40140800. Throughput: 0: 1710.4, 1: 1681.0. Samples: 10035848. Policy #0 lag: (min: 27.0, avg: 52.9, max: 56.0) -[2023-10-14 14:13:03,165][74987] Avg episode reward: [(0, '19.010'), (1, '20.470')] -[2023-10-14 14:13:05,960][75950] Updated weights for policy 1, policy_version 19590 (0.0009) -[2023-10-14 14:13:06,333][75950] Updated weights for policy 1, policy_version 19600 (0.0008) -[2023-10-14 14:13:06,425][75949] Updated weights for policy 0, policy_version 19621 (0.0008) -[2023-10-14 14:13:06,692][75950] Updated weights for policy 1, policy_version 19610 (0.0008) -[2023-10-14 14:13:06,787][75949] Updated weights for policy 0, policy_version 19631 (0.0008) -[2023-10-14 14:13:07,168][75949] Updated weights for policy 0, policy_version 19641 (0.0009) -[2023-10-14 14:13:08,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 40206336. Throughput: 0: 1698.9, 1: 1657.9. Samples: 10055098. Policy #0 lag: (min: 27.0, avg: 52.9, max: 56.0) -[2023-10-14 14:13:08,164][74987] Avg episode reward: [(0, '19.980'), (1, '19.390')] -[2023-10-14 14:13:10,708][75950] Updated weights for policy 1, policy_version 19620 (0.0009) -[2023-10-14 14:13:11,112][75950] Updated weights for policy 1, policy_version 19630 (0.0008) -[2023-10-14 14:13:11,313][75949] Updated weights for policy 0, policy_version 19651 (0.0010) -[2023-10-14 14:13:11,468][75950] Updated weights for policy 1, policy_version 19640 (0.0008) -[2023-10-14 14:13:11,689][75949] Updated weights for policy 0, policy_version 19661 (0.0009) -[2023-10-14 14:13:12,058][75949] Updated weights for policy 0, policy_version 19671 (0.0010) -[2023-10-14 14:13:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 40271872. Throughput: 0: 1674.4, 1: 1672.5. Samples: 10074468. Policy #0 lag: (min: 27.0, avg: 52.9, max: 56.0) -[2023-10-14 14:13:13,164][74987] Avg episode reward: [(0, '19.740'), (1, '19.270')] -[2023-10-14 14:13:15,477][75950] Updated weights for policy 1, policy_version 19650 (0.0010) -[2023-10-14 14:13:15,837][75950] Updated weights for policy 1, policy_version 19660 (0.0008) -[2023-10-14 14:13:16,134][75949] Updated weights for policy 0, policy_version 19681 (0.0009) -[2023-10-14 14:13:16,207][75950] Updated weights for policy 1, policy_version 19670 (0.0008) -[2023-10-14 14:13:16,508][75949] Updated weights for policy 0, policy_version 19691 (0.0009) -[2023-10-14 14:13:16,571][75950] Updated weights for policy 1, policy_version 19680 (0.0007) -[2023-10-14 14:13:16,890][75949] Updated weights for policy 0, policy_version 19701 (0.0008) -[2023-10-14 14:13:17,264][75949] Updated weights for policy 0, policy_version 19711 (0.0007) -[2023-10-14 14:13:18,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40337408. Throughput: 0: 1698.0, 1: 1672.2. Samples: 10086000. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) -[2023-10-14 14:13:18,164][74987] Avg episode reward: [(0, '20.260'), (1, '20.340')] -[2023-10-14 14:13:20,620][75950] Updated weights for policy 1, policy_version 19690 (0.0011) -[2023-10-14 14:13:20,989][75950] Updated weights for policy 1, policy_version 19700 (0.0011) -[2023-10-14 14:13:21,297][75949] Updated weights for policy 0, policy_version 19721 (0.0009) -[2023-10-14 14:13:21,365][75950] Updated weights for policy 1, policy_version 19710 (0.0008) -[2023-10-14 14:13:21,671][75949] Updated weights for policy 0, policy_version 19731 (0.0009) -[2023-10-14 14:13:22,040][75949] Updated weights for policy 0, policy_version 19741 (0.0009) -[2023-10-14 14:13:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 40402944. Throughput: 0: 1679.3, 1: 1657.0. Samples: 10105006. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) -[2023-10-14 14:13:23,164][74987] Avg episode reward: [(0, '19.880'), (1, '20.620')] -[2023-10-14 14:13:25,318][75950] Updated weights for policy 1, policy_version 19720 (0.0009) -[2023-10-14 14:13:25,684][75950] Updated weights for policy 1, policy_version 19730 (0.0008) -[2023-10-14 14:13:26,002][75949] Updated weights for policy 0, policy_version 19751 (0.0008) -[2023-10-14 14:13:26,050][75950] Updated weights for policy 1, policy_version 19740 (0.0009) -[2023-10-14 14:13:26,369][75949] Updated weights for policy 0, policy_version 19761 (0.0009) -[2023-10-14 14:13:26,743][75949] Updated weights for policy 0, policy_version 19771 (0.0011) -[2023-10-14 14:13:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40468480. Throughput: 0: 1676.4, 1: 1680.5. Samples: 10125136. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) -[2023-10-14 14:13:28,164][74987] Avg episode reward: [(0, '19.940'), (1, '21.340')] -[2023-10-14 14:13:30,361][75950] Updated weights for policy 1, policy_version 19750 (0.0008) -[2023-10-14 14:13:30,728][75950] Updated weights for policy 1, policy_version 19760 (0.0008) -[2023-10-14 14:13:30,842][75949] Updated weights for policy 0, policy_version 19781 (0.0011) -[2023-10-14 14:13:31,104][75950] Updated weights for policy 1, policy_version 19770 (0.0009) -[2023-10-14 14:13:31,206][75949] Updated weights for policy 0, policy_version 19791 (0.0010) -[2023-10-14 14:13:31,580][75949] Updated weights for policy 0, policy_version 19801 (0.0009) -[2023-10-14 14:13:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40534016. Throughput: 0: 1686.0, 1: 1669.9. Samples: 10136044. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:13:33,164][74987] Avg episode reward: [(0, '20.740'), (1, '20.310')] -[2023-10-14 14:13:33,165][75615] Saving new best policy, reward=20.740! -[2023-10-14 14:13:35,334][75950] Updated weights for policy 1, policy_version 19780 (0.0009) -[2023-10-14 14:13:35,711][75950] Updated weights for policy 1, policy_version 19790 (0.0008) -[2023-10-14 14:13:35,763][75949] Updated weights for policy 0, policy_version 19811 (0.0009) -[2023-10-14 14:13:36,078][75950] Updated weights for policy 1, policy_version 19800 (0.0007) -[2023-10-14 14:13:36,126][75949] Updated weights for policy 0, policy_version 19821 (0.0010) -[2023-10-14 14:13:36,500][75949] Updated weights for policy 0, policy_version 19831 (0.0009) -[2023-10-14 14:13:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 40599552. Throughput: 0: 1659.0, 1: 1666.1. Samples: 10154614. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:13:38,165][74987] Avg episode reward: [(0, '19.270'), (1, '20.560')] -[2023-10-14 14:13:39,951][75950] Updated weights for policy 1, policy_version 19810 (0.0009) -[2023-10-14 14:13:40,325][75950] Updated weights for policy 1, policy_version 19820 (0.0009) -[2023-10-14 14:13:40,607][75949] Updated weights for policy 0, policy_version 19841 (0.0010) -[2023-10-14 14:13:40,691][75950] Updated weights for policy 1, policy_version 19830 (0.0008) -[2023-10-14 14:13:41,031][75949] Updated weights for policy 0, policy_version 19851 (0.0010) -[2023-10-14 14:13:41,052][75950] Updated weights for policy 1, policy_version 19840 (0.0009) -[2023-10-14 14:13:41,405][75949] Updated weights for policy 0, policy_version 19861 (0.0010) -[2023-10-14 14:13:41,773][75949] Updated weights for policy 0, policy_version 19871 (0.0008) -[2023-10-14 14:13:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 40665088. Throughput: 0: 1675.7, 1: 1678.7. Samples: 10175168. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:13:43,165][74987] Avg episode reward: [(0, '21.000'), (1, '20.180')] -[2023-10-14 14:13:43,175][75615] Saving new best policy, reward=21.000! -[2023-10-14 14:13:45,046][75950] Updated weights for policy 1, policy_version 19850 (0.0010) -[2023-10-14 14:13:45,416][75950] Updated weights for policy 1, policy_version 19860 (0.0010) -[2023-10-14 14:13:45,781][75950] Updated weights for policy 1, policy_version 19870 (0.0008) -[2023-10-14 14:13:45,812][75949] Updated weights for policy 0, policy_version 19881 (0.0009) -[2023-10-14 14:13:46,190][75949] Updated weights for policy 0, policy_version 19891 (0.0008) -[2023-10-14 14:13:46,563][75949] Updated weights for policy 0, policy_version 19901 (0.0009) -[2023-10-14 14:13:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 40730624. Throughput: 0: 1670.2, 1: 1660.7. Samples: 10185740. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:13:48,165][74987] Avg episode reward: [(0, '19.180'), (1, '20.880')] -[2023-10-14 14:13:49,856][75950] Updated weights for policy 1, policy_version 19880 (0.0008) -[2023-10-14 14:13:50,224][75950] Updated weights for policy 1, policy_version 19890 (0.0007) -[2023-10-14 14:13:50,587][75950] Updated weights for policy 1, policy_version 19900 (0.0008) -[2023-10-14 14:13:50,702][75949] Updated weights for policy 0, policy_version 19911 (0.0008) -[2023-10-14 14:13:51,068][75949] Updated weights for policy 0, policy_version 19921 (0.0007) -[2023-10-14 14:13:51,435][75949] Updated weights for policy 0, policy_version 19931 (0.0011) -[2023-10-14 14:13:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 40796160. Throughput: 0: 1651.6, 1: 1677.7. Samples: 10204916. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:13:53,165][74987] Avg episode reward: [(0, '20.370'), (1, '20.420')] -[2023-10-14 14:13:54,318][75950] Updated weights for policy 1, policy_version 19910 (0.0007) -[2023-10-14 14:13:54,689][75950] Updated weights for policy 1, policy_version 19920 (0.0008) -[2023-10-14 14:13:55,063][75950] Updated weights for policy 1, policy_version 19930 (0.0008) -[2023-10-14 14:13:55,524][75949] Updated weights for policy 0, policy_version 19941 (0.0011) -[2023-10-14 14:13:55,893][75949] Updated weights for policy 0, policy_version 19951 (0.0009) -[2023-10-14 14:13:56,271][75949] Updated weights for policy 0, policy_version 19961 (0.0009) -[2023-10-14 14:13:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 40861696. Throughput: 0: 1664.3, 1: 1695.5. Samples: 10225660. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:13:58,165][74987] Avg episode reward: [(0, '18.770'), (1, '20.810')] -[2023-10-14 14:13:59,300][75950] Updated weights for policy 1, policy_version 19940 (0.0010) -[2023-10-14 14:13:59,689][75950] Updated weights for policy 1, policy_version 19950 (0.0010) -[2023-10-14 14:14:00,059][75950] Updated weights for policy 1, policy_version 19960 (0.0007) -[2023-10-14 14:14:00,343][75949] Updated weights for policy 0, policy_version 19971 (0.0009) -[2023-10-14 14:14:00,718][75949] Updated weights for policy 0, policy_version 19981 (0.0008) -[2023-10-14 14:14:01,091][75949] Updated weights for policy 0, policy_version 19991 (0.0009) -[2023-10-14 14:14:03,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 40927232. Throughput: 0: 1656.4, 1: 1668.6. Samples: 10235626. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:14:03,164][74987] Avg episode reward: [(0, '18.870'), (1, '20.500')] -[2023-10-14 14:14:04,165][75950] Updated weights for policy 1, policy_version 19970 (0.0009) -[2023-10-14 14:14:04,531][75950] Updated weights for policy 1, policy_version 19980 (0.0009) -[2023-10-14 14:14:04,894][75950] Updated weights for policy 1, policy_version 19990 (0.0009) -[2023-10-14 14:14:05,141][75949] Updated weights for policy 0, policy_version 20001 (0.0009) -[2023-10-14 14:14:05,266][75950] Updated weights for policy 1, policy_version 20000 (0.0010) -[2023-10-14 14:14:05,516][75949] Updated weights for policy 0, policy_version 20011 (0.0009) -[2023-10-14 14:14:05,891][75949] Updated weights for policy 0, policy_version 20021 (0.0009) -[2023-10-14 14:14:06,257][75949] Updated weights for policy 0, policy_version 20031 (0.0010) -[2023-10-14 14:14:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 40992768. Throughput: 0: 1651.9, 1: 1692.5. Samples: 10255504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:14:08,165][74987] Avg episode reward: [(0, '20.050'), (1, '20.360')] -[2023-10-14 14:14:09,479][75950] Updated weights for policy 1, policy_version 20010 (0.0008) -[2023-10-14 14:14:09,848][75950] Updated weights for policy 1, policy_version 20020 (0.0008) -[2023-10-14 14:14:10,210][75950] Updated weights for policy 1, policy_version 20030 (0.0007) -[2023-10-14 14:14:10,295][75949] Updated weights for policy 0, policy_version 20041 (0.0007) -[2023-10-14 14:14:10,667][75949] Updated weights for policy 0, policy_version 20051 (0.0008) -[2023-10-14 14:14:11,042][75949] Updated weights for policy 0, policy_version 20061 (0.0008) -[2023-10-14 14:14:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41058304. Throughput: 0: 1672.6, 1: 1683.6. Samples: 10276168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:14:13,165][74987] Avg episode reward: [(0, '19.860'), (1, '20.720')] -[2023-10-14 14:14:14,194][75950] Updated weights for policy 1, policy_version 20040 (0.0011) -[2023-10-14 14:14:14,566][75950] Updated weights for policy 1, policy_version 20050 (0.0008) -[2023-10-14 14:14:14,912][75949] Updated weights for policy 0, policy_version 20071 (0.0007) -[2023-10-14 14:14:14,929][75950] Updated weights for policy 1, policy_version 20060 (0.0008) -[2023-10-14 14:14:15,285][75949] Updated weights for policy 0, policy_version 20081 (0.0008) -[2023-10-14 14:14:15,656][75949] Updated weights for policy 0, policy_version 20091 (0.0008) -[2023-10-14 14:14:18,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41123840. Throughput: 0: 1653.6, 1: 1670.0. Samples: 10285604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:14:18,164][74987] Avg episode reward: [(0, '20.480'), (1, '20.790')] -[2023-10-14 14:14:19,146][75950] Updated weights for policy 1, policy_version 20070 (0.0008) -[2023-10-14 14:14:19,513][75950] Updated weights for policy 1, policy_version 20080 (0.0008) -[2023-10-14 14:14:19,833][75949] Updated weights for policy 0, policy_version 20101 (0.0008) -[2023-10-14 14:14:19,877][75950] Updated weights for policy 1, policy_version 20090 (0.0008) -[2023-10-14 14:14:20,204][75949] Updated weights for policy 0, policy_version 20111 (0.0008) -[2023-10-14 14:14:20,572][75949] Updated weights for policy 0, policy_version 20121 (0.0010) -[2023-10-14 14:14:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41189376. Throughput: 0: 1670.2, 1: 1687.6. Samples: 10305714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:14:23,164][74987] Avg episode reward: [(0, '20.590'), (1, '19.550')] -[2023-10-14 14:14:23,934][75950] Updated weights for policy 1, policy_version 20100 (0.0010) -[2023-10-14 14:14:24,302][75950] Updated weights for policy 1, policy_version 20110 (0.0010) -[2023-10-14 14:14:24,617][75949] Updated weights for policy 0, policy_version 20131 (0.0010) -[2023-10-14 14:14:24,664][75950] Updated weights for policy 1, policy_version 20120 (0.0008) -[2023-10-14 14:14:24,990][75949] Updated weights for policy 0, policy_version 20141 (0.0008) -[2023-10-14 14:14:25,364][75949] Updated weights for policy 0, policy_version 20151 (0.0008) -[2023-10-14 14:14:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41254912. Throughput: 0: 1677.8, 1: 1684.2. Samples: 10326458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:14:28,165][74987] Avg episode reward: [(0, '19.600'), (1, '19.200')] -[2023-10-14 14:14:28,850][75950] Updated weights for policy 1, policy_version 20130 (0.0007) -[2023-10-14 14:14:29,221][75950] Updated weights for policy 1, policy_version 20140 (0.0010) -[2023-10-14 14:14:29,563][75949] Updated weights for policy 0, policy_version 20161 (0.0009) -[2023-10-14 14:14:29,586][75950] Updated weights for policy 1, policy_version 20150 (0.0008) -[2023-10-14 14:14:29,964][75950] Updated weights for policy 1, policy_version 20160 (0.0007) -[2023-10-14 14:14:29,966][75949] Updated weights for policy 0, policy_version 20171 (0.0007) -[2023-10-14 14:14:30,347][75949] Updated weights for policy 0, policy_version 20181 (0.0009) -[2023-10-14 14:14:30,711][75949] Updated weights for policy 0, policy_version 20191 (0.0009) -[2023-10-14 14:14:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41320448. Throughput: 0: 1653.5, 1: 1676.4. Samples: 10335586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:14:33,165][74987] Avg episode reward: [(0, '20.250'), (1, '19.430')] -[2023-10-14 14:14:34,015][75950] Updated weights for policy 1, policy_version 20170 (0.0009) -[2023-10-14 14:14:34,387][75950] Updated weights for policy 1, policy_version 20180 (0.0009) -[2023-10-14 14:14:34,752][75949] Updated weights for policy 0, policy_version 20201 (0.0008) -[2023-10-14 14:14:34,758][75950] Updated weights for policy 1, policy_version 20190 (0.0008) -[2023-10-14 14:14:35,117][75949] Updated weights for policy 0, policy_version 20211 (0.0009) -[2023-10-14 14:14:35,508][75949] Updated weights for policy 0, policy_version 20221 (0.0009) -[2023-10-14 14:14:38,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41385984. Throughput: 0: 1682.2, 1: 1681.3. Samples: 10356270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:14:38,164][74987] Avg episode reward: [(0, '19.770'), (1, '21.210')] -[2023-10-14 14:14:38,906][75950] Updated weights for policy 1, policy_version 20200 (0.0010) -[2023-10-14 14:14:39,277][75950] Updated weights for policy 1, policy_version 20210 (0.0010) -[2023-10-14 14:14:39,580][75949] Updated weights for policy 0, policy_version 20231 (0.0008) -[2023-10-14 14:14:39,642][75950] Updated weights for policy 1, policy_version 20220 (0.0008) -[2023-10-14 14:14:39,949][75949] Updated weights for policy 0, policy_version 20241 (0.0009) -[2023-10-14 14:14:40,323][75949] Updated weights for policy 0, policy_version 20251 (0.0008) -[2023-10-14 14:14:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 41451520. Throughput: 0: 1693.7, 1: 1672.6. Samples: 10377144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:14:43,164][74987] Avg episode reward: [(0, '21.070'), (1, '19.260')] -[2023-10-14 14:14:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000020256_20742144.pth... -[2023-10-14 14:14:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000020224_20709376.pth... -[2023-10-14 14:14:43,215][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000018688_19136512.pth -[2023-10-14 14:14:43,220][75615] Saving new best policy, reward=21.370! -[2023-10-14 14:14:43,221][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000018656_19103744.pth -[2023-10-14 14:14:43,586][75950] Updated weights for policy 1, policy_version 20230 (0.0010) -[2023-10-14 14:14:43,947][75950] Updated weights for policy 1, policy_version 20240 (0.0007) -[2023-10-14 14:14:44,302][75949] Updated weights for policy 0, policy_version 20261 (0.0009) -[2023-10-14 14:14:44,311][75950] Updated weights for policy 1, policy_version 20250 (0.0009) -[2023-10-14 14:14:44,678][75949] Updated weights for policy 0, policy_version 20271 (0.0009) -[2023-10-14 14:14:45,052][75949] Updated weights for policy 0, policy_version 20281 (0.0007) -[2023-10-14 14:14:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41517056. Throughput: 0: 1672.4, 1: 1674.6. Samples: 10386238. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-14 14:14:48,165][74987] Avg episode reward: [(0, '19.940'), (1, '20.520')] -[2023-10-14 14:14:48,598][75950] Updated weights for policy 1, policy_version 20260 (0.0009) -[2023-10-14 14:14:48,976][75950] Updated weights for policy 1, policy_version 20270 (0.0009) -[2023-10-14 14:14:49,229][75949] Updated weights for policy 0, policy_version 20291 (0.0008) -[2023-10-14 14:14:49,346][75950] Updated weights for policy 1, policy_version 20280 (0.0008) -[2023-10-14 14:14:49,591][75949] Updated weights for policy 0, policy_version 20301 (0.0008) -[2023-10-14 14:14:49,962][75949] Updated weights for policy 0, policy_version 20311 (0.0007) -[2023-10-14 14:14:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41582592. Throughput: 0: 1688.5, 1: 1669.2. Samples: 10406602. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-14 14:14:53,165][74987] Avg episode reward: [(0, '20.870'), (1, '19.360')] -[2023-10-14 14:14:53,328][75950] Updated weights for policy 1, policy_version 20290 (0.0009) -[2023-10-14 14:14:53,713][75950] Updated weights for policy 1, policy_version 20300 (0.0009) -[2023-10-14 14:14:54,021][75949] Updated weights for policy 0, policy_version 20321 (0.0008) -[2023-10-14 14:14:54,075][75950] Updated weights for policy 1, policy_version 20310 (0.0008) -[2023-10-14 14:14:54,397][75949] Updated weights for policy 0, policy_version 20331 (0.0009) -[2023-10-14 14:14:54,433][75950] Updated weights for policy 1, policy_version 20320 (0.0007) -[2023-10-14 14:14:54,764][75949] Updated weights for policy 0, policy_version 20341 (0.0009) -[2023-10-14 14:14:55,137][75949] Updated weights for policy 0, policy_version 20351 (0.0009) -[2023-10-14 14:14:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 41648128. Throughput: 0: 1686.0, 1: 1675.3. Samples: 10427428. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-14 14:14:58,164][74987] Avg episode reward: [(0, '19.720'), (1, '21.050')] -[2023-10-14 14:14:58,439][75950] Updated weights for policy 1, policy_version 20330 (0.0010) -[2023-10-14 14:14:58,796][75950] Updated weights for policy 1, policy_version 20340 (0.0009) -[2023-10-14 14:14:59,105][75949] Updated weights for policy 0, policy_version 20361 (0.0010) -[2023-10-14 14:14:59,166][75950] Updated weights for policy 1, policy_version 20350 (0.0009) -[2023-10-14 14:14:59,473][75949] Updated weights for policy 0, policy_version 20371 (0.0008) -[2023-10-14 14:14:59,856][75949] Updated weights for policy 0, policy_version 20381 (0.0008) -[2023-10-14 14:15:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41713664. Throughput: 0: 1678.5, 1: 1674.9. Samples: 10436510. Policy #0 lag: (min: 18.0, avg: 25.8, max: 50.0) -[2023-10-14 14:15:03,165][74987] Avg episode reward: [(0, '20.840'), (1, '19.260')] -[2023-10-14 14:15:03,364][75950] Updated weights for policy 1, policy_version 20360 (0.0008) -[2023-10-14 14:15:03,729][75950] Updated weights for policy 1, policy_version 20370 (0.0007) -[2023-10-14 14:15:03,922][75949] Updated weights for policy 0, policy_version 20391 (0.0009) -[2023-10-14 14:15:04,086][75950] Updated weights for policy 1, policy_version 20380 (0.0009) -[2023-10-14 14:15:04,293][75949] Updated weights for policy 0, policy_version 20401 (0.0008) -[2023-10-14 14:15:04,660][75949] Updated weights for policy 0, policy_version 20411 (0.0008) -[2023-10-14 14:15:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41779200. Throughput: 0: 1690.1, 1: 1674.6. Samples: 10457124. Policy #0 lag: (min: 18.0, avg: 25.8, max: 50.0) -[2023-10-14 14:15:08,165][74987] Avg episode reward: [(0, '19.560'), (1, '21.360')] -[2023-10-14 14:15:08,275][75950] Updated weights for policy 1, policy_version 20390 (0.0009) -[2023-10-14 14:15:08,653][75950] Updated weights for policy 1, policy_version 20400 (0.0009) -[2023-10-14 14:15:08,739][75949] Updated weights for policy 0, policy_version 20421 (0.0009) -[2023-10-14 14:15:09,013][75950] Updated weights for policy 1, policy_version 20410 (0.0009) -[2023-10-14 14:15:09,114][75949] Updated weights for policy 0, policy_version 20431 (0.0009) -[2023-10-14 14:15:09,486][75949] Updated weights for policy 0, policy_version 20441 (0.0008) -[2023-10-14 14:15:13,139][75950] Updated weights for policy 1, policy_version 20420 (0.0009) -[2023-10-14 14:15:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41844736. Throughput: 0: 1685.7, 1: 1670.1. Samples: 10477468. Policy #0 lag: (min: 18.0, avg: 25.8, max: 50.0) -[2023-10-14 14:15:13,164][74987] Avg episode reward: [(0, '19.290'), (1, '20.530')] -[2023-10-14 14:15:13,514][75950] Updated weights for policy 1, policy_version 20430 (0.0007) -[2023-10-14 14:15:13,628][75949] Updated weights for policy 0, policy_version 20451 (0.0009) -[2023-10-14 14:15:13,890][75950] Updated weights for policy 1, policy_version 20440 (0.0007) -[2023-10-14 14:15:14,010][75949] Updated weights for policy 0, policy_version 20461 (0.0010) -[2023-10-14 14:15:14,380][75949] Updated weights for policy 0, policy_version 20471 (0.0009) -[2023-10-14 14:15:17,921][75950] Updated weights for policy 1, policy_version 20450 (0.0008) -[2023-10-14 14:15:18,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41910272. Throughput: 0: 1678.9, 1: 1671.7. Samples: 10486360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:15:18,164][74987] Avg episode reward: [(0, '20.560'), (1, '21.110')] -[2023-10-14 14:15:18,287][75950] Updated weights for policy 1, policy_version 20460 (0.0008) -[2023-10-14 14:15:18,433][75949] Updated weights for policy 0, policy_version 20481 (0.0010) -[2023-10-14 14:15:18,656][75950] Updated weights for policy 1, policy_version 20470 (0.0008) -[2023-10-14 14:15:18,800][75949] Updated weights for policy 0, policy_version 20491 (0.0008) -[2023-10-14 14:15:19,024][75950] Updated weights for policy 1, policy_version 20480 (0.0008) -[2023-10-14 14:15:19,180][75949] Updated weights for policy 0, policy_version 20501 (0.0010) -[2023-10-14 14:15:19,541][75949] Updated weights for policy 0, policy_version 20511 (0.0010) -[2023-10-14 14:15:23,102][75950] Updated weights for policy 1, policy_version 20490 (0.0008) -[2023-10-14 14:15:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41975808. Throughput: 0: 1681.1, 1: 1669.0. Samples: 10507024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:15:23,165][74987] Avg episode reward: [(0, '20.310'), (1, '18.090')] -[2023-10-14 14:15:23,473][75950] Updated weights for policy 1, policy_version 20500 (0.0009) -[2023-10-14 14:15:23,655][75949] Updated weights for policy 0, policy_version 20521 (0.0007) -[2023-10-14 14:15:23,844][75950] Updated weights for policy 1, policy_version 20510 (0.0008) -[2023-10-14 14:15:24,014][75949] Updated weights for policy 0, policy_version 20531 (0.0007) -[2023-10-14 14:15:24,389][75949] Updated weights for policy 0, policy_version 20541 (0.0009) -[2023-10-14 14:15:27,895][75950] Updated weights for policy 1, policy_version 20520 (0.0007) -[2023-10-14 14:15:28,164][74987] Fps is (10 sec: 13106.6, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 42041344. Throughput: 0: 1674.0, 1: 1669.4. Samples: 10527598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:15:28,165][74987] Avg episode reward: [(0, '20.290'), (1, '20.880')] -[2023-10-14 14:15:28,273][75950] Updated weights for policy 1, policy_version 20530 (0.0009) -[2023-10-14 14:15:28,537][75949] Updated weights for policy 0, policy_version 20551 (0.0008) -[2023-10-14 14:15:28,646][75950] Updated weights for policy 1, policy_version 20540 (0.0008) -[2023-10-14 14:15:28,915][75949] Updated weights for policy 0, policy_version 20561 (0.0008) -[2023-10-14 14:15:29,292][75949] Updated weights for policy 0, policy_version 20571 (0.0008) -[2023-10-14 14:15:32,596][75950] Updated weights for policy 1, policy_version 20550 (0.0008) -[2023-10-14 14:15:32,964][75950] Updated weights for policy 1, policy_version 20560 (0.0008) -[2023-10-14 14:15:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42106880. Throughput: 0: 1673.6, 1: 1670.8. Samples: 10536734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:15:33,164][74987] Avg episode reward: [(0, '20.900'), (1, '20.840')] -[2023-10-14 14:15:33,322][75950] Updated weights for policy 1, policy_version 20570 (0.0009) -[2023-10-14 14:15:33,464][75949] Updated weights for policy 0, policy_version 20581 (0.0008) -[2023-10-14 14:15:33,823][75949] Updated weights for policy 0, policy_version 20591 (0.0008) -[2023-10-14 14:15:34,189][75949] Updated weights for policy 0, policy_version 20601 (0.0008) -[2023-10-14 14:15:37,531][75950] Updated weights for policy 1, policy_version 20580 (0.0008) -[2023-10-14 14:15:37,923][75950] Updated weights for policy 1, policy_version 20590 (0.0009) -[2023-10-14 14:15:38,163][74987] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42172416. Throughput: 0: 1676.6, 1: 1674.8. Samples: 10557416. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 14:15:38,164][74987] Avg episode reward: [(0, '19.580'), (1, '20.750')] -[2023-10-14 14:15:38,199][75949] Updated weights for policy 0, policy_version 20611 (0.0009) -[2023-10-14 14:15:38,291][75950] Updated weights for policy 1, policy_version 20600 (0.0008) -[2023-10-14 14:15:38,562][75949] Updated weights for policy 0, policy_version 20621 (0.0008) -[2023-10-14 14:15:38,930][75949] Updated weights for policy 0, policy_version 20631 (0.0009) -[2023-10-14 14:15:42,472][75950] Updated weights for policy 1, policy_version 20610 (0.0008) -[2023-10-14 14:15:42,832][75950] Updated weights for policy 1, policy_version 20620 (0.0007) -[2023-10-14 14:15:42,998][75949] Updated weights for policy 0, policy_version 20641 (0.0008) -[2023-10-14 14:15:43,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42237952. Throughput: 0: 1672.8, 1: 1661.2. Samples: 10577458. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 14:15:43,164][74987] Avg episode reward: [(0, '20.410'), (1, '19.650')] -[2023-10-14 14:15:43,208][75950] Updated weights for policy 1, policy_version 20630 (0.0009) -[2023-10-14 14:15:43,371][75949] Updated weights for policy 0, policy_version 20651 (0.0009) -[2023-10-14 14:15:43,576][75950] Updated weights for policy 1, policy_version 20640 (0.0008) -[2023-10-14 14:15:43,742][75949] Updated weights for policy 0, policy_version 20661 (0.0009) -[2023-10-14 14:15:44,109][75949] Updated weights for policy 0, policy_version 20671 (0.0009) -[2023-10-14 14:15:47,638][75950] Updated weights for policy 1, policy_version 20650 (0.0008) -[2023-10-14 14:15:48,013][75950] Updated weights for policy 1, policy_version 20660 (0.0009) -[2023-10-14 14:15:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 42303488. Throughput: 0: 1672.4, 1: 1666.4. Samples: 10586758. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 14:15:48,165][74987] Avg episode reward: [(0, '20.210'), (1, '20.020')] -[2023-10-14 14:15:48,378][75950] Updated weights for policy 1, policy_version 20670 (0.0011) -[2023-10-14 14:15:48,383][75949] Updated weights for policy 0, policy_version 20681 (0.0010) -[2023-10-14 14:15:48,750][75949] Updated weights for policy 0, policy_version 20691 (0.0010) -[2023-10-14 14:15:49,120][75949] Updated weights for policy 0, policy_version 20701 (0.0011) -[2023-10-14 14:15:52,404][75950] Updated weights for policy 1, policy_version 20680 (0.0008) -[2023-10-14 14:15:52,766][75950] Updated weights for policy 1, policy_version 20690 (0.0009) -[2023-10-14 14:15:53,134][75950] Updated weights for policy 1, policy_version 20700 (0.0007) -[2023-10-14 14:15:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42369024. Throughput: 0: 1662.8, 1: 1667.3. Samples: 10606982. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 14:15:53,164][74987] Avg episode reward: [(0, '19.550'), (1, '19.820')] -[2023-10-14 14:15:53,297][75949] Updated weights for policy 0, policy_version 20711 (0.0008) -[2023-10-14 14:15:53,673][75949] Updated weights for policy 0, policy_version 20721 (0.0009) -[2023-10-14 14:15:54,035][75949] Updated weights for policy 0, policy_version 20731 (0.0008) -[2023-10-14 14:15:57,371][75950] Updated weights for policy 1, policy_version 20710 (0.0008) -[2023-10-14 14:15:57,731][75950] Updated weights for policy 1, policy_version 20720 (0.0008) -[2023-10-14 14:15:58,048][75949] Updated weights for policy 0, policy_version 20741 (0.0008) -[2023-10-14 14:15:58,104][75950] Updated weights for policy 1, policy_version 20730 (0.0007) -[2023-10-14 14:15:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 42434560. Throughput: 0: 1667.2, 1: 1656.6. Samples: 10627038. Policy #0 lag: (min: 0.0, avg: 25.4, max: 32.0) -[2023-10-14 14:15:58,164][74987] Avg episode reward: [(0, '20.450'), (1, '19.150')] -[2023-10-14 14:15:58,412][75949] Updated weights for policy 0, policy_version 20751 (0.0010) -[2023-10-14 14:15:58,782][75949] Updated weights for policy 0, policy_version 20761 (0.0008) -[2023-10-14 14:16:02,084][75950] Updated weights for policy 1, policy_version 20740 (0.0007) -[2023-10-14 14:16:02,456][75950] Updated weights for policy 1, policy_version 20750 (0.0007) -[2023-10-14 14:16:02,820][75950] Updated weights for policy 1, policy_version 20760 (0.0009) -[2023-10-14 14:16:02,860][75949] Updated weights for policy 0, policy_version 20771 (0.0007) -[2023-10-14 14:16:03,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 42532864. Throughput: 0: 1672.8, 1: 1670.8. Samples: 10636822. Policy #0 lag: (min: 0.0, avg: 25.4, max: 32.0) -[2023-10-14 14:16:03,164][74987] Avg episode reward: [(0, '19.140'), (1, '20.040')] -[2023-10-14 14:16:03,232][75949] Updated weights for policy 0, policy_version 20781 (0.0009) -[2023-10-14 14:16:03,604][75949] Updated weights for policy 0, policy_version 20791 (0.0007) -[2023-10-14 14:16:06,988][75950] Updated weights for policy 1, policy_version 20770 (0.0007) -[2023-10-14 14:16:07,352][75950] Updated weights for policy 1, policy_version 20780 (0.0008) -[2023-10-14 14:16:07,724][75950] Updated weights for policy 1, policy_version 20790 (0.0010) -[2023-10-14 14:16:07,733][75949] Updated weights for policy 0, policy_version 20801 (0.0008) -[2023-10-14 14:16:08,080][75950] Updated weights for policy 1, policy_version 20800 (0.0007) -[2023-10-14 14:16:08,116][75949] Updated weights for policy 0, policy_version 20811 (0.0008) -[2023-10-14 14:16:08,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42598400. Throughput: 0: 1666.7, 1: 1669.3. Samples: 10657144. Policy #0 lag: (min: 0.0, avg: 25.4, max: 32.0) -[2023-10-14 14:16:08,165][74987] Avg episode reward: [(0, '21.410'), (1, '19.850')] -[2023-10-14 14:16:08,488][75949] Updated weights for policy 0, policy_version 20821 (0.0008) -[2023-10-14 14:16:08,859][75949] Updated weights for policy 0, policy_version 20831 (0.0009) -[2023-10-14 14:16:08,890][75615] Saving new best policy, reward=21.410! -[2023-10-14 14:16:12,190][75950] Updated weights for policy 1, policy_version 20810 (0.0008) -[2023-10-14 14:16:12,553][75950] Updated weights for policy 1, policy_version 20820 (0.0008) -[2023-10-14 14:16:12,908][75949] Updated weights for policy 0, policy_version 20841 (0.0008) -[2023-10-14 14:16:12,916][75950] Updated weights for policy 1, policy_version 20830 (0.0009) -[2023-10-14 14:16:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42663936. Throughput: 0: 1665.7, 1: 1650.9. Samples: 10676844. Policy #0 lag: (min: 20.0, avg: 26.6, max: 52.0) -[2023-10-14 14:16:13,165][74987] Avg episode reward: [(0, '19.330'), (1, '21.190')] -[2023-10-14 14:16:13,276][75949] Updated weights for policy 0, policy_version 20851 (0.0009) -[2023-10-14 14:16:13,657][75949] Updated weights for policy 0, policy_version 20861 (0.0011) -[2023-10-14 14:16:17,029][75950] Updated weights for policy 1, policy_version 20840 (0.0010) -[2023-10-14 14:16:17,396][75950] Updated weights for policy 1, policy_version 20850 (0.0007) -[2023-10-14 14:16:17,773][75950] Updated weights for policy 1, policy_version 20860 (0.0008) -[2023-10-14 14:16:17,814][75949] Updated weights for policy 0, policy_version 20871 (0.0008) -[2023-10-14 14:16:18,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42729472. Throughput: 0: 1666.2, 1: 1666.9. Samples: 10686726. Policy #0 lag: (min: 20.0, avg: 26.6, max: 52.0) -[2023-10-14 14:16:18,164][74987] Avg episode reward: [(0, '20.000'), (1, '19.740')] -[2023-10-14 14:16:18,189][75949] Updated weights for policy 0, policy_version 20881 (0.0008) -[2023-10-14 14:16:18,562][75949] Updated weights for policy 0, policy_version 20891 (0.0009) -[2023-10-14 14:16:22,022][75950] Updated weights for policy 1, policy_version 20870 (0.0008) -[2023-10-14 14:16:22,401][75950] Updated weights for policy 1, policy_version 20880 (0.0009) -[2023-10-14 14:16:22,594][75949] Updated weights for policy 0, policy_version 20901 (0.0009) -[2023-10-14 14:16:22,769][75950] Updated weights for policy 1, policy_version 20890 (0.0009) -[2023-10-14 14:16:22,970][75949] Updated weights for policy 0, policy_version 20911 (0.0007) -[2023-10-14 14:16:23,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 42795008. Throughput: 0: 1662.8, 1: 1667.3. Samples: 10707268. Policy #0 lag: (min: 20.0, avg: 26.6, max: 52.0) -[2023-10-14 14:16:23,164][74987] Avg episode reward: [(0, '20.560'), (1, '21.720')] -[2023-10-14 14:16:23,336][75949] Updated weights for policy 0, policy_version 20921 (0.0012) -[2023-10-14 14:16:26,777][75950] Updated weights for policy 1, policy_version 20900 (0.0009) -[2023-10-14 14:16:27,147][75950] Updated weights for policy 1, policy_version 20910 (0.0009) -[2023-10-14 14:16:27,374][75949] Updated weights for policy 0, policy_version 20931 (0.0009) -[2023-10-14 14:16:27,516][75950] Updated weights for policy 1, policy_version 20920 (0.0010) -[2023-10-14 14:16:27,738][75949] Updated weights for policy 0, policy_version 20941 (0.0009) -[2023-10-14 14:16:28,107][75949] Updated weights for policy 0, policy_version 20951 (0.0007) -[2023-10-14 14:16:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 42860544. Throughput: 0: 1655.2, 1: 1655.6. Samples: 10726446. Policy #0 lag: (min: 20.0, avg: 26.6, max: 52.0) -[2023-10-14 14:16:28,165][74987] Avg episode reward: [(0, '20.750'), (1, '19.980')] -[2023-10-14 14:16:31,741][75950] Updated weights for policy 1, policy_version 20930 (0.0009) -[2023-10-14 14:16:32,102][75949] Updated weights for policy 0, policy_version 20961 (0.0008) -[2023-10-14 14:16:32,104][75950] Updated weights for policy 1, policy_version 20940 (0.0009) -[2023-10-14 14:16:32,472][75950] Updated weights for policy 1, policy_version 20950 (0.0008) -[2023-10-14 14:16:32,475][75949] Updated weights for policy 0, policy_version 20971 (0.0007) -[2023-10-14 14:16:32,834][75950] Updated weights for policy 1, policy_version 20960 (0.0007) -[2023-10-14 14:16:32,850][75949] Updated weights for policy 0, policy_version 20981 (0.0009) -[2023-10-14 14:16:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42926080. Throughput: 0: 1666.0, 1: 1672.2. Samples: 10736978. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) -[2023-10-14 14:16:33,165][74987] Avg episode reward: [(0, '21.940'), (1, '21.350')] -[2023-10-14 14:16:33,228][75949] Updated weights for policy 0, policy_version 20991 (0.0010) -[2023-10-14 14:16:33,258][75615] Saving new best policy, reward=21.940! -[2023-10-14 14:16:36,919][75950] Updated weights for policy 1, policy_version 20970 (0.0009) -[2023-10-14 14:16:37,298][75950] Updated weights for policy 1, policy_version 20980 (0.0009) -[2023-10-14 14:16:37,459][75949] Updated weights for policy 0, policy_version 21001 (0.0008) -[2023-10-14 14:16:37,663][75950] Updated weights for policy 1, policy_version 20990 (0.0009) -[2023-10-14 14:16:37,831][75949] Updated weights for policy 0, policy_version 21011 (0.0008) -[2023-10-14 14:16:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 42991616. Throughput: 0: 1674.8, 1: 1673.2. Samples: 10757646. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) -[2023-10-14 14:16:38,164][74987] Avg episode reward: [(0, '20.050'), (1, '19.670')] -[2023-10-14 14:16:38,203][75949] Updated weights for policy 0, policy_version 21021 (0.0007) -[2023-10-14 14:16:41,547][75950] Updated weights for policy 1, policy_version 21000 (0.0008) -[2023-10-14 14:16:41,911][75950] Updated weights for policy 1, policy_version 21010 (0.0008) -[2023-10-14 14:16:42,273][75950] Updated weights for policy 1, policy_version 21020 (0.0009) -[2023-10-14 14:16:42,300][75949] Updated weights for policy 0, policy_version 21031 (0.0008) -[2023-10-14 14:16:42,679][75949] Updated weights for policy 0, policy_version 21041 (0.0007) -[2023-10-14 14:16:43,052][75949] Updated weights for policy 0, policy_version 21051 (0.0007) -[2023-10-14 14:16:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 43057152. Throughput: 0: 1662.3, 1: 1663.9. Samples: 10776718. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) -[2023-10-14 14:16:43,165][74987] Avg episode reward: [(0, '22.210'), (1, '22.160')] -[2023-10-14 14:16:43,177][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000021024_21528576.pth... -[2023-10-14 14:16:43,213][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000019456_19922944.pth -[2023-10-14 14:16:43,217][75801] Saving new best policy, reward=22.160! -[2023-10-14 14:16:43,228][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000021056_21561344.pth... -[2023-10-14 14:16:43,269][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000019488_19955712.pth -[2023-10-14 14:16:43,273][75615] Saving new best policy, reward=22.210! -[2023-10-14 14:16:46,458][75950] Updated weights for policy 1, policy_version 21030 (0.0008) -[2023-10-14 14:16:46,828][75950] Updated weights for policy 1, policy_version 21040 (0.0009) -[2023-10-14 14:16:47,197][75950] Updated weights for policy 1, policy_version 21050 (0.0008) -[2023-10-14 14:16:47,232][75949] Updated weights for policy 0, policy_version 21061 (0.0009) -[2023-10-14 14:16:47,600][75949] Updated weights for policy 0, policy_version 21071 (0.0007) -[2023-10-14 14:16:47,975][75949] Updated weights for policy 0, policy_version 21081 (0.0008) -[2023-10-14 14:16:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 43122688. Throughput: 0: 1673.7, 1: 1674.0. Samples: 10787468. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) -[2023-10-14 14:16:48,164][74987] Avg episode reward: [(0, '20.350'), (1, '19.890')] -[2023-10-14 14:16:51,302][75950] Updated weights for policy 1, policy_version 21060 (0.0008) -[2023-10-14 14:16:51,674][75950] Updated weights for policy 1, policy_version 21070 (0.0008) -[2023-10-14 14:16:52,010][75949] Updated weights for policy 0, policy_version 21091 (0.0007) -[2023-10-14 14:16:52,045][75950] Updated weights for policy 1, policy_version 21080 (0.0007) -[2023-10-14 14:16:52,378][75949] Updated weights for policy 0, policy_version 21101 (0.0009) -[2023-10-14 14:16:52,755][75949] Updated weights for policy 0, policy_version 21111 (0.0009) -[2023-10-14 14:16:53,164][74987] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 43220992. Throughput: 0: 1676.0, 1: 1667.1. Samples: 10807582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:16:53,164][74987] Avg episode reward: [(0, '21.050'), (1, '21.520')] -[2023-10-14 14:16:55,948][75950] Updated weights for policy 1, policy_version 21090 (0.0008) -[2023-10-14 14:16:56,319][75950] Updated weights for policy 1, policy_version 21100 (0.0009) -[2023-10-14 14:16:56,690][75950] Updated weights for policy 1, policy_version 21110 (0.0009) -[2023-10-14 14:16:56,912][75949] Updated weights for policy 0, policy_version 21121 (0.0009) -[2023-10-14 14:16:57,055][75950] Updated weights for policy 1, policy_version 21120 (0.0007) -[2023-10-14 14:16:57,317][75949] Updated weights for policy 0, policy_version 21131 (0.0010) -[2023-10-14 14:16:57,690][75949] Updated weights for policy 0, policy_version 21141 (0.0009) -[2023-10-14 14:16:58,065][75949] Updated weights for policy 0, policy_version 21151 (0.0007) -[2023-10-14 14:16:58,164][74987] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 43286528. Throughput: 0: 1660.4, 1: 1672.5. Samples: 10826824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:16:58,165][74987] Avg episode reward: [(0, '20.390'), (1, '19.590')] -[2023-10-14 14:17:01,010][75950] Updated weights for policy 1, policy_version 21130 (0.0011) -[2023-10-14 14:17:01,379][75950] Updated weights for policy 1, policy_version 21140 (0.0008) -[2023-10-14 14:17:01,747][75950] Updated weights for policy 1, policy_version 21150 (0.0008) -[2023-10-14 14:17:02,018][75949] Updated weights for policy 0, policy_version 21161 (0.0008) -[2023-10-14 14:17:02,394][75949] Updated weights for policy 0, policy_version 21171 (0.0010) -[2023-10-14 14:17:02,757][75949] Updated weights for policy 0, policy_version 21181 (0.0010) -[2023-10-14 14:17:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 43352064. Throughput: 0: 1677.8, 1: 1689.4. Samples: 10838250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:17:03,165][74987] Avg episode reward: [(0, '20.890'), (1, '20.690')] -[2023-10-14 14:17:05,940][75950] Updated weights for policy 1, policy_version 21160 (0.0008) -[2023-10-14 14:17:06,311][75950] Updated weights for policy 1, policy_version 21170 (0.0009) -[2023-10-14 14:17:06,686][75950] Updated weights for policy 1, policy_version 21180 (0.0007) -[2023-10-14 14:17:06,721][75949] Updated weights for policy 0, policy_version 21191 (0.0010) -[2023-10-14 14:17:07,082][75949] Updated weights for policy 0, policy_version 21201 (0.0009) -[2023-10-14 14:17:07,452][75949] Updated weights for policy 0, policy_version 21211 (0.0010) -[2023-10-14 14:17:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 43417600. Throughput: 0: 1675.9, 1: 1666.7. Samples: 10857690. Policy #0 lag: (min: 11.0, avg: 12.5, max: 38.0) -[2023-10-14 14:17:08,165][74987] Avg episode reward: [(0, '20.820'), (1, '18.390')] -[2023-10-14 14:17:10,864][75950] Updated weights for policy 1, policy_version 21190 (0.0008) -[2023-10-14 14:17:11,258][75950] Updated weights for policy 1, policy_version 21200 (0.0010) -[2023-10-14 14:17:11,432][75949] Updated weights for policy 0, policy_version 21221 (0.0010) -[2023-10-14 14:17:11,631][75950] Updated weights for policy 1, policy_version 21210 (0.0007) -[2023-10-14 14:17:11,800][75949] Updated weights for policy 0, policy_version 21231 (0.0007) -[2023-10-14 14:17:12,173][75949] Updated weights for policy 0, policy_version 21241 (0.0008) -[2023-10-14 14:17:13,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 43483136. Throughput: 0: 1663.7, 1: 1685.3. Samples: 10877152. Policy #0 lag: (min: 11.0, avg: 12.5, max: 38.0) -[2023-10-14 14:17:13,164][74987] Avg episode reward: [(0, '20.310'), (1, '21.510')] -[2023-10-14 14:17:15,447][75950] Updated weights for policy 1, policy_version 21220 (0.0008) -[2023-10-14 14:17:15,820][75950] Updated weights for policy 1, policy_version 21230 (0.0008) -[2023-10-14 14:17:16,189][75950] Updated weights for policy 1, policy_version 21240 (0.0008) -[2023-10-14 14:17:16,261][75949] Updated weights for policy 0, policy_version 21251 (0.0010) -[2023-10-14 14:17:16,640][75949] Updated weights for policy 0, policy_version 21261 (0.0009) -[2023-10-14 14:17:17,012][75949] Updated weights for policy 0, policy_version 21271 (0.0008) -[2023-10-14 14:17:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 43548672. Throughput: 0: 1682.9, 1: 1689.3. Samples: 10888728. Policy #0 lag: (min: 11.0, avg: 12.5, max: 38.0) -[2023-10-14 14:17:18,165][74987] Avg episode reward: [(0, '20.970'), (1, '19.480')] -[2023-10-14 14:17:20,148][75950] Updated weights for policy 1, policy_version 21250 (0.0009) -[2023-10-14 14:17:20,522][75950] Updated weights for policy 1, policy_version 21260 (0.0008) -[2023-10-14 14:17:20,877][75950] Updated weights for policy 1, policy_version 21270 (0.0010) -[2023-10-14 14:17:21,145][75949] Updated weights for policy 0, policy_version 21281 (0.0008) -[2023-10-14 14:17:21,246][75950] Updated weights for policy 1, policy_version 21280 (0.0010) -[2023-10-14 14:17:21,518][75949] Updated weights for policy 0, policy_version 21291 (0.0009) -[2023-10-14 14:17:21,884][75949] Updated weights for policy 0, policy_version 21301 (0.0008) -[2023-10-14 14:17:22,263][75949] Updated weights for policy 0, policy_version 21311 (0.0009) -[2023-10-14 14:17:23,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 43614208. Throughput: 0: 1666.7, 1: 1673.7. Samples: 10907962. Policy #0 lag: (min: 11.0, avg: 12.5, max: 38.0) -[2023-10-14 14:17:23,164][74987] Avg episode reward: [(0, '21.260'), (1, '21.610')] -[2023-10-14 14:17:25,219][75950] Updated weights for policy 1, policy_version 21290 (0.0009) -[2023-10-14 14:17:25,600][75950] Updated weights for policy 1, policy_version 21300 (0.0008) -[2023-10-14 14:17:25,970][75950] Updated weights for policy 1, policy_version 21310 (0.0008) -[2023-10-14 14:17:26,259][75949] Updated weights for policy 0, policy_version 21321 (0.0010) -[2023-10-14 14:17:26,636][75949] Updated weights for policy 0, policy_version 21331 (0.0009) -[2023-10-14 14:17:27,006][75949] Updated weights for policy 0, policy_version 21341 (0.0007) -[2023-10-14 14:17:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 43679744. Throughput: 0: 1663.9, 1: 1699.9. Samples: 10928086. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) -[2023-10-14 14:17:28,165][74987] Avg episode reward: [(0, '20.780'), (1, '18.550')] -[2023-10-14 14:17:30,009][75950] Updated weights for policy 1, policy_version 21320 (0.0008) -[2023-10-14 14:17:30,381][75950] Updated weights for policy 1, policy_version 21330 (0.0009) -[2023-10-14 14:17:30,739][75950] Updated weights for policy 1, policy_version 21340 (0.0009) -[2023-10-14 14:17:30,989][75949] Updated weights for policy 0, policy_version 21351 (0.0008) -[2023-10-14 14:17:31,355][75949] Updated weights for policy 0, policy_version 21361 (0.0009) -[2023-10-14 14:17:31,724][75949] Updated weights for policy 0, policy_version 21371 (0.0010) -[2023-10-14 14:17:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 43745280. Throughput: 0: 1681.6, 1: 1678.2. Samples: 10938656. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) -[2023-10-14 14:17:33,164][74987] Avg episode reward: [(0, '21.390'), (1, '22.420')] -[2023-10-14 14:17:33,165][75801] Saving new best policy, reward=22.420! -[2023-10-14 14:17:34,715][75950] Updated weights for policy 1, policy_version 21350 (0.0008) -[2023-10-14 14:17:35,082][75950] Updated weights for policy 1, policy_version 21360 (0.0007) -[2023-10-14 14:17:35,461][75950] Updated weights for policy 1, policy_version 21370 (0.0008) -[2023-10-14 14:17:35,923][75949] Updated weights for policy 0, policy_version 21381 (0.0011) -[2023-10-14 14:17:36,294][75949] Updated weights for policy 0, policy_version 21391 (0.0009) -[2023-10-14 14:17:36,666][75949] Updated weights for policy 0, policy_version 21401 (0.0010) -[2023-10-14 14:17:38,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 43810816. Throughput: 0: 1661.6, 1: 1682.3. Samples: 10958056. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) -[2023-10-14 14:17:38,164][74987] Avg episode reward: [(0, '20.830'), (1, '21.270')] -[2023-10-14 14:17:39,485][75950] Updated weights for policy 1, policy_version 21380 (0.0008) -[2023-10-14 14:17:39,859][75950] Updated weights for policy 1, policy_version 21390 (0.0008) -[2023-10-14 14:17:40,220][75950] Updated weights for policy 1, policy_version 21400 (0.0007) -[2023-10-14 14:17:40,830][75949] Updated weights for policy 0, policy_version 21411 (0.0011) -[2023-10-14 14:17:41,201][75949] Updated weights for policy 0, policy_version 21421 (0.0008) -[2023-10-14 14:17:41,568][75949] Updated weights for policy 0, policy_version 21431 (0.0010) -[2023-10-14 14:17:43,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 43876352. Throughput: 0: 1671.0, 1: 1692.8. Samples: 10978198. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) -[2023-10-14 14:17:43,165][74987] Avg episode reward: [(0, '21.110'), (1, '23.690')] -[2023-10-14 14:17:43,177][75801] Saving new best policy, reward=23.690! -[2023-10-14 14:17:44,426][75950] Updated weights for policy 1, policy_version 21410 (0.0009) -[2023-10-14 14:17:44,795][75950] Updated weights for policy 1, policy_version 21420 (0.0009) -[2023-10-14 14:17:45,155][75950] Updated weights for policy 1, policy_version 21430 (0.0008) -[2023-10-14 14:17:45,525][75950] Updated weights for policy 1, policy_version 21440 (0.0009) -[2023-10-14 14:17:45,588][75949] Updated weights for policy 0, policy_version 21441 (0.0009) -[2023-10-14 14:17:46,012][75949] Updated weights for policy 0, policy_version 21451 (0.0008) -[2023-10-14 14:17:46,389][75949] Updated weights for policy 0, policy_version 21461 (0.0010) -[2023-10-14 14:17:46,753][75949] Updated weights for policy 0, policy_version 21471 (0.0009) -[2023-10-14 14:17:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 43941888. Throughput: 0: 1680.1, 1: 1655.9. Samples: 10988370. Policy #0 lag: (min: 17.0, avg: 24.5, max: 49.0) -[2023-10-14 14:17:48,165][74987] Avg episode reward: [(0, '20.650'), (1, '21.340')] -[2023-10-14 14:17:49,846][75950] Updated weights for policy 1, policy_version 21450 (0.0008) -[2023-10-14 14:17:50,222][75950] Updated weights for policy 1, policy_version 21460 (0.0009) -[2023-10-14 14:17:50,593][75950] Updated weights for policy 1, policy_version 21470 (0.0008) -[2023-10-14 14:17:51,003][75949] Updated weights for policy 0, policy_version 21481 (0.0008) -[2023-10-14 14:17:51,372][75949] Updated weights for policy 0, policy_version 21491 (0.0009) -[2023-10-14 14:17:51,742][75949] Updated weights for policy 0, policy_version 21501 (0.0009) -[2023-10-14 14:17:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44007424. Throughput: 0: 1658.1, 1: 1676.7. Samples: 11007754. Policy #0 lag: (min: 17.0, avg: 24.5, max: 49.0) -[2023-10-14 14:17:53,165][74987] Avg episode reward: [(0, '21.260'), (1, '21.060')] -[2023-10-14 14:17:54,743][75950] Updated weights for policy 1, policy_version 21480 (0.0009) -[2023-10-14 14:17:55,100][75950] Updated weights for policy 1, policy_version 21490 (0.0008) -[2023-10-14 14:17:55,476][75950] Updated weights for policy 1, policy_version 21500 (0.0009) -[2023-10-14 14:17:55,957][75949] Updated weights for policy 0, policy_version 21511 (0.0007) -[2023-10-14 14:17:56,317][75949] Updated weights for policy 0, policy_version 21521 (0.0008) -[2023-10-14 14:17:56,688][75949] Updated weights for policy 0, policy_version 21531 (0.0010) -[2023-10-14 14:17:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44072960. Throughput: 0: 1671.4, 1: 1681.2. Samples: 11028022. Policy #0 lag: (min: 17.0, avg: 24.5, max: 49.0) -[2023-10-14 14:17:58,164][74987] Avg episode reward: [(0, '19.740'), (1, '20.210')] -[2023-10-14 14:17:59,745][75950] Updated weights for policy 1, policy_version 21510 (0.0010) -[2023-10-14 14:18:00,138][75950] Updated weights for policy 1, policy_version 21520 (0.0009) -[2023-10-14 14:18:00,502][75950] Updated weights for policy 1, policy_version 21530 (0.0010) -[2023-10-14 14:18:00,629][75949] Updated weights for policy 0, policy_version 21541 (0.0008) -[2023-10-14 14:18:01,009][75949] Updated weights for policy 0, policy_version 21551 (0.0008) -[2023-10-14 14:18:01,378][75949] Updated weights for policy 0, policy_version 21561 (0.0010) -[2023-10-14 14:18:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 44138496. Throughput: 0: 1671.0, 1: 1651.8. Samples: 11038252. Policy #0 lag: (min: 17.0, avg: 24.5, max: 49.0) -[2023-10-14 14:18:03,165][74987] Avg episode reward: [(0, '20.120'), (1, '20.790')] -[2023-10-14 14:18:04,271][75950] Updated weights for policy 1, policy_version 21540 (0.0009) -[2023-10-14 14:18:04,640][75950] Updated weights for policy 1, policy_version 21550 (0.0008) -[2023-10-14 14:18:05,012][75950] Updated weights for policy 1, policy_version 21560 (0.0008) -[2023-10-14 14:18:05,285][75949] Updated weights for policy 0, policy_version 21571 (0.0009) -[2023-10-14 14:18:05,663][75949] Updated weights for policy 0, policy_version 21581 (0.0010) -[2023-10-14 14:18:06,043][75949] Updated weights for policy 0, policy_version 21591 (0.0009) -[2023-10-14 14:18:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44204032. Throughput: 0: 1660.9, 1: 1671.3. Samples: 11057910. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) -[2023-10-14 14:18:08,164][74987] Avg episode reward: [(0, '20.520'), (1, '21.360')] -[2023-10-14 14:18:09,238][75950] Updated weights for policy 1, policy_version 21570 (0.0008) -[2023-10-14 14:18:09,600][75950] Updated weights for policy 1, policy_version 21580 (0.0008) -[2023-10-14 14:18:09,963][75950] Updated weights for policy 1, policy_version 21590 (0.0011) -[2023-10-14 14:18:10,146][75949] Updated weights for policy 0, policy_version 21601 (0.0010) -[2023-10-14 14:18:10,329][75950] Updated weights for policy 1, policy_version 21600 (0.0008) -[2023-10-14 14:18:10,525][75949] Updated weights for policy 0, policy_version 21611 (0.0009) -[2023-10-14 14:18:10,901][75949] Updated weights for policy 0, policy_version 21621 (0.0007) -[2023-10-14 14:18:11,268][75949] Updated weights for policy 0, policy_version 21631 (0.0009) -[2023-10-14 14:18:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44269568. Throughput: 0: 1678.5, 1: 1664.8. Samples: 11078534. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) -[2023-10-14 14:18:13,164][74987] Avg episode reward: [(0, '20.710'), (1, '22.150')] -[2023-10-14 14:18:14,593][75950] Updated weights for policy 1, policy_version 21610 (0.0009) -[2023-10-14 14:18:14,961][75950] Updated weights for policy 1, policy_version 21620 (0.0009) -[2023-10-14 14:18:15,323][75950] Updated weights for policy 1, policy_version 21630 (0.0009) -[2023-10-14 14:18:15,332][75949] Updated weights for policy 0, policy_version 21641 (0.0009) -[2023-10-14 14:18:15,689][75949] Updated weights for policy 0, policy_version 21651 (0.0008) -[2023-10-14 14:18:16,061][75949] Updated weights for policy 0, policy_version 21661 (0.0009) -[2023-10-14 14:18:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 44335104. Throughput: 0: 1662.4, 1: 1659.3. Samples: 11088132. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) -[2023-10-14 14:18:18,165][74987] Avg episode reward: [(0, '19.990'), (1, '21.510')] -[2023-10-14 14:18:19,304][75950] Updated weights for policy 1, policy_version 21640 (0.0010) -[2023-10-14 14:18:19,667][75950] Updated weights for policy 1, policy_version 21650 (0.0008) -[2023-10-14 14:18:20,031][75950] Updated weights for policy 1, policy_version 21660 (0.0009) -[2023-10-14 14:18:20,060][75949] Updated weights for policy 0, policy_version 21671 (0.0007) -[2023-10-14 14:18:20,437][75949] Updated weights for policy 0, policy_version 21681 (0.0010) -[2023-10-14 14:18:20,802][75949] Updated weights for policy 0, policy_version 21691 (0.0010) -[2023-10-14 14:18:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44400640. Throughput: 0: 1670.7, 1: 1664.6. Samples: 11108146. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) -[2023-10-14 14:18:23,164][74987] Avg episode reward: [(0, '20.310'), (1, '19.590')] -[2023-10-14 14:18:24,111][75950] Updated weights for policy 1, policy_version 21670 (0.0007) -[2023-10-14 14:18:24,481][75950] Updated weights for policy 1, policy_version 21680 (0.0008) -[2023-10-14 14:18:24,852][75950] Updated weights for policy 1, policy_version 21690 (0.0008) -[2023-10-14 14:18:24,877][75949] Updated weights for policy 0, policy_version 21701 (0.0012) -[2023-10-14 14:18:25,248][75949] Updated weights for policy 0, policy_version 21711 (0.0008) -[2023-10-14 14:18:25,619][75949] Updated weights for policy 0, policy_version 21721 (0.0007) -[2023-10-14 14:18:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 44466176. Throughput: 0: 1681.6, 1: 1666.8. Samples: 11128876. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) -[2023-10-14 14:18:28,165][74987] Avg episode reward: [(0, '20.080'), (1, '22.610')] -[2023-10-14 14:18:28,935][75950] Updated weights for policy 1, policy_version 21700 (0.0010) -[2023-10-14 14:18:29,306][75950] Updated weights for policy 1, policy_version 21710 (0.0008) -[2023-10-14 14:18:29,663][75950] Updated weights for policy 1, policy_version 21720 (0.0008) -[2023-10-14 14:18:29,680][75949] Updated weights for policy 0, policy_version 21731 (0.0008) -[2023-10-14 14:18:30,042][75949] Updated weights for policy 0, policy_version 21741 (0.0007) -[2023-10-14 14:18:30,415][75949] Updated weights for policy 0, policy_version 21751 (0.0007) -[2023-10-14 14:18:33,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44531712. Throughput: 0: 1659.9, 1: 1671.6. Samples: 11138288. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) -[2023-10-14 14:18:33,164][74987] Avg episode reward: [(0, '20.060'), (1, '19.920')] -[2023-10-14 14:18:33,673][75950] Updated weights for policy 1, policy_version 21730 (0.0008) -[2023-10-14 14:18:34,035][75950] Updated weights for policy 1, policy_version 21740 (0.0010) -[2023-10-14 14:18:34,409][75950] Updated weights for policy 1, policy_version 21750 (0.0010) -[2023-10-14 14:18:34,579][75949] Updated weights for policy 0, policy_version 21761 (0.0008) -[2023-10-14 14:18:34,771][75950] Updated weights for policy 1, policy_version 21760 (0.0009) -[2023-10-14 14:18:35,007][75949] Updated weights for policy 0, policy_version 21771 (0.0010) -[2023-10-14 14:18:35,380][75949] Updated weights for policy 0, policy_version 21781 (0.0008) -[2023-10-14 14:18:35,744][75949] Updated weights for policy 0, policy_version 21791 (0.0011) -[2023-10-14 14:18:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44597248. Throughput: 0: 1676.5, 1: 1669.7. Samples: 11158332. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) -[2023-10-14 14:18:38,164][74987] Avg episode reward: [(0, '20.380'), (1, '21.920')] -[2023-10-14 14:18:38,920][75950] Updated weights for policy 1, policy_version 21770 (0.0007) -[2023-10-14 14:18:39,279][75950] Updated weights for policy 1, policy_version 21780 (0.0008) -[2023-10-14 14:18:39,646][75950] Updated weights for policy 1, policy_version 21790 (0.0008) -[2023-10-14 14:18:39,732][75949] Updated weights for policy 0, policy_version 21801 (0.0007) -[2023-10-14 14:18:40,093][75949] Updated weights for policy 0, policy_version 21811 (0.0010) -[2023-10-14 14:18:40,464][75949] Updated weights for policy 0, policy_version 21821 (0.0009) -[2023-10-14 14:18:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44662784. Throughput: 0: 1689.0, 1: 1674.3. Samples: 11179372. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) -[2023-10-14 14:18:43,165][74987] Avg episode reward: [(0, '21.050'), (1, '18.650')] -[2023-10-14 14:18:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000021792_22315008.pth... -[2023-10-14 14:18:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000021824_22347776.pth... -[2023-10-14 14:18:43,216][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000020224_20709376.pth -[2023-10-14 14:18:43,216][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000020256_20742144.pth -[2023-10-14 14:18:43,823][75950] Updated weights for policy 1, policy_version 21800 (0.0008) -[2023-10-14 14:18:44,187][75950] Updated weights for policy 1, policy_version 21810 (0.0009) -[2023-10-14 14:18:44,509][75949] Updated weights for policy 0, policy_version 21831 (0.0009) -[2023-10-14 14:18:44,556][75950] Updated weights for policy 1, policy_version 21820 (0.0009) -[2023-10-14 14:18:44,887][75949] Updated weights for policy 0, policy_version 21841 (0.0008) -[2023-10-14 14:18:45,249][75949] Updated weights for policy 0, policy_version 21851 (0.0010) -[2023-10-14 14:18:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44728320. Throughput: 0: 1664.0, 1: 1676.3. Samples: 11188564. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 14:18:48,165][74987] Avg episode reward: [(0, '20.450'), (1, '21.720')] -[2023-10-14 14:18:48,736][75950] Updated weights for policy 1, policy_version 21830 (0.0008) -[2023-10-14 14:18:49,113][75950] Updated weights for policy 1, policy_version 21840 (0.0009) -[2023-10-14 14:18:49,365][75949] Updated weights for policy 0, policy_version 21861 (0.0009) -[2023-10-14 14:18:49,478][75950] Updated weights for policy 1, policy_version 21850 (0.0009) -[2023-10-14 14:18:49,726][75949] Updated weights for policy 0, policy_version 21871 (0.0008) -[2023-10-14 14:18:50,102][75949] Updated weights for policy 0, policy_version 21881 (0.0008) -[2023-10-14 14:18:53,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 44793856. Throughput: 0: 1687.5, 1: 1669.2. Samples: 11208962. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 14:18:53,164][74987] Avg episode reward: [(0, '20.490'), (1, '19.240')] -[2023-10-14 14:18:53,381][75950] Updated weights for policy 1, policy_version 21860 (0.0009) -[2023-10-14 14:18:53,753][75950] Updated weights for policy 1, policy_version 21870 (0.0007) -[2023-10-14 14:18:54,113][75950] Updated weights for policy 1, policy_version 21880 (0.0008) -[2023-10-14 14:18:54,148][75949] Updated weights for policy 0, policy_version 21891 (0.0008) -[2023-10-14 14:18:54,511][75949] Updated weights for policy 0, policy_version 21901 (0.0007) -[2023-10-14 14:18:54,878][75949] Updated weights for policy 0, policy_version 21911 (0.0008) -[2023-10-14 14:18:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 44859392. Throughput: 0: 1682.9, 1: 1670.0. Samples: 11229414. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 14:18:58,165][74987] Avg episode reward: [(0, '19.760'), (1, '20.220')] -[2023-10-14 14:18:58,305][75950] Updated weights for policy 1, policy_version 21890 (0.0008) -[2023-10-14 14:18:58,674][75950] Updated weights for policy 1, policy_version 21900 (0.0009) -[2023-10-14 14:18:59,037][75950] Updated weights for policy 1, policy_version 21910 (0.0010) -[2023-10-14 14:18:59,098][75949] Updated weights for policy 0, policy_version 21921 (0.0007) -[2023-10-14 14:18:59,407][75950] Updated weights for policy 1, policy_version 21920 (0.0008) -[2023-10-14 14:18:59,477][75949] Updated weights for policy 0, policy_version 21931 (0.0008) -[2023-10-14 14:18:59,843][75949] Updated weights for policy 0, policy_version 21941 (0.0010) -[2023-10-14 14:19:00,215][75949] Updated weights for policy 0, policy_version 21951 (0.0011) -[2023-10-14 14:19:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44924928. Throughput: 0: 1668.3, 1: 1669.9. Samples: 11238350. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 14:19:03,164][74987] Avg episode reward: [(0, '21.720'), (1, '20.090')] -[2023-10-14 14:19:03,597][75950] Updated weights for policy 1, policy_version 21930 (0.0009) -[2023-10-14 14:19:03,968][75950] Updated weights for policy 1, policy_version 21940 (0.0008) -[2023-10-14 14:19:04,252][75949] Updated weights for policy 0, policy_version 21961 (0.0007) -[2023-10-14 14:19:04,325][75950] Updated weights for policy 1, policy_version 21950 (0.0007) -[2023-10-14 14:19:04,627][75949] Updated weights for policy 0, policy_version 21971 (0.0007) -[2023-10-14 14:19:05,001][75949] Updated weights for policy 0, policy_version 21981 (0.0008) -[2023-10-14 14:19:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44990464. Throughput: 0: 1681.2, 1: 1667.3. Samples: 11258832. Policy #0 lag: (min: 18.0, avg: 22.0, max: 50.0) -[2023-10-14 14:19:08,165][74987] Avg episode reward: [(0, '21.660'), (1, '19.510')] -[2023-10-14 14:19:08,590][75950] Updated weights for policy 1, policy_version 21960 (0.0008) -[2023-10-14 14:19:08,956][75950] Updated weights for policy 1, policy_version 21970 (0.0009) -[2023-10-14 14:19:09,205][75949] Updated weights for policy 0, policy_version 21991 (0.0007) -[2023-10-14 14:19:09,323][75950] Updated weights for policy 1, policy_version 21980 (0.0007) -[2023-10-14 14:19:09,580][75949] Updated weights for policy 0, policy_version 22001 (0.0008) -[2023-10-14 14:19:09,946][75949] Updated weights for policy 0, policy_version 22011 (0.0008) -[2023-10-14 14:19:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45056000. Throughput: 0: 1674.5, 1: 1666.7. Samples: 11279232. Policy #0 lag: (min: 18.0, avg: 22.0, max: 50.0) -[2023-10-14 14:19:13,164][74987] Avg episode reward: [(0, '21.460'), (1, '23.070')] -[2023-10-14 14:19:13,374][75950] Updated weights for policy 1, policy_version 21990 (0.0008) -[2023-10-14 14:19:13,749][75950] Updated weights for policy 1, policy_version 22000 (0.0009) -[2023-10-14 14:19:13,902][75949] Updated weights for policy 0, policy_version 22021 (0.0010) -[2023-10-14 14:19:14,110][75950] Updated weights for policy 1, policy_version 22010 (0.0008) -[2023-10-14 14:19:14,275][75949] Updated weights for policy 0, policy_version 22031 (0.0009) -[2023-10-14 14:19:14,638][75949] Updated weights for policy 0, policy_version 22041 (0.0009) -[2023-10-14 14:19:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 45121536. Throughput: 0: 1670.0, 1: 1669.1. Samples: 11288548. Policy #0 lag: (min: 18.0, avg: 22.0, max: 50.0) -[2023-10-14 14:19:18,165][74987] Avg episode reward: [(0, '19.800'), (1, '19.350')] -[2023-10-14 14:19:18,273][75950] Updated weights for policy 1, policy_version 22020 (0.0008) -[2023-10-14 14:19:18,643][75950] Updated weights for policy 1, policy_version 22030 (0.0011) -[2023-10-14 14:19:18,777][75949] Updated weights for policy 0, policy_version 22051 (0.0008) -[2023-10-14 14:19:19,019][75950] Updated weights for policy 1, policy_version 22040 (0.0008) -[2023-10-14 14:19:19,146][75949] Updated weights for policy 0, policy_version 22061 (0.0009) -[2023-10-14 14:19:19,514][75949] Updated weights for policy 0, policy_version 22071 (0.0008) -[2023-10-14 14:19:22,824][75950] Updated weights for policy 1, policy_version 22050 (0.0008) -[2023-10-14 14:19:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45187072. Throughput: 0: 1680.0, 1: 1673.5. Samples: 11309242. Policy #0 lag: (min: 18.0, avg: 22.0, max: 50.0) -[2023-10-14 14:19:23,164][74987] Avg episode reward: [(0, '21.570'), (1, '22.250')] -[2023-10-14 14:19:23,192][75950] Updated weights for policy 1, policy_version 22060 (0.0009) -[2023-10-14 14:19:23,572][75950] Updated weights for policy 1, policy_version 22070 (0.0008) -[2023-10-14 14:19:23,681][75949] Updated weights for policy 0, policy_version 22081 (0.0009) -[2023-10-14 14:19:23,936][75950] Updated weights for policy 1, policy_version 22080 (0.0009) -[2023-10-14 14:19:24,092][75949] Updated weights for policy 0, policy_version 22091 (0.0008) -[2023-10-14 14:19:24,469][75949] Updated weights for policy 0, policy_version 22101 (0.0008) -[2023-10-14 14:19:24,845][75949] Updated weights for policy 0, policy_version 22111 (0.0009) -[2023-10-14 14:19:28,079][75950] Updated weights for policy 1, policy_version 22090 (0.0008) -[2023-10-14 14:19:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45252608. Throughput: 0: 1671.5, 1: 1672.6. Samples: 11329858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 14:19:28,165][74987] Avg episode reward: [(0, '19.970'), (1, '20.030')] -[2023-10-14 14:19:28,448][75950] Updated weights for policy 1, policy_version 22100 (0.0011) -[2023-10-14 14:19:28,811][75950] Updated weights for policy 1, policy_version 22110 (0.0007) -[2023-10-14 14:19:28,868][75949] Updated weights for policy 0, policy_version 22121 (0.0008) -[2023-10-14 14:19:29,237][75949] Updated weights for policy 0, policy_version 22131 (0.0009) -[2023-10-14 14:19:29,606][75949] Updated weights for policy 0, policy_version 22141 (0.0010) -[2023-10-14 14:19:32,916][75950] Updated weights for policy 1, policy_version 22120 (0.0009) -[2023-10-14 14:19:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45318144. Throughput: 0: 1670.3, 1: 1672.7. Samples: 11338998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 14:19:33,164][74987] Avg episode reward: [(0, '19.560'), (1, '21.420')] -[2023-10-14 14:19:33,286][75950] Updated weights for policy 1, policy_version 22130 (0.0008) -[2023-10-14 14:19:33,488][75949] Updated weights for policy 0, policy_version 22151 (0.0007) -[2023-10-14 14:19:33,653][75950] Updated weights for policy 1, policy_version 22140 (0.0007) -[2023-10-14 14:19:33,859][75949] Updated weights for policy 0, policy_version 22161 (0.0009) -[2023-10-14 14:19:34,221][75949] Updated weights for policy 0, policy_version 22171 (0.0009) -[2023-10-14 14:19:37,861][75950] Updated weights for policy 1, policy_version 22150 (0.0009) -[2023-10-14 14:19:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45383680. Throughput: 0: 1672.1, 1: 1679.3. Samples: 11359776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 14:19:38,165][74987] Avg episode reward: [(0, '20.230'), (1, '19.810')] -[2023-10-14 14:19:38,238][75950] Updated weights for policy 1, policy_version 22160 (0.0008) -[2023-10-14 14:19:38,450][75949] Updated weights for policy 0, policy_version 22181 (0.0009) -[2023-10-14 14:19:38,601][75950] Updated weights for policy 1, policy_version 22170 (0.0009) -[2023-10-14 14:19:38,817][75949] Updated weights for policy 0, policy_version 22191 (0.0008) -[2023-10-14 14:19:39,193][75949] Updated weights for policy 0, policy_version 22201 (0.0010) -[2023-10-14 14:19:42,688][75950] Updated weights for policy 1, policy_version 22180 (0.0009) -[2023-10-14 14:19:43,064][75950] Updated weights for policy 1, policy_version 22190 (0.0010) -[2023-10-14 14:19:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45449216. Throughput: 0: 1670.1, 1: 1678.4. Samples: 11380098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 14:19:43,165][74987] Avg episode reward: [(0, '21.310'), (1, '21.140')] -[2023-10-14 14:19:43,367][75949] Updated weights for policy 0, policy_version 22211 (0.0010) -[2023-10-14 14:19:43,425][75950] Updated weights for policy 1, policy_version 22200 (0.0008) -[2023-10-14 14:19:43,735][75949] Updated weights for policy 0, policy_version 22221 (0.0007) -[2023-10-14 14:19:44,111][75949] Updated weights for policy 0, policy_version 22231 (0.0008) -[2023-10-14 14:19:47,215][75950] Updated weights for policy 1, policy_version 22210 (0.0007) -[2023-10-14 14:19:47,591][75950] Updated weights for policy 1, policy_version 22220 (0.0010) -[2023-10-14 14:19:47,945][75950] Updated weights for policy 1, policy_version 22230 (0.0009) -[2023-10-14 14:19:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45514752. Throughput: 0: 1672.3, 1: 1683.6. Samples: 11389366. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) -[2023-10-14 14:19:48,164][74987] Avg episode reward: [(0, '21.030'), (1, '20.230')] -[2023-10-14 14:19:48,272][75949] Updated weights for policy 0, policy_version 22241 (0.0008) -[2023-10-14 14:19:48,311][75950] Updated weights for policy 1, policy_version 22240 (0.0010) -[2023-10-14 14:19:48,640][75949] Updated weights for policy 0, policy_version 22251 (0.0010) -[2023-10-14 14:19:49,012][75949] Updated weights for policy 0, policy_version 22261 (0.0008) -[2023-10-14 14:19:49,390][75949] Updated weights for policy 0, policy_version 22271 (0.0007) -[2023-10-14 14:19:52,464][75950] Updated weights for policy 1, policy_version 22250 (0.0007) -[2023-10-14 14:19:52,828][75950] Updated weights for policy 1, policy_version 22260 (0.0007) -[2023-10-14 14:19:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 45580288. Throughput: 0: 1668.2, 1: 1687.7. Samples: 11409850. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) -[2023-10-14 14:19:53,165][74987] Avg episode reward: [(0, '22.310'), (1, '20.360')] -[2023-10-14 14:19:53,194][75950] Updated weights for policy 1, policy_version 22270 (0.0010) -[2023-10-14 14:19:53,497][75949] Updated weights for policy 0, policy_version 22281 (0.0008) -[2023-10-14 14:19:53,862][75949] Updated weights for policy 0, policy_version 22291 (0.0009) -[2023-10-14 14:19:54,246][75949] Updated weights for policy 0, policy_version 22301 (0.0008) -[2023-10-14 14:19:54,348][75615] Saving new best policy, reward=22.310! -[2023-10-14 14:19:57,340][75950] Updated weights for policy 1, policy_version 22280 (0.0007) -[2023-10-14 14:19:57,695][75950] Updated weights for policy 1, policy_version 22290 (0.0008) -[2023-10-14 14:19:58,064][75950] Updated weights for policy 1, policy_version 22300 (0.0008) -[2023-10-14 14:19:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45645824. Throughput: 0: 1679.2, 1: 1673.8. Samples: 11430116. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) -[2023-10-14 14:19:58,165][74987] Avg episode reward: [(0, '21.000'), (1, '21.850')] -[2023-10-14 14:19:58,189][75949] Updated weights for policy 0, policy_version 22311 (0.0009) -[2023-10-14 14:19:58,559][75949] Updated weights for policy 0, policy_version 22321 (0.0008) -[2023-10-14 14:19:58,935][75949] Updated weights for policy 0, policy_version 22331 (0.0008) -[2023-10-14 14:20:02,222][75950] Updated weights for policy 1, policy_version 22310 (0.0008) -[2023-10-14 14:20:02,592][75950] Updated weights for policy 1, policy_version 22320 (0.0007) -[2023-10-14 14:20:02,945][75949] Updated weights for policy 0, policy_version 22341 (0.0008) -[2023-10-14 14:20:02,954][75950] Updated weights for policy 1, policy_version 22330 (0.0008) -[2023-10-14 14:20:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45711360. Throughput: 0: 1678.5, 1: 1683.4. Samples: 11439836. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) -[2023-10-14 14:20:03,164][74987] Avg episode reward: [(0, '21.530'), (1, '22.540')] -[2023-10-14 14:20:03,324][75949] Updated weights for policy 0, policy_version 22351 (0.0007) -[2023-10-14 14:20:03,690][75949] Updated weights for policy 0, policy_version 22361 (0.0008) -[2023-10-14 14:20:06,970][75950] Updated weights for policy 1, policy_version 22340 (0.0009) -[2023-10-14 14:20:07,338][75950] Updated weights for policy 1, policy_version 22350 (0.0008) -[2023-10-14 14:20:07,710][75950] Updated weights for policy 1, policy_version 22360 (0.0010) -[2023-10-14 14:20:07,754][75949] Updated weights for policy 0, policy_version 22371 (0.0009) -[2023-10-14 14:20:08,133][75949] Updated weights for policy 0, policy_version 22381 (0.0008) -[2023-10-14 14:20:08,164][74987] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 45809664. Throughput: 0: 1682.6, 1: 1681.3. Samples: 11460620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:20:08,164][74987] Avg episode reward: [(0, '20.510'), (1, '21.320')] -[2023-10-14 14:20:08,499][75949] Updated weights for policy 0, policy_version 22391 (0.0009) -[2023-10-14 14:20:11,802][75950] Updated weights for policy 1, policy_version 22370 (0.0007) -[2023-10-14 14:20:12,170][75950] Updated weights for policy 1, policy_version 22380 (0.0011) -[2023-10-14 14:20:12,529][75949] Updated weights for policy 0, policy_version 22401 (0.0007) -[2023-10-14 14:20:12,538][75950] Updated weights for policy 1, policy_version 22390 (0.0010) -[2023-10-14 14:20:12,911][75950] Updated weights for policy 1, policy_version 22400 (0.0007) -[2023-10-14 14:20:12,949][75949] Updated weights for policy 0, policy_version 22411 (0.0007) -[2023-10-14 14:20:13,164][74987] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 45875200. Throughput: 0: 1682.6, 1: 1656.8. Samples: 11480130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:20:13,165][74987] Avg episode reward: [(0, '21.500'), (1, '21.080')] -[2023-10-14 14:20:13,330][75949] Updated weights for policy 0, policy_version 22421 (0.0011) -[2023-10-14 14:20:13,691][75949] Updated weights for policy 0, policy_version 22431 (0.0007) -[2023-10-14 14:20:16,977][75950] Updated weights for policy 1, policy_version 22410 (0.0009) -[2023-10-14 14:20:17,335][75950] Updated weights for policy 1, policy_version 22420 (0.0008) -[2023-10-14 14:20:17,707][75950] Updated weights for policy 1, policy_version 22430 (0.0009) -[2023-10-14 14:20:17,734][75949] Updated weights for policy 0, policy_version 22441 (0.0009) -[2023-10-14 14:20:18,113][75949] Updated weights for policy 0, policy_version 22451 (0.0010) -[2023-10-14 14:20:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 45940736. Throughput: 0: 1681.6, 1: 1679.6. Samples: 11490250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:20:18,164][74987] Avg episode reward: [(0, '21.060'), (1, '20.550')] -[2023-10-14 14:20:18,482][75949] Updated weights for policy 0, policy_version 22461 (0.0008) -[2023-10-14 14:20:21,836][75950] Updated weights for policy 1, policy_version 22440 (0.0007) -[2023-10-14 14:20:22,201][75950] Updated weights for policy 1, policy_version 22450 (0.0007) -[2023-10-14 14:20:22,408][75949] Updated weights for policy 0, policy_version 22471 (0.0008) -[2023-10-14 14:20:22,570][75950] Updated weights for policy 1, policy_version 22460 (0.0007) -[2023-10-14 14:20:22,781][75949] Updated weights for policy 0, policy_version 22481 (0.0008) -[2023-10-14 14:20:23,154][75949] Updated weights for policy 0, policy_version 22491 (0.0010) -[2023-10-14 14:20:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 46006272. Throughput: 0: 1680.9, 1: 1676.0. Samples: 11510834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:20:23,165][74987] Avg episode reward: [(0, '21.920'), (1, '21.560')] -[2023-10-14 14:20:26,727][75950] Updated weights for policy 1, policy_version 22470 (0.0008) -[2023-10-14 14:20:27,112][75950] Updated weights for policy 1, policy_version 22480 (0.0010) -[2023-10-14 14:20:27,297][75949] Updated weights for policy 0, policy_version 22501 (0.0009) -[2023-10-14 14:20:27,481][75950] Updated weights for policy 1, policy_version 22490 (0.0007) -[2023-10-14 14:20:27,667][75949] Updated weights for policy 0, policy_version 22511 (0.0008) -[2023-10-14 14:20:28,038][75949] Updated weights for policy 0, policy_version 22521 (0.0007) -[2023-10-14 14:20:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 46071808. Throughput: 0: 1672.1, 1: 1651.7. Samples: 11529666. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:20:28,164][74987] Avg episode reward: [(0, '21.050'), (1, '20.310')] -[2023-10-14 14:20:31,521][75950] Updated weights for policy 1, policy_version 22500 (0.0009) -[2023-10-14 14:20:31,880][75950] Updated weights for policy 1, policy_version 22510 (0.0008) -[2023-10-14 14:20:32,054][75949] Updated weights for policy 0, policy_version 22531 (0.0009) -[2023-10-14 14:20:32,249][75950] Updated weights for policy 1, policy_version 22520 (0.0008) -[2023-10-14 14:20:32,420][75949] Updated weights for policy 0, policy_version 22541 (0.0010) -[2023-10-14 14:20:32,787][75949] Updated weights for policy 0, policy_version 22551 (0.0010) -[2023-10-14 14:20:33,163][74987] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 46170112. Throughput: 0: 1682.0, 1: 1672.4. Samples: 11540316. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:20:33,164][74987] Avg episode reward: [(0, '21.400'), (1, '20.710')] -[2023-10-14 14:20:36,395][75950] Updated weights for policy 1, policy_version 22530 (0.0009) -[2023-10-14 14:20:36,770][75950] Updated weights for policy 1, policy_version 22540 (0.0009) -[2023-10-14 14:20:36,978][75949] Updated weights for policy 0, policy_version 22561 (0.0008) -[2023-10-14 14:20:37,132][75950] Updated weights for policy 1, policy_version 22550 (0.0010) -[2023-10-14 14:20:37,345][75949] Updated weights for policy 0, policy_version 22571 (0.0007) -[2023-10-14 14:20:37,500][75950] Updated weights for policy 1, policy_version 22560 (0.0008) -[2023-10-14 14:20:37,714][75949] Updated weights for policy 0, policy_version 22581 (0.0007) -[2023-10-14 14:20:38,089][75949] Updated weights for policy 0, policy_version 22591 (0.0008) -[2023-10-14 14:20:38,164][74987] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 46235648. Throughput: 0: 1683.0, 1: 1662.0. Samples: 11560374. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:20:38,165][74987] Avg episode reward: [(0, '19.960'), (1, '19.710')] -[2023-10-14 14:20:41,558][75950] Updated weights for policy 1, policy_version 22570 (0.0008) -[2023-10-14 14:20:41,921][75950] Updated weights for policy 1, policy_version 22580 (0.0008) -[2023-10-14 14:20:42,188][75949] Updated weights for policy 0, policy_version 22601 (0.0008) -[2023-10-14 14:20:42,301][75950] Updated weights for policy 1, policy_version 22590 (0.0008) -[2023-10-14 14:20:42,559][75949] Updated weights for policy 0, policy_version 22611 (0.0007) -[2023-10-14 14:20:42,929][75949] Updated weights for policy 0, policy_version 22621 (0.0007) -[2023-10-14 14:20:43,163][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 46301184. Throughput: 0: 1658.1, 1: 1655.2. Samples: 11579212. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-14 14:20:43,164][74987] Avg episode reward: [(0, '22.670'), (1, '22.620')] -[2023-10-14 14:20:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000022624_23166976.pth... -[2023-10-14 14:20:43,172][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000022592_23134208.pth... -[2023-10-14 14:20:43,207][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000021024_21528576.pth -[2023-10-14 14:20:43,212][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000021056_21561344.pth -[2023-10-14 14:20:43,215][75615] Saving new best policy, reward=22.670! -[2023-10-14 14:20:46,546][75950] Updated weights for policy 1, policy_version 22600 (0.0008) -[2023-10-14 14:20:46,909][75950] Updated weights for policy 1, policy_version 22610 (0.0008) -[2023-10-14 14:20:47,102][75949] Updated weights for policy 0, policy_version 22631 (0.0009) -[2023-10-14 14:20:47,280][75950] Updated weights for policy 1, policy_version 22620 (0.0008) -[2023-10-14 14:20:47,479][75949] Updated weights for policy 0, policy_version 22641 (0.0010) -[2023-10-14 14:20:47,840][75949] Updated weights for policy 0, policy_version 22651 (0.0011) -[2023-10-14 14:20:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 46366720. Throughput: 0: 1676.2, 1: 1666.4. Samples: 11590254. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-14 14:20:48,165][74987] Avg episode reward: [(0, '20.080'), (1, '22.190')] -[2023-10-14 14:20:51,358][75950] Updated weights for policy 1, policy_version 22630 (0.0010) -[2023-10-14 14:20:51,730][75950] Updated weights for policy 1, policy_version 22640 (0.0010) -[2023-10-14 14:20:52,079][75949] Updated weights for policy 0, policy_version 22661 (0.0009) -[2023-10-14 14:20:52,104][75950] Updated weights for policy 1, policy_version 22650 (0.0009) -[2023-10-14 14:20:52,448][75949] Updated weights for policy 0, policy_version 22671 (0.0008) -[2023-10-14 14:20:52,816][75949] Updated weights for policy 0, policy_version 22681 (0.0008) -[2023-10-14 14:20:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 46432256. Throughput: 0: 1669.8, 1: 1656.2. Samples: 11610290. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-14 14:20:53,164][74987] Avg episode reward: [(0, '20.710'), (1, '22.080')] -[2023-10-14 14:20:56,212][75950] Updated weights for policy 1, policy_version 22660 (0.0009) -[2023-10-14 14:20:56,570][75950] Updated weights for policy 1, policy_version 22670 (0.0009) -[2023-10-14 14:20:56,945][75950] Updated weights for policy 1, policy_version 22680 (0.0009) -[2023-10-14 14:20:56,954][75949] Updated weights for policy 0, policy_version 22691 (0.0008) -[2023-10-14 14:20:57,331][75949] Updated weights for policy 0, policy_version 22701 (0.0007) -[2023-10-14 14:20:57,698][75949] Updated weights for policy 0, policy_version 22711 (0.0011) -[2023-10-14 14:20:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 46497792. Throughput: 0: 1653.3, 1: 1662.6. Samples: 11629344. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) -[2023-10-14 14:20:58,164][74987] Avg episode reward: [(0, '21.910'), (1, '22.190')] -[2023-10-14 14:21:00,915][75950] Updated weights for policy 1, policy_version 22690 (0.0008) -[2023-10-14 14:21:01,284][75950] Updated weights for policy 1, policy_version 22700 (0.0008) -[2023-10-14 14:21:01,652][75950] Updated weights for policy 1, policy_version 22710 (0.0009) -[2023-10-14 14:21:01,694][75949] Updated weights for policy 0, policy_version 22721 (0.0008) -[2023-10-14 14:21:02,025][75950] Updated weights for policy 1, policy_version 22720 (0.0007) -[2023-10-14 14:21:02,072][75949] Updated weights for policy 0, policy_version 22731 (0.0011) -[2023-10-14 14:21:02,455][75949] Updated weights for policy 0, policy_version 22741 (0.0010) -[2023-10-14 14:21:02,812][75949] Updated weights for policy 0, policy_version 22751 (0.0010) -[2023-10-14 14:21:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 46563328. Throughput: 0: 1672.6, 1: 1664.8. Samples: 11640432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:21:03,165][74987] Avg episode reward: [(0, '19.750'), (1, '21.870')] -[2023-10-14 14:21:06,174][75950] Updated weights for policy 1, policy_version 22730 (0.0007) -[2023-10-14 14:21:06,539][75950] Updated weights for policy 1, policy_version 22740 (0.0008) -[2023-10-14 14:21:06,881][75949] Updated weights for policy 0, policy_version 22761 (0.0009) -[2023-10-14 14:21:06,902][75950] Updated weights for policy 1, policy_version 22750 (0.0009) -[2023-10-14 14:21:07,247][75949] Updated weights for policy 0, policy_version 22771 (0.0009) -[2023-10-14 14:21:07,616][75949] Updated weights for policy 0, policy_version 22781 (0.0008) -[2023-10-14 14:21:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 46628864. Throughput: 0: 1666.2, 1: 1649.4. Samples: 11660038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:21:08,164][74987] Avg episode reward: [(0, '22.150'), (1, '21.780')] -[2023-10-14 14:21:10,979][75950] Updated weights for policy 1, policy_version 22760 (0.0008) -[2023-10-14 14:21:11,351][75950] Updated weights for policy 1, policy_version 22770 (0.0010) -[2023-10-14 14:21:11,712][75950] Updated weights for policy 1, policy_version 22780 (0.0008) -[2023-10-14 14:21:11,792][75949] Updated weights for policy 0, policy_version 22791 (0.0008) -[2023-10-14 14:21:12,173][75949] Updated weights for policy 0, policy_version 22801 (0.0008) -[2023-10-14 14:21:12,542][75949] Updated weights for policy 0, policy_version 22811 (0.0008) -[2023-10-14 14:21:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 46694400. Throughput: 0: 1656.2, 1: 1668.9. Samples: 11679298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:21:13,165][74987] Avg episode reward: [(0, '19.160'), (1, '20.940')] -[2023-10-14 14:21:15,980][75950] Updated weights for policy 1, policy_version 22790 (0.0008) -[2023-10-14 14:21:16,355][75950] Updated weights for policy 1, policy_version 22800 (0.0008) -[2023-10-14 14:21:16,652][75949] Updated weights for policy 0, policy_version 22821 (0.0009) -[2023-10-14 14:21:16,728][75950] Updated weights for policy 1, policy_version 22810 (0.0007) -[2023-10-14 14:21:17,018][75949] Updated weights for policy 0, policy_version 22831 (0.0008) -[2023-10-14 14:21:17,387][75949] Updated weights for policy 0, policy_version 22841 (0.0009) -[2023-10-14 14:21:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 46759936. Throughput: 0: 1668.8, 1: 1674.4. Samples: 11690760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:21:18,164][74987] Avg episode reward: [(0, '22.170'), (1, '21.320')] -[2023-10-14 14:21:20,783][75950] Updated weights for policy 1, policy_version 22820 (0.0009) -[2023-10-14 14:21:21,154][75950] Updated weights for policy 1, policy_version 22830 (0.0010) -[2023-10-14 14:21:21,482][75949] Updated weights for policy 0, policy_version 22851 (0.0008) -[2023-10-14 14:21:21,517][75950] Updated weights for policy 1, policy_version 22840 (0.0010) -[2023-10-14 14:21:21,855][75949] Updated weights for policy 0, policy_version 22861 (0.0009) -[2023-10-14 14:21:22,226][75949] Updated weights for policy 0, policy_version 22871 (0.0008) -[2023-10-14 14:21:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 46825472. Throughput: 0: 1666.8, 1: 1658.4. Samples: 11710010. Policy #0 lag: (min: 20.0, avg: 45.3, max: 48.0) -[2023-10-14 14:21:23,165][74987] Avg episode reward: [(0, '20.400'), (1, '21.460')] -[2023-10-14 14:21:25,628][75950] Updated weights for policy 1, policy_version 22850 (0.0009) -[2023-10-14 14:21:25,990][75950] Updated weights for policy 1, policy_version 22860 (0.0008) -[2023-10-14 14:21:26,268][75949] Updated weights for policy 0, policy_version 22881 (0.0008) -[2023-10-14 14:21:26,363][75950] Updated weights for policy 1, policy_version 22870 (0.0009) -[2023-10-14 14:21:26,632][75949] Updated weights for policy 0, policy_version 22891 (0.0009) -[2023-10-14 14:21:26,728][75950] Updated weights for policy 1, policy_version 22880 (0.0009) -[2023-10-14 14:21:27,003][75949] Updated weights for policy 0, policy_version 22901 (0.0007) -[2023-10-14 14:21:27,368][75949] Updated weights for policy 0, policy_version 22911 (0.0009) -[2023-10-14 14:21:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 46891008. Throughput: 0: 1667.5, 1: 1673.2. Samples: 11729544. Policy #0 lag: (min: 20.0, avg: 45.3, max: 48.0) -[2023-10-14 14:21:28,164][74987] Avg episode reward: [(0, '23.280'), (1, '21.280')] -[2023-10-14 14:21:28,175][75615] Saving new best policy, reward=23.280! -[2023-10-14 14:21:30,677][75950] Updated weights for policy 1, policy_version 22890 (0.0008) -[2023-10-14 14:21:31,045][75950] Updated weights for policy 1, policy_version 22900 (0.0009) -[2023-10-14 14:21:31,364][75949] Updated weights for policy 0, policy_version 22921 (0.0009) -[2023-10-14 14:21:31,406][75950] Updated weights for policy 1, policy_version 22910 (0.0007) -[2023-10-14 14:21:31,739][75949] Updated weights for policy 0, policy_version 22931 (0.0009) -[2023-10-14 14:21:32,100][75949] Updated weights for policy 0, policy_version 22941 (0.0007) -[2023-10-14 14:21:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 46956544. Throughput: 0: 1679.7, 1: 1668.0. Samples: 11740900. Policy #0 lag: (min: 20.0, avg: 45.3, max: 48.0) -[2023-10-14 14:21:33,164][74987] Avg episode reward: [(0, '21.560'), (1, '21.180')] -[2023-10-14 14:21:35,571][75950] Updated weights for policy 1, policy_version 22920 (0.0007) -[2023-10-14 14:21:35,938][75950] Updated weights for policy 1, policy_version 22930 (0.0008) -[2023-10-14 14:21:36,033][75949] Updated weights for policy 0, policy_version 22951 (0.0008) -[2023-10-14 14:21:36,292][75950] Updated weights for policy 1, policy_version 22940 (0.0009) -[2023-10-14 14:21:36,401][75949] Updated weights for policy 0, policy_version 22961 (0.0008) -[2023-10-14 14:21:36,782][75949] Updated weights for policy 0, policy_version 22971 (0.0012) -[2023-10-14 14:21:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 47022080. Throughput: 0: 1665.9, 1: 1659.4. Samples: 11759928. Policy #0 lag: (min: 20.0, avg: 45.3, max: 48.0) -[2023-10-14 14:21:38,165][74987] Avg episode reward: [(0, '21.750'), (1, '23.070')] -[2023-10-14 14:21:40,397][75950] Updated weights for policy 1, policy_version 22950 (0.0009) -[2023-10-14 14:21:40,765][75950] Updated weights for policy 1, policy_version 22960 (0.0009) -[2023-10-14 14:21:40,833][75949] Updated weights for policy 0, policy_version 22981 (0.0009) -[2023-10-14 14:21:41,136][75950] Updated weights for policy 1, policy_version 22970 (0.0008) -[2023-10-14 14:21:41,206][75949] Updated weights for policy 0, policy_version 22991 (0.0010) -[2023-10-14 14:21:41,570][75949] Updated weights for policy 0, policy_version 23001 (0.0010) -[2023-10-14 14:21:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 47087616. Throughput: 0: 1679.8, 1: 1680.7. Samples: 11780568. Policy #0 lag: (min: 6.0, avg: 19.3, max: 38.0) -[2023-10-14 14:21:43,165][74987] Avg episode reward: [(0, '23.280'), (1, '21.280')] -[2023-10-14 14:21:45,233][75950] Updated weights for policy 1, policy_version 22980 (0.0009) -[2023-10-14 14:21:45,592][75950] Updated weights for policy 1, policy_version 22990 (0.0007) -[2023-10-14 14:21:45,596][75949] Updated weights for policy 0, policy_version 23011 (0.0010) -[2023-10-14 14:21:45,965][75949] Updated weights for policy 0, policy_version 23021 (0.0007) -[2023-10-14 14:21:45,967][75950] Updated weights for policy 1, policy_version 23000 (0.0007) -[2023-10-14 14:21:46,334][75949] Updated weights for policy 0, policy_version 23031 (0.0009) -[2023-10-14 14:21:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47153152. Throughput: 0: 1685.8, 1: 1668.5. Samples: 11791376. Policy #0 lag: (min: 6.0, avg: 19.3, max: 38.0) -[2023-10-14 14:21:48,164][74987] Avg episode reward: [(0, '20.680'), (1, '21.730')] -[2023-10-14 14:21:50,171][75950] Updated weights for policy 1, policy_version 23010 (0.0010) -[2023-10-14 14:21:50,313][75949] Updated weights for policy 0, policy_version 23041 (0.0008) -[2023-10-14 14:21:50,533][75950] Updated weights for policy 1, policy_version 23020 (0.0009) -[2023-10-14 14:21:50,679][75949] Updated weights for policy 0, policy_version 23051 (0.0009) -[2023-10-14 14:21:50,898][75950] Updated weights for policy 1, policy_version 23030 (0.0009) -[2023-10-14 14:21:51,054][75949] Updated weights for policy 0, policy_version 23061 (0.0009) -[2023-10-14 14:21:51,265][75950] Updated weights for policy 1, policy_version 23040 (0.0009) -[2023-10-14 14:21:51,424][75949] Updated weights for policy 0, policy_version 23071 (0.0009) -[2023-10-14 14:21:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47218688. Throughput: 0: 1665.6, 1: 1667.4. Samples: 11810024. Policy #0 lag: (min: 6.0, avg: 19.3, max: 38.0) -[2023-10-14 14:21:53,164][74987] Avg episode reward: [(0, '22.370'), (1, '21.060')] -[2023-10-14 14:21:55,387][75950] Updated weights for policy 1, policy_version 23050 (0.0007) -[2023-10-14 14:21:55,732][75949] Updated weights for policy 0, policy_version 23081 (0.0008) -[2023-10-14 14:21:55,749][75950] Updated weights for policy 1, policy_version 23060 (0.0010) -[2023-10-14 14:21:56,111][75949] Updated weights for policy 0, policy_version 23091 (0.0008) -[2023-10-14 14:21:56,111][75950] Updated weights for policy 1, policy_version 23070 (0.0009) -[2023-10-14 14:21:56,480][75949] Updated weights for policy 0, policy_version 23101 (0.0010) -[2023-10-14 14:21:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47284224. Throughput: 0: 1683.1, 1: 1671.6. Samples: 11830256. Policy #0 lag: (min: 6.0, avg: 19.3, max: 38.0) -[2023-10-14 14:21:58,164][74987] Avg episode reward: [(0, '19.900'), (1, '22.040')] -[2023-10-14 14:22:00,172][75950] Updated weights for policy 1, policy_version 23080 (0.0007) -[2023-10-14 14:22:00,448][75949] Updated weights for policy 0, policy_version 23111 (0.0008) -[2023-10-14 14:22:00,552][75950] Updated weights for policy 1, policy_version 23090 (0.0008) -[2023-10-14 14:22:00,828][75949] Updated weights for policy 0, policy_version 23121 (0.0008) -[2023-10-14 14:22:00,912][75950] Updated weights for policy 1, policy_version 23100 (0.0010) -[2023-10-14 14:22:01,195][75949] Updated weights for policy 0, policy_version 23131 (0.0010) -[2023-10-14 14:22:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 47349760. Throughput: 0: 1681.7, 1: 1650.0. Samples: 11840688. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-14 14:22:03,164][74987] Avg episode reward: [(0, '22.390'), (1, '22.510')] -[2023-10-14 14:22:04,863][75950] Updated weights for policy 1, policy_version 23110 (0.0011) -[2023-10-14 14:22:05,235][75950] Updated weights for policy 1, policy_version 23120 (0.0007) -[2023-10-14 14:22:05,235][75949] Updated weights for policy 0, policy_version 23141 (0.0010) -[2023-10-14 14:22:05,603][75949] Updated weights for policy 0, policy_version 23151 (0.0008) -[2023-10-14 14:22:05,606][75950] Updated weights for policy 1, policy_version 23130 (0.0007) -[2023-10-14 14:22:05,979][75949] Updated weights for policy 0, policy_version 23161 (0.0009) -[2023-10-14 14:22:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 47415296. Throughput: 0: 1669.6, 1: 1669.5. Samples: 11860270. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-14 14:22:08,165][74987] Avg episode reward: [(0, '20.920'), (1, '22.770')] -[2023-10-14 14:22:09,619][75950] Updated weights for policy 1, policy_version 23140 (0.0008) -[2023-10-14 14:22:09,988][75950] Updated weights for policy 1, policy_version 23150 (0.0010) -[2023-10-14 14:22:10,080][75949] Updated weights for policy 0, policy_version 23171 (0.0008) -[2023-10-14 14:22:10,349][75950] Updated weights for policy 1, policy_version 23160 (0.0010) -[2023-10-14 14:22:10,448][75949] Updated weights for policy 0, policy_version 23181 (0.0008) -[2023-10-14 14:22:10,812][75949] Updated weights for policy 0, policy_version 23191 (0.0008) -[2023-10-14 14:22:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47480832. Throughput: 0: 1688.2, 1: 1679.5. Samples: 11881088. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-14 14:22:13,164][74987] Avg episode reward: [(0, '21.410'), (1, '21.920')] -[2023-10-14 14:22:14,512][75950] Updated weights for policy 1, policy_version 23170 (0.0011) -[2023-10-14 14:22:14,879][75950] Updated weights for policy 1, policy_version 23180 (0.0010) -[2023-10-14 14:22:15,055][75949] Updated weights for policy 0, policy_version 23201 (0.0009) -[2023-10-14 14:22:15,241][75950] Updated weights for policy 1, policy_version 23190 (0.0008) -[2023-10-14 14:22:15,418][75949] Updated weights for policy 0, policy_version 23211 (0.0007) -[2023-10-14 14:22:15,607][75950] Updated weights for policy 1, policy_version 23200 (0.0007) -[2023-10-14 14:22:15,794][75949] Updated weights for policy 0, policy_version 23221 (0.0008) -[2023-10-14 14:22:16,157][75949] Updated weights for policy 0, policy_version 23231 (0.0007) -[2023-10-14 14:22:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47546368. Throughput: 0: 1668.1, 1: 1659.2. Samples: 11890632. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) -[2023-10-14 14:22:18,164][74987] Avg episode reward: [(0, '18.080'), (1, '22.970')] -[2023-10-14 14:22:19,650][75950] Updated weights for policy 1, policy_version 23210 (0.0009) -[2023-10-14 14:22:20,006][75950] Updated weights for policy 1, policy_version 23220 (0.0007) -[2023-10-14 14:22:20,177][75949] Updated weights for policy 0, policy_version 23241 (0.0007) -[2023-10-14 14:22:20,379][75950] Updated weights for policy 1, policy_version 23230 (0.0008) -[2023-10-14 14:22:20,537][75949] Updated weights for policy 0, policy_version 23251 (0.0008) -[2023-10-14 14:22:20,907][75949] Updated weights for policy 0, policy_version 23261 (0.0007) -[2023-10-14 14:22:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47611904. Throughput: 0: 1674.1, 1: 1680.7. Samples: 11910892. Policy #0 lag: (min: 0.0, avg: 26.9, max: 32.0) -[2023-10-14 14:22:23,165][74987] Avg episode reward: [(0, '19.280'), (1, '20.310')] -[2023-10-14 14:22:24,514][75950] Updated weights for policy 1, policy_version 23240 (0.0007) -[2023-10-14 14:22:24,818][75949] Updated weights for policy 0, policy_version 23271 (0.0010) -[2023-10-14 14:22:24,878][75950] Updated weights for policy 1, policy_version 23250 (0.0009) -[2023-10-14 14:22:25,191][75949] Updated weights for policy 0, policy_version 23281 (0.0007) -[2023-10-14 14:22:25,241][75950] Updated weights for policy 1, policy_version 23260 (0.0009) -[2023-10-14 14:22:25,561][75949] Updated weights for policy 0, policy_version 23291 (0.0007) -[2023-10-14 14:22:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 47677440. Throughput: 0: 1684.7, 1: 1677.9. Samples: 11931884. Policy #0 lag: (min: 0.0, avg: 26.9, max: 32.0) -[2023-10-14 14:22:28,165][74987] Avg episode reward: [(0, '17.280'), (1, '21.970')] -[2023-10-14 14:22:29,403][75950] Updated weights for policy 1, policy_version 23270 (0.0007) -[2023-10-14 14:22:29,623][75949] Updated weights for policy 0, policy_version 23301 (0.0009) -[2023-10-14 14:22:29,768][75950] Updated weights for policy 1, policy_version 23280 (0.0008) -[2023-10-14 14:22:29,985][75949] Updated weights for policy 0, policy_version 23311 (0.0007) -[2023-10-14 14:22:30,127][75950] Updated weights for policy 1, policy_version 23290 (0.0008) -[2023-10-14 14:22:30,353][75949] Updated weights for policy 0, policy_version 23321 (0.0007) -[2023-10-14 14:22:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 47742976. Throughput: 0: 1657.4, 1: 1664.5. Samples: 11940860. Policy #0 lag: (min: 0.0, avg: 26.9, max: 32.0) -[2023-10-14 14:22:33,165][74987] Avg episode reward: [(0, '20.170'), (1, '20.020')] -[2023-10-14 14:22:34,130][75950] Updated weights for policy 1, policy_version 23300 (0.0010) -[2023-10-14 14:22:34,361][75949] Updated weights for policy 0, policy_version 23331 (0.0008) -[2023-10-14 14:22:34,492][75950] Updated weights for policy 1, policy_version 23310 (0.0008) -[2023-10-14 14:22:34,726][75949] Updated weights for policy 0, policy_version 23341 (0.0008) -[2023-10-14 14:22:34,850][75950] Updated weights for policy 1, policy_version 23320 (0.0010) -[2023-10-14 14:22:35,096][75949] Updated weights for policy 0, policy_version 23351 (0.0008) -[2023-10-14 14:22:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47808512. Throughput: 0: 1684.6, 1: 1685.4. Samples: 11961674. Policy #0 lag: (min: 0.0, avg: 26.9, max: 32.0) -[2023-10-14 14:22:38,165][74987] Avg episode reward: [(0, '18.750'), (1, '21.330')] -[2023-10-14 14:22:38,839][75950] Updated weights for policy 1, policy_version 23330 (0.0008) -[2023-10-14 14:22:39,212][75950] Updated weights for policy 1, policy_version 23340 (0.0008) -[2023-10-14 14:22:39,249][75949] Updated weights for policy 0, policy_version 23361 (0.0007) -[2023-10-14 14:22:39,576][75950] Updated weights for policy 1, policy_version 23350 (0.0009) -[2023-10-14 14:22:39,616][75949] Updated weights for policy 0, policy_version 23371 (0.0008) -[2023-10-14 14:22:39,945][75950] Updated weights for policy 1, policy_version 23360 (0.0007) -[2023-10-14 14:22:39,986][75949] Updated weights for policy 0, policy_version 23381 (0.0009) -[2023-10-14 14:22:40,352][75949] Updated weights for policy 0, policy_version 23391 (0.0007) -[2023-10-14 14:22:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47874048. Throughput: 0: 1694.4, 1: 1689.1. Samples: 11982518. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 14:22:43,164][74987] Avg episode reward: [(0, '20.710'), (1, '21.150')] -[2023-10-14 14:22:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000023392_23953408.pth... -[2023-10-14 14:22:43,172][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000023360_23920640.pth... -[2023-10-14 14:22:43,204][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000021792_22315008.pth -[2023-10-14 14:22:43,208][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000023360_23920640.pth -[2023-10-14 14:22:43,212][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000021824_22347776.pth -[2023-10-14 14:22:43,216][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000023392_23953408.pth -[2023-10-14 14:22:44,027][75950] Updated weights for policy 1, policy_version 23370 (0.0008) -[2023-10-14 14:22:44,382][75949] Updated weights for policy 0, policy_version 23401 (0.0009) -[2023-10-14 14:22:44,390][75950] Updated weights for policy 1, policy_version 23380 (0.0008) -[2023-10-14 14:22:44,749][75949] Updated weights for policy 0, policy_version 23411 (0.0009) -[2023-10-14 14:22:44,763][75950] Updated weights for policy 1, policy_version 23390 (0.0010) -[2023-10-14 14:22:45,113][75949] Updated weights for policy 0, policy_version 23421 (0.0008) -[2023-10-14 14:22:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 47939584. Throughput: 0: 1672.6, 1: 1679.3. Samples: 11991524. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 14:22:48,165][74987] Avg episode reward: [(0, '20.350'), (1, '22.130')] -[2023-10-14 14:22:49,076][75949] Updated weights for policy 0, policy_version 23431 (0.0007) -[2023-10-14 14:22:49,184][75950] Updated weights for policy 1, policy_version 23400 (0.0007) -[2023-10-14 14:22:49,450][75949] Updated weights for policy 0, policy_version 23441 (0.0009) -[2023-10-14 14:22:49,560][75950] Updated weights for policy 1, policy_version 23410 (0.0008) -[2023-10-14 14:22:49,810][75949] Updated weights for policy 0, policy_version 23451 (0.0009) -[2023-10-14 14:22:49,917][75950] Updated weights for policy 1, policy_version 23420 (0.0008) -[2023-10-14 14:22:53,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48005120. Throughput: 0: 1689.9, 1: 1678.1. Samples: 12011832. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 14:22:53,164][74987] Avg episode reward: [(0, '21.600'), (1, '22.850')] -[2023-10-14 14:22:53,856][75950] Updated weights for policy 1, policy_version 23430 (0.0008) -[2023-10-14 14:22:53,931][75949] Updated weights for policy 0, policy_version 23461 (0.0008) -[2023-10-14 14:22:54,222][75950] Updated weights for policy 1, policy_version 23440 (0.0009) -[2023-10-14 14:22:54,303][75949] Updated weights for policy 0, policy_version 23471 (0.0007) -[2023-10-14 14:22:54,585][75950] Updated weights for policy 1, policy_version 23450 (0.0007) -[2023-10-14 14:22:54,673][75949] Updated weights for policy 0, policy_version 23481 (0.0007) -[2023-10-14 14:22:58,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48070656. Throughput: 0: 1693.9, 1: 1673.5. Samples: 12032618. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 14:22:58,164][74987] Avg episode reward: [(0, '20.820'), (1, '21.500')] -[2023-10-14 14:22:58,660][75950] Updated weights for policy 1, policy_version 23460 (0.0008) -[2023-10-14 14:22:58,672][75949] Updated weights for policy 0, policy_version 23491 (0.0009) -[2023-10-14 14:22:59,025][75950] Updated weights for policy 1, policy_version 23470 (0.0008) -[2023-10-14 14:22:59,043][75949] Updated weights for policy 0, policy_version 23501 (0.0008) -[2023-10-14 14:22:59,403][75950] Updated weights for policy 1, policy_version 23480 (0.0009) -[2023-10-14 14:22:59,411][75949] Updated weights for policy 0, policy_version 23511 (0.0009) -[2023-10-14 14:23:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48136192. Throughput: 0: 1682.3, 1: 1672.6. Samples: 12041600. Policy #0 lag: (min: 8.0, avg: 30.3, max: 40.0) -[2023-10-14 14:23:03,164][74987] Avg episode reward: [(0, '19.930'), (1, '22.030')] -[2023-10-14 14:23:03,458][75949] Updated weights for policy 0, policy_version 23521 (0.0009) -[2023-10-14 14:23:03,572][75950] Updated weights for policy 1, policy_version 23490 (0.0009) -[2023-10-14 14:23:03,831][75949] Updated weights for policy 0, policy_version 23531 (0.0008) -[2023-10-14 14:23:03,945][75950] Updated weights for policy 1, policy_version 23500 (0.0009) -[2023-10-14 14:23:04,209][75949] Updated weights for policy 0, policy_version 23541 (0.0008) -[2023-10-14 14:23:04,309][75950] Updated weights for policy 1, policy_version 23510 (0.0008) -[2023-10-14 14:23:04,580][75949] Updated weights for policy 0, policy_version 23551 (0.0007) -[2023-10-14 14:23:04,675][75950] Updated weights for policy 1, policy_version 23520 (0.0008) -[2023-10-14 14:23:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 48201728. Throughput: 0: 1690.4, 1: 1668.7. Samples: 12062048. Policy #0 lag: (min: 8.0, avg: 30.3, max: 40.0) -[2023-10-14 14:23:08,164][74987] Avg episode reward: [(0, '22.080'), (1, '20.660')] -[2023-10-14 14:23:08,749][75949] Updated weights for policy 0, policy_version 23561 (0.0007) -[2023-10-14 14:23:08,847][75950] Updated weights for policy 1, policy_version 23530 (0.0010) -[2023-10-14 14:23:09,116][75949] Updated weights for policy 0, policy_version 23571 (0.0008) -[2023-10-14 14:23:09,215][75950] Updated weights for policy 1, policy_version 23540 (0.0008) -[2023-10-14 14:23:09,490][75949] Updated weights for policy 0, policy_version 23581 (0.0007) -[2023-10-14 14:23:09,583][75950] Updated weights for policy 1, policy_version 23550 (0.0007) -[2023-10-14 14:23:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48267264. Throughput: 0: 1682.2, 1: 1663.4. Samples: 12082438. Policy #0 lag: (min: 8.0, avg: 30.3, max: 40.0) -[2023-10-14 14:23:13,165][74987] Avg episode reward: [(0, '19.620'), (1, '20.280')] -[2023-10-14 14:23:13,493][75949] Updated weights for policy 0, policy_version 23591 (0.0009) -[2023-10-14 14:23:13,577][75950] Updated weights for policy 1, policy_version 23560 (0.0009) -[2023-10-14 14:23:13,868][75949] Updated weights for policy 0, policy_version 23601 (0.0008) -[2023-10-14 14:23:13,937][75950] Updated weights for policy 1, policy_version 23570 (0.0008) -[2023-10-14 14:23:14,230][75949] Updated weights for policy 0, policy_version 23611 (0.0007) -[2023-10-14 14:23:14,297][75950] Updated weights for policy 1, policy_version 23580 (0.0010) -[2023-10-14 14:23:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48332800. Throughput: 0: 1683.5, 1: 1663.9. Samples: 12091492. Policy #0 lag: (min: 8.0, avg: 30.3, max: 40.0) -[2023-10-14 14:23:18,164][74987] Avg episode reward: [(0, '23.040'), (1, '21.870')] -[2023-10-14 14:23:18,497][75949] Updated weights for policy 0, policy_version 23621 (0.0008) -[2023-10-14 14:23:18,550][75950] Updated weights for policy 1, policy_version 23590 (0.0008) -[2023-10-14 14:23:18,865][75949] Updated weights for policy 0, policy_version 23631 (0.0008) -[2023-10-14 14:23:18,908][75950] Updated weights for policy 1, policy_version 23600 (0.0009) -[2023-10-14 14:23:19,229][75949] Updated weights for policy 0, policy_version 23641 (0.0008) -[2023-10-14 14:23:19,283][75950] Updated weights for policy 1, policy_version 23610 (0.0008) -[2023-10-14 14:23:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 48398336. Throughput: 0: 1682.7, 1: 1658.9. Samples: 12112044. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-14 14:23:23,164][74987] Avg episode reward: [(0, '20.070'), (1, '21.500')] -[2023-10-14 14:23:23,255][75949] Updated weights for policy 0, policy_version 23651 (0.0009) -[2023-10-14 14:23:23,298][75950] Updated weights for policy 1, policy_version 23620 (0.0007) -[2023-10-14 14:23:23,622][75949] Updated weights for policy 0, policy_version 23661 (0.0008) -[2023-10-14 14:23:23,666][75950] Updated weights for policy 1, policy_version 23630 (0.0009) -[2023-10-14 14:23:23,999][75949] Updated weights for policy 0, policy_version 23671 (0.0008) -[2023-10-14 14:23:24,031][75950] Updated weights for policy 1, policy_version 23640 (0.0007) -[2023-10-14 14:23:28,107][75949] Updated weights for policy 0, policy_version 23681 (0.0008) -[2023-10-14 14:23:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48463872. Throughput: 0: 1672.6, 1: 1657.4. Samples: 12132368. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-14 14:23:28,164][75950] Updated weights for policy 1, policy_version 23650 (0.0009) -[2023-10-14 14:23:28,164][74987] Avg episode reward: [(0, '20.730'), (1, '22.540')] -[2023-10-14 14:23:28,472][75949] Updated weights for policy 0, policy_version 23691 (0.0007) -[2023-10-14 14:23:28,532][75950] Updated weights for policy 1, policy_version 23660 (0.0007) -[2023-10-14 14:23:28,841][75949] Updated weights for policy 0, policy_version 23701 (0.0007) -[2023-10-14 14:23:28,898][75950] Updated weights for policy 1, policy_version 23670 (0.0008) -[2023-10-14 14:23:29,207][75949] Updated weights for policy 0, policy_version 23711 (0.0009) -[2023-10-14 14:23:29,252][75950] Updated weights for policy 1, policy_version 23680 (0.0011) -[2023-10-14 14:23:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 48529408. Throughput: 0: 1672.6, 1: 1660.3. Samples: 12141504. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-14 14:23:33,165][74987] Avg episode reward: [(0, '20.250'), (1, '21.900')] -[2023-10-14 14:23:33,435][75950] Updated weights for policy 1, policy_version 23690 (0.0007) -[2023-10-14 14:23:33,473][75949] Updated weights for policy 0, policy_version 23721 (0.0008) -[2023-10-14 14:23:33,799][75950] Updated weights for policy 1, policy_version 23700 (0.0007) -[2023-10-14 14:23:33,846][75949] Updated weights for policy 0, policy_version 23731 (0.0008) -[2023-10-14 14:23:34,171][75950] Updated weights for policy 1, policy_version 23710 (0.0008) -[2023-10-14 14:23:34,213][75949] Updated weights for policy 0, policy_version 23741 (0.0010) -[2023-10-14 14:23:38,076][75950] Updated weights for policy 1, policy_version 23720 (0.0008) -[2023-10-14 14:23:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48594944. Throughput: 0: 1664.7, 1: 1668.0. Samples: 12161804. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-14 14:23:38,165][74987] Avg episode reward: [(0, '21.780'), (1, '20.980')] -[2023-10-14 14:23:38,336][75949] Updated weights for policy 0, policy_version 23751 (0.0009) -[2023-10-14 14:23:38,445][75950] Updated weights for policy 1, policy_version 23730 (0.0007) -[2023-10-14 14:23:38,706][75949] Updated weights for policy 0, policy_version 23761 (0.0008) -[2023-10-14 14:23:38,812][75950] Updated weights for policy 1, policy_version 23740 (0.0007) -[2023-10-14 14:23:39,073][75949] Updated weights for policy 0, policy_version 23771 (0.0008) -[2023-10-14 14:23:43,066][75950] Updated weights for policy 1, policy_version 23750 (0.0009) -[2023-10-14 14:23:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48660480. Throughput: 0: 1656.1, 1: 1666.7. Samples: 12182144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:23:43,164][74987] Avg episode reward: [(0, '22.740'), (1, '21.020')] -[2023-10-14 14:23:43,251][75949] Updated weights for policy 0, policy_version 23781 (0.0008) -[2023-10-14 14:23:43,428][75950] Updated weights for policy 1, policy_version 23760 (0.0007) -[2023-10-14 14:23:43,627][75949] Updated weights for policy 0, policy_version 23791 (0.0008) -[2023-10-14 14:23:43,801][75950] Updated weights for policy 1, policy_version 23770 (0.0009) -[2023-10-14 14:23:43,991][75949] Updated weights for policy 0, policy_version 23801 (0.0008) -[2023-10-14 14:23:48,025][75949] Updated weights for policy 0, policy_version 23811 (0.0008) -[2023-10-14 14:23:48,048][75950] Updated weights for policy 1, policy_version 23780 (0.0008) -[2023-10-14 14:23:48,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 48726016. Throughput: 0: 1656.7, 1: 1668.7. Samples: 12191244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:23:48,164][74987] Avg episode reward: [(0, '21.120'), (1, '21.500')] -[2023-10-14 14:23:48,391][75949] Updated weights for policy 0, policy_version 23821 (0.0009) -[2023-10-14 14:23:48,416][75950] Updated weights for policy 1, policy_version 23790 (0.0009) -[2023-10-14 14:23:48,753][75949] Updated weights for policy 0, policy_version 23831 (0.0009) -[2023-10-14 14:23:48,782][75950] Updated weights for policy 1, policy_version 23800 (0.0008) -[2023-10-14 14:23:52,756][75950] Updated weights for policy 1, policy_version 23810 (0.0008) -[2023-10-14 14:23:52,887][75949] Updated weights for policy 0, policy_version 23841 (0.0008) -[2023-10-14 14:23:53,113][75950] Updated weights for policy 1, policy_version 23820 (0.0008) -[2023-10-14 14:23:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48791552. Throughput: 0: 1659.7, 1: 1667.3. Samples: 12211766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:23:53,164][74987] Avg episode reward: [(0, '22.210'), (1, '21.860')] -[2023-10-14 14:23:53,259][75949] Updated weights for policy 0, policy_version 23851 (0.0007) -[2023-10-14 14:23:53,475][75950] Updated weights for policy 1, policy_version 23830 (0.0008) -[2023-10-14 14:23:53,621][75949] Updated weights for policy 0, policy_version 23861 (0.0007) -[2023-10-14 14:23:53,844][75950] Updated weights for policy 1, policy_version 23840 (0.0008) -[2023-10-14 14:23:53,993][75949] Updated weights for policy 0, policy_version 23871 (0.0007) -[2023-10-14 14:23:58,022][75950] Updated weights for policy 1, policy_version 23850 (0.0009) -[2023-10-14 14:23:58,106][75949] Updated weights for policy 0, policy_version 23881 (0.0007) -[2023-10-14 14:23:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48857088. Throughput: 0: 1665.5, 1: 1673.3. Samples: 12232684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:23:58,164][74987] Avg episode reward: [(0, '20.030'), (1, '20.920')] -[2023-10-14 14:23:58,377][75950] Updated weights for policy 1, policy_version 23860 (0.0009) -[2023-10-14 14:23:58,466][75949] Updated weights for policy 0, policy_version 23891 (0.0007) -[2023-10-14 14:23:58,742][75950] Updated weights for policy 1, policy_version 23870 (0.0008) -[2023-10-14 14:23:58,831][75949] Updated weights for policy 0, policy_version 23901 (0.0009) -[2023-10-14 14:24:02,904][75950] Updated weights for policy 1, policy_version 23880 (0.0008) -[2023-10-14 14:24:03,089][75949] Updated weights for policy 0, policy_version 23911 (0.0008) -[2023-10-14 14:24:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48922624. Throughput: 0: 1665.7, 1: 1671.0. Samples: 12241644. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 14:24:03,165][74987] Avg episode reward: [(0, '21.650'), (1, '23.250')] -[2023-10-14 14:24:03,282][75950] Updated weights for policy 1, policy_version 23890 (0.0007) -[2023-10-14 14:24:03,465][75949] Updated weights for policy 0, policy_version 23921 (0.0008) -[2023-10-14 14:24:03,657][75950] Updated weights for policy 1, policy_version 23900 (0.0009) -[2023-10-14 14:24:03,836][75949] Updated weights for policy 0, policy_version 23931 (0.0009) -[2023-10-14 14:24:07,870][75950] Updated weights for policy 1, policy_version 23910 (0.0009) -[2023-10-14 14:24:08,020][75949] Updated weights for policy 0, policy_version 23941 (0.0009) -[2023-10-14 14:24:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48988160. Throughput: 0: 1660.6, 1: 1674.0. Samples: 12262100. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 14:24:08,164][74987] Avg episode reward: [(0, '21.840'), (1, '23.000')] -[2023-10-14 14:24:08,236][75950] Updated weights for policy 1, policy_version 23920 (0.0008) -[2023-10-14 14:24:08,391][75949] Updated weights for policy 0, policy_version 23951 (0.0009) -[2023-10-14 14:24:08,608][75950] Updated weights for policy 1, policy_version 23930 (0.0008) -[2023-10-14 14:24:08,763][75949] Updated weights for policy 0, policy_version 23961 (0.0007) -[2023-10-14 14:24:12,702][75950] Updated weights for policy 1, policy_version 23940 (0.0009) -[2023-10-14 14:24:12,838][75949] Updated weights for policy 0, policy_version 23971 (0.0007) -[2023-10-14 14:24:13,063][75950] Updated weights for policy 1, policy_version 23950 (0.0007) -[2023-10-14 14:24:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 49053696. Throughput: 0: 1664.3, 1: 1675.7. Samples: 12282668. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 14:24:13,165][74987] Avg episode reward: [(0, '20.660'), (1, '23.000')] -[2023-10-14 14:24:13,206][75949] Updated weights for policy 0, policy_version 23981 (0.0007) -[2023-10-14 14:24:13,434][75950] Updated weights for policy 1, policy_version 23960 (0.0007) -[2023-10-14 14:24:13,572][75949] Updated weights for policy 0, policy_version 23991 (0.0008) -[2023-10-14 14:24:17,464][75950] Updated weights for policy 1, policy_version 23970 (0.0009) -[2023-10-14 14:24:17,509][75949] Updated weights for policy 0, policy_version 24001 (0.0010) -[2023-10-14 14:24:17,834][75950] Updated weights for policy 1, policy_version 23980 (0.0007) -[2023-10-14 14:24:17,924][75949] Updated weights for policy 0, policy_version 24011 (0.0008) -[2023-10-14 14:24:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 49119232. Throughput: 0: 1663.6, 1: 1676.9. Samples: 12291824. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 14:24:18,164][74987] Avg episode reward: [(0, '22.450'), (1, '22.950')] -[2023-10-14 14:24:18,189][75950] Updated weights for policy 1, policy_version 23990 (0.0007) -[2023-10-14 14:24:18,298][75949] Updated weights for policy 0, policy_version 24021 (0.0009) -[2023-10-14 14:24:18,550][75950] Updated weights for policy 1, policy_version 24000 (0.0008) -[2023-10-14 14:24:18,685][75949] Updated weights for policy 0, policy_version 24031 (0.0008) -[2023-10-14 14:24:22,676][75950] Updated weights for policy 1, policy_version 24010 (0.0009) -[2023-10-14 14:24:22,777][75949] Updated weights for policy 0, policy_version 24041 (0.0009) -[2023-10-14 14:24:23,047][75950] Updated weights for policy 1, policy_version 24020 (0.0009) -[2023-10-14 14:24:23,148][75949] Updated weights for policy 0, policy_version 24051 (0.0008) -[2023-10-14 14:24:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 49184768. Throughput: 0: 1668.2, 1: 1677.3. Samples: 12312350. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 14:24:23,164][74987] Avg episode reward: [(0, '20.560'), (1, '20.950')] -[2023-10-14 14:24:23,420][75950] Updated weights for policy 1, policy_version 24030 (0.0007) -[2023-10-14 14:24:23,517][75949] Updated weights for policy 0, policy_version 24061 (0.0010) -[2023-10-14 14:24:27,359][75950] Updated weights for policy 1, policy_version 24040 (0.0008) -[2023-10-14 14:24:27,716][75950] Updated weights for policy 1, policy_version 24050 (0.0008) -[2023-10-14 14:24:27,726][75949] Updated weights for policy 0, policy_version 24071 (0.0008) -[2023-10-14 14:24:28,085][75950] Updated weights for policy 1, policy_version 24060 (0.0007) -[2023-10-14 14:24:28,095][75949] Updated weights for policy 0, policy_version 24081 (0.0008) -[2023-10-14 14:24:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 49250304. Throughput: 0: 1667.1, 1: 1663.2. Samples: 12332008. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 14:24:28,164][74987] Avg episode reward: [(0, '22.050'), (1, '21.830')] -[2023-10-14 14:24:28,458][75949] Updated weights for policy 0, policy_version 24091 (0.0010) -[2023-10-14 14:24:32,318][75950] Updated weights for policy 1, policy_version 24070 (0.0007) -[2023-10-14 14:24:32,542][75949] Updated weights for policy 0, policy_version 24101 (0.0008) -[2023-10-14 14:24:32,686][75950] Updated weights for policy 1, policy_version 24080 (0.0007) -[2023-10-14 14:24:32,907][75949] Updated weights for policy 0, policy_version 24111 (0.0008) -[2023-10-14 14:24:33,054][75950] Updated weights for policy 1, policy_version 24090 (0.0008) -[2023-10-14 14:24:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 49315840. Throughput: 0: 1673.3, 1: 1675.8. Samples: 12341956. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 14:24:33,164][74987] Avg episode reward: [(0, '20.330'), (1, '20.680')] -[2023-10-14 14:24:33,285][75949] Updated weights for policy 0, policy_version 24121 (0.0009) -[2023-10-14 14:24:37,144][75950] Updated weights for policy 1, policy_version 24100 (0.0010) -[2023-10-14 14:24:37,341][75949] Updated weights for policy 0, policy_version 24131 (0.0008) -[2023-10-14 14:24:37,511][75950] Updated weights for policy 1, policy_version 24110 (0.0009) -[2023-10-14 14:24:37,709][75949] Updated weights for policy 0, policy_version 24141 (0.0007) -[2023-10-14 14:24:37,880][75950] Updated weights for policy 1, policy_version 24120 (0.0008) -[2023-10-14 14:24:38,079][75949] Updated weights for policy 0, policy_version 24151 (0.0008) -[2023-10-14 14:24:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 49381376. Throughput: 0: 1669.2, 1: 1676.3. Samples: 12362316. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 14:24:38,164][74987] Avg episode reward: [(0, '21.650'), (1, '22.430')] -[2023-10-14 14:24:42,056][75950] Updated weights for policy 1, policy_version 24130 (0.0009) -[2023-10-14 14:24:42,277][75949] Updated weights for policy 0, policy_version 24161 (0.0008) -[2023-10-14 14:24:42,420][75950] Updated weights for policy 1, policy_version 24140 (0.0007) -[2023-10-14 14:24:42,641][75949] Updated weights for policy 0, policy_version 24171 (0.0009) -[2023-10-14 14:24:42,795][75950] Updated weights for policy 1, policy_version 24150 (0.0007) -[2023-10-14 14:24:43,016][75949] Updated weights for policy 0, policy_version 24181 (0.0009) -[2023-10-14 14:24:43,155][75950] Updated weights for policy 1, policy_version 24160 (0.0009) -[2023-10-14 14:24:43,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49479680. Throughput: 0: 1656.4, 1: 1654.4. Samples: 12381666. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 14:24:43,164][74987] Avg episode reward: [(0, '21.190'), (1, '22.000')] -[2023-10-14 14:24:43,170][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000024160_24739840.pth... -[2023-10-14 14:24:43,203][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000022592_23134208.pth -[2023-10-14 14:24:43,382][75949] Updated weights for policy 0, policy_version 24191 (0.0009) -[2023-10-14 14:24:43,418][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000024192_24772608.pth... -[2023-10-14 14:24:43,454][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000022624_23166976.pth -[2023-10-14 14:24:47,218][75950] Updated weights for policy 1, policy_version 24170 (0.0007) -[2023-10-14 14:24:47,485][75949] Updated weights for policy 0, policy_version 24201 (0.0009) -[2023-10-14 14:24:47,587][75950] Updated weights for policy 1, policy_version 24180 (0.0008) -[2023-10-14 14:24:47,855][75949] Updated weights for policy 0, policy_version 24211 (0.0009) -[2023-10-14 14:24:47,951][75950] Updated weights for policy 1, policy_version 24190 (0.0008) -[2023-10-14 14:24:48,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49545216. Throughput: 0: 1666.2, 1: 1669.7. Samples: 12391758. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 14:24:48,165][74987] Avg episode reward: [(0, '20.630'), (1, '22.240')] -[2023-10-14 14:24:48,226][75949] Updated weights for policy 0, policy_version 24221 (0.0011) -[2023-10-14 14:24:52,096][75949] Updated weights for policy 0, policy_version 24231 (0.0009) -[2023-10-14 14:24:52,236][75950] Updated weights for policy 1, policy_version 24200 (0.0009) -[2023-10-14 14:24:52,470][75949] Updated weights for policy 0, policy_version 24241 (0.0011) -[2023-10-14 14:24:52,610][75950] Updated weights for policy 1, policy_version 24210 (0.0008) -[2023-10-14 14:24:52,835][75949] Updated weights for policy 0, policy_version 24251 (0.0009) -[2023-10-14 14:24:52,975][75950] Updated weights for policy 1, policy_version 24220 (0.0009) -[2023-10-14 14:24:53,164][74987] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 49643520. Throughput: 0: 1674.7, 1: 1665.6. Samples: 12412412. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 14:24:53,164][74987] Avg episode reward: [(0, '23.020'), (1, '22.750')] -[2023-10-14 14:24:57,054][75950] Updated weights for policy 1, policy_version 24230 (0.0008) -[2023-10-14 14:24:57,074][75949] Updated weights for policy 0, policy_version 24261 (0.0008) -[2023-10-14 14:24:57,421][75950] Updated weights for policy 1, policy_version 24240 (0.0008) -[2023-10-14 14:24:57,449][75949] Updated weights for policy 0, policy_version 24271 (0.0008) -[2023-10-14 14:24:57,785][75950] Updated weights for policy 1, policy_version 24250 (0.0008) -[2023-10-14 14:24:57,820][75949] Updated weights for policy 0, policy_version 24281 (0.0009) -[2023-10-14 14:24:58,164][74987] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 49709056. Throughput: 0: 1655.2, 1: 1647.2. Samples: 12431274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:24:58,164][74987] Avg episode reward: [(0, '20.510'), (1, '23.480')] -[2023-10-14 14:25:01,814][75949] Updated weights for policy 0, policy_version 24291 (0.0007) -[2023-10-14 14:25:01,868][75950] Updated weights for policy 1, policy_version 24260 (0.0009) -[2023-10-14 14:25:02,177][75949] Updated weights for policy 0, policy_version 24301 (0.0008) -[2023-10-14 14:25:02,232][75950] Updated weights for policy 1, policy_version 24270 (0.0007) -[2023-10-14 14:25:02,545][75949] Updated weights for policy 0, policy_version 24311 (0.0008) -[2023-10-14 14:25:02,597][75950] Updated weights for policy 1, policy_version 24280 (0.0008) -[2023-10-14 14:25:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 49774592. Throughput: 0: 1674.0, 1: 1663.1. Samples: 12441996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:25:03,164][74987] Avg episode reward: [(0, '21.800'), (1, '22.720')] -[2023-10-14 14:25:06,558][75949] Updated weights for policy 0, policy_version 24321 (0.0009) -[2023-10-14 14:25:06,849][75950] Updated weights for policy 1, policy_version 24290 (0.0008) -[2023-10-14 14:25:06,931][75949] Updated weights for policy 0, policy_version 24331 (0.0008) -[2023-10-14 14:25:07,203][75950] Updated weights for policy 1, policy_version 24300 (0.0009) -[2023-10-14 14:25:07,305][75949] Updated weights for policy 0, policy_version 24341 (0.0009) -[2023-10-14 14:25:07,576][75950] Updated weights for policy 1, policy_version 24310 (0.0009) -[2023-10-14 14:25:07,667][75949] Updated weights for policy 0, policy_version 24351 (0.0008) -[2023-10-14 14:25:07,931][75950] Updated weights for policy 1, policy_version 24320 (0.0008) -[2023-10-14 14:25:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 49840128. Throughput: 0: 1674.1, 1: 1659.3. Samples: 12462354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:25:08,165][74987] Avg episode reward: [(0, '18.400'), (1, '21.620')] -[2023-10-14 14:25:11,748][75949] Updated weights for policy 0, policy_version 24361 (0.0008) -[2023-10-14 14:25:12,115][75949] Updated weights for policy 0, policy_version 24371 (0.0007) -[2023-10-14 14:25:12,288][75950] Updated weights for policy 1, policy_version 24330 (0.0007) -[2023-10-14 14:25:12,484][75949] Updated weights for policy 0, policy_version 24381 (0.0007) -[2023-10-14 14:25:12,663][75950] Updated weights for policy 1, policy_version 24340 (0.0010) -[2023-10-14 14:25:13,023][75950] Updated weights for policy 1, policy_version 24350 (0.0008) -[2023-10-14 14:25:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 49905664. Throughput: 0: 1655.5, 1: 1650.9. Samples: 12480798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:25:13,165][74987] Avg episode reward: [(0, '18.860'), (1, '21.990')] -[2023-10-14 14:25:16,619][75949] Updated weights for policy 0, policy_version 24391 (0.0008) -[2023-10-14 14:25:16,992][75949] Updated weights for policy 0, policy_version 24401 (0.0007) -[2023-10-14 14:25:17,040][75950] Updated weights for policy 1, policy_version 24360 (0.0008) -[2023-10-14 14:25:17,358][75949] Updated weights for policy 0, policy_version 24411 (0.0007) -[2023-10-14 14:25:17,419][75950] Updated weights for policy 1, policy_version 24370 (0.0008) -[2023-10-14 14:25:17,781][75950] Updated weights for policy 1, policy_version 24380 (0.0008) -[2023-10-14 14:25:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 49971200. Throughput: 0: 1677.3, 1: 1653.4. Samples: 12491838. Policy #0 lag: (min: 16.0, avg: 39.9, max: 48.0) -[2023-10-14 14:25:18,165][74987] Avg episode reward: [(0, '19.380'), (1, '19.280')] -[2023-10-14 14:25:21,483][75949] Updated weights for policy 0, policy_version 24421 (0.0007) -[2023-10-14 14:25:21,846][75949] Updated weights for policy 0, policy_version 24431 (0.0008) -[2023-10-14 14:25:21,975][75950] Updated weights for policy 1, policy_version 24390 (0.0008) -[2023-10-14 14:25:22,221][75949] Updated weights for policy 0, policy_version 24441 (0.0008) -[2023-10-14 14:25:22,341][75950] Updated weights for policy 1, policy_version 24400 (0.0007) -[2023-10-14 14:25:22,715][75950] Updated weights for policy 1, policy_version 24410 (0.0007) -[2023-10-14 14:25:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 50036736. Throughput: 0: 1669.4, 1: 1654.7. Samples: 12511900. Policy #0 lag: (min: 16.0, avg: 39.9, max: 48.0) -[2023-10-14 14:25:23,165][74987] Avg episode reward: [(0, '19.850'), (1, '19.480')] -[2023-10-14 14:25:26,258][75949] Updated weights for policy 0, policy_version 24451 (0.0009) -[2023-10-14 14:25:26,635][75949] Updated weights for policy 0, policy_version 24461 (0.0009) -[2023-10-14 14:25:26,745][75950] Updated weights for policy 1, policy_version 24420 (0.0009) -[2023-10-14 14:25:26,999][75949] Updated weights for policy 0, policy_version 24471 (0.0008) -[2023-10-14 14:25:27,112][75950] Updated weights for policy 1, policy_version 24430 (0.0009) -[2023-10-14 14:25:27,474][75950] Updated weights for policy 1, policy_version 24440 (0.0010) -[2023-10-14 14:25:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 50102272. Throughput: 0: 1662.2, 1: 1646.8. Samples: 12530572. Policy #0 lag: (min: 16.0, avg: 39.9, max: 48.0) -[2023-10-14 14:25:28,165][74987] Avg episode reward: [(0, '21.720'), (1, '20.220')] -[2023-10-14 14:25:31,033][75949] Updated weights for policy 0, policy_version 24481 (0.0008) -[2023-10-14 14:25:31,394][75949] Updated weights for policy 0, policy_version 24491 (0.0009) -[2023-10-14 14:25:31,442][75950] Updated weights for policy 1, policy_version 24450 (0.0010) -[2023-10-14 14:25:31,762][75949] Updated weights for policy 0, policy_version 24501 (0.0007) -[2023-10-14 14:25:31,802][75950] Updated weights for policy 1, policy_version 24460 (0.0007) -[2023-10-14 14:25:32,138][75949] Updated weights for policy 0, policy_version 24511 (0.0007) -[2023-10-14 14:25:32,174][75950] Updated weights for policy 1, policy_version 24470 (0.0009) -[2023-10-14 14:25:32,536][75950] Updated weights for policy 1, policy_version 24480 (0.0010) -[2023-10-14 14:25:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 50167808. Throughput: 0: 1682.0, 1: 1656.0. Samples: 12541964. Policy #0 lag: (min: 16.0, avg: 39.9, max: 48.0) -[2023-10-14 14:25:33,164][74987] Avg episode reward: [(0, '19.190'), (1, '22.150')] -[2023-10-14 14:25:36,237][75949] Updated weights for policy 0, policy_version 24521 (0.0008) -[2023-10-14 14:25:36,601][75949] Updated weights for policy 0, policy_version 24531 (0.0007) -[2023-10-14 14:25:36,612][75950] Updated weights for policy 1, policy_version 24490 (0.0008) -[2023-10-14 14:25:36,974][75949] Updated weights for policy 0, policy_version 24541 (0.0008) -[2023-10-14 14:25:36,981][75950] Updated weights for policy 1, policy_version 24500 (0.0009) -[2023-10-14 14:25:37,341][75950] Updated weights for policy 1, policy_version 24510 (0.0009) -[2023-10-14 14:25:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 50233344. Throughput: 0: 1659.4, 1: 1650.8. Samples: 12561368. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) -[2023-10-14 14:25:38,164][74987] Avg episode reward: [(0, '21.940'), (1, '23.220')] -[2023-10-14 14:25:41,014][75949] Updated weights for policy 0, policy_version 24551 (0.0009) -[2023-10-14 14:25:41,293][75950] Updated weights for policy 1, policy_version 24520 (0.0008) -[2023-10-14 14:25:41,379][75949] Updated weights for policy 0, policy_version 24561 (0.0010) -[2023-10-14 14:25:41,659][75950] Updated weights for policy 1, policy_version 24530 (0.0009) -[2023-10-14 14:25:41,748][75949] Updated weights for policy 0, policy_version 24571 (0.0008) -[2023-10-14 14:25:42,025][75950] Updated weights for policy 1, policy_version 24540 (0.0008) -[2023-10-14 14:25:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 50298880. Throughput: 0: 1668.5, 1: 1654.9. Samples: 12580828. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) -[2023-10-14 14:25:43,164][74987] Avg episode reward: [(0, '20.590'), (1, '21.890')] -[2023-10-14 14:25:45,817][75949] Updated weights for policy 0, policy_version 24581 (0.0009) -[2023-10-14 14:25:46,188][75949] Updated weights for policy 0, policy_version 24591 (0.0009) -[2023-10-14 14:25:46,293][75950] Updated weights for policy 1, policy_version 24550 (0.0008) -[2023-10-14 14:25:46,554][75949] Updated weights for policy 0, policy_version 24601 (0.0008) -[2023-10-14 14:25:46,660][75950] Updated weights for policy 1, policy_version 24560 (0.0008) -[2023-10-14 14:25:47,030][75950] Updated weights for policy 1, policy_version 24570 (0.0009) -[2023-10-14 14:25:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 50364416. Throughput: 0: 1677.6, 1: 1661.6. Samples: 12592258. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) -[2023-10-14 14:25:48,165][74987] Avg episode reward: [(0, '21.220'), (1, '24.810')] -[2023-10-14 14:25:48,166][75801] Saving new best policy, reward=24.810! -[2023-10-14 14:25:50,793][75949] Updated weights for policy 0, policy_version 24611 (0.0009) -[2023-10-14 14:25:51,017][75950] Updated weights for policy 1, policy_version 24580 (0.0009) -[2023-10-14 14:25:51,170][75949] Updated weights for policy 0, policy_version 24621 (0.0009) -[2023-10-14 14:25:51,390][75950] Updated weights for policy 1, policy_version 24590 (0.0008) -[2023-10-14 14:25:51,528][75949] Updated weights for policy 0, policy_version 24631 (0.0011) -[2023-10-14 14:25:51,749][75950] Updated weights for policy 1, policy_version 24600 (0.0007) -[2023-10-14 14:25:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 50429952. Throughput: 0: 1656.8, 1: 1651.9. Samples: 12611248. Policy #0 lag: (min: 3.0, avg: 10.4, max: 35.0) -[2023-10-14 14:25:53,165][74987] Avg episode reward: [(0, '18.960'), (1, '23.300')] -[2023-10-14 14:25:55,665][75949] Updated weights for policy 0, policy_version 24641 (0.0008) -[2023-10-14 14:25:55,817][75950] Updated weights for policy 1, policy_version 24610 (0.0008) -[2023-10-14 14:25:56,082][75949] Updated weights for policy 0, policy_version 24651 (0.0008) -[2023-10-14 14:25:56,237][75950] Updated weights for policy 1, policy_version 24620 (0.0007) -[2023-10-14 14:25:56,447][75949] Updated weights for policy 0, policy_version 24661 (0.0008) -[2023-10-14 14:25:56,603][75950] Updated weights for policy 1, policy_version 24630 (0.0008) -[2023-10-14 14:25:56,816][75949] Updated weights for policy 0, policy_version 24671 (0.0007) -[2023-10-14 14:25:56,965][75950] Updated weights for policy 1, policy_version 24640 (0.0008) -[2023-10-14 14:25:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50495488. Throughput: 0: 1673.4, 1: 1662.8. Samples: 12630926. Policy #0 lag: (min: 4.0, avg: 5.7, max: 33.0) -[2023-10-14 14:25:58,164][74987] Avg episode reward: [(0, '20.100'), (1, '23.380')] -[2023-10-14 14:26:00,806][75949] Updated weights for policy 0, policy_version 24681 (0.0007) -[2023-10-14 14:26:01,040][75950] Updated weights for policy 1, policy_version 24650 (0.0008) -[2023-10-14 14:26:01,174][75949] Updated weights for policy 0, policy_version 24691 (0.0008) -[2023-10-14 14:26:01,409][75950] Updated weights for policy 1, policy_version 24660 (0.0010) -[2023-10-14 14:26:01,539][75949] Updated weights for policy 0, policy_version 24701 (0.0009) -[2023-10-14 14:26:01,772][75950] Updated weights for policy 1, policy_version 24670 (0.0009) -[2023-10-14 14:26:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50561024. Throughput: 0: 1667.4, 1: 1674.1. Samples: 12642206. Policy #0 lag: (min: 4.0, avg: 5.7, max: 33.0) -[2023-10-14 14:26:03,164][74987] Avg episode reward: [(0, '19.550'), (1, '24.830')] -[2023-10-14 14:26:03,165][75801] Saving new best policy, reward=24.830! -[2023-10-14 14:26:05,650][75949] Updated weights for policy 0, policy_version 24711 (0.0008) -[2023-10-14 14:26:05,996][75950] Updated weights for policy 1, policy_version 24680 (0.0008) -[2023-10-14 14:26:06,024][75949] Updated weights for policy 0, policy_version 24721 (0.0008) -[2023-10-14 14:26:06,353][75950] Updated weights for policy 1, policy_version 24690 (0.0008) -[2023-10-14 14:26:06,386][75949] Updated weights for policy 0, policy_version 24731 (0.0008) -[2023-10-14 14:26:06,729][75950] Updated weights for policy 1, policy_version 24700 (0.0008) -[2023-10-14 14:26:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50626560. Throughput: 0: 1654.9, 1: 1653.2. Samples: 12660768. Policy #0 lag: (min: 4.0, avg: 5.7, max: 33.0) -[2023-10-14 14:26:08,165][74987] Avg episode reward: [(0, '22.650'), (1, '21.560')] -[2023-10-14 14:26:10,432][75949] Updated weights for policy 0, policy_version 24741 (0.0010) -[2023-10-14 14:26:10,803][75949] Updated weights for policy 0, policy_version 24751 (0.0008) -[2023-10-14 14:26:10,823][75950] Updated weights for policy 1, policy_version 24710 (0.0007) -[2023-10-14 14:26:11,175][75949] Updated weights for policy 0, policy_version 24761 (0.0008) -[2023-10-14 14:26:11,188][75950] Updated weights for policy 1, policy_version 24720 (0.0007) -[2023-10-14 14:26:11,555][75950] Updated weights for policy 1, policy_version 24730 (0.0008) -[2023-10-14 14:26:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50692096. Throughput: 0: 1677.6, 1: 1671.0. Samples: 12681256. Policy #0 lag: (min: 4.0, avg: 5.7, max: 33.0) -[2023-10-14 14:26:13,164][74987] Avg episode reward: [(0, '21.240'), (1, '24.430')] -[2023-10-14 14:26:15,052][75949] Updated weights for policy 0, policy_version 24771 (0.0008) -[2023-10-14 14:26:15,420][75949] Updated weights for policy 0, policy_version 24781 (0.0008) -[2023-10-14 14:26:15,637][75950] Updated weights for policy 1, policy_version 24740 (0.0009) -[2023-10-14 14:26:15,796][75949] Updated weights for policy 0, policy_version 24791 (0.0008) -[2023-10-14 14:26:16,002][75950] Updated weights for policy 1, policy_version 24750 (0.0008) -[2023-10-14 14:26:16,378][75950] Updated weights for policy 1, policy_version 24760 (0.0009) -[2023-10-14 14:26:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50757632. Throughput: 0: 1662.0, 1: 1673.6. Samples: 12692062. Policy #0 lag: (min: 7.0, avg: 10.0, max: 39.0) -[2023-10-14 14:26:18,164][74987] Avg episode reward: [(0, '21.510'), (1, '23.620')] -[2023-10-14 14:26:19,941][75949] Updated weights for policy 0, policy_version 24801 (0.0009) -[2023-10-14 14:26:20,315][75949] Updated weights for policy 0, policy_version 24811 (0.0007) -[2023-10-14 14:26:20,495][75950] Updated weights for policy 1, policy_version 24770 (0.0008) -[2023-10-14 14:26:20,684][75949] Updated weights for policy 0, policy_version 24821 (0.0008) -[2023-10-14 14:26:20,859][75950] Updated weights for policy 1, policy_version 24780 (0.0009) -[2023-10-14 14:26:21,055][75949] Updated weights for policy 0, policy_version 24831 (0.0007) -[2023-10-14 14:26:21,228][75950] Updated weights for policy 1, policy_version 24790 (0.0008) -[2023-10-14 14:26:21,591][75950] Updated weights for policy 1, policy_version 24800 (0.0009) -[2023-10-14 14:26:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 50823168. Throughput: 0: 1672.4, 1: 1655.4. Samples: 12711118. Policy #0 lag: (min: 7.0, avg: 10.0, max: 39.0) -[2023-10-14 14:26:23,165][74987] Avg episode reward: [(0, '20.950'), (1, '24.000')] -[2023-10-14 14:26:25,160][75949] Updated weights for policy 0, policy_version 24841 (0.0011) -[2023-10-14 14:26:25,540][75949] Updated weights for policy 0, policy_version 24851 (0.0007) -[2023-10-14 14:26:25,562][75950] Updated weights for policy 1, policy_version 24810 (0.0008) -[2023-10-14 14:26:25,912][75949] Updated weights for policy 0, policy_version 24861 (0.0007) -[2023-10-14 14:26:25,918][75950] Updated weights for policy 1, policy_version 24820 (0.0008) -[2023-10-14 14:26:26,292][75950] Updated weights for policy 1, policy_version 24830 (0.0009) -[2023-10-14 14:26:28,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 50888704. Throughput: 0: 1682.6, 1: 1668.4. Samples: 12731622. Policy #0 lag: (min: 7.0, avg: 10.0, max: 39.0) -[2023-10-14 14:26:28,165][74987] Avg episode reward: [(0, '20.960'), (1, '25.130')] -[2023-10-14 14:26:28,178][75801] Saving new best policy, reward=25.130! -[2023-10-14 14:26:29,838][75949] Updated weights for policy 0, policy_version 24871 (0.0007) -[2023-10-14 14:26:30,207][75949] Updated weights for policy 0, policy_version 24881 (0.0008) -[2023-10-14 14:26:30,486][75950] Updated weights for policy 1, policy_version 24840 (0.0007) -[2023-10-14 14:26:30,574][75949] Updated weights for policy 0, policy_version 24891 (0.0008) -[2023-10-14 14:26:30,853][75950] Updated weights for policy 1, policy_version 24850 (0.0008) -[2023-10-14 14:26:31,229][75950] Updated weights for policy 1, policy_version 24860 (0.0010) -[2023-10-14 14:26:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50954240. Throughput: 0: 1662.4, 1: 1660.8. Samples: 12741806. Policy #0 lag: (min: 7.0, avg: 10.0, max: 39.0) -[2023-10-14 14:26:33,164][74987] Avg episode reward: [(0, '20.910'), (1, '23.210')] -[2023-10-14 14:26:34,609][75949] Updated weights for policy 0, policy_version 24901 (0.0008) -[2023-10-14 14:26:34,986][75949] Updated weights for policy 0, policy_version 24911 (0.0007) -[2023-10-14 14:26:35,353][75949] Updated weights for policy 0, policy_version 24921 (0.0009) -[2023-10-14 14:26:35,526][75950] Updated weights for policy 1, policy_version 24870 (0.0010) -[2023-10-14 14:26:35,891][75950] Updated weights for policy 1, policy_version 24880 (0.0009) -[2023-10-14 14:26:36,259][75950] Updated weights for policy 1, policy_version 24890 (0.0007) -[2023-10-14 14:26:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51019776. Throughput: 0: 1686.1, 1: 1651.5. Samples: 12761440. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:26:38,165][74987] Avg episode reward: [(0, '20.640'), (1, '24.690')] -[2023-10-14 14:26:39,307][75949] Updated weights for policy 0, policy_version 24931 (0.0009) -[2023-10-14 14:26:39,684][75949] Updated weights for policy 0, policy_version 24941 (0.0009) -[2023-10-14 14:26:40,052][75949] Updated weights for policy 0, policy_version 24951 (0.0008) -[2023-10-14 14:26:40,426][75950] Updated weights for policy 1, policy_version 24900 (0.0007) -[2023-10-14 14:26:40,795][75950] Updated weights for policy 1, policy_version 24910 (0.0007) -[2023-10-14 14:26:41,168][75950] Updated weights for policy 1, policy_version 24920 (0.0009) -[2023-10-14 14:26:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51085312. Throughput: 0: 1700.3, 1: 1666.4. Samples: 12782428. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:26:43,164][74987] Avg episode reward: [(0, '20.810'), (1, '22.730')] -[2023-10-14 14:26:43,172][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000024928_25526272.pth... -[2023-10-14 14:26:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000024960_25559040.pth... -[2023-10-14 14:26:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000023360_23920640.pth -[2023-10-14 14:26:43,214][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000023392_23953408.pth -[2023-10-14 14:26:44,208][75949] Updated weights for policy 0, policy_version 24961 (0.0009) -[2023-10-14 14:26:44,615][75949] Updated weights for policy 0, policy_version 24971 (0.0010) -[2023-10-14 14:26:44,986][75949] Updated weights for policy 0, policy_version 24981 (0.0012) -[2023-10-14 14:26:45,334][75950] Updated weights for policy 1, policy_version 24930 (0.0010) -[2023-10-14 14:26:45,352][75949] Updated weights for policy 0, policy_version 24991 (0.0011) -[2023-10-14 14:26:45,702][75950] Updated weights for policy 1, policy_version 24940 (0.0010) -[2023-10-14 14:26:46,073][75950] Updated weights for policy 1, policy_version 24950 (0.0007) -[2023-10-14 14:26:46,448][75950] Updated weights for policy 1, policy_version 24960 (0.0010) -[2023-10-14 14:26:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51150848. Throughput: 0: 1674.9, 1: 1657.9. Samples: 12792180. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:26:48,164][74987] Avg episode reward: [(0, '21.120'), (1, '22.860')] -[2023-10-14 14:26:49,404][75949] Updated weights for policy 0, policy_version 25001 (0.0007) -[2023-10-14 14:26:49,781][75949] Updated weights for policy 0, policy_version 25011 (0.0007) -[2023-10-14 14:26:50,161][75949] Updated weights for policy 0, policy_version 25021 (0.0008) -[2023-10-14 14:26:50,551][75950] Updated weights for policy 1, policy_version 24970 (0.0008) -[2023-10-14 14:26:50,920][75950] Updated weights for policy 1, policy_version 24980 (0.0010) -[2023-10-14 14:26:51,282][75950] Updated weights for policy 1, policy_version 24990 (0.0010) -[2023-10-14 14:26:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51216384. Throughput: 0: 1700.1, 1: 1662.2. Samples: 12812070. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:26:53,164][74987] Avg episode reward: [(0, '21.690'), (1, '24.460')] -[2023-10-14 14:26:54,198][75949] Updated weights for policy 0, policy_version 25031 (0.0009) -[2023-10-14 14:26:54,563][75949] Updated weights for policy 0, policy_version 25041 (0.0009) -[2023-10-14 14:26:54,929][75949] Updated weights for policy 0, policy_version 25051 (0.0008) -[2023-10-14 14:26:55,195][75950] Updated weights for policy 1, policy_version 25000 (0.0010) -[2023-10-14 14:26:55,563][75950] Updated weights for policy 1, policy_version 25010 (0.0008) -[2023-10-14 14:26:55,935][75950] Updated weights for policy 1, policy_version 25020 (0.0007) -[2023-10-14 14:26:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51281920. Throughput: 0: 1698.0, 1: 1671.6. Samples: 12832890. Policy #0 lag: (min: 1.0, avg: 10.5, max: 33.0) -[2023-10-14 14:26:58,164][74987] Avg episode reward: [(0, '20.320'), (1, '24.700')] -[2023-10-14 14:26:58,864][75949] Updated weights for policy 0, policy_version 25061 (0.0009) -[2023-10-14 14:26:59,221][75949] Updated weights for policy 0, policy_version 25071 (0.0009) -[2023-10-14 14:26:59,602][75949] Updated weights for policy 0, policy_version 25081 (0.0008) -[2023-10-14 14:26:59,931][75950] Updated weights for policy 1, policy_version 25030 (0.0008) -[2023-10-14 14:27:00,301][75950] Updated weights for policy 1, policy_version 25040 (0.0009) -[2023-10-14 14:27:00,670][75950] Updated weights for policy 1, policy_version 25050 (0.0010) -[2023-10-14 14:27:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51347456. Throughput: 0: 1684.9, 1: 1658.2. Samples: 12842502. Policy #0 lag: (min: 1.0, avg: 10.5, max: 33.0) -[2023-10-14 14:27:03,164][74987] Avg episode reward: [(0, '21.690'), (1, '24.250')] -[2023-10-14 14:27:03,650][75949] Updated weights for policy 0, policy_version 25091 (0.0009) -[2023-10-14 14:27:04,019][75949] Updated weights for policy 0, policy_version 25101 (0.0008) -[2023-10-14 14:27:04,382][75949] Updated weights for policy 0, policy_version 25111 (0.0007) -[2023-10-14 14:27:04,820][75950] Updated weights for policy 1, policy_version 25060 (0.0007) -[2023-10-14 14:27:05,187][75950] Updated weights for policy 1, policy_version 25070 (0.0007) -[2023-10-14 14:27:05,550][75950] Updated weights for policy 1, policy_version 25080 (0.0009) -[2023-10-14 14:27:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51412992. Throughput: 0: 1694.4, 1: 1673.7. Samples: 12862686. Policy #0 lag: (min: 1.0, avg: 10.5, max: 33.0) -[2023-10-14 14:27:08,165][74987] Avg episode reward: [(0, '19.830'), (1, '25.590')] -[2023-10-14 14:27:08,166][75801] Saving new best policy, reward=25.590! -[2023-10-14 14:27:08,516][75949] Updated weights for policy 0, policy_version 25121 (0.0009) -[2023-10-14 14:27:08,881][75949] Updated weights for policy 0, policy_version 25131 (0.0009) -[2023-10-14 14:27:09,257][75949] Updated weights for policy 0, policy_version 25141 (0.0010) -[2023-10-14 14:27:09,628][75949] Updated weights for policy 0, policy_version 25151 (0.0008) -[2023-10-14 14:27:09,770][75950] Updated weights for policy 1, policy_version 25090 (0.0008) -[2023-10-14 14:27:10,138][75950] Updated weights for policy 1, policy_version 25100 (0.0009) -[2023-10-14 14:27:10,503][75950] Updated weights for policy 1, policy_version 25110 (0.0007) -[2023-10-14 14:27:10,876][75950] Updated weights for policy 1, policy_version 25120 (0.0008) -[2023-10-14 14:27:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 51478528. Throughput: 0: 1695.2, 1: 1666.8. Samples: 12882916. Policy #0 lag: (min: 1.0, avg: 10.5, max: 33.0) -[2023-10-14 14:27:13,165][74987] Avg episode reward: [(0, '20.650'), (1, '22.730')] -[2023-10-14 14:27:13,582][75949] Updated weights for policy 0, policy_version 25161 (0.0007) -[2023-10-14 14:27:13,949][75949] Updated weights for policy 0, policy_version 25171 (0.0007) -[2023-10-14 14:27:14,314][75949] Updated weights for policy 0, policy_version 25181 (0.0008) -[2023-10-14 14:27:14,952][75950] Updated weights for policy 1, policy_version 25130 (0.0011) -[2023-10-14 14:27:15,327][75950] Updated weights for policy 1, policy_version 25140 (0.0010) -[2023-10-14 14:27:15,691][75950] Updated weights for policy 1, policy_version 25150 (0.0011) -[2023-10-14 14:27:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51544064. Throughput: 0: 1689.4, 1: 1654.3. Samples: 12892272. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) -[2023-10-14 14:27:18,164][74987] Avg episode reward: [(0, '21.510'), (1, '23.840')] -[2023-10-14 14:27:18,447][75949] Updated weights for policy 0, policy_version 25191 (0.0007) -[2023-10-14 14:27:18,809][75949] Updated weights for policy 0, policy_version 25201 (0.0008) -[2023-10-14 14:27:19,178][75949] Updated weights for policy 0, policy_version 25211 (0.0009) -[2023-10-14 14:27:19,833][75950] Updated weights for policy 1, policy_version 25160 (0.0009) -[2023-10-14 14:27:20,195][75950] Updated weights for policy 1, policy_version 25170 (0.0008) -[2023-10-14 14:27:20,565][75950] Updated weights for policy 1, policy_version 25180 (0.0011) -[2023-10-14 14:27:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51609600. Throughput: 0: 1689.5, 1: 1672.4. Samples: 12912728. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) -[2023-10-14 14:27:23,165][74987] Avg episode reward: [(0, '19.640'), (1, '24.050')] -[2023-10-14 14:27:23,364][75949] Updated weights for policy 0, policy_version 25221 (0.0009) -[2023-10-14 14:27:23,738][75949] Updated weights for policy 0, policy_version 25231 (0.0010) -[2023-10-14 14:27:24,113][75949] Updated weights for policy 0, policy_version 25241 (0.0007) -[2023-10-14 14:27:24,659][75950] Updated weights for policy 1, policy_version 25190 (0.0009) -[2023-10-14 14:27:25,017][75950] Updated weights for policy 1, policy_version 25200 (0.0011) -[2023-10-14 14:27:25,381][75950] Updated weights for policy 1, policy_version 25210 (0.0007) -[2023-10-14 14:27:28,078][75949] Updated weights for policy 0, policy_version 25251 (0.0009) -[2023-10-14 14:27:28,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 51675136. Throughput: 0: 1683.0, 1: 1677.8. Samples: 12933664. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) -[2023-10-14 14:27:28,164][74987] Avg episode reward: [(0, '20.980'), (1, '22.280')] -[2023-10-14 14:27:28,446][75949] Updated weights for policy 0, policy_version 25261 (0.0007) -[2023-10-14 14:27:28,827][75949] Updated weights for policy 0, policy_version 25271 (0.0009) -[2023-10-14 14:27:29,376][75950] Updated weights for policy 1, policy_version 25220 (0.0007) -[2023-10-14 14:27:29,742][75950] Updated weights for policy 1, policy_version 25230 (0.0009) -[2023-10-14 14:27:30,118][75950] Updated weights for policy 1, policy_version 25240 (0.0007) -[2023-10-14 14:27:32,980][75949] Updated weights for policy 0, policy_version 25281 (0.0009) -[2023-10-14 14:27:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51740672. Throughput: 0: 1688.0, 1: 1659.2. Samples: 12942802. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) -[2023-10-14 14:27:33,164][74987] Avg episode reward: [(0, '21.670'), (1, '26.340')] -[2023-10-14 14:27:33,165][75801] Saving new best policy, reward=26.340! -[2023-10-14 14:27:33,382][75949] Updated weights for policy 0, policy_version 25291 (0.0008) -[2023-10-14 14:27:33,763][75949] Updated weights for policy 0, policy_version 25301 (0.0007) -[2023-10-14 14:27:34,130][75949] Updated weights for policy 0, policy_version 25311 (0.0009) -[2023-10-14 14:27:34,142][75950] Updated weights for policy 1, policy_version 25250 (0.0008) -[2023-10-14 14:27:34,542][75950] Updated weights for policy 1, policy_version 25260 (0.0008) -[2023-10-14 14:27:34,914][75950] Updated weights for policy 1, policy_version 25270 (0.0010) -[2023-10-14 14:27:35,282][75950] Updated weights for policy 1, policy_version 25280 (0.0010) -[2023-10-14 14:27:38,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 51806208. Throughput: 0: 1683.7, 1: 1674.8. Samples: 12963202. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 14:27:38,165][74987] Avg episode reward: [(0, '22.200'), (1, '23.980')] -[2023-10-14 14:27:38,253][75949] Updated weights for policy 0, policy_version 25321 (0.0009) -[2023-10-14 14:27:38,621][75949] Updated weights for policy 0, policy_version 25331 (0.0010) -[2023-10-14 14:27:38,996][75949] Updated weights for policy 0, policy_version 25341 (0.0009) -[2023-10-14 14:27:39,246][75950] Updated weights for policy 1, policy_version 25290 (0.0008) -[2023-10-14 14:27:39,609][75950] Updated weights for policy 1, policy_version 25300 (0.0008) -[2023-10-14 14:27:39,981][75950] Updated weights for policy 1, policy_version 25310 (0.0009) -[2023-10-14 14:27:43,103][75949] Updated weights for policy 0, policy_version 25351 (0.0011) -[2023-10-14 14:27:43,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51871744. Throughput: 0: 1679.2, 1: 1679.2. Samples: 12984022. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 14:27:43,164][74987] Avg episode reward: [(0, '23.280'), (1, '23.740')] -[2023-10-14 14:27:43,464][75949] Updated weights for policy 0, policy_version 25361 (0.0009) -[2023-10-14 14:27:43,833][75949] Updated weights for policy 0, policy_version 25371 (0.0009) -[2023-10-14 14:27:44,075][75950] Updated weights for policy 1, policy_version 25320 (0.0008) -[2023-10-14 14:27:44,434][75950] Updated weights for policy 1, policy_version 25330 (0.0010) -[2023-10-14 14:27:44,799][75950] Updated weights for policy 1, policy_version 25340 (0.0010) -[2023-10-14 14:27:47,787][75949] Updated weights for policy 0, policy_version 25381 (0.0009) -[2023-10-14 14:27:48,156][75949] Updated weights for policy 0, policy_version 25391 (0.0008) -[2023-10-14 14:27:48,163][74987] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51937280. Throughput: 0: 1677.3, 1: 1668.6. Samples: 12993068. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 14:27:48,164][74987] Avg episode reward: [(0, '21.050'), (1, '23.860')] -[2023-10-14 14:27:48,527][75949] Updated weights for policy 0, policy_version 25401 (0.0009) -[2023-10-14 14:27:48,795][75950] Updated weights for policy 1, policy_version 25350 (0.0011) -[2023-10-14 14:27:49,160][75950] Updated weights for policy 1, policy_version 25360 (0.0008) -[2023-10-14 14:27:49,530][75950] Updated weights for policy 1, policy_version 25370 (0.0009) -[2023-10-14 14:27:52,721][75949] Updated weights for policy 0, policy_version 25411 (0.0007) -[2023-10-14 14:27:53,100][75949] Updated weights for policy 0, policy_version 25421 (0.0009) -[2023-10-14 14:27:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52002816. Throughput: 0: 1674.7, 1: 1676.8. Samples: 13013504. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 14:27:53,165][74987] Avg episode reward: [(0, '20.550'), (1, '23.520')] -[2023-10-14 14:27:53,478][75949] Updated weights for policy 0, policy_version 25431 (0.0009) -[2023-10-14 14:27:53,815][75950] Updated weights for policy 1, policy_version 25380 (0.0008) -[2023-10-14 14:27:54,177][75950] Updated weights for policy 1, policy_version 25390 (0.0007) -[2023-10-14 14:27:54,547][75950] Updated weights for policy 1, policy_version 25400 (0.0008) -[2023-10-14 14:27:57,472][75949] Updated weights for policy 0, policy_version 25441 (0.0008) -[2023-10-14 14:27:57,841][75949] Updated weights for policy 0, policy_version 25451 (0.0010) -[2023-10-14 14:27:58,163][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52068352. Throughput: 0: 1672.1, 1: 1683.8. Samples: 13033932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:27:58,164][74987] Avg episode reward: [(0, '21.420'), (1, '23.990')] -[2023-10-14 14:27:58,213][75949] Updated weights for policy 0, policy_version 25461 (0.0009) -[2023-10-14 14:27:58,588][75949] Updated weights for policy 0, policy_version 25471 (0.0010) -[2023-10-14 14:27:58,756][75950] Updated weights for policy 1, policy_version 25410 (0.0008) -[2023-10-14 14:27:59,111][75950] Updated weights for policy 1, policy_version 25420 (0.0010) -[2023-10-14 14:27:59,482][75950] Updated weights for policy 1, policy_version 25430 (0.0008) -[2023-10-14 14:27:59,852][75950] Updated weights for policy 1, policy_version 25440 (0.0007) -[2023-10-14 14:28:02,761][75949] Updated weights for policy 0, policy_version 25481 (0.0008) -[2023-10-14 14:28:03,133][75949] Updated weights for policy 0, policy_version 25491 (0.0009) -[2023-10-14 14:28:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 52133888. Throughput: 0: 1677.6, 1: 1678.3. Samples: 13043288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:28:03,164][74987] Avg episode reward: [(0, '22.790'), (1, '24.530')] -[2023-10-14 14:28:03,505][75949] Updated weights for policy 0, policy_version 25501 (0.0009) -[2023-10-14 14:28:03,926][75950] Updated weights for policy 1, policy_version 25450 (0.0009) -[2023-10-14 14:28:04,294][75950] Updated weights for policy 1, policy_version 25460 (0.0007) -[2023-10-14 14:28:04,656][75950] Updated weights for policy 1, policy_version 25470 (0.0008) -[2023-10-14 14:28:07,688][75949] Updated weights for policy 0, policy_version 25511 (0.0010) -[2023-10-14 14:28:08,053][75949] Updated weights for policy 0, policy_version 25521 (0.0007) -[2023-10-14 14:28:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 52199424. Throughput: 0: 1671.5, 1: 1685.1. Samples: 13063772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:28:08,164][74987] Avg episode reward: [(0, '22.250'), (1, '25.070')] -[2023-10-14 14:28:08,433][75949] Updated weights for policy 0, policy_version 25531 (0.0007) -[2023-10-14 14:28:08,811][75950] Updated weights for policy 1, policy_version 25480 (0.0008) -[2023-10-14 14:28:09,185][75950] Updated weights for policy 1, policy_version 25490 (0.0007) -[2023-10-14 14:28:09,555][75950] Updated weights for policy 1, policy_version 25500 (0.0008) -[2023-10-14 14:28:12,316][75949] Updated weights for policy 0, policy_version 25541 (0.0007) -[2023-10-14 14:28:12,689][75949] Updated weights for policy 0, policy_version 25551 (0.0007) -[2023-10-14 14:28:13,057][75949] Updated weights for policy 0, policy_version 25561 (0.0007) -[2023-10-14 14:28:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 52264960. Throughput: 0: 1660.7, 1: 1686.0. Samples: 13084268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:28:13,164][74987] Avg episode reward: [(0, '22.450'), (1, '24.140')] -[2023-10-14 14:28:13,391][75950] Updated weights for policy 1, policy_version 25510 (0.0008) -[2023-10-14 14:28:13,762][75950] Updated weights for policy 1, policy_version 25520 (0.0008) -[2023-10-14 14:28:14,131][75950] Updated weights for policy 1, policy_version 25530 (0.0008) -[2023-10-14 14:28:17,042][75949] Updated weights for policy 0, policy_version 25571 (0.0008) -[2023-10-14 14:28:17,406][75949] Updated weights for policy 0, policy_version 25581 (0.0011) -[2023-10-14 14:28:17,787][75949] Updated weights for policy 0, policy_version 25591 (0.0011) -[2023-10-14 14:28:18,164][74987] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 52363264. Throughput: 0: 1672.5, 1: 1684.2. Samples: 13093856. Policy #0 lag: (min: 15.0, avg: 19.6, max: 47.0) -[2023-10-14 14:28:18,165][74987] Avg episode reward: [(0, '21.290'), (1, '26.090')] -[2023-10-14 14:28:18,375][75950] Updated weights for policy 1, policy_version 25540 (0.0009) -[2023-10-14 14:28:18,742][75950] Updated weights for policy 1, policy_version 25550 (0.0009) -[2023-10-14 14:28:19,112][75950] Updated weights for policy 1, policy_version 25560 (0.0007) -[2023-10-14 14:28:21,955][75949] Updated weights for policy 0, policy_version 25601 (0.0009) -[2023-10-14 14:28:22,370][75949] Updated weights for policy 0, policy_version 25611 (0.0009) -[2023-10-14 14:28:22,754][75949] Updated weights for policy 0, policy_version 25621 (0.0008) -[2023-10-14 14:28:22,992][75950] Updated weights for policy 1, policy_version 25570 (0.0007) -[2023-10-14 14:28:23,120][75949] Updated weights for policy 0, policy_version 25631 (0.0009) -[2023-10-14 14:28:23,163][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 52428800. Throughput: 0: 1673.6, 1: 1684.8. Samples: 13114330. Policy #0 lag: (min: 15.0, avg: 19.6, max: 47.0) -[2023-10-14 14:28:23,164][74987] Avg episode reward: [(0, '22.740'), (1, '25.800')] -[2023-10-14 14:28:23,406][75950] Updated weights for policy 1, policy_version 25580 (0.0007) -[2023-10-14 14:28:23,776][75950] Updated weights for policy 1, policy_version 25590 (0.0007) -[2023-10-14 14:28:24,137][75950] Updated weights for policy 1, policy_version 25600 (0.0008) -[2023-10-14 14:28:27,251][75949] Updated weights for policy 0, policy_version 25641 (0.0008) -[2023-10-14 14:28:27,616][75949] Updated weights for policy 0, policy_version 25651 (0.0009) -[2023-10-14 14:28:27,986][75949] Updated weights for policy 0, policy_version 25661 (0.0007) -[2023-10-14 14:28:28,153][75950] Updated weights for policy 1, policy_version 25610 (0.0008) -[2023-10-14 14:28:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 52494336. Throughput: 0: 1651.5, 1: 1679.0. Samples: 13133894. Policy #0 lag: (min: 15.0, avg: 19.6, max: 47.0) -[2023-10-14 14:28:28,164][74987] Avg episode reward: [(0, '22.570'), (1, '24.850')] -[2023-10-14 14:28:28,524][75950] Updated weights for policy 1, policy_version 25620 (0.0008) -[2023-10-14 14:28:28,890][75950] Updated weights for policy 1, policy_version 25630 (0.0007) -[2023-10-14 14:28:32,071][75949] Updated weights for policy 0, policy_version 25671 (0.0008) -[2023-10-14 14:28:32,440][75949] Updated weights for policy 0, policy_version 25681 (0.0008) -[2023-10-14 14:28:32,814][75949] Updated weights for policy 0, policy_version 25691 (0.0008) -[2023-10-14 14:28:33,052][75950] Updated weights for policy 1, policy_version 25640 (0.0007) -[2023-10-14 14:28:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 52559872. Throughput: 0: 1672.2, 1: 1676.5. Samples: 13143760. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 14:28:33,164][74987] Avg episode reward: [(0, '22.420'), (1, '25.700')] -[2023-10-14 14:28:33,424][75950] Updated weights for policy 1, policy_version 25650 (0.0009) -[2023-10-14 14:28:33,791][75950] Updated weights for policy 1, policy_version 25660 (0.0009) -[2023-10-14 14:28:36,816][75949] Updated weights for policy 0, policy_version 25701 (0.0008) -[2023-10-14 14:28:37,193][75949] Updated weights for policy 0, policy_version 25711 (0.0008) -[2023-10-14 14:28:37,559][75949] Updated weights for policy 0, policy_version 25721 (0.0009) -[2023-10-14 14:28:37,928][75950] Updated weights for policy 1, policy_version 25670 (0.0008) -[2023-10-14 14:28:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 52625408. Throughput: 0: 1671.8, 1: 1678.7. Samples: 13164276. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 14:28:38,164][74987] Avg episode reward: [(0, '20.550'), (1, '24.910')] -[2023-10-14 14:28:38,304][75950] Updated weights for policy 1, policy_version 25680 (0.0009) -[2023-10-14 14:28:38,671][75950] Updated weights for policy 1, policy_version 25690 (0.0009) -[2023-10-14 14:28:41,696][75949] Updated weights for policy 0, policy_version 25731 (0.0007) -[2023-10-14 14:28:42,064][75949] Updated weights for policy 0, policy_version 25741 (0.0008) -[2023-10-14 14:28:42,443][75949] Updated weights for policy 0, policy_version 25751 (0.0007) -[2023-10-14 14:28:42,823][75950] Updated weights for policy 1, policy_version 25700 (0.0007) -[2023-10-14 14:28:43,164][74987] Fps is (10 sec: 13106.6, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 52690944. Throughput: 0: 1653.1, 1: 1677.8. Samples: 13183826. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 14:28:43,165][74987] Avg episode reward: [(0, '22.930'), (1, '25.590')] -[2023-10-14 14:28:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000025760_26378240.pth... -[2023-10-14 14:28:43,194][75950] Updated weights for policy 1, policy_version 25710 (0.0009) -[2023-10-14 14:28:43,204][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000024192_24772608.pth -[2023-10-14 14:28:43,569][75950] Updated weights for policy 1, policy_version 25720 (0.0009) -[2023-10-14 14:28:43,859][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000025728_26345472.pth... -[2023-10-14 14:28:43,889][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000024160_24739840.pth -[2023-10-14 14:28:46,513][75949] Updated weights for policy 0, policy_version 25761 (0.0007) -[2023-10-14 14:28:46,879][75949] Updated weights for policy 0, policy_version 25771 (0.0009) -[2023-10-14 14:28:47,251][75949] Updated weights for policy 0, policy_version 25781 (0.0008) -[2023-10-14 14:28:47,622][75949] Updated weights for policy 0, policy_version 25791 (0.0010) -[2023-10-14 14:28:47,704][75950] Updated weights for policy 1, policy_version 25730 (0.0008) -[2023-10-14 14:28:48,075][75950] Updated weights for policy 1, policy_version 25740 (0.0009) -[2023-10-14 14:28:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 52756480. Throughput: 0: 1674.4, 1: 1677.0. Samples: 13194100. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 14:28:48,164][74987] Avg episode reward: [(0, '19.850'), (1, '24.610')] -[2023-10-14 14:28:48,441][75950] Updated weights for policy 1, policy_version 25750 (0.0008) -[2023-10-14 14:28:48,799][75950] Updated weights for policy 1, policy_version 25760 (0.0008) -[2023-10-14 14:28:51,657][75949] Updated weights for policy 0, policy_version 25801 (0.0009) -[2023-10-14 14:28:52,034][75949] Updated weights for policy 0, policy_version 25811 (0.0009) -[2023-10-14 14:28:52,407][75949] Updated weights for policy 0, policy_version 25821 (0.0008) -[2023-10-14 14:28:52,865][75950] Updated weights for policy 1, policy_version 25770 (0.0007) -[2023-10-14 14:28:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 52822016. Throughput: 0: 1671.4, 1: 1672.9. Samples: 13214266. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 14:28:53,165][74987] Avg episode reward: [(0, '23.380'), (1, '26.600')] -[2023-10-14 14:28:53,166][75615] Saving new best policy, reward=23.380! -[2023-10-14 14:28:53,238][75950] Updated weights for policy 1, policy_version 25780 (0.0007) -[2023-10-14 14:28:53,599][75950] Updated weights for policy 1, policy_version 25790 (0.0007) -[2023-10-14 14:28:53,670][75801] Saving new best policy, reward=26.600! -[2023-10-14 14:28:56,428][75949] Updated weights for policy 0, policy_version 25831 (0.0010) -[2023-10-14 14:28:56,796][75949] Updated weights for policy 0, policy_version 25841 (0.0008) -[2023-10-14 14:28:57,173][75949] Updated weights for policy 0, policy_version 25851 (0.0008) -[2023-10-14 14:28:57,406][75950] Updated weights for policy 1, policy_version 25800 (0.0010) -[2023-10-14 14:28:57,769][75950] Updated weights for policy 1, policy_version 25810 (0.0010) -[2023-10-14 14:28:58,140][75950] Updated weights for policy 1, policy_version 25820 (0.0010) -[2023-10-14 14:28:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 52887552. Throughput: 0: 1663.8, 1: 1660.9. Samples: 13233878. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 14:28:58,164][74987] Avg episode reward: [(0, '20.950'), (1, '25.110')] -[2023-10-14 14:29:01,213][75949] Updated weights for policy 0, policy_version 25861 (0.0009) -[2023-10-14 14:29:01,591][75949] Updated weights for policy 0, policy_version 25871 (0.0008) -[2023-10-14 14:29:01,965][75949] Updated weights for policy 0, policy_version 25881 (0.0008) -[2023-10-14 14:29:02,223][75950] Updated weights for policy 1, policy_version 25830 (0.0007) -[2023-10-14 14:29:02,585][75950] Updated weights for policy 1, policy_version 25840 (0.0008) -[2023-10-14 14:29:02,961][75950] Updated weights for policy 1, policy_version 25850 (0.0009) -[2023-10-14 14:29:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 52953088. Throughput: 0: 1679.3, 1: 1676.5. Samples: 13244870. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 14:29:03,164][74987] Avg episode reward: [(0, '24.120'), (1, '24.380')] -[2023-10-14 14:29:03,165][75615] Saving new best policy, reward=24.120! -[2023-10-14 14:29:06,097][75949] Updated weights for policy 0, policy_version 25891 (0.0008) -[2023-10-14 14:29:06,465][75949] Updated weights for policy 0, policy_version 25901 (0.0007) -[2023-10-14 14:29:06,833][75949] Updated weights for policy 0, policy_version 25911 (0.0009) -[2023-10-14 14:29:07,057][75950] Updated weights for policy 1, policy_version 25860 (0.0008) -[2023-10-14 14:29:07,422][75950] Updated weights for policy 1, policy_version 25870 (0.0008) -[2023-10-14 14:29:07,791][75950] Updated weights for policy 1, policy_version 25880 (0.0011) -[2023-10-14 14:29:08,164][74987] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 53051392. Throughput: 0: 1665.2, 1: 1681.2. Samples: 13264918. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 14:29:08,165][74987] Avg episode reward: [(0, '20.960'), (1, '25.970')] -[2023-10-14 14:29:11,039][75949] Updated weights for policy 0, policy_version 25921 (0.0009) -[2023-10-14 14:29:11,451][75949] Updated weights for policy 0, policy_version 25931 (0.0008) -[2023-10-14 14:29:11,822][75949] Updated weights for policy 0, policy_version 25941 (0.0007) -[2023-10-14 14:29:12,062][75950] Updated weights for policy 1, policy_version 25890 (0.0010) -[2023-10-14 14:29:12,185][75949] Updated weights for policy 0, policy_version 25951 (0.0007) -[2023-10-14 14:29:12,484][75950] Updated weights for policy 1, policy_version 25900 (0.0007) -[2023-10-14 14:29:12,853][75950] Updated weights for policy 1, policy_version 25910 (0.0007) -[2023-10-14 14:29:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53084160. Throughput: 0: 1676.4, 1: 1661.8. Samples: 13284112. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) -[2023-10-14 14:29:13,164][74987] Avg episode reward: [(0, '21.950'), (1, '24.050')] -[2023-10-14 14:29:13,220][75950] Updated weights for policy 1, policy_version 25920 (0.0007) -[2023-10-14 14:29:16,081][75949] Updated weights for policy 0, policy_version 25961 (0.0009) -[2023-10-14 14:29:16,466][75949] Updated weights for policy 0, policy_version 25971 (0.0009) -[2023-10-14 14:29:16,829][75949] Updated weights for policy 0, policy_version 25981 (0.0010) -[2023-10-14 14:29:17,327][75950] Updated weights for policy 1, policy_version 25930 (0.0007) -[2023-10-14 14:29:17,694][75950] Updated weights for policy 1, policy_version 25940 (0.0008) -[2023-10-14 14:29:18,069][75950] Updated weights for policy 1, policy_version 25950 (0.0009) -[2023-10-14 14:29:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 53182464. Throughput: 0: 1685.9, 1: 1677.9. Samples: 13295132. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) -[2023-10-14 14:29:18,164][74987] Avg episode reward: [(0, '21.810'), (1, '26.650')] -[2023-10-14 14:29:18,165][75801] Saving new best policy, reward=26.650! -[2023-10-14 14:29:21,024][75949] Updated weights for policy 0, policy_version 25991 (0.0008) -[2023-10-14 14:29:21,384][75949] Updated weights for policy 0, policy_version 26001 (0.0008) -[2023-10-14 14:29:21,754][75949] Updated weights for policy 0, policy_version 26011 (0.0010) -[2023-10-14 14:29:22,060][75950] Updated weights for policy 1, policy_version 25960 (0.0008) -[2023-10-14 14:29:22,426][75950] Updated weights for policy 1, policy_version 25970 (0.0007) -[2023-10-14 14:29:22,803][75950] Updated weights for policy 1, policy_version 25980 (0.0007) -[2023-10-14 14:29:23,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 53248000. Throughput: 0: 1662.6, 1: 1676.9. Samples: 13314554. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) -[2023-10-14 14:29:23,165][74987] Avg episode reward: [(0, '21.520'), (1, '26.030')] -[2023-10-14 14:29:25,914][75949] Updated weights for policy 0, policy_version 26021 (0.0010) -[2023-10-14 14:29:26,280][75949] Updated weights for policy 0, policy_version 26031 (0.0009) -[2023-10-14 14:29:26,650][75949] Updated weights for policy 0, policy_version 26041 (0.0008) -[2023-10-14 14:29:26,876][75950] Updated weights for policy 1, policy_version 25990 (0.0009) -[2023-10-14 14:29:27,242][75950] Updated weights for policy 1, policy_version 26000 (0.0007) -[2023-10-14 14:29:27,612][75950] Updated weights for policy 1, policy_version 26010 (0.0009) -[2023-10-14 14:29:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 53313536. Throughput: 0: 1681.5, 1: 1657.0. Samples: 13334060. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) -[2023-10-14 14:29:28,164][74987] Avg episode reward: [(0, '22.110'), (1, '25.830')] -[2023-10-14 14:29:30,630][75949] Updated weights for policy 0, policy_version 26051 (0.0008) -[2023-10-14 14:29:31,015][75949] Updated weights for policy 0, policy_version 26061 (0.0009) -[2023-10-14 14:29:31,381][75949] Updated weights for policy 0, policy_version 26071 (0.0010) -[2023-10-14 14:29:31,767][75950] Updated weights for policy 1, policy_version 26020 (0.0008) -[2023-10-14 14:29:32,136][75950] Updated weights for policy 1, policy_version 26030 (0.0010) -[2023-10-14 14:29:32,492][75950] Updated weights for policy 1, policy_version 26040 (0.0011) -[2023-10-14 14:29:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 53379072. Throughput: 0: 1679.5, 1: 1679.0. Samples: 13345234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:29:33,165][74987] Avg episode reward: [(0, '20.020'), (1, '26.450')] -[2023-10-14 14:29:35,407][75949] Updated weights for policy 0, policy_version 26081 (0.0008) -[2023-10-14 14:29:35,772][75949] Updated weights for policy 0, policy_version 26091 (0.0008) -[2023-10-14 14:29:36,153][75949] Updated weights for policy 0, policy_version 26101 (0.0009) -[2023-10-14 14:29:36,520][75949] Updated weights for policy 0, policy_version 26111 (0.0009) -[2023-10-14 14:29:36,578][75950] Updated weights for policy 1, policy_version 26050 (0.0009) -[2023-10-14 14:29:36,941][75950] Updated weights for policy 1, policy_version 26060 (0.0009) -[2023-10-14 14:29:37,305][75950] Updated weights for policy 1, policy_version 26070 (0.0011) -[2023-10-14 14:29:37,677][75950] Updated weights for policy 1, policy_version 26080 (0.0011) -[2023-10-14 14:29:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 53444608. Throughput: 0: 1668.1, 1: 1675.8. Samples: 13364742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:29:38,165][74987] Avg episode reward: [(0, '21.210'), (1, '25.900')] -[2023-10-14 14:29:40,436][75949] Updated weights for policy 0, policy_version 26121 (0.0007) -[2023-10-14 14:29:40,812][75949] Updated weights for policy 0, policy_version 26131 (0.0007) -[2023-10-14 14:29:41,180][75949] Updated weights for policy 0, policy_version 26141 (0.0008) -[2023-10-14 14:29:41,706][75950] Updated weights for policy 1, policy_version 26090 (0.0009) -[2023-10-14 14:29:42,081][75950] Updated weights for policy 1, policy_version 26100 (0.0010) -[2023-10-14 14:29:42,440][75950] Updated weights for policy 1, policy_version 26110 (0.0010) -[2023-10-14 14:29:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 53510144. Throughput: 0: 1693.7, 1: 1656.4. Samples: 13384630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:29:43,164][74987] Avg episode reward: [(0, '19.560'), (1, '25.080')] -[2023-10-14 14:29:44,985][75949] Updated weights for policy 0, policy_version 26151 (0.0008) -[2023-10-14 14:29:45,357][75949] Updated weights for policy 0, policy_version 26161 (0.0008) -[2023-10-14 14:29:45,723][75949] Updated weights for policy 0, policy_version 26171 (0.0008) -[2023-10-14 14:29:46,634][75950] Updated weights for policy 1, policy_version 26120 (0.0008) -[2023-10-14 14:29:47,010][75950] Updated weights for policy 1, policy_version 26130 (0.0009) -[2023-10-14 14:29:47,374][75950] Updated weights for policy 1, policy_version 26140 (0.0009) -[2023-10-14 14:29:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53575680. Throughput: 0: 1673.9, 1: 1667.4. Samples: 13395226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:29:48,165][74987] Avg episode reward: [(0, '21.610'), (1, '24.490')] -[2023-10-14 14:29:49,778][75949] Updated weights for policy 0, policy_version 26181 (0.0008) -[2023-10-14 14:29:50,155][75949] Updated weights for policy 0, policy_version 26191 (0.0009) -[2023-10-14 14:29:50,530][75949] Updated weights for policy 0, policy_version 26201 (0.0008) -[2023-10-14 14:29:51,618][75950] Updated weights for policy 1, policy_version 26150 (0.0010) -[2023-10-14 14:29:51,992][75950] Updated weights for policy 1, policy_version 26160 (0.0011) -[2023-10-14 14:29:52,352][75950] Updated weights for policy 1, policy_version 26170 (0.0011) -[2023-10-14 14:29:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53641216. Throughput: 0: 1681.8, 1: 1661.3. Samples: 13415356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:29:53,165][74987] Avg episode reward: [(0, '20.150'), (1, '25.420')] -[2023-10-14 14:29:54,507][75949] Updated weights for policy 0, policy_version 26211 (0.0009) -[2023-10-14 14:29:54,878][75949] Updated weights for policy 0, policy_version 26221 (0.0008) -[2023-10-14 14:29:55,245][75949] Updated weights for policy 0, policy_version 26231 (0.0009) -[2023-10-14 14:29:56,506][75950] Updated weights for policy 1, policy_version 26180 (0.0008) -[2023-10-14 14:29:56,869][75950] Updated weights for policy 1, policy_version 26190 (0.0008) -[2023-10-14 14:29:57,239][75950] Updated weights for policy 1, policy_version 26200 (0.0008) -[2023-10-14 14:29:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53706752. Throughput: 0: 1697.3, 1: 1655.7. Samples: 13434998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:29:58,164][74987] Avg episode reward: [(0, '22.960'), (1, '25.510')] -[2023-10-14 14:29:59,400][75949] Updated weights for policy 0, policy_version 26241 (0.0009) -[2023-10-14 14:29:59,806][75949] Updated weights for policy 0, policy_version 26251 (0.0008) -[2023-10-14 14:30:00,185][75949] Updated weights for policy 0, policy_version 26261 (0.0008) -[2023-10-14 14:30:00,566][75949] Updated weights for policy 0, policy_version 26271 (0.0009) -[2023-10-14 14:30:01,377][75950] Updated weights for policy 1, policy_version 26210 (0.0007) -[2023-10-14 14:30:01,766][75950] Updated weights for policy 1, policy_version 26220 (0.0009) -[2023-10-14 14:30:02,136][75950] Updated weights for policy 1, policy_version 26230 (0.0008) -[2023-10-14 14:30:02,518][75950] Updated weights for policy 1, policy_version 26240 (0.0008) -[2023-10-14 14:30:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53772288. Throughput: 0: 1664.2, 1: 1670.2. Samples: 13445182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:30:03,165][74987] Avg episode reward: [(0, '21.070'), (1, '25.140')] -[2023-10-14 14:30:04,753][75949] Updated weights for policy 0, policy_version 26281 (0.0009) -[2023-10-14 14:30:05,126][75949] Updated weights for policy 0, policy_version 26291 (0.0008) -[2023-10-14 14:30:05,494][75949] Updated weights for policy 0, policy_version 26301 (0.0009) -[2023-10-14 14:30:06,624][75950] Updated weights for policy 1, policy_version 26250 (0.0008) -[2023-10-14 14:30:06,992][75950] Updated weights for policy 1, policy_version 26260 (0.0011) -[2023-10-14 14:30:07,359][75950] Updated weights for policy 1, policy_version 26270 (0.0007) -[2023-10-14 14:30:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 53837824. Throughput: 0: 1685.6, 1: 1661.0. Samples: 13465148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:30:08,164][74987] Avg episode reward: [(0, '22.620'), (1, '27.240')] -[2023-10-14 14:30:08,165][75801] Saving new best policy, reward=27.240! -[2023-10-14 14:30:09,587][75949] Updated weights for policy 0, policy_version 26311 (0.0009) -[2023-10-14 14:30:09,953][75949] Updated weights for policy 0, policy_version 26321 (0.0007) -[2023-10-14 14:30:10,319][75949] Updated weights for policy 0, policy_version 26331 (0.0007) -[2023-10-14 14:30:11,282][75950] Updated weights for policy 1, policy_version 26280 (0.0009) -[2023-10-14 14:30:11,652][75950] Updated weights for policy 1, policy_version 26290 (0.0010) -[2023-10-14 14:30:12,009][75950] Updated weights for policy 1, policy_version 26300 (0.0010) -[2023-10-14 14:30:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53903360. Throughput: 0: 1694.4, 1: 1664.4. Samples: 13485208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:30:13,165][74987] Avg episode reward: [(0, '22.360'), (1, '24.260')] -[2023-10-14 14:30:14,622][75949] Updated weights for policy 0, policy_version 26341 (0.0007) -[2023-10-14 14:30:14,993][75949] Updated weights for policy 0, policy_version 26351 (0.0009) -[2023-10-14 14:30:15,369][75949] Updated weights for policy 0, policy_version 26361 (0.0009) -[2023-10-14 14:30:16,198][75950] Updated weights for policy 1, policy_version 26310 (0.0009) -[2023-10-14 14:30:16,556][75950] Updated weights for policy 1, policy_version 26320 (0.0009) -[2023-10-14 14:30:16,928][75950] Updated weights for policy 1, policy_version 26330 (0.0010) -[2023-10-14 14:30:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 53968896. Throughput: 0: 1662.8, 1: 1672.9. Samples: 13495342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:30:18,164][74987] Avg episode reward: [(0, '22.640'), (1, '26.310')] -[2023-10-14 14:30:19,392][75949] Updated weights for policy 0, policy_version 26371 (0.0009) -[2023-10-14 14:30:19,768][75949] Updated weights for policy 0, policy_version 26381 (0.0010) -[2023-10-14 14:30:20,136][75949] Updated weights for policy 0, policy_version 26391 (0.0012) -[2023-10-14 14:30:20,949][75950] Updated weights for policy 1, policy_version 26340 (0.0007) -[2023-10-14 14:30:21,313][75950] Updated weights for policy 1, policy_version 26350 (0.0008) -[2023-10-14 14:30:21,674][75950] Updated weights for policy 1, policy_version 26360 (0.0009) -[2023-10-14 14:30:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54034432. Throughput: 0: 1682.2, 1: 1657.4. Samples: 13515024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:30:23,164][74987] Avg episode reward: [(0, '23.610'), (1, '23.420')] -[2023-10-14 14:30:24,207][75949] Updated weights for policy 0, policy_version 26401 (0.0008) -[2023-10-14 14:30:24,576][75949] Updated weights for policy 0, policy_version 26411 (0.0009) -[2023-10-14 14:30:24,943][75949] Updated weights for policy 0, policy_version 26421 (0.0009) -[2023-10-14 14:30:25,321][75949] Updated weights for policy 0, policy_version 26431 (0.0008) -[2023-10-14 14:30:25,531][75950] Updated weights for policy 1, policy_version 26370 (0.0009) -[2023-10-14 14:30:25,902][75950] Updated weights for policy 1, policy_version 26380 (0.0009) -[2023-10-14 14:30:26,258][75950] Updated weights for policy 1, policy_version 26390 (0.0011) -[2023-10-14 14:30:26,625][75950] Updated weights for policy 1, policy_version 26400 (0.0010) -[2023-10-14 14:30:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 54099968. Throughput: 0: 1674.8, 1: 1675.1. Samples: 13535374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:30:28,165][74987] Avg episode reward: [(0, '22.020'), (1, '23.490')] -[2023-10-14 14:30:29,340][75949] Updated weights for policy 0, policy_version 26441 (0.0007) -[2023-10-14 14:30:29,712][75949] Updated weights for policy 0, policy_version 26451 (0.0008) -[2023-10-14 14:30:30,087][75949] Updated weights for policy 0, policy_version 26461 (0.0009) -[2023-10-14 14:30:30,753][75950] Updated weights for policy 1, policy_version 26410 (0.0007) -[2023-10-14 14:30:31,126][75950] Updated weights for policy 1, policy_version 26420 (0.0008) -[2023-10-14 14:30:31,491][75950] Updated weights for policy 1, policy_version 26430 (0.0011) -[2023-10-14 14:30:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54165504. Throughput: 0: 1665.1, 1: 1678.5. Samples: 13545690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:30:33,164][74987] Avg episode reward: [(0, '23.780'), (1, '26.910')] -[2023-10-14 14:30:34,057][75949] Updated weights for policy 0, policy_version 26471 (0.0011) -[2023-10-14 14:30:34,432][75949] Updated weights for policy 0, policy_version 26481 (0.0008) -[2023-10-14 14:30:34,792][75949] Updated weights for policy 0, policy_version 26491 (0.0009) -[2023-10-14 14:30:35,635][75950] Updated weights for policy 1, policy_version 26440 (0.0008) -[2023-10-14 14:30:36,011][75950] Updated weights for policy 1, policy_version 26450 (0.0008) -[2023-10-14 14:30:36,364][75950] Updated weights for policy 1, policy_version 26460 (0.0009) -[2023-10-14 14:30:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54231040. Throughput: 0: 1673.6, 1: 1656.4. Samples: 13565206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:30:38,164][74987] Avg episode reward: [(0, '21.240'), (1, '23.910')] -[2023-10-14 14:30:38,935][75949] Updated weights for policy 0, policy_version 26501 (0.0010) -[2023-10-14 14:30:39,307][75949] Updated weights for policy 0, policy_version 26511 (0.0010) -[2023-10-14 14:30:39,686][75949] Updated weights for policy 0, policy_version 26521 (0.0010) -[2023-10-14 14:30:40,376][75950] Updated weights for policy 1, policy_version 26470 (0.0010) -[2023-10-14 14:30:40,742][75950] Updated weights for policy 1, policy_version 26480 (0.0008) -[2023-10-14 14:30:41,110][75950] Updated weights for policy 1, policy_version 26490 (0.0010) -[2023-10-14 14:30:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 54296576. Throughput: 0: 1669.3, 1: 1682.8. Samples: 13585844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:30:43,165][74987] Avg episode reward: [(0, '20.970'), (1, '25.890')] -[2023-10-14 14:30:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000026528_27164672.pth... -[2023-10-14 14:30:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000026496_27131904.pth... -[2023-10-14 14:30:43,212][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000024928_25526272.pth -[2023-10-14 14:30:43,216][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000024960_25559040.pth -[2023-10-14 14:30:43,713][75949] Updated weights for policy 0, policy_version 26531 (0.0008) -[2023-10-14 14:30:44,114][75949] Updated weights for policy 0, policy_version 26541 (0.0009) -[2023-10-14 14:30:44,483][75949] Updated weights for policy 0, policy_version 26551 (0.0011) -[2023-10-14 14:30:45,104][75950] Updated weights for policy 1, policy_version 26500 (0.0009) -[2023-10-14 14:30:45,477][75950] Updated weights for policy 1, policy_version 26510 (0.0007) -[2023-10-14 14:30:45,840][75950] Updated weights for policy 1, policy_version 26520 (0.0008) -[2023-10-14 14:30:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54362112. Throughput: 0: 1668.9, 1: 1671.0. Samples: 13595474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:30:48,164][74987] Avg episode reward: [(0, '21.920'), (1, '25.240')] -[2023-10-14 14:30:48,721][75949] Updated weights for policy 0, policy_version 26561 (0.0010) -[2023-10-14 14:30:49,098][75949] Updated weights for policy 0, policy_version 26571 (0.0008) -[2023-10-14 14:30:49,463][75949] Updated weights for policy 0, policy_version 26581 (0.0008) -[2023-10-14 14:30:49,837][75949] Updated weights for policy 0, policy_version 26591 (0.0009) -[2023-10-14 14:30:50,063][75950] Updated weights for policy 1, policy_version 26530 (0.0007) -[2023-10-14 14:30:50,421][75950] Updated weights for policy 1, policy_version 26540 (0.0010) -[2023-10-14 14:30:50,787][75950] Updated weights for policy 1, policy_version 26550 (0.0010) -[2023-10-14 14:30:51,151][75950] Updated weights for policy 1, policy_version 26560 (0.0011) -[2023-10-14 14:30:53,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54427648. Throughput: 0: 1667.9, 1: 1664.8. Samples: 13615118. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-14 14:30:53,164][74987] Avg episode reward: [(0, '21.090'), (1, '24.250')] -[2023-10-14 14:30:53,905][75949] Updated weights for policy 0, policy_version 26601 (0.0010) -[2023-10-14 14:30:54,273][75949] Updated weights for policy 0, policy_version 26611 (0.0009) -[2023-10-14 14:30:54,645][75949] Updated weights for policy 0, policy_version 26621 (0.0008) -[2023-10-14 14:30:55,504][75950] Updated weights for policy 1, policy_version 26570 (0.0009) -[2023-10-14 14:30:55,881][75950] Updated weights for policy 1, policy_version 26580 (0.0007) -[2023-10-14 14:30:56,247][75950] Updated weights for policy 1, policy_version 26590 (0.0009) -[2023-10-14 14:30:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54493184. Throughput: 0: 1667.7, 1: 1680.3. Samples: 13635868. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-14 14:30:58,164][74987] Avg episode reward: [(0, '23.980'), (1, '26.590')] -[2023-10-14 14:30:58,854][75949] Updated weights for policy 0, policy_version 26631 (0.0009) -[2023-10-14 14:30:59,229][75949] Updated weights for policy 0, policy_version 26641 (0.0008) -[2023-10-14 14:30:59,599][75949] Updated weights for policy 0, policy_version 26651 (0.0009) -[2023-10-14 14:31:00,337][75950] Updated weights for policy 1, policy_version 26600 (0.0008) -[2023-10-14 14:31:00,705][75950] Updated weights for policy 1, policy_version 26610 (0.0009) -[2023-10-14 14:31:01,069][75950] Updated weights for policy 1, policy_version 26620 (0.0007) -[2023-10-14 14:31:03,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54558720. Throughput: 0: 1673.6, 1: 1663.6. Samples: 13645516. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-14 14:31:03,164][74987] Avg episode reward: [(0, '22.950'), (1, '26.690')] -[2023-10-14 14:31:03,627][75949] Updated weights for policy 0, policy_version 26661 (0.0009) -[2023-10-14 14:31:03,997][75949] Updated weights for policy 0, policy_version 26671 (0.0009) -[2023-10-14 14:31:04,367][75949] Updated weights for policy 0, policy_version 26681 (0.0008) -[2023-10-14 14:31:05,065][75950] Updated weights for policy 1, policy_version 26630 (0.0007) -[2023-10-14 14:31:05,436][75950] Updated weights for policy 1, policy_version 26640 (0.0007) -[2023-10-14 14:31:05,800][75950] Updated weights for policy 1, policy_version 26650 (0.0007) -[2023-10-14 14:31:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54624256. Throughput: 0: 1678.0, 1: 1672.1. Samples: 13665782. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-10-14 14:31:08,164][74987] Avg episode reward: [(0, '22.500'), (1, '25.460')] -[2023-10-14 14:31:08,316][75949] Updated weights for policy 0, policy_version 26691 (0.0010) -[2023-10-14 14:31:08,676][75949] Updated weights for policy 0, policy_version 26701 (0.0008) -[2023-10-14 14:31:09,045][75949] Updated weights for policy 0, policy_version 26711 (0.0008) -[2023-10-14 14:31:09,908][75950] Updated weights for policy 1, policy_version 26660 (0.0008) -[2023-10-14 14:31:10,277][75950] Updated weights for policy 1, policy_version 26670 (0.0007) -[2023-10-14 14:31:10,642][75950] Updated weights for policy 1, policy_version 26680 (0.0008) -[2023-10-14 14:31:12,934][75949] Updated weights for policy 0, policy_version 26721 (0.0007) -[2023-10-14 14:31:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 54689792. Throughput: 0: 1683.8, 1: 1676.5. Samples: 13686588. Policy #0 lag: (min: 43.0, avg: 55.4, max: 56.0) -[2023-10-14 14:31:13,165][74987] Avg episode reward: [(0, '21.520'), (1, '25.830')] -[2023-10-14 14:31:13,309][75949] Updated weights for policy 0, policy_version 26731 (0.0008) -[2023-10-14 14:31:13,673][75949] Updated weights for policy 0, policy_version 26741 (0.0007) -[2023-10-14 14:31:14,034][75949] Updated weights for policy 0, policy_version 26751 (0.0008) -[2023-10-14 14:31:14,719][75950] Updated weights for policy 1, policy_version 26690 (0.0009) -[2023-10-14 14:31:15,088][75950] Updated weights for policy 1, policy_version 26700 (0.0009) -[2023-10-14 14:31:15,460][75950] Updated weights for policy 1, policy_version 26710 (0.0009) -[2023-10-14 14:31:15,832][75950] Updated weights for policy 1, policy_version 26720 (0.0009) -[2023-10-14 14:31:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54755328. Throughput: 0: 1684.7, 1: 1657.0. Samples: 13696070. Policy #0 lag: (min: 43.0, avg: 55.4, max: 56.0) -[2023-10-14 14:31:18,165][74987] Avg episode reward: [(0, '23.720'), (1, '24.760')] -[2023-10-14 14:31:18,175][75949] Updated weights for policy 0, policy_version 26761 (0.0007) -[2023-10-14 14:31:18,542][75949] Updated weights for policy 0, policy_version 26771 (0.0010) -[2023-10-14 14:31:18,913][75949] Updated weights for policy 0, policy_version 26781 (0.0007) -[2023-10-14 14:31:19,819][75950] Updated weights for policy 1, policy_version 26730 (0.0009) -[2023-10-14 14:31:20,186][75950] Updated weights for policy 1, policy_version 26740 (0.0007) -[2023-10-14 14:31:20,552][75950] Updated weights for policy 1, policy_version 26750 (0.0009) -[2023-10-14 14:31:22,979][75949] Updated weights for policy 0, policy_version 26791 (0.0007) -[2023-10-14 14:31:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54820864. Throughput: 0: 1686.7, 1: 1677.0. Samples: 13716572. Policy #0 lag: (min: 43.0, avg: 55.4, max: 56.0) -[2023-10-14 14:31:23,164][74987] Avg episode reward: [(0, '22.030'), (1, '24.830')] -[2023-10-14 14:31:23,352][75949] Updated weights for policy 0, policy_version 26801 (0.0007) -[2023-10-14 14:31:23,724][75949] Updated weights for policy 0, policy_version 26811 (0.0007) -[2023-10-14 14:31:24,741][75950] Updated weights for policy 1, policy_version 26760 (0.0008) -[2023-10-14 14:31:25,105][75950] Updated weights for policy 1, policy_version 26770 (0.0009) -[2023-10-14 14:31:25,485][75950] Updated weights for policy 1, policy_version 26780 (0.0008) -[2023-10-14 14:31:27,868][75949] Updated weights for policy 0, policy_version 26821 (0.0008) -[2023-10-14 14:31:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54886400. Throughput: 0: 1687.4, 1: 1672.0. Samples: 13737016. Policy #0 lag: (min: 43.0, avg: 55.4, max: 56.0) -[2023-10-14 14:31:28,165][74987] Avg episode reward: [(0, '23.060'), (1, '26.340')] -[2023-10-14 14:31:28,230][75949] Updated weights for policy 0, policy_version 26831 (0.0008) -[2023-10-14 14:31:28,605][75949] Updated weights for policy 0, policy_version 26841 (0.0010) -[2023-10-14 14:31:29,558][75950] Updated weights for policy 1, policy_version 26790 (0.0009) -[2023-10-14 14:31:29,924][75950] Updated weights for policy 1, policy_version 26800 (0.0008) -[2023-10-14 14:31:30,287][75950] Updated weights for policy 1, policy_version 26810 (0.0009) -[2023-10-14 14:31:32,614][75949] Updated weights for policy 0, policy_version 26851 (0.0008) -[2023-10-14 14:31:33,000][75949] Updated weights for policy 0, policy_version 26861 (0.0008) -[2023-10-14 14:31:33,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 54951936. Throughput: 0: 1692.2, 1: 1658.5. Samples: 13746256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:31:33,165][74987] Avg episode reward: [(0, '21.110'), (1, '25.770')] -[2023-10-14 14:31:33,373][75949] Updated weights for policy 0, policy_version 26871 (0.0008) -[2023-10-14 14:31:34,272][75950] Updated weights for policy 1, policy_version 26820 (0.0008) -[2023-10-14 14:31:34,639][75950] Updated weights for policy 1, policy_version 26830 (0.0008) -[2023-10-14 14:31:35,016][75950] Updated weights for policy 1, policy_version 26840 (0.0009) -[2023-10-14 14:31:37,435][75949] Updated weights for policy 0, policy_version 26881 (0.0007) -[2023-10-14 14:31:37,805][75949] Updated weights for policy 0, policy_version 26891 (0.0008) -[2023-10-14 14:31:38,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55017472. Throughput: 0: 1692.4, 1: 1676.9. Samples: 13766738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:31:38,164][74987] Avg episode reward: [(0, '21.980'), (1, '26.030')] -[2023-10-14 14:31:38,177][75949] Updated weights for policy 0, policy_version 26901 (0.0008) -[2023-10-14 14:31:38,550][75949] Updated weights for policy 0, policy_version 26911 (0.0009) -[2023-10-14 14:31:39,340][75950] Updated weights for policy 1, policy_version 26850 (0.0010) -[2023-10-14 14:31:39,710][75950] Updated weights for policy 1, policy_version 26860 (0.0008) -[2023-10-14 14:31:40,082][75950] Updated weights for policy 1, policy_version 26870 (0.0007) -[2023-10-14 14:31:40,452][75950] Updated weights for policy 1, policy_version 26880 (0.0011) -[2023-10-14 14:31:42,649][75949] Updated weights for policy 0, policy_version 26921 (0.0009) -[2023-10-14 14:31:43,019][75949] Updated weights for policy 0, policy_version 26931 (0.0010) -[2023-10-14 14:31:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 55083008. Throughput: 0: 1679.2, 1: 1674.5. Samples: 13786786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:31:43,165][74987] Avg episode reward: [(0, '22.220'), (1, '27.170')] -[2023-10-14 14:31:43,388][75949] Updated weights for policy 0, policy_version 26941 (0.0010) -[2023-10-14 14:31:44,634][75950] Updated weights for policy 1, policy_version 26890 (0.0007) -[2023-10-14 14:31:45,005][75950] Updated weights for policy 1, policy_version 26900 (0.0007) -[2023-10-14 14:31:45,373][75950] Updated weights for policy 1, policy_version 26910 (0.0008) -[2023-10-14 14:31:47,381][75949] Updated weights for policy 0, policy_version 26951 (0.0009) -[2023-10-14 14:31:47,742][75949] Updated weights for policy 0, policy_version 26961 (0.0008) -[2023-10-14 14:31:48,122][75949] Updated weights for policy 0, policy_version 26971 (0.0010) -[2023-10-14 14:31:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 55148544. Throughput: 0: 1688.2, 1: 1660.7. Samples: 13796216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:31:48,165][74987] Avg episode reward: [(0, '22.400'), (1, '25.690')] -[2023-10-14 14:31:49,443][75950] Updated weights for policy 1, policy_version 26920 (0.0010) -[2023-10-14 14:31:49,800][75950] Updated weights for policy 1, policy_version 26930 (0.0009) -[2023-10-14 14:31:50,179][75950] Updated weights for policy 1, policy_version 26940 (0.0011) -[2023-10-14 14:31:52,186][75949] Updated weights for policy 0, policy_version 26981 (0.0009) -[2023-10-14 14:31:52,558][75949] Updated weights for policy 0, policy_version 26991 (0.0008) -[2023-10-14 14:31:52,933][75949] Updated weights for policy 0, policy_version 27001 (0.0009) -[2023-10-14 14:31:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55214080. Throughput: 0: 1686.4, 1: 1667.1. Samples: 13816688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:31:53,164][74987] Avg episode reward: [(0, '22.450'), (1, '25.140')] -[2023-10-14 14:31:54,250][75950] Updated weights for policy 1, policy_version 26950 (0.0011) -[2023-10-14 14:31:54,619][75950] Updated weights for policy 1, policy_version 26960 (0.0010) -[2023-10-14 14:31:54,994][75950] Updated weights for policy 1, policy_version 26970 (0.0007) -[2023-10-14 14:31:57,186][75949] Updated weights for policy 0, policy_version 27011 (0.0008) -[2023-10-14 14:31:57,546][75949] Updated weights for policy 0, policy_version 27021 (0.0009) -[2023-10-14 14:31:57,924][75949] Updated weights for policy 0, policy_version 27031 (0.0008) -[2023-10-14 14:31:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55279616. Throughput: 0: 1664.5, 1: 1671.7. Samples: 13836720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:31:58,164][74987] Avg episode reward: [(0, '22.720'), (1, '27.140')] -[2023-10-14 14:31:59,165][75950] Updated weights for policy 1, policy_version 26980 (0.0009) -[2023-10-14 14:31:59,533][75950] Updated weights for policy 1, policy_version 26990 (0.0009) -[2023-10-14 14:31:59,893][75950] Updated weights for policy 1, policy_version 27000 (0.0008) -[2023-10-14 14:32:01,931][75949] Updated weights for policy 0, policy_version 27041 (0.0007) -[2023-10-14 14:32:02,297][75949] Updated weights for policy 0, policy_version 27051 (0.0007) -[2023-10-14 14:32:02,669][75949] Updated weights for policy 0, policy_version 27061 (0.0007) -[2023-10-14 14:32:03,041][75949] Updated weights for policy 0, policy_version 27071 (0.0007) -[2023-10-14 14:32:03,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 55377920. Throughput: 0: 1679.8, 1: 1662.8. Samples: 13846488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:32:03,164][74987] Avg episode reward: [(0, '23.710'), (1, '25.080')] -[2023-10-14 14:32:03,873][75950] Updated weights for policy 1, policy_version 27010 (0.0011) -[2023-10-14 14:32:04,245][75950] Updated weights for policy 1, policy_version 27020 (0.0009) -[2023-10-14 14:32:04,609][75950] Updated weights for policy 1, policy_version 27030 (0.0009) -[2023-10-14 14:32:04,976][75950] Updated weights for policy 1, policy_version 27040 (0.0009) -[2023-10-14 14:32:07,148][75949] Updated weights for policy 0, policy_version 27081 (0.0007) -[2023-10-14 14:32:07,522][75949] Updated weights for policy 0, policy_version 27091 (0.0009) -[2023-10-14 14:32:07,891][75949] Updated weights for policy 0, policy_version 27101 (0.0008) -[2023-10-14 14:32:08,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 55443456. Throughput: 0: 1675.4, 1: 1665.8. Samples: 13866924. Policy #0 lag: (min: 8.0, avg: 35.2, max: 40.0) -[2023-10-14 14:32:08,165][74987] Avg episode reward: [(0, '22.570'), (1, '24.850')] -[2023-10-14 14:32:09,203][75950] Updated weights for policy 1, policy_version 27050 (0.0008) -[2023-10-14 14:32:09,565][75950] Updated weights for policy 1, policy_version 27060 (0.0008) -[2023-10-14 14:32:09,934][75950] Updated weights for policy 1, policy_version 27070 (0.0008) -[2023-10-14 14:32:11,704][75949] Updated weights for policy 0, policy_version 27111 (0.0008) -[2023-10-14 14:32:12,080][75949] Updated weights for policy 0, policy_version 27121 (0.0008) -[2023-10-14 14:32:12,443][75949] Updated weights for policy 0, policy_version 27131 (0.0010) -[2023-10-14 14:32:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 55508992. Throughput: 0: 1653.2, 1: 1668.9. Samples: 13886514. Policy #0 lag: (min: 8.0, avg: 35.2, max: 40.0) -[2023-10-14 14:32:13,165][74987] Avg episode reward: [(0, '24.140'), (1, '27.000')] -[2023-10-14 14:32:13,175][75615] Saving new best policy, reward=24.140! -[2023-10-14 14:32:13,925][75950] Updated weights for policy 1, policy_version 27080 (0.0010) -[2023-10-14 14:32:14,301][75950] Updated weights for policy 1, policy_version 27090 (0.0010) -[2023-10-14 14:32:14,675][75950] Updated weights for policy 1, policy_version 27100 (0.0009) -[2023-10-14 14:32:16,676][75949] Updated weights for policy 0, policy_version 27141 (0.0009) -[2023-10-14 14:32:17,057][75949] Updated weights for policy 0, policy_version 27151 (0.0008) -[2023-10-14 14:32:17,423][75949] Updated weights for policy 0, policy_version 27161 (0.0008) -[2023-10-14 14:32:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 55574528. Throughput: 0: 1677.9, 1: 1666.5. Samples: 13896754. Policy #0 lag: (min: 8.0, avg: 35.2, max: 40.0) -[2023-10-14 14:32:18,164][74987] Avg episode reward: [(0, '23.790'), (1, '24.160')] -[2023-10-14 14:32:18,585][75950] Updated weights for policy 1, policy_version 27110 (0.0011) -[2023-10-14 14:32:18,957][75950] Updated weights for policy 1, policy_version 27120 (0.0007) -[2023-10-14 14:32:19,327][75950] Updated weights for policy 1, policy_version 27130 (0.0009) -[2023-10-14 14:32:21,612][75949] Updated weights for policy 0, policy_version 27171 (0.0010) -[2023-10-14 14:32:22,020][75949] Updated weights for policy 0, policy_version 27181 (0.0009) -[2023-10-14 14:32:22,399][75949] Updated weights for policy 0, policy_version 27191 (0.0007) -[2023-10-14 14:32:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 55640064. Throughput: 0: 1670.4, 1: 1667.0. Samples: 13916924. Policy #0 lag: (min: 8.0, avg: 35.2, max: 40.0) -[2023-10-14 14:32:23,165][74987] Avg episode reward: [(0, '23.450'), (1, '25.890')] -[2023-10-14 14:32:23,444][75950] Updated weights for policy 1, policy_version 27140 (0.0007) -[2023-10-14 14:32:23,804][75950] Updated weights for policy 1, policy_version 27150 (0.0009) -[2023-10-14 14:32:24,168][75950] Updated weights for policy 1, policy_version 27160 (0.0011) -[2023-10-14 14:32:26,282][75949] Updated weights for policy 0, policy_version 27201 (0.0008) -[2023-10-14 14:32:26,655][75949] Updated weights for policy 0, policy_version 27211 (0.0010) -[2023-10-14 14:32:27,018][75949] Updated weights for policy 0, policy_version 27221 (0.0009) -[2023-10-14 14:32:27,387][75949] Updated weights for policy 0, policy_version 27231 (0.0008) -[2023-10-14 14:32:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 55705600. Throughput: 0: 1655.3, 1: 1667.2. Samples: 13936296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:32:28,164][74987] Avg episode reward: [(0, '22.930'), (1, '27.410')] -[2023-10-14 14:32:28,173][75801] Saving new best policy, reward=27.410! -[2023-10-14 14:32:28,376][75950] Updated weights for policy 1, policy_version 27170 (0.0008) -[2023-10-14 14:32:28,744][75950] Updated weights for policy 1, policy_version 27180 (0.0009) -[2023-10-14 14:32:29,106][75950] Updated weights for policy 1, policy_version 27190 (0.0008) -[2023-10-14 14:32:29,473][75950] Updated weights for policy 1, policy_version 27200 (0.0009) -[2023-10-14 14:32:31,750][75949] Updated weights for policy 0, policy_version 27241 (0.0009) -[2023-10-14 14:32:32,133][75949] Updated weights for policy 0, policy_version 27251 (0.0008) -[2023-10-14 14:32:32,507][75949] Updated weights for policy 0, policy_version 27261 (0.0009) -[2023-10-14 14:32:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 55771136. Throughput: 0: 1675.1, 1: 1666.0. Samples: 13946564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:32:33,164][74987] Avg episode reward: [(0, '23.570'), (1, '25.570')] -[2023-10-14 14:32:33,684][75950] Updated weights for policy 1, policy_version 27210 (0.0009) -[2023-10-14 14:32:34,062][75950] Updated weights for policy 1, policy_version 27220 (0.0011) -[2023-10-14 14:32:34,447][75950] Updated weights for policy 1, policy_version 27230 (0.0010) -[2023-10-14 14:32:36,623][75949] Updated weights for policy 0, policy_version 27271 (0.0009) -[2023-10-14 14:32:36,988][75949] Updated weights for policy 0, policy_version 27281 (0.0011) -[2023-10-14 14:32:37,361][75949] Updated weights for policy 0, policy_version 27291 (0.0010) -[2023-10-14 14:32:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 55836672. Throughput: 0: 1662.0, 1: 1670.5. Samples: 13966650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:32:38,165][74987] Avg episode reward: [(0, '23.390'), (1, '27.150')] -[2023-10-14 14:32:38,474][75950] Updated weights for policy 1, policy_version 27240 (0.0009) -[2023-10-14 14:32:38,835][75950] Updated weights for policy 1, policy_version 27250 (0.0009) -[2023-10-14 14:32:39,207][75950] Updated weights for policy 1, policy_version 27260 (0.0008) -[2023-10-14 14:32:41,415][75949] Updated weights for policy 0, policy_version 27301 (0.0008) -[2023-10-14 14:32:41,786][75949] Updated weights for policy 0, policy_version 27311 (0.0008) -[2023-10-14 14:32:42,167][75949] Updated weights for policy 0, policy_version 27321 (0.0009) -[2023-10-14 14:32:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 55902208. Throughput: 0: 1659.0, 1: 1666.4. Samples: 13986364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:32:43,165][74987] Avg episode reward: [(0, '22.510'), (1, '27.310')] -[2023-10-14 14:32:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000027328_27983872.pth... -[2023-10-14 14:32:43,211][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000025760_26378240.pth -[2023-10-14 14:32:43,379][75950] Updated weights for policy 1, policy_version 27270 (0.0008) -[2023-10-14 14:32:43,743][75950] Updated weights for policy 1, policy_version 27280 (0.0008) -[2023-10-14 14:32:44,116][75950] Updated weights for policy 1, policy_version 27290 (0.0009) -[2023-10-14 14:32:44,330][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000027296_27951104.pth... -[2023-10-14 14:32:44,359][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000025728_26345472.pth -[2023-10-14 14:32:46,310][75949] Updated weights for policy 0, policy_version 27331 (0.0008) -[2023-10-14 14:32:46,674][75949] Updated weights for policy 0, policy_version 27341 (0.0010) -[2023-10-14 14:32:47,048][75949] Updated weights for policy 0, policy_version 27351 (0.0010) -[2023-10-14 14:32:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 55967744. Throughput: 0: 1672.5, 1: 1666.6. Samples: 13996748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:32:48,165][74987] Avg episode reward: [(0, '23.180'), (1, '24.760')] -[2023-10-14 14:32:48,250][75950] Updated weights for policy 1, policy_version 27300 (0.0011) -[2023-10-14 14:32:48,614][75950] Updated weights for policy 1, policy_version 27310 (0.0009) -[2023-10-14 14:32:48,986][75950] Updated weights for policy 1, policy_version 27320 (0.0008) -[2023-10-14 14:32:51,075][75949] Updated weights for policy 0, policy_version 27361 (0.0012) -[2023-10-14 14:32:51,440][75949] Updated weights for policy 0, policy_version 27371 (0.0009) -[2023-10-14 14:32:51,805][75949] Updated weights for policy 0, policy_version 27381 (0.0008) -[2023-10-14 14:32:52,178][75949] Updated weights for policy 0, policy_version 27391 (0.0009) -[2023-10-14 14:32:53,054][75950] Updated weights for policy 1, policy_version 27330 (0.0007) -[2023-10-14 14:32:53,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 56033280. Throughput: 0: 1657.0, 1: 1665.2. Samples: 14016422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:32:53,164][74987] Avg episode reward: [(0, '22.510'), (1, '26.060')] -[2023-10-14 14:32:53,423][75950] Updated weights for policy 1, policy_version 27340 (0.0008) -[2023-10-14 14:32:53,792][75950] Updated weights for policy 1, policy_version 27350 (0.0010) -[2023-10-14 14:32:54,164][75950] Updated weights for policy 1, policy_version 27360 (0.0009) -[2023-10-14 14:32:56,052][75949] Updated weights for policy 0, policy_version 27401 (0.0011) -[2023-10-14 14:32:56,433][75949] Updated weights for policy 0, policy_version 27411 (0.0010) -[2023-10-14 14:32:56,810][75949] Updated weights for policy 0, policy_version 27421 (0.0009) -[2023-10-14 14:32:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 56098816. Throughput: 0: 1668.6, 1: 1668.7. Samples: 14036692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:32:58,165][74987] Avg episode reward: [(0, '22.190'), (1, '24.790')] -[2023-10-14 14:32:58,323][75950] Updated weights for policy 1, policy_version 27370 (0.0008) -[2023-10-14 14:32:58,695][75950] Updated weights for policy 1, policy_version 27380 (0.0007) -[2023-10-14 14:32:59,057][75950] Updated weights for policy 1, policy_version 27390 (0.0009) -[2023-10-14 14:33:01,165][75949] Updated weights for policy 0, policy_version 27431 (0.0008) -[2023-10-14 14:33:01,535][75949] Updated weights for policy 0, policy_version 27441 (0.0008) -[2023-10-14 14:33:01,903][75949] Updated weights for policy 0, policy_version 27451 (0.0008) -[2023-10-14 14:33:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 56164352. Throughput: 0: 1670.4, 1: 1669.8. Samples: 14047062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:33:03,165][74987] Avg episode reward: [(0, '22.250'), (1, '24.190')] -[2023-10-14 14:33:03,188][75950] Updated weights for policy 1, policy_version 27400 (0.0007) -[2023-10-14 14:33:03,558][75950] Updated weights for policy 1, policy_version 27410 (0.0008) -[2023-10-14 14:33:03,922][75950] Updated weights for policy 1, policy_version 27420 (0.0010) -[2023-10-14 14:33:05,978][75949] Updated weights for policy 0, policy_version 27461 (0.0008) -[2023-10-14 14:33:06,351][75949] Updated weights for policy 0, policy_version 27471 (0.0009) -[2023-10-14 14:33:06,720][75949] Updated weights for policy 0, policy_version 27481 (0.0011) -[2023-10-14 14:33:08,110][75950] Updated weights for policy 1, policy_version 27430 (0.0011) -[2023-10-14 14:33:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 56229888. Throughput: 0: 1661.1, 1: 1668.4. Samples: 14066752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:33:08,164][74987] Avg episode reward: [(0, '23.880'), (1, '25.980')] -[2023-10-14 14:33:08,486][75950] Updated weights for policy 1, policy_version 27440 (0.0007) -[2023-10-14 14:33:08,850][75950] Updated weights for policy 1, policy_version 27450 (0.0008) -[2023-10-14 14:33:10,705][75949] Updated weights for policy 0, policy_version 27491 (0.0010) -[2023-10-14 14:33:11,096][75949] Updated weights for policy 0, policy_version 27501 (0.0008) -[2023-10-14 14:33:11,467][75949] Updated weights for policy 0, policy_version 27511 (0.0009) -[2023-10-14 14:33:13,017][75950] Updated weights for policy 1, policy_version 27460 (0.0008) -[2023-10-14 14:33:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56295424. Throughput: 0: 1677.3, 1: 1669.7. Samples: 14086914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:33:13,165][74987] Avg episode reward: [(0, '23.560'), (1, '23.600')] -[2023-10-14 14:33:13,394][75950] Updated weights for policy 1, policy_version 27470 (0.0011) -[2023-10-14 14:33:13,766][75950] Updated weights for policy 1, policy_version 27480 (0.0010) -[2023-10-14 14:33:15,558][75949] Updated weights for policy 0, policy_version 27521 (0.0008) -[2023-10-14 14:33:15,935][75949] Updated weights for policy 0, policy_version 27531 (0.0009) -[2023-10-14 14:33:16,297][75949] Updated weights for policy 0, policy_version 27541 (0.0009) -[2023-10-14 14:33:16,668][75949] Updated weights for policy 0, policy_version 27551 (0.0007) -[2023-10-14 14:33:17,822][75950] Updated weights for policy 1, policy_version 27490 (0.0010) -[2023-10-14 14:33:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 56360960. Throughput: 0: 1672.6, 1: 1668.4. Samples: 14096908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:33:18,165][74987] Avg episode reward: [(0, '24.590'), (1, '25.680')] -[2023-10-14 14:33:18,165][75615] Saving new best policy, reward=24.590! -[2023-10-14 14:33:18,186][75950] Updated weights for policy 1, policy_version 27500 (0.0007) -[2023-10-14 14:33:18,546][75950] Updated weights for policy 1, policy_version 27510 (0.0008) -[2023-10-14 14:33:18,918][75950] Updated weights for policy 1, policy_version 27520 (0.0007) -[2023-10-14 14:33:20,743][75949] Updated weights for policy 0, policy_version 27561 (0.0009) -[2023-10-14 14:33:21,113][75949] Updated weights for policy 0, policy_version 27571 (0.0012) -[2023-10-14 14:33:21,496][75949] Updated weights for policy 0, policy_version 27581 (0.0011) -[2023-10-14 14:33:23,104][75950] Updated weights for policy 1, policy_version 27530 (0.0007) -[2023-10-14 14:33:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56426496. Throughput: 0: 1658.5, 1: 1667.1. Samples: 14116304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:33:23,164][74987] Avg episode reward: [(0, '22.430'), (1, '26.360')] -[2023-10-14 14:33:23,475][75950] Updated weights for policy 1, policy_version 27540 (0.0007) -[2023-10-14 14:33:23,844][75950] Updated weights for policy 1, policy_version 27550 (0.0008) -[2023-10-14 14:33:25,493][75949] Updated weights for policy 0, policy_version 27591 (0.0009) -[2023-10-14 14:33:25,871][75949] Updated weights for policy 0, policy_version 27601 (0.0007) -[2023-10-14 14:33:26,237][75949] Updated weights for policy 0, policy_version 27611 (0.0010) -[2023-10-14 14:33:27,803][75950] Updated weights for policy 1, policy_version 27560 (0.0008) -[2023-10-14 14:33:28,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56492032. Throughput: 0: 1675.8, 1: 1673.5. Samples: 14137080. Policy #0 lag: (min: 10.0, avg: 12.7, max: 36.0) -[2023-10-14 14:33:28,164][74987] Avg episode reward: [(0, '23.950'), (1, '24.050')] -[2023-10-14 14:33:28,176][75950] Updated weights for policy 1, policy_version 27570 (0.0007) -[2023-10-14 14:33:28,540][75950] Updated weights for policy 1, policy_version 27580 (0.0007) -[2023-10-14 14:33:30,377][75949] Updated weights for policy 0, policy_version 27621 (0.0008) -[2023-10-14 14:33:30,745][75949] Updated weights for policy 0, policy_version 27631 (0.0007) -[2023-10-14 14:33:31,113][75949] Updated weights for policy 0, policy_version 27641 (0.0007) -[2023-10-14 14:33:32,496][75950] Updated weights for policy 1, policy_version 27590 (0.0010) -[2023-10-14 14:33:32,868][75950] Updated weights for policy 1, policy_version 27600 (0.0011) -[2023-10-14 14:33:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 56557568. Throughput: 0: 1663.6, 1: 1680.9. Samples: 14147252. Policy #0 lag: (min: 10.0, avg: 12.7, max: 36.0) -[2023-10-14 14:33:33,165][74987] Avg episode reward: [(0, '22.780'), (1, '27.300')] -[2023-10-14 14:33:33,230][75950] Updated weights for policy 1, policy_version 27610 (0.0010) -[2023-10-14 14:33:35,122][75949] Updated weights for policy 0, policy_version 27651 (0.0010) -[2023-10-14 14:33:35,502][75949] Updated weights for policy 0, policy_version 27661 (0.0007) -[2023-10-14 14:33:35,868][75949] Updated weights for policy 0, policy_version 27671 (0.0009) -[2023-10-14 14:33:37,376][75950] Updated weights for policy 1, policy_version 27620 (0.0010) -[2023-10-14 14:33:37,740][75950] Updated weights for policy 1, policy_version 27630 (0.0008) -[2023-10-14 14:33:38,103][75950] Updated weights for policy 1, policy_version 27640 (0.0009) -[2023-10-14 14:33:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56623104. Throughput: 0: 1665.3, 1: 1680.0. Samples: 14166962. Policy #0 lag: (min: 10.0, avg: 12.7, max: 36.0) -[2023-10-14 14:33:38,164][74987] Avg episode reward: [(0, '22.200'), (1, '24.290')] -[2023-10-14 14:33:39,831][75949] Updated weights for policy 0, policy_version 27681 (0.0008) -[2023-10-14 14:33:40,195][75949] Updated weights for policy 0, policy_version 27691 (0.0008) -[2023-10-14 14:33:40,560][75949] Updated weights for policy 0, policy_version 27701 (0.0007) -[2023-10-14 14:33:40,929][75949] Updated weights for policy 0, policy_version 27711 (0.0008) -[2023-10-14 14:33:42,241][75950] Updated weights for policy 1, policy_version 27650 (0.0009) -[2023-10-14 14:33:42,604][75950] Updated weights for policy 1, policy_version 27660 (0.0010) -[2023-10-14 14:33:42,974][75950] Updated weights for policy 1, policy_version 27670 (0.0009) -[2023-10-14 14:33:43,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 56688640. Throughput: 0: 1678.7, 1: 1665.2. Samples: 14187166. Policy #0 lag: (min: 10.0, avg: 12.7, max: 36.0) -[2023-10-14 14:33:43,164][74987] Avg episode reward: [(0, '24.760'), (1, '24.340')] -[2023-10-14 14:33:43,171][75615] Saving new best policy, reward=24.760! -[2023-10-14 14:33:43,336][75950] Updated weights for policy 1, policy_version 27680 (0.0008) -[2023-10-14 14:33:44,989][75949] Updated weights for policy 0, policy_version 27721 (0.0008) -[2023-10-14 14:33:45,367][75949] Updated weights for policy 0, policy_version 27731 (0.0008) -[2023-10-14 14:33:45,742][75949] Updated weights for policy 0, policy_version 27741 (0.0008) -[2023-10-14 14:33:47,524][75950] Updated weights for policy 1, policy_version 27690 (0.0009) -[2023-10-14 14:33:47,902][75950] Updated weights for policy 1, policy_version 27700 (0.0008) -[2023-10-14 14:33:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56754176. Throughput: 0: 1658.8, 1: 1673.2. Samples: 14197002. Policy #0 lag: (min: 10.0, avg: 10.0, max: 11.0) -[2023-10-14 14:33:48,165][74987] Avg episode reward: [(0, '22.660'), (1, '26.760')] -[2023-10-14 14:33:48,265][75950] Updated weights for policy 1, policy_version 27710 (0.0007) -[2023-10-14 14:33:49,847][75949] Updated weights for policy 0, policy_version 27751 (0.0009) -[2023-10-14 14:33:50,226][75949] Updated weights for policy 0, policy_version 27761 (0.0009) -[2023-10-14 14:33:50,597][75949] Updated weights for policy 0, policy_version 27771 (0.0007) -[2023-10-14 14:33:52,202][75950] Updated weights for policy 1, policy_version 27720 (0.0010) -[2023-10-14 14:33:52,570][75950] Updated weights for policy 1, policy_version 27730 (0.0009) -[2023-10-14 14:33:52,935][75950] Updated weights for policy 1, policy_version 27740 (0.0011) -[2023-10-14 14:33:53,164][74987] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 56852480. Throughput: 0: 1666.7, 1: 1670.8. Samples: 14216940. Policy #0 lag: (min: 10.0, avg: 10.0, max: 11.0) -[2023-10-14 14:33:53,165][74987] Avg episode reward: [(0, '22.210'), (1, '24.200')] -[2023-10-14 14:33:54,754][75949] Updated weights for policy 0, policy_version 27781 (0.0008) -[2023-10-14 14:33:55,121][75949] Updated weights for policy 0, policy_version 27791 (0.0007) -[2023-10-14 14:33:55,501][75949] Updated weights for policy 0, policy_version 27801 (0.0010) -[2023-10-14 14:33:57,074][75950] Updated weights for policy 1, policy_version 27750 (0.0010) -[2023-10-14 14:33:57,439][75950] Updated weights for policy 1, policy_version 27760 (0.0009) -[2023-10-14 14:33:57,805][75950] Updated weights for policy 1, policy_version 27770 (0.0008) -[2023-10-14 14:33:58,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 56918016. Throughput: 0: 1677.9, 1: 1652.4. Samples: 14236778. Policy #0 lag: (min: 10.0, avg: 10.0, max: 11.0) -[2023-10-14 14:33:58,165][74987] Avg episode reward: [(0, '23.420'), (1, '25.520')] -[2023-10-14 14:33:59,575][75949] Updated weights for policy 0, policy_version 27811 (0.0009) -[2023-10-14 14:33:59,948][75949] Updated weights for policy 0, policy_version 27821 (0.0010) -[2023-10-14 14:34:00,318][75949] Updated weights for policy 0, policy_version 27831 (0.0008) -[2023-10-14 14:34:01,913][75950] Updated weights for policy 1, policy_version 27780 (0.0007) -[2023-10-14 14:34:02,269][75950] Updated weights for policy 1, policy_version 27790 (0.0008) -[2023-10-14 14:34:02,635][75950] Updated weights for policy 1, policy_version 27800 (0.0007) -[2023-10-14 14:34:03,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 56983552. Throughput: 0: 1652.8, 1: 1679.2. Samples: 14246846. Policy #0 lag: (min: 10.0, avg: 10.0, max: 11.0) -[2023-10-14 14:34:03,164][74987] Avg episode reward: [(0, '22.290'), (1, '26.060')] -[2023-10-14 14:34:04,346][75949] Updated weights for policy 0, policy_version 27841 (0.0009) -[2023-10-14 14:34:04,708][75949] Updated weights for policy 0, policy_version 27851 (0.0010) -[2023-10-14 14:34:05,083][75949] Updated weights for policy 0, policy_version 27861 (0.0009) -[2023-10-14 14:34:05,450][75949] Updated weights for policy 0, policy_version 27871 (0.0008) -[2023-10-14 14:34:06,609][75950] Updated weights for policy 1, policy_version 27810 (0.0007) -[2023-10-14 14:34:06,975][75950] Updated weights for policy 1, policy_version 27820 (0.0008) -[2023-10-14 14:34:07,340][75950] Updated weights for policy 1, policy_version 27830 (0.0009) -[2023-10-14 14:34:07,700][75950] Updated weights for policy 1, policy_version 27840 (0.0009) -[2023-10-14 14:34:08,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 57049088. Throughput: 0: 1677.2, 1: 1679.9. Samples: 14267374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:34:08,164][74987] Avg episode reward: [(0, '24.780'), (1, '24.000')] -[2023-10-14 14:34:08,165][75615] Saving new best policy, reward=24.780! -[2023-10-14 14:34:09,538][75949] Updated weights for policy 0, policy_version 27881 (0.0010) -[2023-10-14 14:34:09,907][75949] Updated weights for policy 0, policy_version 27891 (0.0008) -[2023-10-14 14:34:10,278][75949] Updated weights for policy 0, policy_version 27901 (0.0009) -[2023-10-14 14:34:11,914][75950] Updated weights for policy 1, policy_version 27850 (0.0008) -[2023-10-14 14:34:12,287][75950] Updated weights for policy 1, policy_version 27860 (0.0007) -[2023-10-14 14:34:12,650][75950] Updated weights for policy 1, policy_version 27870 (0.0008) -[2023-10-14 14:34:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 57114624. Throughput: 0: 1684.8, 1: 1646.9. Samples: 14287010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:34:13,165][74987] Avg episode reward: [(0, '23.710'), (1, '26.610')] -[2023-10-14 14:34:14,290][75949] Updated weights for policy 0, policy_version 27911 (0.0007) -[2023-10-14 14:34:14,664][75949] Updated weights for policy 0, policy_version 27921 (0.0011) -[2023-10-14 14:34:15,034][75949] Updated weights for policy 0, policy_version 27931 (0.0011) -[2023-10-14 14:34:16,823][75950] Updated weights for policy 1, policy_version 27880 (0.0010) -[2023-10-14 14:34:17,184][75950] Updated weights for policy 1, policy_version 27890 (0.0010) -[2023-10-14 14:34:17,558][75950] Updated weights for policy 1, policy_version 27900 (0.0007) -[2023-10-14 14:34:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 57180160. Throughput: 0: 1666.5, 1: 1668.8. Samples: 14297338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:34:18,165][74987] Avg episode reward: [(0, '24.290'), (1, '25.160')] -[2023-10-14 14:34:19,201][75949] Updated weights for policy 0, policy_version 27941 (0.0010) -[2023-10-14 14:34:19,570][75949] Updated weights for policy 0, policy_version 27951 (0.0010) -[2023-10-14 14:34:19,939][75949] Updated weights for policy 0, policy_version 27961 (0.0010) -[2023-10-14 14:34:21,522][75950] Updated weights for policy 1, policy_version 27910 (0.0009) -[2023-10-14 14:34:21,889][75950] Updated weights for policy 1, policy_version 27920 (0.0009) -[2023-10-14 14:34:22,266][75950] Updated weights for policy 1, policy_version 27930 (0.0009) -[2023-10-14 14:34:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 57245696. Throughput: 0: 1685.7, 1: 1666.8. Samples: 14317824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:34:23,164][74987] Avg episode reward: [(0, '22.930'), (1, '26.270')] -[2023-10-14 14:34:24,093][75949] Updated weights for policy 0, policy_version 27971 (0.0009) -[2023-10-14 14:34:24,464][75949] Updated weights for policy 0, policy_version 27981 (0.0007) -[2023-10-14 14:34:24,832][75949] Updated weights for policy 0, policy_version 27991 (0.0007) -[2023-10-14 14:34:26,247][75950] Updated weights for policy 1, policy_version 27940 (0.0009) -[2023-10-14 14:34:26,612][75950] Updated weights for policy 1, policy_version 27950 (0.0010) -[2023-10-14 14:34:26,984][75950] Updated weights for policy 1, policy_version 27960 (0.0008) -[2023-10-14 14:34:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 57311232. Throughput: 0: 1682.3, 1: 1660.3. Samples: 14337584. Policy #0 lag: (min: 2.0, avg: 4.1, max: 33.0) -[2023-10-14 14:34:28,165][74987] Avg episode reward: [(0, '22.820'), (1, '27.350')] -[2023-10-14 14:34:28,935][75949] Updated weights for policy 0, policy_version 28001 (0.0007) -[2023-10-14 14:34:29,298][75949] Updated weights for policy 0, policy_version 28011 (0.0007) -[2023-10-14 14:34:29,673][75949] Updated weights for policy 0, policy_version 28021 (0.0008) -[2023-10-14 14:34:30,044][75949] Updated weights for policy 0, policy_version 28031 (0.0008) -[2023-10-14 14:34:31,059][75950] Updated weights for policy 1, policy_version 27970 (0.0007) -[2023-10-14 14:34:31,420][75950] Updated weights for policy 1, policy_version 27980 (0.0010) -[2023-10-14 14:34:31,787][75950] Updated weights for policy 1, policy_version 27990 (0.0008) -[2023-10-14 14:34:32,160][75950] Updated weights for policy 1, policy_version 28000 (0.0008) -[2023-10-14 14:34:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 57376768. Throughput: 0: 1674.0, 1: 1680.1. Samples: 14347932. Policy #0 lag: (min: 2.0, avg: 4.1, max: 33.0) -[2023-10-14 14:34:33,164][74987] Avg episode reward: [(0, '24.050'), (1, '24.380')] -[2023-10-14 14:34:33,932][75949] Updated weights for policy 0, policy_version 28041 (0.0010) -[2023-10-14 14:34:34,304][75949] Updated weights for policy 0, policy_version 28051 (0.0010) -[2023-10-14 14:34:34,685][75949] Updated weights for policy 0, policy_version 28061 (0.0007) -[2023-10-14 14:34:36,365][75950] Updated weights for policy 1, policy_version 28010 (0.0010) -[2023-10-14 14:34:36,726][75950] Updated weights for policy 1, policy_version 28020 (0.0008) -[2023-10-14 14:34:37,097][75950] Updated weights for policy 1, policy_version 28030 (0.0008) -[2023-10-14 14:34:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 57442304. Throughput: 0: 1691.2, 1: 1666.2. Samples: 14368022. Policy #0 lag: (min: 2.0, avg: 4.1, max: 33.0) -[2023-10-14 14:34:38,165][74987] Avg episode reward: [(0, '23.070'), (1, '26.750')] -[2023-10-14 14:34:38,817][75949] Updated weights for policy 0, policy_version 28071 (0.0009) -[2023-10-14 14:34:39,198][75949] Updated weights for policy 0, policy_version 28081 (0.0009) -[2023-10-14 14:34:39,571][75949] Updated weights for policy 0, policy_version 28091 (0.0008) -[2023-10-14 14:34:41,123][75950] Updated weights for policy 1, policy_version 28040 (0.0009) -[2023-10-14 14:34:41,483][75950] Updated weights for policy 1, policy_version 28050 (0.0011) -[2023-10-14 14:34:41,851][75950] Updated weights for policy 1, policy_version 28060 (0.0008) -[2023-10-14 14:34:43,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 57507840. Throughput: 0: 1691.7, 1: 1675.6. Samples: 14388302. Policy #0 lag: (min: 2.0, avg: 4.1, max: 33.0) -[2023-10-14 14:34:43,164][74987] Avg episode reward: [(0, '22.510'), (1, '25.920')] -[2023-10-14 14:34:43,170][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000028096_28770304.pth... -[2023-10-14 14:34:43,171][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000028064_28737536.pth... -[2023-10-14 14:34:43,202][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000026528_27164672.pth -[2023-10-14 14:34:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000026496_27131904.pth -[2023-10-14 14:34:43,556][75949] Updated weights for policy 0, policy_version 28101 (0.0008) -[2023-10-14 14:34:43,920][75949] Updated weights for policy 0, policy_version 28111 (0.0009) -[2023-10-14 14:34:44,296][75949] Updated weights for policy 0, policy_version 28121 (0.0009) -[2023-10-14 14:34:45,833][75950] Updated weights for policy 1, policy_version 28070 (0.0008) -[2023-10-14 14:34:46,199][75950] Updated weights for policy 1, policy_version 28080 (0.0009) -[2023-10-14 14:34:46,563][75950] Updated weights for policy 1, policy_version 28090 (0.0007) -[2023-10-14 14:34:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 57573376. Throughput: 0: 1691.6, 1: 1680.0. Samples: 14398568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:34:48,164][74987] Avg episode reward: [(0, '22.020'), (1, '25.150')] -[2023-10-14 14:34:48,525][75949] Updated weights for policy 0, policy_version 28131 (0.0007) -[2023-10-14 14:34:48,904][75949] Updated weights for policy 0, policy_version 28141 (0.0007) -[2023-10-14 14:34:49,285][75949] Updated weights for policy 0, policy_version 28151 (0.0009) -[2023-10-14 14:34:50,683][75950] Updated weights for policy 1, policy_version 28100 (0.0010) -[2023-10-14 14:34:51,061][75950] Updated weights for policy 1, policy_version 28110 (0.0008) -[2023-10-14 14:34:51,424][75950] Updated weights for policy 1, policy_version 28120 (0.0008) -[2023-10-14 14:34:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 57638912. Throughput: 0: 1688.6, 1: 1655.5. Samples: 14417856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:34:53,165][74987] Avg episode reward: [(0, '22.480'), (1, '26.360')] -[2023-10-14 14:34:53,316][75949] Updated weights for policy 0, policy_version 28161 (0.0010) -[2023-10-14 14:34:53,682][75949] Updated weights for policy 0, policy_version 28171 (0.0011) -[2023-10-14 14:34:54,055][75949] Updated weights for policy 0, policy_version 28181 (0.0009) -[2023-10-14 14:34:54,426][75949] Updated weights for policy 0, policy_version 28191 (0.0007) -[2023-10-14 14:34:55,374][75950] Updated weights for policy 1, policy_version 28130 (0.0007) -[2023-10-14 14:34:55,731][75950] Updated weights for policy 1, policy_version 28140 (0.0007) -[2023-10-14 14:34:56,093][75950] Updated weights for policy 1, policy_version 28150 (0.0008) -[2023-10-14 14:34:56,466][75950] Updated weights for policy 1, policy_version 28160 (0.0009) -[2023-10-14 14:34:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 57704448. Throughput: 0: 1685.7, 1: 1684.0. Samples: 14438644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:34:58,164][74987] Avg episode reward: [(0, '23.150'), (1, '25.490')] -[2023-10-14 14:34:58,380][75949] Updated weights for policy 0, policy_version 28201 (0.0009) -[2023-10-14 14:34:58,753][75949] Updated weights for policy 0, policy_version 28211 (0.0008) -[2023-10-14 14:34:59,132][75949] Updated weights for policy 0, policy_version 28221 (0.0009) -[2023-10-14 14:35:00,647][75950] Updated weights for policy 1, policy_version 28170 (0.0007) -[2023-10-14 14:35:01,023][75950] Updated weights for policy 1, policy_version 28180 (0.0011) -[2023-10-14 14:35:01,392][75950] Updated weights for policy 1, policy_version 28190 (0.0009) -[2023-10-14 14:35:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 57769984. Throughput: 0: 1686.3, 1: 1672.7. Samples: 14448492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:35:03,164][74987] Avg episode reward: [(0, '22.420'), (1, '25.750')] -[2023-10-14 14:35:03,308][75949] Updated weights for policy 0, policy_version 28231 (0.0008) -[2023-10-14 14:35:03,678][75949] Updated weights for policy 0, policy_version 28241 (0.0008) -[2023-10-14 14:35:04,046][75949] Updated weights for policy 0, policy_version 28251 (0.0008) -[2023-10-14 14:35:05,373][75950] Updated weights for policy 1, policy_version 28200 (0.0007) -[2023-10-14 14:35:05,743][75950] Updated weights for policy 1, policy_version 28210 (0.0010) -[2023-10-14 14:35:06,108][75950] Updated weights for policy 1, policy_version 28220 (0.0010) -[2023-10-14 14:35:07,949][75949] Updated weights for policy 0, policy_version 28261 (0.0007) -[2023-10-14 14:35:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 57835520. Throughput: 0: 1684.1, 1: 1663.3. Samples: 14468458. Policy #0 lag: (min: 26.0, avg: 28.6, max: 53.0) -[2023-10-14 14:35:08,164][74987] Avg episode reward: [(0, '24.770'), (1, '27.930')] -[2023-10-14 14:35:08,165][75801] Saving new best policy, reward=27.930! -[2023-10-14 14:35:08,314][75949] Updated weights for policy 0, policy_version 28271 (0.0009) -[2023-10-14 14:35:08,684][75949] Updated weights for policy 0, policy_version 28281 (0.0009) -[2023-10-14 14:35:10,075][75950] Updated weights for policy 1, policy_version 28230 (0.0009) -[2023-10-14 14:35:10,451][75950] Updated weights for policy 1, policy_version 28240 (0.0008) -[2023-10-14 14:35:10,815][75950] Updated weights for policy 1, policy_version 28250 (0.0009) -[2023-10-14 14:35:12,518][75949] Updated weights for policy 0, policy_version 28291 (0.0008) -[2023-10-14 14:35:12,892][75949] Updated weights for policy 0, policy_version 28301 (0.0007) -[2023-10-14 14:35:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 57901056. Throughput: 0: 1683.1, 1: 1685.6. Samples: 14489176. Policy #0 lag: (min: 26.0, avg: 28.6, max: 53.0) -[2023-10-14 14:35:13,165][74987] Avg episode reward: [(0, '24.530'), (1, '24.110')] -[2023-10-14 14:35:13,256][75949] Updated weights for policy 0, policy_version 28311 (0.0008) -[2023-10-14 14:35:14,985][75950] Updated weights for policy 1, policy_version 28260 (0.0008) -[2023-10-14 14:35:15,345][75950] Updated weights for policy 1, policy_version 28270 (0.0009) -[2023-10-14 14:35:15,714][75950] Updated weights for policy 1, policy_version 28280 (0.0010) -[2023-10-14 14:35:17,329][75949] Updated weights for policy 0, policy_version 28321 (0.0007) -[2023-10-14 14:35:17,692][75949] Updated weights for policy 0, policy_version 28331 (0.0009) -[2023-10-14 14:35:18,067][75949] Updated weights for policy 0, policy_version 28341 (0.0007) -[2023-10-14 14:35:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 57966592. Throughput: 0: 1689.4, 1: 1669.2. Samples: 14499068. Policy #0 lag: (min: 26.0, avg: 28.6, max: 53.0) -[2023-10-14 14:35:18,164][74987] Avg episode reward: [(0, '23.540'), (1, '26.250')] -[2023-10-14 14:35:18,441][75949] Updated weights for policy 0, policy_version 28351 (0.0008) -[2023-10-14 14:35:19,850][75950] Updated weights for policy 1, policy_version 28290 (0.0009) -[2023-10-14 14:35:20,224][75950] Updated weights for policy 1, policy_version 28300 (0.0009) -[2023-10-14 14:35:20,601][75950] Updated weights for policy 1, policy_version 28310 (0.0009) -[2023-10-14 14:35:20,963][75950] Updated weights for policy 1, policy_version 28320 (0.0009) -[2023-10-14 14:35:22,590][75949] Updated weights for policy 0, policy_version 28361 (0.0008) -[2023-10-14 14:35:22,968][75949] Updated weights for policy 0, policy_version 28371 (0.0010) -[2023-10-14 14:35:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 58032128. Throughput: 0: 1684.9, 1: 1676.3. Samples: 14519276. Policy #0 lag: (min: 26.0, avg: 28.6, max: 53.0) -[2023-10-14 14:35:23,164][74987] Avg episode reward: [(0, '22.000'), (1, '24.140')] -[2023-10-14 14:35:23,344][75949] Updated weights for policy 0, policy_version 28381 (0.0007) -[2023-10-14 14:35:25,117][75950] Updated weights for policy 1, policy_version 28330 (0.0010) -[2023-10-14 14:35:25,486][75950] Updated weights for policy 1, policy_version 28340 (0.0010) -[2023-10-14 14:35:25,868][75950] Updated weights for policy 1, policy_version 28350 (0.0009) -[2023-10-14 14:35:27,376][75949] Updated weights for policy 0, policy_version 28391 (0.0009) -[2023-10-14 14:35:27,739][75949] Updated weights for policy 0, policy_version 28401 (0.0011) -[2023-10-14 14:35:28,113][75949] Updated weights for policy 0, policy_version 28411 (0.0010) -[2023-10-14 14:35:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 58097664. Throughput: 0: 1669.8, 1: 1683.9. Samples: 14539218. Policy #0 lag: (min: 26.0, avg: 28.6, max: 53.0) -[2023-10-14 14:35:28,164][74987] Avg episode reward: [(0, '22.720'), (1, '24.550')] -[2023-10-14 14:35:29,971][75950] Updated weights for policy 1, policy_version 28360 (0.0008) -[2023-10-14 14:35:30,339][75950] Updated weights for policy 1, policy_version 28370 (0.0008) -[2023-10-14 14:35:30,708][75950] Updated weights for policy 1, policy_version 28380 (0.0007) -[2023-10-14 14:35:32,250][75949] Updated weights for policy 0, policy_version 28421 (0.0009) -[2023-10-14 14:35:32,621][75949] Updated weights for policy 0, policy_version 28431 (0.0009) -[2023-10-14 14:35:32,990][75949] Updated weights for policy 0, policy_version 28441 (0.0008) -[2023-10-14 14:35:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 58163200. Throughput: 0: 1688.3, 1: 1663.4. Samples: 14549396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:35:33,164][74987] Avg episode reward: [(0, '21.280'), (1, '26.760')] -[2023-10-14 14:35:34,681][75950] Updated weights for policy 1, policy_version 28390 (0.0008) -[2023-10-14 14:35:35,043][75950] Updated weights for policy 1, policy_version 28400 (0.0010) -[2023-10-14 14:35:35,415][75950] Updated weights for policy 1, policy_version 28410 (0.0007) -[2023-10-14 14:35:37,280][75949] Updated weights for policy 0, policy_version 28451 (0.0008) -[2023-10-14 14:35:37,678][75949] Updated weights for policy 0, policy_version 28461 (0.0007) -[2023-10-14 14:35:38,045][75949] Updated weights for policy 0, policy_version 28471 (0.0007) -[2023-10-14 14:35:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 58228736. Throughput: 0: 1690.0, 1: 1681.5. Samples: 14569574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:35:38,165][74987] Avg episode reward: [(0, '23.350'), (1, '24.510')] -[2023-10-14 14:35:39,649][75950] Updated weights for policy 1, policy_version 28420 (0.0008) -[2023-10-14 14:35:40,019][75950] Updated weights for policy 1, policy_version 28430 (0.0009) -[2023-10-14 14:35:40,378][75950] Updated weights for policy 1, policy_version 28440 (0.0008) -[2023-10-14 14:35:41,956][75949] Updated weights for policy 0, policy_version 28481 (0.0007) -[2023-10-14 14:35:42,329][75949] Updated weights for policy 0, policy_version 28491 (0.0007) -[2023-10-14 14:35:42,704][75949] Updated weights for policy 0, policy_version 28501 (0.0008) -[2023-10-14 14:35:43,074][75949] Updated weights for policy 0, policy_version 28511 (0.0007) -[2023-10-14 14:35:43,164][74987] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 58327040. Throughput: 0: 1668.6, 1: 1677.2. Samples: 14589204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:35:43,165][74987] Avg episode reward: [(0, '23.720'), (1, '24.360')] -[2023-10-14 14:35:44,556][75950] Updated weights for policy 1, policy_version 28450 (0.0009) -[2023-10-14 14:35:44,922][75950] Updated weights for policy 1, policy_version 28460 (0.0009) -[2023-10-14 14:35:45,286][75950] Updated weights for policy 1, policy_version 28470 (0.0008) -[2023-10-14 14:35:45,654][75950] Updated weights for policy 1, policy_version 28480 (0.0007) -[2023-10-14 14:35:47,265][75949] Updated weights for policy 0, policy_version 28521 (0.0007) -[2023-10-14 14:35:47,638][75949] Updated weights for policy 0, policy_version 28531 (0.0008) -[2023-10-14 14:35:48,004][75949] Updated weights for policy 0, policy_version 28541 (0.0010) -[2023-10-14 14:35:48,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 58392576. Throughput: 0: 1686.2, 1: 1661.2. Samples: 14599122. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 14:35:48,164][74987] Avg episode reward: [(0, '23.930'), (1, '26.290')] -[2023-10-14 14:35:49,827][75950] Updated weights for policy 1, policy_version 28490 (0.0011) -[2023-10-14 14:35:50,201][75950] Updated weights for policy 1, policy_version 28500 (0.0010) -[2023-10-14 14:35:50,579][75950] Updated weights for policy 1, policy_version 28510 (0.0009) -[2023-10-14 14:35:52,226][75949] Updated weights for policy 0, policy_version 28551 (0.0008) -[2023-10-14 14:35:52,602][75949] Updated weights for policy 0, policy_version 28561 (0.0010) -[2023-10-14 14:35:52,975][75949] Updated weights for policy 0, policy_version 28571 (0.0009) -[2023-10-14 14:35:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 58458112. Throughput: 0: 1681.0, 1: 1673.6. Samples: 14619416. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 14:35:53,164][74987] Avg episode reward: [(0, '24.220'), (1, '24.660')] -[2023-10-14 14:35:54,773][75950] Updated weights for policy 1, policy_version 28520 (0.0007) -[2023-10-14 14:35:55,153][75950] Updated weights for policy 1, policy_version 28530 (0.0007) -[2023-10-14 14:35:55,519][75950] Updated weights for policy 1, policy_version 28540 (0.0009) -[2023-10-14 14:35:56,931][75949] Updated weights for policy 0, policy_version 28581 (0.0009) -[2023-10-14 14:35:57,293][75949] Updated weights for policy 0, policy_version 28591 (0.0008) -[2023-10-14 14:35:57,664][75949] Updated weights for policy 0, policy_version 28601 (0.0007) -[2023-10-14 14:35:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 58523648. Throughput: 0: 1665.7, 1: 1663.4. Samples: 14638986. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 14:35:58,164][74987] Avg episode reward: [(0, '21.860'), (1, '26.560')] -[2023-10-14 14:35:59,431][75950] Updated weights for policy 1, policy_version 28550 (0.0008) -[2023-10-14 14:35:59,801][75950] Updated weights for policy 1, policy_version 28560 (0.0009) -[2023-10-14 14:36:00,176][75950] Updated weights for policy 1, policy_version 28570 (0.0010) -[2023-10-14 14:36:01,602][75949] Updated weights for policy 0, policy_version 28611 (0.0007) -[2023-10-14 14:36:01,972][75949] Updated weights for policy 0, policy_version 28621 (0.0008) -[2023-10-14 14:36:02,338][75949] Updated weights for policy 0, policy_version 28631 (0.0009) -[2023-10-14 14:36:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 58589184. Throughput: 0: 1685.2, 1: 1653.4. Samples: 14649306. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 14:36:03,165][74987] Avg episode reward: [(0, '24.250'), (1, '27.100')] -[2023-10-14 14:36:04,322][75950] Updated weights for policy 1, policy_version 28580 (0.0010) -[2023-10-14 14:36:04,683][75950] Updated weights for policy 1, policy_version 28590 (0.0010) -[2023-10-14 14:36:05,058][75950] Updated weights for policy 1, policy_version 28600 (0.0009) -[2023-10-14 14:36:06,335][75949] Updated weights for policy 0, policy_version 28641 (0.0007) -[2023-10-14 14:36:06,718][75949] Updated weights for policy 0, policy_version 28651 (0.0008) -[2023-10-14 14:36:07,078][75949] Updated weights for policy 0, policy_version 28661 (0.0008) -[2023-10-14 14:36:07,457][75949] Updated weights for policy 0, policy_version 28671 (0.0008) -[2023-10-14 14:36:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 58654720. Throughput: 0: 1683.2, 1: 1661.9. Samples: 14669808. Policy #0 lag: (min: 3.0, avg: 10.9, max: 35.0) -[2023-10-14 14:36:08,165][74987] Avg episode reward: [(0, '21.850'), (1, '24.960')] -[2023-10-14 14:36:09,351][75950] Updated weights for policy 1, policy_version 28610 (0.0010) -[2023-10-14 14:36:09,718][75950] Updated weights for policy 1, policy_version 28620 (0.0008) -[2023-10-14 14:36:10,086][75950] Updated weights for policy 1, policy_version 28630 (0.0008) -[2023-10-14 14:36:10,449][75950] Updated weights for policy 1, policy_version 28640 (0.0007) -[2023-10-14 14:36:11,432][75949] Updated weights for policy 0, policy_version 28681 (0.0007) -[2023-10-14 14:36:11,795][75949] Updated weights for policy 0, policy_version 28691 (0.0008) -[2023-10-14 14:36:12,173][75949] Updated weights for policy 0, policy_version 28701 (0.0008) -[2023-10-14 14:36:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 58720256. Throughput: 0: 1676.7, 1: 1659.7. Samples: 14689356. Policy #0 lag: (min: 3.0, avg: 10.9, max: 35.0) -[2023-10-14 14:36:13,164][74987] Avg episode reward: [(0, '24.150'), (1, '24.130')] -[2023-10-14 14:36:14,577][75950] Updated weights for policy 1, policy_version 28650 (0.0007) -[2023-10-14 14:36:14,943][75950] Updated weights for policy 1, policy_version 28660 (0.0009) -[2023-10-14 14:36:15,317][75950] Updated weights for policy 1, policy_version 28670 (0.0008) -[2023-10-14 14:36:16,187][75949] Updated weights for policy 0, policy_version 28711 (0.0008) -[2023-10-14 14:36:16,550][75949] Updated weights for policy 0, policy_version 28721 (0.0011) -[2023-10-14 14:36:16,917][75949] Updated weights for policy 0, policy_version 28731 (0.0010) -[2023-10-14 14:36:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 58785792. Throughput: 0: 1687.9, 1: 1653.6. Samples: 14699762. Policy #0 lag: (min: 3.0, avg: 10.9, max: 35.0) -[2023-10-14 14:36:18,165][74987] Avg episode reward: [(0, '23.920'), (1, '25.040')] -[2023-10-14 14:36:19,401][75950] Updated weights for policy 1, policy_version 28680 (0.0010) -[2023-10-14 14:36:19,772][75950] Updated weights for policy 1, policy_version 28690 (0.0010) -[2023-10-14 14:36:20,141][75950] Updated weights for policy 1, policy_version 28700 (0.0008) -[2023-10-14 14:36:20,873][75949] Updated weights for policy 0, policy_version 28741 (0.0008) -[2023-10-14 14:36:21,254][75949] Updated weights for policy 0, policy_version 28751 (0.0007) -[2023-10-14 14:36:21,630][75949] Updated weights for policy 0, policy_version 28761 (0.0007) -[2023-10-14 14:36:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 58851328. Throughput: 0: 1670.1, 1: 1660.8. Samples: 14719464. Policy #0 lag: (min: 3.0, avg: 10.9, max: 35.0) -[2023-10-14 14:36:23,165][74987] Avg episode reward: [(0, '23.110'), (1, '25.580')] -[2023-10-14 14:36:24,258][75950] Updated weights for policy 1, policy_version 28710 (0.0007) -[2023-10-14 14:36:24,617][75950] Updated weights for policy 1, policy_version 28720 (0.0007) -[2023-10-14 14:36:24,991][75950] Updated weights for policy 1, policy_version 28730 (0.0007) -[2023-10-14 14:36:25,528][75949] Updated weights for policy 0, policy_version 28771 (0.0007) -[2023-10-14 14:36:25,903][75949] Updated weights for policy 0, policy_version 28781 (0.0008) -[2023-10-14 14:36:26,269][75949] Updated weights for policy 0, policy_version 28791 (0.0009) -[2023-10-14 14:36:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.5). Total num frames: 58916864. Throughput: 0: 1686.3, 1: 1663.4. Samples: 14739940. Policy #0 lag: (min: 24.0, avg: 49.8, max: 56.0) -[2023-10-14 14:36:28,164][74987] Avg episode reward: [(0, '23.790'), (1, '25.020')] -[2023-10-14 14:36:29,112][75950] Updated weights for policy 1, policy_version 28740 (0.0008) -[2023-10-14 14:36:29,482][75950] Updated weights for policy 1, policy_version 28750 (0.0008) -[2023-10-14 14:36:29,846][75950] Updated weights for policy 1, policy_version 28760 (0.0008) -[2023-10-14 14:36:30,396][75949] Updated weights for policy 0, policy_version 28801 (0.0009) -[2023-10-14 14:36:30,778][75949] Updated weights for policy 0, policy_version 28811 (0.0009) -[2023-10-14 14:36:31,149][75949] Updated weights for policy 0, policy_version 28821 (0.0008) -[2023-10-14 14:36:31,522][75949] Updated weights for policy 0, policy_version 28831 (0.0008) -[2023-10-14 14:36:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 58982400. Throughput: 0: 1691.7, 1: 1660.1. Samples: 14749956. Policy #0 lag: (min: 24.0, avg: 49.8, max: 56.0) -[2023-10-14 14:36:33,164][74987] Avg episode reward: [(0, '22.660'), (1, '26.710')] -[2023-10-14 14:36:33,950][75950] Updated weights for policy 1, policy_version 28770 (0.0008) -[2023-10-14 14:36:34,327][75950] Updated weights for policy 1, policy_version 28780 (0.0009) -[2023-10-14 14:36:34,694][75950] Updated weights for policy 1, policy_version 28790 (0.0008) -[2023-10-14 14:36:35,065][75950] Updated weights for policy 1, policy_version 28800 (0.0007) -[2023-10-14 14:36:35,512][75949] Updated weights for policy 0, policy_version 28841 (0.0007) -[2023-10-14 14:36:35,882][75949] Updated weights for policy 0, policy_version 28851 (0.0009) -[2023-10-14 14:36:36,257][75949] Updated weights for policy 0, policy_version 28861 (0.0008) -[2023-10-14 14:36:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 59047936. Throughput: 0: 1678.9, 1: 1669.1. Samples: 14770078. Policy #0 lag: (min: 24.0, avg: 49.8, max: 56.0) -[2023-10-14 14:36:38,164][74987] Avg episode reward: [(0, '22.390'), (1, '25.660')] -[2023-10-14 14:36:39,201][75950] Updated weights for policy 1, policy_version 28810 (0.0007) -[2023-10-14 14:36:39,571][75950] Updated weights for policy 1, policy_version 28820 (0.0009) -[2023-10-14 14:36:39,941][75950] Updated weights for policy 1, policy_version 28830 (0.0011) -[2023-10-14 14:36:40,439][75949] Updated weights for policy 0, policy_version 28871 (0.0011) -[2023-10-14 14:36:40,811][75949] Updated weights for policy 0, policy_version 28881 (0.0010) -[2023-10-14 14:36:41,174][75949] Updated weights for policy 0, policy_version 28891 (0.0011) -[2023-10-14 14:36:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 59113472. Throughput: 0: 1700.0, 1: 1671.9. Samples: 14790722. Policy #0 lag: (min: 24.0, avg: 49.8, max: 56.0) -[2023-10-14 14:36:43,165][74987] Avg episode reward: [(0, '24.260'), (1, '25.340')] -[2023-10-14 14:36:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000028896_29589504.pth... -[2023-10-14 14:36:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000028832_29523968.pth... -[2023-10-14 14:36:43,212][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000027296_27951104.pth -[2023-10-14 14:36:43,218][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000027328_27983872.pth -[2023-10-14 14:36:43,851][75950] Updated weights for policy 1, policy_version 28840 (0.0010) -[2023-10-14 14:36:44,215][75950] Updated weights for policy 1, policy_version 28850 (0.0008) -[2023-10-14 14:36:44,588][75950] Updated weights for policy 1, policy_version 28860 (0.0007) -[2023-10-14 14:36:45,217][75949] Updated weights for policy 0, policy_version 28901 (0.0008) -[2023-10-14 14:36:45,584][75949] Updated weights for policy 0, policy_version 28911 (0.0008) -[2023-10-14 14:36:45,955][75949] Updated weights for policy 0, policy_version 28921 (0.0009) -[2023-10-14 14:36:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 59179008. Throughput: 0: 1688.3, 1: 1670.3. Samples: 14800442. Policy #0 lag: (min: 24.0, avg: 49.8, max: 56.0) -[2023-10-14 14:36:48,164][74987] Avg episode reward: [(0, '23.930'), (1, '26.450')] -[2023-10-14 14:36:48,726][75950] Updated weights for policy 1, policy_version 28870 (0.0009) -[2023-10-14 14:36:49,096][75950] Updated weights for policy 1, policy_version 28880 (0.0008) -[2023-10-14 14:36:49,461][75950] Updated weights for policy 1, policy_version 28890 (0.0008) -[2023-10-14 14:36:50,109][75949] Updated weights for policy 0, policy_version 28931 (0.0010) -[2023-10-14 14:36:50,486][75949] Updated weights for policy 0, policy_version 28941 (0.0008) -[2023-10-14 14:36:50,848][75949] Updated weights for policy 0, policy_version 28951 (0.0009) -[2023-10-14 14:36:53,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 59244544. Throughput: 0: 1675.4, 1: 1671.5. Samples: 14820416. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:36:53,164][74987] Avg episode reward: [(0, '23.120'), (1, '24.870')] -[2023-10-14 14:36:53,633][75950] Updated weights for policy 1, policy_version 28900 (0.0008) -[2023-10-14 14:36:54,005][75950] Updated weights for policy 1, policy_version 28910 (0.0010) -[2023-10-14 14:36:54,379][75950] Updated weights for policy 1, policy_version 28920 (0.0009) -[2023-10-14 14:36:54,780][75949] Updated weights for policy 0, policy_version 28961 (0.0009) -[2023-10-14 14:36:55,156][75949] Updated weights for policy 0, policy_version 28971 (0.0009) -[2023-10-14 14:36:55,533][75949] Updated weights for policy 0, policy_version 28981 (0.0008) -[2023-10-14 14:36:55,905][75949] Updated weights for policy 0, policy_version 28991 (0.0008) -[2023-10-14 14:36:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59310080. Throughput: 0: 1693.7, 1: 1678.2. Samples: 14841090. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:36:58,164][74987] Avg episode reward: [(0, '24.200'), (1, '26.890')] -[2023-10-14 14:36:58,389][75950] Updated weights for policy 1, policy_version 28930 (0.0008) -[2023-10-14 14:36:58,763][75950] Updated weights for policy 1, policy_version 28940 (0.0009) -[2023-10-14 14:36:59,132][75950] Updated weights for policy 1, policy_version 28950 (0.0009) -[2023-10-14 14:36:59,508][75950] Updated weights for policy 1, policy_version 28960 (0.0008) -[2023-10-14 14:36:59,841][75949] Updated weights for policy 0, policy_version 29001 (0.0007) -[2023-10-14 14:37:00,212][75949] Updated weights for policy 0, policy_version 29011 (0.0009) -[2023-10-14 14:37:00,572][75949] Updated weights for policy 0, policy_version 29021 (0.0008) -[2023-10-14 14:37:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59375616. Throughput: 0: 1668.1, 1: 1676.8. Samples: 14850282. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:37:03,165][74987] Avg episode reward: [(0, '22.890'), (1, '25.660')] -[2023-10-14 14:37:03,559][75950] Updated weights for policy 1, policy_version 28970 (0.0007) -[2023-10-14 14:37:03,924][75950] Updated weights for policy 1, policy_version 28980 (0.0007) -[2023-10-14 14:37:04,278][75950] Updated weights for policy 1, policy_version 28990 (0.0007) -[2023-10-14 14:37:04,651][75949] Updated weights for policy 0, policy_version 29031 (0.0008) -[2023-10-14 14:37:05,031][75949] Updated weights for policy 0, policy_version 29041 (0.0009) -[2023-10-14 14:37:05,388][75949] Updated weights for policy 0, policy_version 29051 (0.0009) -[2023-10-14 14:37:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59441152. Throughput: 0: 1685.0, 1: 1678.4. Samples: 14870820. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:37:08,165][74987] Avg episode reward: [(0, '24.220'), (1, '25.800')] -[2023-10-14 14:37:08,460][75950] Updated weights for policy 1, policy_version 29000 (0.0007) -[2023-10-14 14:37:08,823][75950] Updated weights for policy 1, policy_version 29010 (0.0009) -[2023-10-14 14:37:09,189][75950] Updated weights for policy 1, policy_version 29020 (0.0010) -[2023-10-14 14:37:09,632][75949] Updated weights for policy 0, policy_version 29061 (0.0009) -[2023-10-14 14:37:10,015][75949] Updated weights for policy 0, policy_version 29071 (0.0009) -[2023-10-14 14:37:10,399][75949] Updated weights for policy 0, policy_version 29081 (0.0008) -[2023-10-14 14:37:13,156][75950] Updated weights for policy 1, policy_version 29030 (0.0007) -[2023-10-14 14:37:13,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59506688. Throughput: 0: 1688.9, 1: 1677.5. Samples: 14891428. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:37:13,164][74987] Avg episode reward: [(0, '23.350'), (1, '28.400')] -[2023-10-14 14:37:13,536][75950] Updated weights for policy 1, policy_version 29040 (0.0008) -[2023-10-14 14:37:13,891][75950] Updated weights for policy 1, policy_version 29050 (0.0010) -[2023-10-14 14:37:14,112][75801] Saving new best policy, reward=28.400! -[2023-10-14 14:37:14,440][75949] Updated weights for policy 0, policy_version 29091 (0.0010) -[2023-10-14 14:37:14,819][75949] Updated weights for policy 0, policy_version 29101 (0.0009) -[2023-10-14 14:37:15,186][75949] Updated weights for policy 0, policy_version 29111 (0.0009) -[2023-10-14 14:37:17,992][75950] Updated weights for policy 1, policy_version 29060 (0.0009) -[2023-10-14 14:37:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59572224. Throughput: 0: 1667.2, 1: 1679.4. Samples: 14900550. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-14 14:37:18,164][74987] Avg episode reward: [(0, '23.730'), (1, '24.460')] -[2023-10-14 14:37:18,364][75950] Updated weights for policy 1, policy_version 29070 (0.0008) -[2023-10-14 14:37:18,743][75950] Updated weights for policy 1, policy_version 29080 (0.0010) -[2023-10-14 14:37:19,316][75949] Updated weights for policy 0, policy_version 29121 (0.0008) -[2023-10-14 14:37:19,694][75949] Updated weights for policy 0, policy_version 29131 (0.0009) -[2023-10-14 14:37:20,065][75949] Updated weights for policy 0, policy_version 29141 (0.0008) -[2023-10-14 14:37:20,438][75949] Updated weights for policy 0, policy_version 29151 (0.0007) -[2023-10-14 14:37:22,890][75950] Updated weights for policy 1, policy_version 29090 (0.0009) -[2023-10-14 14:37:23,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59637760. Throughput: 0: 1684.3, 1: 1670.9. Samples: 14921064. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-14 14:37:23,164][74987] Avg episode reward: [(0, '23.950'), (1, '26.380')] -[2023-10-14 14:37:23,261][75950] Updated weights for policy 1, policy_version 29100 (0.0008) -[2023-10-14 14:37:23,619][75950] Updated weights for policy 1, policy_version 29110 (0.0007) -[2023-10-14 14:37:23,992][75950] Updated weights for policy 1, policy_version 29120 (0.0008) -[2023-10-14 14:37:24,535][75949] Updated weights for policy 0, policy_version 29161 (0.0007) -[2023-10-14 14:37:24,913][75949] Updated weights for policy 0, policy_version 29171 (0.0009) -[2023-10-14 14:37:25,277][75949] Updated weights for policy 0, policy_version 29181 (0.0007) -[2023-10-14 14:37:28,124][75950] Updated weights for policy 1, policy_version 29130 (0.0007) -[2023-10-14 14:37:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59703296. Throughput: 0: 1678.2, 1: 1672.0. Samples: 14941480. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-14 14:37:28,164][74987] Avg episode reward: [(0, '22.540'), (1, '25.350')] -[2023-10-14 14:37:28,492][75950] Updated weights for policy 1, policy_version 29140 (0.0008) -[2023-10-14 14:37:28,860][75950] Updated weights for policy 1, policy_version 29150 (0.0007) -[2023-10-14 14:37:29,317][75949] Updated weights for policy 0, policy_version 29191 (0.0008) -[2023-10-14 14:37:29,681][75949] Updated weights for policy 0, policy_version 29201 (0.0007) -[2023-10-14 14:37:30,051][75949] Updated weights for policy 0, policy_version 29211 (0.0009) -[2023-10-14 14:37:33,002][75950] Updated weights for policy 1, policy_version 29160 (0.0007) -[2023-10-14 14:37:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59768832. Throughput: 0: 1669.0, 1: 1669.9. Samples: 14950692. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-14 14:37:33,165][74987] Avg episode reward: [(0, '22.940'), (1, '23.690')] -[2023-10-14 14:37:33,373][75950] Updated weights for policy 1, policy_version 29170 (0.0008) -[2023-10-14 14:37:33,746][75950] Updated weights for policy 1, policy_version 29180 (0.0008) -[2023-10-14 14:37:34,027][75949] Updated weights for policy 0, policy_version 29221 (0.0009) -[2023-10-14 14:37:34,393][75949] Updated weights for policy 0, policy_version 29231 (0.0010) -[2023-10-14 14:37:34,764][75949] Updated weights for policy 0, policy_version 29241 (0.0008) -[2023-10-14 14:37:37,689][75950] Updated weights for policy 1, policy_version 29190 (0.0009) -[2023-10-14 14:37:38,051][75950] Updated weights for policy 1, policy_version 29200 (0.0008) -[2023-10-14 14:37:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59834368. Throughput: 0: 1688.5, 1: 1674.9. Samples: 14971770. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) -[2023-10-14 14:37:38,164][74987] Avg episode reward: [(0, '22.150'), (1, '26.170')] -[2023-10-14 14:37:38,419][75950] Updated weights for policy 1, policy_version 29210 (0.0008) -[2023-10-14 14:37:38,821][75949] Updated weights for policy 0, policy_version 29251 (0.0009) -[2023-10-14 14:37:39,198][75949] Updated weights for policy 0, policy_version 29261 (0.0008) -[2023-10-14 14:37:39,565][75949] Updated weights for policy 0, policy_version 29271 (0.0011) -[2023-10-14 14:37:42,534][75950] Updated weights for policy 1, policy_version 29220 (0.0009) -[2023-10-14 14:37:42,910][75950] Updated weights for policy 1, policy_version 29230 (0.0008) -[2023-10-14 14:37:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59899904. Throughput: 0: 1694.1, 1: 1667.2. Samples: 14992350. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 14:37:43,165][74987] Avg episode reward: [(0, '24.690'), (1, '23.580')] -[2023-10-14 14:37:43,266][75950] Updated weights for policy 1, policy_version 29240 (0.0008) -[2023-10-14 14:37:43,546][75949] Updated weights for policy 0, policy_version 29281 (0.0007) -[2023-10-14 14:37:43,917][75949] Updated weights for policy 0, policy_version 29291 (0.0009) -[2023-10-14 14:37:44,297][75949] Updated weights for policy 0, policy_version 29301 (0.0008) -[2023-10-14 14:37:44,660][75949] Updated weights for policy 0, policy_version 29311 (0.0008) -[2023-10-14 14:37:47,451][75950] Updated weights for policy 1, policy_version 29250 (0.0008) -[2023-10-14 14:37:47,805][75950] Updated weights for policy 1, policy_version 29260 (0.0010) -[2023-10-14 14:37:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 59965440. Throughput: 0: 1693.2, 1: 1674.7. Samples: 15001840. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 14:37:48,164][74987] Avg episode reward: [(0, '22.300'), (1, '25.590')] -[2023-10-14 14:37:48,169][75950] Updated weights for policy 1, policy_version 29270 (0.0010) -[2023-10-14 14:37:48,539][75950] Updated weights for policy 1, policy_version 29280 (0.0009) -[2023-10-14 14:37:48,704][75949] Updated weights for policy 0, policy_version 29321 (0.0008) -[2023-10-14 14:37:49,080][75949] Updated weights for policy 0, policy_version 29331 (0.0009) -[2023-10-14 14:37:49,448][75949] Updated weights for policy 0, policy_version 29341 (0.0008) -[2023-10-14 14:37:52,545][75950] Updated weights for policy 1, policy_version 29290 (0.0007) -[2023-10-14 14:37:52,909][75950] Updated weights for policy 1, policy_version 29300 (0.0008) -[2023-10-14 14:37:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60030976. Throughput: 0: 1698.4, 1: 1670.8. Samples: 15022434. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 14:37:53,165][74987] Avg episode reward: [(0, '23.650'), (1, '26.980')] -[2023-10-14 14:37:53,276][75950] Updated weights for policy 1, policy_version 29310 (0.0009) -[2023-10-14 14:37:53,487][75949] Updated weights for policy 0, policy_version 29351 (0.0009) -[2023-10-14 14:37:53,863][75949] Updated weights for policy 0, policy_version 29361 (0.0008) -[2023-10-14 14:37:54,225][75949] Updated weights for policy 0, policy_version 29371 (0.0009) -[2023-10-14 14:37:57,505][75950] Updated weights for policy 1, policy_version 29320 (0.0009) -[2023-10-14 14:37:57,867][75950] Updated weights for policy 1, policy_version 29330 (0.0009) -[2023-10-14 14:37:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60096512. Throughput: 0: 1695.8, 1: 1662.1. Samples: 15042536. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 14:37:58,164][74987] Avg episode reward: [(0, '23.020'), (1, '25.150')] -[2023-10-14 14:37:58,233][75950] Updated weights for policy 1, policy_version 29340 (0.0008) -[2023-10-14 14:37:58,340][75949] Updated weights for policy 0, policy_version 29381 (0.0009) -[2023-10-14 14:37:58,722][75949] Updated weights for policy 0, policy_version 29391 (0.0007) -[2023-10-14 14:37:59,091][75949] Updated weights for policy 0, policy_version 29401 (0.0007) -[2023-10-14 14:38:02,286][75950] Updated weights for policy 1, policy_version 29350 (0.0009) -[2023-10-14 14:38:02,662][75950] Updated weights for policy 1, policy_version 29360 (0.0009) -[2023-10-14 14:38:03,020][75950] Updated weights for policy 1, policy_version 29370 (0.0008) -[2023-10-14 14:38:03,091][75949] Updated weights for policy 0, policy_version 29411 (0.0009) -[2023-10-14 14:38:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60162048. Throughput: 0: 1693.2, 1: 1672.9. Samples: 15052026. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 14:38:03,164][74987] Avg episode reward: [(0, '24.290'), (1, '26.710')] -[2023-10-14 14:38:03,472][75949] Updated weights for policy 0, policy_version 29421 (0.0008) -[2023-10-14 14:38:03,842][75949] Updated weights for policy 0, policy_version 29431 (0.0008) -[2023-10-14 14:38:06,972][75950] Updated weights for policy 1, policy_version 29380 (0.0009) -[2023-10-14 14:38:07,333][75950] Updated weights for policy 1, policy_version 29390 (0.0010) -[2023-10-14 14:38:07,699][75950] Updated weights for policy 1, policy_version 29400 (0.0007) -[2023-10-14 14:38:07,966][75949] Updated weights for policy 0, policy_version 29441 (0.0010) -[2023-10-14 14:38:08,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 60260352. Throughput: 0: 1687.5, 1: 1678.1. Samples: 15072514. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:38:08,165][74987] Avg episode reward: [(0, '23.560'), (1, '25.560')] -[2023-10-14 14:38:08,328][75949] Updated weights for policy 0, policy_version 29451 (0.0010) -[2023-10-14 14:38:08,697][75949] Updated weights for policy 0, policy_version 29461 (0.0010) -[2023-10-14 14:38:09,064][75949] Updated weights for policy 0, policy_version 29471 (0.0010) -[2023-10-14 14:38:11,679][75950] Updated weights for policy 1, policy_version 29410 (0.0008) -[2023-10-14 14:38:12,046][75950] Updated weights for policy 1, policy_version 29420 (0.0007) -[2023-10-14 14:38:12,413][75950] Updated weights for policy 1, policy_version 29430 (0.0007) -[2023-10-14 14:38:12,782][75950] Updated weights for policy 1, policy_version 29440 (0.0008) -[2023-10-14 14:38:13,055][75949] Updated weights for policy 0, policy_version 29481 (0.0008) -[2023-10-14 14:38:13,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 60325888. Throughput: 0: 1689.1, 1: 1660.0. Samples: 15092190. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:38:13,164][74987] Avg episode reward: [(0, '23.010'), (1, '24.690')] -[2023-10-14 14:38:13,425][75949] Updated weights for policy 0, policy_version 29491 (0.0008) -[2023-10-14 14:38:13,796][75949] Updated weights for policy 0, policy_version 29501 (0.0008) -[2023-10-14 14:38:16,978][75950] Updated weights for policy 1, policy_version 29450 (0.0009) -[2023-10-14 14:38:17,358][75950] Updated weights for policy 1, policy_version 29460 (0.0011) -[2023-10-14 14:38:17,725][75950] Updated weights for policy 1, policy_version 29470 (0.0008) -[2023-10-14 14:38:17,920][75949] Updated weights for policy 0, policy_version 29511 (0.0009) -[2023-10-14 14:38:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 60391424. Throughput: 0: 1684.9, 1: 1687.8. Samples: 15102462. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:38:18,165][74987] Avg episode reward: [(0, '25.250'), (1, '26.790')] -[2023-10-14 14:38:18,297][75949] Updated weights for policy 0, policy_version 29521 (0.0008) -[2023-10-14 14:38:18,661][75949] Updated weights for policy 0, policy_version 29531 (0.0009) -[2023-10-14 14:38:18,848][75615] Saving new best policy, reward=25.250! -[2023-10-14 14:38:21,706][75950] Updated weights for policy 1, policy_version 29480 (0.0010) -[2023-10-14 14:38:22,075][75950] Updated weights for policy 1, policy_version 29490 (0.0009) -[2023-10-14 14:38:22,439][75950] Updated weights for policy 1, policy_version 29500 (0.0008) -[2023-10-14 14:38:22,715][75949] Updated weights for policy 0, policy_version 29541 (0.0008) -[2023-10-14 14:38:23,080][75949] Updated weights for policy 0, policy_version 29551 (0.0009) -[2023-10-14 14:38:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 60456960. Throughput: 0: 1683.9, 1: 1673.9. Samples: 15122868. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:38:23,164][74987] Avg episode reward: [(0, '22.510'), (1, '25.870')] -[2023-10-14 14:38:23,458][75949] Updated weights for policy 0, policy_version 29561 (0.0008) -[2023-10-14 14:38:26,668][75950] Updated weights for policy 1, policy_version 29510 (0.0008) -[2023-10-14 14:38:27,049][75950] Updated weights for policy 1, policy_version 29520 (0.0010) -[2023-10-14 14:38:27,413][75950] Updated weights for policy 1, policy_version 29530 (0.0010) -[2023-10-14 14:38:27,605][75949] Updated weights for policy 0, policy_version 29571 (0.0008) -[2023-10-14 14:38:27,984][75949] Updated weights for policy 0, policy_version 29581 (0.0007) -[2023-10-14 14:38:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 60522496. Throughput: 0: 1674.7, 1: 1659.0. Samples: 15142368. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:38:28,164][74987] Avg episode reward: [(0, '24.560'), (1, '25.840')] -[2023-10-14 14:38:28,346][75949] Updated weights for policy 0, policy_version 29591 (0.0009) -[2023-10-14 14:38:31,395][75950] Updated weights for policy 1, policy_version 29540 (0.0009) -[2023-10-14 14:38:31,774][75950] Updated weights for policy 1, policy_version 29550 (0.0009) -[2023-10-14 14:38:32,138][75950] Updated weights for policy 1, policy_version 29560 (0.0007) -[2023-10-14 14:38:32,282][75949] Updated weights for policy 0, policy_version 29601 (0.0008) -[2023-10-14 14:38:32,650][75949] Updated weights for policy 0, policy_version 29611 (0.0009) -[2023-10-14 14:38:33,020][75949] Updated weights for policy 0, policy_version 29621 (0.0011) -[2023-10-14 14:38:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 60588032. Throughput: 0: 1675.2, 1: 1681.5. Samples: 15152890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 14:38:33,164][74987] Avg episode reward: [(0, '23.490'), (1, '25.640')] -[2023-10-14 14:38:33,381][75949] Updated weights for policy 0, policy_version 29631 (0.0010) -[2023-10-14 14:38:36,225][75950] Updated weights for policy 1, policy_version 29570 (0.0007) -[2023-10-14 14:38:36,591][75950] Updated weights for policy 1, policy_version 29580 (0.0007) -[2023-10-14 14:38:36,954][75950] Updated weights for policy 1, policy_version 29590 (0.0008) -[2023-10-14 14:38:37,322][75950] Updated weights for policy 1, policy_version 29600 (0.0009) -[2023-10-14 14:38:37,473][75949] Updated weights for policy 0, policy_version 29641 (0.0007) -[2023-10-14 14:38:37,841][75949] Updated weights for policy 0, policy_version 29651 (0.0010) -[2023-10-14 14:38:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 60653568. Throughput: 0: 1676.1, 1: 1673.8. Samples: 15173182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 14:38:38,165][74987] Avg episode reward: [(0, '24.000'), (1, '25.110')] -[2023-10-14 14:38:38,223][75949] Updated weights for policy 0, policy_version 29661 (0.0009) -[2023-10-14 14:38:41,393][75950] Updated weights for policy 1, policy_version 29610 (0.0010) -[2023-10-14 14:38:41,761][75950] Updated weights for policy 1, policy_version 29620 (0.0010) -[2023-10-14 14:38:42,121][75950] Updated weights for policy 1, policy_version 29630 (0.0010) -[2023-10-14 14:38:42,516][75949] Updated weights for policy 0, policy_version 29671 (0.0009) -[2023-10-14 14:38:42,885][75949] Updated weights for policy 0, policy_version 29681 (0.0008) -[2023-10-14 14:38:43,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 60719104. Throughput: 0: 1666.0, 1: 1668.9. Samples: 15192606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 14:38:43,164][74987] Avg episode reward: [(0, '23.900'), (1, '26.370')] -[2023-10-14 14:38:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000029632_30343168.pth... -[2023-10-14 14:38:43,211][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000028064_28737536.pth -[2023-10-14 14:38:43,269][75949] Updated weights for policy 0, policy_version 29691 (0.0007) -[2023-10-14 14:38:43,443][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000029696_30408704.pth... -[2023-10-14 14:38:43,472][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000028096_28770304.pth -[2023-10-14 14:38:46,080][75950] Updated weights for policy 1, policy_version 29640 (0.0009) -[2023-10-14 14:38:46,446][75950] Updated weights for policy 1, policy_version 29650 (0.0010) -[2023-10-14 14:38:46,821][75950] Updated weights for policy 1, policy_version 29660 (0.0009) -[2023-10-14 14:38:47,329][75949] Updated weights for policy 0, policy_version 29701 (0.0008) -[2023-10-14 14:38:47,728][75949] Updated weights for policy 0, policy_version 29711 (0.0010) -[2023-10-14 14:38:48,089][75949] Updated weights for policy 0, policy_version 29721 (0.0011) -[2023-10-14 14:38:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 60784640. Throughput: 0: 1678.5, 1: 1687.9. Samples: 15203512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 14:38:48,165][74987] Avg episode reward: [(0, '24.110'), (1, '26.420')] -[2023-10-14 14:38:50,980][75950] Updated weights for policy 1, policy_version 29670 (0.0008) -[2023-10-14 14:38:51,347][75950] Updated weights for policy 1, policy_version 29680 (0.0008) -[2023-10-14 14:38:51,716][75950] Updated weights for policy 1, policy_version 29690 (0.0009) -[2023-10-14 14:38:52,187][75949] Updated weights for policy 0, policy_version 29731 (0.0010) -[2023-10-14 14:38:52,563][75949] Updated weights for policy 0, policy_version 29741 (0.0008) -[2023-10-14 14:38:52,929][75949] Updated weights for policy 0, policy_version 29751 (0.0009) -[2023-10-14 14:38:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 60850176. Throughput: 0: 1679.6, 1: 1665.2. Samples: 15223032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 14:38:53,165][74987] Avg episode reward: [(0, '22.870'), (1, '26.130')] -[2023-10-14 14:38:55,757][75950] Updated weights for policy 1, policy_version 29700 (0.0008) -[2023-10-14 14:38:56,128][75950] Updated weights for policy 1, policy_version 29710 (0.0008) -[2023-10-14 14:38:56,508][75950] Updated weights for policy 1, policy_version 29720 (0.0010) -[2023-10-14 14:38:56,985][75949] Updated weights for policy 0, policy_version 29761 (0.0007) -[2023-10-14 14:38:57,354][75949] Updated weights for policy 0, policy_version 29771 (0.0008) -[2023-10-14 14:38:57,718][75949] Updated weights for policy 0, policy_version 29781 (0.0010) -[2023-10-14 14:38:58,096][75949] Updated weights for policy 0, policy_version 29791 (0.0008) -[2023-10-14 14:38:58,164][74987] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 60948480. Throughput: 0: 1662.9, 1: 1678.5. Samples: 15242556. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) -[2023-10-14 14:38:58,165][74987] Avg episode reward: [(0, '23.520'), (1, '26.750')] -[2023-10-14 14:39:00,495][75950] Updated weights for policy 1, policy_version 29730 (0.0008) -[2023-10-14 14:39:00,859][75950] Updated weights for policy 1, policy_version 29740 (0.0009) -[2023-10-14 14:39:01,230][75950] Updated weights for policy 1, policy_version 29750 (0.0009) -[2023-10-14 14:39:01,596][75950] Updated weights for policy 1, policy_version 29760 (0.0008) -[2023-10-14 14:39:02,037][75949] Updated weights for policy 0, policy_version 29801 (0.0008) -[2023-10-14 14:39:02,413][75949] Updated weights for policy 0, policy_version 29811 (0.0009) -[2023-10-14 14:39:02,783][75949] Updated weights for policy 0, policy_version 29821 (0.0008) -[2023-10-14 14:39:03,164][74987] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 61014016. Throughput: 0: 1683.5, 1: 1676.4. Samples: 15253658. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) -[2023-10-14 14:39:03,165][74987] Avg episode reward: [(0, '21.810'), (1, '25.570')] -[2023-10-14 14:39:05,908][75950] Updated weights for policy 1, policy_version 29770 (0.0007) -[2023-10-14 14:39:06,276][75950] Updated weights for policy 1, policy_version 29780 (0.0007) -[2023-10-14 14:39:06,640][75950] Updated weights for policy 1, policy_version 29790 (0.0010) -[2023-10-14 14:39:06,852][75949] Updated weights for policy 0, policy_version 29831 (0.0007) -[2023-10-14 14:39:07,221][75949] Updated weights for policy 0, policy_version 29841 (0.0008) -[2023-10-14 14:39:07,586][75949] Updated weights for policy 0, policy_version 29851 (0.0009) -[2023-10-14 14:39:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 61079552. Throughput: 0: 1677.7, 1: 1664.0. Samples: 15273246. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) -[2023-10-14 14:39:08,165][74987] Avg episode reward: [(0, '23.190'), (1, '26.860')] -[2023-10-14 14:39:10,720][75950] Updated weights for policy 1, policy_version 29800 (0.0008) -[2023-10-14 14:39:11,089][75950] Updated weights for policy 1, policy_version 29810 (0.0009) -[2023-10-14 14:39:11,454][75950] Updated weights for policy 1, policy_version 29820 (0.0010) -[2023-10-14 14:39:11,605][75949] Updated weights for policy 0, policy_version 29861 (0.0008) -[2023-10-14 14:39:11,976][75949] Updated weights for policy 0, policy_version 29871 (0.0008) -[2023-10-14 14:39:12,343][75949] Updated weights for policy 0, policy_version 29881 (0.0009) -[2023-10-14 14:39:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 61145088. Throughput: 0: 1654.8, 1: 1684.4. Samples: 15292628. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) -[2023-10-14 14:39:13,164][74987] Avg episode reward: [(0, '23.560'), (1, '26.640')] -[2023-10-14 14:39:15,604][75950] Updated weights for policy 1, policy_version 29830 (0.0008) -[2023-10-14 14:39:15,974][75950] Updated weights for policy 1, policy_version 29840 (0.0009) -[2023-10-14 14:39:16,322][75949] Updated weights for policy 0, policy_version 29891 (0.0009) -[2023-10-14 14:39:16,347][75950] Updated weights for policy 1, policy_version 29850 (0.0008) -[2023-10-14 14:39:16,688][75949] Updated weights for policy 0, policy_version 29901 (0.0008) -[2023-10-14 14:39:17,062][75949] Updated weights for policy 0, policy_version 29911 (0.0009) -[2023-10-14 14:39:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 61210624. Throughput: 0: 1681.3, 1: 1677.5. Samples: 15304036. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 14:39:18,164][74987] Avg episode reward: [(0, '24.660'), (1, '25.060')] -[2023-10-14 14:39:20,370][75950] Updated weights for policy 1, policy_version 29860 (0.0009) -[2023-10-14 14:39:20,737][75950] Updated weights for policy 1, policy_version 29870 (0.0008) -[2023-10-14 14:39:21,106][75950] Updated weights for policy 1, policy_version 29880 (0.0011) -[2023-10-14 14:39:21,229][75949] Updated weights for policy 0, policy_version 29921 (0.0009) -[2023-10-14 14:39:21,590][75949] Updated weights for policy 0, policy_version 29931 (0.0009) -[2023-10-14 14:39:21,962][75949] Updated weights for policy 0, policy_version 29941 (0.0009) -[2023-10-14 14:39:22,328][75949] Updated weights for policy 0, policy_version 29951 (0.0008) -[2023-10-14 14:39:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 61276160. Throughput: 0: 1671.3, 1: 1664.8. Samples: 15323304. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 14:39:23,164][74987] Avg episode reward: [(0, '22.150'), (1, '25.860')] -[2023-10-14 14:39:25,142][75950] Updated weights for policy 1, policy_version 29890 (0.0009) -[2023-10-14 14:39:25,504][75950] Updated weights for policy 1, policy_version 29900 (0.0007) -[2023-10-14 14:39:25,879][75950] Updated weights for policy 1, policy_version 29910 (0.0007) -[2023-10-14 14:39:26,238][75950] Updated weights for policy 1, policy_version 29920 (0.0007) -[2023-10-14 14:39:26,331][75949] Updated weights for policy 0, policy_version 29961 (0.0009) -[2023-10-14 14:39:26,700][75949] Updated weights for policy 0, policy_version 29971 (0.0010) -[2023-10-14 14:39:27,065][75949] Updated weights for policy 0, policy_version 29981 (0.0009) -[2023-10-14 14:39:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 61341696. Throughput: 0: 1664.6, 1: 1683.9. Samples: 15343290. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 14:39:28,165][74987] Avg episode reward: [(0, '23.370'), (1, '24.310')] -[2023-10-14 14:39:30,269][75950] Updated weights for policy 1, policy_version 29930 (0.0009) -[2023-10-14 14:39:30,632][75950] Updated weights for policy 1, policy_version 29940 (0.0008) -[2023-10-14 14:39:31,007][75950] Updated weights for policy 1, policy_version 29950 (0.0008) -[2023-10-14 14:39:31,074][75949] Updated weights for policy 0, policy_version 29991 (0.0008) -[2023-10-14 14:39:31,444][75949] Updated weights for policy 0, policy_version 30001 (0.0010) -[2023-10-14 14:39:31,809][75949] Updated weights for policy 0, policy_version 30011 (0.0007) -[2023-10-14 14:39:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 61407232. Throughput: 0: 1685.6, 1: 1664.3. Samples: 15354258. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 14:39:33,165][74987] Avg episode reward: [(0, '24.880'), (1, '24.450')] -[2023-10-14 14:39:35,134][75950] Updated weights for policy 1, policy_version 29960 (0.0009) -[2023-10-14 14:39:35,509][75950] Updated weights for policy 1, policy_version 29970 (0.0008) -[2023-10-14 14:39:35,886][75950] Updated weights for policy 1, policy_version 29980 (0.0007) -[2023-10-14 14:39:36,053][75949] Updated weights for policy 0, policy_version 30021 (0.0008) -[2023-10-14 14:39:36,446][75949] Updated weights for policy 0, policy_version 30031 (0.0007) -[2023-10-14 14:39:36,812][75949] Updated weights for policy 0, policy_version 30041 (0.0010) -[2023-10-14 14:39:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 61472768. Throughput: 0: 1670.5, 1: 1669.1. Samples: 15373316. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 14:39:38,164][74987] Avg episode reward: [(0, '24.080'), (1, '26.520')] -[2023-10-14 14:39:39,955][75950] Updated weights for policy 1, policy_version 29990 (0.0008) -[2023-10-14 14:39:40,321][75950] Updated weights for policy 1, policy_version 30000 (0.0007) -[2023-10-14 14:39:40,687][75950] Updated weights for policy 1, policy_version 30010 (0.0008) -[2023-10-14 14:39:40,730][75949] Updated weights for policy 0, policy_version 30051 (0.0010) -[2023-10-14 14:39:41,100][75949] Updated weights for policy 0, policy_version 30061 (0.0011) -[2023-10-14 14:39:41,462][75949] Updated weights for policy 0, policy_version 30071 (0.0010) -[2023-10-14 14:39:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 61538304. Throughput: 0: 1681.5, 1: 1677.4. Samples: 15393708. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:39:43,165][74987] Avg episode reward: [(0, '22.970'), (1, '25.440')] -[2023-10-14 14:39:44,853][75950] Updated weights for policy 1, policy_version 30020 (0.0009) -[2023-10-14 14:39:45,218][75950] Updated weights for policy 1, policy_version 30030 (0.0008) -[2023-10-14 14:39:45,558][75949] Updated weights for policy 0, policy_version 30081 (0.0008) -[2023-10-14 14:39:45,591][75950] Updated weights for policy 1, policy_version 30040 (0.0009) -[2023-10-14 14:39:45,932][75949] Updated weights for policy 0, policy_version 30091 (0.0007) -[2023-10-14 14:39:46,307][75949] Updated weights for policy 0, policy_version 30101 (0.0007) -[2023-10-14 14:39:46,684][75949] Updated weights for policy 0, policy_version 30111 (0.0009) -[2023-10-14 14:39:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 61603840. Throughput: 0: 1686.8, 1: 1657.0. Samples: 15404126. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:39:48,164][74987] Avg episode reward: [(0, '20.650'), (1, '24.850')] -[2023-10-14 14:39:49,759][75950] Updated weights for policy 1, policy_version 30050 (0.0010) -[2023-10-14 14:39:50,126][75950] Updated weights for policy 1, policy_version 30060 (0.0009) -[2023-10-14 14:39:50,493][75950] Updated weights for policy 1, policy_version 30070 (0.0009) -[2023-10-14 14:39:50,678][75949] Updated weights for policy 0, policy_version 30121 (0.0009) -[2023-10-14 14:39:50,861][75950] Updated weights for policy 1, policy_version 30080 (0.0007) -[2023-10-14 14:39:51,045][75949] Updated weights for policy 0, policy_version 30131 (0.0010) -[2023-10-14 14:39:51,414][75949] Updated weights for policy 0, policy_version 30141 (0.0010) -[2023-10-14 14:39:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 61669376. Throughput: 0: 1666.1, 1: 1670.7. Samples: 15423404. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:39:53,164][74987] Avg episode reward: [(0, '21.950'), (1, '27.090')] -[2023-10-14 14:39:55,043][75950] Updated weights for policy 1, policy_version 30090 (0.0008) -[2023-10-14 14:39:55,407][75950] Updated weights for policy 1, policy_version 30100 (0.0009) -[2023-10-14 14:39:55,544][75949] Updated weights for policy 0, policy_version 30151 (0.0007) -[2023-10-14 14:39:55,775][75950] Updated weights for policy 1, policy_version 30110 (0.0008) -[2023-10-14 14:39:55,916][75949] Updated weights for policy 0, policy_version 30161 (0.0009) -[2023-10-14 14:39:56,292][75949] Updated weights for policy 0, policy_version 30171 (0.0007) -[2023-10-14 14:39:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 61734912. Throughput: 0: 1692.4, 1: 1672.1. Samples: 15444034. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:39:58,165][74987] Avg episode reward: [(0, '23.690'), (1, '24.440')] -[2023-10-14 14:39:59,880][75950] Updated weights for policy 1, policy_version 30120 (0.0007) -[2023-10-14 14:40:00,243][75950] Updated weights for policy 1, policy_version 30130 (0.0010) -[2023-10-14 14:40:00,343][75949] Updated weights for policy 0, policy_version 30181 (0.0008) -[2023-10-14 14:40:00,615][75950] Updated weights for policy 1, policy_version 30140 (0.0007) -[2023-10-14 14:40:00,703][75949] Updated weights for policy 0, policy_version 30191 (0.0007) -[2023-10-14 14:40:01,073][75949] Updated weights for policy 0, policy_version 30201 (0.0007) -[2023-10-14 14:40:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 61800448. Throughput: 0: 1680.9, 1: 1653.3. Samples: 15454074. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:40:03,164][74987] Avg episode reward: [(0, '25.670'), (1, '25.200')] -[2023-10-14 14:40:03,165][75615] Saving new best policy, reward=25.670! -[2023-10-14 14:40:04,840][75950] Updated weights for policy 1, policy_version 30150 (0.0010) -[2023-10-14 14:40:05,032][75949] Updated weights for policy 0, policy_version 30211 (0.0008) -[2023-10-14 14:40:05,207][75950] Updated weights for policy 1, policy_version 30160 (0.0007) -[2023-10-14 14:40:05,410][75949] Updated weights for policy 0, policy_version 30221 (0.0008) -[2023-10-14 14:40:05,560][75950] Updated weights for policy 1, policy_version 30170 (0.0007) -[2023-10-14 14:40:05,779][75949] Updated weights for policy 0, policy_version 30231 (0.0010) -[2023-10-14 14:40:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 61865984. Throughput: 0: 1673.1, 1: 1670.5. Samples: 15473766. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 14:40:08,164][74987] Avg episode reward: [(0, '23.070'), (1, '26.390')] -[2023-10-14 14:40:09,712][75950] Updated weights for policy 1, policy_version 30180 (0.0007) -[2023-10-14 14:40:09,930][75949] Updated weights for policy 0, policy_version 30241 (0.0009) -[2023-10-14 14:40:10,082][75950] Updated weights for policy 1, policy_version 30190 (0.0007) -[2023-10-14 14:40:10,292][75949] Updated weights for policy 0, policy_version 30251 (0.0009) -[2023-10-14 14:40:10,450][75950] Updated weights for policy 1, policy_version 30200 (0.0009) -[2023-10-14 14:40:10,661][75949] Updated weights for policy 0, policy_version 30261 (0.0007) -[2023-10-14 14:40:11,028][75949] Updated weights for policy 0, policy_version 30271 (0.0008) -[2023-10-14 14:40:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 61931520. Throughput: 0: 1692.0, 1: 1660.4. Samples: 15494144. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 14:40:13,165][74987] Avg episode reward: [(0, '23.760'), (1, '25.140')] -[2023-10-14 14:40:14,583][75950] Updated weights for policy 1, policy_version 30210 (0.0008) -[2023-10-14 14:40:14,952][75950] Updated weights for policy 1, policy_version 30220 (0.0009) -[2023-10-14 14:40:15,100][75949] Updated weights for policy 0, policy_version 30281 (0.0008) -[2023-10-14 14:40:15,310][75950] Updated weights for policy 1, policy_version 30230 (0.0009) -[2023-10-14 14:40:15,480][75949] Updated weights for policy 0, policy_version 30291 (0.0009) -[2023-10-14 14:40:15,675][75950] Updated weights for policy 1, policy_version 30240 (0.0009) -[2023-10-14 14:40:15,858][75949] Updated weights for policy 0, policy_version 30301 (0.0008) -[2023-10-14 14:40:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 61997056. Throughput: 0: 1665.9, 1: 1652.3. Samples: 15503578. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 14:40:18,165][74987] Avg episode reward: [(0, '22.180'), (1, '26.480')] -[2023-10-14 14:40:19,812][75950] Updated weights for policy 1, policy_version 30250 (0.0007) -[2023-10-14 14:40:20,017][75949] Updated weights for policy 0, policy_version 30311 (0.0010) -[2023-10-14 14:40:20,185][75950] Updated weights for policy 1, policy_version 30260 (0.0009) -[2023-10-14 14:40:20,398][75949] Updated weights for policy 0, policy_version 30321 (0.0009) -[2023-10-14 14:40:20,543][75950] Updated weights for policy 1, policy_version 30270 (0.0008) -[2023-10-14 14:40:20,776][75949] Updated weights for policy 0, policy_version 30331 (0.0010) -[2023-10-14 14:40:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 62062592. Throughput: 0: 1675.4, 1: 1662.4. Samples: 15523516. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 14:40:23,164][74987] Avg episode reward: [(0, '25.360'), (1, '27.040')] -[2023-10-14 14:40:24,538][75950] Updated weights for policy 1, policy_version 30280 (0.0007) -[2023-10-14 14:40:24,853][75949] Updated weights for policy 0, policy_version 30341 (0.0008) -[2023-10-14 14:40:24,911][75950] Updated weights for policy 1, policy_version 30290 (0.0009) -[2023-10-14 14:40:25,244][75949] Updated weights for policy 0, policy_version 30351 (0.0008) -[2023-10-14 14:40:25,264][75950] Updated weights for policy 1, policy_version 30300 (0.0009) -[2023-10-14 14:40:25,606][75949] Updated weights for policy 0, policy_version 30361 (0.0007) -[2023-10-14 14:40:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 62128128. Throughput: 0: 1681.2, 1: 1657.1. Samples: 15543930. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 14:40:28,165][74987] Avg episode reward: [(0, '24.940'), (1, '26.450')] -[2023-10-14 14:40:29,362][75950] Updated weights for policy 1, policy_version 30310 (0.0009) -[2023-10-14 14:40:29,726][75950] Updated weights for policy 1, policy_version 30320 (0.0008) -[2023-10-14 14:40:29,742][75949] Updated weights for policy 0, policy_version 30371 (0.0008) -[2023-10-14 14:40:30,098][75950] Updated weights for policy 1, policy_version 30330 (0.0008) -[2023-10-14 14:40:30,098][75949] Updated weights for policy 0, policy_version 30381 (0.0009) -[2023-10-14 14:40:30,464][75949] Updated weights for policy 0, policy_version 30391 (0.0008) -[2023-10-14 14:40:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 62193664. Throughput: 0: 1656.7, 1: 1650.8. Samples: 15552964. Policy #0 lag: (min: 13.0, avg: 13.8, max: 33.0) -[2023-10-14 14:40:33,165][74987] Avg episode reward: [(0, '24.370'), (1, '26.350')] -[2023-10-14 14:40:34,091][75950] Updated weights for policy 1, policy_version 30340 (0.0009) -[2023-10-14 14:40:34,468][75950] Updated weights for policy 1, policy_version 30350 (0.0008) -[2023-10-14 14:40:34,474][75949] Updated weights for policy 0, policy_version 30401 (0.0008) -[2023-10-14 14:40:34,836][75950] Updated weights for policy 1, policy_version 30360 (0.0009) -[2023-10-14 14:40:34,842][75949] Updated weights for policy 0, policy_version 30411 (0.0009) -[2023-10-14 14:40:35,211][75949] Updated weights for policy 0, policy_version 30421 (0.0009) -[2023-10-14 14:40:35,578][75949] Updated weights for policy 0, policy_version 30431 (0.0010) -[2023-10-14 14:40:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 62259200. Throughput: 0: 1678.1, 1: 1658.5. Samples: 15573550. Policy #0 lag: (min: 13.0, avg: 13.8, max: 33.0) -[2023-10-14 14:40:38,165][74987] Avg episode reward: [(0, '23.320'), (1, '26.230')] -[2023-10-14 14:40:39,014][75950] Updated weights for policy 1, policy_version 30370 (0.0008) -[2023-10-14 14:40:39,414][75950] Updated weights for policy 1, policy_version 30380 (0.0007) -[2023-10-14 14:40:39,681][75949] Updated weights for policy 0, policy_version 30441 (0.0009) -[2023-10-14 14:40:39,779][75950] Updated weights for policy 1, policy_version 30390 (0.0009) -[2023-10-14 14:40:40,043][75949] Updated weights for policy 0, policy_version 30451 (0.0007) -[2023-10-14 14:40:40,148][75950] Updated weights for policy 1, policy_version 30400 (0.0008) -[2023-10-14 14:40:40,415][75949] Updated weights for policy 0, policy_version 30461 (0.0009) -[2023-10-14 14:40:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 62324736. Throughput: 0: 1673.1, 1: 1656.5. Samples: 15593864. Policy #0 lag: (min: 13.0, avg: 13.8, max: 33.0) -[2023-10-14 14:40:43,165][74987] Avg episode reward: [(0, '22.600'), (1, '25.860')] -[2023-10-14 14:40:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000030464_31195136.pth... -[2023-10-14 14:40:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000030400_31129600.pth... -[2023-10-14 14:40:43,216][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000028832_29523968.pth -[2023-10-14 14:40:43,217][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000028896_29589504.pth -[2023-10-14 14:40:44,352][75950] Updated weights for policy 1, policy_version 30410 (0.0009) -[2023-10-14 14:40:44,396][75949] Updated weights for policy 0, policy_version 30471 (0.0008) -[2023-10-14 14:40:44,721][75950] Updated weights for policy 1, policy_version 30420 (0.0008) -[2023-10-14 14:40:44,762][75949] Updated weights for policy 0, policy_version 30481 (0.0008) -[2023-10-14 14:40:45,089][75950] Updated weights for policy 1, policy_version 30430 (0.0009) -[2023-10-14 14:40:45,138][75949] Updated weights for policy 0, policy_version 30491 (0.0010) -[2023-10-14 14:40:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 62390272. Throughput: 0: 1656.6, 1: 1651.3. Samples: 15602928. Policy #0 lag: (min: 13.0, avg: 13.8, max: 33.0) -[2023-10-14 14:40:48,165][74987] Avg episode reward: [(0, '23.950'), (1, '26.670')] -[2023-10-14 14:40:49,312][75950] Updated weights for policy 1, policy_version 30440 (0.0009) -[2023-10-14 14:40:49,384][75949] Updated weights for policy 0, policy_version 30501 (0.0009) -[2023-10-14 14:40:49,676][75950] Updated weights for policy 1, policy_version 30450 (0.0008) -[2023-10-14 14:40:49,750][75949] Updated weights for policy 0, policy_version 30511 (0.0008) -[2023-10-14 14:40:50,038][75950] Updated weights for policy 1, policy_version 30460 (0.0009) -[2023-10-14 14:40:50,127][75949] Updated weights for policy 0, policy_version 30521 (0.0010) -[2023-10-14 14:40:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62455808. Throughput: 0: 1667.3, 1: 1656.8. Samples: 15623352. Policy #0 lag: (min: 13.0, avg: 13.8, max: 33.0) -[2023-10-14 14:40:53,164][74987] Avg episode reward: [(0, '24.210'), (1, '27.140')] -[2023-10-14 14:40:54,095][75949] Updated weights for policy 0, policy_version 30531 (0.0009) -[2023-10-14 14:40:54,180][75950] Updated weights for policy 1, policy_version 30470 (0.0007) -[2023-10-14 14:40:54,463][75949] Updated weights for policy 0, policy_version 30541 (0.0007) -[2023-10-14 14:40:54,548][75950] Updated weights for policy 1, policy_version 30480 (0.0007) -[2023-10-14 14:40:54,833][75949] Updated weights for policy 0, policy_version 30551 (0.0009) -[2023-10-14 14:40:54,910][75950] Updated weights for policy 1, policy_version 30490 (0.0009) -[2023-10-14 14:40:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62521344. Throughput: 0: 1666.2, 1: 1657.0. Samples: 15643686. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 14:40:58,164][74987] Avg episode reward: [(0, '24.080'), (1, '26.550')] -[2023-10-14 14:40:59,017][75949] Updated weights for policy 0, policy_version 30561 (0.0008) -[2023-10-14 14:40:59,147][75950] Updated weights for policy 1, policy_version 30500 (0.0008) -[2023-10-14 14:40:59,396][75949] Updated weights for policy 0, policy_version 30571 (0.0009) -[2023-10-14 14:40:59,514][75950] Updated weights for policy 1, policy_version 30510 (0.0008) -[2023-10-14 14:40:59,768][75949] Updated weights for policy 0, policy_version 30581 (0.0007) -[2023-10-14 14:40:59,880][75950] Updated weights for policy 1, policy_version 30520 (0.0008) -[2023-10-14 14:41:00,137][75949] Updated weights for policy 0, policy_version 30591 (0.0008) -[2023-10-14 14:41:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 62586880. Throughput: 0: 1659.7, 1: 1653.0. Samples: 15652650. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 14:41:03,165][74987] Avg episode reward: [(0, '23.620'), (1, '26.910')] -[2023-10-14 14:41:04,011][75950] Updated weights for policy 1, policy_version 30530 (0.0008) -[2023-10-14 14:41:04,197][75949] Updated weights for policy 0, policy_version 30601 (0.0010) -[2023-10-14 14:41:04,385][75950] Updated weights for policy 1, policy_version 30540 (0.0007) -[2023-10-14 14:41:04,562][75949] Updated weights for policy 0, policy_version 30611 (0.0008) -[2023-10-14 14:41:04,748][75950] Updated weights for policy 1, policy_version 30550 (0.0007) -[2023-10-14 14:41:04,938][75949] Updated weights for policy 0, policy_version 30621 (0.0007) -[2023-10-14 14:41:05,115][75950] Updated weights for policy 1, policy_version 30560 (0.0010) -[2023-10-14 14:41:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62652416. Throughput: 0: 1670.0, 1: 1655.6. Samples: 15673168. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 14:41:08,164][74987] Avg episode reward: [(0, '25.160'), (1, '27.030')] -[2023-10-14 14:41:09,105][75949] Updated weights for policy 0, policy_version 30631 (0.0007) -[2023-10-14 14:41:09,194][75950] Updated weights for policy 1, policy_version 30570 (0.0008) -[2023-10-14 14:41:09,470][75949] Updated weights for policy 0, policy_version 30641 (0.0008) -[2023-10-14 14:41:09,560][75950] Updated weights for policy 1, policy_version 30580 (0.0009) -[2023-10-14 14:41:09,850][75949] Updated weights for policy 0, policy_version 30651 (0.0008) -[2023-10-14 14:41:09,933][75950] Updated weights for policy 1, policy_version 30590 (0.0009) -[2023-10-14 14:41:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62717952. Throughput: 0: 1673.3, 1: 1659.9. Samples: 15693924. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 14:41:13,165][74987] Avg episode reward: [(0, '22.540'), (1, '26.660')] -[2023-10-14 14:41:13,795][75949] Updated weights for policy 0, policy_version 30661 (0.0010) -[2023-10-14 14:41:14,147][75950] Updated weights for policy 1, policy_version 30600 (0.0008) -[2023-10-14 14:41:14,195][75949] Updated weights for policy 0, policy_version 30671 (0.0009) -[2023-10-14 14:41:14,512][75950] Updated weights for policy 1, policy_version 30610 (0.0008) -[2023-10-14 14:41:14,566][75949] Updated weights for policy 0, policy_version 30681 (0.0010) -[2023-10-14 14:41:14,882][75950] Updated weights for policy 1, policy_version 30620 (0.0009) -[2023-10-14 14:41:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62783488. Throughput: 0: 1669.6, 1: 1665.0. Samples: 15703018. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 14:41:18,164][74987] Avg episode reward: [(0, '23.940'), (1, '27.830')] -[2023-10-14 14:41:18,610][75949] Updated weights for policy 0, policy_version 30691 (0.0010) -[2023-10-14 14:41:18,907][75950] Updated weights for policy 1, policy_version 30630 (0.0011) -[2023-10-14 14:41:18,993][75949] Updated weights for policy 0, policy_version 30701 (0.0009) -[2023-10-14 14:41:19,276][75950] Updated weights for policy 1, policy_version 30640 (0.0010) -[2023-10-14 14:41:19,358][75949] Updated weights for policy 0, policy_version 30711 (0.0007) -[2023-10-14 14:41:19,650][75950] Updated weights for policy 1, policy_version 30650 (0.0009) -[2023-10-14 14:41:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 62849024. Throughput: 0: 1670.7, 1: 1663.5. Samples: 15723588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:41:23,165][74987] Avg episode reward: [(0, '21.100'), (1, '26.090')] -[2023-10-14 14:41:23,562][75949] Updated weights for policy 0, policy_version 30721 (0.0008) -[2023-10-14 14:41:23,865][75950] Updated weights for policy 1, policy_version 30660 (0.0009) -[2023-10-14 14:41:23,938][75949] Updated weights for policy 0, policy_version 30731 (0.0009) -[2023-10-14 14:41:24,238][75950] Updated weights for policy 1, policy_version 30670 (0.0007) -[2023-10-14 14:41:24,302][75949] Updated weights for policy 0, policy_version 30741 (0.0009) -[2023-10-14 14:41:24,600][75950] Updated weights for policy 1, policy_version 30680 (0.0008) -[2023-10-14 14:41:24,676][75949] Updated weights for policy 0, policy_version 30751 (0.0008) -[2023-10-14 14:41:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 62914560. Throughput: 0: 1672.2, 1: 1659.9. Samples: 15743806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:41:28,165][74987] Avg episode reward: [(0, '23.720'), (1, '26.990')] -[2023-10-14 14:41:28,782][75949] Updated weights for policy 0, policy_version 30761 (0.0009) -[2023-10-14 14:41:28,802][75950] Updated weights for policy 1, policy_version 30690 (0.0009) -[2023-10-14 14:41:29,163][75949] Updated weights for policy 0, policy_version 30771 (0.0008) -[2023-10-14 14:41:29,233][75950] Updated weights for policy 1, policy_version 30700 (0.0009) -[2023-10-14 14:41:29,537][75949] Updated weights for policy 0, policy_version 30781 (0.0009) -[2023-10-14 14:41:29,603][75950] Updated weights for policy 1, policy_version 30710 (0.0008) -[2023-10-14 14:41:29,971][75950] Updated weights for policy 1, policy_version 30720 (0.0009) -[2023-10-14 14:41:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62980096. Throughput: 0: 1672.3, 1: 1655.3. Samples: 15752670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:41:33,164][74987] Avg episode reward: [(0, '20.850'), (1, '27.690')] -[2023-10-14 14:41:33,519][75949] Updated weights for policy 0, policy_version 30791 (0.0008) -[2023-10-14 14:41:33,886][75949] Updated weights for policy 0, policy_version 30801 (0.0008) -[2023-10-14 14:41:33,959][75950] Updated weights for policy 1, policy_version 30730 (0.0009) -[2023-10-14 14:41:34,253][75949] Updated weights for policy 0, policy_version 30811 (0.0008) -[2023-10-14 14:41:34,324][75950] Updated weights for policy 1, policy_version 30740 (0.0010) -[2023-10-14 14:41:34,685][75950] Updated weights for policy 1, policy_version 30750 (0.0008) -[2023-10-14 14:41:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 63045632. Throughput: 0: 1680.2, 1: 1657.4. Samples: 15773544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:41:38,164][74987] Avg episode reward: [(0, '24.560'), (1, '27.190')] -[2023-10-14 14:41:38,407][75949] Updated weights for policy 0, policy_version 30821 (0.0007) -[2023-10-14 14:41:38,615][75950] Updated weights for policy 1, policy_version 30760 (0.0007) -[2023-10-14 14:41:38,774][75949] Updated weights for policy 0, policy_version 30831 (0.0008) -[2023-10-14 14:41:38,983][75950] Updated weights for policy 1, policy_version 30770 (0.0008) -[2023-10-14 14:41:39,143][75949] Updated weights for policy 0, policy_version 30841 (0.0007) -[2023-10-14 14:41:39,340][75950] Updated weights for policy 1, policy_version 30780 (0.0008) -[2023-10-14 14:41:43,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 63111168. Throughput: 0: 1676.0, 1: 1666.3. Samples: 15794090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:41:43,164][74987] Avg episode reward: [(0, '24.940'), (1, '27.640')] -[2023-10-14 14:41:43,316][75949] Updated weights for policy 0, policy_version 30851 (0.0008) -[2023-10-14 14:41:43,517][75950] Updated weights for policy 1, policy_version 30790 (0.0008) -[2023-10-14 14:41:43,690][75949] Updated weights for policy 0, policy_version 30861 (0.0008) -[2023-10-14 14:41:43,883][75950] Updated weights for policy 1, policy_version 30800 (0.0010) -[2023-10-14 14:41:44,052][75949] Updated weights for policy 0, policy_version 30871 (0.0008) -[2023-10-14 14:41:44,251][75950] Updated weights for policy 1, policy_version 30810 (0.0007) -[2023-10-14 14:41:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 63176704. Throughput: 0: 1676.5, 1: 1668.9. Samples: 15803194. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:41:48,164][74987] Avg episode reward: [(0, '26.050'), (1, '28.540')] -[2023-10-14 14:41:48,222][75949] Updated weights for policy 0, policy_version 30881 (0.0008) -[2023-10-14 14:41:48,231][75950] Updated weights for policy 1, policy_version 30820 (0.0009) -[2023-10-14 14:41:48,594][75949] Updated weights for policy 0, policy_version 30891 (0.0008) -[2023-10-14 14:41:48,595][75950] Updated weights for policy 1, policy_version 30830 (0.0009) -[2023-10-14 14:41:48,946][75950] Updated weights for policy 1, policy_version 30840 (0.0008) -[2023-10-14 14:41:48,971][75949] Updated weights for policy 0, policy_version 30901 (0.0007) -[2023-10-14 14:41:49,236][75801] Saving new best policy, reward=28.540! -[2023-10-14 14:41:49,337][75949] Updated weights for policy 0, policy_version 30911 (0.0007) -[2023-10-14 14:41:49,369][75615] Saving new best policy, reward=26.050! -[2023-10-14 14:41:53,118][75950] Updated weights for policy 1, policy_version 30850 (0.0008) -[2023-10-14 14:41:53,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63242240. Throughput: 0: 1671.1, 1: 1668.6. Samples: 15823454. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:41:53,165][74987] Avg episode reward: [(0, '24.080'), (1, '26.630')] -[2023-10-14 14:41:53,418][75949] Updated weights for policy 0, policy_version 30921 (0.0008) -[2023-10-14 14:41:53,485][75950] Updated weights for policy 1, policy_version 30860 (0.0007) -[2023-10-14 14:41:53,799][75949] Updated weights for policy 0, policy_version 30931 (0.0011) -[2023-10-14 14:41:53,849][75950] Updated weights for policy 1, policy_version 30870 (0.0009) -[2023-10-14 14:41:54,158][75949] Updated weights for policy 0, policy_version 30941 (0.0009) -[2023-10-14 14:41:54,213][75950] Updated weights for policy 1, policy_version 30880 (0.0009) -[2023-10-14 14:41:58,085][75950] Updated weights for policy 1, policy_version 30890 (0.0010) -[2023-10-14 14:41:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63307776. Throughput: 0: 1666.0, 1: 1675.2. Samples: 15844276. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:41:58,164][74987] Avg episode reward: [(0, '24.090'), (1, '26.800')] -[2023-10-14 14:41:58,225][75949] Updated weights for policy 0, policy_version 30951 (0.0008) -[2023-10-14 14:41:58,456][75950] Updated weights for policy 1, policy_version 30900 (0.0007) -[2023-10-14 14:41:58,597][75949] Updated weights for policy 0, policy_version 30961 (0.0007) -[2023-10-14 14:41:58,826][75950] Updated weights for policy 1, policy_version 30910 (0.0009) -[2023-10-14 14:41:58,969][75949] Updated weights for policy 0, policy_version 30971 (0.0008) -[2023-10-14 14:42:02,875][75949] Updated weights for policy 0, policy_version 30981 (0.0010) -[2023-10-14 14:42:03,024][75950] Updated weights for policy 1, policy_version 30920 (0.0007) -[2023-10-14 14:42:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63373312. Throughput: 0: 1674.1, 1: 1671.4. Samples: 15853566. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:42:03,164][74987] Avg episode reward: [(0, '23.290'), (1, '26.360')] -[2023-10-14 14:42:03,267][75949] Updated weights for policy 0, policy_version 30991 (0.0009) -[2023-10-14 14:42:03,398][75950] Updated weights for policy 1, policy_version 30930 (0.0007) -[2023-10-14 14:42:03,623][75949] Updated weights for policy 0, policy_version 31001 (0.0008) -[2023-10-14 14:42:03,752][75950] Updated weights for policy 1, policy_version 30940 (0.0008) -[2023-10-14 14:42:07,751][75949] Updated weights for policy 0, policy_version 31011 (0.0008) -[2023-10-14 14:42:07,831][75950] Updated weights for policy 1, policy_version 30950 (0.0009) -[2023-10-14 14:42:08,118][75949] Updated weights for policy 0, policy_version 31021 (0.0007) -[2023-10-14 14:42:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63438848. Throughput: 0: 1673.7, 1: 1670.2. Samples: 15874064. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:42:08,164][74987] Avg episode reward: [(0, '24.620'), (1, '27.030')] -[2023-10-14 14:42:08,195][75950] Updated weights for policy 1, policy_version 30960 (0.0008) -[2023-10-14 14:42:08,484][75949] Updated weights for policy 0, policy_version 31031 (0.0007) -[2023-10-14 14:42:08,561][75950] Updated weights for policy 1, policy_version 30970 (0.0008) -[2023-10-14 14:42:12,634][75949] Updated weights for policy 0, policy_version 31041 (0.0009) -[2023-10-14 14:42:12,684][75950] Updated weights for policy 1, policy_version 30980 (0.0008) -[2023-10-14 14:42:12,999][75949] Updated weights for policy 0, policy_version 31051 (0.0008) -[2023-10-14 14:42:13,053][75950] Updated weights for policy 1, policy_version 30990 (0.0008) -[2023-10-14 14:42:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63504384. Throughput: 0: 1674.2, 1: 1676.4. Samples: 15894582. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:42:13,164][74987] Avg episode reward: [(0, '24.040'), (1, '26.580')] -[2023-10-14 14:42:13,375][75949] Updated weights for policy 0, policy_version 31061 (0.0009) -[2023-10-14 14:42:13,421][75950] Updated weights for policy 1, policy_version 31000 (0.0009) -[2023-10-14 14:42:13,740][75949] Updated weights for policy 0, policy_version 31071 (0.0007) -[2023-10-14 14:42:17,754][75950] Updated weights for policy 1, policy_version 31010 (0.0008) -[2023-10-14 14:42:17,758][75949] Updated weights for policy 0, policy_version 31081 (0.0009) -[2023-10-14 14:42:18,128][75949] Updated weights for policy 0, policy_version 31091 (0.0008) -[2023-10-14 14:42:18,162][75950] Updated weights for policy 1, policy_version 31020 (0.0009) -[2023-10-14 14:42:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63569920. Throughput: 0: 1673.2, 1: 1684.0. Samples: 15903746. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:42:18,164][74987] Avg episode reward: [(0, '23.620'), (1, '25.990')] -[2023-10-14 14:42:18,493][75949] Updated weights for policy 0, policy_version 31101 (0.0008) -[2023-10-14 14:42:18,524][75950] Updated weights for policy 1, policy_version 31030 (0.0009) -[2023-10-14 14:42:18,901][75950] Updated weights for policy 1, policy_version 31040 (0.0008) -[2023-10-14 14:42:22,515][75949] Updated weights for policy 0, policy_version 31111 (0.0007) -[2023-10-14 14:42:22,676][75950] Updated weights for policy 1, policy_version 31050 (0.0007) -[2023-10-14 14:42:22,891][75949] Updated weights for policy 0, policy_version 31121 (0.0008) -[2023-10-14 14:42:23,035][75950] Updated weights for policy 1, policy_version 31060 (0.0007) -[2023-10-14 14:42:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63635456. Throughput: 0: 1670.6, 1: 1680.6. Samples: 15924346. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:42:23,164][74987] Avg episode reward: [(0, '22.720'), (1, '27.890')] -[2023-10-14 14:42:23,248][75949] Updated weights for policy 0, policy_version 31131 (0.0008) -[2023-10-14 14:42:23,402][75950] Updated weights for policy 1, policy_version 31070 (0.0007) -[2023-10-14 14:42:27,516][75949] Updated weights for policy 0, policy_version 31141 (0.0008) -[2023-10-14 14:42:27,569][75950] Updated weights for policy 1, policy_version 31080 (0.0007) -[2023-10-14 14:42:27,888][75949] Updated weights for policy 0, policy_version 31151 (0.0008) -[2023-10-14 14:42:27,939][75950] Updated weights for policy 1, policy_version 31090 (0.0008) -[2023-10-14 14:42:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 63700992. Throughput: 0: 1664.0, 1: 1671.9. Samples: 15944202. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:42:28,164][74987] Avg episode reward: [(0, '22.720'), (1, '27.400')] -[2023-10-14 14:42:28,260][75949] Updated weights for policy 0, policy_version 31161 (0.0009) -[2023-10-14 14:42:28,310][75950] Updated weights for policy 1, policy_version 31100 (0.0007) -[2023-10-14 14:42:32,478][75949] Updated weights for policy 0, policy_version 31171 (0.0009) -[2023-10-14 14:42:32,495][75950] Updated weights for policy 1, policy_version 31110 (0.0007) -[2023-10-14 14:42:32,854][75950] Updated weights for policy 1, policy_version 31120 (0.0007) -[2023-10-14 14:42:32,856][75949] Updated weights for policy 0, policy_version 31181 (0.0007) -[2023-10-14 14:42:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 63766528. Throughput: 0: 1673.0, 1: 1677.9. Samples: 15953982. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:42:33,165][74987] Avg episode reward: [(0, '23.710'), (1, '25.180')] -[2023-10-14 14:42:33,222][75950] Updated weights for policy 1, policy_version 31130 (0.0009) -[2023-10-14 14:42:33,222][75949] Updated weights for policy 0, policy_version 31191 (0.0007) -[2023-10-14 14:42:37,327][75950] Updated weights for policy 1, policy_version 31140 (0.0008) -[2023-10-14 14:42:37,375][75949] Updated weights for policy 0, policy_version 31201 (0.0008) -[2023-10-14 14:42:37,688][75950] Updated weights for policy 1, policy_version 31150 (0.0009) -[2023-10-14 14:42:37,736][75949] Updated weights for policy 0, policy_version 31211 (0.0008) -[2023-10-14 14:42:38,057][75950] Updated weights for policy 1, policy_version 31160 (0.0009) -[2023-10-14 14:42:38,105][75949] Updated weights for policy 0, policy_version 31221 (0.0008) -[2023-10-14 14:42:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63832064. Throughput: 0: 1678.8, 1: 1680.8. Samples: 15974634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:42:38,164][74987] Avg episode reward: [(0, '22.820'), (1, '26.430')] -[2023-10-14 14:42:38,471][75949] Updated weights for policy 0, policy_version 31231 (0.0008) -[2023-10-14 14:42:42,151][75950] Updated weights for policy 1, policy_version 31170 (0.0009) -[2023-10-14 14:42:42,522][75950] Updated weights for policy 1, policy_version 31180 (0.0007) -[2023-10-14 14:42:42,607][75949] Updated weights for policy 0, policy_version 31241 (0.0008) -[2023-10-14 14:42:42,886][75950] Updated weights for policy 1, policy_version 31190 (0.0007) -[2023-10-14 14:42:42,973][75949] Updated weights for policy 0, policy_version 31251 (0.0010) -[2023-10-14 14:42:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 63897600. Throughput: 0: 1670.1, 1: 1657.6. Samples: 15994024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:42:43,165][74987] Avg episode reward: [(0, '23.790'), (1, '25.100')] -[2023-10-14 14:42:43,255][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000031200_31948800.pth... -[2023-10-14 14:42:43,255][75950] Updated weights for policy 1, policy_version 31200 (0.0007) -[2023-10-14 14:42:43,284][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000029632_30343168.pth -[2023-10-14 14:42:43,288][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000031200_31948800.pth -[2023-10-14 14:42:43,347][75949] Updated weights for policy 0, policy_version 31261 (0.0010) -[2023-10-14 14:42:43,451][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000031264_32014336.pth... -[2023-10-14 14:42:43,479][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000029696_30408704.pth -[2023-10-14 14:42:43,483][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000031264_32014336.pth -[2023-10-14 14:42:47,402][75950] Updated weights for policy 1, policy_version 31210 (0.0008) -[2023-10-14 14:42:47,522][75949] Updated weights for policy 0, policy_version 31271 (0.0009) -[2023-10-14 14:42:47,766][75950] Updated weights for policy 1, policy_version 31220 (0.0009) -[2023-10-14 14:42:47,892][75949] Updated weights for policy 0, policy_version 31281 (0.0009) -[2023-10-14 14:42:48,127][75950] Updated weights for policy 1, policy_version 31230 (0.0007) -[2023-10-14 14:42:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63963136. Throughput: 0: 1672.8, 1: 1668.8. Samples: 16003936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:42:48,164][74987] Avg episode reward: [(0, '22.900'), (1, '25.960')] -[2023-10-14 14:42:48,256][75949] Updated weights for policy 0, policy_version 31291 (0.0008) -[2023-10-14 14:42:52,176][75950] Updated weights for policy 1, policy_version 31240 (0.0009) -[2023-10-14 14:42:52,409][75949] Updated weights for policy 0, policy_version 31301 (0.0008) -[2023-10-14 14:42:52,538][75950] Updated weights for policy 1, policy_version 31250 (0.0009) -[2023-10-14 14:42:52,796][75949] Updated weights for policy 0, policy_version 31311 (0.0007) -[2023-10-14 14:42:52,908][75950] Updated weights for policy 1, policy_version 31260 (0.0007) -[2023-10-14 14:42:53,164][74987] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 64061440. Throughput: 0: 1671.7, 1: 1675.8. Samples: 16024704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:42:53,164][74987] Avg episode reward: [(0, '25.020'), (1, '28.250')] -[2023-10-14 14:42:53,166][75949] Updated weights for policy 0, policy_version 31321 (0.0008) -[2023-10-14 14:42:57,010][75950] Updated weights for policy 1, policy_version 31270 (0.0009) -[2023-10-14 14:42:57,197][75949] Updated weights for policy 0, policy_version 31331 (0.0008) -[2023-10-14 14:42:57,374][75950] Updated weights for policy 1, policy_version 31280 (0.0008) -[2023-10-14 14:42:57,562][75949] Updated weights for policy 0, policy_version 31341 (0.0008) -[2023-10-14 14:42:57,736][75950] Updated weights for policy 1, policy_version 31290 (0.0008) -[2023-10-14 14:42:57,939][75949] Updated weights for policy 0, policy_version 31351 (0.0007) -[2023-10-14 14:42:58,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 64126976. Throughput: 0: 1657.7, 1: 1657.9. Samples: 16043788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:42:58,165][74987] Avg episode reward: [(0, '24.530'), (1, '26.990')] -[2023-10-14 14:43:01,923][75950] Updated weights for policy 1, policy_version 31300 (0.0009) -[2023-10-14 14:43:02,037][75949] Updated weights for policy 0, policy_version 31361 (0.0007) -[2023-10-14 14:43:02,281][75950] Updated weights for policy 1, policy_version 31310 (0.0008) -[2023-10-14 14:43:02,407][75949] Updated weights for policy 0, policy_version 31371 (0.0009) -[2023-10-14 14:43:02,644][75950] Updated weights for policy 1, policy_version 31320 (0.0007) -[2023-10-14 14:43:02,783][75949] Updated weights for policy 0, policy_version 31381 (0.0008) -[2023-10-14 14:43:03,161][75949] Updated weights for policy 0, policy_version 31391 (0.0007) -[2023-10-14 14:43:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 64192512. Throughput: 0: 1673.1, 1: 1678.6. Samples: 16054574. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 14:43:03,165][74987] Avg episode reward: [(0, '24.510'), (1, '25.250')] -[2023-10-14 14:43:06,930][75950] Updated weights for policy 1, policy_version 31330 (0.0008) -[2023-10-14 14:43:06,984][75949] Updated weights for policy 0, policy_version 31401 (0.0007) -[2023-10-14 14:43:07,342][75949] Updated weights for policy 0, policy_version 31411 (0.0007) -[2023-10-14 14:43:07,346][75950] Updated weights for policy 1, policy_version 31340 (0.0009) -[2023-10-14 14:43:07,708][75949] Updated weights for policy 0, policy_version 31421 (0.0009) -[2023-10-14 14:43:07,714][75950] Updated weights for policy 1, policy_version 31350 (0.0008) -[2023-10-14 14:43:08,084][75950] Updated weights for policy 1, policy_version 31360 (0.0009) -[2023-10-14 14:43:08,164][74987] Fps is (10 sec: 16384.1, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 64290816. Throughput: 0: 1671.1, 1: 1671.6. Samples: 16074772. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 14:43:08,165][74987] Avg episode reward: [(0, '23.810'), (1, '27.310')] -[2023-10-14 14:43:11,777][75949] Updated weights for policy 0, policy_version 31431 (0.0008) -[2023-10-14 14:43:12,097][75950] Updated weights for policy 1, policy_version 31370 (0.0007) -[2023-10-14 14:43:12,141][75949] Updated weights for policy 0, policy_version 31441 (0.0007) -[2023-10-14 14:43:12,463][75950] Updated weights for policy 1, policy_version 31380 (0.0007) -[2023-10-14 14:43:12,506][75949] Updated weights for policy 0, policy_version 31451 (0.0009) -[2023-10-14 14:43:12,833][75950] Updated weights for policy 1, policy_version 31390 (0.0007) -[2023-10-14 14:43:13,164][74987] Fps is (10 sec: 16384.2, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 64356352. Throughput: 0: 1659.5, 1: 1656.2. Samples: 16093408. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 14:43:13,164][74987] Avg episode reward: [(0, '25.160'), (1, '26.650')] -[2023-10-14 14:43:16,536][75949] Updated weights for policy 0, policy_version 31461 (0.0009) -[2023-10-14 14:43:16,855][75950] Updated weights for policy 1, policy_version 31400 (0.0009) -[2023-10-14 14:43:16,895][75949] Updated weights for policy 0, policy_version 31471 (0.0009) -[2023-10-14 14:43:17,213][75950] Updated weights for policy 1, policy_version 31410 (0.0007) -[2023-10-14 14:43:17,263][75949] Updated weights for policy 0, policy_version 31481 (0.0008) -[2023-10-14 14:43:17,589][75950] Updated weights for policy 1, policy_version 31420 (0.0009) -[2023-10-14 14:43:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 64421888. Throughput: 0: 1676.9, 1: 1670.0. Samples: 16104588. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 14:43:18,164][74987] Avg episode reward: [(0, '25.020'), (1, '25.480')] -[2023-10-14 14:43:21,328][75949] Updated weights for policy 0, policy_version 31491 (0.0008) -[2023-10-14 14:43:21,573][75950] Updated weights for policy 1, policy_version 31430 (0.0008) -[2023-10-14 14:43:21,699][75949] Updated weights for policy 0, policy_version 31501 (0.0007) -[2023-10-14 14:43:21,941][75950] Updated weights for policy 1, policy_version 31440 (0.0008) -[2023-10-14 14:43:22,069][75949] Updated weights for policy 0, policy_version 31511 (0.0009) -[2023-10-14 14:43:22,312][75950] Updated weights for policy 1, policy_version 31450 (0.0010) -[2023-10-14 14:43:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 64487424. Throughput: 0: 1665.7, 1: 1663.7. Samples: 16124456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:43:23,164][74987] Avg episode reward: [(0, '23.850'), (1, '28.490')] -[2023-10-14 14:43:25,924][75949] Updated weights for policy 0, policy_version 31521 (0.0008) -[2023-10-14 14:43:26,296][75949] Updated weights for policy 0, policy_version 31531 (0.0008) -[2023-10-14 14:43:26,626][75950] Updated weights for policy 1, policy_version 31460 (0.0009) -[2023-10-14 14:43:26,672][75949] Updated weights for policy 0, policy_version 31541 (0.0008) -[2023-10-14 14:43:26,993][75950] Updated weights for policy 1, policy_version 31470 (0.0008) -[2023-10-14 14:43:27,036][75949] Updated weights for policy 0, policy_version 31551 (0.0008) -[2023-10-14 14:43:27,361][75950] Updated weights for policy 1, policy_version 31480 (0.0009) -[2023-10-14 14:43:28,164][74987] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 64552960. Throughput: 0: 1660.8, 1: 1654.7. Samples: 16143218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:43:28,165][74987] Avg episode reward: [(0, '22.630'), (1, '27.480')] -[2023-10-14 14:43:31,172][75949] Updated weights for policy 0, policy_version 31561 (0.0008) -[2023-10-14 14:43:31,421][75950] Updated weights for policy 1, policy_version 31490 (0.0009) -[2023-10-14 14:43:31,530][75949] Updated weights for policy 0, policy_version 31571 (0.0009) -[2023-10-14 14:43:31,790][75950] Updated weights for policy 1, policy_version 31500 (0.0008) -[2023-10-14 14:43:31,905][75949] Updated weights for policy 0, policy_version 31581 (0.0009) -[2023-10-14 14:43:32,151][75950] Updated weights for policy 1, policy_version 31510 (0.0009) -[2023-10-14 14:43:32,523][75950] Updated weights for policy 1, policy_version 31520 (0.0009) -[2023-10-14 14:43:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 64618496. Throughput: 0: 1682.8, 1: 1669.3. Samples: 16154782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:43:33,165][74987] Avg episode reward: [(0, '23.780'), (1, '26.150')] -[2023-10-14 14:43:36,098][75949] Updated weights for policy 0, policy_version 31591 (0.0008) -[2023-10-14 14:43:36,468][75949] Updated weights for policy 0, policy_version 31601 (0.0009) -[2023-10-14 14:43:36,609][75950] Updated weights for policy 1, policy_version 31530 (0.0009) -[2023-10-14 14:43:36,840][75949] Updated weights for policy 0, policy_version 31611 (0.0010) -[2023-10-14 14:43:36,971][75950] Updated weights for policy 1, policy_version 31540 (0.0008) -[2023-10-14 14:43:37,337][75950] Updated weights for policy 1, policy_version 31550 (0.0010) -[2023-10-14 14:43:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 64684032. Throughput: 0: 1662.9, 1: 1656.6. Samples: 16174080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:43:38,164][74987] Avg episode reward: [(0, '23.050'), (1, '29.080')] -[2023-10-14 14:43:38,165][75801] Saving new best policy, reward=29.080! -[2023-10-14 14:43:41,173][75949] Updated weights for policy 0, policy_version 31621 (0.0008) -[2023-10-14 14:43:41,268][75950] Updated weights for policy 1, policy_version 31560 (0.0008) -[2023-10-14 14:43:41,560][75949] Updated weights for policy 0, policy_version 31631 (0.0010) -[2023-10-14 14:43:41,629][75950] Updated weights for policy 1, policy_version 31570 (0.0009) -[2023-10-14 14:43:41,923][75949] Updated weights for policy 0, policy_version 31641 (0.0009) -[2023-10-14 14:43:41,999][75950] Updated weights for policy 1, policy_version 31580 (0.0009) -[2023-10-14 14:43:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 64749568. Throughput: 0: 1669.8, 1: 1660.7. Samples: 16193660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:43:43,165][74987] Avg episode reward: [(0, '23.350'), (1, '26.300')] -[2023-10-14 14:43:45,773][75949] Updated weights for policy 0, policy_version 31651 (0.0010) -[2023-10-14 14:43:46,144][75949] Updated weights for policy 0, policy_version 31661 (0.0008) -[2023-10-14 14:43:46,184][75950] Updated weights for policy 1, policy_version 31590 (0.0008) -[2023-10-14 14:43:46,520][75949] Updated weights for policy 0, policy_version 31671 (0.0009) -[2023-10-14 14:43:46,556][75950] Updated weights for policy 1, policy_version 31600 (0.0008) -[2023-10-14 14:43:46,913][75950] Updated weights for policy 1, policy_version 31610 (0.0009) -[2023-10-14 14:43:48,164][74987] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 64815104. Throughput: 0: 1682.7, 1: 1664.2. Samples: 16205184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:43:48,165][74987] Avg episode reward: [(0, '23.960'), (1, '27.600')] -[2023-10-14 14:43:50,759][75949] Updated weights for policy 0, policy_version 31681 (0.0008) -[2023-10-14 14:43:50,947][75950] Updated weights for policy 1, policy_version 31620 (0.0010) -[2023-10-14 14:43:51,125][75949] Updated weights for policy 0, policy_version 31691 (0.0007) -[2023-10-14 14:43:51,324][75950] Updated weights for policy 1, policy_version 31630 (0.0008) -[2023-10-14 14:43:51,491][75949] Updated weights for policy 0, policy_version 31701 (0.0008) -[2023-10-14 14:43:51,692][75950] Updated weights for policy 1, policy_version 31640 (0.0009) -[2023-10-14 14:43:51,858][75949] Updated weights for policy 0, policy_version 31711 (0.0008) -[2023-10-14 14:43:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 64880640. Throughput: 0: 1659.7, 1: 1658.3. Samples: 16224078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:43:53,164][74987] Avg episode reward: [(0, '22.560'), (1, '27.310')] -[2023-10-14 14:43:55,816][75950] Updated weights for policy 1, policy_version 31650 (0.0009) -[2023-10-14 14:43:55,837][75949] Updated weights for policy 0, policy_version 31721 (0.0008) -[2023-10-14 14:43:56,191][75950] Updated weights for policy 1, policy_version 31660 (0.0008) -[2023-10-14 14:43:56,201][75949] Updated weights for policy 0, policy_version 31731 (0.0009) -[2023-10-14 14:43:56,547][75950] Updated weights for policy 1, policy_version 31670 (0.0007) -[2023-10-14 14:43:56,562][75949] Updated weights for policy 0, policy_version 31741 (0.0009) -[2023-10-14 14:43:56,913][75950] Updated weights for policy 1, policy_version 31680 (0.0011) -[2023-10-14 14:43:58,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 64946176. Throughput: 0: 1682.3, 1: 1668.8. Samples: 16244208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:43:58,164][74987] Avg episode reward: [(0, '26.040'), (1, '27.090')] -[2023-10-14 14:44:00,635][75949] Updated weights for policy 0, policy_version 31751 (0.0008) -[2023-10-14 14:44:00,990][75950] Updated weights for policy 1, policy_version 31690 (0.0007) -[2023-10-14 14:44:01,002][75949] Updated weights for policy 0, policy_version 31761 (0.0008) -[2023-10-14 14:44:01,360][75950] Updated weights for policy 1, policy_version 31700 (0.0008) -[2023-10-14 14:44:01,361][75949] Updated weights for policy 0, policy_version 31771 (0.0009) -[2023-10-14 14:44:01,721][75950] Updated weights for policy 1, policy_version 31710 (0.0009) -[2023-10-14 14:44:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 65011712. Throughput: 0: 1678.4, 1: 1675.6. Samples: 16255522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:44:03,164][74987] Avg episode reward: [(0, '22.680'), (1, '28.400')] -[2023-10-14 14:44:05,228][75949] Updated weights for policy 0, policy_version 31781 (0.0008) -[2023-10-14 14:44:05,595][75949] Updated weights for policy 0, policy_version 31791 (0.0011) -[2023-10-14 14:44:05,785][75950] Updated weights for policy 1, policy_version 31720 (0.0008) -[2023-10-14 14:44:05,978][75949] Updated weights for policy 0, policy_version 31801 (0.0009) -[2023-10-14 14:44:06,147][75950] Updated weights for policy 1, policy_version 31730 (0.0010) -[2023-10-14 14:44:06,519][75950] Updated weights for policy 1, policy_version 31740 (0.0008) -[2023-10-14 14:44:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65077248. Throughput: 0: 1667.2, 1: 1656.2. Samples: 16274008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:44:08,164][74987] Avg episode reward: [(0, '25.450'), (1, '25.480')] -[2023-10-14 14:44:10,180][75949] Updated weights for policy 0, policy_version 31811 (0.0008) -[2023-10-14 14:44:10,535][75950] Updated weights for policy 1, policy_version 31750 (0.0010) -[2023-10-14 14:44:10,549][75949] Updated weights for policy 0, policy_version 31821 (0.0007) -[2023-10-14 14:44:10,902][75950] Updated weights for policy 1, policy_version 31760 (0.0007) -[2023-10-14 14:44:10,919][75949] Updated weights for policy 0, policy_version 31831 (0.0007) -[2023-10-14 14:44:11,268][75950] Updated weights for policy 1, policy_version 31770 (0.0008) -[2023-10-14 14:44:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 65142784. Throughput: 0: 1681.5, 1: 1679.8. Samples: 16294474. Policy #0 lag: (min: 24.0, avg: 47.8, max: 56.0) -[2023-10-14 14:44:13,165][74987] Avg episode reward: [(0, '23.650'), (1, '28.030')] -[2023-10-14 14:44:15,189][75950] Updated weights for policy 1, policy_version 31780 (0.0009) -[2023-10-14 14:44:15,190][75949] Updated weights for policy 0, policy_version 31841 (0.0008) -[2023-10-14 14:44:15,549][75950] Updated weights for policy 1, policy_version 31790 (0.0009) -[2023-10-14 14:44:15,559][75949] Updated weights for policy 0, policy_version 31851 (0.0008) -[2023-10-14 14:44:15,919][75950] Updated weights for policy 1, policy_version 31800 (0.0008) -[2023-10-14 14:44:15,926][75949] Updated weights for policy 0, policy_version 31861 (0.0010) -[2023-10-14 14:44:16,299][75949] Updated weights for policy 0, policy_version 31871 (0.0009) -[2023-10-14 14:44:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 65208320. Throughput: 0: 1666.0, 1: 1672.3. Samples: 16305004. Policy #0 lag: (min: 24.0, avg: 47.8, max: 56.0) -[2023-10-14 14:44:18,165][74987] Avg episode reward: [(0, '25.150'), (1, '26.380')] -[2023-10-14 14:44:19,945][75950] Updated weights for policy 1, policy_version 31810 (0.0008) -[2023-10-14 14:44:20,304][75950] Updated weights for policy 1, policy_version 31820 (0.0008) -[2023-10-14 14:44:20,481][75949] Updated weights for policy 0, policy_version 31881 (0.0008) -[2023-10-14 14:44:20,671][75950] Updated weights for policy 1, policy_version 31830 (0.0008) -[2023-10-14 14:44:20,855][75949] Updated weights for policy 0, policy_version 31891 (0.0008) -[2023-10-14 14:44:21,042][75950] Updated weights for policy 1, policy_version 31840 (0.0007) -[2023-10-14 14:44:21,225][75949] Updated weights for policy 0, policy_version 31901 (0.0009) -[2023-10-14 14:44:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65273856. Throughput: 0: 1666.5, 1: 1666.0. Samples: 16324044. Policy #0 lag: (min: 24.0, avg: 47.8, max: 56.0) -[2023-10-14 14:44:23,165][74987] Avg episode reward: [(0, '23.450'), (1, '26.310')] -[2023-10-14 14:44:25,101][75950] Updated weights for policy 1, policy_version 31850 (0.0009) -[2023-10-14 14:44:25,267][75949] Updated weights for policy 0, policy_version 31911 (0.0008) -[2023-10-14 14:44:25,462][75950] Updated weights for policy 1, policy_version 31860 (0.0008) -[2023-10-14 14:44:25,646][75949] Updated weights for policy 0, policy_version 31921 (0.0008) -[2023-10-14 14:44:25,821][75950] Updated weights for policy 1, policy_version 31870 (0.0007) -[2023-10-14 14:44:26,015][75949] Updated weights for policy 0, policy_version 31931 (0.0007) -[2023-10-14 14:44:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65339392. Throughput: 0: 1675.8, 1: 1681.8. Samples: 16344752. Policy #0 lag: (min: 24.0, avg: 47.8, max: 56.0) -[2023-10-14 14:44:28,165][74987] Avg episode reward: [(0, '23.600'), (1, '28.920')] -[2023-10-14 14:44:29,989][75950] Updated weights for policy 1, policy_version 31880 (0.0007) -[2023-10-14 14:44:30,109][75949] Updated weights for policy 0, policy_version 31941 (0.0008) -[2023-10-14 14:44:30,346][75950] Updated weights for policy 1, policy_version 31890 (0.0008) -[2023-10-14 14:44:30,496][75949] Updated weights for policy 0, policy_version 31951 (0.0007) -[2023-10-14 14:44:30,714][75950] Updated weights for policy 1, policy_version 31900 (0.0008) -[2023-10-14 14:44:30,864][75949] Updated weights for policy 0, policy_version 31961 (0.0008) -[2023-10-14 14:44:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65404928. Throughput: 0: 1655.7, 1: 1662.6. Samples: 16354508. Policy #0 lag: (min: 24.0, avg: 47.8, max: 56.0) -[2023-10-14 14:44:33,164][74987] Avg episode reward: [(0, '24.870'), (1, '27.470')] -[2023-10-14 14:44:34,759][75950] Updated weights for policy 1, policy_version 31910 (0.0011) -[2023-10-14 14:44:34,946][75949] Updated weights for policy 0, policy_version 31971 (0.0009) -[2023-10-14 14:44:35,132][75950] Updated weights for policy 1, policy_version 31920 (0.0009) -[2023-10-14 14:44:35,314][75949] Updated weights for policy 0, policy_version 31981 (0.0007) -[2023-10-14 14:44:35,497][75950] Updated weights for policy 1, policy_version 31930 (0.0008) -[2023-10-14 14:44:35,699][75949] Updated weights for policy 0, policy_version 31991 (0.0007) -[2023-10-14 14:44:38,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65470464. Throughput: 0: 1667.4, 1: 1666.7. Samples: 16374112. Policy #0 lag: (min: 16.0, avg: 42.3, max: 48.0) -[2023-10-14 14:44:38,164][74987] Avg episode reward: [(0, '24.010'), (1, '26.190')] -[2023-10-14 14:44:39,681][75950] Updated weights for policy 1, policy_version 31940 (0.0008) -[2023-10-14 14:44:39,713][75949] Updated weights for policy 0, policy_version 32001 (0.0007) -[2023-10-14 14:44:40,076][75949] Updated weights for policy 0, policy_version 32011 (0.0009) -[2023-10-14 14:44:40,079][75950] Updated weights for policy 1, policy_version 31950 (0.0009) -[2023-10-14 14:44:40,453][75950] Updated weights for policy 1, policy_version 31960 (0.0008) -[2023-10-14 14:44:40,453][75949] Updated weights for policy 0, policy_version 32021 (0.0009) -[2023-10-14 14:44:40,820][75949] Updated weights for policy 0, policy_version 32031 (0.0008) -[2023-10-14 14:44:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 65536000. Throughput: 0: 1676.3, 1: 1678.6. Samples: 16395178. Policy #0 lag: (min: 16.0, avg: 42.3, max: 48.0) -[2023-10-14 14:44:43,165][74987] Avg episode reward: [(0, '23.090'), (1, '27.710')] -[2023-10-14 14:44:43,178][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000032032_32800768.pth... -[2023-10-14 14:44:43,178][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000031968_32735232.pth... -[2023-10-14 14:44:43,213][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000030400_31129600.pth -[2023-10-14 14:44:43,215][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000030464_31195136.pth -[2023-10-14 14:44:44,394][75950] Updated weights for policy 1, policy_version 31970 (0.0009) -[2023-10-14 14:44:44,764][75950] Updated weights for policy 1, policy_version 31980 (0.0008) -[2023-10-14 14:44:44,766][75949] Updated weights for policy 0, policy_version 32041 (0.0007) -[2023-10-14 14:44:45,130][75950] Updated weights for policy 1, policy_version 31990 (0.0009) -[2023-10-14 14:44:45,135][75949] Updated weights for policy 0, policy_version 32051 (0.0007) -[2023-10-14 14:44:45,494][75950] Updated weights for policy 1, policy_version 32000 (0.0007) -[2023-10-14 14:44:45,503][75949] Updated weights for policy 0, policy_version 32061 (0.0008) -[2023-10-14 14:44:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 65601536. Throughput: 0: 1653.6, 1: 1651.1. Samples: 16404236. Policy #0 lag: (min: 16.0, avg: 42.3, max: 48.0) -[2023-10-14 14:44:48,165][74987] Avg episode reward: [(0, '23.030'), (1, '26.930')] -[2023-10-14 14:44:49,623][75949] Updated weights for policy 0, policy_version 32071 (0.0009) -[2023-10-14 14:44:49,719][75950] Updated weights for policy 1, policy_version 32010 (0.0008) -[2023-10-14 14:44:49,986][75949] Updated weights for policy 0, policy_version 32081 (0.0009) -[2023-10-14 14:44:50,081][75950] Updated weights for policy 1, policy_version 32020 (0.0009) -[2023-10-14 14:44:50,361][75949] Updated weights for policy 0, policy_version 32091 (0.0007) -[2023-10-14 14:44:50,442][75950] Updated weights for policy 1, policy_version 32030 (0.0009) -[2023-10-14 14:44:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65667072. Throughput: 0: 1670.6, 1: 1675.8. Samples: 16424594. Policy #0 lag: (min: 16.0, avg: 42.3, max: 48.0) -[2023-10-14 14:44:53,165][74987] Avg episode reward: [(0, '22.870'), (1, '25.900')] -[2023-10-14 14:44:54,583][75949] Updated weights for policy 0, policy_version 32101 (0.0009) -[2023-10-14 14:44:54,590][75950] Updated weights for policy 1, policy_version 32040 (0.0008) -[2023-10-14 14:44:54,951][75949] Updated weights for policy 0, policy_version 32111 (0.0008) -[2023-10-14 14:44:54,954][75950] Updated weights for policy 1, policy_version 32050 (0.0009) -[2023-10-14 14:44:55,318][75950] Updated weights for policy 1, policy_version 32060 (0.0009) -[2023-10-14 14:44:55,326][75949] Updated weights for policy 0, policy_version 32121 (0.0008) -[2023-10-14 14:44:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65732608. Throughput: 0: 1670.1, 1: 1674.4. Samples: 16444976. Policy #0 lag: (min: 16.0, avg: 42.3, max: 48.0) -[2023-10-14 14:44:58,164][74987] Avg episode reward: [(0, '24.070'), (1, '26.620')] -[2023-10-14 14:44:59,414][75949] Updated weights for policy 0, policy_version 32131 (0.0010) -[2023-10-14 14:44:59,686][75950] Updated weights for policy 1, policy_version 32070 (0.0009) -[2023-10-14 14:44:59,779][75949] Updated weights for policy 0, policy_version 32141 (0.0009) -[2023-10-14 14:45:00,055][75950] Updated weights for policy 1, policy_version 32080 (0.0008) -[2023-10-14 14:45:00,142][75949] Updated weights for policy 0, policy_version 32151 (0.0010) -[2023-10-14 14:45:00,420][75950] Updated weights for policy 1, policy_version 32090 (0.0008) -[2023-10-14 14:45:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 65798144. Throughput: 0: 1656.4, 1: 1658.8. Samples: 16454186. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:45:03,165][74987] Avg episode reward: [(0, '23.960'), (1, '26.640')] -[2023-10-14 14:45:04,287][75949] Updated weights for policy 0, policy_version 32161 (0.0010) -[2023-10-14 14:45:04,581][75950] Updated weights for policy 1, policy_version 32100 (0.0008) -[2023-10-14 14:45:04,661][75949] Updated weights for policy 0, policy_version 32171 (0.0009) -[2023-10-14 14:45:04,952][75950] Updated weights for policy 1, policy_version 32110 (0.0009) -[2023-10-14 14:45:05,031][75949] Updated weights for policy 0, policy_version 32181 (0.0008) -[2023-10-14 14:45:05,312][75950] Updated weights for policy 1, policy_version 32120 (0.0008) -[2023-10-14 14:45:05,391][75949] Updated weights for policy 0, policy_version 32191 (0.0009) -[2023-10-14 14:45:08,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65863680. Throughput: 0: 1675.7, 1: 1668.6. Samples: 16474536. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:45:08,164][74987] Avg episode reward: [(0, '22.820'), (1, '27.040')] -[2023-10-14 14:45:09,204][75950] Updated weights for policy 1, policy_version 32130 (0.0009) -[2023-10-14 14:45:09,552][75949] Updated weights for policy 0, policy_version 32201 (0.0008) -[2023-10-14 14:45:09,579][75950] Updated weights for policy 1, policy_version 32140 (0.0008) -[2023-10-14 14:45:09,924][75949] Updated weights for policy 0, policy_version 32211 (0.0007) -[2023-10-14 14:45:09,945][75950] Updated weights for policy 1, policy_version 32150 (0.0009) -[2023-10-14 14:45:10,288][75949] Updated weights for policy 0, policy_version 32221 (0.0007) -[2023-10-14 14:45:10,306][75950] Updated weights for policy 1, policy_version 32160 (0.0009) -[2023-10-14 14:45:13,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 65929216. Throughput: 0: 1672.1, 1: 1670.5. Samples: 16495168. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:45:13,164][74987] Avg episode reward: [(0, '22.310'), (1, '26.580')] -[2023-10-14 14:45:14,308][75949] Updated weights for policy 0, policy_version 32231 (0.0007) -[2023-10-14 14:45:14,439][75950] Updated weights for policy 1, policy_version 32170 (0.0007) -[2023-10-14 14:45:14,670][75949] Updated weights for policy 0, policy_version 32241 (0.0008) -[2023-10-14 14:45:14,814][75950] Updated weights for policy 1, policy_version 32180 (0.0009) -[2023-10-14 14:45:15,042][75949] Updated weights for policy 0, policy_version 32251 (0.0008) -[2023-10-14 14:45:15,172][75950] Updated weights for policy 1, policy_version 32190 (0.0007) -[2023-10-14 14:45:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65994752. Throughput: 0: 1665.0, 1: 1664.4. Samples: 16504330. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:45:18,164][74987] Avg episode reward: [(0, '23.010'), (1, '27.850')] -[2023-10-14 14:45:19,231][75949] Updated weights for policy 0, policy_version 32261 (0.0008) -[2023-10-14 14:45:19,361][75950] Updated weights for policy 1, policy_version 32200 (0.0009) -[2023-10-14 14:45:19,593][75949] Updated weights for policy 0, policy_version 32271 (0.0010) -[2023-10-14 14:45:19,721][75950] Updated weights for policy 1, policy_version 32210 (0.0009) -[2023-10-14 14:45:19,967][75949] Updated weights for policy 0, policy_version 32281 (0.0008) -[2023-10-14 14:45:20,097][75950] Updated weights for policy 1, policy_version 32220 (0.0009) -[2023-10-14 14:45:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66060288. Throughput: 0: 1673.9, 1: 1675.6. Samples: 16524842. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 14:45:23,164][74987] Avg episode reward: [(0, '24.390'), (1, '26.170')] -[2023-10-14 14:45:24,067][75949] Updated weights for policy 0, policy_version 32291 (0.0007) -[2023-10-14 14:45:24,083][75950] Updated weights for policy 1, policy_version 32230 (0.0009) -[2023-10-14 14:45:24,431][75949] Updated weights for policy 0, policy_version 32301 (0.0008) -[2023-10-14 14:45:24,477][75950] Updated weights for policy 1, policy_version 32240 (0.0007) -[2023-10-14 14:45:24,802][75949] Updated weights for policy 0, policy_version 32311 (0.0007) -[2023-10-14 14:45:24,852][75950] Updated weights for policy 1, policy_version 32250 (0.0008) -[2023-10-14 14:45:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66125824. Throughput: 0: 1664.8, 1: 1674.8. Samples: 16545460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:45:28,165][74987] Avg episode reward: [(0, '24.460'), (1, '26.000')] -[2023-10-14 14:45:28,890][75949] Updated weights for policy 0, policy_version 32321 (0.0009) -[2023-10-14 14:45:28,895][75950] Updated weights for policy 1, policy_version 32260 (0.0007) -[2023-10-14 14:45:29,252][75949] Updated weights for policy 0, policy_version 32331 (0.0009) -[2023-10-14 14:45:29,268][75950] Updated weights for policy 1, policy_version 32270 (0.0008) -[2023-10-14 14:45:29,630][75949] Updated weights for policy 0, policy_version 32341 (0.0009) -[2023-10-14 14:45:29,631][75950] Updated weights for policy 1, policy_version 32280 (0.0008) -[2023-10-14 14:45:29,993][75949] Updated weights for policy 0, policy_version 32351 (0.0008) -[2023-10-14 14:45:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66191360. Throughput: 0: 1664.9, 1: 1674.7. Samples: 16554518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:45:33,164][74987] Avg episode reward: [(0, '25.540'), (1, '27.740')] -[2023-10-14 14:45:33,648][75950] Updated weights for policy 1, policy_version 32290 (0.0008) -[2023-10-14 14:45:33,884][75949] Updated weights for policy 0, policy_version 32361 (0.0007) -[2023-10-14 14:45:34,013][75950] Updated weights for policy 1, policy_version 32300 (0.0008) -[2023-10-14 14:45:34,250][75949] Updated weights for policy 0, policy_version 32371 (0.0007) -[2023-10-14 14:45:34,379][75950] Updated weights for policy 1, policy_version 32310 (0.0008) -[2023-10-14 14:45:34,618][75949] Updated weights for policy 0, policy_version 32381 (0.0008) -[2023-10-14 14:45:34,750][75950] Updated weights for policy 1, policy_version 32320 (0.0008) -[2023-10-14 14:45:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 66256896. Throughput: 0: 1672.7, 1: 1672.3. Samples: 16575116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:45:38,165][74987] Avg episode reward: [(0, '24.220'), (1, '26.270')] -[2023-10-14 14:45:38,731][75949] Updated weights for policy 0, policy_version 32391 (0.0008) -[2023-10-14 14:45:39,068][75950] Updated weights for policy 1, policy_version 32330 (0.0007) -[2023-10-14 14:45:39,104][75949] Updated weights for policy 0, policy_version 32401 (0.0008) -[2023-10-14 14:45:39,436][75950] Updated weights for policy 1, policy_version 32340 (0.0007) -[2023-10-14 14:45:39,469][75949] Updated weights for policy 0, policy_version 32411 (0.0008) -[2023-10-14 14:45:39,800][75950] Updated weights for policy 1, policy_version 32350 (0.0008) -[2023-10-14 14:45:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 66322432. Throughput: 0: 1677.4, 1: 1673.4. Samples: 16595762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:45:43,164][74987] Avg episode reward: [(0, '23.500'), (1, '28.410')] -[2023-10-14 14:45:43,590][75949] Updated weights for policy 0, policy_version 32421 (0.0007) -[2023-10-14 14:45:43,872][75950] Updated weights for policy 1, policy_version 32360 (0.0008) -[2023-10-14 14:45:43,953][75949] Updated weights for policy 0, policy_version 32431 (0.0008) -[2023-10-14 14:45:44,233][75950] Updated weights for policy 1, policy_version 32370 (0.0008) -[2023-10-14 14:45:44,332][75949] Updated weights for policy 0, policy_version 32441 (0.0007) -[2023-10-14 14:45:44,594][75950] Updated weights for policy 1, policy_version 32380 (0.0008) -[2023-10-14 14:45:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66387968. Throughput: 0: 1676.2, 1: 1671.4. Samples: 16604828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:45:48,164][74987] Avg episode reward: [(0, '22.670'), (1, '28.000')] -[2023-10-14 14:45:48,479][75949] Updated weights for policy 0, policy_version 32451 (0.0009) -[2023-10-14 14:45:48,746][75950] Updated weights for policy 1, policy_version 32390 (0.0010) -[2023-10-14 14:45:48,839][75949] Updated weights for policy 0, policy_version 32461 (0.0007) -[2023-10-14 14:45:49,113][75950] Updated weights for policy 1, policy_version 32400 (0.0009) -[2023-10-14 14:45:49,211][75949] Updated weights for policy 0, policy_version 32471 (0.0007) -[2023-10-14 14:45:49,476][75950] Updated weights for policy 1, policy_version 32410 (0.0008) -[2023-10-14 14:45:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66453504. Throughput: 0: 1676.7, 1: 1671.0. Samples: 16625184. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) -[2023-10-14 14:45:53,165][74987] Avg episode reward: [(0, '23.520'), (1, '25.110')] -[2023-10-14 14:45:53,257][75949] Updated weights for policy 0, policy_version 32481 (0.0007) -[2023-10-14 14:45:53,578][75950] Updated weights for policy 1, policy_version 32420 (0.0009) -[2023-10-14 14:45:53,635][75949] Updated weights for policy 0, policy_version 32491 (0.0007) -[2023-10-14 14:45:53,945][75950] Updated weights for policy 1, policy_version 32430 (0.0008) -[2023-10-14 14:45:54,002][75949] Updated weights for policy 0, policy_version 32501 (0.0009) -[2023-10-14 14:45:54,311][75950] Updated weights for policy 1, policy_version 32440 (0.0009) -[2023-10-14 14:45:54,373][75949] Updated weights for policy 0, policy_version 32511 (0.0008) -[2023-10-14 14:45:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66519040. Throughput: 0: 1678.4, 1: 1666.5. Samples: 16645692. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) -[2023-10-14 14:45:58,164][74987] Avg episode reward: [(0, '23.010'), (1, '28.240')] -[2023-10-14 14:45:58,410][75950] Updated weights for policy 1, policy_version 32450 (0.0008) -[2023-10-14 14:45:58,522][75949] Updated weights for policy 0, policy_version 32521 (0.0007) -[2023-10-14 14:45:58,782][75950] Updated weights for policy 1, policy_version 32460 (0.0009) -[2023-10-14 14:45:58,891][75949] Updated weights for policy 0, policy_version 32531 (0.0008) -[2023-10-14 14:45:59,148][75950] Updated weights for policy 1, policy_version 32470 (0.0008) -[2023-10-14 14:45:59,265][75949] Updated weights for policy 0, policy_version 32541 (0.0010) -[2023-10-14 14:45:59,510][75950] Updated weights for policy 1, policy_version 32480 (0.0010) -[2023-10-14 14:46:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66584576. Throughput: 0: 1675.3, 1: 1663.9. Samples: 16654592. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) -[2023-10-14 14:46:03,165][74987] Avg episode reward: [(0, '23.940'), (1, '26.440')] -[2023-10-14 14:46:03,205][75949] Updated weights for policy 0, policy_version 32551 (0.0008) -[2023-10-14 14:46:03,575][75949] Updated weights for policy 0, policy_version 32561 (0.0009) -[2023-10-14 14:46:03,673][75950] Updated weights for policy 1, policy_version 32490 (0.0009) -[2023-10-14 14:46:03,937][75949] Updated weights for policy 0, policy_version 32571 (0.0008) -[2023-10-14 14:46:04,051][75950] Updated weights for policy 1, policy_version 32500 (0.0007) -[2023-10-14 14:46:04,407][75950] Updated weights for policy 1, policy_version 32510 (0.0007) -[2023-10-14 14:46:08,072][75949] Updated weights for policy 0, policy_version 32581 (0.0007) -[2023-10-14 14:46:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66650112. Throughput: 0: 1679.0, 1: 1667.3. Samples: 16675428. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) -[2023-10-14 14:46:08,165][74987] Avg episode reward: [(0, '23.330'), (1, '24.980')] -[2023-10-14 14:46:08,434][75950] Updated weights for policy 1, policy_version 32520 (0.0007) -[2023-10-14 14:46:08,453][75949] Updated weights for policy 0, policy_version 32591 (0.0007) -[2023-10-14 14:46:08,799][75950] Updated weights for policy 1, policy_version 32530 (0.0007) -[2023-10-14 14:46:08,817][75949] Updated weights for policy 0, policy_version 32601 (0.0007) -[2023-10-14 14:46:09,169][75950] Updated weights for policy 1, policy_version 32540 (0.0008) -[2023-10-14 14:46:12,777][75949] Updated weights for policy 0, policy_version 32611 (0.0008) -[2023-10-14 14:46:13,145][75949] Updated weights for policy 0, policy_version 32621 (0.0007) -[2023-10-14 14:46:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66715648. Throughput: 0: 1679.9, 1: 1666.5. Samples: 16696048. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) -[2023-10-14 14:46:13,164][74987] Avg episode reward: [(0, '24.880'), (1, '28.470')] -[2023-10-14 14:46:13,448][75950] Updated weights for policy 1, policy_version 32550 (0.0008) -[2023-10-14 14:46:13,518][75949] Updated weights for policy 0, policy_version 32631 (0.0008) -[2023-10-14 14:46:13,847][75950] Updated weights for policy 1, policy_version 32560 (0.0008) -[2023-10-14 14:46:14,203][75950] Updated weights for policy 1, policy_version 32570 (0.0011) -[2023-10-14 14:46:17,561][75949] Updated weights for policy 0, policy_version 32641 (0.0008) -[2023-10-14 14:46:17,940][75949] Updated weights for policy 0, policy_version 32651 (0.0008) -[2023-10-14 14:46:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66781184. Throughput: 0: 1684.0, 1: 1659.7. Samples: 16704982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:46:18,165][74987] Avg episode reward: [(0, '24.460'), (1, '27.250')] -[2023-10-14 14:46:18,318][75949] Updated weights for policy 0, policy_version 32661 (0.0010) -[2023-10-14 14:46:18,479][75950] Updated weights for policy 1, policy_version 32580 (0.0010) -[2023-10-14 14:46:18,679][75949] Updated weights for policy 0, policy_version 32671 (0.0008) -[2023-10-14 14:46:18,840][75950] Updated weights for policy 1, policy_version 32590 (0.0010) -[2023-10-14 14:46:19,196][75950] Updated weights for policy 1, policy_version 32600 (0.0010) -[2023-10-14 14:46:23,029][75949] Updated weights for policy 0, policy_version 32681 (0.0008) -[2023-10-14 14:46:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66846720. Throughput: 0: 1680.4, 1: 1658.0. Samples: 16725346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:46:23,164][74987] Avg episode reward: [(0, '24.900'), (1, '25.500')] -[2023-10-14 14:46:23,251][75950] Updated weights for policy 1, policy_version 32610 (0.0010) -[2023-10-14 14:46:23,388][75949] Updated weights for policy 0, policy_version 32691 (0.0007) -[2023-10-14 14:46:23,609][75950] Updated weights for policy 1, policy_version 32620 (0.0009) -[2023-10-14 14:46:23,756][75949] Updated weights for policy 0, policy_version 32701 (0.0008) -[2023-10-14 14:46:23,993][75950] Updated weights for policy 1, policy_version 32630 (0.0009) -[2023-10-14 14:46:24,363][75950] Updated weights for policy 1, policy_version 32640 (0.0009) -[2023-10-14 14:46:27,837][75949] Updated weights for policy 0, policy_version 32711 (0.0008) -[2023-10-14 14:46:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66912256. Throughput: 0: 1673.5, 1: 1662.4. Samples: 16745876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:46:28,164][74987] Avg episode reward: [(0, '22.880'), (1, '29.250')] -[2023-10-14 14:46:28,200][75949] Updated weights for policy 0, policy_version 32721 (0.0007) -[2023-10-14 14:46:28,568][75949] Updated weights for policy 0, policy_version 32731 (0.0007) -[2023-10-14 14:46:28,616][75950] Updated weights for policy 1, policy_version 32650 (0.0009) -[2023-10-14 14:46:28,978][75950] Updated weights for policy 1, policy_version 32660 (0.0009) -[2023-10-14 14:46:29,342][75950] Updated weights for policy 1, policy_version 32670 (0.0008) -[2023-10-14 14:46:29,416][75801] Saving new best policy, reward=29.250! -[2023-10-14 14:46:32,435][75949] Updated weights for policy 0, policy_version 32741 (0.0011) -[2023-10-14 14:46:32,805][75949] Updated weights for policy 0, policy_version 32751 (0.0011) -[2023-10-14 14:46:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66977792. Throughput: 0: 1679.7, 1: 1662.7. Samples: 16755236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:46:33,164][74987] Avg episode reward: [(0, '25.470'), (1, '28.150')] -[2023-10-14 14:46:33,177][75949] Updated weights for policy 0, policy_version 32761 (0.0010) -[2023-10-14 14:46:33,403][75950] Updated weights for policy 1, policy_version 32680 (0.0008) -[2023-10-14 14:46:33,775][75950] Updated weights for policy 1, policy_version 32690 (0.0008) -[2023-10-14 14:46:34,135][75950] Updated weights for policy 1, policy_version 32700 (0.0010) -[2023-10-14 14:46:37,281][75949] Updated weights for policy 0, policy_version 32771 (0.0008) -[2023-10-14 14:46:37,652][75949] Updated weights for policy 0, policy_version 32781 (0.0008) -[2023-10-14 14:46:38,021][75949] Updated weights for policy 0, policy_version 32791 (0.0008) -[2023-10-14 14:46:38,073][75950] Updated weights for policy 1, policy_version 32710 (0.0008) -[2023-10-14 14:46:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 67043328. Throughput: 0: 1675.6, 1: 1664.6. Samples: 16775492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:46:38,165][74987] Avg episode reward: [(0, '22.980'), (1, '26.700')] -[2023-10-14 14:46:38,442][75950] Updated weights for policy 1, policy_version 32720 (0.0008) -[2023-10-14 14:46:38,816][75950] Updated weights for policy 1, policy_version 32730 (0.0008) -[2023-10-14 14:46:42,036][75949] Updated weights for policy 0, policy_version 32801 (0.0008) -[2023-10-14 14:46:42,398][75949] Updated weights for policy 0, policy_version 32811 (0.0008) -[2023-10-14 14:46:42,770][75949] Updated weights for policy 0, policy_version 32821 (0.0008) -[2023-10-14 14:46:42,884][75950] Updated weights for policy 1, policy_version 32740 (0.0008) -[2023-10-14 14:46:43,137][75949] Updated weights for policy 0, policy_version 32831 (0.0007) -[2023-10-14 14:46:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67108864. Throughput: 0: 1668.2, 1: 1667.4. Samples: 16795792. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) -[2023-10-14 14:46:43,164][74987] Avg episode reward: [(0, '25.050'), (1, '28.910')] -[2023-10-14 14:46:43,171][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000032832_33619968.pth... -[2023-10-14 14:46:43,205][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000031264_32014336.pth -[2023-10-14 14:46:43,258][75950] Updated weights for policy 1, policy_version 32750 (0.0008) -[2023-10-14 14:46:43,624][75950] Updated weights for policy 1, policy_version 32760 (0.0009) -[2023-10-14 14:46:43,908][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000032768_33554432.pth... -[2023-10-14 14:46:43,938][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000031200_31948800.pth -[2023-10-14 14:46:47,184][75949] Updated weights for policy 0, policy_version 32841 (0.0010) -[2023-10-14 14:46:47,550][75949] Updated weights for policy 0, policy_version 32851 (0.0009) -[2023-10-14 14:46:47,599][75950] Updated weights for policy 1, policy_version 32770 (0.0010) -[2023-10-14 14:46:47,924][75949] Updated weights for policy 0, policy_version 32861 (0.0010) -[2023-10-14 14:46:47,961][75950] Updated weights for policy 1, policy_version 32780 (0.0008) -[2023-10-14 14:46:48,164][74987] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 67207168. Throughput: 0: 1686.0, 1: 1670.9. Samples: 16805656. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) -[2023-10-14 14:46:48,164][74987] Avg episode reward: [(0, '22.920'), (1, '28.030')] -[2023-10-14 14:46:48,331][75950] Updated weights for policy 1, policy_version 32790 (0.0008) -[2023-10-14 14:46:48,701][75950] Updated weights for policy 1, policy_version 32800 (0.0007) -[2023-10-14 14:46:52,065][75949] Updated weights for policy 0, policy_version 32871 (0.0007) -[2023-10-14 14:46:52,432][75949] Updated weights for policy 0, policy_version 32881 (0.0009) -[2023-10-14 14:46:52,689][75950] Updated weights for policy 1, policy_version 32810 (0.0007) -[2023-10-14 14:46:52,810][75949] Updated weights for policy 0, policy_version 32891 (0.0007) -[2023-10-14 14:46:53,056][75950] Updated weights for policy 1, policy_version 32820 (0.0009) -[2023-10-14 14:46:53,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 67272704. Throughput: 0: 1681.2, 1: 1667.4. Samples: 16826118. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) -[2023-10-14 14:46:53,165][74987] Avg episode reward: [(0, '25.110'), (1, '27.700')] -[2023-10-14 14:46:53,431][75950] Updated weights for policy 1, policy_version 32830 (0.0009) -[2023-10-14 14:46:57,095][75949] Updated weights for policy 0, policy_version 32901 (0.0009) -[2023-10-14 14:46:57,475][75949] Updated weights for policy 0, policy_version 32911 (0.0008) -[2023-10-14 14:46:57,604][75950] Updated weights for policy 1, policy_version 32840 (0.0007) -[2023-10-14 14:46:57,844][75949] Updated weights for policy 0, policy_version 32921 (0.0008) -[2023-10-14 14:46:57,964][75950] Updated weights for policy 1, policy_version 32850 (0.0007) -[2023-10-14 14:46:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 67338240. Throughput: 0: 1661.1, 1: 1664.6. Samples: 16845704. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) -[2023-10-14 14:46:58,164][74987] Avg episode reward: [(0, '23.510'), (1, '27.660')] -[2023-10-14 14:46:58,329][75950] Updated weights for policy 1, policy_version 32860 (0.0007) -[2023-10-14 14:47:01,828][75949] Updated weights for policy 0, policy_version 32931 (0.0010) -[2023-10-14 14:47:02,207][75949] Updated weights for policy 0, policy_version 32941 (0.0010) -[2023-10-14 14:47:02,533][75950] Updated weights for policy 1, policy_version 32870 (0.0009) -[2023-10-14 14:47:02,566][75949] Updated weights for policy 0, policy_version 32951 (0.0007) -[2023-10-14 14:47:02,921][75950] Updated weights for policy 1, policy_version 32880 (0.0009) -[2023-10-14 14:47:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 67403776. Throughput: 0: 1678.3, 1: 1681.8. Samples: 16856186. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 14:47:03,164][74987] Avg episode reward: [(0, '23.720'), (1, '29.450')] -[2023-10-14 14:47:03,282][75950] Updated weights for policy 1, policy_version 32890 (0.0008) -[2023-10-14 14:47:03,495][75801] Saving new best policy, reward=29.450! -[2023-10-14 14:47:06,633][75949] Updated weights for policy 0, policy_version 32961 (0.0008) -[2023-10-14 14:47:07,008][75949] Updated weights for policy 0, policy_version 32971 (0.0007) -[2023-10-14 14:47:07,369][75949] Updated weights for policy 0, policy_version 32981 (0.0007) -[2023-10-14 14:47:07,409][75950] Updated weights for policy 1, policy_version 32900 (0.0008) -[2023-10-14 14:47:07,747][75949] Updated weights for policy 0, policy_version 32991 (0.0008) -[2023-10-14 14:47:07,769][75950] Updated weights for policy 1, policy_version 32910 (0.0009) -[2023-10-14 14:47:08,137][75950] Updated weights for policy 1, policy_version 32920 (0.0008) -[2023-10-14 14:47:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 67469312. Throughput: 0: 1674.8, 1: 1684.0. Samples: 16876496. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 14:47:08,164][74987] Avg episode reward: [(0, '22.370'), (1, '29.290')] -[2023-10-14 14:47:11,595][75949] Updated weights for policy 0, policy_version 33001 (0.0008) -[2023-10-14 14:47:11,969][75949] Updated weights for policy 0, policy_version 33011 (0.0007) -[2023-10-14 14:47:12,304][75950] Updated weights for policy 1, policy_version 32930 (0.0008) -[2023-10-14 14:47:12,332][75949] Updated weights for policy 0, policy_version 33021 (0.0007) -[2023-10-14 14:47:12,676][75950] Updated weights for policy 1, policy_version 32940 (0.0007) -[2023-10-14 14:47:13,047][75950] Updated weights for policy 1, policy_version 32950 (0.0007) -[2023-10-14 14:47:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 67534848. Throughput: 0: 1661.9, 1: 1671.4. Samples: 16895876. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 14:47:13,164][74987] Avg episode reward: [(0, '23.530'), (1, '26.700')] -[2023-10-14 14:47:13,411][75950] Updated weights for policy 1, policy_version 32960 (0.0008) -[2023-10-14 14:47:16,350][75949] Updated weights for policy 0, policy_version 33031 (0.0009) -[2023-10-14 14:47:16,720][75949] Updated weights for policy 0, policy_version 33041 (0.0009) -[2023-10-14 14:47:17,090][75949] Updated weights for policy 0, policy_version 33051 (0.0008) -[2023-10-14 14:47:17,328][75950] Updated weights for policy 1, policy_version 32970 (0.0007) -[2023-10-14 14:47:17,700][75950] Updated weights for policy 1, policy_version 32980 (0.0010) -[2023-10-14 14:47:18,068][75950] Updated weights for policy 1, policy_version 32990 (0.0007) -[2023-10-14 14:47:18,163][74987] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 67633152. Throughput: 0: 1685.1, 1: 1682.3. Samples: 16906770. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 14:47:18,164][74987] Avg episode reward: [(0, '21.800'), (1, '28.010')] -[2023-10-14 14:47:21,154][75949] Updated weights for policy 0, policy_version 33061 (0.0010) -[2023-10-14 14:47:21,533][75949] Updated weights for policy 0, policy_version 33071 (0.0010) -[2023-10-14 14:47:21,899][75949] Updated weights for policy 0, policy_version 33081 (0.0008) -[2023-10-14 14:47:22,197][75950] Updated weights for policy 1, policy_version 33000 (0.0007) -[2023-10-14 14:47:22,567][75950] Updated weights for policy 1, policy_version 33010 (0.0008) -[2023-10-14 14:47:22,936][75950] Updated weights for policy 1, policy_version 33020 (0.0009) -[2023-10-14 14:47:23,164][74987] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 67698688. Throughput: 0: 1674.9, 1: 1684.8. Samples: 16926682. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 14:47:23,165][74987] Avg episode reward: [(0, '24.910'), (1, '26.710')] -[2023-10-14 14:47:26,101][75949] Updated weights for policy 0, policy_version 33091 (0.0009) -[2023-10-14 14:47:26,467][75949] Updated weights for policy 0, policy_version 33101 (0.0007) -[2023-10-14 14:47:26,839][75949] Updated weights for policy 0, policy_version 33111 (0.0008) -[2023-10-14 14:47:27,041][75950] Updated weights for policy 1, policy_version 33030 (0.0008) -[2023-10-14 14:47:27,405][75950] Updated weights for policy 1, policy_version 33040 (0.0007) -[2023-10-14 14:47:27,774][75950] Updated weights for policy 1, policy_version 33050 (0.0008) -[2023-10-14 14:47:28,164][74987] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 67764224. Throughput: 0: 1671.4, 1: 1662.3. Samples: 16945812. Policy #0 lag: (min: 1.0, avg: 5.4, max: 33.0) -[2023-10-14 14:47:28,165][74987] Avg episode reward: [(0, '22.080'), (1, '24.300')] -[2023-10-14 14:47:30,730][75949] Updated weights for policy 0, policy_version 33121 (0.0009) -[2023-10-14 14:47:31,102][75949] Updated weights for policy 0, policy_version 33131 (0.0009) -[2023-10-14 14:47:31,471][75949] Updated weights for policy 0, policy_version 33141 (0.0008) -[2023-10-14 14:47:31,769][75950] Updated weights for policy 1, policy_version 33060 (0.0009) -[2023-10-14 14:47:31,838][75949] Updated weights for policy 0, policy_version 33151 (0.0008) -[2023-10-14 14:47:32,146][75950] Updated weights for policy 1, policy_version 33070 (0.0009) -[2023-10-14 14:47:32,525][75950] Updated weights for policy 1, policy_version 33080 (0.0009) -[2023-10-14 14:47:33,163][74987] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 67829760. Throughput: 0: 1685.4, 1: 1681.8. Samples: 16957180. Policy #0 lag: (min: 1.0, avg: 5.4, max: 33.0) -[2023-10-14 14:47:33,164][74987] Avg episode reward: [(0, '24.160'), (1, '29.490')] -[2023-10-14 14:47:33,165][75801] Saving new best policy, reward=29.490! -[2023-10-14 14:47:35,872][75949] Updated weights for policy 0, policy_version 33161 (0.0008) -[2023-10-14 14:47:36,238][75949] Updated weights for policy 0, policy_version 33171 (0.0009) -[2023-10-14 14:47:36,594][75950] Updated weights for policy 1, policy_version 33090 (0.0009) -[2023-10-14 14:47:36,616][75949] Updated weights for policy 0, policy_version 33181 (0.0010) -[2023-10-14 14:47:36,968][75950] Updated weights for policy 1, policy_version 33100 (0.0010) -[2023-10-14 14:47:37,344][75950] Updated weights for policy 1, policy_version 33110 (0.0008) -[2023-10-14 14:47:37,705][75950] Updated weights for policy 1, policy_version 33120 (0.0011) -[2023-10-14 14:47:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 67895296. Throughput: 0: 1668.9, 1: 1680.2. Samples: 16976828. Policy #0 lag: (min: 1.0, avg: 5.4, max: 33.0) -[2023-10-14 14:47:38,165][74987] Avg episode reward: [(0, '24.120'), (1, '27.750')] -[2023-10-14 14:47:40,641][75949] Updated weights for policy 0, policy_version 33191 (0.0007) -[2023-10-14 14:47:41,025][75949] Updated weights for policy 0, policy_version 33201 (0.0009) -[2023-10-14 14:47:41,392][75949] Updated weights for policy 0, policy_version 33211 (0.0009) -[2023-10-14 14:47:41,773][75950] Updated weights for policy 1, policy_version 33130 (0.0009) -[2023-10-14 14:47:42,140][75950] Updated weights for policy 1, policy_version 33140 (0.0009) -[2023-10-14 14:47:42,513][75950] Updated weights for policy 1, policy_version 33150 (0.0008) -[2023-10-14 14:47:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 67960832. Throughput: 0: 1687.5, 1: 1660.6. Samples: 16996368. Policy #0 lag: (min: 1.0, avg: 5.4, max: 33.0) -[2023-10-14 14:47:43,165][74987] Avg episode reward: [(0, '24.220'), (1, '27.400')] -[2023-10-14 14:47:45,466][75949] Updated weights for policy 0, policy_version 33221 (0.0008) -[2023-10-14 14:47:45,854][75949] Updated weights for policy 0, policy_version 33231 (0.0007) -[2023-10-14 14:47:46,222][75949] Updated weights for policy 0, policy_version 33241 (0.0008) -[2023-10-14 14:47:46,608][75950] Updated weights for policy 1, policy_version 33160 (0.0008) -[2023-10-14 14:47:46,974][75950] Updated weights for policy 1, policy_version 33170 (0.0010) -[2023-10-14 14:47:47,339][75950] Updated weights for policy 1, policy_version 33180 (0.0007) -[2023-10-14 14:47:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 68026368. Throughput: 0: 1683.9, 1: 1681.3. Samples: 17007618. Policy #0 lag: (min: 1.0, avg: 5.4, max: 33.0) -[2023-10-14 14:47:48,165][74987] Avg episode reward: [(0, '24.510'), (1, '27.880')] -[2023-10-14 14:47:50,103][75949] Updated weights for policy 0, policy_version 33251 (0.0007) -[2023-10-14 14:47:50,475][75949] Updated weights for policy 0, policy_version 33261 (0.0007) -[2023-10-14 14:47:50,850][75949] Updated weights for policy 0, policy_version 33271 (0.0008) -[2023-10-14 14:47:51,434][75950] Updated weights for policy 1, policy_version 33190 (0.0008) -[2023-10-14 14:47:51,814][75950] Updated weights for policy 1, policy_version 33200 (0.0008) -[2023-10-14 14:47:52,186][75950] Updated weights for policy 1, policy_version 33210 (0.0008) -[2023-10-14 14:47:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 68091904. Throughput: 0: 1673.7, 1: 1674.6. Samples: 17027170. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 14:47:53,165][74987] Avg episode reward: [(0, '23.300'), (1, '27.820')] -[2023-10-14 14:47:54,881][75949] Updated weights for policy 0, policy_version 33281 (0.0009) -[2023-10-14 14:47:55,254][75949] Updated weights for policy 0, policy_version 33291 (0.0011) -[2023-10-14 14:47:55,631][75949] Updated weights for policy 0, policy_version 33301 (0.0009) -[2023-10-14 14:47:55,996][75949] Updated weights for policy 0, policy_version 33311 (0.0008) -[2023-10-14 14:47:56,071][75950] Updated weights for policy 1, policy_version 33220 (0.0008) -[2023-10-14 14:47:56,436][75950] Updated weights for policy 1, policy_version 33230 (0.0011) -[2023-10-14 14:47:56,804][75950] Updated weights for policy 1, policy_version 33240 (0.0009) -[2023-10-14 14:47:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 68157440. Throughput: 0: 1691.7, 1: 1672.8. Samples: 17047280. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 14:47:58,165][74987] Avg episode reward: [(0, '22.940'), (1, '28.600')] -[2023-10-14 14:48:00,056][75949] Updated weights for policy 0, policy_version 33321 (0.0008) -[2023-10-14 14:48:00,434][75949] Updated weights for policy 0, policy_version 33331 (0.0007) -[2023-10-14 14:48:00,745][75950] Updated weights for policy 1, policy_version 33250 (0.0010) -[2023-10-14 14:48:00,813][75949] Updated weights for policy 0, policy_version 33341 (0.0008) -[2023-10-14 14:48:01,116][75950] Updated weights for policy 1, policy_version 33260 (0.0009) -[2023-10-14 14:48:01,484][75950] Updated weights for policy 1, policy_version 33270 (0.0009) -[2023-10-14 14:48:01,844][75950] Updated weights for policy 1, policy_version 33280 (0.0007) -[2023-10-14 14:48:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 68222976. Throughput: 0: 1668.8, 1: 1693.9. Samples: 17058096. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 14:48:03,165][74987] Avg episode reward: [(0, '22.420'), (1, '30.860')] -[2023-10-14 14:48:03,166][75801] Saving new best policy, reward=30.860! -[2023-10-14 14:48:04,939][75949] Updated weights for policy 0, policy_version 33351 (0.0008) -[2023-10-14 14:48:05,308][75949] Updated weights for policy 0, policy_version 33361 (0.0010) -[2023-10-14 14:48:05,680][75949] Updated weights for policy 0, policy_version 33371 (0.0009) -[2023-10-14 14:48:06,060][75950] Updated weights for policy 1, policy_version 33290 (0.0008) -[2023-10-14 14:48:06,423][75950] Updated weights for policy 1, policy_version 33300 (0.0010) -[2023-10-14 14:48:06,791][75950] Updated weights for policy 1, policy_version 33310 (0.0008) -[2023-10-14 14:48:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 68288512. Throughput: 0: 1676.1, 1: 1672.6. Samples: 17077374. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 14:48:08,165][74987] Avg episode reward: [(0, '25.580'), (1, '28.440')] -[2023-10-14 14:48:09,809][75949] Updated weights for policy 0, policy_version 33381 (0.0007) -[2023-10-14 14:48:10,181][75949] Updated weights for policy 0, policy_version 33391 (0.0009) -[2023-10-14 14:48:10,556][75949] Updated weights for policy 0, policy_version 33401 (0.0010) -[2023-10-14 14:48:10,929][75950] Updated weights for policy 1, policy_version 33320 (0.0010) -[2023-10-14 14:48:11,302][75950] Updated weights for policy 1, policy_version 33330 (0.0009) -[2023-10-14 14:48:11,671][75950] Updated weights for policy 1, policy_version 33340 (0.0007) -[2023-10-14 14:48:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 68354048. Throughput: 0: 1684.8, 1: 1683.2. Samples: 17097368. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 14:48:13,164][74987] Avg episode reward: [(0, '25.140'), (1, '27.570')] -[2023-10-14 14:48:14,730][75949] Updated weights for policy 0, policy_version 33411 (0.0008) -[2023-10-14 14:48:15,104][75949] Updated weights for policy 0, policy_version 33421 (0.0011) -[2023-10-14 14:48:15,475][75949] Updated weights for policy 0, policy_version 33431 (0.0011) -[2023-10-14 14:48:15,902][75950] Updated weights for policy 1, policy_version 33350 (0.0007) -[2023-10-14 14:48:16,279][75950] Updated weights for policy 1, policy_version 33360 (0.0008) -[2023-10-14 14:48:16,642][75950] Updated weights for policy 1, policy_version 33370 (0.0009) -[2023-10-14 14:48:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 68419584. Throughput: 0: 1657.5, 1: 1689.1. Samples: 17107778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:48:18,165][74987] Avg episode reward: [(0, '26.200'), (1, '27.460')] -[2023-10-14 14:48:18,166][75615] Saving new best policy, reward=26.200! -[2023-10-14 14:48:19,639][75949] Updated weights for policy 0, policy_version 33441 (0.0011) -[2023-10-14 14:48:20,001][75949] Updated weights for policy 0, policy_version 33451 (0.0009) -[2023-10-14 14:48:20,376][75949] Updated weights for policy 0, policy_version 33461 (0.0009) -[2023-10-14 14:48:20,601][75950] Updated weights for policy 1, policy_version 33380 (0.0011) -[2023-10-14 14:48:20,746][75949] Updated weights for policy 0, policy_version 33471 (0.0008) -[2023-10-14 14:48:20,972][75950] Updated weights for policy 1, policy_version 33390 (0.0009) -[2023-10-14 14:48:21,327][75950] Updated weights for policy 1, policy_version 33400 (0.0009) -[2023-10-14 14:48:23,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 68485120. Throughput: 0: 1677.8, 1: 1664.6. Samples: 17127236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:48:23,164][74987] Avg episode reward: [(0, '23.730'), (1, '27.760')] -[2023-10-14 14:48:24,710][75949] Updated weights for policy 0, policy_version 33481 (0.0008) -[2023-10-14 14:48:25,077][75949] Updated weights for policy 0, policy_version 33491 (0.0010) -[2023-10-14 14:48:25,292][75950] Updated weights for policy 1, policy_version 33410 (0.0008) -[2023-10-14 14:48:25,459][75949] Updated weights for policy 0, policy_version 33501 (0.0008) -[2023-10-14 14:48:25,651][75950] Updated weights for policy 1, policy_version 33420 (0.0009) -[2023-10-14 14:48:26,018][75950] Updated weights for policy 1, policy_version 33430 (0.0009) -[2023-10-14 14:48:26,385][75950] Updated weights for policy 1, policy_version 33440 (0.0009) -[2023-10-14 14:48:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 68550656. Throughput: 0: 1682.6, 1: 1684.4. Samples: 17147884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:48:28,164][74987] Avg episode reward: [(0, '25.070'), (1, '27.160')] -[2023-10-14 14:48:29,515][75949] Updated weights for policy 0, policy_version 33511 (0.0010) -[2023-10-14 14:48:29,889][75949] Updated weights for policy 0, policy_version 33521 (0.0010) -[2023-10-14 14:48:30,259][75949] Updated weights for policy 0, policy_version 33531 (0.0010) -[2023-10-14 14:48:30,468][75950] Updated weights for policy 1, policy_version 33450 (0.0007) -[2023-10-14 14:48:30,837][75950] Updated weights for policy 1, policy_version 33460 (0.0007) -[2023-10-14 14:48:31,216][75950] Updated weights for policy 1, policy_version 33470 (0.0009) -[2023-10-14 14:48:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 68616192. Throughput: 0: 1665.5, 1: 1665.9. Samples: 17157528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:48:33,165][74987] Avg episode reward: [(0, '22.990'), (1, '29.150')] -[2023-10-14 14:48:34,374][75949] Updated weights for policy 0, policy_version 33541 (0.0008) -[2023-10-14 14:48:34,749][75949] Updated weights for policy 0, policy_version 33551 (0.0008) -[2023-10-14 14:48:35,118][75949] Updated weights for policy 0, policy_version 33561 (0.0009) -[2023-10-14 14:48:35,297][75950] Updated weights for policy 1, policy_version 33480 (0.0007) -[2023-10-14 14:48:35,658][75950] Updated weights for policy 1, policy_version 33490 (0.0007) -[2023-10-14 14:48:36,012][75950] Updated weights for policy 1, policy_version 33500 (0.0012) -[2023-10-14 14:48:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68681728. Throughput: 0: 1680.6, 1: 1661.2. Samples: 17177548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:48:38,165][74987] Avg episode reward: [(0, '25.730'), (1, '26.830')] -[2023-10-14 14:48:39,353][75949] Updated weights for policy 0, policy_version 33571 (0.0007) -[2023-10-14 14:48:39,726][75949] Updated weights for policy 0, policy_version 33581 (0.0008) -[2023-10-14 14:48:40,098][75949] Updated weights for policy 0, policy_version 33591 (0.0009) -[2023-10-14 14:48:40,325][75950] Updated weights for policy 1, policy_version 33510 (0.0008) -[2023-10-14 14:48:40,704][75950] Updated weights for policy 1, policy_version 33520 (0.0007) -[2023-10-14 14:48:41,071][75950] Updated weights for policy 1, policy_version 33530 (0.0007) -[2023-10-14 14:48:43,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68747264. Throughput: 0: 1686.9, 1: 1668.8. Samples: 17198288. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 14:48:43,164][74987] Avg episode reward: [(0, '23.310'), (1, '27.140')] -[2023-10-14 14:48:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000033536_34340864.pth... -[2023-10-14 14:48:43,174][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000033600_34406400.pth... -[2023-10-14 14:48:43,210][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000032032_32800768.pth -[2023-10-14 14:48:43,213][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000031968_32735232.pth -[2023-10-14 14:48:44,073][75949] Updated weights for policy 0, policy_version 33601 (0.0009) -[2023-10-14 14:48:44,451][75949] Updated weights for policy 0, policy_version 33611 (0.0009) -[2023-10-14 14:48:44,820][75949] Updated weights for policy 0, policy_version 33621 (0.0009) -[2023-10-14 14:48:45,122][75950] Updated weights for policy 1, policy_version 33540 (0.0009) -[2023-10-14 14:48:45,195][75949] Updated weights for policy 0, policy_version 33631 (0.0010) -[2023-10-14 14:48:45,481][75950] Updated weights for policy 1, policy_version 33550 (0.0007) -[2023-10-14 14:48:45,845][75950] Updated weights for policy 1, policy_version 33560 (0.0007) -[2023-10-14 14:48:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 68812800. Throughput: 0: 1682.1, 1: 1647.7. Samples: 17207932. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 14:48:48,164][74987] Avg episode reward: [(0, '26.940'), (1, '28.720')] -[2023-10-14 14:48:48,165][75615] Saving new best policy, reward=26.940! -[2023-10-14 14:48:49,222][75949] Updated weights for policy 0, policy_version 33641 (0.0007) -[2023-10-14 14:48:49,597][75949] Updated weights for policy 0, policy_version 33651 (0.0008) -[2023-10-14 14:48:49,969][75949] Updated weights for policy 0, policy_version 33661 (0.0007) -[2023-10-14 14:48:50,053][75950] Updated weights for policy 1, policy_version 33570 (0.0009) -[2023-10-14 14:48:50,407][75950] Updated weights for policy 1, policy_version 33580 (0.0009) -[2023-10-14 14:48:50,771][75950] Updated weights for policy 1, policy_version 33590 (0.0009) -[2023-10-14 14:48:51,142][75950] Updated weights for policy 1, policy_version 33600 (0.0009) -[2023-10-14 14:48:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68878336. Throughput: 0: 1692.1, 1: 1655.2. Samples: 17228000. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 14:48:53,165][74987] Avg episode reward: [(0, '22.240'), (1, '28.520')] -[2023-10-14 14:48:53,950][75949] Updated weights for policy 0, policy_version 33671 (0.0007) -[2023-10-14 14:48:54,310][75949] Updated weights for policy 0, policy_version 33681 (0.0011) -[2023-10-14 14:48:54,680][75949] Updated weights for policy 0, policy_version 33691 (0.0010) -[2023-10-14 14:48:55,206][75950] Updated weights for policy 1, policy_version 33610 (0.0011) -[2023-10-14 14:48:55,573][75950] Updated weights for policy 1, policy_version 33620 (0.0011) -[2023-10-14 14:48:55,950][75950] Updated weights for policy 1, policy_version 33630 (0.0010) -[2023-10-14 14:48:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 68943872. Throughput: 0: 1693.0, 1: 1669.3. Samples: 17248672. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 14:48:58,165][74987] Avg episode reward: [(0, '26.850'), (1, '26.440')] -[2023-10-14 14:48:58,732][75949] Updated weights for policy 0, policy_version 33701 (0.0011) -[2023-10-14 14:48:59,111][75949] Updated weights for policy 0, policy_version 33711 (0.0008) -[2023-10-14 14:48:59,488][75949] Updated weights for policy 0, policy_version 33721 (0.0009) -[2023-10-14 14:49:00,201][75950] Updated weights for policy 1, policy_version 33640 (0.0008) -[2023-10-14 14:49:00,572][75950] Updated weights for policy 1, policy_version 33650 (0.0008) -[2023-10-14 14:49:00,946][75950] Updated weights for policy 1, policy_version 33660 (0.0008) -[2023-10-14 14:49:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 69009408. Throughput: 0: 1689.4, 1: 1656.1. Samples: 17258326. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 14:49:03,164][74987] Avg episode reward: [(0, '21.090'), (1, '29.640')] -[2023-10-14 14:49:03,507][75949] Updated weights for policy 0, policy_version 33731 (0.0010) -[2023-10-14 14:49:03,881][75949] Updated weights for policy 0, policy_version 33741 (0.0008) -[2023-10-14 14:49:04,254][75949] Updated weights for policy 0, policy_version 33751 (0.0007) -[2023-10-14 14:49:04,864][75950] Updated weights for policy 1, policy_version 33670 (0.0008) -[2023-10-14 14:49:05,227][75950] Updated weights for policy 1, policy_version 33680 (0.0008) -[2023-10-14 14:49:05,591][75950] Updated weights for policy 1, policy_version 33690 (0.0010) -[2023-10-14 14:49:08,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 69074944. Throughput: 0: 1688.2, 1: 1670.8. Samples: 17278394. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 14:49:08,164][74987] Avg episode reward: [(0, '25.230'), (1, '28.810')] -[2023-10-14 14:49:08,399][75949] Updated weights for policy 0, policy_version 33761 (0.0008) -[2023-10-14 14:49:08,766][75949] Updated weights for policy 0, policy_version 33771 (0.0007) -[2023-10-14 14:49:09,139][75949] Updated weights for policy 0, policy_version 33781 (0.0011) -[2023-10-14 14:49:09,419][75950] Updated weights for policy 1, policy_version 33700 (0.0010) -[2023-10-14 14:49:09,506][75949] Updated weights for policy 0, policy_version 33791 (0.0009) -[2023-10-14 14:49:09,790][75950] Updated weights for policy 1, policy_version 33710 (0.0011) -[2023-10-14 14:49:10,148][75950] Updated weights for policy 1, policy_version 33720 (0.0007) -[2023-10-14 14:49:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 69140480. Throughput: 0: 1683.9, 1: 1678.4. Samples: 17299188. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 14:49:13,165][74987] Avg episode reward: [(0, '21.560'), (1, '27.080')] -[2023-10-14 14:49:13,760][75949] Updated weights for policy 0, policy_version 33801 (0.0011) -[2023-10-14 14:49:14,130][75949] Updated weights for policy 0, policy_version 33811 (0.0008) -[2023-10-14 14:49:14,177][75950] Updated weights for policy 1, policy_version 33730 (0.0008) -[2023-10-14 14:49:14,490][75949] Updated weights for policy 0, policy_version 33821 (0.0007) -[2023-10-14 14:49:14,537][75950] Updated weights for policy 1, policy_version 33740 (0.0008) -[2023-10-14 14:49:14,898][75950] Updated weights for policy 1, policy_version 33750 (0.0008) -[2023-10-14 14:49:15,270][75950] Updated weights for policy 1, policy_version 33760 (0.0008) -[2023-10-14 14:49:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 69206016. Throughput: 0: 1684.0, 1: 1664.4. Samples: 17308208. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 14:49:18,164][74987] Avg episode reward: [(0, '25.170'), (1, '28.690')] -[2023-10-14 14:49:18,538][75949] Updated weights for policy 0, policy_version 33831 (0.0008) -[2023-10-14 14:49:18,909][75949] Updated weights for policy 0, policy_version 33841 (0.0007) -[2023-10-14 14:49:19,282][75949] Updated weights for policy 0, policy_version 33851 (0.0007) -[2023-10-14 14:49:19,521][75950] Updated weights for policy 1, policy_version 33770 (0.0007) -[2023-10-14 14:49:19,885][75950] Updated weights for policy 1, policy_version 33780 (0.0010) -[2023-10-14 14:49:20,258][75950] Updated weights for policy 1, policy_version 33790 (0.0009) -[2023-10-14 14:49:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 69271552. Throughput: 0: 1682.4, 1: 1678.0. Samples: 17328762. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 14:49:23,165][74987] Avg episode reward: [(0, '24.430'), (1, '28.860')] -[2023-10-14 14:49:23,332][75949] Updated weights for policy 0, policy_version 33861 (0.0009) -[2023-10-14 14:49:23,719][75949] Updated weights for policy 0, policy_version 33871 (0.0008) -[2023-10-14 14:49:24,096][75949] Updated weights for policy 0, policy_version 33881 (0.0009) -[2023-10-14 14:49:24,431][75950] Updated weights for policy 1, policy_version 33800 (0.0007) -[2023-10-14 14:49:24,810][75950] Updated weights for policy 1, policy_version 33810 (0.0009) -[2023-10-14 14:49:25,169][75950] Updated weights for policy 1, policy_version 33820 (0.0011) -[2023-10-14 14:49:28,102][75949] Updated weights for policy 0, policy_version 33891 (0.0008) -[2023-10-14 14:49:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 69337088. Throughput: 0: 1674.1, 1: 1679.3. Samples: 17349192. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 14:49:28,164][74987] Avg episode reward: [(0, '25.130'), (1, '27.950')] -[2023-10-14 14:49:28,482][75949] Updated weights for policy 0, policy_version 33901 (0.0009) -[2023-10-14 14:49:28,858][75949] Updated weights for policy 0, policy_version 33911 (0.0008) -[2023-10-14 14:49:29,304][75950] Updated weights for policy 1, policy_version 33830 (0.0010) -[2023-10-14 14:49:29,679][75950] Updated weights for policy 1, policy_version 33840 (0.0008) -[2023-10-14 14:49:30,041][75950] Updated weights for policy 1, policy_version 33850 (0.0007) -[2023-10-14 14:49:32,806][75949] Updated weights for policy 0, policy_version 33921 (0.0008) -[2023-10-14 14:49:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 69402624. Throughput: 0: 1674.2, 1: 1668.3. Samples: 17358346. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:49:33,165][75949] Updated weights for policy 0, policy_version 33931 (0.0008) -[2023-10-14 14:49:33,165][74987] Avg episode reward: [(0, '26.110'), (1, '28.160')] -[2023-10-14 14:49:33,542][75949] Updated weights for policy 0, policy_version 33941 (0.0007) -[2023-10-14 14:49:33,906][75949] Updated weights for policy 0, policy_version 33951 (0.0009) -[2023-10-14 14:49:34,146][75950] Updated weights for policy 1, policy_version 33860 (0.0008) -[2023-10-14 14:49:34,514][75950] Updated weights for policy 1, policy_version 33870 (0.0007) -[2023-10-14 14:49:34,884][75950] Updated weights for policy 1, policy_version 33880 (0.0008) -[2023-10-14 14:49:38,023][75949] Updated weights for policy 0, policy_version 33961 (0.0009) -[2023-10-14 14:49:38,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 69468160. Throughput: 0: 1677.2, 1: 1681.2. Samples: 17379126. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:49:38,165][74987] Avg episode reward: [(0, '22.710'), (1, '28.240')] -[2023-10-14 14:49:38,392][75949] Updated weights for policy 0, policy_version 33971 (0.0010) -[2023-10-14 14:49:38,764][75949] Updated weights for policy 0, policy_version 33981 (0.0009) -[2023-10-14 14:49:39,114][75950] Updated weights for policy 1, policy_version 33890 (0.0008) -[2023-10-14 14:49:39,471][75950] Updated weights for policy 1, policy_version 33900 (0.0009) -[2023-10-14 14:49:39,841][75950] Updated weights for policy 1, policy_version 33910 (0.0007) -[2023-10-14 14:49:40,220][75950] Updated weights for policy 1, policy_version 33920 (0.0008) -[2023-10-14 14:49:42,816][75949] Updated weights for policy 0, policy_version 33991 (0.0009) -[2023-10-14 14:49:43,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 69533696. Throughput: 0: 1677.8, 1: 1677.2. Samples: 17399644. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:49:43,164][74987] Avg episode reward: [(0, '28.060'), (1, '26.650')] -[2023-10-14 14:49:43,194][75949] Updated weights for policy 0, policy_version 34001 (0.0008) -[2023-10-14 14:49:43,562][75949] Updated weights for policy 0, policy_version 34011 (0.0009) -[2023-10-14 14:49:43,748][75615] Saving new best policy, reward=28.060! -[2023-10-14 14:49:44,300][75950] Updated weights for policy 1, policy_version 33930 (0.0008) -[2023-10-14 14:49:44,667][75950] Updated weights for policy 1, policy_version 33940 (0.0007) -[2023-10-14 14:49:45,032][75950] Updated weights for policy 1, policy_version 33950 (0.0008) -[2023-10-14 14:49:47,665][75949] Updated weights for policy 0, policy_version 34021 (0.0007) -[2023-10-14 14:49:48,032][75949] Updated weights for policy 0, policy_version 34031 (0.0010) -[2023-10-14 14:49:48,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 69599232. Throughput: 0: 1681.2, 1: 1663.7. Samples: 17408848. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:49:48,164][74987] Avg episode reward: [(0, '24.180'), (1, '27.920')] -[2023-10-14 14:49:48,400][75949] Updated weights for policy 0, policy_version 34041 (0.0010) -[2023-10-14 14:49:49,159][75950] Updated weights for policy 1, policy_version 33960 (0.0008) -[2023-10-14 14:49:49,529][75950] Updated weights for policy 1, policy_version 33970 (0.0008) -[2023-10-14 14:49:49,892][75950] Updated weights for policy 1, policy_version 33980 (0.0010) -[2023-10-14 14:49:52,422][75949] Updated weights for policy 0, policy_version 34051 (0.0008) -[2023-10-14 14:49:52,796][75949] Updated weights for policy 0, policy_version 34061 (0.0007) -[2023-10-14 14:49:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 69664768. Throughput: 0: 1683.1, 1: 1670.8. Samples: 17429322. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 14:49:53,165][74987] Avg episode reward: [(0, '27.780'), (1, '27.840')] -[2023-10-14 14:49:53,173][75949] Updated weights for policy 0, policy_version 34071 (0.0007) -[2023-10-14 14:49:53,997][75950] Updated weights for policy 1, policy_version 33990 (0.0008) -[2023-10-14 14:49:54,366][75950] Updated weights for policy 1, policy_version 34000 (0.0008) -[2023-10-14 14:49:54,734][75950] Updated weights for policy 1, policy_version 34010 (0.0009) -[2023-10-14 14:49:57,179][75949] Updated weights for policy 0, policy_version 34081 (0.0008) -[2023-10-14 14:49:57,547][75949] Updated weights for policy 0, policy_version 34091 (0.0009) -[2023-10-14 14:49:57,925][75949] Updated weights for policy 0, policy_version 34101 (0.0007) -[2023-10-14 14:49:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 69730304. Throughput: 0: 1674.4, 1: 1669.5. Samples: 17449660. Policy #0 lag: (min: 4.0, avg: 4.1, max: 9.0) -[2023-10-14 14:49:58,164][74987] Avg episode reward: [(0, '23.580'), (1, '28.180')] -[2023-10-14 14:49:58,294][75949] Updated weights for policy 0, policy_version 34111 (0.0008) -[2023-10-14 14:49:58,824][75950] Updated weights for policy 1, policy_version 34020 (0.0008) -[2023-10-14 14:49:59,193][75950] Updated weights for policy 1, policy_version 34030 (0.0007) -[2023-10-14 14:49:59,567][75950] Updated weights for policy 1, policy_version 34040 (0.0007) -[2023-10-14 14:50:02,202][75949] Updated weights for policy 0, policy_version 34121 (0.0009) -[2023-10-14 14:50:02,572][75949] Updated weights for policy 0, policy_version 34131 (0.0009) -[2023-10-14 14:50:02,947][75949] Updated weights for policy 0, policy_version 34141 (0.0009) -[2023-10-14 14:50:03,163][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 69828608. Throughput: 0: 1690.6, 1: 1670.7. Samples: 17459468. Policy #0 lag: (min: 4.0, avg: 4.1, max: 9.0) -[2023-10-14 14:50:03,164][74987] Avg episode reward: [(0, '26.120'), (1, '27.610')] -[2023-10-14 14:50:03,550][75950] Updated weights for policy 1, policy_version 34050 (0.0010) -[2023-10-14 14:50:03,916][75950] Updated weights for policy 1, policy_version 34060 (0.0008) -[2023-10-14 14:50:04,272][75950] Updated weights for policy 1, policy_version 34070 (0.0008) -[2023-10-14 14:50:04,645][75950] Updated weights for policy 1, policy_version 34080 (0.0010) -[2023-10-14 14:50:06,996][75949] Updated weights for policy 0, policy_version 34151 (0.0008) -[2023-10-14 14:50:07,362][75949] Updated weights for policy 0, policy_version 34161 (0.0009) -[2023-10-14 14:50:07,733][75949] Updated weights for policy 0, policy_version 34171 (0.0007) -[2023-10-14 14:50:08,163][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 69894144. Throughput: 0: 1690.8, 1: 1673.1. Samples: 17480136. Policy #0 lag: (min: 4.0, avg: 4.1, max: 9.0) -[2023-10-14 14:50:08,164][74987] Avg episode reward: [(0, '22.780'), (1, '27.380')] -[2023-10-14 14:50:08,588][75950] Updated weights for policy 1, policy_version 34090 (0.0008) -[2023-10-14 14:50:08,954][75950] Updated weights for policy 1, policy_version 34100 (0.0008) -[2023-10-14 14:50:09,324][75950] Updated weights for policy 1, policy_version 34110 (0.0010) -[2023-10-14 14:50:11,966][75949] Updated weights for policy 0, policy_version 34181 (0.0008) -[2023-10-14 14:50:12,342][75949] Updated weights for policy 0, policy_version 34191 (0.0007) -[2023-10-14 14:50:12,709][75949] Updated weights for policy 0, policy_version 34201 (0.0011) -[2023-10-14 14:50:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 69959680. Throughput: 0: 1669.5, 1: 1672.7. Samples: 17499588. Policy #0 lag: (min: 4.0, avg: 4.1, max: 9.0) -[2023-10-14 14:50:13,164][74987] Avg episode reward: [(0, '25.210'), (1, '27.600')] -[2023-10-14 14:50:13,744][75950] Updated weights for policy 1, policy_version 34120 (0.0008) -[2023-10-14 14:50:14,127][75950] Updated weights for policy 1, policy_version 34130 (0.0007) -[2023-10-14 14:50:14,491][75950] Updated weights for policy 1, policy_version 34140 (0.0008) -[2023-10-14 14:50:16,807][75949] Updated weights for policy 0, policy_version 34211 (0.0009) -[2023-10-14 14:50:17,181][75949] Updated weights for policy 0, policy_version 34221 (0.0007) -[2023-10-14 14:50:17,551][75949] Updated weights for policy 0, policy_version 34231 (0.0007) -[2023-10-14 14:50:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 70025216. Throughput: 0: 1691.2, 1: 1666.0. Samples: 17509420. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-14 14:50:18,164][74987] Avg episode reward: [(0, '24.120'), (1, '29.020')] -[2023-10-14 14:50:18,491][75950] Updated weights for policy 1, policy_version 34150 (0.0008) -[2023-10-14 14:50:18,861][75950] Updated weights for policy 1, policy_version 34160 (0.0009) -[2023-10-14 14:50:19,225][75950] Updated weights for policy 1, policy_version 34170 (0.0010) -[2023-10-14 14:50:21,650][75949] Updated weights for policy 0, policy_version 34241 (0.0009) -[2023-10-14 14:50:22,017][75949] Updated weights for policy 0, policy_version 34251 (0.0010) -[2023-10-14 14:50:22,386][75949] Updated weights for policy 0, policy_version 34261 (0.0009) -[2023-10-14 14:50:22,758][75949] Updated weights for policy 0, policy_version 34271 (0.0010) -[2023-10-14 14:50:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 70090752. Throughput: 0: 1683.8, 1: 1668.5. Samples: 17529982. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-14 14:50:23,165][74987] Avg episode reward: [(0, '23.470'), (1, '28.150')] -[2023-10-14 14:50:23,312][75950] Updated weights for policy 1, policy_version 34180 (0.0009) -[2023-10-14 14:50:23,672][75950] Updated weights for policy 1, policy_version 34190 (0.0007) -[2023-10-14 14:50:24,049][75950] Updated weights for policy 1, policy_version 34200 (0.0010) -[2023-10-14 14:50:26,913][75949] Updated weights for policy 0, policy_version 34281 (0.0009) -[2023-10-14 14:50:27,279][75949] Updated weights for policy 0, policy_version 34291 (0.0007) -[2023-10-14 14:50:27,650][75949] Updated weights for policy 0, policy_version 34301 (0.0008) -[2023-10-14 14:50:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 70156288. Throughput: 0: 1659.5, 1: 1671.5. Samples: 17549540. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-14 14:50:28,164][74987] Avg episode reward: [(0, '22.680'), (1, '26.170')] -[2023-10-14 14:50:28,229][75950] Updated weights for policy 1, policy_version 34210 (0.0009) -[2023-10-14 14:50:28,592][75950] Updated weights for policy 1, policy_version 34220 (0.0007) -[2023-10-14 14:50:28,968][75950] Updated weights for policy 1, policy_version 34230 (0.0010) -[2023-10-14 14:50:29,331][75950] Updated weights for policy 1, policy_version 34240 (0.0008) -[2023-10-14 14:50:31,700][75949] Updated weights for policy 0, policy_version 34311 (0.0007) -[2023-10-14 14:50:32,074][75949] Updated weights for policy 0, policy_version 34321 (0.0008) -[2023-10-14 14:50:32,438][75949] Updated weights for policy 0, policy_version 34331 (0.0007) -[2023-10-14 14:50:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 70221824. Throughput: 0: 1683.1, 1: 1673.2. Samples: 17559880. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-14 14:50:33,164][74987] Avg episode reward: [(0, '21.910'), (1, '29.200')] -[2023-10-14 14:50:33,194][75950] Updated weights for policy 1, policy_version 34250 (0.0008) -[2023-10-14 14:50:33,552][75950] Updated weights for policy 1, policy_version 34260 (0.0008) -[2023-10-14 14:50:33,925][75950] Updated weights for policy 1, policy_version 34270 (0.0009) -[2023-10-14 14:50:36,383][75949] Updated weights for policy 0, policy_version 34341 (0.0008) -[2023-10-14 14:50:36,755][75949] Updated weights for policy 0, policy_version 34351 (0.0010) -[2023-10-14 14:50:37,129][75949] Updated weights for policy 0, policy_version 34361 (0.0009) -[2023-10-14 14:50:37,919][75950] Updated weights for policy 1, policy_version 34280 (0.0008) -[2023-10-14 14:50:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 70287360. Throughput: 0: 1675.7, 1: 1682.1. Samples: 17580422. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) -[2023-10-14 14:50:38,164][74987] Avg episode reward: [(0, '24.840'), (1, '28.650')] -[2023-10-14 14:50:38,280][75950] Updated weights for policy 1, policy_version 34290 (0.0010) -[2023-10-14 14:50:38,643][75950] Updated weights for policy 1, policy_version 34300 (0.0010) -[2023-10-14 14:50:41,249][75949] Updated weights for policy 0, policy_version 34371 (0.0008) -[2023-10-14 14:50:41,625][75949] Updated weights for policy 0, policy_version 34381 (0.0010) -[2023-10-14 14:50:41,996][75949] Updated weights for policy 0, policy_version 34391 (0.0009) -[2023-10-14 14:50:42,824][75950] Updated weights for policy 1, policy_version 34310 (0.0009) -[2023-10-14 14:50:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 70352896. Throughput: 0: 1665.6, 1: 1678.0. Samples: 17600126. Policy #0 lag: (min: 1.0, avg: 13.5, max: 33.0) -[2023-10-14 14:50:43,164][74987] Avg episode reward: [(0, '22.320'), (1, '27.370')] -[2023-10-14 14:50:43,171][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000034400_35225600.pth... -[2023-10-14 14:50:43,192][75950] Updated weights for policy 1, policy_version 34320 (0.0007) -[2023-10-14 14:50:43,205][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000032832_33619968.pth -[2023-10-14 14:50:43,555][75950] Updated weights for policy 1, policy_version 34330 (0.0007) -[2023-10-14 14:50:43,774][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000034336_35160064.pth... -[2023-10-14 14:50:43,803][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000032768_33554432.pth -[2023-10-14 14:50:46,094][75949] Updated weights for policy 0, policy_version 34401 (0.0010) -[2023-10-14 14:50:46,462][75949] Updated weights for policy 0, policy_version 34411 (0.0010) -[2023-10-14 14:50:46,829][75949] Updated weights for policy 0, policy_version 34421 (0.0007) -[2023-10-14 14:50:47,201][75949] Updated weights for policy 0, policy_version 34431 (0.0007) -[2023-10-14 14:50:47,470][75950] Updated weights for policy 1, policy_version 34340 (0.0008) -[2023-10-14 14:50:47,828][75950] Updated weights for policy 1, policy_version 34350 (0.0009) -[2023-10-14 14:50:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 70418432. Throughput: 0: 1679.4, 1: 1681.0. Samples: 17610684. Policy #0 lag: (min: 1.0, avg: 13.5, max: 33.0) -[2023-10-14 14:50:48,164][74987] Avg episode reward: [(0, '25.700'), (1, '28.340')] -[2023-10-14 14:50:48,202][75950] Updated weights for policy 1, policy_version 34360 (0.0011) -[2023-10-14 14:50:51,345][75949] Updated weights for policy 0, policy_version 34441 (0.0008) -[2023-10-14 14:50:51,715][75949] Updated weights for policy 0, policy_version 34451 (0.0007) -[2023-10-14 14:50:52,076][75949] Updated weights for policy 0, policy_version 34461 (0.0008) -[2023-10-14 14:50:52,223][75950] Updated weights for policy 1, policy_version 34370 (0.0009) -[2023-10-14 14:50:52,584][75950] Updated weights for policy 1, policy_version 34380 (0.0007) -[2023-10-14 14:50:52,960][75950] Updated weights for policy 1, policy_version 34390 (0.0007) -[2023-10-14 14:50:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 70483968. Throughput: 0: 1669.6, 1: 1683.7. Samples: 17631036. Policy #0 lag: (min: 1.0, avg: 13.5, max: 33.0) -[2023-10-14 14:50:53,165][74987] Avg episode reward: [(0, '21.930'), (1, '28.380')] -[2023-10-14 14:50:53,333][75950] Updated weights for policy 1, policy_version 34400 (0.0009) -[2023-10-14 14:50:55,960][75949] Updated weights for policy 0, policy_version 34471 (0.0008) -[2023-10-14 14:50:56,324][75949] Updated weights for policy 0, policy_version 34481 (0.0008) -[2023-10-14 14:50:56,688][75949] Updated weights for policy 0, policy_version 34491 (0.0007) -[2023-10-14 14:50:57,326][75950] Updated weights for policy 1, policy_version 34410 (0.0008) -[2023-10-14 14:50:57,688][75950] Updated weights for policy 1, policy_version 34420 (0.0010) -[2023-10-14 14:50:58,047][75950] Updated weights for policy 1, policy_version 34430 (0.0010) -[2023-10-14 14:50:58,164][74987] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 70582272. Throughput: 0: 1682.0, 1: 1676.0. Samples: 17650702. Policy #0 lag: (min: 1.0, avg: 13.5, max: 33.0) -[2023-10-14 14:50:58,165][74987] Avg episode reward: [(0, '23.630'), (1, '27.930')] -[2023-10-14 14:51:00,838][75949] Updated weights for policy 0, policy_version 34501 (0.0009) -[2023-10-14 14:51:01,220][75949] Updated weights for policy 0, policy_version 34511 (0.0010) -[2023-10-14 14:51:01,596][75949] Updated weights for policy 0, policy_version 34521 (0.0011) -[2023-10-14 14:51:02,244][75950] Updated weights for policy 1, policy_version 34440 (0.0009) -[2023-10-14 14:51:02,609][75950] Updated weights for policy 1, policy_version 34450 (0.0007) -[2023-10-14 14:51:02,977][75950] Updated weights for policy 1, policy_version 34460 (0.0008) -[2023-10-14 14:51:03,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 70647808. Throughput: 0: 1682.6, 1: 1699.8. Samples: 17661628. Policy #0 lag: (min: 1.0, avg: 13.5, max: 33.0) -[2023-10-14 14:51:03,164][74987] Avg episode reward: [(0, '20.320'), (1, '27.390')] -[2023-10-14 14:51:05,600][75949] Updated weights for policy 0, policy_version 34531 (0.0009) -[2023-10-14 14:51:05,972][75949] Updated weights for policy 0, policy_version 34541 (0.0010) -[2023-10-14 14:51:06,344][75949] Updated weights for policy 0, policy_version 34551 (0.0011) -[2023-10-14 14:51:06,947][75950] Updated weights for policy 1, policy_version 34470 (0.0010) -[2023-10-14 14:51:07,319][75950] Updated weights for policy 1, policy_version 34480 (0.0009) -[2023-10-14 14:51:07,686][75950] Updated weights for policy 1, policy_version 34490 (0.0010) -[2023-10-14 14:51:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 70713344. Throughput: 0: 1663.2, 1: 1698.1. Samples: 17681242. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) -[2023-10-14 14:51:08,165][74987] Avg episode reward: [(0, '22.250'), (1, '27.640')] -[2023-10-14 14:51:10,458][75949] Updated weights for policy 0, policy_version 34561 (0.0009) -[2023-10-14 14:51:10,838][75949] Updated weights for policy 0, policy_version 34571 (0.0010) -[2023-10-14 14:51:11,203][75949] Updated weights for policy 0, policy_version 34581 (0.0011) -[2023-10-14 14:51:11,582][75949] Updated weights for policy 0, policy_version 34591 (0.0010) -[2023-10-14 14:51:11,921][75950] Updated weights for policy 1, policy_version 34500 (0.0011) -[2023-10-14 14:51:12,292][75950] Updated weights for policy 1, policy_version 34510 (0.0010) -[2023-10-14 14:51:12,660][75950] Updated weights for policy 1, policy_version 34520 (0.0009) -[2023-10-14 14:51:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 70778880. Throughput: 0: 1688.4, 1: 1670.4. Samples: 17700686. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) -[2023-10-14 14:51:13,164][74987] Avg episode reward: [(0, '21.800'), (1, '27.900')] -[2023-10-14 14:51:15,597][75949] Updated weights for policy 0, policy_version 34601 (0.0008) -[2023-10-14 14:51:15,975][75949] Updated weights for policy 0, policy_version 34611 (0.0009) -[2023-10-14 14:51:16,339][75949] Updated weights for policy 0, policy_version 34621 (0.0011) -[2023-10-14 14:51:16,755][75950] Updated weights for policy 1, policy_version 34530 (0.0008) -[2023-10-14 14:51:17,116][75950] Updated weights for policy 1, policy_version 34540 (0.0008) -[2023-10-14 14:51:17,487][75950] Updated weights for policy 1, policy_version 34550 (0.0008) -[2023-10-14 14:51:17,849][75950] Updated weights for policy 1, policy_version 34560 (0.0007) -[2023-10-14 14:51:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 70844416. Throughput: 0: 1679.6, 1: 1688.8. Samples: 17711458. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) -[2023-10-14 14:51:18,164][74987] Avg episode reward: [(0, '23.390'), (1, '29.030')] -[2023-10-14 14:51:20,408][75949] Updated weights for policy 0, policy_version 34631 (0.0010) -[2023-10-14 14:51:20,783][75949] Updated weights for policy 0, policy_version 34641 (0.0009) -[2023-10-14 14:51:21,146][75949] Updated weights for policy 0, policy_version 34651 (0.0009) -[2023-10-14 14:51:21,930][75950] Updated weights for policy 1, policy_version 34570 (0.0007) -[2023-10-14 14:51:22,284][75950] Updated weights for policy 1, policy_version 34580 (0.0007) -[2023-10-14 14:51:22,647][75950] Updated weights for policy 1, policy_version 34590 (0.0007) -[2023-10-14 14:51:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 70909952. Throughput: 0: 1667.1, 1: 1682.4. Samples: 17731152. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) -[2023-10-14 14:51:23,164][74987] Avg episode reward: [(0, '22.290'), (1, '29.180')] -[2023-10-14 14:51:25,218][75949] Updated weights for policy 0, policy_version 34661 (0.0009) -[2023-10-14 14:51:25,584][75949] Updated weights for policy 0, policy_version 34671 (0.0010) -[2023-10-14 14:51:25,949][75949] Updated weights for policy 0, policy_version 34681 (0.0011) -[2023-10-14 14:51:26,617][75950] Updated weights for policy 1, policy_version 34600 (0.0008) -[2023-10-14 14:51:26,987][75950] Updated weights for policy 1, policy_version 34610 (0.0008) -[2023-10-14 14:51:27,363][75950] Updated weights for policy 1, policy_version 34620 (0.0008) -[2023-10-14 14:51:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 70975488. Throughput: 0: 1686.5, 1: 1663.2. Samples: 17750864. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) -[2023-10-14 14:51:28,165][74987] Avg episode reward: [(0, '23.020'), (1, '28.570')] -[2023-10-14 14:51:30,026][75949] Updated weights for policy 0, policy_version 34691 (0.0011) -[2023-10-14 14:51:30,391][75949] Updated weights for policy 0, policy_version 34701 (0.0008) -[2023-10-14 14:51:30,770][75949] Updated weights for policy 0, policy_version 34711 (0.0009) -[2023-10-14 14:51:31,354][75950] Updated weights for policy 1, policy_version 34630 (0.0008) -[2023-10-14 14:51:31,715][75950] Updated weights for policy 1, policy_version 34640 (0.0009) -[2023-10-14 14:51:32,084][75950] Updated weights for policy 1, policy_version 34650 (0.0009) -[2023-10-14 14:51:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 71041024. Throughput: 0: 1666.8, 1: 1690.8. Samples: 17761778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:51:33,165][74987] Avg episode reward: [(0, '24.750'), (1, '26.900')] -[2023-10-14 14:51:34,585][75949] Updated weights for policy 0, policy_version 34721 (0.0010) -[2023-10-14 14:51:34,958][75949] Updated weights for policy 0, policy_version 34731 (0.0009) -[2023-10-14 14:51:35,328][75949] Updated weights for policy 0, policy_version 34741 (0.0008) -[2023-10-14 14:51:35,695][75949] Updated weights for policy 0, policy_version 34751 (0.0007) -[2023-10-14 14:51:36,176][75950] Updated weights for policy 1, policy_version 34660 (0.0009) -[2023-10-14 14:51:36,544][75950] Updated weights for policy 1, policy_version 34670 (0.0008) -[2023-10-14 14:51:36,907][75950] Updated weights for policy 1, policy_version 34680 (0.0007) -[2023-10-14 14:51:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 71106560. Throughput: 0: 1674.0, 1: 1674.8. Samples: 17781730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:51:38,165][74987] Avg episode reward: [(0, '24.550'), (1, '26.650')] -[2023-10-14 14:51:39,690][75949] Updated weights for policy 0, policy_version 34761 (0.0008) -[2023-10-14 14:51:40,067][75949] Updated weights for policy 0, policy_version 34771 (0.0008) -[2023-10-14 14:51:40,436][75949] Updated weights for policy 0, policy_version 34781 (0.0007) -[2023-10-14 14:51:40,910][75950] Updated weights for policy 1, policy_version 34690 (0.0007) -[2023-10-14 14:51:41,272][75950] Updated weights for policy 1, policy_version 34700 (0.0011) -[2023-10-14 14:51:41,636][75950] Updated weights for policy 1, policy_version 34710 (0.0007) -[2023-10-14 14:51:42,011][75950] Updated weights for policy 1, policy_version 34720 (0.0009) -[2023-10-14 14:51:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 71172096. Throughput: 0: 1688.4, 1: 1674.2. Samples: 17802016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:51:43,164][74987] Avg episode reward: [(0, '25.270'), (1, '26.790')] -[2023-10-14 14:51:44,503][75949] Updated weights for policy 0, policy_version 34791 (0.0009) -[2023-10-14 14:51:44,867][75949] Updated weights for policy 0, policy_version 34801 (0.0008) -[2023-10-14 14:51:45,230][75949] Updated weights for policy 0, policy_version 34811 (0.0007) -[2023-10-14 14:51:46,210][75950] Updated weights for policy 1, policy_version 34730 (0.0010) -[2023-10-14 14:51:46,576][75950] Updated weights for policy 1, policy_version 34740 (0.0009) -[2023-10-14 14:51:46,932][75950] Updated weights for policy 1, policy_version 34750 (0.0011) -[2023-10-14 14:51:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 71237632. Throughput: 0: 1666.5, 1: 1686.8. Samples: 17812526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:51:48,164][74987] Avg episode reward: [(0, '26.150'), (1, '28.360')] -[2023-10-14 14:51:49,330][75949] Updated weights for policy 0, policy_version 34821 (0.0009) -[2023-10-14 14:51:49,710][75949] Updated weights for policy 0, policy_version 34831 (0.0009) -[2023-10-14 14:51:50,079][75949] Updated weights for policy 0, policy_version 34841 (0.0009) -[2023-10-14 14:51:50,888][75950] Updated weights for policy 1, policy_version 34760 (0.0008) -[2023-10-14 14:51:51,259][75950] Updated weights for policy 1, policy_version 34770 (0.0007) -[2023-10-14 14:51:51,626][75950] Updated weights for policy 1, policy_version 34780 (0.0008) -[2023-10-14 14:51:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 71303168. Throughput: 0: 1685.3, 1: 1665.6. Samples: 17832036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:51:53,165][74987] Avg episode reward: [(0, '24.590'), (1, '27.910')] -[2023-10-14 14:51:54,335][75949] Updated weights for policy 0, policy_version 34851 (0.0007) -[2023-10-14 14:51:54,735][75949] Updated weights for policy 0, policy_version 34861 (0.0009) -[2023-10-14 14:51:55,106][75949] Updated weights for policy 0, policy_version 34871 (0.0009) -[2023-10-14 14:51:55,891][75950] Updated weights for policy 1, policy_version 34790 (0.0009) -[2023-10-14 14:51:56,259][75950] Updated weights for policy 1, policy_version 34800 (0.0010) -[2023-10-14 14:51:56,627][75950] Updated weights for policy 1, policy_version 34810 (0.0008) -[2023-10-14 14:51:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 71368704. Throughput: 0: 1683.1, 1: 1686.5. Samples: 17852318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:51:58,164][74987] Avg episode reward: [(0, '25.030'), (1, '28.920')] -[2023-10-14 14:51:59,043][75949] Updated weights for policy 0, policy_version 34881 (0.0008) -[2023-10-14 14:51:59,409][75949] Updated weights for policy 0, policy_version 34891 (0.0007) -[2023-10-14 14:51:59,774][75949] Updated weights for policy 0, policy_version 34901 (0.0008) -[2023-10-14 14:52:00,148][75949] Updated weights for policy 0, policy_version 34911 (0.0007) -[2023-10-14 14:52:00,695][75950] Updated weights for policy 1, policy_version 34820 (0.0009) -[2023-10-14 14:52:01,058][75950] Updated weights for policy 1, policy_version 34830 (0.0008) -[2023-10-14 14:52:01,422][75950] Updated weights for policy 1, policy_version 34840 (0.0009) -[2023-10-14 14:52:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 71434240. Throughput: 0: 1665.5, 1: 1692.8. Samples: 17862586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:52:03,165][74987] Avg episode reward: [(0, '23.680'), (1, '29.460')] -[2023-10-14 14:52:04,387][75949] Updated weights for policy 0, policy_version 34921 (0.0007) -[2023-10-14 14:52:04,754][75949] Updated weights for policy 0, policy_version 34931 (0.0008) -[2023-10-14 14:52:05,132][75949] Updated weights for policy 0, policy_version 34941 (0.0008) -[2023-10-14 14:52:05,384][75950] Updated weights for policy 1, policy_version 34850 (0.0008) -[2023-10-14 14:52:05,750][75950] Updated weights for policy 1, policy_version 34860 (0.0007) -[2023-10-14 14:52:06,108][75950] Updated weights for policy 1, policy_version 34870 (0.0007) -[2023-10-14 14:52:06,481][75950] Updated weights for policy 1, policy_version 34880 (0.0008) -[2023-10-14 14:52:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 71499776. Throughput: 0: 1686.6, 1: 1667.5. Samples: 17882086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:52:08,164][74987] Avg episode reward: [(0, '24.590'), (1, '27.930')] -[2023-10-14 14:52:09,108][75949] Updated weights for policy 0, policy_version 34951 (0.0007) -[2023-10-14 14:52:09,496][75949] Updated weights for policy 0, policy_version 34961 (0.0008) -[2023-10-14 14:52:09,861][75949] Updated weights for policy 0, policy_version 34971 (0.0009) -[2023-10-14 14:52:10,528][75950] Updated weights for policy 1, policy_version 34890 (0.0008) -[2023-10-14 14:52:10,895][75950] Updated weights for policy 1, policy_version 34900 (0.0007) -[2023-10-14 14:52:11,260][75950] Updated weights for policy 1, policy_version 34910 (0.0010) -[2023-10-14 14:52:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 71565312. Throughput: 0: 1689.9, 1: 1685.6. Samples: 17902758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:52:13,165][74987] Avg episode reward: [(0, '23.470'), (1, '29.410')] -[2023-10-14 14:52:13,868][75949] Updated weights for policy 0, policy_version 34981 (0.0009) -[2023-10-14 14:52:14,240][75949] Updated weights for policy 0, policy_version 34991 (0.0009) -[2023-10-14 14:52:14,617][75949] Updated weights for policy 0, policy_version 35001 (0.0009) -[2023-10-14 14:52:15,380][75950] Updated weights for policy 1, policy_version 34920 (0.0010) -[2023-10-14 14:52:15,744][75950] Updated weights for policy 1, policy_version 34930 (0.0007) -[2023-10-14 14:52:16,115][75950] Updated weights for policy 1, policy_version 34940 (0.0007) -[2023-10-14 14:52:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71630848. Throughput: 0: 1681.3, 1: 1670.9. Samples: 17912624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:52:18,164][74987] Avg episode reward: [(0, '24.940'), (1, '27.540')] -[2023-10-14 14:52:18,638][75949] Updated weights for policy 0, policy_version 35011 (0.0008) -[2023-10-14 14:52:19,004][75949] Updated weights for policy 0, policy_version 35021 (0.0007) -[2023-10-14 14:52:19,375][75949] Updated weights for policy 0, policy_version 35031 (0.0009) -[2023-10-14 14:52:20,475][75950] Updated weights for policy 1, policy_version 34950 (0.0009) -[2023-10-14 14:52:20,839][75950] Updated weights for policy 1, policy_version 34960 (0.0009) -[2023-10-14 14:52:21,217][75950] Updated weights for policy 1, policy_version 34970 (0.0008) -[2023-10-14 14:52:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71696384. Throughput: 0: 1688.1, 1: 1663.3. Samples: 17932544. Policy #0 lag: (min: 12.0, avg: 15.2, max: 44.0) -[2023-10-14 14:52:23,164][74987] Avg episode reward: [(0, '24.000'), (1, '27.210')] -[2023-10-14 14:52:23,414][75949] Updated weights for policy 0, policy_version 35041 (0.0009) -[2023-10-14 14:52:23,776][75949] Updated weights for policy 0, policy_version 35051 (0.0009) -[2023-10-14 14:52:24,159][75949] Updated weights for policy 0, policy_version 35061 (0.0008) -[2023-10-14 14:52:24,535][75949] Updated weights for policy 0, policy_version 35071 (0.0007) -[2023-10-14 14:52:25,457][75950] Updated weights for policy 1, policy_version 34980 (0.0009) -[2023-10-14 14:52:25,823][75950] Updated weights for policy 1, policy_version 34990 (0.0007) -[2023-10-14 14:52:26,183][75950] Updated weights for policy 1, policy_version 35000 (0.0009) -[2023-10-14 14:52:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71761920. Throughput: 0: 1685.7, 1: 1670.6. Samples: 17953052. Policy #0 lag: (min: 12.0, avg: 15.2, max: 44.0) -[2023-10-14 14:52:28,164][74987] Avg episode reward: [(0, '24.000'), (1, '27.770')] -[2023-10-14 14:52:28,521][75949] Updated weights for policy 0, policy_version 35081 (0.0011) -[2023-10-14 14:52:28,887][75949] Updated weights for policy 0, policy_version 35091 (0.0007) -[2023-10-14 14:52:29,242][75949] Updated weights for policy 0, policy_version 35101 (0.0010) -[2023-10-14 14:52:30,287][75950] Updated weights for policy 1, policy_version 35010 (0.0008) -[2023-10-14 14:52:30,652][75950] Updated weights for policy 1, policy_version 35020 (0.0007) -[2023-10-14 14:52:31,021][75950] Updated weights for policy 1, policy_version 35030 (0.0007) -[2023-10-14 14:52:31,384][75950] Updated weights for policy 1, policy_version 35040 (0.0009) -[2023-10-14 14:52:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71827456. Throughput: 0: 1682.2, 1: 1660.6. Samples: 17962950. Policy #0 lag: (min: 12.0, avg: 15.2, max: 44.0) -[2023-10-14 14:52:33,165][74987] Avg episode reward: [(0, '24.970'), (1, '27.620')] -[2023-10-14 14:52:33,456][75949] Updated weights for policy 0, policy_version 35111 (0.0008) -[2023-10-14 14:52:33,822][75949] Updated weights for policy 0, policy_version 35121 (0.0009) -[2023-10-14 14:52:34,200][75949] Updated weights for policy 0, policy_version 35131 (0.0008) -[2023-10-14 14:52:35,255][75950] Updated weights for policy 1, policy_version 35050 (0.0010) -[2023-10-14 14:52:35,623][75950] Updated weights for policy 1, policy_version 35060 (0.0009) -[2023-10-14 14:52:35,978][75950] Updated weights for policy 1, policy_version 35070 (0.0008) -[2023-10-14 14:52:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71892992. Throughput: 0: 1686.1, 1: 1664.0. Samples: 17982788. Policy #0 lag: (min: 12.0, avg: 15.2, max: 44.0) -[2023-10-14 14:52:38,165][74987] Avg episode reward: [(0, '23.900'), (1, '29.150')] -[2023-10-14 14:52:38,242][75949] Updated weights for policy 0, policy_version 35141 (0.0007) -[2023-10-14 14:52:38,603][75949] Updated weights for policy 0, policy_version 35151 (0.0009) -[2023-10-14 14:52:38,969][75949] Updated weights for policy 0, policy_version 35161 (0.0009) -[2023-10-14 14:52:40,163][75950] Updated weights for policy 1, policy_version 35080 (0.0010) -[2023-10-14 14:52:40,544][75950] Updated weights for policy 1, policy_version 35090 (0.0008) -[2023-10-14 14:52:40,915][75950] Updated weights for policy 1, policy_version 35100 (0.0007) -[2023-10-14 14:52:43,029][75949] Updated weights for policy 0, policy_version 35171 (0.0009) -[2023-10-14 14:52:43,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71958528. Throughput: 0: 1692.2, 1: 1668.8. Samples: 18003560. Policy #0 lag: (min: 12.0, avg: 15.2, max: 44.0) -[2023-10-14 14:52:43,164][74987] Avg episode reward: [(0, '25.140'), (1, '27.180')] -[2023-10-14 14:52:43,171][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000035104_35946496.pth... -[2023-10-14 14:52:43,202][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000033536_34340864.pth -[2023-10-14 14:52:43,424][75949] Updated weights for policy 0, policy_version 35181 (0.0010) -[2023-10-14 14:52:43,790][75949] Updated weights for policy 0, policy_version 35191 (0.0008) -[2023-10-14 14:52:44,128][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000035200_36044800.pth... -[2023-10-14 14:52:44,164][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000033600_34406400.pth -[2023-10-14 14:52:44,935][75950] Updated weights for policy 1, policy_version 35110 (0.0009) -[2023-10-14 14:52:45,298][75950] Updated weights for policy 1, policy_version 35120 (0.0009) -[2023-10-14 14:52:45,669][75950] Updated weights for policy 1, policy_version 35130 (0.0007) -[2023-10-14 14:52:47,911][75949] Updated weights for policy 0, policy_version 35201 (0.0011) -[2023-10-14 14:52:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72024064. Throughput: 0: 1688.6, 1: 1649.7. Samples: 18012812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:52:48,164][74987] Avg episode reward: [(0, '23.610'), (1, '27.780')] -[2023-10-14 14:52:48,273][75949] Updated weights for policy 0, policy_version 35211 (0.0009) -[2023-10-14 14:52:48,650][75949] Updated weights for policy 0, policy_version 35221 (0.0010) -[2023-10-14 14:52:49,020][75949] Updated weights for policy 0, policy_version 35231 (0.0008) -[2023-10-14 14:52:49,752][75950] Updated weights for policy 1, policy_version 35140 (0.0008) -[2023-10-14 14:52:50,122][75950] Updated weights for policy 1, policy_version 35150 (0.0008) -[2023-10-14 14:52:50,484][75950] Updated weights for policy 1, policy_version 35160 (0.0007) -[2023-10-14 14:52:52,963][75949] Updated weights for policy 0, policy_version 35241 (0.0010) -[2023-10-14 14:52:53,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72089600. Throughput: 0: 1691.0, 1: 1666.5. Samples: 18033176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:52:53,164][74987] Avg episode reward: [(0, '26.300'), (1, '28.890')] -[2023-10-14 14:52:53,342][75949] Updated weights for policy 0, policy_version 35251 (0.0008) -[2023-10-14 14:52:53,719][75949] Updated weights for policy 0, policy_version 35261 (0.0009) -[2023-10-14 14:52:54,384][75950] Updated weights for policy 1, policy_version 35170 (0.0007) -[2023-10-14 14:52:54,754][75950] Updated weights for policy 1, policy_version 35180 (0.0008) -[2023-10-14 14:52:55,118][75950] Updated weights for policy 1, policy_version 35190 (0.0008) -[2023-10-14 14:52:55,492][75950] Updated weights for policy 1, policy_version 35200 (0.0007) -[2023-10-14 14:52:57,708][75949] Updated weights for policy 0, policy_version 35271 (0.0009) -[2023-10-14 14:52:58,071][75949] Updated weights for policy 0, policy_version 35281 (0.0010) -[2023-10-14 14:52:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72155136. Throughput: 0: 1684.8, 1: 1675.2. Samples: 18053960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:52:58,164][74987] Avg episode reward: [(0, '23.630'), (1, '27.180')] -[2023-10-14 14:52:58,440][75949] Updated weights for policy 0, policy_version 35291 (0.0009) -[2023-10-14 14:52:59,786][75950] Updated weights for policy 1, policy_version 35210 (0.0009) -[2023-10-14 14:53:00,153][75950] Updated weights for policy 1, policy_version 35220 (0.0009) -[2023-10-14 14:53:00,518][75950] Updated weights for policy 1, policy_version 35230 (0.0007) -[2023-10-14 14:53:02,680][75949] Updated weights for policy 0, policy_version 35301 (0.0009) -[2023-10-14 14:53:03,059][75949] Updated weights for policy 0, policy_version 35311 (0.0009) -[2023-10-14 14:53:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 72220672. Throughput: 0: 1685.6, 1: 1663.2. Samples: 18063318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:53:03,164][74987] Avg episode reward: [(0, '26.150'), (1, '27.490')] -[2023-10-14 14:53:03,436][75949] Updated weights for policy 0, policy_version 35321 (0.0009) -[2023-10-14 14:53:04,273][75950] Updated weights for policy 1, policy_version 35240 (0.0007) -[2023-10-14 14:53:04,640][75950] Updated weights for policy 1, policy_version 35250 (0.0010) -[2023-10-14 14:53:05,008][75950] Updated weights for policy 1, policy_version 35260 (0.0008) -[2023-10-14 14:53:07,542][75949] Updated weights for policy 0, policy_version 35331 (0.0009) -[2023-10-14 14:53:07,910][75949] Updated weights for policy 0, policy_version 35341 (0.0009) -[2023-10-14 14:53:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 72286208. Throughput: 0: 1680.3, 1: 1689.6. Samples: 18084190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:53:08,165][74987] Avg episode reward: [(0, '23.810'), (1, '29.200')] -[2023-10-14 14:53:08,286][75949] Updated weights for policy 0, policy_version 35351 (0.0008) -[2023-10-14 14:53:09,010][75950] Updated weights for policy 1, policy_version 35270 (0.0007) -[2023-10-14 14:53:09,366][75950] Updated weights for policy 1, policy_version 35280 (0.0007) -[2023-10-14 14:53:09,741][75950] Updated weights for policy 1, policy_version 35290 (0.0009) -[2023-10-14 14:53:12,393][75949] Updated weights for policy 0, policy_version 35361 (0.0008) -[2023-10-14 14:53:12,771][75949] Updated weights for policy 0, policy_version 35371 (0.0011) -[2023-10-14 14:53:13,142][75949] Updated weights for policy 0, policy_version 35381 (0.0009) -[2023-10-14 14:53:13,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 72351744. Throughput: 0: 1672.6, 1: 1696.4. Samples: 18104658. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:53:13,164][74987] Avg episode reward: [(0, '26.160'), (1, '27.520')] -[2023-10-14 14:53:13,520][75949] Updated weights for policy 0, policy_version 35391 (0.0009) -[2023-10-14 14:53:13,811][75950] Updated weights for policy 1, policy_version 35300 (0.0008) -[2023-10-14 14:53:14,178][75950] Updated weights for policy 1, policy_version 35310 (0.0007) -[2023-10-14 14:53:14,536][75950] Updated weights for policy 1, policy_version 35320 (0.0011) -[2023-10-14 14:53:17,638][75949] Updated weights for policy 0, policy_version 35401 (0.0007) -[2023-10-14 14:53:18,003][75949] Updated weights for policy 0, policy_version 35411 (0.0007) -[2023-10-14 14:53:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72417280. Throughput: 0: 1680.5, 1: 1675.1. Samples: 18113950. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:53:18,164][74987] Avg episode reward: [(0, '22.870'), (1, '28.500')] -[2023-10-14 14:53:18,371][75949] Updated weights for policy 0, policy_version 35421 (0.0008) -[2023-10-14 14:53:18,695][75950] Updated weights for policy 1, policy_version 35330 (0.0009) -[2023-10-14 14:53:19,068][75950] Updated weights for policy 1, policy_version 35340 (0.0009) -[2023-10-14 14:53:19,428][75950] Updated weights for policy 1, policy_version 35350 (0.0008) -[2023-10-14 14:53:19,799][75950] Updated weights for policy 1, policy_version 35360 (0.0009) -[2023-10-14 14:53:22,355][75949] Updated weights for policy 0, policy_version 35431 (0.0007) -[2023-10-14 14:53:22,716][75949] Updated weights for policy 0, policy_version 35441 (0.0007) -[2023-10-14 14:53:23,086][75949] Updated weights for policy 0, policy_version 35451 (0.0007) -[2023-10-14 14:53:23,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72482816. Throughput: 0: 1680.7, 1: 1690.7. Samples: 18134502. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:53:23,164][74987] Avg episode reward: [(0, '24.220'), (1, '31.020')] -[2023-10-14 14:53:23,165][75801] Saving new best policy, reward=31.020! -[2023-10-14 14:53:23,907][75950] Updated weights for policy 1, policy_version 35370 (0.0008) -[2023-10-14 14:53:24,273][75950] Updated weights for policy 1, policy_version 35380 (0.0010) -[2023-10-14 14:53:24,634][75950] Updated weights for policy 1, policy_version 35390 (0.0010) -[2023-10-14 14:53:27,102][75949] Updated weights for policy 0, policy_version 35461 (0.0010) -[2023-10-14 14:53:27,477][75949] Updated weights for policy 0, policy_version 35471 (0.0008) -[2023-10-14 14:53:27,849][75949] Updated weights for policy 0, policy_version 35481 (0.0008) -[2023-10-14 14:53:28,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 72581120. Throughput: 0: 1661.9, 1: 1687.9. Samples: 18154302. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:53:28,165][74987] Avg episode reward: [(0, '25.220'), (1, '28.790')] -[2023-10-14 14:53:28,939][75950] Updated weights for policy 1, policy_version 35400 (0.0009) -[2023-10-14 14:53:29,325][75950] Updated weights for policy 1, policy_version 35410 (0.0007) -[2023-10-14 14:53:29,693][75950] Updated weights for policy 1, policy_version 35420 (0.0007) -[2023-10-14 14:53:32,114][75949] Updated weights for policy 0, policy_version 35491 (0.0008) -[2023-10-14 14:53:32,509][75949] Updated weights for policy 0, policy_version 35501 (0.0009) -[2023-10-14 14:53:32,883][75949] Updated weights for policy 0, policy_version 35511 (0.0008) -[2023-10-14 14:53:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72613888. Throughput: 0: 1682.4, 1: 1677.1. Samples: 18163990. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 14:53:33,164][74987] Avg episode reward: [(0, '23.880'), (1, '28.150')] -[2023-10-14 14:53:33,741][75950] Updated weights for policy 1, policy_version 35430 (0.0008) -[2023-10-14 14:53:34,101][75950] Updated weights for policy 1, policy_version 35440 (0.0008) -[2023-10-14 14:53:34,465][75950] Updated weights for policy 1, policy_version 35450 (0.0008) -[2023-10-14 14:53:36,724][75949] Updated weights for policy 0, policy_version 35521 (0.0007) -[2023-10-14 14:53:37,089][75949] Updated weights for policy 0, policy_version 35531 (0.0008) -[2023-10-14 14:53:37,455][75949] Updated weights for policy 0, policy_version 35541 (0.0010) -[2023-10-14 14:53:37,820][75949] Updated weights for policy 0, policy_version 35551 (0.0008) -[2023-10-14 14:53:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 72712192. Throughput: 0: 1679.1, 1: 1686.7. Samples: 18184634. Policy #0 lag: (min: 13.0, avg: 18.5, max: 45.0) -[2023-10-14 14:53:38,164][74987] Avg episode reward: [(0, '24.430'), (1, '29.260')] -[2023-10-14 14:53:38,635][75950] Updated weights for policy 1, policy_version 35460 (0.0008) -[2023-10-14 14:53:39,000][75950] Updated weights for policy 1, policy_version 35470 (0.0009) -[2023-10-14 14:53:39,367][75950] Updated weights for policy 1, policy_version 35480 (0.0009) -[2023-10-14 14:53:41,658][75949] Updated weights for policy 0, policy_version 35561 (0.0008) -[2023-10-14 14:53:42,025][75949] Updated weights for policy 0, policy_version 35571 (0.0008) -[2023-10-14 14:53:42,390][75949] Updated weights for policy 0, policy_version 35581 (0.0009) -[2023-10-14 14:53:43,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 72777728. Throughput: 0: 1663.0, 1: 1683.7. Samples: 18204564. Policy #0 lag: (min: 13.0, avg: 18.5, max: 45.0) -[2023-10-14 14:53:43,164][74987] Avg episode reward: [(0, '23.150'), (1, '28.840')] -[2023-10-14 14:53:43,227][75950] Updated weights for policy 1, policy_version 35490 (0.0009) -[2023-10-14 14:53:43,603][75950] Updated weights for policy 1, policy_version 35500 (0.0009) -[2023-10-14 14:53:43,971][75950] Updated weights for policy 1, policy_version 35510 (0.0010) -[2023-10-14 14:53:44,339][75950] Updated weights for policy 1, policy_version 35520 (0.0009) -[2023-10-14 14:53:46,486][75949] Updated weights for policy 0, policy_version 35591 (0.0008) -[2023-10-14 14:53:46,850][75949] Updated weights for policy 0, policy_version 35601 (0.0008) -[2023-10-14 14:53:47,221][75949] Updated weights for policy 0, policy_version 35611 (0.0008) -[2023-10-14 14:53:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 72843264. Throughput: 0: 1688.7, 1: 1678.4. Samples: 18214838. Policy #0 lag: (min: 13.0, avg: 18.5, max: 45.0) -[2023-10-14 14:53:48,164][74987] Avg episode reward: [(0, '25.990'), (1, '28.480')] -[2023-10-14 14:53:48,372][75950] Updated weights for policy 1, policy_version 35530 (0.0009) -[2023-10-14 14:53:48,747][75950] Updated weights for policy 1, policy_version 35540 (0.0009) -[2023-10-14 14:53:49,115][75950] Updated weights for policy 1, policy_version 35550 (0.0008) -[2023-10-14 14:53:51,368][75949] Updated weights for policy 0, policy_version 35621 (0.0009) -[2023-10-14 14:53:51,742][75949] Updated weights for policy 0, policy_version 35631 (0.0011) -[2023-10-14 14:53:52,119][75949] Updated weights for policy 0, policy_version 35641 (0.0009) -[2023-10-14 14:53:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 72908800. Throughput: 0: 1682.0, 1: 1674.5. Samples: 18235236. Policy #0 lag: (min: 13.0, avg: 18.5, max: 45.0) -[2023-10-14 14:53:53,165][74987] Avg episode reward: [(0, '24.510'), (1, '30.280')] -[2023-10-14 14:53:53,255][75950] Updated weights for policy 1, policy_version 35560 (0.0007) -[2023-10-14 14:53:53,630][75950] Updated weights for policy 1, policy_version 35570 (0.0008) -[2023-10-14 14:53:53,989][75950] Updated weights for policy 1, policy_version 35580 (0.0009) -[2023-10-14 14:53:56,160][75949] Updated weights for policy 0, policy_version 35651 (0.0011) -[2023-10-14 14:53:56,538][75949] Updated weights for policy 0, policy_version 35661 (0.0008) -[2023-10-14 14:53:56,918][75949] Updated weights for policy 0, policy_version 35671 (0.0009) -[2023-10-14 14:53:58,024][75950] Updated weights for policy 1, policy_version 35590 (0.0008) -[2023-10-14 14:53:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 72974336. Throughput: 0: 1669.2, 1: 1677.8. Samples: 18255276. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:53:58,165][74987] Avg episode reward: [(0, '26.660'), (1, '28.500')] -[2023-10-14 14:53:58,391][75950] Updated weights for policy 1, policy_version 35600 (0.0009) -[2023-10-14 14:53:58,769][75950] Updated weights for policy 1, policy_version 35610 (0.0010) -[2023-10-14 14:54:01,076][75949] Updated weights for policy 0, policy_version 35681 (0.0008) -[2023-10-14 14:54:01,440][75949] Updated weights for policy 0, policy_version 35691 (0.0008) -[2023-10-14 14:54:01,822][75949] Updated weights for policy 0, policy_version 35701 (0.0009) -[2023-10-14 14:54:02,194][75949] Updated weights for policy 0, policy_version 35711 (0.0008) -[2023-10-14 14:54:02,784][75950] Updated weights for policy 1, policy_version 35620 (0.0010) -[2023-10-14 14:54:03,142][75950] Updated weights for policy 1, policy_version 35630 (0.0007) -[2023-10-14 14:54:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 73039872. Throughput: 0: 1692.4, 1: 1675.6. Samples: 18265510. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:54:03,165][74987] Avg episode reward: [(0, '23.690'), (1, '27.760')] -[2023-10-14 14:54:03,506][75950] Updated weights for policy 1, policy_version 35640 (0.0007) -[2023-10-14 14:54:06,171][75949] Updated weights for policy 0, policy_version 35721 (0.0008) -[2023-10-14 14:54:06,542][75949] Updated weights for policy 0, policy_version 35731 (0.0009) -[2023-10-14 14:54:06,919][75949] Updated weights for policy 0, policy_version 35741 (0.0011) -[2023-10-14 14:54:07,637][75950] Updated weights for policy 1, policy_version 35650 (0.0009) -[2023-10-14 14:54:08,013][75950] Updated weights for policy 1, policy_version 35660 (0.0011) -[2023-10-14 14:54:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 73105408. Throughput: 0: 1673.3, 1: 1678.8. Samples: 18285346. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:54:08,164][74987] Avg episode reward: [(0, '25.370'), (1, '28.760')] -[2023-10-14 14:54:08,386][75950] Updated weights for policy 1, policy_version 35670 (0.0010) -[2023-10-14 14:54:08,744][75950] Updated weights for policy 1, policy_version 35680 (0.0008) -[2023-10-14 14:54:11,168][75949] Updated weights for policy 0, policy_version 35751 (0.0009) -[2023-10-14 14:54:11,532][75949] Updated weights for policy 0, policy_version 35761 (0.0009) -[2023-10-14 14:54:11,906][75949] Updated weights for policy 0, policy_version 35771 (0.0008) -[2023-10-14 14:54:12,916][75950] Updated weights for policy 1, policy_version 35690 (0.0007) -[2023-10-14 14:54:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 73170944. Throughput: 0: 1677.5, 1: 1676.5. Samples: 18305232. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:54:13,164][74987] Avg episode reward: [(0, '23.950'), (1, '26.940')] -[2023-10-14 14:54:13,274][75950] Updated weights for policy 1, policy_version 35700 (0.0007) -[2023-10-14 14:54:13,635][75950] Updated weights for policy 1, policy_version 35710 (0.0007) -[2023-10-14 14:54:16,017][75949] Updated weights for policy 0, policy_version 35781 (0.0009) -[2023-10-14 14:54:16,395][75949] Updated weights for policy 0, policy_version 35791 (0.0010) -[2023-10-14 14:54:16,768][75949] Updated weights for policy 0, policy_version 35801 (0.0011) -[2023-10-14 14:54:17,873][75950] Updated weights for policy 1, policy_version 35720 (0.0008) -[2023-10-14 14:54:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 73236480. Throughput: 0: 1690.0, 1: 1679.7. Samples: 18315624. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) -[2023-10-14 14:54:18,164][74987] Avg episode reward: [(0, '25.230'), (1, '29.270')] -[2023-10-14 14:54:18,256][75950] Updated weights for policy 1, policy_version 35730 (0.0008) -[2023-10-14 14:54:18,613][75950] Updated weights for policy 1, policy_version 35740 (0.0008) -[2023-10-14 14:54:20,815][75949] Updated weights for policy 0, policy_version 35811 (0.0008) -[2023-10-14 14:54:21,203][75949] Updated weights for policy 0, policy_version 35821 (0.0010) -[2023-10-14 14:54:21,571][75949] Updated weights for policy 0, policy_version 35831 (0.0008) -[2023-10-14 14:54:22,927][75950] Updated weights for policy 1, policy_version 35750 (0.0009) -[2023-10-14 14:54:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 73302016. Throughput: 0: 1666.4, 1: 1675.1. Samples: 18335002. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 14:54:23,165][74987] Avg episode reward: [(0, '23.130'), (1, '30.320')] -[2023-10-14 14:54:23,285][75950] Updated weights for policy 1, policy_version 35760 (0.0009) -[2023-10-14 14:54:23,665][75950] Updated weights for policy 1, policy_version 35770 (0.0008) -[2023-10-14 14:54:25,538][75949] Updated weights for policy 0, policy_version 35841 (0.0007) -[2023-10-14 14:54:25,920][75949] Updated weights for policy 0, policy_version 35851 (0.0008) -[2023-10-14 14:54:26,284][75949] Updated weights for policy 0, policy_version 35861 (0.0009) -[2023-10-14 14:54:26,652][75949] Updated weights for policy 0, policy_version 35871 (0.0010) -[2023-10-14 14:54:27,698][75950] Updated weights for policy 1, policy_version 35780 (0.0009) -[2023-10-14 14:54:28,070][75950] Updated weights for policy 1, policy_version 35790 (0.0009) -[2023-10-14 14:54:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 73367552. Throughput: 0: 1679.0, 1: 1668.8. Samples: 18355216. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 14:54:28,165][74987] Avg episode reward: [(0, '25.290'), (1, '29.410')] -[2023-10-14 14:54:28,426][75950] Updated weights for policy 1, policy_version 35800 (0.0010) -[2023-10-14 14:54:30,642][75949] Updated weights for policy 0, policy_version 35881 (0.0008) -[2023-10-14 14:54:31,016][75949] Updated weights for policy 0, policy_version 35891 (0.0009) -[2023-10-14 14:54:31,390][75949] Updated weights for policy 0, policy_version 35901 (0.0009) -[2023-10-14 14:54:32,645][75950] Updated weights for policy 1, policy_version 35810 (0.0010) -[2023-10-14 14:54:33,013][75950] Updated weights for policy 1, policy_version 35820 (0.0008) -[2023-10-14 14:54:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 73433088. Throughput: 0: 1670.3, 1: 1671.4. Samples: 18365216. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 14:54:33,165][74987] Avg episode reward: [(0, '24.360'), (1, '29.100')] -[2023-10-14 14:54:33,383][75950] Updated weights for policy 1, policy_version 35830 (0.0009) -[2023-10-14 14:54:33,760][75950] Updated weights for policy 1, policy_version 35840 (0.0007) -[2023-10-14 14:54:35,461][75949] Updated weights for policy 0, policy_version 35911 (0.0008) -[2023-10-14 14:54:35,822][75949] Updated weights for policy 0, policy_version 35921 (0.0009) -[2023-10-14 14:54:36,203][75949] Updated weights for policy 0, policy_version 35931 (0.0009) -[2023-10-14 14:54:38,059][75950] Updated weights for policy 1, policy_version 35850 (0.0008) -[2023-10-14 14:54:38,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 73498624. Throughput: 0: 1660.3, 1: 1662.9. Samples: 18384778. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 14:54:38,164][74987] Avg episode reward: [(0, '25.930'), (1, '30.580')] -[2023-10-14 14:54:38,426][75950] Updated weights for policy 1, policy_version 35860 (0.0007) -[2023-10-14 14:54:38,791][75950] Updated weights for policy 1, policy_version 35870 (0.0007) -[2023-10-14 14:54:40,280][75949] Updated weights for policy 0, policy_version 35941 (0.0008) -[2023-10-14 14:54:40,655][75949] Updated weights for policy 0, policy_version 35951 (0.0008) -[2023-10-14 14:54:41,024][75949] Updated weights for policy 0, policy_version 35961 (0.0008) -[2023-10-14 14:54:42,681][75950] Updated weights for policy 1, policy_version 35880 (0.0010) -[2023-10-14 14:54:43,053][75950] Updated weights for policy 1, policy_version 35890 (0.0011) -[2023-10-14 14:54:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 73564160. Throughput: 0: 1684.3, 1: 1657.1. Samples: 18405638. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 14:54:43,164][74987] Avg episode reward: [(0, '24.230'), (1, '28.060')] -[2023-10-14 14:54:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000035968_36831232.pth... -[2023-10-14 14:54:43,200][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000034400_35225600.pth -[2023-10-14 14:54:43,418][75950] Updated weights for policy 1, policy_version 35900 (0.0010) -[2023-10-14 14:54:43,562][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000035904_36765696.pth... -[2023-10-14 14:54:43,599][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000034336_35160064.pth -[2023-10-14 14:54:45,017][75949] Updated weights for policy 0, policy_version 35971 (0.0009) -[2023-10-14 14:54:45,390][75949] Updated weights for policy 0, policy_version 35981 (0.0008) -[2023-10-14 14:54:45,761][75949] Updated weights for policy 0, policy_version 35991 (0.0008) -[2023-10-14 14:54:47,489][75950] Updated weights for policy 1, policy_version 35910 (0.0011) -[2023-10-14 14:54:47,852][75950] Updated weights for policy 1, policy_version 35920 (0.0008) -[2023-10-14 14:54:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 73629696. Throughput: 0: 1666.1, 1: 1666.0. Samples: 18415454. Policy #0 lag: (min: 0.0, avg: 22.3, max: 32.0) -[2023-10-14 14:54:48,165][74987] Avg episode reward: [(0, '24.720'), (1, '29.080')] -[2023-10-14 14:54:48,219][75950] Updated weights for policy 1, policy_version 35930 (0.0007) -[2023-10-14 14:54:49,722][75949] Updated weights for policy 0, policy_version 36001 (0.0009) -[2023-10-14 14:54:50,090][75949] Updated weights for policy 0, policy_version 36011 (0.0011) -[2023-10-14 14:54:50,464][75949] Updated weights for policy 0, policy_version 36021 (0.0008) -[2023-10-14 14:54:50,834][75949] Updated weights for policy 0, policy_version 36031 (0.0008) -[2023-10-14 14:54:52,235][75950] Updated weights for policy 1, policy_version 35940 (0.0010) -[2023-10-14 14:54:52,608][75950] Updated weights for policy 1, policy_version 35950 (0.0007) -[2023-10-14 14:54:52,971][75950] Updated weights for policy 1, policy_version 35960 (0.0009) -[2023-10-14 14:54:53,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 73695232. Throughput: 0: 1673.3, 1: 1664.9. Samples: 18435564. Policy #0 lag: (min: 0.0, avg: 22.3, max: 32.0) -[2023-10-14 14:54:53,164][74987] Avg episode reward: [(0, '24.100'), (1, '27.180')] -[2023-10-14 14:54:54,937][75949] Updated weights for policy 0, policy_version 36041 (0.0010) -[2023-10-14 14:54:55,310][75949] Updated weights for policy 0, policy_version 36051 (0.0009) -[2023-10-14 14:54:55,687][75949] Updated weights for policy 0, policy_version 36061 (0.0007) -[2023-10-14 14:54:57,052][75950] Updated weights for policy 1, policy_version 35970 (0.0009) -[2023-10-14 14:54:57,420][75950] Updated weights for policy 1, policy_version 35980 (0.0009) -[2023-10-14 14:54:57,794][75950] Updated weights for policy 1, policy_version 35990 (0.0008) -[2023-10-14 14:54:58,158][75950] Updated weights for policy 1, policy_version 36000 (0.0008) -[2023-10-14 14:54:58,163][74987] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 73793536. Throughput: 0: 1684.6, 1: 1651.3. Samples: 18455346. Policy #0 lag: (min: 0.0, avg: 22.3, max: 32.0) -[2023-10-14 14:54:58,164][74987] Avg episode reward: [(0, '25.210'), (1, '27.870')] -[2023-10-14 14:54:59,763][75949] Updated weights for policy 0, policy_version 36071 (0.0008) -[2023-10-14 14:55:00,143][75949] Updated weights for policy 0, policy_version 36081 (0.0010) -[2023-10-14 14:55:00,519][75949] Updated weights for policy 0, policy_version 36091 (0.0009) -[2023-10-14 14:55:02,067][75950] Updated weights for policy 1, policy_version 36010 (0.0007) -[2023-10-14 14:55:02,432][75950] Updated weights for policy 1, policy_version 36020 (0.0007) -[2023-10-14 14:55:02,802][75950] Updated weights for policy 1, policy_version 36030 (0.0007) -[2023-10-14 14:55:03,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 73859072. Throughput: 0: 1657.5, 1: 1670.6. Samples: 18465392. Policy #0 lag: (min: 0.0, avg: 22.3, max: 32.0) -[2023-10-14 14:55:03,165][74987] Avg episode reward: [(0, '24.200'), (1, '29.430')] -[2023-10-14 14:55:04,616][75949] Updated weights for policy 0, policy_version 36101 (0.0009) -[2023-10-14 14:55:04,989][75949] Updated weights for policy 0, policy_version 36111 (0.0010) -[2023-10-14 14:55:05,370][75949] Updated weights for policy 0, policy_version 36121 (0.0011) -[2023-10-14 14:55:07,023][75950] Updated weights for policy 1, policy_version 36040 (0.0009) -[2023-10-14 14:55:07,401][75950] Updated weights for policy 1, policy_version 36050 (0.0009) -[2023-10-14 14:55:07,770][75950] Updated weights for policy 1, policy_version 36060 (0.0008) -[2023-10-14 14:55:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 73924608. Throughput: 0: 1678.7, 1: 1675.5. Samples: 18485940. Policy #0 lag: (min: 0.0, avg: 22.3, max: 32.0) -[2023-10-14 14:55:08,164][74987] Avg episode reward: [(0, '25.800'), (1, '28.260')] -[2023-10-14 14:55:09,390][75949] Updated weights for policy 0, policy_version 36131 (0.0007) -[2023-10-14 14:55:09,783][75949] Updated weights for policy 0, policy_version 36141 (0.0007) -[2023-10-14 14:55:10,158][75949] Updated weights for policy 0, policy_version 36151 (0.0007) -[2023-10-14 14:55:11,821][75950] Updated weights for policy 1, policy_version 36070 (0.0009) -[2023-10-14 14:55:12,197][75950] Updated weights for policy 1, policy_version 36080 (0.0008) -[2023-10-14 14:55:12,563][75950] Updated weights for policy 1, policy_version 36090 (0.0008) -[2023-10-14 14:55:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 73990144. Throughput: 0: 1684.7, 1: 1652.2. Samples: 18505376. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) -[2023-10-14 14:55:13,164][74987] Avg episode reward: [(0, '23.040'), (1, '29.190')] -[2023-10-14 14:55:14,258][75949] Updated weights for policy 0, policy_version 36161 (0.0009) -[2023-10-14 14:55:14,635][75949] Updated weights for policy 0, policy_version 36171 (0.0008) -[2023-10-14 14:55:15,000][75949] Updated weights for policy 0, policy_version 36181 (0.0010) -[2023-10-14 14:55:15,370][75949] Updated weights for policy 0, policy_version 36191 (0.0007) -[2023-10-14 14:55:16,650][75950] Updated weights for policy 1, policy_version 36100 (0.0008) -[2023-10-14 14:55:17,018][75950] Updated weights for policy 1, policy_version 36110 (0.0009) -[2023-10-14 14:55:17,385][75950] Updated weights for policy 1, policy_version 36120 (0.0009) -[2023-10-14 14:55:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 74055680. Throughput: 0: 1663.9, 1: 1676.2. Samples: 18515522. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) -[2023-10-14 14:55:18,164][74987] Avg episode reward: [(0, '25.110'), (1, '30.670')] -[2023-10-14 14:55:19,605][75949] Updated weights for policy 0, policy_version 36201 (0.0008) -[2023-10-14 14:55:19,978][75949] Updated weights for policy 0, policy_version 36211 (0.0008) -[2023-10-14 14:55:20,345][75949] Updated weights for policy 0, policy_version 36221 (0.0008) -[2023-10-14 14:55:21,402][75950] Updated weights for policy 1, policy_version 36130 (0.0009) -[2023-10-14 14:55:21,764][75950] Updated weights for policy 1, policy_version 36140 (0.0007) -[2023-10-14 14:55:22,132][75950] Updated weights for policy 1, policy_version 36150 (0.0007) -[2023-10-14 14:55:22,496][75950] Updated weights for policy 1, policy_version 36160 (0.0008) -[2023-10-14 14:55:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 74121216. Throughput: 0: 1683.6, 1: 1675.1. Samples: 18535920. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) -[2023-10-14 14:55:23,164][74987] Avg episode reward: [(0, '24.160'), (1, '28.250')] -[2023-10-14 14:55:24,268][75949] Updated weights for policy 0, policy_version 36231 (0.0009) -[2023-10-14 14:55:24,636][75949] Updated weights for policy 0, policy_version 36241 (0.0007) -[2023-10-14 14:55:24,998][75949] Updated weights for policy 0, policy_version 36251 (0.0008) -[2023-10-14 14:55:26,847][75950] Updated weights for policy 1, policy_version 36170 (0.0009) -[2023-10-14 14:55:27,209][75950] Updated weights for policy 1, policy_version 36180 (0.0010) -[2023-10-14 14:55:27,572][75950] Updated weights for policy 1, policy_version 36190 (0.0011) -[2023-10-14 14:55:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 74186752. Throughput: 0: 1685.9, 1: 1651.4. Samples: 18555816. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) -[2023-10-14 14:55:28,164][74987] Avg episode reward: [(0, '24.920'), (1, '26.360')] -[2023-10-14 14:55:29,019][75949] Updated weights for policy 0, policy_version 36261 (0.0008) -[2023-10-14 14:55:29,388][75949] Updated weights for policy 0, policy_version 36271 (0.0010) -[2023-10-14 14:55:29,760][75949] Updated weights for policy 0, policy_version 36281 (0.0008) -[2023-10-14 14:55:31,705][75950] Updated weights for policy 1, policy_version 36200 (0.0009) -[2023-10-14 14:55:32,073][75950] Updated weights for policy 1, policy_version 36210 (0.0010) -[2023-10-14 14:55:32,435][75950] Updated weights for policy 1, policy_version 36220 (0.0007) -[2023-10-14 14:55:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 74252288. Throughput: 0: 1678.4, 1: 1675.4. Samples: 18566376. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) -[2023-10-14 14:55:33,164][74987] Avg episode reward: [(0, '23.600'), (1, '30.610')] -[2023-10-14 14:55:33,797][75949] Updated weights for policy 0, policy_version 36291 (0.0009) -[2023-10-14 14:55:34,180][75949] Updated weights for policy 0, policy_version 36301 (0.0010) -[2023-10-14 14:55:34,566][75949] Updated weights for policy 0, policy_version 36311 (0.0007) -[2023-10-14 14:55:36,326][75950] Updated weights for policy 1, policy_version 36230 (0.0009) -[2023-10-14 14:55:36,696][75950] Updated weights for policy 1, policy_version 36240 (0.0008) -[2023-10-14 14:55:37,055][75950] Updated weights for policy 1, policy_version 36250 (0.0008) -[2023-10-14 14:55:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 74317824. Throughput: 0: 1692.3, 1: 1663.9. Samples: 18586592. Policy #0 lag: (min: 16.0, avg: 35.2, max: 48.0) -[2023-10-14 14:55:38,164][74987] Avg episode reward: [(0, '23.870'), (1, '27.400')] -[2023-10-14 14:55:38,528][75949] Updated weights for policy 0, policy_version 36321 (0.0007) -[2023-10-14 14:55:38,899][75949] Updated weights for policy 0, policy_version 36331 (0.0009) -[2023-10-14 14:55:39,275][75949] Updated weights for policy 0, policy_version 36341 (0.0008) -[2023-10-14 14:55:39,646][75949] Updated weights for policy 0, policy_version 36351 (0.0007) -[2023-10-14 14:55:41,135][75950] Updated weights for policy 1, policy_version 36260 (0.0008) -[2023-10-14 14:55:41,507][75950] Updated weights for policy 1, policy_version 36270 (0.0007) -[2023-10-14 14:55:41,862][75950] Updated weights for policy 1, policy_version 36280 (0.0009) -[2023-10-14 14:55:43,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 74383360. Throughput: 0: 1693.8, 1: 1665.3. Samples: 18606508. Policy #0 lag: (min: 16.0, avg: 35.2, max: 48.0) -[2023-10-14 14:55:43,164][74987] Avg episode reward: [(0, '24.470'), (1, '26.380')] -[2023-10-14 14:55:43,818][75949] Updated weights for policy 0, policy_version 36361 (0.0009) -[2023-10-14 14:55:44,197][75949] Updated weights for policy 0, policy_version 36371 (0.0008) -[2023-10-14 14:55:44,572][75949] Updated weights for policy 0, policy_version 36381 (0.0009) -[2023-10-14 14:55:46,093][75950] Updated weights for policy 1, policy_version 36290 (0.0009) -[2023-10-14 14:55:46,462][75950] Updated weights for policy 1, policy_version 36300 (0.0009) -[2023-10-14 14:55:46,828][75950] Updated weights for policy 1, policy_version 36310 (0.0009) -[2023-10-14 14:55:47,198][75950] Updated weights for policy 1, policy_version 36320 (0.0008) -[2023-10-14 14:55:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 74448896. Throughput: 0: 1691.1, 1: 1676.4. Samples: 18616930. Policy #0 lag: (min: 16.0, avg: 35.2, max: 48.0) -[2023-10-14 14:55:48,164][74987] Avg episode reward: [(0, '24.750'), (1, '31.350')] -[2023-10-14 14:55:48,165][75801] Saving new best policy, reward=31.350! -[2023-10-14 14:55:48,656][75949] Updated weights for policy 0, policy_version 36391 (0.0009) -[2023-10-14 14:55:49,029][75949] Updated weights for policy 0, policy_version 36401 (0.0007) -[2023-10-14 14:55:49,400][75949] Updated weights for policy 0, policy_version 36411 (0.0008) -[2023-10-14 14:55:51,515][75950] Updated weights for policy 1, policy_version 36330 (0.0009) -[2023-10-14 14:55:51,879][75950] Updated weights for policy 1, policy_version 36340 (0.0008) -[2023-10-14 14:55:52,252][75950] Updated weights for policy 1, policy_version 36350 (0.0008) -[2023-10-14 14:55:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 74514432. Throughput: 0: 1692.9, 1: 1661.6. Samples: 18636894. Policy #0 lag: (min: 16.0, avg: 35.2, max: 48.0) -[2023-10-14 14:55:53,164][74987] Avg episode reward: [(0, '23.960'), (1, '27.730')] -[2023-10-14 14:55:53,471][75949] Updated weights for policy 0, policy_version 36421 (0.0008) -[2023-10-14 14:55:53,848][75949] Updated weights for policy 0, policy_version 36431 (0.0008) -[2023-10-14 14:55:54,218][75949] Updated weights for policy 0, policy_version 36441 (0.0009) -[2023-10-14 14:55:56,145][75950] Updated weights for policy 1, policy_version 36360 (0.0007) -[2023-10-14 14:55:56,503][75950] Updated weights for policy 1, policy_version 36370 (0.0008) -[2023-10-14 14:55:56,883][75950] Updated weights for policy 1, policy_version 36380 (0.0011) -[2023-10-14 14:55:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 74579968. Throughput: 0: 1694.9, 1: 1675.2. Samples: 18657030. Policy #0 lag: (min: 16.0, avg: 35.2, max: 48.0) -[2023-10-14 14:55:58,165][74987] Avg episode reward: [(0, '24.580'), (1, '27.710')] -[2023-10-14 14:55:58,215][75949] Updated weights for policy 0, policy_version 36451 (0.0008) -[2023-10-14 14:55:58,608][75949] Updated weights for policy 0, policy_version 36461 (0.0007) -[2023-10-14 14:55:58,970][75949] Updated weights for policy 0, policy_version 36471 (0.0007) -[2023-10-14 14:56:01,104][75950] Updated weights for policy 1, policy_version 36390 (0.0008) -[2023-10-14 14:56:01,457][75950] Updated weights for policy 1, policy_version 36400 (0.0010) -[2023-10-14 14:56:01,826][75950] Updated weights for policy 1, policy_version 36410 (0.0009) -[2023-10-14 14:56:03,006][75949] Updated weights for policy 0, policy_version 36481 (0.0011) -[2023-10-14 14:56:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74645504. Throughput: 0: 1691.2, 1: 1679.7. Samples: 18667210. Policy #0 lag: (min: 23.0, avg: 39.3, max: 40.0) -[2023-10-14 14:56:03,164][74987] Avg episode reward: [(0, '23.570'), (1, '29.460')] -[2023-10-14 14:56:03,372][75949] Updated weights for policy 0, policy_version 36491 (0.0009) -[2023-10-14 14:56:03,743][75949] Updated weights for policy 0, policy_version 36501 (0.0009) -[2023-10-14 14:56:04,112][75949] Updated weights for policy 0, policy_version 36511 (0.0008) -[2023-10-14 14:56:05,639][75950] Updated weights for policy 1, policy_version 36420 (0.0009) -[2023-10-14 14:56:06,005][75950] Updated weights for policy 1, policy_version 36430 (0.0009) -[2023-10-14 14:56:06,375][75950] Updated weights for policy 1, policy_version 36440 (0.0010) -[2023-10-14 14:56:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 74711040. Throughput: 0: 1692.9, 1: 1664.9. Samples: 18687022. Policy #0 lag: (min: 23.0, avg: 39.3, max: 40.0) -[2023-10-14 14:56:08,165][74987] Avg episode reward: [(0, '25.090'), (1, '26.590')] -[2023-10-14 14:56:08,279][75949] Updated weights for policy 0, policy_version 36521 (0.0011) -[2023-10-14 14:56:08,658][75949] Updated weights for policy 0, policy_version 36531 (0.0010) -[2023-10-14 14:56:09,036][75949] Updated weights for policy 0, policy_version 36541 (0.0007) -[2023-10-14 14:56:10,291][75950] Updated weights for policy 1, policy_version 36450 (0.0008) -[2023-10-14 14:56:10,664][75950] Updated weights for policy 1, policy_version 36460 (0.0008) -[2023-10-14 14:56:11,023][75950] Updated weights for policy 1, policy_version 36470 (0.0008) -[2023-10-14 14:56:11,396][75950] Updated weights for policy 1, policy_version 36480 (0.0010) -[2023-10-14 14:56:13,119][75949] Updated weights for policy 0, policy_version 36551 (0.0009) -[2023-10-14 14:56:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 74776576. Throughput: 0: 1684.6, 1: 1689.5. Samples: 18707648. Policy #0 lag: (min: 23.0, avg: 39.3, max: 40.0) -[2023-10-14 14:56:13,165][74987] Avg episode reward: [(0, '25.410'), (1, '26.000')] -[2023-10-14 14:56:13,503][75949] Updated weights for policy 0, policy_version 36561 (0.0011) -[2023-10-14 14:56:13,861][75949] Updated weights for policy 0, policy_version 36571 (0.0010) -[2023-10-14 14:56:15,445][75950] Updated weights for policy 1, policy_version 36490 (0.0010) -[2023-10-14 14:56:15,825][75950] Updated weights for policy 1, policy_version 36500 (0.0008) -[2023-10-14 14:56:16,192][75950] Updated weights for policy 1, policy_version 36510 (0.0009) -[2023-10-14 14:56:17,892][75949] Updated weights for policy 0, policy_version 36581 (0.0008) -[2023-10-14 14:56:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 74842112. Throughput: 0: 1678.4, 1: 1674.3. Samples: 18717250. Policy #0 lag: (min: 23.0, avg: 39.3, max: 40.0) -[2023-10-14 14:56:18,165][74987] Avg episode reward: [(0, '22.440'), (1, '27.450')] -[2023-10-14 14:56:18,260][75949] Updated weights for policy 0, policy_version 36591 (0.0009) -[2023-10-14 14:56:18,620][75949] Updated weights for policy 0, policy_version 36601 (0.0011) -[2023-10-14 14:56:20,166][75950] Updated weights for policy 1, policy_version 36520 (0.0010) -[2023-10-14 14:56:20,526][75950] Updated weights for policy 1, policy_version 36530 (0.0011) -[2023-10-14 14:56:20,897][75950] Updated weights for policy 1, policy_version 36540 (0.0010) -[2023-10-14 14:56:22,745][75949] Updated weights for policy 0, policy_version 36611 (0.0007) -[2023-10-14 14:56:23,111][75949] Updated weights for policy 0, policy_version 36621 (0.0007) -[2023-10-14 14:56:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 74907648. Throughput: 0: 1674.9, 1: 1668.7. Samples: 18737058. Policy #0 lag: (min: 23.0, avg: 39.3, max: 40.0) -[2023-10-14 14:56:23,165][74987] Avg episode reward: [(0, '25.210'), (1, '28.850')] -[2023-10-14 14:56:23,482][75949] Updated weights for policy 0, policy_version 36631 (0.0007) -[2023-10-14 14:56:24,978][75950] Updated weights for policy 1, policy_version 36550 (0.0009) -[2023-10-14 14:56:25,347][75950] Updated weights for policy 1, policy_version 36560 (0.0010) -[2023-10-14 14:56:25,712][75950] Updated weights for policy 1, policy_version 36570 (0.0008) -[2023-10-14 14:56:27,570][75949] Updated weights for policy 0, policy_version 36641 (0.0008) -[2023-10-14 14:56:27,939][75949] Updated weights for policy 0, policy_version 36651 (0.0010) -[2023-10-14 14:56:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74973184. Throughput: 0: 1669.4, 1: 1686.0. Samples: 18757502. Policy #0 lag: (min: 2.0, avg: 2.8, max: 17.0) -[2023-10-14 14:56:28,165][74987] Avg episode reward: [(0, '23.210'), (1, '28.810')] -[2023-10-14 14:56:28,319][75949] Updated weights for policy 0, policy_version 36661 (0.0007) -[2023-10-14 14:56:28,695][75949] Updated weights for policy 0, policy_version 36671 (0.0007) -[2023-10-14 14:56:29,931][75950] Updated weights for policy 1, policy_version 36580 (0.0009) -[2023-10-14 14:56:30,301][75950] Updated weights for policy 1, policy_version 36590 (0.0009) -[2023-10-14 14:56:30,669][75950] Updated weights for policy 1, policy_version 36600 (0.0009) -[2023-10-14 14:56:32,800][75949] Updated weights for policy 0, policy_version 36681 (0.0008) -[2023-10-14 14:56:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75038720. Throughput: 0: 1671.2, 1: 1666.3. Samples: 18767118. Policy #0 lag: (min: 2.0, avg: 2.8, max: 17.0) -[2023-10-14 14:56:33,164][74987] Avg episode reward: [(0, '26.950'), (1, '30.260')] -[2023-10-14 14:56:33,169][75949] Updated weights for policy 0, policy_version 36691 (0.0008) -[2023-10-14 14:56:33,542][75949] Updated weights for policy 0, policy_version 36701 (0.0008) -[2023-10-14 14:56:34,757][75950] Updated weights for policy 1, policy_version 36610 (0.0010) -[2023-10-14 14:56:35,137][75950] Updated weights for policy 1, policy_version 36620 (0.0009) -[2023-10-14 14:56:35,507][75950] Updated weights for policy 1, policy_version 36630 (0.0009) -[2023-10-14 14:56:35,863][75950] Updated weights for policy 1, policy_version 36640 (0.0009) -[2023-10-14 14:56:37,794][75949] Updated weights for policy 0, policy_version 36711 (0.0010) -[2023-10-14 14:56:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75104256. Throughput: 0: 1666.8, 1: 1670.6. Samples: 18787078. Policy #0 lag: (min: 2.0, avg: 2.8, max: 17.0) -[2023-10-14 14:56:38,164][74987] Avg episode reward: [(0, '23.980'), (1, '32.160')] -[2023-10-14 14:56:38,166][75801] Saving new best policy, reward=32.160! -[2023-10-14 14:56:38,172][75949] Updated weights for policy 0, policy_version 36721 (0.0008) -[2023-10-14 14:56:38,547][75949] Updated weights for policy 0, policy_version 36731 (0.0009) -[2023-10-14 14:56:40,206][75950] Updated weights for policy 1, policy_version 36650 (0.0009) -[2023-10-14 14:56:40,572][75950] Updated weights for policy 1, policy_version 36660 (0.0008) -[2023-10-14 14:56:40,934][75950] Updated weights for policy 1, policy_version 36670 (0.0007) -[2023-10-14 14:56:42,470][75949] Updated weights for policy 0, policy_version 36741 (0.0008) -[2023-10-14 14:56:42,844][75949] Updated weights for policy 0, policy_version 36751 (0.0010) -[2023-10-14 14:56:43,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75169792. Throughput: 0: 1656.6, 1: 1677.3. Samples: 18807052. Policy #0 lag: (min: 2.0, avg: 2.8, max: 17.0) -[2023-10-14 14:56:43,164][74987] Avg episode reward: [(0, '26.770'), (1, '29.280')] -[2023-10-14 14:56:43,171][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000036672_37552128.pth... -[2023-10-14 14:56:43,206][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000035104_35946496.pth -[2023-10-14 14:56:43,229][75949] Updated weights for policy 0, policy_version 36761 (0.0009) -[2023-10-14 14:56:43,488][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000036768_37650432.pth... -[2023-10-14 14:56:43,527][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000035200_36044800.pth -[2023-10-14 14:56:45,060][75950] Updated weights for policy 1, policy_version 36680 (0.0010) -[2023-10-14 14:56:45,430][75950] Updated weights for policy 1, policy_version 36690 (0.0010) -[2023-10-14 14:56:45,806][75950] Updated weights for policy 1, policy_version 36700 (0.0010) -[2023-10-14 14:56:47,410][75949] Updated weights for policy 0, policy_version 36771 (0.0010) -[2023-10-14 14:56:47,804][75949] Updated weights for policy 0, policy_version 36781 (0.0007) -[2023-10-14 14:56:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75235328. Throughput: 0: 1671.6, 1: 1656.8. Samples: 18816990. Policy #0 lag: (min: 2.0, avg: 2.8, max: 17.0) -[2023-10-14 14:56:48,164][74987] Avg episode reward: [(0, '23.660'), (1, '28.910')] -[2023-10-14 14:56:48,175][75949] Updated weights for policy 0, policy_version 36791 (0.0008) -[2023-10-14 14:56:49,791][75950] Updated weights for policy 1, policy_version 36710 (0.0010) -[2023-10-14 14:56:50,155][75950] Updated weights for policy 1, policy_version 36720 (0.0010) -[2023-10-14 14:56:50,519][75950] Updated weights for policy 1, policy_version 36730 (0.0010) -[2023-10-14 14:56:52,374][75949] Updated weights for policy 0, policy_version 36801 (0.0008) -[2023-10-14 14:56:52,746][75949] Updated weights for policy 0, policy_version 36811 (0.0008) -[2023-10-14 14:56:53,106][75949] Updated weights for policy 0, policy_version 36821 (0.0009) -[2023-10-14 14:56:53,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75300864. Throughput: 0: 1663.7, 1: 1670.9. Samples: 18837082. Policy #0 lag: (min: 20.0, avg: 32.2, max: 52.0) -[2023-10-14 14:56:53,165][74987] Avg episode reward: [(0, '24.800'), (1, '29.830')] -[2023-10-14 14:56:53,484][75949] Updated weights for policy 0, policy_version 36831 (0.0009) -[2023-10-14 14:56:54,544][75950] Updated weights for policy 1, policy_version 36740 (0.0009) -[2023-10-14 14:56:54,907][75950] Updated weights for policy 1, policy_version 36750 (0.0010) -[2023-10-14 14:56:55,275][75950] Updated weights for policy 1, policy_version 36760 (0.0008) -[2023-10-14 14:56:57,571][75949] Updated weights for policy 0, policy_version 36841 (0.0007) -[2023-10-14 14:56:57,943][75949] Updated weights for policy 0, policy_version 36851 (0.0010) -[2023-10-14 14:56:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75366400. Throughput: 0: 1653.6, 1: 1670.9. Samples: 18857250. Policy #0 lag: (min: 20.0, avg: 32.2, max: 52.0) -[2023-10-14 14:56:58,164][74987] Avg episode reward: [(0, '24.540'), (1, '27.210')] -[2023-10-14 14:56:58,303][75949] Updated weights for policy 0, policy_version 36861 (0.0008) -[2023-10-14 14:56:59,307][75950] Updated weights for policy 1, policy_version 36770 (0.0009) -[2023-10-14 14:56:59,675][75950] Updated weights for policy 1, policy_version 36780 (0.0009) -[2023-10-14 14:57:00,043][75950] Updated weights for policy 1, policy_version 36790 (0.0008) -[2023-10-14 14:57:00,412][75950] Updated weights for policy 1, policy_version 36800 (0.0010) -[2023-10-14 14:57:02,103][75949] Updated weights for policy 0, policy_version 36871 (0.0008) -[2023-10-14 14:57:02,473][75949] Updated weights for policy 0, policy_version 36881 (0.0010) -[2023-10-14 14:57:02,837][75949] Updated weights for policy 0, policy_version 36891 (0.0008) -[2023-10-14 14:57:03,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 75464704. Throughput: 0: 1670.6, 1: 1658.7. Samples: 18867068. Policy #0 lag: (min: 20.0, avg: 32.2, max: 52.0) -[2023-10-14 14:57:03,165][74987] Avg episode reward: [(0, '23.470'), (1, '28.560')] -[2023-10-14 14:57:04,777][75950] Updated weights for policy 1, policy_version 36810 (0.0009) -[2023-10-14 14:57:05,131][75950] Updated weights for policy 1, policy_version 36820 (0.0010) -[2023-10-14 14:57:05,500][75950] Updated weights for policy 1, policy_version 36830 (0.0010) -[2023-10-14 14:57:06,915][75949] Updated weights for policy 0, policy_version 36901 (0.0007) -[2023-10-14 14:57:07,279][75949] Updated weights for policy 0, policy_version 36911 (0.0008) -[2023-10-14 14:57:07,648][75949] Updated weights for policy 0, policy_version 36921 (0.0008) -[2023-10-14 14:57:08,163][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 75530240. Throughput: 0: 1675.0, 1: 1671.1. Samples: 18887630. Policy #0 lag: (min: 20.0, avg: 32.2, max: 52.0) -[2023-10-14 14:57:08,164][74987] Avg episode reward: [(0, '26.500'), (1, '30.240')] -[2023-10-14 14:57:09,569][75950] Updated weights for policy 1, policy_version 36840 (0.0010) -[2023-10-14 14:57:09,938][75950] Updated weights for policy 1, policy_version 36850 (0.0010) -[2023-10-14 14:57:10,311][75950] Updated weights for policy 1, policy_version 36860 (0.0009) -[2023-10-14 14:57:11,813][75949] Updated weights for policy 0, policy_version 36931 (0.0009) -[2023-10-14 14:57:12,185][75949] Updated weights for policy 0, policy_version 36941 (0.0007) -[2023-10-14 14:57:12,556][75949] Updated weights for policy 0, policy_version 36951 (0.0009) -[2023-10-14 14:57:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 75595776. Throughput: 0: 1658.2, 1: 1671.2. Samples: 18907324. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-14 14:57:13,164][74987] Avg episode reward: [(0, '23.610'), (1, '28.610')] -[2023-10-14 14:57:14,244][75950] Updated weights for policy 1, policy_version 36870 (0.0010) -[2023-10-14 14:57:14,611][75950] Updated weights for policy 1, policy_version 36880 (0.0008) -[2023-10-14 14:57:14,969][75950] Updated weights for policy 1, policy_version 36890 (0.0008) -[2023-10-14 14:57:16,584][75949] Updated weights for policy 0, policy_version 36961 (0.0010) -[2023-10-14 14:57:16,959][75949] Updated weights for policy 0, policy_version 36971 (0.0007) -[2023-10-14 14:57:17,317][75949] Updated weights for policy 0, policy_version 36981 (0.0008) -[2023-10-14 14:57:17,694][75949] Updated weights for policy 0, policy_version 36991 (0.0008) -[2023-10-14 14:57:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 75661312. Throughput: 0: 1677.9, 1: 1664.2. Samples: 18917510. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-14 14:57:18,164][74987] Avg episode reward: [(0, '26.780'), (1, '29.420')] -[2023-10-14 14:57:19,037][75950] Updated weights for policy 1, policy_version 36900 (0.0009) -[2023-10-14 14:57:19,400][75950] Updated weights for policy 1, policy_version 36910 (0.0009) -[2023-10-14 14:57:19,772][75950] Updated weights for policy 1, policy_version 36920 (0.0008) -[2023-10-14 14:57:21,634][75949] Updated weights for policy 0, policy_version 37001 (0.0008) -[2023-10-14 14:57:22,004][75949] Updated weights for policy 0, policy_version 37011 (0.0009) -[2023-10-14 14:57:22,369][75949] Updated weights for policy 0, policy_version 37021 (0.0009) -[2023-10-14 14:57:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 75726848. Throughput: 0: 1678.4, 1: 1672.2. Samples: 18937858. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-14 14:57:23,165][74987] Avg episode reward: [(0, '24.310'), (1, '28.590')] -[2023-10-14 14:57:23,693][75950] Updated weights for policy 1, policy_version 36930 (0.0008) -[2023-10-14 14:57:24,053][75950] Updated weights for policy 1, policy_version 36940 (0.0010) -[2023-10-14 14:57:24,414][75950] Updated weights for policy 1, policy_version 36950 (0.0011) -[2023-10-14 14:57:24,785][75950] Updated weights for policy 1, policy_version 36960 (0.0010) -[2023-10-14 14:57:26,646][75949] Updated weights for policy 0, policy_version 37031 (0.0008) -[2023-10-14 14:57:27,019][75949] Updated weights for policy 0, policy_version 37041 (0.0009) -[2023-10-14 14:57:27,382][75949] Updated weights for policy 0, policy_version 37051 (0.0008) -[2023-10-14 14:57:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 75792384. Throughput: 0: 1664.4, 1: 1685.5. Samples: 18957794. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-14 14:57:28,164][74987] Avg episode reward: [(0, '26.550'), (1, '26.990')] -[2023-10-14 14:57:29,042][75950] Updated weights for policy 1, policy_version 36970 (0.0010) -[2023-10-14 14:57:29,409][75950] Updated weights for policy 1, policy_version 36980 (0.0011) -[2023-10-14 14:57:29,776][75950] Updated weights for policy 1, policy_version 36990 (0.0011) -[2023-10-14 14:57:31,472][75949] Updated weights for policy 0, policy_version 37061 (0.0008) -[2023-10-14 14:57:31,846][75949] Updated weights for policy 0, policy_version 37071 (0.0010) -[2023-10-14 14:57:32,219][75949] Updated weights for policy 0, policy_version 37081 (0.0009) -[2023-10-14 14:57:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 75857920. Throughput: 0: 1682.2, 1: 1671.7. Samples: 18967916. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-14 14:57:33,165][74987] Avg episode reward: [(0, '23.000'), (1, '27.590')] -[2023-10-14 14:57:33,863][75950] Updated weights for policy 1, policy_version 37000 (0.0009) -[2023-10-14 14:57:34,229][75950] Updated weights for policy 1, policy_version 37010 (0.0009) -[2023-10-14 14:57:34,595][75950] Updated weights for policy 1, policy_version 37020 (0.0009) -[2023-10-14 14:57:36,177][75949] Updated weights for policy 0, policy_version 37091 (0.0008) -[2023-10-14 14:57:36,555][75949] Updated weights for policy 0, policy_version 37101 (0.0009) -[2023-10-14 14:57:36,940][75949] Updated weights for policy 0, policy_version 37111 (0.0010) -[2023-10-14 14:57:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 75923456. Throughput: 0: 1676.9, 1: 1683.4. Samples: 18988294. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) -[2023-10-14 14:57:38,164][74987] Avg episode reward: [(0, '25.520'), (1, '27.580')] -[2023-10-14 14:57:38,813][75950] Updated weights for policy 1, policy_version 37030 (0.0008) -[2023-10-14 14:57:39,187][75950] Updated weights for policy 1, policy_version 37040 (0.0008) -[2023-10-14 14:57:39,554][75950] Updated weights for policy 1, policy_version 37050 (0.0009) -[2023-10-14 14:57:40,852][75949] Updated weights for policy 0, policy_version 37121 (0.0008) -[2023-10-14 14:57:41,220][75949] Updated weights for policy 0, policy_version 37131 (0.0010) -[2023-10-14 14:57:41,593][75949] Updated weights for policy 0, policy_version 37141 (0.0009) -[2023-10-14 14:57:41,973][75949] Updated weights for policy 0, policy_version 37151 (0.0008) -[2023-10-14 14:57:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 75988992. Throughput: 0: 1675.4, 1: 1682.7. Samples: 19008364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) -[2023-10-14 14:57:43,165][74987] Avg episode reward: [(0, '23.690'), (1, '28.360')] -[2023-10-14 14:57:43,684][75950] Updated weights for policy 1, policy_version 37060 (0.0010) -[2023-10-14 14:57:44,046][75950] Updated weights for policy 1, policy_version 37070 (0.0011) -[2023-10-14 14:57:44,426][75950] Updated weights for policy 1, policy_version 37080 (0.0009) -[2023-10-14 14:57:45,956][75949] Updated weights for policy 0, policy_version 37161 (0.0008) -[2023-10-14 14:57:46,332][75949] Updated weights for policy 0, policy_version 37171 (0.0010) -[2023-10-14 14:57:46,702][75949] Updated weights for policy 0, policy_version 37181 (0.0008) -[2023-10-14 14:57:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 76054528. Throughput: 0: 1691.2, 1: 1678.7. Samples: 19018710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) -[2023-10-14 14:57:48,164][74987] Avg episode reward: [(0, '24.990'), (1, '30.420')] -[2023-10-14 14:57:48,638][75950] Updated weights for policy 1, policy_version 37090 (0.0009) -[2023-10-14 14:57:49,012][75950] Updated weights for policy 1, policy_version 37100 (0.0008) -[2023-10-14 14:57:49,382][75950] Updated weights for policy 1, policy_version 37110 (0.0009) -[2023-10-14 14:57:49,750][75950] Updated weights for policy 1, policy_version 37120 (0.0010) -[2023-10-14 14:57:50,766][75949] Updated weights for policy 0, policy_version 37191 (0.0008) -[2023-10-14 14:57:51,126][75949] Updated weights for policy 0, policy_version 37201 (0.0008) -[2023-10-14 14:57:51,500][75949] Updated weights for policy 0, policy_version 37211 (0.0010) -[2023-10-14 14:57:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 76120064. Throughput: 0: 1664.8, 1: 1682.1. Samples: 19038242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) -[2023-10-14 14:57:53,165][74987] Avg episode reward: [(0, '25.690'), (1, '29.530')] -[2023-10-14 14:57:53,498][75950] Updated weights for policy 1, policy_version 37130 (0.0008) -[2023-10-14 14:57:53,862][75950] Updated weights for policy 1, policy_version 37140 (0.0011) -[2023-10-14 14:57:54,234][75950] Updated weights for policy 1, policy_version 37150 (0.0008) -[2023-10-14 14:57:55,527][75949] Updated weights for policy 0, policy_version 37221 (0.0011) -[2023-10-14 14:57:55,900][75949] Updated weights for policy 0, policy_version 37231 (0.0007) -[2023-10-14 14:57:56,262][75949] Updated weights for policy 0, policy_version 37241 (0.0009) -[2023-10-14 14:57:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 76185600. Throughput: 0: 1686.6, 1: 1687.1. Samples: 19059142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) -[2023-10-14 14:57:58,164][74987] Avg episode reward: [(0, '25.230'), (1, '28.840')] -[2023-10-14 14:57:58,184][75950] Updated weights for policy 1, policy_version 37160 (0.0010) -[2023-10-14 14:57:58,548][75950] Updated weights for policy 1, policy_version 37170 (0.0010) -[2023-10-14 14:57:58,913][75950] Updated weights for policy 1, policy_version 37180 (0.0011) -[2023-10-14 14:58:00,251][75949] Updated weights for policy 0, policy_version 37251 (0.0007) -[2023-10-14 14:58:00,620][75949] Updated weights for policy 0, policy_version 37261 (0.0007) -[2023-10-14 14:58:00,992][75949] Updated weights for policy 0, policy_version 37271 (0.0010) -[2023-10-14 14:58:03,098][75950] Updated weights for policy 1, policy_version 37190 (0.0011) -[2023-10-14 14:58:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 76251136. Throughput: 0: 1686.4, 1: 1683.8. Samples: 19069170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) -[2023-10-14 14:58:03,164][74987] Avg episode reward: [(0, '24.970'), (1, '29.490')] -[2023-10-14 14:58:03,468][75950] Updated weights for policy 1, policy_version 37200 (0.0011) -[2023-10-14 14:58:03,836][75950] Updated weights for policy 1, policy_version 37210 (0.0009) -[2023-10-14 14:58:05,119][75949] Updated weights for policy 0, policy_version 37281 (0.0008) -[2023-10-14 14:58:05,479][75949] Updated weights for policy 0, policy_version 37291 (0.0007) -[2023-10-14 14:58:05,849][75949] Updated weights for policy 0, policy_version 37301 (0.0007) -[2023-10-14 14:58:06,213][75949] Updated weights for policy 0, policy_version 37311 (0.0007) -[2023-10-14 14:58:07,746][75950] Updated weights for policy 1, policy_version 37220 (0.0009) -[2023-10-14 14:58:08,113][75950] Updated weights for policy 1, policy_version 37230 (0.0007) -[2023-10-14 14:58:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 76316672. Throughput: 0: 1675.1, 1: 1691.9. Samples: 19089372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) -[2023-10-14 14:58:08,164][74987] Avg episode reward: [(0, '24.470'), (1, '30.680')] -[2023-10-14 14:58:08,484][75950] Updated weights for policy 1, policy_version 37240 (0.0007) -[2023-10-14 14:58:10,196][75949] Updated weights for policy 0, policy_version 37321 (0.0010) -[2023-10-14 14:58:10,568][75949] Updated weights for policy 0, policy_version 37331 (0.0009) -[2023-10-14 14:58:10,942][75949] Updated weights for policy 0, policy_version 37341 (0.0009) -[2023-10-14 14:58:12,540][75950] Updated weights for policy 1, policy_version 37250 (0.0007) -[2023-10-14 14:58:12,909][75950] Updated weights for policy 1, policy_version 37260 (0.0009) -[2023-10-14 14:58:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 76382208. Throughput: 0: 1700.9, 1: 1685.0. Samples: 19110160. Policy #0 lag: (min: 14.0, avg: 14.1, max: 20.0) -[2023-10-14 14:58:13,164][74987] Avg episode reward: [(0, '24.660'), (1, '27.890')] -[2023-10-14 14:58:13,280][75950] Updated weights for policy 1, policy_version 37270 (0.0011) -[2023-10-14 14:58:13,649][75950] Updated weights for policy 1, policy_version 37280 (0.0010) -[2023-10-14 14:58:15,027][75949] Updated weights for policy 0, policy_version 37351 (0.0009) -[2023-10-14 14:58:15,387][75949] Updated weights for policy 0, policy_version 37361 (0.0009) -[2023-10-14 14:58:15,765][75949] Updated weights for policy 0, policy_version 37371 (0.0007) -[2023-10-14 14:58:17,833][75950] Updated weights for policy 1, policy_version 37290 (0.0009) -[2023-10-14 14:58:18,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 76447744. Throughput: 0: 1680.0, 1: 1695.0. Samples: 19119788. Policy #0 lag: (min: 14.0, avg: 14.1, max: 20.0) -[2023-10-14 14:58:18,164][74987] Avg episode reward: [(0, '24.000'), (1, '29.050')] -[2023-10-14 14:58:18,213][75950] Updated weights for policy 1, policy_version 37300 (0.0007) -[2023-10-14 14:58:18,582][75950] Updated weights for policy 1, policy_version 37310 (0.0010) -[2023-10-14 14:58:19,762][75949] Updated weights for policy 0, policy_version 37381 (0.0008) -[2023-10-14 14:58:20,132][75949] Updated weights for policy 0, policy_version 37391 (0.0007) -[2023-10-14 14:58:20,517][75949] Updated weights for policy 0, policy_version 37401 (0.0007) -[2023-10-14 14:58:22,615][75950] Updated weights for policy 1, policy_version 37320 (0.0009) -[2023-10-14 14:58:22,990][75950] Updated weights for policy 1, policy_version 37330 (0.0009) -[2023-10-14 14:58:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 76513280. Throughput: 0: 1688.4, 1: 1690.7. Samples: 19140354. Policy #0 lag: (min: 14.0, avg: 14.1, max: 20.0) -[2023-10-14 14:58:23,165][74987] Avg episode reward: [(0, '25.780'), (1, '29.520')] -[2023-10-14 14:58:23,356][75950] Updated weights for policy 1, policy_version 37340 (0.0008) -[2023-10-14 14:58:24,553][75949] Updated weights for policy 0, policy_version 37411 (0.0010) -[2023-10-14 14:58:24,939][75949] Updated weights for policy 0, policy_version 37421 (0.0009) -[2023-10-14 14:58:25,306][75949] Updated weights for policy 0, policy_version 37431 (0.0008) -[2023-10-14 14:58:27,372][75950] Updated weights for policy 1, policy_version 37350 (0.0011) -[2023-10-14 14:58:27,747][75950] Updated weights for policy 1, policy_version 37360 (0.0011) -[2023-10-14 14:58:28,113][75950] Updated weights for policy 1, policy_version 37370 (0.0011) -[2023-10-14 14:58:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 76578816. Throughput: 0: 1697.4, 1: 1677.9. Samples: 19160252. Policy #0 lag: (min: 14.0, avg: 14.1, max: 20.0) -[2023-10-14 14:58:28,164][74987] Avg episode reward: [(0, '22.690'), (1, '26.270')] -[2023-10-14 14:58:29,464][75949] Updated weights for policy 0, policy_version 37441 (0.0007) -[2023-10-14 14:58:29,842][75949] Updated weights for policy 0, policy_version 37451 (0.0009) -[2023-10-14 14:58:30,210][75949] Updated weights for policy 0, policy_version 37461 (0.0010) -[2023-10-14 14:58:30,583][75949] Updated weights for policy 0, policy_version 37471 (0.0008) -[2023-10-14 14:58:32,262][75950] Updated weights for policy 1, policy_version 37380 (0.0010) -[2023-10-14 14:58:32,623][75950] Updated weights for policy 1, policy_version 37390 (0.0009) -[2023-10-14 14:58:32,999][75950] Updated weights for policy 1, policy_version 37400 (0.0009) -[2023-10-14 14:58:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 76644352. Throughput: 0: 1666.8, 1: 1686.9. Samples: 19169626. Policy #0 lag: (min: 14.0, avg: 14.1, max: 20.0) -[2023-10-14 14:58:33,164][74987] Avg episode reward: [(0, '26.780'), (1, '29.090')] -[2023-10-14 14:58:34,714][75949] Updated weights for policy 0, policy_version 37481 (0.0008) -[2023-10-14 14:58:35,082][75949] Updated weights for policy 0, policy_version 37491 (0.0009) -[2023-10-14 14:58:35,448][75949] Updated weights for policy 0, policy_version 37501 (0.0008) -[2023-10-14 14:58:37,040][75950] Updated weights for policy 1, policy_version 37410 (0.0008) -[2023-10-14 14:58:37,404][75950] Updated weights for policy 1, policy_version 37420 (0.0008) -[2023-10-14 14:58:37,766][75950] Updated weights for policy 1, policy_version 37430 (0.0008) -[2023-10-14 14:58:38,145][75950] Updated weights for policy 1, policy_version 37440 (0.0009) -[2023-10-14 14:58:38,164][74987] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 76742656. Throughput: 0: 1691.2, 1: 1687.0. Samples: 19190262. Policy #0 lag: (min: 8.0, avg: 30.4, max: 40.0) -[2023-10-14 14:58:38,165][74987] Avg episode reward: [(0, '23.610'), (1, '30.590')] -[2023-10-14 14:58:39,330][75949] Updated weights for policy 0, policy_version 37511 (0.0008) -[2023-10-14 14:58:39,708][75949] Updated weights for policy 0, policy_version 37521 (0.0009) -[2023-10-14 14:58:40,075][75949] Updated weights for policy 0, policy_version 37531 (0.0007) -[2023-10-14 14:58:42,173][75950] Updated weights for policy 1, policy_version 37450 (0.0008) -[2023-10-14 14:58:42,546][75950] Updated weights for policy 1, policy_version 37460 (0.0008) -[2023-10-14 14:58:42,908][75950] Updated weights for policy 1, policy_version 37470 (0.0009) -[2023-10-14 14:58:43,163][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 76808192. Throughput: 0: 1694.6, 1: 1661.4. Samples: 19210162. Policy #0 lag: (min: 8.0, avg: 30.4, max: 40.0) -[2023-10-14 14:58:43,164][74987] Avg episode reward: [(0, '25.450'), (1, '27.890')] -[2023-10-14 14:58:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000037536_38436864.pth... -[2023-10-14 14:58:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000037472_38371328.pth... -[2023-10-14 14:58:43,205][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000035968_36831232.pth -[2023-10-14 14:58:43,211][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000035904_36765696.pth -[2023-10-14 14:58:44,179][75949] Updated weights for policy 0, policy_version 37541 (0.0008) -[2023-10-14 14:58:44,558][75949] Updated weights for policy 0, policy_version 37551 (0.0009) -[2023-10-14 14:58:44,937][75949] Updated weights for policy 0, policy_version 37561 (0.0008) -[2023-10-14 14:58:46,907][75950] Updated weights for policy 1, policy_version 37480 (0.0009) -[2023-10-14 14:58:47,285][75950] Updated weights for policy 1, policy_version 37490 (0.0008) -[2023-10-14 14:58:47,656][75950] Updated weights for policy 1, policy_version 37500 (0.0007) -[2023-10-14 14:58:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 76873728. Throughput: 0: 1673.7, 1: 1682.4. Samples: 19220196. Policy #0 lag: (min: 8.0, avg: 30.4, max: 40.0) -[2023-10-14 14:58:48,165][74987] Avg episode reward: [(0, '25.130'), (1, '27.520')] -[2023-10-14 14:58:49,230][75949] Updated weights for policy 0, policy_version 37571 (0.0009) -[2023-10-14 14:58:49,603][75949] Updated weights for policy 0, policy_version 37581 (0.0007) -[2023-10-14 14:58:49,972][75949] Updated weights for policy 0, policy_version 37591 (0.0007) -[2023-10-14 14:58:51,887][75950] Updated weights for policy 1, policy_version 37510 (0.0008) -[2023-10-14 14:58:52,254][75950] Updated weights for policy 1, policy_version 37520 (0.0008) -[2023-10-14 14:58:52,615][75950] Updated weights for policy 1, policy_version 37530 (0.0007) -[2023-10-14 14:58:53,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 76939264. Throughput: 0: 1688.7, 1: 1674.8. Samples: 19240726. Policy #0 lag: (min: 8.0, avg: 30.4, max: 40.0) -[2023-10-14 14:58:53,164][74987] Avg episode reward: [(0, '25.510'), (1, '29.970')] -[2023-10-14 14:58:53,930][75949] Updated weights for policy 0, policy_version 37601 (0.0007) -[2023-10-14 14:58:54,303][75949] Updated weights for policy 0, policy_version 37611 (0.0007) -[2023-10-14 14:58:54,683][75949] Updated weights for policy 0, policy_version 37621 (0.0008) -[2023-10-14 14:58:55,043][75949] Updated weights for policy 0, policy_version 37631 (0.0010) -[2023-10-14 14:58:56,917][75950] Updated weights for policy 1, policy_version 37540 (0.0009) -[2023-10-14 14:58:57,292][75950] Updated weights for policy 1, policy_version 37550 (0.0008) -[2023-10-14 14:58:57,654][75950] Updated weights for policy 1, policy_version 37560 (0.0009) -[2023-10-14 14:58:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 77004800. Throughput: 0: 1690.8, 1: 1651.4. Samples: 19260560. Policy #0 lag: (min: 8.0, avg: 30.4, max: 40.0) -[2023-10-14 14:58:58,164][74987] Avg episode reward: [(0, '25.000'), (1, '28.070')] -[2023-10-14 14:58:58,914][75949] Updated weights for policy 0, policy_version 37641 (0.0008) -[2023-10-14 14:58:59,293][75949] Updated weights for policy 0, policy_version 37651 (0.0008) -[2023-10-14 14:58:59,653][75949] Updated weights for policy 0, policy_version 37661 (0.0008) -[2023-10-14 14:59:01,728][75950] Updated weights for policy 1, policy_version 37570 (0.0009) -[2023-10-14 14:59:02,102][75950] Updated weights for policy 1, policy_version 37580 (0.0007) -[2023-10-14 14:59:02,458][75950] Updated weights for policy 1, policy_version 37590 (0.0010) -[2023-10-14 14:59:02,823][75950] Updated weights for policy 1, policy_version 37600 (0.0009) -[2023-10-14 14:59:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 77070336. Throughput: 0: 1682.9, 1: 1666.5. Samples: 19270512. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) -[2023-10-14 14:59:03,164][74987] Avg episode reward: [(0, '24.340'), (1, '26.480')] -[2023-10-14 14:59:03,634][75949] Updated weights for policy 0, policy_version 37671 (0.0010) -[2023-10-14 14:59:04,007][75949] Updated weights for policy 0, policy_version 37681 (0.0007) -[2023-10-14 14:59:04,386][75949] Updated weights for policy 0, policy_version 37691 (0.0008) -[2023-10-14 14:59:06,870][75950] Updated weights for policy 1, policy_version 37610 (0.0007) -[2023-10-14 14:59:07,236][75950] Updated weights for policy 1, policy_version 37620 (0.0007) -[2023-10-14 14:59:07,606][75950] Updated weights for policy 1, policy_version 37630 (0.0009) -[2023-10-14 14:59:08,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 77135872. Throughput: 0: 1683.1, 1: 1663.4. Samples: 19290948. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) -[2023-10-14 14:59:08,164][74987] Avg episode reward: [(0, '26.600'), (1, '31.210')] -[2023-10-14 14:59:08,520][75949] Updated weights for policy 0, policy_version 37701 (0.0010) -[2023-10-14 14:59:08,884][75949] Updated weights for policy 0, policy_version 37711 (0.0009) -[2023-10-14 14:59:09,254][75949] Updated weights for policy 0, policy_version 37721 (0.0009) -[2023-10-14 14:59:11,688][75950] Updated weights for policy 1, policy_version 37640 (0.0008) -[2023-10-14 14:59:12,049][75950] Updated weights for policy 1, policy_version 37650 (0.0009) -[2023-10-14 14:59:12,418][75950] Updated weights for policy 1, policy_version 37660 (0.0008) -[2023-10-14 14:59:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 77201408. Throughput: 0: 1690.0, 1: 1652.7. Samples: 19310674. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) -[2023-10-14 14:59:13,164][74987] Avg episode reward: [(0, '23.740'), (1, '29.310')] -[2023-10-14 14:59:13,412][75949] Updated weights for policy 0, policy_version 37731 (0.0007) -[2023-10-14 14:59:13,816][75949] Updated weights for policy 0, policy_version 37741 (0.0010) -[2023-10-14 14:59:14,189][75949] Updated weights for policy 0, policy_version 37751 (0.0009) -[2023-10-14 14:59:16,327][75950] Updated weights for policy 1, policy_version 37670 (0.0008) -[2023-10-14 14:59:16,684][75950] Updated weights for policy 1, policy_version 37680 (0.0010) -[2023-10-14 14:59:17,054][75950] Updated weights for policy 1, policy_version 37690 (0.0009) -[2023-10-14 14:59:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 77266944. Throughput: 0: 1688.4, 1: 1680.7. Samples: 19321238. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) -[2023-10-14 14:59:18,165][74987] Avg episode reward: [(0, '26.270'), (1, '26.460')] -[2023-10-14 14:59:18,396][75949] Updated weights for policy 0, policy_version 37761 (0.0007) -[2023-10-14 14:59:18,776][75949] Updated weights for policy 0, policy_version 37771 (0.0007) -[2023-10-14 14:59:19,145][75949] Updated weights for policy 0, policy_version 37781 (0.0009) -[2023-10-14 14:59:19,516][75949] Updated weights for policy 0, policy_version 37791 (0.0010) -[2023-10-14 14:59:21,230][75950] Updated weights for policy 1, policy_version 37700 (0.0009) -[2023-10-14 14:59:21,605][75950] Updated weights for policy 1, policy_version 37710 (0.0010) -[2023-10-14 14:59:21,965][75950] Updated weights for policy 1, policy_version 37720 (0.0008) -[2023-10-14 14:59:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 77332480. Throughput: 0: 1687.4, 1: 1668.9. Samples: 19341294. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) -[2023-10-14 14:59:23,164][74987] Avg episode reward: [(0, '22.000'), (1, '29.480')] -[2023-10-14 14:59:23,396][75949] Updated weights for policy 0, policy_version 37801 (0.0008) -[2023-10-14 14:59:23,758][75949] Updated weights for policy 0, policy_version 37811 (0.0008) -[2023-10-14 14:59:24,137][75949] Updated weights for policy 0, policy_version 37821 (0.0009) -[2023-10-14 14:59:25,993][75950] Updated weights for policy 1, policy_version 37730 (0.0007) -[2023-10-14 14:59:26,370][75950] Updated weights for policy 1, policy_version 37740 (0.0009) -[2023-10-14 14:59:26,730][75950] Updated weights for policy 1, policy_version 37750 (0.0009) -[2023-10-14 14:59:27,092][75950] Updated weights for policy 1, policy_version 37760 (0.0011) -[2023-10-14 14:59:28,131][75949] Updated weights for policy 0, policy_version 37831 (0.0009) -[2023-10-14 14:59:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 77398016. Throughput: 0: 1686.5, 1: 1674.7. Samples: 19361414. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) -[2023-10-14 14:59:28,164][74987] Avg episode reward: [(0, '25.550'), (1, '27.960')] -[2023-10-14 14:59:28,510][75949] Updated weights for policy 0, policy_version 37841 (0.0009) -[2023-10-14 14:59:28,874][75949] Updated weights for policy 0, policy_version 37851 (0.0008) -[2023-10-14 14:59:31,215][75950] Updated weights for policy 1, policy_version 37770 (0.0009) -[2023-10-14 14:59:31,576][75950] Updated weights for policy 1, policy_version 37780 (0.0010) -[2023-10-14 14:59:31,946][75950] Updated weights for policy 1, policy_version 37790 (0.0007) -[2023-10-14 14:59:32,833][75949] Updated weights for policy 0, policy_version 37861 (0.0008) -[2023-10-14 14:59:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 77463552. Throughput: 0: 1688.1, 1: 1680.8. Samples: 19371798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:59:33,164][74987] Avg episode reward: [(0, '24.740'), (1, '27.900')] -[2023-10-14 14:59:33,201][75949] Updated weights for policy 0, policy_version 37871 (0.0009) -[2023-10-14 14:59:33,572][75949] Updated weights for policy 0, policy_version 37881 (0.0008) -[2023-10-14 14:59:35,976][75950] Updated weights for policy 1, policy_version 37800 (0.0009) -[2023-10-14 14:59:36,347][75950] Updated weights for policy 1, policy_version 37810 (0.0009) -[2023-10-14 14:59:36,719][75950] Updated weights for policy 1, policy_version 37820 (0.0009) -[2023-10-14 14:59:37,592][75949] Updated weights for policy 0, policy_version 37891 (0.0009) -[2023-10-14 14:59:37,964][75949] Updated weights for policy 0, policy_version 37901 (0.0008) -[2023-10-14 14:59:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 77529088. Throughput: 0: 1691.3, 1: 1658.7. Samples: 19391478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:59:38,165][74987] Avg episode reward: [(0, '24.380'), (1, '31.690')] -[2023-10-14 14:59:38,339][75949] Updated weights for policy 0, policy_version 37911 (0.0009) -[2023-10-14 14:59:40,781][75950] Updated weights for policy 1, policy_version 37830 (0.0007) -[2023-10-14 14:59:41,139][75950] Updated weights for policy 1, policy_version 37840 (0.0008) -[2023-10-14 14:59:41,507][75950] Updated weights for policy 1, policy_version 37850 (0.0008) -[2023-10-14 14:59:42,501][75949] Updated weights for policy 0, policy_version 37921 (0.0007) -[2023-10-14 14:59:42,875][75949] Updated weights for policy 0, policy_version 37931 (0.0007) -[2023-10-14 14:59:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 77594624. Throughput: 0: 1679.1, 1: 1680.2. Samples: 19411730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:59:43,164][74987] Avg episode reward: [(0, '25.420'), (1, '29.390')] -[2023-10-14 14:59:43,237][75949] Updated weights for policy 0, policy_version 37941 (0.0008) -[2023-10-14 14:59:43,615][75949] Updated weights for policy 0, policy_version 37951 (0.0009) -[2023-10-14 14:59:45,531][75950] Updated weights for policy 1, policy_version 37860 (0.0009) -[2023-10-14 14:59:45,894][75950] Updated weights for policy 1, policy_version 37870 (0.0007) -[2023-10-14 14:59:46,271][75950] Updated weights for policy 1, policy_version 37880 (0.0009) -[2023-10-14 14:59:47,602][75949] Updated weights for policy 0, policy_version 37961 (0.0008) -[2023-10-14 14:59:47,978][75949] Updated weights for policy 0, policy_version 37971 (0.0008) -[2023-10-14 14:59:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 77660160. Throughput: 0: 1685.2, 1: 1684.3. Samples: 19422138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:59:48,164][74987] Avg episode reward: [(0, '24.490'), (1, '29.120')] -[2023-10-14 14:59:48,351][75949] Updated weights for policy 0, policy_version 37981 (0.0010) -[2023-10-14 14:59:50,220][75950] Updated weights for policy 1, policy_version 37890 (0.0009) -[2023-10-14 14:59:50,583][75950] Updated weights for policy 1, policy_version 37900 (0.0008) -[2023-10-14 14:59:50,948][75950] Updated weights for policy 1, policy_version 37910 (0.0007) -[2023-10-14 14:59:51,316][75950] Updated weights for policy 1, policy_version 37920 (0.0009) -[2023-10-14 14:59:52,367][75949] Updated weights for policy 0, policy_version 37991 (0.0010) -[2023-10-14 14:59:52,743][75949] Updated weights for policy 0, policy_version 38001 (0.0010) -[2023-10-14 14:59:53,118][75949] Updated weights for policy 0, policy_version 38011 (0.0011) -[2023-10-14 14:59:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 77725696. Throughput: 0: 1691.5, 1: 1666.7. Samples: 19442066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:59:53,164][74987] Avg episode reward: [(0, '26.770'), (1, '30.360')] -[2023-10-14 14:59:55,361][75950] Updated weights for policy 1, policy_version 37930 (0.0008) -[2023-10-14 14:59:55,728][75950] Updated weights for policy 1, policy_version 37940 (0.0008) -[2023-10-14 14:59:56,095][75950] Updated weights for policy 1, policy_version 37950 (0.0008) -[2023-10-14 14:59:57,177][75949] Updated weights for policy 0, policy_version 38021 (0.0010) -[2023-10-14 14:59:57,545][75949] Updated weights for policy 0, policy_version 38031 (0.0009) -[2023-10-14 14:59:57,917][75949] Updated weights for policy 0, policy_version 38041 (0.0007) -[2023-10-14 14:59:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 77791232. Throughput: 0: 1669.8, 1: 1693.5. Samples: 19462020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 14:59:58,164][74987] Avg episode reward: [(0, '25.200'), (1, '26.890')] -[2023-10-14 15:00:00,443][75950] Updated weights for policy 1, policy_version 37960 (0.0009) -[2023-10-14 15:00:00,809][75950] Updated weights for policy 1, policy_version 37970 (0.0008) -[2023-10-14 15:00:01,175][75950] Updated weights for policy 1, policy_version 37980 (0.0009) -[2023-10-14 15:00:01,966][75949] Updated weights for policy 0, policy_version 38051 (0.0009) -[2023-10-14 15:00:02,354][75949] Updated weights for policy 0, policy_version 38061 (0.0008) -[2023-10-14 15:00:02,727][75949] Updated weights for policy 0, policy_version 38071 (0.0008) -[2023-10-14 15:00:03,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 77889536. Throughput: 0: 1691.4, 1: 1674.6. Samples: 19472708. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) -[2023-10-14 15:00:03,164][74987] Avg episode reward: [(0, '24.750'), (1, '29.120')] -[2023-10-14 15:00:05,275][75950] Updated weights for policy 1, policy_version 37990 (0.0009) -[2023-10-14 15:00:05,633][75950] Updated weights for policy 1, policy_version 38000 (0.0010) -[2023-10-14 15:00:05,995][75950] Updated weights for policy 1, policy_version 38010 (0.0009) -[2023-10-14 15:00:06,814][75949] Updated weights for policy 0, policy_version 38081 (0.0007) -[2023-10-14 15:00:07,180][75949] Updated weights for policy 0, policy_version 38091 (0.0008) -[2023-10-14 15:00:07,552][75949] Updated weights for policy 0, policy_version 38101 (0.0010) -[2023-10-14 15:00:07,927][75949] Updated weights for policy 0, policy_version 38111 (0.0010) -[2023-10-14 15:00:08,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 77955072. Throughput: 0: 1692.3, 1: 1671.0. Samples: 19492642. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) -[2023-10-14 15:00:08,164][74987] Avg episode reward: [(0, '23.750'), (1, '28.380')] -[2023-10-14 15:00:10,130][75950] Updated weights for policy 1, policy_version 38020 (0.0009) -[2023-10-14 15:00:10,493][75950] Updated weights for policy 1, policy_version 38030 (0.0007) -[2023-10-14 15:00:10,858][75950] Updated weights for policy 1, policy_version 38040 (0.0009) -[2023-10-14 15:00:11,980][75949] Updated weights for policy 0, policy_version 38121 (0.0009) -[2023-10-14 15:00:12,357][75949] Updated weights for policy 0, policy_version 38131 (0.0007) -[2023-10-14 15:00:12,717][75949] Updated weights for policy 0, policy_version 38141 (0.0007) -[2023-10-14 15:00:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 78020608. Throughput: 0: 1666.7, 1: 1686.7. Samples: 19512316. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) -[2023-10-14 15:00:13,165][74987] Avg episode reward: [(0, '26.750'), (1, '27.160')] -[2023-10-14 15:00:14,816][75950] Updated weights for policy 1, policy_version 38050 (0.0011) -[2023-10-14 15:00:15,192][75950] Updated weights for policy 1, policy_version 38060 (0.0009) -[2023-10-14 15:00:15,561][75950] Updated weights for policy 1, policy_version 38070 (0.0010) -[2023-10-14 15:00:15,932][75950] Updated weights for policy 1, policy_version 38080 (0.0011) -[2023-10-14 15:00:16,462][75949] Updated weights for policy 0, policy_version 38151 (0.0010) -[2023-10-14 15:00:16,830][75949] Updated weights for policy 0, policy_version 38161 (0.0010) -[2023-10-14 15:00:17,208][75949] Updated weights for policy 0, policy_version 38171 (0.0011) -[2023-10-14 15:00:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 78086144. Throughput: 0: 1693.4, 1: 1665.0. Samples: 19522928. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) -[2023-10-14 15:00:18,165][74987] Avg episode reward: [(0, '24.550'), (1, '28.840')] -[2023-10-14 15:00:20,046][75950] Updated weights for policy 1, policy_version 38090 (0.0009) -[2023-10-14 15:00:20,409][75950] Updated weights for policy 1, policy_version 38100 (0.0008) -[2023-10-14 15:00:20,774][75950] Updated weights for policy 1, policy_version 38110 (0.0010) -[2023-10-14 15:00:21,425][75949] Updated weights for policy 0, policy_version 38181 (0.0010) -[2023-10-14 15:00:21,795][75949] Updated weights for policy 0, policy_version 38191 (0.0010) -[2023-10-14 15:00:22,164][75949] Updated weights for policy 0, policy_version 38201 (0.0010) -[2023-10-14 15:00:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 78151680. Throughput: 0: 1677.7, 1: 1675.3. Samples: 19542362. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) -[2023-10-14 15:00:23,164][74987] Avg episode reward: [(0, '27.150'), (1, '28.000')] -[2023-10-14 15:00:24,877][75950] Updated weights for policy 1, policy_version 38120 (0.0010) -[2023-10-14 15:00:25,246][75950] Updated weights for policy 1, policy_version 38130 (0.0011) -[2023-10-14 15:00:25,622][75950] Updated weights for policy 1, policy_version 38140 (0.0010) -[2023-10-14 15:00:26,194][75949] Updated weights for policy 0, policy_version 38211 (0.0009) -[2023-10-14 15:00:26,574][75949] Updated weights for policy 0, policy_version 38221 (0.0010) -[2023-10-14 15:00:26,944][75949] Updated weights for policy 0, policy_version 38231 (0.0011) -[2023-10-14 15:00:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 78217216. Throughput: 0: 1664.3, 1: 1682.2. Samples: 19562322. Policy #0 lag: (min: 26.0, avg: 32.2, max: 58.0) -[2023-10-14 15:00:28,165][74987] Avg episode reward: [(0, '24.280'), (1, '27.680')] -[2023-10-14 15:00:29,727][75950] Updated weights for policy 1, policy_version 38150 (0.0008) -[2023-10-14 15:00:30,100][75950] Updated weights for policy 1, policy_version 38160 (0.0008) -[2023-10-14 15:00:30,458][75950] Updated weights for policy 1, policy_version 38170 (0.0009) -[2023-10-14 15:00:31,290][75949] Updated weights for policy 0, policy_version 38241 (0.0010) -[2023-10-14 15:00:31,657][75949] Updated weights for policy 0, policy_version 38251 (0.0008) -[2023-10-14 15:00:32,027][75949] Updated weights for policy 0, policy_version 38261 (0.0008) -[2023-10-14 15:00:32,401][75949] Updated weights for policy 0, policy_version 38271 (0.0009) -[2023-10-14 15:00:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 78282752. Throughput: 0: 1684.9, 1: 1663.2. Samples: 19572800. Policy #0 lag: (min: 26.0, avg: 32.2, max: 58.0) -[2023-10-14 15:00:33,164][74987] Avg episode reward: [(0, '25.160'), (1, '28.980')] -[2023-10-14 15:00:34,425][75950] Updated weights for policy 1, policy_version 38180 (0.0010) -[2023-10-14 15:00:34,802][75950] Updated weights for policy 1, policy_version 38190 (0.0007) -[2023-10-14 15:00:35,163][75950] Updated weights for policy 1, policy_version 38200 (0.0008) -[2023-10-14 15:00:36,342][75949] Updated weights for policy 0, policy_version 38281 (0.0009) -[2023-10-14 15:00:36,714][75949] Updated weights for policy 0, policy_version 38291 (0.0010) -[2023-10-14 15:00:37,084][75949] Updated weights for policy 0, policy_version 38301 (0.0009) -[2023-10-14 15:00:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 78348288. Throughput: 0: 1669.8, 1: 1680.4. Samples: 19592826. Policy #0 lag: (min: 26.0, avg: 32.2, max: 58.0) -[2023-10-14 15:00:38,165][74987] Avg episode reward: [(0, '26.030'), (1, '29.500')] -[2023-10-14 15:00:39,310][75950] Updated weights for policy 1, policy_version 38210 (0.0009) -[2023-10-14 15:00:39,677][75950] Updated weights for policy 1, policy_version 38220 (0.0009) -[2023-10-14 15:00:40,040][75950] Updated weights for policy 1, policy_version 38230 (0.0009) -[2023-10-14 15:00:40,412][75950] Updated weights for policy 1, policy_version 38240 (0.0009) -[2023-10-14 15:00:41,313][75949] Updated weights for policy 0, policy_version 38311 (0.0009) -[2023-10-14 15:00:41,692][75949] Updated weights for policy 0, policy_version 38321 (0.0009) -[2023-10-14 15:00:42,063][75949] Updated weights for policy 0, policy_version 38331 (0.0009) -[2023-10-14 15:00:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 78413824. Throughput: 0: 1676.9, 1: 1678.3. Samples: 19613006. Policy #0 lag: (min: 26.0, avg: 32.2, max: 58.0) -[2023-10-14 15:00:43,164][74987] Avg episode reward: [(0, '24.910'), (1, '29.330')] -[2023-10-14 15:00:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000038336_39256064.pth... -[2023-10-14 15:00:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000038240_39157760.pth... -[2023-10-14 15:00:43,212][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000036768_37650432.pth -[2023-10-14 15:00:43,215][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000036672_37552128.pth -[2023-10-14 15:00:44,496][75950] Updated weights for policy 1, policy_version 38250 (0.0011) -[2023-10-14 15:00:44,872][75950] Updated weights for policy 1, policy_version 38260 (0.0010) -[2023-10-14 15:00:45,241][75950] Updated weights for policy 1, policy_version 38270 (0.0009) -[2023-10-14 15:00:46,166][75949] Updated weights for policy 0, policy_version 38341 (0.0010) -[2023-10-14 15:00:46,540][75949] Updated weights for policy 0, policy_version 38351 (0.0011) -[2023-10-14 15:00:46,908][75949] Updated weights for policy 0, policy_version 38361 (0.0007) -[2023-10-14 15:00:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 78479360. Throughput: 0: 1685.4, 1: 1662.2. Samples: 19623348. Policy #0 lag: (min: 26.0, avg: 32.2, max: 58.0) -[2023-10-14 15:00:48,164][74987] Avg episode reward: [(0, '26.600'), (1, '30.080')] -[2023-10-14 15:00:49,372][75950] Updated weights for policy 1, policy_version 38280 (0.0007) -[2023-10-14 15:00:49,742][75950] Updated weights for policy 1, policy_version 38290 (0.0009) -[2023-10-14 15:00:50,112][75950] Updated weights for policy 1, policy_version 38300 (0.0007) -[2023-10-14 15:00:50,956][75949] Updated weights for policy 0, policy_version 38371 (0.0008) -[2023-10-14 15:00:51,358][75949] Updated weights for policy 0, policy_version 38381 (0.0008) -[2023-10-14 15:00:51,730][75949] Updated weights for policy 0, policy_version 38391 (0.0009) -[2023-10-14 15:00:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 78544896. Throughput: 0: 1662.5, 1: 1680.3. Samples: 19643072. Policy #0 lag: (min: 26.0, avg: 32.2, max: 58.0) -[2023-10-14 15:00:53,165][74987] Avg episode reward: [(0, '25.150'), (1, '27.600')] -[2023-10-14 15:00:54,180][75950] Updated weights for policy 1, policy_version 38310 (0.0008) -[2023-10-14 15:00:54,543][75950] Updated weights for policy 1, policy_version 38320 (0.0010) -[2023-10-14 15:00:54,908][75950] Updated weights for policy 1, policy_version 38330 (0.0011) -[2023-10-14 15:00:55,590][75949] Updated weights for policy 0, policy_version 38401 (0.0010) -[2023-10-14 15:00:55,956][75949] Updated weights for policy 0, policy_version 38411 (0.0009) -[2023-10-14 15:00:56,334][75949] Updated weights for policy 0, policy_version 38421 (0.0010) -[2023-10-14 15:00:56,706][75949] Updated weights for policy 0, policy_version 38431 (0.0010) -[2023-10-14 15:00:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 78610432. Throughput: 0: 1677.0, 1: 1678.3. Samples: 19663302. Policy #0 lag: (min: 20.0, avg: 27.6, max: 52.0) -[2023-10-14 15:00:58,165][74987] Avg episode reward: [(0, '25.000'), (1, '29.790')] -[2023-10-14 15:00:59,087][75950] Updated weights for policy 1, policy_version 38340 (0.0008) -[2023-10-14 15:00:59,456][75950] Updated weights for policy 1, policy_version 38350 (0.0007) -[2023-10-14 15:00:59,826][75950] Updated weights for policy 1, policy_version 38360 (0.0009) -[2023-10-14 15:01:00,713][75949] Updated weights for policy 0, policy_version 38441 (0.0009) -[2023-10-14 15:01:01,087][75949] Updated weights for policy 0, policy_version 38451 (0.0010) -[2023-10-14 15:01:01,459][75949] Updated weights for policy 0, policy_version 38461 (0.0008) -[2023-10-14 15:01:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 78675968. Throughput: 0: 1668.3, 1: 1672.8. Samples: 19673276. Policy #0 lag: (min: 20.0, avg: 27.6, max: 52.0) -[2023-10-14 15:01:03,165][74987] Avg episode reward: [(0, '24.740'), (1, '28.950')] -[2023-10-14 15:01:03,758][75950] Updated weights for policy 1, policy_version 38370 (0.0010) -[2023-10-14 15:01:04,120][75950] Updated weights for policy 1, policy_version 38380 (0.0010) -[2023-10-14 15:01:04,493][75950] Updated weights for policy 1, policy_version 38390 (0.0009) -[2023-10-14 15:01:04,858][75950] Updated weights for policy 1, policy_version 38400 (0.0008) -[2023-10-14 15:01:05,496][75949] Updated weights for policy 0, policy_version 38471 (0.0009) -[2023-10-14 15:01:05,872][75949] Updated weights for policy 0, policy_version 38481 (0.0007) -[2023-10-14 15:01:06,247][75949] Updated weights for policy 0, policy_version 38491 (0.0009) -[2023-10-14 15:01:08,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 78741504. Throughput: 0: 1662.3, 1: 1689.0. Samples: 19693170. Policy #0 lag: (min: 20.0, avg: 27.6, max: 52.0) -[2023-10-14 15:01:08,164][74987] Avg episode reward: [(0, '24.870'), (1, '26.960')] -[2023-10-14 15:01:08,878][75950] Updated weights for policy 1, policy_version 38410 (0.0009) -[2023-10-14 15:01:09,248][75950] Updated weights for policy 1, policy_version 38420 (0.0008) -[2023-10-14 15:01:09,619][75950] Updated weights for policy 1, policy_version 38430 (0.0007) -[2023-10-14 15:01:10,307][75949] Updated weights for policy 0, policy_version 38501 (0.0009) -[2023-10-14 15:01:10,667][75949] Updated weights for policy 0, policy_version 38511 (0.0010) -[2023-10-14 15:01:11,035][75949] Updated weights for policy 0, policy_version 38521 (0.0010) -[2023-10-14 15:01:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 78807040. Throughput: 0: 1684.9, 1: 1691.6. Samples: 19714264. Policy #0 lag: (min: 20.0, avg: 27.6, max: 52.0) -[2023-10-14 15:01:13,165][74987] Avg episode reward: [(0, '24.280'), (1, '30.910')] -[2023-10-14 15:01:13,698][75950] Updated weights for policy 1, policy_version 38440 (0.0007) -[2023-10-14 15:01:14,060][75950] Updated weights for policy 1, policy_version 38450 (0.0008) -[2023-10-14 15:01:14,432][75950] Updated weights for policy 1, policy_version 38460 (0.0007) -[2023-10-14 15:01:15,007][75949] Updated weights for policy 0, policy_version 38531 (0.0009) -[2023-10-14 15:01:15,383][75949] Updated weights for policy 0, policy_version 38541 (0.0008) -[2023-10-14 15:01:15,757][75949] Updated weights for policy 0, policy_version 38551 (0.0008) -[2023-10-14 15:01:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 78872576. Throughput: 0: 1669.3, 1: 1685.3. Samples: 19723758. Policy #0 lag: (min: 20.0, avg: 27.6, max: 52.0) -[2023-10-14 15:01:18,164][74987] Avg episode reward: [(0, '25.300'), (1, '31.090')] -[2023-10-14 15:01:18,628][75950] Updated weights for policy 1, policy_version 38470 (0.0007) -[2023-10-14 15:01:18,992][75950] Updated weights for policy 1, policy_version 38480 (0.0007) -[2023-10-14 15:01:19,360][75950] Updated weights for policy 1, policy_version 38490 (0.0008) -[2023-10-14 15:01:19,793][75949] Updated weights for policy 0, policy_version 38561 (0.0008) -[2023-10-14 15:01:20,162][75949] Updated weights for policy 0, policy_version 38571 (0.0008) -[2023-10-14 15:01:20,531][75949] Updated weights for policy 0, policy_version 38581 (0.0008) -[2023-10-14 15:01:20,905][75949] Updated weights for policy 0, policy_version 38591 (0.0010) -[2023-10-14 15:01:23,157][75950] Updated weights for policy 1, policy_version 38500 (0.0010) -[2023-10-14 15:01:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 78938112. Throughput: 0: 1671.8, 1: 1686.3. Samples: 19743940. Policy #0 lag: (min: 20.0, avg: 27.6, max: 52.0) -[2023-10-14 15:01:23,165][74987] Avg episode reward: [(0, '26.190'), (1, '29.580')] -[2023-10-14 15:01:23,530][75950] Updated weights for policy 1, policy_version 38510 (0.0011) -[2023-10-14 15:01:23,901][75950] Updated weights for policy 1, policy_version 38520 (0.0010) -[2023-10-14 15:01:25,239][75949] Updated weights for policy 0, policy_version 38601 (0.0008) -[2023-10-14 15:01:25,606][75949] Updated weights for policy 0, policy_version 38611 (0.0009) -[2023-10-14 15:01:25,981][75949] Updated weights for policy 0, policy_version 38621 (0.0008) -[2023-10-14 15:01:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 79003648. Throughput: 0: 1682.4, 1: 1684.6. Samples: 19764518. Policy #0 lag: (min: 27.0, avg: 27.3, max: 40.0) -[2023-10-14 15:01:28,165][74987] Avg episode reward: [(0, '25.860'), (1, '30.250')] -[2023-10-14 15:01:28,209][75950] Updated weights for policy 1, policy_version 38530 (0.0009) -[2023-10-14 15:01:28,584][75950] Updated weights for policy 1, policy_version 38540 (0.0010) -[2023-10-14 15:01:28,952][75950] Updated weights for policy 1, policy_version 38550 (0.0008) -[2023-10-14 15:01:29,327][75950] Updated weights for policy 1, policy_version 38560 (0.0009) -[2023-10-14 15:01:30,158][75949] Updated weights for policy 0, policy_version 38631 (0.0007) -[2023-10-14 15:01:30,526][75949] Updated weights for policy 0, policy_version 38641 (0.0007) -[2023-10-14 15:01:30,906][75949] Updated weights for policy 0, policy_version 38651 (0.0008) -[2023-10-14 15:01:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 79069184. Throughput: 0: 1665.3, 1: 1683.3. Samples: 19774036. Policy #0 lag: (min: 27.0, avg: 27.3, max: 40.0) -[2023-10-14 15:01:33,165][74987] Avg episode reward: [(0, '26.710'), (1, '31.370')] -[2023-10-14 15:01:33,343][75950] Updated weights for policy 1, policy_version 38570 (0.0010) -[2023-10-14 15:01:33,722][75950] Updated weights for policy 1, policy_version 38580 (0.0010) -[2023-10-14 15:01:34,076][75950] Updated weights for policy 1, policy_version 38590 (0.0010) -[2023-10-14 15:01:34,960][75949] Updated weights for policy 0, policy_version 38661 (0.0009) -[2023-10-14 15:01:35,335][75949] Updated weights for policy 0, policy_version 38671 (0.0007) -[2023-10-14 15:01:35,705][75949] Updated weights for policy 0, policy_version 38681 (0.0007) -[2023-10-14 15:01:38,039][75950] Updated weights for policy 1, policy_version 38600 (0.0010) -[2023-10-14 15:01:38,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 79134720. Throughput: 0: 1680.7, 1: 1685.6. Samples: 19794556. Policy #0 lag: (min: 27.0, avg: 27.3, max: 40.0) -[2023-10-14 15:01:38,164][74987] Avg episode reward: [(0, '24.740'), (1, '29.310')] -[2023-10-14 15:01:38,409][75950] Updated weights for policy 1, policy_version 38610 (0.0007) -[2023-10-14 15:01:38,779][75950] Updated weights for policy 1, policy_version 38620 (0.0008) -[2023-10-14 15:01:39,664][75949] Updated weights for policy 0, policy_version 38691 (0.0007) -[2023-10-14 15:01:40,067][75949] Updated weights for policy 0, policy_version 38701 (0.0008) -[2023-10-14 15:01:40,440][75949] Updated weights for policy 0, policy_version 38711 (0.0007) -[2023-10-14 15:01:43,005][75950] Updated weights for policy 1, policy_version 38630 (0.0009) -[2023-10-14 15:01:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 79200256. Throughput: 0: 1695.1, 1: 1682.8. Samples: 19815306. Policy #0 lag: (min: 27.0, avg: 27.3, max: 40.0) -[2023-10-14 15:01:43,165][74987] Avg episode reward: [(0, '26.370'), (1, '28.320')] -[2023-10-14 15:01:43,368][75950] Updated weights for policy 1, policy_version 38640 (0.0008) -[2023-10-14 15:01:43,724][75950] Updated weights for policy 1, policy_version 38650 (0.0009) -[2023-10-14 15:01:44,307][75949] Updated weights for policy 0, policy_version 38721 (0.0009) -[2023-10-14 15:01:44,675][75949] Updated weights for policy 0, policy_version 38731 (0.0007) -[2023-10-14 15:01:45,052][75949] Updated weights for policy 0, policy_version 38741 (0.0008) -[2023-10-14 15:01:45,431][75949] Updated weights for policy 0, policy_version 38751 (0.0008) -[2023-10-14 15:01:47,886][75950] Updated weights for policy 1, policy_version 38660 (0.0008) -[2023-10-14 15:01:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 79265792. Throughput: 0: 1675.2, 1: 1684.1. Samples: 19824444. Policy #0 lag: (min: 27.0, avg: 27.3, max: 40.0) -[2023-10-14 15:01:48,164][74987] Avg episode reward: [(0, '23.970'), (1, '31.680')] -[2023-10-14 15:01:48,252][75950] Updated weights for policy 1, policy_version 38670 (0.0008) -[2023-10-14 15:01:48,616][75950] Updated weights for policy 1, policy_version 38680 (0.0008) -[2023-10-14 15:01:49,414][75949] Updated weights for policy 0, policy_version 38761 (0.0010) -[2023-10-14 15:01:49,785][75949] Updated weights for policy 0, policy_version 38771 (0.0010) -[2023-10-14 15:01:50,158][75949] Updated weights for policy 0, policy_version 38781 (0.0008) -[2023-10-14 15:01:52,573][75950] Updated weights for policy 1, policy_version 38690 (0.0008) -[2023-10-14 15:01:52,938][75950] Updated weights for policy 1, policy_version 38700 (0.0009) -[2023-10-14 15:01:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 79331328. Throughput: 0: 1693.4, 1: 1678.5. Samples: 19844904. Policy #0 lag: (min: 27.0, avg: 27.3, max: 40.0) -[2023-10-14 15:01:53,164][74987] Avg episode reward: [(0, '23.110'), (1, '31.030')] -[2023-10-14 15:01:53,308][75950] Updated weights for policy 1, policy_version 38710 (0.0008) -[2023-10-14 15:01:53,680][75950] Updated weights for policy 1, policy_version 38720 (0.0007) -[2023-10-14 15:01:54,171][75949] Updated weights for policy 0, policy_version 38791 (0.0008) -[2023-10-14 15:01:54,557][75949] Updated weights for policy 0, policy_version 38801 (0.0008) -[2023-10-14 15:01:54,920][75949] Updated weights for policy 0, policy_version 38811 (0.0008) -[2023-10-14 15:01:57,700][75950] Updated weights for policy 1, policy_version 38730 (0.0009) -[2023-10-14 15:01:58,060][75950] Updated weights for policy 1, policy_version 38740 (0.0008) -[2023-10-14 15:01:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79396864. Throughput: 0: 1694.0, 1: 1661.1. Samples: 19865244. Policy #0 lag: (min: 3.0, avg: 9.5, max: 35.0) -[2023-10-14 15:01:58,164][74987] Avg episode reward: [(0, '22.630'), (1, '28.160')] -[2023-10-14 15:01:58,422][75950] Updated weights for policy 1, policy_version 38750 (0.0010) -[2023-10-14 15:01:59,071][75949] Updated weights for policy 0, policy_version 38821 (0.0010) -[2023-10-14 15:01:59,441][75949] Updated weights for policy 0, policy_version 38831 (0.0007) -[2023-10-14 15:01:59,807][75949] Updated weights for policy 0, policy_version 38841 (0.0009) -[2023-10-14 15:02:02,541][75950] Updated weights for policy 1, policy_version 38760 (0.0008) -[2023-10-14 15:02:02,909][75950] Updated weights for policy 1, policy_version 38770 (0.0008) -[2023-10-14 15:02:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 79462400. Throughput: 0: 1683.1, 1: 1666.2. Samples: 19874476. Policy #0 lag: (min: 3.0, avg: 9.5, max: 35.0) -[2023-10-14 15:02:03,165][74987] Avg episode reward: [(0, '25.380'), (1, '29.960')] -[2023-10-14 15:02:03,280][75950] Updated weights for policy 1, policy_version 38780 (0.0008) -[2023-10-14 15:02:03,893][75949] Updated weights for policy 0, policy_version 38851 (0.0010) -[2023-10-14 15:02:04,267][75949] Updated weights for policy 0, policy_version 38861 (0.0010) -[2023-10-14 15:02:04,631][75949] Updated weights for policy 0, policy_version 38871 (0.0011) -[2023-10-14 15:02:07,558][75950] Updated weights for policy 1, policy_version 38790 (0.0008) -[2023-10-14 15:02:07,922][75950] Updated weights for policy 1, policy_version 38800 (0.0009) -[2023-10-14 15:02:08,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79527936. Throughput: 0: 1693.7, 1: 1669.0. Samples: 19895260. Policy #0 lag: (min: 3.0, avg: 9.5, max: 35.0) -[2023-10-14 15:02:08,164][74987] Avg episode reward: [(0, '26.830'), (1, '30.630')] -[2023-10-14 15:02:08,287][75950] Updated weights for policy 1, policy_version 38810 (0.0008) -[2023-10-14 15:02:08,510][75949] Updated weights for policy 0, policy_version 38881 (0.0010) -[2023-10-14 15:02:08,883][75949] Updated weights for policy 0, policy_version 38891 (0.0008) -[2023-10-14 15:02:09,257][75949] Updated weights for policy 0, policy_version 38901 (0.0009) -[2023-10-14 15:02:09,623][75949] Updated weights for policy 0, policy_version 38911 (0.0009) -[2023-10-14 15:02:12,261][75950] Updated weights for policy 1, policy_version 38820 (0.0008) -[2023-10-14 15:02:12,635][75950] Updated weights for policy 1, policy_version 38830 (0.0010) -[2023-10-14 15:02:13,011][75950] Updated weights for policy 1, policy_version 38840 (0.0008) -[2023-10-14 15:02:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79593472. Throughput: 0: 1695.3, 1: 1665.0. Samples: 19915732. Policy #0 lag: (min: 3.0, avg: 9.5, max: 35.0) -[2023-10-14 15:02:13,164][74987] Avg episode reward: [(0, '25.620'), (1, '28.120')] -[2023-10-14 15:02:13,837][75949] Updated weights for policy 0, policy_version 38921 (0.0009) -[2023-10-14 15:02:14,198][75949] Updated weights for policy 0, policy_version 38931 (0.0008) -[2023-10-14 15:02:14,574][75949] Updated weights for policy 0, policy_version 38941 (0.0009) -[2023-10-14 15:02:17,175][75950] Updated weights for policy 1, policy_version 38850 (0.0008) -[2023-10-14 15:02:17,543][75950] Updated weights for policy 1, policy_version 38860 (0.0008) -[2023-10-14 15:02:17,901][75950] Updated weights for policy 1, policy_version 38870 (0.0007) -[2023-10-14 15:02:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79659008. Throughput: 0: 1685.3, 1: 1675.1. Samples: 19925254. Policy #0 lag: (min: 3.0, avg: 9.5, max: 35.0) -[2023-10-14 15:02:18,164][74987] Avg episode reward: [(0, '27.160'), (1, '28.230')] -[2023-10-14 15:02:18,271][75950] Updated weights for policy 1, policy_version 38880 (0.0008) -[2023-10-14 15:02:18,707][75949] Updated weights for policy 0, policy_version 38951 (0.0009) -[2023-10-14 15:02:19,078][75949] Updated weights for policy 0, policy_version 38961 (0.0008) -[2023-10-14 15:02:19,448][75949] Updated weights for policy 0, policy_version 38971 (0.0007) -[2023-10-14 15:02:22,624][75950] Updated weights for policy 1, policy_version 38890 (0.0007) -[2023-10-14 15:02:23,000][75950] Updated weights for policy 1, policy_version 38900 (0.0007) -[2023-10-14 15:02:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 79724544. Throughput: 0: 1687.8, 1: 1673.3. Samples: 19945806. Policy #0 lag: (min: 3.0, avg: 9.5, max: 35.0) -[2023-10-14 15:02:23,165][74987] Avg episode reward: [(0, '25.430'), (1, '30.240')] -[2023-10-14 15:02:23,369][75950] Updated weights for policy 1, policy_version 38910 (0.0007) -[2023-10-14 15:02:23,437][75949] Updated weights for policy 0, policy_version 38981 (0.0008) -[2023-10-14 15:02:23,815][75949] Updated weights for policy 0, policy_version 38991 (0.0009) -[2023-10-14 15:02:24,182][75949] Updated weights for policy 0, policy_version 39001 (0.0009) -[2023-10-14 15:02:27,407][75950] Updated weights for policy 1, policy_version 38920 (0.0009) -[2023-10-14 15:02:27,768][75950] Updated weights for policy 1, policy_version 38930 (0.0010) -[2023-10-14 15:02:28,144][75950] Updated weights for policy 1, policy_version 38940 (0.0010) -[2023-10-14 15:02:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79790080. Throughput: 0: 1686.9, 1: 1662.6. Samples: 19966032. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) -[2023-10-14 15:02:28,164][74987] Avg episode reward: [(0, '26.500'), (1, '28.270')] -[2023-10-14 15:02:28,371][75949] Updated weights for policy 0, policy_version 39011 (0.0010) -[2023-10-14 15:02:28,780][75949] Updated weights for policy 0, policy_version 39021 (0.0008) -[2023-10-14 15:02:29,150][75949] Updated weights for policy 0, policy_version 39031 (0.0007) -[2023-10-14 15:02:32,229][75950] Updated weights for policy 1, policy_version 38950 (0.0009) -[2023-10-14 15:02:32,605][75950] Updated weights for policy 1, policy_version 38960 (0.0008) -[2023-10-14 15:02:32,961][75950] Updated weights for policy 1, policy_version 38970 (0.0009) -[2023-10-14 15:02:33,158][75949] Updated weights for policy 0, policy_version 39041 (0.0009) -[2023-10-14 15:02:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79855616. Throughput: 0: 1683.2, 1: 1672.2. Samples: 19975436. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) -[2023-10-14 15:02:33,164][74987] Avg episode reward: [(0, '25.890'), (1, '28.660')] -[2023-10-14 15:02:33,521][75949] Updated weights for policy 0, policy_version 39051 (0.0010) -[2023-10-14 15:02:33,894][75949] Updated weights for policy 0, policy_version 39061 (0.0009) -[2023-10-14 15:02:34,273][75949] Updated weights for policy 0, policy_version 39071 (0.0009) -[2023-10-14 15:02:36,938][75950] Updated weights for policy 1, policy_version 38980 (0.0009) -[2023-10-14 15:02:37,302][75950] Updated weights for policy 1, policy_version 38990 (0.0007) -[2023-10-14 15:02:37,664][75950] Updated weights for policy 1, policy_version 39000 (0.0008) -[2023-10-14 15:02:38,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 79953920. Throughput: 0: 1686.2, 1: 1675.8. Samples: 19996192. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) -[2023-10-14 15:02:38,164][74987] Avg episode reward: [(0, '25.610'), (1, '31.790')] -[2023-10-14 15:02:38,205][75949] Updated weights for policy 0, policy_version 39081 (0.0007) -[2023-10-14 15:02:38,573][75949] Updated weights for policy 0, policy_version 39091 (0.0008) -[2023-10-14 15:02:38,943][75949] Updated weights for policy 0, policy_version 39101 (0.0008) -[2023-10-14 15:02:41,692][75950] Updated weights for policy 1, policy_version 39010 (0.0009) -[2023-10-14 15:02:42,065][75950] Updated weights for policy 1, policy_version 39020 (0.0008) -[2023-10-14 15:02:42,424][75950] Updated weights for policy 1, policy_version 39030 (0.0010) -[2023-10-14 15:02:42,793][75950] Updated weights for policy 1, policy_version 39040 (0.0009) -[2023-10-14 15:02:43,010][75949] Updated weights for policy 0, policy_version 39111 (0.0008) -[2023-10-14 15:02:43,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 80019456. Throughput: 0: 1687.8, 1: 1663.6. Samples: 20016056. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) -[2023-10-14 15:02:43,164][74987] Avg episode reward: [(0, '25.090'), (1, '31.060')] -[2023-10-14 15:02:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000039040_39976960.pth... -[2023-10-14 15:02:43,203][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000037472_38371328.pth -[2023-10-14 15:02:43,207][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000039040_39976960.pth -[2023-10-14 15:02:43,382][75949] Updated weights for policy 0, policy_version 39121 (0.0009) -[2023-10-14 15:02:43,752][75949] Updated weights for policy 0, policy_version 39131 (0.0009) -[2023-10-14 15:02:43,923][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000039136_40075264.pth... -[2023-10-14 15:02:43,952][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000037536_38436864.pth -[2023-10-14 15:02:43,956][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000039136_40075264.pth -[2023-10-14 15:02:46,851][75950] Updated weights for policy 1, policy_version 39050 (0.0007) -[2023-10-14 15:02:47,219][75950] Updated weights for policy 1, policy_version 39060 (0.0007) -[2023-10-14 15:02:47,585][75950] Updated weights for policy 1, policy_version 39070 (0.0007) -[2023-10-14 15:02:47,693][75949] Updated weights for policy 0, policy_version 39141 (0.0010) -[2023-10-14 15:02:48,063][75949] Updated weights for policy 0, policy_version 39151 (0.0010) -[2023-10-14 15:02:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 80084992. Throughput: 0: 1688.5, 1: 1682.9. Samples: 20026188. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) -[2023-10-14 15:02:48,164][74987] Avg episode reward: [(0, '24.740'), (1, '27.850')] -[2023-10-14 15:02:48,435][75949] Updated weights for policy 0, policy_version 39161 (0.0009) -[2023-10-14 15:02:51,515][75950] Updated weights for policy 1, policy_version 39080 (0.0009) -[2023-10-14 15:02:51,885][75950] Updated weights for policy 1, policy_version 39090 (0.0008) -[2023-10-14 15:02:52,245][75950] Updated weights for policy 1, policy_version 39100 (0.0007) -[2023-10-14 15:02:52,474][75949] Updated weights for policy 0, policy_version 39171 (0.0008) -[2023-10-14 15:02:52,852][75949] Updated weights for policy 0, policy_version 39181 (0.0008) -[2023-10-14 15:02:53,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 80150528. Throughput: 0: 1691.6, 1: 1671.5. Samples: 20046600. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) -[2023-10-14 15:02:53,164][74987] Avg episode reward: [(0, '25.760'), (1, '29.360')] -[2023-10-14 15:02:53,228][75949] Updated weights for policy 0, policy_version 39191 (0.0009) -[2023-10-14 15:02:56,421][75950] Updated weights for policy 1, policy_version 39110 (0.0008) -[2023-10-14 15:02:56,777][75950] Updated weights for policy 1, policy_version 39120 (0.0009) -[2023-10-14 15:02:57,148][75949] Updated weights for policy 0, policy_version 39201 (0.0009) -[2023-10-14 15:02:57,148][75950] Updated weights for policy 1, policy_version 39130 (0.0008) -[2023-10-14 15:02:57,514][75949] Updated weights for policy 0, policy_version 39211 (0.0009) -[2023-10-14 15:02:57,882][75949] Updated weights for policy 0, policy_version 39221 (0.0011) -[2023-10-14 15:02:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 80216064. Throughput: 0: 1682.1, 1: 1654.7. Samples: 20065890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:02:58,164][74987] Avg episode reward: [(0, '24.300'), (1, '31.460')] -[2023-10-14 15:02:58,258][75949] Updated weights for policy 0, policy_version 39231 (0.0012) -[2023-10-14 15:03:01,428][75950] Updated weights for policy 1, policy_version 39140 (0.0009) -[2023-10-14 15:03:01,794][75950] Updated weights for policy 1, policy_version 39150 (0.0008) -[2023-10-14 15:03:02,154][75950] Updated weights for policy 1, policy_version 39160 (0.0009) -[2023-10-14 15:03:02,231][75949] Updated weights for policy 0, policy_version 39241 (0.0008) -[2023-10-14 15:03:02,600][75949] Updated weights for policy 0, policy_version 39251 (0.0007) -[2023-10-14 15:03:02,978][75949] Updated weights for policy 0, policy_version 39261 (0.0007) -[2023-10-14 15:03:03,164][74987] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 80314368. Throughput: 0: 1692.4, 1: 1676.4. Samples: 20076852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:03:03,164][74987] Avg episode reward: [(0, '26.020'), (1, '29.650')] -[2023-10-14 15:03:06,382][75950] Updated weights for policy 1, policy_version 39170 (0.0008) -[2023-10-14 15:03:06,743][75950] Updated weights for policy 1, policy_version 39180 (0.0010) -[2023-10-14 15:03:07,072][75949] Updated weights for policy 0, policy_version 39271 (0.0008) -[2023-10-14 15:03:07,109][75950] Updated weights for policy 1, policy_version 39190 (0.0008) -[2023-10-14 15:03:07,449][75949] Updated weights for policy 0, policy_version 39281 (0.0009) -[2023-10-14 15:03:07,478][75950] Updated weights for policy 1, policy_version 39200 (0.0009) -[2023-10-14 15:03:07,818][75949] Updated weights for policy 0, policy_version 39291 (0.0010) -[2023-10-14 15:03:08,163][74987] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 80379904. Throughput: 0: 1694.7, 1: 1663.9. Samples: 20096942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:03:08,164][74987] Avg episode reward: [(0, '24.770'), (1, '30.290')] -[2023-10-14 15:03:11,799][75950] Updated weights for policy 1, policy_version 39210 (0.0009) -[2023-10-14 15:03:11,971][75949] Updated weights for policy 0, policy_version 39301 (0.0009) -[2023-10-14 15:03:12,164][75950] Updated weights for policy 1, policy_version 39220 (0.0009) -[2023-10-14 15:03:12,337][75949] Updated weights for policy 0, policy_version 39311 (0.0008) -[2023-10-14 15:03:12,535][75950] Updated weights for policy 1, policy_version 39230 (0.0007) -[2023-10-14 15:03:12,712][75949] Updated weights for policy 0, policy_version 39321 (0.0009) -[2023-10-14 15:03:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 80445440. Throughput: 0: 1671.9, 1: 1653.9. Samples: 20115696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:03:13,165][74987] Avg episode reward: [(0, '25.390'), (1, '32.540')] -[2023-10-14 15:03:13,179][75801] Saving new best policy, reward=32.540! -[2023-10-14 15:03:16,674][75950] Updated weights for policy 1, policy_version 39240 (0.0007) -[2023-10-14 15:03:16,979][75949] Updated weights for policy 0, policy_version 39331 (0.0008) -[2023-10-14 15:03:17,041][75950] Updated weights for policy 1, policy_version 39250 (0.0007) -[2023-10-14 15:03:17,378][75949] Updated weights for policy 0, policy_version 39341 (0.0008) -[2023-10-14 15:03:17,404][75950] Updated weights for policy 1, policy_version 39260 (0.0009) -[2023-10-14 15:03:17,743][75949] Updated weights for policy 0, policy_version 39351 (0.0008) -[2023-10-14 15:03:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 80510976. Throughput: 0: 1692.9, 1: 1667.0. Samples: 20126632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:03:18,164][74987] Avg episode reward: [(0, '25.920'), (1, '31.020')] -[2023-10-14 15:03:21,599][75950] Updated weights for policy 1, policy_version 39270 (0.0010) -[2023-10-14 15:03:21,806][75949] Updated weights for policy 0, policy_version 39361 (0.0007) -[2023-10-14 15:03:21,965][75950] Updated weights for policy 1, policy_version 39280 (0.0009) -[2023-10-14 15:03:22,172][75949] Updated weights for policy 0, policy_version 39371 (0.0009) -[2023-10-14 15:03:22,333][75950] Updated weights for policy 1, policy_version 39290 (0.0008) -[2023-10-14 15:03:22,548][75949] Updated weights for policy 0, policy_version 39381 (0.0008) -[2023-10-14 15:03:22,915][75949] Updated weights for policy 0, policy_version 39391 (0.0008) -[2023-10-14 15:03:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 80576512. Throughput: 0: 1687.0, 1: 1656.9. Samples: 20146670. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:03:23,164][74987] Avg episode reward: [(0, '24.220'), (1, '30.850')] -[2023-10-14 15:03:26,343][75950] Updated weights for policy 1, policy_version 39300 (0.0007) -[2023-10-14 15:03:26,713][75950] Updated weights for policy 1, policy_version 39310 (0.0009) -[2023-10-14 15:03:26,890][75949] Updated weights for policy 0, policy_version 39401 (0.0009) -[2023-10-14 15:03:27,073][75950] Updated weights for policy 1, policy_version 39320 (0.0007) -[2023-10-14 15:03:27,256][75949] Updated weights for policy 0, policy_version 39411 (0.0009) -[2023-10-14 15:03:27,623][75949] Updated weights for policy 0, policy_version 39421 (0.0010) -[2023-10-14 15:03:28,164][74987] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 80642048. Throughput: 0: 1659.3, 1: 1660.9. Samples: 20165464. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:03:28,165][74987] Avg episode reward: [(0, '26.510'), (1, '29.470')] -[2023-10-14 15:03:30,880][75950] Updated weights for policy 1, policy_version 39330 (0.0008) -[2023-10-14 15:03:31,244][75950] Updated weights for policy 1, policy_version 39340 (0.0009) -[2023-10-14 15:03:31,619][75950] Updated weights for policy 1, policy_version 39350 (0.0008) -[2023-10-14 15:03:31,817][75949] Updated weights for policy 0, policy_version 39431 (0.0009) -[2023-10-14 15:03:31,988][75950] Updated weights for policy 1, policy_version 39360 (0.0010) -[2023-10-14 15:03:32,190][75949] Updated weights for policy 0, policy_version 39441 (0.0008) -[2023-10-14 15:03:32,566][75949] Updated weights for policy 0, policy_version 39451 (0.0008) -[2023-10-14 15:03:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 80707584. Throughput: 0: 1682.2, 1: 1668.6. Samples: 20176974. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:03:33,164][74987] Avg episode reward: [(0, '25.280'), (1, '30.850')] -[2023-10-14 15:03:35,980][75950] Updated weights for policy 1, policy_version 39370 (0.0011) -[2023-10-14 15:03:36,351][75950] Updated weights for policy 1, policy_version 39380 (0.0009) -[2023-10-14 15:03:36,424][75949] Updated weights for policy 0, policy_version 39461 (0.0009) -[2023-10-14 15:03:36,706][75950] Updated weights for policy 1, policy_version 39390 (0.0009) -[2023-10-14 15:03:36,788][75949] Updated weights for policy 0, policy_version 39471 (0.0007) -[2023-10-14 15:03:37,159][75949] Updated weights for policy 0, policy_version 39481 (0.0007) -[2023-10-14 15:03:38,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 80773120. Throughput: 0: 1668.0, 1: 1661.0. Samples: 20196406. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:03:38,164][74987] Avg episode reward: [(0, '27.060'), (1, '30.710')] -[2023-10-14 15:03:40,516][75950] Updated weights for policy 1, policy_version 39400 (0.0007) -[2023-10-14 15:03:40,877][75950] Updated weights for policy 1, policy_version 39410 (0.0007) -[2023-10-14 15:03:41,238][75950] Updated weights for policy 1, policy_version 39420 (0.0008) -[2023-10-14 15:03:41,284][75949] Updated weights for policy 0, policy_version 39491 (0.0009) -[2023-10-14 15:03:41,647][75949] Updated weights for policy 0, policy_version 39501 (0.0008) -[2023-10-14 15:03:42,017][75949] Updated weights for policy 0, policy_version 39511 (0.0009) -[2023-10-14 15:03:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 80838656. Throughput: 0: 1659.5, 1: 1683.0. Samples: 20216302. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:03:43,165][74987] Avg episode reward: [(0, '26.250'), (1, '28.140')] -[2023-10-14 15:03:45,266][75950] Updated weights for policy 1, policy_version 39430 (0.0009) -[2023-10-14 15:03:45,635][75950] Updated weights for policy 1, policy_version 39440 (0.0011) -[2023-10-14 15:03:46,001][75950] Updated weights for policy 1, policy_version 39450 (0.0007) -[2023-10-14 15:03:46,101][75949] Updated weights for policy 0, policy_version 39521 (0.0007) -[2023-10-14 15:03:46,469][75949] Updated weights for policy 0, policy_version 39531 (0.0010) -[2023-10-14 15:03:46,851][75949] Updated weights for policy 0, policy_version 39541 (0.0008) -[2023-10-14 15:03:47,220][75949] Updated weights for policy 0, policy_version 39551 (0.0008) -[2023-10-14 15:03:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 80904192. Throughput: 0: 1679.6, 1: 1668.2. Samples: 20227502. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:03:48,165][74987] Avg episode reward: [(0, '27.090'), (1, '31.330')] -[2023-10-14 15:03:50,172][75950] Updated weights for policy 1, policy_version 39460 (0.0010) -[2023-10-14 15:03:50,541][75950] Updated weights for policy 1, policy_version 39470 (0.0008) -[2023-10-14 15:03:50,904][75950] Updated weights for policy 1, policy_version 39480 (0.0007) -[2023-10-14 15:03:51,406][75949] Updated weights for policy 0, policy_version 39561 (0.0009) -[2023-10-14 15:03:51,783][75949] Updated weights for policy 0, policy_version 39571 (0.0008) -[2023-10-14 15:03:52,153][75949] Updated weights for policy 0, policy_version 39581 (0.0009) -[2023-10-14 15:03:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 80969728. Throughput: 0: 1665.5, 1: 1663.5. Samples: 20246752. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) -[2023-10-14 15:03:53,165][74987] Avg episode reward: [(0, '24.420'), (1, '30.890')] -[2023-10-14 15:03:55,098][75950] Updated weights for policy 1, policy_version 39490 (0.0008) -[2023-10-14 15:03:55,469][75950] Updated weights for policy 1, policy_version 39500 (0.0007) -[2023-10-14 15:03:55,837][75950] Updated weights for policy 1, policy_version 39510 (0.0008) -[2023-10-14 15:03:56,067][75949] Updated weights for policy 0, policy_version 39591 (0.0008) -[2023-10-14 15:03:56,209][75950] Updated weights for policy 1, policy_version 39520 (0.0007) -[2023-10-14 15:03:56,441][75949] Updated weights for policy 0, policy_version 39601 (0.0009) -[2023-10-14 15:03:56,804][75949] Updated weights for policy 0, policy_version 39611 (0.0010) -[2023-10-14 15:03:58,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 81035264. Throughput: 0: 1669.5, 1: 1688.7. Samples: 20266816. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) -[2023-10-14 15:03:58,164][74987] Avg episode reward: [(0, '26.690'), (1, '27.890')] -[2023-10-14 15:04:00,411][75950] Updated weights for policy 1, policy_version 39530 (0.0009) -[2023-10-14 15:04:00,778][75950] Updated weights for policy 1, policy_version 39540 (0.0007) -[2023-10-14 15:04:00,868][75949] Updated weights for policy 0, policy_version 39621 (0.0007) -[2023-10-14 15:04:01,141][75950] Updated weights for policy 1, policy_version 39550 (0.0007) -[2023-10-14 15:04:01,226][75949] Updated weights for policy 0, policy_version 39631 (0.0009) -[2023-10-14 15:04:01,601][75949] Updated weights for policy 0, policy_version 39641 (0.0011) -[2023-10-14 15:04:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 81100800. Throughput: 0: 1685.6, 1: 1674.7. Samples: 20277846. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) -[2023-10-14 15:04:03,164][74987] Avg episode reward: [(0, '25.270'), (1, '29.230')] -[2023-10-14 15:04:05,265][75950] Updated weights for policy 1, policy_version 39560 (0.0008) -[2023-10-14 15:04:05,632][75950] Updated weights for policy 1, policy_version 39570 (0.0008) -[2023-10-14 15:04:05,828][75949] Updated weights for policy 0, policy_version 39651 (0.0010) -[2023-10-14 15:04:05,998][75950] Updated weights for policy 1, policy_version 39580 (0.0009) -[2023-10-14 15:04:06,237][75949] Updated weights for policy 0, policy_version 39661 (0.0009) -[2023-10-14 15:04:06,600][75949] Updated weights for policy 0, policy_version 39671 (0.0007) -[2023-10-14 15:04:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 81166336. Throughput: 0: 1665.9, 1: 1666.5. Samples: 20296630. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) -[2023-10-14 15:04:08,165][74987] Avg episode reward: [(0, '25.170'), (1, '29.570')] -[2023-10-14 15:04:10,267][75950] Updated weights for policy 1, policy_version 39590 (0.0007) -[2023-10-14 15:04:10,612][75949] Updated weights for policy 0, policy_version 39681 (0.0008) -[2023-10-14 15:04:10,638][75950] Updated weights for policy 1, policy_version 39600 (0.0008) -[2023-10-14 15:04:10,974][75949] Updated weights for policy 0, policy_version 39691 (0.0009) -[2023-10-14 15:04:10,997][75950] Updated weights for policy 1, policy_version 39610 (0.0009) -[2023-10-14 15:04:11,345][75949] Updated weights for policy 0, policy_version 39701 (0.0008) -[2023-10-14 15:04:11,719][75949] Updated weights for policy 0, policy_version 39711 (0.0007) -[2023-10-14 15:04:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 81231872. Throughput: 0: 1687.2, 1: 1678.2. Samples: 20316906. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) -[2023-10-14 15:04:13,165][74987] Avg episode reward: [(0, '26.010'), (1, '29.690')] -[2023-10-14 15:04:15,212][75950] Updated weights for policy 1, policy_version 39620 (0.0009) -[2023-10-14 15:04:15,574][75950] Updated weights for policy 1, policy_version 39630 (0.0007) -[2023-10-14 15:04:15,939][75949] Updated weights for policy 0, policy_version 39721 (0.0008) -[2023-10-14 15:04:15,945][75950] Updated weights for policy 1, policy_version 39640 (0.0009) -[2023-10-14 15:04:16,303][75949] Updated weights for policy 0, policy_version 39731 (0.0008) -[2023-10-14 15:04:16,672][75949] Updated weights for policy 0, policy_version 39741 (0.0010) -[2023-10-14 15:04:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 81297408. Throughput: 0: 1688.3, 1: 1657.5. Samples: 20327534. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) -[2023-10-14 15:04:18,164][74987] Avg episode reward: [(0, '24.720'), (1, '28.770')] -[2023-10-14 15:04:19,982][75950] Updated weights for policy 1, policy_version 39650 (0.0009) -[2023-10-14 15:04:20,345][75950] Updated weights for policy 1, policy_version 39660 (0.0008) -[2023-10-14 15:04:20,700][75949] Updated weights for policy 0, policy_version 39751 (0.0009) -[2023-10-14 15:04:20,720][75950] Updated weights for policy 1, policy_version 39670 (0.0008) -[2023-10-14 15:04:21,074][75949] Updated weights for policy 0, policy_version 39761 (0.0009) -[2023-10-14 15:04:21,079][75950] Updated weights for policy 1, policy_version 39680 (0.0008) -[2023-10-14 15:04:21,443][75949] Updated weights for policy 0, policy_version 39771 (0.0011) -[2023-10-14 15:04:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 81362944. Throughput: 0: 1668.5, 1: 1665.3. Samples: 20346426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:04:23,165][74987] Avg episode reward: [(0, '26.590'), (1, '30.710')] -[2023-10-14 15:04:25,117][75950] Updated weights for policy 1, policy_version 39690 (0.0007) -[2023-10-14 15:04:25,480][75950] Updated weights for policy 1, policy_version 39700 (0.0008) -[2023-10-14 15:04:25,545][75949] Updated weights for policy 0, policy_version 39781 (0.0007) -[2023-10-14 15:04:25,848][75950] Updated weights for policy 1, policy_version 39710 (0.0007) -[2023-10-14 15:04:25,913][75949] Updated weights for policy 0, policy_version 39791 (0.0007) -[2023-10-14 15:04:26,288][75949] Updated weights for policy 0, policy_version 39801 (0.0009) -[2023-10-14 15:04:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 81428480. Throughput: 0: 1687.5, 1: 1666.7. Samples: 20367240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:04:28,165][74987] Avg episode reward: [(0, '25.020'), (1, '30.240')] -[2023-10-14 15:04:29,853][75950] Updated weights for policy 1, policy_version 39720 (0.0010) -[2023-10-14 15:04:30,223][75950] Updated weights for policy 1, policy_version 39730 (0.0012) -[2023-10-14 15:04:30,297][75949] Updated weights for policy 0, policy_version 39811 (0.0008) -[2023-10-14 15:04:30,590][75950] Updated weights for policy 1, policy_version 39740 (0.0007) -[2023-10-14 15:04:30,673][75949] Updated weights for policy 0, policy_version 39821 (0.0009) -[2023-10-14 15:04:31,040][75949] Updated weights for policy 0, policy_version 39831 (0.0010) -[2023-10-14 15:04:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 81494016. Throughput: 0: 1670.7, 1: 1656.9. Samples: 20377246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:04:33,164][74987] Avg episode reward: [(0, '26.120'), (1, '28.240')] -[2023-10-14 15:04:34,602][75950] Updated weights for policy 1, policy_version 39750 (0.0008) -[2023-10-14 15:04:34,976][75950] Updated weights for policy 1, policy_version 39760 (0.0009) -[2023-10-14 15:04:35,043][75949] Updated weights for policy 0, policy_version 39841 (0.0010) -[2023-10-14 15:04:35,339][75950] Updated weights for policy 1, policy_version 39770 (0.0009) -[2023-10-14 15:04:35,412][75949] Updated weights for policy 0, policy_version 39851 (0.0008) -[2023-10-14 15:04:35,782][75949] Updated weights for policy 0, policy_version 39861 (0.0008) -[2023-10-14 15:04:36,148][75949] Updated weights for policy 0, policy_version 39871 (0.0010) -[2023-10-14 15:04:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 81559552. Throughput: 0: 1664.5, 1: 1671.0. Samples: 20396850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:04:38,164][74987] Avg episode reward: [(0, '23.790'), (1, '28.800')] -[2023-10-14 15:04:39,504][75950] Updated weights for policy 1, policy_version 39780 (0.0007) -[2023-10-14 15:04:39,857][75950] Updated weights for policy 1, policy_version 39790 (0.0007) -[2023-10-14 15:04:40,219][75950] Updated weights for policy 1, policy_version 39800 (0.0008) -[2023-10-14 15:04:40,238][75949] Updated weights for policy 0, policy_version 39881 (0.0010) -[2023-10-14 15:04:40,617][75949] Updated weights for policy 0, policy_version 39891 (0.0008) -[2023-10-14 15:04:40,992][75949] Updated weights for policy 0, policy_version 39901 (0.0008) -[2023-10-14 15:04:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 81625088. Throughput: 0: 1680.8, 1: 1666.8. Samples: 20417460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:04:43,164][74987] Avg episode reward: [(0, '25.230'), (1, '29.660')] -[2023-10-14 15:04:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000039808_40763392.pth... -[2023-10-14 15:04:43,174][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000039904_40861696.pth... -[2023-10-14 15:04:43,226][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000038336_39256064.pth -[2023-10-14 15:04:43,226][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000038240_39157760.pth -[2023-10-14 15:04:44,388][75950] Updated weights for policy 1, policy_version 39810 (0.0009) -[2023-10-14 15:04:44,749][75950] Updated weights for policy 1, policy_version 39820 (0.0010) -[2023-10-14 15:04:45,012][75949] Updated weights for policy 0, policy_version 39911 (0.0010) -[2023-10-14 15:04:45,122][75950] Updated weights for policy 1, policy_version 39830 (0.0009) -[2023-10-14 15:04:45,381][75949] Updated weights for policy 0, policy_version 39921 (0.0010) -[2023-10-14 15:04:45,484][75950] Updated weights for policy 1, policy_version 39840 (0.0008) -[2023-10-14 15:04:45,749][75949] Updated weights for policy 0, policy_version 39931 (0.0009) -[2023-10-14 15:04:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 81690624. Throughput: 0: 1658.1, 1: 1656.6. Samples: 20427010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:04:48,164][74987] Avg episode reward: [(0, '23.250'), (1, '29.480')] -[2023-10-14 15:04:49,676][75950] Updated weights for policy 1, policy_version 39850 (0.0007) -[2023-10-14 15:04:49,867][75949] Updated weights for policy 0, policy_version 39941 (0.0009) -[2023-10-14 15:04:50,040][75950] Updated weights for policy 1, policy_version 39860 (0.0007) -[2023-10-14 15:04:50,239][75949] Updated weights for policy 0, policy_version 39951 (0.0008) -[2023-10-14 15:04:50,414][75950] Updated weights for policy 1, policy_version 39870 (0.0007) -[2023-10-14 15:04:50,607][75949] Updated weights for policy 0, policy_version 39961 (0.0009) -[2023-10-14 15:04:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 81756160. Throughput: 0: 1674.8, 1: 1672.1. Samples: 20447244. Policy #0 lag: (min: 3.0, avg: 4.7, max: 31.0) -[2023-10-14 15:04:53,165][74987] Avg episode reward: [(0, '25.830'), (1, '30.270')] -[2023-10-14 15:04:54,464][75950] Updated weights for policy 1, policy_version 39880 (0.0007) -[2023-10-14 15:04:54,731][75949] Updated weights for policy 0, policy_version 39971 (0.0010) -[2023-10-14 15:04:54,844][75950] Updated weights for policy 1, policy_version 39890 (0.0007) -[2023-10-14 15:04:55,140][75949] Updated weights for policy 0, policy_version 39981 (0.0009) -[2023-10-14 15:04:55,215][75950] Updated weights for policy 1, policy_version 39900 (0.0008) -[2023-10-14 15:04:55,515][75949] Updated weights for policy 0, policy_version 39991 (0.0008) -[2023-10-14 15:04:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 81821696. Throughput: 0: 1670.5, 1: 1674.9. Samples: 20467448. Policy #0 lag: (min: 3.0, avg: 4.7, max: 31.0) -[2023-10-14 15:04:58,164][74987] Avg episode reward: [(0, '24.830'), (1, '31.200')] -[2023-10-14 15:04:59,184][75950] Updated weights for policy 1, policy_version 39910 (0.0008) -[2023-10-14 15:04:59,544][75950] Updated weights for policy 1, policy_version 39920 (0.0008) -[2023-10-14 15:04:59,719][75949] Updated weights for policy 0, policy_version 40001 (0.0010) -[2023-10-14 15:04:59,917][75950] Updated weights for policy 1, policy_version 39930 (0.0007) -[2023-10-14 15:05:00,091][75949] Updated weights for policy 0, policy_version 40011 (0.0008) -[2023-10-14 15:05:00,452][75949] Updated weights for policy 0, policy_version 40021 (0.0008) -[2023-10-14 15:05:00,819][75949] Updated weights for policy 0, policy_version 40031 (0.0008) -[2023-10-14 15:05:03,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 81887232. Throughput: 0: 1652.2, 1: 1666.0. Samples: 20476854. Policy #0 lag: (min: 3.0, avg: 4.7, max: 31.0) -[2023-10-14 15:05:03,164][74987] Avg episode reward: [(0, '25.810'), (1, '29.310')] -[2023-10-14 15:05:03,943][75950] Updated weights for policy 1, policy_version 39940 (0.0008) -[2023-10-14 15:05:04,314][75950] Updated weights for policy 1, policy_version 39950 (0.0008) -[2023-10-14 15:05:04,677][75950] Updated weights for policy 1, policy_version 39960 (0.0007) -[2023-10-14 15:05:04,880][75949] Updated weights for policy 0, policy_version 40041 (0.0007) -[2023-10-14 15:05:05,257][75949] Updated weights for policy 0, policy_version 40051 (0.0007) -[2023-10-14 15:05:05,633][75949] Updated weights for policy 0, policy_version 40061 (0.0010) -[2023-10-14 15:05:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 81952768. Throughput: 0: 1674.2, 1: 1676.3. Samples: 20497198. Policy #0 lag: (min: 3.0, avg: 4.7, max: 31.0) -[2023-10-14 15:05:08,164][74987] Avg episode reward: [(0, '25.170'), (1, '28.600')] -[2023-10-14 15:05:08,808][75950] Updated weights for policy 1, policy_version 39970 (0.0008) -[2023-10-14 15:05:09,173][75950] Updated weights for policy 1, policy_version 39980 (0.0009) -[2023-10-14 15:05:09,538][75950] Updated weights for policy 1, policy_version 39990 (0.0008) -[2023-10-14 15:05:09,763][75949] Updated weights for policy 0, policy_version 40071 (0.0007) -[2023-10-14 15:05:09,902][75950] Updated weights for policy 1, policy_version 40000 (0.0007) -[2023-10-14 15:05:10,130][75949] Updated weights for policy 0, policy_version 40081 (0.0007) -[2023-10-14 15:05:10,505][75949] Updated weights for policy 0, policy_version 40091 (0.0008) -[2023-10-14 15:05:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82018304. Throughput: 0: 1670.4, 1: 1678.3. Samples: 20517934. Policy #0 lag: (min: 3.0, avg: 4.7, max: 31.0) -[2023-10-14 15:05:13,164][74987] Avg episode reward: [(0, '24.920'), (1, '29.520')] -[2023-10-14 15:05:13,883][75950] Updated weights for policy 1, policy_version 40010 (0.0009) -[2023-10-14 15:05:14,251][75950] Updated weights for policy 1, policy_version 40020 (0.0008) -[2023-10-14 15:05:14,556][75949] Updated weights for policy 0, policy_version 40101 (0.0008) -[2023-10-14 15:05:14,606][75950] Updated weights for policy 1, policy_version 40030 (0.0009) -[2023-10-14 15:05:14,925][75949] Updated weights for policy 0, policy_version 40111 (0.0008) -[2023-10-14 15:05:15,299][75949] Updated weights for policy 0, policy_version 40121 (0.0009) -[2023-10-14 15:05:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 82083840. Throughput: 0: 1658.4, 1: 1672.1. Samples: 20527122. Policy #0 lag: (min: 3.0, avg: 4.7, max: 31.0) -[2023-10-14 15:05:18,165][74987] Avg episode reward: [(0, '26.620'), (1, '28.500')] -[2023-10-14 15:05:18,714][75950] Updated weights for policy 1, policy_version 40040 (0.0010) -[2023-10-14 15:05:19,075][75950] Updated weights for policy 1, policy_version 40050 (0.0008) -[2023-10-14 15:05:19,251][75949] Updated weights for policy 0, policy_version 40131 (0.0009) -[2023-10-14 15:05:19,443][75950] Updated weights for policy 1, policy_version 40060 (0.0007) -[2023-10-14 15:05:19,617][75949] Updated weights for policy 0, policy_version 40141 (0.0010) -[2023-10-14 15:05:19,986][75949] Updated weights for policy 0, policy_version 40151 (0.0009) -[2023-10-14 15:05:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82149376. Throughput: 0: 1679.1, 1: 1672.2. Samples: 20547662. Policy #0 lag: (min: 20.0, avg: 24.4, max: 52.0) -[2023-10-14 15:05:23,165][74987] Avg episode reward: [(0, '24.740'), (1, '28.890')] -[2023-10-14 15:05:23,714][75950] Updated weights for policy 1, policy_version 40070 (0.0009) -[2023-10-14 15:05:24,078][75950] Updated weights for policy 1, policy_version 40080 (0.0008) -[2023-10-14 15:05:24,098][75949] Updated weights for policy 0, policy_version 40161 (0.0009) -[2023-10-14 15:05:24,445][75950] Updated weights for policy 1, policy_version 40090 (0.0007) -[2023-10-14 15:05:24,467][75949] Updated weights for policy 0, policy_version 40171 (0.0007) -[2023-10-14 15:05:24,839][75949] Updated weights for policy 0, policy_version 40181 (0.0008) -[2023-10-14 15:05:25,213][75949] Updated weights for policy 0, policy_version 40191 (0.0010) -[2023-10-14 15:05:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 82214912. Throughput: 0: 1677.3, 1: 1674.1. Samples: 20568274. Policy #0 lag: (min: 20.0, avg: 24.4, max: 52.0) -[2023-10-14 15:05:28,165][74987] Avg episode reward: [(0, '24.800'), (1, '31.560')] -[2023-10-14 15:05:28,571][75950] Updated weights for policy 1, policy_version 40100 (0.0009) -[2023-10-14 15:05:28,941][75950] Updated weights for policy 1, policy_version 40110 (0.0009) -[2023-10-14 15:05:29,315][75950] Updated weights for policy 1, policy_version 40120 (0.0008) -[2023-10-14 15:05:29,321][75949] Updated weights for policy 0, policy_version 40201 (0.0008) -[2023-10-14 15:05:29,694][75949] Updated weights for policy 0, policy_version 40211 (0.0009) -[2023-10-14 15:05:30,066][75949] Updated weights for policy 0, policy_version 40221 (0.0008) -[2023-10-14 15:05:33,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82280448. Throughput: 0: 1666.4, 1: 1677.8. Samples: 20577498. Policy #0 lag: (min: 20.0, avg: 24.4, max: 52.0) -[2023-10-14 15:05:33,164][74987] Avg episode reward: [(0, '25.620'), (1, '31.830')] -[2023-10-14 15:05:33,361][75950] Updated weights for policy 1, policy_version 40130 (0.0008) -[2023-10-14 15:05:33,739][75950] Updated weights for policy 1, policy_version 40140 (0.0009) -[2023-10-14 15:05:34,089][75949] Updated weights for policy 0, policy_version 40231 (0.0008) -[2023-10-14 15:05:34,106][75950] Updated weights for policy 1, policy_version 40150 (0.0011) -[2023-10-14 15:05:34,466][75949] Updated weights for policy 0, policy_version 40241 (0.0008) -[2023-10-14 15:05:34,469][75950] Updated weights for policy 1, policy_version 40160 (0.0009) -[2023-10-14 15:05:34,829][75949] Updated weights for policy 0, policy_version 40251 (0.0009) -[2023-10-14 15:05:38,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82345984. Throughput: 0: 1670.6, 1: 1677.3. Samples: 20597900. Policy #0 lag: (min: 20.0, avg: 24.4, max: 52.0) -[2023-10-14 15:05:38,164][74987] Avg episode reward: [(0, '25.910'), (1, '28.490')] -[2023-10-14 15:05:38,509][75950] Updated weights for policy 1, policy_version 40170 (0.0007) -[2023-10-14 15:05:38,868][75950] Updated weights for policy 1, policy_version 40180 (0.0010) -[2023-10-14 15:05:38,991][75949] Updated weights for policy 0, policy_version 40261 (0.0008) -[2023-10-14 15:05:39,231][75950] Updated weights for policy 1, policy_version 40190 (0.0009) -[2023-10-14 15:05:39,371][75949] Updated weights for policy 0, policy_version 40271 (0.0009) -[2023-10-14 15:05:39,739][75949] Updated weights for policy 0, policy_version 40281 (0.0009) -[2023-10-14 15:05:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 82411520. Throughput: 0: 1678.8, 1: 1683.5. Samples: 20618752. Policy #0 lag: (min: 20.0, avg: 24.4, max: 52.0) -[2023-10-14 15:05:43,165][74987] Avg episode reward: [(0, '25.190'), (1, '31.710')] -[2023-10-14 15:05:43,456][75950] Updated weights for policy 1, policy_version 40200 (0.0008) -[2023-10-14 15:05:43,789][75949] Updated weights for policy 0, policy_version 40291 (0.0010) -[2023-10-14 15:05:43,824][75950] Updated weights for policy 1, policy_version 40210 (0.0008) -[2023-10-14 15:05:44,155][75949] Updated weights for policy 0, policy_version 40301 (0.0009) -[2023-10-14 15:05:44,198][75950] Updated weights for policy 1, policy_version 40220 (0.0009) -[2023-10-14 15:05:44,522][75949] Updated weights for policy 0, policy_version 40311 (0.0010) -[2023-10-14 15:05:48,141][75950] Updated weights for policy 1, policy_version 40230 (0.0007) -[2023-10-14 15:05:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82477056. Throughput: 0: 1674.8, 1: 1682.4. Samples: 20627928. Policy #0 lag: (min: 20.0, avg: 24.4, max: 52.0) -[2023-10-14 15:05:48,164][74987] Avg episode reward: [(0, '26.730'), (1, '31.380')] -[2023-10-14 15:05:48,505][75950] Updated weights for policy 1, policy_version 40240 (0.0010) -[2023-10-14 15:05:48,529][75949] Updated weights for policy 0, policy_version 40321 (0.0010) -[2023-10-14 15:05:48,866][75950] Updated weights for policy 1, policy_version 40250 (0.0007) -[2023-10-14 15:05:48,899][75949] Updated weights for policy 0, policy_version 40331 (0.0009) -[2023-10-14 15:05:49,270][75949] Updated weights for policy 0, policy_version 40341 (0.0009) -[2023-10-14 15:05:49,642][75949] Updated weights for policy 0, policy_version 40351 (0.0009) -[2023-10-14 15:05:52,900][75950] Updated weights for policy 1, policy_version 40260 (0.0009) -[2023-10-14 15:05:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82542592. Throughput: 0: 1680.4, 1: 1685.7. Samples: 20648674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:05:53,165][74987] Avg episode reward: [(0, '23.600'), (1, '30.120')] -[2023-10-14 15:05:53,267][75950] Updated weights for policy 1, policy_version 40270 (0.0010) -[2023-10-14 15:05:53,637][75950] Updated weights for policy 1, policy_version 40280 (0.0008) -[2023-10-14 15:05:53,651][75949] Updated weights for policy 0, policy_version 40361 (0.0008) -[2023-10-14 15:05:54,024][75949] Updated weights for policy 0, policy_version 40371 (0.0007) -[2023-10-14 15:05:54,398][75949] Updated weights for policy 0, policy_version 40381 (0.0008) -[2023-10-14 15:05:57,552][75950] Updated weights for policy 1, policy_version 40290 (0.0009) -[2023-10-14 15:05:57,918][75950] Updated weights for policy 1, policy_version 40300 (0.0008) -[2023-10-14 15:05:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82608128. Throughput: 0: 1684.1, 1: 1678.9. Samples: 20669270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:05:58,164][74987] Avg episode reward: [(0, '25.840'), (1, '30.220')] -[2023-10-14 15:05:58,290][75950] Updated weights for policy 1, policy_version 40310 (0.0009) -[2023-10-14 15:05:58,508][75949] Updated weights for policy 0, policy_version 40391 (0.0007) -[2023-10-14 15:05:58,658][75950] Updated weights for policy 1, policy_version 40320 (0.0007) -[2023-10-14 15:05:58,884][75949] Updated weights for policy 0, policy_version 40401 (0.0007) -[2023-10-14 15:05:59,252][75949] Updated weights for policy 0, policy_version 40411 (0.0007) -[2023-10-14 15:06:02,860][75950] Updated weights for policy 1, policy_version 40330 (0.0008) -[2023-10-14 15:06:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82673664. Throughput: 0: 1680.1, 1: 1682.2. Samples: 20678426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:03,164][74987] Avg episode reward: [(0, '24.450'), (1, '31.020')] -[2023-10-14 15:06:03,188][75949] Updated weights for policy 0, policy_version 40421 (0.0008) -[2023-10-14 15:06:03,235][75950] Updated weights for policy 1, policy_version 40340 (0.0008) -[2023-10-14 15:06:03,559][75949] Updated weights for policy 0, policy_version 40431 (0.0008) -[2023-10-14 15:06:03,602][75950] Updated weights for policy 1, policy_version 40350 (0.0009) -[2023-10-14 15:06:03,929][75949] Updated weights for policy 0, policy_version 40441 (0.0010) -[2023-10-14 15:06:07,763][75950] Updated weights for policy 1, policy_version 40360 (0.0010) -[2023-10-14 15:06:08,000][75949] Updated weights for policy 0, policy_version 40451 (0.0010) -[2023-10-14 15:06:08,140][75950] Updated weights for policy 1, policy_version 40370 (0.0009) -[2023-10-14 15:06:08,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82739200. Throughput: 0: 1683.4, 1: 1680.5. Samples: 20699040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:08,164][74987] Avg episode reward: [(0, '26.230'), (1, '30.020')] -[2023-10-14 15:06:08,358][75949] Updated weights for policy 0, policy_version 40461 (0.0009) -[2023-10-14 15:06:08,495][75950] Updated weights for policy 1, policy_version 40380 (0.0008) -[2023-10-14 15:06:08,728][75949] Updated weights for policy 0, policy_version 40471 (0.0009) -[2023-10-14 15:06:12,685][75950] Updated weights for policy 1, policy_version 40390 (0.0008) -[2023-10-14 15:06:12,757][75949] Updated weights for policy 0, policy_version 40481 (0.0010) -[2023-10-14 15:06:13,048][75950] Updated weights for policy 1, policy_version 40400 (0.0009) -[2023-10-14 15:06:13,120][75949] Updated weights for policy 0, policy_version 40491 (0.0007) -[2023-10-14 15:06:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82804736. Throughput: 0: 1685.6, 1: 1678.5. Samples: 20719660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:13,165][74987] Avg episode reward: [(0, '24.810'), (1, '28.980')] -[2023-10-14 15:06:13,420][75950] Updated weights for policy 1, policy_version 40410 (0.0009) -[2023-10-14 15:06:13,491][75949] Updated weights for policy 0, policy_version 40501 (0.0008) -[2023-10-14 15:06:13,862][75949] Updated weights for policy 0, policy_version 40511 (0.0009) -[2023-10-14 15:06:17,635][75950] Updated weights for policy 1, policy_version 40420 (0.0007) -[2023-10-14 15:06:17,846][75949] Updated weights for policy 0, policy_version 40521 (0.0008) -[2023-10-14 15:06:18,009][75950] Updated weights for policy 1, policy_version 40430 (0.0007) -[2023-10-14 15:06:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82870272. Throughput: 0: 1687.8, 1: 1674.7. Samples: 20728810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:18,164][74987] Avg episode reward: [(0, '25.450'), (1, '32.630')] -[2023-10-14 15:06:18,217][75949] Updated weights for policy 0, policy_version 40531 (0.0009) -[2023-10-14 15:06:18,367][75950] Updated weights for policy 1, policy_version 40440 (0.0007) -[2023-10-14 15:06:18,588][75949] Updated weights for policy 0, policy_version 40541 (0.0008) -[2023-10-14 15:06:18,651][75801] Saving new best policy, reward=32.630! -[2023-10-14 15:06:22,611][75950] Updated weights for policy 1, policy_version 40450 (0.0007) -[2023-10-14 15:06:22,755][75949] Updated weights for policy 0, policy_version 40551 (0.0008) -[2023-10-14 15:06:22,977][75950] Updated weights for policy 1, policy_version 40460 (0.0009) -[2023-10-14 15:06:23,131][75949] Updated weights for policy 0, policy_version 40561 (0.0007) -[2023-10-14 15:06:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 82935808. Throughput: 0: 1691.6, 1: 1669.8. Samples: 20749164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:23,164][74987] Avg episode reward: [(0, '25.470'), (1, '30.810')] -[2023-10-14 15:06:23,340][75950] Updated weights for policy 1, policy_version 40470 (0.0008) -[2023-10-14 15:06:23,490][75949] Updated weights for policy 0, policy_version 40571 (0.0008) -[2023-10-14 15:06:23,706][75950] Updated weights for policy 1, policy_version 40480 (0.0009) -[2023-10-14 15:06:27,577][75949] Updated weights for policy 0, policy_version 40581 (0.0008) -[2023-10-14 15:06:27,761][75950] Updated weights for policy 1, policy_version 40490 (0.0009) -[2023-10-14 15:06:27,938][75949] Updated weights for policy 0, policy_version 40591 (0.0008) -[2023-10-14 15:06:28,117][75950] Updated weights for policy 1, policy_version 40500 (0.0010) -[2023-10-14 15:06:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83001344. Throughput: 0: 1684.9, 1: 1658.7. Samples: 20769214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:28,164][74987] Avg episode reward: [(0, '25.810'), (1, '32.240')] -[2023-10-14 15:06:28,310][75949] Updated weights for policy 0, policy_version 40601 (0.0009) -[2023-10-14 15:06:28,495][75950] Updated weights for policy 1, policy_version 40510 (0.0008) -[2023-10-14 15:06:32,516][75950] Updated weights for policy 1, policy_version 40520 (0.0009) -[2023-10-14 15:06:32,528][75949] Updated weights for policy 0, policy_version 40611 (0.0008) -[2023-10-14 15:06:32,882][75950] Updated weights for policy 1, policy_version 40530 (0.0008) -[2023-10-14 15:06:32,925][75949] Updated weights for policy 0, policy_version 40621 (0.0009) -[2023-10-14 15:06:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 83066880. Throughput: 0: 1687.7, 1: 1666.3. Samples: 20778860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:33,165][74987] Avg episode reward: [(0, '26.370'), (1, '31.860')] -[2023-10-14 15:06:33,251][75950] Updated weights for policy 1, policy_version 40540 (0.0008) -[2023-10-14 15:06:33,301][75949] Updated weights for policy 0, policy_version 40631 (0.0007) -[2023-10-14 15:06:37,353][75949] Updated weights for policy 0, policy_version 40641 (0.0008) -[2023-10-14 15:06:37,504][75950] Updated weights for policy 1, policy_version 40550 (0.0008) -[2023-10-14 15:06:37,722][75949] Updated weights for policy 0, policy_version 40651 (0.0008) -[2023-10-14 15:06:37,875][75950] Updated weights for policy 1, policy_version 40560 (0.0008) -[2023-10-14 15:06:38,098][75949] Updated weights for policy 0, policy_version 40661 (0.0007) -[2023-10-14 15:06:38,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83132416. Throughput: 0: 1678.6, 1: 1657.7. Samples: 20798806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:38,164][74987] Avg episode reward: [(0, '24.670'), (1, '31.270')] -[2023-10-14 15:06:38,239][75950] Updated weights for policy 1, policy_version 40570 (0.0008) -[2023-10-14 15:06:38,473][75949] Updated weights for policy 0, policy_version 40671 (0.0007) -[2023-10-14 15:06:42,410][75950] Updated weights for policy 1, policy_version 40580 (0.0008) -[2023-10-14 15:06:42,617][75949] Updated weights for policy 0, policy_version 40681 (0.0008) -[2023-10-14 15:06:42,778][75950] Updated weights for policy 1, policy_version 40590 (0.0010) -[2023-10-14 15:06:42,988][75949] Updated weights for policy 0, policy_version 40691 (0.0008) -[2023-10-14 15:06:43,144][75950] Updated weights for policy 1, policy_version 40600 (0.0009) -[2023-10-14 15:06:43,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 83197952. Throughput: 0: 1666.5, 1: 1650.6. Samples: 20818540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:43,164][74987] Avg episode reward: [(0, '26.500'), (1, '30.880')] -[2023-10-14 15:06:43,351][75949] Updated weights for policy 0, policy_version 40701 (0.0008) -[2023-10-14 15:06:43,434][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000040608_41582592.pth... -[2023-10-14 15:06:43,462][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000040704_41680896.pth... -[2023-10-14 15:06:43,463][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000039040_39976960.pth -[2023-10-14 15:06:43,491][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000039136_40075264.pth -[2023-10-14 15:06:47,387][75950] Updated weights for policy 1, policy_version 40610 (0.0008) -[2023-10-14 15:06:47,636][75949] Updated weights for policy 0, policy_version 40711 (0.0008) -[2023-10-14 15:06:47,757][75950] Updated weights for policy 1, policy_version 40620 (0.0007) -[2023-10-14 15:06:48,004][75949] Updated weights for policy 0, policy_version 40721 (0.0007) -[2023-10-14 15:06:48,119][75950] Updated weights for policy 1, policy_version 40630 (0.0007) -[2023-10-14 15:06:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 83263488. Throughput: 0: 1676.1, 1: 1654.1. Samples: 20828286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:48,165][74987] Avg episode reward: [(0, '24.180'), (1, '30.470')] -[2023-10-14 15:06:48,381][75949] Updated weights for policy 0, policy_version 40731 (0.0009) -[2023-10-14 15:06:48,478][75950] Updated weights for policy 1, policy_version 40640 (0.0009) -[2023-10-14 15:06:52,457][75950] Updated weights for policy 1, policy_version 40650 (0.0008) -[2023-10-14 15:06:52,469][75949] Updated weights for policy 0, policy_version 40741 (0.0008) -[2023-10-14 15:06:52,822][75950] Updated weights for policy 1, policy_version 40660 (0.0008) -[2023-10-14 15:06:52,829][75949] Updated weights for policy 0, policy_version 40751 (0.0007) -[2023-10-14 15:06:53,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 83329024. Throughput: 0: 1672.4, 1: 1658.0. Samples: 20848904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:53,164][74987] Avg episode reward: [(0, '27.310'), (1, '30.120')] -[2023-10-14 15:06:53,194][75950] Updated weights for policy 1, policy_version 40670 (0.0008) -[2023-10-14 15:06:53,205][75949] Updated weights for policy 0, policy_version 40761 (0.0009) -[2023-10-14 15:06:57,225][75950] Updated weights for policy 1, policy_version 40680 (0.0009) -[2023-10-14 15:06:57,426][75949] Updated weights for policy 0, policy_version 40771 (0.0009) -[2023-10-14 15:06:57,590][75950] Updated weights for policy 1, policy_version 40690 (0.0008) -[2023-10-14 15:06:57,792][75949] Updated weights for policy 0, policy_version 40781 (0.0009) -[2023-10-14 15:06:57,956][75950] Updated weights for policy 1, policy_version 40700 (0.0009) -[2023-10-14 15:06:58,159][75949] Updated weights for policy 0, policy_version 40791 (0.0010) -[2023-10-14 15:06:58,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 83427328. Throughput: 0: 1658.4, 1: 1642.2. Samples: 20868190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:06:58,165][74987] Avg episode reward: [(0, '23.470'), (1, '29.960')] -[2023-10-14 15:07:02,209][75950] Updated weights for policy 1, policy_version 40710 (0.0008) -[2023-10-14 15:07:02,324][75949] Updated weights for policy 0, policy_version 40801 (0.0011) -[2023-10-14 15:07:02,574][75950] Updated weights for policy 1, policy_version 40720 (0.0007) -[2023-10-14 15:07:02,698][75949] Updated weights for policy 0, policy_version 40811 (0.0007) -[2023-10-14 15:07:02,947][75950] Updated weights for policy 1, policy_version 40730 (0.0007) -[2023-10-14 15:07:03,080][75949] Updated weights for policy 0, policy_version 40821 (0.0009) -[2023-10-14 15:07:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83460096. Throughput: 0: 1661.2, 1: 1659.5. Samples: 20878240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:07:03,164][74987] Avg episode reward: [(0, '26.080'), (1, '30.050')] -[2023-10-14 15:07:03,446][75949] Updated weights for policy 0, policy_version 40831 (0.0008) -[2023-10-14 15:07:07,007][75950] Updated weights for policy 1, policy_version 40740 (0.0009) -[2023-10-14 15:07:07,379][75950] Updated weights for policy 1, policy_version 40750 (0.0007) -[2023-10-14 15:07:07,523][75949] Updated weights for policy 0, policy_version 40841 (0.0008) -[2023-10-14 15:07:07,743][75950] Updated weights for policy 1, policy_version 40760 (0.0007) -[2023-10-14 15:07:07,889][75949] Updated weights for policy 0, policy_version 40851 (0.0010) -[2023-10-14 15:07:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 83558400. Throughput: 0: 1658.1, 1: 1667.5. Samples: 20898816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:07:08,164][74987] Avg episode reward: [(0, '24.740'), (1, '30.280')] -[2023-10-14 15:07:08,266][75949] Updated weights for policy 0, policy_version 40861 (0.0009) -[2023-10-14 15:07:11,736][75950] Updated weights for policy 1, policy_version 40770 (0.0007) -[2023-10-14 15:07:12,104][75950] Updated weights for policy 1, policy_version 40780 (0.0007) -[2023-10-14 15:07:12,383][75949] Updated weights for policy 0, policy_version 40871 (0.0009) -[2023-10-14 15:07:12,479][75950] Updated weights for policy 1, policy_version 40790 (0.0008) -[2023-10-14 15:07:12,741][75949] Updated weights for policy 0, policy_version 40881 (0.0010) -[2023-10-14 15:07:12,842][75950] Updated weights for policy 1, policy_version 40800 (0.0008) -[2023-10-14 15:07:13,115][75949] Updated weights for policy 0, policy_version 40891 (0.0009) -[2023-10-14 15:07:13,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 83623936. Throughput: 0: 1648.8, 1: 1654.3. Samples: 20917858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:07:13,165][74987] Avg episode reward: [(0, '26.240'), (1, '29.190')] -[2023-10-14 15:07:16,957][75950] Updated weights for policy 1, policy_version 40810 (0.0007) -[2023-10-14 15:07:17,324][75949] Updated weights for policy 0, policy_version 40901 (0.0008) -[2023-10-14 15:07:17,333][75950] Updated weights for policy 1, policy_version 40820 (0.0007) -[2023-10-14 15:07:17,703][75950] Updated weights for policy 1, policy_version 40830 (0.0007) -[2023-10-14 15:07:17,720][75949] Updated weights for policy 0, policy_version 40911 (0.0007) -[2023-10-14 15:07:18,091][75949] Updated weights for policy 0, policy_version 40921 (0.0007) -[2023-10-14 15:07:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 83689472. Throughput: 0: 1654.6, 1: 1668.7. Samples: 20928408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:07:18,165][74987] Avg episode reward: [(0, '25.440'), (1, '29.950')] -[2023-10-14 15:07:21,943][75950] Updated weights for policy 1, policy_version 40840 (0.0007) -[2023-10-14 15:07:22,071][75949] Updated weights for policy 0, policy_version 40931 (0.0008) -[2023-10-14 15:07:22,315][75950] Updated weights for policy 1, policy_version 40850 (0.0008) -[2023-10-14 15:07:22,437][75949] Updated weights for policy 0, policy_version 40941 (0.0008) -[2023-10-14 15:07:22,672][75950] Updated weights for policy 1, policy_version 40860 (0.0009) -[2023-10-14 15:07:22,815][75949] Updated weights for policy 0, policy_version 40951 (0.0008) -[2023-10-14 15:07:23,163][74987] Fps is (10 sec: 16384.5, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 83787776. Throughput: 0: 1660.4, 1: 1670.3. Samples: 20948688. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) -[2023-10-14 15:07:23,164][74987] Avg episode reward: [(0, '24.110'), (1, '31.060')] -[2023-10-14 15:07:26,628][75950] Updated weights for policy 1, policy_version 40870 (0.0009) -[2023-10-14 15:07:26,911][75949] Updated weights for policy 0, policy_version 40961 (0.0007) -[2023-10-14 15:07:26,990][75950] Updated weights for policy 1, policy_version 40880 (0.0011) -[2023-10-14 15:07:27,283][75949] Updated weights for policy 0, policy_version 40971 (0.0008) -[2023-10-14 15:07:27,364][75950] Updated weights for policy 1, policy_version 40890 (0.0009) -[2023-10-14 15:07:27,647][75949] Updated weights for policy 0, policy_version 40981 (0.0008) -[2023-10-14 15:07:28,022][75949] Updated weights for policy 0, policy_version 40991 (0.0010) -[2023-10-14 15:07:28,164][74987] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 83853312. Throughput: 0: 1648.6, 1: 1660.8. Samples: 20967466. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) -[2023-10-14 15:07:28,165][74987] Avg episode reward: [(0, '25.880'), (1, '29.420')] -[2023-10-14 15:07:31,258][75950] Updated weights for policy 1, policy_version 40900 (0.0009) -[2023-10-14 15:07:31,614][75950] Updated weights for policy 1, policy_version 40910 (0.0009) -[2023-10-14 15:07:31,981][75950] Updated weights for policy 1, policy_version 40920 (0.0010) -[2023-10-14 15:07:32,199][75949] Updated weights for policy 0, policy_version 41001 (0.0010) -[2023-10-14 15:07:32,573][75949] Updated weights for policy 0, policy_version 41011 (0.0008) -[2023-10-14 15:07:32,952][75949] Updated weights for policy 0, policy_version 41021 (0.0010) -[2023-10-14 15:07:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 83918848. Throughput: 0: 1659.6, 1: 1684.9. Samples: 20978786. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) -[2023-10-14 15:07:33,164][74987] Avg episode reward: [(0, '23.480'), (1, '31.120')] -[2023-10-14 15:07:36,185][75950] Updated weights for policy 1, policy_version 40930 (0.0008) -[2023-10-14 15:07:36,558][75950] Updated weights for policy 1, policy_version 40940 (0.0010) -[2023-10-14 15:07:36,925][75950] Updated weights for policy 1, policy_version 40950 (0.0009) -[2023-10-14 15:07:37,270][75949] Updated weights for policy 0, policy_version 41031 (0.0008) -[2023-10-14 15:07:37,290][75950] Updated weights for policy 1, policy_version 40960 (0.0008) -[2023-10-14 15:07:37,649][75949] Updated weights for policy 0, policy_version 41041 (0.0009) -[2023-10-14 15:07:38,017][75949] Updated weights for policy 0, policy_version 41051 (0.0008) -[2023-10-14 15:07:38,164][74987] Fps is (10 sec: 9830.3, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 83951616. Throughput: 0: 1654.2, 1: 1668.0. Samples: 20998404. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) -[2023-10-14 15:07:38,165][74987] Avg episode reward: [(0, '26.780'), (1, '30.450')] -[2023-10-14 15:07:41,327][75950] Updated weights for policy 1, policy_version 40970 (0.0009) -[2023-10-14 15:07:41,692][75950] Updated weights for policy 1, policy_version 40980 (0.0009) -[2023-10-14 15:07:42,060][75950] Updated weights for policy 1, policy_version 40990 (0.0008) -[2023-10-14 15:07:42,094][75949] Updated weights for policy 0, policy_version 41061 (0.0009) -[2023-10-14 15:07:42,472][75949] Updated weights for policy 0, policy_version 41071 (0.0008) -[2023-10-14 15:07:42,852][75949] Updated weights for policy 0, policy_version 41081 (0.0008) -[2023-10-14 15:07:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 84049920. Throughput: 0: 1649.5, 1: 1667.0. Samples: 21017434. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) -[2023-10-14 15:07:43,164][74987] Avg episode reward: [(0, '24.690'), (1, '30.860')] -[2023-10-14 15:07:46,263][75950] Updated weights for policy 1, policy_version 41000 (0.0009) -[2023-10-14 15:07:46,633][75950] Updated weights for policy 1, policy_version 41010 (0.0008) -[2023-10-14 15:07:46,850][75949] Updated weights for policy 0, policy_version 41091 (0.0008) -[2023-10-14 15:07:46,996][75950] Updated weights for policy 1, policy_version 41020 (0.0009) -[2023-10-14 15:07:47,224][75949] Updated weights for policy 0, policy_version 41101 (0.0009) -[2023-10-14 15:07:47,596][75949] Updated weights for policy 0, policy_version 41111 (0.0009) -[2023-10-14 15:07:48,164][74987] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 84115456. Throughput: 0: 1664.8, 1: 1678.6. Samples: 21028696. Policy #0 lag: (min: 26.0, avg: 46.0, max: 48.0) -[2023-10-14 15:07:48,165][74987] Avg episode reward: [(0, '27.700'), (1, '28.370')] -[2023-10-14 15:07:51,073][75950] Updated weights for policy 1, policy_version 41030 (0.0008) -[2023-10-14 15:07:51,440][75950] Updated weights for policy 1, policy_version 41040 (0.0010) -[2023-10-14 15:07:51,684][75949] Updated weights for policy 0, policy_version 41121 (0.0007) -[2023-10-14 15:07:51,818][75950] Updated weights for policy 1, policy_version 41050 (0.0007) -[2023-10-14 15:07:52,058][75949] Updated weights for policy 0, policy_version 41131 (0.0008) -[2023-10-14 15:07:52,428][75949] Updated weights for policy 0, policy_version 41141 (0.0009) -[2023-10-14 15:07:52,790][75949] Updated weights for policy 0, policy_version 41151 (0.0010) -[2023-10-14 15:07:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 84180992. Throughput: 0: 1665.1, 1: 1662.5. Samples: 21048558. Policy #0 lag: (min: 26.0, avg: 46.0, max: 48.0) -[2023-10-14 15:07:53,165][74987] Avg episode reward: [(0, '24.540'), (1, '29.600')] -[2023-10-14 15:07:56,093][75950] Updated weights for policy 1, policy_version 41060 (0.0008) -[2023-10-14 15:07:56,451][75950] Updated weights for policy 1, policy_version 41070 (0.0008) -[2023-10-14 15:07:56,824][75950] Updated weights for policy 1, policy_version 41080 (0.0008) -[2023-10-14 15:07:56,856][75949] Updated weights for policy 0, policy_version 41161 (0.0008) -[2023-10-14 15:07:57,232][75949] Updated weights for policy 0, policy_version 41171 (0.0008) -[2023-10-14 15:07:57,601][75949] Updated weights for policy 0, policy_version 41181 (0.0010) -[2023-10-14 15:07:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 84246528. Throughput: 0: 1655.1, 1: 1673.8. Samples: 21067660. Policy #0 lag: (min: 26.0, avg: 46.0, max: 48.0) -[2023-10-14 15:07:58,164][74987] Avg episode reward: [(0, '25.470'), (1, '29.300')] -[2023-10-14 15:08:00,843][75950] Updated weights for policy 1, policy_version 41090 (0.0007) -[2023-10-14 15:08:01,207][75950] Updated weights for policy 1, policy_version 41100 (0.0009) -[2023-10-14 15:08:01,463][75949] Updated weights for policy 0, policy_version 41191 (0.0009) -[2023-10-14 15:08:01,564][75950] Updated weights for policy 1, policy_version 41110 (0.0009) -[2023-10-14 15:08:01,839][75949] Updated weights for policy 0, policy_version 41201 (0.0008) -[2023-10-14 15:08:01,938][75950] Updated weights for policy 1, policy_version 41120 (0.0007) -[2023-10-14 15:08:02,200][75949] Updated weights for policy 0, policy_version 41211 (0.0009) -[2023-10-14 15:08:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 84312064. Throughput: 0: 1673.8, 1: 1675.7. Samples: 21079138. Policy #0 lag: (min: 26.0, avg: 46.0, max: 48.0) -[2023-10-14 15:08:03,165][74987] Avg episode reward: [(0, '25.110'), (1, '29.560')] -[2023-10-14 15:08:06,073][75950] Updated weights for policy 1, policy_version 41130 (0.0008) -[2023-10-14 15:08:06,330][75949] Updated weights for policy 0, policy_version 41221 (0.0008) -[2023-10-14 15:08:06,443][75950] Updated weights for policy 1, policy_version 41140 (0.0007) -[2023-10-14 15:08:06,722][75949] Updated weights for policy 0, policy_version 41231 (0.0009) -[2023-10-14 15:08:06,797][75950] Updated weights for policy 1, policy_version 41150 (0.0009) -[2023-10-14 15:08:07,084][75949] Updated weights for policy 0, policy_version 41241 (0.0008) -[2023-10-14 15:08:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 84377600. Throughput: 0: 1662.6, 1: 1657.7. Samples: 21098104. Policy #0 lag: (min: 26.0, avg: 46.0, max: 48.0) -[2023-10-14 15:08:08,165][74987] Avg episode reward: [(0, '24.480'), (1, '29.300')] -[2023-10-14 15:08:10,888][75950] Updated weights for policy 1, policy_version 41160 (0.0008) -[2023-10-14 15:08:11,059][75949] Updated weights for policy 0, policy_version 41251 (0.0007) -[2023-10-14 15:08:11,272][75950] Updated weights for policy 1, policy_version 41170 (0.0009) -[2023-10-14 15:08:11,418][75949] Updated weights for policy 0, policy_version 41261 (0.0008) -[2023-10-14 15:08:11,634][75950] Updated weights for policy 1, policy_version 41180 (0.0008) -[2023-10-14 15:08:11,798][75949] Updated weights for policy 0, policy_version 41271 (0.0009) -[2023-10-14 15:08:13,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 84443136. Throughput: 0: 1668.0, 1: 1664.9. Samples: 21117446. Policy #0 lag: (min: 26.0, avg: 46.0, max: 48.0) -[2023-10-14 15:08:13,164][74987] Avg episode reward: [(0, '25.350'), (1, '29.110')] -[2023-10-14 15:08:15,769][75950] Updated weights for policy 1, policy_version 41190 (0.0007) -[2023-10-14 15:08:15,906][75949] Updated weights for policy 0, policy_version 41281 (0.0011) -[2023-10-14 15:08:16,129][75950] Updated weights for policy 1, policy_version 41200 (0.0007) -[2023-10-14 15:08:16,275][75949] Updated weights for policy 0, policy_version 41291 (0.0007) -[2023-10-14 15:08:16,502][75950] Updated weights for policy 1, policy_version 41210 (0.0008) -[2023-10-14 15:08:16,645][75949] Updated weights for policy 0, policy_version 41301 (0.0009) -[2023-10-14 15:08:17,016][75949] Updated weights for policy 0, policy_version 41311 (0.0009) -[2023-10-14 15:08:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 84508672. Throughput: 0: 1677.4, 1: 1658.8. Samples: 21128918. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) -[2023-10-14 15:08:18,165][74987] Avg episode reward: [(0, '26.500'), (1, '29.240')] -[2023-10-14 15:08:20,750][75950] Updated weights for policy 1, policy_version 41220 (0.0008) -[2023-10-14 15:08:21,118][75950] Updated weights for policy 1, policy_version 41230 (0.0008) -[2023-10-14 15:08:21,251][75949] Updated weights for policy 0, policy_version 41321 (0.0008) -[2023-10-14 15:08:21,493][75950] Updated weights for policy 1, policy_version 41240 (0.0010) -[2023-10-14 15:08:21,614][75949] Updated weights for policy 0, policy_version 41331 (0.0008) -[2023-10-14 15:08:21,979][75949] Updated weights for policy 0, policy_version 41341 (0.0007) -[2023-10-14 15:08:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84574208. Throughput: 0: 1666.5, 1: 1647.9. Samples: 21147548. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) -[2023-10-14 15:08:23,164][74987] Avg episode reward: [(0, '25.830'), (1, '29.470')] -[2023-10-14 15:08:25,539][75950] Updated weights for policy 1, policy_version 41250 (0.0007) -[2023-10-14 15:08:25,904][75950] Updated weights for policy 1, policy_version 41260 (0.0009) -[2023-10-14 15:08:25,929][75949] Updated weights for policy 0, policy_version 41351 (0.0007) -[2023-10-14 15:08:26,271][75950] Updated weights for policy 1, policy_version 41270 (0.0007) -[2023-10-14 15:08:26,292][75949] Updated weights for policy 0, policy_version 41361 (0.0008) -[2023-10-14 15:08:26,635][75950] Updated weights for policy 1, policy_version 41280 (0.0008) -[2023-10-14 15:08:26,659][75949] Updated weights for policy 0, policy_version 41371 (0.0008) -[2023-10-14 15:08:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84639744. Throughput: 0: 1673.5, 1: 1659.3. Samples: 21167412. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) -[2023-10-14 15:08:28,164][74987] Avg episode reward: [(0, '25.330'), (1, '30.850')] -[2023-10-14 15:08:30,618][75949] Updated weights for policy 0, policy_version 41381 (0.0008) -[2023-10-14 15:08:30,918][75950] Updated weights for policy 1, policy_version 41290 (0.0008) -[2023-10-14 15:08:30,983][75949] Updated weights for policy 0, policy_version 41391 (0.0008) -[2023-10-14 15:08:31,289][75950] Updated weights for policy 1, policy_version 41300 (0.0009) -[2023-10-14 15:08:31,348][75949] Updated weights for policy 0, policy_version 41401 (0.0009) -[2023-10-14 15:08:31,646][75950] Updated weights for policy 1, policy_version 41310 (0.0009) -[2023-10-14 15:08:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 84705280. Throughput: 0: 1677.4, 1: 1653.7. Samples: 21178596. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) -[2023-10-14 15:08:33,165][74987] Avg episode reward: [(0, '26.710'), (1, '28.420')] -[2023-10-14 15:08:35,518][75949] Updated weights for policy 0, policy_version 41411 (0.0008) -[2023-10-14 15:08:35,693][75950] Updated weights for policy 1, policy_version 41320 (0.0009) -[2023-10-14 15:08:35,890][75949] Updated weights for policy 0, policy_version 41421 (0.0007) -[2023-10-14 15:08:36,057][75950] Updated weights for policy 1, policy_version 41330 (0.0009) -[2023-10-14 15:08:36,249][75949] Updated weights for policy 0, policy_version 41431 (0.0009) -[2023-10-14 15:08:36,433][75950] Updated weights for policy 1, policy_version 41340 (0.0008) -[2023-10-14 15:08:38,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 84770816. Throughput: 0: 1655.2, 1: 1645.6. Samples: 21197094. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) -[2023-10-14 15:08:38,164][74987] Avg episode reward: [(0, '24.960'), (1, '27.770')] -[2023-10-14 15:08:40,387][75949] Updated weights for policy 0, policy_version 41441 (0.0007) -[2023-10-14 15:08:40,466][75950] Updated weights for policy 1, policy_version 41350 (0.0008) -[2023-10-14 15:08:40,751][75949] Updated weights for policy 0, policy_version 41451 (0.0007) -[2023-10-14 15:08:40,836][75950] Updated weights for policy 1, policy_version 41360 (0.0007) -[2023-10-14 15:08:41,131][75949] Updated weights for policy 0, policy_version 41461 (0.0008) -[2023-10-14 15:08:41,201][75950] Updated weights for policy 1, policy_version 41370 (0.0008) -[2023-10-14 15:08:41,504][75949] Updated weights for policy 0, policy_version 41471 (0.0010) -[2023-10-14 15:08:43,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84836352. Throughput: 0: 1678.0, 1: 1658.7. Samples: 21217812. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) -[2023-10-14 15:08:43,164][74987] Avg episode reward: [(0, '24.470'), (1, '26.890')] -[2023-10-14 15:08:43,174][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000041472_42467328.pth... -[2023-10-14 15:08:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000041376_42369024.pth... -[2023-10-14 15:08:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000039808_40763392.pth -[2023-10-14 15:08:43,214][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000039904_40861696.pth -[2023-10-14 15:08:45,373][75950] Updated weights for policy 1, policy_version 41380 (0.0009) -[2023-10-14 15:08:45,689][75949] Updated weights for policy 0, policy_version 41481 (0.0009) -[2023-10-14 15:08:45,745][75950] Updated weights for policy 1, policy_version 41390 (0.0010) -[2023-10-14 15:08:46,065][75949] Updated weights for policy 0, policy_version 41491 (0.0009) -[2023-10-14 15:08:46,118][75950] Updated weights for policy 1, policy_version 41400 (0.0009) -[2023-10-14 15:08:46,444][75949] Updated weights for policy 0, policy_version 41501 (0.0008) -[2023-10-14 15:08:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84901888. Throughput: 0: 1669.9, 1: 1648.1. Samples: 21228450. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 15:08:48,164][74987] Avg episode reward: [(0, '24.610'), (1, '28.030')] -[2023-10-14 15:08:50,239][75950] Updated weights for policy 1, policy_version 41410 (0.0008) -[2023-10-14 15:08:50,403][75949] Updated weights for policy 0, policy_version 41511 (0.0008) -[2023-10-14 15:08:50,608][75950] Updated weights for policy 1, policy_version 41420 (0.0007) -[2023-10-14 15:08:50,773][75949] Updated weights for policy 0, policy_version 41521 (0.0009) -[2023-10-14 15:08:50,975][75950] Updated weights for policy 1, policy_version 41430 (0.0008) -[2023-10-14 15:08:51,140][75949] Updated weights for policy 0, policy_version 41531 (0.0007) -[2023-10-14 15:08:51,331][75950] Updated weights for policy 1, policy_version 41440 (0.0008) -[2023-10-14 15:08:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84967424. Throughput: 0: 1664.2, 1: 1653.6. Samples: 21247404. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 15:08:53,164][74987] Avg episode reward: [(0, '26.540'), (1, '29.430')] -[2023-10-14 15:08:55,208][75950] Updated weights for policy 1, policy_version 41450 (0.0007) -[2023-10-14 15:08:55,253][75949] Updated weights for policy 0, policy_version 41541 (0.0009) -[2023-10-14 15:08:55,580][75950] Updated weights for policy 1, policy_version 41460 (0.0008) -[2023-10-14 15:08:55,632][75949] Updated weights for policy 0, policy_version 41551 (0.0010) -[2023-10-14 15:08:55,940][75950] Updated weights for policy 1, policy_version 41470 (0.0007) -[2023-10-14 15:08:56,003][75949] Updated weights for policy 0, policy_version 41561 (0.0008) -[2023-10-14 15:08:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85032960. Throughput: 0: 1678.8, 1: 1670.4. Samples: 21268162. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 15:08:58,164][74987] Avg episode reward: [(0, '24.930'), (1, '30.180')] -[2023-10-14 15:08:59,952][75949] Updated weights for policy 0, policy_version 41571 (0.0010) -[2023-10-14 15:09:00,044][75950] Updated weights for policy 1, policy_version 41480 (0.0008) -[2023-10-14 15:09:00,320][75949] Updated weights for policy 0, policy_version 41581 (0.0009) -[2023-10-14 15:09:00,412][75950] Updated weights for policy 1, policy_version 41490 (0.0008) -[2023-10-14 15:09:00,698][75949] Updated weights for policy 0, policy_version 41591 (0.0009) -[2023-10-14 15:09:00,776][75950] Updated weights for policy 1, policy_version 41500 (0.0008) -[2023-10-14 15:09:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85098496. Throughput: 0: 1659.5, 1: 1657.6. Samples: 21278186. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 15:09:03,165][74987] Avg episode reward: [(0, '25.720'), (1, '30.660')] -[2023-10-14 15:09:04,711][75949] Updated weights for policy 0, policy_version 41601 (0.0009) -[2023-10-14 15:09:04,869][75950] Updated weights for policy 1, policy_version 41510 (0.0011) -[2023-10-14 15:09:05,082][75949] Updated weights for policy 0, policy_version 41611 (0.0007) -[2023-10-14 15:09:05,235][75950] Updated weights for policy 1, policy_version 41520 (0.0007) -[2023-10-14 15:09:05,448][75949] Updated weights for policy 0, policy_version 41621 (0.0008) -[2023-10-14 15:09:05,600][75950] Updated weights for policy 1, policy_version 41530 (0.0007) -[2023-10-14 15:09:05,818][75949] Updated weights for policy 0, policy_version 41631 (0.0007) -[2023-10-14 15:09:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85164032. Throughput: 0: 1667.2, 1: 1676.9. Samples: 21298034. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 15:09:08,164][74987] Avg episode reward: [(0, '25.560'), (1, '29.620')] -[2023-10-14 15:09:09,590][75950] Updated weights for policy 1, policy_version 41540 (0.0007) -[2023-10-14 15:09:09,936][75949] Updated weights for policy 0, policy_version 41641 (0.0009) -[2023-10-14 15:09:09,962][75950] Updated weights for policy 1, policy_version 41550 (0.0008) -[2023-10-14 15:09:10,304][75949] Updated weights for policy 0, policy_version 41651 (0.0009) -[2023-10-14 15:09:10,330][75950] Updated weights for policy 1, policy_version 41560 (0.0008) -[2023-10-14 15:09:10,665][75949] Updated weights for policy 0, policy_version 41661 (0.0010) -[2023-10-14 15:09:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85229568. Throughput: 0: 1674.5, 1: 1683.6. Samples: 21318528. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 15:09:13,164][74987] Avg episode reward: [(0, '26.150'), (1, '30.530')] -[2023-10-14 15:09:14,574][75950] Updated weights for policy 1, policy_version 41570 (0.0009) -[2023-10-14 15:09:14,940][75950] Updated weights for policy 1, policy_version 41580 (0.0007) -[2023-10-14 15:09:14,989][75949] Updated weights for policy 0, policy_version 41671 (0.0009) -[2023-10-14 15:09:15,308][75950] Updated weights for policy 1, policy_version 41590 (0.0007) -[2023-10-14 15:09:15,358][75949] Updated weights for policy 0, policy_version 41681 (0.0007) -[2023-10-14 15:09:15,669][75950] Updated weights for policy 1, policy_version 41600 (0.0008) -[2023-10-14 15:09:15,730][75949] Updated weights for policy 0, policy_version 41691 (0.0008) -[2023-10-14 15:09:18,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85295104. Throughput: 0: 1654.9, 1: 1661.7. Samples: 21327842. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:09:18,164][74987] Avg episode reward: [(0, '24.920'), (1, '29.750')] -[2023-10-14 15:09:19,625][75950] Updated weights for policy 1, policy_version 41610 (0.0008) -[2023-10-14 15:09:19,778][75949] Updated weights for policy 0, policy_version 41701 (0.0008) -[2023-10-14 15:09:19,991][75950] Updated weights for policy 1, policy_version 41620 (0.0008) -[2023-10-14 15:09:20,141][75949] Updated weights for policy 0, policy_version 41711 (0.0008) -[2023-10-14 15:09:20,355][75950] Updated weights for policy 1, policy_version 41630 (0.0008) -[2023-10-14 15:09:20,500][75949] Updated weights for policy 0, policy_version 41721 (0.0009) -[2023-10-14 15:09:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85360640. Throughput: 0: 1674.3, 1: 1685.5. Samples: 21348286. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:09:23,165][74987] Avg episode reward: [(0, '25.370'), (1, '29.800')] -[2023-10-14 15:09:24,374][75950] Updated weights for policy 1, policy_version 41640 (0.0008) -[2023-10-14 15:09:24,636][75949] Updated weights for policy 0, policy_version 41731 (0.0009) -[2023-10-14 15:09:24,732][75950] Updated weights for policy 1, policy_version 41650 (0.0008) -[2023-10-14 15:09:25,008][75949] Updated weights for policy 0, policy_version 41741 (0.0008) -[2023-10-14 15:09:25,095][75950] Updated weights for policy 1, policy_version 41660 (0.0008) -[2023-10-14 15:09:25,383][75949] Updated weights for policy 0, policy_version 41751 (0.0009) -[2023-10-14 15:09:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85426176. Throughput: 0: 1677.9, 1: 1684.8. Samples: 21369130. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:09:28,164][74987] Avg episode reward: [(0, '26.190'), (1, '29.580')] -[2023-10-14 15:09:29,238][75950] Updated weights for policy 1, policy_version 41670 (0.0007) -[2023-10-14 15:09:29,401][75949] Updated weights for policy 0, policy_version 41761 (0.0009) -[2023-10-14 15:09:29,599][75950] Updated weights for policy 1, policy_version 41680 (0.0007) -[2023-10-14 15:09:29,771][75949] Updated weights for policy 0, policy_version 41771 (0.0008) -[2023-10-14 15:09:29,966][75950] Updated weights for policy 1, policy_version 41690 (0.0007) -[2023-10-14 15:09:30,139][75949] Updated weights for policy 0, policy_version 41781 (0.0007) -[2023-10-14 15:09:30,508][75949] Updated weights for policy 0, policy_version 41791 (0.0008) -[2023-10-14 15:09:33,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 85491712. Throughput: 0: 1662.7, 1: 1675.3. Samples: 21378662. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:09:33,164][74987] Avg episode reward: [(0, '24.740'), (1, '28.840')] -[2023-10-14 15:09:34,148][75950] Updated weights for policy 1, policy_version 41700 (0.0008) -[2023-10-14 15:09:34,507][75949] Updated weights for policy 0, policy_version 41801 (0.0008) -[2023-10-14 15:09:34,513][75950] Updated weights for policy 1, policy_version 41710 (0.0009) -[2023-10-14 15:09:34,876][75949] Updated weights for policy 0, policy_version 41811 (0.0009) -[2023-10-14 15:09:34,881][75950] Updated weights for policy 1, policy_version 41720 (0.0008) -[2023-10-14 15:09:35,235][75949] Updated weights for policy 0, policy_version 41821 (0.0009) -[2023-10-14 15:09:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85557248. Throughput: 0: 1684.4, 1: 1686.6. Samples: 21399098. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:09:38,164][74987] Avg episode reward: [(0, '24.650'), (1, '30.740')] -[2023-10-14 15:09:38,921][75950] Updated weights for policy 1, policy_version 41730 (0.0008) -[2023-10-14 15:09:39,291][75950] Updated weights for policy 1, policy_version 41740 (0.0007) -[2023-10-14 15:09:39,305][75949] Updated weights for policy 0, policy_version 41831 (0.0007) -[2023-10-14 15:09:39,649][75950] Updated weights for policy 1, policy_version 41750 (0.0007) -[2023-10-14 15:09:39,673][75949] Updated weights for policy 0, policy_version 41841 (0.0008) -[2023-10-14 15:09:40,015][75950] Updated weights for policy 1, policy_version 41760 (0.0008) -[2023-10-14 15:09:40,047][75949] Updated weights for policy 0, policy_version 41851 (0.0007) -[2023-10-14 15:09:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 85622784. Throughput: 0: 1683.9, 1: 1676.7. Samples: 21419388. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:09:43,165][74987] Avg episode reward: [(0, '24.600'), (1, '28.860')] -[2023-10-14 15:09:44,258][75949] Updated weights for policy 0, policy_version 41861 (0.0010) -[2023-10-14 15:09:44,270][75950] Updated weights for policy 1, policy_version 41770 (0.0009) -[2023-10-14 15:09:44,638][75949] Updated weights for policy 0, policy_version 41871 (0.0008) -[2023-10-14 15:09:44,643][75950] Updated weights for policy 1, policy_version 41780 (0.0007) -[2023-10-14 15:09:45,009][75950] Updated weights for policy 1, policy_version 41790 (0.0007) -[2023-10-14 15:09:45,012][75949] Updated weights for policy 0, policy_version 41881 (0.0007) -[2023-10-14 15:09:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85688320. Throughput: 0: 1669.8, 1: 1664.4. Samples: 21428226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:09:48,165][74987] Avg episode reward: [(0, '26.660'), (1, '30.150')] -[2023-10-14 15:09:49,126][75949] Updated weights for policy 0, policy_version 41891 (0.0009) -[2023-10-14 15:09:49,166][75950] Updated weights for policy 1, policy_version 41800 (0.0009) -[2023-10-14 15:09:49,495][75949] Updated weights for policy 0, policy_version 41901 (0.0009) -[2023-10-14 15:09:49,532][75950] Updated weights for policy 1, policy_version 41810 (0.0008) -[2023-10-14 15:09:49,859][75949] Updated weights for policy 0, policy_version 41911 (0.0009) -[2023-10-14 15:09:49,899][75950] Updated weights for policy 1, policy_version 41820 (0.0009) -[2023-10-14 15:09:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85753856. Throughput: 0: 1677.1, 1: 1668.4. Samples: 21448580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:09:53,164][74987] Avg episode reward: [(0, '24.500'), (1, '30.270')] -[2023-10-14 15:09:53,950][75949] Updated weights for policy 0, policy_version 41921 (0.0007) -[2023-10-14 15:09:54,113][75950] Updated weights for policy 1, policy_version 41830 (0.0009) -[2023-10-14 15:09:54,323][75949] Updated weights for policy 0, policy_version 41931 (0.0007) -[2023-10-14 15:09:54,472][75950] Updated weights for policy 1, policy_version 41840 (0.0009) -[2023-10-14 15:09:54,698][75949] Updated weights for policy 0, policy_version 41941 (0.0007) -[2023-10-14 15:09:54,848][75950] Updated weights for policy 1, policy_version 41850 (0.0009) -[2023-10-14 15:09:55,059][75949] Updated weights for policy 0, policy_version 41951 (0.0007) -[2023-10-14 15:09:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85819392. Throughput: 0: 1680.0, 1: 1669.8. Samples: 21469268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:09:58,164][74987] Avg episode reward: [(0, '27.240'), (1, '29.080')] -[2023-10-14 15:09:58,990][75950] Updated weights for policy 1, policy_version 41860 (0.0008) -[2023-10-14 15:09:59,177][75949] Updated weights for policy 0, policy_version 41961 (0.0009) -[2023-10-14 15:09:59,364][75950] Updated weights for policy 1, policy_version 41870 (0.0009) -[2023-10-14 15:09:59,546][75949] Updated weights for policy 0, policy_version 41971 (0.0009) -[2023-10-14 15:09:59,732][75950] Updated weights for policy 1, policy_version 41880 (0.0007) -[2023-10-14 15:09:59,916][75949] Updated weights for policy 0, policy_version 41981 (0.0008) -[2023-10-14 15:10:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 85884928. Throughput: 0: 1671.6, 1: 1668.0. Samples: 21478126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:10:03,165][74987] Avg episode reward: [(0, '24.270'), (1, '29.110')] -[2023-10-14 15:10:03,713][75950] Updated weights for policy 1, policy_version 41890 (0.0009) -[2023-10-14 15:10:04,083][75950] Updated weights for policy 1, policy_version 41900 (0.0008) -[2023-10-14 15:10:04,084][75949] Updated weights for policy 0, policy_version 41991 (0.0008) -[2023-10-14 15:10:04,442][75950] Updated weights for policy 1, policy_version 41910 (0.0008) -[2023-10-14 15:10:04,459][75949] Updated weights for policy 0, policy_version 42001 (0.0009) -[2023-10-14 15:10:04,808][75950] Updated weights for policy 1, policy_version 41920 (0.0007) -[2023-10-14 15:10:04,822][75949] Updated weights for policy 0, policy_version 42011 (0.0007) -[2023-10-14 15:10:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85950464. Throughput: 0: 1674.7, 1: 1672.4. Samples: 21498906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:10:08,165][74987] Avg episode reward: [(0, '26.670'), (1, '30.100')] -[2023-10-14 15:10:08,862][75950] Updated weights for policy 1, policy_version 41930 (0.0007) -[2023-10-14 15:10:08,960][75949] Updated weights for policy 0, policy_version 42021 (0.0008) -[2023-10-14 15:10:09,222][75950] Updated weights for policy 1, policy_version 41940 (0.0007) -[2023-10-14 15:10:09,335][75949] Updated weights for policy 0, policy_version 42031 (0.0008) -[2023-10-14 15:10:09,590][75950] Updated weights for policy 1, policy_version 41950 (0.0007) -[2023-10-14 15:10:09,699][75949] Updated weights for policy 0, policy_version 42041 (0.0008) -[2023-10-14 15:10:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86016000. Throughput: 0: 1674.1, 1: 1669.1. Samples: 21519576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:10:13,165][74987] Avg episode reward: [(0, '23.840'), (1, '31.510')] -[2023-10-14 15:10:13,724][75949] Updated weights for policy 0, policy_version 42051 (0.0009) -[2023-10-14 15:10:13,733][75950] Updated weights for policy 1, policy_version 41960 (0.0009) -[2023-10-14 15:10:14,102][75950] Updated weights for policy 1, policy_version 41970 (0.0009) -[2023-10-14 15:10:14,103][75949] Updated weights for policy 0, policy_version 42061 (0.0007) -[2023-10-14 15:10:14,475][75949] Updated weights for policy 0, policy_version 42071 (0.0007) -[2023-10-14 15:10:14,478][75950] Updated weights for policy 1, policy_version 41980 (0.0007) -[2023-10-14 15:10:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86081536. Throughput: 0: 1668.2, 1: 1664.3. Samples: 21528626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:10:18,165][74987] Avg episode reward: [(0, '24.990'), (1, '30.720')] -[2023-10-14 15:10:18,402][75950] Updated weights for policy 1, policy_version 41990 (0.0008) -[2023-10-14 15:10:18,475][75949] Updated weights for policy 0, policy_version 42081 (0.0011) -[2023-10-14 15:10:18,779][75950] Updated weights for policy 1, policy_version 42000 (0.0008) -[2023-10-14 15:10:18,848][75949] Updated weights for policy 0, policy_version 42091 (0.0007) -[2023-10-14 15:10:19,147][75950] Updated weights for policy 1, policy_version 42010 (0.0007) -[2023-10-14 15:10:19,209][75949] Updated weights for policy 0, policy_version 42101 (0.0007) -[2023-10-14 15:10:19,584][75949] Updated weights for policy 0, policy_version 42111 (0.0009) -[2023-10-14 15:10:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 86147072. Throughput: 0: 1667.8, 1: 1670.3. Samples: 21549312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:10:23,164][74987] Avg episode reward: [(0, '23.710'), (1, '29.410')] -[2023-10-14 15:10:23,277][75950] Updated weights for policy 1, policy_version 42020 (0.0009) -[2023-10-14 15:10:23,631][75950] Updated weights for policy 1, policy_version 42030 (0.0007) -[2023-10-14 15:10:23,650][75949] Updated weights for policy 0, policy_version 42121 (0.0007) -[2023-10-14 15:10:23,994][75950] Updated weights for policy 1, policy_version 42040 (0.0008) -[2023-10-14 15:10:24,027][75949] Updated weights for policy 0, policy_version 42131 (0.0008) -[2023-10-14 15:10:24,397][75949] Updated weights for policy 0, policy_version 42141 (0.0008) -[2023-10-14 15:10:28,140][75950] Updated weights for policy 1, policy_version 42050 (0.0008) -[2023-10-14 15:10:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 86212608. Throughput: 0: 1671.2, 1: 1673.2. Samples: 21569884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:10:28,165][74987] Avg episode reward: [(0, '26.240'), (1, '29.070')] -[2023-10-14 15:10:28,455][75949] Updated weights for policy 0, policy_version 42151 (0.0008) -[2023-10-14 15:10:28,549][75950] Updated weights for policy 1, policy_version 42060 (0.0007) -[2023-10-14 15:10:28,824][75949] Updated weights for policy 0, policy_version 42161 (0.0008) -[2023-10-14 15:10:28,914][75950] Updated weights for policy 1, policy_version 42070 (0.0007) -[2023-10-14 15:10:29,204][75949] Updated weights for policy 0, policy_version 42171 (0.0009) -[2023-10-14 15:10:29,283][75950] Updated weights for policy 1, policy_version 42080 (0.0008) -[2023-10-14 15:10:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86278144. Throughput: 0: 1674.2, 1: 1672.4. Samples: 21578822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:10:33,164][74987] Avg episode reward: [(0, '24.670'), (1, '30.200')] -[2023-10-14 15:10:33,378][75950] Updated weights for policy 1, policy_version 42090 (0.0009) -[2023-10-14 15:10:33,381][75949] Updated weights for policy 0, policy_version 42181 (0.0009) -[2023-10-14 15:10:33,743][75950] Updated weights for policy 1, policy_version 42100 (0.0008) -[2023-10-14 15:10:33,763][75949] Updated weights for policy 0, policy_version 42191 (0.0007) -[2023-10-14 15:10:34,103][75950] Updated weights for policy 1, policy_version 42110 (0.0009) -[2023-10-14 15:10:34,134][75949] Updated weights for policy 0, policy_version 42201 (0.0007) -[2023-10-14 15:10:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86343680. Throughput: 0: 1670.9, 1: 1676.0. Samples: 21599192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:10:38,164][74987] Avg episode reward: [(0, '25.530'), (1, '29.990')] -[2023-10-14 15:10:38,203][75949] Updated weights for policy 0, policy_version 42211 (0.0009) -[2023-10-14 15:10:38,311][75950] Updated weights for policy 1, policy_version 42120 (0.0007) -[2023-10-14 15:10:38,572][75949] Updated weights for policy 0, policy_version 42221 (0.0009) -[2023-10-14 15:10:38,677][75950] Updated weights for policy 1, policy_version 42130 (0.0007) -[2023-10-14 15:10:38,947][75949] Updated weights for policy 0, policy_version 42231 (0.0008) -[2023-10-14 15:10:39,046][75950] Updated weights for policy 1, policy_version 42140 (0.0008) -[2023-10-14 15:10:43,019][75949] Updated weights for policy 0, policy_version 42241 (0.0009) -[2023-10-14 15:10:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 86409216. Throughput: 0: 1671.5, 1: 1670.8. Samples: 21619674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:10:43,164][74987] Avg episode reward: [(0, '26.370'), (1, '30.380')] -[2023-10-14 15:10:43,223][75950] Updated weights for policy 1, policy_version 42150 (0.0008) -[2023-10-14 15:10:43,388][75949] Updated weights for policy 0, policy_version 42251 (0.0007) -[2023-10-14 15:10:43,579][75950] Updated weights for policy 1, policy_version 42160 (0.0008) -[2023-10-14 15:10:43,757][75949] Updated weights for policy 0, policy_version 42261 (0.0007) -[2023-10-14 15:10:43,939][75950] Updated weights for policy 1, policy_version 42170 (0.0010) -[2023-10-14 15:10:44,139][75949] Updated weights for policy 0, policy_version 42271 (0.0009) -[2023-10-14 15:10:44,153][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000042176_43188224.pth... -[2023-10-14 15:10:44,169][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000042272_43286528.pth... -[2023-10-14 15:10:44,181][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000040608_41582592.pth -[2023-10-14 15:10:44,199][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000040704_41680896.pth -[2023-10-14 15:10:48,034][75950] Updated weights for policy 1, policy_version 42180 (0.0008) -[2023-10-14 15:10:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86474752. Throughput: 0: 1675.5, 1: 1671.2. Samples: 21628726. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-14 15:10:48,164][74987] Avg episode reward: [(0, '23.300'), (1, '30.470')] -[2023-10-14 15:10:48,386][75949] Updated weights for policy 0, policy_version 42281 (0.0009) -[2023-10-14 15:10:48,402][75950] Updated weights for policy 1, policy_version 42190 (0.0009) -[2023-10-14 15:10:48,759][75949] Updated weights for policy 0, policy_version 42291 (0.0008) -[2023-10-14 15:10:48,761][75950] Updated weights for policy 1, policy_version 42200 (0.0008) -[2023-10-14 15:10:49,120][75949] Updated weights for policy 0, policy_version 42301 (0.0007) -[2023-10-14 15:10:52,634][75950] Updated weights for policy 1, policy_version 42210 (0.0008) -[2023-10-14 15:10:53,010][75950] Updated weights for policy 1, policy_version 42220 (0.0008) -[2023-10-14 15:10:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86540288. Throughput: 0: 1672.2, 1: 1664.0. Samples: 21649038. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-14 15:10:53,164][74987] Avg episode reward: [(0, '26.410'), (1, '29.850')] -[2023-10-14 15:10:53,312][75949] Updated weights for policy 0, policy_version 42311 (0.0009) -[2023-10-14 15:10:53,374][75950] Updated weights for policy 1, policy_version 42230 (0.0009) -[2023-10-14 15:10:53,682][75949] Updated weights for policy 0, policy_version 42321 (0.0008) -[2023-10-14 15:10:53,737][75950] Updated weights for policy 1, policy_version 42240 (0.0007) -[2023-10-14 15:10:54,050][75949] Updated weights for policy 0, policy_version 42331 (0.0009) -[2023-10-14 15:10:57,981][75950] Updated weights for policy 1, policy_version 42250 (0.0008) -[2023-10-14 15:10:58,119][75949] Updated weights for policy 0, policy_version 42341 (0.0009) -[2023-10-14 15:10:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 86605824. Throughput: 0: 1668.8, 1: 1663.2. Samples: 21669518. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-14 15:10:58,165][74987] Avg episode reward: [(0, '22.470'), (1, '31.210')] -[2023-10-14 15:10:58,352][75950] Updated weights for policy 1, policy_version 42260 (0.0009) -[2023-10-14 15:10:58,482][75949] Updated weights for policy 0, policy_version 42351 (0.0008) -[2023-10-14 15:10:58,717][75950] Updated weights for policy 1, policy_version 42270 (0.0009) -[2023-10-14 15:10:58,853][75949] Updated weights for policy 0, policy_version 42361 (0.0008) -[2023-10-14 15:11:02,854][75949] Updated weights for policy 0, policy_version 42371 (0.0007) -[2023-10-14 15:11:02,918][75950] Updated weights for policy 1, policy_version 42280 (0.0009) -[2023-10-14 15:11:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86671360. Throughput: 0: 1671.0, 1: 1662.7. Samples: 21678642. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-14 15:11:03,164][74987] Avg episode reward: [(0, '27.600'), (1, '31.470')] -[2023-10-14 15:11:03,220][75949] Updated weights for policy 0, policy_version 42381 (0.0007) -[2023-10-14 15:11:03,281][75950] Updated weights for policy 1, policy_version 42290 (0.0007) -[2023-10-14 15:11:03,593][75949] Updated weights for policy 0, policy_version 42391 (0.0010) -[2023-10-14 15:11:03,644][75950] Updated weights for policy 1, policy_version 42300 (0.0008) -[2023-10-14 15:11:07,465][75949] Updated weights for policy 0, policy_version 42401 (0.0009) -[2023-10-14 15:11:07,837][75949] Updated weights for policy 0, policy_version 42411 (0.0009) -[2023-10-14 15:11:07,980][75950] Updated weights for policy 1, policy_version 42310 (0.0008) -[2023-10-14 15:11:08,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86736896. Throughput: 0: 1671.8, 1: 1656.1. Samples: 21699068. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-14 15:11:08,164][74987] Avg episode reward: [(0, '22.460'), (1, '32.050')] -[2023-10-14 15:11:08,207][75949] Updated weights for policy 0, policy_version 42421 (0.0008) -[2023-10-14 15:11:08,347][75950] Updated weights for policy 1, policy_version 42320 (0.0008) -[2023-10-14 15:11:08,577][75949] Updated weights for policy 0, policy_version 42431 (0.0007) -[2023-10-14 15:11:08,721][75950] Updated weights for policy 1, policy_version 42330 (0.0009) -[2023-10-14 15:11:12,569][75949] Updated weights for policy 0, policy_version 42441 (0.0010) -[2023-10-14 15:11:12,754][75950] Updated weights for policy 1, policy_version 42340 (0.0008) -[2023-10-14 15:11:12,940][75949] Updated weights for policy 0, policy_version 42451 (0.0008) -[2023-10-14 15:11:13,118][75950] Updated weights for policy 1, policy_version 42350 (0.0009) -[2023-10-14 15:11:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 86802432. Throughput: 0: 1659.2, 1: 1657.2. Samples: 21719120. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) -[2023-10-14 15:11:13,164][74987] Avg episode reward: [(0, '26.600'), (1, '29.990')] -[2023-10-14 15:11:13,309][75949] Updated weights for policy 0, policy_version 42461 (0.0010) -[2023-10-14 15:11:13,488][75950] Updated weights for policy 1, policy_version 42360 (0.0008) -[2023-10-14 15:11:17,582][75949] Updated weights for policy 0, policy_version 42471 (0.0009) -[2023-10-14 15:11:17,594][75950] Updated weights for policy 1, policy_version 42370 (0.0007) -[2023-10-14 15:11:17,957][75949] Updated weights for policy 0, policy_version 42481 (0.0008) -[2023-10-14 15:11:18,026][75950] Updated weights for policy 1, policy_version 42380 (0.0008) -[2023-10-14 15:11:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86867968. Throughput: 0: 1664.9, 1: 1657.3. Samples: 21728320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:11:18,164][74987] Avg episode reward: [(0, '21.590'), (1, '31.620')] -[2023-10-14 15:11:18,321][75949] Updated weights for policy 0, policy_version 42491 (0.0009) -[2023-10-14 15:11:18,388][75950] Updated weights for policy 1, policy_version 42390 (0.0008) -[2023-10-14 15:11:18,758][75950] Updated weights for policy 1, policy_version 42400 (0.0007) -[2023-10-14 15:11:22,295][75949] Updated weights for policy 0, policy_version 42501 (0.0009) -[2023-10-14 15:11:22,686][75949] Updated weights for policy 0, policy_version 42511 (0.0008) -[2023-10-14 15:11:22,975][75950] Updated weights for policy 1, policy_version 42410 (0.0007) -[2023-10-14 15:11:23,051][75949] Updated weights for policy 0, policy_version 42521 (0.0007) -[2023-10-14 15:11:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86933504. Throughput: 0: 1679.0, 1: 1646.9. Samples: 21748856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:11:23,164][74987] Avg episode reward: [(0, '24.960'), (1, '27.710')] -[2023-10-14 15:11:23,333][75950] Updated weights for policy 1, policy_version 42420 (0.0008) -[2023-10-14 15:11:23,700][75950] Updated weights for policy 1, policy_version 42430 (0.0008) -[2023-10-14 15:11:27,240][75949] Updated weights for policy 0, policy_version 42531 (0.0008) -[2023-10-14 15:11:27,618][75949] Updated weights for policy 0, policy_version 42541 (0.0008) -[2023-10-14 15:11:27,805][75950] Updated weights for policy 1, policy_version 42440 (0.0008) -[2023-10-14 15:11:27,983][75949] Updated weights for policy 0, policy_version 42551 (0.0008) -[2023-10-14 15:11:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86999040. Throughput: 0: 1660.5, 1: 1646.4. Samples: 21768484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:11:28,164][74987] Avg episode reward: [(0, '22.310'), (1, '27.020')] -[2023-10-14 15:11:28,171][75950] Updated weights for policy 1, policy_version 42450 (0.0009) -[2023-10-14 15:11:28,541][75950] Updated weights for policy 1, policy_version 42460 (0.0008) -[2023-10-14 15:11:32,039][75949] Updated weights for policy 0, policy_version 42561 (0.0009) -[2023-10-14 15:11:32,410][75949] Updated weights for policy 0, policy_version 42571 (0.0008) -[2023-10-14 15:11:32,782][75949] Updated weights for policy 0, policy_version 42581 (0.0008) -[2023-10-14 15:11:32,920][75950] Updated weights for policy 1, policy_version 42470 (0.0008) -[2023-10-14 15:11:33,155][75949] Updated weights for policy 0, policy_version 42591 (0.0008) -[2023-10-14 15:11:33,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 87064576. Throughput: 0: 1673.9, 1: 1646.3. Samples: 21778136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:11:33,165][74987] Avg episode reward: [(0, '25.260'), (1, '26.870')] -[2023-10-14 15:11:33,287][75950] Updated weights for policy 1, policy_version 42480 (0.0008) -[2023-10-14 15:11:33,658][75950] Updated weights for policy 1, policy_version 42490 (0.0008) -[2023-10-14 15:11:37,076][75949] Updated weights for policy 0, policy_version 42601 (0.0009) -[2023-10-14 15:11:37,444][75949] Updated weights for policy 0, policy_version 42611 (0.0008) -[2023-10-14 15:11:37,673][75950] Updated weights for policy 1, policy_version 42500 (0.0008) -[2023-10-14 15:11:37,818][75949] Updated weights for policy 0, policy_version 42621 (0.0008) -[2023-10-14 15:11:38,031][75950] Updated weights for policy 1, policy_version 42510 (0.0008) -[2023-10-14 15:11:38,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 87162880. Throughput: 0: 1682.4, 1: 1647.4. Samples: 21798878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:11:38,164][74987] Avg episode reward: [(0, '24.000'), (1, '27.120')] -[2023-10-14 15:11:38,396][75950] Updated weights for policy 1, policy_version 42520 (0.0007) -[2023-10-14 15:11:41,796][75949] Updated weights for policy 0, policy_version 42631 (0.0007) -[2023-10-14 15:11:42,168][75949] Updated weights for policy 0, policy_version 42641 (0.0010) -[2023-10-14 15:11:42,550][75949] Updated weights for policy 0, policy_version 42651 (0.0009) -[2023-10-14 15:11:42,638][75950] Updated weights for policy 1, policy_version 42530 (0.0010) -[2023-10-14 15:11:42,996][75950] Updated weights for policy 1, policy_version 42540 (0.0008) -[2023-10-14 15:11:43,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 87228416. Throughput: 0: 1658.5, 1: 1648.8. Samples: 21818346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:11:43,165][74987] Avg episode reward: [(0, '25.520'), (1, '29.250')] -[2023-10-14 15:11:43,358][75950] Updated weights for policy 1, policy_version 42550 (0.0009) -[2023-10-14 15:11:43,712][75950] Updated weights for policy 1, policy_version 42560 (0.0008) -[2023-10-14 15:11:46,633][75949] Updated weights for policy 0, policy_version 42661 (0.0009) -[2023-10-14 15:11:47,005][75949] Updated weights for policy 0, policy_version 42671 (0.0008) -[2023-10-14 15:11:47,380][75949] Updated weights for policy 0, policy_version 42681 (0.0011) -[2023-10-14 15:11:47,689][75950] Updated weights for policy 1, policy_version 42570 (0.0008) -[2023-10-14 15:11:48,062][75950] Updated weights for policy 1, policy_version 42580 (0.0010) -[2023-10-14 15:11:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 87293952. Throughput: 0: 1682.5, 1: 1654.3. Samples: 21828798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:11:48,164][74987] Avg episode reward: [(0, '22.890'), (1, '29.570')] -[2023-10-14 15:11:48,418][75950] Updated weights for policy 1, policy_version 42590 (0.0009) -[2023-10-14 15:11:51,501][75949] Updated weights for policy 0, policy_version 42691 (0.0009) -[2023-10-14 15:11:51,871][75949] Updated weights for policy 0, policy_version 42701 (0.0009) -[2023-10-14 15:11:52,232][75949] Updated weights for policy 0, policy_version 42711 (0.0009) -[2023-10-14 15:11:52,490][75950] Updated weights for policy 1, policy_version 42600 (0.0007) -[2023-10-14 15:11:52,859][75950] Updated weights for policy 1, policy_version 42610 (0.0007) -[2023-10-14 15:11:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 87359488. Throughput: 0: 1673.3, 1: 1660.5. Samples: 21849090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:11:53,164][74987] Avg episode reward: [(0, '22.770'), (1, '30.270')] -[2023-10-14 15:11:53,234][75950] Updated weights for policy 1, policy_version 42620 (0.0007) -[2023-10-14 15:11:56,321][75949] Updated weights for policy 0, policy_version 42721 (0.0009) -[2023-10-14 15:11:56,684][75949] Updated weights for policy 0, policy_version 42731 (0.0009) -[2023-10-14 15:11:57,061][75949] Updated weights for policy 0, policy_version 42741 (0.0007) -[2023-10-14 15:11:57,232][75950] Updated weights for policy 1, policy_version 42630 (0.0008) -[2023-10-14 15:11:57,430][75949] Updated weights for policy 0, policy_version 42751 (0.0008) -[2023-10-14 15:11:57,597][75950] Updated weights for policy 1, policy_version 42640 (0.0009) -[2023-10-14 15:11:57,956][75950] Updated weights for policy 1, policy_version 42650 (0.0009) -[2023-10-14 15:11:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 87425024. Throughput: 0: 1666.0, 1: 1652.6. Samples: 21868458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:11:58,164][74987] Avg episode reward: [(0, '23.180'), (1, '29.700')] -[2023-10-14 15:12:01,626][75949] Updated weights for policy 0, policy_version 42761 (0.0010) -[2023-10-14 15:12:02,002][75949] Updated weights for policy 0, policy_version 42771 (0.0009) -[2023-10-14 15:12:02,149][75950] Updated weights for policy 1, policy_version 42660 (0.0008) -[2023-10-14 15:12:02,374][75949] Updated weights for policy 0, policy_version 42781 (0.0009) -[2023-10-14 15:12:02,511][75950] Updated weights for policy 1, policy_version 42670 (0.0008) -[2023-10-14 15:12:02,877][75950] Updated weights for policy 1, policy_version 42680 (0.0007) -[2023-10-14 15:12:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 87490560. Throughput: 0: 1690.2, 1: 1666.0. Samples: 21879346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:12:03,164][74987] Avg episode reward: [(0, '21.970'), (1, '29.130')] -[2023-10-14 15:12:06,180][75949] Updated weights for policy 0, policy_version 42791 (0.0008) -[2023-10-14 15:12:06,557][75949] Updated weights for policy 0, policy_version 42801 (0.0010) -[2023-10-14 15:12:06,930][75949] Updated weights for policy 0, policy_version 42811 (0.0010) -[2023-10-14 15:12:07,093][75950] Updated weights for policy 1, policy_version 42690 (0.0008) -[2023-10-14 15:12:07,497][75950] Updated weights for policy 1, policy_version 42700 (0.0008) -[2023-10-14 15:12:07,870][75950] Updated weights for policy 1, policy_version 42710 (0.0010) -[2023-10-14 15:12:08,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 87556096. Throughput: 0: 1668.3, 1: 1674.8. Samples: 21899300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:12:08,165][74987] Avg episode reward: [(0, '26.260'), (1, '29.940')] -[2023-10-14 15:12:08,232][75950] Updated weights for policy 1, policy_version 42720 (0.0010) -[2023-10-14 15:12:11,220][75949] Updated weights for policy 0, policy_version 42821 (0.0009) -[2023-10-14 15:12:11,613][75949] Updated weights for policy 0, policy_version 42831 (0.0009) -[2023-10-14 15:12:11,988][75949] Updated weights for policy 0, policy_version 42841 (0.0009) -[2023-10-14 15:12:12,201][75950] Updated weights for policy 1, policy_version 42730 (0.0008) -[2023-10-14 15:12:12,570][75950] Updated weights for policy 1, policy_version 42740 (0.0008) -[2023-10-14 15:12:12,936][75950] Updated weights for policy 1, policy_version 42750 (0.0007) -[2023-10-14 15:12:13,164][74987] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 87654400. Throughput: 0: 1677.4, 1: 1659.6. Samples: 21918652. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 15:12:13,165][74987] Avg episode reward: [(0, '21.810'), (1, '29.100')] -[2023-10-14 15:12:16,016][75949] Updated weights for policy 0, policy_version 42851 (0.0007) -[2023-10-14 15:12:16,387][75949] Updated weights for policy 0, policy_version 42861 (0.0008) -[2023-10-14 15:12:16,756][75949] Updated weights for policy 0, policy_version 42871 (0.0010) -[2023-10-14 15:12:17,050][75950] Updated weights for policy 1, policy_version 42760 (0.0008) -[2023-10-14 15:12:17,414][75950] Updated weights for policy 1, policy_version 42770 (0.0011) -[2023-10-14 15:12:17,780][75950] Updated weights for policy 1, policy_version 42780 (0.0008) -[2023-10-14 15:12:18,164][74987] Fps is (10 sec: 16384.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 87719936. Throughput: 0: 1693.7, 1: 1677.0. Samples: 21929816. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 15:12:18,164][74987] Avg episode reward: [(0, '26.820'), (1, '28.850')] -[2023-10-14 15:12:20,904][75949] Updated weights for policy 0, policy_version 42881 (0.0007) -[2023-10-14 15:12:21,279][75949] Updated weights for policy 0, policy_version 42891 (0.0008) -[2023-10-14 15:12:21,648][75949] Updated weights for policy 0, policy_version 42901 (0.0009) -[2023-10-14 15:12:21,790][75950] Updated weights for policy 1, policy_version 42790 (0.0009) -[2023-10-14 15:12:22,011][75949] Updated weights for policy 0, policy_version 42911 (0.0008) -[2023-10-14 15:12:22,158][75950] Updated weights for policy 1, policy_version 42800 (0.0008) -[2023-10-14 15:12:22,521][75950] Updated weights for policy 1, policy_version 42810 (0.0007) -[2023-10-14 15:12:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 87785472. Throughput: 0: 1668.3, 1: 1676.5. Samples: 21949394. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 15:12:23,164][74987] Avg episode reward: [(0, '23.440'), (1, '30.800')] -[2023-10-14 15:12:25,938][75949] Updated weights for policy 0, policy_version 42921 (0.0008) -[2023-10-14 15:12:26,312][75949] Updated weights for policy 0, policy_version 42931 (0.0008) -[2023-10-14 15:12:26,681][75949] Updated weights for policy 0, policy_version 42941 (0.0008) -[2023-10-14 15:12:26,731][75950] Updated weights for policy 1, policy_version 42820 (0.0007) -[2023-10-14 15:12:27,095][75950] Updated weights for policy 1, policy_version 42830 (0.0008) -[2023-10-14 15:12:27,466][75950] Updated weights for policy 1, policy_version 42840 (0.0010) -[2023-10-14 15:12:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 87851008. Throughput: 0: 1687.9, 1: 1654.6. Samples: 21968758. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 15:12:28,165][74987] Avg episode reward: [(0, '28.180'), (1, '29.020')] -[2023-10-14 15:12:28,177][75615] Saving new best policy, reward=28.180! -[2023-10-14 15:12:30,574][75949] Updated weights for policy 0, policy_version 42951 (0.0010) -[2023-10-14 15:12:30,954][75949] Updated weights for policy 0, policy_version 42961 (0.0009) -[2023-10-14 15:12:31,326][75949] Updated weights for policy 0, policy_version 42971 (0.0008) -[2023-10-14 15:12:31,572][75950] Updated weights for policy 1, policy_version 42850 (0.0009) -[2023-10-14 15:12:31,930][75950] Updated weights for policy 1, policy_version 42860 (0.0007) -[2023-10-14 15:12:32,300][75950] Updated weights for policy 1, policy_version 42870 (0.0008) -[2023-10-14 15:12:32,669][75950] Updated weights for policy 1, policy_version 42880 (0.0009) -[2023-10-14 15:12:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 13440.4). Total num frames: 87916544. Throughput: 0: 1683.5, 1: 1673.0. Samples: 21979840. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 15:12:33,164][74987] Avg episode reward: [(0, '22.570'), (1, '29.120')] -[2023-10-14 15:12:35,385][75949] Updated weights for policy 0, policy_version 42981 (0.0008) -[2023-10-14 15:12:35,760][75949] Updated weights for policy 0, policy_version 42991 (0.0007) -[2023-10-14 15:12:36,128][75949] Updated weights for policy 0, policy_version 43001 (0.0007) -[2023-10-14 15:12:36,806][75950] Updated weights for policy 1, policy_version 42890 (0.0009) -[2023-10-14 15:12:37,184][75950] Updated weights for policy 1, policy_version 42900 (0.0009) -[2023-10-14 15:12:37,540][75950] Updated weights for policy 1, policy_version 42910 (0.0008) -[2023-10-14 15:12:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 87982080. Throughput: 0: 1673.2, 1: 1669.5. Samples: 21999512. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-10-14 15:12:38,165][74987] Avg episode reward: [(0, '26.760'), (1, '28.800')] -[2023-10-14 15:12:40,262][75949] Updated weights for policy 0, policy_version 43011 (0.0010) -[2023-10-14 15:12:40,629][75949] Updated weights for policy 0, policy_version 43021 (0.0009) -[2023-10-14 15:12:40,990][75949] Updated weights for policy 0, policy_version 43031 (0.0009) -[2023-10-14 15:12:41,490][75950] Updated weights for policy 1, policy_version 42920 (0.0008) -[2023-10-14 15:12:41,855][75950] Updated weights for policy 1, policy_version 42930 (0.0008) -[2023-10-14 15:12:42,213][75950] Updated weights for policy 1, policy_version 42940 (0.0007) -[2023-10-14 15:12:43,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 88047616. Throughput: 0: 1696.9, 1: 1659.6. Samples: 22019498. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 15:12:43,164][74987] Avg episode reward: [(0, '22.630'), (1, '28.770')] -[2023-10-14 15:12:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000042944_43974656.pth... -[2023-10-14 15:12:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000043040_44072960.pth... -[2023-10-14 15:12:43,213][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000041376_42369024.pth -[2023-10-14 15:12:43,214][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000041472_42467328.pth -[2023-10-14 15:12:45,249][75949] Updated weights for policy 0, policy_version 43041 (0.0009) -[2023-10-14 15:12:45,621][75949] Updated weights for policy 0, policy_version 43051 (0.0007) -[2023-10-14 15:12:45,993][75949] Updated weights for policy 0, policy_version 43061 (0.0008) -[2023-10-14 15:12:46,280][75950] Updated weights for policy 1, policy_version 42950 (0.0009) -[2023-10-14 15:12:46,366][75949] Updated weights for policy 0, policy_version 43071 (0.0008) -[2023-10-14 15:12:46,655][75950] Updated weights for policy 1, policy_version 42960 (0.0009) -[2023-10-14 15:12:47,022][75950] Updated weights for policy 1, policy_version 42970 (0.0011) -[2023-10-14 15:12:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 88113152. Throughput: 0: 1684.1, 1: 1675.9. Samples: 22030546. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 15:12:48,164][74987] Avg episode reward: [(0, '27.790'), (1, '29.630')] -[2023-10-14 15:12:50,294][75949] Updated weights for policy 0, policy_version 43081 (0.0009) -[2023-10-14 15:12:50,656][75949] Updated weights for policy 0, policy_version 43091 (0.0008) -[2023-10-14 15:12:51,025][75949] Updated weights for policy 0, policy_version 43101 (0.0007) -[2023-10-14 15:12:51,161][75950] Updated weights for policy 1, policy_version 42980 (0.0010) -[2023-10-14 15:12:51,547][75950] Updated weights for policy 1, policy_version 42990 (0.0009) -[2023-10-14 15:12:51,914][75950] Updated weights for policy 1, policy_version 43000 (0.0007) -[2023-10-14 15:12:53,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 88178688. Throughput: 0: 1678.1, 1: 1662.9. Samples: 22049644. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 15:12:53,164][74987] Avg episode reward: [(0, '22.480'), (1, '29.810')] -[2023-10-14 15:12:54,984][75949] Updated weights for policy 0, policy_version 43111 (0.0008) -[2023-10-14 15:12:55,356][75949] Updated weights for policy 0, policy_version 43121 (0.0010) -[2023-10-14 15:12:55,732][75949] Updated weights for policy 0, policy_version 43131 (0.0008) -[2023-10-14 15:12:55,844][75950] Updated weights for policy 1, policy_version 43010 (0.0009) -[2023-10-14 15:12:56,215][75950] Updated weights for policy 1, policy_version 43020 (0.0009) -[2023-10-14 15:12:56,592][75950] Updated weights for policy 1, policy_version 43030 (0.0010) -[2023-10-14 15:12:56,956][75950] Updated weights for policy 1, policy_version 43040 (0.0009) -[2023-10-14 15:12:58,164][74987] Fps is (10 sec: 13106.6, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 88244224. Throughput: 0: 1688.0, 1: 1670.2. Samples: 22069770. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 15:12:58,165][74987] Avg episode reward: [(0, '26.660'), (1, '31.760')] -[2023-10-14 15:12:59,867][75949] Updated weights for policy 0, policy_version 43141 (0.0008) -[2023-10-14 15:13:00,245][75949] Updated weights for policy 0, policy_version 43151 (0.0010) -[2023-10-14 15:13:00,609][75949] Updated weights for policy 0, policy_version 43161 (0.0007) -[2023-10-14 15:13:01,222][75950] Updated weights for policy 1, policy_version 43050 (0.0008) -[2023-10-14 15:13:01,590][75950] Updated weights for policy 1, policy_version 43060 (0.0009) -[2023-10-14 15:13:01,952][75950] Updated weights for policy 1, policy_version 43070 (0.0008) -[2023-10-14 15:13:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 88309760. Throughput: 0: 1661.5, 1: 1681.6. Samples: 22080254. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 15:13:03,164][74987] Avg episode reward: [(0, '21.690'), (1, '30.680')] -[2023-10-14 15:13:04,728][75949] Updated weights for policy 0, policy_version 43171 (0.0007) -[2023-10-14 15:13:05,094][75949] Updated weights for policy 0, policy_version 43181 (0.0007) -[2023-10-14 15:13:05,462][75949] Updated weights for policy 0, policy_version 43191 (0.0007) -[2023-10-14 15:13:05,797][75950] Updated weights for policy 1, policy_version 43080 (0.0009) -[2023-10-14 15:13:06,163][75950] Updated weights for policy 1, policy_version 43090 (0.0009) -[2023-10-14 15:13:06,534][75950] Updated weights for policy 1, policy_version 43100 (0.0009) -[2023-10-14 15:13:08,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.3). Total num frames: 88375296. Throughput: 0: 1677.9, 1: 1662.1. Samples: 22099696. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 15:13:08,165][74987] Avg episode reward: [(0, '26.130'), (1, '29.610')] -[2023-10-14 15:13:09,350][75949] Updated weights for policy 0, policy_version 43201 (0.0008) -[2023-10-14 15:13:09,720][75949] Updated weights for policy 0, policy_version 43211 (0.0008) -[2023-10-14 15:13:10,098][75949] Updated weights for policy 0, policy_version 43221 (0.0008) -[2023-10-14 15:13:10,465][75949] Updated weights for policy 0, policy_version 43231 (0.0008) -[2023-10-14 15:13:10,722][75950] Updated weights for policy 1, policy_version 43110 (0.0008) -[2023-10-14 15:13:11,092][75950] Updated weights for policy 1, policy_version 43120 (0.0007) -[2023-10-14 15:13:11,461][75950] Updated weights for policy 1, policy_version 43130 (0.0009) -[2023-10-14 15:13:13,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 88440832. Throughput: 0: 1688.5, 1: 1679.0. Samples: 22120298. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:13:13,165][74987] Avg episode reward: [(0, '22.130'), (1, '30.050')] -[2023-10-14 15:13:14,460][75949] Updated weights for policy 0, policy_version 43241 (0.0009) -[2023-10-14 15:13:14,815][75949] Updated weights for policy 0, policy_version 43251 (0.0009) -[2023-10-14 15:13:15,199][75949] Updated weights for policy 0, policy_version 43261 (0.0010) -[2023-10-14 15:13:15,379][75950] Updated weights for policy 1, policy_version 43140 (0.0008) -[2023-10-14 15:13:15,747][75950] Updated weights for policy 1, policy_version 43150 (0.0007) -[2023-10-14 15:13:16,108][75950] Updated weights for policy 1, policy_version 43160 (0.0010) -[2023-10-14 15:13:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 88506368. Throughput: 0: 1665.9, 1: 1677.3. Samples: 22130288. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:13:18,165][74987] Avg episode reward: [(0, '24.890'), (1, '30.970')] -[2023-10-14 15:13:19,397][75949] Updated weights for policy 0, policy_version 43271 (0.0010) -[2023-10-14 15:13:19,773][75949] Updated weights for policy 0, policy_version 43281 (0.0008) -[2023-10-14 15:13:20,140][75949] Updated weights for policy 0, policy_version 43291 (0.0009) -[2023-10-14 15:13:20,262][75950] Updated weights for policy 1, policy_version 43170 (0.0010) -[2023-10-14 15:13:20,623][75950] Updated weights for policy 1, policy_version 43180 (0.0007) -[2023-10-14 15:13:20,991][75950] Updated weights for policy 1, policy_version 43190 (0.0007) -[2023-10-14 15:13:21,353][75950] Updated weights for policy 1, policy_version 43200 (0.0009) -[2023-10-14 15:13:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 88571904. Throughput: 0: 1684.8, 1: 1662.8. Samples: 22150152. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:13:23,165][74987] Avg episode reward: [(0, '22.050'), (1, '30.640')] -[2023-10-14 15:13:24,192][75949] Updated weights for policy 0, policy_version 43301 (0.0007) -[2023-10-14 15:13:24,558][75949] Updated weights for policy 0, policy_version 43311 (0.0008) -[2023-10-14 15:13:24,922][75949] Updated weights for policy 0, policy_version 43321 (0.0010) -[2023-10-14 15:13:25,199][75950] Updated weights for policy 1, policy_version 43210 (0.0007) -[2023-10-14 15:13:25,563][75950] Updated weights for policy 1, policy_version 43220 (0.0009) -[2023-10-14 15:13:25,932][75950] Updated weights for policy 1, policy_version 43230 (0.0009) -[2023-10-14 15:13:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 88637440. Throughput: 0: 1682.0, 1: 1687.9. Samples: 22171142. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:13:28,165][74987] Avg episode reward: [(0, '24.100'), (1, '29.780')] -[2023-10-14 15:13:28,979][75949] Updated weights for policy 0, policy_version 43331 (0.0009) -[2023-10-14 15:13:29,339][75949] Updated weights for policy 0, policy_version 43341 (0.0008) -[2023-10-14 15:13:29,716][75949] Updated weights for policy 0, policy_version 43351 (0.0010) -[2023-10-14 15:13:29,997][75950] Updated weights for policy 1, policy_version 43240 (0.0010) -[2023-10-14 15:13:30,364][75950] Updated weights for policy 1, policy_version 43250 (0.0007) -[2023-10-14 15:13:30,737][75950] Updated weights for policy 1, policy_version 43260 (0.0008) -[2023-10-14 15:13:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 88702976. Throughput: 0: 1665.7, 1: 1666.0. Samples: 22180476. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:13:33,165][74987] Avg episode reward: [(0, '25.210'), (1, '30.420')] -[2023-10-14 15:13:33,804][75949] Updated weights for policy 0, policy_version 43361 (0.0008) -[2023-10-14 15:13:34,172][75949] Updated weights for policy 0, policy_version 43371 (0.0011) -[2023-10-14 15:13:34,541][75949] Updated weights for policy 0, policy_version 43381 (0.0012) -[2023-10-14 15:13:34,864][75950] Updated weights for policy 1, policy_version 43270 (0.0007) -[2023-10-14 15:13:34,916][75949] Updated weights for policy 0, policy_version 43391 (0.0010) -[2023-10-14 15:13:35,229][75950] Updated weights for policy 1, policy_version 43280 (0.0007) -[2023-10-14 15:13:35,599][75950] Updated weights for policy 1, policy_version 43290 (0.0007) -[2023-10-14 15:13:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 88768512. Throughput: 0: 1683.2, 1: 1674.9. Samples: 22200762. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:13:38,164][74987] Avg episode reward: [(0, '23.490'), (1, '31.130')] -[2023-10-14 15:13:38,994][75949] Updated weights for policy 0, policy_version 43401 (0.0009) -[2023-10-14 15:13:39,363][75949] Updated weights for policy 0, policy_version 43411 (0.0008) -[2023-10-14 15:13:39,729][75949] Updated weights for policy 0, policy_version 43421 (0.0009) -[2023-10-14 15:13:39,872][75950] Updated weights for policy 1, policy_version 43300 (0.0008) -[2023-10-14 15:13:40,279][75950] Updated weights for policy 1, policy_version 43310 (0.0007) -[2023-10-14 15:13:40,648][75950] Updated weights for policy 1, policy_version 43320 (0.0009) -[2023-10-14 15:13:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 88834048. Throughput: 0: 1683.4, 1: 1684.2. Samples: 22221314. Policy #0 lag: (min: 11.0, avg: 12.1, max: 29.0) -[2023-10-14 15:13:43,165][74987] Avg episode reward: [(0, '26.530'), (1, '29.540')] -[2023-10-14 15:13:43,732][75949] Updated weights for policy 0, policy_version 43431 (0.0008) -[2023-10-14 15:13:44,117][75949] Updated weights for policy 0, policy_version 43441 (0.0010) -[2023-10-14 15:13:44,489][75949] Updated weights for policy 0, policy_version 43451 (0.0009) -[2023-10-14 15:13:44,628][75950] Updated weights for policy 1, policy_version 43330 (0.0008) -[2023-10-14 15:13:45,004][75950] Updated weights for policy 1, policy_version 43340 (0.0008) -[2023-10-14 15:13:45,375][75950] Updated weights for policy 1, policy_version 43350 (0.0009) -[2023-10-14 15:13:45,747][75950] Updated weights for policy 1, policy_version 43360 (0.0009) -[2023-10-14 15:13:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 88899584. Throughput: 0: 1678.5, 1: 1659.9. Samples: 22230480. Policy #0 lag: (min: 11.0, avg: 12.1, max: 29.0) -[2023-10-14 15:13:48,165][74987] Avg episode reward: [(0, '23.620'), (1, '29.820')] -[2023-10-14 15:13:48,800][75949] Updated weights for policy 0, policy_version 43461 (0.0009) -[2023-10-14 15:13:49,170][75949] Updated weights for policy 0, policy_version 43471 (0.0007) -[2023-10-14 15:13:49,542][75949] Updated weights for policy 0, policy_version 43481 (0.0009) -[2023-10-14 15:13:49,945][75950] Updated weights for policy 1, policy_version 43370 (0.0009) -[2023-10-14 15:13:50,309][75950] Updated weights for policy 1, policy_version 43380 (0.0009) -[2023-10-14 15:13:50,687][75950] Updated weights for policy 1, policy_version 43390 (0.0008) -[2023-10-14 15:13:53,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 88965120. Throughput: 0: 1682.9, 1: 1672.8. Samples: 22250700. Policy #0 lag: (min: 11.0, avg: 12.1, max: 29.0) -[2023-10-14 15:13:53,164][74987] Avg episode reward: [(0, '26.900'), (1, '29.680')] -[2023-10-14 15:13:53,655][75949] Updated weights for policy 0, policy_version 43491 (0.0010) -[2023-10-14 15:13:54,030][75949] Updated weights for policy 0, policy_version 43501 (0.0011) -[2023-10-14 15:13:54,398][75949] Updated weights for policy 0, policy_version 43511 (0.0010) -[2023-10-14 15:13:54,756][75950] Updated weights for policy 1, policy_version 43400 (0.0009) -[2023-10-14 15:13:55,128][75950] Updated weights for policy 1, policy_version 43410 (0.0007) -[2023-10-14 15:13:55,499][75950] Updated weights for policy 1, policy_version 43420 (0.0008) -[2023-10-14 15:13:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 89030656. Throughput: 0: 1679.2, 1: 1679.5. Samples: 22271438. Policy #0 lag: (min: 11.0, avg: 12.1, max: 29.0) -[2023-10-14 15:13:58,165][74987] Avg episode reward: [(0, '24.390'), (1, '29.430')] -[2023-10-14 15:13:58,482][75949] Updated weights for policy 0, policy_version 43521 (0.0009) -[2023-10-14 15:13:58,846][75949] Updated weights for policy 0, policy_version 43531 (0.0011) -[2023-10-14 15:13:59,223][75949] Updated weights for policy 0, policy_version 43541 (0.0008) -[2023-10-14 15:13:59,586][75949] Updated weights for policy 0, policy_version 43551 (0.0010) -[2023-10-14 15:13:59,631][75950] Updated weights for policy 1, policy_version 43430 (0.0008) -[2023-10-14 15:13:59,999][75950] Updated weights for policy 1, policy_version 43440 (0.0008) -[2023-10-14 15:14:00,366][75950] Updated weights for policy 1, policy_version 43450 (0.0007) -[2023-10-14 15:14:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89096192. Throughput: 0: 1680.9, 1: 1659.2. Samples: 22280592. Policy #0 lag: (min: 11.0, avg: 12.1, max: 29.0) -[2023-10-14 15:14:03,164][74987] Avg episode reward: [(0, '27.720'), (1, '29.120')] -[2023-10-14 15:14:03,572][75949] Updated weights for policy 0, policy_version 43561 (0.0008) -[2023-10-14 15:14:03,933][75949] Updated weights for policy 0, policy_version 43571 (0.0009) -[2023-10-14 15:14:04,300][75949] Updated weights for policy 0, policy_version 43581 (0.0007) -[2023-10-14 15:14:04,428][75950] Updated weights for policy 1, policy_version 43460 (0.0007) -[2023-10-14 15:14:04,794][75950] Updated weights for policy 1, policy_version 43470 (0.0008) -[2023-10-14 15:14:05,172][75950] Updated weights for policy 1, policy_version 43480 (0.0008) -[2023-10-14 15:14:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89161728. Throughput: 0: 1681.0, 1: 1675.9. Samples: 22301210. Policy #0 lag: (min: 11.0, avg: 12.1, max: 29.0) -[2023-10-14 15:14:08,164][74987] Avg episode reward: [(0, '24.360'), (1, '30.050')] -[2023-10-14 15:14:08,263][75949] Updated weights for policy 0, policy_version 43591 (0.0007) -[2023-10-14 15:14:08,629][75949] Updated weights for policy 0, policy_version 43601 (0.0008) -[2023-10-14 15:14:09,002][75949] Updated weights for policy 0, policy_version 43611 (0.0009) -[2023-10-14 15:14:09,254][75950] Updated weights for policy 1, policy_version 43490 (0.0007) -[2023-10-14 15:14:09,615][75950] Updated weights for policy 1, policy_version 43500 (0.0008) -[2023-10-14 15:14:09,993][75950] Updated weights for policy 1, policy_version 43510 (0.0007) -[2023-10-14 15:14:10,364][75950] Updated weights for policy 1, policy_version 43520 (0.0010) -[2023-10-14 15:14:12,850][75949] Updated weights for policy 0, policy_version 43621 (0.0009) -[2023-10-14 15:14:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 89227264. Throughput: 0: 1685.6, 1: 1670.9. Samples: 22322184. Policy #0 lag: (min: 27.0, avg: 27.0, max: 28.0) -[2023-10-14 15:14:13,165][74987] Avg episode reward: [(0, '26.460'), (1, '30.390')] -[2023-10-14 15:14:13,217][75949] Updated weights for policy 0, policy_version 43631 (0.0008) -[2023-10-14 15:14:13,593][75949] Updated weights for policy 0, policy_version 43641 (0.0008) -[2023-10-14 15:14:14,793][75950] Updated weights for policy 1, policy_version 43530 (0.0010) -[2023-10-14 15:14:15,155][75950] Updated weights for policy 1, policy_version 43540 (0.0009) -[2023-10-14 15:14:15,533][75950] Updated weights for policy 1, policy_version 43550 (0.0008) -[2023-10-14 15:14:17,807][75949] Updated weights for policy 0, policy_version 43651 (0.0009) -[2023-10-14 15:14:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89292800. Throughput: 0: 1686.2, 1: 1661.9. Samples: 22331140. Policy #0 lag: (min: 27.0, avg: 27.0, max: 28.0) -[2023-10-14 15:14:18,165][74987] Avg episode reward: [(0, '23.200'), (1, '29.780')] -[2023-10-14 15:14:18,183][75949] Updated weights for policy 0, policy_version 43661 (0.0009) -[2023-10-14 15:14:18,552][75949] Updated weights for policy 0, policy_version 43671 (0.0009) -[2023-10-14 15:14:19,634][75950] Updated weights for policy 1, policy_version 43560 (0.0009) -[2023-10-14 15:14:20,006][75950] Updated weights for policy 1, policy_version 43570 (0.0010) -[2023-10-14 15:14:20,367][75950] Updated weights for policy 1, policy_version 43580 (0.0008) -[2023-10-14 15:14:22,569][75949] Updated weights for policy 0, policy_version 43681 (0.0008) -[2023-10-14 15:14:22,931][75949] Updated weights for policy 0, policy_version 43691 (0.0007) -[2023-10-14 15:14:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 89358336. Throughput: 0: 1687.6, 1: 1666.4. Samples: 22351690. Policy #0 lag: (min: 27.0, avg: 27.0, max: 28.0) -[2023-10-14 15:14:23,164][74987] Avg episode reward: [(0, '25.620'), (1, '28.870')] -[2023-10-14 15:14:23,309][75949] Updated weights for policy 0, policy_version 43701 (0.0008) -[2023-10-14 15:14:23,667][75949] Updated weights for policy 0, policy_version 43711 (0.0010) -[2023-10-14 15:14:24,396][75950] Updated weights for policy 1, policy_version 43590 (0.0010) -[2023-10-14 15:14:24,760][75950] Updated weights for policy 1, policy_version 43600 (0.0008) -[2023-10-14 15:14:25,127][75950] Updated weights for policy 1, policy_version 43610 (0.0011) -[2023-10-14 15:14:27,708][75949] Updated weights for policy 0, policy_version 43721 (0.0009) -[2023-10-14 15:14:28,074][75949] Updated weights for policy 0, policy_version 43731 (0.0010) -[2023-10-14 15:14:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 89423872. Throughput: 0: 1681.2, 1: 1666.6. Samples: 22371962. Policy #0 lag: (min: 27.0, avg: 27.0, max: 28.0) -[2023-10-14 15:14:28,164][74987] Avg episode reward: [(0, '23.210'), (1, '30.740')] -[2023-10-14 15:14:28,443][75949] Updated weights for policy 0, policy_version 43741 (0.0009) -[2023-10-14 15:14:29,391][75950] Updated weights for policy 1, policy_version 43620 (0.0010) -[2023-10-14 15:14:29,763][75950] Updated weights for policy 1, policy_version 43630 (0.0009) -[2023-10-14 15:14:30,138][75950] Updated weights for policy 1, policy_version 43640 (0.0009) -[2023-10-14 15:14:32,476][75949] Updated weights for policy 0, policy_version 43751 (0.0008) -[2023-10-14 15:14:32,854][75949] Updated weights for policy 0, policy_version 43761 (0.0008) -[2023-10-14 15:14:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 89489408. Throughput: 0: 1691.3, 1: 1661.4. Samples: 22381354. Policy #0 lag: (min: 27.0, avg: 27.0, max: 28.0) -[2023-10-14 15:14:33,165][74987] Avg episode reward: [(0, '24.990'), (1, '29.220')] -[2023-10-14 15:14:33,218][75949] Updated weights for policy 0, policy_version 43771 (0.0008) -[2023-10-14 15:14:34,105][75950] Updated weights for policy 1, policy_version 43650 (0.0010) -[2023-10-14 15:14:34,475][75950] Updated weights for policy 1, policy_version 43660 (0.0008) -[2023-10-14 15:14:34,839][75950] Updated weights for policy 1, policy_version 43670 (0.0008) -[2023-10-14 15:14:35,219][75950] Updated weights for policy 1, policy_version 43680 (0.0011) -[2023-10-14 15:14:37,209][75949] Updated weights for policy 0, policy_version 43781 (0.0009) -[2023-10-14 15:14:37,581][75949] Updated weights for policy 0, policy_version 43791 (0.0010) -[2023-10-14 15:14:37,953][75949] Updated weights for policy 0, policy_version 43801 (0.0008) -[2023-10-14 15:14:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89554944. Throughput: 0: 1691.9, 1: 1672.0. Samples: 22402078. Policy #0 lag: (min: 27.0, avg: 27.0, max: 28.0) -[2023-10-14 15:14:38,165][74987] Avg episode reward: [(0, '24.300'), (1, '29.850')] -[2023-10-14 15:14:39,153][75950] Updated weights for policy 1, policy_version 43690 (0.0008) -[2023-10-14 15:14:39,520][75950] Updated weights for policy 1, policy_version 43700 (0.0010) -[2023-10-14 15:14:39,888][75950] Updated weights for policy 1, policy_version 43710 (0.0010) -[2023-10-14 15:14:42,012][75949] Updated weights for policy 0, policy_version 43811 (0.0008) -[2023-10-14 15:14:42,383][75949] Updated weights for policy 0, policy_version 43821 (0.0007) -[2023-10-14 15:14:42,745][75949] Updated weights for policy 0, policy_version 43831 (0.0008) -[2023-10-14 15:14:43,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 89653248. Throughput: 0: 1675.2, 1: 1673.5. Samples: 22422128. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) -[2023-10-14 15:14:43,164][74987] Avg episode reward: [(0, '25.730'), (1, '31.130')] -[2023-10-14 15:14:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000043712_44761088.pth... -[2023-10-14 15:14:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000043840_44892160.pth... -[2023-10-14 15:14:43,216][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000042176_43188224.pth -[2023-10-14 15:14:43,218][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000042272_43286528.pth -[2023-10-14 15:14:43,993][75950] Updated weights for policy 1, policy_version 43720 (0.0009) -[2023-10-14 15:14:44,362][75950] Updated weights for policy 1, policy_version 43730 (0.0008) -[2023-10-14 15:14:44,733][75950] Updated weights for policy 1, policy_version 43740 (0.0008) -[2023-10-14 15:14:46,713][75949] Updated weights for policy 0, policy_version 43841 (0.0007) -[2023-10-14 15:14:47,074][75949] Updated weights for policy 0, policy_version 43851 (0.0007) -[2023-10-14 15:14:47,453][75949] Updated weights for policy 0, policy_version 43861 (0.0008) -[2023-10-14 15:14:47,831][75949] Updated weights for policy 0, policy_version 43871 (0.0009) -[2023-10-14 15:14:48,163][74987] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 89718784. Throughput: 0: 1694.7, 1: 1670.4. Samples: 22432022. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) -[2023-10-14 15:14:48,164][74987] Avg episode reward: [(0, '23.450'), (1, '29.140')] -[2023-10-14 15:14:48,805][75950] Updated weights for policy 1, policy_version 43750 (0.0010) -[2023-10-14 15:14:49,182][75950] Updated weights for policy 1, policy_version 43760 (0.0009) -[2023-10-14 15:14:49,549][75950] Updated weights for policy 1, policy_version 43770 (0.0008) -[2023-10-14 15:14:51,943][75949] Updated weights for policy 0, policy_version 43881 (0.0010) -[2023-10-14 15:14:52,319][75949] Updated weights for policy 0, policy_version 43891 (0.0009) -[2023-10-14 15:14:52,687][75949] Updated weights for policy 0, policy_version 43901 (0.0010) -[2023-10-14 15:14:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 89784320. Throughput: 0: 1686.6, 1: 1669.5. Samples: 22452232. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) -[2023-10-14 15:14:53,165][74987] Avg episode reward: [(0, '24.520'), (1, '30.750')] -[2023-10-14 15:14:53,718][75950] Updated weights for policy 1, policy_version 43780 (0.0008) -[2023-10-14 15:14:54,086][75950] Updated weights for policy 1, policy_version 43790 (0.0009) -[2023-10-14 15:14:54,445][75950] Updated weights for policy 1, policy_version 43800 (0.0008) -[2023-10-14 15:14:56,963][75949] Updated weights for policy 0, policy_version 43911 (0.0009) -[2023-10-14 15:14:57,324][75949] Updated weights for policy 0, policy_version 43921 (0.0010) -[2023-10-14 15:14:57,694][75949] Updated weights for policy 0, policy_version 43931 (0.0009) -[2023-10-14 15:14:58,164][74987] Fps is (10 sec: 13106.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 89849856. Throughput: 0: 1659.7, 1: 1670.3. Samples: 22472034. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) -[2023-10-14 15:14:58,165][74987] Avg episode reward: [(0, '23.280'), (1, '31.150')] -[2023-10-14 15:14:58,399][75950] Updated weights for policy 1, policy_version 43810 (0.0008) -[2023-10-14 15:14:58,758][75950] Updated weights for policy 1, policy_version 43820 (0.0007) -[2023-10-14 15:14:59,123][75950] Updated weights for policy 1, policy_version 43830 (0.0008) -[2023-10-14 15:14:59,496][75950] Updated weights for policy 1, policy_version 43840 (0.0008) -[2023-10-14 15:15:01,844][75949] Updated weights for policy 0, policy_version 43941 (0.0010) -[2023-10-14 15:15:02,218][75949] Updated weights for policy 0, policy_version 43951 (0.0009) -[2023-10-14 15:15:02,591][75949] Updated weights for policy 0, policy_version 43961 (0.0007) -[2023-10-14 15:15:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 89915392. Throughput: 0: 1684.1, 1: 1676.2. Samples: 22482354. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) -[2023-10-14 15:15:03,164][74987] Avg episode reward: [(0, '23.630'), (1, '28.950')] -[2023-10-14 15:15:03,500][75950] Updated weights for policy 1, policy_version 43850 (0.0008) -[2023-10-14 15:15:03,863][75950] Updated weights for policy 1, policy_version 43860 (0.0008) -[2023-10-14 15:15:04,229][75950] Updated weights for policy 1, policy_version 43870 (0.0008) -[2023-10-14 15:15:06,509][75949] Updated weights for policy 0, policy_version 43971 (0.0008) -[2023-10-14 15:15:06,882][75949] Updated weights for policy 0, policy_version 43981 (0.0008) -[2023-10-14 15:15:07,241][75949] Updated weights for policy 0, policy_version 43991 (0.0011) -[2023-10-14 15:15:08,163][74987] Fps is (10 sec: 13107.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 89980928. Throughput: 0: 1679.9, 1: 1681.5. Samples: 22502952. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-14 15:15:08,164][74987] Avg episode reward: [(0, '26.690'), (1, '30.800')] -[2023-10-14 15:15:08,204][75950] Updated weights for policy 1, policy_version 43880 (0.0008) -[2023-10-14 15:15:08,572][75950] Updated weights for policy 1, policy_version 43890 (0.0009) -[2023-10-14 15:15:08,934][75950] Updated weights for policy 1, policy_version 43900 (0.0010) -[2023-10-14 15:15:11,155][75949] Updated weights for policy 0, policy_version 44001 (0.0011) -[2023-10-14 15:15:11,521][75949] Updated weights for policy 0, policy_version 44011 (0.0010) -[2023-10-14 15:15:11,895][75949] Updated weights for policy 0, policy_version 44021 (0.0009) -[2023-10-14 15:15:12,276][75949] Updated weights for policy 0, policy_version 44031 (0.0010) -[2023-10-14 15:15:13,054][75950] Updated weights for policy 1, policy_version 43910 (0.0008) -[2023-10-14 15:15:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 90046464. Throughput: 0: 1665.6, 1: 1691.4. Samples: 22523028. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-14 15:15:13,165][74987] Avg episode reward: [(0, '23.150'), (1, '30.170')] -[2023-10-14 15:15:13,440][75950] Updated weights for policy 1, policy_version 43920 (0.0009) -[2023-10-14 15:15:13,800][75950] Updated weights for policy 1, policy_version 43930 (0.0009) -[2023-10-14 15:15:16,302][75949] Updated weights for policy 0, policy_version 44041 (0.0007) -[2023-10-14 15:15:16,674][75949] Updated weights for policy 0, policy_version 44051 (0.0010) -[2023-10-14 15:15:17,039][75949] Updated weights for policy 0, policy_version 44061 (0.0009) -[2023-10-14 15:15:17,769][75950] Updated weights for policy 1, policy_version 43940 (0.0010) -[2023-10-14 15:15:18,130][75950] Updated weights for policy 1, policy_version 43950 (0.0009) -[2023-10-14 15:15:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 90112000. Throughput: 0: 1686.8, 1: 1690.5. Samples: 22533336. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-14 15:15:18,165][74987] Avg episode reward: [(0, '25.010'), (1, '30.330')] -[2023-10-14 15:15:18,501][75950] Updated weights for policy 1, policy_version 43960 (0.0008) -[2023-10-14 15:15:21,216][75949] Updated weights for policy 0, policy_version 44071 (0.0008) -[2023-10-14 15:15:21,590][75949] Updated weights for policy 0, policy_version 44081 (0.0010) -[2023-10-14 15:15:21,964][75949] Updated weights for policy 0, policy_version 44091 (0.0010) -[2023-10-14 15:15:22,681][75950] Updated weights for policy 1, policy_version 43970 (0.0009) -[2023-10-14 15:15:23,041][75950] Updated weights for policy 1, policy_version 43980 (0.0011) -[2023-10-14 15:15:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 90177536. Throughput: 0: 1670.4, 1: 1691.0. Samples: 22553340. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-14 15:15:23,164][74987] Avg episode reward: [(0, '24.300'), (1, '29.200')] -[2023-10-14 15:15:23,416][75950] Updated weights for policy 1, policy_version 43990 (0.0009) -[2023-10-14 15:15:23,784][75950] Updated weights for policy 1, policy_version 44000 (0.0010) -[2023-10-14 15:15:25,964][75949] Updated weights for policy 0, policy_version 44101 (0.0009) -[2023-10-14 15:15:26,329][75949] Updated weights for policy 0, policy_version 44111 (0.0007) -[2023-10-14 15:15:26,692][75949] Updated weights for policy 0, policy_version 44121 (0.0010) -[2023-10-14 15:15:27,823][75950] Updated weights for policy 1, policy_version 44010 (0.0010) -[2023-10-14 15:15:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 90243072. Throughput: 0: 1673.0, 1: 1682.2. Samples: 22573112. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-14 15:15:28,165][74987] Avg episode reward: [(0, '25.490'), (1, '28.940')] -[2023-10-14 15:15:28,193][75950] Updated weights for policy 1, policy_version 44020 (0.0009) -[2023-10-14 15:15:28,563][75950] Updated weights for policy 1, policy_version 44030 (0.0007) -[2023-10-14 15:15:30,746][75949] Updated weights for policy 0, policy_version 44131 (0.0009) -[2023-10-14 15:15:31,108][75949] Updated weights for policy 0, policy_version 44141 (0.0011) -[2023-10-14 15:15:31,484][75949] Updated weights for policy 0, policy_version 44151 (0.0009) -[2023-10-14 15:15:32,582][75950] Updated weights for policy 1, policy_version 44040 (0.0007) -[2023-10-14 15:15:32,948][75950] Updated weights for policy 1, policy_version 44050 (0.0008) -[2023-10-14 15:15:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 90308608. Throughput: 0: 1682.4, 1: 1687.3. Samples: 22583660. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) -[2023-10-14 15:15:33,165][74987] Avg episode reward: [(0, '26.950'), (1, '29.280')] -[2023-10-14 15:15:33,309][75950] Updated weights for policy 1, policy_version 44060 (0.0009) -[2023-10-14 15:15:35,512][75949] Updated weights for policy 0, policy_version 44161 (0.0009) -[2023-10-14 15:15:35,887][75949] Updated weights for policy 0, policy_version 44171 (0.0007) -[2023-10-14 15:15:36,253][75949] Updated weights for policy 0, policy_version 44181 (0.0009) -[2023-10-14 15:15:36,632][75949] Updated weights for policy 0, policy_version 44191 (0.0012) -[2023-10-14 15:15:37,506][75950] Updated weights for policy 1, policy_version 44070 (0.0010) -[2023-10-14 15:15:37,875][75950] Updated weights for policy 1, policy_version 44080 (0.0010) -[2023-10-14 15:15:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 90374144. Throughput: 0: 1665.9, 1: 1692.5. Samples: 22603362. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) -[2023-10-14 15:15:38,165][74987] Avg episode reward: [(0, '25.730'), (1, '29.250')] -[2023-10-14 15:15:38,245][75950] Updated weights for policy 1, policy_version 44090 (0.0011) -[2023-10-14 15:15:40,647][75949] Updated weights for policy 0, policy_version 44201 (0.0008) -[2023-10-14 15:15:41,021][75949] Updated weights for policy 0, policy_version 44211 (0.0008) -[2023-10-14 15:15:41,378][75949] Updated weights for policy 0, policy_version 44221 (0.0011) -[2023-10-14 15:15:42,244][75950] Updated weights for policy 1, policy_version 44100 (0.0010) -[2023-10-14 15:15:42,607][75950] Updated weights for policy 1, policy_version 44110 (0.0008) -[2023-10-14 15:15:42,981][75950] Updated weights for policy 1, policy_version 44120 (0.0008) -[2023-10-14 15:15:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 90439680. Throughput: 0: 1687.4, 1: 1678.9. Samples: 22623518. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) -[2023-10-14 15:15:43,165][74987] Avg episode reward: [(0, '23.650'), (1, '30.010')] -[2023-10-14 15:15:45,488][75949] Updated weights for policy 0, policy_version 44231 (0.0007) -[2023-10-14 15:15:45,853][75949] Updated weights for policy 0, policy_version 44241 (0.0008) -[2023-10-14 15:15:46,225][75949] Updated weights for policy 0, policy_version 44251 (0.0008) -[2023-10-14 15:15:47,096][75950] Updated weights for policy 1, policy_version 44130 (0.0008) -[2023-10-14 15:15:47,465][75950] Updated weights for policy 1, policy_version 44140 (0.0008) -[2023-10-14 15:15:47,835][75950] Updated weights for policy 1, policy_version 44150 (0.0009) -[2023-10-14 15:15:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 90505216. Throughput: 0: 1679.9, 1: 1688.3. Samples: 22633924. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) -[2023-10-14 15:15:48,165][74987] Avg episode reward: [(0, '25.190'), (1, '29.460')] -[2023-10-14 15:15:48,192][75950] Updated weights for policy 1, policy_version 44160 (0.0011) -[2023-10-14 15:15:50,346][75949] Updated weights for policy 0, policy_version 44261 (0.0008) -[2023-10-14 15:15:50,726][75949] Updated weights for policy 0, policy_version 44271 (0.0007) -[2023-10-14 15:15:51,094][75949] Updated weights for policy 0, policy_version 44281 (0.0008) -[2023-10-14 15:15:52,375][75950] Updated weights for policy 1, policy_version 44170 (0.0007) -[2023-10-14 15:15:52,743][75950] Updated weights for policy 1, policy_version 44180 (0.0007) -[2023-10-14 15:15:53,112][75950] Updated weights for policy 1, policy_version 44190 (0.0007) -[2023-10-14 15:15:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 90570752. Throughput: 0: 1663.1, 1: 1684.4. Samples: 22653588. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) -[2023-10-14 15:15:53,164][74987] Avg episode reward: [(0, '24.680'), (1, '32.490')] -[2023-10-14 15:15:55,315][75949] Updated weights for policy 0, policy_version 44291 (0.0008) -[2023-10-14 15:15:55,692][75949] Updated weights for policy 0, policy_version 44301 (0.0010) -[2023-10-14 15:15:56,064][75949] Updated weights for policy 0, policy_version 44311 (0.0009) -[2023-10-14 15:15:57,126][75950] Updated weights for policy 1, policy_version 44200 (0.0010) -[2023-10-14 15:15:57,503][75950] Updated weights for policy 1, policy_version 44210 (0.0009) -[2023-10-14 15:15:57,872][75950] Updated weights for policy 1, policy_version 44220 (0.0007) -[2023-10-14 15:15:58,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 90669056. Throughput: 0: 1680.0, 1: 1665.0. Samples: 22673554. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) -[2023-10-14 15:15:58,165][74987] Avg episode reward: [(0, '25.290'), (1, '29.100')] -[2023-10-14 15:16:00,238][75949] Updated weights for policy 0, policy_version 44321 (0.0011) -[2023-10-14 15:16:00,607][75949] Updated weights for policy 0, policy_version 44331 (0.0007) -[2023-10-14 15:16:00,972][75949] Updated weights for policy 0, policy_version 44341 (0.0007) -[2023-10-14 15:16:01,343][75949] Updated weights for policy 0, policy_version 44351 (0.0009) -[2023-10-14 15:16:01,962][75950] Updated weights for policy 1, policy_version 44230 (0.0008) -[2023-10-14 15:16:02,321][75950] Updated weights for policy 1, policy_version 44240 (0.0008) -[2023-10-14 15:16:02,683][75950] Updated weights for policy 1, policy_version 44250 (0.0009) -[2023-10-14 15:16:03,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 90734592. Throughput: 0: 1667.0, 1: 1685.4. Samples: 22684194. Policy #0 lag: (min: 22.0, avg: 26.7, max: 54.0) -[2023-10-14 15:16:03,164][74987] Avg episode reward: [(0, '25.790'), (1, '30.570')] -[2023-10-14 15:16:05,326][75949] Updated weights for policy 0, policy_version 44361 (0.0008) -[2023-10-14 15:16:05,697][75949] Updated weights for policy 0, policy_version 44371 (0.0008) -[2023-10-14 15:16:06,063][75949] Updated weights for policy 0, policy_version 44381 (0.0008) -[2023-10-14 15:16:06,668][75950] Updated weights for policy 1, policy_version 44260 (0.0009) -[2023-10-14 15:16:07,029][75950] Updated weights for policy 1, policy_version 44270 (0.0010) -[2023-10-14 15:16:07,408][75950] Updated weights for policy 1, policy_version 44280 (0.0009) -[2023-10-14 15:16:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 90800128. Throughput: 0: 1667.7, 1: 1679.9. Samples: 22703980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:16:08,164][74987] Avg episode reward: [(0, '23.890'), (1, '33.210')] -[2023-10-14 15:16:08,165][75801] Saving new best policy, reward=33.210! -[2023-10-14 15:16:10,267][75949] Updated weights for policy 0, policy_version 44391 (0.0008) -[2023-10-14 15:16:10,638][75949] Updated weights for policy 0, policy_version 44401 (0.0009) -[2023-10-14 15:16:11,018][75949] Updated weights for policy 0, policy_version 44411 (0.0008) -[2023-10-14 15:16:11,381][75950] Updated weights for policy 1, policy_version 44290 (0.0008) -[2023-10-14 15:16:11,758][75950] Updated weights for policy 1, policy_version 44300 (0.0009) -[2023-10-14 15:16:12,116][75950] Updated weights for policy 1, policy_version 44310 (0.0008) -[2023-10-14 15:16:12,485][75950] Updated weights for policy 1, policy_version 44320 (0.0007) -[2023-10-14 15:16:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 90865664. Throughput: 0: 1681.0, 1: 1667.1. Samples: 22723776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:16:13,165][74987] Avg episode reward: [(0, '24.770'), (1, '29.800')] -[2023-10-14 15:16:15,136][75949] Updated weights for policy 0, policy_version 44421 (0.0009) -[2023-10-14 15:16:15,506][75949] Updated weights for policy 0, policy_version 44431 (0.0011) -[2023-10-14 15:16:15,886][75949] Updated weights for policy 0, policy_version 44441 (0.0009) -[2023-10-14 15:16:16,619][75950] Updated weights for policy 1, policy_version 44330 (0.0008) -[2023-10-14 15:16:16,982][75950] Updated weights for policy 1, policy_version 44340 (0.0009) -[2023-10-14 15:16:17,358][75950] Updated weights for policy 1, policy_version 44350 (0.0008) -[2023-10-14 15:16:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 90931200. Throughput: 0: 1661.0, 1: 1693.1. Samples: 22734594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:16:18,165][74987] Avg episode reward: [(0, '23.530'), (1, '28.750')] -[2023-10-14 15:16:20,047][75949] Updated weights for policy 0, policy_version 44451 (0.0009) -[2023-10-14 15:16:20,411][75949] Updated weights for policy 0, policy_version 44461 (0.0007) -[2023-10-14 15:16:20,777][75949] Updated weights for policy 0, policy_version 44471 (0.0008) -[2023-10-14 15:16:21,384][75950] Updated weights for policy 1, policy_version 44360 (0.0008) -[2023-10-14 15:16:21,751][75950] Updated weights for policy 1, policy_version 44370 (0.0007) -[2023-10-14 15:16:22,118][75950] Updated weights for policy 1, policy_version 44380 (0.0009) -[2023-10-14 15:16:23,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 90996736. Throughput: 0: 1670.8, 1: 1679.8. Samples: 22754136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:16:23,164][74987] Avg episode reward: [(0, '25.160'), (1, '30.970')] -[2023-10-14 15:16:24,967][75949] Updated weights for policy 0, policy_version 44481 (0.0009) -[2023-10-14 15:16:25,337][75949] Updated weights for policy 0, policy_version 44491 (0.0008) -[2023-10-14 15:16:25,694][75949] Updated weights for policy 0, policy_version 44501 (0.0007) -[2023-10-14 15:16:26,063][75949] Updated weights for policy 0, policy_version 44511 (0.0009) -[2023-10-14 15:16:26,076][75950] Updated weights for policy 1, policy_version 44390 (0.0009) -[2023-10-14 15:16:26,450][75950] Updated weights for policy 1, policy_version 44400 (0.0008) -[2023-10-14 15:16:26,807][75950] Updated weights for policy 1, policy_version 44410 (0.0010) -[2023-10-14 15:16:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 91062272. Throughput: 0: 1673.3, 1: 1676.1. Samples: 22774244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:16:28,164][74987] Avg episode reward: [(0, '24.390'), (1, '29.320')] -[2023-10-14 15:16:29,783][75949] Updated weights for policy 0, policy_version 44521 (0.0011) -[2023-10-14 15:16:30,160][75949] Updated weights for policy 0, policy_version 44531 (0.0010) -[2023-10-14 15:16:30,524][75949] Updated weights for policy 0, policy_version 44541 (0.0007) -[2023-10-14 15:16:31,140][75950] Updated weights for policy 1, policy_version 44420 (0.0008) -[2023-10-14 15:16:31,498][75950] Updated weights for policy 1, policy_version 44430 (0.0010) -[2023-10-14 15:16:31,866][75950] Updated weights for policy 1, policy_version 44440 (0.0009) -[2023-10-14 15:16:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 91127808. Throughput: 0: 1661.3, 1: 1688.7. Samples: 22784674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:16:33,164][74987] Avg episode reward: [(0, '25.110'), (1, '26.560')] -[2023-10-14 15:16:34,541][75949] Updated weights for policy 0, policy_version 44551 (0.0008) -[2023-10-14 15:16:34,907][75949] Updated weights for policy 0, policy_version 44561 (0.0010) -[2023-10-14 15:16:35,273][75949] Updated weights for policy 0, policy_version 44571 (0.0010) -[2023-10-14 15:16:35,824][75950] Updated weights for policy 1, policy_version 44450 (0.0008) -[2023-10-14 15:16:36,191][75950] Updated weights for policy 1, policy_version 44460 (0.0007) -[2023-10-14 15:16:36,554][75950] Updated weights for policy 1, policy_version 44470 (0.0011) -[2023-10-14 15:16:36,931][75950] Updated weights for policy 1, policy_version 44480 (0.0008) -[2023-10-14 15:16:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91193344. Throughput: 0: 1686.4, 1: 1669.3. Samples: 22804598. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) -[2023-10-14 15:16:38,165][74987] Avg episode reward: [(0, '23.670'), (1, '29.690')] -[2023-10-14 15:16:39,356][75949] Updated weights for policy 0, policy_version 44581 (0.0009) -[2023-10-14 15:16:39,718][75949] Updated weights for policy 0, policy_version 44591 (0.0010) -[2023-10-14 15:16:40,089][75949] Updated weights for policy 0, policy_version 44601 (0.0010) -[2023-10-14 15:16:41,090][75950] Updated weights for policy 1, policy_version 44490 (0.0007) -[2023-10-14 15:16:41,459][75950] Updated weights for policy 1, policy_version 44500 (0.0009) -[2023-10-14 15:16:41,823][75950] Updated weights for policy 1, policy_version 44510 (0.0009) -[2023-10-14 15:16:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91258880. Throughput: 0: 1687.5, 1: 1673.3. Samples: 22824790. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) -[2023-10-14 15:16:43,165][74987] Avg episode reward: [(0, '24.860'), (1, '29.810')] -[2023-10-14 15:16:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000044608_45678592.pth... -[2023-10-14 15:16:43,177][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000044512_45580288.pth... -[2023-10-14 15:16:43,218][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000043040_44072960.pth -[2023-10-14 15:16:43,218][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000042944_43974656.pth -[2023-10-14 15:16:44,233][75949] Updated weights for policy 0, policy_version 44611 (0.0008) -[2023-10-14 15:16:44,601][75949] Updated weights for policy 0, policy_version 44621 (0.0007) -[2023-10-14 15:16:44,962][75949] Updated weights for policy 0, policy_version 44631 (0.0008) -[2023-10-14 15:16:45,871][75950] Updated weights for policy 1, policy_version 44520 (0.0008) -[2023-10-14 15:16:46,245][75950] Updated weights for policy 1, policy_version 44530 (0.0010) -[2023-10-14 15:16:46,608][75950] Updated weights for policy 1, policy_version 44540 (0.0009) -[2023-10-14 15:16:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91324416. Throughput: 0: 1670.2, 1: 1683.6. Samples: 22835118. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) -[2023-10-14 15:16:48,164][74987] Avg episode reward: [(0, '24.160'), (1, '28.650')] -[2023-10-14 15:16:48,991][75949] Updated weights for policy 0, policy_version 44641 (0.0009) -[2023-10-14 15:16:49,359][75949] Updated weights for policy 0, policy_version 44651 (0.0009) -[2023-10-14 15:16:49,729][75949] Updated weights for policy 0, policy_version 44661 (0.0009) -[2023-10-14 15:16:50,102][75949] Updated weights for policy 0, policy_version 44671 (0.0010) -[2023-10-14 15:16:50,742][75950] Updated weights for policy 1, policy_version 44550 (0.0009) -[2023-10-14 15:16:51,110][75950] Updated weights for policy 1, policy_version 44560 (0.0009) -[2023-10-14 15:16:51,472][75950] Updated weights for policy 1, policy_version 44570 (0.0009) -[2023-10-14 15:16:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91389952. Throughput: 0: 1688.0, 1: 1659.2. Samples: 22854604. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) -[2023-10-14 15:16:53,165][74987] Avg episode reward: [(0, '24.740'), (1, '29.200')] -[2023-10-14 15:16:54,202][75949] Updated weights for policy 0, policy_version 44681 (0.0009) -[2023-10-14 15:16:54,566][75949] Updated weights for policy 0, policy_version 44691 (0.0009) -[2023-10-14 15:16:54,943][75949] Updated weights for policy 0, policy_version 44701 (0.0009) -[2023-10-14 15:16:55,466][75950] Updated weights for policy 1, policy_version 44580 (0.0010) -[2023-10-14 15:16:55,852][75950] Updated weights for policy 1, policy_version 44590 (0.0011) -[2023-10-14 15:16:56,223][75950] Updated weights for policy 1, policy_version 44600 (0.0009) -[2023-10-14 15:16:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 91455488. Throughput: 0: 1690.5, 1: 1677.5. Samples: 22875334. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) -[2023-10-14 15:16:58,165][74987] Avg episode reward: [(0, '24.130'), (1, '31.220')] -[2023-10-14 15:16:59,069][75949] Updated weights for policy 0, policy_version 44711 (0.0011) -[2023-10-14 15:16:59,449][75949] Updated weights for policy 0, policy_version 44721 (0.0009) -[2023-10-14 15:16:59,826][75949] Updated weights for policy 0, policy_version 44731 (0.0008) -[2023-10-14 15:17:00,270][75950] Updated weights for policy 1, policy_version 44610 (0.0011) -[2023-10-14 15:17:00,630][75950] Updated weights for policy 1, policy_version 44620 (0.0007) -[2023-10-14 15:17:01,008][75950] Updated weights for policy 1, policy_version 44630 (0.0008) -[2023-10-14 15:17:01,370][75950] Updated weights for policy 1, policy_version 44640 (0.0010) -[2023-10-14 15:17:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 91521024. Throughput: 0: 1678.0, 1: 1664.4. Samples: 22885000. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) -[2023-10-14 15:17:03,165][74987] Avg episode reward: [(0, '25.520'), (1, '28.750')] -[2023-10-14 15:17:04,011][75949] Updated weights for policy 0, policy_version 44741 (0.0010) -[2023-10-14 15:17:04,384][75949] Updated weights for policy 0, policy_version 44751 (0.0010) -[2023-10-14 15:17:04,756][75949] Updated weights for policy 0, policy_version 44761 (0.0011) -[2023-10-14 15:17:05,351][75950] Updated weights for policy 1, policy_version 44650 (0.0011) -[2023-10-14 15:17:05,719][75950] Updated weights for policy 1, policy_version 44660 (0.0010) -[2023-10-14 15:17:06,093][75950] Updated weights for policy 1, policy_version 44670 (0.0010) -[2023-10-14 15:17:08,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 91586560. Throughput: 0: 1691.0, 1: 1663.2. Samples: 22905074. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 15:17:08,164][74987] Avg episode reward: [(0, '25.040'), (1, '27.580')] -[2023-10-14 15:17:08,661][75949] Updated weights for policy 0, policy_version 44771 (0.0008) -[2023-10-14 15:17:09,023][75949] Updated weights for policy 0, policy_version 44781 (0.0011) -[2023-10-14 15:17:09,394][75949] Updated weights for policy 0, policy_version 44791 (0.0010) -[2023-10-14 15:17:10,182][75950] Updated weights for policy 1, policy_version 44680 (0.0008) -[2023-10-14 15:17:10,545][75950] Updated weights for policy 1, policy_version 44690 (0.0008) -[2023-10-14 15:17:10,914][75950] Updated weights for policy 1, policy_version 44700 (0.0009) -[2023-10-14 15:17:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 91652096. Throughput: 0: 1687.1, 1: 1680.6. Samples: 22925790. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 15:17:13,164][74987] Avg episode reward: [(0, '26.090'), (1, '30.440')] -[2023-10-14 15:17:13,482][75949] Updated weights for policy 0, policy_version 44801 (0.0010) -[2023-10-14 15:17:13,848][75949] Updated weights for policy 0, policy_version 44811 (0.0011) -[2023-10-14 15:17:14,225][75949] Updated weights for policy 0, policy_version 44821 (0.0010) -[2023-10-14 15:17:14,609][75949] Updated weights for policy 0, policy_version 44831 (0.0011) -[2023-10-14 15:17:15,015][75950] Updated weights for policy 1, policy_version 44710 (0.0008) -[2023-10-14 15:17:15,387][75950] Updated weights for policy 1, policy_version 44720 (0.0008) -[2023-10-14 15:17:15,753][75950] Updated weights for policy 1, policy_version 44730 (0.0008) -[2023-10-14 15:17:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 91717632. Throughput: 0: 1682.8, 1: 1662.4. Samples: 22935204. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 15:17:18,164][74987] Avg episode reward: [(0, '24.180'), (1, '30.940')] -[2023-10-14 15:17:18,675][75949] Updated weights for policy 0, policy_version 44841 (0.0009) -[2023-10-14 15:17:19,051][75949] Updated weights for policy 0, policy_version 44851 (0.0007) -[2023-10-14 15:17:19,419][75949] Updated weights for policy 0, policy_version 44861 (0.0010) -[2023-10-14 15:17:19,922][75950] Updated weights for policy 1, policy_version 44740 (0.0008) -[2023-10-14 15:17:20,287][75950] Updated weights for policy 1, policy_version 44750 (0.0007) -[2023-10-14 15:17:20,660][75950] Updated weights for policy 1, policy_version 44760 (0.0007) -[2023-10-14 15:17:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 91783168. Throughput: 0: 1678.7, 1: 1677.5. Samples: 22955626. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 15:17:23,164][74987] Avg episode reward: [(0, '24.450'), (1, '28.440')] -[2023-10-14 15:17:23,436][75949] Updated weights for policy 0, policy_version 44871 (0.0009) -[2023-10-14 15:17:23,810][75949] Updated weights for policy 0, policy_version 44881 (0.0008) -[2023-10-14 15:17:24,181][75949] Updated weights for policy 0, policy_version 44891 (0.0008) -[2023-10-14 15:17:24,740][75950] Updated weights for policy 1, policy_version 44770 (0.0008) -[2023-10-14 15:17:25,115][75950] Updated weights for policy 1, policy_version 44780 (0.0008) -[2023-10-14 15:17:25,480][75950] Updated weights for policy 1, policy_version 44790 (0.0007) -[2023-10-14 15:17:25,840][75950] Updated weights for policy 1, policy_version 44800 (0.0007) -[2023-10-14 15:17:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 91848704. Throughput: 0: 1678.4, 1: 1686.7. Samples: 22976218. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 15:17:28,164][74987] Avg episode reward: [(0, '22.720'), (1, '29.460')] -[2023-10-14 15:17:28,355][75949] Updated weights for policy 0, policy_version 44901 (0.0009) -[2023-10-14 15:17:28,729][75949] Updated weights for policy 0, policy_version 44911 (0.0009) -[2023-10-14 15:17:29,101][75949] Updated weights for policy 0, policy_version 44921 (0.0008) -[2023-10-14 15:17:29,782][75950] Updated weights for policy 1, policy_version 44810 (0.0007) -[2023-10-14 15:17:30,144][75950] Updated weights for policy 1, policy_version 44820 (0.0009) -[2023-10-14 15:17:30,528][75950] Updated weights for policy 1, policy_version 44830 (0.0011) -[2023-10-14 15:17:33,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 91914240. Throughput: 0: 1678.0, 1: 1662.1. Samples: 22985418. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) -[2023-10-14 15:17:33,164][74987] Avg episode reward: [(0, '22.770'), (1, '30.790')] -[2023-10-14 15:17:33,177][75949] Updated weights for policy 0, policy_version 44931 (0.0009) -[2023-10-14 15:17:33,549][75949] Updated weights for policy 0, policy_version 44941 (0.0008) -[2023-10-14 15:17:33,916][75949] Updated weights for policy 0, policy_version 44951 (0.0010) -[2023-10-14 15:17:34,518][75950] Updated weights for policy 1, policy_version 44840 (0.0010) -[2023-10-14 15:17:34,878][75950] Updated weights for policy 1, policy_version 44850 (0.0008) -[2023-10-14 15:17:35,251][75950] Updated weights for policy 1, policy_version 44860 (0.0008) -[2023-10-14 15:17:37,922][75949] Updated weights for policy 0, policy_version 44961 (0.0009) -[2023-10-14 15:17:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 91979776. Throughput: 0: 1679.1, 1: 1687.6. Samples: 23006108. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 15:17:38,164][74987] Avg episode reward: [(0, '25.270'), (1, '29.550')] -[2023-10-14 15:17:38,296][75949] Updated weights for policy 0, policy_version 44971 (0.0009) -[2023-10-14 15:17:38,658][75949] Updated weights for policy 0, policy_version 44981 (0.0009) -[2023-10-14 15:17:39,027][75949] Updated weights for policy 0, policy_version 44991 (0.0008) -[2023-10-14 15:17:39,374][75950] Updated weights for policy 1, policy_version 44870 (0.0008) -[2023-10-14 15:17:39,736][75950] Updated weights for policy 1, policy_version 44880 (0.0009) -[2023-10-14 15:17:40,110][75950] Updated weights for policy 1, policy_version 44890 (0.0009) -[2023-10-14 15:17:43,062][75949] Updated weights for policy 0, policy_version 45001 (0.0009) -[2023-10-14 15:17:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 92045312. Throughput: 0: 1677.5, 1: 1689.0. Samples: 23026828. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 15:17:43,164][74987] Avg episode reward: [(0, '25.180'), (1, '29.360')] -[2023-10-14 15:17:43,433][75949] Updated weights for policy 0, policy_version 45011 (0.0011) -[2023-10-14 15:17:43,811][75949] Updated weights for policy 0, policy_version 45021 (0.0010) -[2023-10-14 15:17:44,322][75950] Updated weights for policy 1, policy_version 44900 (0.0008) -[2023-10-14 15:17:44,712][75950] Updated weights for policy 1, policy_version 44910 (0.0009) -[2023-10-14 15:17:45,079][75950] Updated weights for policy 1, policy_version 44920 (0.0008) -[2023-10-14 15:17:47,945][75949] Updated weights for policy 0, policy_version 45031 (0.0010) -[2023-10-14 15:17:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 92110848. Throughput: 0: 1680.0, 1: 1668.7. Samples: 23035688. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 15:17:48,164][74987] Avg episode reward: [(0, '25.610'), (1, '31.100')] -[2023-10-14 15:17:48,332][75949] Updated weights for policy 0, policy_version 45041 (0.0008) -[2023-10-14 15:17:48,701][75949] Updated weights for policy 0, policy_version 45051 (0.0009) -[2023-10-14 15:17:49,249][75950] Updated weights for policy 1, policy_version 44930 (0.0009) -[2023-10-14 15:17:49,619][75950] Updated weights for policy 1, policy_version 44940 (0.0010) -[2023-10-14 15:17:49,976][75950] Updated weights for policy 1, policy_version 44950 (0.0009) -[2023-10-14 15:17:50,340][75950] Updated weights for policy 1, policy_version 44960 (0.0009) -[2023-10-14 15:17:52,610][75949] Updated weights for policy 0, policy_version 45061 (0.0010) -[2023-10-14 15:17:52,975][75949] Updated weights for policy 0, policy_version 45071 (0.0007) -[2023-10-14 15:17:53,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 92176384. Throughput: 0: 1675.6, 1: 1677.7. Samples: 23055972. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 15:17:53,164][74987] Avg episode reward: [(0, '24.630'), (1, '31.520')] -[2023-10-14 15:17:53,346][75949] Updated weights for policy 0, policy_version 45081 (0.0009) -[2023-10-14 15:17:54,466][75950] Updated weights for policy 1, policy_version 44970 (0.0010) -[2023-10-14 15:17:54,829][75950] Updated weights for policy 1, policy_version 44980 (0.0009) -[2023-10-14 15:17:55,199][75950] Updated weights for policy 1, policy_version 44990 (0.0009) -[2023-10-14 15:17:57,522][75949] Updated weights for policy 0, policy_version 45091 (0.0010) -[2023-10-14 15:17:57,891][75949] Updated weights for policy 0, policy_version 45101 (0.0007) -[2023-10-14 15:17:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 92241920. Throughput: 0: 1670.4, 1: 1669.9. Samples: 23076106. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 15:17:58,165][74987] Avg episode reward: [(0, '25.110'), (1, '27.550')] -[2023-10-14 15:17:58,265][75949] Updated weights for policy 0, policy_version 45111 (0.0007) -[2023-10-14 15:17:59,268][75950] Updated weights for policy 1, policy_version 45000 (0.0008) -[2023-10-14 15:17:59,635][75950] Updated weights for policy 1, policy_version 45010 (0.0009) -[2023-10-14 15:18:00,014][75950] Updated weights for policy 1, policy_version 45020 (0.0007) -[2023-10-14 15:18:02,378][75949] Updated weights for policy 0, policy_version 45121 (0.0008) -[2023-10-14 15:18:02,741][75949] Updated weights for policy 0, policy_version 45131 (0.0007) -[2023-10-14 15:18:03,121][75949] Updated weights for policy 0, policy_version 45141 (0.0008) -[2023-10-14 15:18:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 92307456. Throughput: 0: 1678.1, 1: 1662.9. Samples: 23085552. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 15:18:03,164][74987] Avg episode reward: [(0, '23.610'), (1, '29.360')] -[2023-10-14 15:18:03,484][75949] Updated weights for policy 0, policy_version 45151 (0.0008) -[2023-10-14 15:18:04,065][75950] Updated weights for policy 1, policy_version 45030 (0.0012) -[2023-10-14 15:18:04,434][75950] Updated weights for policy 1, policy_version 45040 (0.0009) -[2023-10-14 15:18:04,785][75950] Updated weights for policy 1, policy_version 45050 (0.0009) -[2023-10-14 15:18:07,489][75949] Updated weights for policy 0, policy_version 45161 (0.0009) -[2023-10-14 15:18:07,857][75949] Updated weights for policy 0, policy_version 45171 (0.0009) -[2023-10-14 15:18:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 92372992. Throughput: 0: 1684.4, 1: 1668.5. Samples: 23106510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:18:08,165][74987] Avg episode reward: [(0, '26.110'), (1, '31.260')] -[2023-10-14 15:18:08,230][75949] Updated weights for policy 0, policy_version 45181 (0.0009) -[2023-10-14 15:18:08,902][75950] Updated weights for policy 1, policy_version 45060 (0.0010) -[2023-10-14 15:18:09,268][75950] Updated weights for policy 1, policy_version 45070 (0.0007) -[2023-10-14 15:18:09,628][75950] Updated weights for policy 1, policy_version 45080 (0.0007) -[2023-10-14 15:18:12,139][75949] Updated weights for policy 0, policy_version 45191 (0.0010) -[2023-10-14 15:18:12,505][75949] Updated weights for policy 0, policy_version 45201 (0.0011) -[2023-10-14 15:18:12,884][75949] Updated weights for policy 0, policy_version 45211 (0.0009) -[2023-10-14 15:18:13,163][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92471296. Throughput: 0: 1670.4, 1: 1677.2. Samples: 23126860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:18:13,164][74987] Avg episode reward: [(0, '24.110'), (1, '28.300')] -[2023-10-14 15:18:13,685][75950] Updated weights for policy 1, policy_version 45090 (0.0007) -[2023-10-14 15:18:14,042][75950] Updated weights for policy 1, policy_version 45100 (0.0010) -[2023-10-14 15:18:14,406][75950] Updated weights for policy 1, policy_version 45110 (0.0007) -[2023-10-14 15:18:14,773][75950] Updated weights for policy 1, policy_version 45120 (0.0008) -[2023-10-14 15:18:17,012][75949] Updated weights for policy 0, policy_version 45221 (0.0008) -[2023-10-14 15:18:17,383][75949] Updated weights for policy 0, policy_version 45231 (0.0009) -[2023-10-14 15:18:17,751][75949] Updated weights for policy 0, policy_version 45241 (0.0009) -[2023-10-14 15:18:18,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92536832. Throughput: 0: 1691.2, 1: 1672.7. Samples: 23136794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:18:18,164][74987] Avg episode reward: [(0, '25.280'), (1, '29.080')] -[2023-10-14 15:18:18,968][75950] Updated weights for policy 1, policy_version 45130 (0.0009) -[2023-10-14 15:18:19,336][75950] Updated weights for policy 1, policy_version 45140 (0.0009) -[2023-10-14 15:18:19,707][75950] Updated weights for policy 1, policy_version 45150 (0.0009) -[2023-10-14 15:18:21,838][75949] Updated weights for policy 0, policy_version 45251 (0.0010) -[2023-10-14 15:18:22,205][75949] Updated weights for policy 0, policy_version 45261 (0.0010) -[2023-10-14 15:18:22,572][75949] Updated weights for policy 0, policy_version 45271 (0.0008) -[2023-10-14 15:18:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92602368. Throughput: 0: 1691.5, 1: 1673.7. Samples: 23157542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:18:23,165][74987] Avg episode reward: [(0, '24.400'), (1, '32.400')] -[2023-10-14 15:18:23,795][75950] Updated weights for policy 1, policy_version 45160 (0.0009) -[2023-10-14 15:18:24,169][75950] Updated weights for policy 1, policy_version 45170 (0.0007) -[2023-10-14 15:18:24,538][75950] Updated weights for policy 1, policy_version 45180 (0.0007) -[2023-10-14 15:18:26,642][75949] Updated weights for policy 0, policy_version 45281 (0.0010) -[2023-10-14 15:18:27,018][75949] Updated weights for policy 0, policy_version 45291 (0.0009) -[2023-10-14 15:18:27,384][75949] Updated weights for policy 0, policy_version 45301 (0.0009) -[2023-10-14 15:18:27,757][75949] Updated weights for policy 0, policy_version 45311 (0.0008) -[2023-10-14 15:18:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92667904. Throughput: 0: 1662.6, 1: 1680.4. Samples: 23177262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:18:28,164][74987] Avg episode reward: [(0, '25.910'), (1, '30.380')] -[2023-10-14 15:18:28,244][75950] Updated weights for policy 1, policy_version 45190 (0.0009) -[2023-10-14 15:18:28,620][75950] Updated weights for policy 1, policy_version 45200 (0.0009) -[2023-10-14 15:18:28,979][75950] Updated weights for policy 1, policy_version 45210 (0.0011) -[2023-10-14 15:18:31,819][75949] Updated weights for policy 0, policy_version 45321 (0.0008) -[2023-10-14 15:18:32,193][75949] Updated weights for policy 0, policy_version 45331 (0.0007) -[2023-10-14 15:18:32,564][75949] Updated weights for policy 0, policy_version 45341 (0.0009) -[2023-10-14 15:18:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92733440. Throughput: 0: 1691.5, 1: 1681.6. Samples: 23187478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:18:33,164][74987] Avg episode reward: [(0, '25.230'), (1, '29.740')] -[2023-10-14 15:18:33,200][75950] Updated weights for policy 1, policy_version 45220 (0.0011) -[2023-10-14 15:18:33,590][75950] Updated weights for policy 1, policy_version 45230 (0.0010) -[2023-10-14 15:18:33,959][75950] Updated weights for policy 1, policy_version 45240 (0.0010) -[2023-10-14 15:18:36,578][75949] Updated weights for policy 0, policy_version 45351 (0.0008) -[2023-10-14 15:18:36,960][75949] Updated weights for policy 0, policy_version 45361 (0.0007) -[2023-10-14 15:18:37,335][75949] Updated weights for policy 0, policy_version 45371 (0.0007) -[2023-10-14 15:18:38,079][75950] Updated weights for policy 1, policy_version 45250 (0.0009) -[2023-10-14 15:18:38,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92798976. Throughput: 0: 1687.2, 1: 1680.6. Samples: 23207524. Policy #0 lag: (min: 17.0, avg: 42.6, max: 48.0) -[2023-10-14 15:18:38,165][74987] Avg episode reward: [(0, '26.470'), (1, '31.370')] -[2023-10-14 15:18:38,454][75950] Updated weights for policy 1, policy_version 45260 (0.0008) -[2023-10-14 15:18:38,816][75950] Updated weights for policy 1, policy_version 45270 (0.0009) -[2023-10-14 15:18:39,185][75950] Updated weights for policy 1, policy_version 45280 (0.0009) -[2023-10-14 15:18:41,325][75949] Updated weights for policy 0, policy_version 45381 (0.0009) -[2023-10-14 15:18:41,695][75949] Updated weights for policy 0, policy_version 45391 (0.0010) -[2023-10-14 15:18:42,066][75949] Updated weights for policy 0, policy_version 45401 (0.0010) -[2023-10-14 15:18:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92864512. Throughput: 0: 1676.2, 1: 1683.2. Samples: 23227280. Policy #0 lag: (min: 17.0, avg: 42.6, max: 48.0) -[2023-10-14 15:18:43,164][74987] Avg episode reward: [(0, '23.940'), (1, '31.350')] -[2023-10-14 15:18:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000045408_46497792.pth... -[2023-10-14 15:18:43,211][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000043840_44892160.pth -[2023-10-14 15:18:43,356][75950] Updated weights for policy 1, policy_version 45290 (0.0007) -[2023-10-14 15:18:43,726][75950] Updated weights for policy 1, policy_version 45300 (0.0008) -[2023-10-14 15:18:44,096][75950] Updated weights for policy 1, policy_version 45310 (0.0008) -[2023-10-14 15:18:44,167][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000045312_46399488.pth... -[2023-10-14 15:18:44,207][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000043712_44761088.pth -[2023-10-14 15:18:46,040][75949] Updated weights for policy 0, policy_version 45411 (0.0009) -[2023-10-14 15:18:46,418][75949] Updated weights for policy 0, policy_version 45421 (0.0008) -[2023-10-14 15:18:46,778][75949] Updated weights for policy 0, policy_version 45431 (0.0008) -[2023-10-14 15:18:48,123][75950] Updated weights for policy 1, policy_version 45320 (0.0007) -[2023-10-14 15:18:48,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92930048. Throughput: 0: 1698.8, 1: 1683.9. Samples: 23237774. Policy #0 lag: (min: 17.0, avg: 42.6, max: 48.0) -[2023-10-14 15:18:48,164][74987] Avg episode reward: [(0, '24.690'), (1, '30.650')] -[2023-10-14 15:18:48,483][75950] Updated weights for policy 1, policy_version 45330 (0.0010) -[2023-10-14 15:18:48,847][75950] Updated weights for policy 1, policy_version 45340 (0.0007) -[2023-10-14 15:18:50,818][75949] Updated weights for policy 0, policy_version 45441 (0.0010) -[2023-10-14 15:18:51,187][75949] Updated weights for policy 0, policy_version 45451 (0.0007) -[2023-10-14 15:18:51,553][75949] Updated weights for policy 0, policy_version 45461 (0.0007) -[2023-10-14 15:18:51,920][75949] Updated weights for policy 0, policy_version 45471 (0.0010) -[2023-10-14 15:18:53,115][75950] Updated weights for policy 1, policy_version 45350 (0.0008) -[2023-10-14 15:18:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92995584. Throughput: 0: 1672.9, 1: 1680.2. Samples: 23257402. Policy #0 lag: (min: 17.0, avg: 42.6, max: 48.0) -[2023-10-14 15:18:53,164][74987] Avg episode reward: [(0, '22.270'), (1, '30.010')] -[2023-10-14 15:18:53,481][75950] Updated weights for policy 1, policy_version 45360 (0.0010) -[2023-10-14 15:18:53,850][75950] Updated weights for policy 1, policy_version 45370 (0.0010) -[2023-10-14 15:18:55,947][75949] Updated weights for policy 0, policy_version 45481 (0.0008) -[2023-10-14 15:18:56,313][75949] Updated weights for policy 0, policy_version 45491 (0.0008) -[2023-10-14 15:18:56,694][75949] Updated weights for policy 0, policy_version 45501 (0.0010) -[2023-10-14 15:18:58,010][75950] Updated weights for policy 1, policy_version 45380 (0.0010) -[2023-10-14 15:18:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 93061120. Throughput: 0: 1684.5, 1: 1668.4. Samples: 23277742. Policy #0 lag: (min: 17.0, avg: 42.6, max: 48.0) -[2023-10-14 15:18:58,164][74987] Avg episode reward: [(0, '24.270'), (1, '32.160')] -[2023-10-14 15:18:58,381][75950] Updated weights for policy 1, policy_version 45390 (0.0008) -[2023-10-14 15:18:58,738][75950] Updated weights for policy 1, policy_version 45400 (0.0009) -[2023-10-14 15:19:00,797][75949] Updated weights for policy 0, policy_version 45511 (0.0009) -[2023-10-14 15:19:01,161][75949] Updated weights for policy 0, policy_version 45521 (0.0008) -[2023-10-14 15:19:01,536][75949] Updated weights for policy 0, policy_version 45531 (0.0010) -[2023-10-14 15:19:02,855][75950] Updated weights for policy 1, policy_version 45410 (0.0007) -[2023-10-14 15:19:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 93126656. Throughput: 0: 1691.2, 1: 1667.9. Samples: 23287952. Policy #0 lag: (min: 17.0, avg: 42.6, max: 48.0) -[2023-10-14 15:19:03,164][74987] Avg episode reward: [(0, '25.660'), (1, '29.110')] -[2023-10-14 15:19:03,219][75950] Updated weights for policy 1, policy_version 45420 (0.0009) -[2023-10-14 15:19:03,592][75950] Updated weights for policy 1, policy_version 45430 (0.0008) -[2023-10-14 15:19:03,949][75950] Updated weights for policy 1, policy_version 45440 (0.0009) -[2023-10-14 15:19:05,498][75949] Updated weights for policy 0, policy_version 45541 (0.0009) -[2023-10-14 15:19:05,873][75949] Updated weights for policy 0, policy_version 45551 (0.0008) -[2023-10-14 15:19:06,237][75949] Updated weights for policy 0, policy_version 45561 (0.0008) -[2023-10-14 15:19:07,936][75950] Updated weights for policy 1, policy_version 45450 (0.0008) -[2023-10-14 15:19:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 93192192. Throughput: 0: 1665.0, 1: 1670.2. Samples: 23307624. Policy #0 lag: (min: 17.0, avg: 42.6, max: 48.0) -[2023-10-14 15:19:08,164][74987] Avg episode reward: [(0, '24.840'), (1, '29.510')] -[2023-10-14 15:19:08,300][75950] Updated weights for policy 1, policy_version 45460 (0.0009) -[2023-10-14 15:19:08,668][75950] Updated weights for policy 1, policy_version 45470 (0.0008) -[2023-10-14 15:19:10,220][75949] Updated weights for policy 0, policy_version 45571 (0.0008) -[2023-10-14 15:19:10,581][75949] Updated weights for policy 0, policy_version 45581 (0.0009) -[2023-10-14 15:19:10,950][75949] Updated weights for policy 0, policy_version 45591 (0.0009) -[2023-10-14 15:19:12,779][75950] Updated weights for policy 1, policy_version 45480 (0.0008) -[2023-10-14 15:19:13,150][75950] Updated weights for policy 1, policy_version 45490 (0.0009) -[2023-10-14 15:19:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93257728. Throughput: 0: 1699.4, 1: 1660.7. Samples: 23328466. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) -[2023-10-14 15:19:13,164][74987] Avg episode reward: [(0, '25.730'), (1, '31.120')] -[2023-10-14 15:19:13,512][75950] Updated weights for policy 1, policy_version 45500 (0.0009) -[2023-10-14 15:19:14,949][75949] Updated weights for policy 0, policy_version 45601 (0.0007) -[2023-10-14 15:19:15,324][75949] Updated weights for policy 0, policy_version 45611 (0.0009) -[2023-10-14 15:19:15,695][75949] Updated weights for policy 0, policy_version 45621 (0.0008) -[2023-10-14 15:19:16,057][75949] Updated weights for policy 0, policy_version 45631 (0.0009) -[2023-10-14 15:19:17,613][75950] Updated weights for policy 1, policy_version 45510 (0.0008) -[2023-10-14 15:19:17,988][75950] Updated weights for policy 1, policy_version 45520 (0.0009) -[2023-10-14 15:19:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93323264. Throughput: 0: 1686.3, 1: 1667.5. Samples: 23338400. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) -[2023-10-14 15:19:18,164][74987] Avg episode reward: [(0, '25.450'), (1, '29.060')] -[2023-10-14 15:19:18,355][75950] Updated weights for policy 1, policy_version 45530 (0.0008) -[2023-10-14 15:19:20,059][75949] Updated weights for policy 0, policy_version 45641 (0.0008) -[2023-10-14 15:19:20,425][75949] Updated weights for policy 0, policy_version 45651 (0.0007) -[2023-10-14 15:19:20,800][75949] Updated weights for policy 0, policy_version 45661 (0.0008) -[2023-10-14 15:19:22,617][75950] Updated weights for policy 1, policy_version 45540 (0.0009) -[2023-10-14 15:19:22,996][75950] Updated weights for policy 1, policy_version 45550 (0.0008) -[2023-10-14 15:19:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93388800. Throughput: 0: 1687.8, 1: 1668.0. Samples: 23358536. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) -[2023-10-14 15:19:23,165][74987] Avg episode reward: [(0, '23.870'), (1, '28.700')] -[2023-10-14 15:19:23,362][75950] Updated weights for policy 1, policy_version 45560 (0.0007) -[2023-10-14 15:19:24,975][75949] Updated weights for policy 0, policy_version 45671 (0.0008) -[2023-10-14 15:19:25,361][75949] Updated weights for policy 0, policy_version 45681 (0.0009) -[2023-10-14 15:19:25,730][75949] Updated weights for policy 0, policy_version 45691 (0.0009) -[2023-10-14 15:19:27,409][75950] Updated weights for policy 1, policy_version 45570 (0.0008) -[2023-10-14 15:19:27,778][75950] Updated weights for policy 1, policy_version 45580 (0.0009) -[2023-10-14 15:19:28,147][75950] Updated weights for policy 1, policy_version 45590 (0.0010) -[2023-10-14 15:19:28,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93454336. Throughput: 0: 1704.2, 1: 1663.3. Samples: 23378820. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) -[2023-10-14 15:19:28,164][74987] Avg episode reward: [(0, '24.330'), (1, '29.540')] -[2023-10-14 15:19:28,517][75950] Updated weights for policy 1, policy_version 45600 (0.0010) -[2023-10-14 15:19:29,760][75949] Updated weights for policy 0, policy_version 45701 (0.0011) -[2023-10-14 15:19:30,133][75949] Updated weights for policy 0, policy_version 45711 (0.0008) -[2023-10-14 15:19:30,499][75949] Updated weights for policy 0, policy_version 45721 (0.0009) -[2023-10-14 15:19:32,663][75950] Updated weights for policy 1, policy_version 45610 (0.0009) -[2023-10-14 15:19:33,037][75950] Updated weights for policy 1, policy_version 45620 (0.0007) -[2023-10-14 15:19:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93519872. Throughput: 0: 1677.8, 1: 1669.8. Samples: 23388414. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) -[2023-10-14 15:19:33,164][74987] Avg episode reward: [(0, '24.980'), (1, '27.380')] -[2023-10-14 15:19:33,398][75950] Updated weights for policy 1, policy_version 45630 (0.0007) -[2023-10-14 15:19:34,599][75949] Updated weights for policy 0, policy_version 45731 (0.0008) -[2023-10-14 15:19:34,966][75949] Updated weights for policy 0, policy_version 45741 (0.0007) -[2023-10-14 15:19:35,338][75949] Updated weights for policy 0, policy_version 45751 (0.0007) -[2023-10-14 15:19:37,421][75950] Updated weights for policy 1, policy_version 45640 (0.0007) -[2023-10-14 15:19:37,795][75950] Updated weights for policy 1, policy_version 45650 (0.0007) -[2023-10-14 15:19:38,161][75950] Updated weights for policy 1, policy_version 45660 (0.0007) -[2023-10-14 15:19:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 93585408. Throughput: 0: 1697.7, 1: 1673.9. Samples: 23409124. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) -[2023-10-14 15:19:38,165][74987] Avg episode reward: [(0, '25.250'), (1, '27.620')] -[2023-10-14 15:19:39,247][75949] Updated weights for policy 0, policy_version 45761 (0.0007) -[2023-10-14 15:19:39,624][75949] Updated weights for policy 0, policy_version 45771 (0.0010) -[2023-10-14 15:19:39,998][75949] Updated weights for policy 0, policy_version 45781 (0.0012) -[2023-10-14 15:19:40,377][75949] Updated weights for policy 0, policy_version 45791 (0.0012) -[2023-10-14 15:19:42,357][75950] Updated weights for policy 1, policy_version 45670 (0.0008) -[2023-10-14 15:19:42,723][75950] Updated weights for policy 1, policy_version 45680 (0.0008) -[2023-10-14 15:19:43,093][75950] Updated weights for policy 1, policy_version 45690 (0.0008) -[2023-10-14 15:19:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 93650944. Throughput: 0: 1705.0, 1: 1664.7. Samples: 23429376. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) -[2023-10-14 15:19:43,165][74987] Avg episode reward: [(0, '26.820'), (1, '28.950')] -[2023-10-14 15:19:44,383][75949] Updated weights for policy 0, policy_version 45801 (0.0008) -[2023-10-14 15:19:44,754][75949] Updated weights for policy 0, policy_version 45811 (0.0008) -[2023-10-14 15:19:45,127][75949] Updated weights for policy 0, policy_version 45821 (0.0009) -[2023-10-14 15:19:46,966][75950] Updated weights for policy 1, policy_version 45700 (0.0007) -[2023-10-14 15:19:47,332][75950] Updated weights for policy 1, policy_version 45710 (0.0007) -[2023-10-14 15:19:47,690][75950] Updated weights for policy 1, policy_version 45720 (0.0007) -[2023-10-14 15:19:48,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 93749248. Throughput: 0: 1679.6, 1: 1681.9. Samples: 23439218. Policy #0 lag: (min: 23.0, avg: 23.6, max: 38.0) -[2023-10-14 15:19:48,165][74987] Avg episode reward: [(0, '25.300'), (1, '28.680')] -[2023-10-14 15:19:49,250][75949] Updated weights for policy 0, policy_version 45831 (0.0009) -[2023-10-14 15:19:49,620][75949] Updated weights for policy 0, policy_version 45841 (0.0008) -[2023-10-14 15:19:49,994][75949] Updated weights for policy 0, policy_version 45851 (0.0007) -[2023-10-14 15:19:51,690][75950] Updated weights for policy 1, policy_version 45730 (0.0011) -[2023-10-14 15:19:52,052][75950] Updated weights for policy 1, policy_version 45740 (0.0009) -[2023-10-14 15:19:52,421][75950] Updated weights for policy 1, policy_version 45750 (0.0008) -[2023-10-14 15:19:52,788][75950] Updated weights for policy 1, policy_version 45760 (0.0007) -[2023-10-14 15:19:53,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 93814784. Throughput: 0: 1700.5, 1: 1681.6. Samples: 23459818. Policy #0 lag: (min: 23.0, avg: 23.6, max: 38.0) -[2023-10-14 15:19:53,164][74987] Avg episode reward: [(0, '27.040'), (1, '28.360')] -[2023-10-14 15:19:53,884][75949] Updated weights for policy 0, policy_version 45861 (0.0009) -[2023-10-14 15:19:54,262][75949] Updated weights for policy 0, policy_version 45871 (0.0009) -[2023-10-14 15:19:54,633][75949] Updated weights for policy 0, policy_version 45881 (0.0011) -[2023-10-14 15:19:56,895][75950] Updated weights for policy 1, policy_version 45770 (0.0008) -[2023-10-14 15:19:57,254][75950] Updated weights for policy 1, policy_version 45780 (0.0011) -[2023-10-14 15:19:57,615][75950] Updated weights for policy 1, policy_version 45790 (0.0009) -[2023-10-14 15:19:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 93880320. Throughput: 0: 1696.3, 1: 1660.0. Samples: 23479500. Policy #0 lag: (min: 23.0, avg: 23.6, max: 38.0) -[2023-10-14 15:19:58,165][74987] Avg episode reward: [(0, '25.530'), (1, '27.660')] -[2023-10-14 15:19:58,767][75949] Updated weights for policy 0, policy_version 45891 (0.0009) -[2023-10-14 15:19:59,137][75949] Updated weights for policy 0, policy_version 45901 (0.0010) -[2023-10-14 15:19:59,521][75949] Updated weights for policy 0, policy_version 45911 (0.0008) -[2023-10-14 15:20:01,713][75950] Updated weights for policy 1, policy_version 45800 (0.0011) -[2023-10-14 15:20:02,093][75950] Updated weights for policy 1, policy_version 45810 (0.0009) -[2023-10-14 15:20:02,454][75950] Updated weights for policy 1, policy_version 45820 (0.0008) -[2023-10-14 15:20:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 93945856. Throughput: 0: 1678.8, 1: 1681.3. Samples: 23489604. Policy #0 lag: (min: 23.0, avg: 23.6, max: 38.0) -[2023-10-14 15:20:03,164][74987] Avg episode reward: [(0, '25.870'), (1, '29.360')] -[2023-10-14 15:20:03,520][75949] Updated weights for policy 0, policy_version 45921 (0.0009) -[2023-10-14 15:20:03,898][75949] Updated weights for policy 0, policy_version 45931 (0.0007) -[2023-10-14 15:20:04,272][75949] Updated weights for policy 0, policy_version 45941 (0.0009) -[2023-10-14 15:20:04,639][75949] Updated weights for policy 0, policy_version 45951 (0.0007) -[2023-10-14 15:20:06,491][75950] Updated weights for policy 1, policy_version 45830 (0.0009) -[2023-10-14 15:20:06,855][75950] Updated weights for policy 1, policy_version 45840 (0.0009) -[2023-10-14 15:20:07,225][75950] Updated weights for policy 1, policy_version 45850 (0.0009) -[2023-10-14 15:20:08,164][74987] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94011392. Throughput: 0: 1690.9, 1: 1678.7. Samples: 23510168. Policy #0 lag: (min: 23.0, avg: 23.6, max: 38.0) -[2023-10-14 15:20:08,164][74987] Avg episode reward: [(0, '24.840'), (1, '29.770')] -[2023-10-14 15:20:08,565][75949] Updated weights for policy 0, policy_version 45961 (0.0009) -[2023-10-14 15:20:08,946][75949] Updated weights for policy 0, policy_version 45971 (0.0010) -[2023-10-14 15:20:09,320][75949] Updated weights for policy 0, policy_version 45981 (0.0010) -[2023-10-14 15:20:11,602][75950] Updated weights for policy 1, policy_version 45860 (0.0008) -[2023-10-14 15:20:11,969][75950] Updated weights for policy 1, policy_version 45870 (0.0010) -[2023-10-14 15:20:12,337][75950] Updated weights for policy 1, policy_version 45880 (0.0010) -[2023-10-14 15:20:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94076928. Throughput: 0: 1692.7, 1: 1661.7. Samples: 23529768. Policy #0 lag: (min: 23.0, avg: 23.6, max: 38.0) -[2023-10-14 15:20:13,164][74987] Avg episode reward: [(0, '27.560'), (1, '30.090')] -[2023-10-14 15:20:13,515][75949] Updated weights for policy 0, policy_version 45991 (0.0008) -[2023-10-14 15:20:13,891][75949] Updated weights for policy 0, policy_version 46001 (0.0010) -[2023-10-14 15:20:14,263][75949] Updated weights for policy 0, policy_version 46011 (0.0010) -[2023-10-14 15:20:16,394][75950] Updated weights for policy 1, policy_version 45890 (0.0010) -[2023-10-14 15:20:16,760][75950] Updated weights for policy 1, policy_version 45900 (0.0008) -[2023-10-14 15:20:17,130][75950] Updated weights for policy 1, policy_version 45910 (0.0008) -[2023-10-14 15:20:17,491][75950] Updated weights for policy 1, policy_version 45920 (0.0009) -[2023-10-14 15:20:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94142464. Throughput: 0: 1685.6, 1: 1681.4. Samples: 23539928. Policy #0 lag: (min: 23.0, avg: 23.6, max: 38.0) -[2023-10-14 15:20:18,164][74987] Avg episode reward: [(0, '25.490'), (1, '30.240')] -[2023-10-14 15:20:18,267][75949] Updated weights for policy 0, policy_version 46021 (0.0009) -[2023-10-14 15:20:18,643][75949] Updated weights for policy 0, policy_version 46031 (0.0009) -[2023-10-14 15:20:19,010][75949] Updated weights for policy 0, policy_version 46041 (0.0009) -[2023-10-14 15:20:21,383][75950] Updated weights for policy 1, policy_version 45930 (0.0009) -[2023-10-14 15:20:21,741][75950] Updated weights for policy 1, policy_version 45940 (0.0007) -[2023-10-14 15:20:22,113][75950] Updated weights for policy 1, policy_version 45950 (0.0011) -[2023-10-14 15:20:23,120][75949] Updated weights for policy 0, policy_version 46051 (0.0010) -[2023-10-14 15:20:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 94208000. Throughput: 0: 1685.8, 1: 1666.6. Samples: 23559982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:20:23,164][74987] Avg episode reward: [(0, '26.100'), (1, '32.000')] -[2023-10-14 15:20:23,495][75949] Updated weights for policy 0, policy_version 46061 (0.0009) -[2023-10-14 15:20:23,865][75949] Updated weights for policy 0, policy_version 46071 (0.0008) -[2023-10-14 15:20:26,159][75950] Updated weights for policy 1, policy_version 45960 (0.0009) -[2023-10-14 15:20:26,518][75950] Updated weights for policy 1, policy_version 45970 (0.0007) -[2023-10-14 15:20:26,885][75950] Updated weights for policy 1, policy_version 45980 (0.0007) -[2023-10-14 15:20:27,923][75949] Updated weights for policy 0, policy_version 46081 (0.0009) -[2023-10-14 15:20:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94273536. Throughput: 0: 1682.1, 1: 1666.5. Samples: 23580058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:20:28,164][74987] Avg episode reward: [(0, '25.190'), (1, '29.500')] -[2023-10-14 15:20:28,287][75949] Updated weights for policy 0, policy_version 46091 (0.0010) -[2023-10-14 15:20:28,652][75949] Updated weights for policy 0, policy_version 46101 (0.0007) -[2023-10-14 15:20:29,028][75949] Updated weights for policy 0, policy_version 46111 (0.0008) -[2023-10-14 15:20:31,005][75950] Updated weights for policy 1, policy_version 45990 (0.0007) -[2023-10-14 15:20:31,372][75950] Updated weights for policy 1, policy_version 46000 (0.0008) -[2023-10-14 15:20:31,741][75950] Updated weights for policy 1, policy_version 46010 (0.0007) -[2023-10-14 15:20:32,998][75949] Updated weights for policy 0, policy_version 46121 (0.0008) -[2023-10-14 15:20:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94339072. Throughput: 0: 1683.0, 1: 1678.3. Samples: 23590476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:20:33,164][74987] Avg episode reward: [(0, '24.470'), (1, '28.070')] -[2023-10-14 15:20:33,355][75949] Updated weights for policy 0, policy_version 46131 (0.0007) -[2023-10-14 15:20:33,732][75949] Updated weights for policy 0, policy_version 46141 (0.0009) -[2023-10-14 15:20:35,886][75950] Updated weights for policy 1, policy_version 46020 (0.0010) -[2023-10-14 15:20:36,255][75950] Updated weights for policy 1, policy_version 46030 (0.0009) -[2023-10-14 15:20:36,615][75950] Updated weights for policy 1, policy_version 46040 (0.0008) -[2023-10-14 15:20:37,938][75949] Updated weights for policy 0, policy_version 46151 (0.0008) -[2023-10-14 15:20:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 94404608. Throughput: 0: 1683.3, 1: 1656.0. Samples: 23610086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:20:38,164][74987] Avg episode reward: [(0, '23.970'), (1, '29.400')] -[2023-10-14 15:20:38,308][75949] Updated weights for policy 0, policy_version 46161 (0.0007) -[2023-10-14 15:20:38,677][75949] Updated weights for policy 0, policy_version 46171 (0.0008) -[2023-10-14 15:20:40,628][75950] Updated weights for policy 1, policy_version 46050 (0.0009) -[2023-10-14 15:20:41,005][75950] Updated weights for policy 1, policy_version 46060 (0.0008) -[2023-10-14 15:20:41,361][75950] Updated weights for policy 1, policy_version 46070 (0.0010) -[2023-10-14 15:20:41,736][75950] Updated weights for policy 1, policy_version 46080 (0.0010) -[2023-10-14 15:20:42,622][75949] Updated weights for policy 0, policy_version 46181 (0.0009) -[2023-10-14 15:20:42,978][75949] Updated weights for policy 0, policy_version 46191 (0.0009) -[2023-10-14 15:20:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94470144. Throughput: 0: 1675.7, 1: 1675.0. Samples: 23630282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:20:43,164][74987] Avg episode reward: [(0, '24.070'), (1, '26.840')] -[2023-10-14 15:20:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000046080_47185920.pth... -[2023-10-14 15:20:43,211][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000044512_45580288.pth -[2023-10-14 15:20:43,352][75949] Updated weights for policy 0, policy_version 46201 (0.0008) -[2023-10-14 15:20:43,605][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000046208_47316992.pth... -[2023-10-14 15:20:43,633][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000044608_45678592.pth -[2023-10-14 15:20:45,926][75950] Updated weights for policy 1, policy_version 46090 (0.0007) -[2023-10-14 15:20:46,297][75950] Updated weights for policy 1, policy_version 46100 (0.0009) -[2023-10-14 15:20:46,662][75950] Updated weights for policy 1, policy_version 46110 (0.0009) -[2023-10-14 15:20:47,370][75949] Updated weights for policy 0, policy_version 46211 (0.0008) -[2023-10-14 15:20:47,742][75949] Updated weights for policy 0, policy_version 46221 (0.0008) -[2023-10-14 15:20:48,111][75949] Updated weights for policy 0, policy_version 46231 (0.0008) -[2023-10-14 15:20:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 94535680. Throughput: 0: 1684.7, 1: 1677.9. Samples: 23640920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:20:48,164][74987] Avg episode reward: [(0, '26.640'), (1, '27.340')] -[2023-10-14 15:20:50,710][75950] Updated weights for policy 1, policy_version 46120 (0.0010) -[2023-10-14 15:20:51,075][75950] Updated weights for policy 1, policy_version 46130 (0.0008) -[2023-10-14 15:20:51,446][75950] Updated weights for policy 1, policy_version 46140 (0.0010) -[2023-10-14 15:20:52,341][75949] Updated weights for policy 0, policy_version 46241 (0.0008) -[2023-10-14 15:20:52,718][75949] Updated weights for policy 0, policy_version 46251 (0.0009) -[2023-10-14 15:20:53,097][75949] Updated weights for policy 0, policy_version 46261 (0.0007) -[2023-10-14 15:20:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94601216. Throughput: 0: 1682.4, 1: 1658.3. Samples: 23660502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:20:53,164][74987] Avg episode reward: [(0, '25.320'), (1, '29.500')] -[2023-10-14 15:20:53,471][75949] Updated weights for policy 0, policy_version 46271 (0.0011) -[2023-10-14 15:20:55,355][75950] Updated weights for policy 1, policy_version 46150 (0.0007) -[2023-10-14 15:20:55,727][75950] Updated weights for policy 1, policy_version 46160 (0.0009) -[2023-10-14 15:20:56,082][75950] Updated weights for policy 1, policy_version 46170 (0.0010) -[2023-10-14 15:20:57,457][75949] Updated weights for policy 0, policy_version 46281 (0.0009) -[2023-10-14 15:20:57,821][75949] Updated weights for policy 0, policy_version 46291 (0.0009) -[2023-10-14 15:20:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 94666752. Throughput: 0: 1668.4, 1: 1688.1. Samples: 23680810. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 15:20:58,164][74987] Avg episode reward: [(0, '25.420'), (1, '27.790')] -[2023-10-14 15:20:58,203][75949] Updated weights for policy 0, policy_version 46301 (0.0007) -[2023-10-14 15:21:00,173][75950] Updated weights for policy 1, policy_version 46180 (0.0008) -[2023-10-14 15:21:00,573][75950] Updated weights for policy 1, policy_version 46190 (0.0007) -[2023-10-14 15:21:00,944][75950] Updated weights for policy 1, policy_version 46200 (0.0007) -[2023-10-14 15:21:02,491][75949] Updated weights for policy 0, policy_version 46311 (0.0007) -[2023-10-14 15:21:02,889][75949] Updated weights for policy 0, policy_version 46321 (0.0007) -[2023-10-14 15:21:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94732288. Throughput: 0: 1685.5, 1: 1673.1. Samples: 23691064. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 15:21:03,164][74987] Avg episode reward: [(0, '24.270'), (1, '29.330')] -[2023-10-14 15:21:03,254][75949] Updated weights for policy 0, policy_version 46331 (0.0007) -[2023-10-14 15:21:05,075][75950] Updated weights for policy 1, policy_version 46210 (0.0007) -[2023-10-14 15:21:05,444][75950] Updated weights for policy 1, policy_version 46220 (0.0008) -[2023-10-14 15:21:05,812][75950] Updated weights for policy 1, policy_version 46230 (0.0010) -[2023-10-14 15:21:06,171][75950] Updated weights for policy 1, policy_version 46240 (0.0009) -[2023-10-14 15:21:07,211][75949] Updated weights for policy 0, policy_version 46341 (0.0008) -[2023-10-14 15:21:07,579][75949] Updated weights for policy 0, policy_version 46351 (0.0009) -[2023-10-14 15:21:07,943][75949] Updated weights for policy 0, policy_version 46361 (0.0007) -[2023-10-14 15:21:08,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94797824. Throughput: 0: 1682.3, 1: 1677.9. Samples: 23711190. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 15:21:08,164][74987] Avg episode reward: [(0, '25.150'), (1, '29.610')] -[2023-10-14 15:21:10,246][75950] Updated weights for policy 1, policy_version 46250 (0.0009) -[2023-10-14 15:21:10,623][75950] Updated weights for policy 1, policy_version 46260 (0.0008) -[2023-10-14 15:21:10,994][75950] Updated weights for policy 1, policy_version 46270 (0.0010) -[2023-10-14 15:21:11,947][75949] Updated weights for policy 0, policy_version 46371 (0.0007) -[2023-10-14 15:21:12,312][75949] Updated weights for policy 0, policy_version 46381 (0.0009) -[2023-10-14 15:21:12,691][75949] Updated weights for policy 0, policy_version 46391 (0.0009) -[2023-10-14 15:21:13,163][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 94896128. Throughput: 0: 1667.1, 1: 1691.2. Samples: 23731180. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 15:21:13,164][74987] Avg episode reward: [(0, '23.810'), (1, '30.350')] -[2023-10-14 15:21:15,109][75950] Updated weights for policy 1, policy_version 46280 (0.0009) -[2023-10-14 15:21:15,465][75950] Updated weights for policy 1, policy_version 46290 (0.0010) -[2023-10-14 15:21:15,834][75950] Updated weights for policy 1, policy_version 46300 (0.0007) -[2023-10-14 15:21:16,792][75949] Updated weights for policy 0, policy_version 46401 (0.0009) -[2023-10-14 15:21:17,160][75949] Updated weights for policy 0, policy_version 46411 (0.0007) -[2023-10-14 15:21:17,523][75949] Updated weights for policy 0, policy_version 46421 (0.0007) -[2023-10-14 15:21:17,900][75949] Updated weights for policy 0, policy_version 46431 (0.0008) -[2023-10-14 15:21:18,163][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94961664. Throughput: 0: 1684.9, 1: 1670.6. Samples: 23741472. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 15:21:18,164][74987] Avg episode reward: [(0, '25.600'), (1, '30.730')] -[2023-10-14 15:21:19,951][75950] Updated weights for policy 1, policy_version 46310 (0.0010) -[2023-10-14 15:21:20,317][75950] Updated weights for policy 1, policy_version 46320 (0.0009) -[2023-10-14 15:21:20,682][75950] Updated weights for policy 1, policy_version 46330 (0.0008) -[2023-10-14 15:21:21,963][75949] Updated weights for policy 0, policy_version 46441 (0.0008) -[2023-10-14 15:21:22,326][75949] Updated weights for policy 0, policy_version 46451 (0.0007) -[2023-10-14 15:21:22,706][75949] Updated weights for policy 0, policy_version 46461 (0.0009) -[2023-10-14 15:21:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95027200. Throughput: 0: 1685.9, 1: 1679.8. Samples: 23761542. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-10-14 15:21:23,165][74987] Avg episode reward: [(0, '23.630'), (1, '31.210')] -[2023-10-14 15:21:24,709][75950] Updated weights for policy 1, policy_version 46340 (0.0008) -[2023-10-14 15:21:25,079][75950] Updated weights for policy 1, policy_version 46350 (0.0007) -[2023-10-14 15:21:25,450][75950] Updated weights for policy 1, policy_version 46360 (0.0007) -[2023-10-14 15:21:26,770][75949] Updated weights for policy 0, policy_version 46471 (0.0008) -[2023-10-14 15:21:27,137][75949] Updated weights for policy 0, policy_version 46481 (0.0007) -[2023-10-14 15:21:27,499][75949] Updated weights for policy 0, policy_version 46491 (0.0010) -[2023-10-14 15:21:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95092736. Throughput: 0: 1665.6, 1: 1690.1. Samples: 23781290. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:21:28,165][74987] Avg episode reward: [(0, '26.630'), (1, '32.160')] -[2023-10-14 15:21:29,326][75950] Updated weights for policy 1, policy_version 46370 (0.0007) -[2023-10-14 15:21:29,695][75950] Updated weights for policy 1, policy_version 46380 (0.0009) -[2023-10-14 15:21:30,062][75950] Updated weights for policy 1, policy_version 46390 (0.0009) -[2023-10-14 15:21:30,437][75950] Updated weights for policy 1, policy_version 46400 (0.0009) -[2023-10-14 15:21:31,539][75949] Updated weights for policy 0, policy_version 46501 (0.0009) -[2023-10-14 15:21:31,913][75949] Updated weights for policy 0, policy_version 46511 (0.0008) -[2023-10-14 15:21:32,275][75949] Updated weights for policy 0, policy_version 46521 (0.0009) -[2023-10-14 15:21:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95158272. Throughput: 0: 1688.1, 1: 1661.7. Samples: 23791662. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:21:33,165][74987] Avg episode reward: [(0, '25.350'), (1, '30.840')] -[2023-10-14 15:21:34,355][75950] Updated weights for policy 1, policy_version 46410 (0.0008) -[2023-10-14 15:21:34,719][75950] Updated weights for policy 1, policy_version 46420 (0.0008) -[2023-10-14 15:21:35,084][75950] Updated weights for policy 1, policy_version 46430 (0.0009) -[2023-10-14 15:21:36,437][75949] Updated weights for policy 0, policy_version 46531 (0.0009) -[2023-10-14 15:21:36,806][75949] Updated weights for policy 0, policy_version 46541 (0.0007) -[2023-10-14 15:21:37,182][75949] Updated weights for policy 0, policy_version 46551 (0.0010) -[2023-10-14 15:21:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95223808. Throughput: 0: 1675.9, 1: 1690.5. Samples: 23811990. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:21:38,165][74987] Avg episode reward: [(0, '27.080'), (1, '31.600')] -[2023-10-14 15:21:39,006][75950] Updated weights for policy 1, policy_version 46440 (0.0008) -[2023-10-14 15:21:39,376][75950] Updated weights for policy 1, policy_version 46450 (0.0007) -[2023-10-14 15:21:39,732][75950] Updated weights for policy 1, policy_version 46460 (0.0007) -[2023-10-14 15:21:41,271][75949] Updated weights for policy 0, policy_version 46561 (0.0007) -[2023-10-14 15:21:41,633][75949] Updated weights for policy 0, policy_version 46571 (0.0008) -[2023-10-14 15:21:42,011][75949] Updated weights for policy 0, policy_version 46581 (0.0009) -[2023-10-14 15:21:42,385][75949] Updated weights for policy 0, policy_version 46591 (0.0010) -[2023-10-14 15:21:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95289344. Throughput: 0: 1667.1, 1: 1693.1. Samples: 23832020. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:21:43,165][74987] Avg episode reward: [(0, '24.020'), (1, '30.610')] -[2023-10-14 15:21:43,842][75950] Updated weights for policy 1, policy_version 46470 (0.0007) -[2023-10-14 15:21:44,203][75950] Updated weights for policy 1, policy_version 46480 (0.0011) -[2023-10-14 15:21:44,565][75950] Updated weights for policy 1, policy_version 46490 (0.0010) -[2023-10-14 15:21:46,413][75949] Updated weights for policy 0, policy_version 46601 (0.0009) -[2023-10-14 15:21:46,788][75949] Updated weights for policy 0, policy_version 46611 (0.0010) -[2023-10-14 15:21:47,163][75949] Updated weights for policy 0, policy_version 46621 (0.0007) -[2023-10-14 15:21:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95354880. Throughput: 0: 1680.9, 1: 1681.7. Samples: 23842380. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:21:48,165][74987] Avg episode reward: [(0, '26.350'), (1, '31.890')] -[2023-10-14 15:21:48,682][75950] Updated weights for policy 1, policy_version 46500 (0.0008) -[2023-10-14 15:21:49,051][75950] Updated weights for policy 1, policy_version 46510 (0.0007) -[2023-10-14 15:21:49,424][75950] Updated weights for policy 1, policy_version 46520 (0.0007) -[2023-10-14 15:21:51,133][75949] Updated weights for policy 0, policy_version 46631 (0.0008) -[2023-10-14 15:21:51,503][75949] Updated weights for policy 0, policy_version 46641 (0.0008) -[2023-10-14 15:21:51,870][75949] Updated weights for policy 0, policy_version 46651 (0.0009) -[2023-10-14 15:21:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95420416. Throughput: 0: 1668.2, 1: 1690.7. Samples: 23862342. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:21:53,164][74987] Avg episode reward: [(0, '25.440'), (1, '31.630')] -[2023-10-14 15:21:53,409][75950] Updated weights for policy 1, policy_version 46530 (0.0009) -[2023-10-14 15:21:53,780][75950] Updated weights for policy 1, policy_version 46540 (0.0008) -[2023-10-14 15:21:54,155][75950] Updated weights for policy 1, policy_version 46550 (0.0008) -[2023-10-14 15:21:54,520][75950] Updated weights for policy 1, policy_version 46560 (0.0011) -[2023-10-14 15:21:55,950][75949] Updated weights for policy 0, policy_version 46661 (0.0007) -[2023-10-14 15:21:56,327][75949] Updated weights for policy 0, policy_version 46671 (0.0011) -[2023-10-14 15:21:56,698][75949] Updated weights for policy 0, policy_version 46681 (0.0009) -[2023-10-14 15:21:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 95485952. Throughput: 0: 1675.3, 1: 1687.3. Samples: 23882500. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:21:58,164][74987] Avg episode reward: [(0, '25.560'), (1, '30.440')] -[2023-10-14 15:21:58,868][75950] Updated weights for policy 1, policy_version 46570 (0.0008) -[2023-10-14 15:21:59,235][75950] Updated weights for policy 1, policy_version 46580 (0.0008) -[2023-10-14 15:21:59,609][75950] Updated weights for policy 1, policy_version 46590 (0.0009) -[2023-10-14 15:22:00,640][75949] Updated weights for policy 0, policy_version 46691 (0.0007) -[2023-10-14 15:22:01,009][75949] Updated weights for policy 0, policy_version 46701 (0.0009) -[2023-10-14 15:22:01,382][75949] Updated weights for policy 0, policy_version 46711 (0.0008) -[2023-10-14 15:22:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95551488. Throughput: 0: 1683.7, 1: 1677.6. Samples: 23892734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 15:22:03,164][74987] Avg episode reward: [(0, '27.080'), (1, '29.800')] -[2023-10-14 15:22:03,635][75950] Updated weights for policy 1, policy_version 46600 (0.0008) -[2023-10-14 15:22:04,005][75950] Updated weights for policy 1, policy_version 46610 (0.0010) -[2023-10-14 15:22:04,369][75950] Updated weights for policy 1, policy_version 46620 (0.0008) -[2023-10-14 15:22:05,530][75949] Updated weights for policy 0, policy_version 46721 (0.0008) -[2023-10-14 15:22:05,902][75949] Updated weights for policy 0, policy_version 46731 (0.0009) -[2023-10-14 15:22:06,268][75949] Updated weights for policy 0, policy_version 46741 (0.0007) -[2023-10-14 15:22:06,648][75949] Updated weights for policy 0, policy_version 46751 (0.0010) -[2023-10-14 15:22:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95617024. Throughput: 0: 1660.7, 1: 1695.1. Samples: 23912554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 15:22:08,165][74987] Avg episode reward: [(0, '24.610'), (1, '26.850')] -[2023-10-14 15:22:08,604][75950] Updated weights for policy 1, policy_version 46630 (0.0007) -[2023-10-14 15:22:08,968][75950] Updated weights for policy 1, policy_version 46640 (0.0009) -[2023-10-14 15:22:09,339][75950] Updated weights for policy 1, policy_version 46650 (0.0008) -[2023-10-14 15:22:10,581][75949] Updated weights for policy 0, policy_version 46761 (0.0008) -[2023-10-14 15:22:10,951][75949] Updated weights for policy 0, policy_version 46771 (0.0009) -[2023-10-14 15:22:11,325][75949] Updated weights for policy 0, policy_version 46781 (0.0010) -[2023-10-14 15:22:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 95682560. Throughput: 0: 1689.9, 1: 1689.7. Samples: 23933370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 15:22:13,165][74987] Avg episode reward: [(0, '26.510'), (1, '28.130')] -[2023-10-14 15:22:13,388][75950] Updated weights for policy 1, policy_version 46660 (0.0008) -[2023-10-14 15:22:13,761][75950] Updated weights for policy 1, policy_version 46670 (0.0008) -[2023-10-14 15:22:14,123][75950] Updated weights for policy 1, policy_version 46680 (0.0009) -[2023-10-14 15:22:15,260][75949] Updated weights for policy 0, policy_version 46791 (0.0009) -[2023-10-14 15:22:15,626][75949] Updated weights for policy 0, policy_version 46801 (0.0011) -[2023-10-14 15:22:15,992][75949] Updated weights for policy 0, policy_version 46811 (0.0010) -[2023-10-14 15:22:18,122][75950] Updated weights for policy 1, policy_version 46690 (0.0009) -[2023-10-14 15:22:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 95748096. Throughput: 0: 1674.9, 1: 1689.0. Samples: 23943038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 15:22:18,165][74987] Avg episode reward: [(0, '24.490'), (1, '29.480')] -[2023-10-14 15:22:18,492][75950] Updated weights for policy 1, policy_version 46700 (0.0008) -[2023-10-14 15:22:18,853][75950] Updated weights for policy 1, policy_version 46710 (0.0008) -[2023-10-14 15:22:19,227][75950] Updated weights for policy 1, policy_version 46720 (0.0008) -[2023-10-14 15:22:20,144][75949] Updated weights for policy 0, policy_version 46821 (0.0009) -[2023-10-14 15:22:20,513][75949] Updated weights for policy 0, policy_version 46831 (0.0008) -[2023-10-14 15:22:20,891][75949] Updated weights for policy 0, policy_version 46841 (0.0007) -[2023-10-14 15:22:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 95813632. Throughput: 0: 1672.7, 1: 1685.5. Samples: 23963110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 15:22:23,164][74987] Avg episode reward: [(0, '25.130'), (1, '30.370')] -[2023-10-14 15:22:23,240][75950] Updated weights for policy 1, policy_version 46730 (0.0007) -[2023-10-14 15:22:23,613][75950] Updated weights for policy 1, policy_version 46740 (0.0007) -[2023-10-14 15:22:23,977][75950] Updated weights for policy 1, policy_version 46750 (0.0007) -[2023-10-14 15:22:24,957][75949] Updated weights for policy 0, policy_version 46851 (0.0007) -[2023-10-14 15:22:25,329][75949] Updated weights for policy 0, policy_version 46861 (0.0007) -[2023-10-14 15:22:25,708][75949] Updated weights for policy 0, policy_version 46871 (0.0007) -[2023-10-14 15:22:28,086][75950] Updated weights for policy 1, policy_version 46760 (0.0007) -[2023-10-14 15:22:28,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 95879168. Throughput: 0: 1694.5, 1: 1680.1. Samples: 23983874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 15:22:28,164][74987] Avg episode reward: [(0, '24.320'), (1, '30.870')] -[2023-10-14 15:22:28,440][75950] Updated weights for policy 1, policy_version 46770 (0.0007) -[2023-10-14 15:22:28,811][75950] Updated weights for policy 1, policy_version 46780 (0.0009) -[2023-10-14 15:22:29,726][75949] Updated weights for policy 0, policy_version 46881 (0.0010) -[2023-10-14 15:22:30,096][75949] Updated weights for policy 0, policy_version 46891 (0.0010) -[2023-10-14 15:22:30,468][75949] Updated weights for policy 0, policy_version 46901 (0.0007) -[2023-10-14 15:22:30,838][75949] Updated weights for policy 0, policy_version 46911 (0.0007) -[2023-10-14 15:22:32,736][75950] Updated weights for policy 1, policy_version 46790 (0.0009) -[2023-10-14 15:22:33,113][75950] Updated weights for policy 1, policy_version 46800 (0.0008) -[2023-10-14 15:22:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 95944704. Throughput: 0: 1673.3, 1: 1680.3. Samples: 23993290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) -[2023-10-14 15:22:33,164][74987] Avg episode reward: [(0, '26.060'), (1, '30.990')] -[2023-10-14 15:22:33,479][75950] Updated weights for policy 1, policy_version 46810 (0.0008) -[2023-10-14 15:22:34,882][75949] Updated weights for policy 0, policy_version 46921 (0.0008) -[2023-10-14 15:22:35,248][75949] Updated weights for policy 0, policy_version 46931 (0.0009) -[2023-10-14 15:22:35,629][75949] Updated weights for policy 0, policy_version 46941 (0.0010) -[2023-10-14 15:22:37,678][75950] Updated weights for policy 1, policy_version 46820 (0.0009) -[2023-10-14 15:22:38,071][75950] Updated weights for policy 1, policy_version 46830 (0.0009) -[2023-10-14 15:22:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 96010240. Throughput: 0: 1684.2, 1: 1681.3. Samples: 24013792. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-14 15:22:38,164][74987] Avg episode reward: [(0, '23.240'), (1, '29.730')] -[2023-10-14 15:22:38,434][75950] Updated weights for policy 1, policy_version 46840 (0.0008) -[2023-10-14 15:22:39,763][75949] Updated weights for policy 0, policy_version 46951 (0.0008) -[2023-10-14 15:22:40,141][75949] Updated weights for policy 0, policy_version 46961 (0.0008) -[2023-10-14 15:22:40,514][75949] Updated weights for policy 0, policy_version 46971 (0.0007) -[2023-10-14 15:22:42,484][75950] Updated weights for policy 1, policy_version 46850 (0.0007) -[2023-10-14 15:22:42,848][75950] Updated weights for policy 1, policy_version 46860 (0.0007) -[2023-10-14 15:22:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 96075776. Throughput: 0: 1695.0, 1: 1677.9. Samples: 24034282. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-14 15:22:43,165][74987] Avg episode reward: [(0, '26.130'), (1, '29.010')] -[2023-10-14 15:22:43,174][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000046976_48103424.pth... -[2023-10-14 15:22:43,208][75950] Updated weights for policy 1, policy_version 46870 (0.0009) -[2023-10-14 15:22:43,209][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000045408_46497792.pth -[2023-10-14 15:22:43,213][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000046976_48103424.pth -[2023-10-14 15:22:43,566][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000046880_48005120.pth... -[2023-10-14 15:22:43,567][75950] Updated weights for policy 1, policy_version 46880 (0.0010) -[2023-10-14 15:22:43,605][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000045312_46399488.pth -[2023-10-14 15:22:43,610][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000046880_48005120.pth -[2023-10-14 15:22:44,621][75949] Updated weights for policy 0, policy_version 46981 (0.0008) -[2023-10-14 15:22:44,981][75949] Updated weights for policy 0, policy_version 46991 (0.0008) -[2023-10-14 15:22:45,361][75949] Updated weights for policy 0, policy_version 47001 (0.0009) -[2023-10-14 15:22:47,715][75950] Updated weights for policy 1, policy_version 46890 (0.0010) -[2023-10-14 15:22:48,082][75950] Updated weights for policy 1, policy_version 46900 (0.0009) -[2023-10-14 15:22:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 96141312. Throughput: 0: 1666.4, 1: 1686.8. Samples: 24043626. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-14 15:22:48,165][74987] Avg episode reward: [(0, '24.910'), (1, '31.330')] -[2023-10-14 15:22:48,446][75950] Updated weights for policy 1, policy_version 46910 (0.0009) -[2023-10-14 15:22:49,484][75949] Updated weights for policy 0, policy_version 47011 (0.0009) -[2023-10-14 15:22:49,858][75949] Updated weights for policy 0, policy_version 47021 (0.0007) -[2023-10-14 15:22:50,227][75949] Updated weights for policy 0, policy_version 47031 (0.0007) -[2023-10-14 15:22:52,545][75950] Updated weights for policy 1, policy_version 46920 (0.0007) -[2023-10-14 15:22:52,907][75950] Updated weights for policy 1, policy_version 46930 (0.0009) -[2023-10-14 15:22:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 96206848. Throughput: 0: 1687.0, 1: 1677.7. Samples: 24063966. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-14 15:22:53,165][74987] Avg episode reward: [(0, '25.760'), (1, '29.390')] -[2023-10-14 15:22:53,270][75950] Updated weights for policy 1, policy_version 46940 (0.0008) -[2023-10-14 15:22:54,450][75949] Updated weights for policy 0, policy_version 47041 (0.0008) -[2023-10-14 15:22:54,820][75949] Updated weights for policy 0, policy_version 47051 (0.0009) -[2023-10-14 15:22:55,189][75949] Updated weights for policy 0, policy_version 47061 (0.0009) -[2023-10-14 15:22:55,552][75949] Updated weights for policy 0, policy_version 47071 (0.0008) -[2023-10-14 15:22:57,444][75950] Updated weights for policy 1, policy_version 46950 (0.0008) -[2023-10-14 15:22:57,809][75950] Updated weights for policy 1, policy_version 46960 (0.0010) -[2023-10-14 15:22:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 96272384. Throughput: 0: 1680.8, 1: 1672.3. Samples: 24084258. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-14 15:22:58,164][74987] Avg episode reward: [(0, '26.480'), (1, '30.670')] -[2023-10-14 15:22:58,182][75950] Updated weights for policy 1, policy_version 46970 (0.0009) -[2023-10-14 15:22:59,693][75949] Updated weights for policy 0, policy_version 47081 (0.0008) -[2023-10-14 15:23:00,066][75949] Updated weights for policy 0, policy_version 47091 (0.0009) -[2023-10-14 15:23:00,433][75949] Updated weights for policy 0, policy_version 47101 (0.0007) -[2023-10-14 15:23:02,203][75950] Updated weights for policy 1, policy_version 46980 (0.0008) -[2023-10-14 15:23:02,566][75950] Updated weights for policy 1, policy_version 46990 (0.0010) -[2023-10-14 15:23:02,931][75950] Updated weights for policy 1, policy_version 47000 (0.0008) -[2023-10-14 15:23:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 96337920. Throughput: 0: 1663.5, 1: 1683.1. Samples: 24093634. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-14 15:23:03,165][74987] Avg episode reward: [(0, '24.960'), (1, '30.120')] -[2023-10-14 15:23:04,331][75949] Updated weights for policy 0, policy_version 47111 (0.0010) -[2023-10-14 15:23:04,706][75949] Updated weights for policy 0, policy_version 47121 (0.0010) -[2023-10-14 15:23:05,084][75949] Updated weights for policy 0, policy_version 47131 (0.0008) -[2023-10-14 15:23:07,185][75950] Updated weights for policy 1, policy_version 47010 (0.0008) -[2023-10-14 15:23:07,554][75950] Updated weights for policy 1, policy_version 47020 (0.0008) -[2023-10-14 15:23:07,917][75950] Updated weights for policy 1, policy_version 47030 (0.0010) -[2023-10-14 15:23:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96403456. Throughput: 0: 1678.0, 1: 1680.7. Samples: 24114248. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) -[2023-10-14 15:23:08,164][74987] Avg episode reward: [(0, '24.910'), (1, '30.970')] -[2023-10-14 15:23:08,286][75950] Updated weights for policy 1, policy_version 47040 (0.0010) -[2023-10-14 15:23:09,149][75949] Updated weights for policy 0, policy_version 47141 (0.0009) -[2023-10-14 15:23:09,517][75949] Updated weights for policy 0, policy_version 47151 (0.0010) -[2023-10-14 15:23:09,888][75949] Updated weights for policy 0, policy_version 47161 (0.0009) -[2023-10-14 15:23:11,983][75950] Updated weights for policy 1, policy_version 47050 (0.0009) -[2023-10-14 15:23:12,349][75950] Updated weights for policy 1, policy_version 47060 (0.0010) -[2023-10-14 15:23:12,710][75950] Updated weights for policy 1, policy_version 47070 (0.0010) -[2023-10-14 15:23:13,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 96501760. Throughput: 0: 1680.8, 1: 1657.0. Samples: 24134072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:23:13,164][74987] Avg episode reward: [(0, '23.340'), (1, '29.430')] -[2023-10-14 15:23:13,904][75949] Updated weights for policy 0, policy_version 47171 (0.0011) -[2023-10-14 15:23:14,271][75949] Updated weights for policy 0, policy_version 47181 (0.0011) -[2023-10-14 15:23:14,641][75949] Updated weights for policy 0, policy_version 47191 (0.0011) -[2023-10-14 15:23:17,104][75950] Updated weights for policy 1, policy_version 47080 (0.0009) -[2023-10-14 15:23:17,469][75950] Updated weights for policy 1, policy_version 47090 (0.0009) -[2023-10-14 15:23:17,846][75950] Updated weights for policy 1, policy_version 47100 (0.0010) -[2023-10-14 15:23:18,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 96567296. Throughput: 0: 1674.8, 1: 1676.4. Samples: 24144094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:23:18,165][74987] Avg episode reward: [(0, '24.150'), (1, '31.430')] -[2023-10-14 15:23:18,689][75949] Updated weights for policy 0, policy_version 47201 (0.0008) -[2023-10-14 15:23:19,056][75949] Updated weights for policy 0, policy_version 47211 (0.0009) -[2023-10-14 15:23:19,423][75949] Updated weights for policy 0, policy_version 47221 (0.0010) -[2023-10-14 15:23:19,789][75949] Updated weights for policy 0, policy_version 47231 (0.0010) -[2023-10-14 15:23:21,986][75950] Updated weights for policy 1, policy_version 47110 (0.0011) -[2023-10-14 15:23:22,355][75950] Updated weights for policy 1, policy_version 47120 (0.0009) -[2023-10-14 15:23:22,725][75950] Updated weights for policy 1, policy_version 47130 (0.0008) -[2023-10-14 15:23:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 96632832. Throughput: 0: 1676.6, 1: 1672.7. Samples: 24164510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:23:23,165][74987] Avg episode reward: [(0, '25.090'), (1, '33.050')] -[2023-10-14 15:23:23,813][75949] Updated weights for policy 0, policy_version 47241 (0.0009) -[2023-10-14 15:23:24,189][75949] Updated weights for policy 0, policy_version 47251 (0.0008) -[2023-10-14 15:23:24,568][75949] Updated weights for policy 0, policy_version 47261 (0.0010) -[2023-10-14 15:23:26,986][75950] Updated weights for policy 1, policy_version 47140 (0.0009) -[2023-10-14 15:23:27,383][75950] Updated weights for policy 1, policy_version 47150 (0.0008) -[2023-10-14 15:23:27,751][75950] Updated weights for policy 1, policy_version 47160 (0.0007) -[2023-10-14 15:23:28,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 96698368. Throughput: 0: 1677.9, 1: 1656.7. Samples: 24184338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:23:28,164][74987] Avg episode reward: [(0, '26.260'), (1, '29.380')] -[2023-10-14 15:23:28,677][75949] Updated weights for policy 0, policy_version 47271 (0.0009) -[2023-10-14 15:23:29,068][75949] Updated weights for policy 0, policy_version 47281 (0.0010) -[2023-10-14 15:23:29,443][75949] Updated weights for policy 0, policy_version 47291 (0.0009) -[2023-10-14 15:23:31,825][75950] Updated weights for policy 1, policy_version 47170 (0.0007) -[2023-10-14 15:23:32,186][75950] Updated weights for policy 1, policy_version 47180 (0.0008) -[2023-10-14 15:23:32,561][75950] Updated weights for policy 1, policy_version 47190 (0.0008) -[2023-10-14 15:23:32,936][75950] Updated weights for policy 1, policy_version 47200 (0.0008) -[2023-10-14 15:23:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 96763904. Throughput: 0: 1674.1, 1: 1667.1. Samples: 24193980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:23:33,164][74987] Avg episode reward: [(0, '24.010'), (1, '31.070')] -[2023-10-14 15:23:33,617][75949] Updated weights for policy 0, policy_version 47301 (0.0010) -[2023-10-14 15:23:33,993][75949] Updated weights for policy 0, policy_version 47311 (0.0010) -[2023-10-14 15:23:34,356][75949] Updated weights for policy 0, policy_version 47321 (0.0007) -[2023-10-14 15:23:36,943][75950] Updated weights for policy 1, policy_version 47210 (0.0010) -[2023-10-14 15:23:37,311][75950] Updated weights for policy 1, policy_version 47220 (0.0009) -[2023-10-14 15:23:37,673][75950] Updated weights for policy 1, policy_version 47230 (0.0011) -[2023-10-14 15:23:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 96829440. Throughput: 0: 1676.0, 1: 1666.8. Samples: 24214390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:23:38,164][74987] Avg episode reward: [(0, '26.470'), (1, '29.990')] -[2023-10-14 15:23:38,521][75949] Updated weights for policy 0, policy_version 47331 (0.0008) -[2023-10-14 15:23:38,901][75949] Updated weights for policy 0, policy_version 47341 (0.0009) -[2023-10-14 15:23:39,270][75949] Updated weights for policy 0, policy_version 47351 (0.0008) -[2023-10-14 15:23:41,865][75950] Updated weights for policy 1, policy_version 47240 (0.0008) -[2023-10-14 15:23:42,235][75950] Updated weights for policy 1, policy_version 47250 (0.0009) -[2023-10-14 15:23:42,595][75950] Updated weights for policy 1, policy_version 47260 (0.0008) -[2023-10-14 15:23:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 96894976. Throughput: 0: 1680.9, 1: 1649.2. Samples: 24234116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:23:43,164][74987] Avg episode reward: [(0, '24.440'), (1, '28.570')] -[2023-10-14 15:23:43,352][75949] Updated weights for policy 0, policy_version 47361 (0.0008) -[2023-10-14 15:23:43,719][75949] Updated weights for policy 0, policy_version 47371 (0.0008) -[2023-10-14 15:23:44,094][75949] Updated weights for policy 0, policy_version 47381 (0.0011) -[2023-10-14 15:23:44,471][75949] Updated weights for policy 0, policy_version 47391 (0.0011) -[2023-10-14 15:23:46,646][75950] Updated weights for policy 1, policy_version 47270 (0.0009) -[2023-10-14 15:23:47,011][75950] Updated weights for policy 1, policy_version 47280 (0.0008) -[2023-10-14 15:23:47,376][75950] Updated weights for policy 1, policy_version 47290 (0.0007) -[2023-10-14 15:23:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 96960512. Throughput: 0: 1686.0, 1: 1662.5. Samples: 24244316. Policy #0 lag: (min: 20.0, avg: 20.0, max: 21.0) -[2023-10-14 15:23:48,164][74987] Avg episode reward: [(0, '25.360'), (1, '27.640')] -[2023-10-14 15:23:48,422][75949] Updated weights for policy 0, policy_version 47401 (0.0010) -[2023-10-14 15:23:48,803][75949] Updated weights for policy 0, policy_version 47411 (0.0010) -[2023-10-14 15:23:49,162][75949] Updated weights for policy 0, policy_version 47421 (0.0008) -[2023-10-14 15:23:51,451][75950] Updated weights for policy 1, policy_version 47300 (0.0007) -[2023-10-14 15:23:51,815][75950] Updated weights for policy 1, policy_version 47310 (0.0007) -[2023-10-14 15:23:52,178][75950] Updated weights for policy 1, policy_version 47320 (0.0007) -[2023-10-14 15:23:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 97026048. Throughput: 0: 1683.5, 1: 1661.6. Samples: 24264780. Policy #0 lag: (min: 20.0, avg: 20.0, max: 21.0) -[2023-10-14 15:23:53,164][74987] Avg episode reward: [(0, '24.820'), (1, '32.090')] -[2023-10-14 15:23:53,186][75949] Updated weights for policy 0, policy_version 47431 (0.0008) -[2023-10-14 15:23:53,558][75949] Updated weights for policy 0, policy_version 47441 (0.0008) -[2023-10-14 15:23:53,936][75949] Updated weights for policy 0, policy_version 47451 (0.0008) -[2023-10-14 15:23:56,050][75950] Updated weights for policy 1, policy_version 47330 (0.0009) -[2023-10-14 15:23:56,417][75950] Updated weights for policy 1, policy_version 47340 (0.0008) -[2023-10-14 15:23:56,778][75950] Updated weights for policy 1, policy_version 47350 (0.0009) -[2023-10-14 15:23:57,138][75950] Updated weights for policy 1, policy_version 47360 (0.0008) -[2023-10-14 15:23:57,864][75949] Updated weights for policy 0, policy_version 47461 (0.0008) -[2023-10-14 15:23:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 97091584. Throughput: 0: 1684.4, 1: 1664.6. Samples: 24284780. Policy #0 lag: (min: 20.0, avg: 20.0, max: 21.0) -[2023-10-14 15:23:58,164][74987] Avg episode reward: [(0, '24.140'), (1, '30.060')] -[2023-10-14 15:23:58,238][75949] Updated weights for policy 0, policy_version 47471 (0.0009) -[2023-10-14 15:23:58,610][75949] Updated weights for policy 0, policy_version 47481 (0.0010) -[2023-10-14 15:24:01,235][75950] Updated weights for policy 1, policy_version 47370 (0.0010) -[2023-10-14 15:24:01,601][75950] Updated weights for policy 1, policy_version 47380 (0.0007) -[2023-10-14 15:24:01,967][75950] Updated weights for policy 1, policy_version 47390 (0.0010) -[2023-10-14 15:24:02,484][75949] Updated weights for policy 0, policy_version 47491 (0.0007) -[2023-10-14 15:24:02,855][75949] Updated weights for policy 0, policy_version 47501 (0.0008) -[2023-10-14 15:24:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 97157120. Throughput: 0: 1684.6, 1: 1674.4. Samples: 24295248. Policy #0 lag: (min: 20.0, avg: 20.0, max: 21.0) -[2023-10-14 15:24:03,165][74987] Avg episode reward: [(0, '25.900'), (1, '28.840')] -[2023-10-14 15:24:03,232][75949] Updated weights for policy 0, policy_version 47511 (0.0010) -[2023-10-14 15:24:06,025][75950] Updated weights for policy 1, policy_version 47400 (0.0009) -[2023-10-14 15:24:06,396][75950] Updated weights for policy 1, policy_version 47410 (0.0008) -[2023-10-14 15:24:06,769][75950] Updated weights for policy 1, policy_version 47420 (0.0009) -[2023-10-14 15:24:07,370][75949] Updated weights for policy 0, policy_version 47521 (0.0010) -[2023-10-14 15:24:07,738][75949] Updated weights for policy 0, policy_version 47531 (0.0008) -[2023-10-14 15:24:08,113][75949] Updated weights for policy 0, policy_version 47541 (0.0007) -[2023-10-14 15:24:08,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 97222656. Throughput: 0: 1689.5, 1: 1659.1. Samples: 24315198. Policy #0 lag: (min: 20.0, avg: 20.0, max: 21.0) -[2023-10-14 15:24:08,164][74987] Avg episode reward: [(0, '25.930'), (1, '28.690')] -[2023-10-14 15:24:08,487][75949] Updated weights for policy 0, policy_version 47551 (0.0010) -[2023-10-14 15:24:10,959][75950] Updated weights for policy 1, policy_version 47430 (0.0008) -[2023-10-14 15:24:11,324][75950] Updated weights for policy 1, policy_version 47440 (0.0007) -[2023-10-14 15:24:11,691][75950] Updated weights for policy 1, policy_version 47450 (0.0009) -[2023-10-14 15:24:12,460][75949] Updated weights for policy 0, policy_version 47561 (0.0009) -[2023-10-14 15:24:12,832][75949] Updated weights for policy 0, policy_version 47571 (0.0009) -[2023-10-14 15:24:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 97288192. Throughput: 0: 1675.9, 1: 1673.7. Samples: 24335072. Policy #0 lag: (min: 20.0, avg: 20.0, max: 21.0) -[2023-10-14 15:24:13,164][74987] Avg episode reward: [(0, '26.660'), (1, '26.720')] -[2023-10-14 15:24:13,206][75949] Updated weights for policy 0, policy_version 47581 (0.0007) -[2023-10-14 15:24:15,699][75950] Updated weights for policy 1, policy_version 47460 (0.0008) -[2023-10-14 15:24:16,094][75950] Updated weights for policy 1, policy_version 47470 (0.0009) -[2023-10-14 15:24:16,462][75950] Updated weights for policy 1, policy_version 47480 (0.0010) -[2023-10-14 15:24:17,351][75949] Updated weights for policy 0, policy_version 47591 (0.0010) -[2023-10-14 15:24:17,738][75949] Updated weights for policy 0, policy_version 47601 (0.0011) -[2023-10-14 15:24:18,104][75949] Updated weights for policy 0, policy_version 47611 (0.0010) -[2023-10-14 15:24:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 97353728. Throughput: 0: 1693.2, 1: 1682.2. Samples: 24345876. Policy #0 lag: (min: 20.0, avg: 20.0, max: 21.0) -[2023-10-14 15:24:18,164][74987] Avg episode reward: [(0, '27.110'), (1, '26.450')] -[2023-10-14 15:24:20,398][75950] Updated weights for policy 1, policy_version 47490 (0.0010) -[2023-10-14 15:24:20,773][75950] Updated weights for policy 1, policy_version 47500 (0.0010) -[2023-10-14 15:24:21,142][75950] Updated weights for policy 1, policy_version 47510 (0.0009) -[2023-10-14 15:24:21,510][75950] Updated weights for policy 1, policy_version 47520 (0.0009) -[2023-10-14 15:24:22,227][75949] Updated weights for policy 0, policy_version 47621 (0.0008) -[2023-10-14 15:24:22,600][75949] Updated weights for policy 0, policy_version 47631 (0.0008) -[2023-10-14 15:24:22,970][75949] Updated weights for policy 0, policy_version 47641 (0.0008) -[2023-10-14 15:24:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 97419264. Throughput: 0: 1693.2, 1: 1661.2. Samples: 24365334. Policy #0 lag: (min: 19.0, avg: 26.1, max: 51.0) -[2023-10-14 15:24:23,164][74987] Avg episode reward: [(0, '24.210'), (1, '29.010')] -[2023-10-14 15:24:25,577][75950] Updated weights for policy 1, policy_version 47530 (0.0007) -[2023-10-14 15:24:25,946][75950] Updated weights for policy 1, policy_version 47540 (0.0007) -[2023-10-14 15:24:26,315][75950] Updated weights for policy 1, policy_version 47550 (0.0009) -[2023-10-14 15:24:26,975][75949] Updated weights for policy 0, policy_version 47651 (0.0008) -[2023-10-14 15:24:27,344][75949] Updated weights for policy 0, policy_version 47661 (0.0008) -[2023-10-14 15:24:27,724][75949] Updated weights for policy 0, policy_version 47671 (0.0011) -[2023-10-14 15:24:28,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 97517568. Throughput: 0: 1671.3, 1: 1682.2. Samples: 24385022. Policy #0 lag: (min: 19.0, avg: 26.1, max: 51.0) -[2023-10-14 15:24:28,165][74987] Avg episode reward: [(0, '27.140'), (1, '29.530')] -[2023-10-14 15:24:30,492][75950] Updated weights for policy 1, policy_version 47560 (0.0007) -[2023-10-14 15:24:30,856][75950] Updated weights for policy 1, policy_version 47570 (0.0007) -[2023-10-14 15:24:31,225][75950] Updated weights for policy 1, policy_version 47580 (0.0009) -[2023-10-14 15:24:31,812][75949] Updated weights for policy 0, policy_version 47681 (0.0010) -[2023-10-14 15:24:32,176][75949] Updated weights for policy 0, policy_version 47691 (0.0008) -[2023-10-14 15:24:32,545][75949] Updated weights for policy 0, policy_version 47701 (0.0008) -[2023-10-14 15:24:32,915][75949] Updated weights for policy 0, policy_version 47711 (0.0007) -[2023-10-14 15:24:33,163][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 97583104. Throughput: 0: 1685.1, 1: 1677.6. Samples: 24395638. Policy #0 lag: (min: 19.0, avg: 26.1, max: 51.0) -[2023-10-14 15:24:33,164][74987] Avg episode reward: [(0, '24.530'), (1, '31.780')] -[2023-10-14 15:24:35,341][75950] Updated weights for policy 1, policy_version 47590 (0.0008) -[2023-10-14 15:24:35,704][75950] Updated weights for policy 1, policy_version 47600 (0.0008) -[2023-10-14 15:24:36,068][75950] Updated weights for policy 1, policy_version 47610 (0.0008) -[2023-10-14 15:24:36,986][75949] Updated weights for policy 0, policy_version 47721 (0.0009) -[2023-10-14 15:24:37,352][75949] Updated weights for policy 0, policy_version 47731 (0.0010) -[2023-10-14 15:24:37,729][75949] Updated weights for policy 0, policy_version 47741 (0.0008) -[2023-10-14 15:24:38,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 97648640. Throughput: 0: 1684.7, 1: 1661.0. Samples: 24415336. Policy #0 lag: (min: 19.0, avg: 26.1, max: 51.0) -[2023-10-14 15:24:38,164][74987] Avg episode reward: [(0, '26.690'), (1, '32.130')] -[2023-10-14 15:24:40,212][75950] Updated weights for policy 1, policy_version 47620 (0.0009) -[2023-10-14 15:24:40,579][75950] Updated weights for policy 1, policy_version 47630 (0.0009) -[2023-10-14 15:24:40,950][75950] Updated weights for policy 1, policy_version 47640 (0.0007) -[2023-10-14 15:24:41,799][75949] Updated weights for policy 0, policy_version 47751 (0.0007) -[2023-10-14 15:24:42,169][75949] Updated weights for policy 0, policy_version 47761 (0.0007) -[2023-10-14 15:24:42,544][75949] Updated weights for policy 0, policy_version 47771 (0.0009) -[2023-10-14 15:24:43,164][74987] Fps is (10 sec: 13106.3, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 97714176. Throughput: 0: 1656.9, 1: 1683.5. Samples: 24435102. Policy #0 lag: (min: 19.0, avg: 26.1, max: 51.0) -[2023-10-14 15:24:43,165][74987] Avg episode reward: [(0, '26.040'), (1, '31.950')] -[2023-10-14 15:24:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000047648_48791552.pth... -[2023-10-14 15:24:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000047776_48922624.pth... -[2023-10-14 15:24:43,211][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000046080_47185920.pth -[2023-10-14 15:24:43,211][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000046208_47316992.pth -[2023-10-14 15:24:45,049][75950] Updated weights for policy 1, policy_version 47650 (0.0008) -[2023-10-14 15:24:45,408][75950] Updated weights for policy 1, policy_version 47660 (0.0008) -[2023-10-14 15:24:45,775][75950] Updated weights for policy 1, policy_version 47670 (0.0009) -[2023-10-14 15:24:46,129][75950] Updated weights for policy 1, policy_version 47680 (0.0009) -[2023-10-14 15:24:46,634][75949] Updated weights for policy 0, policy_version 47781 (0.0009) -[2023-10-14 15:24:46,989][75949] Updated weights for policy 0, policy_version 47791 (0.0009) -[2023-10-14 15:24:47,364][75949] Updated weights for policy 0, policy_version 47801 (0.0008) -[2023-10-14 15:24:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 97779712. Throughput: 0: 1679.2, 1: 1668.0. Samples: 24445870. Policy #0 lag: (min: 19.0, avg: 26.1, max: 51.0) -[2023-10-14 15:24:48,165][74987] Avg episode reward: [(0, '26.300'), (1, '30.280')] -[2023-10-14 15:24:50,103][75950] Updated weights for policy 1, policy_version 47690 (0.0008) -[2023-10-14 15:24:50,475][75950] Updated weights for policy 1, policy_version 47700 (0.0007) -[2023-10-14 15:24:50,840][75950] Updated weights for policy 1, policy_version 47710 (0.0008) -[2023-10-14 15:24:51,446][75949] Updated weights for policy 0, policy_version 47811 (0.0010) -[2023-10-14 15:24:51,810][75949] Updated weights for policy 0, policy_version 47821 (0.0009) -[2023-10-14 15:24:52,185][75949] Updated weights for policy 0, policy_version 47831 (0.0008) -[2023-10-14 15:24:53,164][74987] Fps is (10 sec: 13107.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 97845248. Throughput: 0: 1667.2, 1: 1670.0. Samples: 24465372. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 15:24:53,164][74987] Avg episode reward: [(0, '26.020'), (1, '31.600')] -[2023-10-14 15:24:54,929][75950] Updated weights for policy 1, policy_version 47720 (0.0009) -[2023-10-14 15:24:55,294][75950] Updated weights for policy 1, policy_version 47730 (0.0010) -[2023-10-14 15:24:55,664][75950] Updated weights for policy 1, policy_version 47740 (0.0008) -[2023-10-14 15:24:56,298][75949] Updated weights for policy 0, policy_version 47841 (0.0008) -[2023-10-14 15:24:56,666][75949] Updated weights for policy 0, policy_version 47851 (0.0010) -[2023-10-14 15:24:57,047][75949] Updated weights for policy 0, policy_version 47861 (0.0007) -[2023-10-14 15:24:57,425][75949] Updated weights for policy 0, policy_version 47871 (0.0009) -[2023-10-14 15:24:58,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 97910784. Throughput: 0: 1655.5, 1: 1676.8. Samples: 24485026. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 15:24:58,164][74987] Avg episode reward: [(0, '25.130'), (1, '29.270')] -[2023-10-14 15:24:59,777][75950] Updated weights for policy 1, policy_version 47750 (0.0007) -[2023-10-14 15:25:00,147][75950] Updated weights for policy 1, policy_version 47760 (0.0010) -[2023-10-14 15:25:00,521][75950] Updated weights for policy 1, policy_version 47770 (0.0011) -[2023-10-14 15:25:01,411][75949] Updated weights for policy 0, policy_version 47881 (0.0010) -[2023-10-14 15:25:01,783][75949] Updated weights for policy 0, policy_version 47891 (0.0009) -[2023-10-14 15:25:02,154][75949] Updated weights for policy 0, policy_version 47901 (0.0008) -[2023-10-14 15:25:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 97976320. Throughput: 0: 1675.3, 1: 1657.1. Samples: 24495832. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 15:25:03,164][74987] Avg episode reward: [(0, '25.720'), (1, '31.270')] -[2023-10-14 15:25:04,813][75950] Updated weights for policy 1, policy_version 47780 (0.0010) -[2023-10-14 15:25:05,169][75950] Updated weights for policy 1, policy_version 47790 (0.0011) -[2023-10-14 15:25:05,542][75950] Updated weights for policy 1, policy_version 47800 (0.0008) -[2023-10-14 15:25:06,458][75949] Updated weights for policy 0, policy_version 47911 (0.0009) -[2023-10-14 15:25:06,826][75949] Updated weights for policy 0, policy_version 47921 (0.0010) -[2023-10-14 15:25:07,192][75949] Updated weights for policy 0, policy_version 47931 (0.0011) -[2023-10-14 15:25:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 98041856. Throughput: 0: 1661.7, 1: 1672.8. Samples: 24515390. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 15:25:08,165][74987] Avg episode reward: [(0, '24.540'), (1, '31.660')] -[2023-10-14 15:25:09,685][75950] Updated weights for policy 1, policy_version 47810 (0.0008) -[2023-10-14 15:25:10,057][75950] Updated weights for policy 1, policy_version 47820 (0.0008) -[2023-10-14 15:25:10,425][75950] Updated weights for policy 1, policy_version 47830 (0.0008) -[2023-10-14 15:25:10,782][75950] Updated weights for policy 1, policy_version 47840 (0.0010) -[2023-10-14 15:25:11,386][75949] Updated weights for policy 0, policy_version 47941 (0.0010) -[2023-10-14 15:25:11,756][75949] Updated weights for policy 0, policy_version 47951 (0.0008) -[2023-10-14 15:25:12,131][75949] Updated weights for policy 0, policy_version 47961 (0.0007) -[2023-10-14 15:25:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 98107392. Throughput: 0: 1660.2, 1: 1672.9. Samples: 24535010. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 15:25:13,165][74987] Avg episode reward: [(0, '25.890'), (1, '29.690')] -[2023-10-14 15:25:14,986][75950] Updated weights for policy 1, policy_version 47850 (0.0010) -[2023-10-14 15:25:15,353][75950] Updated weights for policy 1, policy_version 47860 (0.0009) -[2023-10-14 15:25:15,719][75950] Updated weights for policy 1, policy_version 47870 (0.0009) -[2023-10-14 15:25:16,283][75949] Updated weights for policy 0, policy_version 47971 (0.0009) -[2023-10-14 15:25:16,650][75949] Updated weights for policy 0, policy_version 47981 (0.0009) -[2023-10-14 15:25:17,026][75949] Updated weights for policy 0, policy_version 47991 (0.0009) -[2023-10-14 15:25:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 98172928. Throughput: 0: 1672.2, 1: 1656.3. Samples: 24545420. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 15:25:18,165][74987] Avg episode reward: [(0, '24.750'), (1, '28.510')] -[2023-10-14 15:25:19,971][75950] Updated weights for policy 1, policy_version 47880 (0.0008) -[2023-10-14 15:25:20,336][75950] Updated weights for policy 1, policy_version 47890 (0.0007) -[2023-10-14 15:25:20,709][75950] Updated weights for policy 1, policy_version 47900 (0.0007) -[2023-10-14 15:25:20,950][75949] Updated weights for policy 0, policy_version 48001 (0.0009) -[2023-10-14 15:25:21,327][75949] Updated weights for policy 0, policy_version 48011 (0.0009) -[2023-10-14 15:25:21,689][75949] Updated weights for policy 0, policy_version 48021 (0.0011) -[2023-10-14 15:25:22,064][75949] Updated weights for policy 0, policy_version 48031 (0.0009) -[2023-10-14 15:25:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 98238464. Throughput: 0: 1659.6, 1: 1668.6. Samples: 24565104. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 15:25:23,165][74987] Avg episode reward: [(0, '26.450'), (1, '27.380')] -[2023-10-14 15:25:24,895][75950] Updated weights for policy 1, policy_version 47910 (0.0009) -[2023-10-14 15:25:25,266][75950] Updated weights for policy 1, policy_version 47920 (0.0007) -[2023-10-14 15:25:25,632][75950] Updated weights for policy 1, policy_version 47930 (0.0007) -[2023-10-14 15:25:26,005][75949] Updated weights for policy 0, policy_version 48041 (0.0010) -[2023-10-14 15:25:26,379][75949] Updated weights for policy 0, policy_version 48051 (0.0009) -[2023-10-14 15:25:26,752][75949] Updated weights for policy 0, policy_version 48061 (0.0009) -[2023-10-14 15:25:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 98304000. Throughput: 0: 1674.3, 1: 1661.8. Samples: 24585226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:25:28,165][74987] Avg episode reward: [(0, '24.550'), (1, '27.360')] -[2023-10-14 15:25:29,547][75950] Updated weights for policy 1, policy_version 47940 (0.0008) -[2023-10-14 15:25:29,924][75950] Updated weights for policy 1, policy_version 47950 (0.0007) -[2023-10-14 15:25:30,288][75950] Updated weights for policy 1, policy_version 47960 (0.0007) -[2023-10-14 15:25:30,798][75949] Updated weights for policy 0, policy_version 48071 (0.0008) -[2023-10-14 15:25:31,164][75949] Updated weights for policy 0, policy_version 48081 (0.0009) -[2023-10-14 15:25:31,543][75949] Updated weights for policy 0, policy_version 48091 (0.0009) -[2023-10-14 15:25:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 98369536. Throughput: 0: 1676.9, 1: 1644.9. Samples: 24595354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:25:33,165][74987] Avg episode reward: [(0, '26.660'), (1, '26.960')] -[2023-10-14 15:25:34,516][75950] Updated weights for policy 1, policy_version 47970 (0.0008) -[2023-10-14 15:25:34,875][75950] Updated weights for policy 1, policy_version 47980 (0.0010) -[2023-10-14 15:25:35,251][75950] Updated weights for policy 1, policy_version 47990 (0.0009) -[2023-10-14 15:25:35,561][75949] Updated weights for policy 0, policy_version 48101 (0.0009) -[2023-10-14 15:25:35,610][75950] Updated weights for policy 1, policy_version 48000 (0.0009) -[2023-10-14 15:25:35,922][75949] Updated weights for policy 0, policy_version 48111 (0.0012) -[2023-10-14 15:25:36,299][75949] Updated weights for policy 0, policy_version 48121 (0.0010) -[2023-10-14 15:25:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 98435072. Throughput: 0: 1659.9, 1: 1661.4. Samples: 24614830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:25:38,164][74987] Avg episode reward: [(0, '24.370'), (1, '29.430')] -[2023-10-14 15:25:39,669][75950] Updated weights for policy 1, policy_version 48010 (0.0007) -[2023-10-14 15:25:40,039][75950] Updated weights for policy 1, policy_version 48020 (0.0007) -[2023-10-14 15:25:40,415][75950] Updated weights for policy 1, policy_version 48030 (0.0007) -[2023-10-14 15:25:40,435][75949] Updated weights for policy 0, policy_version 48131 (0.0009) -[2023-10-14 15:25:40,806][75949] Updated weights for policy 0, policy_version 48141 (0.0007) -[2023-10-14 15:25:41,181][75949] Updated weights for policy 0, policy_version 48151 (0.0008) -[2023-10-14 15:25:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 98500608. Throughput: 0: 1683.7, 1: 1667.3. Samples: 24635822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:25:43,164][74987] Avg episode reward: [(0, '27.020'), (1, '30.850')] -[2023-10-14 15:25:44,338][75950] Updated weights for policy 1, policy_version 48040 (0.0008) -[2023-10-14 15:25:44,703][75950] Updated weights for policy 1, policy_version 48050 (0.0010) -[2023-10-14 15:25:45,079][75950] Updated weights for policy 1, policy_version 48060 (0.0008) -[2023-10-14 15:25:45,164][75949] Updated weights for policy 0, policy_version 48161 (0.0007) -[2023-10-14 15:25:45,536][75949] Updated weights for policy 0, policy_version 48171 (0.0008) -[2023-10-14 15:25:45,895][75949] Updated weights for policy 0, policy_version 48181 (0.0011) -[2023-10-14 15:25:46,270][75949] Updated weights for policy 0, policy_version 48191 (0.0010) -[2023-10-14 15:25:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 98566144. Throughput: 0: 1666.8, 1: 1662.6. Samples: 24645658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:25:48,164][74987] Avg episode reward: [(0, '24.980'), (1, '31.300')] -[2023-10-14 15:25:49,154][75950] Updated weights for policy 1, policy_version 48070 (0.0009) -[2023-10-14 15:25:49,520][75950] Updated weights for policy 1, policy_version 48080 (0.0009) -[2023-10-14 15:25:49,888][75950] Updated weights for policy 1, policy_version 48090 (0.0011) -[2023-10-14 15:25:50,415][75949] Updated weights for policy 0, policy_version 48201 (0.0011) -[2023-10-14 15:25:50,788][75949] Updated weights for policy 0, policy_version 48211 (0.0010) -[2023-10-14 15:25:51,165][75949] Updated weights for policy 0, policy_version 48221 (0.0007) -[2023-10-14 15:25:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 98631680. Throughput: 0: 1664.8, 1: 1670.0. Samples: 24665456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:25:53,165][74987] Avg episode reward: [(0, '27.060'), (1, '32.720')] -[2023-10-14 15:25:53,972][75950] Updated weights for policy 1, policy_version 48100 (0.0007) -[2023-10-14 15:25:54,361][75950] Updated weights for policy 1, policy_version 48110 (0.0008) -[2023-10-14 15:25:54,735][75950] Updated weights for policy 1, policy_version 48120 (0.0007) -[2023-10-14 15:25:55,285][75949] Updated weights for policy 0, policy_version 48231 (0.0007) -[2023-10-14 15:25:55,668][75949] Updated weights for policy 0, policy_version 48241 (0.0007) -[2023-10-14 15:25:56,044][75949] Updated weights for policy 0, policy_version 48251 (0.0008) -[2023-10-14 15:25:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 98697216. Throughput: 0: 1685.7, 1: 1673.3. Samples: 24686164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:25:58,164][74987] Avg episode reward: [(0, '25.010'), (1, '31.280')] -[2023-10-14 15:25:58,879][75950] Updated weights for policy 1, policy_version 48130 (0.0008) -[2023-10-14 15:25:59,245][75950] Updated weights for policy 1, policy_version 48140 (0.0008) -[2023-10-14 15:25:59,609][75950] Updated weights for policy 1, policy_version 48150 (0.0007) -[2023-10-14 15:25:59,978][75950] Updated weights for policy 1, policy_version 48160 (0.0009) -[2023-10-14 15:26:00,048][75949] Updated weights for policy 0, policy_version 48261 (0.0008) -[2023-10-14 15:26:00,427][75949] Updated weights for policy 0, policy_version 48271 (0.0007) -[2023-10-14 15:26:00,802][75949] Updated weights for policy 0, policy_version 48281 (0.0007) -[2023-10-14 15:26:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 98762752. Throughput: 0: 1674.2, 1: 1671.5. Samples: 24695976. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 15:26:03,164][74987] Avg episode reward: [(0, '26.520'), (1, '31.370')] -[2023-10-14 15:26:04,066][75950] Updated weights for policy 1, policy_version 48170 (0.0011) -[2023-10-14 15:26:04,430][75950] Updated weights for policy 1, policy_version 48180 (0.0011) -[2023-10-14 15:26:04,795][75950] Updated weights for policy 1, policy_version 48190 (0.0008) -[2023-10-14 15:26:04,867][75949] Updated weights for policy 0, policy_version 48291 (0.0008) -[2023-10-14 15:26:05,233][75949] Updated weights for policy 0, policy_version 48301 (0.0011) -[2023-10-14 15:26:05,606][75949] Updated weights for policy 0, policy_version 48311 (0.0008) -[2023-10-14 15:26:08,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 98828288. Throughput: 0: 1674.0, 1: 1685.0. Samples: 24716256. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 15:26:08,164][74987] Avg episode reward: [(0, '24.670'), (1, '31.250')] -[2023-10-14 15:26:08,644][75950] Updated weights for policy 1, policy_version 48200 (0.0010) -[2023-10-14 15:26:09,017][75950] Updated weights for policy 1, policy_version 48210 (0.0007) -[2023-10-14 15:26:09,376][75950] Updated weights for policy 1, policy_version 48220 (0.0009) -[2023-10-14 15:26:09,650][75949] Updated weights for policy 0, policy_version 48321 (0.0009) -[2023-10-14 15:26:10,021][75949] Updated weights for policy 0, policy_version 48331 (0.0008) -[2023-10-14 15:26:10,389][75949] Updated weights for policy 0, policy_version 48341 (0.0008) -[2023-10-14 15:26:10,767][75949] Updated weights for policy 0, policy_version 48351 (0.0009) -[2023-10-14 15:26:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 98893824. Throughput: 0: 1684.6, 1: 1691.5. Samples: 24737152. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 15:26:13,164][74987] Avg episode reward: [(0, '26.580'), (1, '31.770')] -[2023-10-14 15:26:13,433][75950] Updated weights for policy 1, policy_version 48230 (0.0009) -[2023-10-14 15:26:13,805][75950] Updated weights for policy 1, policy_version 48240 (0.0008) -[2023-10-14 15:26:14,174][75950] Updated weights for policy 1, policy_version 48250 (0.0008) -[2023-10-14 15:26:14,949][75949] Updated weights for policy 0, policy_version 48361 (0.0009) -[2023-10-14 15:26:15,320][75949] Updated weights for policy 0, policy_version 48371 (0.0008) -[2023-10-14 15:26:15,698][75949] Updated weights for policy 0, policy_version 48381 (0.0007) -[2023-10-14 15:26:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 98959360. Throughput: 0: 1662.6, 1: 1694.0. Samples: 24746398. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 15:26:18,165][74987] Avg episode reward: [(0, '25.260'), (1, '31.510')] -[2023-10-14 15:26:18,338][75950] Updated weights for policy 1, policy_version 48260 (0.0008) -[2023-10-14 15:26:18,718][75950] Updated weights for policy 1, policy_version 48270 (0.0009) -[2023-10-14 15:26:19,083][75950] Updated weights for policy 1, policy_version 48280 (0.0008) -[2023-10-14 15:26:19,867][75949] Updated weights for policy 0, policy_version 48391 (0.0008) -[2023-10-14 15:26:20,238][75949] Updated weights for policy 0, policy_version 48401 (0.0010) -[2023-10-14 15:26:20,602][75949] Updated weights for policy 0, policy_version 48411 (0.0009) -[2023-10-14 15:26:23,067][75950] Updated weights for policy 1, policy_version 48290 (0.0007) -[2023-10-14 15:26:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 99024896. Throughput: 0: 1684.5, 1: 1691.2. Samples: 24766736. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 15:26:23,164][74987] Avg episode reward: [(0, '24.880'), (1, '30.960')] -[2023-10-14 15:26:23,422][75950] Updated weights for policy 1, policy_version 48300 (0.0008) -[2023-10-14 15:26:23,790][75950] Updated weights for policy 1, policy_version 48310 (0.0009) -[2023-10-14 15:26:24,152][75950] Updated weights for policy 1, policy_version 48320 (0.0007) -[2023-10-14 15:26:24,708][75949] Updated weights for policy 0, policy_version 48421 (0.0009) -[2023-10-14 15:26:25,078][75949] Updated weights for policy 0, policy_version 48431 (0.0008) -[2023-10-14 15:26:25,448][75949] Updated weights for policy 0, policy_version 48441 (0.0008) -[2023-10-14 15:26:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99090432. Throughput: 0: 1684.4, 1: 1687.0. Samples: 24787534. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 15:26:28,164][74987] Avg episode reward: [(0, '25.490'), (1, '29.750')] -[2023-10-14 15:26:28,337][75950] Updated weights for policy 1, policy_version 48330 (0.0008) -[2023-10-14 15:26:28,704][75950] Updated weights for policy 1, policy_version 48340 (0.0008) -[2023-10-14 15:26:29,068][75950] Updated weights for policy 1, policy_version 48350 (0.0010) -[2023-10-14 15:26:29,282][75949] Updated weights for policy 0, policy_version 48451 (0.0008) -[2023-10-14 15:26:29,662][75949] Updated weights for policy 0, policy_version 48461 (0.0008) -[2023-10-14 15:26:30,030][75949] Updated weights for policy 0, policy_version 48471 (0.0007) -[2023-10-14 15:26:33,067][75950] Updated weights for policy 1, policy_version 48360 (0.0008) -[2023-10-14 15:26:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99155968. Throughput: 0: 1671.4, 1: 1687.5. Samples: 24796806. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 15:26:33,165][74987] Avg episode reward: [(0, '26.670'), (1, '30.690')] -[2023-10-14 15:26:33,434][75950] Updated weights for policy 1, policy_version 48370 (0.0010) -[2023-10-14 15:26:33,802][75950] Updated weights for policy 1, policy_version 48380 (0.0011) -[2023-10-14 15:26:34,050][75949] Updated weights for policy 0, policy_version 48481 (0.0007) -[2023-10-14 15:26:34,418][75949] Updated weights for policy 0, policy_version 48491 (0.0007) -[2023-10-14 15:26:34,779][75949] Updated weights for policy 0, policy_version 48501 (0.0007) -[2023-10-14 15:26:35,154][75949] Updated weights for policy 0, policy_version 48511 (0.0010) -[2023-10-14 15:26:37,854][75950] Updated weights for policy 1, policy_version 48390 (0.0009) -[2023-10-14 15:26:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99221504. Throughput: 0: 1690.9, 1: 1685.8. Samples: 24817406. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:26:38,164][74987] Avg episode reward: [(0, '26.440'), (1, '29.430')] -[2023-10-14 15:26:38,224][75950] Updated weights for policy 1, policy_version 48400 (0.0010) -[2023-10-14 15:26:38,604][75950] Updated weights for policy 1, policy_version 48410 (0.0008) -[2023-10-14 15:26:39,205][75949] Updated weights for policy 0, policy_version 48521 (0.0007) -[2023-10-14 15:26:39,567][75949] Updated weights for policy 0, policy_version 48531 (0.0010) -[2023-10-14 15:26:39,948][75949] Updated weights for policy 0, policy_version 48541 (0.0010) -[2023-10-14 15:26:42,818][75950] Updated weights for policy 1, policy_version 48420 (0.0008) -[2023-10-14 15:26:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99287040. Throughput: 0: 1695.2, 1: 1684.1. Samples: 24838234. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:26:43,164][74987] Avg episode reward: [(0, '25.750'), (1, '28.180')] -[2023-10-14 15:26:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000048544_49709056.pth... -[2023-10-14 15:26:43,207][75950] Updated weights for policy 1, policy_version 48430 (0.0008) -[2023-10-14 15:26:43,209][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000046976_48103424.pth -[2023-10-14 15:26:43,574][75950] Updated weights for policy 1, policy_version 48440 (0.0007) -[2023-10-14 15:26:43,863][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000048448_49610752.pth... -[2023-10-14 15:26:43,892][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000046880_48005120.pth -[2023-10-14 15:26:44,014][75949] Updated weights for policy 0, policy_version 48551 (0.0011) -[2023-10-14 15:26:44,409][75949] Updated weights for policy 0, policy_version 48561 (0.0010) -[2023-10-14 15:26:44,788][75949] Updated weights for policy 0, policy_version 48571 (0.0010) -[2023-10-14 15:26:47,632][75950] Updated weights for policy 1, policy_version 48450 (0.0009) -[2023-10-14 15:26:48,008][75950] Updated weights for policy 1, policy_version 48460 (0.0009) -[2023-10-14 15:26:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99352576. Throughput: 0: 1675.4, 1: 1684.0. Samples: 24847152. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:26:48,164][74987] Avg episode reward: [(0, '24.350'), (1, '29.670')] -[2023-10-14 15:26:48,370][75950] Updated weights for policy 1, policy_version 48470 (0.0008) -[2023-10-14 15:26:48,727][75950] Updated weights for policy 1, policy_version 48480 (0.0008) -[2023-10-14 15:26:48,779][75949] Updated weights for policy 0, policy_version 48581 (0.0010) -[2023-10-14 15:26:49,150][75949] Updated weights for policy 0, policy_version 48591 (0.0009) -[2023-10-14 15:26:49,519][75949] Updated weights for policy 0, policy_version 48601 (0.0007) -[2023-10-14 15:26:52,808][75950] Updated weights for policy 1, policy_version 48490 (0.0010) -[2023-10-14 15:26:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 99418112. Throughput: 0: 1690.1, 1: 1680.7. Samples: 24867942. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:26:53,165][74987] Avg episode reward: [(0, '25.240'), (1, '27.400')] -[2023-10-14 15:26:53,183][75950] Updated weights for policy 1, policy_version 48500 (0.0010) -[2023-10-14 15:26:53,560][75950] Updated weights for policy 1, policy_version 48510 (0.0009) -[2023-10-14 15:26:53,673][75949] Updated weights for policy 0, policy_version 48611 (0.0009) -[2023-10-14 15:26:54,035][75949] Updated weights for policy 0, policy_version 48621 (0.0008) -[2023-10-14 15:26:54,403][75949] Updated weights for policy 0, policy_version 48631 (0.0008) -[2023-10-14 15:26:57,385][75950] Updated weights for policy 1, policy_version 48520 (0.0009) -[2023-10-14 15:26:57,754][75950] Updated weights for policy 1, policy_version 48530 (0.0009) -[2023-10-14 15:26:58,132][75950] Updated weights for policy 1, policy_version 48540 (0.0009) -[2023-10-14 15:26:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99483648. Throughput: 0: 1687.5, 1: 1666.4. Samples: 24888078. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:26:58,164][74987] Avg episode reward: [(0, '27.540'), (1, '29.310')] -[2023-10-14 15:26:58,594][75949] Updated weights for policy 0, policy_version 48641 (0.0009) -[2023-10-14 15:26:58,964][75949] Updated weights for policy 0, policy_version 48651 (0.0009) -[2023-10-14 15:26:59,349][75949] Updated weights for policy 0, policy_version 48661 (0.0009) -[2023-10-14 15:26:59,718][75949] Updated weights for policy 0, policy_version 48671 (0.0008) -[2023-10-14 15:27:02,246][75950] Updated weights for policy 1, policy_version 48550 (0.0010) -[2023-10-14 15:27:02,611][75950] Updated weights for policy 1, policy_version 48560 (0.0007) -[2023-10-14 15:27:02,978][75950] Updated weights for policy 1, policy_version 48570 (0.0008) -[2023-10-14 15:27:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99549184. Throughput: 0: 1686.2, 1: 1677.9. Samples: 24897782. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:27:03,164][74987] Avg episode reward: [(0, '26.660'), (1, '31.480')] -[2023-10-14 15:27:03,633][75949] Updated weights for policy 0, policy_version 48681 (0.0007) -[2023-10-14 15:27:04,003][75949] Updated weights for policy 0, policy_version 48691 (0.0008) -[2023-10-14 15:27:04,374][75949] Updated weights for policy 0, policy_version 48701 (0.0008) -[2023-10-14 15:27:07,002][75950] Updated weights for policy 1, policy_version 48580 (0.0009) -[2023-10-14 15:27:07,371][75950] Updated weights for policy 1, policy_version 48590 (0.0008) -[2023-10-14 15:27:07,733][75950] Updated weights for policy 1, policy_version 48600 (0.0009) -[2023-10-14 15:27:08,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 99647488. Throughput: 0: 1696.3, 1: 1679.4. Samples: 24918640. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:27:08,164][74987] Avg episode reward: [(0, '27.380'), (1, '29.260')] -[2023-10-14 15:27:08,536][75949] Updated weights for policy 0, policy_version 48711 (0.0008) -[2023-10-14 15:27:08,913][75949] Updated weights for policy 0, policy_version 48721 (0.0008) -[2023-10-14 15:27:09,293][75949] Updated weights for policy 0, policy_version 48731 (0.0008) -[2023-10-14 15:27:11,830][75950] Updated weights for policy 1, policy_version 48610 (0.0009) -[2023-10-14 15:27:12,188][75950] Updated weights for policy 1, policy_version 48620 (0.0007) -[2023-10-14 15:27:12,556][75950] Updated weights for policy 1, policy_version 48630 (0.0007) -[2023-10-14 15:27:12,924][75950] Updated weights for policy 1, policy_version 48640 (0.0008) -[2023-10-14 15:27:13,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 99713024. Throughput: 0: 1692.7, 1: 1659.5. Samples: 24938388. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 15:27:13,165][74987] Avg episode reward: [(0, '26.070'), (1, '32.100')] -[2023-10-14 15:27:13,235][75949] Updated weights for policy 0, policy_version 48741 (0.0011) -[2023-10-14 15:27:13,620][75949] Updated weights for policy 0, policy_version 48751 (0.0007) -[2023-10-14 15:27:13,980][75949] Updated weights for policy 0, policy_version 48761 (0.0010) -[2023-10-14 15:27:17,132][75950] Updated weights for policy 1, policy_version 48650 (0.0009) -[2023-10-14 15:27:17,491][75950] Updated weights for policy 1, policy_version 48660 (0.0008) -[2023-10-14 15:27:17,862][75950] Updated weights for policy 1, policy_version 48670 (0.0010) -[2023-10-14 15:27:17,930][75949] Updated weights for policy 0, policy_version 48771 (0.0008) -[2023-10-14 15:27:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 99778560. Throughput: 0: 1691.5, 1: 1679.1. Samples: 24948482. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 15:27:18,165][74987] Avg episode reward: [(0, '24.500'), (1, '31.900')] -[2023-10-14 15:27:18,312][75949] Updated weights for policy 0, policy_version 48781 (0.0010) -[2023-10-14 15:27:18,678][75949] Updated weights for policy 0, policy_version 48791 (0.0008) -[2023-10-14 15:27:21,900][75950] Updated weights for policy 1, policy_version 48680 (0.0009) -[2023-10-14 15:27:22,277][75950] Updated weights for policy 1, policy_version 48690 (0.0009) -[2023-10-14 15:27:22,634][75950] Updated weights for policy 1, policy_version 48700 (0.0007) -[2023-10-14 15:27:22,741][75949] Updated weights for policy 0, policy_version 48801 (0.0009) -[2023-10-14 15:27:23,117][75949] Updated weights for policy 0, policy_version 48811 (0.0008) -[2023-10-14 15:27:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 99844096. Throughput: 0: 1692.4, 1: 1682.9. Samples: 24969296. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 15:27:23,164][74987] Avg episode reward: [(0, '25.890'), (1, '31.090')] -[2023-10-14 15:27:23,495][75949] Updated weights for policy 0, policy_version 48821 (0.0009) -[2023-10-14 15:27:23,865][75949] Updated weights for policy 0, policy_version 48831 (0.0010) -[2023-10-14 15:27:26,882][75950] Updated weights for policy 1, policy_version 48710 (0.0009) -[2023-10-14 15:27:27,255][75950] Updated weights for policy 1, policy_version 48720 (0.0009) -[2023-10-14 15:27:27,619][75950] Updated weights for policy 1, policy_version 48730 (0.0008) -[2023-10-14 15:27:28,061][75949] Updated weights for policy 0, policy_version 48841 (0.0009) -[2023-10-14 15:27:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 99909632. Throughput: 0: 1685.4, 1: 1658.0. Samples: 24988688. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 15:27:28,164][74987] Avg episode reward: [(0, '24.610'), (1, '31.550')] -[2023-10-14 15:27:28,426][75949] Updated weights for policy 0, policy_version 48851 (0.0010) -[2023-10-14 15:27:28,799][75949] Updated weights for policy 0, policy_version 48861 (0.0007) -[2023-10-14 15:27:31,746][75950] Updated weights for policy 1, policy_version 48740 (0.0008) -[2023-10-14 15:27:32,147][75950] Updated weights for policy 1, policy_version 48750 (0.0007) -[2023-10-14 15:27:32,509][75950] Updated weights for policy 1, policy_version 48760 (0.0007) -[2023-10-14 15:27:32,757][75949] Updated weights for policy 0, policy_version 48871 (0.0008) -[2023-10-14 15:27:33,138][75949] Updated weights for policy 0, policy_version 48881 (0.0008) -[2023-10-14 15:27:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 99975168. Throughput: 0: 1688.6, 1: 1682.3. Samples: 24998840. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 15:27:33,164][74987] Avg episode reward: [(0, '25.860'), (1, '33.290')] -[2023-10-14 15:27:33,165][75801] Saving new best policy, reward=33.290! -[2023-10-14 15:27:33,518][75949] Updated weights for policy 0, policy_version 48891 (0.0007) -[2023-10-14 15:27:36,411][75950] Updated weights for policy 1, policy_version 48770 (0.0008) -[2023-10-14 15:27:36,783][75950] Updated weights for policy 1, policy_version 48780 (0.0007) -[2023-10-14 15:27:37,150][75950] Updated weights for policy 1, policy_version 48790 (0.0007) -[2023-10-14 15:27:37,511][75950] Updated weights for policy 1, policy_version 48800 (0.0008) -[2023-10-14 15:27:37,597][75949] Updated weights for policy 0, policy_version 48901 (0.0009) -[2023-10-14 15:27:37,959][75949] Updated weights for policy 0, policy_version 48911 (0.0008) -[2023-10-14 15:27:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100040704. Throughput: 0: 1684.2, 1: 1674.5. Samples: 25019084. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 15:27:38,165][74987] Avg episode reward: [(0, '24.740'), (1, '30.120')] -[2023-10-14 15:27:38,325][75949] Updated weights for policy 0, policy_version 48921 (0.0009) -[2023-10-14 15:27:41,722][75950] Updated weights for policy 1, policy_version 48810 (0.0010) -[2023-10-14 15:27:42,094][75950] Updated weights for policy 1, policy_version 48820 (0.0011) -[2023-10-14 15:27:42,435][75949] Updated weights for policy 0, policy_version 48931 (0.0009) -[2023-10-14 15:27:42,466][75950] Updated weights for policy 1, policy_version 48830 (0.0010) -[2023-10-14 15:27:42,807][75949] Updated weights for policy 0, policy_version 48941 (0.0007) -[2023-10-14 15:27:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100106240. Throughput: 0: 1676.6, 1: 1661.1. Samples: 25038278. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 15:27:43,165][74987] Avg episode reward: [(0, '26.230'), (1, '29.630')] -[2023-10-14 15:27:43,173][75949] Updated weights for policy 0, policy_version 48951 (0.0008) -[2023-10-14 15:27:46,429][75950] Updated weights for policy 1, policy_version 48840 (0.0010) -[2023-10-14 15:27:46,790][75950] Updated weights for policy 1, policy_version 48850 (0.0011) -[2023-10-14 15:27:47,159][75950] Updated weights for policy 1, policy_version 48860 (0.0011) -[2023-10-14 15:27:47,302][75949] Updated weights for policy 0, policy_version 48961 (0.0010) -[2023-10-14 15:27:47,659][75949] Updated weights for policy 0, policy_version 48971 (0.0010) -[2023-10-14 15:27:48,028][75949] Updated weights for policy 0, policy_version 48981 (0.0010) -[2023-10-14 15:27:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 100171776. Throughput: 0: 1679.8, 1: 1679.1. Samples: 25048932. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:27:48,164][74987] Avg episode reward: [(0, '24.540'), (1, '29.420')] -[2023-10-14 15:27:48,402][75949] Updated weights for policy 0, policy_version 48991 (0.0009) -[2023-10-14 15:27:51,148][75950] Updated weights for policy 1, policy_version 48870 (0.0009) -[2023-10-14 15:27:51,521][75950] Updated weights for policy 1, policy_version 48880 (0.0008) -[2023-10-14 15:27:51,876][75950] Updated weights for policy 1, policy_version 48890 (0.0009) -[2023-10-14 15:27:52,544][75949] Updated weights for policy 0, policy_version 49001 (0.0009) -[2023-10-14 15:27:52,905][75949] Updated weights for policy 0, policy_version 49011 (0.0008) -[2023-10-14 15:27:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 100237312. Throughput: 0: 1676.1, 1: 1666.0. Samples: 25069036. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:27:53,164][74987] Avg episode reward: [(0, '26.080'), (1, '26.540')] -[2023-10-14 15:27:53,278][75949] Updated weights for policy 0, policy_version 49021 (0.0010) -[2023-10-14 15:27:55,970][75950] Updated weights for policy 1, policy_version 48900 (0.0007) -[2023-10-14 15:27:56,334][75950] Updated weights for policy 1, policy_version 48910 (0.0009) -[2023-10-14 15:27:56,695][75950] Updated weights for policy 1, policy_version 48920 (0.0010) -[2023-10-14 15:27:57,453][75949] Updated weights for policy 0, policy_version 49031 (0.0010) -[2023-10-14 15:27:57,824][75949] Updated weights for policy 0, policy_version 49041 (0.0010) -[2023-10-14 15:27:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100302848. Throughput: 0: 1666.7, 1: 1677.2. Samples: 25088862. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:27:58,165][74987] Avg episode reward: [(0, '24.380'), (1, '26.470')] -[2023-10-14 15:27:58,194][75949] Updated weights for policy 0, policy_version 49051 (0.0007) -[2023-10-14 15:28:00,777][75950] Updated weights for policy 1, policy_version 48930 (0.0009) -[2023-10-14 15:28:01,142][75950] Updated weights for policy 1, policy_version 48940 (0.0010) -[2023-10-14 15:28:01,512][75950] Updated weights for policy 1, policy_version 48950 (0.0009) -[2023-10-14 15:28:01,872][75950] Updated weights for policy 1, policy_version 48960 (0.0008) -[2023-10-14 15:28:02,162][75949] Updated weights for policy 0, policy_version 49061 (0.0010) -[2023-10-14 15:28:02,539][75949] Updated weights for policy 0, policy_version 49071 (0.0011) -[2023-10-14 15:28:02,916][75949] Updated weights for policy 0, policy_version 49081 (0.0012) -[2023-10-14 15:28:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100368384. Throughput: 0: 1677.7, 1: 1679.3. Samples: 25099544. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:28:03,164][74987] Avg episode reward: [(0, '26.150'), (1, '28.720')] -[2023-10-14 15:28:06,080][75950] Updated weights for policy 1, policy_version 48970 (0.0007) -[2023-10-14 15:28:06,452][75950] Updated weights for policy 1, policy_version 48980 (0.0009) -[2023-10-14 15:28:06,827][75950] Updated weights for policy 1, policy_version 48990 (0.0008) -[2023-10-14 15:28:07,063][75949] Updated weights for policy 0, policy_version 49091 (0.0010) -[2023-10-14 15:28:07,436][75949] Updated weights for policy 0, policy_version 49101 (0.0010) -[2023-10-14 15:28:07,812][75949] Updated weights for policy 0, policy_version 49111 (0.0011) -[2023-10-14 15:28:08,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100466688. Throughput: 0: 1673.0, 1: 1658.1. Samples: 25119196. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:28:08,165][74987] Avg episode reward: [(0, '26.110'), (1, '28.990')] -[2023-10-14 15:28:11,014][75950] Updated weights for policy 1, policy_version 49000 (0.0007) -[2023-10-14 15:28:11,377][75950] Updated weights for policy 1, policy_version 49010 (0.0008) -[2023-10-14 15:28:11,739][75950] Updated weights for policy 1, policy_version 49020 (0.0009) -[2023-10-14 15:28:11,934][75949] Updated weights for policy 0, policy_version 49121 (0.0011) -[2023-10-14 15:28:12,300][75949] Updated weights for policy 0, policy_version 49131 (0.0007) -[2023-10-14 15:28:12,667][75949] Updated weights for policy 0, policy_version 49141 (0.0010) -[2023-10-14 15:28:13,036][75949] Updated weights for policy 0, policy_version 49151 (0.0011) -[2023-10-14 15:28:13,164][74987] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100532224. Throughput: 0: 1656.8, 1: 1674.0. Samples: 25138578. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 15:28:13,165][74987] Avg episode reward: [(0, '25.560'), (1, '28.870')] -[2023-10-14 15:28:15,985][75950] Updated weights for policy 1, policy_version 49030 (0.0008) -[2023-10-14 15:28:16,358][75950] Updated weights for policy 1, policy_version 49040 (0.0009) -[2023-10-14 15:28:16,723][75950] Updated weights for policy 1, policy_version 49050 (0.0009) -[2023-10-14 15:28:17,223][75949] Updated weights for policy 0, policy_version 49161 (0.0007) -[2023-10-14 15:28:17,586][75949] Updated weights for policy 0, policy_version 49171 (0.0009) -[2023-10-14 15:28:17,952][75949] Updated weights for policy 0, policy_version 49181 (0.0010) -[2023-10-14 15:28:18,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 100597760. Throughput: 0: 1672.5, 1: 1676.8. Samples: 25149558. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:28:18,164][74987] Avg episode reward: [(0, '26.170'), (1, '31.120')] -[2023-10-14 15:28:20,907][75950] Updated weights for policy 1, policy_version 49060 (0.0008) -[2023-10-14 15:28:21,322][75950] Updated weights for policy 1, policy_version 49070 (0.0009) -[2023-10-14 15:28:21,684][75950] Updated weights for policy 1, policy_version 49080 (0.0008) -[2023-10-14 15:28:21,947][75949] Updated weights for policy 0, policy_version 49191 (0.0010) -[2023-10-14 15:28:22,319][75949] Updated weights for policy 0, policy_version 49201 (0.0009) -[2023-10-14 15:28:22,701][75949] Updated weights for policy 0, policy_version 49211 (0.0007) -[2023-10-14 15:28:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100663296. Throughput: 0: 1674.8, 1: 1659.0. Samples: 25169104. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:28:23,165][74987] Avg episode reward: [(0, '25.870'), (1, '28.550')] -[2023-10-14 15:28:25,687][75950] Updated weights for policy 1, policy_version 49090 (0.0010) -[2023-10-14 15:28:26,051][75950] Updated weights for policy 1, policy_version 49100 (0.0009) -[2023-10-14 15:28:26,421][75950] Updated weights for policy 1, policy_version 49110 (0.0010) -[2023-10-14 15:28:26,670][75949] Updated weights for policy 0, policy_version 49221 (0.0008) -[2023-10-14 15:28:26,784][75950] Updated weights for policy 1, policy_version 49120 (0.0010) -[2023-10-14 15:28:27,039][75949] Updated weights for policy 0, policy_version 49231 (0.0011) -[2023-10-14 15:28:27,402][75949] Updated weights for policy 0, policy_version 49241 (0.0009) -[2023-10-14 15:28:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100728832. Throughput: 0: 1657.6, 1: 1675.5. Samples: 25188268. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:28:28,165][74987] Avg episode reward: [(0, '26.490'), (1, '27.500')] -[2023-10-14 15:28:30,679][75950] Updated weights for policy 1, policy_version 49130 (0.0009) -[2023-10-14 15:28:31,047][75950] Updated weights for policy 1, policy_version 49140 (0.0008) -[2023-10-14 15:28:31,408][75950] Updated weights for policy 1, policy_version 49150 (0.0009) -[2023-10-14 15:28:31,416][75949] Updated weights for policy 0, policy_version 49251 (0.0007) -[2023-10-14 15:28:31,783][75949] Updated weights for policy 0, policy_version 49261 (0.0007) -[2023-10-14 15:28:32,144][75949] Updated weights for policy 0, policy_version 49271 (0.0007) -[2023-10-14 15:28:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100794368. Throughput: 0: 1680.6, 1: 1670.0. Samples: 25199712. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:28:33,165][74987] Avg episode reward: [(0, '26.670'), (1, '30.330')] -[2023-10-14 15:28:35,238][75950] Updated weights for policy 1, policy_version 49160 (0.0009) -[2023-10-14 15:28:35,613][75950] Updated weights for policy 1, policy_version 49170 (0.0007) -[2023-10-14 15:28:35,969][75950] Updated weights for policy 1, policy_version 49180 (0.0007) -[2023-10-14 15:28:36,143][75949] Updated weights for policy 0, policy_version 49281 (0.0008) -[2023-10-14 15:28:36,510][75949] Updated weights for policy 0, policy_version 49291 (0.0011) -[2023-10-14 15:28:36,889][75949] Updated weights for policy 0, policy_version 49301 (0.0007) -[2023-10-14 15:28:37,255][75949] Updated weights for policy 0, policy_version 49311 (0.0009) -[2023-10-14 15:28:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 100859904. Throughput: 0: 1671.1, 1: 1663.3. Samples: 25219082. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:28:38,164][74987] Avg episode reward: [(0, '26.660'), (1, '29.760')] -[2023-10-14 15:28:40,236][75950] Updated weights for policy 1, policy_version 49190 (0.0009) -[2023-10-14 15:28:40,602][75950] Updated weights for policy 1, policy_version 49200 (0.0009) -[2023-10-14 15:28:40,963][75950] Updated weights for policy 1, policy_version 49210 (0.0007) -[2023-10-14 15:28:41,288][75949] Updated weights for policy 0, policy_version 49321 (0.0013) -[2023-10-14 15:28:41,656][75949] Updated weights for policy 0, policy_version 49331 (0.0008) -[2023-10-14 15:28:42,027][75949] Updated weights for policy 0, policy_version 49341 (0.0010) -[2023-10-14 15:28:43,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 100925440. Throughput: 0: 1669.1, 1: 1672.3. Samples: 25239224. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:28:43,164][74987] Avg episode reward: [(0, '25.510'), (1, '29.650')] -[2023-10-14 15:28:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000049216_50397184.pth... -[2023-10-14 15:28:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000049344_50528256.pth... -[2023-10-14 15:28:43,214][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000047648_48791552.pth -[2023-10-14 15:28:43,215][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000047776_48922624.pth -[2023-10-14 15:28:45,019][75950] Updated weights for policy 1, policy_version 49220 (0.0008) -[2023-10-14 15:28:45,387][75950] Updated weights for policy 1, policy_version 49230 (0.0007) -[2023-10-14 15:28:45,749][75950] Updated weights for policy 1, policy_version 49240 (0.0009) -[2023-10-14 15:28:46,042][75949] Updated weights for policy 0, policy_version 49351 (0.0009) -[2023-10-14 15:28:46,415][75949] Updated weights for policy 0, policy_version 49361 (0.0009) -[2023-10-14 15:28:46,787][75949] Updated weights for policy 0, policy_version 49371 (0.0007) -[2023-10-14 15:28:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100990976. Throughput: 0: 1688.7, 1: 1661.3. Samples: 25250296. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:28:48,164][74987] Avg episode reward: [(0, '25.800'), (1, '30.390')] -[2023-10-14 15:28:49,892][75950] Updated weights for policy 1, policy_version 49250 (0.0009) -[2023-10-14 15:28:50,261][75950] Updated weights for policy 1, policy_version 49260 (0.0009) -[2023-10-14 15:28:50,629][75950] Updated weights for policy 1, policy_version 49270 (0.0007) -[2023-10-14 15:28:50,855][75949] Updated weights for policy 0, policy_version 49381 (0.0009) -[2023-10-14 15:28:51,001][75950] Updated weights for policy 1, policy_version 49280 (0.0008) -[2023-10-14 15:28:51,217][75949] Updated weights for policy 0, policy_version 49391 (0.0008) -[2023-10-14 15:28:51,589][75949] Updated weights for policy 0, policy_version 49401 (0.0008) -[2023-10-14 15:28:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 101056512. Throughput: 0: 1667.4, 1: 1668.7. Samples: 25269320. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 15:28:53,165][74987] Avg episode reward: [(0, '26.010'), (1, '31.760')] -[2023-10-14 15:28:55,196][75950] Updated weights for policy 1, policy_version 49290 (0.0007) -[2023-10-14 15:28:55,569][75950] Updated weights for policy 1, policy_version 49300 (0.0008) -[2023-10-14 15:28:55,638][75949] Updated weights for policy 0, policy_version 49411 (0.0008) -[2023-10-14 15:28:55,939][75950] Updated weights for policy 1, policy_version 49310 (0.0008) -[2023-10-14 15:28:56,013][75949] Updated weights for policy 0, policy_version 49421 (0.0008) -[2023-10-14 15:28:56,377][75949] Updated weights for policy 0, policy_version 49431 (0.0009) -[2023-10-14 15:28:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 101122048. Throughput: 0: 1681.4, 1: 1680.3. Samples: 25289854. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 15:28:58,165][74987] Avg episode reward: [(0, '27.620'), (1, '30.750')] -[2023-10-14 15:28:59,973][75950] Updated weights for policy 1, policy_version 49320 (0.0011) -[2023-10-14 15:29:00,330][75950] Updated weights for policy 1, policy_version 49330 (0.0011) -[2023-10-14 15:29:00,545][75949] Updated weights for policy 0, policy_version 49441 (0.0009) -[2023-10-14 15:29:00,694][75950] Updated weights for policy 1, policy_version 49340 (0.0009) -[2023-10-14 15:29:00,911][75949] Updated weights for policy 0, policy_version 49451 (0.0008) -[2023-10-14 15:29:01,282][75949] Updated weights for policy 0, policy_version 49461 (0.0008) -[2023-10-14 15:29:01,657][75949] Updated weights for policy 0, policy_version 49471 (0.0008) -[2023-10-14 15:29:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 101187584. Throughput: 0: 1689.1, 1: 1656.5. Samples: 25300110. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 15:29:03,164][74987] Avg episode reward: [(0, '26.490'), (1, '28.930')] -[2023-10-14 15:29:04,748][75950] Updated weights for policy 1, policy_version 49350 (0.0009) -[2023-10-14 15:29:05,114][75950] Updated weights for policy 1, policy_version 49360 (0.0009) -[2023-10-14 15:29:05,481][75950] Updated weights for policy 1, policy_version 49370 (0.0009) -[2023-10-14 15:29:05,792][75949] Updated weights for policy 0, policy_version 49481 (0.0007) -[2023-10-14 15:29:06,159][75949] Updated weights for policy 0, policy_version 49491 (0.0008) -[2023-10-14 15:29:06,538][75949] Updated weights for policy 0, policy_version 49501 (0.0008) -[2023-10-14 15:29:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 101253120. Throughput: 0: 1668.8, 1: 1674.7. Samples: 25319560. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 15:29:08,164][74987] Avg episode reward: [(0, '26.260'), (1, '29.970')] -[2023-10-14 15:29:09,682][75950] Updated weights for policy 1, policy_version 49380 (0.0008) -[2023-10-14 15:29:10,077][75950] Updated weights for policy 1, policy_version 49390 (0.0008) -[2023-10-14 15:29:10,449][75950] Updated weights for policy 1, policy_version 49400 (0.0008) -[2023-10-14 15:29:10,630][75949] Updated weights for policy 0, policy_version 49511 (0.0007) -[2023-10-14 15:29:11,005][75949] Updated weights for policy 0, policy_version 49521 (0.0009) -[2023-10-14 15:29:11,370][75949] Updated weights for policy 0, policy_version 49531 (0.0011) -[2023-10-14 15:29:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 101318656. Throughput: 0: 1690.2, 1: 1675.5. Samples: 25339724. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 15:29:13,165][74987] Avg episode reward: [(0, '26.230'), (1, '27.950')] -[2023-10-14 15:29:14,632][75950] Updated weights for policy 1, policy_version 49410 (0.0009) -[2023-10-14 15:29:15,001][75950] Updated weights for policy 1, policy_version 49420 (0.0007) -[2023-10-14 15:29:15,373][75950] Updated weights for policy 1, policy_version 49430 (0.0010) -[2023-10-14 15:29:15,454][75949] Updated weights for policy 0, policy_version 49541 (0.0011) -[2023-10-14 15:29:15,732][75950] Updated weights for policy 1, policy_version 49440 (0.0007) -[2023-10-14 15:29:15,820][75949] Updated weights for policy 0, policy_version 49551 (0.0009) -[2023-10-14 15:29:16,195][75949] Updated weights for policy 0, policy_version 49561 (0.0010) -[2023-10-14 15:29:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 101384192. Throughput: 0: 1680.8, 1: 1653.6. Samples: 25349758. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 15:29:18,164][74987] Avg episode reward: [(0, '26.210'), (1, '27.620')] -[2023-10-14 15:29:19,829][75950] Updated weights for policy 1, policy_version 49450 (0.0007) -[2023-10-14 15:29:20,187][75950] Updated weights for policy 1, policy_version 49460 (0.0009) -[2023-10-14 15:29:20,231][75949] Updated weights for policy 0, policy_version 49571 (0.0008) -[2023-10-14 15:29:20,553][75950] Updated weights for policy 1, policy_version 49470 (0.0008) -[2023-10-14 15:29:20,596][75949] Updated weights for policy 0, policy_version 49581 (0.0009) -[2023-10-14 15:29:20,973][75949] Updated weights for policy 0, policy_version 49591 (0.0010) -[2023-10-14 15:29:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 101449728. Throughput: 0: 1670.3, 1: 1668.4. Samples: 25369324. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 15:29:23,164][74987] Avg episode reward: [(0, '25.040'), (1, '28.510')] -[2023-10-14 15:29:24,700][75950] Updated weights for policy 1, policy_version 49480 (0.0009) -[2023-10-14 15:29:25,072][75950] Updated weights for policy 1, policy_version 49490 (0.0011) -[2023-10-14 15:29:25,097][75949] Updated weights for policy 0, policy_version 49601 (0.0009) -[2023-10-14 15:29:25,444][75950] Updated weights for policy 1, policy_version 49500 (0.0010) -[2023-10-14 15:29:25,474][75949] Updated weights for policy 0, policy_version 49611 (0.0009) -[2023-10-14 15:29:25,838][75949] Updated weights for policy 0, policy_version 49621 (0.0009) -[2023-10-14 15:29:26,222][75949] Updated weights for policy 0, policy_version 49631 (0.0010) -[2023-10-14 15:29:28,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 101515264. Throughput: 0: 1678.9, 1: 1663.6. Samples: 25389636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:29:28,165][74987] Avg episode reward: [(0, '27.880'), (1, '29.990')] -[2023-10-14 15:29:29,698][75950] Updated weights for policy 1, policy_version 49510 (0.0008) -[2023-10-14 15:29:30,070][75950] Updated weights for policy 1, policy_version 49520 (0.0008) -[2023-10-14 15:29:30,297][75949] Updated weights for policy 0, policy_version 49641 (0.0008) -[2023-10-14 15:29:30,446][75950] Updated weights for policy 1, policy_version 49530 (0.0008) -[2023-10-14 15:29:30,671][75949] Updated weights for policy 0, policy_version 49651 (0.0007) -[2023-10-14 15:29:31,044][75949] Updated weights for policy 0, policy_version 49661 (0.0008) -[2023-10-14 15:29:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 101580800. Throughput: 0: 1657.8, 1: 1652.0. Samples: 25399234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:29:33,164][74987] Avg episode reward: [(0, '24.330'), (1, '29.450')] -[2023-10-14 15:29:34,559][75950] Updated weights for policy 1, policy_version 49540 (0.0010) -[2023-10-14 15:29:34,933][75950] Updated weights for policy 1, policy_version 49550 (0.0011) -[2023-10-14 15:29:35,129][75949] Updated weights for policy 0, policy_version 49671 (0.0008) -[2023-10-14 15:29:35,287][75950] Updated weights for policy 1, policy_version 49560 (0.0009) -[2023-10-14 15:29:35,507][75949] Updated weights for policy 0, policy_version 49681 (0.0008) -[2023-10-14 15:29:35,864][75949] Updated weights for policy 0, policy_version 49691 (0.0011) -[2023-10-14 15:29:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 101646336. Throughput: 0: 1666.3, 1: 1665.1. Samples: 25419236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:29:38,165][74987] Avg episode reward: [(0, '25.480'), (1, '29.420')] -[2023-10-14 15:29:39,373][75950] Updated weights for policy 1, policy_version 49570 (0.0008) -[2023-10-14 15:29:39,731][75950] Updated weights for policy 1, policy_version 49580 (0.0008) -[2023-10-14 15:29:40,081][75949] Updated weights for policy 0, policy_version 49701 (0.0009) -[2023-10-14 15:29:40,102][75950] Updated weights for policy 1, policy_version 49590 (0.0007) -[2023-10-14 15:29:40,452][75949] Updated weights for policy 0, policy_version 49711 (0.0009) -[2023-10-14 15:29:40,463][75950] Updated weights for policy 1, policy_version 49600 (0.0008) -[2023-10-14 15:29:40,823][75949] Updated weights for policy 0, policy_version 49721 (0.0009) -[2023-10-14 15:29:43,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 101711872. Throughput: 0: 1665.0, 1: 1660.0. Samples: 25439480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:29:43,164][74987] Avg episode reward: [(0, '25.600'), (1, '29.590')] -[2023-10-14 15:29:44,580][75950] Updated weights for policy 1, policy_version 49610 (0.0009) -[2023-10-14 15:29:44,950][75950] Updated weights for policy 1, policy_version 49620 (0.0007) -[2023-10-14 15:29:44,987][75949] Updated weights for policy 0, policy_version 49731 (0.0009) -[2023-10-14 15:29:45,322][75950] Updated weights for policy 1, policy_version 49630 (0.0009) -[2023-10-14 15:29:45,353][75949] Updated weights for policy 0, policy_version 49741 (0.0008) -[2023-10-14 15:29:45,727][75949] Updated weights for policy 0, policy_version 49751 (0.0009) -[2023-10-14 15:29:48,163][74987] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 101777408. Throughput: 0: 1654.8, 1: 1656.2. Samples: 25449108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:29:48,164][74987] Avg episode reward: [(0, '26.000'), (1, '27.180')] -[2023-10-14 15:29:49,380][75950] Updated weights for policy 1, policy_version 49640 (0.0009) -[2023-10-14 15:29:49,750][75950] Updated weights for policy 1, policy_version 49650 (0.0010) -[2023-10-14 15:29:49,830][75949] Updated weights for policy 0, policy_version 49761 (0.0012) -[2023-10-14 15:29:50,115][75950] Updated weights for policy 1, policy_version 49660 (0.0011) -[2023-10-14 15:29:50,201][75949] Updated weights for policy 0, policy_version 49771 (0.0009) -[2023-10-14 15:29:50,568][75949] Updated weights for policy 0, policy_version 49781 (0.0009) -[2023-10-14 15:29:50,951][75949] Updated weights for policy 0, policy_version 49791 (0.0008) -[2023-10-14 15:29:53,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 101842944. Throughput: 0: 1663.3, 1: 1657.9. Samples: 25469018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:29:53,165][74987] Avg episode reward: [(0, '26.240'), (1, '28.350')] -[2023-10-14 15:29:54,379][75950] Updated weights for policy 1, policy_version 49670 (0.0010) -[2023-10-14 15:29:54,751][75950] Updated weights for policy 1, policy_version 49680 (0.0009) -[2023-10-14 15:29:55,007][75949] Updated weights for policy 0, policy_version 49801 (0.0008) -[2023-10-14 15:29:55,119][75950] Updated weights for policy 1, policy_version 49690 (0.0008) -[2023-10-14 15:29:55,370][75949] Updated weights for policy 0, policy_version 49811 (0.0010) -[2023-10-14 15:29:55,735][75949] Updated weights for policy 0, policy_version 49821 (0.0011) -[2023-10-14 15:29:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 101908480. Throughput: 0: 1664.1, 1: 1661.2. Samples: 25489362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:29:58,165][74987] Avg episode reward: [(0, '26.610'), (1, '28.780')] -[2023-10-14 15:29:59,266][75950] Updated weights for policy 1, policy_version 49700 (0.0007) -[2023-10-14 15:29:59,663][75950] Updated weights for policy 1, policy_version 49710 (0.0008) -[2023-10-14 15:29:59,942][75949] Updated weights for policy 0, policy_version 49831 (0.0010) -[2023-10-14 15:30:00,025][75950] Updated weights for policy 1, policy_version 49720 (0.0009) -[2023-10-14 15:30:00,316][75949] Updated weights for policy 0, policy_version 49841 (0.0008) -[2023-10-14 15:30:00,692][75949] Updated weights for policy 0, policy_version 49851 (0.0008) -[2023-10-14 15:30:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 101974016. Throughput: 0: 1648.2, 1: 1657.6. Samples: 25498520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:30:03,165][74987] Avg episode reward: [(0, '25.970'), (1, '28.510')] -[2023-10-14 15:30:04,029][75950] Updated weights for policy 1, policy_version 49730 (0.0008) -[2023-10-14 15:30:04,398][75950] Updated weights for policy 1, policy_version 49740 (0.0008) -[2023-10-14 15:30:04,756][75950] Updated weights for policy 1, policy_version 49750 (0.0008) -[2023-10-14 15:30:04,798][75949] Updated weights for policy 0, policy_version 49861 (0.0009) -[2023-10-14 15:30:05,124][75950] Updated weights for policy 1, policy_version 49760 (0.0009) -[2023-10-14 15:30:05,163][75949] Updated weights for policy 0, policy_version 49871 (0.0008) -[2023-10-14 15:30:05,531][75949] Updated weights for policy 0, policy_version 49881 (0.0007) -[2023-10-14 15:30:08,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102039552. Throughput: 0: 1653.2, 1: 1659.3. Samples: 25518384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:30:08,164][74987] Avg episode reward: [(0, '25.980'), (1, '28.590')] -[2023-10-14 15:30:09,131][75950] Updated weights for policy 1, policy_version 49770 (0.0010) -[2023-10-14 15:30:09,510][75950] Updated weights for policy 1, policy_version 49780 (0.0009) -[2023-10-14 15:30:09,600][75949] Updated weights for policy 0, policy_version 49891 (0.0008) -[2023-10-14 15:30:09,877][75950] Updated weights for policy 1, policy_version 49790 (0.0009) -[2023-10-14 15:30:09,964][75949] Updated weights for policy 0, policy_version 49901 (0.0008) -[2023-10-14 15:30:10,336][75949] Updated weights for policy 0, policy_version 49911 (0.0008) -[2023-10-14 15:30:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102105088. Throughput: 0: 1657.9, 1: 1656.5. Samples: 25538784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:30:13,165][74987] Avg episode reward: [(0, '26.930'), (1, '30.100')] -[2023-10-14 15:30:14,018][75950] Updated weights for policy 1, policy_version 49800 (0.0010) -[2023-10-14 15:30:14,338][75949] Updated weights for policy 0, policy_version 49921 (0.0008) -[2023-10-14 15:30:14,377][75950] Updated weights for policy 1, policy_version 49810 (0.0009) -[2023-10-14 15:30:14,706][75949] Updated weights for policy 0, policy_version 49931 (0.0008) -[2023-10-14 15:30:14,749][75950] Updated weights for policy 1, policy_version 49820 (0.0009) -[2023-10-14 15:30:15,066][75949] Updated weights for policy 0, policy_version 49941 (0.0010) -[2023-10-14 15:30:15,434][75949] Updated weights for policy 0, policy_version 49951 (0.0010) -[2023-10-14 15:30:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102170624. Throughput: 0: 1644.1, 1: 1660.2. Samples: 25547928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:30:18,164][74987] Avg episode reward: [(0, '25.500'), (1, '30.680')] -[2023-10-14 15:30:18,883][75950] Updated weights for policy 1, policy_version 49830 (0.0008) -[2023-10-14 15:30:19,252][75950] Updated weights for policy 1, policy_version 49840 (0.0009) -[2023-10-14 15:30:19,627][75950] Updated weights for policy 1, policy_version 49850 (0.0008) -[2023-10-14 15:30:19,658][75949] Updated weights for policy 0, policy_version 49961 (0.0009) -[2023-10-14 15:30:20,013][75949] Updated weights for policy 0, policy_version 49971 (0.0009) -[2023-10-14 15:30:20,390][75949] Updated weights for policy 0, policy_version 49981 (0.0008) -[2023-10-14 15:30:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102236160. Throughput: 0: 1657.9, 1: 1659.0. Samples: 25568494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:30:23,165][74987] Avg episode reward: [(0, '26.620'), (1, '30.150')] -[2023-10-14 15:30:23,840][75950] Updated weights for policy 1, policy_version 49860 (0.0007) -[2023-10-14 15:30:24,207][75950] Updated weights for policy 1, policy_version 49870 (0.0009) -[2023-10-14 15:30:24,453][75949] Updated weights for policy 0, policy_version 49991 (0.0008) -[2023-10-14 15:30:24,561][75950] Updated weights for policy 1, policy_version 49880 (0.0007) -[2023-10-14 15:30:24,827][75949] Updated weights for policy 0, policy_version 50001 (0.0007) -[2023-10-14 15:30:25,198][75949] Updated weights for policy 0, policy_version 50011 (0.0007) -[2023-10-14 15:30:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102301696. Throughput: 0: 1663.1, 1: 1665.2. Samples: 25589256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:30:28,165][74987] Avg episode reward: [(0, '25.830'), (1, '33.060')] -[2023-10-14 15:30:28,417][75950] Updated weights for policy 1, policy_version 49890 (0.0008) -[2023-10-14 15:30:28,780][75950] Updated weights for policy 1, policy_version 49900 (0.0010) -[2023-10-14 15:30:29,153][75950] Updated weights for policy 1, policy_version 49910 (0.0011) -[2023-10-14 15:30:29,379][75949] Updated weights for policy 0, policy_version 50021 (0.0008) -[2023-10-14 15:30:29,509][75950] Updated weights for policy 1, policy_version 49920 (0.0008) -[2023-10-14 15:30:29,741][75949] Updated weights for policy 0, policy_version 50031 (0.0009) -[2023-10-14 15:30:30,116][75949] Updated weights for policy 0, policy_version 50041 (0.0008) -[2023-10-14 15:30:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102367232. Throughput: 0: 1651.1, 1: 1667.4. Samples: 25598438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:30:33,164][74987] Avg episode reward: [(0, '27.490'), (1, '33.890')] -[2023-10-14 15:30:33,551][75950] Updated weights for policy 1, policy_version 49930 (0.0010) -[2023-10-14 15:30:33,921][75950] Updated weights for policy 1, policy_version 49940 (0.0009) -[2023-10-14 15:30:34,190][75949] Updated weights for policy 0, policy_version 50051 (0.0007) -[2023-10-14 15:30:34,284][75950] Updated weights for policy 1, policy_version 49950 (0.0009) -[2023-10-14 15:30:34,362][75801] Saving new best policy, reward=33.890! -[2023-10-14 15:30:34,558][75949] Updated weights for policy 0, policy_version 50061 (0.0009) -[2023-10-14 15:30:34,926][75949] Updated weights for policy 0, policy_version 50071 (0.0010) -[2023-10-14 15:30:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 102432768. Throughput: 0: 1661.7, 1: 1670.4. Samples: 25618964. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 15:30:38,165][74987] Avg episode reward: [(0, '25.830'), (1, '29.300')] -[2023-10-14 15:30:38,520][75950] Updated weights for policy 1, policy_version 49960 (0.0009) -[2023-10-14 15:30:38,891][75950] Updated weights for policy 1, policy_version 49970 (0.0010) -[2023-10-14 15:30:38,948][75949] Updated weights for policy 0, policy_version 50081 (0.0010) -[2023-10-14 15:30:39,266][75950] Updated weights for policy 1, policy_version 49980 (0.0008) -[2023-10-14 15:30:39,320][75949] Updated weights for policy 0, policy_version 50091 (0.0008) -[2023-10-14 15:30:39,679][75949] Updated weights for policy 0, policy_version 50101 (0.0009) -[2023-10-14 15:30:40,054][75949] Updated weights for policy 0, policy_version 50111 (0.0008) -[2023-10-14 15:30:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102498304. Throughput: 0: 1670.6, 1: 1673.9. Samples: 25639866. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 15:30:43,164][74987] Avg episode reward: [(0, '25.130'), (1, '32.330')] -[2023-10-14 15:30:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000050112_51314688.pth... -[2023-10-14 15:30:43,212][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000048544_49709056.pth -[2023-10-14 15:30:43,292][75950] Updated weights for policy 1, policy_version 49990 (0.0009) -[2023-10-14 15:30:43,653][75950] Updated weights for policy 1, policy_version 50000 (0.0011) -[2023-10-14 15:30:44,014][75950] Updated weights for policy 1, policy_version 50010 (0.0008) -[2023-10-14 15:30:44,174][75949] Updated weights for policy 0, policy_version 50121 (0.0008) -[2023-10-14 15:30:44,230][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000050016_51216384.pth... -[2023-10-14 15:30:44,266][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000048448_49610752.pth -[2023-10-14 15:30:44,553][75949] Updated weights for policy 0, policy_version 50131 (0.0008) -[2023-10-14 15:30:44,928][75949] Updated weights for policy 0, policy_version 50141 (0.0009) -[2023-10-14 15:30:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102563840. Throughput: 0: 1666.0, 1: 1673.7. Samples: 25648808. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 15:30:48,164][74987] Avg episode reward: [(0, '25.290'), (1, '31.150')] -[2023-10-14 15:30:48,183][75950] Updated weights for policy 1, policy_version 50020 (0.0008) -[2023-10-14 15:30:48,548][75950] Updated weights for policy 1, policy_version 50030 (0.0009) -[2023-10-14 15:30:48,908][75950] Updated weights for policy 1, policy_version 50040 (0.0007) -[2023-10-14 15:30:49,028][75949] Updated weights for policy 0, policy_version 50151 (0.0008) -[2023-10-14 15:30:49,394][75949] Updated weights for policy 0, policy_version 50161 (0.0008) -[2023-10-14 15:30:49,764][75949] Updated weights for policy 0, policy_version 50171 (0.0009) -[2023-10-14 15:30:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102629376. Throughput: 0: 1676.3, 1: 1678.0. Samples: 25669328. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 15:30:53,164][74987] Avg episode reward: [(0, '25.270'), (1, '28.220')] -[2023-10-14 15:30:53,172][75950] Updated weights for policy 1, policy_version 50050 (0.0009) -[2023-10-14 15:30:53,539][75950] Updated weights for policy 1, policy_version 50060 (0.0010) -[2023-10-14 15:30:53,880][75949] Updated weights for policy 0, policy_version 50181 (0.0008) -[2023-10-14 15:30:53,916][75950] Updated weights for policy 1, policy_version 50070 (0.0009) -[2023-10-14 15:30:54,250][75949] Updated weights for policy 0, policy_version 50191 (0.0008) -[2023-10-14 15:30:54,286][75950] Updated weights for policy 1, policy_version 50080 (0.0009) -[2023-10-14 15:30:54,619][75949] Updated weights for policy 0, policy_version 50201 (0.0010) -[2023-10-14 15:30:58,164][74987] Fps is (10 sec: 13106.6, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 102694912. Throughput: 0: 1673.0, 1: 1684.2. Samples: 25689858. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 15:30:58,165][74987] Avg episode reward: [(0, '26.250'), (1, '30.480')] -[2023-10-14 15:30:58,306][75950] Updated weights for policy 1, policy_version 50090 (0.0011) -[2023-10-14 15:30:58,680][75950] Updated weights for policy 1, policy_version 50100 (0.0011) -[2023-10-14 15:30:58,752][75949] Updated weights for policy 0, policy_version 50211 (0.0010) -[2023-10-14 15:30:59,054][75950] Updated weights for policy 1, policy_version 50110 (0.0009) -[2023-10-14 15:30:59,118][75949] Updated weights for policy 0, policy_version 50221 (0.0007) -[2023-10-14 15:30:59,488][75949] Updated weights for policy 0, policy_version 50231 (0.0009) -[2023-10-14 15:31:02,983][75950] Updated weights for policy 1, policy_version 50120 (0.0008) -[2023-10-14 15:31:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 102760448. Throughput: 0: 1678.1, 1: 1678.2. Samples: 25698962. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 15:31:03,164][74987] Avg episode reward: [(0, '23.910'), (1, '29.830')] -[2023-10-14 15:31:03,347][75950] Updated weights for policy 1, policy_version 50130 (0.0008) -[2023-10-14 15:31:03,501][75949] Updated weights for policy 0, policy_version 50241 (0.0010) -[2023-10-14 15:31:03,722][75950] Updated weights for policy 1, policy_version 50140 (0.0008) -[2023-10-14 15:31:03,872][75949] Updated weights for policy 0, policy_version 50251 (0.0008) -[2023-10-14 15:31:04,245][75949] Updated weights for policy 0, policy_version 50261 (0.0010) -[2023-10-14 15:31:04,615][75949] Updated weights for policy 0, policy_version 50271 (0.0010) -[2023-10-14 15:31:07,694][75950] Updated weights for policy 1, policy_version 50150 (0.0008) -[2023-10-14 15:31:08,061][75950] Updated weights for policy 1, policy_version 50160 (0.0007) -[2023-10-14 15:31:08,163][74987] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102825984. Throughput: 0: 1679.2, 1: 1684.7. Samples: 25719870. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 15:31:08,164][74987] Avg episode reward: [(0, '25.610'), (1, '29.710')] -[2023-10-14 15:31:08,435][75950] Updated weights for policy 1, policy_version 50170 (0.0008) -[2023-10-14 15:31:08,753][75949] Updated weights for policy 0, policy_version 50281 (0.0007) -[2023-10-14 15:31:09,127][75949] Updated weights for policy 0, policy_version 50291 (0.0008) -[2023-10-14 15:31:09,503][75949] Updated weights for policy 0, policy_version 50301 (0.0008) -[2023-10-14 15:31:12,430][75950] Updated weights for policy 1, policy_version 50180 (0.0008) -[2023-10-14 15:31:12,789][75950] Updated weights for policy 1, policy_version 50190 (0.0009) -[2023-10-14 15:31:13,160][75950] Updated weights for policy 1, policy_version 50200 (0.0010) -[2023-10-14 15:31:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102891520. Throughput: 0: 1674.8, 1: 1676.5. Samples: 25740064. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:31:13,164][74987] Avg episode reward: [(0, '24.360'), (1, '30.520')] -[2023-10-14 15:31:13,635][75949] Updated weights for policy 0, policy_version 50311 (0.0008) -[2023-10-14 15:31:14,000][75949] Updated weights for policy 0, policy_version 50321 (0.0010) -[2023-10-14 15:31:14,377][75949] Updated weights for policy 0, policy_version 50331 (0.0009) -[2023-10-14 15:31:17,126][75950] Updated weights for policy 1, policy_version 50210 (0.0008) -[2023-10-14 15:31:17,490][75950] Updated weights for policy 1, policy_version 50220 (0.0009) -[2023-10-14 15:31:17,850][75950] Updated weights for policy 1, policy_version 50230 (0.0010) -[2023-10-14 15:31:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102957056. Throughput: 0: 1672.8, 1: 1685.2. Samples: 25749548. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:31:18,164][74987] Avg episode reward: [(0, '26.920'), (1, '30.720')] -[2023-10-14 15:31:18,225][75950] Updated weights for policy 1, policy_version 50240 (0.0007) -[2023-10-14 15:31:18,689][75949] Updated weights for policy 0, policy_version 50341 (0.0008) -[2023-10-14 15:31:19,051][75949] Updated weights for policy 0, policy_version 50351 (0.0009) -[2023-10-14 15:31:19,431][75949] Updated weights for policy 0, policy_version 50361 (0.0008) -[2023-10-14 15:31:22,212][75950] Updated weights for policy 1, policy_version 50250 (0.0009) -[2023-10-14 15:31:22,575][75950] Updated weights for policy 1, policy_version 50260 (0.0010) -[2023-10-14 15:31:22,953][75950] Updated weights for policy 1, policy_version 50270 (0.0008) -[2023-10-14 15:31:23,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 103055360. Throughput: 0: 1671.2, 1: 1693.4. Samples: 25770368. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:31:23,164][74987] Avg episode reward: [(0, '25.650'), (1, '30.920')] -[2023-10-14 15:31:23,541][75949] Updated weights for policy 0, policy_version 50371 (0.0011) -[2023-10-14 15:31:23,908][75949] Updated weights for policy 0, policy_version 50381 (0.0008) -[2023-10-14 15:31:24,276][75949] Updated weights for policy 0, policy_version 50391 (0.0008) -[2023-10-14 15:31:26,982][75950] Updated weights for policy 1, policy_version 50280 (0.0009) -[2023-10-14 15:31:27,347][75950] Updated weights for policy 1, policy_version 50290 (0.0010) -[2023-10-14 15:31:27,717][75950] Updated weights for policy 1, policy_version 50300 (0.0011) -[2023-10-14 15:31:28,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 103120896. Throughput: 0: 1669.0, 1: 1676.1. Samples: 25790394. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:31:28,164][74987] Avg episode reward: [(0, '27.570'), (1, '30.370')] -[2023-10-14 15:31:28,275][75949] Updated weights for policy 0, policy_version 50401 (0.0008) -[2023-10-14 15:31:28,644][75949] Updated weights for policy 0, policy_version 50411 (0.0009) -[2023-10-14 15:31:29,013][75949] Updated weights for policy 0, policy_version 50421 (0.0009) -[2023-10-14 15:31:29,399][75949] Updated weights for policy 0, policy_version 50431 (0.0009) -[2023-10-14 15:31:31,920][75950] Updated weights for policy 1, policy_version 50310 (0.0010) -[2023-10-14 15:31:32,306][75950] Updated weights for policy 1, policy_version 50320 (0.0008) -[2023-10-14 15:31:32,663][75950] Updated weights for policy 1, policy_version 50330 (0.0008) -[2023-10-14 15:31:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 103186432. Throughput: 0: 1669.3, 1: 1698.5. Samples: 25800362. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:31:33,164][74987] Avg episode reward: [(0, '23.930'), (1, '30.540')] -[2023-10-14 15:31:33,425][75949] Updated weights for policy 0, policy_version 50441 (0.0010) -[2023-10-14 15:31:33,799][75949] Updated weights for policy 0, policy_version 50451 (0.0009) -[2023-10-14 15:31:34,166][75949] Updated weights for policy 0, policy_version 50461 (0.0010) -[2023-10-14 15:31:36,765][75950] Updated weights for policy 1, policy_version 50340 (0.0010) -[2023-10-14 15:31:37,136][75950] Updated weights for policy 1, policy_version 50350 (0.0011) -[2023-10-14 15:31:37,509][75950] Updated weights for policy 1, policy_version 50360 (0.0011) -[2023-10-14 15:31:38,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 103251968. Throughput: 0: 1674.1, 1: 1694.4. Samples: 25820910. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:31:38,164][74987] Avg episode reward: [(0, '27.310'), (1, '30.110')] -[2023-10-14 15:31:38,195][75949] Updated weights for policy 0, policy_version 50471 (0.0010) -[2023-10-14 15:31:38,559][75949] Updated weights for policy 0, policy_version 50481 (0.0008) -[2023-10-14 15:31:38,939][75949] Updated weights for policy 0, policy_version 50491 (0.0007) -[2023-10-14 15:31:41,519][75950] Updated weights for policy 1, policy_version 50370 (0.0011) -[2023-10-14 15:31:41,882][75950] Updated weights for policy 1, policy_version 50380 (0.0008) -[2023-10-14 15:31:42,245][75950] Updated weights for policy 1, policy_version 50390 (0.0010) -[2023-10-14 15:31:42,609][75950] Updated weights for policy 1, policy_version 50400 (0.0009) -[2023-10-14 15:31:42,935][75949] Updated weights for policy 0, policy_version 50501 (0.0007) -[2023-10-14 15:31:43,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 103317504. Throughput: 0: 1680.0, 1: 1669.7. Samples: 25840596. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:31:43,164][74987] Avg episode reward: [(0, '25.140'), (1, '30.190')] -[2023-10-14 15:31:43,308][75949] Updated weights for policy 0, policy_version 50511 (0.0009) -[2023-10-14 15:31:43,679][75949] Updated weights for policy 0, policy_version 50521 (0.0009) -[2023-10-14 15:31:46,922][75950] Updated weights for policy 1, policy_version 50410 (0.0009) -[2023-10-14 15:31:47,289][75950] Updated weights for policy 1, policy_version 50420 (0.0009) -[2023-10-14 15:31:47,568][75949] Updated weights for policy 0, policy_version 50531 (0.0010) -[2023-10-14 15:31:47,655][75950] Updated weights for policy 1, policy_version 50430 (0.0008) -[2023-10-14 15:31:47,933][75949] Updated weights for policy 0, policy_version 50541 (0.0009) -[2023-10-14 15:31:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 103383040. Throughput: 0: 1674.9, 1: 1696.7. Samples: 25850686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:31:48,165][74987] Avg episode reward: [(0, '25.230'), (1, '30.390')] -[2023-10-14 15:31:48,303][75949] Updated weights for policy 0, policy_version 50551 (0.0012) -[2023-10-14 15:31:51,760][75950] Updated weights for policy 1, policy_version 50440 (0.0008) -[2023-10-14 15:31:52,122][75950] Updated weights for policy 1, policy_version 50450 (0.0008) -[2023-10-14 15:31:52,356][75949] Updated weights for policy 0, policy_version 50561 (0.0008) -[2023-10-14 15:31:52,493][75950] Updated weights for policy 1, policy_version 50460 (0.0008) -[2023-10-14 15:31:52,719][75949] Updated weights for policy 0, policy_version 50571 (0.0009) -[2023-10-14 15:31:53,103][75949] Updated weights for policy 0, policy_version 50581 (0.0008) -[2023-10-14 15:31:53,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 103448576. Throughput: 0: 1676.7, 1: 1685.2. Samples: 25871154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:31:53,164][74987] Avg episode reward: [(0, '23.780'), (1, '31.480')] -[2023-10-14 15:31:53,460][75949] Updated weights for policy 0, policy_version 50591 (0.0009) -[2023-10-14 15:31:56,684][75950] Updated weights for policy 1, policy_version 50470 (0.0008) -[2023-10-14 15:31:57,058][75950] Updated weights for policy 1, policy_version 50480 (0.0009) -[2023-10-14 15:31:57,431][75950] Updated weights for policy 1, policy_version 50490 (0.0009) -[2023-10-14 15:31:57,472][75949] Updated weights for policy 0, policy_version 50601 (0.0008) -[2023-10-14 15:31:57,841][75949] Updated weights for policy 0, policy_version 50611 (0.0009) -[2023-10-14 15:31:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 103514112. Throughput: 0: 1675.1, 1: 1666.4. Samples: 25890430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:31:58,164][74987] Avg episode reward: [(0, '25.480'), (1, '30.700')] -[2023-10-14 15:31:58,215][75949] Updated weights for policy 0, policy_version 50621 (0.0008) -[2023-10-14 15:32:01,469][75950] Updated weights for policy 1, policy_version 50500 (0.0009) -[2023-10-14 15:32:01,842][75950] Updated weights for policy 1, policy_version 50510 (0.0007) -[2023-10-14 15:32:02,207][75950] Updated weights for policy 1, policy_version 50520 (0.0008) -[2023-10-14 15:32:02,449][75949] Updated weights for policy 0, policy_version 50631 (0.0007) -[2023-10-14 15:32:02,827][75949] Updated weights for policy 0, policy_version 50641 (0.0008) -[2023-10-14 15:32:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 103579648. Throughput: 0: 1686.3, 1: 1684.2. Samples: 25901218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:32:03,164][74987] Avg episode reward: [(0, '25.700'), (1, '29.850')] -[2023-10-14 15:32:03,196][75949] Updated weights for policy 0, policy_version 50651 (0.0007) -[2023-10-14 15:32:06,206][75950] Updated weights for policy 1, policy_version 50530 (0.0008) -[2023-10-14 15:32:06,572][75950] Updated weights for policy 1, policy_version 50540 (0.0010) -[2023-10-14 15:32:06,946][75950] Updated weights for policy 1, policy_version 50550 (0.0009) -[2023-10-14 15:32:07,285][75949] Updated weights for policy 0, policy_version 50661 (0.0007) -[2023-10-14 15:32:07,315][75950] Updated weights for policy 1, policy_version 50560 (0.0007) -[2023-10-14 15:32:07,654][75949] Updated weights for policy 0, policy_version 50671 (0.0007) -[2023-10-14 15:32:08,024][75949] Updated weights for policy 0, policy_version 50681 (0.0009) -[2023-10-14 15:32:08,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 103645184. Throughput: 0: 1691.2, 1: 1659.1. Samples: 25921128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:32:08,164][74987] Avg episode reward: [(0, '24.390'), (1, '29.760')] -[2023-10-14 15:32:11,363][75950] Updated weights for policy 1, policy_version 50570 (0.0009) -[2023-10-14 15:32:11,729][75950] Updated weights for policy 1, policy_version 50580 (0.0007) -[2023-10-14 15:32:12,094][75950] Updated weights for policy 1, policy_version 50590 (0.0010) -[2023-10-14 15:32:12,160][75949] Updated weights for policy 0, policy_version 50691 (0.0009) -[2023-10-14 15:32:12,543][75949] Updated weights for policy 0, policy_version 50701 (0.0008) -[2023-10-14 15:32:12,908][75949] Updated weights for policy 0, policy_version 50711 (0.0007) -[2023-10-14 15:32:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 103710720. Throughput: 0: 1673.4, 1: 1660.7. Samples: 25940428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:32:13,164][74987] Avg episode reward: [(0, '25.860'), (1, '28.700')] -[2023-10-14 15:32:16,265][75950] Updated weights for policy 1, policy_version 50600 (0.0009) -[2023-10-14 15:32:16,626][75950] Updated weights for policy 1, policy_version 50610 (0.0009) -[2023-10-14 15:32:16,984][75950] Updated weights for policy 1, policy_version 50620 (0.0010) -[2023-10-14 15:32:17,009][75949] Updated weights for policy 0, policy_version 50721 (0.0008) -[2023-10-14 15:32:17,373][75949] Updated weights for policy 0, policy_version 50731 (0.0007) -[2023-10-14 15:32:17,746][75949] Updated weights for policy 0, policy_version 50741 (0.0009) -[2023-10-14 15:32:18,105][75949] Updated weights for policy 0, policy_version 50751 (0.0008) -[2023-10-14 15:32:18,163][74987] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 103809024. Throughput: 0: 1683.9, 1: 1670.9. Samples: 25951328. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-14 15:32:18,164][74987] Avg episode reward: [(0, '22.980'), (1, '28.870')] -[2023-10-14 15:32:21,262][75950] Updated weights for policy 1, policy_version 50630 (0.0011) -[2023-10-14 15:32:21,648][75950] Updated weights for policy 1, policy_version 50640 (0.0011) -[2023-10-14 15:32:22,010][75950] Updated weights for policy 1, policy_version 50650 (0.0010) -[2023-10-14 15:32:22,334][75949] Updated weights for policy 0, policy_version 50761 (0.0009) -[2023-10-14 15:32:22,705][75949] Updated weights for policy 0, policy_version 50771 (0.0008) -[2023-10-14 15:32:23,071][75949] Updated weights for policy 0, policy_version 50781 (0.0007) -[2023-10-14 15:32:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 103841792. Throughput: 0: 1687.1, 1: 1654.5. Samples: 25971284. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-14 15:32:23,165][74987] Avg episode reward: [(0, '26.910'), (1, '30.000')] -[2023-10-14 15:32:26,109][75950] Updated weights for policy 1, policy_version 50660 (0.0008) -[2023-10-14 15:32:26,481][75950] Updated weights for policy 1, policy_version 50670 (0.0010) -[2023-10-14 15:32:26,848][75950] Updated weights for policy 1, policy_version 50680 (0.0009) -[2023-10-14 15:32:26,918][75949] Updated weights for policy 0, policy_version 50791 (0.0007) -[2023-10-14 15:32:27,290][75949] Updated weights for policy 0, policy_version 50801 (0.0008) -[2023-10-14 15:32:27,672][75949] Updated weights for policy 0, policy_version 50811 (0.0009) -[2023-10-14 15:32:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 103940096. Throughput: 0: 1657.9, 1: 1665.1. Samples: 25990128. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-14 15:32:28,164][74987] Avg episode reward: [(0, '21.750'), (1, '29.500')] -[2023-10-14 15:32:30,696][75950] Updated weights for policy 1, policy_version 50690 (0.0009) -[2023-10-14 15:32:31,063][75950] Updated weights for policy 1, policy_version 50700 (0.0010) -[2023-10-14 15:32:31,427][75950] Updated weights for policy 1, policy_version 50710 (0.0008) -[2023-10-14 15:32:31,681][75949] Updated weights for policy 0, policy_version 50821 (0.0008) -[2023-10-14 15:32:31,796][75950] Updated weights for policy 1, policy_version 50720 (0.0010) -[2023-10-14 15:32:32,041][75949] Updated weights for policy 0, policy_version 50831 (0.0007) -[2023-10-14 15:32:32,416][75949] Updated weights for policy 0, policy_version 50841 (0.0007) -[2023-10-14 15:32:33,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104005632. Throughput: 0: 1684.5, 1: 1667.9. Samples: 26001544. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-14 15:32:33,165][74987] Avg episode reward: [(0, '24.890'), (1, '30.840')] -[2023-10-14 15:32:35,961][75950] Updated weights for policy 1, policy_version 50730 (0.0008) -[2023-10-14 15:32:36,328][75950] Updated weights for policy 1, policy_version 50740 (0.0008) -[2023-10-14 15:32:36,509][75949] Updated weights for policy 0, policy_version 50851 (0.0008) -[2023-10-14 15:32:36,689][75950] Updated weights for policy 1, policy_version 50750 (0.0009) -[2023-10-14 15:32:36,880][75949] Updated weights for policy 0, policy_version 50861 (0.0010) -[2023-10-14 15:32:37,259][75949] Updated weights for policy 0, policy_version 50871 (0.0009) -[2023-10-14 15:32:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104071168. Throughput: 0: 1676.9, 1: 1648.0. Samples: 26020776. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-14 15:32:38,164][74987] Avg episode reward: [(0, '20.200'), (1, '31.740')] -[2023-10-14 15:32:40,753][75950] Updated weights for policy 1, policy_version 50760 (0.0007) -[2023-10-14 15:32:41,121][75950] Updated weights for policy 1, policy_version 50770 (0.0011) -[2023-10-14 15:32:41,343][75949] Updated weights for policy 0, policy_version 50881 (0.0009) -[2023-10-14 15:32:41,492][75950] Updated weights for policy 1, policy_version 50780 (0.0009) -[2023-10-14 15:32:41,705][75949] Updated weights for policy 0, policy_version 50891 (0.0008) -[2023-10-14 15:32:42,075][75949] Updated weights for policy 0, policy_version 50901 (0.0007) -[2023-10-14 15:32:42,445][75949] Updated weights for policy 0, policy_version 50911 (0.0008) -[2023-10-14 15:32:43,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104136704. Throughput: 0: 1660.9, 1: 1670.5. Samples: 26040344. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-14 15:32:43,164][74987] Avg episode reward: [(0, '26.900'), (1, '31.000')] -[2023-10-14 15:32:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000050784_52002816.pth... -[2023-10-14 15:32:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000050912_52133888.pth... -[2023-10-14 15:32:43,203][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000049216_50397184.pth -[2023-10-14 15:32:43,212][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000049344_50528256.pth -[2023-10-14 15:32:45,670][75950] Updated weights for policy 1, policy_version 50790 (0.0008) -[2023-10-14 15:32:46,046][75950] Updated weights for policy 1, policy_version 50800 (0.0008) -[2023-10-14 15:32:46,409][75950] Updated weights for policy 1, policy_version 50810 (0.0008) -[2023-10-14 15:32:46,650][75949] Updated weights for policy 0, policy_version 50921 (0.0009) -[2023-10-14 15:32:47,023][75949] Updated weights for policy 0, policy_version 50931 (0.0008) -[2023-10-14 15:32:47,402][75949] Updated weights for policy 0, policy_version 50941 (0.0008) -[2023-10-14 15:32:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104202240. Throughput: 0: 1678.1, 1: 1665.5. Samples: 26051680. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) -[2023-10-14 15:32:48,165][74987] Avg episode reward: [(0, '24.210'), (1, '30.010')] -[2023-10-14 15:32:50,570][75950] Updated weights for policy 1, policy_version 50820 (0.0008) -[2023-10-14 15:32:50,939][75950] Updated weights for policy 1, policy_version 50830 (0.0009) -[2023-10-14 15:32:51,311][75950] Updated weights for policy 1, policy_version 50840 (0.0008) -[2023-10-14 15:32:51,633][75949] Updated weights for policy 0, policy_version 50951 (0.0007) -[2023-10-14 15:32:52,002][75949] Updated weights for policy 0, policy_version 50961 (0.0007) -[2023-10-14 15:32:52,371][75949] Updated weights for policy 0, policy_version 50971 (0.0007) -[2023-10-14 15:32:53,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104267776. Throughput: 0: 1665.6, 1: 1658.5. Samples: 26070714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:32:53,165][74987] Avg episode reward: [(0, '25.590'), (1, '31.640')] -[2023-10-14 15:32:55,356][75950] Updated weights for policy 1, policy_version 50850 (0.0009) -[2023-10-14 15:32:55,725][75950] Updated weights for policy 1, policy_version 50860 (0.0010) -[2023-10-14 15:32:56,095][75950] Updated weights for policy 1, policy_version 50870 (0.0008) -[2023-10-14 15:32:56,398][75949] Updated weights for policy 0, policy_version 50981 (0.0007) -[2023-10-14 15:32:56,458][75950] Updated weights for policy 1, policy_version 50880 (0.0008) -[2023-10-14 15:32:56,769][75949] Updated weights for policy 0, policy_version 50991 (0.0010) -[2023-10-14 15:32:57,139][75949] Updated weights for policy 0, policy_version 51001 (0.0007) -[2023-10-14 15:32:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104333312. Throughput: 0: 1657.8, 1: 1673.7. Samples: 26090346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:32:58,165][74987] Avg episode reward: [(0, '24.680'), (1, '30.550')] -[2023-10-14 15:33:00,588][75950] Updated weights for policy 1, policy_version 50890 (0.0008) -[2023-10-14 15:33:00,957][75950] Updated weights for policy 1, policy_version 50900 (0.0009) -[2023-10-14 15:33:01,082][75949] Updated weights for policy 0, policy_version 51011 (0.0008) -[2023-10-14 15:33:01,317][75950] Updated weights for policy 1, policy_version 50910 (0.0010) -[2023-10-14 15:33:01,450][75949] Updated weights for policy 0, policy_version 51021 (0.0009) -[2023-10-14 15:33:01,828][75949] Updated weights for policy 0, policy_version 51031 (0.0008) -[2023-10-14 15:33:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 104398848. Throughput: 0: 1678.0, 1: 1657.5. Samples: 26101428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:33:03,165][74987] Avg episode reward: [(0, '26.070'), (1, '28.820')] -[2023-10-14 15:33:05,453][75950] Updated weights for policy 1, policy_version 50920 (0.0007) -[2023-10-14 15:33:05,816][75950] Updated weights for policy 1, policy_version 50930 (0.0007) -[2023-10-14 15:33:06,059][75949] Updated weights for policy 0, policy_version 51041 (0.0010) -[2023-10-14 15:33:06,169][75950] Updated weights for policy 1, policy_version 50940 (0.0009) -[2023-10-14 15:33:06,432][75949] Updated weights for policy 0, policy_version 51051 (0.0008) -[2023-10-14 15:33:06,811][75949] Updated weights for policy 0, policy_version 51061 (0.0009) -[2023-10-14 15:33:07,187][75949] Updated weights for policy 0, policy_version 51071 (0.0009) -[2023-10-14 15:33:08,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 104464384. Throughput: 0: 1656.8, 1: 1657.7. Samples: 26120436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:33:08,164][74987] Avg episode reward: [(0, '24.030'), (1, '31.200')] -[2023-10-14 15:33:10,268][75950] Updated weights for policy 1, policy_version 50950 (0.0009) -[2023-10-14 15:33:10,659][75950] Updated weights for policy 1, policy_version 50960 (0.0010) -[2023-10-14 15:33:11,033][75950] Updated weights for policy 1, policy_version 50970 (0.0009) -[2023-10-14 15:33:11,296][75949] Updated weights for policy 0, policy_version 51081 (0.0009) -[2023-10-14 15:33:11,668][75949] Updated weights for policy 0, policy_version 51091 (0.0010) -[2023-10-14 15:33:12,037][75949] Updated weights for policy 0, policy_version 51101 (0.0008) -[2023-10-14 15:33:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 104529920. Throughput: 0: 1663.2, 1: 1673.3. Samples: 26140270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:33:13,165][74987] Avg episode reward: [(0, '25.610'), (1, '31.600')] -[2023-10-14 15:33:15,047][75950] Updated weights for policy 1, policy_version 50980 (0.0010) -[2023-10-14 15:33:15,414][75950] Updated weights for policy 1, policy_version 50990 (0.0010) -[2023-10-14 15:33:15,784][75950] Updated weights for policy 1, policy_version 51000 (0.0009) -[2023-10-14 15:33:15,944][75949] Updated weights for policy 0, policy_version 51111 (0.0010) -[2023-10-14 15:33:16,305][75949] Updated weights for policy 0, policy_version 51121 (0.0010) -[2023-10-14 15:33:16,673][75949] Updated weights for policy 0, policy_version 51131 (0.0007) -[2023-10-14 15:33:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104595456. Throughput: 0: 1671.4, 1: 1656.6. Samples: 26151304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:33:18,165][74987] Avg episode reward: [(0, '26.820'), (1, '29.520')] -[2023-10-14 15:33:19,824][75950] Updated weights for policy 1, policy_version 51010 (0.0008) -[2023-10-14 15:33:20,185][75950] Updated weights for policy 1, policy_version 51020 (0.0011) -[2023-10-14 15:33:20,548][75950] Updated weights for policy 1, policy_version 51030 (0.0007) -[2023-10-14 15:33:20,870][75949] Updated weights for policy 0, policy_version 51141 (0.0009) -[2023-10-14 15:33:20,911][75950] Updated weights for policy 1, policy_version 51040 (0.0007) -[2023-10-14 15:33:21,240][75949] Updated weights for policy 0, policy_version 51151 (0.0010) -[2023-10-14 15:33:21,615][75949] Updated weights for policy 0, policy_version 51161 (0.0011) -[2023-10-14 15:33:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 104660992. Throughput: 0: 1650.8, 1: 1670.8. Samples: 26170250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:33:23,164][74987] Avg episode reward: [(0, '27.910'), (1, '33.000')] -[2023-10-14 15:33:25,143][75950] Updated weights for policy 1, policy_version 51050 (0.0008) -[2023-10-14 15:33:25,510][75950] Updated weights for policy 1, policy_version 51060 (0.0009) -[2023-10-14 15:33:25,552][75949] Updated weights for policy 0, policy_version 51171 (0.0009) -[2023-10-14 15:33:25,884][75950] Updated weights for policy 1, policy_version 51070 (0.0009) -[2023-10-14 15:33:25,926][75949] Updated weights for policy 0, policy_version 51181 (0.0007) -[2023-10-14 15:33:26,290][75949] Updated weights for policy 0, policy_version 51191 (0.0007) -[2023-10-14 15:33:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104726528. Throughput: 0: 1670.3, 1: 1671.5. Samples: 26190730. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:33:28,165][74987] Avg episode reward: [(0, '24.720'), (1, '31.730')] -[2023-10-14 15:33:29,908][75950] Updated weights for policy 1, policy_version 51080 (0.0008) -[2023-10-14 15:33:30,282][75950] Updated weights for policy 1, policy_version 51090 (0.0008) -[2023-10-14 15:33:30,552][75949] Updated weights for policy 0, policy_version 51201 (0.0008) -[2023-10-14 15:33:30,642][75950] Updated weights for policy 1, policy_version 51100 (0.0009) -[2023-10-14 15:33:30,917][75949] Updated weights for policy 0, policy_version 51211 (0.0009) -[2023-10-14 15:33:31,288][75949] Updated weights for policy 0, policy_version 51221 (0.0009) -[2023-10-14 15:33:31,647][75949] Updated weights for policy 0, policy_version 51231 (0.0009) -[2023-10-14 15:33:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 104792064. Throughput: 0: 1669.9, 1: 1653.9. Samples: 26201250. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:33:33,165][74987] Avg episode reward: [(0, '26.900'), (1, '30.560')] -[2023-10-14 15:33:34,757][75950] Updated weights for policy 1, policy_version 51110 (0.0007) -[2023-10-14 15:33:35,119][75950] Updated weights for policy 1, policy_version 51120 (0.0008) -[2023-10-14 15:33:35,429][75949] Updated weights for policy 0, policy_version 51241 (0.0008) -[2023-10-14 15:33:35,483][75950] Updated weights for policy 1, policy_version 51130 (0.0009) -[2023-10-14 15:33:35,800][75949] Updated weights for policy 0, policy_version 51251 (0.0007) -[2023-10-14 15:33:36,174][75949] Updated weights for policy 0, policy_version 51261 (0.0008) -[2023-10-14 15:33:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 104857600. Throughput: 0: 1661.4, 1: 1667.7. Samples: 26220524. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:33:38,165][74987] Avg episode reward: [(0, '25.840'), (1, '29.490')] -[2023-10-14 15:33:39,568][75950] Updated weights for policy 1, policy_version 51140 (0.0008) -[2023-10-14 15:33:39,933][75950] Updated weights for policy 1, policy_version 51150 (0.0010) -[2023-10-14 15:33:40,213][75949] Updated weights for policy 0, policy_version 51271 (0.0008) -[2023-10-14 15:33:40,290][75950] Updated weights for policy 1, policy_version 51160 (0.0008) -[2023-10-14 15:33:40,593][75949] Updated weights for policy 0, policy_version 51281 (0.0009) -[2023-10-14 15:33:40,951][75949] Updated weights for policy 0, policy_version 51291 (0.0008) -[2023-10-14 15:33:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104923136. Throughput: 0: 1687.6, 1: 1674.5. Samples: 26241636. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:33:43,164][74987] Avg episode reward: [(0, '27.970'), (1, '30.110')] -[2023-10-14 15:33:44,407][75950] Updated weights for policy 1, policy_version 51170 (0.0009) -[2023-10-14 15:33:44,774][75950] Updated weights for policy 1, policy_version 51180 (0.0009) -[2023-10-14 15:33:45,044][75949] Updated weights for policy 0, policy_version 51301 (0.0008) -[2023-10-14 15:33:45,136][75950] Updated weights for policy 1, policy_version 51190 (0.0009) -[2023-10-14 15:33:45,406][75949] Updated weights for policy 0, policy_version 51311 (0.0007) -[2023-10-14 15:33:45,496][75950] Updated weights for policy 1, policy_version 51200 (0.0008) -[2023-10-14 15:33:45,790][75949] Updated weights for policy 0, policy_version 51321 (0.0010) -[2023-10-14 15:33:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104988672. Throughput: 0: 1664.6, 1: 1658.9. Samples: 26250986. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:33:48,164][74987] Avg episode reward: [(0, '25.230'), (1, '28.980')] -[2023-10-14 15:33:49,546][75950] Updated weights for policy 1, policy_version 51210 (0.0008) -[2023-10-14 15:33:49,897][75949] Updated weights for policy 0, policy_version 51331 (0.0008) -[2023-10-14 15:33:49,910][75950] Updated weights for policy 1, policy_version 51220 (0.0009) -[2023-10-14 15:33:50,268][75949] Updated weights for policy 0, policy_version 51341 (0.0008) -[2023-10-14 15:33:50,272][75950] Updated weights for policy 1, policy_version 51230 (0.0007) -[2023-10-14 15:33:50,647][75949] Updated weights for policy 0, policy_version 51351 (0.0009) -[2023-10-14 15:33:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105054208. Throughput: 0: 1673.1, 1: 1682.1. Samples: 26271424. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:33:53,165][74987] Avg episode reward: [(0, '26.830'), (1, '27.490')] -[2023-10-14 15:33:54,379][75950] Updated weights for policy 1, policy_version 51240 (0.0009) -[2023-10-14 15:33:54,748][75950] Updated weights for policy 1, policy_version 51250 (0.0010) -[2023-10-14 15:33:54,787][75949] Updated weights for policy 0, policy_version 51361 (0.0009) -[2023-10-14 15:33:55,112][75950] Updated weights for policy 1, policy_version 51260 (0.0007) -[2023-10-14 15:33:55,158][75949] Updated weights for policy 0, policy_version 51371 (0.0010) -[2023-10-14 15:33:55,515][75949] Updated weights for policy 0, policy_version 51381 (0.0009) -[2023-10-14 15:33:55,883][75949] Updated weights for policy 0, policy_version 51391 (0.0010) -[2023-10-14 15:33:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 105119744. Throughput: 0: 1691.2, 1: 1682.7. Samples: 26292096. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:33:58,165][74987] Avg episode reward: [(0, '21.970'), (1, '29.530')] -[2023-10-14 15:33:59,203][75950] Updated weights for policy 1, policy_version 51270 (0.0008) -[2023-10-14 15:33:59,576][75950] Updated weights for policy 1, policy_version 51280 (0.0007) -[2023-10-14 15:33:59,934][75950] Updated weights for policy 1, policy_version 51290 (0.0008) -[2023-10-14 15:34:00,192][75949] Updated weights for policy 0, policy_version 51401 (0.0009) -[2023-10-14 15:34:00,560][75949] Updated weights for policy 0, policy_version 51411 (0.0009) -[2023-10-14 15:34:00,940][75949] Updated weights for policy 0, policy_version 51421 (0.0007) -[2023-10-14 15:34:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105185280. Throughput: 0: 1666.5, 1: 1670.7. Samples: 26301476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:34:03,164][74987] Avg episode reward: [(0, '25.040'), (1, '29.520')] -[2023-10-14 15:34:03,801][75950] Updated weights for policy 1, policy_version 51300 (0.0007) -[2023-10-14 15:34:04,154][75950] Updated weights for policy 1, policy_version 51310 (0.0008) -[2023-10-14 15:34:04,532][75950] Updated weights for policy 1, policy_version 51320 (0.0010) -[2023-10-14 15:34:05,058][75949] Updated weights for policy 0, policy_version 51431 (0.0009) -[2023-10-14 15:34:05,431][75949] Updated weights for policy 0, policy_version 51441 (0.0007) -[2023-10-14 15:34:05,791][75949] Updated weights for policy 0, policy_version 51451 (0.0008) -[2023-10-14 15:34:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105250816. Throughput: 0: 1684.0, 1: 1687.6. Samples: 26321976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:34:08,164][74987] Avg episode reward: [(0, '23.380'), (1, '29.150')] -[2023-10-14 15:34:08,583][75950] Updated weights for policy 1, policy_version 51330 (0.0010) -[2023-10-14 15:34:08,951][75950] Updated weights for policy 1, policy_version 51340 (0.0009) -[2023-10-14 15:34:09,327][75950] Updated weights for policy 1, policy_version 51350 (0.0009) -[2023-10-14 15:34:09,678][75950] Updated weights for policy 1, policy_version 51360 (0.0010) -[2023-10-14 15:34:09,768][75949] Updated weights for policy 0, policy_version 51461 (0.0008) -[2023-10-14 15:34:10,134][75949] Updated weights for policy 0, policy_version 51471 (0.0007) -[2023-10-14 15:34:10,500][75949] Updated weights for policy 0, policy_version 51481 (0.0007) -[2023-10-14 15:34:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 105316352. Throughput: 0: 1695.7, 1: 1682.8. Samples: 26342762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:34:13,164][74987] Avg episode reward: [(0, '26.620'), (1, '30.750')] -[2023-10-14 15:34:13,994][75950] Updated weights for policy 1, policy_version 51370 (0.0009) -[2023-10-14 15:34:14,366][75950] Updated weights for policy 1, policy_version 51380 (0.0011) -[2023-10-14 15:34:14,477][75949] Updated weights for policy 0, policy_version 51491 (0.0007) -[2023-10-14 15:34:14,731][75950] Updated weights for policy 1, policy_version 51390 (0.0011) -[2023-10-14 15:34:14,852][75949] Updated weights for policy 0, policy_version 51501 (0.0009) -[2023-10-14 15:34:15,220][75949] Updated weights for policy 0, policy_version 51511 (0.0010) -[2023-10-14 15:34:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105381888. Throughput: 0: 1668.5, 1: 1676.0. Samples: 26351754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:34:18,164][74987] Avg episode reward: [(0, '25.560'), (1, '31.500')] -[2023-10-14 15:34:18,876][75950] Updated weights for policy 1, policy_version 51400 (0.0008) -[2023-10-14 15:34:19,237][75950] Updated weights for policy 1, policy_version 51410 (0.0007) -[2023-10-14 15:34:19,318][75949] Updated weights for policy 0, policy_version 51521 (0.0010) -[2023-10-14 15:34:19,607][75950] Updated weights for policy 1, policy_version 51420 (0.0007) -[2023-10-14 15:34:19,689][75949] Updated weights for policy 0, policy_version 51531 (0.0007) -[2023-10-14 15:34:20,055][75949] Updated weights for policy 0, policy_version 51541 (0.0007) -[2023-10-14 15:34:20,419][75949] Updated weights for policy 0, policy_version 51551 (0.0008) -[2023-10-14 15:34:23,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105447424. Throughput: 0: 1687.7, 1: 1689.4. Samples: 26372492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:34:23,164][74987] Avg episode reward: [(0, '26.290'), (1, '30.830')] -[2023-10-14 15:34:23,488][75950] Updated weights for policy 1, policy_version 51430 (0.0007) -[2023-10-14 15:34:23,861][75950] Updated weights for policy 1, policy_version 51440 (0.0008) -[2023-10-14 15:34:24,220][75950] Updated weights for policy 1, policy_version 51450 (0.0008) -[2023-10-14 15:34:24,562][75949] Updated weights for policy 0, policy_version 51561 (0.0010) -[2023-10-14 15:34:24,930][75949] Updated weights for policy 0, policy_version 51571 (0.0010) -[2023-10-14 15:34:25,306][75949] Updated weights for policy 0, policy_version 51581 (0.0009) -[2023-10-14 15:34:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 105512960. Throughput: 0: 1687.2, 1: 1686.4. Samples: 26393450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:34:28,164][74987] Avg episode reward: [(0, '23.820'), (1, '29.420')] -[2023-10-14 15:34:28,308][75950] Updated weights for policy 1, policy_version 51460 (0.0007) -[2023-10-14 15:34:28,672][75950] Updated weights for policy 1, policy_version 51470 (0.0008) -[2023-10-14 15:34:29,035][75950] Updated weights for policy 1, policy_version 51480 (0.0009) -[2023-10-14 15:34:29,262][75949] Updated weights for policy 0, policy_version 51591 (0.0007) -[2023-10-14 15:34:29,640][75949] Updated weights for policy 0, policy_version 51601 (0.0009) -[2023-10-14 15:34:30,007][75949] Updated weights for policy 0, policy_version 51611 (0.0011) -[2023-10-14 15:34:33,162][75950] Updated weights for policy 1, policy_version 51490 (0.0007) -[2023-10-14 15:34:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105578496. Throughput: 0: 1680.3, 1: 1689.3. Samples: 26402620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:34:33,164][74987] Avg episode reward: [(0, '23.950'), (1, '30.410')] -[2023-10-14 15:34:33,535][75950] Updated weights for policy 1, policy_version 51500 (0.0010) -[2023-10-14 15:34:33,900][75950] Updated weights for policy 1, policy_version 51510 (0.0008) -[2023-10-14 15:34:34,101][75949] Updated weights for policy 0, policy_version 51621 (0.0009) -[2023-10-14 15:34:34,268][75950] Updated weights for policy 1, policy_version 51520 (0.0008) -[2023-10-14 15:34:34,483][75949] Updated weights for policy 0, policy_version 51631 (0.0009) -[2023-10-14 15:34:34,861][75949] Updated weights for policy 0, policy_version 51641 (0.0008) -[2023-10-14 15:34:38,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105644032. Throughput: 0: 1689.2, 1: 1686.5. Samples: 26423328. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) -[2023-10-14 15:34:38,164][74987] Avg episode reward: [(0, '24.050'), (1, '30.960')] -[2023-10-14 15:34:38,389][75950] Updated weights for policy 1, policy_version 51530 (0.0010) -[2023-10-14 15:34:38,747][75949] Updated weights for policy 0, policy_version 51651 (0.0010) -[2023-10-14 15:34:38,757][75950] Updated weights for policy 1, policy_version 51540 (0.0009) -[2023-10-14 15:34:39,107][75949] Updated weights for policy 0, policy_version 51661 (0.0007) -[2023-10-14 15:34:39,120][75950] Updated weights for policy 1, policy_version 51550 (0.0009) -[2023-10-14 15:34:39,484][75949] Updated weights for policy 0, policy_version 51671 (0.0010) -[2023-10-14 15:34:43,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 105709568. Throughput: 0: 1696.9, 1: 1681.5. Samples: 26444126. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) -[2023-10-14 15:34:43,165][74987] Avg episode reward: [(0, '25.160'), (1, '29.120')] -[2023-10-14 15:34:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000051680_52920320.pth... -[2023-10-14 15:34:43,209][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000050112_51314688.pth -[2023-10-14 15:34:43,247][75950] Updated weights for policy 1, policy_version 51560 (0.0008) -[2023-10-14 15:34:43,535][75949] Updated weights for policy 0, policy_version 51681 (0.0010) -[2023-10-14 15:34:43,623][75950] Updated weights for policy 1, policy_version 51570 (0.0008) -[2023-10-14 15:34:43,895][75949] Updated weights for policy 0, policy_version 51691 (0.0009) -[2023-10-14 15:34:43,989][75950] Updated weights for policy 1, policy_version 51580 (0.0010) -[2023-10-14 15:34:44,134][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000051584_52822016.pth... -[2023-10-14 15:34:44,171][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000050016_51216384.pth -[2023-10-14 15:34:44,268][75949] Updated weights for policy 0, policy_version 51701 (0.0008) -[2023-10-14 15:34:44,635][75949] Updated weights for policy 0, policy_version 51711 (0.0009) -[2023-10-14 15:34:48,102][75950] Updated weights for policy 1, policy_version 51590 (0.0009) -[2023-10-14 15:34:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105775104. Throughput: 0: 1692.5, 1: 1680.5. Samples: 26453264. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) -[2023-10-14 15:34:48,164][74987] Avg episode reward: [(0, '23.160'), (1, '29.790')] -[2023-10-14 15:34:48,472][75950] Updated weights for policy 1, policy_version 51600 (0.0009) -[2023-10-14 15:34:48,632][75949] Updated weights for policy 0, policy_version 51721 (0.0009) -[2023-10-14 15:34:48,833][75950] Updated weights for policy 1, policy_version 51610 (0.0008) -[2023-10-14 15:34:49,005][75949] Updated weights for policy 0, policy_version 51731 (0.0009) -[2023-10-14 15:34:49,376][75949] Updated weights for policy 0, policy_version 51741 (0.0010) -[2023-10-14 15:34:53,056][75950] Updated weights for policy 1, policy_version 51620 (0.0009) -[2023-10-14 15:34:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105840640. Throughput: 0: 1699.4, 1: 1666.1. Samples: 26473426. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) -[2023-10-14 15:34:53,164][74987] Avg episode reward: [(0, '23.600'), (1, '31.490')] -[2023-10-14 15:34:53,361][75949] Updated weights for policy 0, policy_version 51751 (0.0009) -[2023-10-14 15:34:53,426][75950] Updated weights for policy 1, policy_version 51630 (0.0008) -[2023-10-14 15:34:53,731][75949] Updated weights for policy 0, policy_version 51761 (0.0009) -[2023-10-14 15:34:53,795][75950] Updated weights for policy 1, policy_version 51640 (0.0007) -[2023-10-14 15:34:54,087][75949] Updated weights for policy 0, policy_version 51771 (0.0008) -[2023-10-14 15:34:57,920][75950] Updated weights for policy 1, policy_version 51650 (0.0007) -[2023-10-14 15:34:58,145][75949] Updated weights for policy 0, policy_version 51781 (0.0008) -[2023-10-14 15:34:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105906176. Throughput: 0: 1693.7, 1: 1673.8. Samples: 26494302. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) -[2023-10-14 15:34:58,164][74987] Avg episode reward: [(0, '24.650'), (1, '31.660')] -[2023-10-14 15:34:58,285][75950] Updated weights for policy 1, policy_version 51660 (0.0007) -[2023-10-14 15:34:58,527][75949] Updated weights for policy 0, policy_version 51791 (0.0007) -[2023-10-14 15:34:58,653][75950] Updated weights for policy 1, policy_version 51670 (0.0007) -[2023-10-14 15:34:58,896][75949] Updated weights for policy 0, policy_version 51801 (0.0007) -[2023-10-14 15:34:59,021][75950] Updated weights for policy 1, policy_version 51680 (0.0008) -[2023-10-14 15:35:03,000][75949] Updated weights for policy 0, policy_version 51811 (0.0008) -[2023-10-14 15:35:03,079][75950] Updated weights for policy 1, policy_version 51690 (0.0007) -[2023-10-14 15:35:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105971712. Throughput: 0: 1693.8, 1: 1675.0. Samples: 26503352. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) -[2023-10-14 15:35:03,164][74987] Avg episode reward: [(0, '25.070'), (1, '30.670')] -[2023-10-14 15:35:03,370][75949] Updated weights for policy 0, policy_version 51821 (0.0009) -[2023-10-14 15:35:03,442][75950] Updated weights for policy 1, policy_version 51700 (0.0009) -[2023-10-14 15:35:03,736][75949] Updated weights for policy 0, policy_version 51831 (0.0009) -[2023-10-14 15:35:03,807][75950] Updated weights for policy 1, policy_version 51710 (0.0008) -[2023-10-14 15:35:07,946][75950] Updated weights for policy 1, policy_version 51720 (0.0008) -[2023-10-14 15:35:08,057][75949] Updated weights for policy 0, policy_version 51841 (0.0009) -[2023-10-14 15:35:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106037248. Throughput: 0: 1689.6, 1: 1667.7. Samples: 26523572. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) -[2023-10-14 15:35:08,165][74987] Avg episode reward: [(0, '25.750'), (1, '33.050')] -[2023-10-14 15:35:08,318][75950] Updated weights for policy 1, policy_version 51730 (0.0009) -[2023-10-14 15:35:08,422][75949] Updated weights for policy 0, policy_version 51851 (0.0008) -[2023-10-14 15:35:08,683][75950] Updated weights for policy 1, policy_version 51740 (0.0009) -[2023-10-14 15:35:08,793][75949] Updated weights for policy 0, policy_version 51861 (0.0007) -[2023-10-14 15:35:09,165][75949] Updated weights for policy 0, policy_version 51871 (0.0011) -[2023-10-14 15:35:12,777][75950] Updated weights for policy 1, policy_version 51750 (0.0009) -[2023-10-14 15:35:13,143][75950] Updated weights for policy 1, policy_version 51760 (0.0010) -[2023-10-14 15:35:13,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106102784. Throughput: 0: 1681.9, 1: 1661.8. Samples: 26543914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:35:13,164][74987] Avg episode reward: [(0, '26.470'), (1, '31.180')] -[2023-10-14 15:35:13,302][75949] Updated weights for policy 0, policy_version 51881 (0.0010) -[2023-10-14 15:35:13,515][75950] Updated weights for policy 1, policy_version 51770 (0.0008) -[2023-10-14 15:35:13,676][75949] Updated weights for policy 0, policy_version 51891 (0.0008) -[2023-10-14 15:35:14,048][75949] Updated weights for policy 0, policy_version 51901 (0.0009) -[2023-10-14 15:35:17,503][75950] Updated weights for policy 1, policy_version 51780 (0.0007) -[2023-10-14 15:35:17,877][75950] Updated weights for policy 1, policy_version 51790 (0.0009) -[2023-10-14 15:35:17,981][75949] Updated weights for policy 0, policy_version 51911 (0.0008) -[2023-10-14 15:35:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106168320. Throughput: 0: 1683.6, 1: 1663.3. Samples: 26553234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:35:18,164][74987] Avg episode reward: [(0, '27.360'), (1, '30.260')] -[2023-10-14 15:35:18,248][75950] Updated weights for policy 1, policy_version 51800 (0.0009) -[2023-10-14 15:35:18,347][75949] Updated weights for policy 0, policy_version 51921 (0.0008) -[2023-10-14 15:35:18,704][75949] Updated weights for policy 0, policy_version 51931 (0.0009) -[2023-10-14 15:35:22,258][75950] Updated weights for policy 1, policy_version 51810 (0.0009) -[2023-10-14 15:35:22,623][75950] Updated weights for policy 1, policy_version 51820 (0.0009) -[2023-10-14 15:35:22,822][75949] Updated weights for policy 0, policy_version 51941 (0.0008) -[2023-10-14 15:35:22,990][75950] Updated weights for policy 1, policy_version 51830 (0.0007) -[2023-10-14 15:35:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106233856. Throughput: 0: 1681.6, 1: 1663.5. Samples: 26573854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:35:23,164][74987] Avg episode reward: [(0, '25.350'), (1, '30.400')] -[2023-10-14 15:35:23,189][75949] Updated weights for policy 0, policy_version 51951 (0.0008) -[2023-10-14 15:35:23,355][75950] Updated weights for policy 1, policy_version 51840 (0.0007) -[2023-10-14 15:35:23,554][75949] Updated weights for policy 0, policy_version 51961 (0.0010) -[2023-10-14 15:35:27,414][75950] Updated weights for policy 1, policy_version 51850 (0.0010) -[2023-10-14 15:35:27,654][75949] Updated weights for policy 0, policy_version 51971 (0.0009) -[2023-10-14 15:35:27,785][75950] Updated weights for policy 1, policy_version 51860 (0.0009) -[2023-10-14 15:35:28,031][75949] Updated weights for policy 0, policy_version 51981 (0.0008) -[2023-10-14 15:35:28,141][75950] Updated weights for policy 1, policy_version 51870 (0.0008) -[2023-10-14 15:35:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 106299392. Throughput: 0: 1670.7, 1: 1653.1. Samples: 26593696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:35:28,165][74987] Avg episode reward: [(0, '27.500'), (1, '30.040')] -[2023-10-14 15:35:28,394][75949] Updated weights for policy 0, policy_version 51991 (0.0007) -[2023-10-14 15:35:32,297][75950] Updated weights for policy 1, policy_version 51880 (0.0008) -[2023-10-14 15:35:32,494][75949] Updated weights for policy 0, policy_version 52001 (0.0008) -[2023-10-14 15:35:32,658][75950] Updated weights for policy 1, policy_version 51890 (0.0009) -[2023-10-14 15:35:32,854][75949] Updated weights for policy 0, policy_version 52011 (0.0009) -[2023-10-14 15:35:33,028][75950] Updated weights for policy 1, policy_version 51900 (0.0008) -[2023-10-14 15:35:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106364928. Throughput: 0: 1672.5, 1: 1669.2. Samples: 26603640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:35:33,164][74987] Avg episode reward: [(0, '26.000'), (1, '30.620')] -[2023-10-14 15:35:33,231][75949] Updated weights for policy 0, policy_version 52021 (0.0008) -[2023-10-14 15:35:33,598][75949] Updated weights for policy 0, policy_version 52031 (0.0010) -[2023-10-14 15:35:37,068][75950] Updated weights for policy 1, policy_version 51910 (0.0008) -[2023-10-14 15:35:37,462][75950] Updated weights for policy 1, policy_version 51920 (0.0009) -[2023-10-14 15:35:37,693][75949] Updated weights for policy 0, policy_version 52041 (0.0007) -[2023-10-14 15:35:37,832][75950] Updated weights for policy 1, policy_version 51930 (0.0007) -[2023-10-14 15:35:38,072][75949] Updated weights for policy 0, policy_version 52051 (0.0007) -[2023-10-14 15:35:38,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 106463232. Throughput: 0: 1676.6, 1: 1679.5. Samples: 26624450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:35:38,165][74987] Avg episode reward: [(0, '27.310'), (1, '28.110')] -[2023-10-14 15:35:38,443][75949] Updated weights for policy 0, policy_version 52061 (0.0007) -[2023-10-14 15:35:41,974][75950] Updated weights for policy 1, policy_version 51940 (0.0007) -[2023-10-14 15:35:42,342][75950] Updated weights for policy 1, policy_version 51950 (0.0007) -[2023-10-14 15:35:42,393][75949] Updated weights for policy 0, policy_version 52071 (0.0010) -[2023-10-14 15:35:42,695][75950] Updated weights for policy 1, policy_version 51960 (0.0009) -[2023-10-14 15:35:42,763][75949] Updated weights for policy 0, policy_version 52081 (0.0009) -[2023-10-14 15:35:43,136][75949] Updated weights for policy 0, policy_version 52091 (0.0010) -[2023-10-14 15:35:43,163][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 106528768. Throughput: 0: 1661.1, 1: 1658.0. Samples: 26643662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:35:43,164][74987] Avg episode reward: [(0, '23.750'), (1, '30.060')] -[2023-10-14 15:35:46,602][75950] Updated weights for policy 1, policy_version 51970 (0.0009) -[2023-10-14 15:35:46,977][75950] Updated weights for policy 1, policy_version 51980 (0.0007) -[2023-10-14 15:35:47,337][75949] Updated weights for policy 0, policy_version 52101 (0.0008) -[2023-10-14 15:35:47,340][75950] Updated weights for policy 1, policy_version 51990 (0.0007) -[2023-10-14 15:35:47,707][75950] Updated weights for policy 1, policy_version 52000 (0.0010) -[2023-10-14 15:35:47,712][75949] Updated weights for policy 0, policy_version 52111 (0.0007) -[2023-10-14 15:35:48,074][75949] Updated weights for policy 0, policy_version 52121 (0.0008) -[2023-10-14 15:35:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 106594304. Throughput: 0: 1671.8, 1: 1683.3. Samples: 26654330. Policy #0 lag: (min: 16.0, avg: 38.2, max: 48.0) -[2023-10-14 15:35:48,164][74987] Avg episode reward: [(0, '27.060'), (1, '31.750')] -[2023-10-14 15:35:51,962][75949] Updated weights for policy 0, policy_version 52131 (0.0008) -[2023-10-14 15:35:52,009][75950] Updated weights for policy 1, policy_version 52010 (0.0008) -[2023-10-14 15:35:52,342][75949] Updated weights for policy 0, policy_version 52141 (0.0008) -[2023-10-14 15:35:52,376][75950] Updated weights for policy 1, policy_version 52020 (0.0008) -[2023-10-14 15:35:52,706][75949] Updated weights for policy 0, policy_version 52151 (0.0007) -[2023-10-14 15:35:52,743][75950] Updated weights for policy 1, policy_version 52030 (0.0009) -[2023-10-14 15:35:53,163][74987] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 106692608. Throughput: 0: 1680.3, 1: 1681.9. Samples: 26674870. Policy #0 lag: (min: 16.0, avg: 38.2, max: 48.0) -[2023-10-14 15:35:53,164][74987] Avg episode reward: [(0, '23.920'), (1, '29.380')] -[2023-10-14 15:35:56,771][75950] Updated weights for policy 1, policy_version 52040 (0.0008) -[2023-10-14 15:35:56,827][75949] Updated weights for policy 0, policy_version 52161 (0.0009) -[2023-10-14 15:35:57,143][75950] Updated weights for policy 1, policy_version 52050 (0.0008) -[2023-10-14 15:35:57,190][75949] Updated weights for policy 0, policy_version 52171 (0.0009) -[2023-10-14 15:35:57,505][75950] Updated weights for policy 1, policy_version 52060 (0.0007) -[2023-10-14 15:35:57,562][75949] Updated weights for policy 0, policy_version 52181 (0.0009) -[2023-10-14 15:35:57,932][75949] Updated weights for policy 0, policy_version 52191 (0.0008) -[2023-10-14 15:35:58,163][74987] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 106758144. Throughput: 0: 1662.9, 1: 1661.4. Samples: 26693508. Policy #0 lag: (min: 16.0, avg: 38.2, max: 48.0) -[2023-10-14 15:35:58,164][74987] Avg episode reward: [(0, '25.790'), (1, '30.550')] -[2023-10-14 15:36:01,584][75950] Updated weights for policy 1, policy_version 52070 (0.0007) -[2023-10-14 15:36:01,945][75950] Updated weights for policy 1, policy_version 52080 (0.0007) -[2023-10-14 15:36:01,958][75949] Updated weights for policy 0, policy_version 52201 (0.0007) -[2023-10-14 15:36:02,315][75950] Updated weights for policy 1, policy_version 52090 (0.0009) -[2023-10-14 15:36:02,330][75949] Updated weights for policy 0, policy_version 52211 (0.0008) -[2023-10-14 15:36:02,710][75949] Updated weights for policy 0, policy_version 52221 (0.0010) -[2023-10-14 15:36:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 106823680. Throughput: 0: 1681.7, 1: 1687.5. Samples: 26704850. Policy #0 lag: (min: 16.0, avg: 38.2, max: 48.0) -[2023-10-14 15:36:03,165][74987] Avg episode reward: [(0, '24.250'), (1, '30.070')] -[2023-10-14 15:36:06,339][75950] Updated weights for policy 1, policy_version 52100 (0.0010) -[2023-10-14 15:36:06,697][75950] Updated weights for policy 1, policy_version 52110 (0.0009) -[2023-10-14 15:36:06,769][75949] Updated weights for policy 0, policy_version 52231 (0.0010) -[2023-10-14 15:36:07,064][75950] Updated weights for policy 1, policy_version 52120 (0.0008) -[2023-10-14 15:36:07,142][75949] Updated weights for policy 0, policy_version 52241 (0.0009) -[2023-10-14 15:36:07,507][75949] Updated weights for policy 0, policy_version 52251 (0.0007) -[2023-10-14 15:36:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 106889216. Throughput: 0: 1677.5, 1: 1676.0. Samples: 26724758. Policy #0 lag: (min: 16.0, avg: 38.2, max: 48.0) -[2023-10-14 15:36:08,164][74987] Avg episode reward: [(0, '25.380'), (1, '28.720')] -[2023-10-14 15:36:11,078][75950] Updated weights for policy 1, policy_version 52130 (0.0008) -[2023-10-14 15:36:11,446][75950] Updated weights for policy 1, policy_version 52140 (0.0010) -[2023-10-14 15:36:11,594][75949] Updated weights for policy 0, policy_version 52261 (0.0009) -[2023-10-14 15:36:11,797][75950] Updated weights for policy 1, policy_version 52150 (0.0007) -[2023-10-14 15:36:11,964][75949] Updated weights for policy 0, policy_version 52271 (0.0007) -[2023-10-14 15:36:12,164][75950] Updated weights for policy 1, policy_version 52160 (0.0008) -[2023-10-14 15:36:12,340][75949] Updated weights for policy 0, policy_version 52281 (0.0009) -[2023-10-14 15:36:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 106954752. Throughput: 0: 1659.0, 1: 1672.3. Samples: 26743604. Policy #0 lag: (min: 16.0, avg: 38.2, max: 48.0) -[2023-10-14 15:36:13,165][74987] Avg episode reward: [(0, '24.370'), (1, '29.750')] -[2023-10-14 15:36:16,186][75950] Updated weights for policy 1, policy_version 52170 (0.0008) -[2023-10-14 15:36:16,391][75949] Updated weights for policy 0, policy_version 52291 (0.0007) -[2023-10-14 15:36:16,549][75950] Updated weights for policy 1, policy_version 52180 (0.0007) -[2023-10-14 15:36:16,759][75949] Updated weights for policy 0, policy_version 52301 (0.0008) -[2023-10-14 15:36:16,901][75950] Updated weights for policy 1, policy_version 52190 (0.0008) -[2023-10-14 15:36:17,123][75949] Updated weights for policy 0, policy_version 52311 (0.0009) -[2023-10-14 15:36:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 107020288. Throughput: 0: 1680.2, 1: 1687.6. Samples: 26755194. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:36:18,165][74987] Avg episode reward: [(0, '25.740'), (1, '32.400')] -[2023-10-14 15:36:21,116][75950] Updated weights for policy 1, policy_version 52200 (0.0007) -[2023-10-14 15:36:21,307][75949] Updated weights for policy 0, policy_version 52321 (0.0011) -[2023-10-14 15:36:21,472][75950] Updated weights for policy 1, policy_version 52210 (0.0009) -[2023-10-14 15:36:21,676][75949] Updated weights for policy 0, policy_version 52331 (0.0007) -[2023-10-14 15:36:21,847][75950] Updated weights for policy 1, policy_version 52220 (0.0008) -[2023-10-14 15:36:22,030][75949] Updated weights for policy 0, policy_version 52341 (0.0009) -[2023-10-14 15:36:22,408][75949] Updated weights for policy 0, policy_version 52351 (0.0009) -[2023-10-14 15:36:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 107085824. Throughput: 0: 1669.6, 1: 1665.7. Samples: 26774540. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:36:23,164][74987] Avg episode reward: [(0, '25.640'), (1, '28.890')] -[2023-10-14 15:36:26,144][75950] Updated weights for policy 1, policy_version 52230 (0.0008) -[2023-10-14 15:36:26,538][75950] Updated weights for policy 1, policy_version 52240 (0.0008) -[2023-10-14 15:36:26,591][75949] Updated weights for policy 0, policy_version 52361 (0.0008) -[2023-10-14 15:36:26,910][75950] Updated weights for policy 1, policy_version 52250 (0.0009) -[2023-10-14 15:36:26,961][75949] Updated weights for policy 0, policy_version 52371 (0.0009) -[2023-10-14 15:36:27,330][75949] Updated weights for policy 0, policy_version 52381 (0.0010) -[2023-10-14 15:36:28,163][74987] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 107151360. Throughput: 0: 1662.7, 1: 1672.7. Samples: 26793756. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:36:28,164][74987] Avg episode reward: [(0, '25.030'), (1, '29.960')] -[2023-10-14 15:36:31,118][75950] Updated weights for policy 1, policy_version 52260 (0.0007) -[2023-10-14 15:36:31,200][75949] Updated weights for policy 0, policy_version 52391 (0.0009) -[2023-10-14 15:36:31,485][75950] Updated weights for policy 1, policy_version 52270 (0.0008) -[2023-10-14 15:36:31,560][75949] Updated weights for policy 0, policy_version 52401 (0.0010) -[2023-10-14 15:36:31,847][75950] Updated weights for policy 1, policy_version 52280 (0.0009) -[2023-10-14 15:36:31,932][75949] Updated weights for policy 0, policy_version 52411 (0.0007) -[2023-10-14 15:36:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 107216896. Throughput: 0: 1683.9, 1: 1672.6. Samples: 26805372. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:36:33,164][74987] Avg episode reward: [(0, '25.730'), (1, '31.670')] -[2023-10-14 15:36:35,850][75950] Updated weights for policy 1, policy_version 52290 (0.0009) -[2023-10-14 15:36:35,986][75949] Updated weights for policy 0, policy_version 52421 (0.0007) -[2023-10-14 15:36:36,209][75950] Updated weights for policy 1, policy_version 52300 (0.0008) -[2023-10-14 15:36:36,353][75949] Updated weights for policy 0, policy_version 52431 (0.0009) -[2023-10-14 15:36:36,571][75950] Updated weights for policy 1, policy_version 52310 (0.0007) -[2023-10-14 15:36:36,718][75949] Updated weights for policy 0, policy_version 52441 (0.0008) -[2023-10-14 15:36:36,937][75950] Updated weights for policy 1, policy_version 52320 (0.0009) -[2023-10-14 15:36:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 107282432. Throughput: 0: 1661.7, 1: 1656.8. Samples: 26824206. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:36:38,164][74987] Avg episode reward: [(0, '26.130'), (1, '30.300')] -[2023-10-14 15:36:40,886][75949] Updated weights for policy 0, policy_version 52451 (0.0008) -[2023-10-14 15:36:41,061][75950] Updated weights for policy 1, policy_version 52330 (0.0008) -[2023-10-14 15:36:41,254][75949] Updated weights for policy 0, policy_version 52461 (0.0010) -[2023-10-14 15:36:41,423][75950] Updated weights for policy 1, policy_version 52340 (0.0008) -[2023-10-14 15:36:41,618][75949] Updated weights for policy 0, policy_version 52471 (0.0010) -[2023-10-14 15:36:41,789][75950] Updated weights for policy 1, policy_version 52350 (0.0009) -[2023-10-14 15:36:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 107347968. Throughput: 0: 1670.1, 1: 1673.8. Samples: 26843984. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:36:43,164][74987] Avg episode reward: [(0, '24.320'), (1, '30.540')] -[2023-10-14 15:36:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000052352_53608448.pth... -[2023-10-14 15:36:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000052480_53739520.pth... -[2023-10-14 15:36:43,211][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000050784_52002816.pth -[2023-10-14 15:36:43,212][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000050912_52133888.pth -[2023-10-14 15:36:45,703][75949] Updated weights for policy 0, policy_version 52481 (0.0010) -[2023-10-14 15:36:45,852][75950] Updated weights for policy 1, policy_version 52360 (0.0009) -[2023-10-14 15:36:46,076][75949] Updated weights for policy 0, policy_version 52491 (0.0009) -[2023-10-14 15:36:46,214][75950] Updated weights for policy 1, policy_version 52370 (0.0008) -[2023-10-14 15:36:46,447][75949] Updated weights for policy 0, policy_version 52501 (0.0009) -[2023-10-14 15:36:46,582][75950] Updated weights for policy 1, policy_version 52380 (0.0007) -[2023-10-14 15:36:46,819][75949] Updated weights for policy 0, policy_version 52511 (0.0009) -[2023-10-14 15:36:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 107413504. Throughput: 0: 1679.2, 1: 1667.6. Samples: 26855458. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:36:48,164][74987] Avg episode reward: [(0, '25.870'), (1, '31.250')] -[2023-10-14 15:36:50,639][75950] Updated weights for policy 1, policy_version 52390 (0.0009) -[2023-10-14 15:36:50,983][75949] Updated weights for policy 0, policy_version 52521 (0.0008) -[2023-10-14 15:36:51,004][75950] Updated weights for policy 1, policy_version 52400 (0.0007) -[2023-10-14 15:36:51,342][75949] Updated weights for policy 0, policy_version 52531 (0.0010) -[2023-10-14 15:36:51,369][75950] Updated weights for policy 1, policy_version 52410 (0.0009) -[2023-10-14 15:36:51,709][75949] Updated weights for policy 0, policy_version 52541 (0.0009) -[2023-10-14 15:36:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 107479040. Throughput: 0: 1659.8, 1: 1655.1. Samples: 26873928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:36:53,165][74987] Avg episode reward: [(0, '24.530'), (1, '29.550')] -[2023-10-14 15:36:55,630][75950] Updated weights for policy 1, policy_version 52420 (0.0009) -[2023-10-14 15:36:55,804][75949] Updated weights for policy 0, policy_version 52551 (0.0008) -[2023-10-14 15:36:56,000][75950] Updated weights for policy 1, policy_version 52430 (0.0009) -[2023-10-14 15:36:56,173][75949] Updated weights for policy 0, policy_version 52561 (0.0009) -[2023-10-14 15:36:56,369][75950] Updated weights for policy 1, policy_version 52440 (0.0009) -[2023-10-14 15:36:56,554][75949] Updated weights for policy 0, policy_version 52571 (0.0008) -[2023-10-14 15:36:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 107544576. Throughput: 0: 1679.7, 1: 1676.4. Samples: 26894630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:36:58,165][74987] Avg episode reward: [(0, '27.000'), (1, '29.600')] -[2023-10-14 15:37:00,203][75950] Updated weights for policy 1, policy_version 52450 (0.0010) -[2023-10-14 15:37:00,572][75950] Updated weights for policy 1, policy_version 52460 (0.0007) -[2023-10-14 15:37:00,589][75949] Updated weights for policy 0, policy_version 52581 (0.0009) -[2023-10-14 15:37:00,949][75950] Updated weights for policy 1, policy_version 52470 (0.0009) -[2023-10-14 15:37:00,962][75949] Updated weights for policy 0, policy_version 52591 (0.0009) -[2023-10-14 15:37:01,314][75950] Updated weights for policy 1, policy_version 52480 (0.0009) -[2023-10-14 15:37:01,327][75949] Updated weights for policy 0, policy_version 52601 (0.0009) -[2023-10-14 15:37:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 107610112. Throughput: 0: 1676.4, 1: 1667.2. Samples: 26905656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:37:03,165][74987] Avg episode reward: [(0, '25.520'), (1, '31.030')] -[2023-10-14 15:37:05,385][75949] Updated weights for policy 0, policy_version 52611 (0.0010) -[2023-10-14 15:37:05,481][75950] Updated weights for policy 1, policy_version 52490 (0.0008) -[2023-10-14 15:37:05,753][75949] Updated weights for policy 0, policy_version 52621 (0.0009) -[2023-10-14 15:37:05,856][75950] Updated weights for policy 1, policy_version 52500 (0.0007) -[2023-10-14 15:37:06,123][75949] Updated weights for policy 0, policy_version 52631 (0.0009) -[2023-10-14 15:37:06,222][75950] Updated weights for policy 1, policy_version 52510 (0.0008) -[2023-10-14 15:37:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 107675648. Throughput: 0: 1663.8, 1: 1672.0. Samples: 26924648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:37:08,164][74987] Avg episode reward: [(0, '26.940'), (1, '29.480')] -[2023-10-14 15:37:10,074][75949] Updated weights for policy 0, policy_version 52641 (0.0009) -[2023-10-14 15:37:10,191][75950] Updated weights for policy 1, policy_version 52520 (0.0007) -[2023-10-14 15:37:10,433][75949] Updated weights for policy 0, policy_version 52651 (0.0008) -[2023-10-14 15:37:10,556][75950] Updated weights for policy 1, policy_version 52530 (0.0007) -[2023-10-14 15:37:10,806][75949] Updated weights for policy 0, policy_version 52661 (0.0007) -[2023-10-14 15:37:10,924][75950] Updated weights for policy 1, policy_version 52540 (0.0008) -[2023-10-14 15:37:11,180][75949] Updated weights for policy 0, policy_version 52671 (0.0007) -[2023-10-14 15:37:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 107741184. Throughput: 0: 1689.4, 1: 1682.6. Samples: 26945494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:37:13,165][74987] Avg episode reward: [(0, '25.350'), (1, '30.160')] -[2023-10-14 15:37:15,140][75949] Updated weights for policy 0, policy_version 52681 (0.0008) -[2023-10-14 15:37:15,235][75950] Updated weights for policy 1, policy_version 52550 (0.0008) -[2023-10-14 15:37:15,509][75949] Updated weights for policy 0, policy_version 52691 (0.0008) -[2023-10-14 15:37:15,619][75950] Updated weights for policy 1, policy_version 52560 (0.0009) -[2023-10-14 15:37:15,876][75949] Updated weights for policy 0, policy_version 52701 (0.0007) -[2023-10-14 15:37:15,990][75950] Updated weights for policy 1, policy_version 52570 (0.0008) -[2023-10-14 15:37:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 107806720. Throughput: 0: 1664.1, 1: 1666.6. Samples: 26955254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:37:18,164][74987] Avg episode reward: [(0, '26.720'), (1, '30.430')] -[2023-10-14 15:37:19,890][75949] Updated weights for policy 0, policy_version 52711 (0.0010) -[2023-10-14 15:37:20,020][75950] Updated weights for policy 1, policy_version 52580 (0.0009) -[2023-10-14 15:37:20,255][75949] Updated weights for policy 0, policy_version 52721 (0.0008) -[2023-10-14 15:37:20,387][75950] Updated weights for policy 1, policy_version 52590 (0.0008) -[2023-10-14 15:37:20,620][75949] Updated weights for policy 0, policy_version 52731 (0.0007) -[2023-10-14 15:37:20,753][75950] Updated weights for policy 1, policy_version 52600 (0.0009) -[2023-10-14 15:37:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 107872256. Throughput: 0: 1676.6, 1: 1671.2. Samples: 26974854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:37:23,164][74987] Avg episode reward: [(0, '23.860'), (1, '28.850')] -[2023-10-14 15:37:24,720][75949] Updated weights for policy 0, policy_version 52741 (0.0007) -[2023-10-14 15:37:24,878][75950] Updated weights for policy 1, policy_version 52610 (0.0009) -[2023-10-14 15:37:25,089][75949] Updated weights for policy 0, policy_version 52751 (0.0008) -[2023-10-14 15:37:25,233][75950] Updated weights for policy 1, policy_version 52620 (0.0007) -[2023-10-14 15:37:25,467][75949] Updated weights for policy 0, policy_version 52761 (0.0007) -[2023-10-14 15:37:25,612][75950] Updated weights for policy 1, policy_version 52630 (0.0008) -[2023-10-14 15:37:25,977][75950] Updated weights for policy 1, policy_version 52640 (0.0009) -[2023-10-14 15:37:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 107937792. Throughput: 0: 1690.1, 1: 1671.8. Samples: 26995272. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 15:37:28,164][74987] Avg episode reward: [(0, '26.650'), (1, '27.210')] -[2023-10-14 15:37:29,484][75949] Updated weights for policy 0, policy_version 52771 (0.0009) -[2023-10-14 15:37:29,852][75949] Updated weights for policy 0, policy_version 52781 (0.0010) -[2023-10-14 15:37:30,174][75950] Updated weights for policy 1, policy_version 52650 (0.0009) -[2023-10-14 15:37:30,218][75949] Updated weights for policy 0, policy_version 52791 (0.0008) -[2023-10-14 15:37:30,545][75950] Updated weights for policy 1, policy_version 52660 (0.0010) -[2023-10-14 15:37:30,910][75950] Updated weights for policy 1, policy_version 52670 (0.0010) -[2023-10-14 15:37:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108003328. Throughput: 0: 1662.8, 1: 1654.0. Samples: 27004716. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 15:37:33,164][74987] Avg episode reward: [(0, '25.930'), (1, '28.400')] -[2023-10-14 15:37:34,396][75949] Updated weights for policy 0, policy_version 52801 (0.0007) -[2023-10-14 15:37:34,766][75949] Updated weights for policy 0, policy_version 52811 (0.0010) -[2023-10-14 15:37:34,943][75950] Updated weights for policy 1, policy_version 52680 (0.0007) -[2023-10-14 15:37:35,130][75949] Updated weights for policy 0, policy_version 52821 (0.0010) -[2023-10-14 15:37:35,313][75950] Updated weights for policy 1, policy_version 52690 (0.0009) -[2023-10-14 15:37:35,500][75949] Updated weights for policy 0, policy_version 52831 (0.0007) -[2023-10-14 15:37:35,677][75950] Updated weights for policy 1, policy_version 52700 (0.0007) -[2023-10-14 15:37:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 108068864. Throughput: 0: 1683.9, 1: 1669.4. Samples: 27024824. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 15:37:38,165][74987] Avg episode reward: [(0, '23.220'), (1, '28.820')] -[2023-10-14 15:37:39,556][75949] Updated weights for policy 0, policy_version 52841 (0.0010) -[2023-10-14 15:37:39,830][75950] Updated weights for policy 1, policy_version 52710 (0.0008) -[2023-10-14 15:37:39,922][75949] Updated weights for policy 0, policy_version 52851 (0.0008) -[2023-10-14 15:37:40,195][75950] Updated weights for policy 1, policy_version 52720 (0.0007) -[2023-10-14 15:37:40,285][75949] Updated weights for policy 0, policy_version 52861 (0.0007) -[2023-10-14 15:37:40,557][75950] Updated weights for policy 1, policy_version 52730 (0.0007) -[2023-10-14 15:37:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108134400. Throughput: 0: 1692.3, 1: 1667.7. Samples: 27045830. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 15:37:43,165][74987] Avg episode reward: [(0, '25.350'), (1, '28.990')] -[2023-10-14 15:37:44,365][75949] Updated weights for policy 0, policy_version 52871 (0.0008) -[2023-10-14 15:37:44,484][75950] Updated weights for policy 1, policy_version 52740 (0.0008) -[2023-10-14 15:37:44,736][75949] Updated weights for policy 0, policy_version 52881 (0.0009) -[2023-10-14 15:37:44,843][75950] Updated weights for policy 1, policy_version 52750 (0.0008) -[2023-10-14 15:37:45,104][75949] Updated weights for policy 0, policy_version 52891 (0.0009) -[2023-10-14 15:37:45,224][75950] Updated weights for policy 1, policy_version 52760 (0.0009) -[2023-10-14 15:37:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108199936. Throughput: 0: 1666.6, 1: 1647.6. Samples: 27054792. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 15:37:48,165][74987] Avg episode reward: [(0, '24.730'), (1, '30.510')] -[2023-10-14 15:37:49,160][75949] Updated weights for policy 0, policy_version 52901 (0.0009) -[2023-10-14 15:37:49,378][75950] Updated weights for policy 1, policy_version 52770 (0.0009) -[2023-10-14 15:37:49,525][75949] Updated weights for policy 0, policy_version 52911 (0.0010) -[2023-10-14 15:37:49,747][75950] Updated weights for policy 1, policy_version 52780 (0.0009) -[2023-10-14 15:37:49,900][75949] Updated weights for policy 0, policy_version 52921 (0.0008) -[2023-10-14 15:37:50,121][75950] Updated weights for policy 1, policy_version 52790 (0.0008) -[2023-10-14 15:37:50,478][75950] Updated weights for policy 1, policy_version 52800 (0.0007) -[2023-10-14 15:37:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108265472. Throughput: 0: 1687.1, 1: 1662.1. Samples: 27075360. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 15:37:53,164][74987] Avg episode reward: [(0, '28.780'), (1, '30.800')] -[2023-10-14 15:37:53,165][75615] Saving new best policy, reward=28.780! -[2023-10-14 15:37:53,873][75949] Updated weights for policy 0, policy_version 52931 (0.0007) -[2023-10-14 15:37:54,249][75949] Updated weights for policy 0, policy_version 52941 (0.0010) -[2023-10-14 15:37:54,623][75949] Updated weights for policy 0, policy_version 52951 (0.0008) -[2023-10-14 15:37:54,623][75950] Updated weights for policy 1, policy_version 52810 (0.0008) -[2023-10-14 15:37:55,000][75950] Updated weights for policy 1, policy_version 52820 (0.0009) -[2023-10-14 15:37:55,358][75950] Updated weights for policy 1, policy_version 52830 (0.0009) -[2023-10-14 15:37:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108331008. Throughput: 0: 1682.1, 1: 1666.2. Samples: 27096168. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 15:37:58,164][74987] Avg episode reward: [(0, '23.880'), (1, '31.600')] -[2023-10-14 15:37:58,720][75949] Updated weights for policy 0, policy_version 52961 (0.0008) -[2023-10-14 15:37:59,085][75949] Updated weights for policy 0, policy_version 52971 (0.0010) -[2023-10-14 15:37:59,441][75950] Updated weights for policy 1, policy_version 52840 (0.0007) -[2023-10-14 15:37:59,460][75949] Updated weights for policy 0, policy_version 52981 (0.0008) -[2023-10-14 15:37:59,821][75950] Updated weights for policy 1, policy_version 52850 (0.0008) -[2023-10-14 15:37:59,832][75949] Updated weights for policy 0, policy_version 52991 (0.0009) -[2023-10-14 15:38:00,188][75950] Updated weights for policy 1, policy_version 52860 (0.0009) -[2023-10-14 15:38:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 108396544. Throughput: 0: 1675.9, 1: 1653.5. Samples: 27105076. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:38:03,165][74987] Avg episode reward: [(0, '28.820'), (1, '31.050')] -[2023-10-14 15:38:03,166][75615] Saving new best policy, reward=28.820! -[2023-10-14 15:38:04,157][75949] Updated weights for policy 0, policy_version 53001 (0.0009) -[2023-10-14 15:38:04,320][75950] Updated weights for policy 1, policy_version 52870 (0.0009) -[2023-10-14 15:38:04,533][75949] Updated weights for policy 0, policy_version 53011 (0.0009) -[2023-10-14 15:38:04,692][75950] Updated weights for policy 1, policy_version 52880 (0.0008) -[2023-10-14 15:38:04,904][75949] Updated weights for policy 0, policy_version 53021 (0.0008) -[2023-10-14 15:38:05,068][75950] Updated weights for policy 1, policy_version 52890 (0.0008) -[2023-10-14 15:38:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108462080. Throughput: 0: 1676.4, 1: 1666.5. Samples: 27125282. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:38:08,164][74987] Avg episode reward: [(0, '24.570'), (1, '32.610')] -[2023-10-14 15:38:09,005][75949] Updated weights for policy 0, policy_version 53031 (0.0009) -[2023-10-14 15:38:09,152][75950] Updated weights for policy 1, policy_version 52900 (0.0008) -[2023-10-14 15:38:09,375][75949] Updated weights for policy 0, policy_version 53041 (0.0007) -[2023-10-14 15:38:09,524][75950] Updated weights for policy 1, policy_version 52910 (0.0008) -[2023-10-14 15:38:09,743][75949] Updated weights for policy 0, policy_version 53051 (0.0007) -[2023-10-14 15:38:09,888][75950] Updated weights for policy 1, policy_version 52920 (0.0007) -[2023-10-14 15:38:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 108527616. Throughput: 0: 1677.2, 1: 1668.3. Samples: 27145818. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:38:13,164][74987] Avg episode reward: [(0, '28.310'), (1, '30.430')] -[2023-10-14 15:38:13,883][75949] Updated weights for policy 0, policy_version 53061 (0.0009) -[2023-10-14 15:38:14,122][75950] Updated weights for policy 1, policy_version 52930 (0.0007) -[2023-10-14 15:38:14,266][75949] Updated weights for policy 0, policy_version 53071 (0.0008) -[2023-10-14 15:38:14,486][75950] Updated weights for policy 1, policy_version 52940 (0.0008) -[2023-10-14 15:38:14,622][75949] Updated weights for policy 0, policy_version 53081 (0.0009) -[2023-10-14 15:38:14,861][75950] Updated weights for policy 1, policy_version 52950 (0.0008) -[2023-10-14 15:38:15,226][75950] Updated weights for policy 1, policy_version 52960 (0.0007) -[2023-10-14 15:38:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108593152. Throughput: 0: 1673.2, 1: 1661.2. Samples: 27154764. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:38:18,165][74987] Avg episode reward: [(0, '23.910'), (1, '29.000')] -[2023-10-14 15:38:18,743][75949] Updated weights for policy 0, policy_version 53091 (0.0009) -[2023-10-14 15:38:19,114][75949] Updated weights for policy 0, policy_version 53101 (0.0009) -[2023-10-14 15:38:19,436][75950] Updated weights for policy 1, policy_version 52970 (0.0009) -[2023-10-14 15:38:19,481][75949] Updated weights for policy 0, policy_version 53111 (0.0008) -[2023-10-14 15:38:19,800][75950] Updated weights for policy 1, policy_version 52980 (0.0010) -[2023-10-14 15:38:20,171][75950] Updated weights for policy 1, policy_version 52990 (0.0008) -[2023-10-14 15:38:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108658688. Throughput: 0: 1678.7, 1: 1666.8. Samples: 27175368. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:38:23,165][74987] Avg episode reward: [(0, '28.310'), (1, '28.930')] -[2023-10-14 15:38:23,658][75949] Updated weights for policy 0, policy_version 53121 (0.0009) -[2023-10-14 15:38:24,028][75949] Updated weights for policy 0, policy_version 53131 (0.0009) -[2023-10-14 15:38:24,249][75950] Updated weights for policy 1, policy_version 53000 (0.0007) -[2023-10-14 15:38:24,392][75949] Updated weights for policy 0, policy_version 53141 (0.0009) -[2023-10-14 15:38:24,610][75950] Updated weights for policy 1, policy_version 53010 (0.0008) -[2023-10-14 15:38:24,758][75949] Updated weights for policy 0, policy_version 53151 (0.0010) -[2023-10-14 15:38:24,977][75950] Updated weights for policy 1, policy_version 53020 (0.0010) -[2023-10-14 15:38:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108724224. Throughput: 0: 1669.9, 1: 1665.7. Samples: 27195932. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:38:28,165][74987] Avg episode reward: [(0, '24.170'), (1, '29.960')] -[2023-10-14 15:38:28,882][75949] Updated weights for policy 0, policy_version 53161 (0.0010) -[2023-10-14 15:38:28,956][75950] Updated weights for policy 1, policy_version 53030 (0.0010) -[2023-10-14 15:38:29,245][75949] Updated weights for policy 0, policy_version 53171 (0.0011) -[2023-10-14 15:38:29,317][75950] Updated weights for policy 1, policy_version 53040 (0.0007) -[2023-10-14 15:38:29,626][75949] Updated weights for policy 0, policy_version 53181 (0.0009) -[2023-10-14 15:38:29,689][75950] Updated weights for policy 1, policy_version 53050 (0.0008) -[2023-10-14 15:38:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108789760. Throughput: 0: 1673.1, 1: 1664.2. Samples: 27204968. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:38:33,164][74987] Avg episode reward: [(0, '25.470'), (1, '30.130')] -[2023-10-14 15:38:33,551][75949] Updated weights for policy 0, policy_version 53191 (0.0010) -[2023-10-14 15:38:33,892][75950] Updated weights for policy 1, policy_version 53060 (0.0009) -[2023-10-14 15:38:33,912][75949] Updated weights for policy 0, policy_version 53201 (0.0011) -[2023-10-14 15:38:34,249][75950] Updated weights for policy 1, policy_version 53070 (0.0007) -[2023-10-14 15:38:34,288][75949] Updated weights for policy 0, policy_version 53211 (0.0009) -[2023-10-14 15:38:34,617][75950] Updated weights for policy 1, policy_version 53080 (0.0009) -[2023-10-14 15:38:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108855296. Throughput: 0: 1678.7, 1: 1662.4. Samples: 27225710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:38:38,164][74987] Avg episode reward: [(0, '27.110'), (1, '28.830')] -[2023-10-14 15:38:38,291][75949] Updated weights for policy 0, policy_version 53221 (0.0007) -[2023-10-14 15:38:38,664][75949] Updated weights for policy 0, policy_version 53231 (0.0009) -[2023-10-14 15:38:38,873][75950] Updated weights for policy 1, policy_version 53090 (0.0009) -[2023-10-14 15:38:39,038][75949] Updated weights for policy 0, policy_version 53241 (0.0008) -[2023-10-14 15:38:39,247][75950] Updated weights for policy 1, policy_version 53100 (0.0008) -[2023-10-14 15:38:39,607][75950] Updated weights for policy 1, policy_version 53110 (0.0010) -[2023-10-14 15:38:39,973][75950] Updated weights for policy 1, policy_version 53120 (0.0008) -[2023-10-14 15:38:43,086][75949] Updated weights for policy 0, policy_version 53251 (0.0010) -[2023-10-14 15:38:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 108920832. Throughput: 0: 1680.5, 1: 1661.9. Samples: 27246578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:38:43,165][74987] Avg episode reward: [(0, '26.290'), (1, '30.470')] -[2023-10-14 15:38:43,177][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000053120_54394880.pth... -[2023-10-14 15:38:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000051584_52822016.pth -[2023-10-14 15:38:43,455][75949] Updated weights for policy 0, policy_version 53261 (0.0009) -[2023-10-14 15:38:43,833][75949] Updated weights for policy 0, policy_version 53271 (0.0008) -[2023-10-14 15:38:44,155][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000053280_54558720.pth... -[2023-10-14 15:38:44,169][75950] Updated weights for policy 1, policy_version 53130 (0.0009) -[2023-10-14 15:38:44,185][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000051680_52920320.pth -[2023-10-14 15:38:44,538][75950] Updated weights for policy 1, policy_version 53140 (0.0007) -[2023-10-14 15:38:44,905][75950] Updated weights for policy 1, policy_version 53150 (0.0007) -[2023-10-14 15:38:47,927][75949] Updated weights for policy 0, policy_version 53281 (0.0008) -[2023-10-14 15:38:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108986368. Throughput: 0: 1679.2, 1: 1666.0. Samples: 27255610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:38:48,164][74987] Avg episode reward: [(0, '29.280'), (1, '30.670')] -[2023-10-14 15:38:48,316][75949] Updated weights for policy 0, policy_version 53291 (0.0010) -[2023-10-14 15:38:48,694][75949] Updated weights for policy 0, policy_version 53301 (0.0008) -[2023-10-14 15:38:48,803][75950] Updated weights for policy 1, policy_version 53160 (0.0008) -[2023-10-14 15:38:49,079][75949] Updated weights for policy 0, policy_version 53311 (0.0009) -[2023-10-14 15:38:49,110][75615] Saving new best policy, reward=29.280! -[2023-10-14 15:38:49,180][75950] Updated weights for policy 1, policy_version 53170 (0.0008) -[2023-10-14 15:38:49,535][75950] Updated weights for policy 1, policy_version 53180 (0.0008) -[2023-10-14 15:38:53,111][75949] Updated weights for policy 0, policy_version 53321 (0.0009) -[2023-10-14 15:38:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109051904. Throughput: 0: 1682.4, 1: 1665.0. Samples: 27275916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:38:53,164][74987] Avg episode reward: [(0, '25.550'), (1, '30.840')] -[2023-10-14 15:38:53,484][75949] Updated weights for policy 0, policy_version 53331 (0.0010) -[2023-10-14 15:38:53,592][75950] Updated weights for policy 1, policy_version 53190 (0.0010) -[2023-10-14 15:38:53,857][75949] Updated weights for policy 0, policy_version 53341 (0.0008) -[2023-10-14 15:38:53,962][75950] Updated weights for policy 1, policy_version 53200 (0.0009) -[2023-10-14 15:38:54,325][75950] Updated weights for policy 1, policy_version 53210 (0.0008) -[2023-10-14 15:38:57,889][75949] Updated weights for policy 0, policy_version 53351 (0.0008) -[2023-10-14 15:38:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 109117440. Throughput: 0: 1682.4, 1: 1667.3. Samples: 27296554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:38:58,165][74987] Avg episode reward: [(0, '28.300'), (1, '32.060')] -[2023-10-14 15:38:58,258][75949] Updated weights for policy 0, policy_version 53361 (0.0007) -[2023-10-14 15:38:58,514][75950] Updated weights for policy 1, policy_version 53220 (0.0010) -[2023-10-14 15:38:58,637][75949] Updated weights for policy 0, policy_version 53371 (0.0008) -[2023-10-14 15:38:58,878][75950] Updated weights for policy 1, policy_version 53230 (0.0008) -[2023-10-14 15:38:59,244][75950] Updated weights for policy 1, policy_version 53240 (0.0010) -[2023-10-14 15:39:02,574][75949] Updated weights for policy 0, policy_version 53381 (0.0007) -[2023-10-14 15:39:02,943][75949] Updated weights for policy 0, policy_version 53391 (0.0007) -[2023-10-14 15:39:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 109182976. Throughput: 0: 1686.8, 1: 1669.2. Samples: 27305782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:39:03,164][74987] Avg episode reward: [(0, '24.360'), (1, '30.840')] -[2023-10-14 15:39:03,321][75949] Updated weights for policy 0, policy_version 53401 (0.0007) -[2023-10-14 15:39:03,437][75950] Updated weights for policy 1, policy_version 53250 (0.0009) -[2023-10-14 15:39:03,807][75950] Updated weights for policy 1, policy_version 53260 (0.0008) -[2023-10-14 15:39:04,165][75950] Updated weights for policy 1, policy_version 53270 (0.0009) -[2023-10-14 15:39:04,540][75950] Updated weights for policy 1, policy_version 53280 (0.0008) -[2023-10-14 15:39:07,402][75949] Updated weights for policy 0, policy_version 53411 (0.0008) -[2023-10-14 15:39:07,760][75949] Updated weights for policy 0, policy_version 53421 (0.0010) -[2023-10-14 15:39:08,127][75949] Updated weights for policy 0, policy_version 53431 (0.0009) -[2023-10-14 15:39:08,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109248512. Throughput: 0: 1687.4, 1: 1677.0. Samples: 27326764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:39:08,164][74987] Avg episode reward: [(0, '28.550'), (1, '31.280')] -[2023-10-14 15:39:08,516][75950] Updated weights for policy 1, policy_version 53290 (0.0008) -[2023-10-14 15:39:08,890][75950] Updated weights for policy 1, policy_version 53300 (0.0007) -[2023-10-14 15:39:09,257][75950] Updated weights for policy 1, policy_version 53310 (0.0009) -[2023-10-14 15:39:12,303][75949] Updated weights for policy 0, policy_version 53441 (0.0009) -[2023-10-14 15:39:12,667][75949] Updated weights for policy 0, policy_version 53451 (0.0010) -[2023-10-14 15:39:13,040][75949] Updated weights for policy 0, policy_version 53461 (0.0007) -[2023-10-14 15:39:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109314048. Throughput: 0: 1680.2, 1: 1677.1. Samples: 27347008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:39:13,164][74987] Avg episode reward: [(0, '25.940'), (1, '31.560')] -[2023-10-14 15:39:13,188][75950] Updated weights for policy 1, policy_version 53320 (0.0008) -[2023-10-14 15:39:13,400][75949] Updated weights for policy 0, policy_version 53471 (0.0007) -[2023-10-14 15:39:13,561][75950] Updated weights for policy 1, policy_version 53330 (0.0007) -[2023-10-14 15:39:13,924][75950] Updated weights for policy 1, policy_version 53340 (0.0007) -[2023-10-14 15:39:17,460][75949] Updated weights for policy 0, policy_version 53481 (0.0010) -[2023-10-14 15:39:17,832][75949] Updated weights for policy 0, policy_version 53491 (0.0010) -[2023-10-14 15:39:18,084][75950] Updated weights for policy 1, policy_version 53350 (0.0007) -[2023-10-14 15:39:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 109379584. Throughput: 0: 1690.0, 1: 1677.0. Samples: 27356486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:39:18,165][74987] Avg episode reward: [(0, '27.580'), (1, '31.850')] -[2023-10-14 15:39:18,207][75949] Updated weights for policy 0, policy_version 53501 (0.0009) -[2023-10-14 15:39:18,449][75950] Updated weights for policy 1, policy_version 53360 (0.0007) -[2023-10-14 15:39:18,825][75950] Updated weights for policy 1, policy_version 53370 (0.0007) -[2023-10-14 15:39:22,206][75949] Updated weights for policy 0, policy_version 53511 (0.0011) -[2023-10-14 15:39:22,578][75949] Updated weights for policy 0, policy_version 53521 (0.0010) -[2023-10-14 15:39:22,864][75950] Updated weights for policy 1, policy_version 53380 (0.0008) -[2023-10-14 15:39:22,953][75949] Updated weights for policy 0, policy_version 53531 (0.0007) -[2023-10-14 15:39:23,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 109477888. Throughput: 0: 1687.0, 1: 1682.5. Samples: 27377336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:39:23,165][74987] Avg episode reward: [(0, '24.640'), (1, '30.630')] -[2023-10-14 15:39:23,228][75950] Updated weights for policy 1, policy_version 53390 (0.0008) -[2023-10-14 15:39:23,590][75950] Updated weights for policy 1, policy_version 53400 (0.0009) -[2023-10-14 15:39:26,701][75949] Updated weights for policy 0, policy_version 53541 (0.0009) -[2023-10-14 15:39:27,075][75949] Updated weights for policy 0, policy_version 53551 (0.0008) -[2023-10-14 15:39:27,434][75949] Updated weights for policy 0, policy_version 53561 (0.0009) -[2023-10-14 15:39:27,484][75950] Updated weights for policy 1, policy_version 53410 (0.0009) -[2023-10-14 15:39:27,858][75950] Updated weights for policy 1, policy_version 53420 (0.0009) -[2023-10-14 15:39:28,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 109543424. Throughput: 0: 1659.6, 1: 1676.9. Samples: 27396718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:39:28,164][74987] Avg episode reward: [(0, '26.030'), (1, '29.120')] -[2023-10-14 15:39:28,216][75950] Updated weights for policy 1, policy_version 53430 (0.0010) -[2023-10-14 15:39:28,588][75950] Updated weights for policy 1, policy_version 53440 (0.0011) -[2023-10-14 15:39:31,608][75949] Updated weights for policy 0, policy_version 53571 (0.0008) -[2023-10-14 15:39:31,981][75949] Updated weights for policy 0, policy_version 53581 (0.0007) -[2023-10-14 15:39:32,350][75949] Updated weights for policy 0, policy_version 53591 (0.0007) -[2023-10-14 15:39:32,772][75950] Updated weights for policy 1, policy_version 53450 (0.0010) -[2023-10-14 15:39:33,138][75950] Updated weights for policy 1, policy_version 53460 (0.0012) -[2023-10-14 15:39:33,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 109608960. Throughput: 0: 1687.0, 1: 1681.8. Samples: 27407206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:39:33,164][74987] Avg episode reward: [(0, '23.110'), (1, '30.250')] -[2023-10-14 15:39:33,501][75950] Updated weights for policy 1, policy_version 53470 (0.0010) -[2023-10-14 15:39:36,400][75949] Updated weights for policy 0, policy_version 53601 (0.0007) -[2023-10-14 15:39:36,777][75949] Updated weights for policy 0, policy_version 53611 (0.0008) -[2023-10-14 15:39:37,155][75949] Updated weights for policy 0, policy_version 53621 (0.0007) -[2023-10-14 15:39:37,518][75949] Updated weights for policy 0, policy_version 53631 (0.0009) -[2023-10-14 15:39:37,671][75950] Updated weights for policy 1, policy_version 53480 (0.0009) -[2023-10-14 15:39:38,052][75950] Updated weights for policy 1, policy_version 53490 (0.0008) -[2023-10-14 15:39:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 109674496. Throughput: 0: 1685.4, 1: 1682.8. Samples: 27427486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:39:38,165][74987] Avg episode reward: [(0, '25.600'), (1, '29.290')] -[2023-10-14 15:39:38,430][75950] Updated weights for policy 1, policy_version 53500 (0.0010) -[2023-10-14 15:39:41,691][75949] Updated weights for policy 0, policy_version 53641 (0.0009) -[2023-10-14 15:39:42,058][75949] Updated weights for policy 0, policy_version 53651 (0.0010) -[2023-10-14 15:39:42,431][75949] Updated weights for policy 0, policy_version 53661 (0.0008) -[2023-10-14 15:39:42,611][75950] Updated weights for policy 1, policy_version 53510 (0.0008) -[2023-10-14 15:39:42,979][75950] Updated weights for policy 1, policy_version 53520 (0.0008) -[2023-10-14 15:39:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 109740032. Throughput: 0: 1660.0, 1: 1675.6. Samples: 27446654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:39:43,164][74987] Avg episode reward: [(0, '26.090'), (1, '28.460')] -[2023-10-14 15:39:43,342][75950] Updated weights for policy 1, policy_version 53530 (0.0009) -[2023-10-14 15:39:46,393][75949] Updated weights for policy 0, policy_version 53671 (0.0008) -[2023-10-14 15:39:46,760][75949] Updated weights for policy 0, policy_version 53681 (0.0008) -[2023-10-14 15:39:47,130][75949] Updated weights for policy 0, policy_version 53691 (0.0008) -[2023-10-14 15:39:47,441][75950] Updated weights for policy 1, policy_version 53540 (0.0009) -[2023-10-14 15:39:47,810][75950] Updated weights for policy 1, policy_version 53550 (0.0011) -[2023-10-14 15:39:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 109805568. Throughput: 0: 1685.9, 1: 1679.2. Samples: 27457210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:39:48,164][74987] Avg episode reward: [(0, '27.800'), (1, '29.070')] -[2023-10-14 15:39:48,178][75950] Updated weights for policy 1, policy_version 53560 (0.0007) -[2023-10-14 15:39:51,114][75949] Updated weights for policy 0, policy_version 53701 (0.0008) -[2023-10-14 15:39:51,473][75949] Updated weights for policy 0, policy_version 53711 (0.0009) -[2023-10-14 15:39:51,849][75949] Updated weights for policy 0, policy_version 53721 (0.0008) -[2023-10-14 15:39:52,215][75950] Updated weights for policy 1, policy_version 53570 (0.0010) -[2023-10-14 15:39:52,578][75950] Updated weights for policy 1, policy_version 53580 (0.0008) -[2023-10-14 15:39:52,952][75950] Updated weights for policy 1, policy_version 53590 (0.0009) -[2023-10-14 15:39:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 109871104. Throughput: 0: 1670.6, 1: 1673.3. Samples: 27477240. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 15:39:53,165][74987] Avg episode reward: [(0, '26.950'), (1, '31.300')] -[2023-10-14 15:39:53,316][75950] Updated weights for policy 1, policy_version 53600 (0.0008) -[2023-10-14 15:39:55,978][75949] Updated weights for policy 0, policy_version 53731 (0.0008) -[2023-10-14 15:39:56,359][75949] Updated weights for policy 0, policy_version 53741 (0.0009) -[2023-10-14 15:39:56,724][75949] Updated weights for policy 0, policy_version 53751 (0.0009) -[2023-10-14 15:39:57,333][75950] Updated weights for policy 1, policy_version 53610 (0.0009) -[2023-10-14 15:39:57,699][75950] Updated weights for policy 1, policy_version 53620 (0.0007) -[2023-10-14 15:39:58,075][75950] Updated weights for policy 1, policy_version 53630 (0.0009) -[2023-10-14 15:39:58,164][74987] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 109969408. Throughput: 0: 1670.8, 1: 1654.2. Samples: 27496634. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 15:39:58,164][74987] Avg episode reward: [(0, '27.340'), (1, '29.560')] -[2023-10-14 15:40:00,830][75949] Updated weights for policy 0, policy_version 53761 (0.0009) -[2023-10-14 15:40:01,200][75949] Updated weights for policy 0, policy_version 53771 (0.0008) -[2023-10-14 15:40:01,575][75949] Updated weights for policy 0, policy_version 53781 (0.0011) -[2023-10-14 15:40:01,944][75949] Updated weights for policy 0, policy_version 53791 (0.0008) -[2023-10-14 15:40:02,066][75950] Updated weights for policy 1, policy_version 53640 (0.0008) -[2023-10-14 15:40:02,445][75950] Updated weights for policy 1, policy_version 53650 (0.0009) -[2023-10-14 15:40:02,811][75950] Updated weights for policy 1, policy_version 53660 (0.0008) -[2023-10-14 15:40:03,163][74987] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 110034944. Throughput: 0: 1689.7, 1: 1671.0. Samples: 27507718. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 15:40:03,164][74987] Avg episode reward: [(0, '26.000'), (1, '28.570')] -[2023-10-14 15:40:06,068][75949] Updated weights for policy 0, policy_version 53801 (0.0007) -[2023-10-14 15:40:06,435][75949] Updated weights for policy 0, policy_version 53811 (0.0008) -[2023-10-14 15:40:06,808][75949] Updated weights for policy 0, policy_version 53821 (0.0007) -[2023-10-14 15:40:07,041][75950] Updated weights for policy 1, policy_version 53670 (0.0010) -[2023-10-14 15:40:07,414][75950] Updated weights for policy 1, policy_version 53680 (0.0010) -[2023-10-14 15:40:07,774][75950] Updated weights for policy 1, policy_version 53690 (0.0008) -[2023-10-14 15:40:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 110100480. Throughput: 0: 1668.2, 1: 1670.7. Samples: 27527586. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 15:40:08,164][74987] Avg episode reward: [(0, '25.470'), (1, '31.910')] -[2023-10-14 15:40:10,855][75949] Updated weights for policy 0, policy_version 53831 (0.0007) -[2023-10-14 15:40:11,220][75949] Updated weights for policy 0, policy_version 53841 (0.0010) -[2023-10-14 15:40:11,593][75949] Updated weights for policy 0, policy_version 53851 (0.0009) -[2023-10-14 15:40:11,939][75950] Updated weights for policy 1, policy_version 53700 (0.0008) -[2023-10-14 15:40:12,304][75950] Updated weights for policy 1, policy_version 53710 (0.0009) -[2023-10-14 15:40:12,673][75950] Updated weights for policy 1, policy_version 53720 (0.0008) -[2023-10-14 15:40:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 110166016. Throughput: 0: 1688.2, 1: 1653.7. Samples: 27547104. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 15:40:13,165][74987] Avg episode reward: [(0, '21.800'), (1, '31.610')] -[2023-10-14 15:40:15,698][75949] Updated weights for policy 0, policy_version 53861 (0.0008) -[2023-10-14 15:40:16,076][75949] Updated weights for policy 0, policy_version 53871 (0.0008) -[2023-10-14 15:40:16,441][75949] Updated weights for policy 0, policy_version 53881 (0.0010) -[2023-10-14 15:40:16,654][75950] Updated weights for policy 1, policy_version 53730 (0.0008) -[2023-10-14 15:40:17,016][75950] Updated weights for policy 1, policy_version 53740 (0.0009) -[2023-10-14 15:40:17,382][75950] Updated weights for policy 1, policy_version 53750 (0.0007) -[2023-10-14 15:40:17,757][75950] Updated weights for policy 1, policy_version 53760 (0.0009) -[2023-10-14 15:40:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 110231552. Throughput: 0: 1688.4, 1: 1669.2. Samples: 27558298. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 15:40:18,164][74987] Avg episode reward: [(0, '23.820'), (1, '28.410')] -[2023-10-14 15:40:20,625][75949] Updated weights for policy 0, policy_version 53891 (0.0008) -[2023-10-14 15:40:20,987][75949] Updated weights for policy 0, policy_version 53901 (0.0008) -[2023-10-14 15:40:21,354][75949] Updated weights for policy 0, policy_version 53911 (0.0009) -[2023-10-14 15:40:21,902][75950] Updated weights for policy 1, policy_version 53770 (0.0007) -[2023-10-14 15:40:22,270][75950] Updated weights for policy 1, policy_version 53780 (0.0007) -[2023-10-14 15:40:22,650][75950] Updated weights for policy 1, policy_version 53790 (0.0008) -[2023-10-14 15:40:23,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 110297088. Throughput: 0: 1668.8, 1: 1670.2. Samples: 27577740. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) -[2023-10-14 15:40:23,164][74987] Avg episode reward: [(0, '22.990'), (1, '31.260')] -[2023-10-14 15:40:25,426][75949] Updated weights for policy 0, policy_version 53921 (0.0009) -[2023-10-14 15:40:25,848][75949] Updated weights for policy 0, policy_version 53931 (0.0007) -[2023-10-14 15:40:26,210][75949] Updated weights for policy 0, policy_version 53941 (0.0008) -[2023-10-14 15:40:26,584][75949] Updated weights for policy 0, policy_version 53951 (0.0008) -[2023-10-14 15:40:26,885][75950] Updated weights for policy 1, policy_version 53800 (0.0010) -[2023-10-14 15:40:27,251][75950] Updated weights for policy 1, policy_version 53810 (0.0008) -[2023-10-14 15:40:27,626][75950] Updated weights for policy 1, policy_version 53820 (0.0007) -[2023-10-14 15:40:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 110362624. Throughput: 0: 1691.5, 1: 1652.7. Samples: 27597140. Policy #0 lag: (min: 21.0, avg: 39.2, max: 40.0) -[2023-10-14 15:40:28,165][74987] Avg episode reward: [(0, '24.690'), (1, '32.400')] -[2023-10-14 15:40:30,662][75949] Updated weights for policy 0, policy_version 53961 (0.0007) -[2023-10-14 15:40:31,032][75949] Updated weights for policy 0, policy_version 53971 (0.0008) -[2023-10-14 15:40:31,394][75949] Updated weights for policy 0, policy_version 53981 (0.0008) -[2023-10-14 15:40:31,474][75950] Updated weights for policy 1, policy_version 53830 (0.0008) -[2023-10-14 15:40:31,845][75950] Updated weights for policy 1, policy_version 53840 (0.0007) -[2023-10-14 15:40:32,210][75950] Updated weights for policy 1, policy_version 53850 (0.0007) -[2023-10-14 15:40:33,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 110428160. Throughput: 0: 1681.8, 1: 1675.2. Samples: 27608274. Policy #0 lag: (min: 21.0, avg: 39.2, max: 40.0) -[2023-10-14 15:40:33,165][74987] Avg episode reward: [(0, '26.800'), (1, '28.790')] -[2023-10-14 15:40:35,311][75949] Updated weights for policy 0, policy_version 53991 (0.0007) -[2023-10-14 15:40:35,683][75949] Updated weights for policy 0, policy_version 54001 (0.0008) -[2023-10-14 15:40:36,050][75949] Updated weights for policy 0, policy_version 54011 (0.0009) -[2023-10-14 15:40:36,499][75950] Updated weights for policy 1, policy_version 53860 (0.0008) -[2023-10-14 15:40:36,858][75950] Updated weights for policy 1, policy_version 53870 (0.0009) -[2023-10-14 15:40:37,235][75950] Updated weights for policy 1, policy_version 53880 (0.0008) -[2023-10-14 15:40:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 110493696. Throughput: 0: 1674.6, 1: 1667.1. Samples: 27627616. Policy #0 lag: (min: 21.0, avg: 39.2, max: 40.0) -[2023-10-14 15:40:38,165][74987] Avg episode reward: [(0, '24.640'), (1, '31.440')] -[2023-10-14 15:40:40,164][75949] Updated weights for policy 0, policy_version 54021 (0.0009) -[2023-10-14 15:40:40,529][75949] Updated weights for policy 0, policy_version 54031 (0.0008) -[2023-10-14 15:40:40,902][75949] Updated weights for policy 0, policy_version 54041 (0.0008) -[2023-10-14 15:40:41,234][75950] Updated weights for policy 1, policy_version 53890 (0.0007) -[2023-10-14 15:40:41,603][75950] Updated weights for policy 1, policy_version 53900 (0.0008) -[2023-10-14 15:40:41,962][75950] Updated weights for policy 1, policy_version 53910 (0.0008) -[2023-10-14 15:40:42,330][75950] Updated weights for policy 1, policy_version 53920 (0.0007) -[2023-10-14 15:40:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 110559232. Throughput: 0: 1685.4, 1: 1666.5. Samples: 27647472. Policy #0 lag: (min: 21.0, avg: 39.2, max: 40.0) -[2023-10-14 15:40:43,165][74987] Avg episode reward: [(0, '25.350'), (1, '31.150')] -[2023-10-14 15:40:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000053920_55214080.pth... -[2023-10-14 15:40:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000054048_55345152.pth... -[2023-10-14 15:40:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000052352_53608448.pth -[2023-10-14 15:40:43,213][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000052480_53739520.pth -[2023-10-14 15:40:44,995][75949] Updated weights for policy 0, policy_version 54051 (0.0009) -[2023-10-14 15:40:45,357][75949] Updated weights for policy 0, policy_version 54061 (0.0009) -[2023-10-14 15:40:45,739][75949] Updated weights for policy 0, policy_version 54071 (0.0008) -[2023-10-14 15:40:46,239][75950] Updated weights for policy 1, policy_version 53930 (0.0009) -[2023-10-14 15:40:46,604][75950] Updated weights for policy 1, policy_version 53940 (0.0009) -[2023-10-14 15:40:46,968][75950] Updated weights for policy 1, policy_version 53950 (0.0007) -[2023-10-14 15:40:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 110624768. Throughput: 0: 1666.5, 1: 1681.7. Samples: 27658386. Policy #0 lag: (min: 21.0, avg: 39.2, max: 40.0) -[2023-10-14 15:40:48,164][74987] Avg episode reward: [(0, '25.020'), (1, '30.970')] -[2023-10-14 15:40:49,913][75949] Updated weights for policy 0, policy_version 54081 (0.0009) -[2023-10-14 15:40:50,277][75949] Updated weights for policy 0, policy_version 54091 (0.0009) -[2023-10-14 15:40:50,655][75949] Updated weights for policy 0, policy_version 54101 (0.0008) -[2023-10-14 15:40:50,999][75950] Updated weights for policy 1, policy_version 53960 (0.0007) -[2023-10-14 15:40:51,030][75949] Updated weights for policy 0, policy_version 54111 (0.0008) -[2023-10-14 15:40:51,366][75950] Updated weights for policy 1, policy_version 53970 (0.0009) -[2023-10-14 15:40:51,731][75950] Updated weights for policy 1, policy_version 53980 (0.0010) -[2023-10-14 15:40:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 110690304. Throughput: 0: 1670.8, 1: 1660.2. Samples: 27677484. Policy #0 lag: (min: 21.0, avg: 39.2, max: 40.0) -[2023-10-14 15:40:53,165][74987] Avg episode reward: [(0, '28.330'), (1, '30.920')] -[2023-10-14 15:40:54,960][75949] Updated weights for policy 0, policy_version 54121 (0.0010) -[2023-10-14 15:40:55,322][75949] Updated weights for policy 0, policy_version 54131 (0.0007) -[2023-10-14 15:40:55,694][75949] Updated weights for policy 0, policy_version 54141 (0.0007) -[2023-10-14 15:40:56,061][75950] Updated weights for policy 1, policy_version 53990 (0.0010) -[2023-10-14 15:40:56,433][75950] Updated weights for policy 1, policy_version 54000 (0.0010) -[2023-10-14 15:40:56,799][75950] Updated weights for policy 1, policy_version 54010 (0.0009) -[2023-10-14 15:40:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110755840. Throughput: 0: 1674.4, 1: 1668.8. Samples: 27697552. Policy #0 lag: (min: 21.0, avg: 39.2, max: 40.0) -[2023-10-14 15:40:58,165][74987] Avg episode reward: [(0, '25.520'), (1, '32.050')] -[2023-10-14 15:40:59,796][75949] Updated weights for policy 0, policy_version 54151 (0.0009) -[2023-10-14 15:41:00,161][75949] Updated weights for policy 0, policy_version 54161 (0.0009) -[2023-10-14 15:41:00,537][75949] Updated weights for policy 0, policy_version 54171 (0.0010) -[2023-10-14 15:41:00,866][75950] Updated weights for policy 1, policy_version 54020 (0.0009) -[2023-10-14 15:41:01,242][75950] Updated weights for policy 1, policy_version 54030 (0.0008) -[2023-10-14 15:41:01,598][75950] Updated weights for policy 1, policy_version 54040 (0.0010) -[2023-10-14 15:41:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110821376. Throughput: 0: 1649.4, 1: 1676.5. Samples: 27707966. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) -[2023-10-14 15:41:03,164][74987] Avg episode reward: [(0, '25.820'), (1, '32.230')] -[2023-10-14 15:41:04,689][75949] Updated weights for policy 0, policy_version 54181 (0.0009) -[2023-10-14 15:41:05,076][75949] Updated weights for policy 0, policy_version 54191 (0.0011) -[2023-10-14 15:41:05,449][75949] Updated weights for policy 0, policy_version 54201 (0.0010) -[2023-10-14 15:41:05,738][75950] Updated weights for policy 1, policy_version 54050 (0.0009) -[2023-10-14 15:41:06,115][75950] Updated weights for policy 1, policy_version 54060 (0.0009) -[2023-10-14 15:41:06,484][75950] Updated weights for policy 1, policy_version 54070 (0.0009) -[2023-10-14 15:41:06,842][75950] Updated weights for policy 1, policy_version 54080 (0.0007) -[2023-10-14 15:41:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110886912. Throughput: 0: 1669.7, 1: 1658.8. Samples: 27727524. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) -[2023-10-14 15:41:08,165][74987] Avg episode reward: [(0, '24.100'), (1, '29.680')] -[2023-10-14 15:41:09,564][75949] Updated weights for policy 0, policy_version 54211 (0.0007) -[2023-10-14 15:41:09,930][75949] Updated weights for policy 0, policy_version 54221 (0.0008) -[2023-10-14 15:41:10,302][75949] Updated weights for policy 0, policy_version 54231 (0.0011) -[2023-10-14 15:41:10,940][75950] Updated weights for policy 1, policy_version 54090 (0.0010) -[2023-10-14 15:41:11,306][75950] Updated weights for policy 1, policy_version 54100 (0.0009) -[2023-10-14 15:41:11,676][75950] Updated weights for policy 1, policy_version 54110 (0.0008) -[2023-10-14 15:41:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 110952448. Throughput: 0: 1670.5, 1: 1678.9. Samples: 27747862. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) -[2023-10-14 15:41:13,164][74987] Avg episode reward: [(0, '24.070'), (1, '32.670')] -[2023-10-14 15:41:14,340][75949] Updated weights for policy 0, policy_version 54241 (0.0010) -[2023-10-14 15:41:14,727][75949] Updated weights for policy 0, policy_version 54251 (0.0009) -[2023-10-14 15:41:15,091][75949] Updated weights for policy 0, policy_version 54261 (0.0008) -[2023-10-14 15:41:15,471][75949] Updated weights for policy 0, policy_version 54271 (0.0007) -[2023-10-14 15:41:15,782][75950] Updated weights for policy 1, policy_version 54120 (0.0008) -[2023-10-14 15:41:16,158][75950] Updated weights for policy 1, policy_version 54130 (0.0010) -[2023-10-14 15:41:16,517][75950] Updated weights for policy 1, policy_version 54140 (0.0010) -[2023-10-14 15:41:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111017984. Throughput: 0: 1649.3, 1: 1676.1. Samples: 27757916. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) -[2023-10-14 15:41:18,164][74987] Avg episode reward: [(0, '23.660'), (1, '30.930')] -[2023-10-14 15:41:19,641][75949] Updated weights for policy 0, policy_version 54281 (0.0008) -[2023-10-14 15:41:20,011][75949] Updated weights for policy 0, policy_version 54291 (0.0010) -[2023-10-14 15:41:20,376][75949] Updated weights for policy 0, policy_version 54301 (0.0008) -[2023-10-14 15:41:20,464][75950] Updated weights for policy 1, policy_version 54150 (0.0008) -[2023-10-14 15:41:20,840][75950] Updated weights for policy 1, policy_version 54160 (0.0010) -[2023-10-14 15:41:21,212][75950] Updated weights for policy 1, policy_version 54170 (0.0009) -[2023-10-14 15:41:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 111083520. Throughput: 0: 1670.6, 1: 1665.7. Samples: 27777750. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) -[2023-10-14 15:41:23,165][74987] Avg episode reward: [(0, '24.620'), (1, '28.890')] -[2023-10-14 15:41:24,336][75949] Updated weights for policy 0, policy_version 54311 (0.0010) -[2023-10-14 15:41:24,704][75949] Updated weights for policy 0, policy_version 54321 (0.0010) -[2023-10-14 15:41:25,077][75949] Updated weights for policy 0, policy_version 54331 (0.0010) -[2023-10-14 15:41:25,254][75950] Updated weights for policy 1, policy_version 54180 (0.0010) -[2023-10-14 15:41:25,620][75950] Updated weights for policy 1, policy_version 54190 (0.0008) -[2023-10-14 15:41:25,988][75950] Updated weights for policy 1, policy_version 54200 (0.0008) -[2023-10-14 15:41:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111149056. Throughput: 0: 1669.9, 1: 1684.7. Samples: 27798430. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) -[2023-10-14 15:41:28,164][74987] Avg episode reward: [(0, '24.270'), (1, '31.120')] -[2023-10-14 15:41:29,181][75949] Updated weights for policy 0, policy_version 54341 (0.0010) -[2023-10-14 15:41:29,548][75949] Updated weights for policy 0, policy_version 54351 (0.0009) -[2023-10-14 15:41:29,919][75949] Updated weights for policy 0, policy_version 54361 (0.0009) -[2023-10-14 15:41:30,155][75950] Updated weights for policy 1, policy_version 54210 (0.0009) -[2023-10-14 15:41:30,523][75950] Updated weights for policy 1, policy_version 54220 (0.0009) -[2023-10-14 15:41:30,893][75950] Updated weights for policy 1, policy_version 54230 (0.0008) -[2023-10-14 15:41:31,255][75950] Updated weights for policy 1, policy_version 54240 (0.0011) -[2023-10-14 15:41:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111214592. Throughput: 0: 1657.1, 1: 1668.1. Samples: 27808022. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) -[2023-10-14 15:41:33,165][74987] Avg episode reward: [(0, '26.700'), (1, '31.800')] -[2023-10-14 15:41:34,010][75949] Updated weights for policy 0, policy_version 54371 (0.0011) -[2023-10-14 15:41:34,378][75949] Updated weights for policy 0, policy_version 54381 (0.0011) -[2023-10-14 15:41:34,746][75949] Updated weights for policy 0, policy_version 54391 (0.0007) -[2023-10-14 15:41:35,370][75950] Updated weights for policy 1, policy_version 54250 (0.0008) -[2023-10-14 15:41:35,745][75950] Updated weights for policy 1, policy_version 54260 (0.0007) -[2023-10-14 15:41:36,108][75950] Updated weights for policy 1, policy_version 54270 (0.0008) -[2023-10-14 15:41:38,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111280128. Throughput: 0: 1670.2, 1: 1671.3. Samples: 27827852. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) -[2023-10-14 15:41:38,164][74987] Avg episode reward: [(0, '25.860'), (1, '31.550')] -[2023-10-14 15:41:38,715][75949] Updated weights for policy 0, policy_version 54401 (0.0008) -[2023-10-14 15:41:39,083][75949] Updated weights for policy 0, policy_version 54411 (0.0007) -[2023-10-14 15:41:39,459][75949] Updated weights for policy 0, policy_version 54421 (0.0009) -[2023-10-14 15:41:39,828][75949] Updated weights for policy 0, policy_version 54431 (0.0009) -[2023-10-14 15:41:40,137][75950] Updated weights for policy 1, policy_version 54280 (0.0010) -[2023-10-14 15:41:40,511][75950] Updated weights for policy 1, policy_version 54290 (0.0009) -[2023-10-14 15:41:40,874][75950] Updated weights for policy 1, policy_version 54300 (0.0009) -[2023-10-14 15:41:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111345664. Throughput: 0: 1676.9, 1: 1686.5. Samples: 27848906. Policy #0 lag: (min: 25.0, avg: 27.9, max: 57.0) -[2023-10-14 15:41:43,164][74987] Avg episode reward: [(0, '25.470'), (1, '30.660')] -[2023-10-14 15:41:43,936][75949] Updated weights for policy 0, policy_version 54441 (0.0011) -[2023-10-14 15:41:44,301][75949] Updated weights for policy 0, policy_version 54451 (0.0011) -[2023-10-14 15:41:44,665][75949] Updated weights for policy 0, policy_version 54461 (0.0009) -[2023-10-14 15:41:45,005][75950] Updated weights for policy 1, policy_version 54310 (0.0010) -[2023-10-14 15:41:45,372][75950] Updated weights for policy 1, policy_version 54320 (0.0009) -[2023-10-14 15:41:45,741][75950] Updated weights for policy 1, policy_version 54330 (0.0010) -[2023-10-14 15:41:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111411200. Throughput: 0: 1674.8, 1: 1666.6. Samples: 27858330. Policy #0 lag: (min: 25.0, avg: 27.9, max: 57.0) -[2023-10-14 15:41:48,164][74987] Avg episode reward: [(0, '23.730'), (1, '28.560')] -[2023-10-14 15:41:48,931][75949] Updated weights for policy 0, policy_version 54471 (0.0009) -[2023-10-14 15:41:49,307][75949] Updated weights for policy 0, policy_version 54481 (0.0008) -[2023-10-14 15:41:49,677][75949] Updated weights for policy 0, policy_version 54491 (0.0007) -[2023-10-14 15:41:49,803][75950] Updated weights for policy 1, policy_version 54340 (0.0009) -[2023-10-14 15:41:50,168][75950] Updated weights for policy 1, policy_version 54350 (0.0009) -[2023-10-14 15:41:50,544][75950] Updated weights for policy 1, policy_version 54360 (0.0009) -[2023-10-14 15:41:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111476736. Throughput: 0: 1682.1, 1: 1674.2. Samples: 27878560. Policy #0 lag: (min: 25.0, avg: 27.9, max: 57.0) -[2023-10-14 15:41:53,164][74987] Avg episode reward: [(0, '27.270'), (1, '29.390')] -[2023-10-14 15:41:53,425][75949] Updated weights for policy 0, policy_version 54501 (0.0007) -[2023-10-14 15:41:53,787][75949] Updated weights for policy 0, policy_version 54511 (0.0008) -[2023-10-14 15:41:54,156][75949] Updated weights for policy 0, policy_version 54521 (0.0008) -[2023-10-14 15:41:54,573][75950] Updated weights for policy 1, policy_version 54370 (0.0008) -[2023-10-14 15:41:54,938][75950] Updated weights for policy 1, policy_version 54380 (0.0009) -[2023-10-14 15:41:55,315][75950] Updated weights for policy 1, policy_version 54390 (0.0007) -[2023-10-14 15:41:55,677][75950] Updated weights for policy 1, policy_version 54400 (0.0009) -[2023-10-14 15:41:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 111542272. Throughput: 0: 1687.5, 1: 1681.5. Samples: 27899466. Policy #0 lag: (min: 25.0, avg: 27.9, max: 57.0) -[2023-10-14 15:41:58,164][74987] Avg episode reward: [(0, '24.920'), (1, '29.040')] -[2023-10-14 15:41:58,230][75949] Updated weights for policy 0, policy_version 54531 (0.0009) -[2023-10-14 15:41:58,602][75949] Updated weights for policy 0, policy_version 54541 (0.0008) -[2023-10-14 15:41:58,973][75949] Updated weights for policy 0, policy_version 54551 (0.0009) -[2023-10-14 15:41:59,936][75950] Updated weights for policy 1, policy_version 54410 (0.0007) -[2023-10-14 15:42:00,298][75950] Updated weights for policy 1, policy_version 54420 (0.0007) -[2023-10-14 15:42:00,671][75950] Updated weights for policy 1, policy_version 54430 (0.0007) -[2023-10-14 15:42:03,093][75949] Updated weights for policy 0, policy_version 54561 (0.0009) -[2023-10-14 15:42:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 111607808. Throughput: 0: 1690.7, 1: 1660.1. Samples: 27908700. Policy #0 lag: (min: 25.0, avg: 27.9, max: 57.0) -[2023-10-14 15:42:03,165][74987] Avg episode reward: [(0, '28.570'), (1, '29.060')] -[2023-10-14 15:42:03,502][75949] Updated weights for policy 0, policy_version 54571 (0.0009) -[2023-10-14 15:42:03,872][75949] Updated weights for policy 0, policy_version 54581 (0.0007) -[2023-10-14 15:42:04,245][75949] Updated weights for policy 0, policy_version 54591 (0.0007) -[2023-10-14 15:42:04,827][75950] Updated weights for policy 1, policy_version 54440 (0.0009) -[2023-10-14 15:42:05,200][75950] Updated weights for policy 1, policy_version 54450 (0.0007) -[2023-10-14 15:42:05,575][75950] Updated weights for policy 1, policy_version 54460 (0.0007) -[2023-10-14 15:42:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111673344. Throughput: 0: 1688.9, 1: 1670.9. Samples: 27928936. Policy #0 lag: (min: 25.0, avg: 27.9, max: 57.0) -[2023-10-14 15:42:08,164][74987] Avg episode reward: [(0, '24.660'), (1, '29.200')] -[2023-10-14 15:42:08,275][75949] Updated weights for policy 0, policy_version 54601 (0.0009) -[2023-10-14 15:42:08,647][75949] Updated weights for policy 0, policy_version 54611 (0.0009) -[2023-10-14 15:42:09,020][75949] Updated weights for policy 0, policy_version 54621 (0.0008) -[2023-10-14 15:42:09,623][75950] Updated weights for policy 1, policy_version 54470 (0.0008) -[2023-10-14 15:42:09,982][75950] Updated weights for policy 1, policy_version 54480 (0.0009) -[2023-10-14 15:42:10,354][75950] Updated weights for policy 1, policy_version 54490 (0.0008) -[2023-10-14 15:42:13,160][75949] Updated weights for policy 0, policy_version 54631 (0.0009) -[2023-10-14 15:42:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111738880. Throughput: 0: 1687.3, 1: 1670.4. Samples: 27949526. Policy #0 lag: (min: 25.0, avg: 27.9, max: 57.0) -[2023-10-14 15:42:13,164][74987] Avg episode reward: [(0, '27.790'), (1, '28.060')] -[2023-10-14 15:42:13,531][75949] Updated weights for policy 0, policy_version 54641 (0.0008) -[2023-10-14 15:42:13,908][75949] Updated weights for policy 0, policy_version 54651 (0.0008) -[2023-10-14 15:42:14,450][75950] Updated weights for policy 1, policy_version 54500 (0.0007) -[2023-10-14 15:42:14,811][75950] Updated weights for policy 1, policy_version 54510 (0.0009) -[2023-10-14 15:42:15,178][75950] Updated weights for policy 1, policy_version 54520 (0.0010) -[2023-10-14 15:42:18,127][75949] Updated weights for policy 0, policy_version 54661 (0.0009) -[2023-10-14 15:42:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 111804416. Throughput: 0: 1690.0, 1: 1653.9. Samples: 27958496. Policy #0 lag: (min: 25.0, avg: 27.9, max: 57.0) -[2023-10-14 15:42:18,165][74987] Avg episode reward: [(0, '23.650'), (1, '29.480')] -[2023-10-14 15:42:18,495][75949] Updated weights for policy 0, policy_version 54671 (0.0009) -[2023-10-14 15:42:18,875][75949] Updated weights for policy 0, policy_version 54681 (0.0009) -[2023-10-14 15:42:19,259][75950] Updated weights for policy 1, policy_version 54530 (0.0011) -[2023-10-14 15:42:19,627][75950] Updated weights for policy 1, policy_version 54540 (0.0010) -[2023-10-14 15:42:19,996][75950] Updated weights for policy 1, policy_version 54550 (0.0011) -[2023-10-14 15:42:20,362][75950] Updated weights for policy 1, policy_version 54560 (0.0007) -[2023-10-14 15:42:22,868][75949] Updated weights for policy 0, policy_version 54691 (0.0009) -[2023-10-14 15:42:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111869952. Throughput: 0: 1692.7, 1: 1671.4. Samples: 27979238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) -[2023-10-14 15:42:23,164][74987] Avg episode reward: [(0, '27.510'), (1, '29.600')] -[2023-10-14 15:42:23,234][75949] Updated weights for policy 0, policy_version 54701 (0.0009) -[2023-10-14 15:42:23,612][75949] Updated weights for policy 0, policy_version 54711 (0.0010) -[2023-10-14 15:42:24,471][75950] Updated weights for policy 1, policy_version 54570 (0.0009) -[2023-10-14 15:42:24,826][75950] Updated weights for policy 1, policy_version 54580 (0.0008) -[2023-10-14 15:42:25,188][75950] Updated weights for policy 1, policy_version 54590 (0.0010) -[2023-10-14 15:42:27,777][75949] Updated weights for policy 0, policy_version 54721 (0.0011) -[2023-10-14 15:42:28,147][75949] Updated weights for policy 0, policy_version 54731 (0.0011) -[2023-10-14 15:42:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111935488. Throughput: 0: 1684.8, 1: 1665.9. Samples: 27999684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) -[2023-10-14 15:42:28,164][74987] Avg episode reward: [(0, '23.690'), (1, '30.150')] -[2023-10-14 15:42:28,507][75949] Updated weights for policy 0, policy_version 54741 (0.0009) -[2023-10-14 15:42:28,881][75949] Updated weights for policy 0, policy_version 54751 (0.0010) -[2023-10-14 15:42:29,293][75950] Updated weights for policy 1, policy_version 54600 (0.0009) -[2023-10-14 15:42:29,660][75950] Updated weights for policy 1, policy_version 54610 (0.0010) -[2023-10-14 15:42:30,018][75950] Updated weights for policy 1, policy_version 54620 (0.0009) -[2023-10-14 15:42:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112001024. Throughput: 0: 1675.7, 1: 1651.1. Samples: 28008034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) -[2023-10-14 15:42:33,164][74987] Avg episode reward: [(0, '24.810'), (1, '32.890')] -[2023-10-14 15:42:33,526][75949] Updated weights for policy 0, policy_version 54761 (0.0008) -[2023-10-14 15:42:33,904][75949] Updated weights for policy 0, policy_version 54771 (0.0008) -[2023-10-14 15:42:34,275][75949] Updated weights for policy 0, policy_version 54781 (0.0009) -[2023-10-14 15:42:34,367][75950] Updated weights for policy 1, policy_version 54630 (0.0008) -[2023-10-14 15:42:34,732][75950] Updated weights for policy 1, policy_version 54640 (0.0011) -[2023-10-14 15:42:35,109][75950] Updated weights for policy 1, policy_version 54650 (0.0010) -[2023-10-14 15:42:38,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112066560. Throughput: 0: 1660.5, 1: 1653.8. Samples: 28027702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) -[2023-10-14 15:42:38,164][74987] Avg episode reward: [(0, '24.660'), (1, '31.250')] -[2023-10-14 15:42:38,617][75949] Updated weights for policy 0, policy_version 54791 (0.0010) -[2023-10-14 15:42:38,991][75949] Updated weights for policy 0, policy_version 54801 (0.0010) -[2023-10-14 15:42:39,362][75949] Updated weights for policy 0, policy_version 54811 (0.0007) -[2023-10-14 15:42:39,425][75950] Updated weights for policy 1, policy_version 54660 (0.0008) -[2023-10-14 15:42:39,792][75950] Updated weights for policy 1, policy_version 54670 (0.0009) -[2023-10-14 15:42:40,155][75950] Updated weights for policy 1, policy_version 54680 (0.0008) -[2023-10-14 15:42:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 112132096. Throughput: 0: 1634.2, 1: 1633.5. Samples: 28046516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) -[2023-10-14 15:42:43,165][74987] Avg episode reward: [(0, '24.080'), (1, '29.490')] -[2023-10-14 15:42:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000054816_56131584.pth... -[2023-10-14 15:42:43,177][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000054688_56000512.pth... -[2023-10-14 15:42:43,211][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000053120_54394880.pth -[2023-10-14 15:42:43,215][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000054688_56000512.pth -[2023-10-14 15:42:43,217][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000053280_54558720.pth -[2023-10-14 15:42:43,223][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000054816_56131584.pth -[2023-10-14 15:42:43,776][75949] Updated weights for policy 0, policy_version 54821 (0.0010) -[2023-10-14 15:42:44,154][75949] Updated weights for policy 0, policy_version 54831 (0.0010) -[2023-10-14 15:42:44,517][75949] Updated weights for policy 0, policy_version 54841 (0.0011) -[2023-10-14 15:42:44,746][75950] Updated weights for policy 1, policy_version 54690 (0.0011) -[2023-10-14 15:42:45,110][75950] Updated weights for policy 1, policy_version 54700 (0.0009) -[2023-10-14 15:42:45,473][75950] Updated weights for policy 1, policy_version 54710 (0.0009) -[2023-10-14 15:42:45,841][75950] Updated weights for policy 1, policy_version 54720 (0.0009) -[2023-10-14 15:42:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112197632. Throughput: 0: 1627.2, 1: 1627.7. Samples: 28055170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) -[2023-10-14 15:42:48,164][74987] Avg episode reward: [(0, '29.960'), (1, '30.660')] -[2023-10-14 15:42:48,165][75615] Saving new best policy, reward=29.960! -[2023-10-14 15:42:49,118][75949] Updated weights for policy 0, policy_version 54851 (0.0010) -[2023-10-14 15:42:49,510][75949] Updated weights for policy 0, policy_version 54861 (0.0009) -[2023-10-14 15:42:49,871][75949] Updated weights for policy 0, policy_version 54871 (0.0009) -[2023-10-14 15:42:50,409][75950] Updated weights for policy 1, policy_version 54730 (0.0009) -[2023-10-14 15:42:50,769][75950] Updated weights for policy 1, policy_version 54740 (0.0011) -[2023-10-14 15:42:51,135][75950] Updated weights for policy 1, policy_version 54750 (0.0010) -[2023-10-14 15:42:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112263168. Throughput: 0: 1606.1, 1: 1609.6. Samples: 28073644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) -[2023-10-14 15:42:53,164][74987] Avg episode reward: [(0, '25.710'), (1, '31.490')] -[2023-10-14 15:42:54,063][75949] Updated weights for policy 0, policy_version 54881 (0.0009) -[2023-10-14 15:42:54,441][75949] Updated weights for policy 0, policy_version 54891 (0.0010) -[2023-10-14 15:42:54,802][75949] Updated weights for policy 0, policy_version 54901 (0.0009) -[2023-10-14 15:42:55,171][75949] Updated weights for policy 0, policy_version 54911 (0.0009) -[2023-10-14 15:42:55,664][75950] Updated weights for policy 1, policy_version 54760 (0.0009) -[2023-10-14 15:42:56,037][75950] Updated weights for policy 1, policy_version 54770 (0.0009) -[2023-10-14 15:42:56,410][75950] Updated weights for policy 1, policy_version 54780 (0.0009) -[2023-10-14 15:42:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112328704. Throughput: 0: 1592.6, 1: 1587.7. Samples: 28092642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:42:58,164][74987] Avg episode reward: [(0, '28.570'), (1, '30.150')] -[2023-10-14 15:42:59,555][75949] Updated weights for policy 0, policy_version 54921 (0.0010) -[2023-10-14 15:42:59,927][75949] Updated weights for policy 0, policy_version 54931 (0.0010) -[2023-10-14 15:43:00,293][75949] Updated weights for policy 0, policy_version 54941 (0.0008) -[2023-10-14 15:43:00,992][75950] Updated weights for policy 1, policy_version 54790 (0.0010) -[2023-10-14 15:43:01,360][75950] Updated weights for policy 1, policy_version 54800 (0.0009) -[2023-10-14 15:43:01,727][75950] Updated weights for policy 1, policy_version 54810 (0.0009) -[2023-10-14 15:43:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 112394240. Throughput: 0: 1583.7, 1: 1608.7. Samples: 28102154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:43:03,164][74987] Avg episode reward: [(0, '24.030'), (1, '29.730')] -[2023-10-14 15:43:04,720][75949] Updated weights for policy 0, policy_version 54951 (0.0010) -[2023-10-14 15:43:05,078][75949] Updated weights for policy 0, policy_version 54961 (0.0010) -[2023-10-14 15:43:05,452][75949] Updated weights for policy 0, policy_version 54971 (0.0010) -[2023-10-14 15:43:06,073][75950] Updated weights for policy 1, policy_version 54820 (0.0010) -[2023-10-14 15:43:06,437][75950] Updated weights for policy 1, policy_version 54830 (0.0011) -[2023-10-14 15:43:06,807][75950] Updated weights for policy 1, policy_version 54840 (0.0009) -[2023-10-14 15:43:08,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112459776. Throughput: 0: 1570.8, 1: 1577.5. Samples: 28120910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:43:08,164][74987] Avg episode reward: [(0, '26.590'), (1, '31.340')] -[2023-10-14 15:43:09,316][75949] Updated weights for policy 0, policy_version 54981 (0.0010) -[2023-10-14 15:43:09,687][75949] Updated weights for policy 0, policy_version 54991 (0.0009) -[2023-10-14 15:43:10,053][75949] Updated weights for policy 0, policy_version 55001 (0.0007) -[2023-10-14 15:43:10,941][75950] Updated weights for policy 1, policy_version 54850 (0.0007) -[2023-10-14 15:43:11,311][75950] Updated weights for policy 1, policy_version 54860 (0.0010) -[2023-10-14 15:43:11,675][75950] Updated weights for policy 1, policy_version 54870 (0.0010) -[2023-10-14 15:43:12,036][75950] Updated weights for policy 1, policy_version 54880 (0.0007) -[2023-10-14 15:43:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112525312. Throughput: 0: 1577.4, 1: 1568.4. Samples: 28141246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:43:13,165][74987] Avg episode reward: [(0, '24.480'), (1, '29.520')] -[2023-10-14 15:43:14,216][75949] Updated weights for policy 0, policy_version 55011 (0.0008) -[2023-10-14 15:43:14,582][75949] Updated weights for policy 0, policy_version 55021 (0.0011) -[2023-10-14 15:43:14,961][75949] Updated weights for policy 0, policy_version 55031 (0.0010) -[2023-10-14 15:43:16,082][75950] Updated weights for policy 1, policy_version 54890 (0.0008) -[2023-10-14 15:43:16,444][75950] Updated weights for policy 1, policy_version 54900 (0.0009) -[2023-10-14 15:43:16,813][75950] Updated weights for policy 1, policy_version 54910 (0.0012) -[2023-10-14 15:43:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112590848. Throughput: 0: 1585.0, 1: 1602.5. Samples: 28151474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:43:18,165][74987] Avg episode reward: [(0, '26.170'), (1, '27.930')] -[2023-10-14 15:43:18,931][75949] Updated weights for policy 0, policy_version 55041 (0.0009) -[2023-10-14 15:43:19,297][75949] Updated weights for policy 0, policy_version 55051 (0.0010) -[2023-10-14 15:43:19,668][75949] Updated weights for policy 0, policy_version 55061 (0.0010) -[2023-10-14 15:43:20,038][75949] Updated weights for policy 0, policy_version 55071 (0.0011) -[2023-10-14 15:43:20,772][75950] Updated weights for policy 1, policy_version 54920 (0.0009) -[2023-10-14 15:43:21,142][75950] Updated weights for policy 1, policy_version 54930 (0.0008) -[2023-10-14 15:43:21,507][75950] Updated weights for policy 1, policy_version 54940 (0.0007) -[2023-10-14 15:43:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112656384. Throughput: 0: 1600.7, 1: 1588.0. Samples: 28171194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:43:23,164][74987] Avg episode reward: [(0, '24.520'), (1, '31.470')] -[2023-10-14 15:43:23,961][75949] Updated weights for policy 0, policy_version 55081 (0.0009) -[2023-10-14 15:43:24,327][75949] Updated weights for policy 0, policy_version 55091 (0.0008) -[2023-10-14 15:43:24,700][75949] Updated weights for policy 0, policy_version 55101 (0.0008) -[2023-10-14 15:43:25,372][75950] Updated weights for policy 1, policy_version 54950 (0.0007) -[2023-10-14 15:43:25,739][75950] Updated weights for policy 1, policy_version 54960 (0.0008) -[2023-10-14 15:43:26,114][75950] Updated weights for policy 1, policy_version 54970 (0.0008) -[2023-10-14 15:43:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112721920. Throughput: 0: 1630.6, 1: 1606.1. Samples: 28192168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:43:28,164][74987] Avg episode reward: [(0, '25.150'), (1, '31.850')] -[2023-10-14 15:43:28,936][75949] Updated weights for policy 0, policy_version 55111 (0.0009) -[2023-10-14 15:43:29,304][75949] Updated weights for policy 0, policy_version 55121 (0.0009) -[2023-10-14 15:43:29,672][75949] Updated weights for policy 0, policy_version 55131 (0.0009) -[2023-10-14 15:43:30,307][75950] Updated weights for policy 1, policy_version 54980 (0.0008) -[2023-10-14 15:43:30,677][75950] Updated weights for policy 1, policy_version 54990 (0.0009) -[2023-10-14 15:43:31,041][75950] Updated weights for policy 1, policy_version 55000 (0.0009) -[2023-10-14 15:43:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 112787456. Throughput: 0: 1635.6, 1: 1628.3. Samples: 28202050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:43:33,165][74987] Avg episode reward: [(0, '24.860'), (1, '30.830')] -[2023-10-14 15:43:33,857][75949] Updated weights for policy 0, policy_version 55141 (0.0010) -[2023-10-14 15:43:34,243][75949] Updated weights for policy 0, policy_version 55151 (0.0010) -[2023-10-14 15:43:34,605][75949] Updated weights for policy 0, policy_version 55161 (0.0009) -[2023-10-14 15:43:35,133][75950] Updated weights for policy 1, policy_version 55010 (0.0009) -[2023-10-14 15:43:35,495][75950] Updated weights for policy 1, policy_version 55020 (0.0007) -[2023-10-14 15:43:35,867][75950] Updated weights for policy 1, policy_version 55030 (0.0009) -[2023-10-14 15:43:36,225][75950] Updated weights for policy 1, policy_version 55040 (0.0010) -[2023-10-14 15:43:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112852992. Throughput: 0: 1659.1, 1: 1638.3. Samples: 28222028. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) -[2023-10-14 15:43:38,165][74987] Avg episode reward: [(0, '25.590'), (1, '31.270')] -[2023-10-14 15:43:38,446][75949] Updated weights for policy 0, policy_version 55171 (0.0008) -[2023-10-14 15:43:38,815][75949] Updated weights for policy 0, policy_version 55181 (0.0009) -[2023-10-14 15:43:39,174][75949] Updated weights for policy 0, policy_version 55191 (0.0007) -[2023-10-14 15:43:40,600][75950] Updated weights for policy 1, policy_version 55050 (0.0008) -[2023-10-14 15:43:40,967][75950] Updated weights for policy 1, policy_version 55060 (0.0009) -[2023-10-14 15:43:41,341][75950] Updated weights for policy 1, policy_version 55070 (0.0008) -[2023-10-14 15:43:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 112918528. Throughput: 0: 1675.6, 1: 1658.3. Samples: 28242666. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) -[2023-10-14 15:43:43,165][74987] Avg episode reward: [(0, '26.320'), (1, '32.790')] -[2023-10-14 15:43:43,322][75949] Updated weights for policy 0, policy_version 55201 (0.0009) -[2023-10-14 15:43:43,688][75949] Updated weights for policy 0, policy_version 55211 (0.0008) -[2023-10-14 15:43:44,052][75949] Updated weights for policy 0, policy_version 55221 (0.0009) -[2023-10-14 15:43:44,424][75949] Updated weights for policy 0, policy_version 55231 (0.0009) -[2023-10-14 15:43:45,318][75950] Updated weights for policy 1, policy_version 55080 (0.0008) -[2023-10-14 15:43:45,699][75950] Updated weights for policy 1, policy_version 55090 (0.0008) -[2023-10-14 15:43:46,069][75950] Updated weights for policy 1, policy_version 55100 (0.0010) -[2023-10-14 15:43:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 112984064. Throughput: 0: 1683.3, 1: 1657.4. Samples: 28252486. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) -[2023-10-14 15:43:48,165][74987] Avg episode reward: [(0, '25.070'), (1, '30.800')] -[2023-10-14 15:43:48,504][75949] Updated weights for policy 0, policy_version 55241 (0.0008) -[2023-10-14 15:43:48,880][75949] Updated weights for policy 0, policy_version 55251 (0.0008) -[2023-10-14 15:43:49,250][75949] Updated weights for policy 0, policy_version 55261 (0.0008) -[2023-10-14 15:43:50,248][75950] Updated weights for policy 1, policy_version 55110 (0.0008) -[2023-10-14 15:43:50,615][75950] Updated weights for policy 1, policy_version 55120 (0.0008) -[2023-10-14 15:43:50,973][75950] Updated weights for policy 1, policy_version 55130 (0.0008) -[2023-10-14 15:43:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 113049600. Throughput: 0: 1693.5, 1: 1671.9. Samples: 28272356. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) -[2023-10-14 15:43:53,165][74987] Avg episode reward: [(0, '26.170'), (1, '30.000')] -[2023-10-14 15:43:53,296][75949] Updated weights for policy 0, policy_version 55271 (0.0011) -[2023-10-14 15:43:53,667][75949] Updated weights for policy 0, policy_version 55281 (0.0008) -[2023-10-14 15:43:54,033][75949] Updated weights for policy 0, policy_version 55291 (0.0011) -[2023-10-14 15:43:55,100][75950] Updated weights for policy 1, policy_version 55140 (0.0008) -[2023-10-14 15:43:55,472][75950] Updated weights for policy 1, policy_version 55150 (0.0009) -[2023-10-14 15:43:55,844][75950] Updated weights for policy 1, policy_version 55160 (0.0009) -[2023-10-14 15:43:58,159][75949] Updated weights for policy 0, policy_version 55301 (0.0008) -[2023-10-14 15:43:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 113115136. Throughput: 0: 1687.6, 1: 1680.9. Samples: 28292830. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) -[2023-10-14 15:43:58,165][74987] Avg episode reward: [(0, '25.710'), (1, '31.480')] -[2023-10-14 15:43:58,536][75949] Updated weights for policy 0, policy_version 55311 (0.0007) -[2023-10-14 15:43:58,904][75949] Updated weights for policy 0, policy_version 55321 (0.0007) -[2023-10-14 15:43:59,808][75950] Updated weights for policy 1, policy_version 55170 (0.0008) -[2023-10-14 15:44:00,175][75950] Updated weights for policy 1, policy_version 55180 (0.0008) -[2023-10-14 15:44:00,543][75950] Updated weights for policy 1, policy_version 55190 (0.0009) -[2023-10-14 15:44:00,912][75950] Updated weights for policy 1, policy_version 55200 (0.0011) -[2023-10-14 15:44:03,031][75949] Updated weights for policy 0, policy_version 55331 (0.0009) -[2023-10-14 15:44:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 113180672. Throughput: 0: 1692.8, 1: 1660.4. Samples: 28302366. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) -[2023-10-14 15:44:03,165][74987] Avg episode reward: [(0, '25.200'), (1, '31.470')] -[2023-10-14 15:44:03,403][75949] Updated weights for policy 0, policy_version 55341 (0.0009) -[2023-10-14 15:44:03,770][75949] Updated weights for policy 0, policy_version 55351 (0.0011) -[2023-10-14 15:44:04,905][75950] Updated weights for policy 1, policy_version 55210 (0.0009) -[2023-10-14 15:44:05,264][75950] Updated weights for policy 1, policy_version 55220 (0.0008) -[2023-10-14 15:44:05,636][75950] Updated weights for policy 1, policy_version 55230 (0.0007) -[2023-10-14 15:44:07,893][75949] Updated weights for policy 0, policy_version 55361 (0.0010) -[2023-10-14 15:44:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 113246208. Throughput: 0: 1685.7, 1: 1676.5. Samples: 28322494. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) -[2023-10-14 15:44:08,165][74987] Avg episode reward: [(0, '23.600'), (1, '28.940')] -[2023-10-14 15:44:08,253][75949] Updated weights for policy 0, policy_version 55371 (0.0011) -[2023-10-14 15:44:08,635][75949] Updated weights for policy 0, policy_version 55381 (0.0012) -[2023-10-14 15:44:09,009][75949] Updated weights for policy 0, policy_version 55391 (0.0011) -[2023-10-14 15:44:09,893][75950] Updated weights for policy 1, policy_version 55240 (0.0008) -[2023-10-14 15:44:10,260][75950] Updated weights for policy 1, policy_version 55250 (0.0010) -[2023-10-14 15:44:10,628][75950] Updated weights for policy 1, policy_version 55260 (0.0010) -[2023-10-14 15:44:13,023][75949] Updated weights for policy 0, policy_version 55401 (0.0008) -[2023-10-14 15:44:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 113311744. Throughput: 0: 1679.1, 1: 1672.0. Samples: 28342970. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) -[2023-10-14 15:44:13,165][74987] Avg episode reward: [(0, '26.010'), (1, '29.960')] -[2023-10-14 15:44:13,396][75949] Updated weights for policy 0, policy_version 55411 (0.0009) -[2023-10-14 15:44:13,778][75949] Updated weights for policy 0, policy_version 55421 (0.0010) -[2023-10-14 15:44:14,675][75950] Updated weights for policy 1, policy_version 55270 (0.0009) -[2023-10-14 15:44:15,051][75950] Updated weights for policy 1, policy_version 55280 (0.0007) -[2023-10-14 15:44:15,412][75950] Updated weights for policy 1, policy_version 55290 (0.0007) -[2023-10-14 15:44:17,904][75949] Updated weights for policy 0, policy_version 55431 (0.0008) -[2023-10-14 15:44:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113377280. Throughput: 0: 1679.6, 1: 1655.6. Samples: 28352136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:44:18,165][74987] Avg episode reward: [(0, '24.500'), (1, '32.730')] -[2023-10-14 15:44:18,276][75949] Updated weights for policy 0, policy_version 55441 (0.0008) -[2023-10-14 15:44:18,640][75949] Updated weights for policy 0, policy_version 55451 (0.0009) -[2023-10-14 15:44:19,450][75950] Updated weights for policy 1, policy_version 55300 (0.0008) -[2023-10-14 15:44:19,816][75950] Updated weights for policy 1, policy_version 55310 (0.0007) -[2023-10-14 15:44:20,183][75950] Updated weights for policy 1, policy_version 55320 (0.0007) -[2023-10-14 15:44:22,711][75949] Updated weights for policy 0, policy_version 55461 (0.0008) -[2023-10-14 15:44:23,097][75949] Updated weights for policy 0, policy_version 55471 (0.0008) -[2023-10-14 15:44:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113442816. Throughput: 0: 1676.1, 1: 1668.4. Samples: 28372530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:44:23,164][74987] Avg episode reward: [(0, '28.120'), (1, '30.420')] -[2023-10-14 15:44:23,459][75949] Updated weights for policy 0, policy_version 55481 (0.0007) -[2023-10-14 15:44:24,293][75950] Updated weights for policy 1, policy_version 55330 (0.0007) -[2023-10-14 15:44:24,704][75950] Updated weights for policy 1, policy_version 55340 (0.0009) -[2023-10-14 15:44:25,073][75950] Updated weights for policy 1, policy_version 55350 (0.0008) -[2023-10-14 15:44:25,431][75950] Updated weights for policy 1, policy_version 55360 (0.0007) -[2023-10-14 15:44:27,521][75949] Updated weights for policy 0, policy_version 55491 (0.0008) -[2023-10-14 15:44:27,895][75949] Updated weights for policy 0, policy_version 55501 (0.0007) -[2023-10-14 15:44:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113508352. Throughput: 0: 1667.8, 1: 1663.2. Samples: 28392562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:44:28,164][74987] Avg episode reward: [(0, '23.750'), (1, '29.210')] -[2023-10-14 15:44:28,266][75949] Updated weights for policy 0, policy_version 55511 (0.0008) -[2023-10-14 15:44:29,514][75950] Updated weights for policy 1, policy_version 55370 (0.0007) -[2023-10-14 15:44:29,882][75950] Updated weights for policy 1, policy_version 55380 (0.0007) -[2023-10-14 15:44:30,248][75950] Updated weights for policy 1, policy_version 55390 (0.0007) -[2023-10-14 15:44:32,392][75949] Updated weights for policy 0, policy_version 55521 (0.0008) -[2023-10-14 15:44:32,759][75949] Updated weights for policy 0, policy_version 55531 (0.0010) -[2023-10-14 15:44:33,130][75949] Updated weights for policy 0, policy_version 55541 (0.0009) -[2023-10-14 15:44:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113573888. Throughput: 0: 1673.7, 1: 1647.3. Samples: 28401932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:44:33,165][74987] Avg episode reward: [(0, '26.870'), (1, '32.830')] -[2023-10-14 15:44:33,493][75949] Updated weights for policy 0, policy_version 55551 (0.0008) -[2023-10-14 15:44:34,324][75950] Updated weights for policy 1, policy_version 55400 (0.0009) -[2023-10-14 15:44:34,691][75950] Updated weights for policy 1, policy_version 55410 (0.0010) -[2023-10-14 15:44:35,058][75950] Updated weights for policy 1, policy_version 55420 (0.0010) -[2023-10-14 15:44:37,602][75949] Updated weights for policy 0, policy_version 55561 (0.0009) -[2023-10-14 15:44:37,973][75949] Updated weights for policy 0, policy_version 55571 (0.0008) -[2023-10-14 15:44:38,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 113639424. Throughput: 0: 1676.9, 1: 1664.7. Samples: 28422726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:44:38,164][74987] Avg episode reward: [(0, '24.440'), (1, '30.620')] -[2023-10-14 15:44:38,343][75949] Updated weights for policy 0, policy_version 55581 (0.0008) -[2023-10-14 15:44:39,167][75950] Updated weights for policy 1, policy_version 55430 (0.0008) -[2023-10-14 15:44:39,535][75950] Updated weights for policy 1, policy_version 55440 (0.0007) -[2023-10-14 15:44:39,907][75950] Updated weights for policy 1, policy_version 55450 (0.0010) -[2023-10-14 15:44:42,456][75949] Updated weights for policy 0, policy_version 55591 (0.0008) -[2023-10-14 15:44:42,828][75949] Updated weights for policy 0, policy_version 55601 (0.0007) -[2023-10-14 15:44:43,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 113704960. Throughput: 0: 1666.1, 1: 1667.9. Samples: 28442856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:44:43,164][74987] Avg episode reward: [(0, '25.860'), (1, '30.230')] -[2023-10-14 15:44:43,172][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000055456_56786944.pth... -[2023-10-14 15:44:43,198][75949] Updated weights for policy 0, policy_version 55611 (0.0008) -[2023-10-14 15:44:43,208][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000053920_55214080.pth -[2023-10-14 15:44:43,377][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000055616_56950784.pth... -[2023-10-14 15:44:43,407][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000054048_55345152.pth -[2023-10-14 15:44:44,029][75950] Updated weights for policy 1, policy_version 55460 (0.0008) -[2023-10-14 15:44:44,391][75950] Updated weights for policy 1, policy_version 55470 (0.0008) -[2023-10-14 15:44:44,756][75950] Updated weights for policy 1, policy_version 55480 (0.0010) -[2023-10-14 15:44:47,026][75949] Updated weights for policy 0, policy_version 55621 (0.0008) -[2023-10-14 15:44:47,390][75949] Updated weights for policy 0, policy_version 55631 (0.0009) -[2023-10-14 15:44:47,758][75949] Updated weights for policy 0, policy_version 55641 (0.0011) -[2023-10-14 15:44:48,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 113803264. Throughput: 0: 1675.9, 1: 1657.3. Samples: 28452358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:44:48,164][74987] Avg episode reward: [(0, '25.870'), (1, '32.910')] -[2023-10-14 15:44:48,851][75950] Updated weights for policy 1, policy_version 55490 (0.0012) -[2023-10-14 15:44:49,222][75950] Updated weights for policy 1, policy_version 55500 (0.0011) -[2023-10-14 15:44:49,587][75950] Updated weights for policy 1, policy_version 55510 (0.0010) -[2023-10-14 15:44:49,954][75950] Updated weights for policy 1, policy_version 55520 (0.0011) -[2023-10-14 15:44:52,035][75949] Updated weights for policy 0, policy_version 55651 (0.0008) -[2023-10-14 15:44:52,406][75949] Updated weights for policy 0, policy_version 55661 (0.0011) -[2023-10-14 15:44:52,785][75949] Updated weights for policy 0, policy_version 55671 (0.0008) -[2023-10-14 15:44:53,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 113868800. Throughput: 0: 1673.9, 1: 1666.4. Samples: 28472808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:44:53,165][74987] Avg episode reward: [(0, '25.370'), (1, '32.800')] -[2023-10-14 15:44:54,216][75950] Updated weights for policy 1, policy_version 55530 (0.0007) -[2023-10-14 15:44:54,592][75950] Updated weights for policy 1, policy_version 55540 (0.0008) -[2023-10-14 15:44:54,956][75950] Updated weights for policy 1, policy_version 55550 (0.0009) -[2023-10-14 15:44:56,885][75949] Updated weights for policy 0, policy_version 55681 (0.0008) -[2023-10-14 15:44:57,256][75949] Updated weights for policy 0, policy_version 55691 (0.0009) -[2023-10-14 15:44:57,622][75949] Updated weights for policy 0, policy_version 55701 (0.0008) -[2023-10-14 15:44:57,995][75949] Updated weights for policy 0, policy_version 55711 (0.0008) -[2023-10-14 15:44:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 113934336. Throughput: 0: 1650.8, 1: 1677.6. Samples: 28492746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:44:58,164][74987] Avg episode reward: [(0, '27.260'), (1, '30.870')] -[2023-10-14 15:44:58,966][75950] Updated weights for policy 1, policy_version 55560 (0.0009) -[2023-10-14 15:44:59,337][75950] Updated weights for policy 1, policy_version 55570 (0.0010) -[2023-10-14 15:44:59,704][75950] Updated weights for policy 1, policy_version 55580 (0.0009) -[2023-10-14 15:45:02,065][75949] Updated weights for policy 0, policy_version 55721 (0.0011) -[2023-10-14 15:45:02,436][75949] Updated weights for policy 0, policy_version 55731 (0.0010) -[2023-10-14 15:45:02,815][75949] Updated weights for policy 0, policy_version 55741 (0.0011) -[2023-10-14 15:45:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 113999872. Throughput: 0: 1670.2, 1: 1674.0. Samples: 28502624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:45:03,165][74987] Avg episode reward: [(0, '25.200'), (1, '32.180')] -[2023-10-14 15:45:03,529][75950] Updated weights for policy 1, policy_version 55590 (0.0008) -[2023-10-14 15:45:03,893][75950] Updated weights for policy 1, policy_version 55600 (0.0011) -[2023-10-14 15:45:04,262][75950] Updated weights for policy 1, policy_version 55610 (0.0009) -[2023-10-14 15:45:06,826][75949] Updated weights for policy 0, policy_version 55751 (0.0009) -[2023-10-14 15:45:07,195][75949] Updated weights for policy 0, policy_version 55761 (0.0010) -[2023-10-14 15:45:07,564][75949] Updated weights for policy 0, policy_version 55771 (0.0011) -[2023-10-14 15:45:08,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 114065408. Throughput: 0: 1669.1, 1: 1681.6. Samples: 28523312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:45:08,164][74987] Avg episode reward: [(0, '27.650'), (1, '31.250')] -[2023-10-14 15:45:08,271][75950] Updated weights for policy 1, policy_version 55620 (0.0008) -[2023-10-14 15:45:08,641][75950] Updated weights for policy 1, policy_version 55630 (0.0009) -[2023-10-14 15:45:09,010][75950] Updated weights for policy 1, policy_version 55640 (0.0008) -[2023-10-14 15:45:11,758][75949] Updated weights for policy 0, policy_version 55781 (0.0009) -[2023-10-14 15:45:12,136][75949] Updated weights for policy 0, policy_version 55791 (0.0007) -[2023-10-14 15:45:12,515][75949] Updated weights for policy 0, policy_version 55801 (0.0009) -[2023-10-14 15:45:13,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 114130944. Throughput: 0: 1651.0, 1: 1695.1. Samples: 28543136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:45:13,164][74987] Avg episode reward: [(0, '25.090'), (1, '31.160')] -[2023-10-14 15:45:13,205][75950] Updated weights for policy 1, policy_version 55650 (0.0008) -[2023-10-14 15:45:13,618][75950] Updated weights for policy 1, policy_version 55660 (0.0008) -[2023-10-14 15:45:13,995][75950] Updated weights for policy 1, policy_version 55670 (0.0010) -[2023-10-14 15:45:14,358][75950] Updated weights for policy 1, policy_version 55680 (0.0011) -[2023-10-14 15:45:16,443][75949] Updated weights for policy 0, policy_version 55811 (0.0009) -[2023-10-14 15:45:16,826][75949] Updated weights for policy 0, policy_version 55821 (0.0010) -[2023-10-14 15:45:17,194][75949] Updated weights for policy 0, policy_version 55831 (0.0011) -[2023-10-14 15:45:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 114196480. Throughput: 0: 1671.2, 1: 1689.4. Samples: 28553160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:45:18,164][74987] Avg episode reward: [(0, '26.270'), (1, '29.830')] -[2023-10-14 15:45:18,380][75950] Updated weights for policy 1, policy_version 55690 (0.0007) -[2023-10-14 15:45:18,752][75950] Updated weights for policy 1, policy_version 55700 (0.0007) -[2023-10-14 15:45:19,120][75950] Updated weights for policy 1, policy_version 55710 (0.0009) -[2023-10-14 15:45:21,208][75949] Updated weights for policy 0, policy_version 55841 (0.0009) -[2023-10-14 15:45:21,561][75949] Updated weights for policy 0, policy_version 55851 (0.0010) -[2023-10-14 15:45:21,931][75949] Updated weights for policy 0, policy_version 55861 (0.0009) -[2023-10-14 15:45:22,296][75949] Updated weights for policy 0, policy_version 55871 (0.0007) -[2023-10-14 15:45:23,057][75950] Updated weights for policy 1, policy_version 55720 (0.0007) -[2023-10-14 15:45:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 114262016. Throughput: 0: 1664.7, 1: 1686.8. Samples: 28573544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:45:23,165][74987] Avg episode reward: [(0, '25.770'), (1, '32.140')] -[2023-10-14 15:45:23,425][75950] Updated weights for policy 1, policy_version 55730 (0.0007) -[2023-10-14 15:45:23,794][75950] Updated weights for policy 1, policy_version 55740 (0.0008) -[2023-10-14 15:45:26,440][75949] Updated weights for policy 0, policy_version 55881 (0.0010) -[2023-10-14 15:45:26,814][75949] Updated weights for policy 0, policy_version 55891 (0.0009) -[2023-10-14 15:45:27,178][75949] Updated weights for policy 0, policy_version 55901 (0.0008) -[2023-10-14 15:45:27,971][75950] Updated weights for policy 1, policy_version 55750 (0.0008) -[2023-10-14 15:45:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 114327552. Throughput: 0: 1655.9, 1: 1691.7. Samples: 28593502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:45:28,165][74987] Avg episode reward: [(0, '26.270'), (1, '32.450')] -[2023-10-14 15:45:28,335][75950] Updated weights for policy 1, policy_version 55760 (0.0009) -[2023-10-14 15:45:28,705][75950] Updated weights for policy 1, policy_version 55770 (0.0009) -[2023-10-14 15:45:31,401][75949] Updated weights for policy 0, policy_version 55911 (0.0010) -[2023-10-14 15:45:31,777][75949] Updated weights for policy 0, policy_version 55921 (0.0009) -[2023-10-14 15:45:32,153][75949] Updated weights for policy 0, policy_version 55931 (0.0008) -[2023-10-14 15:45:32,586][75950] Updated weights for policy 1, policy_version 55780 (0.0009) -[2023-10-14 15:45:32,949][75950] Updated weights for policy 1, policy_version 55790 (0.0010) -[2023-10-14 15:45:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 114393088. Throughput: 0: 1670.5, 1: 1698.2. Samples: 28603948. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:45:33,165][74987] Avg episode reward: [(0, '25.720'), (1, '29.890')] -[2023-10-14 15:45:33,325][75950] Updated weights for policy 1, policy_version 55800 (0.0009) -[2023-10-14 15:45:36,053][75949] Updated weights for policy 0, policy_version 55941 (0.0008) -[2023-10-14 15:45:36,429][75949] Updated weights for policy 0, policy_version 55951 (0.0009) -[2023-10-14 15:45:36,782][75949] Updated weights for policy 0, policy_version 55961 (0.0009) -[2023-10-14 15:45:37,434][75950] Updated weights for policy 1, policy_version 55810 (0.0008) -[2023-10-14 15:45:37,803][75950] Updated weights for policy 1, policy_version 55820 (0.0008) -[2023-10-14 15:45:38,163][75950] Updated weights for policy 1, policy_version 55830 (0.0007) -[2023-10-14 15:45:38,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 114458624. Throughput: 0: 1657.1, 1: 1696.8. Samples: 28623730. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:45:38,164][74987] Avg episode reward: [(0, '26.680'), (1, '31.230')] -[2023-10-14 15:45:38,535][75950] Updated weights for policy 1, policy_version 55840 (0.0008) -[2023-10-14 15:45:40,694][75949] Updated weights for policy 0, policy_version 55971 (0.0008) -[2023-10-14 15:45:41,064][75949] Updated weights for policy 0, policy_version 55981 (0.0008) -[2023-10-14 15:45:41,434][75949] Updated weights for policy 0, policy_version 55991 (0.0010) -[2023-10-14 15:45:42,742][75950] Updated weights for policy 1, policy_version 55850 (0.0009) -[2023-10-14 15:45:43,101][75950] Updated weights for policy 1, policy_version 55860 (0.0008) -[2023-10-14 15:45:43,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 114524160. Throughput: 0: 1672.2, 1: 1685.8. Samples: 28643856. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:45:43,164][74987] Avg episode reward: [(0, '26.500'), (1, '32.510')] -[2023-10-14 15:45:43,471][75950] Updated weights for policy 1, policy_version 55870 (0.0010) -[2023-10-14 15:45:45,496][75949] Updated weights for policy 0, policy_version 56001 (0.0008) -[2023-10-14 15:45:45,873][75949] Updated weights for policy 0, policy_version 56011 (0.0008) -[2023-10-14 15:45:46,242][75949] Updated weights for policy 0, policy_version 56021 (0.0007) -[2023-10-14 15:45:46,607][75949] Updated weights for policy 0, policy_version 56031 (0.0008) -[2023-10-14 15:45:47,563][75950] Updated weights for policy 1, policy_version 55880 (0.0010) -[2023-10-14 15:45:47,929][75950] Updated weights for policy 1, policy_version 55890 (0.0010) -[2023-10-14 15:45:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 114589696. Throughput: 0: 1677.5, 1: 1694.1. Samples: 28654346. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:45:48,165][74987] Avg episode reward: [(0, '25.510'), (1, '29.770')] -[2023-10-14 15:45:48,294][75950] Updated weights for policy 1, policy_version 55900 (0.0008) -[2023-10-14 15:45:50,575][75949] Updated weights for policy 0, policy_version 56041 (0.0008) -[2023-10-14 15:45:50,952][75949] Updated weights for policy 0, policy_version 56051 (0.0009) -[2023-10-14 15:45:51,328][75949] Updated weights for policy 0, policy_version 56061 (0.0009) -[2023-10-14 15:45:52,338][75950] Updated weights for policy 1, policy_version 55910 (0.0009) -[2023-10-14 15:45:52,704][75950] Updated weights for policy 1, policy_version 55920 (0.0009) -[2023-10-14 15:45:53,077][75950] Updated weights for policy 1, policy_version 55930 (0.0009) -[2023-10-14 15:45:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 114655232. Throughput: 0: 1659.8, 1: 1693.1. Samples: 28674192. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:45:53,165][74987] Avg episode reward: [(0, '27.390'), (1, '29.320')] -[2023-10-14 15:45:55,365][75949] Updated weights for policy 0, policy_version 56071 (0.0009) -[2023-10-14 15:45:55,726][75949] Updated weights for policy 0, policy_version 56081 (0.0009) -[2023-10-14 15:45:56,089][75949] Updated weights for policy 0, policy_version 56091 (0.0009) -[2023-10-14 15:45:57,183][75950] Updated weights for policy 1, policy_version 55940 (0.0008) -[2023-10-14 15:45:57,551][75950] Updated weights for policy 1, policy_version 55950 (0.0008) -[2023-10-14 15:45:57,912][75950] Updated weights for policy 1, policy_version 55960 (0.0009) -[2023-10-14 15:45:58,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 114720768. Throughput: 0: 1688.1, 1: 1672.0. Samples: 28694340. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:45:58,164][74987] Avg episode reward: [(0, '25.450'), (1, '30.190')] -[2023-10-14 15:46:00,410][75949] Updated weights for policy 0, policy_version 56101 (0.0007) -[2023-10-14 15:46:00,801][75949] Updated weights for policy 0, policy_version 56111 (0.0009) -[2023-10-14 15:46:01,173][75949] Updated weights for policy 0, policy_version 56121 (0.0008) -[2023-10-14 15:46:02,056][75950] Updated weights for policy 1, policy_version 55970 (0.0009) -[2023-10-14 15:46:02,477][75950] Updated weights for policy 1, policy_version 55980 (0.0008) -[2023-10-14 15:46:02,834][75950] Updated weights for policy 1, policy_version 55990 (0.0008) -[2023-10-14 15:46:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 114786304. Throughput: 0: 1678.4, 1: 1693.1. Samples: 28704876. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:46:03,164][74987] Avg episode reward: [(0, '26.680'), (1, '30.310')] -[2023-10-14 15:46:03,204][75950] Updated weights for policy 1, policy_version 56000 (0.0007) -[2023-10-14 15:46:05,281][75949] Updated weights for policy 0, policy_version 56131 (0.0009) -[2023-10-14 15:46:05,650][75949] Updated weights for policy 0, policy_version 56141 (0.0007) -[2023-10-14 15:46:06,014][75949] Updated weights for policy 0, policy_version 56151 (0.0009) -[2023-10-14 15:46:07,266][75950] Updated weights for policy 1, policy_version 56010 (0.0008) -[2023-10-14 15:46:07,636][75950] Updated weights for policy 1, policy_version 56020 (0.0009) -[2023-10-14 15:46:08,010][75950] Updated weights for policy 1, policy_version 56030 (0.0010) -[2023-10-14 15:46:08,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 114884608. Throughput: 0: 1669.1, 1: 1689.1. Samples: 28724662. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:46:08,164][74987] Avg episode reward: [(0, '24.250'), (1, '27.920')] -[2023-10-14 15:46:09,907][75949] Updated weights for policy 0, policy_version 56161 (0.0007) -[2023-10-14 15:46:10,280][75949] Updated weights for policy 0, policy_version 56171 (0.0008) -[2023-10-14 15:46:10,645][75949] Updated weights for policy 0, policy_version 56181 (0.0007) -[2023-10-14 15:46:11,018][75949] Updated weights for policy 0, policy_version 56191 (0.0007) -[2023-10-14 15:46:12,126][75950] Updated weights for policy 1, policy_version 56040 (0.0009) -[2023-10-14 15:46:12,491][75950] Updated weights for policy 1, policy_version 56050 (0.0010) -[2023-10-14 15:46:12,847][75950] Updated weights for policy 1, policy_version 56060 (0.0009) -[2023-10-14 15:46:13,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 114950144. Throughput: 0: 1696.1, 1: 1665.6. Samples: 28744778. Policy #0 lag: (min: 6.0, avg: 8.8, max: 38.0) -[2023-10-14 15:46:13,164][74987] Avg episode reward: [(0, '27.290'), (1, '30.820')] -[2023-10-14 15:46:15,091][75949] Updated weights for policy 0, policy_version 56201 (0.0007) -[2023-10-14 15:46:15,456][75949] Updated weights for policy 0, policy_version 56211 (0.0007) -[2023-10-14 15:46:15,819][75949] Updated weights for policy 0, policy_version 56221 (0.0007) -[2023-10-14 15:46:16,985][75950] Updated weights for policy 1, policy_version 56070 (0.0008) -[2023-10-14 15:46:17,371][75950] Updated weights for policy 1, policy_version 56080 (0.0009) -[2023-10-14 15:46:17,739][75950] Updated weights for policy 1, policy_version 56090 (0.0009) -[2023-10-14 15:46:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 115015680. Throughput: 0: 1677.3, 1: 1679.3. Samples: 28754990. Policy #0 lag: (min: 6.0, avg: 8.8, max: 38.0) -[2023-10-14 15:46:18,164][74987] Avg episode reward: [(0, '26.150'), (1, '31.080')] -[2023-10-14 15:46:19,881][75949] Updated weights for policy 0, policy_version 56231 (0.0009) -[2023-10-14 15:46:20,247][75949] Updated weights for policy 0, policy_version 56241 (0.0007) -[2023-10-14 15:46:20,623][75949] Updated weights for policy 0, policy_version 56251 (0.0009) -[2023-10-14 15:46:21,756][75950] Updated weights for policy 1, policy_version 56100 (0.0009) -[2023-10-14 15:46:22,113][75950] Updated weights for policy 1, policy_version 56110 (0.0008) -[2023-10-14 15:46:22,483][75950] Updated weights for policy 1, policy_version 56120 (0.0007) -[2023-10-14 15:46:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 115081216. Throughput: 0: 1687.0, 1: 1681.2. Samples: 28775300. Policy #0 lag: (min: 6.0, avg: 8.8, max: 38.0) -[2023-10-14 15:46:23,164][74987] Avg episode reward: [(0, '26.930'), (1, '28.290')] -[2023-10-14 15:46:24,753][75949] Updated weights for policy 0, policy_version 56261 (0.0008) -[2023-10-14 15:46:25,127][75949] Updated weights for policy 0, policy_version 56271 (0.0009) -[2023-10-14 15:46:25,496][75949] Updated weights for policy 0, policy_version 56281 (0.0009) -[2023-10-14 15:46:26,351][75950] Updated weights for policy 1, policy_version 56130 (0.0008) -[2023-10-14 15:46:26,709][75950] Updated weights for policy 1, policy_version 56140 (0.0009) -[2023-10-14 15:46:27,074][75950] Updated weights for policy 1, policy_version 56150 (0.0010) -[2023-10-14 15:46:27,442][75950] Updated weights for policy 1, policy_version 56160 (0.0007) -[2023-10-14 15:46:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 115146752. Throughput: 0: 1693.5, 1: 1665.9. Samples: 28795028. Policy #0 lag: (min: 6.0, avg: 8.8, max: 38.0) -[2023-10-14 15:46:28,165][74987] Avg episode reward: [(0, '24.780'), (1, '30.090')] -[2023-10-14 15:46:29,633][75949] Updated weights for policy 0, policy_version 56291 (0.0008) -[2023-10-14 15:46:30,010][75949] Updated weights for policy 0, policy_version 56301 (0.0011) -[2023-10-14 15:46:30,365][75949] Updated weights for policy 0, policy_version 56311 (0.0009) -[2023-10-14 15:46:31,454][75950] Updated weights for policy 1, policy_version 56170 (0.0008) -[2023-10-14 15:46:31,825][75950] Updated weights for policy 1, policy_version 56180 (0.0008) -[2023-10-14 15:46:32,193][75950] Updated weights for policy 1, policy_version 56190 (0.0009) -[2023-10-14 15:46:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13329.3). Total num frames: 115212288. Throughput: 0: 1668.8, 1: 1690.7. Samples: 28805522. Policy #0 lag: (min: 6.0, avg: 8.8, max: 38.0) -[2023-10-14 15:46:33,165][74987] Avg episode reward: [(0, '26.210'), (1, '30.680')] -[2023-10-14 15:46:34,653][75949] Updated weights for policy 0, policy_version 56321 (0.0008) -[2023-10-14 15:46:35,013][75949] Updated weights for policy 0, policy_version 56331 (0.0008) -[2023-10-14 15:46:35,381][75949] Updated weights for policy 0, policy_version 56341 (0.0008) -[2023-10-14 15:46:35,753][75949] Updated weights for policy 0, policy_version 56351 (0.0007) -[2023-10-14 15:46:35,994][75950] Updated weights for policy 1, policy_version 56200 (0.0008) -[2023-10-14 15:46:36,359][75950] Updated weights for policy 1, policy_version 56210 (0.0007) -[2023-10-14 15:46:36,730][75950] Updated weights for policy 1, policy_version 56220 (0.0007) -[2023-10-14 15:46:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 115277824. Throughput: 0: 1685.8, 1: 1673.8. Samples: 28825376. Policy #0 lag: (min: 6.0, avg: 8.8, max: 38.0) -[2023-10-14 15:46:38,165][74987] Avg episode reward: [(0, '26.950'), (1, '30.810')] -[2023-10-14 15:46:39,627][75949] Updated weights for policy 0, policy_version 56361 (0.0008) -[2023-10-14 15:46:39,991][75949] Updated weights for policy 0, policy_version 56371 (0.0008) -[2023-10-14 15:46:40,367][75949] Updated weights for policy 0, policy_version 56381 (0.0009) -[2023-10-14 15:46:40,924][75950] Updated weights for policy 1, policy_version 56230 (0.0007) -[2023-10-14 15:46:41,286][75950] Updated weights for policy 1, policy_version 56240 (0.0007) -[2023-10-14 15:46:41,653][75950] Updated weights for policy 1, policy_version 56250 (0.0008) -[2023-10-14 15:46:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 115343360. Throughput: 0: 1687.7, 1: 1677.6. Samples: 28845780. Policy #0 lag: (min: 6.0, avg: 8.8, max: 38.0) -[2023-10-14 15:46:43,165][74987] Avg episode reward: [(0, '28.190'), (1, '30.760')] -[2023-10-14 15:46:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000056256_57606144.pth... -[2023-10-14 15:46:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000056384_57737216.pth... -[2023-10-14 15:46:43,208][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000054688_56000512.pth -[2023-10-14 15:46:43,219][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000054816_56131584.pth -[2023-10-14 15:46:44,359][75949] Updated weights for policy 0, policy_version 56391 (0.0008) -[2023-10-14 15:46:44,725][75949] Updated weights for policy 0, policy_version 56401 (0.0007) -[2023-10-14 15:46:45,093][75949] Updated weights for policy 0, policy_version 56411 (0.0008) -[2023-10-14 15:46:45,699][75950] Updated weights for policy 1, policy_version 56260 (0.0008) -[2023-10-14 15:46:46,066][75950] Updated weights for policy 1, policy_version 56270 (0.0011) -[2023-10-14 15:46:46,443][75950] Updated weights for policy 1, policy_version 56280 (0.0010) -[2023-10-14 15:46:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 115408896. Throughput: 0: 1674.0, 1: 1687.4. Samples: 28856136. Policy #0 lag: (min: 6.0, avg: 8.8, max: 38.0) -[2023-10-14 15:46:48,164][74987] Avg episode reward: [(0, '25.860'), (1, '30.810')] -[2023-10-14 15:46:49,277][75949] Updated weights for policy 0, policy_version 56421 (0.0010) -[2023-10-14 15:46:49,653][75949] Updated weights for policy 0, policy_version 56431 (0.0008) -[2023-10-14 15:46:50,026][75949] Updated weights for policy 0, policy_version 56441 (0.0007) -[2023-10-14 15:46:50,403][75950] Updated weights for policy 1, policy_version 56290 (0.0009) -[2023-10-14 15:46:50,775][75950] Updated weights for policy 1, policy_version 56300 (0.0007) -[2023-10-14 15:46:51,144][75950] Updated weights for policy 1, policy_version 56310 (0.0009) -[2023-10-14 15:46:51,509][75950] Updated weights for policy 1, policy_version 56320 (0.0011) -[2023-10-14 15:46:53,164][74987] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 115474432. Throughput: 0: 1685.3, 1: 1664.6. Samples: 28875406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:46:53,165][74987] Avg episode reward: [(0, '26.240'), (1, '30.290')] -[2023-10-14 15:46:54,083][75949] Updated weights for policy 0, policy_version 56451 (0.0008) -[2023-10-14 15:46:54,491][75949] Updated weights for policy 0, policy_version 56461 (0.0009) -[2023-10-14 15:46:54,864][75949] Updated weights for policy 0, policy_version 56471 (0.0009) -[2023-10-14 15:46:55,677][75950] Updated weights for policy 1, policy_version 56330 (0.0009) -[2023-10-14 15:46:56,054][75950] Updated weights for policy 1, policy_version 56340 (0.0010) -[2023-10-14 15:46:56,419][75950] Updated weights for policy 1, policy_version 56350 (0.0009) -[2023-10-14 15:46:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 115539968. Throughput: 0: 1674.5, 1: 1681.0. Samples: 28895774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:46:58,164][74987] Avg episode reward: [(0, '24.970'), (1, '29.250')] -[2023-10-14 15:46:59,091][75949] Updated weights for policy 0, policy_version 56481 (0.0007) -[2023-10-14 15:46:59,460][75949] Updated weights for policy 0, policy_version 56491 (0.0007) -[2023-10-14 15:46:59,826][75949] Updated weights for policy 0, policy_version 56501 (0.0008) -[2023-10-14 15:47:00,200][75949] Updated weights for policy 0, policy_version 56511 (0.0007) -[2023-10-14 15:47:00,455][75950] Updated weights for policy 1, policy_version 56360 (0.0009) -[2023-10-14 15:47:00,814][75950] Updated weights for policy 1, policy_version 56370 (0.0009) -[2023-10-14 15:47:01,177][75950] Updated weights for policy 1, policy_version 56380 (0.0009) -[2023-10-14 15:47:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 115605504. Throughput: 0: 1664.1, 1: 1685.5. Samples: 28905722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:47:03,164][74987] Avg episode reward: [(0, '27.690'), (1, '31.420')] -[2023-10-14 15:47:04,088][75949] Updated weights for policy 0, policy_version 56521 (0.0010) -[2023-10-14 15:47:04,461][75949] Updated weights for policy 0, policy_version 56531 (0.0007) -[2023-10-14 15:47:04,821][75949] Updated weights for policy 0, policy_version 56541 (0.0007) -[2023-10-14 15:47:05,355][75950] Updated weights for policy 1, policy_version 56390 (0.0008) -[2023-10-14 15:47:05,713][75950] Updated weights for policy 1, policy_version 56400 (0.0008) -[2023-10-14 15:47:06,093][75950] Updated weights for policy 1, policy_version 56410 (0.0010) -[2023-10-14 15:47:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 115671040. Throughput: 0: 1678.7, 1: 1669.9. Samples: 28925988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:47:08,164][74987] Avg episode reward: [(0, '26.360'), (1, '31.360')] -[2023-10-14 15:47:08,812][75949] Updated weights for policy 0, policy_version 56551 (0.0008) -[2023-10-14 15:47:09,170][75949] Updated weights for policy 0, policy_version 56561 (0.0011) -[2023-10-14 15:47:09,540][75949] Updated weights for policy 0, policy_version 56571 (0.0010) -[2023-10-14 15:47:10,135][75950] Updated weights for policy 1, policy_version 56420 (0.0009) -[2023-10-14 15:47:10,493][75950] Updated weights for policy 1, policy_version 56430 (0.0009) -[2023-10-14 15:47:10,856][75950] Updated weights for policy 1, policy_version 56440 (0.0010) -[2023-10-14 15:47:13,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 115736576. Throughput: 0: 1682.1, 1: 1688.5. Samples: 28946708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:47:13,164][74987] Avg episode reward: [(0, '27.080'), (1, '30.610')] -[2023-10-14 15:47:13,784][75949] Updated weights for policy 0, policy_version 56581 (0.0008) -[2023-10-14 15:47:14,151][75949] Updated weights for policy 0, policy_version 56591 (0.0009) -[2023-10-14 15:47:14,519][75949] Updated weights for policy 0, policy_version 56601 (0.0010) -[2023-10-14 15:47:14,925][75950] Updated weights for policy 1, policy_version 56450 (0.0010) -[2023-10-14 15:47:15,294][75950] Updated weights for policy 1, policy_version 56460 (0.0008) -[2023-10-14 15:47:15,668][75950] Updated weights for policy 1, policy_version 56470 (0.0009) -[2023-10-14 15:47:16,033][75950] Updated weights for policy 1, policy_version 56480 (0.0008) -[2023-10-14 15:47:18,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 115802112. Throughput: 0: 1682.1, 1: 1669.3. Samples: 28956330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:47:18,164][74987] Avg episode reward: [(0, '26.330'), (1, '30.400')] -[2023-10-14 15:47:18,490][75949] Updated weights for policy 0, policy_version 56611 (0.0010) -[2023-10-14 15:47:18,864][75949] Updated weights for policy 0, policy_version 56621 (0.0010) -[2023-10-14 15:47:19,235][75949] Updated weights for policy 0, policy_version 56631 (0.0009) -[2023-10-14 15:47:20,177][75950] Updated weights for policy 1, policy_version 56490 (0.0007) -[2023-10-14 15:47:20,545][75950] Updated weights for policy 1, policy_version 56500 (0.0007) -[2023-10-14 15:47:20,915][75950] Updated weights for policy 1, policy_version 56510 (0.0010) -[2023-10-14 15:47:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 115867648. Throughput: 0: 1685.8, 1: 1670.3. Samples: 28976398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:47:23,165][74987] Avg episode reward: [(0, '25.950'), (1, '30.760')] -[2023-10-14 15:47:23,368][75949] Updated weights for policy 0, policy_version 56641 (0.0007) -[2023-10-14 15:47:23,742][75949] Updated weights for policy 0, policy_version 56651 (0.0009) -[2023-10-14 15:47:24,108][75949] Updated weights for policy 0, policy_version 56661 (0.0009) -[2023-10-14 15:47:24,474][75949] Updated weights for policy 0, policy_version 56671 (0.0008) -[2023-10-14 15:47:25,040][75950] Updated weights for policy 1, policy_version 56520 (0.0008) -[2023-10-14 15:47:25,412][75950] Updated weights for policy 1, policy_version 56530 (0.0009) -[2023-10-14 15:47:25,779][75950] Updated weights for policy 1, policy_version 56540 (0.0009) -[2023-10-14 15:47:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 115933184. Throughput: 0: 1680.5, 1: 1678.6. Samples: 28996940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:47:28,164][74987] Avg episode reward: [(0, '25.990'), (1, '29.960')] -[2023-10-14 15:47:28,639][75949] Updated weights for policy 0, policy_version 56681 (0.0007) -[2023-10-14 15:47:29,004][75949] Updated weights for policy 0, policy_version 56691 (0.0008) -[2023-10-14 15:47:29,380][75949] Updated weights for policy 0, policy_version 56701 (0.0008) -[2023-10-14 15:47:29,876][75950] Updated weights for policy 1, policy_version 56550 (0.0008) -[2023-10-14 15:47:30,229][75950] Updated weights for policy 1, policy_version 56560 (0.0008) -[2023-10-14 15:47:30,594][75950] Updated weights for policy 1, policy_version 56570 (0.0007) -[2023-10-14 15:47:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 115998720. Throughput: 0: 1680.0, 1: 1658.9. Samples: 29006388. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 15:47:33,165][74987] Avg episode reward: [(0, '24.680'), (1, '30.130')] -[2023-10-14 15:47:33,390][75949] Updated weights for policy 0, policy_version 56711 (0.0010) -[2023-10-14 15:47:33,768][75949] Updated weights for policy 0, policy_version 56721 (0.0010) -[2023-10-14 15:47:34,142][75949] Updated weights for policy 0, policy_version 56731 (0.0011) -[2023-10-14 15:47:34,718][75950] Updated weights for policy 1, policy_version 56580 (0.0007) -[2023-10-14 15:47:35,081][75950] Updated weights for policy 1, policy_version 56590 (0.0008) -[2023-10-14 15:47:35,452][75950] Updated weights for policy 1, policy_version 56600 (0.0008) -[2023-10-14 15:47:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116064256. Throughput: 0: 1683.9, 1: 1683.3. Samples: 29026932. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 15:47:38,165][74987] Avg episode reward: [(0, '28.440'), (1, '31.950')] -[2023-10-14 15:47:38,184][75949] Updated weights for policy 0, policy_version 56741 (0.0009) -[2023-10-14 15:47:38,559][75949] Updated weights for policy 0, policy_version 56751 (0.0009) -[2023-10-14 15:47:38,934][75949] Updated weights for policy 0, policy_version 56761 (0.0009) -[2023-10-14 15:47:39,533][75950] Updated weights for policy 1, policy_version 56610 (0.0008) -[2023-10-14 15:47:39,898][75950] Updated weights for policy 1, policy_version 56620 (0.0010) -[2023-10-14 15:47:40,274][75950] Updated weights for policy 1, policy_version 56630 (0.0008) -[2023-10-14 15:47:40,644][75950] Updated weights for policy 1, policy_version 56640 (0.0007) -[2023-10-14 15:47:42,965][75949] Updated weights for policy 0, policy_version 56771 (0.0008) -[2023-10-14 15:47:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.3). Total num frames: 116129792. Throughput: 0: 1688.2, 1: 1681.1. Samples: 29047392. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 15:47:43,165][74987] Avg episode reward: [(0, '24.110'), (1, '32.920')] -[2023-10-14 15:47:43,373][75949] Updated weights for policy 0, policy_version 56781 (0.0008) -[2023-10-14 15:47:43,738][75949] Updated weights for policy 0, policy_version 56791 (0.0009) -[2023-10-14 15:47:44,859][75950] Updated weights for policy 1, policy_version 56650 (0.0008) -[2023-10-14 15:47:45,226][75950] Updated weights for policy 1, policy_version 56660 (0.0007) -[2023-10-14 15:47:45,596][75950] Updated weights for policy 1, policy_version 56670 (0.0007) -[2023-10-14 15:47:47,796][75949] Updated weights for policy 0, policy_version 56801 (0.0010) -[2023-10-14 15:47:48,161][75949] Updated weights for policy 0, policy_version 56811 (0.0007) -[2023-10-14 15:47:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116195328. Throughput: 0: 1688.6, 1: 1657.2. Samples: 29056282. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 15:47:48,164][74987] Avg episode reward: [(0, '28.910'), (1, '30.350')] -[2023-10-14 15:47:48,523][75949] Updated weights for policy 0, policy_version 56821 (0.0008) -[2023-10-14 15:47:48,886][75949] Updated weights for policy 0, policy_version 56831 (0.0011) -[2023-10-14 15:47:49,560][75950] Updated weights for policy 1, policy_version 56680 (0.0008) -[2023-10-14 15:47:49,922][75950] Updated weights for policy 1, policy_version 56690 (0.0011) -[2023-10-14 15:47:50,287][75950] Updated weights for policy 1, policy_version 56700 (0.0010) -[2023-10-14 15:47:53,044][75949] Updated weights for policy 0, policy_version 56841 (0.0010) -[2023-10-14 15:47:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116260864. Throughput: 0: 1679.0, 1: 1666.8. Samples: 29076548. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 15:47:53,165][74987] Avg episode reward: [(0, '23.020'), (1, '30.140')] -[2023-10-14 15:47:53,422][75949] Updated weights for policy 0, policy_version 56851 (0.0010) -[2023-10-14 15:47:53,794][75949] Updated weights for policy 0, policy_version 56861 (0.0011) -[2023-10-14 15:47:54,296][75950] Updated weights for policy 1, policy_version 56710 (0.0009) -[2023-10-14 15:47:54,666][75950] Updated weights for policy 1, policy_version 56720 (0.0008) -[2023-10-14 15:47:55,029][75950] Updated weights for policy 1, policy_version 56730 (0.0007) -[2023-10-14 15:47:57,764][75949] Updated weights for policy 0, policy_version 56871 (0.0011) -[2023-10-14 15:47:58,137][75949] Updated weights for policy 0, policy_version 56881 (0.0008) -[2023-10-14 15:47:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 116326400. Throughput: 0: 1670.9, 1: 1675.5. Samples: 29097296. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 15:47:58,165][74987] Avg episode reward: [(0, '27.010'), (1, '31.040')] -[2023-10-14 15:47:58,506][75949] Updated weights for policy 0, policy_version 56891 (0.0008) -[2023-10-14 15:47:59,012][75950] Updated weights for policy 1, policy_version 56740 (0.0009) -[2023-10-14 15:47:59,386][75950] Updated weights for policy 1, policy_version 56750 (0.0011) -[2023-10-14 15:47:59,754][75950] Updated weights for policy 1, policy_version 56760 (0.0010) -[2023-10-14 15:48:02,641][75949] Updated weights for policy 0, policy_version 56901 (0.0009) -[2023-10-14 15:48:02,999][75949] Updated weights for policy 0, policy_version 56911 (0.0007) -[2023-10-14 15:48:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116391936. Throughput: 0: 1673.1, 1: 1666.0. Samples: 29106590. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 15:48:03,164][74987] Avg episode reward: [(0, '23.890'), (1, '32.060')] -[2023-10-14 15:48:03,367][75949] Updated weights for policy 0, policy_version 56921 (0.0007) -[2023-10-14 15:48:03,922][75950] Updated weights for policy 1, policy_version 56770 (0.0010) -[2023-10-14 15:48:04,294][75950] Updated weights for policy 1, policy_version 56780 (0.0009) -[2023-10-14 15:48:04,653][75950] Updated weights for policy 1, policy_version 56790 (0.0009) -[2023-10-14 15:48:05,020][75950] Updated weights for policy 1, policy_version 56800 (0.0008) -[2023-10-14 15:48:07,402][75949] Updated weights for policy 0, policy_version 56931 (0.0008) -[2023-10-14 15:48:07,774][75949] Updated weights for policy 0, policy_version 56941 (0.0009) -[2023-10-14 15:48:08,145][75949] Updated weights for policy 0, policy_version 56951 (0.0008) -[2023-10-14 15:48:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116457472. Throughput: 0: 1671.8, 1: 1679.0. Samples: 29127184. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 15:48:08,164][74987] Avg episode reward: [(0, '26.320'), (1, '29.570')] -[2023-10-14 15:48:09,235][75950] Updated weights for policy 1, policy_version 56810 (0.0010) -[2023-10-14 15:48:09,610][75950] Updated weights for policy 1, policy_version 56820 (0.0008) -[2023-10-14 15:48:09,973][75950] Updated weights for policy 1, policy_version 56830 (0.0010) -[2023-10-14 15:48:12,252][75949] Updated weights for policy 0, policy_version 56961 (0.0011) -[2023-10-14 15:48:12,621][75949] Updated weights for policy 0, policy_version 56971 (0.0009) -[2023-10-14 15:48:12,988][75949] Updated weights for policy 0, policy_version 56981 (0.0008) -[2023-10-14 15:48:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116523008. Throughput: 0: 1660.2, 1: 1681.4. Samples: 29147312. Policy #0 lag: (min: 26.0, avg: 26.7, max: 45.0) -[2023-10-14 15:48:13,164][74987] Avg episode reward: [(0, '24.150'), (1, '33.760')] -[2023-10-14 15:48:13,347][75949] Updated weights for policy 0, policy_version 56991 (0.0009) -[2023-10-14 15:48:14,041][75950] Updated weights for policy 1, policy_version 56840 (0.0009) -[2023-10-14 15:48:14,409][75950] Updated weights for policy 1, policy_version 56850 (0.0007) -[2023-10-14 15:48:14,770][75950] Updated weights for policy 1, policy_version 56860 (0.0008) -[2023-10-14 15:48:17,476][75949] Updated weights for policy 0, policy_version 57001 (0.0008) -[2023-10-14 15:48:17,847][75949] Updated weights for policy 0, policy_version 57011 (0.0007) -[2023-10-14 15:48:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116588544. Throughput: 0: 1669.8, 1: 1674.5. Samples: 29156884. Policy #0 lag: (min: 26.0, avg: 26.7, max: 45.0) -[2023-10-14 15:48:18,164][74987] Avg episode reward: [(0, '25.850'), (1, '32.000')] -[2023-10-14 15:48:18,207][75949] Updated weights for policy 0, policy_version 57021 (0.0008) -[2023-10-14 15:48:18,912][75950] Updated weights for policy 1, policy_version 56870 (0.0007) -[2023-10-14 15:48:19,266][75950] Updated weights for policy 1, policy_version 56880 (0.0008) -[2023-10-14 15:48:19,636][75950] Updated weights for policy 1, policy_version 56890 (0.0011) -[2023-10-14 15:48:22,464][75949] Updated weights for policy 0, policy_version 57031 (0.0011) -[2023-10-14 15:48:22,836][75949] Updated weights for policy 0, policy_version 57041 (0.0007) -[2023-10-14 15:48:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116654080. Throughput: 0: 1673.9, 1: 1678.0. Samples: 29177768. Policy #0 lag: (min: 26.0, avg: 26.7, max: 45.0) -[2023-10-14 15:48:23,164][74987] Avg episode reward: [(0, '25.750'), (1, '29.630')] -[2023-10-14 15:48:23,204][75949] Updated weights for policy 0, policy_version 57051 (0.0008) -[2023-10-14 15:48:23,677][75950] Updated weights for policy 1, policy_version 56900 (0.0009) -[2023-10-14 15:48:24,040][75950] Updated weights for policy 1, policy_version 56910 (0.0008) -[2023-10-14 15:48:24,403][75950] Updated weights for policy 1, policy_version 56920 (0.0009) -[2023-10-14 15:48:27,219][75949] Updated weights for policy 0, policy_version 57061 (0.0008) -[2023-10-14 15:48:27,585][75949] Updated weights for policy 0, policy_version 57071 (0.0007) -[2023-10-14 15:48:27,962][75949] Updated weights for policy 0, policy_version 57081 (0.0012) -[2023-10-14 15:48:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116719616. Throughput: 0: 1660.2, 1: 1685.8. Samples: 29197964. Policy #0 lag: (min: 26.0, avg: 26.7, max: 45.0) -[2023-10-14 15:48:28,164][74987] Avg episode reward: [(0, '26.690'), (1, '32.360')] -[2023-10-14 15:48:28,494][75950] Updated weights for policy 1, policy_version 56930 (0.0009) -[2023-10-14 15:48:28,857][75950] Updated weights for policy 1, policy_version 56940 (0.0010) -[2023-10-14 15:48:29,230][75950] Updated weights for policy 1, policy_version 56950 (0.0008) -[2023-10-14 15:48:29,590][75950] Updated weights for policy 1, policy_version 56960 (0.0008) -[2023-10-14 15:48:32,074][75949] Updated weights for policy 0, policy_version 57091 (0.0009) -[2023-10-14 15:48:32,451][75949] Updated weights for policy 0, policy_version 57101 (0.0008) -[2023-10-14 15:48:32,830][75949] Updated weights for policy 0, policy_version 57111 (0.0007) -[2023-10-14 15:48:33,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 116817920. Throughput: 0: 1676.9, 1: 1688.2. Samples: 29207710. Policy #0 lag: (min: 26.0, avg: 26.7, max: 45.0) -[2023-10-14 15:48:33,164][74987] Avg episode reward: [(0, '26.440'), (1, '30.170')] -[2023-10-14 15:48:33,658][75950] Updated weights for policy 1, policy_version 56970 (0.0007) -[2023-10-14 15:48:34,025][75950] Updated weights for policy 1, policy_version 56980 (0.0009) -[2023-10-14 15:48:34,397][75950] Updated weights for policy 1, policy_version 56990 (0.0011) -[2023-10-14 15:48:36,820][75949] Updated weights for policy 0, policy_version 57121 (0.0008) -[2023-10-14 15:48:37,196][75949] Updated weights for policy 0, policy_version 57131 (0.0007) -[2023-10-14 15:48:37,566][75949] Updated weights for policy 0, policy_version 57141 (0.0009) -[2023-10-14 15:48:37,944][75949] Updated weights for policy 0, policy_version 57151 (0.0008) -[2023-10-14 15:48:38,163][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.5). Total num frames: 116883456. Throughput: 0: 1679.0, 1: 1693.2. Samples: 29228294. Policy #0 lag: (min: 26.0, avg: 26.7, max: 45.0) -[2023-10-14 15:48:38,164][74987] Avg episode reward: [(0, '25.670'), (1, '29.570')] -[2023-10-14 15:48:38,343][75950] Updated weights for policy 1, policy_version 57000 (0.0008) -[2023-10-14 15:48:38,718][75950] Updated weights for policy 1, policy_version 57010 (0.0008) -[2023-10-14 15:48:39,090][75950] Updated weights for policy 1, policy_version 57020 (0.0010) -[2023-10-14 15:48:41,779][75949] Updated weights for policy 0, policy_version 57161 (0.0009) -[2023-10-14 15:48:42,143][75949] Updated weights for policy 0, policy_version 57171 (0.0009) -[2023-10-14 15:48:42,515][75949] Updated weights for policy 0, policy_version 57181 (0.0009) -[2023-10-14 15:48:43,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 116948992. Throughput: 0: 1658.1, 1: 1688.0. Samples: 29247874. Policy #0 lag: (min: 26.0, avg: 26.7, max: 45.0) -[2023-10-14 15:48:43,165][74987] Avg episode reward: [(0, '25.670'), (1, '30.130')] -[2023-10-14 15:48:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000057184_58556416.pth... -[2023-10-14 15:48:43,209][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000055616_56950784.pth -[2023-10-14 15:48:43,243][75950] Updated weights for policy 1, policy_version 57030 (0.0011) -[2023-10-14 15:48:43,623][75950] Updated weights for policy 1, policy_version 57040 (0.0007) -[2023-10-14 15:48:43,991][75950] Updated weights for policy 1, policy_version 57050 (0.0010) -[2023-10-14 15:48:44,207][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000057056_58425344.pth... -[2023-10-14 15:48:44,239][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000055456_56786944.pth -[2023-10-14 15:48:46,648][75949] Updated weights for policy 0, policy_version 57191 (0.0010) -[2023-10-14 15:48:47,026][75949] Updated weights for policy 0, policy_version 57201 (0.0011) -[2023-10-14 15:48:47,387][75949] Updated weights for policy 0, policy_version 57211 (0.0009) -[2023-10-14 15:48:48,055][75950] Updated weights for policy 1, policy_version 57060 (0.0009) -[2023-10-14 15:48:48,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117014528. Throughput: 0: 1683.8, 1: 1682.7. Samples: 29258084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:48:48,164][74987] Avg episode reward: [(0, '26.620'), (1, '31.180')] -[2023-10-14 15:48:48,416][75950] Updated weights for policy 1, policy_version 57070 (0.0009) -[2023-10-14 15:48:48,789][75950] Updated weights for policy 1, policy_version 57080 (0.0007) -[2023-10-14 15:48:51,466][75949] Updated weights for policy 0, policy_version 57221 (0.0008) -[2023-10-14 15:48:51,842][75949] Updated weights for policy 0, policy_version 57231 (0.0007) -[2023-10-14 15:48:52,212][75949] Updated weights for policy 0, policy_version 57241 (0.0007) -[2023-10-14 15:48:53,058][75950] Updated weights for policy 1, policy_version 57090 (0.0009) -[2023-10-14 15:48:53,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 117080064. Throughput: 0: 1679.2, 1: 1678.0. Samples: 29278256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:48:53,164][74987] Avg episode reward: [(0, '26.060'), (1, '30.600')] -[2023-10-14 15:48:53,430][75950] Updated weights for policy 1, policy_version 57100 (0.0011) -[2023-10-14 15:48:53,788][75950] Updated weights for policy 1, policy_version 57110 (0.0007) -[2023-10-14 15:48:54,156][75950] Updated weights for policy 1, policy_version 57120 (0.0008) -[2023-10-14 15:48:56,115][75949] Updated weights for policy 0, policy_version 57251 (0.0009) -[2023-10-14 15:48:56,484][75949] Updated weights for policy 0, policy_version 57261 (0.0008) -[2023-10-14 15:48:56,860][75949] Updated weights for policy 0, policy_version 57271 (0.0007) -[2023-10-14 15:48:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 117145600. Throughput: 0: 1674.7, 1: 1676.7. Samples: 29298124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:48:58,164][74987] Avg episode reward: [(0, '26.030'), (1, '30.880')] -[2023-10-14 15:48:58,306][75950] Updated weights for policy 1, policy_version 57130 (0.0010) -[2023-10-14 15:48:58,670][75950] Updated weights for policy 1, policy_version 57140 (0.0010) -[2023-10-14 15:48:59,033][75950] Updated weights for policy 1, policy_version 57150 (0.0010) -[2023-10-14 15:49:00,982][75949] Updated weights for policy 0, policy_version 57281 (0.0008) -[2023-10-14 15:49:01,358][75949] Updated weights for policy 0, policy_version 57291 (0.0009) -[2023-10-14 15:49:01,719][75949] Updated weights for policy 0, policy_version 57301 (0.0010) -[2023-10-14 15:49:02,091][75949] Updated weights for policy 0, policy_version 57311 (0.0009) -[2023-10-14 15:49:03,037][75950] Updated weights for policy 1, policy_version 57160 (0.0009) -[2023-10-14 15:49:03,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117211136. Throughput: 0: 1696.9, 1: 1676.4. Samples: 29308684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:49:03,164][74987] Avg episode reward: [(0, '26.110'), (1, '30.670')] -[2023-10-14 15:49:03,404][75950] Updated weights for policy 1, policy_version 57170 (0.0008) -[2023-10-14 15:49:03,764][75950] Updated weights for policy 1, policy_version 57180 (0.0008) -[2023-10-14 15:49:06,055][75949] Updated weights for policy 0, policy_version 57321 (0.0010) -[2023-10-14 15:49:06,420][75949] Updated weights for policy 0, policy_version 57331 (0.0009) -[2023-10-14 15:49:06,790][75949] Updated weights for policy 0, policy_version 57341 (0.0010) -[2023-10-14 15:49:07,862][75950] Updated weights for policy 1, policy_version 57190 (0.0009) -[2023-10-14 15:49:08,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 117276672. Throughput: 0: 1672.7, 1: 1678.4. Samples: 29328564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:49:08,164][74987] Avg episode reward: [(0, '27.530'), (1, '29.050')] -[2023-10-14 15:49:08,234][75950] Updated weights for policy 1, policy_version 57200 (0.0007) -[2023-10-14 15:49:08,601][75950] Updated weights for policy 1, policy_version 57210 (0.0007) -[2023-10-14 15:49:10,905][75949] Updated weights for policy 0, policy_version 57351 (0.0009) -[2023-10-14 15:49:11,274][75949] Updated weights for policy 0, policy_version 57361 (0.0008) -[2023-10-14 15:49:11,648][75949] Updated weights for policy 0, policy_version 57371 (0.0009) -[2023-10-14 15:49:12,654][75950] Updated weights for policy 1, policy_version 57220 (0.0008) -[2023-10-14 15:49:13,023][75950] Updated weights for policy 1, policy_version 57230 (0.0009) -[2023-10-14 15:49:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117342208. Throughput: 0: 1680.8, 1: 1671.3. Samples: 29348806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:49:13,164][74987] Avg episode reward: [(0, '26.570'), (1, '27.920')] -[2023-10-14 15:49:13,390][75950] Updated weights for policy 1, policy_version 57240 (0.0010) -[2023-10-14 15:49:15,769][75949] Updated weights for policy 0, policy_version 57381 (0.0008) -[2023-10-14 15:49:16,138][75949] Updated weights for policy 0, policy_version 57391 (0.0009) -[2023-10-14 15:49:16,505][75949] Updated weights for policy 0, policy_version 57401 (0.0008) -[2023-10-14 15:49:17,610][75950] Updated weights for policy 1, policy_version 57250 (0.0008) -[2023-10-14 15:49:17,972][75950] Updated weights for policy 1, policy_version 57260 (0.0007) -[2023-10-14 15:49:18,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117407744. Throughput: 0: 1691.9, 1: 1670.6. Samples: 29359024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:49:18,165][74987] Avg episode reward: [(0, '26.790'), (1, '29.520')] -[2023-10-14 15:49:18,348][75950] Updated weights for policy 1, policy_version 57270 (0.0007) -[2023-10-14 15:49:18,715][75950] Updated weights for policy 1, policy_version 57280 (0.0007) -[2023-10-14 15:49:20,504][75949] Updated weights for policy 0, policy_version 57411 (0.0008) -[2023-10-14 15:49:20,882][75949] Updated weights for policy 0, policy_version 57421 (0.0010) -[2023-10-14 15:49:21,253][75949] Updated weights for policy 0, policy_version 57431 (0.0011) -[2023-10-14 15:49:22,824][75950] Updated weights for policy 1, policy_version 57290 (0.0008) -[2023-10-14 15:49:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117473280. Throughput: 0: 1669.9, 1: 1671.2. Samples: 29378642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:49:23,164][74987] Avg episode reward: [(0, '24.980'), (1, '29.230')] -[2023-10-14 15:49:23,204][75950] Updated weights for policy 1, policy_version 57300 (0.0008) -[2023-10-14 15:49:23,566][75950] Updated weights for policy 1, policy_version 57310 (0.0008) -[2023-10-14 15:49:25,324][75949] Updated weights for policy 0, policy_version 57441 (0.0011) -[2023-10-14 15:49:25,738][75949] Updated weights for policy 0, policy_version 57451 (0.0008) -[2023-10-14 15:49:26,104][75949] Updated weights for policy 0, policy_version 57461 (0.0009) -[2023-10-14 15:49:26,473][75949] Updated weights for policy 0, policy_version 57471 (0.0009) -[2023-10-14 15:49:27,639][75950] Updated weights for policy 1, policy_version 57320 (0.0009) -[2023-10-14 15:49:28,008][75950] Updated weights for policy 1, policy_version 57330 (0.0008) -[2023-10-14 15:49:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117538816. Throughput: 0: 1697.5, 1: 1659.4. Samples: 29398932. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:49:28,165][74987] Avg episode reward: [(0, '25.630'), (1, '29.150')] -[2023-10-14 15:49:28,382][75950] Updated weights for policy 1, policy_version 57340 (0.0008) -[2023-10-14 15:49:30,529][75949] Updated weights for policy 0, policy_version 57481 (0.0010) -[2023-10-14 15:49:30,904][75949] Updated weights for policy 0, policy_version 57491 (0.0011) -[2023-10-14 15:49:31,273][75949] Updated weights for policy 0, policy_version 57501 (0.0008) -[2023-10-14 15:49:32,257][75950] Updated weights for policy 1, policy_version 57350 (0.0008) -[2023-10-14 15:49:32,618][75950] Updated weights for policy 1, policy_version 57360 (0.0008) -[2023-10-14 15:49:32,986][75950] Updated weights for policy 1, policy_version 57370 (0.0009) -[2023-10-14 15:49:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 117604352. Throughput: 0: 1684.4, 1: 1669.6. Samples: 29409014. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:49:33,164][74987] Avg episode reward: [(0, '25.520'), (1, '30.810')] -[2023-10-14 15:49:35,238][75949] Updated weights for policy 0, policy_version 57511 (0.0009) -[2023-10-14 15:49:35,610][75949] Updated weights for policy 0, policy_version 57521 (0.0009) -[2023-10-14 15:49:35,977][75949] Updated weights for policy 0, policy_version 57531 (0.0008) -[2023-10-14 15:49:37,267][75950] Updated weights for policy 1, policy_version 57380 (0.0009) -[2023-10-14 15:49:37,633][75950] Updated weights for policy 1, policy_version 57390 (0.0008) -[2023-10-14 15:49:37,998][75950] Updated weights for policy 1, policy_version 57400 (0.0008) -[2023-10-14 15:49:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 117669888. Throughput: 0: 1677.6, 1: 1671.1. Samples: 29428948. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:49:38,165][74987] Avg episode reward: [(0, '24.260'), (1, '30.650')] -[2023-10-14 15:49:39,960][75949] Updated weights for policy 0, policy_version 57541 (0.0008) -[2023-10-14 15:49:40,330][75949] Updated weights for policy 0, policy_version 57551 (0.0011) -[2023-10-14 15:49:40,691][75949] Updated weights for policy 0, policy_version 57561 (0.0010) -[2023-10-14 15:49:42,145][75950] Updated weights for policy 1, policy_version 57410 (0.0008) -[2023-10-14 15:49:42,509][75950] Updated weights for policy 1, policy_version 57420 (0.0009) -[2023-10-14 15:49:42,889][75950] Updated weights for policy 1, policy_version 57430 (0.0010) -[2023-10-14 15:49:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 117735424. Throughput: 0: 1699.0, 1: 1658.0. Samples: 29449186. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:49:43,165][74987] Avg episode reward: [(0, '29.100'), (1, '29.850')] -[2023-10-14 15:49:43,251][75950] Updated weights for policy 1, policy_version 57440 (0.0008) -[2023-10-14 15:49:44,759][75949] Updated weights for policy 0, policy_version 57571 (0.0010) -[2023-10-14 15:49:45,137][75949] Updated weights for policy 0, policy_version 57581 (0.0008) -[2023-10-14 15:49:45,499][75949] Updated weights for policy 0, policy_version 57591 (0.0007) -[2023-10-14 15:49:47,318][75950] Updated weights for policy 1, policy_version 57450 (0.0009) -[2023-10-14 15:49:47,686][75950] Updated weights for policy 1, policy_version 57460 (0.0007) -[2023-10-14 15:49:48,053][75950] Updated weights for policy 1, policy_version 57470 (0.0007) -[2023-10-14 15:49:48,163][74987] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117833728. Throughput: 0: 1672.2, 1: 1673.1. Samples: 29459224. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:49:48,164][74987] Avg episode reward: [(0, '24.660'), (1, '30.920')] -[2023-10-14 15:49:49,561][75949] Updated weights for policy 0, policy_version 57601 (0.0007) -[2023-10-14 15:49:49,924][75949] Updated weights for policy 0, policy_version 57611 (0.0009) -[2023-10-14 15:49:50,296][75949] Updated weights for policy 0, policy_version 57621 (0.0012) -[2023-10-14 15:49:50,662][75949] Updated weights for policy 0, policy_version 57631 (0.0011) -[2023-10-14 15:49:52,281][75950] Updated weights for policy 1, policy_version 57480 (0.0008) -[2023-10-14 15:49:52,640][75950] Updated weights for policy 1, policy_version 57490 (0.0008) -[2023-10-14 15:49:53,007][75950] Updated weights for policy 1, policy_version 57500 (0.0007) -[2023-10-14 15:49:53,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117899264. Throughput: 0: 1682.7, 1: 1670.1. Samples: 29479442. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:49:53,165][74987] Avg episode reward: [(0, '28.980'), (1, '29.500')] -[2023-10-14 15:49:54,754][75949] Updated weights for policy 0, policy_version 57641 (0.0009) -[2023-10-14 15:49:55,129][75949] Updated weights for policy 0, policy_version 57651 (0.0009) -[2023-10-14 15:49:55,502][75949] Updated weights for policy 0, policy_version 57661 (0.0007) -[2023-10-14 15:49:57,108][75950] Updated weights for policy 1, policy_version 57510 (0.0009) -[2023-10-14 15:49:57,484][75950] Updated weights for policy 1, policy_version 57520 (0.0008) -[2023-10-14 15:49:57,852][75950] Updated weights for policy 1, policy_version 57530 (0.0010) -[2023-10-14 15:49:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 117964800. Throughput: 0: 1695.9, 1: 1654.4. Samples: 29499572. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:49:58,164][74987] Avg episode reward: [(0, '23.120'), (1, '29.870')] -[2023-10-14 15:49:59,373][75949] Updated weights for policy 0, policy_version 57671 (0.0008) -[2023-10-14 15:49:59,750][75949] Updated weights for policy 0, policy_version 57681 (0.0010) -[2023-10-14 15:50:00,112][75949] Updated weights for policy 0, policy_version 57691 (0.0009) -[2023-10-14 15:50:01,899][75950] Updated weights for policy 1, policy_version 57540 (0.0009) -[2023-10-14 15:50:02,259][75950] Updated weights for policy 1, policy_version 57550 (0.0008) -[2023-10-14 15:50:02,623][75950] Updated weights for policy 1, policy_version 57560 (0.0008) -[2023-10-14 15:50:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118030336. Throughput: 0: 1671.8, 1: 1671.9. Samples: 29509490. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) -[2023-10-14 15:50:03,164][74987] Avg episode reward: [(0, '27.960'), (1, '31.490')] -[2023-10-14 15:50:04,142][75949] Updated weights for policy 0, policy_version 57701 (0.0009) -[2023-10-14 15:50:04,516][75949] Updated weights for policy 0, policy_version 57711 (0.0008) -[2023-10-14 15:50:04,871][75949] Updated weights for policy 0, policy_version 57721 (0.0008) -[2023-10-14 15:50:06,578][75950] Updated weights for policy 1, policy_version 57570 (0.0010) -[2023-10-14 15:50:06,945][75950] Updated weights for policy 1, policy_version 57580 (0.0008) -[2023-10-14 15:50:07,324][75950] Updated weights for policy 1, policy_version 57590 (0.0009) -[2023-10-14 15:50:07,686][75950] Updated weights for policy 1, policy_version 57600 (0.0010) -[2023-10-14 15:50:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118095872. Throughput: 0: 1694.7, 1: 1672.3. Samples: 29530158. Policy #0 lag: (min: 6.0, avg: 7.0, max: 28.0) -[2023-10-14 15:50:08,165][74987] Avg episode reward: [(0, '24.670'), (1, '31.670')] -[2023-10-14 15:50:09,038][75949] Updated weights for policy 0, policy_version 57731 (0.0010) -[2023-10-14 15:50:09,407][75949] Updated weights for policy 0, policy_version 57741 (0.0008) -[2023-10-14 15:50:09,771][75949] Updated weights for policy 0, policy_version 57751 (0.0009) -[2023-10-14 15:50:11,881][75950] Updated weights for policy 1, policy_version 57610 (0.0008) -[2023-10-14 15:50:12,259][75950] Updated weights for policy 1, policy_version 57620 (0.0010) -[2023-10-14 15:50:12,624][75950] Updated weights for policy 1, policy_version 57630 (0.0009) -[2023-10-14 15:50:13,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118161408. Throughput: 0: 1694.5, 1: 1657.3. Samples: 29549766. Policy #0 lag: (min: 6.0, avg: 7.0, max: 28.0) -[2023-10-14 15:50:13,165][74987] Avg episode reward: [(0, '27.590'), (1, '30.540')] -[2023-10-14 15:50:13,850][75949] Updated weights for policy 0, policy_version 57761 (0.0007) -[2023-10-14 15:50:14,268][75949] Updated weights for policy 0, policy_version 57771 (0.0011) -[2023-10-14 15:50:14,639][75949] Updated weights for policy 0, policy_version 57781 (0.0008) -[2023-10-14 15:50:15,012][75949] Updated weights for policy 0, policy_version 57791 (0.0010) -[2023-10-14 15:50:16,789][75950] Updated weights for policy 1, policy_version 57640 (0.0009) -[2023-10-14 15:50:17,145][75950] Updated weights for policy 1, policy_version 57650 (0.0010) -[2023-10-14 15:50:17,513][75950] Updated weights for policy 1, policy_version 57660 (0.0008) -[2023-10-14 15:50:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 118226944. Throughput: 0: 1676.0, 1: 1674.6. Samples: 29559790. Policy #0 lag: (min: 6.0, avg: 7.0, max: 28.0) -[2023-10-14 15:50:18,164][74987] Avg episode reward: [(0, '23.770'), (1, '31.790')] -[2023-10-14 15:50:19,121][75949] Updated weights for policy 0, policy_version 57801 (0.0008) -[2023-10-14 15:50:19,494][75949] Updated weights for policy 0, policy_version 57811 (0.0007) -[2023-10-14 15:50:19,869][75949] Updated weights for policy 0, policy_version 57821 (0.0008) -[2023-10-14 15:50:21,506][75950] Updated weights for policy 1, policy_version 57670 (0.0009) -[2023-10-14 15:50:21,884][75950] Updated weights for policy 1, policy_version 57680 (0.0007) -[2023-10-14 15:50:22,246][75950] Updated weights for policy 1, policy_version 57690 (0.0011) -[2023-10-14 15:50:23,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118292480. Throughput: 0: 1687.2, 1: 1670.4. Samples: 29580042. Policy #0 lag: (min: 6.0, avg: 7.0, max: 28.0) -[2023-10-14 15:50:23,164][74987] Avg episode reward: [(0, '25.930'), (1, '30.700')] -[2023-10-14 15:50:23,833][75949] Updated weights for policy 0, policy_version 57831 (0.0009) -[2023-10-14 15:50:24,198][75949] Updated weights for policy 0, policy_version 57841 (0.0007) -[2023-10-14 15:50:24,569][75949] Updated weights for policy 0, policy_version 57851 (0.0007) -[2023-10-14 15:50:26,209][75950] Updated weights for policy 1, policy_version 57700 (0.0010) -[2023-10-14 15:50:26,583][75950] Updated weights for policy 1, policy_version 57710 (0.0010) -[2023-10-14 15:50:26,950][75950] Updated weights for policy 1, policy_version 57720 (0.0009) -[2023-10-14 15:50:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118358016. Throughput: 0: 1686.5, 1: 1666.6. Samples: 29600076. Policy #0 lag: (min: 6.0, avg: 7.0, max: 28.0) -[2023-10-14 15:50:28,165][74987] Avg episode reward: [(0, '25.750'), (1, '30.780')] -[2023-10-14 15:50:28,725][75949] Updated weights for policy 0, policy_version 57861 (0.0009) -[2023-10-14 15:50:29,093][75949] Updated weights for policy 0, policy_version 57871 (0.0007) -[2023-10-14 15:50:29,463][75949] Updated weights for policy 0, policy_version 57881 (0.0007) -[2023-10-14 15:50:30,902][75950] Updated weights for policy 1, policy_version 57730 (0.0009) -[2023-10-14 15:50:31,268][75950] Updated weights for policy 1, policy_version 57740 (0.0009) -[2023-10-14 15:50:31,635][75950] Updated weights for policy 1, policy_version 57750 (0.0007) -[2023-10-14 15:50:32,004][75950] Updated weights for policy 1, policy_version 57760 (0.0007) -[2023-10-14 15:50:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118423552. Throughput: 0: 1679.2, 1: 1681.7. Samples: 29610462. Policy #0 lag: (min: 6.0, avg: 7.0, max: 28.0) -[2023-10-14 15:50:33,164][74987] Avg episode reward: [(0, '26.370'), (1, '31.390')] -[2023-10-14 15:50:33,635][75949] Updated weights for policy 0, policy_version 57891 (0.0008) -[2023-10-14 15:50:33,998][75949] Updated weights for policy 0, policy_version 57901 (0.0009) -[2023-10-14 15:50:34,376][75949] Updated weights for policy 0, policy_version 57911 (0.0010) -[2023-10-14 15:50:35,833][75950] Updated weights for policy 1, policy_version 57770 (0.0009) -[2023-10-14 15:50:36,206][75950] Updated weights for policy 1, policy_version 57780 (0.0007) -[2023-10-14 15:50:36,570][75950] Updated weights for policy 1, policy_version 57790 (0.0009) -[2023-10-14 15:50:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 118489088. Throughput: 0: 1683.7, 1: 1664.0. Samples: 29630088. Policy #0 lag: (min: 6.0, avg: 7.0, max: 28.0) -[2023-10-14 15:50:38,164][74987] Avg episode reward: [(0, '27.540'), (1, '31.260')] -[2023-10-14 15:50:38,428][75949] Updated weights for policy 0, policy_version 57921 (0.0009) -[2023-10-14 15:50:38,807][75949] Updated weights for policy 0, policy_version 57931 (0.0007) -[2023-10-14 15:50:39,172][75949] Updated weights for policy 0, policy_version 57941 (0.0007) -[2023-10-14 15:50:39,540][75949] Updated weights for policy 0, policy_version 57951 (0.0008) -[2023-10-14 15:50:40,653][75950] Updated weights for policy 1, policy_version 57800 (0.0008) -[2023-10-14 15:50:41,026][75950] Updated weights for policy 1, policy_version 57810 (0.0007) -[2023-10-14 15:50:41,394][75950] Updated weights for policy 1, policy_version 57820 (0.0008) -[2023-10-14 15:50:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118554624. Throughput: 0: 1677.9, 1: 1680.3. Samples: 29650692. Policy #0 lag: (min: 6.0, avg: 7.0, max: 28.0) -[2023-10-14 15:50:43,165][74987] Avg episode reward: [(0, '25.950'), (1, '32.050')] -[2023-10-14 15:50:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000057824_59211776.pth... -[2023-10-14 15:50:43,214][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000056256_57606144.pth -[2023-10-14 15:50:43,530][75949] Updated weights for policy 0, policy_version 57961 (0.0007) -[2023-10-14 15:50:43,899][75949] Updated weights for policy 0, policy_version 57971 (0.0008) -[2023-10-14 15:50:44,273][75949] Updated weights for policy 0, policy_version 57981 (0.0008) -[2023-10-14 15:50:44,377][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000057984_59375616.pth... -[2023-10-14 15:50:44,406][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000056384_57737216.pth -[2023-10-14 15:50:45,657][75950] Updated weights for policy 1, policy_version 57830 (0.0009) -[2023-10-14 15:50:46,027][75950] Updated weights for policy 1, policy_version 57840 (0.0007) -[2023-10-14 15:50:46,409][75950] Updated weights for policy 1, policy_version 57850 (0.0010) -[2023-10-14 15:50:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 118620160. Throughput: 0: 1676.7, 1: 1687.4. Samples: 29660872. Policy #0 lag: (min: 23.0, avg: 24.1, max: 45.0) -[2023-10-14 15:50:48,164][74987] Avg episode reward: [(0, '27.640'), (1, '32.280')] -[2023-10-14 15:50:48,396][75949] Updated weights for policy 0, policy_version 57991 (0.0008) -[2023-10-14 15:50:48,770][75949] Updated weights for policy 0, policy_version 58001 (0.0011) -[2023-10-14 15:50:49,148][75949] Updated weights for policy 0, policy_version 58011 (0.0009) -[2023-10-14 15:50:50,634][75950] Updated weights for policy 1, policy_version 57860 (0.0010) -[2023-10-14 15:50:50,998][75950] Updated weights for policy 1, policy_version 57870 (0.0009) -[2023-10-14 15:50:51,355][75950] Updated weights for policy 1, policy_version 57880 (0.0008) -[2023-10-14 15:50:53,077][75949] Updated weights for policy 0, policy_version 58021 (0.0007) -[2023-10-14 15:50:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 118685696. Throughput: 0: 1679.2, 1: 1665.2. Samples: 29680660. Policy #0 lag: (min: 23.0, avg: 24.1, max: 45.0) -[2023-10-14 15:50:53,164][74987] Avg episode reward: [(0, '25.580'), (1, '32.680')] -[2023-10-14 15:50:53,449][75949] Updated weights for policy 0, policy_version 58031 (0.0007) -[2023-10-14 15:50:53,818][75949] Updated weights for policy 0, policy_version 58041 (0.0008) -[2023-10-14 15:50:55,260][75950] Updated weights for policy 1, policy_version 57890 (0.0007) -[2023-10-14 15:50:55,615][75950] Updated weights for policy 1, policy_version 57900 (0.0008) -[2023-10-14 15:50:55,986][75950] Updated weights for policy 1, policy_version 57910 (0.0008) -[2023-10-14 15:50:56,348][75950] Updated weights for policy 1, policy_version 57920 (0.0008) -[2023-10-14 15:50:57,650][75949] Updated weights for policy 0, policy_version 58051 (0.0008) -[2023-10-14 15:50:58,016][75949] Updated weights for policy 0, policy_version 58061 (0.0008) -[2023-10-14 15:50:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 118751232. Throughput: 0: 1683.8, 1: 1689.4. Samples: 29701560. Policy #0 lag: (min: 23.0, avg: 24.1, max: 45.0) -[2023-10-14 15:50:58,165][74987] Avg episode reward: [(0, '27.180'), (1, '31.930')] -[2023-10-14 15:50:58,375][75949] Updated weights for policy 0, policy_version 58071 (0.0009) -[2023-10-14 15:51:00,640][75950] Updated weights for policy 1, policy_version 57930 (0.0010) -[2023-10-14 15:51:01,015][75950] Updated weights for policy 1, policy_version 57940 (0.0010) -[2023-10-14 15:51:01,376][75950] Updated weights for policy 1, policy_version 57950 (0.0009) -[2023-10-14 15:51:02,322][75949] Updated weights for policy 0, policy_version 58081 (0.0009) -[2023-10-14 15:51:02,690][75949] Updated weights for policy 0, policy_version 58091 (0.0011) -[2023-10-14 15:51:03,070][75949] Updated weights for policy 0, policy_version 58101 (0.0009) -[2023-10-14 15:51:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 118816768. Throughput: 0: 1696.2, 1: 1679.2. Samples: 29711682. Policy #0 lag: (min: 23.0, avg: 24.1, max: 45.0) -[2023-10-14 15:51:03,164][74987] Avg episode reward: [(0, '25.260'), (1, '32.330')] -[2023-10-14 15:51:03,426][75949] Updated weights for policy 0, policy_version 58111 (0.0008) -[2023-10-14 15:51:05,463][75950] Updated weights for policy 1, policy_version 57960 (0.0010) -[2023-10-14 15:51:05,835][75950] Updated weights for policy 1, policy_version 57970 (0.0011) -[2023-10-14 15:51:06,196][75950] Updated weights for policy 1, policy_version 57980 (0.0011) -[2023-10-14 15:51:07,557][75949] Updated weights for policy 0, policy_version 58121 (0.0008) -[2023-10-14 15:51:07,926][75949] Updated weights for policy 0, policy_version 58131 (0.0007) -[2023-10-14 15:51:08,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 118882304. Throughput: 0: 1696.2, 1: 1667.0. Samples: 29731386. Policy #0 lag: (min: 23.0, avg: 24.1, max: 45.0) -[2023-10-14 15:51:08,164][74987] Avg episode reward: [(0, '26.800'), (1, '32.840')] -[2023-10-14 15:51:08,293][75949] Updated weights for policy 0, policy_version 58141 (0.0008) -[2023-10-14 15:51:10,354][75950] Updated weights for policy 1, policy_version 57990 (0.0009) -[2023-10-14 15:51:10,717][75950] Updated weights for policy 1, policy_version 58000 (0.0008) -[2023-10-14 15:51:11,093][75950] Updated weights for policy 1, policy_version 58010 (0.0008) -[2023-10-14 15:51:12,245][75949] Updated weights for policy 0, policy_version 58151 (0.0007) -[2023-10-14 15:51:12,613][75949] Updated weights for policy 0, policy_version 58161 (0.0007) -[2023-10-14 15:51:12,991][75949] Updated weights for policy 0, policy_version 58171 (0.0007) -[2023-10-14 15:51:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 118947840. Throughput: 0: 1679.5, 1: 1684.2. Samples: 29751440. Policy #0 lag: (min: 23.0, avg: 24.1, max: 45.0) -[2023-10-14 15:51:13,164][74987] Avg episode reward: [(0, '23.610'), (1, '32.670')] -[2023-10-14 15:51:15,188][75950] Updated weights for policy 1, policy_version 58020 (0.0010) -[2023-10-14 15:51:15,550][75950] Updated weights for policy 1, policy_version 58030 (0.0010) -[2023-10-14 15:51:15,929][75950] Updated weights for policy 1, policy_version 58040 (0.0010) -[2023-10-14 15:51:17,120][75949] Updated weights for policy 0, policy_version 58181 (0.0010) -[2023-10-14 15:51:17,500][75949] Updated weights for policy 0, policy_version 58191 (0.0010) -[2023-10-14 15:51:17,861][75949] Updated weights for policy 0, policy_version 58201 (0.0009) -[2023-10-14 15:51:18,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 119046144. Throughput: 0: 1694.2, 1: 1668.1. Samples: 29761766. Policy #0 lag: (min: 23.0, avg: 24.1, max: 45.0) -[2023-10-14 15:51:18,164][74987] Avg episode reward: [(0, '27.650'), (1, '31.240')] -[2023-10-14 15:51:19,878][75950] Updated weights for policy 1, policy_version 58050 (0.0009) -[2023-10-14 15:51:20,240][75950] Updated weights for policy 1, policy_version 58060 (0.0009) -[2023-10-14 15:51:20,604][75950] Updated weights for policy 1, policy_version 58070 (0.0007) -[2023-10-14 15:51:20,960][75950] Updated weights for policy 1, policy_version 58080 (0.0008) -[2023-10-14 15:51:21,875][75949] Updated weights for policy 0, policy_version 58211 (0.0011) -[2023-10-14 15:51:22,234][75949] Updated weights for policy 0, policy_version 58221 (0.0010) -[2023-10-14 15:51:22,600][75949] Updated weights for policy 0, policy_version 58231 (0.0008) -[2023-10-14 15:51:23,164][74987] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 119111680. Throughput: 0: 1697.0, 1: 1671.2. Samples: 29781658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:51:23,165][74987] Avg episode reward: [(0, '25.830'), (1, '31.380')] -[2023-10-14 15:51:24,867][75950] Updated weights for policy 1, policy_version 58090 (0.0009) -[2023-10-14 15:51:25,235][75950] Updated weights for policy 1, policy_version 58100 (0.0007) -[2023-10-14 15:51:25,588][75950] Updated weights for policy 1, policy_version 58110 (0.0009) -[2023-10-14 15:51:26,710][75949] Updated weights for policy 0, policy_version 58241 (0.0009) -[2023-10-14 15:51:27,086][75949] Updated weights for policy 0, policy_version 58251 (0.0009) -[2023-10-14 15:51:27,462][75949] Updated weights for policy 0, policy_version 58261 (0.0010) -[2023-10-14 15:51:27,829][75949] Updated weights for policy 0, policy_version 58271 (0.0009) -[2023-10-14 15:51:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 119177216. Throughput: 0: 1674.5, 1: 1671.4. Samples: 29801254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:51:28,164][74987] Avg episode reward: [(0, '26.930'), (1, '30.890')] -[2023-10-14 15:51:29,869][75950] Updated weights for policy 1, policy_version 58120 (0.0009) -[2023-10-14 15:51:30,236][75950] Updated weights for policy 1, policy_version 58130 (0.0007) -[2023-10-14 15:51:30,595][75950] Updated weights for policy 1, policy_version 58140 (0.0009) -[2023-10-14 15:51:31,949][75949] Updated weights for policy 0, policy_version 58281 (0.0010) -[2023-10-14 15:51:32,316][75949] Updated weights for policy 0, policy_version 58291 (0.0007) -[2023-10-14 15:51:32,685][75949] Updated weights for policy 0, policy_version 58301 (0.0007) -[2023-10-14 15:51:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 119242752. Throughput: 0: 1697.5, 1: 1650.5. Samples: 29811536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:51:33,165][74987] Avg episode reward: [(0, '25.430'), (1, '29.410')] -[2023-10-14 15:51:34,752][75950] Updated weights for policy 1, policy_version 58150 (0.0010) -[2023-10-14 15:51:35,118][75950] Updated weights for policy 1, policy_version 58160 (0.0010) -[2023-10-14 15:51:35,477][75950] Updated weights for policy 1, policy_version 58170 (0.0010) -[2023-10-14 15:51:36,636][75949] Updated weights for policy 0, policy_version 58311 (0.0007) -[2023-10-14 15:51:37,004][75949] Updated weights for policy 0, policy_version 58321 (0.0010) -[2023-10-14 15:51:37,377][75949] Updated weights for policy 0, policy_version 58331 (0.0011) -[2023-10-14 15:51:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 119308288. Throughput: 0: 1689.1, 1: 1664.9. Samples: 29831588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:51:38,165][74987] Avg episode reward: [(0, '25.380'), (1, '31.530')] -[2023-10-14 15:51:39,639][75950] Updated weights for policy 1, policy_version 58180 (0.0008) -[2023-10-14 15:51:40,012][75950] Updated weights for policy 1, policy_version 58190 (0.0007) -[2023-10-14 15:51:40,378][75950] Updated weights for policy 1, policy_version 58200 (0.0007) -[2023-10-14 15:51:41,710][75949] Updated weights for policy 0, policy_version 58341 (0.0009) -[2023-10-14 15:51:42,091][75949] Updated weights for policy 0, policy_version 58351 (0.0007) -[2023-10-14 15:51:42,461][75949] Updated weights for policy 0, policy_version 58361 (0.0008) -[2023-10-14 15:51:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 119373824. Throughput: 0: 1659.3, 1: 1668.7. Samples: 29851320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:51:43,164][74987] Avg episode reward: [(0, '27.330'), (1, '33.320')] -[2023-10-14 15:51:44,560][75950] Updated weights for policy 1, policy_version 58210 (0.0008) -[2023-10-14 15:51:44,932][75950] Updated weights for policy 1, policy_version 58220 (0.0008) -[2023-10-14 15:51:45,296][75950] Updated weights for policy 1, policy_version 58230 (0.0008) -[2023-10-14 15:51:45,658][75950] Updated weights for policy 1, policy_version 58240 (0.0008) -[2023-10-14 15:51:46,882][75949] Updated weights for policy 0, policy_version 58371 (0.0008) -[2023-10-14 15:51:47,258][75949] Updated weights for policy 0, policy_version 58381 (0.0010) -[2023-10-14 15:51:47,624][75949] Updated weights for policy 0, policy_version 58391 (0.0008) -[2023-10-14 15:51:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 119439360. Throughput: 0: 1672.8, 1: 1654.5. Samples: 29861412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:51:48,165][74987] Avg episode reward: [(0, '25.610'), (1, '34.680')] -[2023-10-14 15:51:48,166][75801] Saving new best policy, reward=34.680! -[2023-10-14 15:51:49,739][75950] Updated weights for policy 1, policy_version 58250 (0.0007) -[2023-10-14 15:51:50,108][75950] Updated weights for policy 1, policy_version 58260 (0.0009) -[2023-10-14 15:51:50,477][75950] Updated weights for policy 1, policy_version 58270 (0.0010) -[2023-10-14 15:51:51,784][75949] Updated weights for policy 0, policy_version 58401 (0.0009) -[2023-10-14 15:51:52,204][75949] Updated weights for policy 0, policy_version 58411 (0.0009) -[2023-10-14 15:51:52,570][75949] Updated weights for policy 0, policy_version 58421 (0.0008) -[2023-10-14 15:51:52,951][75949] Updated weights for policy 0, policy_version 58431 (0.0010) -[2023-10-14 15:51:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 119504896. Throughput: 0: 1674.0, 1: 1670.8. Samples: 29881900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:51:53,164][74987] Avg episode reward: [(0, '28.080'), (1, '32.670')] -[2023-10-14 15:51:54,474][75950] Updated weights for policy 1, policy_version 58280 (0.0008) -[2023-10-14 15:51:54,836][75950] Updated weights for policy 1, policy_version 58290 (0.0007) -[2023-10-14 15:51:55,218][75950] Updated weights for policy 1, policy_version 58300 (0.0009) -[2023-10-14 15:51:56,832][75949] Updated weights for policy 0, policy_version 58441 (0.0010) -[2023-10-14 15:51:57,208][75949] Updated weights for policy 0, policy_version 58451 (0.0010) -[2023-10-14 15:51:57,588][75949] Updated weights for policy 0, policy_version 58461 (0.0010) -[2023-10-14 15:51:58,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 119570432. Throughput: 0: 1658.0, 1: 1676.6. Samples: 29901498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:51:58,164][74987] Avg episode reward: [(0, '24.920'), (1, '31.940')] -[2023-10-14 15:51:59,263][75950] Updated weights for policy 1, policy_version 58310 (0.0007) -[2023-10-14 15:51:59,627][75950] Updated weights for policy 1, policy_version 58320 (0.0008) -[2023-10-14 15:51:59,993][75950] Updated weights for policy 1, policy_version 58330 (0.0008) -[2023-10-14 15:52:01,628][75949] Updated weights for policy 0, policy_version 58471 (0.0008) -[2023-10-14 15:52:01,996][75949] Updated weights for policy 0, policy_version 58481 (0.0007) -[2023-10-14 15:52:02,370][75949] Updated weights for policy 0, policy_version 58491 (0.0007) -[2023-10-14 15:52:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 119635968. Throughput: 0: 1673.8, 1: 1660.9. Samples: 29911826. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) -[2023-10-14 15:52:03,165][74987] Avg episode reward: [(0, '27.330'), (1, '32.030')] -[2023-10-14 15:52:04,057][75950] Updated weights for policy 1, policy_version 58340 (0.0009) -[2023-10-14 15:52:04,419][75950] Updated weights for policy 1, policy_version 58350 (0.0008) -[2023-10-14 15:52:04,777][75950] Updated weights for policy 1, policy_version 58360 (0.0008) -[2023-10-14 15:52:06,496][75949] Updated weights for policy 0, policy_version 58501 (0.0009) -[2023-10-14 15:52:06,872][75949] Updated weights for policy 0, policy_version 58511 (0.0010) -[2023-10-14 15:52:07,235][75949] Updated weights for policy 0, policy_version 58521 (0.0011) -[2023-10-14 15:52:08,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 119701504. Throughput: 0: 1665.1, 1: 1676.5. Samples: 29932032. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) -[2023-10-14 15:52:08,165][74987] Avg episode reward: [(0, '23.890'), (1, '29.690')] -[2023-10-14 15:52:08,795][75950] Updated weights for policy 1, policy_version 58370 (0.0010) -[2023-10-14 15:52:09,155][75950] Updated weights for policy 1, policy_version 58380 (0.0008) -[2023-10-14 15:52:09,526][75950] Updated weights for policy 1, policy_version 58390 (0.0010) -[2023-10-14 15:52:09,888][75950] Updated weights for policy 1, policy_version 58400 (0.0010) -[2023-10-14 15:52:11,263][75949] Updated weights for policy 0, policy_version 58531 (0.0009) -[2023-10-14 15:52:11,632][75949] Updated weights for policy 0, policy_version 58541 (0.0010) -[2023-10-14 15:52:12,001][75949] Updated weights for policy 0, policy_version 58551 (0.0010) -[2023-10-14 15:52:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 119767040. Throughput: 0: 1664.0, 1: 1682.0. Samples: 29951822. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) -[2023-10-14 15:52:13,165][74987] Avg episode reward: [(0, '25.980'), (1, '32.540')] -[2023-10-14 15:52:13,976][75950] Updated weights for policy 1, policy_version 58410 (0.0008) -[2023-10-14 15:52:14,340][75950] Updated weights for policy 1, policy_version 58420 (0.0009) -[2023-10-14 15:52:14,701][75950] Updated weights for policy 1, policy_version 58430 (0.0009) -[2023-10-14 15:52:15,957][75949] Updated weights for policy 0, policy_version 58561 (0.0010) -[2023-10-14 15:52:16,327][75949] Updated weights for policy 0, policy_version 58571 (0.0008) -[2023-10-14 15:52:16,696][75949] Updated weights for policy 0, policy_version 58581 (0.0009) -[2023-10-14 15:52:17,077][75949] Updated weights for policy 0, policy_version 58591 (0.0011) -[2023-10-14 15:52:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 119832576. Throughput: 0: 1668.4, 1: 1676.5. Samples: 29962056. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) -[2023-10-14 15:52:18,165][74987] Avg episode reward: [(0, '27.290'), (1, '32.820')] -[2023-10-14 15:52:18,559][75950] Updated weights for policy 1, policy_version 58440 (0.0010) -[2023-10-14 15:52:18,931][75950] Updated weights for policy 1, policy_version 58450 (0.0008) -[2023-10-14 15:52:19,291][75950] Updated weights for policy 1, policy_version 58460 (0.0009) -[2023-10-14 15:52:21,101][75949] Updated weights for policy 0, policy_version 58601 (0.0011) -[2023-10-14 15:52:21,468][75949] Updated weights for policy 0, policy_version 58611 (0.0007) -[2023-10-14 15:52:21,837][75949] Updated weights for policy 0, policy_version 58621 (0.0008) -[2023-10-14 15:52:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 119898112. Throughput: 0: 1657.3, 1: 1690.4. Samples: 29982232. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) -[2023-10-14 15:52:23,164][74987] Avg episode reward: [(0, '26.000'), (1, '31.600')] -[2023-10-14 15:52:23,364][75950] Updated weights for policy 1, policy_version 58470 (0.0007) -[2023-10-14 15:52:23,746][75950] Updated weights for policy 1, policy_version 58480 (0.0008) -[2023-10-14 15:52:24,113][75950] Updated weights for policy 1, policy_version 58490 (0.0009) -[2023-10-14 15:52:25,936][75949] Updated weights for policy 0, policy_version 58631 (0.0008) -[2023-10-14 15:52:26,300][75949] Updated weights for policy 0, policy_version 58641 (0.0008) -[2023-10-14 15:52:26,675][75949] Updated weights for policy 0, policy_version 58651 (0.0008) -[2023-10-14 15:52:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 119963648. Throughput: 0: 1670.4, 1: 1687.8. Samples: 30002438. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) -[2023-10-14 15:52:28,164][74987] Avg episode reward: [(0, '27.050'), (1, '32.200')] -[2023-10-14 15:52:28,192][75950] Updated weights for policy 1, policy_version 58500 (0.0008) -[2023-10-14 15:52:28,567][75950] Updated weights for policy 1, policy_version 58510 (0.0008) -[2023-10-14 15:52:28,926][75950] Updated weights for policy 1, policy_version 58520 (0.0008) -[2023-10-14 15:52:30,780][75949] Updated weights for policy 0, policy_version 58661 (0.0010) -[2023-10-14 15:52:31,157][75949] Updated weights for policy 0, policy_version 58671 (0.0008) -[2023-10-14 15:52:31,525][75949] Updated weights for policy 0, policy_version 58681 (0.0008) -[2023-10-14 15:52:33,027][75950] Updated weights for policy 1, policy_version 58530 (0.0007) -[2023-10-14 15:52:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120029184. Throughput: 0: 1674.9, 1: 1688.8. Samples: 30012778. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) -[2023-10-14 15:52:33,164][74987] Avg episode reward: [(0, '25.470'), (1, '32.600')] -[2023-10-14 15:52:33,400][75950] Updated weights for policy 1, policy_version 58540 (0.0008) -[2023-10-14 15:52:33,764][75950] Updated weights for policy 1, policy_version 58550 (0.0012) -[2023-10-14 15:52:34,131][75950] Updated weights for policy 1, policy_version 58560 (0.0009) -[2023-10-14 15:52:35,491][75949] Updated weights for policy 0, policy_version 58691 (0.0008) -[2023-10-14 15:52:35,857][75949] Updated weights for policy 0, policy_version 58701 (0.0010) -[2023-10-14 15:52:36,233][75949] Updated weights for policy 0, policy_version 58711 (0.0007) -[2023-10-14 15:52:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120094720. Throughput: 0: 1650.7, 1: 1693.6. Samples: 30032398. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) -[2023-10-14 15:52:38,165][74987] Avg episode reward: [(0, '26.290'), (1, '31.730')] -[2023-10-14 15:52:38,198][75950] Updated weights for policy 1, policy_version 58570 (0.0008) -[2023-10-14 15:52:38,574][75950] Updated weights for policy 1, policy_version 58580 (0.0009) -[2023-10-14 15:52:38,934][75950] Updated weights for policy 1, policy_version 58590 (0.0008) -[2023-10-14 15:52:40,246][75949] Updated weights for policy 0, policy_version 58721 (0.0008) -[2023-10-14 15:52:40,655][75949] Updated weights for policy 0, policy_version 58731 (0.0009) -[2023-10-14 15:52:41,033][75949] Updated weights for policy 0, policy_version 58741 (0.0009) -[2023-10-14 15:52:41,403][75949] Updated weights for policy 0, policy_version 58751 (0.0011) -[2023-10-14 15:52:42,964][75950] Updated weights for policy 1, policy_version 58600 (0.0010) -[2023-10-14 15:52:43,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120160256. Throughput: 0: 1683.0, 1: 1689.1. Samples: 30053244. Policy #0 lag: (min: 2.0, avg: 2.3, max: 13.0) -[2023-10-14 15:52:43,165][74987] Avg episode reward: [(0, '25.070'), (1, '30.910')] -[2023-10-14 15:52:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000058752_60162048.pth... -[2023-10-14 15:52:43,211][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000057184_58556416.pth -[2023-10-14 15:52:43,330][75950] Updated weights for policy 1, policy_version 58610 (0.0009) -[2023-10-14 15:52:43,696][75950] Updated weights for policy 1, policy_version 58620 (0.0007) -[2023-10-14 15:52:43,839][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000058624_60030976.pth... -[2023-10-14 15:52:43,868][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000057056_58425344.pth -[2023-10-14 15:52:45,405][75949] Updated weights for policy 0, policy_version 58761 (0.0010) -[2023-10-14 15:52:45,769][75949] Updated weights for policy 0, policy_version 58771 (0.0009) -[2023-10-14 15:52:46,135][75949] Updated weights for policy 0, policy_version 58781 (0.0010) -[2023-10-14 15:52:47,923][75950] Updated weights for policy 1, policy_version 58630 (0.0009) -[2023-10-14 15:52:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120225792. Throughput: 0: 1672.0, 1: 1688.8. Samples: 30063066. Policy #0 lag: (min: 2.0, avg: 2.3, max: 13.0) -[2023-10-14 15:52:48,165][74987] Avg episode reward: [(0, '28.800'), (1, '32.580')] -[2023-10-14 15:52:48,287][75950] Updated weights for policy 1, policy_version 58640 (0.0008) -[2023-10-14 15:52:48,647][75950] Updated weights for policy 1, policy_version 58650 (0.0008) -[2023-10-14 15:52:50,366][75949] Updated weights for policy 0, policy_version 58791 (0.0008) -[2023-10-14 15:52:50,729][75949] Updated weights for policy 0, policy_version 58801 (0.0010) -[2023-10-14 15:52:51,103][75949] Updated weights for policy 0, policy_version 58811 (0.0010) -[2023-10-14 15:52:52,840][75950] Updated weights for policy 1, policy_version 58660 (0.0008) -[2023-10-14 15:52:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120291328. Throughput: 0: 1662.0, 1: 1686.5. Samples: 30082712. Policy #0 lag: (min: 2.0, avg: 2.3, max: 13.0) -[2023-10-14 15:52:53,164][74987] Avg episode reward: [(0, '25.360'), (1, '33.900')] -[2023-10-14 15:52:53,206][75950] Updated weights for policy 1, policy_version 58670 (0.0007) -[2023-10-14 15:52:53,578][75950] Updated weights for policy 1, policy_version 58680 (0.0008) -[2023-10-14 15:52:55,176][75949] Updated weights for policy 0, policy_version 58821 (0.0010) -[2023-10-14 15:52:55,544][75949] Updated weights for policy 0, policy_version 58831 (0.0008) -[2023-10-14 15:52:55,921][75949] Updated weights for policy 0, policy_version 58841 (0.0009) -[2023-10-14 15:52:57,639][75950] Updated weights for policy 1, policy_version 58690 (0.0008) -[2023-10-14 15:52:58,007][75950] Updated weights for policy 1, policy_version 58700 (0.0008) -[2023-10-14 15:52:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120356864. Throughput: 0: 1686.8, 1: 1680.0. Samples: 30103324. Policy #0 lag: (min: 2.0, avg: 2.3, max: 13.0) -[2023-10-14 15:52:58,164][74987] Avg episode reward: [(0, '29.500'), (1, '32.110')] -[2023-10-14 15:52:58,376][75950] Updated weights for policy 1, policy_version 58710 (0.0009) -[2023-10-14 15:52:58,735][75950] Updated weights for policy 1, policy_version 58720 (0.0008) -[2023-10-14 15:52:59,906][75949] Updated weights for policy 0, policy_version 58851 (0.0008) -[2023-10-14 15:53:00,270][75949] Updated weights for policy 0, policy_version 58861 (0.0009) -[2023-10-14 15:53:00,643][75949] Updated weights for policy 0, policy_version 58871 (0.0008) -[2023-10-14 15:53:02,871][75950] Updated weights for policy 1, policy_version 58730 (0.0009) -[2023-10-14 15:53:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120422400. Throughput: 0: 1668.9, 1: 1685.1. Samples: 30112984. Policy #0 lag: (min: 2.0, avg: 2.3, max: 13.0) -[2023-10-14 15:53:03,164][74987] Avg episode reward: [(0, '25.500'), (1, '32.150')] -[2023-10-14 15:53:03,239][75950] Updated weights for policy 1, policy_version 58740 (0.0008) -[2023-10-14 15:53:03,605][75950] Updated weights for policy 1, policy_version 58750 (0.0008) -[2023-10-14 15:53:04,719][75949] Updated weights for policy 0, policy_version 58881 (0.0008) -[2023-10-14 15:53:05,084][75949] Updated weights for policy 0, policy_version 58891 (0.0009) -[2023-10-14 15:53:05,455][75949] Updated weights for policy 0, policy_version 58901 (0.0010) -[2023-10-14 15:53:05,826][75949] Updated weights for policy 0, policy_version 58911 (0.0008) -[2023-10-14 15:53:07,674][75950] Updated weights for policy 1, policy_version 58760 (0.0009) -[2023-10-14 15:53:08,037][75950] Updated weights for policy 1, policy_version 58770 (0.0010) -[2023-10-14 15:53:08,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 120487936. Throughput: 0: 1678.0, 1: 1675.9. Samples: 30133158. Policy #0 lag: (min: 2.0, avg: 2.3, max: 13.0) -[2023-10-14 15:53:08,164][74987] Avg episode reward: [(0, '30.360'), (1, '33.570')] -[2023-10-14 15:53:08,165][75615] Saving new best policy, reward=30.360! -[2023-10-14 15:53:08,406][75950] Updated weights for policy 1, policy_version 58780 (0.0009) -[2023-10-14 15:53:09,869][75949] Updated weights for policy 0, policy_version 58921 (0.0008) -[2023-10-14 15:53:10,245][75949] Updated weights for policy 0, policy_version 58931 (0.0008) -[2023-10-14 15:53:10,628][75949] Updated weights for policy 0, policy_version 58941 (0.0008) -[2023-10-14 15:53:12,534][75950] Updated weights for policy 1, policy_version 58790 (0.0009) -[2023-10-14 15:53:12,891][75950] Updated weights for policy 1, policy_version 58800 (0.0007) -[2023-10-14 15:53:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 120553472. Throughput: 0: 1688.9, 1: 1664.2. Samples: 30153328. Policy #0 lag: (min: 2.0, avg: 2.3, max: 13.0) -[2023-10-14 15:53:13,164][74987] Avg episode reward: [(0, '25.550'), (1, '30.680')] -[2023-10-14 15:53:13,250][75950] Updated weights for policy 1, policy_version 58810 (0.0007) -[2023-10-14 15:53:14,681][75949] Updated weights for policy 0, policy_version 58951 (0.0008) -[2023-10-14 15:53:15,065][75949] Updated weights for policy 0, policy_version 58961 (0.0009) -[2023-10-14 15:53:15,443][75949] Updated weights for policy 0, policy_version 58971 (0.0008) -[2023-10-14 15:53:17,467][75950] Updated weights for policy 1, policy_version 58820 (0.0008) -[2023-10-14 15:53:17,860][75950] Updated weights for policy 1, policy_version 58830 (0.0010) -[2023-10-14 15:53:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120619008. Throughput: 0: 1663.7, 1: 1669.6. Samples: 30162776. Policy #0 lag: (min: 2.0, avg: 2.3, max: 13.0) -[2023-10-14 15:53:18,165][74987] Avg episode reward: [(0, '28.810'), (1, '31.910')] -[2023-10-14 15:53:18,228][75950] Updated weights for policy 1, policy_version 58840 (0.0008) -[2023-10-14 15:53:19,598][75949] Updated weights for policy 0, policy_version 58981 (0.0008) -[2023-10-14 15:53:19,975][75949] Updated weights for policy 0, policy_version 58991 (0.0007) -[2023-10-14 15:53:20,340][75949] Updated weights for policy 0, policy_version 59001 (0.0007) -[2023-10-14 15:53:22,360][75950] Updated weights for policy 1, policy_version 58850 (0.0010) -[2023-10-14 15:53:22,736][75950] Updated weights for policy 1, policy_version 58860 (0.0007) -[2023-10-14 15:53:23,096][75950] Updated weights for policy 1, policy_version 58870 (0.0007) -[2023-10-14 15:53:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120684544. Throughput: 0: 1687.9, 1: 1664.3. Samples: 30183248. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:53:23,164][74987] Avg episode reward: [(0, '23.520'), (1, '32.010')] -[2023-10-14 15:53:23,467][75950] Updated weights for policy 1, policy_version 58880 (0.0009) -[2023-10-14 15:53:24,311][75949] Updated weights for policy 0, policy_version 59011 (0.0009) -[2023-10-14 15:53:24,677][75949] Updated weights for policy 0, policy_version 59021 (0.0010) -[2023-10-14 15:53:25,048][75949] Updated weights for policy 0, policy_version 59031 (0.0008) -[2023-10-14 15:53:27,447][75950] Updated weights for policy 1, policy_version 58890 (0.0009) -[2023-10-14 15:53:27,813][75950] Updated weights for policy 1, policy_version 58900 (0.0009) -[2023-10-14 15:53:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 120750080. Throughput: 0: 1685.5, 1: 1656.1. Samples: 30203614. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:53:28,164][74987] Avg episode reward: [(0, '27.680'), (1, '31.160')] -[2023-10-14 15:53:28,174][75950] Updated weights for policy 1, policy_version 58910 (0.0009) -[2023-10-14 15:53:29,156][75949] Updated weights for policy 0, policy_version 59041 (0.0009) -[2023-10-14 15:53:29,558][75949] Updated weights for policy 0, policy_version 59051 (0.0009) -[2023-10-14 15:53:29,934][75949] Updated weights for policy 0, policy_version 59061 (0.0010) -[2023-10-14 15:53:30,298][75949] Updated weights for policy 0, policy_version 59071 (0.0010) -[2023-10-14 15:53:32,080][75950] Updated weights for policy 1, policy_version 58920 (0.0008) -[2023-10-14 15:53:32,450][75950] Updated weights for policy 1, policy_version 58930 (0.0008) -[2023-10-14 15:53:32,829][75950] Updated weights for policy 1, policy_version 58940 (0.0007) -[2023-10-14 15:53:33,163][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 120848384. Throughput: 0: 1668.1, 1: 1672.8. Samples: 30213402. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:53:33,164][74987] Avg episode reward: [(0, '25.100'), (1, '31.460')] -[2023-10-14 15:53:34,392][75949] Updated weights for policy 0, policy_version 59081 (0.0007) -[2023-10-14 15:53:34,761][75949] Updated weights for policy 0, policy_version 59091 (0.0009) -[2023-10-14 15:53:35,145][75949] Updated weights for policy 0, policy_version 59101 (0.0010) -[2023-10-14 15:53:36,809][75950] Updated weights for policy 1, policy_version 58950 (0.0009) -[2023-10-14 15:53:37,179][75950] Updated weights for policy 1, policy_version 58960 (0.0008) -[2023-10-14 15:53:37,546][75950] Updated weights for policy 1, policy_version 58970 (0.0008) -[2023-10-14 15:53:38,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 120913920. Throughput: 0: 1688.4, 1: 1681.2. Samples: 30234342. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:53:38,165][74987] Avg episode reward: [(0, '26.190'), (1, '31.760')] -[2023-10-14 15:53:39,103][75949] Updated weights for policy 0, policy_version 59111 (0.0010) -[2023-10-14 15:53:39,473][75949] Updated weights for policy 0, policy_version 59121 (0.0010) -[2023-10-14 15:53:39,847][75949] Updated weights for policy 0, policy_version 59131 (0.0010) -[2023-10-14 15:53:41,813][75950] Updated weights for policy 1, policy_version 58980 (0.0008) -[2023-10-14 15:53:42,169][75950] Updated weights for policy 1, policy_version 58990 (0.0007) -[2023-10-14 15:53:42,530][75950] Updated weights for policy 1, policy_version 59000 (0.0008) -[2023-10-14 15:53:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 120979456. Throughput: 0: 1692.3, 1: 1659.0. Samples: 30254130. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:53:43,164][74987] Avg episode reward: [(0, '26.850'), (1, '30.440')] -[2023-10-14 15:53:43,927][75949] Updated weights for policy 0, policy_version 59141 (0.0010) -[2023-10-14 15:53:44,305][75949] Updated weights for policy 0, policy_version 59151 (0.0008) -[2023-10-14 15:53:44,683][75949] Updated weights for policy 0, policy_version 59161 (0.0011) -[2023-10-14 15:53:46,488][75950] Updated weights for policy 1, policy_version 59010 (0.0008) -[2023-10-14 15:53:46,853][75950] Updated weights for policy 1, policy_version 59020 (0.0009) -[2023-10-14 15:53:47,212][75950] Updated weights for policy 1, policy_version 59030 (0.0007) -[2023-10-14 15:53:47,578][75950] Updated weights for policy 1, policy_version 59040 (0.0009) -[2023-10-14 15:53:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 121044992. Throughput: 0: 1681.8, 1: 1680.1. Samples: 30264272. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:53:48,164][74987] Avg episode reward: [(0, '25.430'), (1, '30.420')] -[2023-10-14 15:53:48,643][75949] Updated weights for policy 0, policy_version 59171 (0.0008) -[2023-10-14 15:53:49,014][75949] Updated weights for policy 0, policy_version 59181 (0.0007) -[2023-10-14 15:53:49,391][75949] Updated weights for policy 0, policy_version 59191 (0.0008) -[2023-10-14 15:53:51,803][75950] Updated weights for policy 1, policy_version 59050 (0.0007) -[2023-10-14 15:53:52,180][75950] Updated weights for policy 1, policy_version 59060 (0.0007) -[2023-10-14 15:53:52,541][75950] Updated weights for policy 1, policy_version 59070 (0.0009) -[2023-10-14 15:53:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121110528. Throughput: 0: 1694.8, 1: 1674.1. Samples: 30284758. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:53:53,165][74987] Avg episode reward: [(0, '28.620'), (1, '31.270')] -[2023-10-14 15:53:53,253][75949] Updated weights for policy 0, policy_version 59201 (0.0008) -[2023-10-14 15:53:53,624][75949] Updated weights for policy 0, policy_version 59211 (0.0007) -[2023-10-14 15:53:54,001][75949] Updated weights for policy 0, policy_version 59221 (0.0008) -[2023-10-14 15:53:54,369][75949] Updated weights for policy 0, policy_version 59231 (0.0008) -[2023-10-14 15:53:56,574][75950] Updated weights for policy 1, policy_version 59080 (0.0009) -[2023-10-14 15:53:56,947][75950] Updated weights for policy 1, policy_version 59090 (0.0007) -[2023-10-14 15:53:57,307][75950] Updated weights for policy 1, policy_version 59100 (0.0008) -[2023-10-14 15:53:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121176064. Throughput: 0: 1698.8, 1: 1664.1. Samples: 30304658. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) -[2023-10-14 15:53:58,165][74987] Avg episode reward: [(0, '25.700'), (1, '31.960')] -[2023-10-14 15:53:58,415][75949] Updated weights for policy 0, policy_version 59241 (0.0008) -[2023-10-14 15:53:58,783][75949] Updated weights for policy 0, policy_version 59251 (0.0009) -[2023-10-14 15:53:59,157][75949] Updated weights for policy 0, policy_version 59261 (0.0009) -[2023-10-14 15:54:01,253][75950] Updated weights for policy 1, policy_version 59110 (0.0009) -[2023-10-14 15:54:01,609][75950] Updated weights for policy 1, policy_version 59120 (0.0007) -[2023-10-14 15:54:01,981][75950] Updated weights for policy 1, policy_version 59130 (0.0007) -[2023-10-14 15:54:03,144][75949] Updated weights for policy 0, policy_version 59271 (0.0007) -[2023-10-14 15:54:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121241600. Throughput: 0: 1698.9, 1: 1692.3. Samples: 30315378. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) -[2023-10-14 15:54:03,164][74987] Avg episode reward: [(0, '27.180'), (1, '30.660')] -[2023-10-14 15:54:03,504][75949] Updated weights for policy 0, policy_version 59281 (0.0008) -[2023-10-14 15:54:03,870][75949] Updated weights for policy 0, policy_version 59291 (0.0007) -[2023-10-14 15:54:06,171][75950] Updated weights for policy 1, policy_version 59140 (0.0009) -[2023-10-14 15:54:06,544][75950] Updated weights for policy 1, policy_version 59150 (0.0008) -[2023-10-14 15:54:06,912][75950] Updated weights for policy 1, policy_version 59160 (0.0008) -[2023-10-14 15:54:07,979][75949] Updated weights for policy 0, policy_version 59301 (0.0007) -[2023-10-14 15:54:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121307136. Throughput: 0: 1701.4, 1: 1684.1. Samples: 30335598. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) -[2023-10-14 15:54:08,164][74987] Avg episode reward: [(0, '25.510'), (1, '31.610')] -[2023-10-14 15:54:08,348][75949] Updated weights for policy 0, policy_version 59311 (0.0008) -[2023-10-14 15:54:08,720][75949] Updated weights for policy 0, policy_version 59321 (0.0009) -[2023-10-14 15:54:10,838][75950] Updated weights for policy 1, policy_version 59170 (0.0008) -[2023-10-14 15:54:11,196][75950] Updated weights for policy 1, policy_version 59180 (0.0010) -[2023-10-14 15:54:11,566][75950] Updated weights for policy 1, policy_version 59190 (0.0008) -[2023-10-14 15:54:11,924][75950] Updated weights for policy 1, policy_version 59200 (0.0009) -[2023-10-14 15:54:12,826][75949] Updated weights for policy 0, policy_version 59331 (0.0010) -[2023-10-14 15:54:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121372672. Throughput: 0: 1699.8, 1: 1681.2. Samples: 30355760. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) -[2023-10-14 15:54:13,165][74987] Avg episode reward: [(0, '26.480'), (1, '32.460')] -[2023-10-14 15:54:13,197][75949] Updated weights for policy 0, policy_version 59341 (0.0011) -[2023-10-14 15:54:13,566][75949] Updated weights for policy 0, policy_version 59351 (0.0007) -[2023-10-14 15:54:16,027][75950] Updated weights for policy 1, policy_version 59210 (0.0009) -[2023-10-14 15:54:16,395][75950] Updated weights for policy 1, policy_version 59220 (0.0011) -[2023-10-14 15:54:16,769][75950] Updated weights for policy 1, policy_version 59230 (0.0010) -[2023-10-14 15:54:17,505][75949] Updated weights for policy 0, policy_version 59361 (0.0009) -[2023-10-14 15:54:17,912][75949] Updated weights for policy 0, policy_version 59371 (0.0010) -[2023-10-14 15:54:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 121438208. Throughput: 0: 1698.8, 1: 1697.7. Samples: 30366244. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) -[2023-10-14 15:54:18,164][74987] Avg episode reward: [(0, '27.070'), (1, '33.430')] -[2023-10-14 15:54:18,285][75949] Updated weights for policy 0, policy_version 59381 (0.0007) -[2023-10-14 15:54:18,642][75949] Updated weights for policy 0, policy_version 59391 (0.0009) -[2023-10-14 15:54:20,893][75950] Updated weights for policy 1, policy_version 59240 (0.0007) -[2023-10-14 15:54:21,255][75950] Updated weights for policy 1, policy_version 59250 (0.0010) -[2023-10-14 15:54:21,624][75950] Updated weights for policy 1, policy_version 59260 (0.0010) -[2023-10-14 15:54:22,738][75949] Updated weights for policy 0, policy_version 59401 (0.0008) -[2023-10-14 15:54:23,119][75949] Updated weights for policy 0, policy_version 59411 (0.0009) -[2023-10-14 15:54:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121503744. Throughput: 0: 1702.2, 1: 1671.2. Samples: 30386148. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) -[2023-10-14 15:54:23,165][74987] Avg episode reward: [(0, '25.620'), (1, '31.040')] -[2023-10-14 15:54:23,490][75949] Updated weights for policy 0, policy_version 59421 (0.0009) -[2023-10-14 15:54:25,655][75950] Updated weights for policy 1, policy_version 59270 (0.0009) -[2023-10-14 15:54:26,021][75950] Updated weights for policy 1, policy_version 59280 (0.0008) -[2023-10-14 15:54:26,396][75950] Updated weights for policy 1, policy_version 59290 (0.0009) -[2023-10-14 15:54:27,496][75949] Updated weights for policy 0, policy_version 59431 (0.0010) -[2023-10-14 15:54:27,871][75949] Updated weights for policy 0, policy_version 59441 (0.0010) -[2023-10-14 15:54:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121569280. Throughput: 0: 1687.8, 1: 1689.9. Samples: 30406126. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) -[2023-10-14 15:54:28,165][74987] Avg episode reward: [(0, '26.870'), (1, '32.350')] -[2023-10-14 15:54:28,242][75949] Updated weights for policy 0, policy_version 59451 (0.0010) -[2023-10-14 15:54:30,327][75950] Updated weights for policy 1, policy_version 59300 (0.0009) -[2023-10-14 15:54:30,685][75950] Updated weights for policy 1, policy_version 59310 (0.0008) -[2023-10-14 15:54:31,046][75950] Updated weights for policy 1, policy_version 59320 (0.0008) -[2023-10-14 15:54:32,392][75949] Updated weights for policy 0, policy_version 59461 (0.0008) -[2023-10-14 15:54:32,761][75949] Updated weights for policy 0, policy_version 59471 (0.0009) -[2023-10-14 15:54:33,140][75949] Updated weights for policy 0, policy_version 59481 (0.0008) -[2023-10-14 15:54:33,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 121634816. Throughput: 0: 1698.3, 1: 1688.2. Samples: 30416662. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) -[2023-10-14 15:54:33,164][74987] Avg episode reward: [(0, '25.530'), (1, '32.140')] -[2023-10-14 15:54:35,003][75950] Updated weights for policy 1, policy_version 59330 (0.0010) -[2023-10-14 15:54:35,377][75950] Updated weights for policy 1, policy_version 59340 (0.0009) -[2023-10-14 15:54:35,754][75950] Updated weights for policy 1, policy_version 59350 (0.0007) -[2023-10-14 15:54:36,121][75950] Updated weights for policy 1, policy_version 59360 (0.0008) -[2023-10-14 15:54:37,055][75949] Updated weights for policy 0, policy_version 59491 (0.0008) -[2023-10-14 15:54:37,427][75949] Updated weights for policy 0, policy_version 59501 (0.0010) -[2023-10-14 15:54:37,813][75949] Updated weights for policy 0, policy_version 59511 (0.0010) -[2023-10-14 15:54:38,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 121733120. Throughput: 0: 1693.0, 1: 1678.8. Samples: 30436490. Policy #0 lag: (min: 8.0, avg: 27.9, max: 40.0) -[2023-10-14 15:54:38,165][74987] Avg episode reward: [(0, '28.560'), (1, '30.630')] -[2023-10-14 15:54:40,191][75950] Updated weights for policy 1, policy_version 59370 (0.0009) -[2023-10-14 15:54:40,558][75950] Updated weights for policy 1, policy_version 59380 (0.0010) -[2023-10-14 15:54:40,918][75950] Updated weights for policy 1, policy_version 59390 (0.0008) -[2023-10-14 15:54:41,879][75949] Updated weights for policy 0, policy_version 59521 (0.0008) -[2023-10-14 15:54:42,241][75949] Updated weights for policy 0, policy_version 59531 (0.0009) -[2023-10-14 15:54:42,612][75949] Updated weights for policy 0, policy_version 59541 (0.0009) -[2023-10-14 15:54:42,990][75949] Updated weights for policy 0, policy_version 59551 (0.0009) -[2023-10-14 15:54:43,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121798656. Throughput: 0: 1669.5, 1: 1701.9. Samples: 30456370. Policy #0 lag: (min: 8.0, avg: 27.9, max: 40.0) -[2023-10-14 15:54:43,164][74987] Avg episode reward: [(0, '25.680'), (1, '32.570')] -[2023-10-14 15:54:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000059552_60981248.pth... -[2023-10-14 15:54:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000059392_60817408.pth... -[2023-10-14 15:54:43,209][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000057824_59211776.pth -[2023-10-14 15:54:43,212][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000057984_59375616.pth -[2023-10-14 15:54:44,989][75950] Updated weights for policy 1, policy_version 59400 (0.0009) -[2023-10-14 15:54:45,354][75950] Updated weights for policy 1, policy_version 59410 (0.0009) -[2023-10-14 15:54:45,719][75950] Updated weights for policy 1, policy_version 59420 (0.0009) -[2023-10-14 15:54:47,141][75949] Updated weights for policy 0, policy_version 59561 (0.0011) -[2023-10-14 15:54:47,505][75949] Updated weights for policy 0, policy_version 59571 (0.0009) -[2023-10-14 15:54:47,884][75949] Updated weights for policy 0, policy_version 59581 (0.0011) -[2023-10-14 15:54:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121864192. Throughput: 0: 1686.7, 1: 1673.6. Samples: 30466594. Policy #0 lag: (min: 8.0, avg: 27.9, max: 40.0) -[2023-10-14 15:54:48,164][74987] Avg episode reward: [(0, '29.090'), (1, '33.490')] -[2023-10-14 15:54:49,985][75950] Updated weights for policy 1, policy_version 59430 (0.0008) -[2023-10-14 15:54:50,348][75950] Updated weights for policy 1, policy_version 59440 (0.0008) -[2023-10-14 15:54:50,722][75950] Updated weights for policy 1, policy_version 59450 (0.0009) -[2023-10-14 15:54:52,246][75949] Updated weights for policy 0, policy_version 59591 (0.0009) -[2023-10-14 15:54:52,619][75949] Updated weights for policy 0, policy_version 59601 (0.0007) -[2023-10-14 15:54:52,985][75949] Updated weights for policy 0, policy_version 59611 (0.0008) -[2023-10-14 15:54:53,164][74987] Fps is (10 sec: 9830.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 121896960. Throughput: 0: 1678.3, 1: 1672.4. Samples: 30486384. Policy #0 lag: (min: 8.0, avg: 27.9, max: 40.0) -[2023-10-14 15:54:53,165][74987] Avg episode reward: [(0, '24.870'), (1, '30.690')] -[2023-10-14 15:54:54,883][75950] Updated weights for policy 1, policy_version 59460 (0.0009) -[2023-10-14 15:54:55,279][75950] Updated weights for policy 1, policy_version 59470 (0.0010) -[2023-10-14 15:54:55,642][75950] Updated weights for policy 1, policy_version 59480 (0.0008) -[2023-10-14 15:54:56,875][75949] Updated weights for policy 0, policy_version 59621 (0.0010) -[2023-10-14 15:54:57,255][75949] Updated weights for policy 0, policy_version 59631 (0.0008) -[2023-10-14 15:54:57,622][75949] Updated weights for policy 0, policy_version 59641 (0.0007) -[2023-10-14 15:54:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 121995264. Throughput: 0: 1660.7, 1: 1678.6. Samples: 30506028. Policy #0 lag: (min: 8.0, avg: 27.9, max: 40.0) -[2023-10-14 15:54:58,164][74987] Avg episode reward: [(0, '27.030'), (1, '30.280')] -[2023-10-14 15:54:59,886][75950] Updated weights for policy 1, policy_version 59490 (0.0009) -[2023-10-14 15:55:00,251][75950] Updated weights for policy 1, policy_version 59500 (0.0009) -[2023-10-14 15:55:00,621][75950] Updated weights for policy 1, policy_version 59510 (0.0008) -[2023-10-14 15:55:00,990][75950] Updated weights for policy 1, policy_version 59520 (0.0007) -[2023-10-14 15:55:01,770][75949] Updated weights for policy 0, policy_version 59651 (0.0010) -[2023-10-14 15:55:02,143][75949] Updated weights for policy 0, policy_version 59661 (0.0007) -[2023-10-14 15:55:02,508][75949] Updated weights for policy 0, policy_version 59671 (0.0008) -[2023-10-14 15:55:03,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 122060800. Throughput: 0: 1683.8, 1: 1653.8. Samples: 30516436. Policy #0 lag: (min: 8.0, avg: 27.9, max: 40.0) -[2023-10-14 15:55:03,164][74987] Avg episode reward: [(0, '27.600'), (1, '30.860')] -[2023-10-14 15:55:05,141][75950] Updated weights for policy 1, policy_version 59530 (0.0010) -[2023-10-14 15:55:05,512][75950] Updated weights for policy 1, policy_version 59540 (0.0010) -[2023-10-14 15:55:05,880][75950] Updated weights for policy 1, policy_version 59550 (0.0009) -[2023-10-14 15:55:06,651][75949] Updated weights for policy 0, policy_version 59681 (0.0007) -[2023-10-14 15:55:07,065][75949] Updated weights for policy 0, policy_version 59691 (0.0009) -[2023-10-14 15:55:07,441][75949] Updated weights for policy 0, policy_version 59701 (0.0008) -[2023-10-14 15:55:07,803][75949] Updated weights for policy 0, policy_version 59711 (0.0007) -[2023-10-14 15:55:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 122126336. Throughput: 0: 1675.2, 1: 1664.1. Samples: 30536414. Policy #0 lag: (min: 8.0, avg: 27.9, max: 40.0) -[2023-10-14 15:55:08,165][74987] Avg episode reward: [(0, '26.690'), (1, '29.770')] -[2023-10-14 15:55:09,682][75950] Updated weights for policy 1, policy_version 59560 (0.0008) -[2023-10-14 15:55:10,049][75950] Updated weights for policy 1, policy_version 59570 (0.0009) -[2023-10-14 15:55:10,407][75950] Updated weights for policy 1, policy_version 59580 (0.0008) -[2023-10-14 15:55:11,705][75949] Updated weights for policy 0, policy_version 59721 (0.0011) -[2023-10-14 15:55:12,072][75949] Updated weights for policy 0, policy_version 59731 (0.0007) -[2023-10-14 15:55:12,439][75949] Updated weights for policy 0, policy_version 59741 (0.0009) -[2023-10-14 15:55:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 122191872. Throughput: 0: 1657.7, 1: 1683.3. Samples: 30556470. Policy #0 lag: (min: 8.0, avg: 27.9, max: 40.0) -[2023-10-14 15:55:13,164][74987] Avg episode reward: [(0, '27.920'), (1, '30.140')] -[2023-10-14 15:55:14,415][75950] Updated weights for policy 1, policy_version 59590 (0.0008) -[2023-10-14 15:55:14,785][75950] Updated weights for policy 1, policy_version 59600 (0.0009) -[2023-10-14 15:55:15,156][75950] Updated weights for policy 1, policy_version 59610 (0.0009) -[2023-10-14 15:55:16,450][75949] Updated weights for policy 0, policy_version 59751 (0.0010) -[2023-10-14 15:55:16,825][75949] Updated weights for policy 0, policy_version 59761 (0.0010) -[2023-10-14 15:55:17,199][75949] Updated weights for policy 0, policy_version 59771 (0.0011) -[2023-10-14 15:55:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 122257408. Throughput: 0: 1676.8, 1: 1659.6. Samples: 30566804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-14 15:55:18,165][74987] Avg episode reward: [(0, '25.430'), (1, '30.080')] -[2023-10-14 15:55:19,217][75950] Updated weights for policy 1, policy_version 59620 (0.0008) -[2023-10-14 15:55:19,578][75950] Updated weights for policy 1, policy_version 59630 (0.0008) -[2023-10-14 15:55:19,950][75950] Updated weights for policy 1, policy_version 59640 (0.0007) -[2023-10-14 15:55:21,331][75949] Updated weights for policy 0, policy_version 59781 (0.0010) -[2023-10-14 15:55:21,710][75949] Updated weights for policy 0, policy_version 59791 (0.0007) -[2023-10-14 15:55:22,080][75949] Updated weights for policy 0, policy_version 59801 (0.0007) -[2023-10-14 15:55:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 122322944. Throughput: 0: 1666.7, 1: 1676.9. Samples: 30586950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-14 15:55:23,164][74987] Avg episode reward: [(0, '27.120'), (1, '30.160')] -[2023-10-14 15:55:23,862][75950] Updated weights for policy 1, policy_version 59650 (0.0010) -[2023-10-14 15:55:24,223][75950] Updated weights for policy 1, policy_version 59660 (0.0009) -[2023-10-14 15:55:24,586][75950] Updated weights for policy 1, policy_version 59670 (0.0008) -[2023-10-14 15:55:24,956][75950] Updated weights for policy 1, policy_version 59680 (0.0007) -[2023-10-14 15:55:26,274][75949] Updated weights for policy 0, policy_version 59811 (0.0008) -[2023-10-14 15:55:26,638][75949] Updated weights for policy 0, policy_version 59821 (0.0008) -[2023-10-14 15:55:27,009][75949] Updated weights for policy 0, policy_version 59831 (0.0007) -[2023-10-14 15:55:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 122388480. Throughput: 0: 1668.7, 1: 1680.8. Samples: 30607100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-14 15:55:28,165][74987] Avg episode reward: [(0, '24.960'), (1, '30.740')] -[2023-10-14 15:55:29,183][75950] Updated weights for policy 1, policy_version 59690 (0.0007) -[2023-10-14 15:55:29,557][75950] Updated weights for policy 1, policy_version 59700 (0.0008) -[2023-10-14 15:55:29,919][75950] Updated weights for policy 1, policy_version 59710 (0.0009) -[2023-10-14 15:55:30,895][75949] Updated weights for policy 0, policy_version 59841 (0.0007) -[2023-10-14 15:55:31,261][75949] Updated weights for policy 0, policy_version 59851 (0.0009) -[2023-10-14 15:55:31,631][75949] Updated weights for policy 0, policy_version 59861 (0.0008) -[2023-10-14 15:55:31,995][75949] Updated weights for policy 0, policy_version 59871 (0.0008) -[2023-10-14 15:55:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 122454016. Throughput: 0: 1684.6, 1: 1670.9. Samples: 30617590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-14 15:55:33,164][74987] Avg episode reward: [(0, '26.720'), (1, '31.710')] -[2023-10-14 15:55:33,896][75950] Updated weights for policy 1, policy_version 59720 (0.0010) -[2023-10-14 15:55:34,273][75950] Updated weights for policy 1, policy_version 59730 (0.0010) -[2023-10-14 15:55:34,641][75950] Updated weights for policy 1, policy_version 59740 (0.0009) -[2023-10-14 15:55:36,098][75949] Updated weights for policy 0, policy_version 59881 (0.0008) -[2023-10-14 15:55:36,467][75949] Updated weights for policy 0, policy_version 59891 (0.0009) -[2023-10-14 15:55:36,834][75949] Updated weights for policy 0, policy_version 59901 (0.0008) -[2023-10-14 15:55:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 122519552. Throughput: 0: 1670.0, 1: 1683.0. Samples: 30637270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-14 15:55:38,164][74987] Avg episode reward: [(0, '26.590'), (1, '30.680')] -[2023-10-14 15:55:38,857][75950] Updated weights for policy 1, policy_version 59750 (0.0008) -[2023-10-14 15:55:39,226][75950] Updated weights for policy 1, policy_version 59760 (0.0009) -[2023-10-14 15:55:39,588][75950] Updated weights for policy 1, policy_version 59770 (0.0009) -[2023-10-14 15:55:40,983][75949] Updated weights for policy 0, policy_version 59911 (0.0010) -[2023-10-14 15:55:41,344][75949] Updated weights for policy 0, policy_version 59921 (0.0009) -[2023-10-14 15:55:41,721][75949] Updated weights for policy 0, policy_version 59931 (0.0010) -[2023-10-14 15:55:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 122585088. Throughput: 0: 1680.2, 1: 1690.9. Samples: 30657728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-14 15:55:43,165][74987] Avg episode reward: [(0, '28.100'), (1, '31.610')] -[2023-10-14 15:55:43,540][75950] Updated weights for policy 1, policy_version 59780 (0.0010) -[2023-10-14 15:55:43,934][75950] Updated weights for policy 1, policy_version 59790 (0.0009) -[2023-10-14 15:55:44,306][75950] Updated weights for policy 1, policy_version 59800 (0.0010) -[2023-10-14 15:55:45,631][75949] Updated weights for policy 0, policy_version 59941 (0.0009) -[2023-10-14 15:55:46,002][75949] Updated weights for policy 0, policy_version 59951 (0.0007) -[2023-10-14 15:55:46,365][75949] Updated weights for policy 0, policy_version 59961 (0.0009) -[2023-10-14 15:55:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 122650624. Throughput: 0: 1682.5, 1: 1678.7. Samples: 30667692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-14 15:55:48,164][74987] Avg episode reward: [(0, '27.250'), (1, '31.850')] -[2023-10-14 15:55:48,495][75950] Updated weights for policy 1, policy_version 59810 (0.0009) -[2023-10-14 15:55:48,860][75950] Updated weights for policy 1, policy_version 59820 (0.0007) -[2023-10-14 15:55:49,232][75950] Updated weights for policy 1, policy_version 59830 (0.0007) -[2023-10-14 15:55:49,594][75950] Updated weights for policy 1, policy_version 59840 (0.0009) -[2023-10-14 15:55:50,541][75949] Updated weights for policy 0, policy_version 59971 (0.0010) -[2023-10-14 15:55:50,911][75949] Updated weights for policy 0, policy_version 59981 (0.0008) -[2023-10-14 15:55:51,273][75949] Updated weights for policy 0, policy_version 59991 (0.0008) -[2023-10-14 15:55:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 122716160. Throughput: 0: 1667.7, 1: 1690.0. Samples: 30687512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) -[2023-10-14 15:55:53,165][74987] Avg episode reward: [(0, '25.910'), (1, '30.460')] -[2023-10-14 15:55:53,626][75950] Updated weights for policy 1, policy_version 59850 (0.0007) -[2023-10-14 15:55:53,989][75950] Updated weights for policy 1, policy_version 59860 (0.0007) -[2023-10-14 15:55:54,366][75950] Updated weights for policy 1, policy_version 59870 (0.0009) -[2023-10-14 15:55:55,229][75949] Updated weights for policy 0, policy_version 60001 (0.0008) -[2023-10-14 15:55:55,658][75949] Updated weights for policy 0, policy_version 60011 (0.0007) -[2023-10-14 15:55:56,025][75949] Updated weights for policy 0, policy_version 60021 (0.0008) -[2023-10-14 15:55:56,397][75949] Updated weights for policy 0, policy_version 60031 (0.0011) -[2023-10-14 15:55:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 122781696. Throughput: 0: 1690.1, 1: 1679.5. Samples: 30708100. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:55:58,164][74987] Avg episode reward: [(0, '27.530'), (1, '30.680')] -[2023-10-14 15:55:58,593][75950] Updated weights for policy 1, policy_version 59880 (0.0009) -[2023-10-14 15:55:58,964][75950] Updated weights for policy 1, policy_version 59890 (0.0008) -[2023-10-14 15:55:59,339][75950] Updated weights for policy 1, policy_version 59900 (0.0009) -[2023-10-14 15:56:00,303][75949] Updated weights for policy 0, policy_version 60041 (0.0009) -[2023-10-14 15:56:00,682][75949] Updated weights for policy 0, policy_version 60051 (0.0007) -[2023-10-14 15:56:01,046][75949] Updated weights for policy 0, policy_version 60061 (0.0008) -[2023-10-14 15:56:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 122847232. Throughput: 0: 1677.0, 1: 1681.9. Samples: 30717954. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:56:03,164][74987] Avg episode reward: [(0, '26.280'), (1, '32.620')] -[2023-10-14 15:56:03,185][75950] Updated weights for policy 1, policy_version 59910 (0.0008) -[2023-10-14 15:56:03,541][75950] Updated weights for policy 1, policy_version 59920 (0.0008) -[2023-10-14 15:56:03,904][75950] Updated weights for policy 1, policy_version 59930 (0.0010) -[2023-10-14 15:56:05,056][75949] Updated weights for policy 0, policy_version 60071 (0.0010) -[2023-10-14 15:56:05,423][75949] Updated weights for policy 0, policy_version 60081 (0.0009) -[2023-10-14 15:56:05,798][75949] Updated weights for policy 0, policy_version 60091 (0.0008) -[2023-10-14 15:56:07,963][75950] Updated weights for policy 1, policy_version 59940 (0.0009) -[2023-10-14 15:56:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 122912768. Throughput: 0: 1677.5, 1: 1682.1. Samples: 30738134. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:56:08,164][74987] Avg episode reward: [(0, '27.260'), (1, '32.230')] -[2023-10-14 15:56:08,321][75950] Updated weights for policy 1, policy_version 59950 (0.0009) -[2023-10-14 15:56:08,694][75950] Updated weights for policy 1, policy_version 59960 (0.0008) -[2023-10-14 15:56:10,046][75949] Updated weights for policy 0, policy_version 60101 (0.0008) -[2023-10-14 15:56:10,413][75949] Updated weights for policy 0, policy_version 60111 (0.0007) -[2023-10-14 15:56:10,791][75949] Updated weights for policy 0, policy_version 60121 (0.0009) -[2023-10-14 15:56:12,811][75950] Updated weights for policy 1, policy_version 59970 (0.0009) -[2023-10-14 15:56:13,161][75950] Updated weights for policy 1, policy_version 59980 (0.0007) -[2023-10-14 15:56:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122978304. Throughput: 0: 1693.6, 1: 1678.1. Samples: 30758824. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:56:13,164][74987] Avg episode reward: [(0, '26.820'), (1, '29.890')] -[2023-10-14 15:56:13,533][75950] Updated weights for policy 1, policy_version 59990 (0.0008) -[2023-10-14 15:56:13,900][75950] Updated weights for policy 1, policy_version 60000 (0.0008) -[2023-10-14 15:56:14,575][75949] Updated weights for policy 0, policy_version 60131 (0.0009) -[2023-10-14 15:56:14,941][75949] Updated weights for policy 0, policy_version 60141 (0.0011) -[2023-10-14 15:56:15,313][75949] Updated weights for policy 0, policy_version 60151 (0.0010) -[2023-10-14 15:56:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123043840. Throughput: 0: 1661.2, 1: 1682.2. Samples: 30768044. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:56:18,165][74987] Avg episode reward: [(0, '26.390'), (1, '31.150')] -[2023-10-14 15:56:18,233][75950] Updated weights for policy 1, policy_version 60010 (0.0008) -[2023-10-14 15:56:18,589][75950] Updated weights for policy 1, policy_version 60020 (0.0008) -[2023-10-14 15:56:18,966][75950] Updated weights for policy 1, policy_version 60030 (0.0011) -[2023-10-14 15:56:19,566][75949] Updated weights for policy 0, policy_version 60161 (0.0009) -[2023-10-14 15:56:19,947][75949] Updated weights for policy 0, policy_version 60171 (0.0009) -[2023-10-14 15:56:20,318][75949] Updated weights for policy 0, policy_version 60181 (0.0010) -[2023-10-14 15:56:20,693][75949] Updated weights for policy 0, policy_version 60191 (0.0009) -[2023-10-14 15:56:23,001][75950] Updated weights for policy 1, policy_version 60040 (0.0008) -[2023-10-14 15:56:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123109376. Throughput: 0: 1677.6, 1: 1686.1. Samples: 30788638. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:56:23,164][74987] Avg episode reward: [(0, '25.760'), (1, '27.720')] -[2023-10-14 15:56:23,366][75950] Updated weights for policy 1, policy_version 60050 (0.0011) -[2023-10-14 15:56:23,728][75950] Updated weights for policy 1, policy_version 60060 (0.0009) -[2023-10-14 15:56:24,641][75949] Updated weights for policy 0, policy_version 60201 (0.0010) -[2023-10-14 15:56:25,022][75949] Updated weights for policy 0, policy_version 60211 (0.0007) -[2023-10-14 15:56:25,380][75949] Updated weights for policy 0, policy_version 60221 (0.0009) -[2023-10-14 15:56:27,771][75950] Updated weights for policy 1, policy_version 60070 (0.0010) -[2023-10-14 15:56:28,138][75950] Updated weights for policy 1, policy_version 60080 (0.0009) -[2023-10-14 15:56:28,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123174912. Throughput: 0: 1691.6, 1: 1681.6. Samples: 30809518. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:56:28,164][74987] Avg episode reward: [(0, '26.290'), (1, '28.020')] -[2023-10-14 15:56:28,502][75950] Updated weights for policy 1, policy_version 60090 (0.0007) -[2023-10-14 15:56:29,537][75949] Updated weights for policy 0, policy_version 60231 (0.0008) -[2023-10-14 15:56:29,915][75949] Updated weights for policy 0, policy_version 60241 (0.0009) -[2023-10-14 15:56:30,287][75949] Updated weights for policy 0, policy_version 60251 (0.0009) -[2023-10-14 15:56:32,596][75950] Updated weights for policy 1, policy_version 60100 (0.0008) -[2023-10-14 15:56:32,995][75950] Updated weights for policy 1, policy_version 60110 (0.0008) -[2023-10-14 15:56:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123240448. Throughput: 0: 1668.3, 1: 1688.8. Samples: 30818760. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-10-14 15:56:33,164][74987] Avg episode reward: [(0, '27.230'), (1, '31.780')] -[2023-10-14 15:56:33,358][75950] Updated weights for policy 1, policy_version 60120 (0.0009) -[2023-10-14 15:56:34,248][75949] Updated weights for policy 0, policy_version 60261 (0.0008) -[2023-10-14 15:56:34,614][75949] Updated weights for policy 0, policy_version 60271 (0.0009) -[2023-10-14 15:56:34,993][75949] Updated weights for policy 0, policy_version 60281 (0.0009) -[2023-10-14 15:56:37,401][75950] Updated weights for policy 1, policy_version 60130 (0.0011) -[2023-10-14 15:56:37,765][75950] Updated weights for policy 1, policy_version 60140 (0.0008) -[2023-10-14 15:56:38,133][75950] Updated weights for policy 1, policy_version 60150 (0.0008) -[2023-10-14 15:56:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123305984. Throughput: 0: 1691.7, 1: 1682.8. Samples: 30839366. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:56:38,164][74987] Avg episode reward: [(0, '27.680'), (1, '29.470')] -[2023-10-14 15:56:38,503][75950] Updated weights for policy 1, policy_version 60160 (0.0009) -[2023-10-14 15:56:39,031][75949] Updated weights for policy 0, policy_version 60291 (0.0009) -[2023-10-14 15:56:39,403][75949] Updated weights for policy 0, policy_version 60301 (0.0008) -[2023-10-14 15:56:39,783][75949] Updated weights for policy 0, policy_version 60311 (0.0008) -[2023-10-14 15:56:42,618][75950] Updated weights for policy 1, policy_version 60170 (0.0008) -[2023-10-14 15:56:42,988][75950] Updated weights for policy 1, policy_version 60180 (0.0009) -[2023-10-14 15:56:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123371520. Throughput: 0: 1704.0, 1: 1671.3. Samples: 30859986. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:56:43,165][74987] Avg episode reward: [(0, '27.330'), (1, '32.140')] -[2023-10-14 15:56:43,171][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000060320_61767680.pth... -[2023-10-14 15:56:43,205][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000058752_60162048.pth -[2023-10-14 15:56:43,342][75950] Updated weights for policy 1, policy_version 60190 (0.0007) -[2023-10-14 15:56:43,413][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000060192_61636608.pth... -[2023-10-14 15:56:43,442][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000058624_60030976.pth -[2023-10-14 15:56:43,718][75949] Updated weights for policy 0, policy_version 60321 (0.0009) -[2023-10-14 15:56:44,105][75949] Updated weights for policy 0, policy_version 60331 (0.0009) -[2023-10-14 15:56:44,474][75949] Updated weights for policy 0, policy_version 60341 (0.0008) -[2023-10-14 15:56:44,853][75949] Updated weights for policy 0, policy_version 60351 (0.0008) -[2023-10-14 15:56:47,653][75950] Updated weights for policy 1, policy_version 60200 (0.0008) -[2023-10-14 15:56:48,009][75950] Updated weights for policy 1, policy_version 60210 (0.0010) -[2023-10-14 15:56:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123437056. Throughput: 0: 1686.4, 1: 1676.3. Samples: 30869276. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:56:48,164][74987] Avg episode reward: [(0, '26.060'), (1, '31.670')] -[2023-10-14 15:56:48,382][75950] Updated weights for policy 1, policy_version 60220 (0.0010) -[2023-10-14 15:56:49,020][75949] Updated weights for policy 0, policy_version 60361 (0.0007) -[2023-10-14 15:56:49,395][75949] Updated weights for policy 0, policy_version 60371 (0.0007) -[2023-10-14 15:56:49,774][75949] Updated weights for policy 0, policy_version 60381 (0.0008) -[2023-10-14 15:56:52,313][75950] Updated weights for policy 1, policy_version 60230 (0.0009) -[2023-10-14 15:56:52,681][75950] Updated weights for policy 1, policy_version 60240 (0.0007) -[2023-10-14 15:56:53,058][75950] Updated weights for policy 1, policy_version 60250 (0.0008) -[2023-10-14 15:56:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 123502592. Throughput: 0: 1694.5, 1: 1680.0. Samples: 30889986. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:56:53,165][74987] Avg episode reward: [(0, '26.200'), (1, '29.210')] -[2023-10-14 15:56:53,769][75949] Updated weights for policy 0, policy_version 60391 (0.0008) -[2023-10-14 15:56:54,145][75949] Updated weights for policy 0, policy_version 60401 (0.0009) -[2023-10-14 15:56:54,521][75949] Updated weights for policy 0, policy_version 60411 (0.0007) -[2023-10-14 15:56:57,134][75950] Updated weights for policy 1, policy_version 60260 (0.0009) -[2023-10-14 15:56:57,506][75950] Updated weights for policy 1, policy_version 60270 (0.0007) -[2023-10-14 15:56:57,865][75950] Updated weights for policy 1, policy_version 60280 (0.0007) -[2023-10-14 15:56:58,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123600896. Throughput: 0: 1694.3, 1: 1665.4. Samples: 30910014. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:56:58,165][74987] Avg episode reward: [(0, '25.580'), (1, '31.320')] -[2023-10-14 15:56:58,557][75949] Updated weights for policy 0, policy_version 60421 (0.0007) -[2023-10-14 15:56:58,932][75949] Updated weights for policy 0, policy_version 60431 (0.0007) -[2023-10-14 15:56:59,291][75949] Updated weights for policy 0, policy_version 60441 (0.0008) -[2023-10-14 15:57:01,911][75950] Updated weights for policy 1, policy_version 60290 (0.0009) -[2023-10-14 15:57:02,274][75950] Updated weights for policy 1, policy_version 60300 (0.0008) -[2023-10-14 15:57:02,644][75950] Updated weights for policy 1, policy_version 60310 (0.0008) -[2023-10-14 15:57:03,021][75950] Updated weights for policy 1, policy_version 60320 (0.0008) -[2023-10-14 15:57:03,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123666432. Throughput: 0: 1691.7, 1: 1683.0. Samples: 30919908. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:57:03,164][74987] Avg episode reward: [(0, '28.780'), (1, '31.780')] -[2023-10-14 15:57:03,438][75949] Updated weights for policy 0, policy_version 60451 (0.0008) -[2023-10-14 15:57:03,814][75949] Updated weights for policy 0, policy_version 60461 (0.0009) -[2023-10-14 15:57:04,191][75949] Updated weights for policy 0, policy_version 60471 (0.0010) -[2023-10-14 15:57:06,959][75950] Updated weights for policy 1, policy_version 60330 (0.0008) -[2023-10-14 15:57:07,313][75950] Updated weights for policy 1, policy_version 60340 (0.0007) -[2023-10-14 15:57:07,678][75950] Updated weights for policy 1, policy_version 60350 (0.0008) -[2023-10-14 15:57:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123731968. Throughput: 0: 1698.6, 1: 1680.3. Samples: 30940686. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:57:08,165][74987] Avg episode reward: [(0, '26.370'), (1, '30.950')] -[2023-10-14 15:57:08,210][75949] Updated weights for policy 0, policy_version 60481 (0.0007) -[2023-10-14 15:57:08,587][75949] Updated weights for policy 0, policy_version 60491 (0.0007) -[2023-10-14 15:57:08,963][75949] Updated weights for policy 0, policy_version 60501 (0.0008) -[2023-10-14 15:57:09,337][75949] Updated weights for policy 0, policy_version 60511 (0.0009) -[2023-10-14 15:57:11,603][75950] Updated weights for policy 1, policy_version 60360 (0.0008) -[2023-10-14 15:57:11,973][75950] Updated weights for policy 1, policy_version 60370 (0.0008) -[2023-10-14 15:57:12,337][75950] Updated weights for policy 1, policy_version 60380 (0.0008) -[2023-10-14 15:57:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123797504. Throughput: 0: 1695.2, 1: 1655.5. Samples: 30960296. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 15:57:13,164][74987] Avg episode reward: [(0, '28.900'), (1, '31.970')] -[2023-10-14 15:57:13,193][75949] Updated weights for policy 0, policy_version 60521 (0.0008) -[2023-10-14 15:57:13,566][75949] Updated weights for policy 0, policy_version 60531 (0.0008) -[2023-10-14 15:57:13,933][75949] Updated weights for policy 0, policy_version 60541 (0.0010) -[2023-10-14 15:57:16,360][75950] Updated weights for policy 1, policy_version 60390 (0.0008) -[2023-10-14 15:57:16,722][75950] Updated weights for policy 1, policy_version 60400 (0.0008) -[2023-10-14 15:57:17,096][75950] Updated weights for policy 1, policy_version 60410 (0.0008) -[2023-10-14 15:57:17,893][75949] Updated weights for policy 0, policy_version 60551 (0.0010) -[2023-10-14 15:57:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123863040. Throughput: 0: 1691.9, 1: 1681.9. Samples: 30970580. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-14 15:57:18,165][74987] Avg episode reward: [(0, '24.380'), (1, '31.260')] -[2023-10-14 15:57:18,268][75949] Updated weights for policy 0, policy_version 60561 (0.0011) -[2023-10-14 15:57:18,622][75949] Updated weights for policy 0, policy_version 60571 (0.0011) -[2023-10-14 15:57:21,555][75950] Updated weights for policy 1, policy_version 60420 (0.0009) -[2023-10-14 15:57:21,949][75950] Updated weights for policy 1, policy_version 60430 (0.0009) -[2023-10-14 15:57:22,305][75950] Updated weights for policy 1, policy_version 60440 (0.0009) -[2023-10-14 15:57:22,771][75949] Updated weights for policy 0, policy_version 60581 (0.0009) -[2023-10-14 15:57:23,140][75949] Updated weights for policy 0, policy_version 60591 (0.0009) -[2023-10-14 15:57:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123928576. Throughput: 0: 1691.8, 1: 1672.2. Samples: 30990746. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-14 15:57:23,164][74987] Avg episode reward: [(0, '27.280'), (1, '30.690')] -[2023-10-14 15:57:23,512][75949] Updated weights for policy 0, policy_version 60601 (0.0009) -[2023-10-14 15:57:26,323][75950] Updated weights for policy 1, policy_version 60450 (0.0009) -[2023-10-14 15:57:26,691][75950] Updated weights for policy 1, policy_version 60460 (0.0009) -[2023-10-14 15:57:27,055][75950] Updated weights for policy 1, policy_version 60470 (0.0008) -[2023-10-14 15:57:27,430][75950] Updated weights for policy 1, policy_version 60480 (0.0008) -[2023-10-14 15:57:27,785][75949] Updated weights for policy 0, policy_version 60611 (0.0009) -[2023-10-14 15:57:28,145][75949] Updated weights for policy 0, policy_version 60621 (0.0008) -[2023-10-14 15:57:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 123994112. Throughput: 0: 1681.7, 1: 1659.2. Samples: 31010330. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-14 15:57:28,165][74987] Avg episode reward: [(0, '25.290'), (1, '32.100')] -[2023-10-14 15:57:28,510][75949] Updated weights for policy 0, policy_version 60631 (0.0008) -[2023-10-14 15:57:31,589][75950] Updated weights for policy 1, policy_version 60490 (0.0007) -[2023-10-14 15:57:31,952][75950] Updated weights for policy 1, policy_version 60500 (0.0008) -[2023-10-14 15:57:32,322][75950] Updated weights for policy 1, policy_version 60510 (0.0007) -[2023-10-14 15:57:32,573][75949] Updated weights for policy 0, policy_version 60641 (0.0009) -[2023-10-14 15:57:32,986][75949] Updated weights for policy 0, policy_version 60651 (0.0007) -[2023-10-14 15:57:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124059648. Throughput: 0: 1685.4, 1: 1680.7. Samples: 31020752. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-14 15:57:33,165][74987] Avg episode reward: [(0, '27.850'), (1, '30.970')] -[2023-10-14 15:57:33,357][75949] Updated weights for policy 0, policy_version 60661 (0.0007) -[2023-10-14 15:57:33,726][75949] Updated weights for policy 0, policy_version 60671 (0.0009) -[2023-10-14 15:57:36,501][75950] Updated weights for policy 1, policy_version 60520 (0.0009) -[2023-10-14 15:57:36,873][75950] Updated weights for policy 1, policy_version 60530 (0.0009) -[2023-10-14 15:57:37,245][75950] Updated weights for policy 1, policy_version 60540 (0.0009) -[2023-10-14 15:57:37,653][75949] Updated weights for policy 0, policy_version 60681 (0.0009) -[2023-10-14 15:57:38,024][75949] Updated weights for policy 0, policy_version 60691 (0.0007) -[2023-10-14 15:57:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124125184. Throughput: 0: 1687.3, 1: 1667.8. Samples: 31040964. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-14 15:57:38,165][74987] Avg episode reward: [(0, '26.510'), (1, '31.700')] -[2023-10-14 15:57:38,401][75949] Updated weights for policy 0, policy_version 60701 (0.0009) -[2023-10-14 15:57:41,054][75950] Updated weights for policy 1, policy_version 60550 (0.0007) -[2023-10-14 15:57:41,423][75950] Updated weights for policy 1, policy_version 60560 (0.0008) -[2023-10-14 15:57:41,792][75950] Updated weights for policy 1, policy_version 60570 (0.0008) -[2023-10-14 15:57:42,574][75949] Updated weights for policy 0, policy_version 60711 (0.0008) -[2023-10-14 15:57:42,951][75949] Updated weights for policy 0, policy_version 60721 (0.0009) -[2023-10-14 15:57:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 124190720. Throughput: 0: 1681.9, 1: 1667.8. Samples: 31060748. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-14 15:57:43,164][74987] Avg episode reward: [(0, '26.790'), (1, '33.580')] -[2023-10-14 15:57:43,322][75949] Updated weights for policy 0, policy_version 60731 (0.0008) -[2023-10-14 15:57:45,927][75950] Updated weights for policy 1, policy_version 60580 (0.0008) -[2023-10-14 15:57:46,292][75950] Updated weights for policy 1, policy_version 60590 (0.0010) -[2023-10-14 15:57:46,661][75950] Updated weights for policy 1, policy_version 60600 (0.0009) -[2023-10-14 15:57:47,384][75949] Updated weights for policy 0, policy_version 60741 (0.0008) -[2023-10-14 15:57:47,762][75949] Updated weights for policy 0, policy_version 60751 (0.0010) -[2023-10-14 15:57:48,125][75949] Updated weights for policy 0, policy_version 60761 (0.0009) -[2023-10-14 15:57:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124256256. Throughput: 0: 1688.9, 1: 1675.9. Samples: 31071324. Policy #0 lag: (min: 2.0, avg: 11.2, max: 34.0) -[2023-10-14 15:57:48,164][74987] Avg episode reward: [(0, '26.560'), (1, '29.980')] -[2023-10-14 15:57:50,749][75950] Updated weights for policy 1, policy_version 60610 (0.0009) -[2023-10-14 15:57:51,117][75950] Updated weights for policy 1, policy_version 60620 (0.0007) -[2023-10-14 15:57:51,472][75950] Updated weights for policy 1, policy_version 60630 (0.0008) -[2023-10-14 15:57:51,846][75950] Updated weights for policy 1, policy_version 60640 (0.0008) -[2023-10-14 15:57:51,981][75949] Updated weights for policy 0, policy_version 60771 (0.0009) -[2023-10-14 15:57:52,361][75949] Updated weights for policy 0, policy_version 60781 (0.0008) -[2023-10-14 15:57:52,718][75949] Updated weights for policy 0, policy_version 60791 (0.0010) -[2023-10-14 15:57:53,164][74987] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 124354560. Throughput: 0: 1688.8, 1: 1654.9. Samples: 31091154. Policy #0 lag: (min: 10.0, avg: 16.1, max: 42.0) -[2023-10-14 15:57:53,165][74987] Avg episode reward: [(0, '26.950'), (1, '34.000')] -[2023-10-14 15:57:55,948][75950] Updated weights for policy 1, policy_version 60650 (0.0008) -[2023-10-14 15:57:56,306][75950] Updated weights for policy 1, policy_version 60660 (0.0009) -[2023-10-14 15:57:56,671][75950] Updated weights for policy 1, policy_version 60670 (0.0008) -[2023-10-14 15:57:56,910][75949] Updated weights for policy 0, policy_version 60801 (0.0009) -[2023-10-14 15:57:57,285][75949] Updated weights for policy 0, policy_version 60811 (0.0007) -[2023-10-14 15:57:57,652][75949] Updated weights for policy 0, policy_version 60821 (0.0008) -[2023-10-14 15:57:58,014][75949] Updated weights for policy 0, policy_version 60831 (0.0007) -[2023-10-14 15:57:58,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 124420096. Throughput: 0: 1665.8, 1: 1678.8. Samples: 31110800. Policy #0 lag: (min: 10.0, avg: 16.1, max: 42.0) -[2023-10-14 15:57:58,164][74987] Avg episode reward: [(0, '24.940'), (1, '33.350')] -[2023-10-14 15:58:00,728][75950] Updated weights for policy 1, policy_version 60680 (0.0010) -[2023-10-14 15:58:01,106][75950] Updated weights for policy 1, policy_version 60690 (0.0007) -[2023-10-14 15:58:01,470][75950] Updated weights for policy 1, policy_version 60700 (0.0008) -[2023-10-14 15:58:01,987][75949] Updated weights for policy 0, policy_version 60841 (0.0009) -[2023-10-14 15:58:02,355][75949] Updated weights for policy 0, policy_version 60851 (0.0010) -[2023-10-14 15:58:02,731][75949] Updated weights for policy 0, policy_version 60861 (0.0010) -[2023-10-14 15:58:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 124485632. Throughput: 0: 1689.4, 1: 1675.7. Samples: 31122008. Policy #0 lag: (min: 10.0, avg: 16.1, max: 42.0) -[2023-10-14 15:58:03,165][74987] Avg episode reward: [(0, '26.840'), (1, '33.280')] -[2023-10-14 15:58:05,481][75950] Updated weights for policy 1, policy_version 60710 (0.0008) -[2023-10-14 15:58:05,851][75950] Updated weights for policy 1, policy_version 60720 (0.0009) -[2023-10-14 15:58:06,222][75950] Updated weights for policy 1, policy_version 60730 (0.0010) -[2023-10-14 15:58:06,839][75949] Updated weights for policy 0, policy_version 60871 (0.0008) -[2023-10-14 15:58:07,212][75949] Updated weights for policy 0, policy_version 60881 (0.0007) -[2023-10-14 15:58:07,576][75949] Updated weights for policy 0, policy_version 60891 (0.0008) -[2023-10-14 15:58:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 124551168. Throughput: 0: 1682.8, 1: 1664.5. Samples: 31141376. Policy #0 lag: (min: 10.0, avg: 16.1, max: 42.0) -[2023-10-14 15:58:08,165][74987] Avg episode reward: [(0, '26.180'), (1, '31.830')] -[2023-10-14 15:58:10,232][75950] Updated weights for policy 1, policy_version 60740 (0.0009) -[2023-10-14 15:58:10,624][75950] Updated weights for policy 1, policy_version 60750 (0.0010) -[2023-10-14 15:58:10,984][75950] Updated weights for policy 1, policy_version 60760 (0.0009) -[2023-10-14 15:58:11,589][75949] Updated weights for policy 0, policy_version 60901 (0.0009) -[2023-10-14 15:58:11,954][75949] Updated weights for policy 0, policy_version 60911 (0.0010) -[2023-10-14 15:58:12,328][75949] Updated weights for policy 0, policy_version 60921 (0.0009) -[2023-10-14 15:58:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 124616704. Throughput: 0: 1662.1, 1: 1691.7. Samples: 31161252. Policy #0 lag: (min: 10.0, avg: 16.1, max: 42.0) -[2023-10-14 15:58:13,164][74987] Avg episode reward: [(0, '28.010'), (1, '30.240')] -[2023-10-14 15:58:14,916][75950] Updated weights for policy 1, policy_version 60770 (0.0010) -[2023-10-14 15:58:15,279][75950] Updated weights for policy 1, policy_version 60780 (0.0007) -[2023-10-14 15:58:15,649][75950] Updated weights for policy 1, policy_version 60790 (0.0008) -[2023-10-14 15:58:16,015][75950] Updated weights for policy 1, policy_version 60800 (0.0009) -[2023-10-14 15:58:16,310][75949] Updated weights for policy 0, policy_version 60931 (0.0009) -[2023-10-14 15:58:16,683][75949] Updated weights for policy 0, policy_version 60941 (0.0009) -[2023-10-14 15:58:17,048][75949] Updated weights for policy 0, policy_version 60951 (0.0011) -[2023-10-14 15:58:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 124682240. Throughput: 0: 1688.5, 1: 1676.3. Samples: 31172168. Policy #0 lag: (min: 10.0, avg: 16.1, max: 42.0) -[2023-10-14 15:58:18,165][74987] Avg episode reward: [(0, '27.630'), (1, '30.070')] -[2023-10-14 15:58:20,194][75950] Updated weights for policy 1, policy_version 60810 (0.0007) -[2023-10-14 15:58:20,567][75950] Updated weights for policy 1, policy_version 60820 (0.0008) -[2023-10-14 15:58:20,927][75950] Updated weights for policy 1, policy_version 60830 (0.0009) -[2023-10-14 15:58:21,283][75949] Updated weights for policy 0, policy_version 60961 (0.0009) -[2023-10-14 15:58:21,687][75949] Updated weights for policy 0, policy_version 60971 (0.0010) -[2023-10-14 15:58:22,051][75949] Updated weights for policy 0, policy_version 60981 (0.0009) -[2023-10-14 15:58:22,413][75949] Updated weights for policy 0, policy_version 60991 (0.0009) -[2023-10-14 15:58:23,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 124747776. Throughput: 0: 1675.1, 1: 1676.4. Samples: 31191778. Policy #0 lag: (min: 10.0, avg: 16.1, max: 42.0) -[2023-10-14 15:58:23,164][74987] Avg episode reward: [(0, '24.720'), (1, '33.020')] -[2023-10-14 15:58:24,754][75950] Updated weights for policy 1, policy_version 60840 (0.0009) -[2023-10-14 15:58:25,124][75950] Updated weights for policy 1, policy_version 60850 (0.0008) -[2023-10-14 15:58:25,490][75950] Updated weights for policy 1, policy_version 60860 (0.0009) -[2023-10-14 15:58:26,471][75949] Updated weights for policy 0, policy_version 61001 (0.0009) -[2023-10-14 15:58:26,847][75949] Updated weights for policy 0, policy_version 61011 (0.0009) -[2023-10-14 15:58:27,219][75949] Updated weights for policy 0, policy_version 61021 (0.0008) -[2023-10-14 15:58:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 124813312. Throughput: 0: 1658.7, 1: 1697.7. Samples: 31211788. Policy #0 lag: (min: 10.0, avg: 16.1, max: 42.0) -[2023-10-14 15:58:28,165][74987] Avg episode reward: [(0, '28.490'), (1, '33.770')] -[2023-10-14 15:58:29,472][75950] Updated weights for policy 1, policy_version 60870 (0.0008) -[2023-10-14 15:58:29,847][75950] Updated weights for policy 1, policy_version 60880 (0.0009) -[2023-10-14 15:58:30,210][75950] Updated weights for policy 1, policy_version 60890 (0.0007) -[2023-10-14 15:58:31,158][75949] Updated weights for policy 0, policy_version 61031 (0.0009) -[2023-10-14 15:58:31,531][75949] Updated weights for policy 0, policy_version 61041 (0.0011) -[2023-10-14 15:58:31,894][75949] Updated weights for policy 0, policy_version 61051 (0.0010) -[2023-10-14 15:58:33,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124878848. Throughput: 0: 1683.2, 1: 1668.5. Samples: 31222150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:58:33,165][74987] Avg episode reward: [(0, '25.150'), (1, '31.650')] -[2023-10-14 15:58:34,467][75950] Updated weights for policy 1, policy_version 60900 (0.0007) -[2023-10-14 15:58:34,824][75950] Updated weights for policy 1, policy_version 60910 (0.0008) -[2023-10-14 15:58:35,186][75950] Updated weights for policy 1, policy_version 60920 (0.0010) -[2023-10-14 15:58:35,903][75949] Updated weights for policy 0, policy_version 61061 (0.0012) -[2023-10-14 15:58:36,263][75949] Updated weights for policy 0, policy_version 61071 (0.0011) -[2023-10-14 15:58:36,630][75949] Updated weights for policy 0, policy_version 61081 (0.0010) -[2023-10-14 15:58:38,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 124944384. Throughput: 0: 1660.1, 1: 1686.9. Samples: 31241768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:58:38,164][74987] Avg episode reward: [(0, '28.060'), (1, '32.550')] -[2023-10-14 15:58:39,263][75950] Updated weights for policy 1, policy_version 60930 (0.0008) -[2023-10-14 15:58:39,630][75950] Updated weights for policy 1, policy_version 60940 (0.0008) -[2023-10-14 15:58:39,997][75950] Updated weights for policy 1, policy_version 60950 (0.0008) -[2023-10-14 15:58:40,360][75950] Updated weights for policy 1, policy_version 60960 (0.0007) -[2023-10-14 15:58:40,931][75949] Updated weights for policy 0, policy_version 61091 (0.0009) -[2023-10-14 15:58:41,303][75949] Updated weights for policy 0, policy_version 61101 (0.0010) -[2023-10-14 15:58:41,686][75949] Updated weights for policy 0, policy_version 61111 (0.0009) -[2023-10-14 15:58:43,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 125009920. Throughput: 0: 1671.4, 1: 1687.8. Samples: 31261964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:58:43,164][74987] Avg episode reward: [(0, '25.460'), (1, '33.880')] -[2023-10-14 15:58:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000060960_62423040.pth... -[2023-10-14 15:58:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000061120_62586880.pth... -[2023-10-14 15:58:43,209][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000059552_60981248.pth -[2023-10-14 15:58:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000059392_60817408.pth -[2023-10-14 15:58:44,483][75950] Updated weights for policy 1, policy_version 60970 (0.0010) -[2023-10-14 15:58:44,845][75950] Updated weights for policy 1, policy_version 60980 (0.0010) -[2023-10-14 15:58:45,220][75950] Updated weights for policy 1, policy_version 60990 (0.0010) -[2023-10-14 15:58:45,664][75949] Updated weights for policy 0, policy_version 61121 (0.0011) -[2023-10-14 15:58:46,024][75949] Updated weights for policy 0, policy_version 61131 (0.0009) -[2023-10-14 15:58:46,391][75949] Updated weights for policy 0, policy_version 61141 (0.0009) -[2023-10-14 15:58:46,765][75949] Updated weights for policy 0, policy_version 61151 (0.0010) -[2023-10-14 15:58:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 125075456. Throughput: 0: 1676.8, 1: 1662.2. Samples: 31272260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:58:48,165][74987] Avg episode reward: [(0, '27.800'), (1, '32.020')] -[2023-10-14 15:58:49,205][75950] Updated weights for policy 1, policy_version 61000 (0.0008) -[2023-10-14 15:58:49,573][75950] Updated weights for policy 1, policy_version 61010 (0.0009) -[2023-10-14 15:58:49,939][75950] Updated weights for policy 1, policy_version 61020 (0.0008) -[2023-10-14 15:58:50,890][75949] Updated weights for policy 0, policy_version 61161 (0.0008) -[2023-10-14 15:58:51,265][75949] Updated weights for policy 0, policy_version 61171 (0.0009) -[2023-10-14 15:58:51,634][75949] Updated weights for policy 0, policy_version 61181 (0.0008) -[2023-10-14 15:58:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 125140992. Throughput: 0: 1654.2, 1: 1692.3. Samples: 31291970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:58:53,164][74987] Avg episode reward: [(0, '24.210'), (1, '31.130')] -[2023-10-14 15:58:54,093][75950] Updated weights for policy 1, policy_version 61030 (0.0010) -[2023-10-14 15:58:54,459][75950] Updated weights for policy 1, policy_version 61040 (0.0009) -[2023-10-14 15:58:54,820][75950] Updated weights for policy 1, policy_version 61050 (0.0008) -[2023-10-14 15:58:55,516][75949] Updated weights for policy 0, policy_version 61191 (0.0009) -[2023-10-14 15:58:55,886][75949] Updated weights for policy 0, policy_version 61201 (0.0008) -[2023-10-14 15:58:56,259][75949] Updated weights for policy 0, policy_version 61211 (0.0010) -[2023-10-14 15:58:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 125206528. Throughput: 0: 1677.9, 1: 1691.3. Samples: 31312864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:58:58,165][74987] Avg episode reward: [(0, '28.040'), (1, '31.570')] -[2023-10-14 15:58:58,907][75950] Updated weights for policy 1, policy_version 61060 (0.0008) -[2023-10-14 15:58:59,306][75950] Updated weights for policy 1, policy_version 61070 (0.0008) -[2023-10-14 15:58:59,666][75950] Updated weights for policy 1, policy_version 61080 (0.0010) -[2023-10-14 15:59:00,433][75949] Updated weights for policy 0, policy_version 61221 (0.0007) -[2023-10-14 15:59:00,802][75949] Updated weights for policy 0, policy_version 61231 (0.0007) -[2023-10-14 15:59:01,173][75949] Updated weights for policy 0, policy_version 61241 (0.0009) -[2023-10-14 15:59:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 125272064. Throughput: 0: 1666.6, 1: 1677.2. Samples: 31322636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:59:03,164][74987] Avg episode reward: [(0, '24.710'), (1, '31.520')] -[2023-10-14 15:59:03,804][75950] Updated weights for policy 1, policy_version 61090 (0.0010) -[2023-10-14 15:59:04,175][75950] Updated weights for policy 1, policy_version 61100 (0.0007) -[2023-10-14 15:59:04,537][75950] Updated weights for policy 1, policy_version 61110 (0.0009) -[2023-10-14 15:59:04,899][75950] Updated weights for policy 1, policy_version 61120 (0.0010) -[2023-10-14 15:59:05,321][75949] Updated weights for policy 0, policy_version 61251 (0.0008) -[2023-10-14 15:59:05,698][75949] Updated weights for policy 0, policy_version 61261 (0.0007) -[2023-10-14 15:59:06,059][75949] Updated weights for policy 0, policy_version 61271 (0.0008) -[2023-10-14 15:59:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 125337600. Throughput: 0: 1660.0, 1: 1689.7. Samples: 31342518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 15:59:08,165][74987] Avg episode reward: [(0, '28.560'), (1, '30.580')] -[2023-10-14 15:59:09,010][75950] Updated weights for policy 1, policy_version 61130 (0.0011) -[2023-10-14 15:59:09,381][75950] Updated weights for policy 1, policy_version 61140 (0.0009) -[2023-10-14 15:59:09,740][75950] Updated weights for policy 1, policy_version 61150 (0.0009) -[2023-10-14 15:59:10,087][75949] Updated weights for policy 0, policy_version 61281 (0.0009) -[2023-10-14 15:59:10,504][75949] Updated weights for policy 0, policy_version 61291 (0.0010) -[2023-10-14 15:59:10,877][75949] Updated weights for policy 0, policy_version 61301 (0.0008) -[2023-10-14 15:59:11,253][75949] Updated weights for policy 0, policy_version 61311 (0.0008) -[2023-10-14 15:59:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 125403136. Throughput: 0: 1680.3, 1: 1678.5. Samples: 31362934. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 15:59:13,164][74987] Avg episode reward: [(0, '22.830'), (1, '32.040')] -[2023-10-14 15:59:13,825][75950] Updated weights for policy 1, policy_version 61160 (0.0008) -[2023-10-14 15:59:14,188][75950] Updated weights for policy 1, policy_version 61170 (0.0011) -[2023-10-14 15:59:14,555][75950] Updated weights for policy 1, policy_version 61180 (0.0007) -[2023-10-14 15:59:15,189][75949] Updated weights for policy 0, policy_version 61321 (0.0008) -[2023-10-14 15:59:15,563][75949] Updated weights for policy 0, policy_version 61331 (0.0007) -[2023-10-14 15:59:15,931][75949] Updated weights for policy 0, policy_version 61341 (0.0007) -[2023-10-14 15:59:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 125468672. Throughput: 0: 1660.0, 1: 1682.4. Samples: 31372554. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 15:59:18,165][74987] Avg episode reward: [(0, '28.080'), (1, '34.990')] -[2023-10-14 15:59:18,166][75801] Saving new best policy, reward=34.990! -[2023-10-14 15:59:18,665][75950] Updated weights for policy 1, policy_version 61190 (0.0008) -[2023-10-14 15:59:19,030][75950] Updated weights for policy 1, policy_version 61200 (0.0008) -[2023-10-14 15:59:19,391][75950] Updated weights for policy 1, policy_version 61210 (0.0010) -[2023-10-14 15:59:20,126][75949] Updated weights for policy 0, policy_version 61351 (0.0011) -[2023-10-14 15:59:20,492][75949] Updated weights for policy 0, policy_version 61361 (0.0007) -[2023-10-14 15:59:20,864][75949] Updated weights for policy 0, policy_version 61371 (0.0008) -[2023-10-14 15:59:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 125534208. Throughput: 0: 1672.4, 1: 1685.8. Samples: 31392888. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 15:59:23,164][74987] Avg episode reward: [(0, '23.820'), (1, '32.480')] -[2023-10-14 15:59:23,302][75950] Updated weights for policy 1, policy_version 61220 (0.0007) -[2023-10-14 15:59:23,666][75950] Updated weights for policy 1, policy_version 61230 (0.0007) -[2023-10-14 15:59:24,034][75950] Updated weights for policy 1, policy_version 61240 (0.0009) -[2023-10-14 15:59:24,813][75949] Updated weights for policy 0, policy_version 61381 (0.0008) -[2023-10-14 15:59:25,179][75949] Updated weights for policy 0, policy_version 61391 (0.0009) -[2023-10-14 15:59:25,544][75949] Updated weights for policy 0, policy_version 61401 (0.0009) -[2023-10-14 15:59:28,081][75950] Updated weights for policy 1, policy_version 61250 (0.0009) -[2023-10-14 15:59:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 125599744. Throughput: 0: 1685.2, 1: 1686.6. Samples: 31413694. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 15:59:28,164][74987] Avg episode reward: [(0, '27.650'), (1, '32.260')] -[2023-10-14 15:59:28,447][75950] Updated weights for policy 1, policy_version 61260 (0.0009) -[2023-10-14 15:59:28,815][75950] Updated weights for policy 1, policy_version 61270 (0.0010) -[2023-10-14 15:59:29,183][75950] Updated weights for policy 1, policy_version 61280 (0.0008) -[2023-10-14 15:59:29,635][75949] Updated weights for policy 0, policy_version 61411 (0.0011) -[2023-10-14 15:59:29,997][75949] Updated weights for policy 0, policy_version 61421 (0.0008) -[2023-10-14 15:59:30,367][75949] Updated weights for policy 0, policy_version 61431 (0.0010) -[2023-10-14 15:59:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125665280. Throughput: 0: 1661.4, 1: 1687.3. Samples: 31422954. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 15:59:33,165][74987] Avg episode reward: [(0, '22.890'), (1, '34.610')] -[2023-10-14 15:59:33,204][75950] Updated weights for policy 1, policy_version 61290 (0.0009) -[2023-10-14 15:59:33,572][75950] Updated weights for policy 1, policy_version 61300 (0.0009) -[2023-10-14 15:59:33,937][75950] Updated weights for policy 1, policy_version 61310 (0.0007) -[2023-10-14 15:59:34,637][75949] Updated weights for policy 0, policy_version 61441 (0.0009) -[2023-10-14 15:59:35,004][75949] Updated weights for policy 0, policy_version 61451 (0.0009) -[2023-10-14 15:59:35,373][75949] Updated weights for policy 0, policy_version 61461 (0.0007) -[2023-10-14 15:59:35,742][75949] Updated weights for policy 0, policy_version 61471 (0.0009) -[2023-10-14 15:59:38,094][75950] Updated weights for policy 1, policy_version 61320 (0.0007) -[2023-10-14 15:59:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125730816. Throughput: 0: 1681.7, 1: 1685.8. Samples: 31443506. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 15:59:38,164][74987] Avg episode reward: [(0, '25.930'), (1, '34.310')] -[2023-10-14 15:59:38,455][75950] Updated weights for policy 1, policy_version 61330 (0.0011) -[2023-10-14 15:59:38,831][75950] Updated weights for policy 1, policy_version 61340 (0.0010) -[2023-10-14 15:59:39,780][75949] Updated weights for policy 0, policy_version 61481 (0.0008) -[2023-10-14 15:59:40,159][75949] Updated weights for policy 0, policy_version 61491 (0.0007) -[2023-10-14 15:59:40,528][75949] Updated weights for policy 0, policy_version 61501 (0.0009) -[2023-10-14 15:59:42,884][75950] Updated weights for policy 1, policy_version 61350 (0.0009) -[2023-10-14 15:59:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 125796352. Throughput: 0: 1681.2, 1: 1681.0. Samples: 31464160. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 15:59:43,165][74987] Avg episode reward: [(0, '24.330'), (1, '31.660')] -[2023-10-14 15:59:43,248][75950] Updated weights for policy 1, policy_version 61360 (0.0007) -[2023-10-14 15:59:43,617][75950] Updated weights for policy 1, policy_version 61370 (0.0008) -[2023-10-14 15:59:44,636][75949] Updated weights for policy 0, policy_version 61511 (0.0010) -[2023-10-14 15:59:45,005][75949] Updated weights for policy 0, policy_version 61521 (0.0010) -[2023-10-14 15:59:45,371][75949] Updated weights for policy 0, policy_version 61531 (0.0010) -[2023-10-14 15:59:47,771][75950] Updated weights for policy 1, policy_version 61380 (0.0008) -[2023-10-14 15:59:48,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 125861888. Throughput: 0: 1661.7, 1: 1683.2. Samples: 31473156. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 15:59:48,165][74987] Avg episode reward: [(0, '24.490'), (1, '30.940')] -[2023-10-14 15:59:48,177][75950] Updated weights for policy 1, policy_version 61390 (0.0011) -[2023-10-14 15:59:48,544][75950] Updated weights for policy 1, policy_version 61400 (0.0010) -[2023-10-14 15:59:49,560][75949] Updated weights for policy 0, policy_version 61541 (0.0009) -[2023-10-14 15:59:49,920][75949] Updated weights for policy 0, policy_version 61551 (0.0009) -[2023-10-14 15:59:50,300][75949] Updated weights for policy 0, policy_version 61561 (0.0007) -[2023-10-14 15:59:52,570][75950] Updated weights for policy 1, policy_version 61410 (0.0008) -[2023-10-14 15:59:52,940][75950] Updated weights for policy 1, policy_version 61420 (0.0009) -[2023-10-14 15:59:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125927424. Throughput: 0: 1679.4, 1: 1677.7. Samples: 31493590. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 15:59:53,164][74987] Avg episode reward: [(0, '28.080'), (1, '32.530')] -[2023-10-14 15:59:53,302][75950] Updated weights for policy 1, policy_version 61430 (0.0011) -[2023-10-14 15:59:53,667][75950] Updated weights for policy 1, policy_version 61440 (0.0008) -[2023-10-14 15:59:54,208][75949] Updated weights for policy 0, policy_version 61571 (0.0008) -[2023-10-14 15:59:54,573][75949] Updated weights for policy 0, policy_version 61581 (0.0009) -[2023-10-14 15:59:54,936][75949] Updated weights for policy 0, policy_version 61591 (0.0007) -[2023-10-14 15:59:57,792][75950] Updated weights for policy 1, policy_version 61450 (0.0007) -[2023-10-14 15:59:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125992960. Throughput: 0: 1685.0, 1: 1677.6. Samples: 31514248. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 15:59:58,165][75950] Updated weights for policy 1, policy_version 61460 (0.0008) -[2023-10-14 15:59:58,164][74987] Avg episode reward: [(0, '24.780'), (1, '33.350')] -[2023-10-14 15:59:58,525][75950] Updated weights for policy 1, policy_version 61470 (0.0008) -[2023-10-14 15:59:59,138][75949] Updated weights for policy 0, policy_version 61601 (0.0007) -[2023-10-14 15:59:59,572][75949] Updated weights for policy 0, policy_version 61611 (0.0008) -[2023-10-14 15:59:59,945][75949] Updated weights for policy 0, policy_version 61621 (0.0011) -[2023-10-14 16:00:00,323][75949] Updated weights for policy 0, policy_version 61631 (0.0011) -[2023-10-14 16:00:02,845][75950] Updated weights for policy 1, policy_version 61480 (0.0008) -[2023-10-14 16:00:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126058496. Throughput: 0: 1671.2, 1: 1678.8. Samples: 31523306. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 16:00:03,165][74987] Avg episode reward: [(0, '27.720'), (1, '29.210')] -[2023-10-14 16:00:03,216][75950] Updated weights for policy 1, policy_version 61490 (0.0008) -[2023-10-14 16:00:03,582][75950] Updated weights for policy 1, policy_version 61500 (0.0007) -[2023-10-14 16:00:04,385][75949] Updated weights for policy 0, policy_version 61641 (0.0009) -[2023-10-14 16:00:04,758][75949] Updated weights for policy 0, policy_version 61651 (0.0008) -[2023-10-14 16:00:05,130][75949] Updated weights for policy 0, policy_version 61661 (0.0008) -[2023-10-14 16:00:07,719][75950] Updated weights for policy 1, policy_version 61510 (0.0007) -[2023-10-14 16:00:08,095][75950] Updated weights for policy 1, policy_version 61520 (0.0007) -[2023-10-14 16:00:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126124032. Throughput: 0: 1678.4, 1: 1676.1. Samples: 31543842. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 16:00:08,164][74987] Avg episode reward: [(0, '24.790'), (1, '30.580')] -[2023-10-14 16:00:08,458][75950] Updated weights for policy 1, policy_version 61530 (0.0008) -[2023-10-14 16:00:09,171][75949] Updated weights for policy 0, policy_version 61671 (0.0008) -[2023-10-14 16:00:09,547][75949] Updated weights for policy 0, policy_version 61681 (0.0010) -[2023-10-14 16:00:09,919][75949] Updated weights for policy 0, policy_version 61691 (0.0009) -[2023-10-14 16:00:12,468][75950] Updated weights for policy 1, policy_version 61540 (0.0009) -[2023-10-14 16:00:12,838][75950] Updated weights for policy 1, policy_version 61550 (0.0007) -[2023-10-14 16:00:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126189568. Throughput: 0: 1678.7, 1: 1670.1. Samples: 31564388. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 16:00:13,164][74987] Avg episode reward: [(0, '29.660'), (1, '32.390')] -[2023-10-14 16:00:13,200][75950] Updated weights for policy 1, policy_version 61560 (0.0007) -[2023-10-14 16:00:13,850][75949] Updated weights for policy 0, policy_version 61701 (0.0008) -[2023-10-14 16:00:14,225][75949] Updated weights for policy 0, policy_version 61711 (0.0009) -[2023-10-14 16:00:14,597][75949] Updated weights for policy 0, policy_version 61721 (0.0007) -[2023-10-14 16:00:17,199][75950] Updated weights for policy 1, policy_version 61570 (0.0009) -[2023-10-14 16:00:17,564][75950] Updated weights for policy 1, policy_version 61580 (0.0009) -[2023-10-14 16:00:17,943][75950] Updated weights for policy 1, policy_version 61590 (0.0009) -[2023-10-14 16:00:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126255104. Throughput: 0: 1675.3, 1: 1678.2. Samples: 31573864. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 16:00:18,164][74987] Avg episode reward: [(0, '23.340'), (1, '30.060')] -[2023-10-14 16:00:18,304][75950] Updated weights for policy 1, policy_version 61600 (0.0009) -[2023-10-14 16:00:18,798][75949] Updated weights for policy 0, policy_version 61731 (0.0008) -[2023-10-14 16:00:19,164][75949] Updated weights for policy 0, policy_version 61741 (0.0008) -[2023-10-14 16:00:19,535][75949] Updated weights for policy 0, policy_version 61751 (0.0008) -[2023-10-14 16:00:22,399][75950] Updated weights for policy 1, policy_version 61610 (0.0007) -[2023-10-14 16:00:22,767][75950] Updated weights for policy 1, policy_version 61620 (0.0008) -[2023-10-14 16:00:23,135][75950] Updated weights for policy 1, policy_version 61630 (0.0007) -[2023-10-14 16:00:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126320640. Throughput: 0: 1674.4, 1: 1673.3. Samples: 31594154. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 16:00:23,165][74987] Avg episode reward: [(0, '29.210'), (1, '30.500')] -[2023-10-14 16:00:23,585][75949] Updated weights for policy 0, policy_version 61761 (0.0011) -[2023-10-14 16:00:23,958][75949] Updated weights for policy 0, policy_version 61771 (0.0008) -[2023-10-14 16:00:24,318][75949] Updated weights for policy 0, policy_version 61781 (0.0007) -[2023-10-14 16:00:24,683][75949] Updated weights for policy 0, policy_version 61791 (0.0007) -[2023-10-14 16:00:26,980][75950] Updated weights for policy 1, policy_version 61640 (0.0008) -[2023-10-14 16:00:27,354][75950] Updated weights for policy 1, policy_version 61650 (0.0007) -[2023-10-14 16:00:27,726][75950] Updated weights for policy 1, policy_version 61660 (0.0010) -[2023-10-14 16:00:28,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 126418944. Throughput: 0: 1672.0, 1: 1656.8. Samples: 31613958. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) -[2023-10-14 16:00:28,165][74987] Avg episode reward: [(0, '23.670'), (1, '35.030')] -[2023-10-14 16:00:28,173][75801] Saving new best policy, reward=35.030! -[2023-10-14 16:00:28,812][75949] Updated weights for policy 0, policy_version 61801 (0.0011) -[2023-10-14 16:00:29,192][75949] Updated weights for policy 0, policy_version 61811 (0.0009) -[2023-10-14 16:00:29,556][75949] Updated weights for policy 0, policy_version 61821 (0.0009) -[2023-10-14 16:00:31,758][75950] Updated weights for policy 1, policy_version 61670 (0.0007) -[2023-10-14 16:00:32,120][75950] Updated weights for policy 1, policy_version 61680 (0.0009) -[2023-10-14 16:00:32,503][75950] Updated weights for policy 1, policy_version 61690 (0.0007) -[2023-10-14 16:00:33,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 126484480. Throughput: 0: 1671.1, 1: 1681.9. Samples: 31624038. Policy #0 lag: (min: 11.0, avg: 11.2, max: 21.0) -[2023-10-14 16:00:33,164][74987] Avg episode reward: [(0, '25.960'), (1, '36.260')] -[2023-10-14 16:00:33,165][75801] Saving new best policy, reward=36.260! -[2023-10-14 16:00:33,796][75949] Updated weights for policy 0, policy_version 61831 (0.0009) -[2023-10-14 16:00:34,173][75949] Updated weights for policy 0, policy_version 61841 (0.0009) -[2023-10-14 16:00:34,538][75949] Updated weights for policy 0, policy_version 61851 (0.0009) -[2023-10-14 16:00:36,836][75950] Updated weights for policy 1, policy_version 61700 (0.0008) -[2023-10-14 16:00:37,239][75950] Updated weights for policy 1, policy_version 61710 (0.0009) -[2023-10-14 16:00:37,609][75950] Updated weights for policy 1, policy_version 61720 (0.0008) -[2023-10-14 16:00:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 126550016. Throughput: 0: 1678.3, 1: 1675.8. Samples: 31644522. Policy #0 lag: (min: 11.0, avg: 11.2, max: 21.0) -[2023-10-14 16:00:38,165][74987] Avg episode reward: [(0, '24.270'), (1, '31.920')] -[2023-10-14 16:00:38,499][75949] Updated weights for policy 0, policy_version 61861 (0.0008) -[2023-10-14 16:00:38,868][75949] Updated weights for policy 0, policy_version 61871 (0.0007) -[2023-10-14 16:00:39,246][75949] Updated weights for policy 0, policy_version 61881 (0.0010) -[2023-10-14 16:00:41,641][75950] Updated weights for policy 1, policy_version 61730 (0.0008) -[2023-10-14 16:00:42,012][75950] Updated weights for policy 1, policy_version 61740 (0.0009) -[2023-10-14 16:00:42,376][75950] Updated weights for policy 1, policy_version 61750 (0.0008) -[2023-10-14 16:00:42,749][75950] Updated weights for policy 1, policy_version 61760 (0.0009) -[2023-10-14 16:00:43,048][75949] Updated weights for policy 0, policy_version 61891 (0.0010) -[2023-10-14 16:00:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 126615552. Throughput: 0: 1684.0, 1: 1650.7. Samples: 31664312. Policy #0 lag: (min: 11.0, avg: 11.2, max: 21.0) -[2023-10-14 16:00:43,164][74987] Avg episode reward: [(0, '26.620'), (1, '31.240')] -[2023-10-14 16:00:43,170][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000061760_63242240.pth... -[2023-10-14 16:00:43,199][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000060192_61636608.pth -[2023-10-14 16:00:43,424][75949] Updated weights for policy 0, policy_version 61901 (0.0009) -[2023-10-14 16:00:43,803][75949] Updated weights for policy 0, policy_version 61911 (0.0007) -[2023-10-14 16:00:44,133][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000061920_63406080.pth... -[2023-10-14 16:00:44,164][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000060320_61767680.pth -[2023-10-14 16:00:46,865][75950] Updated weights for policy 1, policy_version 61770 (0.0008) -[2023-10-14 16:00:47,236][75950] Updated weights for policy 1, policy_version 61780 (0.0009) -[2023-10-14 16:00:47,608][75950] Updated weights for policy 1, policy_version 61790 (0.0009) -[2023-10-14 16:00:47,722][75949] Updated weights for policy 0, policy_version 61921 (0.0010) -[2023-10-14 16:00:48,141][75949] Updated weights for policy 0, policy_version 61931 (0.0011) -[2023-10-14 16:00:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 126681088. Throughput: 0: 1687.7, 1: 1668.9. Samples: 31674354. Policy #0 lag: (min: 11.0, avg: 11.2, max: 21.0) -[2023-10-14 16:00:48,165][74987] Avg episode reward: [(0, '25.400'), (1, '32.370')] -[2023-10-14 16:00:48,511][75949] Updated weights for policy 0, policy_version 61941 (0.0009) -[2023-10-14 16:00:48,880][75949] Updated weights for policy 0, policy_version 61951 (0.0007) -[2023-10-14 16:00:51,700][75950] Updated weights for policy 1, policy_version 61800 (0.0009) -[2023-10-14 16:00:52,069][75950] Updated weights for policy 1, policy_version 61810 (0.0009) -[2023-10-14 16:00:52,446][75950] Updated weights for policy 1, policy_version 61820 (0.0007) -[2023-10-14 16:00:52,928][75949] Updated weights for policy 0, policy_version 61961 (0.0008) -[2023-10-14 16:00:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 126746624. Throughput: 0: 1687.5, 1: 1665.2. Samples: 31694716. Policy #0 lag: (min: 11.0, avg: 11.2, max: 21.0) -[2023-10-14 16:00:53,165][74987] Avg episode reward: [(0, '26.760'), (1, '30.140')] -[2023-10-14 16:00:53,298][75949] Updated weights for policy 0, policy_version 61971 (0.0007) -[2023-10-14 16:00:53,669][75949] Updated weights for policy 0, policy_version 61981 (0.0007) -[2023-10-14 16:00:56,611][75950] Updated weights for policy 1, policy_version 61830 (0.0009) -[2023-10-14 16:00:56,981][75950] Updated weights for policy 1, policy_version 61840 (0.0010) -[2023-10-14 16:00:57,345][75950] Updated weights for policy 1, policy_version 61850 (0.0010) -[2023-10-14 16:00:57,717][75949] Updated weights for policy 0, policy_version 61991 (0.0008) -[2023-10-14 16:00:58,088][75949] Updated weights for policy 0, policy_version 62001 (0.0007) -[2023-10-14 16:00:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 126812160. Throughput: 0: 1689.9, 1: 1645.5. Samples: 31714482. Policy #0 lag: (min: 11.0, avg: 11.2, max: 21.0) -[2023-10-14 16:00:58,164][74987] Avg episode reward: [(0, '25.390'), (1, '29.320')] -[2023-10-14 16:00:58,454][75949] Updated weights for policy 0, policy_version 62011 (0.0010) -[2023-10-14 16:01:01,614][75950] Updated weights for policy 1, policy_version 61860 (0.0010) -[2023-10-14 16:01:01,967][75950] Updated weights for policy 1, policy_version 61870 (0.0010) -[2023-10-14 16:01:02,345][75950] Updated weights for policy 1, policy_version 61880 (0.0009) -[2023-10-14 16:01:02,496][75949] Updated weights for policy 0, policy_version 62021 (0.0010) -[2023-10-14 16:01:02,879][75949] Updated weights for policy 0, policy_version 62031 (0.0008) -[2023-10-14 16:01:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 126877696. Throughput: 0: 1691.3, 1: 1666.3. Samples: 31724954. Policy #0 lag: (min: 11.0, avg: 11.2, max: 21.0) -[2023-10-14 16:01:03,165][74987] Avg episode reward: [(0, '24.360'), (1, '32.220')] -[2023-10-14 16:01:03,247][75949] Updated weights for policy 0, policy_version 62041 (0.0009) -[2023-10-14 16:01:06,437][75950] Updated weights for policy 1, policy_version 61890 (0.0008) -[2023-10-14 16:01:06,800][75950] Updated weights for policy 1, policy_version 61900 (0.0007) -[2023-10-14 16:01:07,164][75950] Updated weights for policy 1, policy_version 61910 (0.0009) -[2023-10-14 16:01:07,410][75949] Updated weights for policy 0, policy_version 62051 (0.0008) -[2023-10-14 16:01:07,535][75950] Updated weights for policy 1, policy_version 61920 (0.0008) -[2023-10-14 16:01:07,787][75949] Updated weights for policy 0, policy_version 62061 (0.0009) -[2023-10-14 16:01:08,156][75949] Updated weights for policy 0, policy_version 62071 (0.0008) -[2023-10-14 16:01:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 126943232. Throughput: 0: 1698.0, 1: 1662.5. Samples: 31745378. Policy #0 lag: (min: 11.0, avg: 11.2, max: 21.0) -[2023-10-14 16:01:08,164][74987] Avg episode reward: [(0, '26.880'), (1, '31.430')] -[2023-10-14 16:01:11,871][75950] Updated weights for policy 1, policy_version 61930 (0.0008) -[2023-10-14 16:01:12,176][75949] Updated weights for policy 0, policy_version 62081 (0.0008) -[2023-10-14 16:01:12,244][75950] Updated weights for policy 1, policy_version 61940 (0.0009) -[2023-10-14 16:01:12,551][75949] Updated weights for policy 0, policy_version 62091 (0.0009) -[2023-10-14 16:01:12,605][75950] Updated weights for policy 1, policy_version 61950 (0.0007) -[2023-10-14 16:01:12,906][75949] Updated weights for policy 0, policy_version 62101 (0.0010) -[2023-10-14 16:01:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 127008768. Throughput: 0: 1688.2, 1: 1657.3. Samples: 31764506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:01:13,165][74987] Avg episode reward: [(0, '27.120'), (1, '29.440')] -[2023-10-14 16:01:13,287][75949] Updated weights for policy 0, policy_version 62111 (0.0011) -[2023-10-14 16:01:16,671][75950] Updated weights for policy 1, policy_version 61960 (0.0008) -[2023-10-14 16:01:17,036][75950] Updated weights for policy 1, policy_version 61970 (0.0008) -[2023-10-14 16:01:17,402][75950] Updated weights for policy 1, policy_version 61980 (0.0008) -[2023-10-14 16:01:17,457][75949] Updated weights for policy 0, policy_version 62121 (0.0009) -[2023-10-14 16:01:17,838][75949] Updated weights for policy 0, policy_version 62131 (0.0009) -[2023-10-14 16:01:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 127074304. Throughput: 0: 1700.5, 1: 1658.2. Samples: 31775180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:01:18,165][74987] Avg episode reward: [(0, '28.770'), (1, '30.180')] -[2023-10-14 16:01:18,214][75949] Updated weights for policy 0, policy_version 62141 (0.0010) -[2023-10-14 16:01:21,664][75950] Updated weights for policy 1, policy_version 61990 (0.0009) -[2023-10-14 16:01:22,047][75950] Updated weights for policy 1, policy_version 62000 (0.0011) -[2023-10-14 16:01:22,417][75950] Updated weights for policy 1, policy_version 62010 (0.0007) -[2023-10-14 16:01:22,431][75949] Updated weights for policy 0, policy_version 62151 (0.0008) -[2023-10-14 16:01:22,797][75949] Updated weights for policy 0, policy_version 62161 (0.0007) -[2023-10-14 16:01:23,159][75949] Updated weights for policy 0, policy_version 62171 (0.0007) -[2023-10-14 16:01:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 127139840. Throughput: 0: 1691.3, 1: 1658.9. Samples: 31795280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:01:23,165][74987] Avg episode reward: [(0, '24.100'), (1, '32.230')] -[2023-10-14 16:01:26,308][75950] Updated weights for policy 1, policy_version 62020 (0.0010) -[2023-10-14 16:01:26,674][75950] Updated weights for policy 1, policy_version 62030 (0.0010) -[2023-10-14 16:01:27,033][75950] Updated weights for policy 1, policy_version 62040 (0.0009) -[2023-10-14 16:01:27,198][75949] Updated weights for policy 0, policy_version 62181 (0.0008) -[2023-10-14 16:01:27,558][75949] Updated weights for policy 0, policy_version 62191 (0.0008) -[2023-10-14 16:01:27,929][75949] Updated weights for policy 0, policy_version 62201 (0.0007) -[2023-10-14 16:01:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 127205376. Throughput: 0: 1662.6, 1: 1667.5. Samples: 31814168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:01:28,165][74987] Avg episode reward: [(0, '28.090'), (1, '31.200')] -[2023-10-14 16:01:31,059][75950] Updated weights for policy 1, policy_version 62050 (0.0009) -[2023-10-14 16:01:31,425][75950] Updated weights for policy 1, policy_version 62060 (0.0008) -[2023-10-14 16:01:31,787][75950] Updated weights for policy 1, policy_version 62070 (0.0009) -[2023-10-14 16:01:32,150][75950] Updated weights for policy 1, policy_version 62080 (0.0008) -[2023-10-14 16:01:32,261][75949] Updated weights for policy 0, policy_version 62211 (0.0008) -[2023-10-14 16:01:32,632][75949] Updated weights for policy 0, policy_version 62221 (0.0008) -[2023-10-14 16:01:33,010][75949] Updated weights for policy 0, policy_version 62231 (0.0008) -[2023-10-14 16:01:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 127270912. Throughput: 0: 1676.7, 1: 1673.4. Samples: 31825108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:01:33,165][74987] Avg episode reward: [(0, '23.450'), (1, '31.620')] -[2023-10-14 16:01:36,182][75950] Updated weights for policy 1, policy_version 62090 (0.0007) -[2023-10-14 16:01:36,555][75950] Updated weights for policy 1, policy_version 62100 (0.0009) -[2023-10-14 16:01:36,917][75950] Updated weights for policy 1, policy_version 62110 (0.0008) -[2023-10-14 16:01:36,959][75949] Updated weights for policy 0, policy_version 62241 (0.0008) -[2023-10-14 16:01:37,379][75949] Updated weights for policy 0, policy_version 62251 (0.0009) -[2023-10-14 16:01:37,745][75949] Updated weights for policy 0, policy_version 62261 (0.0008) -[2023-10-14 16:01:38,121][75949] Updated weights for policy 0, policy_version 62271 (0.0008) -[2023-10-14 16:01:38,163][74987] Fps is (10 sec: 16384.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 127369216. Throughput: 0: 1677.4, 1: 1661.7. Samples: 31844974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:01:38,164][74987] Avg episode reward: [(0, '28.500'), (1, '32.650')] -[2023-10-14 16:01:41,048][75950] Updated weights for policy 1, policy_version 62120 (0.0007) -[2023-10-14 16:01:41,407][75950] Updated weights for policy 1, policy_version 62130 (0.0007) -[2023-10-14 16:01:41,775][75950] Updated weights for policy 1, policy_version 62140 (0.0007) -[2023-10-14 16:01:42,117][75949] Updated weights for policy 0, policy_version 62281 (0.0008) -[2023-10-14 16:01:42,489][75949] Updated weights for policy 0, policy_version 62291 (0.0009) -[2023-10-14 16:01:42,860][75949] Updated weights for policy 0, policy_version 62301 (0.0010) -[2023-10-14 16:01:43,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 127434752. Throughput: 0: 1648.4, 1: 1678.0. Samples: 31864172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:01:43,165][74987] Avg episode reward: [(0, '23.610'), (1, '29.940')] -[2023-10-14 16:01:45,960][75950] Updated weights for policy 1, policy_version 62150 (0.0007) -[2023-10-14 16:01:46,316][75950] Updated weights for policy 1, policy_version 62160 (0.0008) -[2023-10-14 16:01:46,686][75950] Updated weights for policy 1, policy_version 62170 (0.0007) -[2023-10-14 16:01:46,847][75949] Updated weights for policy 0, policy_version 62311 (0.0009) -[2023-10-14 16:01:47,213][75949] Updated weights for policy 0, policy_version 62321 (0.0008) -[2023-10-14 16:01:47,581][75949] Updated weights for policy 0, policy_version 62331 (0.0008) -[2023-10-14 16:01:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 127500288. Throughput: 0: 1669.0, 1: 1676.7. Samples: 31875512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:01:48,164][74987] Avg episode reward: [(0, '29.110'), (1, '31.830')] -[2023-10-14 16:01:50,799][75950] Updated weights for policy 1, policy_version 62180 (0.0007) -[2023-10-14 16:01:51,168][75950] Updated weights for policy 1, policy_version 62190 (0.0009) -[2023-10-14 16:01:51,537][75950] Updated weights for policy 1, policy_version 62200 (0.0008) -[2023-10-14 16:01:51,666][75949] Updated weights for policy 0, policy_version 62341 (0.0009) -[2023-10-14 16:01:52,030][75949] Updated weights for policy 0, policy_version 62351 (0.0008) -[2023-10-14 16:01:52,409][75949] Updated weights for policy 0, policy_version 62361 (0.0008) -[2023-10-14 16:01:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 127565824. Throughput: 0: 1661.9, 1: 1655.2. Samples: 31894646. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) -[2023-10-14 16:01:53,164][74987] Avg episode reward: [(0, '23.410'), (1, '30.900')] -[2023-10-14 16:01:55,375][75950] Updated weights for policy 1, policy_version 62210 (0.0009) -[2023-10-14 16:01:55,737][75950] Updated weights for policy 1, policy_version 62220 (0.0007) -[2023-10-14 16:01:56,105][75950] Updated weights for policy 1, policy_version 62230 (0.0009) -[2023-10-14 16:01:56,443][75949] Updated weights for policy 0, policy_version 62371 (0.0009) -[2023-10-14 16:01:56,471][75950] Updated weights for policy 1, policy_version 62240 (0.0010) -[2023-10-14 16:01:56,811][75949] Updated weights for policy 0, policy_version 62381 (0.0009) -[2023-10-14 16:01:57,180][75949] Updated weights for policy 0, policy_version 62391 (0.0011) -[2023-10-14 16:01:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 127631360. Throughput: 0: 1649.9, 1: 1675.3. Samples: 31914140. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) -[2023-10-14 16:01:58,165][74987] Avg episode reward: [(0, '27.700'), (1, '31.940')] -[2023-10-14 16:02:00,625][75950] Updated weights for policy 1, policy_version 62250 (0.0011) -[2023-10-14 16:02:00,992][75950] Updated weights for policy 1, policy_version 62260 (0.0011) -[2023-10-14 16:02:01,357][75950] Updated weights for policy 1, policy_version 62270 (0.0009) -[2023-10-14 16:02:01,376][75949] Updated weights for policy 0, policy_version 62401 (0.0008) -[2023-10-14 16:02:01,744][75949] Updated weights for policy 0, policy_version 62411 (0.0010) -[2023-10-14 16:02:02,116][75949] Updated weights for policy 0, policy_version 62421 (0.0007) -[2023-10-14 16:02:02,484][75949] Updated weights for policy 0, policy_version 62431 (0.0008) -[2023-10-14 16:02:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 127696896. Throughput: 0: 1666.9, 1: 1664.8. Samples: 31925104. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) -[2023-10-14 16:02:03,165][74987] Avg episode reward: [(0, '23.120'), (1, '32.330')] -[2023-10-14 16:02:05,405][75950] Updated weights for policy 1, policy_version 62280 (0.0009) -[2023-10-14 16:02:05,773][75950] Updated weights for policy 1, policy_version 62290 (0.0010) -[2023-10-14 16:02:06,138][75950] Updated weights for policy 1, policy_version 62300 (0.0008) -[2023-10-14 16:02:06,452][75949] Updated weights for policy 0, policy_version 62441 (0.0009) -[2023-10-14 16:02:06,818][75949] Updated weights for policy 0, policy_version 62451 (0.0008) -[2023-10-14 16:02:07,199][75949] Updated weights for policy 0, policy_version 62461 (0.0007) -[2023-10-14 16:02:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 127762432. Throughput: 0: 1664.1, 1: 1651.4. Samples: 31944476. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) -[2023-10-14 16:02:08,165][74987] Avg episode reward: [(0, '26.570'), (1, '29.800')] -[2023-10-14 16:02:10,470][75950] Updated weights for policy 1, policy_version 62310 (0.0008) -[2023-10-14 16:02:10,867][75950] Updated weights for policy 1, policy_version 62320 (0.0011) -[2023-10-14 16:02:11,097][75949] Updated weights for policy 0, policy_version 62471 (0.0007) -[2023-10-14 16:02:11,225][75950] Updated weights for policy 1, policy_version 62330 (0.0008) -[2023-10-14 16:02:11,453][75949] Updated weights for policy 0, policy_version 62481 (0.0008) -[2023-10-14 16:02:11,828][75949] Updated weights for policy 0, policy_version 62491 (0.0009) -[2023-10-14 16:02:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 127827968. Throughput: 0: 1673.1, 1: 1673.1. Samples: 31964744. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) -[2023-10-14 16:02:13,165][74987] Avg episode reward: [(0, '25.250'), (1, '29.730')] -[2023-10-14 16:02:15,177][75950] Updated weights for policy 1, policy_version 62340 (0.0007) -[2023-10-14 16:02:15,549][75950] Updated weights for policy 1, policy_version 62350 (0.0009) -[2023-10-14 16:02:15,831][75949] Updated weights for policy 0, policy_version 62501 (0.0007) -[2023-10-14 16:02:15,916][75950] Updated weights for policy 1, policy_version 62360 (0.0008) -[2023-10-14 16:02:16,192][75949] Updated weights for policy 0, policy_version 62511 (0.0008) -[2023-10-14 16:02:16,566][75949] Updated weights for policy 0, policy_version 62521 (0.0009) -[2023-10-14 16:02:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 127893504. Throughput: 0: 1689.4, 1: 1660.5. Samples: 31975856. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) -[2023-10-14 16:02:18,164][74987] Avg episode reward: [(0, '25.570'), (1, '30.780')] -[2023-10-14 16:02:20,040][75950] Updated weights for policy 1, policy_version 62370 (0.0012) -[2023-10-14 16:02:20,397][75950] Updated weights for policy 1, policy_version 62380 (0.0008) -[2023-10-14 16:02:20,769][75950] Updated weights for policy 1, policy_version 62390 (0.0009) -[2023-10-14 16:02:20,786][75949] Updated weights for policy 0, policy_version 62531 (0.0009) -[2023-10-14 16:02:21,131][75950] Updated weights for policy 1, policy_version 62400 (0.0009) -[2023-10-14 16:02:21,154][75949] Updated weights for policy 0, policy_version 62541 (0.0007) -[2023-10-14 16:02:21,524][75949] Updated weights for policy 0, policy_version 62551 (0.0010) -[2023-10-14 16:02:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 127959040. Throughput: 0: 1669.1, 1: 1661.5. Samples: 31994852. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) -[2023-10-14 16:02:23,165][74987] Avg episode reward: [(0, '25.110'), (1, '29.990')] -[2023-10-14 16:02:25,175][75950] Updated weights for policy 1, policy_version 62410 (0.0009) -[2023-10-14 16:02:25,547][75950] Updated weights for policy 1, policy_version 62420 (0.0009) -[2023-10-14 16:02:25,562][75949] Updated weights for policy 0, policy_version 62561 (0.0010) -[2023-10-14 16:02:25,917][75950] Updated weights for policy 1, policy_version 62430 (0.0009) -[2023-10-14 16:02:25,967][75949] Updated weights for policy 0, policy_version 62571 (0.0009) -[2023-10-14 16:02:26,333][75949] Updated weights for policy 0, policy_version 62581 (0.0007) -[2023-10-14 16:02:26,705][75949] Updated weights for policy 0, policy_version 62591 (0.0008) -[2023-10-14 16:02:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 128024576. Throughput: 0: 1682.4, 1: 1671.0. Samples: 32015074. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) -[2023-10-14 16:02:28,165][74987] Avg episode reward: [(0, '25.450'), (1, '31.420')] -[2023-10-14 16:02:30,064][75950] Updated weights for policy 1, policy_version 62440 (0.0009) -[2023-10-14 16:02:30,424][75950] Updated weights for policy 1, policy_version 62450 (0.0009) -[2023-10-14 16:02:30,592][75949] Updated weights for policy 0, policy_version 62601 (0.0008) -[2023-10-14 16:02:30,790][75950] Updated weights for policy 1, policy_version 62460 (0.0008) -[2023-10-14 16:02:30,956][75949] Updated weights for policy 0, policy_version 62611 (0.0009) -[2023-10-14 16:02:31,333][75949] Updated weights for policy 0, policy_version 62621 (0.0010) -[2023-10-14 16:02:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 128090112. Throughput: 0: 1681.0, 1: 1651.7. Samples: 32025482. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) -[2023-10-14 16:02:33,165][74987] Avg episode reward: [(0, '27.000'), (1, '31.600')] -[2023-10-14 16:02:34,894][75950] Updated weights for policy 1, policy_version 62470 (0.0009) -[2023-10-14 16:02:35,274][75950] Updated weights for policy 1, policy_version 62480 (0.0008) -[2023-10-14 16:02:35,340][75949] Updated weights for policy 0, policy_version 62631 (0.0008) -[2023-10-14 16:02:35,636][75950] Updated weights for policy 1, policy_version 62490 (0.0007) -[2023-10-14 16:02:35,706][75949] Updated weights for policy 0, policy_version 62641 (0.0007) -[2023-10-14 16:02:36,075][75949] Updated weights for policy 0, policy_version 62651 (0.0007) -[2023-10-14 16:02:38,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 128155648. Throughput: 0: 1672.0, 1: 1670.2. Samples: 32045046. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 16:02:38,164][74987] Avg episode reward: [(0, '26.190'), (1, '30.430')] -[2023-10-14 16:02:39,806][75950] Updated weights for policy 1, policy_version 62500 (0.0007) -[2023-10-14 16:02:40,164][75950] Updated weights for policy 1, policy_version 62510 (0.0008) -[2023-10-14 16:02:40,235][75949] Updated weights for policy 0, policy_version 62661 (0.0009) -[2023-10-14 16:02:40,526][75950] Updated weights for policy 1, policy_version 62520 (0.0008) -[2023-10-14 16:02:40,610][75949] Updated weights for policy 0, policy_version 62671 (0.0007) -[2023-10-14 16:02:40,986][75949] Updated weights for policy 0, policy_version 62681 (0.0008) -[2023-10-14 16:02:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 128221184. Throughput: 0: 1696.5, 1: 1667.1. Samples: 32065502. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 16:02:43,165][74987] Avg episode reward: [(0, '27.810'), (1, '32.160')] -[2023-10-14 16:02:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000062528_64028672.pth... -[2023-10-14 16:02:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000062688_64192512.pth... -[2023-10-14 16:02:43,217][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000060960_62423040.pth -[2023-10-14 16:02:43,218][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000061120_62586880.pth -[2023-10-14 16:02:43,222][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000062528_64028672.pth -[2023-10-14 16:02:43,224][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000062688_64192512.pth -[2023-10-14 16:02:44,640][75950] Updated weights for policy 1, policy_version 62530 (0.0008) -[2023-10-14 16:02:44,937][75949] Updated weights for policy 0, policy_version 62691 (0.0008) -[2023-10-14 16:02:44,999][75950] Updated weights for policy 1, policy_version 62540 (0.0008) -[2023-10-14 16:02:45,299][75949] Updated weights for policy 0, policy_version 62701 (0.0007) -[2023-10-14 16:02:45,367][75950] Updated weights for policy 1, policy_version 62550 (0.0008) -[2023-10-14 16:02:45,667][75949] Updated weights for policy 0, policy_version 62711 (0.0008) -[2023-10-14 16:02:45,734][75950] Updated weights for policy 1, policy_version 62560 (0.0008) -[2023-10-14 16:02:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128286720. Throughput: 0: 1680.1, 1: 1656.4. Samples: 32075246. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 16:02:48,165][74987] Avg episode reward: [(0, '27.070'), (1, '30.610')] -[2023-10-14 16:02:49,753][75949] Updated weights for policy 0, policy_version 62721 (0.0010) -[2023-10-14 16:02:49,758][75950] Updated weights for policy 1, policy_version 62570 (0.0010) -[2023-10-14 16:02:50,124][75950] Updated weights for policy 1, policy_version 62580 (0.0007) -[2023-10-14 16:02:50,131][75949] Updated weights for policy 0, policy_version 62731 (0.0008) -[2023-10-14 16:02:50,498][75949] Updated weights for policy 0, policy_version 62741 (0.0008) -[2023-10-14 16:02:50,501][75950] Updated weights for policy 1, policy_version 62590 (0.0009) -[2023-10-14 16:02:50,863][75949] Updated weights for policy 0, policy_version 62751 (0.0007) -[2023-10-14 16:02:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128352256. Throughput: 0: 1676.7, 1: 1675.3. Samples: 32095318. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 16:02:53,164][74987] Avg episode reward: [(0, '28.690'), (1, '31.150')] -[2023-10-14 16:02:54,649][75950] Updated weights for policy 1, policy_version 62600 (0.0009) -[2023-10-14 16:02:55,019][75950] Updated weights for policy 1, policy_version 62610 (0.0007) -[2023-10-14 16:02:55,023][75949] Updated weights for policy 0, policy_version 62761 (0.0007) -[2023-10-14 16:02:55,391][75949] Updated weights for policy 0, policy_version 62771 (0.0007) -[2023-10-14 16:02:55,393][75950] Updated weights for policy 1, policy_version 62620 (0.0009) -[2023-10-14 16:02:55,763][75949] Updated weights for policy 0, policy_version 62781 (0.0007) -[2023-10-14 16:02:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128417792. Throughput: 0: 1691.5, 1: 1671.6. Samples: 32116086. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 16:02:58,165][74987] Avg episode reward: [(0, '26.310'), (1, '33.320')] -[2023-10-14 16:02:59,519][75950] Updated weights for policy 1, policy_version 62630 (0.0008) -[2023-10-14 16:02:59,808][75949] Updated weights for policy 0, policy_version 62791 (0.0008) -[2023-10-14 16:02:59,887][75950] Updated weights for policy 1, policy_version 62640 (0.0010) -[2023-10-14 16:03:00,174][75949] Updated weights for policy 0, policy_version 62801 (0.0007) -[2023-10-14 16:03:00,256][75950] Updated weights for policy 1, policy_version 62650 (0.0008) -[2023-10-14 16:03:00,536][75949] Updated weights for policy 0, policy_version 62811 (0.0009) -[2023-10-14 16:03:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128483328. Throughput: 0: 1662.7, 1: 1655.1. Samples: 32125158. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 16:03:03,164][74987] Avg episode reward: [(0, '27.840'), (1, '30.260')] -[2023-10-14 16:03:04,347][75950] Updated weights for policy 1, policy_version 62660 (0.0009) -[2023-10-14 16:03:04,702][75949] Updated weights for policy 0, policy_version 62821 (0.0009) -[2023-10-14 16:03:04,713][75950] Updated weights for policy 1, policy_version 62670 (0.0007) -[2023-10-14 16:03:05,067][75949] Updated weights for policy 0, policy_version 62831 (0.0008) -[2023-10-14 16:03:05,071][75950] Updated weights for policy 1, policy_version 62680 (0.0008) -[2023-10-14 16:03:05,438][75949] Updated weights for policy 0, policy_version 62841 (0.0007) -[2023-10-14 16:03:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 128548864. Throughput: 0: 1673.9, 1: 1669.6. Samples: 32145308. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 16:03:08,165][74987] Avg episode reward: [(0, '26.300'), (1, '30.160')] -[2023-10-14 16:03:09,140][75950] Updated weights for policy 1, policy_version 62690 (0.0010) -[2023-10-14 16:03:09,429][75949] Updated weights for policy 0, policy_version 62851 (0.0007) -[2023-10-14 16:03:09,502][75950] Updated weights for policy 1, policy_version 62700 (0.0009) -[2023-10-14 16:03:09,794][75949] Updated weights for policy 0, policy_version 62861 (0.0007) -[2023-10-14 16:03:09,873][75950] Updated weights for policy 1, policy_version 62710 (0.0008) -[2023-10-14 16:03:10,162][75949] Updated weights for policy 0, policy_version 62871 (0.0009) -[2023-10-14 16:03:10,235][75950] Updated weights for policy 1, policy_version 62720 (0.0009) -[2023-10-14 16:03:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 128614400. Throughput: 0: 1686.0, 1: 1667.4. Samples: 32165978. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-10-14 16:03:13,164][74987] Avg episode reward: [(0, '28.050'), (1, '34.420')] -[2023-10-14 16:03:14,330][75949] Updated weights for policy 0, policy_version 62881 (0.0007) -[2023-10-14 16:03:14,539][75950] Updated weights for policy 1, policy_version 62730 (0.0009) -[2023-10-14 16:03:14,754][75949] Updated weights for policy 0, policy_version 62891 (0.0008) -[2023-10-14 16:03:14,904][75950] Updated weights for policy 1, policy_version 62740 (0.0008) -[2023-10-14 16:03:15,132][75949] Updated weights for policy 0, policy_version 62901 (0.0007) -[2023-10-14 16:03:15,273][75950] Updated weights for policy 1, policy_version 62750 (0.0007) -[2023-10-14 16:03:15,501][75949] Updated weights for policy 0, policy_version 62911 (0.0008) -[2023-10-14 16:03:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128679936. Throughput: 0: 1664.4, 1: 1655.2. Samples: 32174860. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) -[2023-10-14 16:03:18,164][74987] Avg episode reward: [(0, '26.640'), (1, '31.610')] -[2023-10-14 16:03:19,364][75950] Updated weights for policy 1, policy_version 62760 (0.0007) -[2023-10-14 16:03:19,562][75949] Updated weights for policy 0, policy_version 62921 (0.0008) -[2023-10-14 16:03:19,723][75950] Updated weights for policy 1, policy_version 62770 (0.0008) -[2023-10-14 16:03:19,927][75949] Updated weights for policy 0, policy_version 62931 (0.0007) -[2023-10-14 16:03:20,091][75950] Updated weights for policy 1, policy_version 62780 (0.0008) -[2023-10-14 16:03:20,298][75949] Updated weights for policy 0, policy_version 62941 (0.0008) -[2023-10-14 16:03:23,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128745472. Throughput: 0: 1680.4, 1: 1660.2. Samples: 32195376. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) -[2023-10-14 16:03:23,165][74987] Avg episode reward: [(0, '27.690'), (1, '32.750')] -[2023-10-14 16:03:24,253][75950] Updated weights for policy 1, policy_version 62790 (0.0007) -[2023-10-14 16:03:24,295][75949] Updated weights for policy 0, policy_version 62951 (0.0007) -[2023-10-14 16:03:24,619][75950] Updated weights for policy 1, policy_version 62800 (0.0008) -[2023-10-14 16:03:24,670][75949] Updated weights for policy 0, policy_version 62961 (0.0008) -[2023-10-14 16:03:24,979][75950] Updated weights for policy 1, policy_version 62810 (0.0008) -[2023-10-14 16:03:25,040][75949] Updated weights for policy 0, policy_version 62971 (0.0007) -[2023-10-14 16:03:28,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 128811008. Throughput: 0: 1678.8, 1: 1665.1. Samples: 32215978. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) -[2023-10-14 16:03:28,164][74987] Avg episode reward: [(0, '26.400'), (1, '34.190')] -[2023-10-14 16:03:29,011][75950] Updated weights for policy 1, policy_version 62820 (0.0008) -[2023-10-14 16:03:29,159][75949] Updated weights for policy 0, policy_version 62981 (0.0008) -[2023-10-14 16:03:29,382][75950] Updated weights for policy 1, policy_version 62830 (0.0007) -[2023-10-14 16:03:29,536][75949] Updated weights for policy 0, policy_version 62991 (0.0008) -[2023-10-14 16:03:29,738][75950] Updated weights for policy 1, policy_version 62840 (0.0008) -[2023-10-14 16:03:29,905][75949] Updated weights for policy 0, policy_version 63001 (0.0009) -[2023-10-14 16:03:33,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 128876544. Throughput: 0: 1668.3, 1: 1663.7. Samples: 32225182. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) -[2023-10-14 16:03:33,164][74987] Avg episode reward: [(0, '27.900'), (1, '32.470')] -[2023-10-14 16:03:33,696][75950] Updated weights for policy 1, policy_version 62850 (0.0007) -[2023-10-14 16:03:33,951][75949] Updated weights for policy 0, policy_version 63011 (0.0008) -[2023-10-14 16:03:34,064][75950] Updated weights for policy 1, policy_version 62860 (0.0010) -[2023-10-14 16:03:34,320][75949] Updated weights for policy 0, policy_version 63021 (0.0009) -[2023-10-14 16:03:34,424][75950] Updated weights for policy 1, policy_version 62870 (0.0008) -[2023-10-14 16:03:34,689][75949] Updated weights for policy 0, policy_version 63031 (0.0009) -[2023-10-14 16:03:34,791][75950] Updated weights for policy 1, policy_version 62880 (0.0008) -[2023-10-14 16:03:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128942080. Throughput: 0: 1678.0, 1: 1665.6. Samples: 32245780. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) -[2023-10-14 16:03:38,164][74987] Avg episode reward: [(0, '25.790'), (1, '31.440')] -[2023-10-14 16:03:38,741][75949] Updated weights for policy 0, policy_version 63041 (0.0009) -[2023-10-14 16:03:38,973][75950] Updated weights for policy 1, policy_version 62890 (0.0008) -[2023-10-14 16:03:39,114][75949] Updated weights for policy 0, policy_version 63051 (0.0008) -[2023-10-14 16:03:39,344][75950] Updated weights for policy 1, policy_version 62900 (0.0009) -[2023-10-14 16:03:39,481][75949] Updated weights for policy 0, policy_version 63061 (0.0009) -[2023-10-14 16:03:39,707][75950] Updated weights for policy 1, policy_version 62910 (0.0007) -[2023-10-14 16:03:39,852][75949] Updated weights for policy 0, policy_version 63071 (0.0009) -[2023-10-14 16:03:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129007616. Throughput: 0: 1672.1, 1: 1665.2. Samples: 32266266. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) -[2023-10-14 16:03:43,165][74987] Avg episode reward: [(0, '25.780'), (1, '32.360')] -[2023-10-14 16:03:43,983][75949] Updated weights for policy 0, policy_version 63081 (0.0008) -[2023-10-14 16:03:44,025][75950] Updated weights for policy 1, policy_version 62920 (0.0009) -[2023-10-14 16:03:44,343][75949] Updated weights for policy 0, policy_version 63091 (0.0007) -[2023-10-14 16:03:44,396][75950] Updated weights for policy 1, policy_version 62930 (0.0009) -[2023-10-14 16:03:44,718][75949] Updated weights for policy 0, policy_version 63101 (0.0007) -[2023-10-14 16:03:44,770][75950] Updated weights for policy 1, policy_version 62940 (0.0007) -[2023-10-14 16:03:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 129073152. Throughput: 0: 1670.2, 1: 1661.7. Samples: 32275096. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) -[2023-10-14 16:03:48,165][74987] Avg episode reward: [(0, '25.740'), (1, '35.510')] -[2023-10-14 16:03:48,683][75950] Updated weights for policy 1, policy_version 62950 (0.0008) -[2023-10-14 16:03:48,872][75949] Updated weights for policy 0, policy_version 63111 (0.0007) -[2023-10-14 16:03:49,048][75950] Updated weights for policy 1, policy_version 62960 (0.0008) -[2023-10-14 16:03:49,238][75949] Updated weights for policy 0, policy_version 63121 (0.0009) -[2023-10-14 16:03:49,408][75950] Updated weights for policy 1, policy_version 62970 (0.0009) -[2023-10-14 16:03:49,607][75949] Updated weights for policy 0, policy_version 63131 (0.0009) -[2023-10-14 16:03:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129138688. Throughput: 0: 1675.4, 1: 1667.7. Samples: 32295744. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) -[2023-10-14 16:03:53,164][74987] Avg episode reward: [(0, '26.900'), (1, '32.630')] -[2023-10-14 16:03:53,528][75950] Updated weights for policy 1, policy_version 62980 (0.0008) -[2023-10-14 16:03:53,734][75949] Updated weights for policy 0, policy_version 63141 (0.0008) -[2023-10-14 16:03:53,899][75950] Updated weights for policy 1, policy_version 62990 (0.0009) -[2023-10-14 16:03:54,098][75949] Updated weights for policy 0, policy_version 63151 (0.0009) -[2023-10-14 16:03:54,257][75950] Updated weights for policy 1, policy_version 63000 (0.0009) -[2023-10-14 16:03:54,466][75949] Updated weights for policy 0, policy_version 63161 (0.0007) -[2023-10-14 16:03:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129204224. Throughput: 0: 1675.3, 1: 1667.9. Samples: 32316424. Policy #0 lag: (min: 31.0, avg: 31.2, max: 42.0) -[2023-10-14 16:03:58,164][74987] Avg episode reward: [(0, '27.260'), (1, '33.030')] -[2023-10-14 16:03:58,458][75950] Updated weights for policy 1, policy_version 63010 (0.0009) -[2023-10-14 16:03:58,485][75949] Updated weights for policy 0, policy_version 63171 (0.0009) -[2023-10-14 16:03:58,828][75950] Updated weights for policy 1, policy_version 63020 (0.0009) -[2023-10-14 16:03:58,854][75949] Updated weights for policy 0, policy_version 63181 (0.0007) -[2023-10-14 16:03:59,194][75950] Updated weights for policy 1, policy_version 63030 (0.0008) -[2023-10-14 16:03:59,220][75949] Updated weights for policy 0, policy_version 63191 (0.0007) -[2023-10-14 16:03:59,558][75950] Updated weights for policy 1, policy_version 63040 (0.0008) -[2023-10-14 16:04:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129269760. Throughput: 0: 1673.1, 1: 1673.1. Samples: 32325436. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 16:04:03,164][74987] Avg episode reward: [(0, '27.510'), (1, '34.060')] -[2023-10-14 16:04:03,438][75949] Updated weights for policy 0, policy_version 63201 (0.0008) -[2023-10-14 16:04:03,637][75950] Updated weights for policy 1, policy_version 63050 (0.0009) -[2023-10-14 16:04:03,831][75949] Updated weights for policy 0, policy_version 63211 (0.0009) -[2023-10-14 16:04:03,993][75950] Updated weights for policy 1, policy_version 63060 (0.0009) -[2023-10-14 16:04:04,201][75949] Updated weights for policy 0, policy_version 63221 (0.0008) -[2023-10-14 16:04:04,356][75950] Updated weights for policy 1, policy_version 63070 (0.0009) -[2023-10-14 16:04:04,573][75949] Updated weights for policy 0, policy_version 63231 (0.0008) -[2023-10-14 16:04:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129335296. Throughput: 0: 1672.1, 1: 1676.3. Samples: 32346054. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 16:04:08,164][74987] Avg episode reward: [(0, '26.290'), (1, '34.500')] -[2023-10-14 16:04:08,408][75950] Updated weights for policy 1, policy_version 63080 (0.0010) -[2023-10-14 16:04:08,547][75949] Updated weights for policy 0, policy_version 63241 (0.0009) -[2023-10-14 16:04:08,782][75950] Updated weights for policy 1, policy_version 63090 (0.0009) -[2023-10-14 16:04:08,914][75949] Updated weights for policy 0, policy_version 63251 (0.0008) -[2023-10-14 16:04:09,145][75950] Updated weights for policy 1, policy_version 63100 (0.0010) -[2023-10-14 16:04:09,282][75949] Updated weights for policy 0, policy_version 63261 (0.0007) -[2023-10-14 16:04:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129400832. Throughput: 0: 1682.8, 1: 1678.0. Samples: 32367214. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 16:04:13,164][74987] Avg episode reward: [(0, '27.070'), (1, '31.820')] -[2023-10-14 16:04:13,176][75950] Updated weights for policy 1, policy_version 63110 (0.0007) -[2023-10-14 16:04:13,239][75949] Updated weights for policy 0, policy_version 63271 (0.0009) -[2023-10-14 16:04:13,552][75950] Updated weights for policy 1, policy_version 63120 (0.0008) -[2023-10-14 16:04:13,610][75949] Updated weights for policy 0, policy_version 63281 (0.0007) -[2023-10-14 16:04:13,923][75950] Updated weights for policy 1, policy_version 63130 (0.0009) -[2023-10-14 16:04:13,980][75949] Updated weights for policy 0, policy_version 63291 (0.0007) -[2023-10-14 16:04:18,031][75949] Updated weights for policy 0, policy_version 63301 (0.0008) -[2023-10-14 16:04:18,087][75950] Updated weights for policy 1, policy_version 63140 (0.0009) -[2023-10-14 16:04:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 129466368. Throughput: 0: 1684.3, 1: 1673.5. Samples: 32376282. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 16:04:18,165][74987] Avg episode reward: [(0, '25.450'), (1, '31.860')] -[2023-10-14 16:04:18,396][75949] Updated weights for policy 0, policy_version 63311 (0.0007) -[2023-10-14 16:04:18,459][75950] Updated weights for policy 1, policy_version 63150 (0.0007) -[2023-10-14 16:04:18,760][75949] Updated weights for policy 0, policy_version 63321 (0.0009) -[2023-10-14 16:04:18,819][75950] Updated weights for policy 1, policy_version 63160 (0.0007) -[2023-10-14 16:04:22,869][75950] Updated weights for policy 1, policy_version 63170 (0.0007) -[2023-10-14 16:04:22,949][75949] Updated weights for policy 0, policy_version 63331 (0.0008) -[2023-10-14 16:04:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129531904. Throughput: 0: 1683.9, 1: 1669.8. Samples: 32396694. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 16:04:23,164][74987] Avg episode reward: [(0, '25.350'), (1, '30.740')] -[2023-10-14 16:04:23,231][75950] Updated weights for policy 1, policy_version 63180 (0.0007) -[2023-10-14 16:04:23,314][75949] Updated weights for policy 0, policy_version 63341 (0.0007) -[2023-10-14 16:04:23,603][75950] Updated weights for policy 1, policy_version 63190 (0.0008) -[2023-10-14 16:04:23,687][75949] Updated weights for policy 0, policy_version 63351 (0.0007) -[2023-10-14 16:04:23,970][75950] Updated weights for policy 1, policy_version 63200 (0.0008) -[2023-10-14 16:04:27,771][75949] Updated weights for policy 0, policy_version 63361 (0.0009) -[2023-10-14 16:04:28,036][75950] Updated weights for policy 1, policy_version 63210 (0.0007) -[2023-10-14 16:04:28,135][75949] Updated weights for policy 0, policy_version 63371 (0.0007) -[2023-10-14 16:04:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129597440. Throughput: 0: 1683.2, 1: 1671.5. Samples: 32417224. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 16:04:28,164][74987] Avg episode reward: [(0, '27.020'), (1, '29.610')] -[2023-10-14 16:04:28,403][75950] Updated weights for policy 1, policy_version 63220 (0.0007) -[2023-10-14 16:04:28,503][75949] Updated weights for policy 0, policy_version 63381 (0.0007) -[2023-10-14 16:04:28,763][75950] Updated weights for policy 1, policy_version 63230 (0.0008) -[2023-10-14 16:04:28,870][75949] Updated weights for policy 0, policy_version 63391 (0.0007) -[2023-10-14 16:04:32,855][75949] Updated weights for policy 0, policy_version 63401 (0.0007) -[2023-10-14 16:04:32,924][75950] Updated weights for policy 1, policy_version 63240 (0.0009) -[2023-10-14 16:04:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129662976. Throughput: 0: 1684.5, 1: 1679.8. Samples: 32426490. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 16:04:33,164][74987] Avg episode reward: [(0, '26.170'), (1, '29.890')] -[2023-10-14 16:04:33,219][75949] Updated weights for policy 0, policy_version 63411 (0.0008) -[2023-10-14 16:04:33,294][75950] Updated weights for policy 1, policy_version 63250 (0.0007) -[2023-10-14 16:04:33,589][75949] Updated weights for policy 0, policy_version 63421 (0.0008) -[2023-10-14 16:04:33,655][75950] Updated weights for policy 1, policy_version 63260 (0.0009) -[2023-10-14 16:04:37,661][75949] Updated weights for policy 0, policy_version 63431 (0.0008) -[2023-10-14 16:04:37,696][75950] Updated weights for policy 1, policy_version 63270 (0.0008) -[2023-10-14 16:04:38,021][75949] Updated weights for policy 0, policy_version 63441 (0.0007) -[2023-10-14 16:04:38,065][75950] Updated weights for policy 1, policy_version 63280 (0.0007) -[2023-10-14 16:04:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129728512. Throughput: 0: 1689.6, 1: 1671.2. Samples: 32446982. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 16:04:38,164][74987] Avg episode reward: [(0, '26.610'), (1, '32.460')] -[2023-10-14 16:04:38,399][75949] Updated weights for policy 0, policy_version 63451 (0.0008) -[2023-10-14 16:04:38,439][75950] Updated weights for policy 1, policy_version 63290 (0.0007) -[2023-10-14 16:04:42,412][75949] Updated weights for policy 0, policy_version 63461 (0.0009) -[2023-10-14 16:04:42,460][75950] Updated weights for policy 1, policy_version 63300 (0.0009) -[2023-10-14 16:04:42,782][75949] Updated weights for policy 0, policy_version 63471 (0.0009) -[2023-10-14 16:04:42,826][75950] Updated weights for policy 1, policy_version 63310 (0.0009) -[2023-10-14 16:04:43,152][75949] Updated weights for policy 0, policy_version 63481 (0.0007) -[2023-10-14 16:04:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129794048. Throughput: 0: 1676.2, 1: 1667.0. Samples: 32466866. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-10-14 16:04:43,165][74987] Avg episode reward: [(0, '26.340'), (1, '34.030')] -[2023-10-14 16:04:43,194][75950] Updated weights for policy 1, policy_version 63320 (0.0008) -[2023-10-14 16:04:43,412][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000063488_65011712.pth... -[2023-10-14 16:04:43,449][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000061920_63406080.pth -[2023-10-14 16:04:43,485][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000063328_64847872.pth... -[2023-10-14 16:04:43,514][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000061760_63242240.pth -[2023-10-14 16:04:47,288][75949] Updated weights for policy 0, policy_version 63491 (0.0008) -[2023-10-14 16:04:47,379][75950] Updated weights for policy 1, policy_version 63330 (0.0008) -[2023-10-14 16:04:47,655][75949] Updated weights for policy 0, policy_version 63501 (0.0007) -[2023-10-14 16:04:47,741][75950] Updated weights for policy 1, policy_version 63340 (0.0009) -[2023-10-14 16:04:48,021][75949] Updated weights for policy 0, policy_version 63511 (0.0008) -[2023-10-14 16:04:48,101][75950] Updated weights for policy 1, policy_version 63350 (0.0007) -[2023-10-14 16:04:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129859584. Throughput: 0: 1688.1, 1: 1673.0. Samples: 32476688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:04:48,165][74987] Avg episode reward: [(0, '26.040'), (1, '35.490')] -[2023-10-14 16:04:48,469][75950] Updated weights for policy 1, policy_version 63360 (0.0008) -[2023-10-14 16:04:52,180][75949] Updated weights for policy 0, policy_version 63521 (0.0007) -[2023-10-14 16:04:52,580][75949] Updated weights for policy 0, policy_version 63531 (0.0007) -[2023-10-14 16:04:52,646][75950] Updated weights for policy 1, policy_version 63370 (0.0009) -[2023-10-14 16:04:52,954][75949] Updated weights for policy 0, policy_version 63541 (0.0007) -[2023-10-14 16:04:53,018][75950] Updated weights for policy 1, policy_version 63380 (0.0008) -[2023-10-14 16:04:53,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129925120. Throughput: 0: 1686.6, 1: 1669.5. Samples: 32497080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:04:53,164][74987] Avg episode reward: [(0, '26.170'), (1, '32.840')] -[2023-10-14 16:04:53,323][75949] Updated weights for policy 0, policy_version 63551 (0.0007) -[2023-10-14 16:04:53,382][75950] Updated weights for policy 1, policy_version 63390 (0.0009) -[2023-10-14 16:04:57,330][75949] Updated weights for policy 0, policy_version 63561 (0.0009) -[2023-10-14 16:04:57,535][75950] Updated weights for policy 1, policy_version 63400 (0.0010) -[2023-10-14 16:04:57,702][75949] Updated weights for policy 0, policy_version 63571 (0.0009) -[2023-10-14 16:04:57,900][75950] Updated weights for policy 1, policy_version 63410 (0.0008) -[2023-10-14 16:04:58,069][75949] Updated weights for policy 0, policy_version 63581 (0.0008) -[2023-10-14 16:04:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129990656. Throughput: 0: 1661.4, 1: 1656.5. Samples: 32516520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:04:58,165][74987] Avg episode reward: [(0, '30.440'), (1, '34.820')] -[2023-10-14 16:04:58,175][75615] Saving new best policy, reward=30.440! -[2023-10-14 16:04:58,259][75950] Updated weights for policy 1, policy_version 63420 (0.0010) -[2023-10-14 16:05:02,171][75949] Updated weights for policy 0, policy_version 63591 (0.0008) -[2023-10-14 16:05:02,378][75950] Updated weights for policy 1, policy_version 63430 (0.0009) -[2023-10-14 16:05:02,532][75949] Updated weights for policy 0, policy_version 63601 (0.0007) -[2023-10-14 16:05:02,745][75950] Updated weights for policy 1, policy_version 63440 (0.0008) -[2023-10-14 16:05:02,904][75949] Updated weights for policy 0, policy_version 63611 (0.0007) -[2023-10-14 16:05:03,116][75950] Updated weights for policy 1, policy_version 63450 (0.0007) -[2023-10-14 16:05:03,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 130088960. Throughput: 0: 1675.5, 1: 1666.5. Samples: 32526674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:05:03,165][74987] Avg episode reward: [(0, '28.070'), (1, '33.630')] -[2023-10-14 16:05:06,926][75949] Updated weights for policy 0, policy_version 63621 (0.0007) -[2023-10-14 16:05:07,299][75949] Updated weights for policy 0, policy_version 63631 (0.0007) -[2023-10-14 16:05:07,471][75950] Updated weights for policy 1, policy_version 63460 (0.0008) -[2023-10-14 16:05:07,665][75949] Updated weights for policy 0, policy_version 63641 (0.0009) -[2023-10-14 16:05:07,837][75950] Updated weights for policy 1, policy_version 63470 (0.0008) -[2023-10-14 16:05:08,163][74987] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 130154496. Throughput: 0: 1679.9, 1: 1668.8. Samples: 32547386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:05:08,164][74987] Avg episode reward: [(0, '28.460'), (1, '32.160')] -[2023-10-14 16:05:08,196][75950] Updated weights for policy 1, policy_version 63480 (0.0010) -[2023-10-14 16:05:11,686][75949] Updated weights for policy 0, policy_version 63651 (0.0008) -[2023-10-14 16:05:12,049][75949] Updated weights for policy 0, policy_version 63661 (0.0008) -[2023-10-14 16:05:12,394][75950] Updated weights for policy 1, policy_version 63490 (0.0010) -[2023-10-14 16:05:12,427][75949] Updated weights for policy 0, policy_version 63671 (0.0009) -[2023-10-14 16:05:12,756][75950] Updated weights for policy 1, policy_version 63500 (0.0009) -[2023-10-14 16:05:13,116][75950] Updated weights for policy 1, policy_version 63510 (0.0011) -[2023-10-14 16:05:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 130220032. Throughput: 0: 1656.0, 1: 1655.0. Samples: 32566220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:05:13,164][74987] Avg episode reward: [(0, '26.870'), (1, '32.470')] -[2023-10-14 16:05:13,482][75950] Updated weights for policy 1, policy_version 63520 (0.0009) -[2023-10-14 16:05:16,525][75949] Updated weights for policy 0, policy_version 63681 (0.0008) -[2023-10-14 16:05:16,907][75949] Updated weights for policy 0, policy_version 63691 (0.0007) -[2023-10-14 16:05:17,273][75949] Updated weights for policy 0, policy_version 63701 (0.0008) -[2023-10-14 16:05:17,650][75949] Updated weights for policy 0, policy_version 63711 (0.0007) -[2023-10-14 16:05:17,841][75950] Updated weights for policy 1, policy_version 63530 (0.0009) -[2023-10-14 16:05:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 130285568. Throughput: 0: 1678.0, 1: 1656.0. Samples: 32576518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:05:18,164][74987] Avg episode reward: [(0, '26.970'), (1, '34.080')] -[2023-10-14 16:05:18,211][75950] Updated weights for policy 1, policy_version 63540 (0.0007) -[2023-10-14 16:05:18,579][75950] Updated weights for policy 1, policy_version 63550 (0.0007) -[2023-10-14 16:05:21,848][75949] Updated weights for policy 0, policy_version 63721 (0.0008) -[2023-10-14 16:05:22,216][75949] Updated weights for policy 0, policy_version 63731 (0.0007) -[2023-10-14 16:05:22,539][75950] Updated weights for policy 1, policy_version 63560 (0.0008) -[2023-10-14 16:05:22,584][75949] Updated weights for policy 0, policy_version 63741 (0.0008) -[2023-10-14 16:05:22,908][75950] Updated weights for policy 1, policy_version 63570 (0.0008) -[2023-10-14 16:05:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 130351104. Throughput: 0: 1673.1, 1: 1654.4. Samples: 32596718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:05:23,164][74987] Avg episode reward: [(0, '25.390'), (1, '33.650')] -[2023-10-14 16:05:23,267][75950] Updated weights for policy 1, policy_version 63580 (0.0011) -[2023-10-14 16:05:26,958][75949] Updated weights for policy 0, policy_version 63751 (0.0009) -[2023-10-14 16:05:27,326][75949] Updated weights for policy 0, policy_version 63761 (0.0007) -[2023-10-14 16:05:27,360][75950] Updated weights for policy 1, policy_version 63590 (0.0008) -[2023-10-14 16:05:27,691][75949] Updated weights for policy 0, policy_version 63771 (0.0007) -[2023-10-14 16:05:27,720][75950] Updated weights for policy 1, policy_version 63600 (0.0008) -[2023-10-14 16:05:28,081][75950] Updated weights for policy 1, policy_version 63610 (0.0010) -[2023-10-14 16:05:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 130416640. Throughput: 0: 1661.2, 1: 1649.0. Samples: 32615822. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 16:05:28,165][74987] Avg episode reward: [(0, '27.780'), (1, '33.170')] -[2023-10-14 16:05:31,809][75949] Updated weights for policy 0, policy_version 63781 (0.0008) -[2023-10-14 16:05:32,181][75949] Updated weights for policy 0, policy_version 63791 (0.0010) -[2023-10-14 16:05:32,273][75950] Updated weights for policy 1, policy_version 63620 (0.0008) -[2023-10-14 16:05:32,543][75949] Updated weights for policy 0, policy_version 63801 (0.0008) -[2023-10-14 16:05:32,643][75950] Updated weights for policy 1, policy_version 63630 (0.0008) -[2023-10-14 16:05:33,009][75950] Updated weights for policy 1, policy_version 63640 (0.0010) -[2023-10-14 16:05:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 130482176. Throughput: 0: 1672.3, 1: 1654.7. Samples: 32626402. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 16:05:33,164][74987] Avg episode reward: [(0, '25.250'), (1, '32.490')] -[2023-10-14 16:05:36,617][75949] Updated weights for policy 0, policy_version 63811 (0.0009) -[2023-10-14 16:05:36,992][75949] Updated weights for policy 0, policy_version 63821 (0.0009) -[2023-10-14 16:05:37,110][75950] Updated weights for policy 1, policy_version 63650 (0.0010) -[2023-10-14 16:05:37,368][75949] Updated weights for policy 0, policy_version 63831 (0.0010) -[2023-10-14 16:05:37,478][75950] Updated weights for policy 1, policy_version 63660 (0.0009) -[2023-10-14 16:05:37,843][75950] Updated weights for policy 1, policy_version 63670 (0.0009) -[2023-10-14 16:05:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 130547712. Throughput: 0: 1665.0, 1: 1658.4. Samples: 32646632. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 16:05:38,165][74987] Avg episode reward: [(0, '28.490'), (1, '32.790')] -[2023-10-14 16:05:38,206][75950] Updated weights for policy 1, policy_version 63680 (0.0007) -[2023-10-14 16:05:41,574][75949] Updated weights for policy 0, policy_version 63841 (0.0008) -[2023-10-14 16:05:41,982][75949] Updated weights for policy 0, policy_version 63851 (0.0008) -[2023-10-14 16:05:42,322][75950] Updated weights for policy 1, policy_version 63690 (0.0007) -[2023-10-14 16:05:42,342][75949] Updated weights for policy 0, policy_version 63861 (0.0008) -[2023-10-14 16:05:42,689][75950] Updated weights for policy 1, policy_version 63700 (0.0009) -[2023-10-14 16:05:42,703][75949] Updated weights for policy 0, policy_version 63871 (0.0007) -[2023-10-14 16:05:43,050][75950] Updated weights for policy 1, policy_version 63710 (0.0009) -[2023-10-14 16:05:43,164][74987] Fps is (10 sec: 16383.3, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 130646016. Throughput: 0: 1652.0, 1: 1657.2. Samples: 32665436. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 16:05:43,165][74987] Avg episode reward: [(0, '25.950'), (1, '34.200')] -[2023-10-14 16:05:46,798][75949] Updated weights for policy 0, policy_version 63881 (0.0009) -[2023-10-14 16:05:47,057][75950] Updated weights for policy 1, policy_version 63720 (0.0008) -[2023-10-14 16:05:47,163][75949] Updated weights for policy 0, policy_version 63891 (0.0008) -[2023-10-14 16:05:47,427][75950] Updated weights for policy 1, policy_version 63730 (0.0008) -[2023-10-14 16:05:47,535][75949] Updated weights for policy 0, policy_version 63901 (0.0009) -[2023-10-14 16:05:47,802][75950] Updated weights for policy 1, policy_version 63740 (0.0009) -[2023-10-14 16:05:48,164][74987] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 130711552. Throughput: 0: 1664.8, 1: 1667.0. Samples: 32676602. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 16:05:48,164][74987] Avg episode reward: [(0, '28.000'), (1, '32.380')] -[2023-10-14 16:05:51,740][75949] Updated weights for policy 0, policy_version 63911 (0.0009) -[2023-10-14 16:05:51,860][75950] Updated weights for policy 1, policy_version 63750 (0.0009) -[2023-10-14 16:05:52,108][75949] Updated weights for policy 0, policy_version 63921 (0.0009) -[2023-10-14 16:05:52,214][75950] Updated weights for policy 1, policy_version 63760 (0.0008) -[2023-10-14 16:05:52,469][75949] Updated weights for policy 0, policy_version 63931 (0.0007) -[2023-10-14 16:05:52,589][75950] Updated weights for policy 1, policy_version 63770 (0.0008) -[2023-10-14 16:05:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 130777088. Throughput: 0: 1651.5, 1: 1670.1. Samples: 32696860. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 16:05:53,164][74987] Avg episode reward: [(0, '25.730'), (1, '33.560')] -[2023-10-14 16:05:56,572][75949] Updated weights for policy 0, policy_version 63941 (0.0008) -[2023-10-14 16:05:56,728][75950] Updated weights for policy 1, policy_version 63780 (0.0009) -[2023-10-14 16:05:56,943][75949] Updated weights for policy 0, policy_version 63951 (0.0007) -[2023-10-14 16:05:57,083][75950] Updated weights for policy 1, policy_version 63790 (0.0008) -[2023-10-14 16:05:57,304][75949] Updated weights for policy 0, policy_version 63961 (0.0009) -[2023-10-14 16:05:57,449][75950] Updated weights for policy 1, policy_version 63800 (0.0007) -[2023-10-14 16:05:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 130842624. Throughput: 0: 1653.8, 1: 1658.8. Samples: 32715290. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 16:05:58,164][74987] Avg episode reward: [(0, '27.150'), (1, '34.730')] -[2023-10-14 16:06:01,366][75950] Updated weights for policy 1, policy_version 63810 (0.0007) -[2023-10-14 16:06:01,470][75949] Updated weights for policy 0, policy_version 63971 (0.0008) -[2023-10-14 16:06:01,733][75950] Updated weights for policy 1, policy_version 63820 (0.0009) -[2023-10-14 16:06:01,842][75949] Updated weights for policy 0, policy_version 63981 (0.0008) -[2023-10-14 16:06:02,097][75950] Updated weights for policy 1, policy_version 63830 (0.0007) -[2023-10-14 16:06:02,213][75949] Updated weights for policy 0, policy_version 63991 (0.0008) -[2023-10-14 16:06:02,458][75950] Updated weights for policy 1, policy_version 63840 (0.0008) -[2023-10-14 16:06:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 130908160. Throughput: 0: 1658.5, 1: 1685.0. Samples: 32726978. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 16:06:03,165][74987] Avg episode reward: [(0, '25.980'), (1, '31.530')] -[2023-10-14 16:06:06,315][75949] Updated weights for policy 0, policy_version 64001 (0.0009) -[2023-10-14 16:06:06,433][75950] Updated weights for policy 1, policy_version 63850 (0.0007) -[2023-10-14 16:06:06,691][75949] Updated weights for policy 0, policy_version 64011 (0.0008) -[2023-10-14 16:06:06,793][75950] Updated weights for policy 1, policy_version 63860 (0.0007) -[2023-10-14 16:06:07,055][75949] Updated weights for policy 0, policy_version 64021 (0.0008) -[2023-10-14 16:06:07,157][75950] Updated weights for policy 1, policy_version 63870 (0.0009) -[2023-10-14 16:06:07,425][75949] Updated weights for policy 0, policy_version 64031 (0.0010) -[2023-10-14 16:06:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 130973696. Throughput: 0: 1651.5, 1: 1679.2. Samples: 32746598. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) -[2023-10-14 16:06:08,164][74987] Avg episode reward: [(0, '25.990'), (1, '32.970')] -[2023-10-14 16:06:11,280][75950] Updated weights for policy 1, policy_version 63880 (0.0009) -[2023-10-14 16:06:11,567][75949] Updated weights for policy 0, policy_version 64041 (0.0008) -[2023-10-14 16:06:11,638][75950] Updated weights for policy 1, policy_version 63890 (0.0008) -[2023-10-14 16:06:11,950][75949] Updated weights for policy 0, policy_version 64051 (0.0008) -[2023-10-14 16:06:12,009][75950] Updated weights for policy 1, policy_version 63900 (0.0008) -[2023-10-14 16:06:12,324][75949] Updated weights for policy 0, policy_version 64061 (0.0010) -[2023-10-14 16:06:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 131039232. Throughput: 0: 1649.8, 1: 1676.1. Samples: 32765486. Policy #0 lag: (min: 43.0, avg: 47.9, max: 48.0) -[2023-10-14 16:06:13,165][74987] Avg episode reward: [(0, '27.760'), (1, '32.720')] -[2023-10-14 16:06:16,066][75950] Updated weights for policy 1, policy_version 63910 (0.0007) -[2023-10-14 16:06:16,392][75949] Updated weights for policy 0, policy_version 64071 (0.0010) -[2023-10-14 16:06:16,434][75950] Updated weights for policy 1, policy_version 63920 (0.0009) -[2023-10-14 16:06:16,760][75949] Updated weights for policy 0, policy_version 64081 (0.0009) -[2023-10-14 16:06:16,795][75950] Updated weights for policy 1, policy_version 63930 (0.0008) -[2023-10-14 16:06:17,145][75949] Updated weights for policy 0, policy_version 64091 (0.0008) -[2023-10-14 16:06:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 131104768. Throughput: 0: 1659.6, 1: 1689.9. Samples: 32777130. Policy #0 lag: (min: 43.0, avg: 47.9, max: 48.0) -[2023-10-14 16:06:18,164][74987] Avg episode reward: [(0, '25.950'), (1, '31.080')] -[2023-10-14 16:06:20,910][75950] Updated weights for policy 1, policy_version 63940 (0.0007) -[2023-10-14 16:06:21,270][75949] Updated weights for policy 0, policy_version 64101 (0.0009) -[2023-10-14 16:06:21,282][75950] Updated weights for policy 1, policy_version 63950 (0.0009) -[2023-10-14 16:06:21,635][75949] Updated weights for policy 0, policy_version 64111 (0.0008) -[2023-10-14 16:06:21,640][75950] Updated weights for policy 1, policy_version 63960 (0.0009) -[2023-10-14 16:06:22,004][75949] Updated weights for policy 0, policy_version 64121 (0.0010) -[2023-10-14 16:06:23,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 131170304. Throughput: 0: 1655.2, 1: 1671.0. Samples: 32796310. Policy #0 lag: (min: 43.0, avg: 47.9, max: 48.0) -[2023-10-14 16:06:23,164][74987] Avg episode reward: [(0, '29.220'), (1, '32.100')] -[2023-10-14 16:06:25,618][75950] Updated weights for policy 1, policy_version 63970 (0.0007) -[2023-10-14 16:06:25,986][75950] Updated weights for policy 1, policy_version 63980 (0.0009) -[2023-10-14 16:06:26,140][75949] Updated weights for policy 0, policy_version 64131 (0.0008) -[2023-10-14 16:06:26,354][75950] Updated weights for policy 1, policy_version 63990 (0.0008) -[2023-10-14 16:06:26,547][75949] Updated weights for policy 0, policy_version 64141 (0.0009) -[2023-10-14 16:06:26,719][75950] Updated weights for policy 1, policy_version 64000 (0.0009) -[2023-10-14 16:06:26,908][75949] Updated weights for policy 0, policy_version 64151 (0.0009) -[2023-10-14 16:06:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 131235840. Throughput: 0: 1664.0, 1: 1678.1. Samples: 32815826. Policy #0 lag: (min: 43.0, avg: 47.9, max: 48.0) -[2023-10-14 16:06:28,165][74987] Avg episode reward: [(0, '25.980'), (1, '34.030')] -[2023-10-14 16:06:30,789][75950] Updated weights for policy 1, policy_version 64010 (0.0008) -[2023-10-14 16:06:30,907][75949] Updated weights for policy 0, policy_version 64161 (0.0009) -[2023-10-14 16:06:31,149][75950] Updated weights for policy 1, policy_version 64020 (0.0008) -[2023-10-14 16:06:31,280][75949] Updated weights for policy 0, policy_version 64171 (0.0008) -[2023-10-14 16:06:31,519][75950] Updated weights for policy 1, policy_version 64030 (0.0008) -[2023-10-14 16:06:31,640][75949] Updated weights for policy 0, policy_version 64181 (0.0008) -[2023-10-14 16:06:32,012][75949] Updated weights for policy 0, policy_version 64191 (0.0008) -[2023-10-14 16:06:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 131301376. Throughput: 0: 1666.8, 1: 1680.0. Samples: 32827208. Policy #0 lag: (min: 43.0, avg: 47.9, max: 48.0) -[2023-10-14 16:06:33,164][74987] Avg episode reward: [(0, '30.560'), (1, '33.690')] -[2023-10-14 16:06:33,165][75615] Saving new best policy, reward=30.560! -[2023-10-14 16:06:35,706][75950] Updated weights for policy 1, policy_version 64040 (0.0009) -[2023-10-14 16:06:35,942][75949] Updated weights for policy 0, policy_version 64201 (0.0008) -[2023-10-14 16:06:36,068][75950] Updated weights for policy 1, policy_version 64050 (0.0010) -[2023-10-14 16:06:36,302][75949] Updated weights for policy 0, policy_version 64211 (0.0008) -[2023-10-14 16:06:36,435][75950] Updated weights for policy 1, policy_version 64060 (0.0009) -[2023-10-14 16:06:36,669][75949] Updated weights for policy 0, policy_version 64221 (0.0009) -[2023-10-14 16:06:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 131366912. Throughput: 0: 1658.4, 1: 1653.5. Samples: 32845898. Policy #0 lag: (min: 43.0, avg: 47.9, max: 48.0) -[2023-10-14 16:06:38,165][74987] Avg episode reward: [(0, '25.630'), (1, '33.020')] -[2023-10-14 16:06:40,691][75950] Updated weights for policy 1, policy_version 64070 (0.0009) -[2023-10-14 16:06:40,811][75949] Updated weights for policy 0, policy_version 64231 (0.0008) -[2023-10-14 16:06:41,056][75950] Updated weights for policy 1, policy_version 64080 (0.0009) -[2023-10-14 16:06:41,185][75949] Updated weights for policy 0, policy_version 64241 (0.0008) -[2023-10-14 16:06:41,421][75950] Updated weights for policy 1, policy_version 64090 (0.0008) -[2023-10-14 16:06:41,558][75949] Updated weights for policy 0, policy_version 64251 (0.0008) -[2023-10-14 16:06:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 131432448. Throughput: 0: 1673.1, 1: 1677.8. Samples: 32866080. Policy #0 lag: (min: 43.0, avg: 47.9, max: 48.0) -[2023-10-14 16:06:43,165][74987] Avg episode reward: [(0, '31.140'), (1, '33.150')] -[2023-10-14 16:06:43,177][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000064096_65634304.pth... -[2023-10-14 16:06:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000064256_65798144.pth... -[2023-10-14 16:06:43,211][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000062688_64192512.pth -[2023-10-14 16:06:43,214][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000062528_64028672.pth -[2023-10-14 16:06:43,215][75615] Saving new best policy, reward=31.140! -[2023-10-14 16:06:45,529][75950] Updated weights for policy 1, policy_version 64100 (0.0008) -[2023-10-14 16:06:45,534][75949] Updated weights for policy 0, policy_version 64261 (0.0007) -[2023-10-14 16:06:45,898][75949] Updated weights for policy 0, policy_version 64271 (0.0008) -[2023-10-14 16:06:45,903][75950] Updated weights for policy 1, policy_version 64110 (0.0008) -[2023-10-14 16:06:46,269][75949] Updated weights for policy 0, policy_version 64281 (0.0009) -[2023-10-14 16:06:46,271][75950] Updated weights for policy 1, policy_version 64120 (0.0007) -[2023-10-14 16:06:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 131497984. Throughput: 0: 1667.3, 1: 1667.0. Samples: 32877020. Policy #0 lag: (min: 43.0, avg: 47.9, max: 48.0) -[2023-10-14 16:06:48,164][74987] Avg episode reward: [(0, '24.580'), (1, '31.900')] -[2023-10-14 16:06:50,340][75949] Updated weights for policy 0, policy_version 64291 (0.0009) -[2023-10-14 16:06:50,437][75950] Updated weights for policy 1, policy_version 64130 (0.0007) -[2023-10-14 16:06:50,698][75949] Updated weights for policy 0, policy_version 64301 (0.0007) -[2023-10-14 16:06:50,811][75950] Updated weights for policy 1, policy_version 64140 (0.0008) -[2023-10-14 16:06:51,072][75949] Updated weights for policy 0, policy_version 64311 (0.0008) -[2023-10-14 16:06:51,171][75950] Updated weights for policy 1, policy_version 64150 (0.0008) -[2023-10-14 16:06:51,531][75950] Updated weights for policy 1, policy_version 64160 (0.0009) -[2023-10-14 16:06:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 131563520. Throughput: 0: 1654.5, 1: 1656.0. Samples: 32895570. Policy #0 lag: (min: 43.0, avg: 47.9, max: 48.0) -[2023-10-14 16:06:53,165][74987] Avg episode reward: [(0, '27.990'), (1, '30.410')] -[2023-10-14 16:06:55,211][75949] Updated weights for policy 0, policy_version 64321 (0.0009) -[2023-10-14 16:06:55,579][75949] Updated weights for policy 0, policy_version 64331 (0.0008) -[2023-10-14 16:06:55,626][75950] Updated weights for policy 1, policy_version 64170 (0.0007) -[2023-10-14 16:06:55,942][75949] Updated weights for policy 0, policy_version 64341 (0.0007) -[2023-10-14 16:06:55,984][75950] Updated weights for policy 1, policy_version 64180 (0.0010) -[2023-10-14 16:06:56,310][75949] Updated weights for policy 0, policy_version 64351 (0.0009) -[2023-10-14 16:06:56,346][75950] Updated weights for policy 1, policy_version 64190 (0.0009) -[2023-10-14 16:06:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 131629056. Throughput: 0: 1682.2, 1: 1666.6. Samples: 32916184. Policy #0 lag: (min: 17.0, avg: 28.2, max: 49.0) -[2023-10-14 16:06:58,165][74987] Avg episode reward: [(0, '23.950'), (1, '30.390')] -[2023-10-14 16:07:00,258][75949] Updated weights for policy 0, policy_version 64361 (0.0008) -[2023-10-14 16:07:00,384][75950] Updated weights for policy 1, policy_version 64200 (0.0008) -[2023-10-14 16:07:00,639][75949] Updated weights for policy 0, policy_version 64371 (0.0008) -[2023-10-14 16:07:00,753][75950] Updated weights for policy 1, policy_version 64210 (0.0009) -[2023-10-14 16:07:01,004][75949] Updated weights for policy 0, policy_version 64381 (0.0009) -[2023-10-14 16:07:01,113][75950] Updated weights for policy 1, policy_version 64220 (0.0008) -[2023-10-14 16:07:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 131694592. Throughput: 0: 1665.7, 1: 1653.6. Samples: 32926502. Policy #0 lag: (min: 17.0, avg: 28.2, max: 49.0) -[2023-10-14 16:07:03,164][74987] Avg episode reward: [(0, '30.280'), (1, '33.430')] -[2023-10-14 16:07:05,045][75949] Updated weights for policy 0, policy_version 64391 (0.0008) -[2023-10-14 16:07:05,102][75950] Updated weights for policy 1, policy_version 64230 (0.0007) -[2023-10-14 16:07:05,418][75949] Updated weights for policy 0, policy_version 64401 (0.0007) -[2023-10-14 16:07:05,467][75950] Updated weights for policy 1, policy_version 64240 (0.0007) -[2023-10-14 16:07:05,787][75949] Updated weights for policy 0, policy_version 64411 (0.0008) -[2023-10-14 16:07:05,840][75950] Updated weights for policy 1, policy_version 64250 (0.0007) -[2023-10-14 16:07:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 131760128. Throughput: 0: 1670.7, 1: 1660.0. Samples: 32946196. Policy #0 lag: (min: 17.0, avg: 28.2, max: 49.0) -[2023-10-14 16:07:08,165][74987] Avg episode reward: [(0, '24.720'), (1, '31.990')] -[2023-10-14 16:07:09,902][75949] Updated weights for policy 0, policy_version 64421 (0.0008) -[2023-10-14 16:07:09,920][75950] Updated weights for policy 1, policy_version 64260 (0.0008) -[2023-10-14 16:07:10,270][75949] Updated weights for policy 0, policy_version 64431 (0.0009) -[2023-10-14 16:07:10,280][75950] Updated weights for policy 1, policy_version 64270 (0.0008) -[2023-10-14 16:07:10,635][75949] Updated weights for policy 0, policy_version 64441 (0.0007) -[2023-10-14 16:07:10,644][75950] Updated weights for policy 1, policy_version 64280 (0.0007) -[2023-10-14 16:07:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 131825664. Throughput: 0: 1691.1, 1: 1666.8. Samples: 32966932. Policy #0 lag: (min: 17.0, avg: 28.2, max: 49.0) -[2023-10-14 16:07:13,165][74987] Avg episode reward: [(0, '27.120'), (1, '32.940')] -[2023-10-14 16:07:14,755][75949] Updated weights for policy 0, policy_version 64451 (0.0009) -[2023-10-14 16:07:14,931][75950] Updated weights for policy 1, policy_version 64290 (0.0008) -[2023-10-14 16:07:15,147][75949] Updated weights for policy 0, policy_version 64461 (0.0008) -[2023-10-14 16:07:15,294][75950] Updated weights for policy 1, policy_version 64300 (0.0008) -[2023-10-14 16:07:15,525][75949] Updated weights for policy 0, policy_version 64471 (0.0009) -[2023-10-14 16:07:15,656][75950] Updated weights for policy 1, policy_version 64310 (0.0010) -[2023-10-14 16:07:16,025][75950] Updated weights for policy 1, policy_version 64320 (0.0008) -[2023-10-14 16:07:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 131891200. Throughput: 0: 1665.5, 1: 1653.4. Samples: 32976558. Policy #0 lag: (min: 17.0, avg: 28.2, max: 49.0) -[2023-10-14 16:07:18,164][74987] Avg episode reward: [(0, '25.190'), (1, '35.660')] -[2023-10-14 16:07:19,486][75949] Updated weights for policy 0, policy_version 64481 (0.0007) -[2023-10-14 16:07:19,855][75949] Updated weights for policy 0, policy_version 64491 (0.0009) -[2023-10-14 16:07:20,215][75949] Updated weights for policy 0, policy_version 64501 (0.0007) -[2023-10-14 16:07:20,248][75950] Updated weights for policy 1, policy_version 64330 (0.0007) -[2023-10-14 16:07:20,586][75949] Updated weights for policy 0, policy_version 64511 (0.0009) -[2023-10-14 16:07:20,626][75950] Updated weights for policy 1, policy_version 64340 (0.0009) -[2023-10-14 16:07:20,988][75950] Updated weights for policy 1, policy_version 64350 (0.0009) -[2023-10-14 16:07:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 131956736. Throughput: 0: 1681.3, 1: 1663.8. Samples: 32996428. Policy #0 lag: (min: 17.0, avg: 28.2, max: 49.0) -[2023-10-14 16:07:23,165][74987] Avg episode reward: [(0, '26.640'), (1, '32.070')] -[2023-10-14 16:07:24,572][75949] Updated weights for policy 0, policy_version 64521 (0.0009) -[2023-10-14 16:07:24,868][75950] Updated weights for policy 1, policy_version 64360 (0.0009) -[2023-10-14 16:07:24,946][75949] Updated weights for policy 0, policy_version 64531 (0.0009) -[2023-10-14 16:07:25,231][75950] Updated weights for policy 1, policy_version 64370 (0.0008) -[2023-10-14 16:07:25,329][75949] Updated weights for policy 0, policy_version 64541 (0.0010) -[2023-10-14 16:07:25,604][75950] Updated weights for policy 1, policy_version 64380 (0.0007) -[2023-10-14 16:07:28,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132022272. Throughput: 0: 1690.3, 1: 1665.9. Samples: 33017106. Policy #0 lag: (min: 17.0, avg: 28.2, max: 49.0) -[2023-10-14 16:07:28,165][74987] Avg episode reward: [(0, '26.530'), (1, '30.970')] -[2023-10-14 16:07:29,597][75949] Updated weights for policy 0, policy_version 64551 (0.0009) -[2023-10-14 16:07:29,764][75950] Updated weights for policy 1, policy_version 64390 (0.0007) -[2023-10-14 16:07:29,961][75949] Updated weights for policy 0, policy_version 64561 (0.0008) -[2023-10-14 16:07:30,124][75950] Updated weights for policy 1, policy_version 64400 (0.0007) -[2023-10-14 16:07:30,328][75949] Updated weights for policy 0, policy_version 64571 (0.0007) -[2023-10-14 16:07:30,487][75950] Updated weights for policy 1, policy_version 64410 (0.0008) -[2023-10-14 16:07:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 132087808. Throughput: 0: 1664.9, 1: 1648.9. Samples: 33026142. Policy #0 lag: (min: 17.0, avg: 28.2, max: 49.0) -[2023-10-14 16:07:33,165][74987] Avg episode reward: [(0, '23.570'), (1, '30.100')] -[2023-10-14 16:07:34,130][75949] Updated weights for policy 0, policy_version 64581 (0.0008) -[2023-10-14 16:07:34,504][75949] Updated weights for policy 0, policy_version 64591 (0.0009) -[2023-10-14 16:07:34,657][75950] Updated weights for policy 1, policy_version 64420 (0.0011) -[2023-10-14 16:07:34,877][75949] Updated weights for policy 0, policy_version 64601 (0.0009) -[2023-10-14 16:07:35,022][75950] Updated weights for policy 1, policy_version 64430 (0.0007) -[2023-10-14 16:07:35,390][75950] Updated weights for policy 1, policy_version 64440 (0.0009) -[2023-10-14 16:07:38,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132153344. Throughput: 0: 1696.1, 1: 1667.2. Samples: 33046918. Policy #0 lag: (min: 17.0, avg: 28.2, max: 49.0) -[2023-10-14 16:07:38,164][74987] Avg episode reward: [(0, '29.750'), (1, '31.740')] -[2023-10-14 16:07:38,896][75949] Updated weights for policy 0, policy_version 64611 (0.0009) -[2023-10-14 16:07:39,265][75949] Updated weights for policy 0, policy_version 64621 (0.0011) -[2023-10-14 16:07:39,558][75950] Updated weights for policy 1, policy_version 64450 (0.0008) -[2023-10-14 16:07:39,629][75949] Updated weights for policy 0, policy_version 64631 (0.0008) -[2023-10-14 16:07:39,917][75950] Updated weights for policy 1, policy_version 64460 (0.0008) -[2023-10-14 16:07:40,281][75950] Updated weights for policy 1, policy_version 64470 (0.0008) -[2023-10-14 16:07:40,648][75950] Updated weights for policy 1, policy_version 64480 (0.0010) -[2023-10-14 16:07:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132218880. Throughput: 0: 1693.0, 1: 1672.0. Samples: 33067610. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) -[2023-10-14 16:07:43,164][74987] Avg episode reward: [(0, '25.110'), (1, '31.900')] -[2023-10-14 16:07:43,784][75949] Updated weights for policy 0, policy_version 64641 (0.0009) -[2023-10-14 16:07:44,142][75949] Updated weights for policy 0, policy_version 64651 (0.0008) -[2023-10-14 16:07:44,518][75949] Updated weights for policy 0, policy_version 64661 (0.0008) -[2023-10-14 16:07:44,802][75950] Updated weights for policy 1, policy_version 64490 (0.0007) -[2023-10-14 16:07:44,882][75949] Updated weights for policy 0, policy_version 64671 (0.0008) -[2023-10-14 16:07:45,180][75950] Updated weights for policy 1, policy_version 64500 (0.0008) -[2023-10-14 16:07:45,548][75950] Updated weights for policy 1, policy_version 64510 (0.0010) -[2023-10-14 16:07:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 132284416. Throughput: 0: 1679.0, 1: 1654.5. Samples: 33076510. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) -[2023-10-14 16:07:48,165][74987] Avg episode reward: [(0, '30.280'), (1, '32.290')] -[2023-10-14 16:07:49,094][75949] Updated weights for policy 0, policy_version 64681 (0.0008) -[2023-10-14 16:07:49,463][75949] Updated weights for policy 0, policy_version 64691 (0.0007) -[2023-10-14 16:07:49,617][75950] Updated weights for policy 1, policy_version 64520 (0.0009) -[2023-10-14 16:07:49,830][75949] Updated weights for policy 0, policy_version 64701 (0.0008) -[2023-10-14 16:07:49,993][75950] Updated weights for policy 1, policy_version 64530 (0.0009) -[2023-10-14 16:07:50,362][75950] Updated weights for policy 1, policy_version 64540 (0.0009) -[2023-10-14 16:07:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132349952. Throughput: 0: 1684.3, 1: 1662.4. Samples: 33096794. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) -[2023-10-14 16:07:53,165][74987] Avg episode reward: [(0, '24.810'), (1, '32.340')] -[2023-10-14 16:07:53,893][75949] Updated weights for policy 0, policy_version 64711 (0.0009) -[2023-10-14 16:07:54,264][75949] Updated weights for policy 0, policy_version 64721 (0.0010) -[2023-10-14 16:07:54,513][75950] Updated weights for policy 1, policy_version 64550 (0.0010) -[2023-10-14 16:07:54,640][75949] Updated weights for policy 0, policy_version 64731 (0.0009) -[2023-10-14 16:07:54,887][75950] Updated weights for policy 1, policy_version 64560 (0.0009) -[2023-10-14 16:07:55,248][75950] Updated weights for policy 1, policy_version 64570 (0.0009) -[2023-10-14 16:07:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 132415488. Throughput: 0: 1691.0, 1: 1660.7. Samples: 33117758. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) -[2023-10-14 16:07:58,165][74987] Avg episode reward: [(0, '30.210'), (1, '33.610')] -[2023-10-14 16:07:58,553][75949] Updated weights for policy 0, policy_version 64741 (0.0008) -[2023-10-14 16:07:58,923][75949] Updated weights for policy 0, policy_version 64751 (0.0009) -[2023-10-14 16:07:59,175][75950] Updated weights for policy 1, policy_version 64580 (0.0008) -[2023-10-14 16:07:59,285][75949] Updated weights for policy 0, policy_version 64761 (0.0007) -[2023-10-14 16:07:59,527][75950] Updated weights for policy 1, policy_version 64590 (0.0008) -[2023-10-14 16:07:59,900][75950] Updated weights for policy 1, policy_version 64600 (0.0007) -[2023-10-14 16:08:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132481024. Throughput: 0: 1682.4, 1: 1659.5. Samples: 33126946. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) -[2023-10-14 16:08:03,164][74987] Avg episode reward: [(0, '23.880'), (1, '32.260')] -[2023-10-14 16:08:03,655][75949] Updated weights for policy 0, policy_version 64771 (0.0008) -[2023-10-14 16:08:04,051][75949] Updated weights for policy 0, policy_version 64781 (0.0008) -[2023-10-14 16:08:04,174][75950] Updated weights for policy 1, policy_version 64610 (0.0008) -[2023-10-14 16:08:04,414][75949] Updated weights for policy 0, policy_version 64791 (0.0008) -[2023-10-14 16:08:04,542][75950] Updated weights for policy 1, policy_version 64620 (0.0008) -[2023-10-14 16:08:04,909][75950] Updated weights for policy 1, policy_version 64630 (0.0011) -[2023-10-14 16:08:05,271][75950] Updated weights for policy 1, policy_version 64640 (0.0011) -[2023-10-14 16:08:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 132546560. Throughput: 0: 1681.7, 1: 1670.8. Samples: 33147292. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) -[2023-10-14 16:08:08,165][74987] Avg episode reward: [(0, '27.790'), (1, '31.540')] -[2023-10-14 16:08:08,471][75949] Updated weights for policy 0, policy_version 64801 (0.0009) -[2023-10-14 16:08:08,845][75949] Updated weights for policy 0, policy_version 64811 (0.0008) -[2023-10-14 16:08:09,220][75949] Updated weights for policy 0, policy_version 64821 (0.0009) -[2023-10-14 16:08:09,359][75950] Updated weights for policy 1, policy_version 64650 (0.0007) -[2023-10-14 16:08:09,585][75949] Updated weights for policy 0, policy_version 64831 (0.0009) -[2023-10-14 16:08:09,717][75950] Updated weights for policy 1, policy_version 64660 (0.0009) -[2023-10-14 16:08:10,083][75950] Updated weights for policy 1, policy_version 64670 (0.0010) -[2023-10-14 16:08:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132612096. Throughput: 0: 1683.6, 1: 1672.7. Samples: 33168142. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) -[2023-10-14 16:08:13,164][74987] Avg episode reward: [(0, '21.600'), (1, '33.700')] -[2023-10-14 16:08:13,532][75949] Updated weights for policy 0, policy_version 64841 (0.0010) -[2023-10-14 16:08:13,908][75949] Updated weights for policy 0, policy_version 64851 (0.0008) -[2023-10-14 16:08:14,000][75950] Updated weights for policy 1, policy_version 64680 (0.0009) -[2023-10-14 16:08:14,286][75949] Updated weights for policy 0, policy_version 64861 (0.0007) -[2023-10-14 16:08:14,368][75950] Updated weights for policy 1, policy_version 64690 (0.0009) -[2023-10-14 16:08:14,739][75950] Updated weights for policy 1, policy_version 64700 (0.0010) -[2023-10-14 16:08:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132677632. Throughput: 0: 1690.0, 1: 1669.6. Samples: 33177324. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) -[2023-10-14 16:08:18,164][74987] Avg episode reward: [(0, '27.700'), (1, '32.520')] -[2023-10-14 16:08:18,202][75949] Updated weights for policy 0, policy_version 64871 (0.0009) -[2023-10-14 16:08:18,581][75949] Updated weights for policy 0, policy_version 64881 (0.0009) -[2023-10-14 16:08:18,952][75949] Updated weights for policy 0, policy_version 64891 (0.0009) -[2023-10-14 16:08:19,139][75950] Updated weights for policy 1, policy_version 64710 (0.0010) -[2023-10-14 16:08:19,509][75950] Updated weights for policy 1, policy_version 64720 (0.0011) -[2023-10-14 16:08:19,877][75950] Updated weights for policy 1, policy_version 64730 (0.0009) -[2023-10-14 16:08:23,116][75949] Updated weights for policy 0, policy_version 64901 (0.0009) -[2023-10-14 16:08:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 132743168. Throughput: 0: 1681.1, 1: 1668.2. Samples: 33197634. Policy #0 lag: (min: 23.0, avg: 30.4, max: 55.0) -[2023-10-14 16:08:23,164][74987] Avg episode reward: [(0, '25.010'), (1, '32.240')] -[2023-10-14 16:08:23,490][75949] Updated weights for policy 0, policy_version 64911 (0.0007) -[2023-10-14 16:08:23,865][75949] Updated weights for policy 0, policy_version 64921 (0.0010) -[2023-10-14 16:08:23,964][75950] Updated weights for policy 1, policy_version 64740 (0.0009) -[2023-10-14 16:08:24,328][75950] Updated weights for policy 1, policy_version 64750 (0.0009) -[2023-10-14 16:08:24,689][75950] Updated weights for policy 1, policy_version 64760 (0.0010) -[2023-10-14 16:08:27,923][75949] Updated weights for policy 0, policy_version 64931 (0.0008) -[2023-10-14 16:08:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 132808704. Throughput: 0: 1679.1, 1: 1667.5. Samples: 33218204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:08:28,164][74987] Avg episode reward: [(0, '28.020'), (1, '33.880')] -[2023-10-14 16:08:28,294][75949] Updated weights for policy 0, policy_version 64941 (0.0008) -[2023-10-14 16:08:28,590][75950] Updated weights for policy 1, policy_version 64770 (0.0009) -[2023-10-14 16:08:28,662][75949] Updated weights for policy 0, policy_version 64951 (0.0009) -[2023-10-14 16:08:28,964][75950] Updated weights for policy 1, policy_version 64780 (0.0007) -[2023-10-14 16:08:29,322][75950] Updated weights for policy 1, policy_version 64790 (0.0007) -[2023-10-14 16:08:29,682][75950] Updated weights for policy 1, policy_version 64800 (0.0008) -[2023-10-14 16:08:32,800][75949] Updated weights for policy 0, policy_version 64961 (0.0008) -[2023-10-14 16:08:33,163][75949] Updated weights for policy 0, policy_version 64971 (0.0011) -[2023-10-14 16:08:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132874240. Throughput: 0: 1678.2, 1: 1676.2. Samples: 33227460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:08:33,164][74987] Avg episode reward: [(0, '25.130'), (1, '35.340')] -[2023-10-14 16:08:33,535][75949] Updated weights for policy 0, policy_version 64981 (0.0008) -[2023-10-14 16:08:33,900][75949] Updated weights for policy 0, policy_version 64991 (0.0009) -[2023-10-14 16:08:33,905][75950] Updated weights for policy 1, policy_version 64810 (0.0008) -[2023-10-14 16:08:34,272][75950] Updated weights for policy 1, policy_version 64820 (0.0007) -[2023-10-14 16:08:34,648][75950] Updated weights for policy 1, policy_version 64830 (0.0007) -[2023-10-14 16:08:37,939][75949] Updated weights for policy 0, policy_version 65001 (0.0009) -[2023-10-14 16:08:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132939776. Throughput: 0: 1681.6, 1: 1679.1. Samples: 33248026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:08:38,164][74987] Avg episode reward: [(0, '27.350'), (1, '34.620')] -[2023-10-14 16:08:38,310][75949] Updated weights for policy 0, policy_version 65011 (0.0007) -[2023-10-14 16:08:38,591][75950] Updated weights for policy 1, policy_version 64840 (0.0009) -[2023-10-14 16:08:38,682][75949] Updated weights for policy 0, policy_version 65021 (0.0008) -[2023-10-14 16:08:38,964][75950] Updated weights for policy 1, policy_version 64850 (0.0007) -[2023-10-14 16:08:39,332][75950] Updated weights for policy 1, policy_version 64860 (0.0008) -[2023-10-14 16:08:42,936][75949] Updated weights for policy 0, policy_version 65031 (0.0009) -[2023-10-14 16:08:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133005312. Throughput: 0: 1669.5, 1: 1685.0. Samples: 33268708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:08:43,165][74987] Avg episode reward: [(0, '25.220'), (1, '32.510')] -[2023-10-14 16:08:43,312][75949] Updated weights for policy 0, policy_version 65041 (0.0007) -[2023-10-14 16:08:43,345][75950] Updated weights for policy 1, policy_version 64870 (0.0008) -[2023-10-14 16:08:43,676][75949] Updated weights for policy 0, policy_version 65051 (0.0007) -[2023-10-14 16:08:43,716][75950] Updated weights for policy 1, policy_version 64880 (0.0008) -[2023-10-14 16:08:43,851][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000065056_66617344.pth... -[2023-10-14 16:08:43,879][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000063488_65011712.pth -[2023-10-14 16:08:44,085][75950] Updated weights for policy 1, policy_version 64890 (0.0007) -[2023-10-14 16:08:44,309][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000064896_66453504.pth... -[2023-10-14 16:08:44,348][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000063328_64847872.pth -[2023-10-14 16:08:47,822][75949] Updated weights for policy 0, policy_version 65061 (0.0008) -[2023-10-14 16:08:48,115][75950] Updated weights for policy 1, policy_version 64900 (0.0008) -[2023-10-14 16:08:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133070848. Throughput: 0: 1673.3, 1: 1677.9. Samples: 33277752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:08:48,164][74987] Avg episode reward: [(0, '29.050'), (1, '34.250')] -[2023-10-14 16:08:48,193][75949] Updated weights for policy 0, policy_version 65071 (0.0009) -[2023-10-14 16:08:48,475][75950] Updated weights for policy 1, policy_version 64910 (0.0009) -[2023-10-14 16:08:48,562][75949] Updated weights for policy 0, policy_version 65081 (0.0008) -[2023-10-14 16:08:48,847][75950] Updated weights for policy 1, policy_version 64920 (0.0009) -[2023-10-14 16:08:52,625][75949] Updated weights for policy 0, policy_version 65091 (0.0008) -[2023-10-14 16:08:52,998][75950] Updated weights for policy 1, policy_version 64930 (0.0009) -[2023-10-14 16:08:53,030][75949] Updated weights for policy 0, policy_version 65101 (0.0008) -[2023-10-14 16:08:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133136384. Throughput: 0: 1677.3, 1: 1679.4. Samples: 33298344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:08:53,164][74987] Avg episode reward: [(0, '26.920'), (1, '33.880')] -[2023-10-14 16:08:53,362][75950] Updated weights for policy 1, policy_version 64940 (0.0009) -[2023-10-14 16:08:53,389][75949] Updated weights for policy 0, policy_version 65111 (0.0009) -[2023-10-14 16:08:53,730][75950] Updated weights for policy 1, policy_version 64950 (0.0008) -[2023-10-14 16:08:54,090][75950] Updated weights for policy 1, policy_version 64960 (0.0010) -[2023-10-14 16:08:57,518][75949] Updated weights for policy 0, policy_version 65121 (0.0008) -[2023-10-14 16:08:57,894][75949] Updated weights for policy 0, policy_version 65131 (0.0011) -[2023-10-14 16:08:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133201920. Throughput: 0: 1665.1, 1: 1679.6. Samples: 33318652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:08:58,164][74987] Avg episode reward: [(0, '27.560'), (1, '31.800')] -[2023-10-14 16:08:58,251][75949] Updated weights for policy 0, policy_version 65141 (0.0009) -[2023-10-14 16:08:58,354][75950] Updated weights for policy 1, policy_version 64970 (0.0008) -[2023-10-14 16:08:58,618][75949] Updated weights for policy 0, policy_version 65151 (0.0008) -[2023-10-14 16:08:58,721][75950] Updated weights for policy 1, policy_version 64980 (0.0009) -[2023-10-14 16:08:59,092][75950] Updated weights for policy 1, policy_version 64990 (0.0007) -[2023-10-14 16:09:02,663][75949] Updated weights for policy 0, policy_version 65161 (0.0010) -[2023-10-14 16:09:03,030][75950] Updated weights for policy 1, policy_version 65000 (0.0009) -[2023-10-14 16:09:03,038][75949] Updated weights for policy 0, policy_version 65171 (0.0007) -[2023-10-14 16:09:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133267456. Throughput: 0: 1669.2, 1: 1680.6. Samples: 33328066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:09:03,164][74987] Avg episode reward: [(0, '28.460'), (1, '32.030')] -[2023-10-14 16:09:03,395][75950] Updated weights for policy 1, policy_version 65010 (0.0009) -[2023-10-14 16:09:03,401][75949] Updated weights for policy 0, policy_version 65181 (0.0009) -[2023-10-14 16:09:03,755][75950] Updated weights for policy 1, policy_version 65020 (0.0008) -[2023-10-14 16:09:07,440][75949] Updated weights for policy 0, policy_version 65191 (0.0008) -[2023-10-14 16:09:07,813][75949] Updated weights for policy 0, policy_version 65201 (0.0007) -[2023-10-14 16:09:07,917][75950] Updated weights for policy 1, policy_version 65030 (0.0009) -[2023-10-14 16:09:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133332992. Throughput: 0: 1670.4, 1: 1694.1. Samples: 33349040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:09:08,164][74987] Avg episode reward: [(0, '27.220'), (1, '33.530')] -[2023-10-14 16:09:08,178][75949] Updated weights for policy 0, policy_version 65211 (0.0008) -[2023-10-14 16:09:08,288][75950] Updated weights for policy 1, policy_version 65040 (0.0008) -[2023-10-14 16:09:08,658][75950] Updated weights for policy 1, policy_version 65050 (0.0009) -[2023-10-14 16:09:12,168][75949] Updated weights for policy 0, policy_version 65221 (0.0009) -[2023-10-14 16:09:12,538][75949] Updated weights for policy 0, policy_version 65231 (0.0008) -[2023-10-14 16:09:12,554][75950] Updated weights for policy 1, policy_version 65060 (0.0008) -[2023-10-14 16:09:12,903][75949] Updated weights for policy 0, policy_version 65241 (0.0007) -[2023-10-14 16:09:12,926][75950] Updated weights for policy 1, policy_version 65070 (0.0009) -[2023-10-14 16:09:13,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 133431296. Throughput: 0: 1653.9, 1: 1692.9. Samples: 33368814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:09:13,164][74987] Avg episode reward: [(0, '28.710'), (1, '31.730')] -[2023-10-14 16:09:13,294][75950] Updated weights for policy 1, policy_version 65080 (0.0007) -[2023-10-14 16:09:17,066][75949] Updated weights for policy 0, policy_version 65251 (0.0009) -[2023-10-14 16:09:17,408][75950] Updated weights for policy 1, policy_version 65090 (0.0007) -[2023-10-14 16:09:17,441][75949] Updated weights for policy 0, policy_version 65261 (0.0007) -[2023-10-14 16:09:17,768][75950] Updated weights for policy 1, policy_version 65100 (0.0008) -[2023-10-14 16:09:17,808][75949] Updated weights for policy 0, policy_version 65271 (0.0007) -[2023-10-14 16:09:18,135][75950] Updated weights for policy 1, policy_version 65110 (0.0007) -[2023-10-14 16:09:18,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 133496832. Throughput: 0: 1669.3, 1: 1694.1. Samples: 33378814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:09:18,164][74987] Avg episode reward: [(0, '24.970'), (1, '31.650')] -[2023-10-14 16:09:18,506][75950] Updated weights for policy 1, policy_version 65120 (0.0010) -[2023-10-14 16:09:21,817][75949] Updated weights for policy 0, policy_version 65281 (0.0009) -[2023-10-14 16:09:22,170][75949] Updated weights for policy 0, policy_version 65291 (0.0008) -[2023-10-14 16:09:22,552][75949] Updated weights for policy 0, policy_version 65301 (0.0008) -[2023-10-14 16:09:22,745][75950] Updated weights for policy 1, policy_version 65130 (0.0007) -[2023-10-14 16:09:22,927][75949] Updated weights for policy 0, policy_version 65311 (0.0008) -[2023-10-14 16:09:23,117][75950] Updated weights for policy 1, policy_version 65140 (0.0007) -[2023-10-14 16:09:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 133562368. Throughput: 0: 1670.3, 1: 1694.1. Samples: 33399426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:09:23,164][74987] Avg episode reward: [(0, '29.080'), (1, '33.280')] -[2023-10-14 16:09:23,490][75950] Updated weights for policy 1, policy_version 65150 (0.0008) -[2023-10-14 16:09:27,007][75949] Updated weights for policy 0, policy_version 65321 (0.0011) -[2023-10-14 16:09:27,362][75949] Updated weights for policy 0, policy_version 65331 (0.0010) -[2023-10-14 16:09:27,574][75950] Updated weights for policy 1, policy_version 65160 (0.0008) -[2023-10-14 16:09:27,736][75949] Updated weights for policy 0, policy_version 65341 (0.0008) -[2023-10-14 16:09:27,935][75950] Updated weights for policy 1, policy_version 65170 (0.0008) -[2023-10-14 16:09:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 133627904. Throughput: 0: 1648.6, 1: 1679.5. Samples: 33418474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:09:28,164][74987] Avg episode reward: [(0, '25.410'), (1, '32.840')] -[2023-10-14 16:09:28,296][75950] Updated weights for policy 1, policy_version 65180 (0.0008) -[2023-10-14 16:09:31,827][75949] Updated weights for policy 0, policy_version 65351 (0.0008) -[2023-10-14 16:09:32,196][75949] Updated weights for policy 0, policy_version 65361 (0.0009) -[2023-10-14 16:09:32,248][75950] Updated weights for policy 1, policy_version 65190 (0.0009) -[2023-10-14 16:09:32,572][75949] Updated weights for policy 0, policy_version 65371 (0.0007) -[2023-10-14 16:09:32,613][75950] Updated weights for policy 1, policy_version 65200 (0.0008) -[2023-10-14 16:09:32,978][75950] Updated weights for policy 1, policy_version 65210 (0.0011) -[2023-10-14 16:09:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 133693440. Throughput: 0: 1674.0, 1: 1689.0. Samples: 33429086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:09:33,164][74987] Avg episode reward: [(0, '27.350'), (1, '32.420')] -[2023-10-14 16:09:36,621][75949] Updated weights for policy 0, policy_version 65381 (0.0007) -[2023-10-14 16:09:36,993][75949] Updated weights for policy 0, policy_version 65391 (0.0007) -[2023-10-14 16:09:37,136][75950] Updated weights for policy 1, policy_version 65220 (0.0010) -[2023-10-14 16:09:37,354][75949] Updated weights for policy 0, policy_version 65401 (0.0009) -[2023-10-14 16:09:37,509][75950] Updated weights for policy 1, policy_version 65230 (0.0008) -[2023-10-14 16:09:37,870][75950] Updated weights for policy 1, policy_version 65240 (0.0007) -[2023-10-14 16:09:38,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 133758976. Throughput: 0: 1670.4, 1: 1690.4. Samples: 33449578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:09:38,164][74987] Avg episode reward: [(0, '24.800'), (1, '33.970')] -[2023-10-14 16:09:41,480][75949] Updated weights for policy 0, policy_version 65411 (0.0007) -[2023-10-14 16:09:41,780][75950] Updated weights for policy 1, policy_version 65250 (0.0009) -[2023-10-14 16:09:41,889][75949] Updated weights for policy 0, policy_version 65421 (0.0008) -[2023-10-14 16:09:42,145][75950] Updated weights for policy 1, policy_version 65260 (0.0009) -[2023-10-14 16:09:42,265][75949] Updated weights for policy 0, policy_version 65431 (0.0007) -[2023-10-14 16:09:42,516][75950] Updated weights for policy 1, policy_version 65270 (0.0008) -[2023-10-14 16:09:42,871][75950] Updated weights for policy 1, policy_version 65280 (0.0009) -[2023-10-14 16:09:43,163][74987] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 133857280. Throughput: 0: 1661.0, 1: 1666.6. Samples: 33468394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:09:43,164][74987] Avg episode reward: [(0, '27.730'), (1, '35.030')] -[2023-10-14 16:09:46,419][75949] Updated weights for policy 0, policy_version 65441 (0.0009) -[2023-10-14 16:09:46,788][75949] Updated weights for policy 0, policy_version 65451 (0.0009) -[2023-10-14 16:09:46,965][75950] Updated weights for policy 1, policy_version 65290 (0.0008) -[2023-10-14 16:09:47,150][75949] Updated weights for policy 0, policy_version 65461 (0.0009) -[2023-10-14 16:09:47,328][75950] Updated weights for policy 1, policy_version 65300 (0.0008) -[2023-10-14 16:09:47,524][75949] Updated weights for policy 0, policy_version 65471 (0.0008) -[2023-10-14 16:09:47,693][75950] Updated weights for policy 1, policy_version 65310 (0.0008) -[2023-10-14 16:09:48,163][74987] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 133922816. Throughput: 0: 1678.3, 1: 1685.7. Samples: 33479448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:09:48,164][74987] Avg episode reward: [(0, '26.080'), (1, '34.770')] -[2023-10-14 16:09:51,655][75949] Updated weights for policy 0, policy_version 65481 (0.0009) -[2023-10-14 16:09:51,927][75950] Updated weights for policy 1, policy_version 65320 (0.0007) -[2023-10-14 16:09:52,026][75949] Updated weights for policy 0, policy_version 65491 (0.0008) -[2023-10-14 16:09:52,283][75950] Updated weights for policy 1, policy_version 65330 (0.0007) -[2023-10-14 16:09:52,390][75949] Updated weights for policy 0, policy_version 65501 (0.0009) -[2023-10-14 16:09:52,653][75950] Updated weights for policy 1, policy_version 65340 (0.0008) -[2023-10-14 16:09:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 133988352. Throughput: 0: 1666.6, 1: 1679.7. Samples: 33499624. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 16:09:53,164][74987] Avg episode reward: [(0, '28.480'), (1, '35.340')] -[2023-10-14 16:09:56,481][75949] Updated weights for policy 0, policy_version 65511 (0.0009) -[2023-10-14 16:09:56,800][75950] Updated weights for policy 1, policy_version 65350 (0.0008) -[2023-10-14 16:09:56,848][75949] Updated weights for policy 0, policy_version 65521 (0.0009) -[2023-10-14 16:09:57,169][75950] Updated weights for policy 1, policy_version 65360 (0.0008) -[2023-10-14 16:09:57,228][75949] Updated weights for policy 0, policy_version 65531 (0.0008) -[2023-10-14 16:09:57,531][75950] Updated weights for policy 1, policy_version 65370 (0.0009) -[2023-10-14 16:09:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 134053888. Throughput: 0: 1665.9, 1: 1658.1. Samples: 33518394. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 16:09:58,164][74987] Avg episode reward: [(0, '26.030'), (1, '31.890')] -[2023-10-14 16:10:01,211][75949] Updated weights for policy 0, policy_version 65541 (0.0008) -[2023-10-14 16:10:01,581][75949] Updated weights for policy 0, policy_version 65551 (0.0008) -[2023-10-14 16:10:01,643][75950] Updated weights for policy 1, policy_version 65380 (0.0008) -[2023-10-14 16:10:01,960][75949] Updated weights for policy 0, policy_version 65561 (0.0010) -[2023-10-14 16:10:02,012][75950] Updated weights for policy 1, policy_version 65390 (0.0009) -[2023-10-14 16:10:02,379][75950] Updated weights for policy 1, policy_version 65400 (0.0008) -[2023-10-14 16:10:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 134119424. Throughput: 0: 1680.3, 1: 1675.8. Samples: 33529836. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 16:10:03,164][74987] Avg episode reward: [(0, '27.990'), (1, '34.350')] -[2023-10-14 16:10:05,928][75949] Updated weights for policy 0, policy_version 65571 (0.0009) -[2023-10-14 16:10:06,310][75949] Updated weights for policy 0, policy_version 65581 (0.0011) -[2023-10-14 16:10:06,348][75950] Updated weights for policy 1, policy_version 65410 (0.0008) -[2023-10-14 16:10:06,682][75949] Updated weights for policy 0, policy_version 65591 (0.0010) -[2023-10-14 16:10:06,714][75950] Updated weights for policy 1, policy_version 65420 (0.0008) -[2023-10-14 16:10:07,082][75950] Updated weights for policy 1, policy_version 65430 (0.0009) -[2023-10-14 16:10:07,448][75950] Updated weights for policy 1, policy_version 65440 (0.0010) -[2023-10-14 16:10:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 134184960. Throughput: 0: 1663.1, 1: 1671.8. Samples: 33549494. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 16:10:08,165][74987] Avg episode reward: [(0, '26.200'), (1, '32.660')] -[2023-10-14 16:10:10,873][75949] Updated weights for policy 0, policy_version 65601 (0.0009) -[2023-10-14 16:10:11,244][75949] Updated weights for policy 0, policy_version 65611 (0.0008) -[2023-10-14 16:10:11,334][75950] Updated weights for policy 1, policy_version 65450 (0.0010) -[2023-10-14 16:10:11,615][75949] Updated weights for policy 0, policy_version 65621 (0.0008) -[2023-10-14 16:10:11,699][75950] Updated weights for policy 1, policy_version 65460 (0.0008) -[2023-10-14 16:10:11,985][75949] Updated weights for policy 0, policy_version 65631 (0.0010) -[2023-10-14 16:10:12,068][75950] Updated weights for policy 1, policy_version 65470 (0.0009) -[2023-10-14 16:10:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 134250496. Throughput: 0: 1674.9, 1: 1665.2. Samples: 33568778. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 16:10:13,165][74987] Avg episode reward: [(0, '26.350'), (1, '31.140')] -[2023-10-14 16:10:15,984][75949] Updated weights for policy 0, policy_version 65641 (0.0008) -[2023-10-14 16:10:16,207][75950] Updated weights for policy 1, policy_version 65480 (0.0010) -[2023-10-14 16:10:16,359][75949] Updated weights for policy 0, policy_version 65651 (0.0010) -[2023-10-14 16:10:16,571][75950] Updated weights for policy 1, policy_version 65490 (0.0010) -[2023-10-14 16:10:16,718][75949] Updated weights for policy 0, policy_version 65661 (0.0008) -[2023-10-14 16:10:16,941][75950] Updated weights for policy 1, policy_version 65500 (0.0010) -[2023-10-14 16:10:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 134316032. Throughput: 0: 1679.6, 1: 1684.0. Samples: 33580452. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 16:10:18,165][74987] Avg episode reward: [(0, '26.480'), (1, '31.990')] -[2023-10-14 16:10:20,892][75949] Updated weights for policy 0, policy_version 65671 (0.0007) -[2023-10-14 16:10:21,259][75949] Updated weights for policy 0, policy_version 65681 (0.0008) -[2023-10-14 16:10:21,299][75950] Updated weights for policy 1, policy_version 65510 (0.0007) -[2023-10-14 16:10:21,621][75949] Updated weights for policy 0, policy_version 65691 (0.0008) -[2023-10-14 16:10:21,668][75950] Updated weights for policy 1, policy_version 65520 (0.0008) -[2023-10-14 16:10:22,036][75950] Updated weights for policy 1, policy_version 65530 (0.0008) -[2023-10-14 16:10:23,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 134381568. Throughput: 0: 1660.2, 1: 1662.8. Samples: 33599114. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 16:10:23,164][74987] Avg episode reward: [(0, '26.380'), (1, '31.700')] -[2023-10-14 16:10:25,679][75949] Updated weights for policy 0, policy_version 65701 (0.0009) -[2023-10-14 16:10:26,045][75949] Updated weights for policy 0, policy_version 65711 (0.0008) -[2023-10-14 16:10:26,075][75950] Updated weights for policy 1, policy_version 65540 (0.0009) -[2023-10-14 16:10:26,411][75949] Updated weights for policy 0, policy_version 65721 (0.0009) -[2023-10-14 16:10:26,434][75950] Updated weights for policy 1, policy_version 65550 (0.0008) -[2023-10-14 16:10:26,805][75950] Updated weights for policy 1, policy_version 65560 (0.0009) -[2023-10-14 16:10:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 134447104. Throughput: 0: 1677.3, 1: 1671.0. Samples: 33619066. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 16:10:28,164][74987] Avg episode reward: [(0, '27.490'), (1, '31.610')] -[2023-10-14 16:10:30,453][75949] Updated weights for policy 0, policy_version 65731 (0.0008) -[2023-10-14 16:10:30,815][75950] Updated weights for policy 1, policy_version 65570 (0.0009) -[2023-10-14 16:10:30,849][75949] Updated weights for policy 0, policy_version 65741 (0.0008) -[2023-10-14 16:10:31,191][75950] Updated weights for policy 1, policy_version 65580 (0.0009) -[2023-10-14 16:10:31,219][75949] Updated weights for policy 0, policy_version 65751 (0.0009) -[2023-10-14 16:10:31,558][75950] Updated weights for policy 1, policy_version 65590 (0.0010) -[2023-10-14 16:10:31,926][75950] Updated weights for policy 1, policy_version 65600 (0.0009) -[2023-10-14 16:10:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 134512640. Throughput: 0: 1672.9, 1: 1682.2. Samples: 33630426. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 16:10:33,164][74987] Avg episode reward: [(0, '25.360'), (1, '31.450')] -[2023-10-14 16:10:35,303][75949] Updated weights for policy 0, policy_version 65761 (0.0008) -[2023-10-14 16:10:35,667][75949] Updated weights for policy 0, policy_version 65771 (0.0008) -[2023-10-14 16:10:36,011][75950] Updated weights for policy 1, policy_version 65610 (0.0008) -[2023-10-14 16:10:36,037][75949] Updated weights for policy 0, policy_version 65781 (0.0008) -[2023-10-14 16:10:36,369][75950] Updated weights for policy 1, policy_version 65620 (0.0009) -[2023-10-14 16:10:36,399][75949] Updated weights for policy 0, policy_version 65791 (0.0010) -[2023-10-14 16:10:36,733][75950] Updated weights for policy 1, policy_version 65630 (0.0008) -[2023-10-14 16:10:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 134578176. Throughput: 0: 1666.5, 1: 1657.9. Samples: 33649222. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 16:10:38,164][74987] Avg episode reward: [(0, '28.280'), (1, '34.000')] -[2023-10-14 16:10:40,444][75949] Updated weights for policy 0, policy_version 65801 (0.0008) -[2023-10-14 16:10:40,819][75949] Updated weights for policy 0, policy_version 65811 (0.0009) -[2023-10-14 16:10:40,836][75950] Updated weights for policy 1, policy_version 65640 (0.0008) -[2023-10-14 16:10:41,178][75949] Updated weights for policy 0, policy_version 65821 (0.0009) -[2023-10-14 16:10:41,195][75950] Updated weights for policy 1, policy_version 65650 (0.0009) -[2023-10-14 16:10:41,563][75950] Updated weights for policy 1, policy_version 65660 (0.0008) -[2023-10-14 16:10:43,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134643712. Throughput: 0: 1690.5, 1: 1674.8. Samples: 33669828. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 16:10:43,164][74987] Avg episode reward: [(0, '26.220'), (1, '34.710')] -[2023-10-14 16:10:43,174][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000065824_67403776.pth... -[2023-10-14 16:10:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000065664_67239936.pth... -[2023-10-14 16:10:43,211][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000064096_65634304.pth -[2023-10-14 16:10:43,214][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000064256_65798144.pth -[2023-10-14 16:10:45,292][75949] Updated weights for policy 0, policy_version 65831 (0.0008) -[2023-10-14 16:10:45,662][75949] Updated weights for policy 0, policy_version 65841 (0.0008) -[2023-10-14 16:10:45,773][75950] Updated weights for policy 1, policy_version 65670 (0.0007) -[2023-10-14 16:10:46,026][75949] Updated weights for policy 0, policy_version 65851 (0.0008) -[2023-10-14 16:10:46,142][75950] Updated weights for policy 1, policy_version 65680 (0.0007) -[2023-10-14 16:10:46,504][75950] Updated weights for policy 1, policy_version 65690 (0.0007) -[2023-10-14 16:10:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134709248. Throughput: 0: 1675.6, 1: 1673.6. Samples: 33680546. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 16:10:48,164][74987] Avg episode reward: [(0, '28.760'), (1, '33.660')] -[2023-10-14 16:10:50,103][75949] Updated weights for policy 0, policy_version 65861 (0.0009) -[2023-10-14 16:10:50,470][75949] Updated weights for policy 0, policy_version 65871 (0.0009) -[2023-10-14 16:10:50,582][75950] Updated weights for policy 1, policy_version 65700 (0.0007) -[2023-10-14 16:10:50,832][75949] Updated weights for policy 0, policy_version 65881 (0.0007) -[2023-10-14 16:10:50,948][75950] Updated weights for policy 1, policy_version 65710 (0.0008) -[2023-10-14 16:10:51,311][75950] Updated weights for policy 1, policy_version 65720 (0.0010) -[2023-10-14 16:10:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134774784. Throughput: 0: 1676.6, 1: 1653.2. Samples: 33699334. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 16:10:53,164][74987] Avg episode reward: [(0, '26.040'), (1, '34.140')] -[2023-10-14 16:10:54,931][75949] Updated weights for policy 0, policy_version 65891 (0.0009) -[2023-10-14 16:10:55,299][75949] Updated weights for policy 0, policy_version 65901 (0.0007) -[2023-10-14 16:10:55,303][75950] Updated weights for policy 1, policy_version 65730 (0.0009) -[2023-10-14 16:10:55,664][75950] Updated weights for policy 1, policy_version 65740 (0.0009) -[2023-10-14 16:10:55,665][75949] Updated weights for policy 0, policy_version 65911 (0.0009) -[2023-10-14 16:10:56,026][75950] Updated weights for policy 1, policy_version 65750 (0.0008) -[2023-10-14 16:10:56,394][75950] Updated weights for policy 1, policy_version 65760 (0.0007) -[2023-10-14 16:10:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134840320. Throughput: 0: 1690.4, 1: 1672.0. Samples: 33720084. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 16:10:58,165][74987] Avg episode reward: [(0, '27.910'), (1, '35.430')] -[2023-10-14 16:10:59,711][75949] Updated weights for policy 0, policy_version 65921 (0.0009) -[2023-10-14 16:11:00,074][75949] Updated weights for policy 0, policy_version 65931 (0.0009) -[2023-10-14 16:11:00,447][75949] Updated weights for policy 0, policy_version 65941 (0.0008) -[2023-10-14 16:11:00,589][75950] Updated weights for policy 1, policy_version 65770 (0.0007) -[2023-10-14 16:11:00,805][75949] Updated weights for policy 0, policy_version 65951 (0.0008) -[2023-10-14 16:11:00,951][75950] Updated weights for policy 1, policy_version 65780 (0.0008) -[2023-10-14 16:11:01,324][75950] Updated weights for policy 1, policy_version 65790 (0.0008) -[2023-10-14 16:11:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 134905856. Throughput: 0: 1665.9, 1: 1662.4. Samples: 33730224. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 16:11:03,165][74987] Avg episode reward: [(0, '25.310'), (1, '34.440')] -[2023-10-14 16:11:04,802][75949] Updated weights for policy 0, policy_version 65961 (0.0008) -[2023-10-14 16:11:05,174][75949] Updated weights for policy 0, policy_version 65971 (0.0008) -[2023-10-14 16:11:05,394][75950] Updated weights for policy 1, policy_version 65800 (0.0008) -[2023-10-14 16:11:05,540][75949] Updated weights for policy 0, policy_version 65981 (0.0007) -[2023-10-14 16:11:05,760][75950] Updated weights for policy 1, policy_version 65810 (0.0007) -[2023-10-14 16:11:06,129][75950] Updated weights for policy 1, policy_version 65820 (0.0009) -[2023-10-14 16:11:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134971392. Throughput: 0: 1687.0, 1: 1663.5. Samples: 33749884. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 16:11:08,165][74987] Avg episode reward: [(0, '28.320'), (1, '31.710')] -[2023-10-14 16:11:09,565][75949] Updated weights for policy 0, policy_version 65991 (0.0007) -[2023-10-14 16:11:09,942][75949] Updated weights for policy 0, policy_version 66001 (0.0009) -[2023-10-14 16:11:10,125][75950] Updated weights for policy 1, policy_version 65830 (0.0007) -[2023-10-14 16:11:10,300][75949] Updated weights for policy 0, policy_version 66011 (0.0009) -[2023-10-14 16:11:10,493][75950] Updated weights for policy 1, policy_version 65840 (0.0009) -[2023-10-14 16:11:10,861][75950] Updated weights for policy 1, policy_version 65850 (0.0009) -[2023-10-14 16:11:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 135036928. Throughput: 0: 1690.4, 1: 1679.1. Samples: 33770696. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 16:11:13,165][74987] Avg episode reward: [(0, '25.940'), (1, '33.730')] -[2023-10-14 16:11:14,457][75949] Updated weights for policy 0, policy_version 66021 (0.0009) -[2023-10-14 16:11:14,831][75949] Updated weights for policy 0, policy_version 66031 (0.0008) -[2023-10-14 16:11:15,119][75950] Updated weights for policy 1, policy_version 65860 (0.0009) -[2023-10-14 16:11:15,203][75949] Updated weights for policy 0, policy_version 66041 (0.0007) -[2023-10-14 16:11:15,490][75950] Updated weights for policy 1, policy_version 65870 (0.0007) -[2023-10-14 16:11:15,850][75950] Updated weights for policy 1, policy_version 65880 (0.0012) -[2023-10-14 16:11:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135102464. Throughput: 0: 1670.9, 1: 1662.9. Samples: 33780448. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 16:11:18,164][74987] Avg episode reward: [(0, '28.470'), (1, '32.350')] -[2023-10-14 16:11:19,100][75949] Updated weights for policy 0, policy_version 66051 (0.0008) -[2023-10-14 16:11:19,474][75949] Updated weights for policy 0, policy_version 66061 (0.0009) -[2023-10-14 16:11:19,808][75950] Updated weights for policy 1, policy_version 65890 (0.0008) -[2023-10-14 16:11:19,849][75949] Updated weights for policy 0, policy_version 66071 (0.0008) -[2023-10-14 16:11:20,177][75950] Updated weights for policy 1, policy_version 65900 (0.0010) -[2023-10-14 16:11:20,531][75950] Updated weights for policy 1, policy_version 65910 (0.0008) -[2023-10-14 16:11:20,901][75950] Updated weights for policy 1, policy_version 65920 (0.0009) -[2023-10-14 16:11:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135168000. Throughput: 0: 1690.4, 1: 1668.6. Samples: 33800380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:11:23,164][74987] Avg episode reward: [(0, '26.150'), (1, '30.570')] -[2023-10-14 16:11:23,986][75949] Updated weights for policy 0, policy_version 66081 (0.0007) -[2023-10-14 16:11:24,377][75949] Updated weights for policy 0, policy_version 66091 (0.0008) -[2023-10-14 16:11:24,742][75949] Updated weights for policy 0, policy_version 66101 (0.0009) -[2023-10-14 16:11:25,062][75950] Updated weights for policy 1, policy_version 65930 (0.0008) -[2023-10-14 16:11:25,124][75949] Updated weights for policy 0, policy_version 66111 (0.0009) -[2023-10-14 16:11:25,432][75950] Updated weights for policy 1, policy_version 65940 (0.0008) -[2023-10-14 16:11:25,801][75950] Updated weights for policy 1, policy_version 65950 (0.0010) -[2023-10-14 16:11:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135233536. Throughput: 0: 1681.2, 1: 1667.6. Samples: 33820522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:11:28,164][74987] Avg episode reward: [(0, '27.280'), (1, '31.260')] -[2023-10-14 16:11:29,240][75949] Updated weights for policy 0, policy_version 66121 (0.0010) -[2023-10-14 16:11:29,611][75949] Updated weights for policy 0, policy_version 66131 (0.0010) -[2023-10-14 16:11:29,898][75950] Updated weights for policy 1, policy_version 65960 (0.0007) -[2023-10-14 16:11:29,986][75949] Updated weights for policy 0, policy_version 66141 (0.0008) -[2023-10-14 16:11:30,265][75950] Updated weights for policy 1, policy_version 65970 (0.0007) -[2023-10-14 16:11:30,628][75950] Updated weights for policy 1, policy_version 65980 (0.0007) -[2023-10-14 16:11:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135299072. Throughput: 0: 1670.1, 1: 1652.4. Samples: 33830060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:11:33,165][74987] Avg episode reward: [(0, '26.940'), (1, '33.440')] -[2023-10-14 16:11:34,056][75949] Updated weights for policy 0, policy_version 66151 (0.0009) -[2023-10-14 16:11:34,417][75949] Updated weights for policy 0, policy_version 66161 (0.0010) -[2023-10-14 16:11:34,582][75950] Updated weights for policy 1, policy_version 65990 (0.0007) -[2023-10-14 16:11:34,790][75949] Updated weights for policy 0, policy_version 66171 (0.0008) -[2023-10-14 16:11:34,950][75950] Updated weights for policy 1, policy_version 66000 (0.0009) -[2023-10-14 16:11:35,320][75950] Updated weights for policy 1, policy_version 66010 (0.0010) -[2023-10-14 16:11:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135364608. Throughput: 0: 1684.7, 1: 1677.8. Samples: 33850646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:11:38,165][74987] Avg episode reward: [(0, '27.430'), (1, '31.220')] -[2023-10-14 16:11:38,727][75949] Updated weights for policy 0, policy_version 66181 (0.0007) -[2023-10-14 16:11:39,090][75949] Updated weights for policy 0, policy_version 66191 (0.0008) -[2023-10-14 16:11:39,392][75950] Updated weights for policy 1, policy_version 66020 (0.0007) -[2023-10-14 16:11:39,464][75949] Updated weights for policy 0, policy_version 66201 (0.0009) -[2023-10-14 16:11:39,768][75950] Updated weights for policy 1, policy_version 66030 (0.0008) -[2023-10-14 16:11:40,120][75950] Updated weights for policy 1, policy_version 66040 (0.0010) -[2023-10-14 16:11:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 135430144. Throughput: 0: 1682.6, 1: 1675.3. Samples: 33871188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:11:43,165][74987] Avg episode reward: [(0, '26.530'), (1, '30.520')] -[2023-10-14 16:11:43,647][75949] Updated weights for policy 0, policy_version 66211 (0.0009) -[2023-10-14 16:11:44,017][75949] Updated weights for policy 0, policy_version 66221 (0.0008) -[2023-10-14 16:11:44,312][75950] Updated weights for policy 1, policy_version 66050 (0.0008) -[2023-10-14 16:11:44,388][75949] Updated weights for policy 0, policy_version 66231 (0.0010) -[2023-10-14 16:11:44,671][75950] Updated weights for policy 1, policy_version 66060 (0.0010) -[2023-10-14 16:11:45,043][75950] Updated weights for policy 1, policy_version 66070 (0.0009) -[2023-10-14 16:11:45,414][75950] Updated weights for policy 1, policy_version 66080 (0.0008) -[2023-10-14 16:11:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135495680. Throughput: 0: 1675.6, 1: 1656.8. Samples: 33880178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:11:48,164][74987] Avg episode reward: [(0, '26.730'), (1, '33.770')] -[2023-10-14 16:11:48,489][75949] Updated weights for policy 0, policy_version 66241 (0.0008) -[2023-10-14 16:11:48,856][75949] Updated weights for policy 0, policy_version 66251 (0.0010) -[2023-10-14 16:11:49,231][75949] Updated weights for policy 0, policy_version 66261 (0.0009) -[2023-10-14 16:11:49,549][75950] Updated weights for policy 1, policy_version 66090 (0.0007) -[2023-10-14 16:11:49,595][75949] Updated weights for policy 0, policy_version 66271 (0.0007) -[2023-10-14 16:11:49,913][75950] Updated weights for policy 1, policy_version 66100 (0.0007) -[2023-10-14 16:11:50,278][75950] Updated weights for policy 1, policy_version 66110 (0.0011) -[2023-10-14 16:11:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135561216. Throughput: 0: 1674.1, 1: 1679.0. Samples: 33900772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:11:53,165][74987] Avg episode reward: [(0, '27.540'), (1, '33.470')] -[2023-10-14 16:11:53,594][75949] Updated weights for policy 0, policy_version 66281 (0.0007) -[2023-10-14 16:11:53,961][75949] Updated weights for policy 0, policy_version 66291 (0.0008) -[2023-10-14 16:11:54,331][75949] Updated weights for policy 0, policy_version 66301 (0.0009) -[2023-10-14 16:11:54,336][75950] Updated weights for policy 1, policy_version 66120 (0.0008) -[2023-10-14 16:11:54,708][75950] Updated weights for policy 1, policy_version 66130 (0.0010) -[2023-10-14 16:11:55,078][75950] Updated weights for policy 1, policy_version 66140 (0.0009) -[2023-10-14 16:11:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 135626752. Throughput: 0: 1675.2, 1: 1675.9. Samples: 33921494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:11:58,164][74987] Avg episode reward: [(0, '24.790'), (1, '31.960')] -[2023-10-14 16:11:58,492][75949] Updated weights for policy 0, policy_version 66311 (0.0008) -[2023-10-14 16:11:58,859][75949] Updated weights for policy 0, policy_version 66321 (0.0007) -[2023-10-14 16:11:59,072][75950] Updated weights for policy 1, policy_version 66150 (0.0007) -[2023-10-14 16:11:59,232][75949] Updated weights for policy 0, policy_version 66331 (0.0008) -[2023-10-14 16:11:59,440][75950] Updated weights for policy 1, policy_version 66160 (0.0008) -[2023-10-14 16:11:59,803][75950] Updated weights for policy 1, policy_version 66170 (0.0010) -[2023-10-14 16:12:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135692288. Throughput: 0: 1674.5, 1: 1665.2. Samples: 33930738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:12:03,164][74987] Avg episode reward: [(0, '28.390'), (1, '34.610')] -[2023-10-14 16:12:03,262][75949] Updated weights for policy 0, policy_version 66341 (0.0008) -[2023-10-14 16:12:03,633][75949] Updated weights for policy 0, policy_version 66351 (0.0007) -[2023-10-14 16:12:03,958][75950] Updated weights for policy 1, policy_version 66180 (0.0009) -[2023-10-14 16:12:03,999][75949] Updated weights for policy 0, policy_version 66361 (0.0007) -[2023-10-14 16:12:04,320][75950] Updated weights for policy 1, policy_version 66190 (0.0010) -[2023-10-14 16:12:04,688][75950] Updated weights for policy 1, policy_version 66200 (0.0008) -[2023-10-14 16:12:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135757824. Throughput: 0: 1674.5, 1: 1682.5. Samples: 33951446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:12:08,165][74987] Avg episode reward: [(0, '24.270'), (1, '32.910')] -[2023-10-14 16:12:08,252][75949] Updated weights for policy 0, policy_version 66371 (0.0008) -[2023-10-14 16:12:08,612][75950] Updated weights for policy 1, policy_version 66210 (0.0008) -[2023-10-14 16:12:08,621][75949] Updated weights for policy 0, policy_version 66381 (0.0007) -[2023-10-14 16:12:08,967][75950] Updated weights for policy 1, policy_version 66220 (0.0007) -[2023-10-14 16:12:08,991][75949] Updated weights for policy 0, policy_version 66391 (0.0008) -[2023-10-14 16:12:09,339][75950] Updated weights for policy 1, policy_version 66230 (0.0007) -[2023-10-14 16:12:09,710][75950] Updated weights for policy 1, policy_version 66240 (0.0008) -[2023-10-14 16:12:13,013][75949] Updated weights for policy 0, policy_version 66401 (0.0008) -[2023-10-14 16:12:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 135823360. Throughput: 0: 1680.7, 1: 1688.6. Samples: 33972140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:12:13,164][74987] Avg episode reward: [(0, '28.320'), (1, '31.720')] -[2023-10-14 16:12:13,375][75949] Updated weights for policy 0, policy_version 66411 (0.0008) -[2023-10-14 16:12:13,745][75949] Updated weights for policy 0, policy_version 66421 (0.0008) -[2023-10-14 16:12:13,892][75950] Updated weights for policy 1, policy_version 66250 (0.0009) -[2023-10-14 16:12:14,110][75949] Updated weights for policy 0, policy_version 66431 (0.0009) -[2023-10-14 16:12:14,257][75950] Updated weights for policy 1, policy_version 66260 (0.0010) -[2023-10-14 16:12:14,619][75950] Updated weights for policy 1, policy_version 66270 (0.0008) -[2023-10-14 16:12:18,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135888896. Throughput: 0: 1675.2, 1: 1680.6. Samples: 33981072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:12:18,164][74987] Avg episode reward: [(0, '23.830'), (1, '33.390')] -[2023-10-14 16:12:18,394][75949] Updated weights for policy 0, policy_version 66441 (0.0010) -[2023-10-14 16:12:18,647][75950] Updated weights for policy 1, policy_version 66280 (0.0008) -[2023-10-14 16:12:18,751][75949] Updated weights for policy 0, policy_version 66451 (0.0010) -[2023-10-14 16:12:19,007][75950] Updated weights for policy 1, policy_version 66290 (0.0008) -[2023-10-14 16:12:19,117][75949] Updated weights for policy 0, policy_version 66461 (0.0009) -[2023-10-14 16:12:19,384][75950] Updated weights for policy 1, policy_version 66300 (0.0009) -[2023-10-14 16:12:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 135954432. Throughput: 0: 1673.1, 1: 1680.2. Samples: 34001546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:12:23,164][74987] Avg episode reward: [(0, '28.160'), (1, '33.080')] -[2023-10-14 16:12:23,194][75949] Updated weights for policy 0, policy_version 66471 (0.0009) -[2023-10-14 16:12:23,493][75950] Updated weights for policy 1, policy_version 66310 (0.0009) -[2023-10-14 16:12:23,567][75949] Updated weights for policy 0, policy_version 66481 (0.0008) -[2023-10-14 16:12:23,849][75950] Updated weights for policy 1, policy_version 66320 (0.0008) -[2023-10-14 16:12:23,930][75949] Updated weights for policy 0, policy_version 66491 (0.0009) -[2023-10-14 16:12:24,219][75950] Updated weights for policy 1, policy_version 66330 (0.0009) -[2023-10-14 16:12:28,052][75949] Updated weights for policy 0, policy_version 66501 (0.0009) -[2023-10-14 16:12:28,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136019968. Throughput: 0: 1675.1, 1: 1679.2. Samples: 34022130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:12:28,165][74987] Avg episode reward: [(0, '25.090'), (1, '33.960')] -[2023-10-14 16:12:28,315][75950] Updated weights for policy 1, policy_version 66340 (0.0009) -[2023-10-14 16:12:28,422][75949] Updated weights for policy 0, policy_version 66511 (0.0008) -[2023-10-14 16:12:28,680][75950] Updated weights for policy 1, policy_version 66350 (0.0009) -[2023-10-14 16:12:28,780][75949] Updated weights for policy 0, policy_version 66521 (0.0008) -[2023-10-14 16:12:29,046][75950] Updated weights for policy 1, policy_version 66360 (0.0008) -[2023-10-14 16:12:32,794][75949] Updated weights for policy 0, policy_version 66531 (0.0007) -[2023-10-14 16:12:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136085504. Throughput: 0: 1677.5, 1: 1680.5. Samples: 34031288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:12:33,164][74987] Avg episode reward: [(0, '29.080'), (1, '33.700')] -[2023-10-14 16:12:33,169][75949] Updated weights for policy 0, policy_version 66541 (0.0008) -[2023-10-14 16:12:33,284][75950] Updated weights for policy 1, policy_version 66370 (0.0010) -[2023-10-14 16:12:33,537][75949] Updated weights for policy 0, policy_version 66551 (0.0008) -[2023-10-14 16:12:33,641][75950] Updated weights for policy 1, policy_version 66380 (0.0008) -[2023-10-14 16:12:34,002][75950] Updated weights for policy 1, policy_version 66390 (0.0009) -[2023-10-14 16:12:34,376][75950] Updated weights for policy 1, policy_version 66400 (0.0007) -[2023-10-14 16:12:37,438][75949] Updated weights for policy 0, policy_version 66561 (0.0009) -[2023-10-14 16:12:37,810][75949] Updated weights for policy 0, policy_version 66571 (0.0009) -[2023-10-14 16:12:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136151040. Throughput: 0: 1683.7, 1: 1676.7. Samples: 34051990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:12:38,165][74987] Avg episode reward: [(0, '24.950'), (1, '35.010')] -[2023-10-14 16:12:38,169][75949] Updated weights for policy 0, policy_version 66581 (0.0008) -[2023-10-14 16:12:38,547][75949] Updated weights for policy 0, policy_version 66591 (0.0007) -[2023-10-14 16:12:38,581][75950] Updated weights for policy 1, policy_version 66410 (0.0009) -[2023-10-14 16:12:38,953][75950] Updated weights for policy 1, policy_version 66420 (0.0008) -[2023-10-14 16:12:39,325][75950] Updated weights for policy 1, policy_version 66430 (0.0009) -[2023-10-14 16:12:42,641][75949] Updated weights for policy 0, policy_version 66601 (0.0010) -[2023-10-14 16:12:43,007][75949] Updated weights for policy 0, policy_version 66611 (0.0007) -[2023-10-14 16:12:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136216576. Throughput: 0: 1673.0, 1: 1674.2. Samples: 34072118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:12:43,165][74987] Avg episode reward: [(0, '26.760'), (1, '34.120')] -[2023-10-14 16:12:43,377][75949] Updated weights for policy 0, policy_version 66621 (0.0009) -[2023-10-14 16:12:43,448][75950] Updated weights for policy 1, policy_version 66440 (0.0010) -[2023-10-14 16:12:43,479][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000066624_68222976.pth... -[2023-10-14 16:12:43,511][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000065056_66617344.pth -[2023-10-14 16:12:43,817][75950] Updated weights for policy 1, policy_version 66450 (0.0009) -[2023-10-14 16:12:44,192][75950] Updated weights for policy 1, policy_version 66460 (0.0010) -[2023-10-14 16:12:44,333][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000066464_68059136.pth... -[2023-10-14 16:12:44,368][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000064896_66453504.pth -[2023-10-14 16:12:47,412][75949] Updated weights for policy 0, policy_version 66631 (0.0008) -[2023-10-14 16:12:47,780][75949] Updated weights for policy 0, policy_version 66641 (0.0009) -[2023-10-14 16:12:48,148][75949] Updated weights for policy 0, policy_version 66651 (0.0008) -[2023-10-14 16:12:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136282112. Throughput: 0: 1683.4, 1: 1668.0. Samples: 34081552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:12:48,164][74987] Avg episode reward: [(0, '25.830'), (1, '33.180')] -[2023-10-14 16:12:48,370][75950] Updated weights for policy 1, policy_version 66470 (0.0009) -[2023-10-14 16:12:48,751][75950] Updated weights for policy 1, policy_version 66480 (0.0008) -[2023-10-14 16:12:49,116][75950] Updated weights for policy 1, policy_version 66490 (0.0011) -[2023-10-14 16:12:52,205][75949] Updated weights for policy 0, policy_version 66661 (0.0009) -[2023-10-14 16:12:52,567][75949] Updated weights for policy 0, policy_version 66671 (0.0009) -[2023-10-14 16:12:52,945][75949] Updated weights for policy 0, policy_version 66681 (0.0009) -[2023-10-14 16:12:53,123][75950] Updated weights for policy 1, policy_version 66500 (0.0009) -[2023-10-14 16:12:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136347648. Throughput: 0: 1680.0, 1: 1661.3. Samples: 34101804. Policy #0 lag: (min: 17.0, avg: 23.6, max: 49.0) -[2023-10-14 16:12:53,165][74987] Avg episode reward: [(0, '27.780'), (1, '32.870')] -[2023-10-14 16:12:53,489][75950] Updated weights for policy 1, policy_version 66510 (0.0010) -[2023-10-14 16:12:53,855][75950] Updated weights for policy 1, policy_version 66520 (0.0010) -[2023-10-14 16:12:57,200][75949] Updated weights for policy 0, policy_version 66691 (0.0007) -[2023-10-14 16:12:57,612][75949] Updated weights for policy 0, policy_version 66701 (0.0010) -[2023-10-14 16:12:57,872][75950] Updated weights for policy 1, policy_version 66530 (0.0009) -[2023-10-14 16:12:57,974][75949] Updated weights for policy 0, policy_version 66711 (0.0010) -[2023-10-14 16:12:58,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136413184. Throughput: 0: 1660.3, 1: 1669.6. Samples: 34121986. Policy #0 lag: (min: 17.0, avg: 23.6, max: 49.0) -[2023-10-14 16:12:58,164][74987] Avg episode reward: [(0, '25.000'), (1, '34.040')] -[2023-10-14 16:12:58,240][75950] Updated weights for policy 1, policy_version 66540 (0.0008) -[2023-10-14 16:12:58,601][75950] Updated weights for policy 1, policy_version 66550 (0.0007) -[2023-10-14 16:12:58,960][75950] Updated weights for policy 1, policy_version 66560 (0.0008) -[2023-10-14 16:13:01,928][75949] Updated weights for policy 0, policy_version 66721 (0.0008) -[2023-10-14 16:13:02,286][75949] Updated weights for policy 0, policy_version 66731 (0.0009) -[2023-10-14 16:13:02,653][75949] Updated weights for policy 0, policy_version 66741 (0.0007) -[2023-10-14 16:13:03,026][75949] Updated weights for policy 0, policy_version 66751 (0.0010) -[2023-10-14 16:13:03,060][75950] Updated weights for policy 1, policy_version 66570 (0.0009) -[2023-10-14 16:13:03,164][74987] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 136511488. Throughput: 0: 1679.0, 1: 1669.2. Samples: 34131742. Policy #0 lag: (min: 17.0, avg: 23.6, max: 49.0) -[2023-10-14 16:13:03,164][74987] Avg episode reward: [(0, '25.920'), (1, '31.600')] -[2023-10-14 16:13:03,428][75950] Updated weights for policy 1, policy_version 66580 (0.0008) -[2023-10-14 16:13:03,798][75950] Updated weights for policy 1, policy_version 66590 (0.0008) -[2023-10-14 16:13:07,145][75949] Updated weights for policy 0, policy_version 66761 (0.0008) -[2023-10-14 16:13:07,519][75949] Updated weights for policy 0, policy_version 66771 (0.0007) -[2023-10-14 16:13:07,835][75950] Updated weights for policy 1, policy_version 66600 (0.0008) -[2023-10-14 16:13:07,886][75949] Updated weights for policy 0, policy_version 66781 (0.0007) -[2023-10-14 16:13:08,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 136577024. Throughput: 0: 1683.7, 1: 1668.0. Samples: 34152376. Policy #0 lag: (min: 17.0, avg: 23.6, max: 49.0) -[2023-10-14 16:13:08,165][74987] Avg episode reward: [(0, '26.400'), (1, '33.430')] -[2023-10-14 16:13:08,205][75950] Updated weights for policy 1, policy_version 66610 (0.0008) -[2023-10-14 16:13:08,573][75950] Updated weights for policy 1, policy_version 66620 (0.0009) -[2023-10-14 16:13:11,924][75949] Updated weights for policy 0, policy_version 66791 (0.0007) -[2023-10-14 16:13:12,300][75949] Updated weights for policy 0, policy_version 66801 (0.0008) -[2023-10-14 16:13:12,611][75950] Updated weights for policy 1, policy_version 66630 (0.0008) -[2023-10-14 16:13:12,670][75949] Updated weights for policy 0, policy_version 66811 (0.0007) -[2023-10-14 16:13:12,979][75950] Updated weights for policy 1, policy_version 66640 (0.0009) -[2023-10-14 16:13:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 136642560. Throughput: 0: 1663.9, 1: 1667.2. Samples: 34172028. Policy #0 lag: (min: 17.0, avg: 23.6, max: 49.0) -[2023-10-14 16:13:13,164][74987] Avg episode reward: [(0, '25.950'), (1, '31.820')] -[2023-10-14 16:13:13,343][75950] Updated weights for policy 1, policy_version 66650 (0.0010) -[2023-10-14 16:13:16,797][75949] Updated weights for policy 0, policy_version 66821 (0.0008) -[2023-10-14 16:13:17,165][75949] Updated weights for policy 0, policy_version 66831 (0.0009) -[2023-10-14 16:13:17,354][75950] Updated weights for policy 1, policy_version 66660 (0.0009) -[2023-10-14 16:13:17,534][75949] Updated weights for policy 0, policy_version 66841 (0.0008) -[2023-10-14 16:13:17,723][75950] Updated weights for policy 1, policy_version 66670 (0.0008) -[2023-10-14 16:13:18,083][75950] Updated weights for policy 1, policy_version 66680 (0.0009) -[2023-10-14 16:13:18,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 136708096. Throughput: 0: 1684.3, 1: 1673.7. Samples: 34182398. Policy #0 lag: (min: 17.0, avg: 23.6, max: 49.0) -[2023-10-14 16:13:18,164][74987] Avg episode reward: [(0, '28.410'), (1, '31.920')] -[2023-10-14 16:13:21,409][75949] Updated weights for policy 0, policy_version 66851 (0.0010) -[2023-10-14 16:13:21,791][75949] Updated weights for policy 0, policy_version 66861 (0.0008) -[2023-10-14 16:13:22,160][75949] Updated weights for policy 0, policy_version 66871 (0.0009) -[2023-10-14 16:13:22,465][75950] Updated weights for policy 1, policy_version 66690 (0.0011) -[2023-10-14 16:13:22,832][75950] Updated weights for policy 1, policy_version 66700 (0.0010) -[2023-10-14 16:13:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 136773632. Throughput: 0: 1673.4, 1: 1668.1. Samples: 34202360. Policy #0 lag: (min: 17.0, avg: 23.6, max: 49.0) -[2023-10-14 16:13:23,165][74987] Avg episode reward: [(0, '24.490'), (1, '32.480')] -[2023-10-14 16:13:23,194][75950] Updated weights for policy 1, policy_version 66710 (0.0010) -[2023-10-14 16:13:23,560][75950] Updated weights for policy 1, policy_version 66720 (0.0010) -[2023-10-14 16:13:26,233][75949] Updated weights for policy 0, policy_version 66881 (0.0008) -[2023-10-14 16:13:26,602][75949] Updated weights for policy 0, policy_version 66891 (0.0010) -[2023-10-14 16:13:26,975][75949] Updated weights for policy 0, policy_version 66901 (0.0010) -[2023-10-14 16:13:27,348][75949] Updated weights for policy 0, policy_version 66911 (0.0009) -[2023-10-14 16:13:27,609][75950] Updated weights for policy 1, policy_version 66730 (0.0009) -[2023-10-14 16:13:27,975][75950] Updated weights for policy 1, policy_version 66740 (0.0010) -[2023-10-14 16:13:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 136839168. Throughput: 0: 1661.5, 1: 1662.8. Samples: 34221710. Policy #0 lag: (min: 17.0, avg: 23.6, max: 49.0) -[2023-10-14 16:13:28,165][74987] Avg episode reward: [(0, '29.470'), (1, '32.650')] -[2023-10-14 16:13:28,330][75950] Updated weights for policy 1, policy_version 66750 (0.0008) -[2023-10-14 16:13:31,459][75949] Updated weights for policy 0, policy_version 66921 (0.0011) -[2023-10-14 16:13:31,828][75949] Updated weights for policy 0, policy_version 66931 (0.0011) -[2023-10-14 16:13:32,200][75949] Updated weights for policy 0, policy_version 66941 (0.0010) -[2023-10-14 16:13:32,454][75950] Updated weights for policy 1, policy_version 66760 (0.0008) -[2023-10-14 16:13:32,824][75950] Updated weights for policy 1, policy_version 66770 (0.0009) -[2023-10-14 16:13:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 136904704. Throughput: 0: 1678.5, 1: 1676.0. Samples: 34232502. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-14 16:13:33,164][74987] Avg episode reward: [(0, '24.130'), (1, '32.820')] -[2023-10-14 16:13:33,185][75950] Updated weights for policy 1, policy_version 66780 (0.0008) -[2023-10-14 16:13:36,139][75949] Updated weights for policy 0, policy_version 66951 (0.0007) -[2023-10-14 16:13:36,513][75949] Updated weights for policy 0, policy_version 66961 (0.0008) -[2023-10-14 16:13:36,893][75949] Updated weights for policy 0, policy_version 66971 (0.0007) -[2023-10-14 16:13:37,176][75950] Updated weights for policy 1, policy_version 66790 (0.0007) -[2023-10-14 16:13:37,542][75950] Updated weights for policy 1, policy_version 66800 (0.0008) -[2023-10-14 16:13:37,911][75950] Updated weights for policy 1, policy_version 66810 (0.0007) -[2023-10-14 16:13:38,164][74987] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 137003008. Throughput: 0: 1667.0, 1: 1681.4. Samples: 34252484. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-14 16:13:38,164][74987] Avg episode reward: [(0, '28.610'), (1, '33.240')] -[2023-10-14 16:13:40,954][75949] Updated weights for policy 0, policy_version 66981 (0.0007) -[2023-10-14 16:13:41,323][75949] Updated weights for policy 0, policy_version 66991 (0.0009) -[2023-10-14 16:13:41,698][75949] Updated weights for policy 0, policy_version 67001 (0.0008) -[2023-10-14 16:13:42,054][75950] Updated weights for policy 1, policy_version 66820 (0.0008) -[2023-10-14 16:13:42,426][75950] Updated weights for policy 1, policy_version 66830 (0.0008) -[2023-10-14 16:13:42,789][75950] Updated weights for policy 1, policy_version 66840 (0.0007) -[2023-10-14 16:13:43,164][74987] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 137068544. Throughput: 0: 1678.7, 1: 1659.2. Samples: 34272192. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-14 16:13:43,164][74987] Avg episode reward: [(0, '24.940'), (1, '33.920')] -[2023-10-14 16:13:45,812][75949] Updated weights for policy 0, policy_version 67011 (0.0008) -[2023-10-14 16:13:46,211][75949] Updated weights for policy 0, policy_version 67021 (0.0008) -[2023-10-14 16:13:46,580][75949] Updated weights for policy 0, policy_version 67031 (0.0008) -[2023-10-14 16:13:46,689][75950] Updated weights for policy 1, policy_version 66850 (0.0007) -[2023-10-14 16:13:47,054][75950] Updated weights for policy 1, policy_version 66860 (0.0007) -[2023-10-14 16:13:47,418][75950] Updated weights for policy 1, policy_version 66870 (0.0008) -[2023-10-14 16:13:47,780][75950] Updated weights for policy 1, policy_version 66880 (0.0009) -[2023-10-14 16:13:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 137134080. Throughput: 0: 1690.1, 1: 1679.3. Samples: 34283364. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-14 16:13:48,164][74987] Avg episode reward: [(0, '27.790'), (1, '32.000')] -[2023-10-14 16:13:50,593][75949] Updated weights for policy 0, policy_version 67041 (0.0009) -[2023-10-14 16:13:50,968][75949] Updated weights for policy 0, policy_version 67051 (0.0010) -[2023-10-14 16:13:51,335][75949] Updated weights for policy 0, policy_version 67061 (0.0009) -[2023-10-14 16:13:51,705][75949] Updated weights for policy 0, policy_version 67071 (0.0008) -[2023-10-14 16:13:51,847][75950] Updated weights for policy 1, policy_version 66890 (0.0007) -[2023-10-14 16:13:52,223][75950] Updated weights for policy 1, policy_version 66900 (0.0007) -[2023-10-14 16:13:52,596][75950] Updated weights for policy 1, policy_version 66910 (0.0007) -[2023-10-14 16:13:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 137199616. Throughput: 0: 1662.4, 1: 1686.2. Samples: 34303062. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-14 16:13:53,165][74987] Avg episode reward: [(0, '23.910'), (1, '34.780')] -[2023-10-14 16:13:55,814][75949] Updated weights for policy 0, policy_version 67081 (0.0008) -[2023-10-14 16:13:56,179][75949] Updated weights for policy 0, policy_version 67091 (0.0007) -[2023-10-14 16:13:56,545][75949] Updated weights for policy 0, policy_version 67101 (0.0008) -[2023-10-14 16:13:56,769][75950] Updated weights for policy 1, policy_version 66920 (0.0008) -[2023-10-14 16:13:57,144][75950] Updated weights for policy 1, policy_version 66930 (0.0008) -[2023-10-14 16:13:57,513][75950] Updated weights for policy 1, policy_version 66940 (0.0007) -[2023-10-14 16:13:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 137265152. Throughput: 0: 1682.8, 1: 1663.1. Samples: 34322590. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-14 16:13:58,164][74987] Avg episode reward: [(0, '27.820'), (1, '34.540')] -[2023-10-14 16:14:00,390][75949] Updated weights for policy 0, policy_version 67111 (0.0008) -[2023-10-14 16:14:00,767][75949] Updated weights for policy 0, policy_version 67121 (0.0007) -[2023-10-14 16:14:01,134][75949] Updated weights for policy 0, policy_version 67131 (0.0007) -[2023-10-14 16:14:01,646][75950] Updated weights for policy 1, policy_version 66950 (0.0010) -[2023-10-14 16:14:02,011][75950] Updated weights for policy 1, policy_version 66960 (0.0009) -[2023-10-14 16:14:02,374][75950] Updated weights for policy 1, policy_version 66970 (0.0010) -[2023-10-14 16:14:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 137330688. Throughput: 0: 1680.5, 1: 1682.2. Samples: 34333718. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-14 16:14:03,164][74987] Avg episode reward: [(0, '24.900'), (1, '32.900')] -[2023-10-14 16:14:05,281][75949] Updated weights for policy 0, policy_version 67141 (0.0009) -[2023-10-14 16:14:05,642][75949] Updated weights for policy 0, policy_version 67151 (0.0007) -[2023-10-14 16:14:06,016][75949] Updated weights for policy 0, policy_version 67161 (0.0009) -[2023-10-14 16:14:06,340][75950] Updated weights for policy 1, policy_version 66980 (0.0010) -[2023-10-14 16:14:06,708][75950] Updated weights for policy 1, policy_version 66990 (0.0009) -[2023-10-14 16:14:07,083][75950] Updated weights for policy 1, policy_version 67000 (0.0007) -[2023-10-14 16:14:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137396224. Throughput: 0: 1673.9, 1: 1682.5. Samples: 34353400. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-14 16:14:08,165][74987] Avg episode reward: [(0, '26.210'), (1, '31.920')] -[2023-10-14 16:14:10,121][75949] Updated weights for policy 0, policy_version 67171 (0.0008) -[2023-10-14 16:14:10,487][75949] Updated weights for policy 0, policy_version 67181 (0.0009) -[2023-10-14 16:14:10,865][75949] Updated weights for policy 0, policy_version 67191 (0.0010) -[2023-10-14 16:14:11,021][75950] Updated weights for policy 1, policy_version 67010 (0.0008) -[2023-10-14 16:14:11,381][75950] Updated weights for policy 1, policy_version 67020 (0.0009) -[2023-10-14 16:14:11,764][75950] Updated weights for policy 1, policy_version 67030 (0.0007) -[2023-10-14 16:14:12,119][75950] Updated weights for policy 1, policy_version 67040 (0.0009) -[2023-10-14 16:14:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137461760. Throughput: 0: 1692.8, 1: 1675.8. Samples: 34373298. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) -[2023-10-14 16:14:13,165][74987] Avg episode reward: [(0, '24.940'), (1, '33.060')] -[2023-10-14 16:14:14,968][75949] Updated weights for policy 0, policy_version 67201 (0.0008) -[2023-10-14 16:14:15,337][75949] Updated weights for policy 0, policy_version 67211 (0.0009) -[2023-10-14 16:14:15,704][75949] Updated weights for policy 0, policy_version 67221 (0.0008) -[2023-10-14 16:14:16,078][75949] Updated weights for policy 0, policy_version 67231 (0.0007) -[2023-10-14 16:14:16,330][75950] Updated weights for policy 1, policy_version 67050 (0.0010) -[2023-10-14 16:14:16,699][75950] Updated weights for policy 1, policy_version 67060 (0.0011) -[2023-10-14 16:14:17,073][75950] Updated weights for policy 1, policy_version 67070 (0.0010) -[2023-10-14 16:14:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137527296. Throughput: 0: 1676.9, 1: 1695.3. Samples: 34384254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:14:18,164][74987] Avg episode reward: [(0, '25.190'), (1, '31.470')] -[2023-10-14 16:14:20,139][75949] Updated weights for policy 0, policy_version 67241 (0.0007) -[2023-10-14 16:14:20,506][75949] Updated weights for policy 0, policy_version 67251 (0.0008) -[2023-10-14 16:14:20,881][75949] Updated weights for policy 0, policy_version 67261 (0.0008) -[2023-10-14 16:14:21,064][75950] Updated weights for policy 1, policy_version 67080 (0.0008) -[2023-10-14 16:14:21,427][75950] Updated weights for policy 1, policy_version 67090 (0.0010) -[2023-10-14 16:14:21,801][75950] Updated weights for policy 1, policy_version 67100 (0.0008) -[2023-10-14 16:14:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137592832. Throughput: 0: 1680.2, 1: 1673.6. Samples: 34403406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:14:23,165][74987] Avg episode reward: [(0, '24.370'), (1, '32.490')] -[2023-10-14 16:14:24,855][75949] Updated weights for policy 0, policy_version 67271 (0.0009) -[2023-10-14 16:14:25,232][75949] Updated weights for policy 0, policy_version 67281 (0.0010) -[2023-10-14 16:14:25,597][75949] Updated weights for policy 0, policy_version 67291 (0.0009) -[2023-10-14 16:14:25,758][75950] Updated weights for policy 1, policy_version 67110 (0.0009) -[2023-10-14 16:14:26,120][75950] Updated weights for policy 1, policy_version 67120 (0.0011) -[2023-10-14 16:14:26,496][75950] Updated weights for policy 1, policy_version 67130 (0.0008) -[2023-10-14 16:14:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 137658368. Throughput: 0: 1691.0, 1: 1685.7. Samples: 34424142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:14:28,164][74987] Avg episode reward: [(0, '23.510'), (1, '35.070')] -[2023-10-14 16:14:29,690][75949] Updated weights for policy 0, policy_version 67301 (0.0007) -[2023-10-14 16:14:30,055][75949] Updated weights for policy 0, policy_version 67311 (0.0007) -[2023-10-14 16:14:30,414][75949] Updated weights for policy 0, policy_version 67321 (0.0007) -[2023-10-14 16:14:30,484][75950] Updated weights for policy 1, policy_version 67140 (0.0008) -[2023-10-14 16:14:30,851][75950] Updated weights for policy 1, policy_version 67150 (0.0009) -[2023-10-14 16:14:31,219][75950] Updated weights for policy 1, policy_version 67160 (0.0009) -[2023-10-14 16:14:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 137723904. Throughput: 0: 1665.5, 1: 1691.6. Samples: 34434432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:14:33,164][74987] Avg episode reward: [(0, '27.200'), (1, '33.220')] -[2023-10-14 16:14:34,570][75949] Updated weights for policy 0, policy_version 67331 (0.0008) -[2023-10-14 16:14:34,968][75949] Updated weights for policy 0, policy_version 67341 (0.0011) -[2023-10-14 16:14:35,310][75950] Updated weights for policy 1, policy_version 67170 (0.0008) -[2023-10-14 16:14:35,343][75949] Updated weights for policy 0, policy_version 67351 (0.0010) -[2023-10-14 16:14:35,668][75950] Updated weights for policy 1, policy_version 67180 (0.0009) -[2023-10-14 16:14:36,036][75950] Updated weights for policy 1, policy_version 67190 (0.0007) -[2023-10-14 16:14:36,402][75950] Updated weights for policy 1, policy_version 67200 (0.0009) -[2023-10-14 16:14:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 137789440. Throughput: 0: 1688.5, 1: 1670.2. Samples: 34454202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:14:38,165][74987] Avg episode reward: [(0, '23.380'), (1, '31.470')] -[2023-10-14 16:14:39,269][75949] Updated weights for policy 0, policy_version 67361 (0.0008) -[2023-10-14 16:14:39,632][75949] Updated weights for policy 0, policy_version 67371 (0.0009) -[2023-10-14 16:14:39,998][75949] Updated weights for policy 0, policy_version 67381 (0.0008) -[2023-10-14 16:14:40,370][75949] Updated weights for policy 0, policy_version 67391 (0.0008) -[2023-10-14 16:14:40,456][75950] Updated weights for policy 1, policy_version 67210 (0.0008) -[2023-10-14 16:14:40,823][75950] Updated weights for policy 1, policy_version 67220 (0.0007) -[2023-10-14 16:14:41,189][75950] Updated weights for policy 1, policy_version 67230 (0.0009) -[2023-10-14 16:14:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 137854976. Throughput: 0: 1690.1, 1: 1696.7. Samples: 34474998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:14:43,165][74987] Avg episode reward: [(0, '26.870'), (1, '34.380')] -[2023-10-14 16:14:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000067392_69009408.pth... -[2023-10-14 16:14:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000067232_68845568.pth... -[2023-10-14 16:14:43,205][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000065824_67403776.pth -[2023-10-14 16:14:43,209][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000065664_67239936.pth -[2023-10-14 16:14:44,426][75949] Updated weights for policy 0, policy_version 67401 (0.0008) -[2023-10-14 16:14:44,797][75949] Updated weights for policy 0, policy_version 67411 (0.0007) -[2023-10-14 16:14:45,165][75949] Updated weights for policy 0, policy_version 67421 (0.0009) -[2023-10-14 16:14:45,269][75950] Updated weights for policy 1, policy_version 67240 (0.0009) -[2023-10-14 16:14:45,636][75950] Updated weights for policy 1, policy_version 67250 (0.0008) -[2023-10-14 16:14:46,001][75950] Updated weights for policy 1, policy_version 67260 (0.0008) -[2023-10-14 16:14:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137920512. Throughput: 0: 1668.5, 1: 1685.1. Samples: 34484632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:14:48,164][74987] Avg episode reward: [(0, '22.640'), (1, '31.140')] -[2023-10-14 16:14:49,328][75949] Updated weights for policy 0, policy_version 67431 (0.0008) -[2023-10-14 16:14:49,698][75949] Updated weights for policy 0, policy_version 67441 (0.0008) -[2023-10-14 16:14:50,061][75949] Updated weights for policy 0, policy_version 67451 (0.0007) -[2023-10-14 16:14:50,131][75950] Updated weights for policy 1, policy_version 67270 (0.0008) -[2023-10-14 16:14:50,494][75950] Updated weights for policy 1, policy_version 67280 (0.0009) -[2023-10-14 16:14:50,866][75950] Updated weights for policy 1, policy_version 67290 (0.0008) -[2023-10-14 16:14:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137986048. Throughput: 0: 1682.0, 1: 1675.2. Samples: 34504474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:14:53,164][74987] Avg episode reward: [(0, '28.080'), (1, '30.400')] -[2023-10-14 16:14:54,215][75949] Updated weights for policy 0, policy_version 67461 (0.0008) -[2023-10-14 16:14:54,581][75949] Updated weights for policy 0, policy_version 67471 (0.0008) -[2023-10-14 16:14:54,952][75949] Updated weights for policy 0, policy_version 67481 (0.0008) -[2023-10-14 16:14:54,963][75950] Updated weights for policy 1, policy_version 67300 (0.0008) -[2023-10-14 16:14:55,332][75950] Updated weights for policy 1, policy_version 67310 (0.0008) -[2023-10-14 16:14:55,699][75950] Updated weights for policy 1, policy_version 67320 (0.0008) -[2023-10-14 16:14:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138051584. Throughput: 0: 1679.8, 1: 1691.4. Samples: 34525002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:14:58,164][74987] Avg episode reward: [(0, '24.040'), (1, '30.810')] -[2023-10-14 16:14:59,015][75949] Updated weights for policy 0, policy_version 67491 (0.0009) -[2023-10-14 16:14:59,387][75949] Updated weights for policy 0, policy_version 67501 (0.0010) -[2023-10-14 16:14:59,743][75949] Updated weights for policy 0, policy_version 67511 (0.0010) -[2023-10-14 16:14:59,832][75950] Updated weights for policy 1, policy_version 67330 (0.0009) -[2023-10-14 16:15:00,199][75950] Updated weights for policy 1, policy_version 67340 (0.0008) -[2023-10-14 16:15:00,562][75950] Updated weights for policy 1, policy_version 67350 (0.0008) -[2023-10-14 16:15:00,925][75950] Updated weights for policy 1, policy_version 67360 (0.0007) -[2023-10-14 16:15:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138117120. Throughput: 0: 1668.0, 1: 1670.0. Samples: 34534462. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 16:15:03,164][74987] Avg episode reward: [(0, '29.780'), (1, '29.930')] -[2023-10-14 16:15:03,727][75949] Updated weights for policy 0, policy_version 67521 (0.0009) -[2023-10-14 16:15:04,087][75949] Updated weights for policy 0, policy_version 67531 (0.0008) -[2023-10-14 16:15:04,457][75949] Updated weights for policy 0, policy_version 67541 (0.0008) -[2023-10-14 16:15:04,831][75949] Updated weights for policy 0, policy_version 67551 (0.0008) -[2023-10-14 16:15:04,910][75950] Updated weights for policy 1, policy_version 67370 (0.0008) -[2023-10-14 16:15:05,279][75950] Updated weights for policy 1, policy_version 67380 (0.0010) -[2023-10-14 16:15:05,652][75950] Updated weights for policy 1, policy_version 67390 (0.0008) -[2023-10-14 16:15:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138182656. Throughput: 0: 1685.2, 1: 1683.0. Samples: 34554972. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 16:15:08,164][74987] Avg episode reward: [(0, '26.520'), (1, '32.140')] -[2023-10-14 16:15:08,972][75949] Updated weights for policy 0, policy_version 67561 (0.0007) -[2023-10-14 16:15:09,345][75949] Updated weights for policy 0, policy_version 67571 (0.0007) -[2023-10-14 16:15:09,709][75949] Updated weights for policy 0, policy_version 67581 (0.0008) -[2023-10-14 16:15:09,714][75950] Updated weights for policy 1, policy_version 67400 (0.0009) -[2023-10-14 16:15:10,085][75950] Updated weights for policy 1, policy_version 67410 (0.0009) -[2023-10-14 16:15:10,443][75950] Updated weights for policy 1, policy_version 67420 (0.0008) -[2023-10-14 16:15:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 138248192. Throughput: 0: 1682.9, 1: 1683.5. Samples: 34575628. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 16:15:13,164][74987] Avg episode reward: [(0, '28.860'), (1, '32.740')] -[2023-10-14 16:15:13,542][75949] Updated weights for policy 0, policy_version 67591 (0.0007) -[2023-10-14 16:15:13,916][75949] Updated weights for policy 0, policy_version 67601 (0.0009) -[2023-10-14 16:15:14,294][75949] Updated weights for policy 0, policy_version 67611 (0.0008) -[2023-10-14 16:15:14,631][75950] Updated weights for policy 1, policy_version 67430 (0.0009) -[2023-10-14 16:15:15,001][75950] Updated weights for policy 1, policy_version 67440 (0.0008) -[2023-10-14 16:15:15,356][75950] Updated weights for policy 1, policy_version 67450 (0.0008) -[2023-10-14 16:15:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138313728. Throughput: 0: 1684.9, 1: 1657.9. Samples: 34584860. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 16:15:18,164][74987] Avg episode reward: [(0, '25.330'), (1, '32.680')] -[2023-10-14 16:15:18,598][75949] Updated weights for policy 0, policy_version 67621 (0.0009) -[2023-10-14 16:15:18,969][75949] Updated weights for policy 0, policy_version 67631 (0.0009) -[2023-10-14 16:15:19,346][75949] Updated weights for policy 0, policy_version 67641 (0.0008) -[2023-10-14 16:15:19,500][75950] Updated weights for policy 1, policy_version 67460 (0.0008) -[2023-10-14 16:15:19,877][75950] Updated weights for policy 1, policy_version 67470 (0.0009) -[2023-10-14 16:15:20,240][75950] Updated weights for policy 1, policy_version 67480 (0.0007) -[2023-10-14 16:15:23,154][75949] Updated weights for policy 0, policy_version 67651 (0.0008) -[2023-10-14 16:15:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138379264. Throughput: 0: 1686.8, 1: 1672.8. Samples: 34605380. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 16:15:23,164][74987] Avg episode reward: [(0, '26.560'), (1, '31.880')] -[2023-10-14 16:15:23,550][75949] Updated weights for policy 0, policy_version 67661 (0.0009) -[2023-10-14 16:15:23,914][75949] Updated weights for policy 0, policy_version 67671 (0.0009) -[2023-10-14 16:15:24,219][75950] Updated weights for policy 1, policy_version 67490 (0.0007) -[2023-10-14 16:15:24,581][75950] Updated weights for policy 1, policy_version 67500 (0.0008) -[2023-10-14 16:15:24,953][75950] Updated weights for policy 1, policy_version 67510 (0.0007) -[2023-10-14 16:15:25,312][75950] Updated weights for policy 1, policy_version 67520 (0.0007) -[2023-10-14 16:15:28,021][75949] Updated weights for policy 0, policy_version 67681 (0.0008) -[2023-10-14 16:15:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138444800. Throughput: 0: 1683.4, 1: 1676.4. Samples: 34626188. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 16:15:28,165][74987] Avg episode reward: [(0, '25.000'), (1, '30.540')] -[2023-10-14 16:15:28,388][75949] Updated weights for policy 0, policy_version 67691 (0.0009) -[2023-10-14 16:15:28,753][75949] Updated weights for policy 0, policy_version 67701 (0.0008) -[2023-10-14 16:15:29,130][75949] Updated weights for policy 0, policy_version 67711 (0.0007) -[2023-10-14 16:15:29,306][75950] Updated weights for policy 1, policy_version 67530 (0.0010) -[2023-10-14 16:15:29,673][75950] Updated weights for policy 1, policy_version 67540 (0.0009) -[2023-10-14 16:15:30,032][75950] Updated weights for policy 1, policy_version 67550 (0.0009) -[2023-10-14 16:15:33,145][75949] Updated weights for policy 0, policy_version 67721 (0.0008) -[2023-10-14 16:15:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138510336. Throughput: 0: 1686.3, 1: 1660.5. Samples: 34635238. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 16:15:33,164][74987] Avg episode reward: [(0, '26.190'), (1, '31.330')] -[2023-10-14 16:15:33,514][75949] Updated weights for policy 0, policy_version 67731 (0.0008) -[2023-10-14 16:15:33,888][75949] Updated weights for policy 0, policy_version 67741 (0.0007) -[2023-10-14 16:15:34,210][75950] Updated weights for policy 1, policy_version 67560 (0.0008) -[2023-10-14 16:15:34,568][75950] Updated weights for policy 1, policy_version 67570 (0.0010) -[2023-10-14 16:15:34,931][75950] Updated weights for policy 1, policy_version 67580 (0.0008) -[2023-10-14 16:15:37,831][75949] Updated weights for policy 0, policy_version 67751 (0.0008) -[2023-10-14 16:15:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138575872. Throughput: 0: 1693.5, 1: 1675.5. Samples: 34656080. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 16:15:38,164][74987] Avg episode reward: [(0, '27.350'), (1, '33.360')] -[2023-10-14 16:15:38,195][75949] Updated weights for policy 0, policy_version 67761 (0.0008) -[2023-10-14 16:15:38,567][75949] Updated weights for policy 0, policy_version 67771 (0.0007) -[2023-10-14 16:15:39,128][75950] Updated weights for policy 1, policy_version 67590 (0.0010) -[2023-10-14 16:15:39,495][75950] Updated weights for policy 1, policy_version 67600 (0.0009) -[2023-10-14 16:15:39,858][75950] Updated weights for policy 1, policy_version 67610 (0.0010) -[2023-10-14 16:15:42,867][75949] Updated weights for policy 0, policy_version 67781 (0.0008) -[2023-10-14 16:15:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 138641408. Throughput: 0: 1698.1, 1: 1671.0. Samples: 34676612. Policy #0 lag: (min: 10.0, avg: 10.1, max: 16.0) -[2023-10-14 16:15:43,164][74987] Avg episode reward: [(0, '26.390'), (1, '32.710')] -[2023-10-14 16:15:43,231][75949] Updated weights for policy 0, policy_version 67791 (0.0009) -[2023-10-14 16:15:43,602][75949] Updated weights for policy 0, policy_version 67801 (0.0007) -[2023-10-14 16:15:43,899][75950] Updated weights for policy 1, policy_version 67620 (0.0010) -[2023-10-14 16:15:44,272][75950] Updated weights for policy 1, policy_version 67630 (0.0009) -[2023-10-14 16:15:44,643][75950] Updated weights for policy 1, policy_version 67640 (0.0007) -[2023-10-14 16:15:47,676][75949] Updated weights for policy 0, policy_version 67811 (0.0007) -[2023-10-14 16:15:48,047][75949] Updated weights for policy 0, policy_version 67821 (0.0009) -[2023-10-14 16:15:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138706944. Throughput: 0: 1698.1, 1: 1667.2. Samples: 34685902. Policy #0 lag: (min: 8.0, avg: 32.2, max: 40.0) -[2023-10-14 16:15:48,164][74987] Avg episode reward: [(0, '26.880'), (1, '33.390')] -[2023-10-14 16:15:48,418][75949] Updated weights for policy 0, policy_version 67831 (0.0008) -[2023-10-14 16:15:48,819][75950] Updated weights for policy 1, policy_version 67650 (0.0007) -[2023-10-14 16:15:49,190][75950] Updated weights for policy 1, policy_version 67660 (0.0009) -[2023-10-14 16:15:49,550][75950] Updated weights for policy 1, policy_version 67670 (0.0009) -[2023-10-14 16:15:49,917][75950] Updated weights for policy 1, policy_version 67680 (0.0009) -[2023-10-14 16:15:52,489][75949] Updated weights for policy 0, policy_version 67841 (0.0009) -[2023-10-14 16:15:52,853][75949] Updated weights for policy 0, policy_version 67851 (0.0007) -[2023-10-14 16:15:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138772480. Throughput: 0: 1688.9, 1: 1676.5. Samples: 34706418. Policy #0 lag: (min: 8.0, avg: 32.2, max: 40.0) -[2023-10-14 16:15:53,164][74987] Avg episode reward: [(0, '23.670'), (1, '32.750')] -[2023-10-14 16:15:53,224][75949] Updated weights for policy 0, policy_version 67861 (0.0007) -[2023-10-14 16:15:53,592][75949] Updated weights for policy 0, policy_version 67871 (0.0008) -[2023-10-14 16:15:53,968][75950] Updated weights for policy 1, policy_version 67690 (0.0008) -[2023-10-14 16:15:54,342][75950] Updated weights for policy 1, policy_version 67700 (0.0009) -[2023-10-14 16:15:54,708][75950] Updated weights for policy 1, policy_version 67710 (0.0009) -[2023-10-14 16:15:57,560][75949] Updated weights for policy 0, policy_version 67881 (0.0008) -[2023-10-14 16:15:57,940][75949] Updated weights for policy 0, policy_version 67891 (0.0008) -[2023-10-14 16:15:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138838016. Throughput: 0: 1682.3, 1: 1679.0. Samples: 34726886. Policy #0 lag: (min: 8.0, avg: 32.2, max: 40.0) -[2023-10-14 16:15:58,164][74987] Avg episode reward: [(0, '28.560'), (1, '31.170')] -[2023-10-14 16:15:58,305][75949] Updated weights for policy 0, policy_version 67901 (0.0007) -[2023-10-14 16:15:58,746][75950] Updated weights for policy 1, policy_version 67720 (0.0007) -[2023-10-14 16:15:59,102][75950] Updated weights for policy 1, policy_version 67730 (0.0009) -[2023-10-14 16:15:59,467][75950] Updated weights for policy 1, policy_version 67740 (0.0010) -[2023-10-14 16:16:02,558][75949] Updated weights for policy 0, policy_version 67911 (0.0008) -[2023-10-14 16:16:02,929][75949] Updated weights for policy 0, policy_version 67921 (0.0011) -[2023-10-14 16:16:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138903552. Throughput: 0: 1687.6, 1: 1681.5. Samples: 34736472. Policy #0 lag: (min: 8.0, avg: 32.2, max: 40.0) -[2023-10-14 16:16:03,165][74987] Avg episode reward: [(0, '26.630'), (1, '32.640')] -[2023-10-14 16:16:03,290][75949] Updated weights for policy 0, policy_version 67931 (0.0012) -[2023-10-14 16:16:03,543][75950] Updated weights for policy 1, policy_version 67750 (0.0008) -[2023-10-14 16:16:03,919][75950] Updated weights for policy 1, policy_version 67760 (0.0008) -[2023-10-14 16:16:04,295][75950] Updated weights for policy 1, policy_version 67770 (0.0009) -[2023-10-14 16:16:07,370][75949] Updated weights for policy 0, policy_version 67941 (0.0009) -[2023-10-14 16:16:07,737][75949] Updated weights for policy 0, policy_version 67951 (0.0008) -[2023-10-14 16:16:08,118][75949] Updated weights for policy 0, policy_version 67961 (0.0007) -[2023-10-14 16:16:08,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138969088. Throughput: 0: 1684.0, 1: 1685.1. Samples: 34756986. Policy #0 lag: (min: 8.0, avg: 32.2, max: 40.0) -[2023-10-14 16:16:08,164][74987] Avg episode reward: [(0, '29.330'), (1, '32.450')] -[2023-10-14 16:16:08,387][75950] Updated weights for policy 1, policy_version 67780 (0.0008) -[2023-10-14 16:16:08,747][75950] Updated weights for policy 1, policy_version 67790 (0.0010) -[2023-10-14 16:16:09,116][75950] Updated weights for policy 1, policy_version 67800 (0.0010) -[2023-10-14 16:16:11,986][75949] Updated weights for policy 0, policy_version 67971 (0.0008) -[2023-10-14 16:16:12,389][75949] Updated weights for policy 0, policy_version 67981 (0.0010) -[2023-10-14 16:16:12,751][75949] Updated weights for policy 0, policy_version 67991 (0.0010) -[2023-10-14 16:16:13,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 139067392. Throughput: 0: 1672.4, 1: 1677.7. Samples: 34776942. Policy #0 lag: (min: 8.0, avg: 32.2, max: 40.0) -[2023-10-14 16:16:13,164][74987] Avg episode reward: [(0, '25.110'), (1, '33.290')] -[2023-10-14 16:16:13,193][75950] Updated weights for policy 1, policy_version 67810 (0.0008) -[2023-10-14 16:16:13,572][75950] Updated weights for policy 1, policy_version 67820 (0.0009) -[2023-10-14 16:16:13,935][75950] Updated weights for policy 1, policy_version 67830 (0.0009) -[2023-10-14 16:16:14,308][75950] Updated weights for policy 1, policy_version 67840 (0.0009) -[2023-10-14 16:16:16,710][75949] Updated weights for policy 0, policy_version 68001 (0.0009) -[2023-10-14 16:16:17,069][75949] Updated weights for policy 0, policy_version 68011 (0.0009) -[2023-10-14 16:16:17,436][75949] Updated weights for policy 0, policy_version 68021 (0.0007) -[2023-10-14 16:16:17,800][75949] Updated weights for policy 0, policy_version 68031 (0.0009) -[2023-10-14 16:16:18,163][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 139132928. Throughput: 0: 1692.3, 1: 1677.9. Samples: 34786898. Policy #0 lag: (min: 8.0, avg: 32.2, max: 40.0) -[2023-10-14 16:16:18,164][74987] Avg episode reward: [(0, '26.850'), (1, '31.260')] -[2023-10-14 16:16:18,572][75950] Updated weights for policy 1, policy_version 67850 (0.0010) -[2023-10-14 16:16:18,935][75950] Updated weights for policy 1, policy_version 67860 (0.0008) -[2023-10-14 16:16:19,302][75950] Updated weights for policy 1, policy_version 67870 (0.0008) -[2023-10-14 16:16:21,813][75949] Updated weights for policy 0, policy_version 68041 (0.0009) -[2023-10-14 16:16:22,193][75949] Updated weights for policy 0, policy_version 68051 (0.0011) -[2023-10-14 16:16:22,555][75949] Updated weights for policy 0, policy_version 68061 (0.0011) -[2023-10-14 16:16:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 139198464. Throughput: 0: 1682.6, 1: 1679.6. Samples: 34807382. Policy #0 lag: (min: 8.0, avg: 32.2, max: 40.0) -[2023-10-14 16:16:23,165][74987] Avg episode reward: [(0, '25.920'), (1, '32.130')] -[2023-10-14 16:16:23,426][75950] Updated weights for policy 1, policy_version 67880 (0.0007) -[2023-10-14 16:16:23,788][75950] Updated weights for policy 1, policy_version 67890 (0.0009) -[2023-10-14 16:16:24,155][75950] Updated weights for policy 1, policy_version 67900 (0.0008) -[2023-10-14 16:16:26,490][75949] Updated weights for policy 0, policy_version 68071 (0.0009) -[2023-10-14 16:16:26,850][75949] Updated weights for policy 0, policy_version 68081 (0.0008) -[2023-10-14 16:16:27,225][75949] Updated weights for policy 0, policy_version 68091 (0.0009) -[2023-10-14 16:16:28,113][75950] Updated weights for policy 1, policy_version 67910 (0.0009) -[2023-10-14 16:16:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 139264000. Throughput: 0: 1661.5, 1: 1686.3. Samples: 34827262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:16:28,164][74987] Avg episode reward: [(0, '26.630'), (1, '33.090')] -[2023-10-14 16:16:28,481][75950] Updated weights for policy 1, policy_version 67920 (0.0008) -[2023-10-14 16:16:28,838][75950] Updated weights for policy 1, policy_version 67930 (0.0010) -[2023-10-14 16:16:31,265][75949] Updated weights for policy 0, policy_version 68101 (0.0009) -[2023-10-14 16:16:31,640][75949] Updated weights for policy 0, policy_version 68111 (0.0009) -[2023-10-14 16:16:31,997][75949] Updated weights for policy 0, policy_version 68121 (0.0008) -[2023-10-14 16:16:32,854][75950] Updated weights for policy 1, policy_version 67940 (0.0010) -[2023-10-14 16:16:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 139329536. Throughput: 0: 1693.7, 1: 1681.8. Samples: 34837798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:16:33,164][74987] Avg episode reward: [(0, '25.870'), (1, '30.800')] -[2023-10-14 16:16:33,222][75950] Updated weights for policy 1, policy_version 67950 (0.0009) -[2023-10-14 16:16:33,581][75950] Updated weights for policy 1, policy_version 67960 (0.0008) -[2023-10-14 16:16:36,045][75949] Updated weights for policy 0, policy_version 68131 (0.0008) -[2023-10-14 16:16:36,418][75949] Updated weights for policy 0, policy_version 68141 (0.0008) -[2023-10-14 16:16:36,784][75949] Updated weights for policy 0, policy_version 68151 (0.0009) -[2023-10-14 16:16:37,763][75950] Updated weights for policy 1, policy_version 67970 (0.0009) -[2023-10-14 16:16:38,133][75950] Updated weights for policy 1, policy_version 67980 (0.0008) -[2023-10-14 16:16:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 139395072. Throughput: 0: 1680.1, 1: 1683.4. Samples: 34857776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:16:38,165][74987] Avg episode reward: [(0, '28.020'), (1, '31.850')] -[2023-10-14 16:16:38,510][75950] Updated weights for policy 1, policy_version 67990 (0.0009) -[2023-10-14 16:16:38,879][75950] Updated weights for policy 1, policy_version 68000 (0.0010) -[2023-10-14 16:16:40,897][75949] Updated weights for policy 0, policy_version 68161 (0.0007) -[2023-10-14 16:16:41,268][75949] Updated weights for policy 0, policy_version 68171 (0.0009) -[2023-10-14 16:16:41,639][75949] Updated weights for policy 0, policy_version 68181 (0.0009) -[2023-10-14 16:16:42,003][75949] Updated weights for policy 0, policy_version 68191 (0.0007) -[2023-10-14 16:16:42,920][75950] Updated weights for policy 1, policy_version 68010 (0.0007) -[2023-10-14 16:16:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 139460608. Throughput: 0: 1672.8, 1: 1683.0. Samples: 34877898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:16:43,165][74987] Avg episode reward: [(0, '24.730'), (1, '34.080')] -[2023-10-14 16:16:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000068192_69828608.pth... -[2023-10-14 16:16:43,210][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000066624_68222976.pth -[2023-10-14 16:16:43,296][75950] Updated weights for policy 1, policy_version 68020 (0.0007) -[2023-10-14 16:16:43,670][75950] Updated weights for policy 1, policy_version 68030 (0.0010) -[2023-10-14 16:16:43,732][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000068032_69664768.pth... -[2023-10-14 16:16:43,762][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000066464_68059136.pth -[2023-10-14 16:16:46,193][75949] Updated weights for policy 0, policy_version 68201 (0.0007) -[2023-10-14 16:16:46,559][75949] Updated weights for policy 0, policy_version 68211 (0.0008) -[2023-10-14 16:16:46,928][75949] Updated weights for policy 0, policy_version 68221 (0.0010) -[2023-10-14 16:16:47,648][75950] Updated weights for policy 1, policy_version 68040 (0.0007) -[2023-10-14 16:16:48,012][75950] Updated weights for policy 1, policy_version 68050 (0.0008) -[2023-10-14 16:16:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 139526144. Throughput: 0: 1691.2, 1: 1680.4. Samples: 34888192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:16:48,164][74987] Avg episode reward: [(0, '27.970'), (1, '32.840')] -[2023-10-14 16:16:48,385][75950] Updated weights for policy 1, policy_version 68060 (0.0010) -[2023-10-14 16:16:50,947][75949] Updated weights for policy 0, policy_version 68231 (0.0009) -[2023-10-14 16:16:51,326][75949] Updated weights for policy 0, policy_version 68241 (0.0009) -[2023-10-14 16:16:51,691][75949] Updated weights for policy 0, policy_version 68251 (0.0007) -[2023-10-14 16:16:52,598][75950] Updated weights for policy 1, policy_version 68070 (0.0008) -[2023-10-14 16:16:52,966][75950] Updated weights for policy 1, policy_version 68080 (0.0007) -[2023-10-14 16:16:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 139591680. Throughput: 0: 1671.7, 1: 1677.7. Samples: 34907710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:16:53,164][74987] Avg episode reward: [(0, '26.660'), (1, '32.110')] -[2023-10-14 16:16:53,329][75950] Updated weights for policy 1, policy_version 68090 (0.0007) -[2023-10-14 16:16:55,671][75949] Updated weights for policy 0, policy_version 68261 (0.0007) -[2023-10-14 16:16:56,043][75949] Updated weights for policy 0, policy_version 68271 (0.0009) -[2023-10-14 16:16:56,413][75949] Updated weights for policy 0, policy_version 68281 (0.0011) -[2023-10-14 16:16:57,400][75950] Updated weights for policy 1, policy_version 68100 (0.0007) -[2023-10-14 16:16:57,769][75950] Updated weights for policy 1, policy_version 68110 (0.0009) -[2023-10-14 16:16:58,135][75950] Updated weights for policy 1, policy_version 68120 (0.0009) -[2023-10-14 16:16:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 139657216. Throughput: 0: 1682.7, 1: 1673.2. Samples: 34927958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:16:58,164][74987] Avg episode reward: [(0, '28.300'), (1, '33.350')] -[2023-10-14 16:17:00,493][75949] Updated weights for policy 0, policy_version 68291 (0.0009) -[2023-10-14 16:17:00,886][75949] Updated weights for policy 0, policy_version 68301 (0.0007) -[2023-10-14 16:17:01,257][75949] Updated weights for policy 0, policy_version 68311 (0.0008) -[2023-10-14 16:17:02,270][75950] Updated weights for policy 1, policy_version 68130 (0.0008) -[2023-10-14 16:17:02,646][75950] Updated weights for policy 1, policy_version 68140 (0.0008) -[2023-10-14 16:17:03,013][75950] Updated weights for policy 1, policy_version 68150 (0.0007) -[2023-10-14 16:17:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 139722752. Throughput: 0: 1682.0, 1: 1681.9. Samples: 34938278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:17:03,165][74987] Avg episode reward: [(0, '25.530'), (1, '32.860')] -[2023-10-14 16:17:03,379][75950] Updated weights for policy 1, policy_version 68160 (0.0009) -[2023-10-14 16:17:05,281][75949] Updated weights for policy 0, policy_version 68321 (0.0010) -[2023-10-14 16:17:05,656][75949] Updated weights for policy 0, policy_version 68331 (0.0008) -[2023-10-14 16:17:06,025][75949] Updated weights for policy 0, policy_version 68341 (0.0008) -[2023-10-14 16:17:06,399][75949] Updated weights for policy 0, policy_version 68351 (0.0008) -[2023-10-14 16:17:07,385][75950] Updated weights for policy 1, policy_version 68170 (0.0009) -[2023-10-14 16:17:07,758][75950] Updated weights for policy 1, policy_version 68180 (0.0007) -[2023-10-14 16:17:08,119][75950] Updated weights for policy 1, policy_version 68190 (0.0007) -[2023-10-14 16:17:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 139788288. Throughput: 0: 1664.1, 1: 1680.8. Samples: 34957906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:17:08,165][74987] Avg episode reward: [(0, '27.470'), (1, '32.170')] -[2023-10-14 16:17:10,450][75949] Updated weights for policy 0, policy_version 68361 (0.0007) -[2023-10-14 16:17:10,818][75949] Updated weights for policy 0, policy_version 68371 (0.0011) -[2023-10-14 16:17:11,183][75949] Updated weights for policy 0, policy_version 68381 (0.0009) -[2023-10-14 16:17:12,127][75950] Updated weights for policy 1, policy_version 68200 (0.0008) -[2023-10-14 16:17:12,498][75950] Updated weights for policy 1, policy_version 68210 (0.0008) -[2023-10-14 16:17:12,863][75950] Updated weights for policy 1, policy_version 68220 (0.0009) -[2023-10-14 16:17:13,163][74987] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 139886592. Throughput: 0: 1684.8, 1: 1659.5. Samples: 34977752. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 16:17:13,164][74987] Avg episode reward: [(0, '25.350'), (1, '34.660')] -[2023-10-14 16:17:15,260][75949] Updated weights for policy 0, policy_version 68391 (0.0007) -[2023-10-14 16:17:15,631][75949] Updated weights for policy 0, policy_version 68401 (0.0008) -[2023-10-14 16:17:16,005][75949] Updated weights for policy 0, policy_version 68411 (0.0008) -[2023-10-14 16:17:16,803][75950] Updated weights for policy 1, policy_version 68230 (0.0008) -[2023-10-14 16:17:17,175][75950] Updated weights for policy 1, policy_version 68240 (0.0007) -[2023-10-14 16:17:17,543][75950] Updated weights for policy 1, policy_version 68250 (0.0009) -[2023-10-14 16:17:18,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 139952128. Throughput: 0: 1667.1, 1: 1680.4. Samples: 34988438. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 16:17:18,165][74987] Avg episode reward: [(0, '27.270'), (1, '33.520')] -[2023-10-14 16:17:20,155][75949] Updated weights for policy 0, policy_version 68421 (0.0007) -[2023-10-14 16:17:20,530][75949] Updated weights for policy 0, policy_version 68431 (0.0007) -[2023-10-14 16:17:20,901][75949] Updated weights for policy 0, policy_version 68441 (0.0007) -[2023-10-14 16:17:21,629][75950] Updated weights for policy 1, policy_version 68260 (0.0009) -[2023-10-14 16:17:21,996][75950] Updated weights for policy 1, policy_version 68270 (0.0010) -[2023-10-14 16:17:22,361][75950] Updated weights for policy 1, policy_version 68280 (0.0007) -[2023-10-14 16:17:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 140017664. Throughput: 0: 1673.2, 1: 1674.0. Samples: 35008400. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 16:17:23,164][74987] Avg episode reward: [(0, '25.830'), (1, '32.930')] -[2023-10-14 16:17:25,026][75949] Updated weights for policy 0, policy_version 68451 (0.0008) -[2023-10-14 16:17:25,395][75949] Updated weights for policy 0, policy_version 68461 (0.0011) -[2023-10-14 16:17:25,765][75949] Updated weights for policy 0, policy_version 68471 (0.0009) -[2023-10-14 16:17:26,369][75950] Updated weights for policy 1, policy_version 68290 (0.0008) -[2023-10-14 16:17:26,729][75950] Updated weights for policy 1, policy_version 68300 (0.0008) -[2023-10-14 16:17:27,097][75950] Updated weights for policy 1, policy_version 68310 (0.0007) -[2023-10-14 16:17:27,474][75950] Updated weights for policy 1, policy_version 68320 (0.0007) -[2023-10-14 16:17:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 140083200. Throughput: 0: 1681.3, 1: 1650.3. Samples: 35027818. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 16:17:28,164][74987] Avg episode reward: [(0, '27.420'), (1, '32.050')] -[2023-10-14 16:17:29,847][75949] Updated weights for policy 0, policy_version 68481 (0.0009) -[2023-10-14 16:17:30,222][75949] Updated weights for policy 0, policy_version 68491 (0.0008) -[2023-10-14 16:17:30,592][75949] Updated weights for policy 0, policy_version 68501 (0.0007) -[2023-10-14 16:17:30,961][75949] Updated weights for policy 0, policy_version 68511 (0.0007) -[2023-10-14 16:17:31,714][75950] Updated weights for policy 1, policy_version 68330 (0.0010) -[2023-10-14 16:17:32,086][75950] Updated weights for policy 1, policy_version 68340 (0.0008) -[2023-10-14 16:17:32,453][75950] Updated weights for policy 1, policy_version 68350 (0.0008) -[2023-10-14 16:17:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 140148736. Throughput: 0: 1663.9, 1: 1679.1. Samples: 35038628. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 16:17:33,165][74987] Avg episode reward: [(0, '25.810'), (1, '30.370')] -[2023-10-14 16:17:35,021][75949] Updated weights for policy 0, policy_version 68521 (0.0010) -[2023-10-14 16:17:35,396][75949] Updated weights for policy 0, policy_version 68531 (0.0009) -[2023-10-14 16:17:35,765][75949] Updated weights for policy 0, policy_version 68541 (0.0008) -[2023-10-14 16:17:36,804][75950] Updated weights for policy 1, policy_version 68360 (0.0008) -[2023-10-14 16:17:37,173][75950] Updated weights for policy 1, policy_version 68370 (0.0009) -[2023-10-14 16:17:37,544][75950] Updated weights for policy 1, policy_version 68380 (0.0007) -[2023-10-14 16:17:38,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 140214272. Throughput: 0: 1678.2, 1: 1671.1. Samples: 35058430. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 16:17:38,164][74987] Avg episode reward: [(0, '27.120'), (1, '32.670')] -[2023-10-14 16:17:39,945][75949] Updated weights for policy 0, policy_version 68551 (0.0009) -[2023-10-14 16:17:40,312][75949] Updated weights for policy 0, policy_version 68561 (0.0009) -[2023-10-14 16:17:40,677][75949] Updated weights for policy 0, policy_version 68571 (0.0010) -[2023-10-14 16:17:41,556][75950] Updated weights for policy 1, policy_version 68390 (0.0007) -[2023-10-14 16:17:41,928][75950] Updated weights for policy 1, policy_version 68400 (0.0009) -[2023-10-14 16:17:42,294][75950] Updated weights for policy 1, policy_version 68410 (0.0008) -[2023-10-14 16:17:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 140279808. Throughput: 0: 1679.5, 1: 1656.4. Samples: 35078076. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 16:17:43,165][74987] Avg episode reward: [(0, '27.140'), (1, '32.460')] -[2023-10-14 16:17:44,689][75949] Updated weights for policy 0, policy_version 68581 (0.0008) -[2023-10-14 16:17:45,061][75949] Updated weights for policy 0, policy_version 68591 (0.0009) -[2023-10-14 16:17:45,430][75949] Updated weights for policy 0, policy_version 68601 (0.0007) -[2023-10-14 16:17:46,250][75950] Updated weights for policy 1, policy_version 68420 (0.0009) -[2023-10-14 16:17:46,612][75950] Updated weights for policy 1, policy_version 68430 (0.0010) -[2023-10-14 16:17:46,985][75950] Updated weights for policy 1, policy_version 68440 (0.0011) -[2023-10-14 16:17:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 140345344. Throughput: 0: 1660.6, 1: 1680.0. Samples: 35088604. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 16:17:48,164][74987] Avg episode reward: [(0, '26.750'), (1, '35.510')] -[2023-10-14 16:17:49,488][75949] Updated weights for policy 0, policy_version 68611 (0.0007) -[2023-10-14 16:17:49,856][75949] Updated weights for policy 0, policy_version 68621 (0.0009) -[2023-10-14 16:17:50,231][75949] Updated weights for policy 0, policy_version 68631 (0.0010) -[2023-10-14 16:17:51,152][75950] Updated weights for policy 1, policy_version 68450 (0.0009) -[2023-10-14 16:17:51,515][75950] Updated weights for policy 1, policy_version 68460 (0.0011) -[2023-10-14 16:17:51,883][75950] Updated weights for policy 1, policy_version 68470 (0.0010) -[2023-10-14 16:17:52,252][75950] Updated weights for policy 1, policy_version 68480 (0.0010) -[2023-10-14 16:17:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 140410880. Throughput: 0: 1681.2, 1: 1670.7. Samples: 35108742. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) -[2023-10-14 16:17:53,164][74987] Avg episode reward: [(0, '25.540'), (1, '33.890')] -[2023-10-14 16:17:54,344][75949] Updated weights for policy 0, policy_version 68641 (0.0008) -[2023-10-14 16:17:54,765][75949] Updated weights for policy 0, policy_version 68651 (0.0008) -[2023-10-14 16:17:55,126][75949] Updated weights for policy 0, policy_version 68661 (0.0008) -[2023-10-14 16:17:55,491][75949] Updated weights for policy 0, policy_version 68671 (0.0009) -[2023-10-14 16:17:56,214][75950] Updated weights for policy 1, policy_version 68490 (0.0008) -[2023-10-14 16:17:56,571][75950] Updated weights for policy 1, policy_version 68500 (0.0007) -[2023-10-14 16:17:56,941][75950] Updated weights for policy 1, policy_version 68510 (0.0008) -[2023-10-14 16:17:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 140476416. Throughput: 0: 1679.2, 1: 1672.7. Samples: 35128586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:17:58,164][74987] Avg episode reward: [(0, '25.880'), (1, '33.990')] -[2023-10-14 16:17:59,635][75949] Updated weights for policy 0, policy_version 68681 (0.0010) -[2023-10-14 16:18:00,007][75949] Updated weights for policy 0, policy_version 68691 (0.0011) -[2023-10-14 16:18:00,385][75949] Updated weights for policy 0, policy_version 68701 (0.0009) -[2023-10-14 16:18:00,797][75950] Updated weights for policy 1, policy_version 68520 (0.0009) -[2023-10-14 16:18:01,172][75950] Updated weights for policy 1, policy_version 68530 (0.0009) -[2023-10-14 16:18:01,541][75950] Updated weights for policy 1, policy_version 68540 (0.0009) -[2023-10-14 16:18:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 140541952. Throughput: 0: 1659.1, 1: 1678.3. Samples: 35138618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:03,165][74987] Avg episode reward: [(0, '25.560'), (1, '35.160')] -[2023-10-14 16:18:04,496][75949] Updated weights for policy 0, policy_version 68711 (0.0008) -[2023-10-14 16:18:04,866][75949] Updated weights for policy 0, policy_version 68721 (0.0009) -[2023-10-14 16:18:05,229][75949] Updated weights for policy 0, policy_version 68731 (0.0009) -[2023-10-14 16:18:05,604][75950] Updated weights for policy 1, policy_version 68550 (0.0010) -[2023-10-14 16:18:05,967][75950] Updated weights for policy 1, policy_version 68560 (0.0007) -[2023-10-14 16:18:06,338][75950] Updated weights for policy 1, policy_version 68570 (0.0010) -[2023-10-14 16:18:08,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 140607488. Throughput: 0: 1668.3, 1: 1661.4. Samples: 35158236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:08,164][74987] Avg episode reward: [(0, '25.250'), (1, '34.170')] -[2023-10-14 16:18:09,258][75949] Updated weights for policy 0, policy_version 68741 (0.0008) -[2023-10-14 16:18:09,634][75949] Updated weights for policy 0, policy_version 68751 (0.0008) -[2023-10-14 16:18:09,999][75949] Updated weights for policy 0, policy_version 68761 (0.0008) -[2023-10-14 16:18:10,424][75950] Updated weights for policy 1, policy_version 68580 (0.0008) -[2023-10-14 16:18:10,788][75950] Updated weights for policy 1, policy_version 68590 (0.0008) -[2023-10-14 16:18:11,149][75950] Updated weights for policy 1, policy_version 68600 (0.0008) -[2023-10-14 16:18:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 140673024. Throughput: 0: 1673.3, 1: 1685.5. Samples: 35178964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:13,164][74987] Avg episode reward: [(0, '26.690'), (1, '33.780')] -[2023-10-14 16:18:14,183][75949] Updated weights for policy 0, policy_version 68771 (0.0009) -[2023-10-14 16:18:14,545][75949] Updated weights for policy 0, policy_version 68781 (0.0008) -[2023-10-14 16:18:14,913][75949] Updated weights for policy 0, policy_version 68791 (0.0011) -[2023-10-14 16:18:15,373][75950] Updated weights for policy 1, policy_version 68610 (0.0009) -[2023-10-14 16:18:15,741][75950] Updated weights for policy 1, policy_version 68620 (0.0007) -[2023-10-14 16:18:16,104][75950] Updated weights for policy 1, policy_version 68630 (0.0007) -[2023-10-14 16:18:16,471][75950] Updated weights for policy 1, policy_version 68640 (0.0007) -[2023-10-14 16:18:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 140738560. Throughput: 0: 1662.8, 1: 1677.0. Samples: 35188920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:18,164][74987] Avg episode reward: [(0, '25.930'), (1, '34.500')] -[2023-10-14 16:18:19,205][75949] Updated weights for policy 0, policy_version 68801 (0.0009) -[2023-10-14 16:18:19,575][75949] Updated weights for policy 0, policy_version 68811 (0.0007) -[2023-10-14 16:18:19,958][75949] Updated weights for policy 0, policy_version 68821 (0.0008) -[2023-10-14 16:18:20,329][75949] Updated weights for policy 0, policy_version 68831 (0.0008) -[2023-10-14 16:18:20,628][75950] Updated weights for policy 1, policy_version 68650 (0.0010) -[2023-10-14 16:18:21,000][75950] Updated weights for policy 1, policy_version 68660 (0.0008) -[2023-10-14 16:18:21,365][75950] Updated weights for policy 1, policy_version 68670 (0.0009) -[2023-10-14 16:18:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 140804096. Throughput: 0: 1667.7, 1: 1667.1. Samples: 35208498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:23,165][74987] Avg episode reward: [(0, '29.490'), (1, '37.190')] -[2023-10-14 16:18:23,166][75801] Saving new best policy, reward=37.190! -[2023-10-14 16:18:24,387][75949] Updated weights for policy 0, policy_version 68841 (0.0009) -[2023-10-14 16:18:24,751][75949] Updated weights for policy 0, policy_version 68851 (0.0009) -[2023-10-14 16:18:25,120][75949] Updated weights for policy 0, policy_version 68861 (0.0009) -[2023-10-14 16:18:25,381][75950] Updated weights for policy 1, policy_version 68680 (0.0008) -[2023-10-14 16:18:25,744][75950] Updated weights for policy 1, policy_version 68690 (0.0010) -[2023-10-14 16:18:26,115][75950] Updated weights for policy 1, policy_version 68700 (0.0008) -[2023-10-14 16:18:28,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 140869632. Throughput: 0: 1672.8, 1: 1689.1. Samples: 35229358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:28,165][74987] Avg episode reward: [(0, '26.000'), (1, '35.970')] -[2023-10-14 16:18:29,096][75949] Updated weights for policy 0, policy_version 68871 (0.0009) -[2023-10-14 16:18:29,456][75949] Updated weights for policy 0, policy_version 68881 (0.0010) -[2023-10-14 16:18:29,822][75949] Updated weights for policy 0, policy_version 68891 (0.0009) -[2023-10-14 16:18:30,046][75950] Updated weights for policy 1, policy_version 68710 (0.0010) -[2023-10-14 16:18:30,408][75950] Updated weights for policy 1, policy_version 68720 (0.0010) -[2023-10-14 16:18:30,778][75950] Updated weights for policy 1, policy_version 68730 (0.0009) -[2023-10-14 16:18:33,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 140935168. Throughput: 0: 1672.3, 1: 1671.9. Samples: 35239094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:33,164][74987] Avg episode reward: [(0, '28.130'), (1, '34.630')] -[2023-10-14 16:18:33,730][75949] Updated weights for policy 0, policy_version 68901 (0.0010) -[2023-10-14 16:18:34,097][75949] Updated weights for policy 0, policy_version 68911 (0.0008) -[2023-10-14 16:18:34,467][75949] Updated weights for policy 0, policy_version 68921 (0.0010) -[2023-10-14 16:18:34,937][75950] Updated weights for policy 1, policy_version 68740 (0.0009) -[2023-10-14 16:18:35,310][75950] Updated weights for policy 1, policy_version 68750 (0.0008) -[2023-10-14 16:18:35,683][75950] Updated weights for policy 1, policy_version 68760 (0.0009) -[2023-10-14 16:18:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141000704. Throughput: 0: 1679.5, 1: 1673.8. Samples: 35259640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:38,164][74987] Avg episode reward: [(0, '27.060'), (1, '34.940')] -[2023-10-14 16:18:38,490][75949] Updated weights for policy 0, policy_version 68931 (0.0008) -[2023-10-14 16:18:38,854][75949] Updated weights for policy 0, policy_version 68941 (0.0008) -[2023-10-14 16:18:39,227][75949] Updated weights for policy 0, policy_version 68951 (0.0007) -[2023-10-14 16:18:39,769][75950] Updated weights for policy 1, policy_version 68770 (0.0010) -[2023-10-14 16:18:40,135][75950] Updated weights for policy 1, policy_version 68780 (0.0007) -[2023-10-14 16:18:40,503][75950] Updated weights for policy 1, policy_version 68790 (0.0010) -[2023-10-14 16:18:40,863][75950] Updated weights for policy 1, policy_version 68800 (0.0010) -[2023-10-14 16:18:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141066240. Throughput: 0: 1683.5, 1: 1690.8. Samples: 35280434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:43,164][74987] Avg episode reward: [(0, '27.650'), (1, '33.130')] -[2023-10-14 16:18:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000068800_70451200.pth... -[2023-10-14 16:18:43,208][75949] Updated weights for policy 0, policy_version 68961 (0.0010) -[2023-10-14 16:18:43,214][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000067232_68845568.pth -[2023-10-14 16:18:43,628][75949] Updated weights for policy 0, policy_version 68971 (0.0011) -[2023-10-14 16:18:44,003][75949] Updated weights for policy 0, policy_version 68981 (0.0011) -[2023-10-14 16:18:44,369][75949] Updated weights for policy 0, policy_version 68991 (0.0011) -[2023-10-14 16:18:44,406][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000068992_70647808.pth... -[2023-10-14 16:18:44,447][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000067392_69009408.pth -[2023-10-14 16:18:44,734][75950] Updated weights for policy 1, policy_version 68810 (0.0008) -[2023-10-14 16:18:45,100][75950] Updated weights for policy 1, policy_version 68820 (0.0009) -[2023-10-14 16:18:45,463][75950] Updated weights for policy 1, policy_version 68830 (0.0008) -[2023-10-14 16:18:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141131776. Throughput: 0: 1685.5, 1: 1667.5. Samples: 35289502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:48,164][74987] Avg episode reward: [(0, '26.470'), (1, '33.250')] -[2023-10-14 16:18:48,687][75949] Updated weights for policy 0, policy_version 69001 (0.0009) -[2023-10-14 16:18:49,043][75949] Updated weights for policy 0, policy_version 69011 (0.0009) -[2023-10-14 16:18:49,417][75949] Updated weights for policy 0, policy_version 69021 (0.0007) -[2023-10-14 16:18:49,522][75950] Updated weights for policy 1, policy_version 68840 (0.0009) -[2023-10-14 16:18:49,892][75950] Updated weights for policy 1, policy_version 68850 (0.0010) -[2023-10-14 16:18:50,247][75950] Updated weights for policy 1, policy_version 68860 (0.0009) -[2023-10-14 16:18:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141197312. Throughput: 0: 1682.9, 1: 1688.8. Samples: 35309962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:53,164][74987] Avg episode reward: [(0, '27.660'), (1, '32.740')] -[2023-10-14 16:18:53,494][75949] Updated weights for policy 0, policy_version 69031 (0.0009) -[2023-10-14 16:18:53,864][75949] Updated weights for policy 0, policy_version 69041 (0.0010) -[2023-10-14 16:18:54,233][75949] Updated weights for policy 0, policy_version 69051 (0.0007) -[2023-10-14 16:18:54,359][75950] Updated weights for policy 1, policy_version 68870 (0.0008) -[2023-10-14 16:18:54,735][75950] Updated weights for policy 1, policy_version 68880 (0.0011) -[2023-10-14 16:18:55,098][75950] Updated weights for policy 1, policy_version 68890 (0.0010) -[2023-10-14 16:18:57,959][75949] Updated weights for policy 0, policy_version 69061 (0.0009) -[2023-10-14 16:18:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 141262848. Throughput: 0: 1685.2, 1: 1685.1. Samples: 35330628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:18:58,165][74987] Avg episode reward: [(0, '26.320'), (1, '31.900')] -[2023-10-14 16:18:58,326][75949] Updated weights for policy 0, policy_version 69071 (0.0008) -[2023-10-14 16:18:58,691][75949] Updated weights for policy 0, policy_version 69081 (0.0007) -[2023-10-14 16:18:59,361][75950] Updated weights for policy 1, policy_version 68900 (0.0008) -[2023-10-14 16:18:59,725][75950] Updated weights for policy 1, policy_version 68910 (0.0008) -[2023-10-14 16:19:00,097][75950] Updated weights for policy 1, policy_version 68920 (0.0007) -[2023-10-14 16:19:02,849][75949] Updated weights for policy 0, policy_version 69091 (0.0010) -[2023-10-14 16:19:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141328384. Throughput: 0: 1686.0, 1: 1666.6. Samples: 35339788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:03,165][74987] Avg episode reward: [(0, '27.390'), (1, '34.800')] -[2023-10-14 16:19:03,219][75949] Updated weights for policy 0, policy_version 69101 (0.0008) -[2023-10-14 16:19:03,600][75949] Updated weights for policy 0, policy_version 69111 (0.0009) -[2023-10-14 16:19:04,146][75950] Updated weights for policy 1, policy_version 68930 (0.0007) -[2023-10-14 16:19:04,517][75950] Updated weights for policy 1, policy_version 68940 (0.0008) -[2023-10-14 16:19:04,890][75950] Updated weights for policy 1, policy_version 68950 (0.0009) -[2023-10-14 16:19:05,254][75950] Updated weights for policy 1, policy_version 68960 (0.0009) -[2023-10-14 16:19:07,630][75949] Updated weights for policy 0, policy_version 69121 (0.0009) -[2023-10-14 16:19:07,991][75949] Updated weights for policy 0, policy_version 69131 (0.0007) -[2023-10-14 16:19:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141393920. Throughput: 0: 1691.6, 1: 1685.6. Samples: 35360470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:08,164][74987] Avg episode reward: [(0, '26.290'), (1, '34.960')] -[2023-10-14 16:19:08,364][75949] Updated weights for policy 0, policy_version 69141 (0.0008) -[2023-10-14 16:19:08,734][75949] Updated weights for policy 0, policy_version 69151 (0.0010) -[2023-10-14 16:19:09,393][75950] Updated weights for policy 1, policy_version 68970 (0.0007) -[2023-10-14 16:19:09,755][75950] Updated weights for policy 1, policy_version 68980 (0.0010) -[2023-10-14 16:19:10,121][75950] Updated weights for policy 1, policy_version 68990 (0.0007) -[2023-10-14 16:19:12,701][75949] Updated weights for policy 0, policy_version 69161 (0.0009) -[2023-10-14 16:19:13,070][75949] Updated weights for policy 0, policy_version 69171 (0.0008) -[2023-10-14 16:19:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141459456. Throughput: 0: 1685.2, 1: 1681.6. Samples: 35380860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:13,164][74987] Avg episode reward: [(0, '27.000'), (1, '33.400')] -[2023-10-14 16:19:13,437][75949] Updated weights for policy 0, policy_version 69181 (0.0009) -[2023-10-14 16:19:14,347][75950] Updated weights for policy 1, policy_version 69000 (0.0007) -[2023-10-14 16:19:14,708][75950] Updated weights for policy 1, policy_version 69010 (0.0008) -[2023-10-14 16:19:15,070][75950] Updated weights for policy 1, policy_version 69020 (0.0010) -[2023-10-14 16:19:17,516][75949] Updated weights for policy 0, policy_version 69191 (0.0009) -[2023-10-14 16:19:17,888][75949] Updated weights for policy 0, policy_version 69201 (0.0011) -[2023-10-14 16:19:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141524992. Throughput: 0: 1692.0, 1: 1664.7. Samples: 35390146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:18,165][74987] Avg episode reward: [(0, '25.710'), (1, '34.170')] -[2023-10-14 16:19:18,263][75949] Updated weights for policy 0, policy_version 69211 (0.0008) -[2023-10-14 16:19:19,135][75950] Updated weights for policy 1, policy_version 69030 (0.0008) -[2023-10-14 16:19:19,496][75950] Updated weights for policy 1, policy_version 69040 (0.0008) -[2023-10-14 16:19:19,860][75950] Updated weights for policy 1, policy_version 69050 (0.0008) -[2023-10-14 16:19:22,226][75949] Updated weights for policy 0, policy_version 69221 (0.0009) -[2023-10-14 16:19:22,599][75949] Updated weights for policy 0, policy_version 69231 (0.0008) -[2023-10-14 16:19:22,974][75949] Updated weights for policy 0, policy_version 69241 (0.0008) -[2023-10-14 16:19:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141590528. Throughput: 0: 1686.4, 1: 1678.9. Samples: 35411080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:23,164][74987] Avg episode reward: [(0, '27.680'), (1, '35.810')] -[2023-10-14 16:19:23,897][75950] Updated weights for policy 1, policy_version 69060 (0.0009) -[2023-10-14 16:19:24,269][75950] Updated weights for policy 1, policy_version 69070 (0.0007) -[2023-10-14 16:19:24,638][75950] Updated weights for policy 1, policy_version 69080 (0.0009) -[2023-10-14 16:19:27,076][75949] Updated weights for policy 0, policy_version 69251 (0.0009) -[2023-10-14 16:19:27,441][75949] Updated weights for policy 0, policy_version 69261 (0.0008) -[2023-10-14 16:19:27,816][75949] Updated weights for policy 0, policy_version 69271 (0.0008) -[2023-10-14 16:19:28,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 141688832. Throughput: 0: 1669.2, 1: 1679.6. Samples: 35431132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:28,165][74987] Avg episode reward: [(0, '24.390'), (1, '33.410')] -[2023-10-14 16:19:28,756][75950] Updated weights for policy 1, policy_version 69090 (0.0008) -[2023-10-14 16:19:29,125][75950] Updated weights for policy 1, policy_version 69100 (0.0008) -[2023-10-14 16:19:29,487][75950] Updated weights for policy 1, policy_version 69110 (0.0008) -[2023-10-14 16:19:29,853][75950] Updated weights for policy 1, policy_version 69120 (0.0009) -[2023-10-14 16:19:31,931][75949] Updated weights for policy 0, policy_version 69281 (0.0010) -[2023-10-14 16:19:32,346][75949] Updated weights for policy 0, policy_version 69291 (0.0008) -[2023-10-14 16:19:32,722][75949] Updated weights for policy 0, policy_version 69301 (0.0008) -[2023-10-14 16:19:33,080][75949] Updated weights for policy 0, policy_version 69311 (0.0008) -[2023-10-14 16:19:33,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 141754368. Throughput: 0: 1690.9, 1: 1676.4. Samples: 35441032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:33,165][74987] Avg episode reward: [(0, '27.860'), (1, '32.610')] -[2023-10-14 16:19:33,818][75950] Updated weights for policy 1, policy_version 69130 (0.0009) -[2023-10-14 16:19:34,192][75950] Updated weights for policy 1, policy_version 69140 (0.0008) -[2023-10-14 16:19:34,557][75950] Updated weights for policy 1, policy_version 69150 (0.0007) -[2023-10-14 16:19:37,268][75949] Updated weights for policy 0, policy_version 69321 (0.0007) -[2023-10-14 16:19:37,641][75949] Updated weights for policy 0, policy_version 69331 (0.0009) -[2023-10-14 16:19:38,011][75949] Updated weights for policy 0, policy_version 69341 (0.0007) -[2023-10-14 16:19:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 141819904. Throughput: 0: 1690.0, 1: 1678.6. Samples: 35461546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:38,164][74987] Avg episode reward: [(0, '25.250'), (1, '33.270')] -[2023-10-14 16:19:38,711][75950] Updated weights for policy 1, policy_version 69160 (0.0008) -[2023-10-14 16:19:39,073][75950] Updated weights for policy 1, policy_version 69170 (0.0010) -[2023-10-14 16:19:39,445][75950] Updated weights for policy 1, policy_version 69180 (0.0009) -[2023-10-14 16:19:42,044][75949] Updated weights for policy 0, policy_version 69351 (0.0010) -[2023-10-14 16:19:42,407][75949] Updated weights for policy 0, policy_version 69361 (0.0007) -[2023-10-14 16:19:42,785][75949] Updated weights for policy 0, policy_version 69371 (0.0007) -[2023-10-14 16:19:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 141885440. Throughput: 0: 1662.7, 1: 1684.4. Samples: 35481248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:43,165][74987] Avg episode reward: [(0, '27.360'), (1, '33.050')] -[2023-10-14 16:19:43,488][75950] Updated weights for policy 1, policy_version 69190 (0.0008) -[2023-10-14 16:19:43,852][75950] Updated weights for policy 1, policy_version 69200 (0.0008) -[2023-10-14 16:19:44,215][75950] Updated weights for policy 1, policy_version 69210 (0.0010) -[2023-10-14 16:19:46,766][75949] Updated weights for policy 0, policy_version 69381 (0.0007) -[2023-10-14 16:19:47,139][75949] Updated weights for policy 0, policy_version 69391 (0.0009) -[2023-10-14 16:19:47,505][75949] Updated weights for policy 0, policy_version 69401 (0.0010) -[2023-10-14 16:19:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 141950976. Throughput: 0: 1684.1, 1: 1681.7. Samples: 35491246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:48,164][74987] Avg episode reward: [(0, '25.180'), (1, '33.240')] -[2023-10-14 16:19:48,290][75950] Updated weights for policy 1, policy_version 69220 (0.0010) -[2023-10-14 16:19:48,660][75950] Updated weights for policy 1, policy_version 69230 (0.0009) -[2023-10-14 16:19:49,016][75950] Updated weights for policy 1, policy_version 69240 (0.0008) -[2023-10-14 16:19:51,590][75949] Updated weights for policy 0, policy_version 69411 (0.0010) -[2023-10-14 16:19:51,969][75949] Updated weights for policy 0, policy_version 69421 (0.0007) -[2023-10-14 16:19:52,338][75949] Updated weights for policy 0, policy_version 69431 (0.0008) -[2023-10-14 16:19:53,106][75950] Updated weights for policy 1, policy_version 69250 (0.0008) -[2023-10-14 16:19:53,164][74987] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 142016512. Throughput: 0: 1683.1, 1: 1679.2. Samples: 35511774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:53,164][74987] Avg episode reward: [(0, '27.690'), (1, '34.810')] -[2023-10-14 16:19:53,481][75950] Updated weights for policy 1, policy_version 69260 (0.0010) -[2023-10-14 16:19:53,851][75950] Updated weights for policy 1, policy_version 69270 (0.0009) -[2023-10-14 16:19:54,218][75950] Updated weights for policy 1, policy_version 69280 (0.0010) -[2023-10-14 16:19:56,492][75949] Updated weights for policy 0, policy_version 69441 (0.0009) -[2023-10-14 16:19:56,854][75949] Updated weights for policy 0, policy_version 69451 (0.0008) -[2023-10-14 16:19:57,227][75949] Updated weights for policy 0, policy_version 69461 (0.0009) -[2023-10-14 16:19:57,605][75949] Updated weights for policy 0, policy_version 69471 (0.0010) -[2023-10-14 16:19:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 142082048. Throughput: 0: 1662.1, 1: 1685.5. Samples: 35531504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:19:58,165][74987] Avg episode reward: [(0, '27.240'), (1, '34.160')] -[2023-10-14 16:19:58,204][75950] Updated weights for policy 1, policy_version 69290 (0.0009) -[2023-10-14 16:19:58,568][75950] Updated weights for policy 1, policy_version 69300 (0.0007) -[2023-10-14 16:19:58,937][75950] Updated weights for policy 1, policy_version 69310 (0.0008) -[2023-10-14 16:20:01,561][75949] Updated weights for policy 0, policy_version 69481 (0.0009) -[2023-10-14 16:20:01,931][75949] Updated weights for policy 0, policy_version 69491 (0.0008) -[2023-10-14 16:20:02,295][75949] Updated weights for policy 0, policy_version 69501 (0.0007) -[2023-10-14 16:20:02,985][75950] Updated weights for policy 1, policy_version 69320 (0.0008) -[2023-10-14 16:20:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 142147584. Throughput: 0: 1684.4, 1: 1686.7. Samples: 35541846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:20:03,164][74987] Avg episode reward: [(0, '28.050'), (1, '32.270')] -[2023-10-14 16:20:03,352][75950] Updated weights for policy 1, policy_version 69330 (0.0008) -[2023-10-14 16:20:03,710][75950] Updated weights for policy 1, policy_version 69340 (0.0009) -[2023-10-14 16:20:06,349][75949] Updated weights for policy 0, policy_version 69511 (0.0007) -[2023-10-14 16:20:06,706][75949] Updated weights for policy 0, policy_version 69521 (0.0009) -[2023-10-14 16:20:07,085][75949] Updated weights for policy 0, policy_version 69531 (0.0009) -[2023-10-14 16:20:07,854][75950] Updated weights for policy 1, policy_version 69350 (0.0008) -[2023-10-14 16:20:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 142213120. Throughput: 0: 1670.0, 1: 1681.4. Samples: 35561894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:20:08,164][74987] Avg episode reward: [(0, '24.680'), (1, '32.150')] -[2023-10-14 16:20:08,227][75950] Updated weights for policy 1, policy_version 69360 (0.0007) -[2023-10-14 16:20:08,593][75950] Updated weights for policy 1, policy_version 69370 (0.0007) -[2023-10-14 16:20:10,992][75949] Updated weights for policy 0, policy_version 69541 (0.0008) -[2023-10-14 16:20:11,365][75949] Updated weights for policy 0, policy_version 69551 (0.0009) -[2023-10-14 16:20:11,727][75949] Updated weights for policy 0, policy_version 69561 (0.0008) -[2023-10-14 16:20:12,706][75950] Updated weights for policy 1, policy_version 69380 (0.0008) -[2023-10-14 16:20:13,081][75950] Updated weights for policy 1, policy_version 69390 (0.0008) -[2023-10-14 16:20:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 142278656. Throughput: 0: 1672.1, 1: 1676.6. Samples: 35581822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:20:13,165][74987] Avg episode reward: [(0, '28.040'), (1, '31.990')] -[2023-10-14 16:20:13,456][75950] Updated weights for policy 1, policy_version 69400 (0.0008) -[2023-10-14 16:20:15,771][75949] Updated weights for policy 0, policy_version 69571 (0.0008) -[2023-10-14 16:20:16,139][75949] Updated weights for policy 0, policy_version 69581 (0.0008) -[2023-10-14 16:20:16,518][75949] Updated weights for policy 0, policy_version 69591 (0.0009) -[2023-10-14 16:20:17,584][75950] Updated weights for policy 1, policy_version 69410 (0.0009) -[2023-10-14 16:20:17,956][75950] Updated weights for policy 1, policy_version 69420 (0.0008) -[2023-10-14 16:20:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 142344192. Throughput: 0: 1684.3, 1: 1677.1. Samples: 35592294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:20:18,165][74987] Avg episode reward: [(0, '25.330'), (1, '34.990')] -[2023-10-14 16:20:18,322][75950] Updated weights for policy 1, policy_version 69430 (0.0007) -[2023-10-14 16:20:18,684][75950] Updated weights for policy 1, policy_version 69440 (0.0007) -[2023-10-14 16:20:20,713][75949] Updated weights for policy 0, policy_version 69601 (0.0007) -[2023-10-14 16:20:21,092][75949] Updated weights for policy 0, policy_version 69611 (0.0009) -[2023-10-14 16:20:21,458][75949] Updated weights for policy 0, policy_version 69621 (0.0008) -[2023-10-14 16:20:21,831][75949] Updated weights for policy 0, policy_version 69631 (0.0008) -[2023-10-14 16:20:22,939][75950] Updated weights for policy 1, policy_version 69450 (0.0008) -[2023-10-14 16:20:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 142409728. Throughput: 0: 1665.2, 1: 1672.6. Samples: 35611746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:20:23,164][74987] Avg episode reward: [(0, '25.580'), (1, '34.170')] -[2023-10-14 16:20:23,308][75950] Updated weights for policy 1, policy_version 69460 (0.0009) -[2023-10-14 16:20:23,669][75950] Updated weights for policy 1, policy_version 69470 (0.0008) -[2023-10-14 16:20:25,938][75949] Updated weights for policy 0, policy_version 69641 (0.0009) -[2023-10-14 16:20:26,309][75949] Updated weights for policy 0, policy_version 69651 (0.0009) -[2023-10-14 16:20:26,675][75949] Updated weights for policy 0, policy_version 69661 (0.0011) -[2023-10-14 16:20:27,826][75950] Updated weights for policy 1, policy_version 69480 (0.0008) -[2023-10-14 16:20:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 142475264. Throughput: 0: 1688.7, 1: 1667.1. Samples: 35632258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:20:28,165][74987] Avg episode reward: [(0, '24.640'), (1, '34.110')] -[2023-10-14 16:20:28,190][75950] Updated weights for policy 1, policy_version 69490 (0.0007) -[2023-10-14 16:20:28,556][75950] Updated weights for policy 1, policy_version 69500 (0.0007) -[2023-10-14 16:20:30,640][75949] Updated weights for policy 0, policy_version 69671 (0.0008) -[2023-10-14 16:20:31,023][75949] Updated weights for policy 0, policy_version 69681 (0.0009) -[2023-10-14 16:20:31,390][75949] Updated weights for policy 0, policy_version 69691 (0.0008) -[2023-10-14 16:20:32,678][75950] Updated weights for policy 1, policy_version 69510 (0.0008) -[2023-10-14 16:20:33,044][75950] Updated weights for policy 1, policy_version 69520 (0.0007) -[2023-10-14 16:20:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 142540800. Throughput: 0: 1689.2, 1: 1668.6. Samples: 35642348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:20:33,164][74987] Avg episode reward: [(0, '25.340'), (1, '34.600')] -[2023-10-14 16:20:33,417][75950] Updated weights for policy 1, policy_version 69530 (0.0007) -[2023-10-14 16:20:35,474][75949] Updated weights for policy 0, policy_version 69701 (0.0010) -[2023-10-14 16:20:35,844][75949] Updated weights for policy 0, policy_version 69711 (0.0008) -[2023-10-14 16:20:36,216][75949] Updated weights for policy 0, policy_version 69721 (0.0008) -[2023-10-14 16:20:37,319][75950] Updated weights for policy 1, policy_version 69540 (0.0008) -[2023-10-14 16:20:37,688][75950] Updated weights for policy 1, policy_version 69550 (0.0008) -[2023-10-14 16:20:38,053][75950] Updated weights for policy 1, policy_version 69560 (0.0009) -[2023-10-14 16:20:38,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 142606336. Throughput: 0: 1670.0, 1: 1670.4. Samples: 35662090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:20:38,164][74987] Avg episode reward: [(0, '26.800'), (1, '36.270')] -[2023-10-14 16:20:40,187][75949] Updated weights for policy 0, policy_version 69731 (0.0008) -[2023-10-14 16:20:40,554][75949] Updated weights for policy 0, policy_version 69741 (0.0009) -[2023-10-14 16:20:40,924][75949] Updated weights for policy 0, policy_version 69751 (0.0007) -[2023-10-14 16:20:42,259][75950] Updated weights for policy 1, policy_version 69570 (0.0009) -[2023-10-14 16:20:42,633][75950] Updated weights for policy 1, policy_version 69580 (0.0008) -[2023-10-14 16:20:43,002][75950] Updated weights for policy 1, policy_version 69590 (0.0009) -[2023-10-14 16:20:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 142671872. Throughput: 0: 1697.2, 1: 1659.4. Samples: 35682554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:20:43,165][74987] Avg episode reward: [(0, '25.130'), (1, '33.090')] -[2023-10-14 16:20:43,174][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000069760_71434240.pth... -[2023-10-14 16:20:43,217][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000068192_69828608.pth -[2023-10-14 16:20:43,363][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000069600_71270400.pth... -[2023-10-14 16:20:43,363][75950] Updated weights for policy 1, policy_version 69600 (0.0009) -[2023-10-14 16:20:43,392][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000068032_69664768.pth -[2023-10-14 16:20:45,026][75949] Updated weights for policy 0, policy_version 69761 (0.0008) -[2023-10-14 16:20:45,401][75949] Updated weights for policy 0, policy_version 69771 (0.0010) -[2023-10-14 16:20:45,772][75949] Updated weights for policy 0, policy_version 69781 (0.0009) -[2023-10-14 16:20:46,147][75949] Updated weights for policy 0, policy_version 69791 (0.0009) -[2023-10-14 16:20:47,434][75950] Updated weights for policy 1, policy_version 69610 (0.0008) -[2023-10-14 16:20:47,805][75950] Updated weights for policy 1, policy_version 69620 (0.0008) -[2023-10-14 16:20:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 142737408. Throughput: 0: 1677.5, 1: 1671.0. Samples: 35692526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:20:48,164][74987] Avg episode reward: [(0, '27.330'), (1, '35.410')] -[2023-10-14 16:20:48,167][75950] Updated weights for policy 1, policy_version 69630 (0.0010) -[2023-10-14 16:20:50,229][75949] Updated weights for policy 0, policy_version 69801 (0.0007) -[2023-10-14 16:20:50,599][75949] Updated weights for policy 0, policy_version 69811 (0.0008) -[2023-10-14 16:20:50,974][75949] Updated weights for policy 0, policy_version 69821 (0.0009) -[2023-10-14 16:20:52,244][75950] Updated weights for policy 1, policy_version 69640 (0.0008) -[2023-10-14 16:20:52,625][75950] Updated weights for policy 1, policy_version 69650 (0.0008) -[2023-10-14 16:20:52,990][75950] Updated weights for policy 1, policy_version 69660 (0.0007) -[2023-10-14 16:20:53,164][74987] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 142835712. Throughput: 0: 1678.4, 1: 1667.4. Samples: 35712454. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) -[2023-10-14 16:20:53,164][74987] Avg episode reward: [(0, '25.080'), (1, '33.700')] -[2023-10-14 16:20:55,027][75949] Updated weights for policy 0, policy_version 69831 (0.0009) -[2023-10-14 16:20:55,397][75949] Updated weights for policy 0, policy_version 69841 (0.0008) -[2023-10-14 16:20:55,765][75949] Updated weights for policy 0, policy_version 69851 (0.0010) -[2023-10-14 16:20:56,964][75950] Updated weights for policy 1, policy_version 69670 (0.0007) -[2023-10-14 16:20:57,334][75950] Updated weights for policy 1, policy_version 69680 (0.0010) -[2023-10-14 16:20:57,692][75950] Updated weights for policy 1, policy_version 69690 (0.0009) -[2023-10-14 16:20:58,163][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 142901248. Throughput: 0: 1694.0, 1: 1653.5. Samples: 35732460. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) -[2023-10-14 16:20:58,164][74987] Avg episode reward: [(0, '28.210'), (1, '33.350')] -[2023-10-14 16:20:59,863][75949] Updated weights for policy 0, policy_version 69861 (0.0009) -[2023-10-14 16:21:00,234][75949] Updated weights for policy 0, policy_version 69871 (0.0007) -[2023-10-14 16:21:00,611][75949] Updated weights for policy 0, policy_version 69881 (0.0007) -[2023-10-14 16:21:01,712][75950] Updated weights for policy 1, policy_version 69700 (0.0008) -[2023-10-14 16:21:02,084][75950] Updated weights for policy 1, policy_version 69710 (0.0008) -[2023-10-14 16:21:02,448][75950] Updated weights for policy 1, policy_version 69720 (0.0009) -[2023-10-14 16:21:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 142966784. Throughput: 0: 1670.4, 1: 1674.6. Samples: 35742818. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) -[2023-10-14 16:21:03,164][74987] Avg episode reward: [(0, '25.510'), (1, '33.080')] -[2023-10-14 16:21:04,589][75949] Updated weights for policy 0, policy_version 69891 (0.0009) -[2023-10-14 16:21:04,956][75949] Updated weights for policy 0, policy_version 69901 (0.0009) -[2023-10-14 16:21:05,332][75949] Updated weights for policy 0, policy_version 69911 (0.0010) -[2023-10-14 16:21:06,656][75950] Updated weights for policy 1, policy_version 69730 (0.0007) -[2023-10-14 16:21:07,019][75950] Updated weights for policy 1, policy_version 69740 (0.0008) -[2023-10-14 16:21:07,393][75950] Updated weights for policy 1, policy_version 69750 (0.0007) -[2023-10-14 16:21:07,761][75950] Updated weights for policy 1, policy_version 69760 (0.0009) -[2023-10-14 16:21:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 143032320. Throughput: 0: 1693.9, 1: 1675.9. Samples: 35763386. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) -[2023-10-14 16:21:08,164][74987] Avg episode reward: [(0, '28.350'), (1, '32.740')] -[2023-10-14 16:21:09,368][75949] Updated weights for policy 0, policy_version 69921 (0.0009) -[2023-10-14 16:21:09,744][75949] Updated weights for policy 0, policy_version 69931 (0.0008) -[2023-10-14 16:21:10,113][75949] Updated weights for policy 0, policy_version 69941 (0.0010) -[2023-10-14 16:21:10,488][75949] Updated weights for policy 0, policy_version 69951 (0.0008) -[2023-10-14 16:21:11,788][75950] Updated weights for policy 1, policy_version 69770 (0.0008) -[2023-10-14 16:21:12,161][75950] Updated weights for policy 1, policy_version 69780 (0.0008) -[2023-10-14 16:21:12,526][75950] Updated weights for policy 1, policy_version 69790 (0.0009) -[2023-10-14 16:21:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 143097856. Throughput: 0: 1690.6, 1: 1654.3. Samples: 35782778. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) -[2023-10-14 16:21:13,164][74987] Avg episode reward: [(0, '27.340'), (1, '33.210')] -[2023-10-14 16:21:14,537][75949] Updated weights for policy 0, policy_version 69961 (0.0009) -[2023-10-14 16:21:14,910][75949] Updated weights for policy 0, policy_version 69971 (0.0009) -[2023-10-14 16:21:15,278][75949] Updated weights for policy 0, policy_version 69981 (0.0007) -[2023-10-14 16:21:16,729][75950] Updated weights for policy 1, policy_version 69800 (0.0008) -[2023-10-14 16:21:17,099][75950] Updated weights for policy 1, policy_version 69810 (0.0009) -[2023-10-14 16:21:17,458][75950] Updated weights for policy 1, policy_version 69820 (0.0007) -[2023-10-14 16:21:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 143163392. Throughput: 0: 1665.5, 1: 1679.5. Samples: 35792874. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) -[2023-10-14 16:21:18,165][74987] Avg episode reward: [(0, '27.930'), (1, '34.280')] -[2023-10-14 16:21:19,288][75949] Updated weights for policy 0, policy_version 69991 (0.0011) -[2023-10-14 16:21:19,657][75949] Updated weights for policy 0, policy_version 70001 (0.0010) -[2023-10-14 16:21:20,033][75949] Updated weights for policy 0, policy_version 70011 (0.0008) -[2023-10-14 16:21:21,536][75950] Updated weights for policy 1, policy_version 69830 (0.0007) -[2023-10-14 16:21:21,905][75950] Updated weights for policy 1, policy_version 69840 (0.0009) -[2023-10-14 16:21:22,279][75950] Updated weights for policy 1, policy_version 69850 (0.0009) -[2023-10-14 16:21:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 143228928. Throughput: 0: 1692.5, 1: 1671.2. Samples: 35813458. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) -[2023-10-14 16:21:23,164][74987] Avg episode reward: [(0, '26.230'), (1, '33.800')] -[2023-10-14 16:21:23,911][75949] Updated weights for policy 0, policy_version 70021 (0.0009) -[2023-10-14 16:21:24,272][75949] Updated weights for policy 0, policy_version 70031 (0.0009) -[2023-10-14 16:21:24,640][75949] Updated weights for policy 0, policy_version 70041 (0.0009) -[2023-10-14 16:21:26,383][75950] Updated weights for policy 1, policy_version 69860 (0.0008) -[2023-10-14 16:21:26,741][75950] Updated weights for policy 1, policy_version 69870 (0.0010) -[2023-10-14 16:21:27,115][75950] Updated weights for policy 1, policy_version 69880 (0.0007) -[2023-10-14 16:21:28,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 143294464. Throughput: 0: 1694.2, 1: 1661.2. Samples: 35833548. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) -[2023-10-14 16:21:28,164][74987] Avg episode reward: [(0, '28.950'), (1, '32.810')] -[2023-10-14 16:21:28,711][75949] Updated weights for policy 0, policy_version 70051 (0.0010) -[2023-10-14 16:21:29,070][75949] Updated weights for policy 0, policy_version 70061 (0.0010) -[2023-10-14 16:21:29,448][75949] Updated weights for policy 0, policy_version 70071 (0.0010) -[2023-10-14 16:21:31,035][75950] Updated weights for policy 1, policy_version 69890 (0.0008) -[2023-10-14 16:21:31,403][75950] Updated weights for policy 1, policy_version 69900 (0.0009) -[2023-10-14 16:21:31,774][75950] Updated weights for policy 1, policy_version 69910 (0.0007) -[2023-10-14 16:21:32,134][75950] Updated weights for policy 1, policy_version 69920 (0.0007) -[2023-10-14 16:21:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 143360000. Throughput: 0: 1681.6, 1: 1681.1. Samples: 35843846. Policy #0 lag: (min: 24.0, avg: 49.9, max: 56.0) -[2023-10-14 16:21:33,164][74987] Avg episode reward: [(0, '26.250'), (1, '34.360')] -[2023-10-14 16:21:33,662][75949] Updated weights for policy 0, policy_version 70081 (0.0010) -[2023-10-14 16:21:34,029][75949] Updated weights for policy 0, policy_version 70091 (0.0009) -[2023-10-14 16:21:34,392][75949] Updated weights for policy 0, policy_version 70101 (0.0008) -[2023-10-14 16:21:34,760][75949] Updated weights for policy 0, policy_version 70111 (0.0008) -[2023-10-14 16:21:36,060][75950] Updated weights for policy 1, policy_version 69930 (0.0010) -[2023-10-14 16:21:36,433][75950] Updated weights for policy 1, policy_version 69940 (0.0009) -[2023-10-14 16:21:36,797][75950] Updated weights for policy 1, policy_version 69950 (0.0010) -[2023-10-14 16:21:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 143425536. Throughput: 0: 1697.4, 1: 1668.3. Samples: 35863908. Policy #0 lag: (min: 9.0, avg: 16.0, max: 41.0) -[2023-10-14 16:21:38,164][74987] Avg episode reward: [(0, '27.590'), (1, '32.970')] -[2023-10-14 16:21:38,686][75949] Updated weights for policy 0, policy_version 70121 (0.0008) -[2023-10-14 16:21:39,046][75949] Updated weights for policy 0, policy_version 70131 (0.0007) -[2023-10-14 16:21:39,427][75949] Updated weights for policy 0, policy_version 70141 (0.0010) -[2023-10-14 16:21:41,126][75950] Updated weights for policy 1, policy_version 69960 (0.0010) -[2023-10-14 16:21:41,510][75950] Updated weights for policy 1, policy_version 69970 (0.0008) -[2023-10-14 16:21:41,874][75950] Updated weights for policy 1, policy_version 69980 (0.0007) -[2023-10-14 16:21:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 143491072. Throughput: 0: 1698.0, 1: 1671.8. Samples: 35884100. Policy #0 lag: (min: 9.0, avg: 16.0, max: 41.0) -[2023-10-14 16:21:43,164][74987] Avg episode reward: [(0, '26.630'), (1, '31.110')] -[2023-10-14 16:21:43,531][75949] Updated weights for policy 0, policy_version 70151 (0.0008) -[2023-10-14 16:21:43,908][75949] Updated weights for policy 0, policy_version 70161 (0.0010) -[2023-10-14 16:21:44,274][75949] Updated weights for policy 0, policy_version 70171 (0.0009) -[2023-10-14 16:21:45,821][75950] Updated weights for policy 1, policy_version 69990 (0.0007) -[2023-10-14 16:21:46,184][75950] Updated weights for policy 1, policy_version 70000 (0.0007) -[2023-10-14 16:21:46,551][75950] Updated weights for policy 1, policy_version 70010 (0.0009) -[2023-10-14 16:21:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 143556608. Throughput: 0: 1689.6, 1: 1678.0. Samples: 35894362. Policy #0 lag: (min: 9.0, avg: 16.0, max: 41.0) -[2023-10-14 16:21:48,165][74987] Avg episode reward: [(0, '28.000'), (1, '33.280')] -[2023-10-14 16:21:48,270][75949] Updated weights for policy 0, policy_version 70181 (0.0009) -[2023-10-14 16:21:48,636][75949] Updated weights for policy 0, policy_version 70191 (0.0010) -[2023-10-14 16:21:49,012][75949] Updated weights for policy 0, policy_version 70201 (0.0010) -[2023-10-14 16:21:50,837][75950] Updated weights for policy 1, policy_version 70020 (0.0009) -[2023-10-14 16:21:51,208][75950] Updated weights for policy 1, policy_version 70030 (0.0008) -[2023-10-14 16:21:51,571][75950] Updated weights for policy 1, policy_version 70040 (0.0010) -[2023-10-14 16:21:52,987][75949] Updated weights for policy 0, policy_version 70211 (0.0008) -[2023-10-14 16:21:53,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 143622144. Throughput: 0: 1689.5, 1: 1658.6. Samples: 35914052. Policy #0 lag: (min: 9.0, avg: 16.0, max: 41.0) -[2023-10-14 16:21:53,165][74987] Avg episode reward: [(0, '27.120'), (1, '31.780')] -[2023-10-14 16:21:53,352][75949] Updated weights for policy 0, policy_version 70221 (0.0008) -[2023-10-14 16:21:53,732][75949] Updated weights for policy 0, policy_version 70231 (0.0008) -[2023-10-14 16:21:55,517][75950] Updated weights for policy 1, policy_version 70050 (0.0009) -[2023-10-14 16:21:55,887][75950] Updated weights for policy 1, policy_version 70060 (0.0007) -[2023-10-14 16:21:56,256][75950] Updated weights for policy 1, policy_version 70070 (0.0009) -[2023-10-14 16:21:56,624][75950] Updated weights for policy 1, policy_version 70080 (0.0008) -[2023-10-14 16:21:57,870][75949] Updated weights for policy 0, policy_version 70241 (0.0009) -[2023-10-14 16:21:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 143687680. Throughput: 0: 1692.6, 1: 1680.7. Samples: 35934578. Policy #0 lag: (min: 9.0, avg: 16.0, max: 41.0) -[2023-10-14 16:21:58,165][74987] Avg episode reward: [(0, '27.780'), (1, '31.700')] -[2023-10-14 16:21:58,296][75949] Updated weights for policy 0, policy_version 70251 (0.0008) -[2023-10-14 16:21:58,669][75949] Updated weights for policy 0, policy_version 70261 (0.0008) -[2023-10-14 16:21:59,050][75949] Updated weights for policy 0, policy_version 70271 (0.0008) -[2023-10-14 16:22:00,693][75950] Updated weights for policy 1, policy_version 70090 (0.0009) -[2023-10-14 16:22:01,069][75950] Updated weights for policy 1, policy_version 70100 (0.0007) -[2023-10-14 16:22:01,435][75950] Updated weights for policy 1, policy_version 70110 (0.0008) -[2023-10-14 16:22:03,056][75949] Updated weights for policy 0, policy_version 70281 (0.0011) -[2023-10-14 16:22:03,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 143753216. Throughput: 0: 1691.8, 1: 1679.2. Samples: 35944570. Policy #0 lag: (min: 9.0, avg: 16.0, max: 41.0) -[2023-10-14 16:22:03,165][74987] Avg episode reward: [(0, '26.520'), (1, '33.020')] -[2023-10-14 16:22:03,426][75949] Updated weights for policy 0, policy_version 70291 (0.0009) -[2023-10-14 16:22:03,804][75949] Updated weights for policy 0, policy_version 70301 (0.0009) -[2023-10-14 16:22:05,419][75950] Updated weights for policy 1, policy_version 70120 (0.0007) -[2023-10-14 16:22:05,789][75950] Updated weights for policy 1, policy_version 70130 (0.0008) -[2023-10-14 16:22:06,149][75950] Updated weights for policy 1, policy_version 70140 (0.0009) -[2023-10-14 16:22:07,600][75949] Updated weights for policy 0, policy_version 70311 (0.0010) -[2023-10-14 16:22:07,964][75949] Updated weights for policy 0, policy_version 70321 (0.0008) -[2023-10-14 16:22:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 143818752. Throughput: 0: 1687.6, 1: 1663.9. Samples: 35964278. Policy #0 lag: (min: 9.0, avg: 16.0, max: 41.0) -[2023-10-14 16:22:08,165][74987] Avg episode reward: [(0, '27.800'), (1, '31.220')] -[2023-10-14 16:22:08,337][75949] Updated weights for policy 0, policy_version 70331 (0.0007) -[2023-10-14 16:22:10,271][75950] Updated weights for policy 1, policy_version 70150 (0.0008) -[2023-10-14 16:22:10,636][75950] Updated weights for policy 1, policy_version 70160 (0.0008) -[2023-10-14 16:22:10,998][75950] Updated weights for policy 1, policy_version 70170 (0.0010) -[2023-10-14 16:22:12,363][75949] Updated weights for policy 0, policy_version 70341 (0.0008) -[2023-10-14 16:22:12,732][75949] Updated weights for policy 0, policy_version 70351 (0.0009) -[2023-10-14 16:22:13,094][75949] Updated weights for policy 0, policy_version 70361 (0.0007) -[2023-10-14 16:22:13,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 143884288. Throughput: 0: 1676.4, 1: 1680.7. Samples: 35984618. Policy #0 lag: (min: 9.0, avg: 16.0, max: 41.0) -[2023-10-14 16:22:13,164][74987] Avg episode reward: [(0, '27.750'), (1, '31.210')] -[2023-10-14 16:22:15,104][75950] Updated weights for policy 1, policy_version 70180 (0.0009) -[2023-10-14 16:22:15,479][75950] Updated weights for policy 1, policy_version 70190 (0.0009) -[2023-10-14 16:22:15,848][75950] Updated weights for policy 1, policy_version 70200 (0.0008) -[2023-10-14 16:22:17,281][75949] Updated weights for policy 0, policy_version 70371 (0.0008) -[2023-10-14 16:22:17,657][75949] Updated weights for policy 0, policy_version 70381 (0.0008) -[2023-10-14 16:22:18,028][75949] Updated weights for policy 0, policy_version 70391 (0.0007) -[2023-10-14 16:22:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 143949824. Throughput: 0: 1688.5, 1: 1666.6. Samples: 35994824. Policy #0 lag: (min: 9.0, avg: 16.0, max: 41.0) -[2023-10-14 16:22:18,164][74987] Avg episode reward: [(0, '25.130'), (1, '31.540')] -[2023-10-14 16:22:19,876][75950] Updated weights for policy 1, policy_version 70210 (0.0010) -[2023-10-14 16:22:20,247][75950] Updated weights for policy 1, policy_version 70220 (0.0008) -[2023-10-14 16:22:20,610][75950] Updated weights for policy 1, policy_version 70230 (0.0007) -[2023-10-14 16:22:20,973][75950] Updated weights for policy 1, policy_version 70240 (0.0007) -[2023-10-14 16:22:22,179][75949] Updated weights for policy 0, policy_version 70401 (0.0008) -[2023-10-14 16:22:22,545][75949] Updated weights for policy 0, policy_version 70411 (0.0007) -[2023-10-14 16:22:22,917][75949] Updated weights for policy 0, policy_version 70421 (0.0007) -[2023-10-14 16:22:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144015360. Throughput: 0: 1684.9, 1: 1668.9. Samples: 36014830. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:22:23,165][74987] Avg episode reward: [(0, '26.960'), (1, '30.710')] -[2023-10-14 16:22:23,287][75949] Updated weights for policy 0, policy_version 70431 (0.0008) -[2023-10-14 16:22:25,304][75950] Updated weights for policy 1, policy_version 70250 (0.0011) -[2023-10-14 16:22:25,673][75950] Updated weights for policy 1, policy_version 70260 (0.0009) -[2023-10-14 16:22:26,033][75950] Updated weights for policy 1, policy_version 70270 (0.0009) -[2023-10-14 16:22:27,285][75949] Updated weights for policy 0, policy_version 70441 (0.0008) -[2023-10-14 16:22:27,658][75949] Updated weights for policy 0, policy_version 70451 (0.0008) -[2023-10-14 16:22:28,030][75949] Updated weights for policy 0, policy_version 70461 (0.0008) -[2023-10-14 16:22:28,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 144113664. Throughput: 0: 1667.6, 1: 1682.6. Samples: 36034862. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:22:28,165][74987] Avg episode reward: [(0, '26.120'), (1, '31.960')] -[2023-10-14 16:22:30,274][75950] Updated weights for policy 1, policy_version 70280 (0.0009) -[2023-10-14 16:22:30,649][75950] Updated weights for policy 1, policy_version 70290 (0.0008) -[2023-10-14 16:22:31,019][75950] Updated weights for policy 1, policy_version 70300 (0.0010) -[2023-10-14 16:22:32,095][75949] Updated weights for policy 0, policy_version 70471 (0.0010) -[2023-10-14 16:22:32,471][75949] Updated weights for policy 0, policy_version 70481 (0.0009) -[2023-10-14 16:22:32,842][75949] Updated weights for policy 0, policy_version 70491 (0.0009) -[2023-10-14 16:22:33,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 144179200. Throughput: 0: 1688.4, 1: 1661.5. Samples: 36045106. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:22:33,165][74987] Avg episode reward: [(0, '27.150'), (1, '30.960')] -[2023-10-14 16:22:35,154][75950] Updated weights for policy 1, policy_version 70310 (0.0010) -[2023-10-14 16:22:35,512][75950] Updated weights for policy 1, policy_version 70320 (0.0010) -[2023-10-14 16:22:35,891][75950] Updated weights for policy 1, policy_version 70330 (0.0011) -[2023-10-14 16:22:36,860][75949] Updated weights for policy 0, policy_version 70501 (0.0009) -[2023-10-14 16:22:37,227][75949] Updated weights for policy 0, policy_version 70511 (0.0007) -[2023-10-14 16:22:37,596][75949] Updated weights for policy 0, policy_version 70521 (0.0011) -[2023-10-14 16:22:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 144244736. Throughput: 0: 1690.0, 1: 1664.1. Samples: 36064982. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:22:38,164][74987] Avg episode reward: [(0, '26.900'), (1, '32.940')] -[2023-10-14 16:22:39,999][75950] Updated weights for policy 1, policy_version 70340 (0.0009) -[2023-10-14 16:22:40,376][75950] Updated weights for policy 1, policy_version 70350 (0.0010) -[2023-10-14 16:22:40,739][75950] Updated weights for policy 1, policy_version 70360 (0.0009) -[2023-10-14 16:22:41,794][75949] Updated weights for policy 0, policy_version 70531 (0.0010) -[2023-10-14 16:22:42,165][75949] Updated weights for policy 0, policy_version 70541 (0.0008) -[2023-10-14 16:22:42,532][75949] Updated weights for policy 0, policy_version 70551 (0.0008) -[2023-10-14 16:22:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 144310272. Throughput: 0: 1666.5, 1: 1666.2. Samples: 36084552. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:22:43,165][74987] Avg episode reward: [(0, '26.240'), (1, '33.460')] -[2023-10-14 16:22:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000070560_72253440.pth... -[2023-10-14 16:22:43,177][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000070368_72056832.pth... -[2023-10-14 16:22:43,217][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000068800_70451200.pth -[2023-10-14 16:22:43,219][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000068992_70647808.pth -[2023-10-14 16:22:43,223][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000070368_72056832.pth -[2023-10-14 16:22:43,225][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000070560_72253440.pth -[2023-10-14 16:22:44,677][75950] Updated weights for policy 1, policy_version 70370 (0.0009) -[2023-10-14 16:22:45,053][75950] Updated weights for policy 1, policy_version 70380 (0.0010) -[2023-10-14 16:22:45,423][75950] Updated weights for policy 1, policy_version 70390 (0.0009) -[2023-10-14 16:22:45,784][75950] Updated weights for policy 1, policy_version 70400 (0.0009) -[2023-10-14 16:22:46,519][75949] Updated weights for policy 0, policy_version 70561 (0.0007) -[2023-10-14 16:22:46,912][75949] Updated weights for policy 0, policy_version 70571 (0.0007) -[2023-10-14 16:22:47,287][75949] Updated weights for policy 0, policy_version 70581 (0.0009) -[2023-10-14 16:22:47,656][75949] Updated weights for policy 0, policy_version 70591 (0.0010) -[2023-10-14 16:22:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 144375808. Throughput: 0: 1692.8, 1: 1649.4. Samples: 36094970. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:22:48,165][74987] Avg episode reward: [(0, '25.810'), (1, '33.180')] -[2023-10-14 16:22:49,856][75950] Updated weights for policy 1, policy_version 70410 (0.0008) -[2023-10-14 16:22:50,219][75950] Updated weights for policy 1, policy_version 70420 (0.0007) -[2023-10-14 16:22:50,584][75950] Updated weights for policy 1, policy_version 70430 (0.0007) -[2023-10-14 16:22:51,756][75949] Updated weights for policy 0, policy_version 70601 (0.0008) -[2023-10-14 16:22:52,128][75949] Updated weights for policy 0, policy_version 70611 (0.0008) -[2023-10-14 16:22:52,507][75949] Updated weights for policy 0, policy_version 70621 (0.0010) -[2023-10-14 16:22:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 144441344. Throughput: 0: 1681.0, 1: 1666.8. Samples: 36114926. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:22:53,165][74987] Avg episode reward: [(0, '26.180'), (1, '33.200')] -[2023-10-14 16:22:54,595][75950] Updated weights for policy 1, policy_version 70440 (0.0008) -[2023-10-14 16:22:54,976][75950] Updated weights for policy 1, policy_version 70450 (0.0009) -[2023-10-14 16:22:55,347][75950] Updated weights for policy 1, policy_version 70460 (0.0008) -[2023-10-14 16:22:56,527][75949] Updated weights for policy 0, policy_version 70631 (0.0008) -[2023-10-14 16:22:56,889][75949] Updated weights for policy 0, policy_version 70641 (0.0008) -[2023-10-14 16:22:57,262][75949] Updated weights for policy 0, policy_version 70651 (0.0009) -[2023-10-14 16:22:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 144506880. Throughput: 0: 1665.7, 1: 1672.5. Samples: 36134838. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:22:58,165][74987] Avg episode reward: [(0, '27.610'), (1, '34.390')] -[2023-10-14 16:22:59,393][75950] Updated weights for policy 1, policy_version 70470 (0.0008) -[2023-10-14 16:22:59,758][75950] Updated weights for policy 1, policy_version 70480 (0.0009) -[2023-10-14 16:23:00,129][75950] Updated weights for policy 1, policy_version 70490 (0.0008) -[2023-10-14 16:23:01,456][75949] Updated weights for policy 0, policy_version 70661 (0.0010) -[2023-10-14 16:23:01,827][75949] Updated weights for policy 0, policy_version 70671 (0.0009) -[2023-10-14 16:23:02,193][75949] Updated weights for policy 0, policy_version 70681 (0.0009) -[2023-10-14 16:23:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 144572416. Throughput: 0: 1681.3, 1: 1656.7. Samples: 36145032. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 16:23:03,165][74987] Avg episode reward: [(0, '27.740'), (1, '32.480')] -[2023-10-14 16:23:04,424][75950] Updated weights for policy 1, policy_version 70500 (0.0010) -[2023-10-14 16:23:04,790][75950] Updated weights for policy 1, policy_version 70510 (0.0009) -[2023-10-14 16:23:05,156][75950] Updated weights for policy 1, policy_version 70520 (0.0007) -[2023-10-14 16:23:06,245][75949] Updated weights for policy 0, policy_version 70691 (0.0011) -[2023-10-14 16:23:06,616][75949] Updated weights for policy 0, policy_version 70701 (0.0009) -[2023-10-14 16:23:06,988][75949] Updated weights for policy 0, policy_version 70711 (0.0009) -[2023-10-14 16:23:08,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 144637952. Throughput: 0: 1674.0, 1: 1670.1. Samples: 36165316. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 16:23:08,164][74987] Avg episode reward: [(0, '26.820'), (1, '33.230')] -[2023-10-14 16:23:09,145][75950] Updated weights for policy 1, policy_version 70530 (0.0009) -[2023-10-14 16:23:09,507][75950] Updated weights for policy 1, policy_version 70540 (0.0010) -[2023-10-14 16:23:09,870][75950] Updated weights for policy 1, policy_version 70550 (0.0009) -[2023-10-14 16:23:10,238][75950] Updated weights for policy 1, policy_version 70560 (0.0008) -[2023-10-14 16:23:11,090][75949] Updated weights for policy 0, policy_version 70721 (0.0007) -[2023-10-14 16:23:11,446][75949] Updated weights for policy 0, policy_version 70731 (0.0007) -[2023-10-14 16:23:11,821][75949] Updated weights for policy 0, policy_version 70741 (0.0008) -[2023-10-14 16:23:12,192][75949] Updated weights for policy 0, policy_version 70751 (0.0011) -[2023-10-14 16:23:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 144703488. Throughput: 0: 1676.8, 1: 1674.6. Samples: 36185672. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 16:23:13,164][74987] Avg episode reward: [(0, '25.320'), (1, '33.730')] -[2023-10-14 16:23:14,201][75950] Updated weights for policy 1, policy_version 70570 (0.0009) -[2023-10-14 16:23:14,569][75950] Updated weights for policy 1, policy_version 70580 (0.0008) -[2023-10-14 16:23:14,938][75950] Updated weights for policy 1, policy_version 70590 (0.0007) -[2023-10-14 16:23:16,027][75949] Updated weights for policy 0, policy_version 70761 (0.0007) -[2023-10-14 16:23:16,400][75949] Updated weights for policy 0, policy_version 70771 (0.0011) -[2023-10-14 16:23:16,764][75949] Updated weights for policy 0, policy_version 70781 (0.0009) -[2023-10-14 16:23:18,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 144769024. Throughput: 0: 1695.1, 1: 1668.0. Samples: 36196444. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 16:23:18,165][74987] Avg episode reward: [(0, '28.040'), (1, '33.310')] -[2023-10-14 16:23:19,003][75950] Updated weights for policy 1, policy_version 70600 (0.0009) -[2023-10-14 16:23:19,376][75950] Updated weights for policy 1, policy_version 70610 (0.0009) -[2023-10-14 16:23:19,743][75950] Updated weights for policy 1, policy_version 70620 (0.0009) -[2023-10-14 16:23:20,954][75949] Updated weights for policy 0, policy_version 70791 (0.0010) -[2023-10-14 16:23:21,319][75949] Updated weights for policy 0, policy_version 70801 (0.0009) -[2023-10-14 16:23:21,684][75949] Updated weights for policy 0, policy_version 70811 (0.0008) -[2023-10-14 16:23:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 144834560. Throughput: 0: 1671.2, 1: 1686.6. Samples: 36216084. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 16:23:23,165][74987] Avg episode reward: [(0, '26.720'), (1, '31.870')] -[2023-10-14 16:23:23,979][75950] Updated weights for policy 1, policy_version 70630 (0.0008) -[2023-10-14 16:23:24,380][75950] Updated weights for policy 1, policy_version 70640 (0.0010) -[2023-10-14 16:23:24,754][75950] Updated weights for policy 1, policy_version 70650 (0.0010) -[2023-10-14 16:23:25,573][75949] Updated weights for policy 0, policy_version 70821 (0.0011) -[2023-10-14 16:23:25,933][75949] Updated weights for policy 0, policy_version 70831 (0.0010) -[2023-10-14 16:23:26,309][75949] Updated weights for policy 0, policy_version 70841 (0.0011) -[2023-10-14 16:23:28,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 144900096. Throughput: 0: 1691.3, 1: 1684.0. Samples: 36236438. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 16:23:28,164][74987] Avg episode reward: [(0, '28.190'), (1, '33.360')] -[2023-10-14 16:23:28,814][75950] Updated weights for policy 1, policy_version 70660 (0.0007) -[2023-10-14 16:23:29,177][75950] Updated weights for policy 1, policy_version 70670 (0.0007) -[2023-10-14 16:23:29,542][75950] Updated weights for policy 1, policy_version 70680 (0.0008) -[2023-10-14 16:23:30,471][75949] Updated weights for policy 0, policy_version 70851 (0.0010) -[2023-10-14 16:23:30,831][75949] Updated weights for policy 0, policy_version 70861 (0.0009) -[2023-10-14 16:23:31,199][75949] Updated weights for policy 0, policy_version 70871 (0.0009) -[2023-10-14 16:23:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 144965632. Throughput: 0: 1692.0, 1: 1678.7. Samples: 36246650. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 16:23:33,164][74987] Avg episode reward: [(0, '25.730'), (1, '32.920')] -[2023-10-14 16:23:33,454][75950] Updated weights for policy 1, policy_version 70690 (0.0009) -[2023-10-14 16:23:33,821][75950] Updated weights for policy 1, policy_version 70700 (0.0009) -[2023-10-14 16:23:34,183][75950] Updated weights for policy 1, policy_version 70710 (0.0008) -[2023-10-14 16:23:34,546][75950] Updated weights for policy 1, policy_version 70720 (0.0007) -[2023-10-14 16:23:35,377][75949] Updated weights for policy 0, policy_version 70881 (0.0010) -[2023-10-14 16:23:35,767][75949] Updated weights for policy 0, policy_version 70891 (0.0007) -[2023-10-14 16:23:36,131][75949] Updated weights for policy 0, policy_version 70901 (0.0008) -[2023-10-14 16:23:36,497][75949] Updated weights for policy 0, policy_version 70911 (0.0008) -[2023-10-14 16:23:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 145031168. Throughput: 0: 1679.6, 1: 1686.5. Samples: 36266400. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 16:23:38,165][74987] Avg episode reward: [(0, '28.540'), (1, '30.300')] -[2023-10-14 16:23:38,750][75950] Updated weights for policy 1, policy_version 70730 (0.0008) -[2023-10-14 16:23:39,109][75950] Updated weights for policy 1, policy_version 70740 (0.0008) -[2023-10-14 16:23:39,477][75950] Updated weights for policy 1, policy_version 70750 (0.0007) -[2023-10-14 16:23:40,512][75949] Updated weights for policy 0, policy_version 70921 (0.0010) -[2023-10-14 16:23:40,893][75949] Updated weights for policy 0, policy_version 70931 (0.0009) -[2023-10-14 16:23:41,255][75949] Updated weights for policy 0, policy_version 70941 (0.0009) -[2023-10-14 16:23:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 145096704. Throughput: 0: 1701.2, 1: 1690.2. Samples: 36287450. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 16:23:43,165][74987] Avg episode reward: [(0, '27.540'), (1, '32.070')] -[2023-10-14 16:23:43,258][75950] Updated weights for policy 1, policy_version 70760 (0.0010) -[2023-10-14 16:23:43,628][75950] Updated weights for policy 1, policy_version 70770 (0.0008) -[2023-10-14 16:23:44,005][75950] Updated weights for policy 1, policy_version 70780 (0.0010) -[2023-10-14 16:23:45,189][75949] Updated weights for policy 0, policy_version 70951 (0.0009) -[2023-10-14 16:23:45,556][75949] Updated weights for policy 0, policy_version 70961 (0.0007) -[2023-10-14 16:23:45,930][75949] Updated weights for policy 0, policy_version 70971 (0.0008) -[2023-10-14 16:23:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 145162240. Throughput: 0: 1688.5, 1: 1691.5. Samples: 36297132. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 16:23:48,164][74987] Avg episode reward: [(0, '27.540'), (1, '31.720')] -[2023-10-14 16:23:48,173][75950] Updated weights for policy 1, policy_version 70790 (0.0008) -[2023-10-14 16:23:48,546][75950] Updated weights for policy 1, policy_version 70800 (0.0007) -[2023-10-14 16:23:48,904][75950] Updated weights for policy 1, policy_version 70810 (0.0007) -[2023-10-14 16:23:49,892][75949] Updated weights for policy 0, policy_version 70981 (0.0010) -[2023-10-14 16:23:50,261][75949] Updated weights for policy 0, policy_version 70991 (0.0008) -[2023-10-14 16:23:50,621][75949] Updated weights for policy 0, policy_version 71001 (0.0008) -[2023-10-14 16:23:53,048][75950] Updated weights for policy 1, policy_version 70820 (0.0008) -[2023-10-14 16:23:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 145227776. Throughput: 0: 1684.0, 1: 1688.8. Samples: 36317096. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 16:23:53,165][74987] Avg episode reward: [(0, '27.540'), (1, '31.130')] -[2023-10-14 16:23:53,413][75950] Updated weights for policy 1, policy_version 70830 (0.0008) -[2023-10-14 16:23:53,780][75950] Updated weights for policy 1, policy_version 70840 (0.0010) -[2023-10-14 16:23:54,636][75949] Updated weights for policy 0, policy_version 71011 (0.0008) -[2023-10-14 16:23:54,995][75949] Updated weights for policy 0, policy_version 71021 (0.0010) -[2023-10-14 16:23:55,379][75949] Updated weights for policy 0, policy_version 71031 (0.0011) -[2023-10-14 16:23:57,746][75950] Updated weights for policy 1, policy_version 70850 (0.0009) -[2023-10-14 16:23:58,113][75950] Updated weights for policy 1, policy_version 70860 (0.0008) -[2023-10-14 16:23:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 145293312. Throughput: 0: 1698.1, 1: 1687.9. Samples: 36338044. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 16:23:58,164][74987] Avg episode reward: [(0, '26.330'), (1, '34.460')] -[2023-10-14 16:23:58,478][75950] Updated weights for policy 1, policy_version 70870 (0.0009) -[2023-10-14 16:23:58,842][75950] Updated weights for policy 1, policy_version 70880 (0.0008) -[2023-10-14 16:23:59,479][75949] Updated weights for policy 0, policy_version 71041 (0.0011) -[2023-10-14 16:23:59,838][75949] Updated weights for policy 0, policy_version 71051 (0.0009) -[2023-10-14 16:24:00,210][75949] Updated weights for policy 0, policy_version 71061 (0.0008) -[2023-10-14 16:24:00,566][75949] Updated weights for policy 0, policy_version 71071 (0.0008) -[2023-10-14 16:24:02,915][75950] Updated weights for policy 1, policy_version 70890 (0.0010) -[2023-10-14 16:24:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 145358848. Throughput: 0: 1664.5, 1: 1688.6. Samples: 36347334. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 16:24:03,164][74987] Avg episode reward: [(0, '26.170'), (1, '32.480')] -[2023-10-14 16:24:03,280][75950] Updated weights for policy 1, policy_version 70900 (0.0010) -[2023-10-14 16:24:03,650][75950] Updated weights for policy 1, policy_version 70910 (0.0010) -[2023-10-14 16:24:04,679][75949] Updated weights for policy 0, policy_version 71081 (0.0009) -[2023-10-14 16:24:05,044][75949] Updated weights for policy 0, policy_version 71091 (0.0007) -[2023-10-14 16:24:05,418][75949] Updated weights for policy 0, policy_version 71101 (0.0008) -[2023-10-14 16:24:07,793][75950] Updated weights for policy 1, policy_version 70920 (0.0009) -[2023-10-14 16:24:08,160][75950] Updated weights for policy 1, policy_version 70930 (0.0007) -[2023-10-14 16:24:08,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 145424384. Throughput: 0: 1687.4, 1: 1687.3. Samples: 36367944. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 16:24:08,164][74987] Avg episode reward: [(0, '26.560'), (1, '33.180')] -[2023-10-14 16:24:08,537][75950] Updated weights for policy 1, policy_version 70940 (0.0007) -[2023-10-14 16:24:09,459][75949] Updated weights for policy 0, policy_version 71111 (0.0009) -[2023-10-14 16:24:09,827][75949] Updated weights for policy 0, policy_version 71121 (0.0009) -[2023-10-14 16:24:10,191][75949] Updated weights for policy 0, policy_version 71131 (0.0009) -[2023-10-14 16:24:12,521][75950] Updated weights for policy 1, policy_version 70950 (0.0007) -[2023-10-14 16:24:12,892][75950] Updated weights for policy 1, policy_version 70960 (0.0007) -[2023-10-14 16:24:13,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 145489920. Throughput: 0: 1688.6, 1: 1682.7. Samples: 36388146. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 16:24:13,164][74987] Avg episode reward: [(0, '26.750'), (1, '32.150')] -[2023-10-14 16:24:13,265][75950] Updated weights for policy 1, policy_version 70970 (0.0007) -[2023-10-14 16:24:14,310][75949] Updated weights for policy 0, policy_version 71141 (0.0009) -[2023-10-14 16:24:14,685][75949] Updated weights for policy 0, policy_version 71151 (0.0008) -[2023-10-14 16:24:15,059][75949] Updated weights for policy 0, policy_version 71161 (0.0011) -[2023-10-14 16:24:17,425][75950] Updated weights for policy 1, policy_version 70980 (0.0008) -[2023-10-14 16:24:17,789][75950] Updated weights for policy 1, policy_version 70990 (0.0009) -[2023-10-14 16:24:18,153][75950] Updated weights for policy 1, policy_version 71000 (0.0009) -[2023-10-14 16:24:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 145555456. Throughput: 0: 1665.3, 1: 1691.6. Samples: 36397712. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 16:24:18,164][74987] Avg episode reward: [(0, '26.030'), (1, '31.120')] -[2023-10-14 16:24:19,098][75949] Updated weights for policy 0, policy_version 71171 (0.0010) -[2023-10-14 16:24:19,476][75949] Updated weights for policy 0, policy_version 71181 (0.0007) -[2023-10-14 16:24:19,836][75949] Updated weights for policy 0, policy_version 71191 (0.0010) -[2023-10-14 16:24:22,133][75950] Updated weights for policy 1, policy_version 71010 (0.0008) -[2023-10-14 16:24:22,497][75950] Updated weights for policy 1, policy_version 71020 (0.0007) -[2023-10-14 16:24:22,865][75950] Updated weights for policy 1, policy_version 71030 (0.0008) -[2023-10-14 16:24:23,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145620992. Throughput: 0: 1684.5, 1: 1694.4. Samples: 36418446. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 16:24:23,164][74987] Avg episode reward: [(0, '27.650'), (1, '32.680')] -[2023-10-14 16:24:23,231][75950] Updated weights for policy 1, policy_version 71040 (0.0007) -[2023-10-14 16:24:23,999][75949] Updated weights for policy 0, policy_version 71201 (0.0011) -[2023-10-14 16:24:24,379][75949] Updated weights for policy 0, policy_version 71211 (0.0009) -[2023-10-14 16:24:24,756][75949] Updated weights for policy 0, policy_version 71221 (0.0009) -[2023-10-14 16:24:25,121][75949] Updated weights for policy 0, policy_version 71231 (0.0009) -[2023-10-14 16:24:27,240][75950] Updated weights for policy 1, policy_version 71050 (0.0010) -[2023-10-14 16:24:27,600][75950] Updated weights for policy 1, policy_version 71060 (0.0011) -[2023-10-14 16:24:27,975][75950] Updated weights for policy 1, policy_version 71070 (0.0011) -[2023-10-14 16:24:28,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 145719296. Throughput: 0: 1687.1, 1: 1671.2. Samples: 36438572. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 16:24:28,165][74987] Avg episode reward: [(0, '25.950'), (1, '31.040')] -[2023-10-14 16:24:29,239][75949] Updated weights for policy 0, policy_version 71241 (0.0008) -[2023-10-14 16:24:29,609][75949] Updated weights for policy 0, policy_version 71251 (0.0009) -[2023-10-14 16:24:29,984][75949] Updated weights for policy 0, policy_version 71261 (0.0008) -[2023-10-14 16:24:32,113][75950] Updated weights for policy 1, policy_version 71080 (0.0009) -[2023-10-14 16:24:32,472][75950] Updated weights for policy 1, policy_version 71090 (0.0009) -[2023-10-14 16:24:32,836][75950] Updated weights for policy 1, policy_version 71100 (0.0009) -[2023-10-14 16:24:33,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 145784832. Throughput: 0: 1673.6, 1: 1690.2. Samples: 36448504. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 16:24:33,164][74987] Avg episode reward: [(0, '27.490'), (1, '32.870')] -[2023-10-14 16:24:34,228][75949] Updated weights for policy 0, policy_version 71271 (0.0008) -[2023-10-14 16:24:34,592][75949] Updated weights for policy 0, policy_version 71281 (0.0011) -[2023-10-14 16:24:34,962][75949] Updated weights for policy 0, policy_version 71291 (0.0008) -[2023-10-14 16:24:36,843][75950] Updated weights for policy 1, policy_version 71110 (0.0010) -[2023-10-14 16:24:37,206][75950] Updated weights for policy 1, policy_version 71120 (0.0010) -[2023-10-14 16:24:37,572][75950] Updated weights for policy 1, policy_version 71130 (0.0009) -[2023-10-14 16:24:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 145850368. Throughput: 0: 1681.8, 1: 1687.8. Samples: 36468730. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) -[2023-10-14 16:24:38,165][74987] Avg episode reward: [(0, '27.050'), (1, '32.890')] -[2023-10-14 16:24:38,843][75949] Updated weights for policy 0, policy_version 71301 (0.0010) -[2023-10-14 16:24:39,220][75949] Updated weights for policy 0, policy_version 71311 (0.0009) -[2023-10-14 16:24:39,591][75949] Updated weights for policy 0, policy_version 71321 (0.0010) -[2023-10-14 16:24:41,701][75950] Updated weights for policy 1, policy_version 71140 (0.0009) -[2023-10-14 16:24:42,079][75950] Updated weights for policy 1, policy_version 71150 (0.0009) -[2023-10-14 16:24:42,436][75950] Updated weights for policy 1, policy_version 71160 (0.0011) -[2023-10-14 16:24:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 145915904. Throughput: 0: 1681.2, 1: 1653.9. Samples: 36488122. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:24:43,165][74987] Avg episode reward: [(0, '26.980'), (1, '32.950')] -[2023-10-14 16:24:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000071328_73039872.pth... -[2023-10-14 16:24:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000071168_72876032.pth... -[2023-10-14 16:24:43,206][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000069760_71434240.pth -[2023-10-14 16:24:43,214][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000069600_71270400.pth -[2023-10-14 16:24:43,490][75949] Updated weights for policy 0, policy_version 71331 (0.0010) -[2023-10-14 16:24:43,857][75949] Updated weights for policy 0, policy_version 71341 (0.0009) -[2023-10-14 16:24:44,225][75949] Updated weights for policy 0, policy_version 71351 (0.0008) -[2023-10-14 16:24:46,464][75950] Updated weights for policy 1, policy_version 71170 (0.0009) -[2023-10-14 16:24:46,818][75950] Updated weights for policy 1, policy_version 71180 (0.0011) -[2023-10-14 16:24:47,190][75950] Updated weights for policy 1, policy_version 71190 (0.0010) -[2023-10-14 16:24:47,555][75950] Updated weights for policy 1, policy_version 71200 (0.0011) -[2023-10-14 16:24:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 145981440. Throughput: 0: 1677.4, 1: 1678.9. Samples: 36498368. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:24:48,165][74987] Avg episode reward: [(0, '27.060'), (1, '34.760')] -[2023-10-14 16:24:48,406][75949] Updated weights for policy 0, policy_version 71361 (0.0008) -[2023-10-14 16:24:48,768][75949] Updated weights for policy 0, policy_version 71371 (0.0009) -[2023-10-14 16:24:49,144][75949] Updated weights for policy 0, policy_version 71381 (0.0008) -[2023-10-14 16:24:49,516][75949] Updated weights for policy 0, policy_version 71391 (0.0009) -[2023-10-14 16:24:51,794][75950] Updated weights for policy 1, policy_version 71210 (0.0007) -[2023-10-14 16:24:52,165][75950] Updated weights for policy 1, policy_version 71220 (0.0010) -[2023-10-14 16:24:52,520][75950] Updated weights for policy 1, policy_version 71230 (0.0008) -[2023-10-14 16:24:53,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 146046976. Throughput: 0: 1675.0, 1: 1671.9. Samples: 36518556. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:24:53,164][74987] Avg episode reward: [(0, '27.130'), (1, '34.620')] -[2023-10-14 16:24:53,545][75949] Updated weights for policy 0, policy_version 71401 (0.0008) -[2023-10-14 16:24:53,913][75949] Updated weights for policy 0, policy_version 71411 (0.0008) -[2023-10-14 16:24:54,291][75949] Updated weights for policy 0, policy_version 71421 (0.0007) -[2023-10-14 16:24:56,593][75950] Updated weights for policy 1, policy_version 71240 (0.0009) -[2023-10-14 16:24:56,962][75950] Updated weights for policy 1, policy_version 71250 (0.0008) -[2023-10-14 16:24:57,320][75950] Updated weights for policy 1, policy_version 71260 (0.0007) -[2023-10-14 16:24:58,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 146112512. Throughput: 0: 1677.1, 1: 1657.8. Samples: 36538216. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:24:58,164][74987] Avg episode reward: [(0, '25.230'), (1, '31.920')] -[2023-10-14 16:24:58,439][75949] Updated weights for policy 0, policy_version 71431 (0.0009) -[2023-10-14 16:24:58,817][75949] Updated weights for policy 0, policy_version 71441 (0.0010) -[2023-10-14 16:24:59,183][75949] Updated weights for policy 0, policy_version 71451 (0.0009) -[2023-10-14 16:25:01,354][75950] Updated weights for policy 1, policy_version 71270 (0.0009) -[2023-10-14 16:25:01,734][75950] Updated weights for policy 1, policy_version 71280 (0.0010) -[2023-10-14 16:25:02,092][75950] Updated weights for policy 1, policy_version 71290 (0.0009) -[2023-10-14 16:25:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 146178048. Throughput: 0: 1677.0, 1: 1683.4. Samples: 36548930. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:25:03,164][74987] Avg episode reward: [(0, '25.910'), (1, '32.920')] -[2023-10-14 16:25:03,267][75949] Updated weights for policy 0, policy_version 71461 (0.0008) -[2023-10-14 16:25:03,635][75949] Updated weights for policy 0, policy_version 71471 (0.0008) -[2023-10-14 16:25:03,998][75949] Updated weights for policy 0, policy_version 71481 (0.0009) -[2023-10-14 16:25:06,384][75950] Updated weights for policy 1, policy_version 71300 (0.0009) -[2023-10-14 16:25:06,757][75950] Updated weights for policy 1, policy_version 71310 (0.0008) -[2023-10-14 16:25:07,125][75950] Updated weights for policy 1, policy_version 71320 (0.0007) -[2023-10-14 16:25:08,115][75949] Updated weights for policy 0, policy_version 71491 (0.0010) -[2023-10-14 16:25:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 146243584. Throughput: 0: 1681.4, 1: 1667.5. Samples: 36569146. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:25:08,164][74987] Avg episode reward: [(0, '24.610'), (1, '34.120')] -[2023-10-14 16:25:08,533][75949] Updated weights for policy 0, policy_version 71501 (0.0008) -[2023-10-14 16:25:08,906][75949] Updated weights for policy 0, policy_version 71511 (0.0008) -[2023-10-14 16:25:11,111][75950] Updated weights for policy 1, policy_version 71330 (0.0008) -[2023-10-14 16:25:11,471][75950] Updated weights for policy 1, policy_version 71340 (0.0007) -[2023-10-14 16:25:11,834][75950] Updated weights for policy 1, policy_version 71350 (0.0007) -[2023-10-14 16:25:12,209][75950] Updated weights for policy 1, policy_version 71360 (0.0009) -[2023-10-14 16:25:13,043][75949] Updated weights for policy 0, policy_version 71521 (0.0008) -[2023-10-14 16:25:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 146309120. Throughput: 0: 1680.0, 1: 1669.3. Samples: 36589288. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:25:13,164][74987] Avg episode reward: [(0, '26.490'), (1, '33.840')] -[2023-10-14 16:25:13,411][75949] Updated weights for policy 0, policy_version 71531 (0.0007) -[2023-10-14 16:25:13,773][75949] Updated weights for policy 0, policy_version 71541 (0.0007) -[2023-10-14 16:25:14,149][75949] Updated weights for policy 0, policy_version 71551 (0.0011) -[2023-10-14 16:25:16,183][75950] Updated weights for policy 1, policy_version 71370 (0.0008) -[2023-10-14 16:25:16,549][75950] Updated weights for policy 1, policy_version 71380 (0.0008) -[2023-10-14 16:25:16,911][75950] Updated weights for policy 1, policy_version 71390 (0.0009) -[2023-10-14 16:25:18,086][75949] Updated weights for policy 0, policy_version 71561 (0.0008) -[2023-10-14 16:25:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 146374656. Throughput: 0: 1677.8, 1: 1677.0. Samples: 36599470. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:25:18,164][74987] Avg episode reward: [(0, '24.900'), (1, '31.800')] -[2023-10-14 16:25:18,463][75949] Updated weights for policy 0, policy_version 71571 (0.0008) -[2023-10-14 16:25:18,835][75949] Updated weights for policy 0, policy_version 71581 (0.0009) -[2023-10-14 16:25:20,969][75950] Updated weights for policy 1, policy_version 71400 (0.0010) -[2023-10-14 16:25:21,330][75950] Updated weights for policy 1, policy_version 71410 (0.0010) -[2023-10-14 16:25:21,701][75950] Updated weights for policy 1, policy_version 71420 (0.0010) -[2023-10-14 16:25:23,058][75949] Updated weights for policy 0, policy_version 71591 (0.0010) -[2023-10-14 16:25:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 146440192. Throughput: 0: 1680.8, 1: 1663.6. Samples: 36619228. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:25:23,165][74987] Avg episode reward: [(0, '27.670'), (1, '34.680')] -[2023-10-14 16:25:23,429][75949] Updated weights for policy 0, policy_version 71601 (0.0011) -[2023-10-14 16:25:23,803][75949] Updated weights for policy 0, policy_version 71611 (0.0011) -[2023-10-14 16:25:25,768][75950] Updated weights for policy 1, policy_version 71430 (0.0010) -[2023-10-14 16:25:26,134][75950] Updated weights for policy 1, policy_version 71440 (0.0008) -[2023-10-14 16:25:26,508][75950] Updated weights for policy 1, policy_version 71450 (0.0007) -[2023-10-14 16:25:27,731][75949] Updated weights for policy 0, policy_version 71621 (0.0009) -[2023-10-14 16:25:28,092][75949] Updated weights for policy 0, policy_version 71631 (0.0009) -[2023-10-14 16:25:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 146505728. Throughput: 0: 1676.0, 1: 1687.8. Samples: 36639496. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:25:28,165][74987] Avg episode reward: [(0, '25.150'), (1, '34.260')] -[2023-10-14 16:25:28,469][75949] Updated weights for policy 0, policy_version 71641 (0.0009) -[2023-10-14 16:25:30,598][75950] Updated weights for policy 1, policy_version 71460 (0.0008) -[2023-10-14 16:25:30,962][75950] Updated weights for policy 1, policy_version 71470 (0.0007) -[2023-10-14 16:25:31,333][75950] Updated weights for policy 1, policy_version 71480 (0.0009) -[2023-10-14 16:25:32,733][75949] Updated weights for policy 0, policy_version 71651 (0.0010) -[2023-10-14 16:25:33,099][75949] Updated weights for policy 0, policy_version 71661 (0.0008) -[2023-10-14 16:25:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 146571264. Throughput: 0: 1680.0, 1: 1686.8. Samples: 36649874. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 16:25:33,164][74987] Avg episode reward: [(0, '27.530'), (1, '30.640')] -[2023-10-14 16:25:33,466][75949] Updated weights for policy 0, policy_version 71671 (0.0008) -[2023-10-14 16:25:35,526][75950] Updated weights for policy 1, policy_version 71490 (0.0010) -[2023-10-14 16:25:35,893][75950] Updated weights for policy 1, policy_version 71500 (0.0009) -[2023-10-14 16:25:36,253][75950] Updated weights for policy 1, policy_version 71510 (0.0011) -[2023-10-14 16:25:36,621][75950] Updated weights for policy 1, policy_version 71520 (0.0009) -[2023-10-14 16:25:37,452][75949] Updated weights for policy 0, policy_version 71681 (0.0009) -[2023-10-14 16:25:37,824][75949] Updated weights for policy 0, policy_version 71691 (0.0009) -[2023-10-14 16:25:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 146636800. Throughput: 0: 1683.5, 1: 1670.7. Samples: 36669498. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 16:25:38,165][74987] Avg episode reward: [(0, '26.100'), (1, '32.160')] -[2023-10-14 16:25:38,199][75949] Updated weights for policy 0, policy_version 71701 (0.0008) -[2023-10-14 16:25:38,563][75949] Updated weights for policy 0, policy_version 71711 (0.0008) -[2023-10-14 16:25:40,756][75950] Updated weights for policy 1, policy_version 71530 (0.0008) -[2023-10-14 16:25:41,125][75950] Updated weights for policy 1, policy_version 71540 (0.0008) -[2023-10-14 16:25:41,489][75950] Updated weights for policy 1, policy_version 71550 (0.0010) -[2023-10-14 16:25:42,417][75949] Updated weights for policy 0, policy_version 71721 (0.0008) -[2023-10-14 16:25:42,791][75949] Updated weights for policy 0, policy_version 71731 (0.0007) -[2023-10-14 16:25:43,163][75949] Updated weights for policy 0, policy_version 71741 (0.0010) -[2023-10-14 16:25:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 146702336. Throughput: 0: 1676.2, 1: 1692.9. Samples: 36689826. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 16:25:43,165][74987] Avg episode reward: [(0, '28.850'), (1, '34.850')] -[2023-10-14 16:25:45,562][75950] Updated weights for policy 1, policy_version 71560 (0.0008) -[2023-10-14 16:25:45,927][75950] Updated weights for policy 1, policy_version 71570 (0.0007) -[2023-10-14 16:25:46,293][75950] Updated weights for policy 1, policy_version 71580 (0.0009) -[2023-10-14 16:25:47,263][75949] Updated weights for policy 0, policy_version 71751 (0.0009) -[2023-10-14 16:25:47,634][75949] Updated weights for policy 0, policy_version 71761 (0.0010) -[2023-10-14 16:25:48,001][75949] Updated weights for policy 0, policy_version 71771 (0.0007) -[2023-10-14 16:25:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 146767872. Throughput: 0: 1692.9, 1: 1675.2. Samples: 36700494. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 16:25:48,164][74987] Avg episode reward: [(0, '26.180'), (1, '31.320')] -[2023-10-14 16:25:50,495][75950] Updated weights for policy 1, policy_version 71590 (0.0007) -[2023-10-14 16:25:50,870][75950] Updated weights for policy 1, policy_version 71600 (0.0008) -[2023-10-14 16:25:51,241][75950] Updated weights for policy 1, policy_version 71610 (0.0008) -[2023-10-14 16:25:51,820][75949] Updated weights for policy 0, policy_version 71781 (0.0008) -[2023-10-14 16:25:52,192][75949] Updated weights for policy 0, policy_version 71791 (0.0011) -[2023-10-14 16:25:52,556][75949] Updated weights for policy 0, policy_version 71801 (0.0011) -[2023-10-14 16:25:53,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 146866176. Throughput: 0: 1693.1, 1: 1662.5. Samples: 36720148. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 16:25:53,165][74987] Avg episode reward: [(0, '25.380'), (1, '31.980')] -[2023-10-14 16:25:55,450][75950] Updated weights for policy 1, policy_version 71620 (0.0007) -[2023-10-14 16:25:55,827][75950] Updated weights for policy 1, policy_version 71630 (0.0009) -[2023-10-14 16:25:56,192][75950] Updated weights for policy 1, policy_version 71640 (0.0008) -[2023-10-14 16:25:56,739][75949] Updated weights for policy 0, policy_version 71811 (0.0009) -[2023-10-14 16:25:57,141][75949] Updated weights for policy 0, policy_version 71821 (0.0009) -[2023-10-14 16:25:57,516][75949] Updated weights for policy 0, policy_version 71831 (0.0008) -[2023-10-14 16:25:58,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 146931712. Throughput: 0: 1669.0, 1: 1674.8. Samples: 36739758. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 16:25:58,165][74987] Avg episode reward: [(0, '25.350'), (1, '33.200')] -[2023-10-14 16:26:00,048][75950] Updated weights for policy 1, policy_version 71650 (0.0008) -[2023-10-14 16:26:00,412][75950] Updated weights for policy 1, policy_version 71660 (0.0008) -[2023-10-14 16:26:00,783][75950] Updated weights for policy 1, policy_version 71670 (0.0009) -[2023-10-14 16:26:01,156][75950] Updated weights for policy 1, policy_version 71680 (0.0010) -[2023-10-14 16:26:01,541][75949] Updated weights for policy 0, policy_version 71841 (0.0009) -[2023-10-14 16:26:01,907][75949] Updated weights for policy 0, policy_version 71851 (0.0009) -[2023-10-14 16:26:02,283][75949] Updated weights for policy 0, policy_version 71861 (0.0009) -[2023-10-14 16:26:02,660][75949] Updated weights for policy 0, policy_version 71871 (0.0010) -[2023-10-14 16:26:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 146997248. Throughput: 0: 1695.7, 1: 1664.9. Samples: 36750700. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 16:26:03,164][74987] Avg episode reward: [(0, '26.660'), (1, '33.060')] -[2023-10-14 16:26:05,059][75950] Updated weights for policy 1, policy_version 71690 (0.0010) -[2023-10-14 16:26:05,432][75950] Updated weights for policy 1, policy_version 71700 (0.0007) -[2023-10-14 16:26:05,795][75950] Updated weights for policy 1, policy_version 71710 (0.0009) -[2023-10-14 16:26:06,815][75949] Updated weights for policy 0, policy_version 71881 (0.0009) -[2023-10-14 16:26:07,181][75949] Updated weights for policy 0, policy_version 71891 (0.0008) -[2023-10-14 16:26:07,562][75949] Updated weights for policy 0, policy_version 71901 (0.0008) -[2023-10-14 16:26:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 147062784. Throughput: 0: 1692.0, 1: 1668.7. Samples: 36770460. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 16:26:08,165][74987] Avg episode reward: [(0, '27.700'), (1, '31.280')] -[2023-10-14 16:26:09,992][75950] Updated weights for policy 1, policy_version 71720 (0.0009) -[2023-10-14 16:26:10,365][75950] Updated weights for policy 1, policy_version 71730 (0.0007) -[2023-10-14 16:26:10,732][75950] Updated weights for policy 1, policy_version 71740 (0.0009) -[2023-10-14 16:26:11,358][75949] Updated weights for policy 0, policy_version 71911 (0.0010) -[2023-10-14 16:26:11,737][75949] Updated weights for policy 0, policy_version 71921 (0.0009) -[2023-10-14 16:26:12,102][75949] Updated weights for policy 0, policy_version 71931 (0.0007) -[2023-10-14 16:26:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 147128320. Throughput: 0: 1677.3, 1: 1674.2. Samples: 36790312. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) -[2023-10-14 16:26:13,165][74987] Avg episode reward: [(0, '25.370'), (1, '32.060')] -[2023-10-14 16:26:14,846][75950] Updated weights for policy 1, policy_version 71750 (0.0009) -[2023-10-14 16:26:15,218][75950] Updated weights for policy 1, policy_version 71760 (0.0009) -[2023-10-14 16:26:15,583][75950] Updated weights for policy 1, policy_version 71770 (0.0009) -[2023-10-14 16:26:16,030][75949] Updated weights for policy 0, policy_version 71941 (0.0009) -[2023-10-14 16:26:16,410][75949] Updated weights for policy 0, policy_version 71951 (0.0009) -[2023-10-14 16:26:16,775][75949] Updated weights for policy 0, policy_version 71961 (0.0009) -[2023-10-14 16:26:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 147193856. Throughput: 0: 1708.3, 1: 1650.9. Samples: 36801042. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) -[2023-10-14 16:26:18,165][74987] Avg episode reward: [(0, '27.070'), (1, '36.010')] -[2023-10-14 16:26:19,695][75950] Updated weights for policy 1, policy_version 71780 (0.0010) -[2023-10-14 16:26:20,058][75950] Updated weights for policy 1, policy_version 71790 (0.0010) -[2023-10-14 16:26:20,422][75950] Updated weights for policy 1, policy_version 71800 (0.0007) -[2023-10-14 16:26:20,981][75949] Updated weights for policy 0, policy_version 71971 (0.0009) -[2023-10-14 16:26:21,348][75949] Updated weights for policy 0, policy_version 71981 (0.0010) -[2023-10-14 16:26:21,727][75949] Updated weights for policy 0, policy_version 71991 (0.0010) -[2023-10-14 16:26:23,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 147259392. Throughput: 0: 1687.9, 1: 1668.0. Samples: 36820510. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) -[2023-10-14 16:26:23,164][74987] Avg episode reward: [(0, '25.430'), (1, '33.220')] -[2023-10-14 16:26:24,435][75950] Updated weights for policy 1, policy_version 71810 (0.0009) -[2023-10-14 16:26:24,809][75950] Updated weights for policy 1, policy_version 71820 (0.0009) -[2023-10-14 16:26:25,187][75950] Updated weights for policy 1, policy_version 71830 (0.0010) -[2023-10-14 16:26:25,555][75950] Updated weights for policy 1, policy_version 71840 (0.0009) -[2023-10-14 16:26:25,710][75949] Updated weights for policy 0, policy_version 72001 (0.0010) -[2023-10-14 16:26:26,082][75949] Updated weights for policy 0, policy_version 72011 (0.0008) -[2023-10-14 16:26:26,446][75949] Updated weights for policy 0, policy_version 72021 (0.0007) -[2023-10-14 16:26:26,816][75949] Updated weights for policy 0, policy_version 72031 (0.0011) -[2023-10-14 16:26:28,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 147324928. Throughput: 0: 1684.8, 1: 1672.3. Samples: 36840896. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) -[2023-10-14 16:26:28,164][74987] Avg episode reward: [(0, '27.560'), (1, '31.660')] -[2023-10-14 16:26:29,496][75950] Updated weights for policy 1, policy_version 71850 (0.0007) -[2023-10-14 16:26:29,863][75950] Updated weights for policy 1, policy_version 71860 (0.0009) -[2023-10-14 16:26:30,227][75950] Updated weights for policy 1, policy_version 71870 (0.0009) -[2023-10-14 16:26:31,027][75949] Updated weights for policy 0, policy_version 72041 (0.0009) -[2023-10-14 16:26:31,390][75949] Updated weights for policy 0, policy_version 72051 (0.0010) -[2023-10-14 16:26:31,770][75949] Updated weights for policy 0, policy_version 72061 (0.0011) -[2023-10-14 16:26:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 147390464. Throughput: 0: 1698.0, 1: 1654.8. Samples: 36851368. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) -[2023-10-14 16:26:33,164][74987] Avg episode reward: [(0, '24.070'), (1, '34.060')] -[2023-10-14 16:26:34,292][75950] Updated weights for policy 1, policy_version 71880 (0.0008) -[2023-10-14 16:26:34,658][75950] Updated weights for policy 1, policy_version 71890 (0.0008) -[2023-10-14 16:26:35,028][75950] Updated weights for policy 1, policy_version 71900 (0.0010) -[2023-10-14 16:26:35,820][75949] Updated weights for policy 0, policy_version 72071 (0.0009) -[2023-10-14 16:26:36,184][75949] Updated weights for policy 0, policy_version 72081 (0.0009) -[2023-10-14 16:26:36,553][75949] Updated weights for policy 0, policy_version 72091 (0.0009) -[2023-10-14 16:26:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 147456000. Throughput: 0: 1669.1, 1: 1687.1. Samples: 36871176. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) -[2023-10-14 16:26:38,165][74987] Avg episode reward: [(0, '25.650'), (1, '34.870')] -[2023-10-14 16:26:39,094][75950] Updated weights for policy 1, policy_version 71910 (0.0007) -[2023-10-14 16:26:39,463][75950] Updated weights for policy 1, policy_version 71920 (0.0007) -[2023-10-14 16:26:39,826][75950] Updated weights for policy 1, policy_version 71930 (0.0009) -[2023-10-14 16:26:40,288][75949] Updated weights for policy 0, policy_version 72101 (0.0009) -[2023-10-14 16:26:40,651][75949] Updated weights for policy 0, policy_version 72111 (0.0008) -[2023-10-14 16:26:41,019][75949] Updated weights for policy 0, policy_version 72121 (0.0007) -[2023-10-14 16:26:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 147521536. Throughput: 0: 1698.7, 1: 1689.5. Samples: 36892228. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) -[2023-10-14 16:26:43,165][74987] Avg episode reward: [(0, '25.190'), (1, '30.820')] -[2023-10-14 16:26:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000072128_73859072.pth... -[2023-10-14 16:26:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000071936_73662464.pth... -[2023-10-14 16:26:43,212][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000070368_72056832.pth -[2023-10-14 16:26:43,216][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000070560_72253440.pth -[2023-10-14 16:26:44,026][75950] Updated weights for policy 1, policy_version 71940 (0.0009) -[2023-10-14 16:26:44,431][75950] Updated weights for policy 1, policy_version 71950 (0.0009) -[2023-10-14 16:26:44,792][75950] Updated weights for policy 1, policy_version 71960 (0.0008) -[2023-10-14 16:26:45,110][75949] Updated weights for policy 0, policy_version 72131 (0.0008) -[2023-10-14 16:26:45,507][75949] Updated weights for policy 0, policy_version 72141 (0.0009) -[2023-10-14 16:26:45,876][75949] Updated weights for policy 0, policy_version 72151 (0.0008) -[2023-10-14 16:26:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 147587072. Throughput: 0: 1684.2, 1: 1673.0. Samples: 36901772. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) -[2023-10-14 16:26:48,165][74987] Avg episode reward: [(0, '27.870'), (1, '32.440')] -[2023-10-14 16:26:48,826][75950] Updated weights for policy 1, policy_version 71970 (0.0007) -[2023-10-14 16:26:49,200][75950] Updated weights for policy 1, policy_version 71980 (0.0008) -[2023-10-14 16:26:49,573][75950] Updated weights for policy 1, policy_version 71990 (0.0008) -[2023-10-14 16:26:49,881][75949] Updated weights for policy 0, policy_version 72161 (0.0008) -[2023-10-14 16:26:49,931][75950] Updated weights for policy 1, policy_version 72000 (0.0007) -[2023-10-14 16:26:50,245][75949] Updated weights for policy 0, policy_version 72171 (0.0010) -[2023-10-14 16:26:50,609][75949] Updated weights for policy 0, policy_version 72181 (0.0010) -[2023-10-14 16:26:50,979][75949] Updated weights for policy 0, policy_version 72191 (0.0010) -[2023-10-14 16:26:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 147652608. Throughput: 0: 1675.0, 1: 1692.9. Samples: 36922014. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) -[2023-10-14 16:26:53,165][74987] Avg episode reward: [(0, '26.240'), (1, '34.140')] -[2023-10-14 16:26:53,939][75950] Updated weights for policy 1, policy_version 72010 (0.0007) -[2023-10-14 16:26:54,300][75950] Updated weights for policy 1, policy_version 72020 (0.0009) -[2023-10-14 16:26:54,664][75950] Updated weights for policy 1, policy_version 72030 (0.0009) -[2023-10-14 16:26:55,005][75949] Updated weights for policy 0, policy_version 72201 (0.0008) -[2023-10-14 16:26:55,381][75949] Updated weights for policy 0, policy_version 72211 (0.0008) -[2023-10-14 16:26:55,752][75949] Updated weights for policy 0, policy_version 72221 (0.0008) -[2023-10-14 16:26:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 147718144. Throughput: 0: 1695.2, 1: 1695.1. Samples: 36942872. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) -[2023-10-14 16:26:58,164][74987] Avg episode reward: [(0, '27.740'), (1, '31.420')] -[2023-10-14 16:26:58,693][75950] Updated weights for policy 1, policy_version 72040 (0.0011) -[2023-10-14 16:26:59,054][75950] Updated weights for policy 1, policy_version 72050 (0.0010) -[2023-10-14 16:26:59,421][75950] Updated weights for policy 1, policy_version 72060 (0.0010) -[2023-10-14 16:26:59,741][75949] Updated weights for policy 0, policy_version 72231 (0.0009) -[2023-10-14 16:27:00,110][75949] Updated weights for policy 0, policy_version 72241 (0.0008) -[2023-10-14 16:27:00,486][75949] Updated weights for policy 0, policy_version 72251 (0.0009) -[2023-10-14 16:27:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 147783680. Throughput: 0: 1667.3, 1: 1693.7. Samples: 36952284. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) -[2023-10-14 16:27:03,164][74987] Avg episode reward: [(0, '24.440'), (1, '32.950')] -[2023-10-14 16:27:03,467][75950] Updated weights for policy 1, policy_version 72070 (0.0008) -[2023-10-14 16:27:03,833][75950] Updated weights for policy 1, policy_version 72080 (0.0007) -[2023-10-14 16:27:04,205][75950] Updated weights for policy 1, policy_version 72090 (0.0007) -[2023-10-14 16:27:04,527][75949] Updated weights for policy 0, policy_version 72261 (0.0008) -[2023-10-14 16:27:04,899][75949] Updated weights for policy 0, policy_version 72271 (0.0009) -[2023-10-14 16:27:05,269][75949] Updated weights for policy 0, policy_version 72281 (0.0008) -[2023-10-14 16:27:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 147849216. Throughput: 0: 1686.7, 1: 1704.2. Samples: 36973100. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 16:27:08,165][74987] Avg episode reward: [(0, '28.760'), (1, '33.960')] -[2023-10-14 16:27:08,197][75950] Updated weights for policy 1, policy_version 72100 (0.0008) -[2023-10-14 16:27:08,566][75950] Updated weights for policy 1, policy_version 72110 (0.0007) -[2023-10-14 16:27:08,939][75950] Updated weights for policy 1, policy_version 72120 (0.0007) -[2023-10-14 16:27:09,430][75949] Updated weights for policy 0, policy_version 72291 (0.0009) -[2023-10-14 16:27:09,791][75949] Updated weights for policy 0, policy_version 72301 (0.0010) -[2023-10-14 16:27:10,165][75949] Updated weights for policy 0, policy_version 72311 (0.0011) -[2023-10-14 16:27:12,766][75950] Updated weights for policy 1, policy_version 72130 (0.0008) -[2023-10-14 16:27:13,130][75950] Updated weights for policy 1, policy_version 72140 (0.0007) -[2023-10-14 16:27:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 147914752. Throughput: 0: 1696.9, 1: 1701.0. Samples: 36993802. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 16:27:13,164][74987] Avg episode reward: [(0, '26.000'), (1, '32.150')] -[2023-10-14 16:27:13,486][75950] Updated weights for policy 1, policy_version 72150 (0.0007) -[2023-10-14 16:27:13,852][75950] Updated weights for policy 1, policy_version 72160 (0.0008) -[2023-10-14 16:27:14,115][75949] Updated weights for policy 0, policy_version 72321 (0.0011) -[2023-10-14 16:27:14,493][75949] Updated weights for policy 0, policy_version 72331 (0.0010) -[2023-10-14 16:27:14,865][75949] Updated weights for policy 0, policy_version 72341 (0.0008) -[2023-10-14 16:27:15,229][75949] Updated weights for policy 0, policy_version 72351 (0.0008) -[2023-10-14 16:27:17,963][75950] Updated weights for policy 1, policy_version 72170 (0.0008) -[2023-10-14 16:27:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 147980288. Throughput: 0: 1668.8, 1: 1700.9. Samples: 37003006. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 16:27:18,165][74987] Avg episode reward: [(0, '29.440'), (1, '30.600')] -[2023-10-14 16:27:18,329][75950] Updated weights for policy 1, policy_version 72180 (0.0009) -[2023-10-14 16:27:18,695][75950] Updated weights for policy 1, policy_version 72190 (0.0009) -[2023-10-14 16:27:19,248][75949] Updated weights for policy 0, policy_version 72361 (0.0007) -[2023-10-14 16:27:19,615][75949] Updated weights for policy 0, policy_version 72371 (0.0008) -[2023-10-14 16:27:19,989][75949] Updated weights for policy 0, policy_version 72381 (0.0008) -[2023-10-14 16:27:22,832][75950] Updated weights for policy 1, policy_version 72200 (0.0010) -[2023-10-14 16:27:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148045824. Throughput: 0: 1699.1, 1: 1694.3. Samples: 37023876. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 16:27:23,164][74987] Avg episode reward: [(0, '26.520'), (1, '30.990')] -[2023-10-14 16:27:23,193][75950] Updated weights for policy 1, policy_version 72210 (0.0008) -[2023-10-14 16:27:23,561][75950] Updated weights for policy 1, policy_version 72220 (0.0007) -[2023-10-14 16:27:24,002][75949] Updated weights for policy 0, policy_version 72391 (0.0009) -[2023-10-14 16:27:24,366][75949] Updated weights for policy 0, policy_version 72401 (0.0009) -[2023-10-14 16:27:24,738][75949] Updated weights for policy 0, policy_version 72411 (0.0010) -[2023-10-14 16:27:27,790][75950] Updated weights for policy 1, policy_version 72230 (0.0008) -[2023-10-14 16:27:28,159][75950] Updated weights for policy 1, policy_version 72240 (0.0008) -[2023-10-14 16:27:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148111360. Throughput: 0: 1694.9, 1: 1690.1. Samples: 37044548. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 16:27:28,164][74987] Avg episode reward: [(0, '28.810'), (1, '33.260')] -[2023-10-14 16:27:28,528][75950] Updated weights for policy 1, policy_version 72250 (0.0010) -[2023-10-14 16:27:28,869][75949] Updated weights for policy 0, policy_version 72421 (0.0010) -[2023-10-14 16:27:29,245][75949] Updated weights for policy 0, policy_version 72431 (0.0008) -[2023-10-14 16:27:29,613][75949] Updated weights for policy 0, policy_version 72441 (0.0009) -[2023-10-14 16:27:32,554][75950] Updated weights for policy 1, policy_version 72260 (0.0009) -[2023-10-14 16:27:32,953][75950] Updated weights for policy 1, policy_version 72270 (0.0008) -[2023-10-14 16:27:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148176896. Throughput: 0: 1685.7, 1: 1693.5. Samples: 37053834. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 16:27:33,164][74987] Avg episode reward: [(0, '27.930'), (1, '30.300')] -[2023-10-14 16:27:33,318][75950] Updated weights for policy 1, policy_version 72280 (0.0009) -[2023-10-14 16:27:33,676][75949] Updated weights for policy 0, policy_version 72451 (0.0008) -[2023-10-14 16:27:34,072][75949] Updated weights for policy 0, policy_version 72461 (0.0009) -[2023-10-14 16:27:34,431][75949] Updated weights for policy 0, policy_version 72471 (0.0008) -[2023-10-14 16:27:37,383][75950] Updated weights for policy 1, policy_version 72290 (0.0009) -[2023-10-14 16:27:37,752][75950] Updated weights for policy 1, policy_version 72300 (0.0009) -[2023-10-14 16:27:38,124][75950] Updated weights for policy 1, policy_version 72310 (0.0008) -[2023-10-14 16:27:38,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 148242432. Throughput: 0: 1698.3, 1: 1686.4. Samples: 37074326. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 16:27:38,164][74987] Avg episode reward: [(0, '28.260'), (1, '30.080')] -[2023-10-14 16:27:38,436][75949] Updated weights for policy 0, policy_version 72481 (0.0008) -[2023-10-14 16:27:38,489][75950] Updated weights for policy 1, policy_version 72320 (0.0008) -[2023-10-14 16:27:38,802][75949] Updated weights for policy 0, policy_version 72491 (0.0007) -[2023-10-14 16:27:39,183][75949] Updated weights for policy 0, policy_version 72501 (0.0008) -[2023-10-14 16:27:39,553][75949] Updated weights for policy 0, policy_version 72511 (0.0008) -[2023-10-14 16:27:42,519][75950] Updated weights for policy 1, policy_version 72330 (0.0008) -[2023-10-14 16:27:42,884][75950] Updated weights for policy 1, policy_version 72340 (0.0008) -[2023-10-14 16:27:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148307968. Throughput: 0: 1700.2, 1: 1675.5. Samples: 37094780. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 16:27:43,164][74987] Avg episode reward: [(0, '26.780'), (1, '32.890')] -[2023-10-14 16:27:43,246][75950] Updated weights for policy 1, policy_version 72350 (0.0009) -[2023-10-14 16:27:43,523][75949] Updated weights for policy 0, policy_version 72521 (0.0010) -[2023-10-14 16:27:43,896][75949] Updated weights for policy 0, policy_version 72531 (0.0008) -[2023-10-14 16:27:44,268][75949] Updated weights for policy 0, policy_version 72541 (0.0008) -[2023-10-14 16:27:47,357][75950] Updated weights for policy 1, policy_version 72360 (0.0010) -[2023-10-14 16:27:47,728][75950] Updated weights for policy 1, policy_version 72370 (0.0009) -[2023-10-14 16:27:48,091][75950] Updated weights for policy 1, policy_version 72380 (0.0009) -[2023-10-14 16:27:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148373504. Throughput: 0: 1691.6, 1: 1686.8. Samples: 37104314. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 16:27:48,164][74987] Avg episode reward: [(0, '28.090'), (1, '35.140')] -[2023-10-14 16:27:48,350][75949] Updated weights for policy 0, policy_version 72551 (0.0009) -[2023-10-14 16:27:48,721][75949] Updated weights for policy 0, policy_version 72561 (0.0007) -[2023-10-14 16:27:49,088][75949] Updated weights for policy 0, policy_version 72571 (0.0008) -[2023-10-14 16:27:52,246][75950] Updated weights for policy 1, policy_version 72390 (0.0011) -[2023-10-14 16:27:52,615][75950] Updated weights for policy 1, policy_version 72400 (0.0009) -[2023-10-14 16:27:52,975][75950] Updated weights for policy 1, policy_version 72410 (0.0008) -[2023-10-14 16:27:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148439040. Throughput: 0: 1684.7, 1: 1678.8. Samples: 37124460. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) -[2023-10-14 16:27:53,165][74987] Avg episode reward: [(0, '28.280'), (1, '33.410')] -[2023-10-14 16:27:53,327][75949] Updated weights for policy 0, policy_version 72581 (0.0008) -[2023-10-14 16:27:53,702][75949] Updated weights for policy 0, policy_version 72591 (0.0008) -[2023-10-14 16:27:54,076][75949] Updated weights for policy 0, policy_version 72601 (0.0008) -[2023-10-14 16:27:57,074][75950] Updated weights for policy 1, policy_version 72420 (0.0007) -[2023-10-14 16:27:57,449][75950] Updated weights for policy 1, policy_version 72430 (0.0010) -[2023-10-14 16:27:57,816][75950] Updated weights for policy 1, policy_version 72440 (0.0007) -[2023-10-14 16:27:58,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 148537344. Throughput: 0: 1684.2, 1: 1662.8. Samples: 37144420. Policy #0 lag: (min: 0.0, avg: 18.5, max: 32.0) -[2023-10-14 16:27:58,164][74987] Avg episode reward: [(0, '27.740'), (1, '34.490')] -[2023-10-14 16:27:58,175][75949] Updated weights for policy 0, policy_version 72611 (0.0009) -[2023-10-14 16:27:58,550][75949] Updated weights for policy 0, policy_version 72621 (0.0011) -[2023-10-14 16:27:58,929][75949] Updated weights for policy 0, policy_version 72631 (0.0008) -[2023-10-14 16:28:01,876][75950] Updated weights for policy 1, policy_version 72450 (0.0008) -[2023-10-14 16:28:02,242][75950] Updated weights for policy 1, policy_version 72460 (0.0009) -[2023-10-14 16:28:02,612][75950] Updated weights for policy 1, policy_version 72470 (0.0011) -[2023-10-14 16:28:02,883][75949] Updated weights for policy 0, policy_version 72641 (0.0008) -[2023-10-14 16:28:02,979][75950] Updated weights for policy 1, policy_version 72480 (0.0010) -[2023-10-14 16:28:03,163][74987] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 148602880. Throughput: 0: 1684.1, 1: 1678.8. Samples: 37154332. Policy #0 lag: (min: 0.0, avg: 18.5, max: 32.0) -[2023-10-14 16:28:03,164][74987] Avg episode reward: [(0, '28.570'), (1, '35.090')] -[2023-10-14 16:28:03,253][75949] Updated weights for policy 0, policy_version 72651 (0.0008) -[2023-10-14 16:28:03,617][75949] Updated weights for policy 0, policy_version 72661 (0.0007) -[2023-10-14 16:28:03,985][75949] Updated weights for policy 0, policy_version 72671 (0.0009) -[2023-10-14 16:28:07,067][75950] Updated weights for policy 1, policy_version 72490 (0.0010) -[2023-10-14 16:28:07,441][75950] Updated weights for policy 1, policy_version 72500 (0.0008) -[2023-10-14 16:28:07,812][75950] Updated weights for policy 1, policy_version 72510 (0.0010) -[2023-10-14 16:28:08,157][75949] Updated weights for policy 0, policy_version 72681 (0.0010) -[2023-10-14 16:28:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 148668416. Throughput: 0: 1679.6, 1: 1682.7. Samples: 37175182. Policy #0 lag: (min: 0.0, avg: 18.5, max: 32.0) -[2023-10-14 16:28:08,165][74987] Avg episode reward: [(0, '26.840'), (1, '31.180')] -[2023-10-14 16:28:08,521][75949] Updated weights for policy 0, policy_version 72691 (0.0011) -[2023-10-14 16:28:08,884][75949] Updated weights for policy 0, policy_version 72701 (0.0009) -[2023-10-14 16:28:11,966][75950] Updated weights for policy 1, policy_version 72520 (0.0009) -[2023-10-14 16:28:12,344][75950] Updated weights for policy 1, policy_version 72530 (0.0008) -[2023-10-14 16:28:12,708][75950] Updated weights for policy 1, policy_version 72540 (0.0009) -[2023-10-14 16:28:12,851][75949] Updated weights for policy 0, policy_version 72711 (0.0009) -[2023-10-14 16:28:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 148733952. Throughput: 0: 1673.0, 1: 1662.8. Samples: 37194662. Policy #0 lag: (min: 0.0, avg: 18.5, max: 32.0) -[2023-10-14 16:28:13,165][74987] Avg episode reward: [(0, '29.560'), (1, '31.270')] -[2023-10-14 16:28:13,213][75949] Updated weights for policy 0, policy_version 72721 (0.0008) -[2023-10-14 16:28:13,594][75949] Updated weights for policy 0, policy_version 72731 (0.0008) -[2023-10-14 16:28:16,843][75950] Updated weights for policy 1, policy_version 72550 (0.0007) -[2023-10-14 16:28:17,208][75950] Updated weights for policy 1, policy_version 72560 (0.0009) -[2023-10-14 16:28:17,573][75950] Updated weights for policy 1, policy_version 72570 (0.0008) -[2023-10-14 16:28:17,653][75949] Updated weights for policy 0, policy_version 72741 (0.0009) -[2023-10-14 16:28:18,036][75949] Updated weights for policy 0, policy_version 72751 (0.0008) -[2023-10-14 16:28:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 148799488. Throughput: 0: 1675.3, 1: 1676.4. Samples: 37204664. Policy #0 lag: (min: 0.0, avg: 18.5, max: 32.0) -[2023-10-14 16:28:18,165][74987] Avg episode reward: [(0, '26.580'), (1, '32.930')] -[2023-10-14 16:28:18,422][75949] Updated weights for policy 0, policy_version 72761 (0.0009) -[2023-10-14 16:28:21,715][75950] Updated weights for policy 1, policy_version 72580 (0.0007) -[2023-10-14 16:28:22,122][75950] Updated weights for policy 1, policy_version 72590 (0.0009) -[2023-10-14 16:28:22,480][75950] Updated weights for policy 1, policy_version 72600 (0.0007) -[2023-10-14 16:28:22,711][75949] Updated weights for policy 0, policy_version 72771 (0.0008) -[2023-10-14 16:28:23,095][75949] Updated weights for policy 0, policy_version 72781 (0.0009) -[2023-10-14 16:28:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 148865024. Throughput: 0: 1674.5, 1: 1673.9. Samples: 37225006. Policy #0 lag: (min: 0.0, avg: 18.5, max: 32.0) -[2023-10-14 16:28:23,165][74987] Avg episode reward: [(0, '28.880'), (1, '33.010')] -[2023-10-14 16:28:23,473][75949] Updated weights for policy 0, policy_version 72791 (0.0009) -[2023-10-14 16:28:26,654][75950] Updated weights for policy 1, policy_version 72610 (0.0008) -[2023-10-14 16:28:27,018][75950] Updated weights for policy 1, policy_version 72620 (0.0008) -[2023-10-14 16:28:27,328][75949] Updated weights for policy 0, policy_version 72801 (0.0010) -[2023-10-14 16:28:27,378][75950] Updated weights for policy 1, policy_version 72630 (0.0007) -[2023-10-14 16:28:27,705][75949] Updated weights for policy 0, policy_version 72811 (0.0009) -[2023-10-14 16:28:27,743][75950] Updated weights for policy 1, policy_version 72640 (0.0007) -[2023-10-14 16:28:28,077][75949] Updated weights for policy 0, policy_version 72821 (0.0007) -[2023-10-14 16:28:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 148930560. Throughput: 0: 1664.9, 1: 1656.9. Samples: 37244262. Policy #0 lag: (min: 0.0, avg: 18.5, max: 32.0) -[2023-10-14 16:28:28,165][74987] Avg episode reward: [(0, '24.550'), (1, '32.410')] -[2023-10-14 16:28:28,447][75949] Updated weights for policy 0, policy_version 72831 (0.0009) -[2023-10-14 16:28:31,847][75950] Updated weights for policy 1, policy_version 72650 (0.0007) -[2023-10-14 16:28:32,212][75950] Updated weights for policy 1, policy_version 72660 (0.0007) -[2023-10-14 16:28:32,580][75950] Updated weights for policy 1, policy_version 72670 (0.0008) -[2023-10-14 16:28:32,606][75949] Updated weights for policy 0, policy_version 72841 (0.0008) -[2023-10-14 16:28:32,974][75949] Updated weights for policy 0, policy_version 72851 (0.0010) -[2023-10-14 16:28:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 148996096. Throughput: 0: 1673.3, 1: 1671.1. Samples: 37254810. Policy #0 lag: (min: 0.0, avg: 18.5, max: 32.0) -[2023-10-14 16:28:33,164][74987] Avg episode reward: [(0, '27.670'), (1, '33.010')] -[2023-10-14 16:28:33,350][75949] Updated weights for policy 0, policy_version 72861 (0.0009) -[2023-10-14 16:28:36,486][75950] Updated weights for policy 1, policy_version 72680 (0.0010) -[2023-10-14 16:28:36,842][75950] Updated weights for policy 1, policy_version 72690 (0.0011) -[2023-10-14 16:28:37,217][75950] Updated weights for policy 1, policy_version 72700 (0.0009) -[2023-10-14 16:28:37,528][75949] Updated weights for policy 0, policy_version 72871 (0.0009) -[2023-10-14 16:28:37,902][75949] Updated weights for policy 0, policy_version 72881 (0.0008) -[2023-10-14 16:28:38,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 149061632. Throughput: 0: 1673.6, 1: 1663.4. Samples: 37274624. Policy #0 lag: (min: 0.0, avg: 18.5, max: 32.0) -[2023-10-14 16:28:38,164][74987] Avg episode reward: [(0, '24.280'), (1, '35.030')] -[2023-10-14 16:28:38,269][75949] Updated weights for policy 0, policy_version 72891 (0.0007) -[2023-10-14 16:28:41,266][75950] Updated weights for policy 1, policy_version 72710 (0.0010) -[2023-10-14 16:28:41,639][75950] Updated weights for policy 1, policy_version 72720 (0.0010) -[2023-10-14 16:28:42,005][75950] Updated weights for policy 1, policy_version 72730 (0.0008) -[2023-10-14 16:28:42,411][75949] Updated weights for policy 0, policy_version 72901 (0.0008) -[2023-10-14 16:28:42,796][75949] Updated weights for policy 0, policy_version 72911 (0.0007) -[2023-10-14 16:28:43,163][75949] Updated weights for policy 0, policy_version 72921 (0.0007) -[2023-10-14 16:28:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 149127168. Throughput: 0: 1663.1, 1: 1661.7. Samples: 37294036. Policy #0 lag: (min: 0.0, avg: 18.5, max: 32.0) -[2023-10-14 16:28:43,164][74987] Avg episode reward: [(0, '28.030'), (1, '33.100')] -[2023-10-14 16:28:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000072736_74481664.pth... -[2023-10-14 16:28:43,206][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000071168_72876032.pth -[2023-10-14 16:28:43,418][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000072928_74678272.pth... -[2023-10-14 16:28:43,456][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000071328_73039872.pth -[2023-10-14 16:28:46,025][75950] Updated weights for policy 1, policy_version 72740 (0.0010) -[2023-10-14 16:28:46,397][75950] Updated weights for policy 1, policy_version 72750 (0.0009) -[2023-10-14 16:28:46,762][75950] Updated weights for policy 1, policy_version 72760 (0.0011) -[2023-10-14 16:28:47,138][75949] Updated weights for policy 0, policy_version 72931 (0.0009) -[2023-10-14 16:28:47,510][75949] Updated weights for policy 0, policy_version 72941 (0.0008) -[2023-10-14 16:28:47,879][75949] Updated weights for policy 0, policy_version 72951 (0.0008) -[2023-10-14 16:28:48,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 149192704. Throughput: 0: 1674.7, 1: 1674.3. Samples: 37305038. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) -[2023-10-14 16:28:48,165][74987] Avg episode reward: [(0, '25.510'), (1, '34.180')] -[2023-10-14 16:28:51,071][75950] Updated weights for policy 1, policy_version 72770 (0.0010) -[2023-10-14 16:28:51,433][75950] Updated weights for policy 1, policy_version 72780 (0.0008) -[2023-10-14 16:28:51,795][75950] Updated weights for policy 1, policy_version 72790 (0.0010) -[2023-10-14 16:28:51,972][75949] Updated weights for policy 0, policy_version 72961 (0.0007) -[2023-10-14 16:28:52,162][75950] Updated weights for policy 1, policy_version 72800 (0.0008) -[2023-10-14 16:28:52,344][75949] Updated weights for policy 0, policy_version 72971 (0.0008) -[2023-10-14 16:28:52,713][75949] Updated weights for policy 0, policy_version 72981 (0.0008) -[2023-10-14 16:28:53,090][75949] Updated weights for policy 0, policy_version 72991 (0.0007) -[2023-10-14 16:28:53,164][74987] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 149291008. Throughput: 0: 1674.7, 1: 1652.0. Samples: 37324882. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) -[2023-10-14 16:28:53,164][74987] Avg episode reward: [(0, '27.950'), (1, '35.770')] -[2023-10-14 16:28:56,300][75950] Updated weights for policy 1, policy_version 72810 (0.0012) -[2023-10-14 16:28:56,670][75950] Updated weights for policy 1, policy_version 72820 (0.0008) -[2023-10-14 16:28:56,885][75949] Updated weights for policy 0, policy_version 73001 (0.0008) -[2023-10-14 16:28:57,032][75950] Updated weights for policy 1, policy_version 72830 (0.0007) -[2023-10-14 16:28:57,256][75949] Updated weights for policy 0, policy_version 73011 (0.0007) -[2023-10-14 16:28:57,634][75949] Updated weights for policy 0, policy_version 73021 (0.0009) -[2023-10-14 16:28:58,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 149356544. Throughput: 0: 1660.8, 1: 1659.6. Samples: 37344080. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) -[2023-10-14 16:28:58,164][74987] Avg episode reward: [(0, '25.510'), (1, '35.890')] -[2023-10-14 16:29:01,089][75950] Updated weights for policy 1, policy_version 72840 (0.0008) -[2023-10-14 16:29:01,450][75950] Updated weights for policy 1, policy_version 72850 (0.0009) -[2023-10-14 16:29:01,760][75949] Updated weights for policy 0, policy_version 73031 (0.0009) -[2023-10-14 16:29:01,816][75950] Updated weights for policy 1, policy_version 72860 (0.0007) -[2023-10-14 16:29:02,132][75949] Updated weights for policy 0, policy_version 73041 (0.0009) -[2023-10-14 16:29:02,505][75949] Updated weights for policy 0, policy_version 73051 (0.0008) -[2023-10-14 16:29:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 149422080. Throughput: 0: 1683.0, 1: 1671.6. Samples: 37355620. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) -[2023-10-14 16:29:03,165][74987] Avg episode reward: [(0, '27.570'), (1, '33.790')] -[2023-10-14 16:29:05,795][75950] Updated weights for policy 1, policy_version 72870 (0.0008) -[2023-10-14 16:29:06,158][75950] Updated weights for policy 1, policy_version 72880 (0.0009) -[2023-10-14 16:29:06,523][75950] Updated weights for policy 1, policy_version 72890 (0.0008) -[2023-10-14 16:29:06,551][75949] Updated weights for policy 0, policy_version 73061 (0.0007) -[2023-10-14 16:29:06,923][75949] Updated weights for policy 0, policy_version 73071 (0.0008) -[2023-10-14 16:29:07,283][75949] Updated weights for policy 0, policy_version 73081 (0.0011) -[2023-10-14 16:29:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 149487616. Throughput: 0: 1679.8, 1: 1653.8. Samples: 37375020. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) -[2023-10-14 16:29:08,165][74987] Avg episode reward: [(0, '25.920'), (1, '34.080')] -[2023-10-14 16:29:10,751][75950] Updated weights for policy 1, policy_version 72900 (0.0008) -[2023-10-14 16:29:11,129][75950] Updated weights for policy 1, policy_version 72910 (0.0009) -[2023-10-14 16:29:11,496][75950] Updated weights for policy 1, policy_version 72920 (0.0010) -[2023-10-14 16:29:11,546][75949] Updated weights for policy 0, policy_version 73091 (0.0010) -[2023-10-14 16:29:11,949][75949] Updated weights for policy 0, policy_version 73101 (0.0009) -[2023-10-14 16:29:12,316][75949] Updated weights for policy 0, policy_version 73111 (0.0010) -[2023-10-14 16:29:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 149553152. Throughput: 0: 1657.5, 1: 1674.5. Samples: 37394200. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) -[2023-10-14 16:29:13,164][74987] Avg episode reward: [(0, '26.430'), (1, '34.420')] -[2023-10-14 16:29:15,673][75950] Updated weights for policy 1, policy_version 72930 (0.0009) -[2023-10-14 16:29:16,036][75950] Updated weights for policy 1, policy_version 72940 (0.0010) -[2023-10-14 16:29:16,389][75949] Updated weights for policy 0, policy_version 73121 (0.0009) -[2023-10-14 16:29:16,408][75950] Updated weights for policy 1, policy_version 72950 (0.0008) -[2023-10-14 16:29:16,749][75949] Updated weights for policy 0, policy_version 73131 (0.0008) -[2023-10-14 16:29:16,767][75950] Updated weights for policy 1, policy_version 72960 (0.0008) -[2023-10-14 16:29:17,128][75949] Updated weights for policy 0, policy_version 73141 (0.0008) -[2023-10-14 16:29:17,501][75949] Updated weights for policy 0, policy_version 73151 (0.0007) -[2023-10-14 16:29:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 149618688. Throughput: 0: 1676.9, 1: 1673.1. Samples: 37405562. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) -[2023-10-14 16:29:18,165][74987] Avg episode reward: [(0, '27.070'), (1, '31.240')] -[2023-10-14 16:29:20,855][75950] Updated weights for policy 1, policy_version 72970 (0.0007) -[2023-10-14 16:29:21,229][75950] Updated weights for policy 1, policy_version 72980 (0.0008) -[2023-10-14 16:29:21,488][75949] Updated weights for policy 0, policy_version 73161 (0.0009) -[2023-10-14 16:29:21,592][75950] Updated weights for policy 1, policy_version 72990 (0.0010) -[2023-10-14 16:29:21,864][75949] Updated weights for policy 0, policy_version 73171 (0.0008) -[2023-10-14 16:29:22,239][75949] Updated weights for policy 0, policy_version 73181 (0.0008) -[2023-10-14 16:29:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 149684224. Throughput: 0: 1674.1, 1: 1661.4. Samples: 37424722. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) -[2023-10-14 16:29:23,164][74987] Avg episode reward: [(0, '25.970'), (1, '32.220')] -[2023-10-14 16:29:25,849][75950] Updated weights for policy 1, policy_version 73000 (0.0008) -[2023-10-14 16:29:26,217][75950] Updated weights for policy 1, policy_version 73010 (0.0009) -[2023-10-14 16:29:26,418][75949] Updated weights for policy 0, policy_version 73191 (0.0009) -[2023-10-14 16:29:26,578][75950] Updated weights for policy 1, policy_version 73020 (0.0008) -[2023-10-14 16:29:26,782][75949] Updated weights for policy 0, policy_version 73201 (0.0008) -[2023-10-14 16:29:27,152][75949] Updated weights for policy 0, policy_version 73211 (0.0008) -[2023-10-14 16:29:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 149749760. Throughput: 0: 1668.4, 1: 1674.8. Samples: 37444478. Policy #0 lag: (min: 26.0, avg: 28.8, max: 58.0) -[2023-10-14 16:29:28,165][74987] Avg episode reward: [(0, '28.130'), (1, '34.220')] -[2023-10-14 16:29:30,498][75950] Updated weights for policy 1, policy_version 73030 (0.0008) -[2023-10-14 16:29:30,871][75950] Updated weights for policy 1, policy_version 73040 (0.0009) -[2023-10-14 16:29:31,140][75949] Updated weights for policy 0, policy_version 73221 (0.0009) -[2023-10-14 16:29:31,229][75950] Updated weights for policy 1, policy_version 73050 (0.0008) -[2023-10-14 16:29:31,503][75949] Updated weights for policy 0, policy_version 73231 (0.0009) -[2023-10-14 16:29:31,875][75949] Updated weights for policy 0, policy_version 73241 (0.0009) -[2023-10-14 16:29:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 149815296. Throughput: 0: 1684.7, 1: 1666.5. Samples: 37455840. Policy #0 lag: (min: 10.0, avg: 19.9, max: 42.0) -[2023-10-14 16:29:33,164][74987] Avg episode reward: [(0, '25.360'), (1, '32.960')] -[2023-10-14 16:29:35,350][75950] Updated weights for policy 1, policy_version 73060 (0.0008) -[2023-10-14 16:29:35,714][75950] Updated weights for policy 1, policy_version 73070 (0.0009) -[2023-10-14 16:29:35,833][75949] Updated weights for policy 0, policy_version 73251 (0.0008) -[2023-10-14 16:29:36,069][75950] Updated weights for policy 1, policy_version 73080 (0.0009) -[2023-10-14 16:29:36,205][75949] Updated weights for policy 0, policy_version 73261 (0.0009) -[2023-10-14 16:29:36,579][75949] Updated weights for policy 0, policy_version 73271 (0.0010) -[2023-10-14 16:29:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 149880832. Throughput: 0: 1668.5, 1: 1666.1. Samples: 37474942. Policy #0 lag: (min: 10.0, avg: 19.9, max: 42.0) -[2023-10-14 16:29:38,165][74987] Avg episode reward: [(0, '29.140'), (1, '33.510')] -[2023-10-14 16:29:40,299][75950] Updated weights for policy 1, policy_version 73090 (0.0010) -[2023-10-14 16:29:40,601][75949] Updated weights for policy 0, policy_version 73281 (0.0008) -[2023-10-14 16:29:40,658][75950] Updated weights for policy 1, policy_version 73100 (0.0007) -[2023-10-14 16:29:40,969][75949] Updated weights for policy 0, policy_version 73291 (0.0007) -[2023-10-14 16:29:41,023][75950] Updated weights for policy 1, policy_version 73110 (0.0008) -[2023-10-14 16:29:41,337][75949] Updated weights for policy 0, policy_version 73301 (0.0008) -[2023-10-14 16:29:41,378][75950] Updated weights for policy 1, policy_version 73120 (0.0010) -[2023-10-14 16:29:41,699][75949] Updated weights for policy 0, policy_version 73311 (0.0010) -[2023-10-14 16:29:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 149946368. Throughput: 0: 1676.8, 1: 1680.0. Samples: 37495136. Policy #0 lag: (min: 10.0, avg: 19.9, max: 42.0) -[2023-10-14 16:29:43,165][74987] Avg episode reward: [(0, '26.400'), (1, '34.190')] -[2023-10-14 16:29:45,374][75950] Updated weights for policy 1, policy_version 73130 (0.0010) -[2023-10-14 16:29:45,725][75949] Updated weights for policy 0, policy_version 73321 (0.0008) -[2023-10-14 16:29:45,740][75950] Updated weights for policy 1, policy_version 73140 (0.0008) -[2023-10-14 16:29:46,099][75949] Updated weights for policy 0, policy_version 73331 (0.0008) -[2023-10-14 16:29:46,105][75950] Updated weights for policy 1, policy_version 73150 (0.0007) -[2023-10-14 16:29:46,474][75949] Updated weights for policy 0, policy_version 73341 (0.0011) -[2023-10-14 16:29:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 150011904. Throughput: 0: 1678.1, 1: 1661.2. Samples: 37505888. Policy #0 lag: (min: 10.0, avg: 19.9, max: 42.0) -[2023-10-14 16:29:48,164][74987] Avg episode reward: [(0, '28.540'), (1, '32.670')] -[2023-10-14 16:29:50,158][75950] Updated weights for policy 1, policy_version 73160 (0.0010) -[2023-10-14 16:29:50,517][75950] Updated weights for policy 1, policy_version 73170 (0.0009) -[2023-10-14 16:29:50,614][75949] Updated weights for policy 0, policy_version 73351 (0.0008) -[2023-10-14 16:29:50,884][75950] Updated weights for policy 1, policy_version 73180 (0.0007) -[2023-10-14 16:29:50,983][75949] Updated weights for policy 0, policy_version 73361 (0.0008) -[2023-10-14 16:29:51,355][75949] Updated weights for policy 0, policy_version 73371 (0.0010) -[2023-10-14 16:29:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 150077440. Throughput: 0: 1660.6, 1: 1667.6. Samples: 37524788. Policy #0 lag: (min: 10.0, avg: 19.9, max: 42.0) -[2023-10-14 16:29:53,165][74987] Avg episode reward: [(0, '26.330'), (1, '33.240')] -[2023-10-14 16:29:55,005][75950] Updated weights for policy 1, policy_version 73190 (0.0008) -[2023-10-14 16:29:55,363][75950] Updated weights for policy 1, policy_version 73200 (0.0007) -[2023-10-14 16:29:55,469][75949] Updated weights for policy 0, policy_version 73381 (0.0010) -[2023-10-14 16:29:55,733][75950] Updated weights for policy 1, policy_version 73210 (0.0008) -[2023-10-14 16:29:55,832][75949] Updated weights for policy 0, policy_version 73391 (0.0008) -[2023-10-14 16:29:56,199][75949] Updated weights for policy 0, policy_version 73401 (0.0009) -[2023-10-14 16:29:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 150142976. Throughput: 0: 1686.6, 1: 1673.4. Samples: 37545400. Policy #0 lag: (min: 10.0, avg: 19.9, max: 42.0) -[2023-10-14 16:29:58,164][74987] Avg episode reward: [(0, '28.970'), (1, '33.330')] -[2023-10-14 16:29:59,965][75950] Updated weights for policy 1, policy_version 73220 (0.0010) -[2023-10-14 16:30:00,336][75949] Updated weights for policy 0, policy_version 73411 (0.0008) -[2023-10-14 16:30:00,356][75950] Updated weights for policy 1, policy_version 73230 (0.0009) -[2023-10-14 16:30:00,721][75950] Updated weights for policy 1, policy_version 73240 (0.0008) -[2023-10-14 16:30:00,736][75949] Updated weights for policy 0, policy_version 73421 (0.0008) -[2023-10-14 16:30:01,103][75949] Updated weights for policy 0, policy_version 73431 (0.0008) -[2023-10-14 16:30:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 150208512. Throughput: 0: 1674.7, 1: 1656.2. Samples: 37555454. Policy #0 lag: (min: 10.0, avg: 19.9, max: 42.0) -[2023-10-14 16:30:03,165][74987] Avg episode reward: [(0, '27.390'), (1, '32.260')] -[2023-10-14 16:30:04,813][75950] Updated weights for policy 1, policy_version 73250 (0.0008) -[2023-10-14 16:30:05,166][75949] Updated weights for policy 0, policy_version 73441 (0.0009) -[2023-10-14 16:30:05,171][75950] Updated weights for policy 1, policy_version 73260 (0.0008) -[2023-10-14 16:30:05,519][75949] Updated weights for policy 0, policy_version 73451 (0.0007) -[2023-10-14 16:30:05,536][75950] Updated weights for policy 1, policy_version 73270 (0.0010) -[2023-10-14 16:30:05,888][75949] Updated weights for policy 0, policy_version 73461 (0.0008) -[2023-10-14 16:30:05,916][75950] Updated weights for policy 1, policy_version 73280 (0.0009) -[2023-10-14 16:30:06,247][75949] Updated weights for policy 0, policy_version 73471 (0.0010) -[2023-10-14 16:30:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 150274048. Throughput: 0: 1665.3, 1: 1672.1. Samples: 37574906. Policy #0 lag: (min: 10.0, avg: 19.9, max: 42.0) -[2023-10-14 16:30:08,165][74987] Avg episode reward: [(0, '27.580'), (1, '32.880')] -[2023-10-14 16:30:09,916][75950] Updated weights for policy 1, policy_version 73290 (0.0010) -[2023-10-14 16:30:10,284][75950] Updated weights for policy 1, policy_version 73300 (0.0009) -[2023-10-14 16:30:10,404][75949] Updated weights for policy 0, policy_version 73481 (0.0007) -[2023-10-14 16:30:10,657][75950] Updated weights for policy 1, policy_version 73310 (0.0008) -[2023-10-14 16:30:10,779][75949] Updated weights for policy 0, policy_version 73491 (0.0008) -[2023-10-14 16:30:11,141][75949] Updated weights for policy 0, policy_version 73501 (0.0010) -[2023-10-14 16:30:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 150339584. Throughput: 0: 1680.0, 1: 1671.8. Samples: 37595308. Policy #0 lag: (min: 10.0, avg: 19.9, max: 42.0) -[2023-10-14 16:30:13,165][74987] Avg episode reward: [(0, '26.970'), (1, '34.270')] -[2023-10-14 16:30:14,689][75950] Updated weights for policy 1, policy_version 73320 (0.0009) -[2023-10-14 16:30:15,059][75950] Updated weights for policy 1, policy_version 73330 (0.0010) -[2023-10-14 16:30:15,213][75949] Updated weights for policy 0, policy_version 73511 (0.0007) -[2023-10-14 16:30:15,423][75950] Updated weights for policy 1, policy_version 73340 (0.0008) -[2023-10-14 16:30:15,577][75949] Updated weights for policy 0, policy_version 73521 (0.0008) -[2023-10-14 16:30:15,951][75949] Updated weights for policy 0, policy_version 73531 (0.0009) -[2023-10-14 16:30:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 150405120. Throughput: 0: 1664.0, 1: 1650.5. Samples: 37604992. Policy #0 lag: (min: 10.0, avg: 19.9, max: 42.0) -[2023-10-14 16:30:18,165][74987] Avg episode reward: [(0, '27.810'), (1, '34.210')] -[2023-10-14 16:30:19,543][75950] Updated weights for policy 1, policy_version 73350 (0.0008) -[2023-10-14 16:30:19,909][75950] Updated weights for policy 1, policy_version 73360 (0.0009) -[2023-10-14 16:30:20,061][75949] Updated weights for policy 0, policy_version 73541 (0.0009) -[2023-10-14 16:30:20,282][75950] Updated weights for policy 1, policy_version 73370 (0.0008) -[2023-10-14 16:30:20,433][75949] Updated weights for policy 0, policy_version 73551 (0.0008) -[2023-10-14 16:30:20,797][75949] Updated weights for policy 0, policy_version 73561 (0.0009) -[2023-10-14 16:30:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 150470656. Throughput: 0: 1665.6, 1: 1665.6. Samples: 37624844. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 16:30:23,164][74987] Avg episode reward: [(0, '28.120'), (1, '33.350')] -[2023-10-14 16:30:24,398][75950] Updated weights for policy 1, policy_version 73380 (0.0010) -[2023-10-14 16:30:24,767][75950] Updated weights for policy 1, policy_version 73390 (0.0007) -[2023-10-14 16:30:24,907][75949] Updated weights for policy 0, policy_version 73571 (0.0009) -[2023-10-14 16:30:25,120][75950] Updated weights for policy 1, policy_version 73400 (0.0007) -[2023-10-14 16:30:25,283][75949] Updated weights for policy 0, policy_version 73581 (0.0009) -[2023-10-14 16:30:25,663][75949] Updated weights for policy 0, policy_version 73591 (0.0011) -[2023-10-14 16:30:28,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 150536192. Throughput: 0: 1675.6, 1: 1665.7. Samples: 37645490. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 16:30:28,164][74987] Avg episode reward: [(0, '26.820'), (1, '34.100')] -[2023-10-14 16:30:29,152][75950] Updated weights for policy 1, policy_version 73410 (0.0008) -[2023-10-14 16:30:29,526][75950] Updated weights for policy 1, policy_version 73420 (0.0009) -[2023-10-14 16:30:29,641][75949] Updated weights for policy 0, policy_version 73601 (0.0010) -[2023-10-14 16:30:29,891][75950] Updated weights for policy 1, policy_version 73430 (0.0008) -[2023-10-14 16:30:30,014][75949] Updated weights for policy 0, policy_version 73611 (0.0009) -[2023-10-14 16:30:30,249][75950] Updated weights for policy 1, policy_version 73440 (0.0007) -[2023-10-14 16:30:30,376][75949] Updated weights for policy 0, policy_version 73621 (0.0008) -[2023-10-14 16:30:30,743][75949] Updated weights for policy 0, policy_version 73631 (0.0007) -[2023-10-14 16:30:33,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 150601728. Throughput: 0: 1657.0, 1: 1653.9. Samples: 37654882. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 16:30:33,165][74987] Avg episode reward: [(0, '27.740'), (1, '33.890')] -[2023-10-14 16:30:34,529][75950] Updated weights for policy 1, policy_version 73450 (0.0009) -[2023-10-14 16:30:34,892][75950] Updated weights for policy 1, policy_version 73460 (0.0009) -[2023-10-14 16:30:34,930][75949] Updated weights for policy 0, policy_version 73641 (0.0009) -[2023-10-14 16:30:35,258][75950] Updated weights for policy 1, policy_version 73470 (0.0010) -[2023-10-14 16:30:35,298][75949] Updated weights for policy 0, policy_version 73651 (0.0008) -[2023-10-14 16:30:35,665][75949] Updated weights for policy 0, policy_version 73661 (0.0010) -[2023-10-14 16:30:38,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 150667264. Throughput: 0: 1674.9, 1: 1665.2. Samples: 37675094. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 16:30:38,164][74987] Avg episode reward: [(0, '26.810'), (1, '34.420')] -[2023-10-14 16:30:39,513][75950] Updated weights for policy 1, policy_version 73480 (0.0007) -[2023-10-14 16:30:39,577][75949] Updated weights for policy 0, policy_version 73671 (0.0010) -[2023-10-14 16:30:39,877][75950] Updated weights for policy 1, policy_version 73490 (0.0007) -[2023-10-14 16:30:39,953][75949] Updated weights for policy 0, policy_version 73681 (0.0009) -[2023-10-14 16:30:40,241][75950] Updated weights for policy 1, policy_version 73500 (0.0008) -[2023-10-14 16:30:40,326][75949] Updated weights for policy 0, policy_version 73691 (0.0007) -[2023-10-14 16:30:43,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 150732800. Throughput: 0: 1680.8, 1: 1661.5. Samples: 37695806. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 16:30:43,164][74987] Avg episode reward: [(0, '28.990'), (1, '33.820')] -[2023-10-14 16:30:43,178][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000073504_75268096.pth... -[2023-10-14 16:30:43,178][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000073696_75464704.pth... -[2023-10-14 16:30:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000071936_73662464.pth -[2023-10-14 16:30:43,213][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000072128_73859072.pth -[2023-10-14 16:30:44,379][75949] Updated weights for policy 0, policy_version 73701 (0.0008) -[2023-10-14 16:30:44,415][75950] Updated weights for policy 1, policy_version 73510 (0.0008) -[2023-10-14 16:30:44,739][75949] Updated weights for policy 0, policy_version 73711 (0.0008) -[2023-10-14 16:30:44,778][75950] Updated weights for policy 1, policy_version 73520 (0.0008) -[2023-10-14 16:30:45,116][75949] Updated weights for policy 0, policy_version 73721 (0.0009) -[2023-10-14 16:30:45,146][75950] Updated weights for policy 1, policy_version 73530 (0.0008) -[2023-10-14 16:30:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 150798336. Throughput: 0: 1662.6, 1: 1654.4. Samples: 37704718. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 16:30:48,164][74987] Avg episode reward: [(0, '27.090'), (1, '33.590')] -[2023-10-14 16:30:49,144][75950] Updated weights for policy 1, policy_version 73540 (0.0007) -[2023-10-14 16:30:49,246][75949] Updated weights for policy 0, policy_version 73731 (0.0008) -[2023-10-14 16:30:49,509][75950] Updated weights for policy 1, policy_version 73550 (0.0007) -[2023-10-14 16:30:49,626][75949] Updated weights for policy 0, policy_version 73741 (0.0008) -[2023-10-14 16:30:49,888][75950] Updated weights for policy 1, policy_version 73560 (0.0010) -[2023-10-14 16:30:49,990][75949] Updated weights for policy 0, policy_version 73751 (0.0008) -[2023-10-14 16:30:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 150863872. Throughput: 0: 1679.5, 1: 1661.0. Samples: 37725228. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 16:30:53,165][74987] Avg episode reward: [(0, '27.590'), (1, '34.630')] -[2023-10-14 16:30:54,035][75950] Updated weights for policy 1, policy_version 73570 (0.0011) -[2023-10-14 16:30:54,259][75949] Updated weights for policy 0, policy_version 73761 (0.0008) -[2023-10-14 16:30:54,453][75950] Updated weights for policy 1, policy_version 73580 (0.0010) -[2023-10-14 16:30:54,677][75949] Updated weights for policy 0, policy_version 73771 (0.0007) -[2023-10-14 16:30:54,820][75950] Updated weights for policy 1, policy_version 73590 (0.0010) -[2023-10-14 16:30:55,044][75949] Updated weights for policy 0, policy_version 73781 (0.0007) -[2023-10-14 16:30:55,178][75950] Updated weights for policy 1, policy_version 73600 (0.0008) -[2023-10-14 16:30:55,414][75949] Updated weights for policy 0, policy_version 73791 (0.0007) -[2023-10-14 16:30:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 150929408. Throughput: 0: 1680.3, 1: 1662.7. Samples: 37745742. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 16:30:58,165][74987] Avg episode reward: [(0, '27.780'), (1, '31.860')] -[2023-10-14 16:30:59,070][75950] Updated weights for policy 1, policy_version 73610 (0.0007) -[2023-10-14 16:30:59,405][75949] Updated weights for policy 0, policy_version 73801 (0.0009) -[2023-10-14 16:30:59,428][75950] Updated weights for policy 1, policy_version 73620 (0.0007) -[2023-10-14 16:30:59,773][75949] Updated weights for policy 0, policy_version 73811 (0.0009) -[2023-10-14 16:30:59,792][75950] Updated weights for policy 1, policy_version 73630 (0.0007) -[2023-10-14 16:31:00,142][75949] Updated weights for policy 0, policy_version 73821 (0.0008) -[2023-10-14 16:31:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 150994944. Throughput: 0: 1667.4, 1: 1664.8. Samples: 37754942. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 16:31:03,165][74987] Avg episode reward: [(0, '26.310'), (1, '31.780')] -[2023-10-14 16:31:03,874][75950] Updated weights for policy 1, policy_version 73640 (0.0009) -[2023-10-14 16:31:04,200][75949] Updated weights for policy 0, policy_version 73831 (0.0009) -[2023-10-14 16:31:04,236][75950] Updated weights for policy 1, policy_version 73650 (0.0008) -[2023-10-14 16:31:04,573][75949] Updated weights for policy 0, policy_version 73841 (0.0008) -[2023-10-14 16:31:04,600][75950] Updated weights for policy 1, policy_version 73660 (0.0009) -[2023-10-14 16:31:04,939][75949] Updated weights for policy 0, policy_version 73851 (0.0009) -[2023-10-14 16:31:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151060480. Throughput: 0: 1686.2, 1: 1667.6. Samples: 37775768. Policy #0 lag: (min: 1.0, avg: 5.2, max: 33.0) -[2023-10-14 16:31:08,165][74987] Avg episode reward: [(0, '27.810'), (1, '32.350')] -[2023-10-14 16:31:08,630][75950] Updated weights for policy 1, policy_version 73670 (0.0008) -[2023-10-14 16:31:08,946][75949] Updated weights for policy 0, policy_version 73861 (0.0008) -[2023-10-14 16:31:09,005][75950] Updated weights for policy 1, policy_version 73680 (0.0008) -[2023-10-14 16:31:09,312][75949] Updated weights for policy 0, policy_version 73871 (0.0007) -[2023-10-14 16:31:09,380][75950] Updated weights for policy 1, policy_version 73690 (0.0008) -[2023-10-14 16:31:09,681][75949] Updated weights for policy 0, policy_version 73881 (0.0008) -[2023-10-14 16:31:13,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 151126016. Throughput: 0: 1687.5, 1: 1669.0. Samples: 37796534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:31:13,164][74987] Avg episode reward: [(0, '24.610'), (1, '33.800')] -[2023-10-14 16:31:13,393][75950] Updated weights for policy 1, policy_version 73700 (0.0008) -[2023-10-14 16:31:13,753][75950] Updated weights for policy 1, policy_version 73710 (0.0009) -[2023-10-14 16:31:13,764][75949] Updated weights for policy 0, policy_version 73891 (0.0008) -[2023-10-14 16:31:14,109][75950] Updated weights for policy 1, policy_version 73720 (0.0008) -[2023-10-14 16:31:14,139][75949] Updated weights for policy 0, policy_version 73901 (0.0008) -[2023-10-14 16:31:14,502][75949] Updated weights for policy 0, policy_version 73911 (0.0007) -[2023-10-14 16:31:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 151191552. Throughput: 0: 1680.2, 1: 1670.9. Samples: 37805682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:31:18,164][74987] Avg episode reward: [(0, '28.510'), (1, '34.340')] -[2023-10-14 16:31:18,299][75950] Updated weights for policy 1, policy_version 73730 (0.0007) -[2023-10-14 16:31:18,665][75950] Updated weights for policy 1, policy_version 73740 (0.0009) -[2023-10-14 16:31:18,670][75949] Updated weights for policy 0, policy_version 73921 (0.0008) -[2023-10-14 16:31:19,030][75950] Updated weights for policy 1, policy_version 73750 (0.0007) -[2023-10-14 16:31:19,037][75949] Updated weights for policy 0, policy_version 73931 (0.0008) -[2023-10-14 16:31:19,396][75950] Updated weights for policy 1, policy_version 73760 (0.0009) -[2023-10-14 16:31:19,398][75949] Updated weights for policy 0, policy_version 73941 (0.0008) -[2023-10-14 16:31:19,764][75949] Updated weights for policy 0, policy_version 73951 (0.0010) -[2023-10-14 16:31:23,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151257088. Throughput: 0: 1683.8, 1: 1676.2. Samples: 37826294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:31:23,165][74987] Avg episode reward: [(0, '25.030'), (1, '35.590')] -[2023-10-14 16:31:23,403][75950] Updated weights for policy 1, policy_version 73770 (0.0007) -[2023-10-14 16:31:23,773][75950] Updated weights for policy 1, policy_version 73780 (0.0009) -[2023-10-14 16:31:23,823][75949] Updated weights for policy 0, policy_version 73961 (0.0008) -[2023-10-14 16:31:24,140][75950] Updated weights for policy 1, policy_version 73790 (0.0008) -[2023-10-14 16:31:24,198][75949] Updated weights for policy 0, policy_version 73971 (0.0008) -[2023-10-14 16:31:24,560][75949] Updated weights for policy 0, policy_version 73981 (0.0008) -[2023-10-14 16:31:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151322624. Throughput: 0: 1682.2, 1: 1681.9. Samples: 37847192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:31:28,164][74987] Avg episode reward: [(0, '29.660'), (1, '35.030')] -[2023-10-14 16:31:28,355][75950] Updated weights for policy 1, policy_version 73800 (0.0008) -[2023-10-14 16:31:28,616][75949] Updated weights for policy 0, policy_version 73991 (0.0007) -[2023-10-14 16:31:28,725][75950] Updated weights for policy 1, policy_version 73810 (0.0008) -[2023-10-14 16:31:28,992][75949] Updated weights for policy 0, policy_version 74001 (0.0008) -[2023-10-14 16:31:29,094][75950] Updated weights for policy 1, policy_version 73820 (0.0008) -[2023-10-14 16:31:29,356][75949] Updated weights for policy 0, policy_version 74011 (0.0008) -[2023-10-14 16:31:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151388160. Throughput: 0: 1688.2, 1: 1677.9. Samples: 37856194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:31:33,165][74987] Avg episode reward: [(0, '27.810'), (1, '33.640')] -[2023-10-14 16:31:33,334][75950] Updated weights for policy 1, policy_version 73830 (0.0008) -[2023-10-14 16:31:33,396][75949] Updated weights for policy 0, policy_version 74021 (0.0007) -[2023-10-14 16:31:33,699][75950] Updated weights for policy 1, policy_version 73840 (0.0007) -[2023-10-14 16:31:33,763][75949] Updated weights for policy 0, policy_version 74031 (0.0007) -[2023-10-14 16:31:34,065][75950] Updated weights for policy 1, policy_version 73850 (0.0008) -[2023-10-14 16:31:34,138][75949] Updated weights for policy 0, policy_version 74041 (0.0008) -[2023-10-14 16:31:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151453696. Throughput: 0: 1687.7, 1: 1675.7. Samples: 37876582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:31:38,164][74987] Avg episode reward: [(0, '30.130'), (1, '33.600')] -[2023-10-14 16:31:38,188][75950] Updated weights for policy 1, policy_version 73860 (0.0009) -[2023-10-14 16:31:38,263][75949] Updated weights for policy 0, policy_version 74051 (0.0010) -[2023-10-14 16:31:38,550][75950] Updated weights for policy 1, policy_version 73870 (0.0007) -[2023-10-14 16:31:38,637][75949] Updated weights for policy 0, policy_version 74061 (0.0009) -[2023-10-14 16:31:38,917][75950] Updated weights for policy 1, policy_version 73880 (0.0008) -[2023-10-14 16:31:39,009][75949] Updated weights for policy 0, policy_version 74071 (0.0009) -[2023-10-14 16:31:43,019][75950] Updated weights for policy 1, policy_version 73890 (0.0009) -[2023-10-14 16:31:43,081][75949] Updated weights for policy 0, policy_version 74081 (0.0010) -[2023-10-14 16:31:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151519232. Throughput: 0: 1693.6, 1: 1680.7. Samples: 37897584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:31:43,165][74987] Avg episode reward: [(0, '26.050'), (1, '35.030')] -[2023-10-14 16:31:43,428][75950] Updated weights for policy 1, policy_version 73900 (0.0007) -[2023-10-14 16:31:43,485][75949] Updated weights for policy 0, policy_version 74091 (0.0008) -[2023-10-14 16:31:43,794][75950] Updated weights for policy 1, policy_version 73910 (0.0009) -[2023-10-14 16:31:43,863][75949] Updated weights for policy 0, policy_version 74101 (0.0009) -[2023-10-14 16:31:44,158][75950] Updated weights for policy 1, policy_version 73920 (0.0008) -[2023-10-14 16:31:44,223][75949] Updated weights for policy 0, policy_version 74111 (0.0010) -[2023-10-14 16:31:48,035][75950] Updated weights for policy 1, policy_version 73930 (0.0008) -[2023-10-14 16:31:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151584768. Throughput: 0: 1688.9, 1: 1677.6. Samples: 37906432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:31:48,164][74987] Avg episode reward: [(0, '27.430'), (1, '34.420')] -[2023-10-14 16:31:48,201][75949] Updated weights for policy 0, policy_version 74121 (0.0008) -[2023-10-14 16:31:48,401][75950] Updated weights for policy 1, policy_version 73940 (0.0008) -[2023-10-14 16:31:48,574][75949] Updated weights for policy 0, policy_version 74131 (0.0008) -[2023-10-14 16:31:48,766][75950] Updated weights for policy 1, policy_version 73950 (0.0008) -[2023-10-14 16:31:48,948][75949] Updated weights for policy 0, policy_version 74141 (0.0009) -[2023-10-14 16:31:52,920][75949] Updated weights for policy 0, policy_version 74151 (0.0010) -[2023-10-14 16:31:53,068][75950] Updated weights for policy 1, policy_version 73960 (0.0007) -[2023-10-14 16:31:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151650304. Throughput: 0: 1684.7, 1: 1676.0. Samples: 37927002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:31:53,164][74987] Avg episode reward: [(0, '26.530'), (1, '33.020')] -[2023-10-14 16:31:53,282][75949] Updated weights for policy 0, policy_version 74161 (0.0009) -[2023-10-14 16:31:53,422][75950] Updated weights for policy 1, policy_version 73970 (0.0007) -[2023-10-14 16:31:53,658][75949] Updated weights for policy 0, policy_version 74171 (0.0007) -[2023-10-14 16:31:53,792][75950] Updated weights for policy 1, policy_version 73980 (0.0008) -[2023-10-14 16:31:57,623][75950] Updated weights for policy 1, policy_version 73990 (0.0008) -[2023-10-14 16:31:57,743][75949] Updated weights for policy 0, policy_version 74181 (0.0009) -[2023-10-14 16:31:57,992][75950] Updated weights for policy 1, policy_version 74000 (0.0008) -[2023-10-14 16:31:58,108][75949] Updated weights for policy 0, policy_version 74191 (0.0008) -[2023-10-14 16:31:58,164][74987] Fps is (10 sec: 13106.6, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 151715840. Throughput: 0: 1680.0, 1: 1676.0. Samples: 37947556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:31:58,165][74987] Avg episode reward: [(0, '28.650'), (1, '34.370')] -[2023-10-14 16:31:58,356][75950] Updated weights for policy 1, policy_version 74010 (0.0008) -[2023-10-14 16:31:58,480][75949] Updated weights for policy 0, policy_version 74201 (0.0007) -[2023-10-14 16:32:02,592][75950] Updated weights for policy 1, policy_version 74020 (0.0009) -[2023-10-14 16:32:02,668][75949] Updated weights for policy 0, policy_version 74211 (0.0008) -[2023-10-14 16:32:02,955][75950] Updated weights for policy 1, policy_version 74030 (0.0008) -[2023-10-14 16:32:03,024][75949] Updated weights for policy 0, policy_version 74221 (0.0007) -[2023-10-14 16:32:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151781376. Throughput: 0: 1683.9, 1: 1678.7. Samples: 37957000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:32:03,165][74987] Avg episode reward: [(0, '28.180'), (1, '30.220')] -[2023-10-14 16:32:03,327][75950] Updated weights for policy 1, policy_version 74040 (0.0008) -[2023-10-14 16:32:03,397][75949] Updated weights for policy 0, policy_version 74231 (0.0008) -[2023-10-14 16:32:07,385][75949] Updated weights for policy 0, policy_version 74241 (0.0008) -[2023-10-14 16:32:07,423][75950] Updated weights for policy 1, policy_version 74050 (0.0010) -[2023-10-14 16:32:07,746][75949] Updated weights for policy 0, policy_version 74251 (0.0007) -[2023-10-14 16:32:07,795][75950] Updated weights for policy 1, policy_version 74060 (0.0009) -[2023-10-14 16:32:08,110][75949] Updated weights for policy 0, policy_version 74261 (0.0008) -[2023-10-14 16:32:08,157][75950] Updated weights for policy 1, policy_version 74070 (0.0007) -[2023-10-14 16:32:08,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151846912. Throughput: 0: 1683.1, 1: 1677.0. Samples: 37977498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:32:08,165][74987] Avg episode reward: [(0, '27.250'), (1, '31.380')] -[2023-10-14 16:32:08,481][75949] Updated weights for policy 0, policy_version 74271 (0.0007) -[2023-10-14 16:32:08,527][75950] Updated weights for policy 1, policy_version 74080 (0.0008) -[2023-10-14 16:32:12,522][75949] Updated weights for policy 0, policy_version 74281 (0.0009) -[2023-10-14 16:32:12,591][75950] Updated weights for policy 1, policy_version 74090 (0.0007) -[2023-10-14 16:32:12,898][75949] Updated weights for policy 0, policy_version 74291 (0.0009) -[2023-10-14 16:32:12,960][75950] Updated weights for policy 1, policy_version 74100 (0.0007) -[2023-10-14 16:32:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151912448. Throughput: 0: 1667.5, 1: 1667.6. Samples: 37997272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:32:13,164][74987] Avg episode reward: [(0, '25.090'), (1, '33.110')] -[2023-10-14 16:32:13,256][75949] Updated weights for policy 0, policy_version 74301 (0.0009) -[2023-10-14 16:32:13,326][75950] Updated weights for policy 1, policy_version 74110 (0.0008) -[2023-10-14 16:32:17,315][75950] Updated weights for policy 1, policy_version 74120 (0.0008) -[2023-10-14 16:32:17,553][75949] Updated weights for policy 0, policy_version 74311 (0.0008) -[2023-10-14 16:32:17,677][75950] Updated weights for policy 1, policy_version 74130 (0.0009) -[2023-10-14 16:32:17,925][75949] Updated weights for policy 0, policy_version 74321 (0.0007) -[2023-10-14 16:32:18,043][75950] Updated weights for policy 1, policy_version 74140 (0.0009) -[2023-10-14 16:32:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151977984. Throughput: 0: 1676.2, 1: 1677.5. Samples: 38007110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:32:18,164][74987] Avg episode reward: [(0, '24.020'), (1, '30.700')] -[2023-10-14 16:32:18,295][75949] Updated weights for policy 0, policy_version 74331 (0.0009) -[2023-10-14 16:32:22,123][75950] Updated weights for policy 1, policy_version 74150 (0.0007) -[2023-10-14 16:32:22,463][75949] Updated weights for policy 0, policy_version 74341 (0.0009) -[2023-10-14 16:32:22,479][75950] Updated weights for policy 1, policy_version 74160 (0.0008) -[2023-10-14 16:32:22,833][75949] Updated weights for policy 0, policy_version 74351 (0.0009) -[2023-10-14 16:32:22,853][75950] Updated weights for policy 1, policy_version 74170 (0.0008) -[2023-10-14 16:32:23,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 152076288. Throughput: 0: 1673.0, 1: 1682.8. Samples: 38027596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:32:23,164][74987] Avg episode reward: [(0, '26.810'), (1, '30.690')] -[2023-10-14 16:32:23,204][75949] Updated weights for policy 0, policy_version 74361 (0.0009) -[2023-10-14 16:32:26,805][75950] Updated weights for policy 1, policy_version 74180 (0.0010) -[2023-10-14 16:32:27,170][75950] Updated weights for policy 1, policy_version 74190 (0.0009) -[2023-10-14 16:32:27,214][75949] Updated weights for policy 0, policy_version 74371 (0.0010) -[2023-10-14 16:32:27,525][75950] Updated weights for policy 1, policy_version 74200 (0.0007) -[2023-10-14 16:32:27,583][75949] Updated weights for policy 0, policy_version 74381 (0.0008) -[2023-10-14 16:32:27,952][75949] Updated weights for policy 0, policy_version 74391 (0.0009) -[2023-10-14 16:32:28,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 152141824. Throughput: 0: 1656.2, 1: 1657.1. Samples: 38046682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:32:28,164][74987] Avg episode reward: [(0, '24.760'), (1, '34.350')] -[2023-10-14 16:32:31,841][75950] Updated weights for policy 1, policy_version 74210 (0.0009) -[2023-10-14 16:32:32,013][75949] Updated weights for policy 0, policy_version 74401 (0.0008) -[2023-10-14 16:32:32,241][75950] Updated weights for policy 1, policy_version 74220 (0.0007) -[2023-10-14 16:32:32,438][75949] Updated weights for policy 0, policy_version 74411 (0.0007) -[2023-10-14 16:32:32,609][75950] Updated weights for policy 1, policy_version 74230 (0.0008) -[2023-10-14 16:32:32,802][75949] Updated weights for policy 0, policy_version 74421 (0.0008) -[2023-10-14 16:32:32,971][75950] Updated weights for policy 1, policy_version 74240 (0.0007) -[2023-10-14 16:32:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 152207360. Throughput: 0: 1672.9, 1: 1680.5. Samples: 38057336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:32:33,164][74987] Avg episode reward: [(0, '26.950'), (1, '33.570')] -[2023-10-14 16:32:33,165][75949] Updated weights for policy 0, policy_version 74431 (0.0010) -[2023-10-14 16:32:36,958][75950] Updated weights for policy 1, policy_version 74250 (0.0007) -[2023-10-14 16:32:37,185][75949] Updated weights for policy 0, policy_version 74441 (0.0010) -[2023-10-14 16:32:37,328][75950] Updated weights for policy 1, policy_version 74260 (0.0008) -[2023-10-14 16:32:37,547][75949] Updated weights for policy 0, policy_version 74451 (0.0008) -[2023-10-14 16:32:37,691][75950] Updated weights for policy 1, policy_version 74270 (0.0009) -[2023-10-14 16:32:37,917][75949] Updated weights for policy 0, policy_version 74461 (0.0009) -[2023-10-14 16:32:38,163][74987] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 152305664. Throughput: 0: 1669.9, 1: 1679.4. Samples: 38077720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:32:38,164][74987] Avg episode reward: [(0, '28.690'), (1, '32.170')] -[2023-10-14 16:32:41,677][75950] Updated weights for policy 1, policy_version 74280 (0.0008) -[2023-10-14 16:32:41,999][75949] Updated weights for policy 0, policy_version 74471 (0.0009) -[2023-10-14 16:32:42,048][75950] Updated weights for policy 1, policy_version 74290 (0.0007) -[2023-10-14 16:32:42,359][75949] Updated weights for policy 0, policy_version 74481 (0.0008) -[2023-10-14 16:32:42,406][75950] Updated weights for policy 1, policy_version 74300 (0.0007) -[2023-10-14 16:32:42,728][75949] Updated weights for policy 0, policy_version 74491 (0.0010) -[2023-10-14 16:32:43,164][74987] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 152371200. Throughput: 0: 1651.6, 1: 1656.2. Samples: 38096406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:32:43,164][74987] Avg episode reward: [(0, '26.080'), (1, '33.220')] -[2023-10-14 16:32:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000074304_76087296.pth... -[2023-10-14 16:32:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000074496_76283904.pth... -[2023-10-14 16:32:43,210][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000072928_74678272.pth -[2023-10-14 16:32:43,212][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000072736_74481664.pth -[2023-10-14 16:32:46,541][75950] Updated weights for policy 1, policy_version 74310 (0.0009) -[2023-10-14 16:32:46,900][75949] Updated weights for policy 0, policy_version 74501 (0.0009) -[2023-10-14 16:32:46,912][75950] Updated weights for policy 1, policy_version 74320 (0.0009) -[2023-10-14 16:32:47,271][75949] Updated weights for policy 0, policy_version 74511 (0.0007) -[2023-10-14 16:32:47,274][75950] Updated weights for policy 1, policy_version 74330 (0.0007) -[2023-10-14 16:32:47,636][75949] Updated weights for policy 0, policy_version 74521 (0.0008) -[2023-10-14 16:32:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 152436736. Throughput: 0: 1666.4, 1: 1679.9. Samples: 38107582. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) -[2023-10-14 16:32:48,164][74987] Avg episode reward: [(0, '28.450'), (1, '32.400')] -[2023-10-14 16:32:51,637][75950] Updated weights for policy 1, policy_version 74340 (0.0007) -[2023-10-14 16:32:51,816][75949] Updated weights for policy 0, policy_version 74531 (0.0009) -[2023-10-14 16:32:52,004][75950] Updated weights for policy 1, policy_version 74350 (0.0008) -[2023-10-14 16:32:52,176][75949] Updated weights for policy 0, policy_version 74541 (0.0007) -[2023-10-14 16:32:52,364][75950] Updated weights for policy 1, policy_version 74360 (0.0010) -[2023-10-14 16:32:52,539][75949] Updated weights for policy 0, policy_version 74551 (0.0007) -[2023-10-14 16:32:53,163][74987] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 152502272. Throughput: 0: 1666.9, 1: 1672.4. Samples: 38127766. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) -[2023-10-14 16:32:53,164][74987] Avg episode reward: [(0, '25.730'), (1, '31.090')] -[2023-10-14 16:32:56,326][75950] Updated weights for policy 1, policy_version 74370 (0.0010) -[2023-10-14 16:32:56,694][75950] Updated weights for policy 1, policy_version 74380 (0.0008) -[2023-10-14 16:32:56,733][75949] Updated weights for policy 0, policy_version 74561 (0.0007) -[2023-10-14 16:32:57,067][75950] Updated weights for policy 1, policy_version 74390 (0.0007) -[2023-10-14 16:32:57,104][75949] Updated weights for policy 0, policy_version 74571 (0.0008) -[2023-10-14 16:32:57,439][75950] Updated weights for policy 1, policy_version 74400 (0.0008) -[2023-10-14 16:32:57,473][75949] Updated weights for policy 0, policy_version 74581 (0.0008) -[2023-10-14 16:32:57,842][75949] Updated weights for policy 0, policy_version 74591 (0.0007) -[2023-10-14 16:32:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 152567808. Throughput: 0: 1653.7, 1: 1653.3. Samples: 38146088. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) -[2023-10-14 16:32:58,165][74987] Avg episode reward: [(0, '30.050'), (1, '33.780')] -[2023-10-14 16:33:01,661][75950] Updated weights for policy 1, policy_version 74410 (0.0009) -[2023-10-14 16:33:02,008][75949] Updated weights for policy 0, policy_version 74601 (0.0008) -[2023-10-14 16:33:02,026][75950] Updated weights for policy 1, policy_version 74420 (0.0008) -[2023-10-14 16:33:02,379][75949] Updated weights for policy 0, policy_version 74611 (0.0007) -[2023-10-14 16:33:02,383][75950] Updated weights for policy 1, policy_version 74430 (0.0008) -[2023-10-14 16:33:02,743][75949] Updated weights for policy 0, policy_version 74621 (0.0007) -[2023-10-14 16:33:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 152633344. Throughput: 0: 1665.4, 1: 1672.1. Samples: 38157298. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) -[2023-10-14 16:33:03,165][74987] Avg episode reward: [(0, '25.950'), (1, '34.590')] -[2023-10-14 16:33:06,436][75950] Updated weights for policy 1, policy_version 74440 (0.0008) -[2023-10-14 16:33:06,710][75949] Updated weights for policy 0, policy_version 74631 (0.0008) -[2023-10-14 16:33:06,806][75950] Updated weights for policy 1, policy_version 74450 (0.0009) -[2023-10-14 16:33:07,074][75949] Updated weights for policy 0, policy_version 74641 (0.0008) -[2023-10-14 16:33:07,163][75950] Updated weights for policy 1, policy_version 74460 (0.0008) -[2023-10-14 16:33:07,442][75949] Updated weights for policy 0, policy_version 74651 (0.0007) -[2023-10-14 16:33:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 152698880. Throughput: 0: 1667.4, 1: 1658.2. Samples: 38177246. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) -[2023-10-14 16:33:08,164][74987] Avg episode reward: [(0, '29.500'), (1, '34.170')] -[2023-10-14 16:33:11,428][75950] Updated weights for policy 1, policy_version 74470 (0.0009) -[2023-10-14 16:33:11,629][75949] Updated weights for policy 0, policy_version 74661 (0.0009) -[2023-10-14 16:33:11,782][75950] Updated weights for policy 1, policy_version 74480 (0.0008) -[2023-10-14 16:33:12,004][75949] Updated weights for policy 0, policy_version 74671 (0.0009) -[2023-10-14 16:33:12,154][75950] Updated weights for policy 1, policy_version 74490 (0.0007) -[2023-10-14 16:33:12,377][75949] Updated weights for policy 0, policy_version 74681 (0.0007) -[2023-10-14 16:33:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 152764416. Throughput: 0: 1655.2, 1: 1662.4. Samples: 38195972. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) -[2023-10-14 16:33:13,164][74987] Avg episode reward: [(0, '26.690'), (1, '35.720')] -[2023-10-14 16:33:16,241][75950] Updated weights for policy 1, policy_version 74500 (0.0007) -[2023-10-14 16:33:16,357][75949] Updated weights for policy 0, policy_version 74691 (0.0009) -[2023-10-14 16:33:16,613][75950] Updated weights for policy 1, policy_version 74510 (0.0008) -[2023-10-14 16:33:16,751][75949] Updated weights for policy 0, policy_version 74701 (0.0009) -[2023-10-14 16:33:16,974][75950] Updated weights for policy 1, policy_version 74520 (0.0008) -[2023-10-14 16:33:17,120][75949] Updated weights for policy 0, policy_version 74711 (0.0008) -[2023-10-14 16:33:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 152829952. Throughput: 0: 1675.0, 1: 1670.7. Samples: 38207890. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) -[2023-10-14 16:33:18,165][74987] Avg episode reward: [(0, '30.270'), (1, '34.380')] -[2023-10-14 16:33:21,189][75950] Updated weights for policy 1, policy_version 74530 (0.0007) -[2023-10-14 16:33:21,209][75949] Updated weights for policy 0, policy_version 74721 (0.0007) -[2023-10-14 16:33:21,571][75949] Updated weights for policy 0, policy_version 74731 (0.0010) -[2023-10-14 16:33:21,602][75950] Updated weights for policy 1, policy_version 74540 (0.0010) -[2023-10-14 16:33:21,937][75949] Updated weights for policy 0, policy_version 74741 (0.0009) -[2023-10-14 16:33:21,970][75950] Updated weights for policy 1, policy_version 74550 (0.0008) -[2023-10-14 16:33:22,310][75949] Updated weights for policy 0, policy_version 74751 (0.0009) -[2023-10-14 16:33:22,338][75950] Updated weights for policy 1, policy_version 74560 (0.0008) -[2023-10-14 16:33:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 152895488. Throughput: 0: 1662.8, 1: 1664.3. Samples: 38227442. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) -[2023-10-14 16:33:23,165][74987] Avg episode reward: [(0, '26.890'), (1, '33.160')] -[2023-10-14 16:33:26,250][75950] Updated weights for policy 1, policy_version 74570 (0.0008) -[2023-10-14 16:33:26,520][75949] Updated weights for policy 0, policy_version 74761 (0.0009) -[2023-10-14 16:33:26,611][75950] Updated weights for policy 1, policy_version 74580 (0.0007) -[2023-10-14 16:33:26,891][75949] Updated weights for policy 0, policy_version 74771 (0.0008) -[2023-10-14 16:33:26,977][75950] Updated weights for policy 1, policy_version 74590 (0.0008) -[2023-10-14 16:33:27,256][75949] Updated weights for policy 0, policy_version 74781 (0.0008) -[2023-10-14 16:33:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 152961024. Throughput: 0: 1663.1, 1: 1673.3. Samples: 38246540. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) -[2023-10-14 16:33:28,164][74987] Avg episode reward: [(0, '29.240'), (1, '33.300')] -[2023-10-14 16:33:30,888][75950] Updated weights for policy 1, policy_version 74600 (0.0008) -[2023-10-14 16:33:31,206][75949] Updated weights for policy 0, policy_version 74791 (0.0010) -[2023-10-14 16:33:31,257][75950] Updated weights for policy 1, policy_version 74610 (0.0008) -[2023-10-14 16:33:31,572][75949] Updated weights for policy 0, policy_version 74801 (0.0009) -[2023-10-14 16:33:31,622][75950] Updated weights for policy 1, policy_version 74620 (0.0008) -[2023-10-14 16:33:31,939][75949] Updated weights for policy 0, policy_version 74811 (0.0008) -[2023-10-14 16:33:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 153026560. Throughput: 0: 1677.5, 1: 1671.3. Samples: 38258276. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) -[2023-10-14 16:33:33,164][74987] Avg episode reward: [(0, '27.890'), (1, '32.950')] -[2023-10-14 16:33:35,785][75950] Updated weights for policy 1, policy_version 74630 (0.0007) -[2023-10-14 16:33:36,086][75949] Updated weights for policy 0, policy_version 74821 (0.0008) -[2023-10-14 16:33:36,138][75950] Updated weights for policy 1, policy_version 74640 (0.0007) -[2023-10-14 16:33:36,445][75949] Updated weights for policy 0, policy_version 74831 (0.0009) -[2023-10-14 16:33:36,505][75950] Updated weights for policy 1, policy_version 74650 (0.0007) -[2023-10-14 16:33:36,815][75949] Updated weights for policy 0, policy_version 74841 (0.0007) -[2023-10-14 16:33:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 153092096. Throughput: 0: 1662.3, 1: 1656.4. Samples: 38277108. Policy #0 lag: (min: 19.0, avg: 33.7, max: 51.0) -[2023-10-14 16:33:38,164][74987] Avg episode reward: [(0, '26.140'), (1, '32.540')] -[2023-10-14 16:33:40,417][75950] Updated weights for policy 1, policy_version 74660 (0.0008) -[2023-10-14 16:33:40,568][75949] Updated weights for policy 0, policy_version 74851 (0.0008) -[2023-10-14 16:33:40,788][75950] Updated weights for policy 1, policy_version 74670 (0.0007) -[2023-10-14 16:33:40,941][75949] Updated weights for policy 0, policy_version 74861 (0.0007) -[2023-10-14 16:33:41,156][75950] Updated weights for policy 1, policy_version 74680 (0.0008) -[2023-10-14 16:33:41,298][75949] Updated weights for policy 0, policy_version 74871 (0.0009) -[2023-10-14 16:33:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 153157632. Throughput: 0: 1679.7, 1: 1684.0. Samples: 38297458. Policy #0 lag: (min: 19.0, avg: 33.7, max: 51.0) -[2023-10-14 16:33:43,164][74987] Avg episode reward: [(0, '26.980'), (1, '33.970')] -[2023-10-14 16:33:45,123][75950] Updated weights for policy 1, policy_version 74690 (0.0009) -[2023-10-14 16:33:45,458][75949] Updated weights for policy 0, policy_version 74881 (0.0009) -[2023-10-14 16:33:45,482][75950] Updated weights for policy 1, policy_version 74700 (0.0010) -[2023-10-14 16:33:45,823][75949] Updated weights for policy 0, policy_version 74891 (0.0008) -[2023-10-14 16:33:45,849][75950] Updated weights for policy 1, policy_version 74710 (0.0008) -[2023-10-14 16:33:46,191][75949] Updated weights for policy 0, policy_version 74901 (0.0009) -[2023-10-14 16:33:46,207][75950] Updated weights for policy 1, policy_version 74720 (0.0009) -[2023-10-14 16:33:46,563][75949] Updated weights for policy 0, policy_version 74911 (0.0007) -[2023-10-14 16:33:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153223168. Throughput: 0: 1685.3, 1: 1673.4. Samples: 38308440. Policy #0 lag: (min: 19.0, avg: 33.7, max: 51.0) -[2023-10-14 16:33:48,164][74987] Avg episode reward: [(0, '25.560'), (1, '33.350')] -[2023-10-14 16:33:50,355][75950] Updated weights for policy 1, policy_version 74730 (0.0009) -[2023-10-14 16:33:50,515][75949] Updated weights for policy 0, policy_version 74921 (0.0007) -[2023-10-14 16:33:50,717][75950] Updated weights for policy 1, policy_version 74740 (0.0009) -[2023-10-14 16:33:50,885][75949] Updated weights for policy 0, policy_version 74931 (0.0008) -[2023-10-14 16:33:51,084][75950] Updated weights for policy 1, policy_version 74750 (0.0008) -[2023-10-14 16:33:51,254][75949] Updated weights for policy 0, policy_version 74941 (0.0009) -[2023-10-14 16:33:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153288704. Throughput: 0: 1666.3, 1: 1668.0. Samples: 38327286. Policy #0 lag: (min: 19.0, avg: 33.7, max: 51.0) -[2023-10-14 16:33:53,165][74987] Avg episode reward: [(0, '29.440'), (1, '34.260')] -[2023-10-14 16:33:55,387][75950] Updated weights for policy 1, policy_version 74760 (0.0009) -[2023-10-14 16:33:55,398][75949] Updated weights for policy 0, policy_version 74951 (0.0009) -[2023-10-14 16:33:55,760][75950] Updated weights for policy 1, policy_version 74770 (0.0007) -[2023-10-14 16:33:55,772][75949] Updated weights for policy 0, policy_version 74961 (0.0008) -[2023-10-14 16:33:56,126][75950] Updated weights for policy 1, policy_version 74780 (0.0008) -[2023-10-14 16:33:56,144][75949] Updated weights for policy 0, policy_version 74971 (0.0010) -[2023-10-14 16:33:58,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 153354240. Throughput: 0: 1691.1, 1: 1683.5. Samples: 38347828. Policy #0 lag: (min: 19.0, avg: 33.7, max: 51.0) -[2023-10-14 16:33:58,165][74987] Avg episode reward: [(0, '24.660'), (1, '33.350')] -[2023-10-14 16:34:00,218][75949] Updated weights for policy 0, policy_version 74981 (0.0008) -[2023-10-14 16:34:00,459][75950] Updated weights for policy 1, policy_version 74790 (0.0007) -[2023-10-14 16:34:00,586][75949] Updated weights for policy 0, policy_version 74991 (0.0007) -[2023-10-14 16:34:00,823][75950] Updated weights for policy 1, policy_version 74800 (0.0008) -[2023-10-14 16:34:00,964][75949] Updated weights for policy 0, policy_version 75001 (0.0009) -[2023-10-14 16:34:01,195][75950] Updated weights for policy 1, policy_version 74810 (0.0008) -[2023-10-14 16:34:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153419776. Throughput: 0: 1671.3, 1: 1669.4. Samples: 38358222. Policy #0 lag: (min: 19.0, avg: 33.7, max: 51.0) -[2023-10-14 16:34:03,164][74987] Avg episode reward: [(0, '28.640'), (1, '33.100')] -[2023-10-14 16:34:05,084][75949] Updated weights for policy 0, policy_version 75011 (0.0008) -[2023-10-14 16:34:05,257][75950] Updated weights for policy 1, policy_version 74820 (0.0008) -[2023-10-14 16:34:05,477][75949] Updated weights for policy 0, policy_version 75021 (0.0007) -[2023-10-14 16:34:05,619][75950] Updated weights for policy 1, policy_version 74830 (0.0008) -[2023-10-14 16:34:05,851][75949] Updated weights for policy 0, policy_version 75031 (0.0007) -[2023-10-14 16:34:05,991][75950] Updated weights for policy 1, policy_version 74840 (0.0009) -[2023-10-14 16:34:08,164][74987] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153485312. Throughput: 0: 1667.9, 1: 1661.3. Samples: 38377256. Policy #0 lag: (min: 19.0, avg: 33.7, max: 51.0) -[2023-10-14 16:34:08,164][74987] Avg episode reward: [(0, '25.080'), (1, '32.250')] -[2023-10-14 16:34:09,934][75949] Updated weights for policy 0, policy_version 75041 (0.0008) -[2023-10-14 16:34:10,154][75950] Updated weights for policy 1, policy_version 74850 (0.0007) -[2023-10-14 16:34:10,308][75949] Updated weights for policy 0, policy_version 75051 (0.0009) -[2023-10-14 16:34:10,571][75950] Updated weights for policy 1, policy_version 74860 (0.0010) -[2023-10-14 16:34:10,676][75949] Updated weights for policy 0, policy_version 75061 (0.0009) -[2023-10-14 16:34:10,936][75950] Updated weights for policy 1, policy_version 74870 (0.0009) -[2023-10-14 16:34:11,040][75949] Updated weights for policy 0, policy_version 75071 (0.0009) -[2023-10-14 16:34:11,294][75950] Updated weights for policy 1, policy_version 74880 (0.0009) -[2023-10-14 16:34:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 153550848. Throughput: 0: 1686.2, 1: 1670.0. Samples: 38397570. Policy #0 lag: (min: 19.0, avg: 33.7, max: 51.0) -[2023-10-14 16:34:13,165][74987] Avg episode reward: [(0, '30.140'), (1, '33.270')] -[2023-10-14 16:34:15,181][75949] Updated weights for policy 0, policy_version 75081 (0.0010) -[2023-10-14 16:34:15,230][75950] Updated weights for policy 1, policy_version 74890 (0.0007) -[2023-10-14 16:34:15,551][75949] Updated weights for policy 0, policy_version 75091 (0.0007) -[2023-10-14 16:34:15,610][75950] Updated weights for policy 1, policy_version 74900 (0.0008) -[2023-10-14 16:34:15,931][75949] Updated weights for policy 0, policy_version 75101 (0.0007) -[2023-10-14 16:34:15,984][75950] Updated weights for policy 1, policy_version 74910 (0.0008) -[2023-10-14 16:34:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 153616384. Throughput: 0: 1663.7, 1: 1655.9. Samples: 38407660. Policy #0 lag: (min: 19.0, avg: 33.7, max: 51.0) -[2023-10-14 16:34:18,165][74987] Avg episode reward: [(0, '25.050'), (1, '33.290')] -[2023-10-14 16:34:20,058][75949] Updated weights for policy 0, policy_version 75111 (0.0008) -[2023-10-14 16:34:20,114][75950] Updated weights for policy 1, policy_version 74920 (0.0009) -[2023-10-14 16:34:20,424][75949] Updated weights for policy 0, policy_version 75121 (0.0009) -[2023-10-14 16:34:20,466][75950] Updated weights for policy 1, policy_version 74930 (0.0008) -[2023-10-14 16:34:20,796][75949] Updated weights for policy 0, policy_version 75131 (0.0009) -[2023-10-14 16:34:20,839][75950] Updated weights for policy 1, policy_version 74940 (0.0007) -[2023-10-14 16:34:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153681920. Throughput: 0: 1673.7, 1: 1663.2. Samples: 38427270. Policy #0 lag: (min: 19.0, avg: 33.7, max: 51.0) -[2023-10-14 16:34:23,164][74987] Avg episode reward: [(0, '29.970'), (1, '33.520')] -[2023-10-14 16:34:24,849][75949] Updated weights for policy 0, policy_version 75141 (0.0009) -[2023-10-14 16:34:25,092][75950] Updated weights for policy 1, policy_version 74950 (0.0009) -[2023-10-14 16:34:25,204][75949] Updated weights for policy 0, policy_version 75151 (0.0010) -[2023-10-14 16:34:25,459][75950] Updated weights for policy 1, policy_version 74960 (0.0008) -[2023-10-14 16:34:25,579][75949] Updated weights for policy 0, policy_version 75161 (0.0008) -[2023-10-14 16:34:25,830][75950] Updated weights for policy 1, policy_version 74970 (0.0007) -[2023-10-14 16:34:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 153747456. Throughput: 0: 1681.1, 1: 1654.6. Samples: 38447564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:34:28,165][74987] Avg episode reward: [(0, '24.570'), (1, '34.160')] -[2023-10-14 16:34:29,628][75949] Updated weights for policy 0, policy_version 75171 (0.0008) -[2023-10-14 16:34:29,939][75950] Updated weights for policy 1, policy_version 74980 (0.0007) -[2023-10-14 16:34:29,992][75949] Updated weights for policy 0, policy_version 75181 (0.0009) -[2023-10-14 16:34:30,303][75950] Updated weights for policy 1, policy_version 74990 (0.0009) -[2023-10-14 16:34:30,355][75949] Updated weights for policy 0, policy_version 75191 (0.0009) -[2023-10-14 16:34:30,664][75950] Updated weights for policy 1, policy_version 75000 (0.0008) -[2023-10-14 16:34:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153812992. Throughput: 0: 1656.1, 1: 1649.3. Samples: 38457184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:34:33,164][74987] Avg episode reward: [(0, '28.580'), (1, '35.280')] -[2023-10-14 16:34:34,293][75949] Updated weights for policy 0, policy_version 75201 (0.0008) -[2023-10-14 16:34:34,651][75949] Updated weights for policy 0, policy_version 75211 (0.0007) -[2023-10-14 16:34:34,900][75950] Updated weights for policy 1, policy_version 75010 (0.0008) -[2023-10-14 16:34:35,025][75949] Updated weights for policy 0, policy_version 75221 (0.0009) -[2023-10-14 16:34:35,268][75950] Updated weights for policy 1, policy_version 75020 (0.0008) -[2023-10-14 16:34:35,397][75949] Updated weights for policy 0, policy_version 75231 (0.0008) -[2023-10-14 16:34:35,630][75950] Updated weights for policy 1, policy_version 75030 (0.0007) -[2023-10-14 16:34:36,008][75950] Updated weights for policy 1, policy_version 75040 (0.0009) -[2023-10-14 16:34:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153878528. Throughput: 0: 1683.0, 1: 1653.2. Samples: 38477414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:34:38,165][74987] Avg episode reward: [(0, '24.420'), (1, '33.410')] -[2023-10-14 16:34:39,484][75949] Updated weights for policy 0, policy_version 75241 (0.0008) -[2023-10-14 16:34:39,850][75949] Updated weights for policy 0, policy_version 75251 (0.0007) -[2023-10-14 16:34:40,076][75950] Updated weights for policy 1, policy_version 75050 (0.0007) -[2023-10-14 16:34:40,220][75949] Updated weights for policy 0, policy_version 75261 (0.0008) -[2023-10-14 16:34:40,446][75950] Updated weights for policy 1, policy_version 75060 (0.0008) -[2023-10-14 16:34:40,802][75950] Updated weights for policy 1, policy_version 75070 (0.0008) -[2023-10-14 16:34:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153944064. Throughput: 0: 1685.7, 1: 1660.4. Samples: 38498404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:34:43,164][74987] Avg episode reward: [(0, '28.580'), (1, '32.010')] -[2023-10-14 16:34:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000075264_77070336.pth... -[2023-10-14 16:34:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000075072_76873728.pth... -[2023-10-14 16:34:43,204][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000073696_75464704.pth -[2023-10-14 16:34:43,218][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000073504_75268096.pth -[2023-10-14 16:34:44,378][75949] Updated weights for policy 0, policy_version 75271 (0.0008) -[2023-10-14 16:34:44,648][75950] Updated weights for policy 1, policy_version 75080 (0.0009) -[2023-10-14 16:34:44,740][75949] Updated weights for policy 0, policy_version 75281 (0.0010) -[2023-10-14 16:34:45,010][75950] Updated weights for policy 1, policy_version 75090 (0.0009) -[2023-10-14 16:34:45,111][75949] Updated weights for policy 0, policy_version 75291 (0.0008) -[2023-10-14 16:34:45,382][75950] Updated weights for policy 1, policy_version 75100 (0.0008) -[2023-10-14 16:34:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154009600. Throughput: 0: 1669.6, 1: 1647.4. Samples: 38507488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:34:48,165][74987] Avg episode reward: [(0, '24.470'), (1, '33.000')] -[2023-10-14 16:34:49,162][75949] Updated weights for policy 0, policy_version 75301 (0.0008) -[2023-10-14 16:34:49,521][75949] Updated weights for policy 0, policy_version 75311 (0.0007) -[2023-10-14 16:34:49,589][75950] Updated weights for policy 1, policy_version 75110 (0.0008) -[2023-10-14 16:34:49,895][75949] Updated weights for policy 0, policy_version 75321 (0.0009) -[2023-10-14 16:34:49,964][75950] Updated weights for policy 1, policy_version 75120 (0.0008) -[2023-10-14 16:34:50,333][75950] Updated weights for policy 1, policy_version 75130 (0.0008) -[2023-10-14 16:34:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 154075136. Throughput: 0: 1687.5, 1: 1657.1. Samples: 38527762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:34:53,165][74987] Avg episode reward: [(0, '28.020'), (1, '34.230')] -[2023-10-14 16:34:53,916][75949] Updated weights for policy 0, policy_version 75331 (0.0009) -[2023-10-14 16:34:54,314][75949] Updated weights for policy 0, policy_version 75341 (0.0010) -[2023-10-14 16:34:54,532][75950] Updated weights for policy 1, policy_version 75140 (0.0009) -[2023-10-14 16:34:54,673][75949] Updated weights for policy 0, policy_version 75351 (0.0008) -[2023-10-14 16:34:54,891][75950] Updated weights for policy 1, policy_version 75150 (0.0007) -[2023-10-14 16:34:55,256][75950] Updated weights for policy 1, policy_version 75160 (0.0007) -[2023-10-14 16:34:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154140672. Throughput: 0: 1683.8, 1: 1664.1. Samples: 38548228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:34:58,165][74987] Avg episode reward: [(0, '26.840'), (1, '33.650')] -[2023-10-14 16:34:58,896][75949] Updated weights for policy 0, policy_version 75361 (0.0009) -[2023-10-14 16:34:59,270][75949] Updated weights for policy 0, policy_version 75371 (0.0009) -[2023-10-14 16:34:59,362][75950] Updated weights for policy 1, policy_version 75170 (0.0007) -[2023-10-14 16:34:59,638][75949] Updated weights for policy 0, policy_version 75381 (0.0008) -[2023-10-14 16:34:59,731][75950] Updated weights for policy 1, policy_version 75180 (0.0008) -[2023-10-14 16:35:00,002][75949] Updated weights for policy 0, policy_version 75391 (0.0008) -[2023-10-14 16:35:00,094][75950] Updated weights for policy 1, policy_version 75190 (0.0009) -[2023-10-14 16:35:00,471][75950] Updated weights for policy 1, policy_version 75200 (0.0008) -[2023-10-14 16:35:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154206208. Throughput: 0: 1669.7, 1: 1653.2. Samples: 38557190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:03,164][74987] Avg episode reward: [(0, '27.080'), (1, '33.450')] -[2023-10-14 16:35:03,989][75949] Updated weights for policy 0, policy_version 75401 (0.0008) -[2023-10-14 16:35:04,360][75949] Updated weights for policy 0, policy_version 75411 (0.0009) -[2023-10-14 16:35:04,512][75950] Updated weights for policy 1, policy_version 75210 (0.0008) -[2023-10-14 16:35:04,720][75949] Updated weights for policy 0, policy_version 75421 (0.0008) -[2023-10-14 16:35:04,878][75950] Updated weights for policy 1, policy_version 75220 (0.0009) -[2023-10-14 16:35:05,228][75950] Updated weights for policy 1, policy_version 75230 (0.0009) -[2023-10-14 16:35:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154271744. Throughput: 0: 1682.7, 1: 1666.7. Samples: 38577992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:08,165][74987] Avg episode reward: [(0, '26.550'), (1, '34.770')] -[2023-10-14 16:35:08,600][75949] Updated weights for policy 0, policy_version 75431 (0.0010) -[2023-10-14 16:35:08,974][75949] Updated weights for policy 0, policy_version 75441 (0.0010) -[2023-10-14 16:35:09,343][75949] Updated weights for policy 0, policy_version 75451 (0.0009) -[2023-10-14 16:35:09,435][75950] Updated weights for policy 1, policy_version 75240 (0.0008) -[2023-10-14 16:35:09,800][75950] Updated weights for policy 1, policy_version 75250 (0.0009) -[2023-10-14 16:35:10,166][75950] Updated weights for policy 1, policy_version 75260 (0.0010) -[2023-10-14 16:35:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154337280. Throughput: 0: 1682.0, 1: 1673.7. Samples: 38598572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:13,165][74987] Avg episode reward: [(0, '26.680'), (1, '34.590')] -[2023-10-14 16:35:13,522][75949] Updated weights for policy 0, policy_version 75461 (0.0009) -[2023-10-14 16:35:13,887][75949] Updated weights for policy 0, policy_version 75471 (0.0010) -[2023-10-14 16:35:14,260][75950] Updated weights for policy 1, policy_version 75270 (0.0008) -[2023-10-14 16:35:14,261][75949] Updated weights for policy 0, policy_version 75481 (0.0010) -[2023-10-14 16:35:14,620][75950] Updated weights for policy 1, policy_version 75280 (0.0007) -[2023-10-14 16:35:14,994][75950] Updated weights for policy 1, policy_version 75290 (0.0007) -[2023-10-14 16:35:18,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154402816. Throughput: 0: 1677.8, 1: 1667.3. Samples: 38607714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:18,164][74987] Avg episode reward: [(0, '26.480'), (1, '34.410')] -[2023-10-14 16:35:18,312][75949] Updated weights for policy 0, policy_version 75491 (0.0007) -[2023-10-14 16:35:18,684][75949] Updated weights for policy 0, policy_version 75501 (0.0007) -[2023-10-14 16:35:19,056][75949] Updated weights for policy 0, policy_version 75511 (0.0007) -[2023-10-14 16:35:19,174][75950] Updated weights for policy 1, policy_version 75300 (0.0009) -[2023-10-14 16:35:19,539][75950] Updated weights for policy 1, policy_version 75310 (0.0008) -[2023-10-14 16:35:19,893][75950] Updated weights for policy 1, policy_version 75320 (0.0009) -[2023-10-14 16:35:23,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154468352. Throughput: 0: 1673.7, 1: 1680.1. Samples: 38628338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:23,164][74987] Avg episode reward: [(0, '27.750'), (1, '32.750')] -[2023-10-14 16:35:23,283][75949] Updated weights for policy 0, policy_version 75521 (0.0009) -[2023-10-14 16:35:23,660][75949] Updated weights for policy 0, policy_version 75531 (0.0008) -[2023-10-14 16:35:23,940][75950] Updated weights for policy 1, policy_version 75330 (0.0010) -[2023-10-14 16:35:24,036][75949] Updated weights for policy 0, policy_version 75541 (0.0008) -[2023-10-14 16:35:24,305][75950] Updated weights for policy 1, policy_version 75340 (0.0008) -[2023-10-14 16:35:24,412][75949] Updated weights for policy 0, policy_version 75551 (0.0008) -[2023-10-14 16:35:24,663][75950] Updated weights for policy 1, policy_version 75350 (0.0007) -[2023-10-14 16:35:25,031][75950] Updated weights for policy 1, policy_version 75360 (0.0008) -[2023-10-14 16:35:28,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154533888. Throughput: 0: 1673.1, 1: 1675.4. Samples: 38649088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:28,165][74987] Avg episode reward: [(0, '27.380'), (1, '34.610')] -[2023-10-14 16:35:28,469][75949] Updated weights for policy 0, policy_version 75561 (0.0008) -[2023-10-14 16:35:28,833][75949] Updated weights for policy 0, policy_version 75571 (0.0008) -[2023-10-14 16:35:29,000][75950] Updated weights for policy 1, policy_version 75370 (0.0009) -[2023-10-14 16:35:29,204][75949] Updated weights for policy 0, policy_version 75581 (0.0008) -[2023-10-14 16:35:29,364][75950] Updated weights for policy 1, policy_version 75380 (0.0009) -[2023-10-14 16:35:29,736][75950] Updated weights for policy 1, policy_version 75390 (0.0009) -[2023-10-14 16:35:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 154599424. Throughput: 0: 1675.4, 1: 1675.0. Samples: 38658254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:33,165][74987] Avg episode reward: [(0, '29.380'), (1, '34.350')] -[2023-10-14 16:35:33,295][75949] Updated weights for policy 0, policy_version 75591 (0.0009) -[2023-10-14 16:35:33,671][75949] Updated weights for policy 0, policy_version 75601 (0.0007) -[2023-10-14 16:35:33,810][75950] Updated weights for policy 1, policy_version 75400 (0.0009) -[2023-10-14 16:35:34,038][75949] Updated weights for policy 0, policy_version 75611 (0.0008) -[2023-10-14 16:35:34,172][75950] Updated weights for policy 1, policy_version 75410 (0.0007) -[2023-10-14 16:35:34,529][75950] Updated weights for policy 1, policy_version 75420 (0.0010) -[2023-10-14 16:35:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154664960. Throughput: 0: 1675.0, 1: 1681.2. Samples: 38678788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:38,165][74987] Avg episode reward: [(0, '26.490'), (1, '33.860')] -[2023-10-14 16:35:38,202][75949] Updated weights for policy 0, policy_version 75621 (0.0007) -[2023-10-14 16:35:38,564][75950] Updated weights for policy 1, policy_version 75430 (0.0008) -[2023-10-14 16:35:38,571][75949] Updated weights for policy 0, policy_version 75631 (0.0007) -[2023-10-14 16:35:38,934][75950] Updated weights for policy 1, policy_version 75440 (0.0007) -[2023-10-14 16:35:38,943][75949] Updated weights for policy 0, policy_version 75641 (0.0009) -[2023-10-14 16:35:39,298][75950] Updated weights for policy 1, policy_version 75450 (0.0007) -[2023-10-14 16:35:43,051][75949] Updated weights for policy 0, policy_version 75651 (0.0009) -[2023-10-14 16:35:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 154730496. Throughput: 0: 1681.5, 1: 1678.6. Samples: 38699432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:43,165][74987] Avg episode reward: [(0, '26.920'), (1, '34.030')] -[2023-10-14 16:35:43,378][75950] Updated weights for policy 1, policy_version 75460 (0.0008) -[2023-10-14 16:35:43,431][75949] Updated weights for policy 0, policy_version 75661 (0.0008) -[2023-10-14 16:35:43,737][75950] Updated weights for policy 1, policy_version 75470 (0.0008) -[2023-10-14 16:35:43,805][75949] Updated weights for policy 0, policy_version 75671 (0.0008) -[2023-10-14 16:35:44,108][75950] Updated weights for policy 1, policy_version 75480 (0.0008) -[2023-10-14 16:35:47,797][75949] Updated weights for policy 0, policy_version 75681 (0.0009) -[2023-10-14 16:35:48,161][75949] Updated weights for policy 0, policy_version 75691 (0.0010) -[2023-10-14 16:35:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154796032. Throughput: 0: 1684.0, 1: 1677.3. Samples: 38708446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:48,164][74987] Avg episode reward: [(0, '26.490'), (1, '35.580')] -[2023-10-14 16:35:48,388][75950] Updated weights for policy 1, policy_version 75490 (0.0009) -[2023-10-14 16:35:48,533][75949] Updated weights for policy 0, policy_version 75701 (0.0009) -[2023-10-14 16:35:48,781][75950] Updated weights for policy 1, policy_version 75500 (0.0008) -[2023-10-14 16:35:48,906][75949] Updated weights for policy 0, policy_version 75711 (0.0008) -[2023-10-14 16:35:49,141][75950] Updated weights for policy 1, policy_version 75510 (0.0009) -[2023-10-14 16:35:49,508][75950] Updated weights for policy 1, policy_version 75520 (0.0007) -[2023-10-14 16:35:52,974][75949] Updated weights for policy 0, policy_version 75721 (0.0009) -[2023-10-14 16:35:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 154861568. Throughput: 0: 1673.1, 1: 1673.5. Samples: 38728588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:53,164][74987] Avg episode reward: [(0, '27.680'), (1, '32.850')] -[2023-10-14 16:35:53,336][75949] Updated weights for policy 0, policy_version 75731 (0.0007) -[2023-10-14 16:35:53,628][75950] Updated weights for policy 1, policy_version 75530 (0.0007) -[2023-10-14 16:35:53,714][75949] Updated weights for policy 0, policy_version 75741 (0.0007) -[2023-10-14 16:35:53,991][75950] Updated weights for policy 1, policy_version 75540 (0.0008) -[2023-10-14 16:35:54,349][75950] Updated weights for policy 1, policy_version 75550 (0.0009) -[2023-10-14 16:35:57,927][75949] Updated weights for policy 0, policy_version 75751 (0.0009) -[2023-10-14 16:35:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154927104. Throughput: 0: 1673.0, 1: 1678.1. Samples: 38749368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:35:58,164][74987] Avg episode reward: [(0, '30.040'), (1, '32.410')] -[2023-10-14 16:35:58,302][75949] Updated weights for policy 0, policy_version 75761 (0.0008) -[2023-10-14 16:35:58,369][75950] Updated weights for policy 1, policy_version 75560 (0.0008) -[2023-10-14 16:35:58,671][75949] Updated weights for policy 0, policy_version 75771 (0.0008) -[2023-10-14 16:35:58,727][75950] Updated weights for policy 1, policy_version 75570 (0.0008) -[2023-10-14 16:35:59,095][75950] Updated weights for policy 1, policy_version 75580 (0.0007) -[2023-10-14 16:36:02,444][75949] Updated weights for policy 0, policy_version 75781 (0.0009) -[2023-10-14 16:36:02,810][75949] Updated weights for policy 0, policy_version 75791 (0.0007) -[2023-10-14 16:36:03,081][75950] Updated weights for policy 1, policy_version 75590 (0.0008) -[2023-10-14 16:36:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 154992640. Throughput: 0: 1681.5, 1: 1673.7. Samples: 38758696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:36:03,164][74987] Avg episode reward: [(0, '28.090'), (1, '33.280')] -[2023-10-14 16:36:03,177][75949] Updated weights for policy 0, policy_version 75801 (0.0008) -[2023-10-14 16:36:03,437][75950] Updated weights for policy 1, policy_version 75600 (0.0008) -[2023-10-14 16:36:03,808][75950] Updated weights for policy 1, policy_version 75610 (0.0010) -[2023-10-14 16:36:07,221][75949] Updated weights for policy 0, policy_version 75811 (0.0009) -[2023-10-14 16:36:07,590][75949] Updated weights for policy 0, policy_version 75821 (0.0009) -[2023-10-14 16:36:07,949][75950] Updated weights for policy 1, policy_version 75620 (0.0009) -[2023-10-14 16:36:07,967][75949] Updated weights for policy 0, policy_version 75831 (0.0010) -[2023-10-14 16:36:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 155058176. Throughput: 0: 1683.3, 1: 1677.0. Samples: 38779554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:36:08,164][74987] Avg episode reward: [(0, '31.350'), (1, '32.720')] -[2023-10-14 16:36:08,287][75615] Saving new best policy, reward=31.350! -[2023-10-14 16:36:08,316][75950] Updated weights for policy 1, policy_version 75630 (0.0008) -[2023-10-14 16:36:08,687][75950] Updated weights for policy 1, policy_version 75640 (0.0007) -[2023-10-14 16:36:12,151][75949] Updated weights for policy 0, policy_version 75841 (0.0009) -[2023-10-14 16:36:12,447][75950] Updated weights for policy 1, policy_version 75650 (0.0008) -[2023-10-14 16:36:12,516][75949] Updated weights for policy 0, policy_version 75851 (0.0009) -[2023-10-14 16:36:12,817][75950] Updated weights for policy 1, policy_version 75660 (0.0010) -[2023-10-14 16:36:12,886][75949] Updated weights for policy 0, policy_version 75861 (0.0009) -[2023-10-14 16:36:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 155123712. Throughput: 0: 1665.5, 1: 1675.0. Samples: 38799410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:36:13,164][74987] Avg episode reward: [(0, '26.710'), (1, '31.120')] -[2023-10-14 16:36:13,183][75950] Updated weights for policy 1, policy_version 75670 (0.0008) -[2023-10-14 16:36:13,255][75949] Updated weights for policy 0, policy_version 75871 (0.0009) -[2023-10-14 16:36:13,539][75950] Updated weights for policy 1, policy_version 75680 (0.0010) -[2023-10-14 16:36:17,366][75949] Updated weights for policy 0, policy_version 75881 (0.0008) -[2023-10-14 16:36:17,735][75949] Updated weights for policy 0, policy_version 75891 (0.0009) -[2023-10-14 16:36:17,820][75950] Updated weights for policy 1, policy_version 75690 (0.0008) -[2023-10-14 16:36:18,097][75949] Updated weights for policy 0, policy_version 75901 (0.0009) -[2023-10-14 16:36:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 155189248. Throughput: 0: 1678.9, 1: 1676.3. Samples: 38809238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:36:18,164][74987] Avg episode reward: [(0, '27.590'), (1, '33.100')] -[2023-10-14 16:36:18,183][75950] Updated weights for policy 1, policy_version 75700 (0.0008) -[2023-10-14 16:36:18,547][75950] Updated weights for policy 1, policy_version 75710 (0.0007) -[2023-10-14 16:36:22,140][75949] Updated weights for policy 0, policy_version 75911 (0.0008) -[2023-10-14 16:36:22,505][75949] Updated weights for policy 0, policy_version 75921 (0.0008) -[2023-10-14 16:36:22,837][75950] Updated weights for policy 1, policy_version 75720 (0.0008) -[2023-10-14 16:36:22,874][75949] Updated weights for policy 0, policy_version 75931 (0.0009) -[2023-10-14 16:36:23,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 155287552. Throughput: 0: 1683.2, 1: 1674.1. Samples: 38829868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:36:23,165][74987] Avg episode reward: [(0, '25.860'), (1, '31.580')] -[2023-10-14 16:36:23,202][75950] Updated weights for policy 1, policy_version 75730 (0.0008) -[2023-10-14 16:36:23,575][75950] Updated weights for policy 1, policy_version 75740 (0.0011) -[2023-10-14 16:36:26,900][75949] Updated weights for policy 0, policy_version 75941 (0.0009) -[2023-10-14 16:36:27,271][75949] Updated weights for policy 0, policy_version 75951 (0.0007) -[2023-10-14 16:36:27,637][75949] Updated weights for policy 0, policy_version 75961 (0.0008) -[2023-10-14 16:36:27,854][75950] Updated weights for policy 1, policy_version 75750 (0.0011) -[2023-10-14 16:36:28,164][74987] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 155353088. Throughput: 0: 1659.3, 1: 1671.5. Samples: 38849320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:36:28,165][74987] Avg episode reward: [(0, '28.240'), (1, '30.980')] -[2023-10-14 16:36:28,214][75950] Updated weights for policy 1, policy_version 75760 (0.0008) -[2023-10-14 16:36:28,586][75950] Updated weights for policy 1, policy_version 75770 (0.0009) -[2023-10-14 16:36:31,578][75949] Updated weights for policy 0, policy_version 75971 (0.0008) -[2023-10-14 16:36:31,948][75949] Updated weights for policy 0, policy_version 75981 (0.0009) -[2023-10-14 16:36:32,324][75949] Updated weights for policy 0, policy_version 75991 (0.0008) -[2023-10-14 16:36:32,819][75950] Updated weights for policy 1, policy_version 75780 (0.0009) -[2023-10-14 16:36:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 155418624. Throughput: 0: 1686.0, 1: 1672.3. Samples: 38859568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:36:33,164][74987] Avg episode reward: [(0, '27.390'), (1, '31.940')] -[2023-10-14 16:36:33,215][75950] Updated weights for policy 1, policy_version 75790 (0.0009) -[2023-10-14 16:36:33,579][75950] Updated weights for policy 1, policy_version 75800 (0.0008) -[2023-10-14 16:36:36,618][75949] Updated weights for policy 0, policy_version 76001 (0.0007) -[2023-10-14 16:36:36,980][75949] Updated weights for policy 0, policy_version 76011 (0.0008) -[2023-10-14 16:36:37,359][75949] Updated weights for policy 0, policy_version 76021 (0.0009) -[2023-10-14 16:36:37,546][75950] Updated weights for policy 1, policy_version 75810 (0.0009) -[2023-10-14 16:36:37,721][75949] Updated weights for policy 0, policy_version 76031 (0.0007) -[2023-10-14 16:36:37,915][75950] Updated weights for policy 1, policy_version 75820 (0.0008) -[2023-10-14 16:36:38,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 155484160. Throughput: 0: 1690.8, 1: 1670.6. Samples: 38879852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:36:38,164][74987] Avg episode reward: [(0, '28.090'), (1, '31.950')] -[2023-10-14 16:36:38,283][75950] Updated weights for policy 1, policy_version 75830 (0.0011) -[2023-10-14 16:36:38,652][75950] Updated weights for policy 1, policy_version 75840 (0.0009) -[2023-10-14 16:36:41,910][75949] Updated weights for policy 0, policy_version 76041 (0.0007) -[2023-10-14 16:36:42,270][75949] Updated weights for policy 0, policy_version 76051 (0.0007) -[2023-10-14 16:36:42,599][75950] Updated weights for policy 1, policy_version 75850 (0.0008) -[2023-10-14 16:36:42,644][75949] Updated weights for policy 0, policy_version 76061 (0.0009) -[2023-10-14 16:36:42,960][75950] Updated weights for policy 1, policy_version 75860 (0.0008) -[2023-10-14 16:36:43,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 155549696. Throughput: 0: 1667.2, 1: 1657.3. Samples: 38898972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:36:43,164][74987] Avg episode reward: [(0, '25.510'), (1, '34.050')] -[2023-10-14 16:36:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000076064_77889536.pth... -[2023-10-14 16:36:43,207][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000074496_76283904.pth -[2023-10-14 16:36:43,327][75950] Updated weights for policy 1, policy_version 75870 (0.0009) -[2023-10-14 16:36:43,399][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000075872_77692928.pth... -[2023-10-14 16:36:43,440][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000074304_76087296.pth -[2023-10-14 16:36:46,697][75949] Updated weights for policy 0, policy_version 76071 (0.0009) -[2023-10-14 16:36:47,058][75949] Updated weights for policy 0, policy_version 76081 (0.0008) -[2023-10-14 16:36:47,422][75949] Updated weights for policy 0, policy_version 76091 (0.0010) -[2023-10-14 16:36:47,507][75950] Updated weights for policy 1, policy_version 75880 (0.0008) -[2023-10-14 16:36:47,881][75950] Updated weights for policy 1, policy_version 75890 (0.0008) -[2023-10-14 16:36:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 155615232. Throughput: 0: 1686.1, 1: 1665.9. Samples: 38909536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:36:48,165][74987] Avg episode reward: [(0, '25.940'), (1, '33.180')] -[2023-10-14 16:36:48,242][75950] Updated weights for policy 1, policy_version 75900 (0.0008) -[2023-10-14 16:36:51,594][75949] Updated weights for policy 0, policy_version 76101 (0.0009) -[2023-10-14 16:36:51,954][75949] Updated weights for policy 0, policy_version 76111 (0.0009) -[2023-10-14 16:36:52,283][75950] Updated weights for policy 1, policy_version 75910 (0.0008) -[2023-10-14 16:36:52,331][75949] Updated weights for policy 0, policy_version 76121 (0.0008) -[2023-10-14 16:36:52,641][75950] Updated weights for policy 1, policy_version 75920 (0.0009) -[2023-10-14 16:36:53,005][75950] Updated weights for policy 1, policy_version 75930 (0.0008) -[2023-10-14 16:36:53,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 155680768. Throughput: 0: 1669.6, 1: 1665.5. Samples: 38929630. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:36:53,164][74987] Avg episode reward: [(0, '27.250'), (1, '32.620')] -[2023-10-14 16:36:56,410][75949] Updated weights for policy 0, policy_version 76131 (0.0007) -[2023-10-14 16:36:56,768][75949] Updated weights for policy 0, policy_version 76141 (0.0008) -[2023-10-14 16:36:57,139][75949] Updated weights for policy 0, policy_version 76151 (0.0008) -[2023-10-14 16:36:57,188][75950] Updated weights for policy 1, policy_version 75940 (0.0008) -[2023-10-14 16:36:57,556][75950] Updated weights for policy 1, policy_version 75950 (0.0008) -[2023-10-14 16:36:57,915][75950] Updated weights for policy 1, policy_version 75960 (0.0009) -[2023-10-14 16:36:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 155746304. Throughput: 0: 1663.5, 1: 1651.1. Samples: 38948564. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:36:58,164][74987] Avg episode reward: [(0, '27.010'), (1, '32.140')] -[2023-10-14 16:37:01,100][75949] Updated weights for policy 0, policy_version 76161 (0.0008) -[2023-10-14 16:37:01,464][75949] Updated weights for policy 0, policy_version 76171 (0.0011) -[2023-10-14 16:37:01,835][75949] Updated weights for policy 0, policy_version 76181 (0.0010) -[2023-10-14 16:37:02,056][75950] Updated weights for policy 1, policy_version 75970 (0.0010) -[2023-10-14 16:37:02,200][75949] Updated weights for policy 0, policy_version 76191 (0.0010) -[2023-10-14 16:37:02,431][75950] Updated weights for policy 1, policy_version 75980 (0.0009) -[2023-10-14 16:37:02,798][75950] Updated weights for policy 1, policy_version 75990 (0.0010) -[2023-10-14 16:37:03,160][75950] Updated weights for policy 1, policy_version 76000 (0.0010) -[2023-10-14 16:37:03,164][74987] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 155844608. Throughput: 0: 1680.4, 1: 1662.8. Samples: 38959682. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:37:03,165][74987] Avg episode reward: [(0, '27.410'), (1, '33.560')] -[2023-10-14 16:37:06,450][75949] Updated weights for policy 0, policy_version 76201 (0.0011) -[2023-10-14 16:37:06,816][75949] Updated weights for policy 0, policy_version 76211 (0.0008) -[2023-10-14 16:37:07,183][75949] Updated weights for policy 0, policy_version 76221 (0.0007) -[2023-10-14 16:37:07,363][75950] Updated weights for policy 1, policy_version 76010 (0.0009) -[2023-10-14 16:37:07,733][75950] Updated weights for policy 1, policy_version 76020 (0.0011) -[2023-10-14 16:37:08,092][75950] Updated weights for policy 1, policy_version 76030 (0.0010) -[2023-10-14 16:37:08,164][74987] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 155910144. Throughput: 0: 1660.9, 1: 1667.1. Samples: 38979630. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:37:08,165][74987] Avg episode reward: [(0, '27.280'), (1, '33.730')] -[2023-10-14 16:37:11,377][75949] Updated weights for policy 0, policy_version 76231 (0.0007) -[2023-10-14 16:37:11,741][75949] Updated weights for policy 0, policy_version 76241 (0.0008) -[2023-10-14 16:37:12,108][75949] Updated weights for policy 0, policy_version 76251 (0.0008) -[2023-10-14 16:37:12,330][75950] Updated weights for policy 1, policy_version 76040 (0.0008) -[2023-10-14 16:37:12,691][75950] Updated weights for policy 1, policy_version 76050 (0.0009) -[2023-10-14 16:37:13,045][75950] Updated weights for policy 1, policy_version 76060 (0.0008) -[2023-10-14 16:37:13,164][74987] Fps is (10 sec: 9830.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 155942912. Throughput: 0: 1663.2, 1: 1655.1. Samples: 38998640. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:37:13,164][74987] Avg episode reward: [(0, '26.430'), (1, '34.330')] -[2023-10-14 16:37:16,177][75949] Updated weights for policy 0, policy_version 76261 (0.0008) -[2023-10-14 16:37:16,562][75949] Updated weights for policy 0, policy_version 76271 (0.0011) -[2023-10-14 16:37:16,930][75949] Updated weights for policy 0, policy_version 76281 (0.0009) -[2023-10-14 16:37:17,250][75950] Updated weights for policy 1, policy_version 76070 (0.0008) -[2023-10-14 16:37:17,620][75950] Updated weights for policy 1, policy_version 76080 (0.0007) -[2023-10-14 16:37:17,986][75950] Updated weights for policy 1, policy_version 76090 (0.0007) -[2023-10-14 16:37:18,164][74987] Fps is (10 sec: 9830.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 156008448. Throughput: 0: 1666.7, 1: 1668.0. Samples: 39009628. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:37:18,164][74987] Avg episode reward: [(0, '27.900'), (1, '32.590')] -[2023-10-14 16:37:20,852][75949] Updated weights for policy 0, policy_version 76291 (0.0008) -[2023-10-14 16:37:21,221][75949] Updated weights for policy 0, policy_version 76301 (0.0008) -[2023-10-14 16:37:21,597][75949] Updated weights for policy 0, policy_version 76311 (0.0007) -[2023-10-14 16:37:22,211][75950] Updated weights for policy 1, policy_version 76100 (0.0009) -[2023-10-14 16:37:22,602][75950] Updated weights for policy 1, policy_version 76110 (0.0009) -[2023-10-14 16:37:22,972][75950] Updated weights for policy 1, policy_version 76120 (0.0009) -[2023-10-14 16:37:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156073984. Throughput: 0: 1646.5, 1: 1671.7. Samples: 39029170. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:37:23,164][74987] Avg episode reward: [(0, '26.240'), (1, '34.210')] -[2023-10-14 16:37:25,636][75949] Updated weights for policy 0, policy_version 76321 (0.0010) -[2023-10-14 16:37:26,003][75949] Updated weights for policy 0, policy_version 76331 (0.0008) -[2023-10-14 16:37:26,370][75949] Updated weights for policy 0, policy_version 76341 (0.0009) -[2023-10-14 16:37:26,741][75949] Updated weights for policy 0, policy_version 76351 (0.0008) -[2023-10-14 16:37:26,944][75950] Updated weights for policy 1, policy_version 76130 (0.0007) -[2023-10-14 16:37:27,314][75950] Updated weights for policy 1, policy_version 76140 (0.0008) -[2023-10-14 16:37:27,682][75950] Updated weights for policy 1, policy_version 76150 (0.0007) -[2023-10-14 16:37:28,046][75950] Updated weights for policy 1, policy_version 76160 (0.0008) -[2023-10-14 16:37:28,163][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 156172288. Throughput: 0: 1666.7, 1: 1659.5. Samples: 39048652. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:37:28,164][74987] Avg episode reward: [(0, '28.450'), (1, '33.290')] -[2023-10-14 16:37:30,824][75949] Updated weights for policy 0, policy_version 76361 (0.0009) -[2023-10-14 16:37:31,195][75949] Updated weights for policy 0, policy_version 76371 (0.0008) -[2023-10-14 16:37:31,557][75949] Updated weights for policy 0, policy_version 76381 (0.0008) -[2023-10-14 16:37:32,024][75950] Updated weights for policy 1, policy_version 76170 (0.0008) -[2023-10-14 16:37:32,385][75950] Updated weights for policy 1, policy_version 76180 (0.0008) -[2023-10-14 16:37:32,753][75950] Updated weights for policy 1, policy_version 76190 (0.0008) -[2023-10-14 16:37:33,163][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 156237824. Throughput: 0: 1667.2, 1: 1671.7. Samples: 39059784. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:37:33,164][74987] Avg episode reward: [(0, '26.830'), (1, '30.850')] -[2023-10-14 16:37:35,699][75949] Updated weights for policy 0, policy_version 76391 (0.0008) -[2023-10-14 16:37:36,073][75949] Updated weights for policy 0, policy_version 76401 (0.0007) -[2023-10-14 16:37:36,440][75949] Updated weights for policy 0, policy_version 76411 (0.0007) -[2023-10-14 16:37:36,742][75950] Updated weights for policy 1, policy_version 76200 (0.0008) -[2023-10-14 16:37:37,110][75950] Updated weights for policy 1, policy_version 76210 (0.0007) -[2023-10-14 16:37:37,487][75950] Updated weights for policy 1, policy_version 76220 (0.0008) -[2023-10-14 16:37:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 156303360. Throughput: 0: 1656.3, 1: 1663.6. Samples: 39079024. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-10-14 16:37:38,164][74987] Avg episode reward: [(0, '30.980'), (1, '32.640')] -[2023-10-14 16:37:40,366][75949] Updated weights for policy 0, policy_version 76421 (0.0008) -[2023-10-14 16:37:40,748][75949] Updated weights for policy 0, policy_version 76431 (0.0008) -[2023-10-14 16:37:41,123][75949] Updated weights for policy 0, policy_version 76441 (0.0008) -[2023-10-14 16:37:41,532][75950] Updated weights for policy 1, policy_version 76230 (0.0008) -[2023-10-14 16:37:41,893][75950] Updated weights for policy 1, policy_version 76240 (0.0008) -[2023-10-14 16:37:42,257][75950] Updated weights for policy 1, policy_version 76250 (0.0009) -[2023-10-14 16:37:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 156368896. Throughput: 0: 1684.5, 1: 1656.5. Samples: 39098908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:37:43,164][74987] Avg episode reward: [(0, '26.520'), (1, '34.330')] -[2023-10-14 16:37:45,129][75949] Updated weights for policy 0, policy_version 76451 (0.0009) -[2023-10-14 16:37:45,495][75949] Updated weights for policy 0, policy_version 76461 (0.0009) -[2023-10-14 16:37:45,871][75949] Updated weights for policy 0, policy_version 76471 (0.0007) -[2023-10-14 16:37:46,275][75950] Updated weights for policy 1, policy_version 76260 (0.0008) -[2023-10-14 16:37:46,637][75950] Updated weights for policy 1, policy_version 76270 (0.0009) -[2023-10-14 16:37:47,011][75950] Updated weights for policy 1, policy_version 76280 (0.0007) -[2023-10-14 16:37:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 156434432. Throughput: 0: 1667.8, 1: 1675.0. Samples: 39110108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:37:48,165][74987] Avg episode reward: [(0, '28.980'), (1, '31.780')] -[2023-10-14 16:37:49,799][75949] Updated weights for policy 0, policy_version 76481 (0.0009) -[2023-10-14 16:37:50,166][75949] Updated weights for policy 0, policy_version 76491 (0.0008) -[2023-10-14 16:37:50,541][75949] Updated weights for policy 0, policy_version 76501 (0.0007) -[2023-10-14 16:37:50,915][75949] Updated weights for policy 0, policy_version 76511 (0.0008) -[2023-10-14 16:37:51,046][75950] Updated weights for policy 1, policy_version 76290 (0.0008) -[2023-10-14 16:37:51,417][75950] Updated weights for policy 1, policy_version 76300 (0.0010) -[2023-10-14 16:37:51,778][75950] Updated weights for policy 1, policy_version 76310 (0.0008) -[2023-10-14 16:37:52,145][75950] Updated weights for policy 1, policy_version 76320 (0.0007) -[2023-10-14 16:37:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 156499968. Throughput: 0: 1674.6, 1: 1660.8. Samples: 39129726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:37:53,164][74987] Avg episode reward: [(0, '26.280'), (1, '33.230')] -[2023-10-14 16:37:54,994][75949] Updated weights for policy 0, policy_version 76521 (0.0008) -[2023-10-14 16:37:55,368][75949] Updated weights for policy 0, policy_version 76531 (0.0007) -[2023-10-14 16:37:55,732][75949] Updated weights for policy 0, policy_version 76541 (0.0008) -[2023-10-14 16:37:56,306][75950] Updated weights for policy 1, policy_version 76330 (0.0009) -[2023-10-14 16:37:56,673][75950] Updated weights for policy 1, policy_version 76340 (0.0008) -[2023-10-14 16:37:57,028][75950] Updated weights for policy 1, policy_version 76350 (0.0007) -[2023-10-14 16:37:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 156565504. Throughput: 0: 1697.9, 1: 1669.7. Samples: 39150182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:37:58,164][74987] Avg episode reward: [(0, '28.040'), (1, '34.860')] -[2023-10-14 16:37:59,622][75949] Updated weights for policy 0, policy_version 76551 (0.0008) -[2023-10-14 16:37:59,991][75949] Updated weights for policy 0, policy_version 76561 (0.0010) -[2023-10-14 16:38:00,362][75949] Updated weights for policy 0, policy_version 76571 (0.0007) -[2023-10-14 16:38:01,051][75950] Updated weights for policy 1, policy_version 76360 (0.0007) -[2023-10-14 16:38:01,412][75950] Updated weights for policy 1, policy_version 76370 (0.0007) -[2023-10-14 16:38:01,771][75950] Updated weights for policy 1, policy_version 76380 (0.0007) -[2023-10-14 16:38:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156631040. Throughput: 0: 1672.4, 1: 1684.8. Samples: 39160702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:38:03,164][74987] Avg episode reward: [(0, '27.830'), (1, '33.600')] -[2023-10-14 16:38:04,712][75949] Updated weights for policy 0, policy_version 76581 (0.0009) -[2023-10-14 16:38:05,099][75949] Updated weights for policy 0, policy_version 76591 (0.0010) -[2023-10-14 16:38:05,470][75949] Updated weights for policy 0, policy_version 76601 (0.0009) -[2023-10-14 16:38:05,774][75950] Updated weights for policy 1, policy_version 76390 (0.0008) -[2023-10-14 16:38:06,142][75950] Updated weights for policy 1, policy_version 76400 (0.0008) -[2023-10-14 16:38:06,516][75950] Updated weights for policy 1, policy_version 76410 (0.0008) -[2023-10-14 16:38:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156696576. Throughput: 0: 1690.5, 1: 1670.3. Samples: 39180408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:38:08,165][74987] Avg episode reward: [(0, '27.760'), (1, '33.710')] -[2023-10-14 16:38:09,490][75949] Updated weights for policy 0, policy_version 76611 (0.0009) -[2023-10-14 16:38:09,869][75949] Updated weights for policy 0, policy_version 76621 (0.0009) -[2023-10-14 16:38:10,237][75949] Updated weights for policy 0, policy_version 76631 (0.0008) -[2023-10-14 16:38:10,541][75950] Updated weights for policy 1, policy_version 76420 (0.0010) -[2023-10-14 16:38:10,938][75950] Updated weights for policy 1, policy_version 76430 (0.0007) -[2023-10-14 16:38:11,295][75950] Updated weights for policy 1, policy_version 76440 (0.0008) -[2023-10-14 16:38:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 156762112. Throughput: 0: 1693.6, 1: 1687.1. Samples: 39200784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:38:13,164][74987] Avg episode reward: [(0, '27.980'), (1, '33.080')] -[2023-10-14 16:38:14,413][75949] Updated weights for policy 0, policy_version 76641 (0.0008) -[2023-10-14 16:38:14,782][75949] Updated weights for policy 0, policy_version 76651 (0.0007) -[2023-10-14 16:38:15,153][75949] Updated weights for policy 0, policy_version 76661 (0.0011) -[2023-10-14 16:38:15,329][75950] Updated weights for policy 1, policy_version 76450 (0.0010) -[2023-10-14 16:38:15,533][75949] Updated weights for policy 0, policy_version 76671 (0.0009) -[2023-10-14 16:38:15,695][75950] Updated weights for policy 1, policy_version 76460 (0.0009) -[2023-10-14 16:38:16,060][75950] Updated weights for policy 1, policy_version 76470 (0.0010) -[2023-10-14 16:38:16,426][75950] Updated weights for policy 1, policy_version 76480 (0.0007) -[2023-10-14 16:38:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 156827648. Throughput: 0: 1663.6, 1: 1688.3. Samples: 39210622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:38:18,164][74987] Avg episode reward: [(0, '26.050'), (1, '33.790')] -[2023-10-14 16:38:19,495][75949] Updated weights for policy 0, policy_version 76681 (0.0008) -[2023-10-14 16:38:19,857][75949] Updated weights for policy 0, policy_version 76691 (0.0010) -[2023-10-14 16:38:20,231][75949] Updated weights for policy 0, policy_version 76701 (0.0008) -[2023-10-14 16:38:20,433][75950] Updated weights for policy 1, policy_version 76490 (0.0008) -[2023-10-14 16:38:20,813][75950] Updated weights for policy 1, policy_version 76500 (0.0009) -[2023-10-14 16:38:21,176][75950] Updated weights for policy 1, policy_version 76510 (0.0010) -[2023-10-14 16:38:23,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 156893184. Throughput: 0: 1690.3, 1: 1675.4. Samples: 39230480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:38:23,165][74987] Avg episode reward: [(0, '29.130'), (1, '32.250')] -[2023-10-14 16:38:24,329][75949] Updated weights for policy 0, policy_version 76711 (0.0009) -[2023-10-14 16:38:24,689][75949] Updated weights for policy 0, policy_version 76721 (0.0009) -[2023-10-14 16:38:25,067][75949] Updated weights for policy 0, policy_version 76731 (0.0008) -[2023-10-14 16:38:25,281][75950] Updated weights for policy 1, policy_version 76520 (0.0008) -[2023-10-14 16:38:25,654][75950] Updated weights for policy 1, policy_version 76530 (0.0009) -[2023-10-14 16:38:26,012][75950] Updated weights for policy 1, policy_version 76540 (0.0010) -[2023-10-14 16:38:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 156958720. Throughput: 0: 1680.8, 1: 1694.3. Samples: 39250788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:38:28,165][74987] Avg episode reward: [(0, '26.990'), (1, '32.990')] -[2023-10-14 16:38:29,224][75949] Updated weights for policy 0, policy_version 76741 (0.0008) -[2023-10-14 16:38:29,582][75949] Updated weights for policy 0, policy_version 76751 (0.0009) -[2023-10-14 16:38:29,957][75949] Updated weights for policy 0, policy_version 76761 (0.0007) -[2023-10-14 16:38:30,162][75950] Updated weights for policy 1, policy_version 76550 (0.0007) -[2023-10-14 16:38:30,526][75950] Updated weights for policy 1, policy_version 76560 (0.0010) -[2023-10-14 16:38:30,891][75950] Updated weights for policy 1, policy_version 76570 (0.0010) -[2023-10-14 16:38:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157024256. Throughput: 0: 1669.6, 1: 1674.0. Samples: 39260572. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-10-14 16:38:33,165][74987] Avg episode reward: [(0, '29.490'), (1, '34.030')] -[2023-10-14 16:38:33,921][75949] Updated weights for policy 0, policy_version 76771 (0.0008) -[2023-10-14 16:38:34,290][75949] Updated weights for policy 0, policy_version 76781 (0.0009) -[2023-10-14 16:38:34,667][75949] Updated weights for policy 0, policy_version 76791 (0.0009) -[2023-10-14 16:38:34,888][75950] Updated weights for policy 1, policy_version 76580 (0.0009) -[2023-10-14 16:38:35,259][75950] Updated weights for policy 1, policy_version 76590 (0.0008) -[2023-10-14 16:38:35,625][75950] Updated weights for policy 1, policy_version 76600 (0.0008) -[2023-10-14 16:38:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 157089792. Throughput: 0: 1684.2, 1: 1676.9. Samples: 39280976. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-10-14 16:38:38,165][74987] Avg episode reward: [(0, '26.020'), (1, '32.900')] -[2023-10-14 16:38:38,729][75949] Updated weights for policy 0, policy_version 76801 (0.0009) -[2023-10-14 16:38:39,102][75949] Updated weights for policy 0, policy_version 76811 (0.0008) -[2023-10-14 16:38:39,468][75949] Updated weights for policy 0, policy_version 76821 (0.0009) -[2023-10-14 16:38:39,522][75950] Updated weights for policy 1, policy_version 76610 (0.0008) -[2023-10-14 16:38:39,836][75949] Updated weights for policy 0, policy_version 76831 (0.0008) -[2023-10-14 16:38:39,896][75950] Updated weights for policy 1, policy_version 76620 (0.0009) -[2023-10-14 16:38:40,251][75950] Updated weights for policy 1, policy_version 76630 (0.0010) -[2023-10-14 16:38:40,613][75950] Updated weights for policy 1, policy_version 76640 (0.0010) -[2023-10-14 16:38:43,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 157155328. Throughput: 0: 1687.0, 1: 1680.5. Samples: 39301720. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-10-14 16:38:43,165][74987] Avg episode reward: [(0, '29.500'), (1, '32.780')] -[2023-10-14 16:38:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000076640_78479360.pth... -[2023-10-14 16:38:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000076832_78675968.pth... -[2023-10-14 16:38:43,210][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000075264_77070336.pth -[2023-10-14 16:38:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000075072_76873728.pth -[2023-10-14 16:38:43,796][75949] Updated weights for policy 0, policy_version 76841 (0.0010) -[2023-10-14 16:38:44,167][75949] Updated weights for policy 0, policy_version 76851 (0.0010) -[2023-10-14 16:38:44,534][75949] Updated weights for policy 0, policy_version 76861 (0.0009) -[2023-10-14 16:38:45,003][75950] Updated weights for policy 1, policy_version 76650 (0.0009) -[2023-10-14 16:38:45,376][75950] Updated weights for policy 1, policy_version 76660 (0.0007) -[2023-10-14 16:38:45,743][75950] Updated weights for policy 1, policy_version 76670 (0.0008) -[2023-10-14 16:38:48,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 157220864. Throughput: 0: 1682.0, 1: 1659.0. Samples: 39311048. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-10-14 16:38:48,164][74987] Avg episode reward: [(0, '26.460'), (1, '33.390')] -[2023-10-14 16:38:48,691][75949] Updated weights for policy 0, policy_version 76871 (0.0007) -[2023-10-14 16:38:49,060][75949] Updated weights for policy 0, policy_version 76881 (0.0008) -[2023-10-14 16:38:49,438][75949] Updated weights for policy 0, policy_version 76891 (0.0010) -[2023-10-14 16:38:49,811][75950] Updated weights for policy 1, policy_version 76680 (0.0009) -[2023-10-14 16:38:50,178][75950] Updated weights for policy 1, policy_version 76690 (0.0009) -[2023-10-14 16:38:50,542][75950] Updated weights for policy 1, policy_version 76700 (0.0009) -[2023-10-14 16:38:53,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157286400. Throughput: 0: 1682.5, 1: 1672.4. Samples: 39331382. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-10-14 16:38:53,165][74987] Avg episode reward: [(0, '27.330'), (1, '33.070')] -[2023-10-14 16:38:53,633][75949] Updated weights for policy 0, policy_version 76901 (0.0007) -[2023-10-14 16:38:54,015][75949] Updated weights for policy 0, policy_version 76911 (0.0009) -[2023-10-14 16:38:54,377][75949] Updated weights for policy 0, policy_version 76921 (0.0009) -[2023-10-14 16:38:54,632][75950] Updated weights for policy 1, policy_version 76710 (0.0009) -[2023-10-14 16:38:54,990][75950] Updated weights for policy 1, policy_version 76720 (0.0009) -[2023-10-14 16:38:55,362][75950] Updated weights for policy 1, policy_version 76730 (0.0008) -[2023-10-14 16:38:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157351936. Throughput: 0: 1682.5, 1: 1677.3. Samples: 39351978. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-10-14 16:38:58,165][74987] Avg episode reward: [(0, '27.250'), (1, '32.770')] -[2023-10-14 16:38:58,462][75949] Updated weights for policy 0, policy_version 76931 (0.0007) -[2023-10-14 16:38:58,829][75949] Updated weights for policy 0, policy_version 76941 (0.0008) -[2023-10-14 16:38:59,205][75949] Updated weights for policy 0, policy_version 76951 (0.0008) -[2023-10-14 16:38:59,594][75950] Updated weights for policy 1, policy_version 76740 (0.0009) -[2023-10-14 16:39:00,015][75950] Updated weights for policy 1, policy_version 76750 (0.0009) -[2023-10-14 16:39:00,374][75950] Updated weights for policy 1, policy_version 76760 (0.0009) -[2023-10-14 16:39:03,116][75949] Updated weights for policy 0, policy_version 76961 (0.0008) -[2023-10-14 16:39:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 157417472. Throughput: 0: 1688.6, 1: 1655.7. Samples: 39361118. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-10-14 16:39:03,165][74987] Avg episode reward: [(0, '26.160'), (1, '31.680')] -[2023-10-14 16:39:03,487][75949] Updated weights for policy 0, policy_version 76971 (0.0010) -[2023-10-14 16:39:03,859][75949] Updated weights for policy 0, policy_version 76981 (0.0010) -[2023-10-14 16:39:04,232][75949] Updated weights for policy 0, policy_version 76991 (0.0009) -[2023-10-14 16:39:04,325][75950] Updated weights for policy 1, policy_version 76770 (0.0008) -[2023-10-14 16:39:04,684][75950] Updated weights for policy 1, policy_version 76780 (0.0009) -[2023-10-14 16:39:05,049][75950] Updated weights for policy 1, policy_version 76790 (0.0007) -[2023-10-14 16:39:05,422][75950] Updated weights for policy 1, policy_version 76800 (0.0008) -[2023-10-14 16:39:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157483008. Throughput: 0: 1685.4, 1: 1676.0. Samples: 39381744. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-10-14 16:39:08,165][74987] Avg episode reward: [(0, '26.500'), (1, '33.420')] -[2023-10-14 16:39:08,426][75949] Updated weights for policy 0, policy_version 77001 (0.0011) -[2023-10-14 16:39:08,796][75949] Updated weights for policy 0, policy_version 77011 (0.0007) -[2023-10-14 16:39:09,173][75949] Updated weights for policy 0, policy_version 77021 (0.0009) -[2023-10-14 16:39:09,600][75950] Updated weights for policy 1, policy_version 76810 (0.0008) -[2023-10-14 16:39:09,969][75950] Updated weights for policy 1, policy_version 76820 (0.0009) -[2023-10-14 16:39:10,331][75950] Updated weights for policy 1, policy_version 76830 (0.0009) -[2023-10-14 16:39:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157548544. Throughput: 0: 1687.2, 1: 1681.9. Samples: 39402400. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-10-14 16:39:13,164][74987] Avg episode reward: [(0, '26.400'), (1, '32.530')] -[2023-10-14 16:39:13,198][75949] Updated weights for policy 0, policy_version 77031 (0.0007) -[2023-10-14 16:39:13,568][75949] Updated weights for policy 0, policy_version 77041 (0.0010) -[2023-10-14 16:39:13,940][75949] Updated weights for policy 0, policy_version 77051 (0.0010) -[2023-10-14 16:39:14,438][75950] Updated weights for policy 1, policy_version 76840 (0.0008) -[2023-10-14 16:39:14,804][75950] Updated weights for policy 1, policy_version 76850 (0.0009) -[2023-10-14 16:39:15,168][75950] Updated weights for policy 1, policy_version 76860 (0.0010) -[2023-10-14 16:39:18,019][75949] Updated weights for policy 0, policy_version 77061 (0.0009) -[2023-10-14 16:39:18,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157614080. Throughput: 0: 1687.3, 1: 1668.2. Samples: 39411570. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-10-14 16:39:18,164][74987] Avg episode reward: [(0, '27.780'), (1, '32.530')] -[2023-10-14 16:39:18,384][75949] Updated weights for policy 0, policy_version 77071 (0.0008) -[2023-10-14 16:39:18,756][75949] Updated weights for policy 0, policy_version 77081 (0.0008) -[2023-10-14 16:39:19,253][75950] Updated weights for policy 1, policy_version 76870 (0.0009) -[2023-10-14 16:39:19,609][75950] Updated weights for policy 1, policy_version 76880 (0.0010) -[2023-10-14 16:39:19,982][75950] Updated weights for policy 1, policy_version 76890 (0.0010) -[2023-10-14 16:39:22,833][75949] Updated weights for policy 0, policy_version 77091 (0.0008) -[2023-10-14 16:39:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157679616. Throughput: 0: 1681.4, 1: 1675.1. Samples: 39432018. Policy #0 lag: (min: 17.0, avg: 20.8, max: 49.0) -[2023-10-14 16:39:23,165][74987] Avg episode reward: [(0, '27.550'), (1, '32.090')] -[2023-10-14 16:39:23,205][75949] Updated weights for policy 0, policy_version 77101 (0.0010) -[2023-10-14 16:39:23,577][75949] Updated weights for policy 0, policy_version 77111 (0.0007) -[2023-10-14 16:39:24,026][75950] Updated weights for policy 1, policy_version 76900 (0.0010) -[2023-10-14 16:39:24,388][75950] Updated weights for policy 1, policy_version 76910 (0.0010) -[2023-10-14 16:39:24,747][75950] Updated weights for policy 1, policy_version 76920 (0.0011) -[2023-10-14 16:39:27,495][75949] Updated weights for policy 0, policy_version 77121 (0.0008) -[2023-10-14 16:39:27,863][75949] Updated weights for policy 0, policy_version 77131 (0.0010) -[2023-10-14 16:39:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 157745152. Throughput: 0: 1672.7, 1: 1677.3. Samples: 39452466. Policy #0 lag: (min: 17.0, avg: 20.8, max: 49.0) -[2023-10-14 16:39:28,164][74987] Avg episode reward: [(0, '28.810'), (1, '34.250')] -[2023-10-14 16:39:28,233][75949] Updated weights for policy 0, policy_version 77141 (0.0008) -[2023-10-14 16:39:28,598][75949] Updated weights for policy 0, policy_version 77151 (0.0007) -[2023-10-14 16:39:28,829][75950] Updated weights for policy 1, policy_version 76930 (0.0010) -[2023-10-14 16:39:29,195][75950] Updated weights for policy 1, policy_version 76940 (0.0007) -[2023-10-14 16:39:29,565][75950] Updated weights for policy 1, policy_version 76950 (0.0010) -[2023-10-14 16:39:29,931][75950] Updated weights for policy 1, policy_version 76960 (0.0008) -[2023-10-14 16:39:32,691][75949] Updated weights for policy 0, policy_version 77161 (0.0009) -[2023-10-14 16:39:33,054][75949] Updated weights for policy 0, policy_version 77171 (0.0007) -[2023-10-14 16:39:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157810688. Throughput: 0: 1680.0, 1: 1672.6. Samples: 39461916. Policy #0 lag: (min: 17.0, avg: 20.8, max: 49.0) -[2023-10-14 16:39:33,165][74987] Avg episode reward: [(0, '26.930'), (1, '34.180')] -[2023-10-14 16:39:33,430][75949] Updated weights for policy 0, policy_version 77181 (0.0009) -[2023-10-14 16:39:33,996][75950] Updated weights for policy 1, policy_version 76970 (0.0008) -[2023-10-14 16:39:34,359][75950] Updated weights for policy 1, policy_version 76980 (0.0007) -[2023-10-14 16:39:34,716][75950] Updated weights for policy 1, policy_version 76990 (0.0009) -[2023-10-14 16:39:37,617][75949] Updated weights for policy 0, policy_version 77191 (0.0010) -[2023-10-14 16:39:37,975][75949] Updated weights for policy 0, policy_version 77201 (0.0011) -[2023-10-14 16:39:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157876224. Throughput: 0: 1681.1, 1: 1676.0. Samples: 39482448. Policy #0 lag: (min: 17.0, avg: 20.8, max: 49.0) -[2023-10-14 16:39:38,164][74987] Avg episode reward: [(0, '29.200'), (1, '32.820')] -[2023-10-14 16:39:38,339][75949] Updated weights for policy 0, policy_version 77211 (0.0008) -[2023-10-14 16:39:38,762][75950] Updated weights for policy 1, policy_version 77000 (0.0008) -[2023-10-14 16:39:39,126][75950] Updated weights for policy 1, policy_version 77010 (0.0007) -[2023-10-14 16:39:39,491][75950] Updated weights for policy 1, policy_version 77020 (0.0007) -[2023-10-14 16:39:42,409][75949] Updated weights for policy 0, policy_version 77221 (0.0008) -[2023-10-14 16:39:42,792][75949] Updated weights for policy 0, policy_version 77231 (0.0009) -[2023-10-14 16:39:43,159][75949] Updated weights for policy 0, policy_version 77241 (0.0007) -[2023-10-14 16:39:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 157941760. Throughput: 0: 1669.4, 1: 1679.8. Samples: 39502692. Policy #0 lag: (min: 17.0, avg: 20.8, max: 49.0) -[2023-10-14 16:39:43,165][74987] Avg episode reward: [(0, '26.660'), (1, '34.950')] -[2023-10-14 16:39:43,530][75950] Updated weights for policy 1, policy_version 77030 (0.0009) -[2023-10-14 16:39:43,888][75950] Updated weights for policy 1, policy_version 77040 (0.0011) -[2023-10-14 16:39:44,247][75950] Updated weights for policy 1, policy_version 77050 (0.0010) -[2023-10-14 16:39:47,123][75949] Updated weights for policy 0, policy_version 77251 (0.0007) -[2023-10-14 16:39:47,486][75949] Updated weights for policy 0, policy_version 77261 (0.0008) -[2023-10-14 16:39:47,849][75949] Updated weights for policy 0, policy_version 77271 (0.0009) -[2023-10-14 16:39:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 158007296. Throughput: 0: 1679.8, 1: 1682.8. Samples: 39512432. Policy #0 lag: (min: 17.0, avg: 20.8, max: 49.0) -[2023-10-14 16:39:48,165][74987] Avg episode reward: [(0, '27.650'), (1, '33.530')] -[2023-10-14 16:39:48,309][75950] Updated weights for policy 1, policy_version 77060 (0.0010) -[2023-10-14 16:39:48,689][75950] Updated weights for policy 1, policy_version 77070 (0.0009) -[2023-10-14 16:39:49,065][75950] Updated weights for policy 1, policy_version 77080 (0.0008) -[2023-10-14 16:39:52,084][75949] Updated weights for policy 0, policy_version 77281 (0.0008) -[2023-10-14 16:39:52,447][75949] Updated weights for policy 0, policy_version 77291 (0.0008) -[2023-10-14 16:39:52,815][75949] Updated weights for policy 0, policy_version 77301 (0.0009) -[2023-10-14 16:39:53,135][75950] Updated weights for policy 1, policy_version 77090 (0.0008) -[2023-10-14 16:39:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 158072832. Throughput: 0: 1681.8, 1: 1680.9. Samples: 39533066. Policy #0 lag: (min: 17.0, avg: 20.8, max: 49.0) -[2023-10-14 16:39:53,164][74987] Avg episode reward: [(0, '26.760'), (1, '32.120')] -[2023-10-14 16:39:53,192][75949] Updated weights for policy 0, policy_version 77311 (0.0008) -[2023-10-14 16:39:53,498][75950] Updated weights for policy 1, policy_version 77100 (0.0009) -[2023-10-14 16:39:53,867][75950] Updated weights for policy 1, policy_version 77110 (0.0008) -[2023-10-14 16:39:54,229][75950] Updated weights for policy 1, policy_version 77120 (0.0011) -[2023-10-14 16:39:57,343][75949] Updated weights for policy 0, policy_version 77321 (0.0007) -[2023-10-14 16:39:57,714][75949] Updated weights for policy 0, policy_version 77331 (0.0010) -[2023-10-14 16:39:58,081][75949] Updated weights for policy 0, policy_version 77341 (0.0011) -[2023-10-14 16:39:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158138368. Throughput: 0: 1666.5, 1: 1680.1. Samples: 39552998. Policy #0 lag: (min: 17.0, avg: 20.8, max: 49.0) -[2023-10-14 16:39:58,164][74987] Avg episode reward: [(0, '28.140'), (1, '32.690')] -[2023-10-14 16:39:58,371][75950] Updated weights for policy 1, policy_version 77130 (0.0009) -[2023-10-14 16:39:58,744][75950] Updated weights for policy 1, policy_version 77140 (0.0009) -[2023-10-14 16:39:59,104][75950] Updated weights for policy 1, policy_version 77150 (0.0008) -[2023-10-14 16:40:02,167][75949] Updated weights for policy 0, policy_version 77351 (0.0009) -[2023-10-14 16:40:02,530][75949] Updated weights for policy 0, policy_version 77361 (0.0008) -[2023-10-14 16:40:02,898][75949] Updated weights for policy 0, policy_version 77371 (0.0010) -[2023-10-14 16:40:03,125][75950] Updated weights for policy 1, policy_version 77160 (0.0009) -[2023-10-14 16:40:03,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 158236672. Throughput: 0: 1680.9, 1: 1678.6. Samples: 39562748. Policy #0 lag: (min: 17.0, avg: 20.8, max: 49.0) -[2023-10-14 16:40:03,164][74987] Avg episode reward: [(0, '27.080'), (1, '34.830')] -[2023-10-14 16:40:03,493][75950] Updated weights for policy 1, policy_version 77170 (0.0009) -[2023-10-14 16:40:03,865][75950] Updated weights for policy 1, policy_version 77180 (0.0011) -[2023-10-14 16:40:06,820][75949] Updated weights for policy 0, policy_version 77381 (0.0010) -[2023-10-14 16:40:07,184][75949] Updated weights for policy 0, policy_version 77391 (0.0011) -[2023-10-14 16:40:07,551][75949] Updated weights for policy 0, policy_version 77401 (0.0008) -[2023-10-14 16:40:08,016][75950] Updated weights for policy 1, policy_version 77190 (0.0009) -[2023-10-14 16:40:08,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 158302208. Throughput: 0: 1680.2, 1: 1686.1. Samples: 39583504. Policy #0 lag: (min: 25.0, avg: 29.5, max: 57.0) -[2023-10-14 16:40:08,165][74987] Avg episode reward: [(0, '28.560'), (1, '33.580')] -[2023-10-14 16:40:08,382][75950] Updated weights for policy 1, policy_version 77200 (0.0011) -[2023-10-14 16:40:08,746][75950] Updated weights for policy 1, policy_version 77210 (0.0011) -[2023-10-14 16:40:11,688][75949] Updated weights for policy 0, policy_version 77411 (0.0008) -[2023-10-14 16:40:12,055][75949] Updated weights for policy 0, policy_version 77421 (0.0010) -[2023-10-14 16:40:12,428][75949] Updated weights for policy 0, policy_version 77431 (0.0008) -[2023-10-14 16:40:12,626][75950] Updated weights for policy 1, policy_version 77220 (0.0010) -[2023-10-14 16:40:12,986][75950] Updated weights for policy 1, policy_version 77230 (0.0008) -[2023-10-14 16:40:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 158367744. Throughput: 0: 1661.3, 1: 1688.7. Samples: 39603218. Policy #0 lag: (min: 25.0, avg: 29.5, max: 57.0) -[2023-10-14 16:40:13,164][74987] Avg episode reward: [(0, '25.560'), (1, '34.540')] -[2023-10-14 16:40:13,347][75950] Updated weights for policy 1, policy_version 77240 (0.0008) -[2023-10-14 16:40:16,451][75949] Updated weights for policy 0, policy_version 77441 (0.0008) -[2023-10-14 16:40:16,816][75949] Updated weights for policy 0, policy_version 77451 (0.0008) -[2023-10-14 16:40:17,183][75949] Updated weights for policy 0, policy_version 77461 (0.0007) -[2023-10-14 16:40:17,417][75950] Updated weights for policy 1, policy_version 77250 (0.0007) -[2023-10-14 16:40:17,558][75949] Updated weights for policy 0, policy_version 77471 (0.0007) -[2023-10-14 16:40:17,783][75950] Updated weights for policy 1, policy_version 77260 (0.0009) -[2023-10-14 16:40:18,157][75950] Updated weights for policy 1, policy_version 77270 (0.0009) -[2023-10-14 16:40:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 158433280. Throughput: 0: 1680.6, 1: 1691.9. Samples: 39613676. Policy #0 lag: (min: 25.0, avg: 29.5, max: 57.0) -[2023-10-14 16:40:18,165][74987] Avg episode reward: [(0, '29.170'), (1, '33.390')] -[2023-10-14 16:40:18,523][75950] Updated weights for policy 1, policy_version 77280 (0.0010) -[2023-10-14 16:40:21,619][75949] Updated weights for policy 0, policy_version 77481 (0.0011) -[2023-10-14 16:40:21,979][75949] Updated weights for policy 0, policy_version 77491 (0.0008) -[2023-10-14 16:40:22,353][75949] Updated weights for policy 0, policy_version 77501 (0.0011) -[2023-10-14 16:40:22,825][75950] Updated weights for policy 1, policy_version 77290 (0.0007) -[2023-10-14 16:40:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 158498816. Throughput: 0: 1675.6, 1: 1688.1. Samples: 39633816. Policy #0 lag: (min: 25.0, avg: 29.5, max: 57.0) -[2023-10-14 16:40:23,165][74987] Avg episode reward: [(0, '26.280'), (1, '33.380')] -[2023-10-14 16:40:23,186][75950] Updated weights for policy 1, policy_version 77300 (0.0010) -[2023-10-14 16:40:23,549][75950] Updated weights for policy 1, policy_version 77310 (0.0010) -[2023-10-14 16:40:26,355][75949] Updated weights for policy 0, policy_version 77511 (0.0008) -[2023-10-14 16:40:26,726][75949] Updated weights for policy 0, policy_version 77521 (0.0009) -[2023-10-14 16:40:27,096][75949] Updated weights for policy 0, policy_version 77531 (0.0008) -[2023-10-14 16:40:27,661][75950] Updated weights for policy 1, policy_version 77320 (0.0010) -[2023-10-14 16:40:28,023][75950] Updated weights for policy 1, policy_version 77330 (0.0010) -[2023-10-14 16:40:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 158564352. Throughput: 0: 1671.3, 1: 1680.0. Samples: 39653500. Policy #0 lag: (min: 25.0, avg: 29.5, max: 57.0) -[2023-10-14 16:40:28,164][74987] Avg episode reward: [(0, '28.330'), (1, '32.760')] -[2023-10-14 16:40:28,398][75950] Updated weights for policy 1, policy_version 77340 (0.0010) -[2023-10-14 16:40:31,284][75949] Updated weights for policy 0, policy_version 77541 (0.0009) -[2023-10-14 16:40:31,680][75949] Updated weights for policy 0, policy_version 77551 (0.0008) -[2023-10-14 16:40:32,052][75949] Updated weights for policy 0, policy_version 77561 (0.0008) -[2023-10-14 16:40:32,450][75950] Updated weights for policy 1, policy_version 77350 (0.0007) -[2023-10-14 16:40:32,818][75950] Updated weights for policy 1, policy_version 77360 (0.0007) -[2023-10-14 16:40:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 158629888. Throughput: 0: 1684.4, 1: 1684.7. Samples: 39664040. Policy #0 lag: (min: 25.0, avg: 29.5, max: 57.0) -[2023-10-14 16:40:33,164][74987] Avg episode reward: [(0, '25.790'), (1, '33.400')] -[2023-10-14 16:40:33,179][75950] Updated weights for policy 1, policy_version 77370 (0.0007) -[2023-10-14 16:40:36,170][75949] Updated weights for policy 0, policy_version 77571 (0.0009) -[2023-10-14 16:40:36,531][75949] Updated weights for policy 0, policy_version 77581 (0.0008) -[2023-10-14 16:40:36,900][75949] Updated weights for policy 0, policy_version 77591 (0.0010) -[2023-10-14 16:40:37,353][75950] Updated weights for policy 1, policy_version 77380 (0.0008) -[2023-10-14 16:40:37,738][75950] Updated weights for policy 1, policy_version 77390 (0.0007) -[2023-10-14 16:40:38,110][75950] Updated weights for policy 1, policy_version 77400 (0.0008) -[2023-10-14 16:40:38,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 158695424. Throughput: 0: 1663.1, 1: 1689.0. Samples: 39683910. Policy #0 lag: (min: 25.0, avg: 29.5, max: 57.0) -[2023-10-14 16:40:38,164][74987] Avg episode reward: [(0, '28.380'), (1, '33.170')] -[2023-10-14 16:40:41,032][75949] Updated weights for policy 0, policy_version 77601 (0.0009) -[2023-10-14 16:40:41,391][75949] Updated weights for policy 0, policy_version 77611 (0.0010) -[2023-10-14 16:40:41,762][75949] Updated weights for policy 0, policy_version 77621 (0.0010) -[2023-10-14 16:40:42,136][75949] Updated weights for policy 0, policy_version 77631 (0.0007) -[2023-10-14 16:40:42,165][75950] Updated weights for policy 1, policy_version 77410 (0.0008) -[2023-10-14 16:40:42,529][75950] Updated weights for policy 1, policy_version 77420 (0.0008) -[2023-10-14 16:40:42,897][75950] Updated weights for policy 1, policy_version 77430 (0.0008) -[2023-10-14 16:40:43,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 158760960. Throughput: 0: 1662.7, 1: 1673.4. Samples: 39703122. Policy #0 lag: (min: 25.0, avg: 29.5, max: 57.0) -[2023-10-14 16:40:43,164][74987] Avg episode reward: [(0, '27.160'), (1, '32.730')] -[2023-10-14 16:40:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000077632_79495168.pth... -[2023-10-14 16:40:43,207][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000076064_77889536.pth -[2023-10-14 16:40:43,262][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000077440_79298560.pth... -[2023-10-14 16:40:43,262][75950] Updated weights for policy 1, policy_version 77440 (0.0009) -[2023-10-14 16:40:43,300][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000075872_77692928.pth -[2023-10-14 16:40:46,165][75949] Updated weights for policy 0, policy_version 77641 (0.0009) -[2023-10-14 16:40:46,538][75949] Updated weights for policy 0, policy_version 77651 (0.0009) -[2023-10-14 16:40:46,911][75949] Updated weights for policy 0, policy_version 77661 (0.0009) -[2023-10-14 16:40:47,207][75950] Updated weights for policy 1, policy_version 77450 (0.0010) -[2023-10-14 16:40:47,578][75950] Updated weights for policy 1, policy_version 77460 (0.0010) -[2023-10-14 16:40:47,937][75950] Updated weights for policy 1, policy_version 77470 (0.0010) -[2023-10-14 16:40:48,164][74987] Fps is (10 sec: 16383.6, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 158859264. Throughput: 0: 1677.2, 1: 1688.7. Samples: 39714216. Policy #0 lag: (min: 25.0, avg: 29.5, max: 57.0) -[2023-10-14 16:40:48,165][74987] Avg episode reward: [(0, '27.390'), (1, '32.160')] -[2023-10-14 16:40:51,029][75949] Updated weights for policy 0, policy_version 77671 (0.0008) -[2023-10-14 16:40:51,401][75949] Updated weights for policy 0, policy_version 77681 (0.0010) -[2023-10-14 16:40:51,774][75949] Updated weights for policy 0, policy_version 77691 (0.0010) -[2023-10-14 16:40:51,849][75950] Updated weights for policy 1, policy_version 77480 (0.0008) -[2023-10-14 16:40:52,211][75950] Updated weights for policy 1, policy_version 77490 (0.0009) -[2023-10-14 16:40:52,583][75950] Updated weights for policy 1, policy_version 77500 (0.0009) -[2023-10-14 16:40:53,164][74987] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 158924800. Throughput: 0: 1657.4, 1: 1685.1. Samples: 39733916. Policy #0 lag: (min: 25.0, avg: 29.5, max: 57.0) -[2023-10-14 16:40:53,164][74987] Avg episode reward: [(0, '27.480'), (1, '31.720')] -[2023-10-14 16:40:55,737][75949] Updated weights for policy 0, policy_version 77701 (0.0007) -[2023-10-14 16:40:56,100][75949] Updated weights for policy 0, policy_version 77711 (0.0010) -[2023-10-14 16:40:56,467][75949] Updated weights for policy 0, policy_version 77721 (0.0009) -[2023-10-14 16:40:56,666][75950] Updated weights for policy 1, policy_version 77510 (0.0008) -[2023-10-14 16:40:57,026][75950] Updated weights for policy 1, policy_version 77520 (0.0007) -[2023-10-14 16:40:57,389][75950] Updated weights for policy 1, policy_version 77530 (0.0009) -[2023-10-14 16:40:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 158990336. Throughput: 0: 1675.0, 1: 1661.5. Samples: 39753358. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 16:40:58,164][74987] Avg episode reward: [(0, '25.970'), (1, '32.640')] -[2023-10-14 16:41:00,527][75949] Updated weights for policy 0, policy_version 77731 (0.0009) -[2023-10-14 16:41:00,901][75949] Updated weights for policy 0, policy_version 77741 (0.0007) -[2023-10-14 16:41:01,262][75949] Updated weights for policy 0, policy_version 77751 (0.0009) -[2023-10-14 16:41:01,428][75950] Updated weights for policy 1, policy_version 77540 (0.0009) -[2023-10-14 16:41:01,796][75950] Updated weights for policy 1, policy_version 77550 (0.0008) -[2023-10-14 16:41:02,158][75950] Updated weights for policy 1, policy_version 77560 (0.0009) -[2023-10-14 16:41:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 159055872. Throughput: 0: 1671.5, 1: 1686.0. Samples: 39764762. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 16:41:03,165][74987] Avg episode reward: [(0, '28.860'), (1, '34.180')] -[2023-10-14 16:41:05,294][75949] Updated weights for policy 0, policy_version 77761 (0.0007) -[2023-10-14 16:41:05,669][75949] Updated weights for policy 0, policy_version 77771 (0.0008) -[2023-10-14 16:41:06,032][75949] Updated weights for policy 0, policy_version 77781 (0.0009) -[2023-10-14 16:41:06,324][75950] Updated weights for policy 1, policy_version 77570 (0.0009) -[2023-10-14 16:41:06,397][75949] Updated weights for policy 0, policy_version 77791 (0.0008) -[2023-10-14 16:41:06,684][75950] Updated weights for policy 1, policy_version 77580 (0.0009) -[2023-10-14 16:41:07,050][75950] Updated weights for policy 1, policy_version 77590 (0.0011) -[2023-10-14 16:41:07,417][75950] Updated weights for policy 1, policy_version 77600 (0.0008) -[2023-10-14 16:41:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 159121408. Throughput: 0: 1655.7, 1: 1680.4. Samples: 39783944. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 16:41:08,165][74987] Avg episode reward: [(0, '26.240'), (1, '33.620')] -[2023-10-14 16:41:10,380][75949] Updated weights for policy 0, policy_version 77801 (0.0007) -[2023-10-14 16:41:10,746][75949] Updated weights for policy 0, policy_version 77811 (0.0010) -[2023-10-14 16:41:11,123][75949] Updated weights for policy 0, policy_version 77821 (0.0011) -[2023-10-14 16:41:11,553][75950] Updated weights for policy 1, policy_version 77610 (0.0007) -[2023-10-14 16:41:11,921][75950] Updated weights for policy 1, policy_version 77620 (0.0007) -[2023-10-14 16:41:12,277][75950] Updated weights for policy 1, policy_version 77630 (0.0007) -[2023-10-14 16:41:13,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 159186944. Throughput: 0: 1679.1, 1: 1666.2. Samples: 39804038. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 16:41:13,164][74987] Avg episode reward: [(0, '30.910'), (1, '32.090')] -[2023-10-14 16:41:15,265][75949] Updated weights for policy 0, policy_version 77831 (0.0009) -[2023-10-14 16:41:15,629][75949] Updated weights for policy 0, policy_version 77841 (0.0008) -[2023-10-14 16:41:16,013][75949] Updated weights for policy 0, policy_version 77851 (0.0009) -[2023-10-14 16:41:16,256][75950] Updated weights for policy 1, policy_version 77640 (0.0010) -[2023-10-14 16:41:16,625][75950] Updated weights for policy 1, policy_version 77650 (0.0010) -[2023-10-14 16:41:16,995][75950] Updated weights for policy 1, policy_version 77660 (0.0010) -[2023-10-14 16:41:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 159252480. Throughput: 0: 1665.3, 1: 1688.1. Samples: 39814944. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 16:41:18,164][74987] Avg episode reward: [(0, '26.210'), (1, '34.170')] -[2023-10-14 16:41:20,095][75949] Updated weights for policy 0, policy_version 77861 (0.0008) -[2023-10-14 16:41:20,470][75949] Updated weights for policy 0, policy_version 77871 (0.0009) -[2023-10-14 16:41:20,841][75949] Updated weights for policy 0, policy_version 77881 (0.0007) -[2023-10-14 16:41:21,333][75950] Updated weights for policy 1, policy_version 77670 (0.0009) -[2023-10-14 16:41:21,719][75950] Updated weights for policy 1, policy_version 77680 (0.0007) -[2023-10-14 16:41:22,079][75950] Updated weights for policy 1, policy_version 77690 (0.0008) -[2023-10-14 16:41:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 159318016. Throughput: 0: 1674.6, 1: 1666.4. Samples: 39834258. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 16:41:23,165][74987] Avg episode reward: [(0, '30.590'), (1, '32.660')] -[2023-10-14 16:41:24,988][75949] Updated weights for policy 0, policy_version 77891 (0.0009) -[2023-10-14 16:41:25,388][75949] Updated weights for policy 0, policy_version 77901 (0.0010) -[2023-10-14 16:41:25,763][75949] Updated weights for policy 0, policy_version 77911 (0.0010) -[2023-10-14 16:41:26,154][75950] Updated weights for policy 1, policy_version 77700 (0.0007) -[2023-10-14 16:41:26,512][75950] Updated weights for policy 1, policy_version 77710 (0.0008) -[2023-10-14 16:41:26,876][75950] Updated weights for policy 1, policy_version 77720 (0.0009) -[2023-10-14 16:41:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 159383552. Throughput: 0: 1690.0, 1: 1667.1. Samples: 39854190. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 16:41:28,164][74987] Avg episode reward: [(0, '26.870'), (1, '33.610')] -[2023-10-14 16:41:29,839][75949] Updated weights for policy 0, policy_version 77921 (0.0009) -[2023-10-14 16:41:30,209][75949] Updated weights for policy 0, policy_version 77931 (0.0008) -[2023-10-14 16:41:30,573][75949] Updated weights for policy 0, policy_version 77941 (0.0009) -[2023-10-14 16:41:30,954][75949] Updated weights for policy 0, policy_version 77951 (0.0009) -[2023-10-14 16:41:31,046][75950] Updated weights for policy 1, policy_version 77730 (0.0009) -[2023-10-14 16:41:31,407][75950] Updated weights for policy 1, policy_version 77740 (0.0010) -[2023-10-14 16:41:31,778][75950] Updated weights for policy 1, policy_version 77750 (0.0009) -[2023-10-14 16:41:32,145][75950] Updated weights for policy 1, policy_version 77760 (0.0007) -[2023-10-14 16:41:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 159449088. Throughput: 0: 1667.3, 1: 1679.0. Samples: 39864802. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 16:41:33,164][74987] Avg episode reward: [(0, '28.540'), (1, '34.340')] -[2023-10-14 16:41:34,885][75949] Updated weights for policy 0, policy_version 77961 (0.0008) -[2023-10-14 16:41:35,263][75949] Updated weights for policy 0, policy_version 77971 (0.0007) -[2023-10-14 16:41:35,634][75949] Updated weights for policy 0, policy_version 77981 (0.0007) -[2023-10-14 16:41:36,296][75950] Updated weights for policy 1, policy_version 77770 (0.0008) -[2023-10-14 16:41:36,659][75950] Updated weights for policy 1, policy_version 77780 (0.0009) -[2023-10-14 16:41:37,026][75950] Updated weights for policy 1, policy_version 77790 (0.0009) -[2023-10-14 16:41:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 159514624. Throughput: 0: 1684.4, 1: 1659.0. Samples: 39884366. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 16:41:38,165][74987] Avg episode reward: [(0, '25.890'), (1, '33.410')] -[2023-10-14 16:41:39,741][75949] Updated weights for policy 0, policy_version 77991 (0.0007) -[2023-10-14 16:41:40,108][75949] Updated weights for policy 0, policy_version 78001 (0.0008) -[2023-10-14 16:41:40,484][75949] Updated weights for policy 0, policy_version 78011 (0.0009) -[2023-10-14 16:41:40,957][75950] Updated weights for policy 1, policy_version 77800 (0.0009) -[2023-10-14 16:41:41,322][75950] Updated weights for policy 1, policy_version 77810 (0.0010) -[2023-10-14 16:41:41,687][75950] Updated weights for policy 1, policy_version 77820 (0.0010) -[2023-10-14 16:41:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 159580160. Throughput: 0: 1688.5, 1: 1672.9. Samples: 39904620. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) -[2023-10-14 16:41:43,164][74987] Avg episode reward: [(0, '27.680'), (1, '32.790')] -[2023-10-14 16:41:44,460][75949] Updated weights for policy 0, policy_version 78021 (0.0008) -[2023-10-14 16:41:44,827][75949] Updated weights for policy 0, policy_version 78031 (0.0010) -[2023-10-14 16:41:45,203][75949] Updated weights for policy 0, policy_version 78041 (0.0008) -[2023-10-14 16:41:45,606][75950] Updated weights for policy 1, policy_version 77830 (0.0008) -[2023-10-14 16:41:45,972][75950] Updated weights for policy 1, policy_version 77840 (0.0008) -[2023-10-14 16:41:46,348][75950] Updated weights for policy 1, policy_version 77850 (0.0008) -[2023-10-14 16:41:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 159645696. Throughput: 0: 1667.9, 1: 1667.3. Samples: 39914846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:41:48,164][74987] Avg episode reward: [(0, '29.060'), (1, '31.310')] -[2023-10-14 16:41:49,313][75949] Updated weights for policy 0, policy_version 78051 (0.0008) -[2023-10-14 16:41:49,675][75949] Updated weights for policy 0, policy_version 78061 (0.0009) -[2023-10-14 16:41:50,052][75949] Updated weights for policy 0, policy_version 78071 (0.0009) -[2023-10-14 16:41:50,601][75950] Updated weights for policy 1, policy_version 77860 (0.0007) -[2023-10-14 16:41:50,962][75950] Updated weights for policy 1, policy_version 77870 (0.0007) -[2023-10-14 16:41:51,326][75950] Updated weights for policy 1, policy_version 77880 (0.0009) -[2023-10-14 16:41:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 159711232. Throughput: 0: 1688.3, 1: 1657.6. Samples: 39934508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:41:53,165][74987] Avg episode reward: [(0, '29.140'), (1, '31.960')] -[2023-10-14 16:41:54,084][75949] Updated weights for policy 0, policy_version 78081 (0.0009) -[2023-10-14 16:41:54,450][75949] Updated weights for policy 0, policy_version 78091 (0.0009) -[2023-10-14 16:41:54,824][75949] Updated weights for policy 0, policy_version 78101 (0.0009) -[2023-10-14 16:41:55,195][75949] Updated weights for policy 0, policy_version 78111 (0.0008) -[2023-10-14 16:41:55,301][75950] Updated weights for policy 1, policy_version 77890 (0.0009) -[2023-10-14 16:41:55,671][75950] Updated weights for policy 1, policy_version 77900 (0.0008) -[2023-10-14 16:41:56,035][75950] Updated weights for policy 1, policy_version 77910 (0.0008) -[2023-10-14 16:41:56,405][75950] Updated weights for policy 1, policy_version 77920 (0.0009) -[2023-10-14 16:41:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159776768. Throughput: 0: 1689.0, 1: 1676.7. Samples: 39955494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:41:58,165][74987] Avg episode reward: [(0, '28.300'), (1, '32.290')] -[2023-10-14 16:41:59,255][75949] Updated weights for policy 0, policy_version 78121 (0.0010) -[2023-10-14 16:41:59,623][75949] Updated weights for policy 0, policy_version 78131 (0.0008) -[2023-10-14 16:41:59,997][75949] Updated weights for policy 0, policy_version 78141 (0.0008) -[2023-10-14 16:42:00,373][75950] Updated weights for policy 1, policy_version 77930 (0.0010) -[2023-10-14 16:42:00,737][75950] Updated weights for policy 1, policy_version 77940 (0.0010) -[2023-10-14 16:42:01,108][75950] Updated weights for policy 1, policy_version 77950 (0.0008) -[2023-10-14 16:42:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159842304. Throughput: 0: 1677.6, 1: 1663.7. Samples: 39965302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:42:03,165][74987] Avg episode reward: [(0, '27.770'), (1, '32.250')] -[2023-10-14 16:42:03,937][75949] Updated weights for policy 0, policy_version 78151 (0.0008) -[2023-10-14 16:42:04,315][75949] Updated weights for policy 0, policy_version 78161 (0.0008) -[2023-10-14 16:42:04,695][75949] Updated weights for policy 0, policy_version 78171 (0.0009) -[2023-10-14 16:42:05,126][75950] Updated weights for policy 1, policy_version 77960 (0.0010) -[2023-10-14 16:42:05,495][75950] Updated weights for policy 1, policy_version 77970 (0.0010) -[2023-10-14 16:42:05,854][75950] Updated weights for policy 1, policy_version 77980 (0.0010) -[2023-10-14 16:42:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 159907840. Throughput: 0: 1695.2, 1: 1669.1. Samples: 39985654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:42:08,165][74987] Avg episode reward: [(0, '27.370'), (1, '32.710')] -[2023-10-14 16:42:08,647][75949] Updated weights for policy 0, policy_version 78181 (0.0008) -[2023-10-14 16:42:09,017][75949] Updated weights for policy 0, policy_version 78191 (0.0009) -[2023-10-14 16:42:09,387][75949] Updated weights for policy 0, policy_version 78201 (0.0009) -[2023-10-14 16:42:09,928][75950] Updated weights for policy 1, policy_version 77990 (0.0009) -[2023-10-14 16:42:10,316][75950] Updated weights for policy 1, policy_version 78000 (0.0008) -[2023-10-14 16:42:10,688][75950] Updated weights for policy 1, policy_version 78010 (0.0009) -[2023-10-14 16:42:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 159973376. Throughput: 0: 1697.9, 1: 1680.5. Samples: 40006220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:42:13,164][74987] Avg episode reward: [(0, '26.030'), (1, '32.170')] -[2023-10-14 16:42:13,546][75949] Updated weights for policy 0, policy_version 78211 (0.0008) -[2023-10-14 16:42:13,942][75949] Updated weights for policy 0, policy_version 78221 (0.0009) -[2023-10-14 16:42:14,308][75949] Updated weights for policy 0, policy_version 78231 (0.0009) -[2023-10-14 16:42:14,880][75950] Updated weights for policy 1, policy_version 78020 (0.0009) -[2023-10-14 16:42:15,253][75950] Updated weights for policy 1, policy_version 78030 (0.0007) -[2023-10-14 16:42:15,620][75950] Updated weights for policy 1, policy_version 78040 (0.0009) -[2023-10-14 16:42:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 160038912. Throughput: 0: 1688.6, 1: 1664.1. Samples: 40015674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:42:18,164][74987] Avg episode reward: [(0, '28.720'), (1, '33.120')] -[2023-10-14 16:42:18,410][75949] Updated weights for policy 0, policy_version 78241 (0.0009) -[2023-10-14 16:42:18,774][75949] Updated weights for policy 0, policy_version 78251 (0.0009) -[2023-10-14 16:42:19,158][75949] Updated weights for policy 0, policy_version 78261 (0.0009) -[2023-10-14 16:42:19,530][75949] Updated weights for policy 0, policy_version 78271 (0.0010) -[2023-10-14 16:42:19,799][75950] Updated weights for policy 1, policy_version 78050 (0.0008) -[2023-10-14 16:42:20,163][75950] Updated weights for policy 1, policy_version 78060 (0.0008) -[2023-10-14 16:42:20,527][75950] Updated weights for policy 1, policy_version 78070 (0.0007) -[2023-10-14 16:42:20,887][75950] Updated weights for policy 1, policy_version 78080 (0.0010) -[2023-10-14 16:42:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160104448. Throughput: 0: 1694.9, 1: 1672.1. Samples: 40035878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:42:23,164][74987] Avg episode reward: [(0, '27.890'), (1, '31.670')] -[2023-10-14 16:42:23,583][75949] Updated weights for policy 0, policy_version 78281 (0.0009) -[2023-10-14 16:42:23,953][75949] Updated weights for policy 0, policy_version 78291 (0.0011) -[2023-10-14 16:42:24,329][75949] Updated weights for policy 0, policy_version 78301 (0.0008) -[2023-10-14 16:42:25,099][75950] Updated weights for policy 1, policy_version 78090 (0.0011) -[2023-10-14 16:42:25,464][75950] Updated weights for policy 1, policy_version 78100 (0.0009) -[2023-10-14 16:42:25,827][75950] Updated weights for policy 1, policy_version 78110 (0.0007) -[2023-10-14 16:42:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160169984. Throughput: 0: 1697.1, 1: 1683.2. Samples: 40056734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:42:28,164][74987] Avg episode reward: [(0, '28.650'), (1, '32.370')] -[2023-10-14 16:42:28,287][75949] Updated weights for policy 0, policy_version 78311 (0.0008) -[2023-10-14 16:42:28,665][75949] Updated weights for policy 0, policy_version 78321 (0.0008) -[2023-10-14 16:42:29,042][75949] Updated weights for policy 0, policy_version 78331 (0.0007) -[2023-10-14 16:42:29,926][75950] Updated weights for policy 1, policy_version 78120 (0.0009) -[2023-10-14 16:42:30,296][75950] Updated weights for policy 1, policy_version 78130 (0.0009) -[2023-10-14 16:42:30,668][75950] Updated weights for policy 1, policy_version 78140 (0.0009) -[2023-10-14 16:42:33,085][75949] Updated weights for policy 0, policy_version 78341 (0.0010) -[2023-10-14 16:42:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160235520. Throughput: 0: 1696.2, 1: 1662.4. Samples: 40065984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:42:33,164][74987] Avg episode reward: [(0, '23.480'), (1, '32.760')] -[2023-10-14 16:42:33,465][75949] Updated weights for policy 0, policy_version 78351 (0.0011) -[2023-10-14 16:42:33,829][75949] Updated weights for policy 0, policy_version 78361 (0.0008) -[2023-10-14 16:42:34,773][75950] Updated weights for policy 1, policy_version 78150 (0.0009) -[2023-10-14 16:42:35,142][75950] Updated weights for policy 1, policy_version 78160 (0.0011) -[2023-10-14 16:42:35,517][75950] Updated weights for policy 1, policy_version 78170 (0.0007) -[2023-10-14 16:42:37,986][75949] Updated weights for policy 0, policy_version 78371 (0.0009) -[2023-10-14 16:42:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160301056. Throughput: 0: 1696.3, 1: 1672.6. Samples: 40086106. Policy #0 lag: (min: 6.0, avg: 6.7, max: 24.0) -[2023-10-14 16:42:38,165][74987] Avg episode reward: [(0, '27.360'), (1, '32.950')] -[2023-10-14 16:42:38,352][75949] Updated weights for policy 0, policy_version 78381 (0.0010) -[2023-10-14 16:42:38,730][75949] Updated weights for policy 0, policy_version 78391 (0.0010) -[2023-10-14 16:42:39,628][75950] Updated weights for policy 1, policy_version 78180 (0.0009) -[2023-10-14 16:42:39,992][75950] Updated weights for policy 1, policy_version 78190 (0.0008) -[2023-10-14 16:42:40,355][75950] Updated weights for policy 1, policy_version 78200 (0.0010) -[2023-10-14 16:42:42,572][75949] Updated weights for policy 0, policy_version 78401 (0.0008) -[2023-10-14 16:42:42,945][75949] Updated weights for policy 0, policy_version 78411 (0.0009) -[2023-10-14 16:42:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160366592. Throughput: 0: 1689.4, 1: 1670.5. Samples: 40106690. Policy #0 lag: (min: 6.0, avg: 6.7, max: 24.0) -[2023-10-14 16:42:43,164][74987] Avg episode reward: [(0, '25.390'), (1, '34.350')] -[2023-10-14 16:42:43,172][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000078208_80084992.pth... -[2023-10-14 16:42:43,206][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000076640_78479360.pth -[2023-10-14 16:42:43,210][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000078208_80084992.pth -[2023-10-14 16:42:43,315][75949] Updated weights for policy 0, policy_version 78421 (0.0008) -[2023-10-14 16:42:43,684][75949] Updated weights for policy 0, policy_version 78431 (0.0009) -[2023-10-14 16:42:43,712][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000078432_80314368.pth... -[2023-10-14 16:42:43,749][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000076832_78675968.pth -[2023-10-14 16:42:43,755][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000078432_80314368.pth -[2023-10-14 16:42:44,483][75950] Updated weights for policy 1, policy_version 78210 (0.0008) -[2023-10-14 16:42:44,838][75950] Updated weights for policy 1, policy_version 78220 (0.0008) -[2023-10-14 16:42:45,201][75950] Updated weights for policy 1, policy_version 78230 (0.0009) -[2023-10-14 16:42:45,561][75950] Updated weights for policy 1, policy_version 78240 (0.0007) -[2023-10-14 16:42:47,556][75949] Updated weights for policy 0, policy_version 78441 (0.0008) -[2023-10-14 16:42:47,925][75949] Updated weights for policy 0, policy_version 78451 (0.0008) -[2023-10-14 16:42:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160432128. Throughput: 0: 1698.3, 1: 1653.3. Samples: 40116124. Policy #0 lag: (min: 6.0, avg: 6.7, max: 24.0) -[2023-10-14 16:42:48,164][74987] Avg episode reward: [(0, '28.580'), (1, '33.650')] -[2023-10-14 16:42:48,294][75949] Updated weights for policy 0, policy_version 78461 (0.0007) -[2023-10-14 16:42:49,550][75950] Updated weights for policy 1, policy_version 78250 (0.0007) -[2023-10-14 16:42:49,922][75950] Updated weights for policy 1, policy_version 78260 (0.0007) -[2023-10-14 16:42:50,288][75950] Updated weights for policy 1, policy_version 78270 (0.0008) -[2023-10-14 16:42:52,465][75949] Updated weights for policy 0, policy_version 78471 (0.0009) -[2023-10-14 16:42:52,828][75949] Updated weights for policy 0, policy_version 78481 (0.0009) -[2023-10-14 16:42:53,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160497664. Throughput: 0: 1692.7, 1: 1669.2. Samples: 40136940. Policy #0 lag: (min: 6.0, avg: 6.7, max: 24.0) -[2023-10-14 16:42:53,164][74987] Avg episode reward: [(0, '28.140'), (1, '35.210')] -[2023-10-14 16:42:53,195][75949] Updated weights for policy 0, policy_version 78491 (0.0010) -[2023-10-14 16:42:54,245][75950] Updated weights for policy 1, policy_version 78280 (0.0008) -[2023-10-14 16:42:54,614][75950] Updated weights for policy 1, policy_version 78290 (0.0008) -[2023-10-14 16:42:54,991][75950] Updated weights for policy 1, policy_version 78300 (0.0009) -[2023-10-14 16:42:57,308][75949] Updated weights for policy 0, policy_version 78501 (0.0007) -[2023-10-14 16:42:57,667][75949] Updated weights for policy 0, policy_version 78511 (0.0009) -[2023-10-14 16:42:58,046][75949] Updated weights for policy 0, policy_version 78521 (0.0008) -[2023-10-14 16:42:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160563200. Throughput: 0: 1676.1, 1: 1677.0. Samples: 40157108. Policy #0 lag: (min: 6.0, avg: 6.7, max: 24.0) -[2023-10-14 16:42:58,164][74987] Avg episode reward: [(0, '28.800'), (1, '32.510')] -[2023-10-14 16:42:59,159][75950] Updated weights for policy 1, policy_version 78310 (0.0008) -[2023-10-14 16:42:59,542][75950] Updated weights for policy 1, policy_version 78320 (0.0009) -[2023-10-14 16:42:59,911][75950] Updated weights for policy 1, policy_version 78330 (0.0007) -[2023-10-14 16:43:02,068][75949] Updated weights for policy 0, policy_version 78531 (0.0010) -[2023-10-14 16:43:02,448][75949] Updated weights for policy 0, policy_version 78541 (0.0007) -[2023-10-14 16:43:02,816][75949] Updated weights for policy 0, policy_version 78551 (0.0007) -[2023-10-14 16:43:03,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 160661504. Throughput: 0: 1691.2, 1: 1666.6. Samples: 40166774. Policy #0 lag: (min: 6.0, avg: 6.7, max: 24.0) -[2023-10-14 16:43:03,164][74987] Avg episode reward: [(0, '29.210'), (1, '33.970')] -[2023-10-14 16:43:03,789][75950] Updated weights for policy 1, policy_version 78340 (0.0012) -[2023-10-14 16:43:04,160][75950] Updated weights for policy 1, policy_version 78350 (0.0012) -[2023-10-14 16:43:04,530][75950] Updated weights for policy 1, policy_version 78360 (0.0008) -[2023-10-14 16:43:06,792][75949] Updated weights for policy 0, policy_version 78561 (0.0008) -[2023-10-14 16:43:07,156][75949] Updated weights for policy 0, policy_version 78571 (0.0007) -[2023-10-14 16:43:07,529][75949] Updated weights for policy 0, policy_version 78581 (0.0008) -[2023-10-14 16:43:07,902][75949] Updated weights for policy 0, policy_version 78591 (0.0008) -[2023-10-14 16:43:08,164][74987] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 160727040. Throughput: 0: 1691.5, 1: 1680.4. Samples: 40187612. Policy #0 lag: (min: 6.0, avg: 6.7, max: 24.0) -[2023-10-14 16:43:08,165][74987] Avg episode reward: [(0, '28.470'), (1, '34.230')] -[2023-10-14 16:43:08,737][75950] Updated weights for policy 1, policy_version 78370 (0.0009) -[2023-10-14 16:43:09,103][75950] Updated weights for policy 1, policy_version 78380 (0.0008) -[2023-10-14 16:43:09,469][75950] Updated weights for policy 1, policy_version 78390 (0.0007) -[2023-10-14 16:43:09,841][75950] Updated weights for policy 1, policy_version 78400 (0.0009) -[2023-10-14 16:43:12,024][75949] Updated weights for policy 0, policy_version 78601 (0.0009) -[2023-10-14 16:43:12,403][75949] Updated weights for policy 0, policy_version 78611 (0.0007) -[2023-10-14 16:43:12,764][75949] Updated weights for policy 0, policy_version 78621 (0.0008) -[2023-10-14 16:43:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 160792576. Throughput: 0: 1668.6, 1: 1676.4. Samples: 40207262. Policy #0 lag: (min: 6.0, avg: 6.7, max: 24.0) -[2023-10-14 16:43:13,164][74987] Avg episode reward: [(0, '26.150'), (1, '33.170')] -[2023-10-14 16:43:14,040][75950] Updated weights for policy 1, policy_version 78410 (0.0008) -[2023-10-14 16:43:14,412][75950] Updated weights for policy 1, policy_version 78420 (0.0009) -[2023-10-14 16:43:14,778][75950] Updated weights for policy 1, policy_version 78430 (0.0009) -[2023-10-14 16:43:16,705][75949] Updated weights for policy 0, policy_version 78631 (0.0009) -[2023-10-14 16:43:17,085][75949] Updated weights for policy 0, policy_version 78641 (0.0009) -[2023-10-14 16:43:17,461][75949] Updated weights for policy 0, policy_version 78651 (0.0010) -[2023-10-14 16:43:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 160858112. Throughput: 0: 1692.5, 1: 1674.5. Samples: 40217498. Policy #0 lag: (min: 6.0, avg: 6.7, max: 24.0) -[2023-10-14 16:43:18,164][74987] Avg episode reward: [(0, '26.380'), (1, '32.540')] -[2023-10-14 16:43:18,837][75950] Updated weights for policy 1, policy_version 78440 (0.0008) -[2023-10-14 16:43:19,206][75950] Updated weights for policy 1, policy_version 78450 (0.0007) -[2023-10-14 16:43:19,572][75950] Updated weights for policy 1, policy_version 78460 (0.0008) -[2023-10-14 16:43:21,658][75949] Updated weights for policy 0, policy_version 78661 (0.0009) -[2023-10-14 16:43:22,036][75949] Updated weights for policy 0, policy_version 78671 (0.0007) -[2023-10-14 16:43:22,406][75949] Updated weights for policy 0, policy_version 78681 (0.0008) -[2023-10-14 16:43:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 160923648. Throughput: 0: 1684.9, 1: 1689.6. Samples: 40237960. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 16:43:23,164][74987] Avg episode reward: [(0, '27.160'), (1, '32.400')] -[2023-10-14 16:43:23,543][75950] Updated weights for policy 1, policy_version 78470 (0.0009) -[2023-10-14 16:43:23,900][75950] Updated weights for policy 1, policy_version 78480 (0.0011) -[2023-10-14 16:43:24,263][75950] Updated weights for policy 1, policy_version 78490 (0.0009) -[2023-10-14 16:43:26,536][75949] Updated weights for policy 0, policy_version 78691 (0.0008) -[2023-10-14 16:43:26,904][75949] Updated weights for policy 0, policy_version 78701 (0.0008) -[2023-10-14 16:43:27,267][75949] Updated weights for policy 0, policy_version 78711 (0.0009) -[2023-10-14 16:43:28,164][74987] Fps is (10 sec: 13106.6, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 160989184. Throughput: 0: 1660.7, 1: 1688.9. Samples: 40257424. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 16:43:28,165][74987] Avg episode reward: [(0, '26.920'), (1, '31.050')] -[2023-10-14 16:43:28,409][75950] Updated weights for policy 1, policy_version 78500 (0.0009) -[2023-10-14 16:43:28,779][75950] Updated weights for policy 1, policy_version 78510 (0.0007) -[2023-10-14 16:43:29,137][75950] Updated weights for policy 1, policy_version 78520 (0.0009) -[2023-10-14 16:43:31,556][75949] Updated weights for policy 0, policy_version 78721 (0.0009) -[2023-10-14 16:43:31,925][75949] Updated weights for policy 0, policy_version 78731 (0.0008) -[2023-10-14 16:43:32,296][75949] Updated weights for policy 0, policy_version 78741 (0.0007) -[2023-10-14 16:43:32,663][75949] Updated weights for policy 0, policy_version 78751 (0.0008) -[2023-10-14 16:43:33,121][75950] Updated weights for policy 1, policy_version 78530 (0.0009) -[2023-10-14 16:43:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161054720. Throughput: 0: 1675.5, 1: 1690.8. Samples: 40267608. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 16:43:33,165][74987] Avg episode reward: [(0, '26.740'), (1, '29.860')] -[2023-10-14 16:43:33,497][75950] Updated weights for policy 1, policy_version 78540 (0.0012) -[2023-10-14 16:43:33,862][75950] Updated weights for policy 1, policy_version 78550 (0.0011) -[2023-10-14 16:43:34,226][75950] Updated weights for policy 1, policy_version 78560 (0.0009) -[2023-10-14 16:43:36,600][75949] Updated weights for policy 0, policy_version 78761 (0.0010) -[2023-10-14 16:43:36,966][75949] Updated weights for policy 0, policy_version 78771 (0.0010) -[2023-10-14 16:43:37,340][75949] Updated weights for policy 0, policy_version 78781 (0.0010) -[2023-10-14 16:43:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161120256. Throughput: 0: 1667.9, 1: 1688.2. Samples: 40287966. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 16:43:38,165][74987] Avg episode reward: [(0, '27.730'), (1, '32.570')] -[2023-10-14 16:43:38,338][75950] Updated weights for policy 1, policy_version 78570 (0.0008) -[2023-10-14 16:43:38,699][75950] Updated weights for policy 1, policy_version 78580 (0.0008) -[2023-10-14 16:43:39,063][75950] Updated weights for policy 1, policy_version 78590 (0.0007) -[2023-10-14 16:43:41,311][75949] Updated weights for policy 0, policy_version 78791 (0.0009) -[2023-10-14 16:43:41,684][75949] Updated weights for policy 0, policy_version 78801 (0.0009) -[2023-10-14 16:43:42,055][75949] Updated weights for policy 0, policy_version 78811 (0.0008) -[2023-10-14 16:43:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161185792. Throughput: 0: 1666.9, 1: 1685.5. Samples: 40307968. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 16:43:43,164][74987] Avg episode reward: [(0, '29.780'), (1, '32.600')] -[2023-10-14 16:43:43,205][75950] Updated weights for policy 1, policy_version 78600 (0.0008) -[2023-10-14 16:43:43,576][75950] Updated weights for policy 1, policy_version 78610 (0.0007) -[2023-10-14 16:43:43,940][75950] Updated weights for policy 1, policy_version 78620 (0.0010) -[2023-10-14 16:43:46,082][75949] Updated weights for policy 0, policy_version 78821 (0.0010) -[2023-10-14 16:43:46,454][75949] Updated weights for policy 0, policy_version 78831 (0.0011) -[2023-10-14 16:43:46,824][75949] Updated weights for policy 0, policy_version 78841 (0.0008) -[2023-10-14 16:43:47,805][75950] Updated weights for policy 1, policy_version 78630 (0.0009) -[2023-10-14 16:43:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161251328. Throughput: 0: 1685.4, 1: 1688.8. Samples: 40318610. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 16:43:48,164][74987] Avg episode reward: [(0, '26.680'), (1, '32.130')] -[2023-10-14 16:43:48,178][75950] Updated weights for policy 1, policy_version 78640 (0.0011) -[2023-10-14 16:43:48,539][75950] Updated weights for policy 1, policy_version 78650 (0.0008) -[2023-10-14 16:43:51,135][75949] Updated weights for policy 0, policy_version 78851 (0.0008) -[2023-10-14 16:43:51,529][75949] Updated weights for policy 0, policy_version 78861 (0.0008) -[2023-10-14 16:43:51,899][75949] Updated weights for policy 0, policy_version 78871 (0.0008) -[2023-10-14 16:43:52,588][75950] Updated weights for policy 1, policy_version 78660 (0.0009) -[2023-10-14 16:43:52,956][75950] Updated weights for policy 1, policy_version 78670 (0.0007) -[2023-10-14 16:43:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161316864. Throughput: 0: 1664.5, 1: 1686.2. Samples: 40338394. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 16:43:53,165][74987] Avg episode reward: [(0, '29.280'), (1, '34.440')] -[2023-10-14 16:43:53,324][75950] Updated weights for policy 1, policy_version 78680 (0.0007) -[2023-10-14 16:43:55,885][75949] Updated weights for policy 0, policy_version 78881 (0.0008) -[2023-10-14 16:43:56,255][75949] Updated weights for policy 0, policy_version 78891 (0.0008) -[2023-10-14 16:43:56,617][75949] Updated weights for policy 0, policy_version 78901 (0.0008) -[2023-10-14 16:43:56,990][75949] Updated weights for policy 0, policy_version 78911 (0.0008) -[2023-10-14 16:43:57,495][75950] Updated weights for policy 1, policy_version 78690 (0.0010) -[2023-10-14 16:43:57,863][75950] Updated weights for policy 1, policy_version 78700 (0.0008) -[2023-10-14 16:43:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161382400. Throughput: 0: 1674.2, 1: 1684.2. Samples: 40358388. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 16:43:58,164][74987] Avg episode reward: [(0, '26.050'), (1, '33.330')] -[2023-10-14 16:43:58,233][75950] Updated weights for policy 1, policy_version 78710 (0.0009) -[2023-10-14 16:43:58,586][75950] Updated weights for policy 1, policy_version 78720 (0.0008) -[2023-10-14 16:44:01,014][75949] Updated weights for policy 0, policy_version 78921 (0.0011) -[2023-10-14 16:44:01,381][75949] Updated weights for policy 0, policy_version 78931 (0.0011) -[2023-10-14 16:44:01,751][75949] Updated weights for policy 0, policy_version 78941 (0.0010) -[2023-10-14 16:44:02,658][75950] Updated weights for policy 1, policy_version 78730 (0.0009) -[2023-10-14 16:44:03,023][75950] Updated weights for policy 1, policy_version 78740 (0.0007) -[2023-10-14 16:44:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 161447936. Throughput: 0: 1681.9, 1: 1685.4. Samples: 40369028. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 16:44:03,164][74987] Avg episode reward: [(0, '28.990'), (1, '30.190')] -[2023-10-14 16:44:03,396][75950] Updated weights for policy 1, policy_version 78750 (0.0008) -[2023-10-14 16:44:05,877][75949] Updated weights for policy 0, policy_version 78951 (0.0009) -[2023-10-14 16:44:06,237][75949] Updated weights for policy 0, policy_version 78961 (0.0009) -[2023-10-14 16:44:06,608][75949] Updated weights for policy 0, policy_version 78971 (0.0008) -[2023-10-14 16:44:07,470][75950] Updated weights for policy 1, policy_version 78760 (0.0010) -[2023-10-14 16:44:07,838][75950] Updated weights for policy 1, policy_version 78770 (0.0008) -[2023-10-14 16:44:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 161513472. Throughput: 0: 1667.5, 1: 1684.0. Samples: 40388778. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) -[2023-10-14 16:44:08,165][74987] Avg episode reward: [(0, '28.080'), (1, '32.660')] -[2023-10-14 16:44:08,197][75950] Updated weights for policy 1, policy_version 78780 (0.0008) -[2023-10-14 16:44:10,646][75949] Updated weights for policy 0, policy_version 78981 (0.0009) -[2023-10-14 16:44:11,014][75949] Updated weights for policy 0, policy_version 78991 (0.0009) -[2023-10-14 16:44:11,384][75949] Updated weights for policy 0, policy_version 79001 (0.0008) -[2023-10-14 16:44:12,339][75950] Updated weights for policy 1, policy_version 78790 (0.0007) -[2023-10-14 16:44:12,700][75950] Updated weights for policy 1, policy_version 78800 (0.0008) -[2023-10-14 16:44:13,058][75950] Updated weights for policy 1, policy_version 78810 (0.0008) -[2023-10-14 16:44:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 161579008. Throughput: 0: 1692.1, 1: 1674.1. Samples: 40408900. Policy #0 lag: (min: 29.0, avg: 30.4, max: 54.0) -[2023-10-14 16:44:13,164][74987] Avg episode reward: [(0, '30.030'), (1, '34.310')] -[2023-10-14 16:44:15,199][75949] Updated weights for policy 0, policy_version 79011 (0.0008) -[2023-10-14 16:44:15,567][75949] Updated weights for policy 0, policy_version 79021 (0.0009) -[2023-10-14 16:44:15,945][75949] Updated weights for policy 0, policy_version 79031 (0.0008) -[2023-10-14 16:44:16,921][75950] Updated weights for policy 1, policy_version 78820 (0.0009) -[2023-10-14 16:44:17,280][75950] Updated weights for policy 1, policy_version 78830 (0.0009) -[2023-10-14 16:44:17,645][75950] Updated weights for policy 1, policy_version 78840 (0.0009) -[2023-10-14 16:44:18,164][74987] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 161677312. Throughput: 0: 1686.5, 1: 1685.2. Samples: 40419332. Policy #0 lag: (min: 29.0, avg: 30.4, max: 54.0) -[2023-10-14 16:44:18,164][74987] Avg episode reward: [(0, '27.340'), (1, '31.600')] -[2023-10-14 16:44:19,977][75949] Updated weights for policy 0, policy_version 79041 (0.0008) -[2023-10-14 16:44:20,353][75949] Updated weights for policy 0, policy_version 79051 (0.0008) -[2023-10-14 16:44:20,726][75949] Updated weights for policy 0, policy_version 79061 (0.0007) -[2023-10-14 16:44:21,090][75949] Updated weights for policy 0, policy_version 79071 (0.0007) -[2023-10-14 16:44:21,747][75950] Updated weights for policy 1, policy_version 78850 (0.0009) -[2023-10-14 16:44:22,125][75950] Updated weights for policy 1, policy_version 78860 (0.0009) -[2023-10-14 16:44:22,482][75950] Updated weights for policy 1, policy_version 78870 (0.0009) -[2023-10-14 16:44:22,844][75950] Updated weights for policy 1, policy_version 78880 (0.0012) -[2023-10-14 16:44:23,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 161742848. Throughput: 0: 1681.5, 1: 1684.0. Samples: 40439412. Policy #0 lag: (min: 29.0, avg: 30.4, max: 54.0) -[2023-10-14 16:44:23,164][74987] Avg episode reward: [(0, '29.900'), (1, '34.210')] -[2023-10-14 16:44:25,198][75949] Updated weights for policy 0, policy_version 79081 (0.0008) -[2023-10-14 16:44:25,562][75949] Updated weights for policy 0, policy_version 79091 (0.0007) -[2023-10-14 16:44:25,936][75949] Updated weights for policy 0, policy_version 79101 (0.0008) -[2023-10-14 16:44:27,006][75950] Updated weights for policy 1, policy_version 78890 (0.0007) -[2023-10-14 16:44:27,365][75950] Updated weights for policy 1, policy_version 78900 (0.0009) -[2023-10-14 16:44:27,736][75950] Updated weights for policy 1, policy_version 78910 (0.0010) -[2023-10-14 16:44:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 161808384. Throughput: 0: 1699.8, 1: 1658.7. Samples: 40459098. Policy #0 lag: (min: 29.0, avg: 30.4, max: 54.0) -[2023-10-14 16:44:28,164][74987] Avg episode reward: [(0, '23.980'), (1, '36.360')] -[2023-10-14 16:44:29,748][75949] Updated weights for policy 0, policy_version 79111 (0.0010) -[2023-10-14 16:44:30,125][75949] Updated weights for policy 0, policy_version 79121 (0.0009) -[2023-10-14 16:44:30,499][75949] Updated weights for policy 0, policy_version 79131 (0.0007) -[2023-10-14 16:44:31,951][75950] Updated weights for policy 1, policy_version 78920 (0.0008) -[2023-10-14 16:44:32,320][75950] Updated weights for policy 1, policy_version 78930 (0.0008) -[2023-10-14 16:44:32,683][75950] Updated weights for policy 1, policy_version 78940 (0.0007) -[2023-10-14 16:44:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 161873920. Throughput: 0: 1670.7, 1: 1679.7. Samples: 40469378. Policy #0 lag: (min: 29.0, avg: 30.4, max: 54.0) -[2023-10-14 16:44:33,164][74987] Avg episode reward: [(0, '28.720'), (1, '33.060')] -[2023-10-14 16:44:34,746][75949] Updated weights for policy 0, policy_version 79141 (0.0010) -[2023-10-14 16:44:35,121][75949] Updated weights for policy 0, policy_version 79151 (0.0007) -[2023-10-14 16:44:35,487][75949] Updated weights for policy 0, policy_version 79161 (0.0008) -[2023-10-14 16:44:36,887][75950] Updated weights for policy 1, policy_version 78950 (0.0009) -[2023-10-14 16:44:37,265][75950] Updated weights for policy 1, policy_version 78960 (0.0009) -[2023-10-14 16:44:37,634][75950] Updated weights for policy 1, policy_version 78970 (0.0009) -[2023-10-14 16:44:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 161939456. Throughput: 0: 1685.9, 1: 1678.5. Samples: 40489794. Policy #0 lag: (min: 29.0, avg: 30.4, max: 54.0) -[2023-10-14 16:44:38,164][74987] Avg episode reward: [(0, '25.960'), (1, '33.080')] -[2023-10-14 16:44:39,538][75949] Updated weights for policy 0, policy_version 79171 (0.0008) -[2023-10-14 16:44:39,942][75949] Updated weights for policy 0, policy_version 79181 (0.0009) -[2023-10-14 16:44:40,315][75949] Updated weights for policy 0, policy_version 79191 (0.0009) -[2023-10-14 16:44:41,441][75950] Updated weights for policy 1, policy_version 78980 (0.0009) -[2023-10-14 16:44:41,807][75950] Updated weights for policy 1, policy_version 78990 (0.0008) -[2023-10-14 16:44:42,170][75950] Updated weights for policy 1, policy_version 79000 (0.0008) -[2023-10-14 16:44:43,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 162004992. Throughput: 0: 1701.1, 1: 1656.9. Samples: 40509498. Policy #0 lag: (min: 29.0, avg: 30.4, max: 54.0) -[2023-10-14 16:44:43,165][74987] Avg episode reward: [(0, '28.630'), (1, '34.330')] -[2023-10-14 16:44:43,175][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000079008_80904192.pth... -[2023-10-14 16:44:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000079200_81100800.pth... -[2023-10-14 16:44:43,211][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000077632_79495168.pth -[2023-10-14 16:44:43,216][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000077440_79298560.pth -[2023-10-14 16:44:44,221][75949] Updated weights for policy 0, policy_version 79201 (0.0008) -[2023-10-14 16:44:44,588][75949] Updated weights for policy 0, policy_version 79211 (0.0007) -[2023-10-14 16:44:44,958][75949] Updated weights for policy 0, policy_version 79221 (0.0007) -[2023-10-14 16:44:45,335][75949] Updated weights for policy 0, policy_version 79231 (0.0007) -[2023-10-14 16:44:46,286][75950] Updated weights for policy 1, policy_version 79010 (0.0008) -[2023-10-14 16:44:46,642][75950] Updated weights for policy 1, policy_version 79020 (0.0008) -[2023-10-14 16:44:47,008][75950] Updated weights for policy 1, policy_version 79030 (0.0009) -[2023-10-14 16:44:47,371][75950] Updated weights for policy 1, policy_version 79040 (0.0007) -[2023-10-14 16:44:48,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 162070528. Throughput: 0: 1669.2, 1: 1685.0. Samples: 40519966. Policy #0 lag: (min: 29.0, avg: 30.4, max: 54.0) -[2023-10-14 16:44:48,164][74987] Avg episode reward: [(0, '26.160'), (1, '33.710')] -[2023-10-14 16:44:49,527][75949] Updated weights for policy 0, policy_version 79241 (0.0008) -[2023-10-14 16:44:49,902][75949] Updated weights for policy 0, policy_version 79251 (0.0008) -[2023-10-14 16:44:50,267][75949] Updated weights for policy 0, policy_version 79261 (0.0007) -[2023-10-14 16:44:51,465][75950] Updated weights for policy 1, policy_version 79050 (0.0009) -[2023-10-14 16:44:51,829][75950] Updated weights for policy 1, policy_version 79060 (0.0008) -[2023-10-14 16:44:52,199][75950] Updated weights for policy 1, policy_version 79070 (0.0008) -[2023-10-14 16:44:53,164][74987] Fps is (10 sec: 13107.7, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 162136064. Throughput: 0: 1689.4, 1: 1672.7. Samples: 40540072. Policy #0 lag: (min: 29.0, avg: 30.4, max: 54.0) -[2023-10-14 16:44:53,164][74987] Avg episode reward: [(0, '27.710'), (1, '33.390')] -[2023-10-14 16:44:54,409][75949] Updated weights for policy 0, policy_version 79271 (0.0007) -[2023-10-14 16:44:54,765][75949] Updated weights for policy 0, policy_version 79281 (0.0007) -[2023-10-14 16:44:55,137][75949] Updated weights for policy 0, policy_version 79291 (0.0009) -[2023-10-14 16:44:56,173][75950] Updated weights for policy 1, policy_version 79080 (0.0008) -[2023-10-14 16:44:56,544][75950] Updated weights for policy 1, policy_version 79090 (0.0008) -[2023-10-14 16:44:56,905][75950] Updated weights for policy 1, policy_version 79100 (0.0008) -[2023-10-14 16:44:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 162201600. Throughput: 0: 1693.5, 1: 1669.7. Samples: 40560244. Policy #0 lag: (min: 29.0, avg: 30.4, max: 54.0) -[2023-10-14 16:44:58,164][74987] Avg episode reward: [(0, '27.550'), (1, '33.550')] -[2023-10-14 16:44:59,123][75949] Updated weights for policy 0, policy_version 79301 (0.0009) -[2023-10-14 16:44:59,489][75949] Updated weights for policy 0, policy_version 79311 (0.0008) -[2023-10-14 16:44:59,857][75949] Updated weights for policy 0, policy_version 79321 (0.0010) -[2023-10-14 16:45:01,002][75950] Updated weights for policy 1, policy_version 79110 (0.0008) -[2023-10-14 16:45:01,362][75950] Updated weights for policy 1, policy_version 79120 (0.0010) -[2023-10-14 16:45:01,731][75950] Updated weights for policy 1, policy_version 79130 (0.0010) -[2023-10-14 16:45:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 162267136. Throughput: 0: 1673.4, 1: 1688.3. Samples: 40570606. Policy #0 lag: (min: 29.0, avg: 30.4, max: 54.0) -[2023-10-14 16:45:03,165][74987] Avg episode reward: [(0, '27.820'), (1, '33.810')] -[2023-10-14 16:45:03,982][75949] Updated weights for policy 0, policy_version 79331 (0.0009) -[2023-10-14 16:45:04,357][75949] Updated weights for policy 0, policy_version 79341 (0.0008) -[2023-10-14 16:45:04,724][75949] Updated weights for policy 0, policy_version 79351 (0.0009) -[2023-10-14 16:45:05,672][75950] Updated weights for policy 1, policy_version 79140 (0.0010) -[2023-10-14 16:45:06,030][75950] Updated weights for policy 1, policy_version 79150 (0.0010) -[2023-10-14 16:45:06,394][75950] Updated weights for policy 1, policy_version 79160 (0.0009) -[2023-10-14 16:45:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 162332672. Throughput: 0: 1688.7, 1: 1665.1. Samples: 40590332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:45:08,164][74987] Avg episode reward: [(0, '28.890'), (1, '32.610')] -[2023-10-14 16:45:08,759][75949] Updated weights for policy 0, policy_version 79361 (0.0008) -[2023-10-14 16:45:09,132][75949] Updated weights for policy 0, policy_version 79371 (0.0007) -[2023-10-14 16:45:09,493][75949] Updated weights for policy 0, policy_version 79381 (0.0010) -[2023-10-14 16:45:09,864][75949] Updated weights for policy 0, policy_version 79391 (0.0007) -[2023-10-14 16:45:10,499][75950] Updated weights for policy 1, policy_version 79170 (0.0010) -[2023-10-14 16:45:10,873][75950] Updated weights for policy 1, policy_version 79180 (0.0012) -[2023-10-14 16:45:11,249][75950] Updated weights for policy 1, policy_version 79190 (0.0010) -[2023-10-14 16:45:11,613][75950] Updated weights for policy 1, policy_version 79200 (0.0010) -[2023-10-14 16:45:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 162398208. Throughput: 0: 1690.8, 1: 1682.9. Samples: 40610916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:45:13,165][74987] Avg episode reward: [(0, '25.400'), (1, '31.620')] -[2023-10-14 16:45:13,730][75949] Updated weights for policy 0, policy_version 79401 (0.0010) -[2023-10-14 16:45:14,104][75949] Updated weights for policy 0, policy_version 79411 (0.0009) -[2023-10-14 16:45:14,465][75949] Updated weights for policy 0, policy_version 79421 (0.0007) -[2023-10-14 16:45:15,936][75950] Updated weights for policy 1, policy_version 79210 (0.0009) -[2023-10-14 16:45:16,318][75950] Updated weights for policy 1, policy_version 79220 (0.0009) -[2023-10-14 16:45:16,669][75950] Updated weights for policy 1, policy_version 79230 (0.0009) -[2023-10-14 16:45:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 162463744. Throughput: 0: 1688.2, 1: 1682.5. Samples: 40621062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:45:18,165][74987] Avg episode reward: [(0, '29.230'), (1, '33.970')] -[2023-10-14 16:45:18,484][75949] Updated weights for policy 0, policy_version 79431 (0.0007) -[2023-10-14 16:45:18,850][75949] Updated weights for policy 0, policy_version 79441 (0.0007) -[2023-10-14 16:45:19,217][75949] Updated weights for policy 0, policy_version 79451 (0.0007) -[2023-10-14 16:45:20,697][75950] Updated weights for policy 1, policy_version 79240 (0.0008) -[2023-10-14 16:45:21,060][75950] Updated weights for policy 1, policy_version 79250 (0.0008) -[2023-10-14 16:45:21,429][75950] Updated weights for policy 1, policy_version 79260 (0.0008) -[2023-10-14 16:45:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 162529280. Throughput: 0: 1695.2, 1: 1660.6. Samples: 40640802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:45:23,164][74987] Avg episode reward: [(0, '24.470'), (1, '33.090')] -[2023-10-14 16:45:23,188][75949] Updated weights for policy 0, policy_version 79461 (0.0008) -[2023-10-14 16:45:23,571][75949] Updated weights for policy 0, policy_version 79471 (0.0007) -[2023-10-14 16:45:23,939][75949] Updated weights for policy 0, policy_version 79481 (0.0010) -[2023-10-14 16:45:25,645][75950] Updated weights for policy 1, policy_version 79270 (0.0009) -[2023-10-14 16:45:26,023][75950] Updated weights for policy 1, policy_version 79280 (0.0008) -[2023-10-14 16:45:26,390][75950] Updated weights for policy 1, policy_version 79290 (0.0008) -[2023-10-14 16:45:28,101][75949] Updated weights for policy 0, policy_version 79491 (0.0010) -[2023-10-14 16:45:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 162594816. Throughput: 0: 1691.4, 1: 1680.2. Samples: 40661220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:45:28,165][74987] Avg episode reward: [(0, '30.960'), (1, '32.830')] -[2023-10-14 16:45:28,498][75949] Updated weights for policy 0, policy_version 79501 (0.0009) -[2023-10-14 16:45:28,872][75949] Updated weights for policy 0, policy_version 79511 (0.0009) -[2023-10-14 16:45:30,606][75950] Updated weights for policy 1, policy_version 79300 (0.0009) -[2023-10-14 16:45:30,962][75950] Updated weights for policy 1, policy_version 79310 (0.0008) -[2023-10-14 16:45:31,322][75950] Updated weights for policy 1, policy_version 79320 (0.0008) -[2023-10-14 16:45:32,844][75949] Updated weights for policy 0, policy_version 79521 (0.0010) -[2023-10-14 16:45:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 162660352. Throughput: 0: 1690.9, 1: 1673.3. Samples: 40671356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:45:33,164][74987] Avg episode reward: [(0, '25.390'), (1, '34.160')] -[2023-10-14 16:45:33,210][75949] Updated weights for policy 0, policy_version 79531 (0.0008) -[2023-10-14 16:45:33,588][75949] Updated weights for policy 0, policy_version 79541 (0.0008) -[2023-10-14 16:45:33,957][75949] Updated weights for policy 0, policy_version 79551 (0.0011) -[2023-10-14 16:45:35,422][75950] Updated weights for policy 1, policy_version 79330 (0.0007) -[2023-10-14 16:45:35,791][75950] Updated weights for policy 1, policy_version 79340 (0.0007) -[2023-10-14 16:45:36,153][75950] Updated weights for policy 1, policy_version 79350 (0.0008) -[2023-10-14 16:45:36,517][75950] Updated weights for policy 1, policy_version 79360 (0.0008) -[2023-10-14 16:45:38,032][75949] Updated weights for policy 0, policy_version 79561 (0.0008) -[2023-10-14 16:45:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 162725888. Throughput: 0: 1699.2, 1: 1659.5. Samples: 40691214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:45:38,164][74987] Avg episode reward: [(0, '31.850'), (1, '34.120')] -[2023-10-14 16:45:38,405][75949] Updated weights for policy 0, policy_version 79571 (0.0007) -[2023-10-14 16:45:38,772][75949] Updated weights for policy 0, policy_version 79581 (0.0009) -[2023-10-14 16:45:38,882][75615] Saving new best policy, reward=31.850! -[2023-10-14 16:45:40,534][75950] Updated weights for policy 1, policy_version 79370 (0.0007) -[2023-10-14 16:45:40,892][75950] Updated weights for policy 1, policy_version 79380 (0.0007) -[2023-10-14 16:45:41,257][75950] Updated weights for policy 1, policy_version 79390 (0.0010) -[2023-10-14 16:45:42,908][75949] Updated weights for policy 0, policy_version 79591 (0.0008) -[2023-10-14 16:45:43,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 162791424. Throughput: 0: 1697.2, 1: 1677.2. Samples: 40712092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:45:43,164][74987] Avg episode reward: [(0, '26.390'), (1, '33.370')] -[2023-10-14 16:45:43,278][75949] Updated weights for policy 0, policy_version 79601 (0.0009) -[2023-10-14 16:45:43,652][75949] Updated weights for policy 0, policy_version 79611 (0.0008) -[2023-10-14 16:45:45,315][75950] Updated weights for policy 1, policy_version 79400 (0.0010) -[2023-10-14 16:45:45,673][75950] Updated weights for policy 1, policy_version 79410 (0.0008) -[2023-10-14 16:45:46,049][75950] Updated weights for policy 1, policy_version 79420 (0.0011) -[2023-10-14 16:45:47,714][75949] Updated weights for policy 0, policy_version 79621 (0.0009) -[2023-10-14 16:45:48,077][75949] Updated weights for policy 0, policy_version 79631 (0.0009) -[2023-10-14 16:45:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 162856960. Throughput: 0: 1698.0, 1: 1663.2. Samples: 40721860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:45:48,165][74987] Avg episode reward: [(0, '30.730'), (1, '32.940')] -[2023-10-14 16:45:48,446][75949] Updated weights for policy 0, policy_version 79641 (0.0011) -[2023-10-14 16:45:50,238][75950] Updated weights for policy 1, policy_version 79430 (0.0008) -[2023-10-14 16:45:50,603][75950] Updated weights for policy 1, policy_version 79440 (0.0007) -[2023-10-14 16:45:50,974][75950] Updated weights for policy 1, policy_version 79450 (0.0008) -[2023-10-14 16:45:52,466][75949] Updated weights for policy 0, policy_version 79651 (0.0009) -[2023-10-14 16:45:52,837][75949] Updated weights for policy 0, policy_version 79661 (0.0009) -[2023-10-14 16:45:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162922496. Throughput: 0: 1695.7, 1: 1666.9. Samples: 40741652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:45:53,164][74987] Avg episode reward: [(0, '26.470'), (1, '33.640')] -[2023-10-14 16:45:53,206][75949] Updated weights for policy 0, policy_version 79671 (0.0009) -[2023-10-14 16:45:55,174][75950] Updated weights for policy 1, policy_version 79460 (0.0008) -[2023-10-14 16:45:55,549][75950] Updated weights for policy 1, policy_version 79470 (0.0007) -[2023-10-14 16:45:55,911][75950] Updated weights for policy 1, policy_version 79480 (0.0009) -[2023-10-14 16:45:57,155][75949] Updated weights for policy 0, policy_version 79681 (0.0009) -[2023-10-14 16:45:57,524][75949] Updated weights for policy 0, policy_version 79691 (0.0009) -[2023-10-14 16:45:57,890][75949] Updated weights for policy 0, policy_version 79701 (0.0009) -[2023-10-14 16:45:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162988032. Throughput: 0: 1680.4, 1: 1671.3. Samples: 40761744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:45:58,164][74987] Avg episode reward: [(0, '30.060'), (1, '35.450')] -[2023-10-14 16:45:58,268][75949] Updated weights for policy 0, policy_version 79711 (0.0009) -[2023-10-14 16:45:59,955][75950] Updated weights for policy 1, policy_version 79490 (0.0008) -[2023-10-14 16:46:00,315][75950] Updated weights for policy 1, policy_version 79500 (0.0007) -[2023-10-14 16:46:00,685][75950] Updated weights for policy 1, policy_version 79510 (0.0007) -[2023-10-14 16:46:01,057][75950] Updated weights for policy 1, policy_version 79520 (0.0008) -[2023-10-14 16:46:02,324][75949] Updated weights for policy 0, policy_version 79721 (0.0009) -[2023-10-14 16:46:02,683][75949] Updated weights for policy 0, policy_version 79731 (0.0010) -[2023-10-14 16:46:03,059][75949] Updated weights for policy 0, policy_version 79741 (0.0011) -[2023-10-14 16:46:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163053568. Throughput: 0: 1692.6, 1: 1660.6. Samples: 40771956. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:46:03,164][74987] Avg episode reward: [(0, '26.190'), (1, '34.130')] -[2023-10-14 16:46:04,994][75950] Updated weights for policy 1, policy_version 79530 (0.0007) -[2023-10-14 16:46:05,368][75950] Updated weights for policy 1, policy_version 79540 (0.0008) -[2023-10-14 16:46:05,728][75950] Updated weights for policy 1, policy_version 79550 (0.0009) -[2023-10-14 16:46:07,040][75949] Updated weights for policy 0, policy_version 79751 (0.0009) -[2023-10-14 16:46:07,400][75949] Updated weights for policy 0, policy_version 79761 (0.0009) -[2023-10-14 16:46:07,771][75949] Updated weights for policy 0, policy_version 79771 (0.0008) -[2023-10-14 16:46:08,163][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 163151872. Throughput: 0: 1690.3, 1: 1673.3. Samples: 40792164. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:46:08,164][74987] Avg episode reward: [(0, '28.770'), (1, '34.000')] -[2023-10-14 16:46:09,898][75950] Updated weights for policy 1, policy_version 79560 (0.0008) -[2023-10-14 16:46:10,273][75950] Updated weights for policy 1, policy_version 79570 (0.0008) -[2023-10-14 16:46:10,642][75950] Updated weights for policy 1, policy_version 79580 (0.0008) -[2023-10-14 16:46:11,749][75949] Updated weights for policy 0, policy_version 79781 (0.0008) -[2023-10-14 16:46:12,108][75949] Updated weights for policy 0, policy_version 79791 (0.0009) -[2023-10-14 16:46:12,484][75949] Updated weights for policy 0, policy_version 79801 (0.0008) -[2023-10-14 16:46:13,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 163217408. Throughput: 0: 1668.8, 1: 1677.3. Samples: 40811794. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:46:13,164][74987] Avg episode reward: [(0, '27.650'), (1, '34.840')] -[2023-10-14 16:46:14,679][75950] Updated weights for policy 1, policy_version 79590 (0.0008) -[2023-10-14 16:46:15,039][75950] Updated weights for policy 1, policy_version 79600 (0.0007) -[2023-10-14 16:46:15,403][75950] Updated weights for policy 1, policy_version 79610 (0.0007) -[2023-10-14 16:46:16,808][75949] Updated weights for policy 0, policy_version 79811 (0.0010) -[2023-10-14 16:46:17,196][75949] Updated weights for policy 0, policy_version 79821 (0.0011) -[2023-10-14 16:46:17,563][75949] Updated weights for policy 0, policy_version 79831 (0.0010) -[2023-10-14 16:46:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 163282944. Throughput: 0: 1692.0, 1: 1656.0. Samples: 40822014. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:46:18,165][74987] Avg episode reward: [(0, '28.310'), (1, '35.340')] -[2023-10-14 16:46:19,519][75950] Updated weights for policy 1, policy_version 79620 (0.0008) -[2023-10-14 16:46:19,891][75950] Updated weights for policy 1, policy_version 79630 (0.0008) -[2023-10-14 16:46:20,250][75950] Updated weights for policy 1, policy_version 79640 (0.0008) -[2023-10-14 16:46:21,581][75949] Updated weights for policy 0, policy_version 79841 (0.0010) -[2023-10-14 16:46:21,938][75949] Updated weights for policy 0, policy_version 79851 (0.0009) -[2023-10-14 16:46:22,314][75949] Updated weights for policy 0, policy_version 79861 (0.0010) -[2023-10-14 16:46:22,682][75949] Updated weights for policy 0, policy_version 79871 (0.0008) -[2023-10-14 16:46:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 163348480. Throughput: 0: 1679.6, 1: 1675.2. Samples: 40842180. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:46:23,164][74987] Avg episode reward: [(0, '29.210'), (1, '32.320')] -[2023-10-14 16:46:24,308][75950] Updated weights for policy 1, policy_version 79650 (0.0008) -[2023-10-14 16:46:24,670][75950] Updated weights for policy 1, policy_version 79660 (0.0007) -[2023-10-14 16:46:25,035][75950] Updated weights for policy 1, policy_version 79670 (0.0010) -[2023-10-14 16:46:25,408][75950] Updated weights for policy 1, policy_version 79680 (0.0008) -[2023-10-14 16:46:26,700][75949] Updated weights for policy 0, policy_version 79881 (0.0008) -[2023-10-14 16:46:27,062][75949] Updated weights for policy 0, policy_version 79891 (0.0010) -[2023-10-14 16:46:27,435][75949] Updated weights for policy 0, policy_version 79901 (0.0009) -[2023-10-14 16:46:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 163414016. Throughput: 0: 1651.3, 1: 1675.9. Samples: 40861816. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:46:28,165][74987] Avg episode reward: [(0, '28.020'), (1, '32.450')] -[2023-10-14 16:46:29,434][75950] Updated weights for policy 1, policy_version 79690 (0.0007) -[2023-10-14 16:46:29,797][75950] Updated weights for policy 1, policy_version 79700 (0.0008) -[2023-10-14 16:46:30,162][75950] Updated weights for policy 1, policy_version 79710 (0.0009) -[2023-10-14 16:46:31,486][75949] Updated weights for policy 0, policy_version 79911 (0.0010) -[2023-10-14 16:46:31,853][75949] Updated weights for policy 0, policy_version 79921 (0.0008) -[2023-10-14 16:46:32,215][75949] Updated weights for policy 0, policy_version 79931 (0.0008) -[2023-10-14 16:46:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 163479552. Throughput: 0: 1682.6, 1: 1658.5. Samples: 40872210. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:46:33,165][74987] Avg episode reward: [(0, '29.780'), (1, '33.500')] -[2023-10-14 16:46:34,423][75950] Updated weights for policy 1, policy_version 79720 (0.0010) -[2023-10-14 16:46:34,794][75950] Updated weights for policy 1, policy_version 79730 (0.0008) -[2023-10-14 16:46:35,152][75950] Updated weights for policy 1, policy_version 79740 (0.0007) -[2023-10-14 16:46:36,385][75949] Updated weights for policy 0, policy_version 79941 (0.0009) -[2023-10-14 16:46:36,756][75949] Updated weights for policy 0, policy_version 79951 (0.0009) -[2023-10-14 16:46:37,124][75949] Updated weights for policy 0, policy_version 79961 (0.0009) -[2023-10-14 16:46:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 163545088. Throughput: 0: 1671.0, 1: 1679.2. Samples: 40892410. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:46:38,165][74987] Avg episode reward: [(0, '27.000'), (1, '31.960')] -[2023-10-14 16:46:39,252][75950] Updated weights for policy 1, policy_version 79750 (0.0010) -[2023-10-14 16:46:39,619][75950] Updated weights for policy 1, policy_version 79760 (0.0008) -[2023-10-14 16:46:39,985][75950] Updated weights for policy 1, policy_version 79770 (0.0008) -[2023-10-14 16:46:41,042][75949] Updated weights for policy 0, policy_version 79971 (0.0010) -[2023-10-14 16:46:41,423][75949] Updated weights for policy 0, policy_version 79981 (0.0009) -[2023-10-14 16:46:41,781][75949] Updated weights for policy 0, policy_version 79991 (0.0009) -[2023-10-14 16:46:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 163610624. Throughput: 0: 1671.9, 1: 1681.4. Samples: 40912642. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:46:43,164][74987] Avg episode reward: [(0, '29.800'), (1, '33.860')] -[2023-10-14 16:46:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000079776_81690624.pth... -[2023-10-14 16:46:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000080000_81920000.pth... -[2023-10-14 16:46:43,213][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000078208_80084992.pth -[2023-10-14 16:46:43,214][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000078432_80314368.pth -[2023-10-14 16:46:43,981][75950] Updated weights for policy 1, policy_version 79780 (0.0008) -[2023-10-14 16:46:44,350][75950] Updated weights for policy 1, policy_version 79790 (0.0007) -[2023-10-14 16:46:44,712][75950] Updated weights for policy 1, policy_version 79800 (0.0008) -[2023-10-14 16:46:45,874][75949] Updated weights for policy 0, policy_version 80001 (0.0008) -[2023-10-14 16:46:46,252][75949] Updated weights for policy 0, policy_version 80011 (0.0010) -[2023-10-14 16:46:46,620][75949] Updated weights for policy 0, policy_version 80021 (0.0009) -[2023-10-14 16:46:46,998][75949] Updated weights for policy 0, policy_version 80031 (0.0010) -[2023-10-14 16:46:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 163676160. Throughput: 0: 1688.8, 1: 1667.3. Samples: 40922978. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:46:48,165][74987] Avg episode reward: [(0, '26.250'), (1, '34.830')] -[2023-10-14 16:46:48,860][75950] Updated weights for policy 1, policy_version 79810 (0.0008) -[2023-10-14 16:46:49,232][75950] Updated weights for policy 1, policy_version 79820 (0.0010) -[2023-10-14 16:46:49,601][75950] Updated weights for policy 1, policy_version 79830 (0.0007) -[2023-10-14 16:46:49,961][75950] Updated weights for policy 1, policy_version 79840 (0.0009) -[2023-10-14 16:46:51,148][75949] Updated weights for policy 0, policy_version 80041 (0.0009) -[2023-10-14 16:46:51,517][75949] Updated weights for policy 0, policy_version 80051 (0.0008) -[2023-10-14 16:46:51,886][75949] Updated weights for policy 0, policy_version 80061 (0.0008) -[2023-10-14 16:46:53,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 163741696. Throughput: 0: 1668.4, 1: 1676.7. Samples: 40942694. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 16:46:53,164][74987] Avg episode reward: [(0, '28.460'), (1, '35.760')] -[2023-10-14 16:46:53,936][75950] Updated weights for policy 1, policy_version 79850 (0.0010) -[2023-10-14 16:46:54,298][75950] Updated weights for policy 1, policy_version 79860 (0.0010) -[2023-10-14 16:46:54,668][75950] Updated weights for policy 1, policy_version 79870 (0.0007) -[2023-10-14 16:46:55,917][75949] Updated weights for policy 0, policy_version 80071 (0.0010) -[2023-10-14 16:46:56,289][75949] Updated weights for policy 0, policy_version 80081 (0.0011) -[2023-10-14 16:46:56,659][75949] Updated weights for policy 0, policy_version 80091 (0.0010) -[2023-10-14 16:46:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 163807232. Throughput: 0: 1681.6, 1: 1683.4. Samples: 40963218. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 16:46:58,164][74987] Avg episode reward: [(0, '25.330'), (1, '32.180')] -[2023-10-14 16:46:58,934][75950] Updated weights for policy 1, policy_version 79880 (0.0008) -[2023-10-14 16:46:59,316][75950] Updated weights for policy 1, policy_version 79890 (0.0009) -[2023-10-14 16:46:59,685][75950] Updated weights for policy 1, policy_version 79900 (0.0010) -[2023-10-14 16:47:00,791][75949] Updated weights for policy 0, policy_version 80101 (0.0009) -[2023-10-14 16:47:01,157][75949] Updated weights for policy 0, policy_version 80111 (0.0007) -[2023-10-14 16:47:01,521][75949] Updated weights for policy 0, policy_version 80121 (0.0008) -[2023-10-14 16:47:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 163872768. Throughput: 0: 1688.2, 1: 1678.4. Samples: 40973510. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 16:47:03,165][74987] Avg episode reward: [(0, '29.850'), (1, '34.530')] -[2023-10-14 16:47:03,386][75950] Updated weights for policy 1, policy_version 79910 (0.0008) -[2023-10-14 16:47:03,751][75950] Updated weights for policy 1, policy_version 79920 (0.0008) -[2023-10-14 16:47:04,130][75950] Updated weights for policy 1, policy_version 79930 (0.0008) -[2023-10-14 16:47:05,596][75949] Updated weights for policy 0, policy_version 80131 (0.0009) -[2023-10-14 16:47:06,016][75949] Updated weights for policy 0, policy_version 80141 (0.0007) -[2023-10-14 16:47:06,386][75949] Updated weights for policy 0, policy_version 80151 (0.0007) -[2023-10-14 16:47:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 163938304. Throughput: 0: 1670.6, 1: 1689.7. Samples: 40993394. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 16:47:08,164][74987] Avg episode reward: [(0, '27.330'), (1, '34.370')] -[2023-10-14 16:47:08,171][75950] Updated weights for policy 1, policy_version 79940 (0.0008) -[2023-10-14 16:47:08,535][75950] Updated weights for policy 1, policy_version 79950 (0.0008) -[2023-10-14 16:47:08,905][75950] Updated weights for policy 1, policy_version 79960 (0.0010) -[2023-10-14 16:47:10,358][75949] Updated weights for policy 0, policy_version 80161 (0.0010) -[2023-10-14 16:47:10,723][75949] Updated weights for policy 0, policy_version 80171 (0.0008) -[2023-10-14 16:47:11,091][75949] Updated weights for policy 0, policy_version 80181 (0.0009) -[2023-10-14 16:47:11,461][75949] Updated weights for policy 0, policy_version 80191 (0.0008) -[2023-10-14 16:47:12,999][75950] Updated weights for policy 1, policy_version 79970 (0.0007) -[2023-10-14 16:47:13,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 164003840. Throughput: 0: 1698.0, 1: 1685.0. Samples: 41014052. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 16:47:13,164][74987] Avg episode reward: [(0, '30.600'), (1, '32.300')] -[2023-10-14 16:47:13,359][75950] Updated weights for policy 1, policy_version 79980 (0.0009) -[2023-10-14 16:47:13,719][75950] Updated weights for policy 1, policy_version 79990 (0.0009) -[2023-10-14 16:47:14,089][75950] Updated weights for policy 1, policy_version 80000 (0.0009) -[2023-10-14 16:47:15,352][75949] Updated weights for policy 0, policy_version 80201 (0.0007) -[2023-10-14 16:47:15,714][75949] Updated weights for policy 0, policy_version 80211 (0.0008) -[2023-10-14 16:47:16,095][75949] Updated weights for policy 0, policy_version 80221 (0.0010) -[2023-10-14 16:47:18,085][75950] Updated weights for policy 1, policy_version 80010 (0.0009) -[2023-10-14 16:47:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 164069376. Throughput: 0: 1685.3, 1: 1688.3. Samples: 41024024. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 16:47:18,165][74987] Avg episode reward: [(0, '27.420'), (1, '34.410')] -[2023-10-14 16:47:18,464][75950] Updated weights for policy 1, policy_version 80020 (0.0008) -[2023-10-14 16:47:18,831][75950] Updated weights for policy 1, policy_version 80030 (0.0009) -[2023-10-14 16:47:20,108][75949] Updated weights for policy 0, policy_version 80231 (0.0010) -[2023-10-14 16:47:20,482][75949] Updated weights for policy 0, policy_version 80241 (0.0008) -[2023-10-14 16:47:20,854][75949] Updated weights for policy 0, policy_version 80251 (0.0008) -[2023-10-14 16:47:22,809][75950] Updated weights for policy 1, policy_version 80040 (0.0008) -[2023-10-14 16:47:23,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 164134912. Throughput: 0: 1684.1, 1: 1690.6. Samples: 41044270. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 16:47:23,164][74987] Avg episode reward: [(0, '31.070'), (1, '33.720')] -[2023-10-14 16:47:23,180][75950] Updated weights for policy 1, policy_version 80050 (0.0009) -[2023-10-14 16:47:23,536][75950] Updated weights for policy 1, policy_version 80060 (0.0012) -[2023-10-14 16:47:24,983][75949] Updated weights for policy 0, policy_version 80261 (0.0008) -[2023-10-14 16:47:25,344][75949] Updated weights for policy 0, policy_version 80271 (0.0009) -[2023-10-14 16:47:25,725][75949] Updated weights for policy 0, policy_version 80281 (0.0008) -[2023-10-14 16:47:27,696][75950] Updated weights for policy 1, policy_version 80070 (0.0009) -[2023-10-14 16:47:28,067][75950] Updated weights for policy 1, policy_version 80080 (0.0009) -[2023-10-14 16:47:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 164200448. Throughput: 0: 1691.3, 1: 1682.9. Samples: 41064482. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 16:47:28,164][74987] Avg episode reward: [(0, '27.830'), (1, '32.060')] -[2023-10-14 16:47:28,433][75950] Updated weights for policy 1, policy_version 80090 (0.0008) -[2023-10-14 16:47:29,906][75949] Updated weights for policy 0, policy_version 80291 (0.0010) -[2023-10-14 16:47:30,271][75949] Updated weights for policy 0, policy_version 80301 (0.0011) -[2023-10-14 16:47:30,639][75949] Updated weights for policy 0, policy_version 80311 (0.0010) -[2023-10-14 16:47:32,417][75950] Updated weights for policy 1, policy_version 80100 (0.0007) -[2023-10-14 16:47:32,786][75950] Updated weights for policy 1, policy_version 80110 (0.0007) -[2023-10-14 16:47:33,149][75950] Updated weights for policy 1, policy_version 80120 (0.0007) -[2023-10-14 16:47:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 164265984. Throughput: 0: 1665.9, 1: 1688.9. Samples: 41073942. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 16:47:33,165][74987] Avg episode reward: [(0, '30.010'), (1, '32.670')] -[2023-10-14 16:47:34,707][75949] Updated weights for policy 0, policy_version 80321 (0.0008) -[2023-10-14 16:47:35,068][75949] Updated weights for policy 0, policy_version 80331 (0.0007) -[2023-10-14 16:47:35,432][75949] Updated weights for policy 0, policy_version 80341 (0.0007) -[2023-10-14 16:47:35,812][75949] Updated weights for policy 0, policy_version 80351 (0.0007) -[2023-10-14 16:47:37,367][75950] Updated weights for policy 1, policy_version 80130 (0.0007) -[2023-10-14 16:47:37,741][75950] Updated weights for policy 1, policy_version 80140 (0.0009) -[2023-10-14 16:47:38,104][75950] Updated weights for policy 1, policy_version 80150 (0.0007) -[2023-10-14 16:47:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 164331520. Throughput: 0: 1676.8, 1: 1693.6. Samples: 41094366. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 16:47:38,164][74987] Avg episode reward: [(0, '27.980'), (1, '35.090')] -[2023-10-14 16:47:38,477][75950] Updated weights for policy 1, policy_version 80160 (0.0008) -[2023-10-14 16:47:39,831][75949] Updated weights for policy 0, policy_version 80361 (0.0009) -[2023-10-14 16:47:40,199][75949] Updated weights for policy 0, policy_version 80371 (0.0008) -[2023-10-14 16:47:40,569][75949] Updated weights for policy 0, policy_version 80381 (0.0010) -[2023-10-14 16:47:42,520][75950] Updated weights for policy 1, policy_version 80170 (0.0007) -[2023-10-14 16:47:42,880][75950] Updated weights for policy 1, policy_version 80180 (0.0007) -[2023-10-14 16:47:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 164397056. Throughput: 0: 1687.8, 1: 1679.7. Samples: 41114758. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) -[2023-10-14 16:47:43,164][74987] Avg episode reward: [(0, '29.510'), (1, '33.990')] -[2023-10-14 16:47:43,248][75950] Updated weights for policy 1, policy_version 80190 (0.0008) -[2023-10-14 16:47:44,699][75949] Updated weights for policy 0, policy_version 80391 (0.0008) -[2023-10-14 16:47:45,075][75949] Updated weights for policy 0, policy_version 80401 (0.0008) -[2023-10-14 16:47:45,450][75949] Updated weights for policy 0, policy_version 80411 (0.0008) -[2023-10-14 16:47:47,295][75950] Updated weights for policy 1, policy_version 80200 (0.0010) -[2023-10-14 16:47:47,654][75950] Updated weights for policy 1, policy_version 80210 (0.0010) -[2023-10-14 16:47:48,019][75950] Updated weights for policy 1, policy_version 80220 (0.0009) -[2023-10-14 16:47:48,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 164462592. Throughput: 0: 1656.9, 1: 1691.2. Samples: 41124176. Policy #0 lag: (min: 0.0, avg: 28.7, max: 32.0) -[2023-10-14 16:47:48,164][74987] Avg episode reward: [(0, '26.900'), (1, '33.980')] -[2023-10-14 16:47:49,362][75949] Updated weights for policy 0, policy_version 80421 (0.0010) -[2023-10-14 16:47:49,730][75949] Updated weights for policy 0, policy_version 80431 (0.0008) -[2023-10-14 16:47:50,109][75949] Updated weights for policy 0, policy_version 80441 (0.0009) -[2023-10-14 16:47:52,123][75950] Updated weights for policy 1, policy_version 80230 (0.0007) -[2023-10-14 16:47:52,481][75950] Updated weights for policy 1, policy_version 80240 (0.0008) -[2023-10-14 16:47:52,853][75950] Updated weights for policy 1, policy_version 80250 (0.0008) -[2023-10-14 16:47:53,163][74987] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 164560896. Throughput: 0: 1681.2, 1: 1684.6. Samples: 41144852. Policy #0 lag: (min: 0.0, avg: 28.7, max: 32.0) -[2023-10-14 16:47:53,164][74987] Avg episode reward: [(0, '28.040'), (1, '34.910')] -[2023-10-14 16:47:54,371][75949] Updated weights for policy 0, policy_version 80451 (0.0010) -[2023-10-14 16:47:54,764][75949] Updated weights for policy 0, policy_version 80461 (0.0009) -[2023-10-14 16:47:55,139][75949] Updated weights for policy 0, policy_version 80471 (0.0008) -[2023-10-14 16:47:56,955][75950] Updated weights for policy 1, policy_version 80260 (0.0008) -[2023-10-14 16:47:57,325][75950] Updated weights for policy 1, policy_version 80270 (0.0007) -[2023-10-14 16:47:57,688][75950] Updated weights for policy 1, policy_version 80280 (0.0007) -[2023-10-14 16:47:58,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 164626432. Throughput: 0: 1674.4, 1: 1668.9. Samples: 41164502. Policy #0 lag: (min: 0.0, avg: 28.7, max: 32.0) -[2023-10-14 16:47:58,164][74987] Avg episode reward: [(0, '27.230'), (1, '32.750')] -[2023-10-14 16:47:59,202][75949] Updated weights for policy 0, policy_version 80481 (0.0009) -[2023-10-14 16:47:59,570][75949] Updated weights for policy 0, policy_version 80491 (0.0010) -[2023-10-14 16:47:59,945][75949] Updated weights for policy 0, policy_version 80501 (0.0009) -[2023-10-14 16:48:00,315][75949] Updated weights for policy 0, policy_version 80511 (0.0008) -[2023-10-14 16:48:01,587][75950] Updated weights for policy 1, policy_version 80290 (0.0009) -[2023-10-14 16:48:01,951][75950] Updated weights for policy 1, policy_version 80300 (0.0009) -[2023-10-14 16:48:02,312][75950] Updated weights for policy 1, policy_version 80310 (0.0008) -[2023-10-14 16:48:02,679][75950] Updated weights for policy 1, policy_version 80320 (0.0008) -[2023-10-14 16:48:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 164691968. Throughput: 0: 1656.3, 1: 1691.3. Samples: 41174668. Policy #0 lag: (min: 0.0, avg: 28.7, max: 32.0) -[2023-10-14 16:48:03,164][74987] Avg episode reward: [(0, '26.980'), (1, '34.470')] -[2023-10-14 16:48:04,264][75949] Updated weights for policy 0, policy_version 80521 (0.0008) -[2023-10-14 16:48:04,626][75949] Updated weights for policy 0, policy_version 80531 (0.0010) -[2023-10-14 16:48:04,991][75949] Updated weights for policy 0, policy_version 80541 (0.0011) -[2023-10-14 16:48:06,901][75950] Updated weights for policy 1, policy_version 80330 (0.0011) -[2023-10-14 16:48:07,260][75950] Updated weights for policy 1, policy_version 80340 (0.0009) -[2023-10-14 16:48:07,628][75950] Updated weights for policy 1, policy_version 80350 (0.0010) -[2023-10-14 16:48:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 164757504. Throughput: 0: 1678.4, 1: 1679.6. Samples: 41195378. Policy #0 lag: (min: 0.0, avg: 28.7, max: 32.0) -[2023-10-14 16:48:08,164][74987] Avg episode reward: [(0, '27.580'), (1, '34.630')] -[2023-10-14 16:48:08,932][75949] Updated weights for policy 0, policy_version 80551 (0.0009) -[2023-10-14 16:48:09,303][75949] Updated weights for policy 0, policy_version 80561 (0.0007) -[2023-10-14 16:48:09,671][75949] Updated weights for policy 0, policy_version 80571 (0.0009) -[2023-10-14 16:48:11,805][75950] Updated weights for policy 1, policy_version 80360 (0.0010) -[2023-10-14 16:48:12,176][75950] Updated weights for policy 1, policy_version 80370 (0.0009) -[2023-10-14 16:48:12,543][75950] Updated weights for policy 1, policy_version 80380 (0.0007) -[2023-10-14 16:48:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 164823040. Throughput: 0: 1684.9, 1: 1663.8. Samples: 41215176. Policy #0 lag: (min: 0.0, avg: 28.7, max: 32.0) -[2023-10-14 16:48:13,164][74987] Avg episode reward: [(0, '26.570'), (1, '33.910')] -[2023-10-14 16:48:13,750][75949] Updated weights for policy 0, policy_version 80581 (0.0010) -[2023-10-14 16:48:14,119][75949] Updated weights for policy 0, policy_version 80591 (0.0007) -[2023-10-14 16:48:14,490][75949] Updated weights for policy 0, policy_version 80601 (0.0007) -[2023-10-14 16:48:16,691][75950] Updated weights for policy 1, policy_version 80390 (0.0008) -[2023-10-14 16:48:17,062][75950] Updated weights for policy 1, policy_version 80400 (0.0007) -[2023-10-14 16:48:17,428][75950] Updated weights for policy 1, policy_version 80410 (0.0007) -[2023-10-14 16:48:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 164888576. Throughput: 0: 1680.0, 1: 1686.4. Samples: 41225430. Policy #0 lag: (min: 0.0, avg: 28.7, max: 32.0) -[2023-10-14 16:48:18,165][74987] Avg episode reward: [(0, '28.880'), (1, '35.270')] -[2023-10-14 16:48:18,693][75949] Updated weights for policy 0, policy_version 80611 (0.0009) -[2023-10-14 16:48:19,064][75949] Updated weights for policy 0, policy_version 80621 (0.0010) -[2023-10-14 16:48:19,440][75949] Updated weights for policy 0, policy_version 80631 (0.0008) -[2023-10-14 16:48:21,507][75950] Updated weights for policy 1, policy_version 80420 (0.0008) -[2023-10-14 16:48:21,865][75950] Updated weights for policy 1, policy_version 80430 (0.0008) -[2023-10-14 16:48:22,231][75950] Updated weights for policy 1, policy_version 80440 (0.0008) -[2023-10-14 16:48:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 164954112. Throughput: 0: 1685.7, 1: 1674.9. Samples: 41245594. Policy #0 lag: (min: 0.0, avg: 28.7, max: 32.0) -[2023-10-14 16:48:23,165][74987] Avg episode reward: [(0, '28.040'), (1, '33.910')] -[2023-10-14 16:48:23,378][75949] Updated weights for policy 0, policy_version 80641 (0.0008) -[2023-10-14 16:48:23,749][75949] Updated weights for policy 0, policy_version 80651 (0.0008) -[2023-10-14 16:48:24,125][75949] Updated weights for policy 0, policy_version 80661 (0.0008) -[2023-10-14 16:48:24,486][75949] Updated weights for policy 0, policy_version 80671 (0.0008) -[2023-10-14 16:48:26,278][75950] Updated weights for policy 1, policy_version 80450 (0.0008) -[2023-10-14 16:48:26,641][75950] Updated weights for policy 1, policy_version 80460 (0.0009) -[2023-10-14 16:48:27,011][75950] Updated weights for policy 1, policy_version 80470 (0.0008) -[2023-10-14 16:48:27,379][75950] Updated weights for policy 1, policy_version 80480 (0.0008) -[2023-10-14 16:48:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 165019648. Throughput: 0: 1684.9, 1: 1660.5. Samples: 41265304. Policy #0 lag: (min: 0.0, avg: 28.7, max: 32.0) -[2023-10-14 16:48:28,164][74987] Avg episode reward: [(0, '29.880'), (1, '34.100')] -[2023-10-14 16:48:28,661][75949] Updated weights for policy 0, policy_version 80681 (0.0007) -[2023-10-14 16:48:29,032][75949] Updated weights for policy 0, policy_version 80691 (0.0009) -[2023-10-14 16:48:29,403][75949] Updated weights for policy 0, policy_version 80701 (0.0011) -[2023-10-14 16:48:31,415][75950] Updated weights for policy 1, policy_version 80490 (0.0010) -[2023-10-14 16:48:31,776][75950] Updated weights for policy 1, policy_version 80500 (0.0008) -[2023-10-14 16:48:32,148][75950] Updated weights for policy 1, policy_version 80510 (0.0008) -[2023-10-14 16:48:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 165085184. Throughput: 0: 1686.0, 1: 1679.5. Samples: 41275628. Policy #0 lag: (min: 0.0, avg: 28.7, max: 32.0) -[2023-10-14 16:48:33,165][74987] Avg episode reward: [(0, '28.970'), (1, '33.270')] -[2023-10-14 16:48:33,351][75949] Updated weights for policy 0, policy_version 80711 (0.0008) -[2023-10-14 16:48:33,713][75949] Updated weights for policy 0, policy_version 80721 (0.0008) -[2023-10-14 16:48:34,089][75949] Updated weights for policy 0, policy_version 80731 (0.0007) -[2023-10-14 16:48:36,436][75950] Updated weights for policy 1, policy_version 80520 (0.0008) -[2023-10-14 16:48:36,818][75950] Updated weights for policy 1, policy_version 80530 (0.0012) -[2023-10-14 16:48:37,186][75950] Updated weights for policy 1, policy_version 80540 (0.0008) -[2023-10-14 16:48:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 165150720. Throughput: 0: 1687.4, 1: 1663.3. Samples: 41295632. Policy #0 lag: (min: 0.0, avg: 28.7, max: 32.0) -[2023-10-14 16:48:38,165][74987] Avg episode reward: [(0, '29.420'), (1, '32.280')] -[2023-10-14 16:48:38,169][75949] Updated weights for policy 0, policy_version 80741 (0.0007) -[2023-10-14 16:48:38,534][75949] Updated weights for policy 0, policy_version 80751 (0.0008) -[2023-10-14 16:48:38,898][75949] Updated weights for policy 0, policy_version 80761 (0.0008) -[2023-10-14 16:48:41,221][75950] Updated weights for policy 1, policy_version 80550 (0.0008) -[2023-10-14 16:48:41,580][75950] Updated weights for policy 1, policy_version 80560 (0.0008) -[2023-10-14 16:48:41,947][75950] Updated weights for policy 1, policy_version 80570 (0.0008) -[2023-10-14 16:48:42,883][75949] Updated weights for policy 0, policy_version 80771 (0.0007) -[2023-10-14 16:48:43,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 165216256. Throughput: 0: 1696.9, 1: 1664.1. Samples: 41315746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:48:43,164][74987] Avg episode reward: [(0, '28.360'), (1, '34.400')] -[2023-10-14 16:48:43,172][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000080576_82509824.pth... -[2023-10-14 16:48:43,212][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000079008_80904192.pth -[2023-10-14 16:48:43,292][75949] Updated weights for policy 0, policy_version 80781 (0.0008) -[2023-10-14 16:48:43,678][75949] Updated weights for policy 0, policy_version 80791 (0.0010) -[2023-10-14 16:48:44,004][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000080800_82739200.pth... -[2023-10-14 16:48:44,034][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000079200_81100800.pth -[2023-10-14 16:48:45,977][75950] Updated weights for policy 1, policy_version 80580 (0.0009) -[2023-10-14 16:48:46,358][75950] Updated weights for policy 1, policy_version 80590 (0.0009) -[2023-10-14 16:48:46,722][75950] Updated weights for policy 1, policy_version 80600 (0.0008) -[2023-10-14 16:48:47,710][75949] Updated weights for policy 0, policy_version 80801 (0.0011) -[2023-10-14 16:48:48,076][75949] Updated weights for policy 0, policy_version 80811 (0.0007) -[2023-10-14 16:48:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 165281792. Throughput: 0: 1692.7, 1: 1668.6. Samples: 41325924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:48:48,164][74987] Avg episode reward: [(0, '29.970'), (1, '34.060')] -[2023-10-14 16:48:48,450][75949] Updated weights for policy 0, policy_version 80821 (0.0008) -[2023-10-14 16:48:48,818][75949] Updated weights for policy 0, policy_version 80831 (0.0009) -[2023-10-14 16:48:50,677][75950] Updated weights for policy 1, policy_version 80610 (0.0007) -[2023-10-14 16:48:51,048][75950] Updated weights for policy 1, policy_version 80620 (0.0009) -[2023-10-14 16:48:51,414][75950] Updated weights for policy 1, policy_version 80630 (0.0009) -[2023-10-14 16:48:51,777][75950] Updated weights for policy 1, policy_version 80640 (0.0008) -[2023-10-14 16:48:52,994][75949] Updated weights for policy 0, policy_version 80841 (0.0007) -[2023-10-14 16:48:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 165347328. Throughput: 0: 1682.0, 1: 1655.5. Samples: 41345566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:48:53,164][74987] Avg episode reward: [(0, '27.980'), (1, '35.250')] -[2023-10-14 16:48:53,361][75949] Updated weights for policy 0, policy_version 80851 (0.0008) -[2023-10-14 16:48:53,738][75949] Updated weights for policy 0, policy_version 80861 (0.0007) -[2023-10-14 16:48:55,974][75950] Updated weights for policy 1, policy_version 80650 (0.0007) -[2023-10-14 16:48:56,338][75950] Updated weights for policy 1, policy_version 80660 (0.0009) -[2023-10-14 16:48:56,709][75950] Updated weights for policy 1, policy_version 80670 (0.0009) -[2023-10-14 16:48:58,017][75949] Updated weights for policy 0, policy_version 80871 (0.0011) -[2023-10-14 16:48:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 165412864. Throughput: 0: 1678.6, 1: 1672.9. Samples: 41365994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:48:58,164][74987] Avg episode reward: [(0, '29.240'), (1, '33.990')] -[2023-10-14 16:48:58,374][75949] Updated weights for policy 0, policy_version 80881 (0.0011) -[2023-10-14 16:48:58,742][75949] Updated weights for policy 0, policy_version 80891 (0.0011) -[2023-10-14 16:49:00,720][75950] Updated weights for policy 1, policy_version 80680 (0.0011) -[2023-10-14 16:49:01,092][75950] Updated weights for policy 1, policy_version 80690 (0.0011) -[2023-10-14 16:49:01,462][75950] Updated weights for policy 1, policy_version 80700 (0.0010) -[2023-10-14 16:49:02,898][75949] Updated weights for policy 0, policy_version 80901 (0.0010) -[2023-10-14 16:49:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 165478400. Throughput: 0: 1677.4, 1: 1669.8. Samples: 41376054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:49:03,164][74987] Avg episode reward: [(0, '27.860'), (1, '33.570')] -[2023-10-14 16:49:03,278][75949] Updated weights for policy 0, policy_version 80911 (0.0007) -[2023-10-14 16:49:03,643][75949] Updated weights for policy 0, policy_version 80921 (0.0009) -[2023-10-14 16:49:05,635][75950] Updated weights for policy 1, policy_version 80710 (0.0007) -[2023-10-14 16:49:06,009][75950] Updated weights for policy 1, policy_version 80720 (0.0007) -[2023-10-14 16:49:06,374][75950] Updated weights for policy 1, policy_version 80730 (0.0009) -[2023-10-14 16:49:07,560][75949] Updated weights for policy 0, policy_version 80931 (0.0010) -[2023-10-14 16:49:07,921][75949] Updated weights for policy 0, policy_version 80941 (0.0008) -[2023-10-14 16:49:08,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 165543936. Throughput: 0: 1685.5, 1: 1652.9. Samples: 41395820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:49:08,164][74987] Avg episode reward: [(0, '28.690'), (1, '35.320')] -[2023-10-14 16:49:08,294][75949] Updated weights for policy 0, policy_version 80951 (0.0007) -[2023-10-14 16:49:10,517][75950] Updated weights for policy 1, policy_version 80740 (0.0010) -[2023-10-14 16:49:10,890][75950] Updated weights for policy 1, policy_version 80750 (0.0010) -[2023-10-14 16:49:11,250][75950] Updated weights for policy 1, policy_version 80760 (0.0010) -[2023-10-14 16:49:12,395][75949] Updated weights for policy 0, policy_version 80961 (0.0008) -[2023-10-14 16:49:12,753][75949] Updated weights for policy 0, policy_version 80971 (0.0007) -[2023-10-14 16:49:13,122][75949] Updated weights for policy 0, policy_version 80981 (0.0010) -[2023-10-14 16:49:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 165609472. Throughput: 0: 1676.1, 1: 1677.4. Samples: 41416210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:49:13,164][74987] Avg episode reward: [(0, '25.200'), (1, '35.680')] -[2023-10-14 16:49:13,498][75949] Updated weights for policy 0, policy_version 80991 (0.0008) -[2023-10-14 16:49:15,288][75950] Updated weights for policy 1, policy_version 80770 (0.0009) -[2023-10-14 16:49:15,656][75950] Updated weights for policy 1, policy_version 80780 (0.0008) -[2023-10-14 16:49:16,020][75950] Updated weights for policy 1, policy_version 80790 (0.0008) -[2023-10-14 16:49:16,390][75950] Updated weights for policy 1, policy_version 80800 (0.0008) -[2023-10-14 16:49:17,525][75949] Updated weights for policy 0, policy_version 81001 (0.0009) -[2023-10-14 16:49:17,901][75949] Updated weights for policy 0, policy_version 81011 (0.0009) -[2023-10-14 16:49:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 165675008. Throughput: 0: 1683.9, 1: 1668.6. Samples: 41426488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:49:18,164][74987] Avg episode reward: [(0, '28.220'), (1, '32.310')] -[2023-10-14 16:49:18,262][75949] Updated weights for policy 0, policy_version 81021 (0.0009) -[2023-10-14 16:49:20,528][75950] Updated weights for policy 1, policy_version 80810 (0.0007) -[2023-10-14 16:49:20,904][75950] Updated weights for policy 1, policy_version 80820 (0.0007) -[2023-10-14 16:49:21,271][75950] Updated weights for policy 1, policy_version 80830 (0.0007) -[2023-10-14 16:49:22,194][75949] Updated weights for policy 0, policy_version 81031 (0.0010) -[2023-10-14 16:49:22,574][75949] Updated weights for policy 0, policy_version 81041 (0.0010) -[2023-10-14 16:49:22,942][75949] Updated weights for policy 0, policy_version 81051 (0.0007) -[2023-10-14 16:49:23,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 165773312. Throughput: 0: 1685.7, 1: 1666.0. Samples: 41446456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:49:23,165][74987] Avg episode reward: [(0, '26.760'), (1, '32.700')] -[2023-10-14 16:49:25,281][75950] Updated weights for policy 1, policy_version 80840 (0.0008) -[2023-10-14 16:49:25,659][75950] Updated weights for policy 1, policy_version 80850 (0.0009) -[2023-10-14 16:49:26,016][75950] Updated weights for policy 1, policy_version 80860 (0.0009) -[2023-10-14 16:49:26,874][75949] Updated weights for policy 0, policy_version 81061 (0.0009) -[2023-10-14 16:49:27,240][75949] Updated weights for policy 0, policy_version 81071 (0.0008) -[2023-10-14 16:49:27,608][75949] Updated weights for policy 0, policy_version 81081 (0.0009) -[2023-10-14 16:49:28,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 165838848. Throughput: 0: 1663.2, 1: 1678.3. Samples: 41466112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:49:28,165][74987] Avg episode reward: [(0, '31.190'), (1, '36.410')] -[2023-10-14 16:49:30,255][75950] Updated weights for policy 1, policy_version 80870 (0.0009) -[2023-10-14 16:49:30,619][75950] Updated weights for policy 1, policy_version 80880 (0.0008) -[2023-10-14 16:49:30,988][75950] Updated weights for policy 1, policy_version 80890 (0.0007) -[2023-10-14 16:49:31,680][75949] Updated weights for policy 0, policy_version 81091 (0.0010) -[2023-10-14 16:49:32,074][75949] Updated weights for policy 0, policy_version 81101 (0.0009) -[2023-10-14 16:49:32,444][75949] Updated weights for policy 0, policy_version 81111 (0.0010) -[2023-10-14 16:49:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 165904384. Throughput: 0: 1691.7, 1: 1661.2. Samples: 41476804. Policy #0 lag: (min: 27.0, avg: 28.1, max: 45.0) -[2023-10-14 16:49:33,164][74987] Avg episode reward: [(0, '28.060'), (1, '35.460')] -[2023-10-14 16:49:35,199][75950] Updated weights for policy 1, policy_version 80900 (0.0008) -[2023-10-14 16:49:35,577][75950] Updated weights for policy 1, policy_version 80910 (0.0009) -[2023-10-14 16:49:35,949][75950] Updated weights for policy 1, policy_version 80920 (0.0011) -[2023-10-14 16:49:36,406][75949] Updated weights for policy 0, policy_version 81121 (0.0009) -[2023-10-14 16:49:36,779][75949] Updated weights for policy 0, policy_version 81131 (0.0010) -[2023-10-14 16:49:37,148][75949] Updated weights for policy 0, policy_version 81141 (0.0010) -[2023-10-14 16:49:37,525][75949] Updated weights for policy 0, policy_version 81151 (0.0008) -[2023-10-14 16:49:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 165969920. Throughput: 0: 1689.3, 1: 1667.2. Samples: 41496610. Policy #0 lag: (min: 27.0, avg: 28.1, max: 45.0) -[2023-10-14 16:49:38,165][74987] Avg episode reward: [(0, '29.200'), (1, '35.930')] -[2023-10-14 16:49:40,048][75950] Updated weights for policy 1, policy_version 80930 (0.0007) -[2023-10-14 16:49:40,421][75950] Updated weights for policy 1, policy_version 80940 (0.0011) -[2023-10-14 16:49:40,787][75950] Updated weights for policy 1, policy_version 80950 (0.0008) -[2023-10-14 16:49:41,149][75950] Updated weights for policy 1, policy_version 80960 (0.0008) -[2023-10-14 16:49:41,325][75949] Updated weights for policy 0, policy_version 81161 (0.0008) -[2023-10-14 16:49:41,698][75949] Updated weights for policy 0, policy_version 81171 (0.0008) -[2023-10-14 16:49:42,064][75949] Updated weights for policy 0, policy_version 81181 (0.0008) -[2023-10-14 16:49:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 166035456. Throughput: 0: 1672.7, 1: 1668.3. Samples: 41516338. Policy #0 lag: (min: 27.0, avg: 28.1, max: 45.0) -[2023-10-14 16:49:43,165][74987] Avg episode reward: [(0, '26.120'), (1, '36.930')] -[2023-10-14 16:49:45,428][75950] Updated weights for policy 1, policy_version 80970 (0.0008) -[2023-10-14 16:49:45,797][75950] Updated weights for policy 1, policy_version 80980 (0.0009) -[2023-10-14 16:49:46,153][75950] Updated weights for policy 1, policy_version 80990 (0.0009) -[2023-10-14 16:49:46,245][75949] Updated weights for policy 0, policy_version 81191 (0.0008) -[2023-10-14 16:49:46,622][75949] Updated weights for policy 0, policy_version 81201 (0.0008) -[2023-10-14 16:49:46,989][75949] Updated weights for policy 0, policy_version 81211 (0.0009) -[2023-10-14 16:49:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 166100992. Throughput: 0: 1702.7, 1: 1657.6. Samples: 41527266. Policy #0 lag: (min: 27.0, avg: 28.1, max: 45.0) -[2023-10-14 16:49:48,164][74987] Avg episode reward: [(0, '26.870'), (1, '37.580')] -[2023-10-14 16:49:48,165][75801] Saving new best policy, reward=37.580! -[2023-10-14 16:49:50,219][75950] Updated weights for policy 1, policy_version 81000 (0.0011) -[2023-10-14 16:49:50,587][75950] Updated weights for policy 1, policy_version 81010 (0.0010) -[2023-10-14 16:49:50,959][75950] Updated weights for policy 1, policy_version 81020 (0.0012) -[2023-10-14 16:49:51,156][75949] Updated weights for policy 0, policy_version 81221 (0.0008) -[2023-10-14 16:49:51,526][75949] Updated weights for policy 0, policy_version 81231 (0.0009) -[2023-10-14 16:49:51,899][75949] Updated weights for policy 0, policy_version 81241 (0.0009) -[2023-10-14 16:49:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 166166528. Throughput: 0: 1680.9, 1: 1667.5. Samples: 41546500. Policy #0 lag: (min: 27.0, avg: 28.1, max: 45.0) -[2023-10-14 16:49:53,164][74987] Avg episode reward: [(0, '27.090'), (1, '33.870')] -[2023-10-14 16:49:54,995][75950] Updated weights for policy 1, policy_version 81030 (0.0010) -[2023-10-14 16:49:55,372][75950] Updated weights for policy 1, policy_version 81040 (0.0009) -[2023-10-14 16:49:55,736][75950] Updated weights for policy 1, policy_version 81050 (0.0008) -[2023-10-14 16:49:55,896][75949] Updated weights for policy 0, policy_version 81251 (0.0008) -[2023-10-14 16:49:56,262][75949] Updated weights for policy 0, policy_version 81261 (0.0010) -[2023-10-14 16:49:56,628][75949] Updated weights for policy 0, policy_version 81271 (0.0009) -[2023-10-14 16:49:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 166232064. Throughput: 0: 1679.7, 1: 1666.6. Samples: 41566794. Policy #0 lag: (min: 27.0, avg: 28.1, max: 45.0) -[2023-10-14 16:49:58,165][74987] Avg episode reward: [(0, '28.170'), (1, '33.120')] -[2023-10-14 16:49:59,778][75950] Updated weights for policy 1, policy_version 81060 (0.0008) -[2023-10-14 16:50:00,150][75950] Updated weights for policy 1, policy_version 81070 (0.0008) -[2023-10-14 16:50:00,506][75950] Updated weights for policy 1, policy_version 81080 (0.0008) -[2023-10-14 16:50:00,707][75949] Updated weights for policy 0, policy_version 81281 (0.0008) -[2023-10-14 16:50:01,075][75949] Updated weights for policy 0, policy_version 81291 (0.0009) -[2023-10-14 16:50:01,447][75949] Updated weights for policy 0, policy_version 81301 (0.0009) -[2023-10-14 16:50:01,823][75949] Updated weights for policy 0, policy_version 81311 (0.0011) -[2023-10-14 16:50:03,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 166297600. Throughput: 0: 1701.6, 1: 1652.0. Samples: 41577404. Policy #0 lag: (min: 27.0, avg: 28.1, max: 45.0) -[2023-10-14 16:50:03,164][74987] Avg episode reward: [(0, '29.300'), (1, '35.040')] -[2023-10-14 16:50:04,617][75950] Updated weights for policy 1, policy_version 81090 (0.0007) -[2023-10-14 16:50:04,977][75950] Updated weights for policy 1, policy_version 81100 (0.0007) -[2023-10-14 16:50:05,341][75950] Updated weights for policy 1, policy_version 81110 (0.0008) -[2023-10-14 16:50:05,707][75950] Updated weights for policy 1, policy_version 81120 (0.0008) -[2023-10-14 16:50:05,818][75949] Updated weights for policy 0, policy_version 81321 (0.0009) -[2023-10-14 16:50:06,179][75949] Updated weights for policy 0, policy_version 81331 (0.0009) -[2023-10-14 16:50:06,548][75949] Updated weights for policy 0, policy_version 81341 (0.0008) -[2023-10-14 16:50:08,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 166363136. Throughput: 0: 1674.8, 1: 1663.4. Samples: 41596674. Policy #0 lag: (min: 27.0, avg: 28.1, max: 45.0) -[2023-10-14 16:50:08,165][74987] Avg episode reward: [(0, '28.170'), (1, '33.840')] -[2023-10-14 16:50:09,885][75950] Updated weights for policy 1, policy_version 81130 (0.0007) -[2023-10-14 16:50:10,248][75950] Updated weights for policy 1, policy_version 81140 (0.0008) -[2023-10-14 16:50:10,620][75950] Updated weights for policy 1, policy_version 81150 (0.0007) -[2023-10-14 16:50:10,624][75949] Updated weights for policy 0, policy_version 81351 (0.0009) -[2023-10-14 16:50:10,990][75949] Updated weights for policy 0, policy_version 81361 (0.0011) -[2023-10-14 16:50:11,361][75949] Updated weights for policy 0, policy_version 81371 (0.0009) -[2023-10-14 16:50:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 166428672. Throughput: 0: 1695.0, 1: 1666.3. Samples: 41617370. Policy #0 lag: (min: 27.0, avg: 28.1, max: 45.0) -[2023-10-14 16:50:13,165][74987] Avg episode reward: [(0, '29.660'), (1, '32.900')] -[2023-10-14 16:50:14,684][75950] Updated weights for policy 1, policy_version 81160 (0.0008) -[2023-10-14 16:50:15,048][75950] Updated weights for policy 1, policy_version 81170 (0.0007) -[2023-10-14 16:50:15,415][75950] Updated weights for policy 1, policy_version 81180 (0.0008) -[2023-10-14 16:50:15,439][75949] Updated weights for policy 0, policy_version 81381 (0.0009) -[2023-10-14 16:50:15,821][75949] Updated weights for policy 0, policy_version 81391 (0.0009) -[2023-10-14 16:50:16,181][75949] Updated weights for policy 0, policy_version 81401 (0.0009) -[2023-10-14 16:50:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 166494208. Throughput: 0: 1686.1, 1: 1655.5. Samples: 41627176. Policy #0 lag: (min: 27.0, avg: 28.1, max: 45.0) -[2023-10-14 16:50:18,165][74987] Avg episode reward: [(0, '28.130'), (1, '32.460')] -[2023-10-14 16:50:19,468][75950] Updated weights for policy 1, policy_version 81190 (0.0010) -[2023-10-14 16:50:19,837][75950] Updated weights for policy 1, policy_version 81200 (0.0010) -[2023-10-14 16:50:20,197][75950] Updated weights for policy 1, policy_version 81210 (0.0009) -[2023-10-14 16:50:20,310][75949] Updated weights for policy 0, policy_version 81411 (0.0010) -[2023-10-14 16:50:20,693][75949] Updated weights for policy 0, policy_version 81421 (0.0008) -[2023-10-14 16:50:21,063][75949] Updated weights for policy 0, policy_version 81431 (0.0009) -[2023-10-14 16:50:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 166559744. Throughput: 0: 1666.0, 1: 1668.3. Samples: 41646652. Policy #0 lag: (min: 27.0, avg: 28.1, max: 45.0) -[2023-10-14 16:50:23,165][74987] Avg episode reward: [(0, '31.510'), (1, '35.520')] -[2023-10-14 16:50:24,360][75950] Updated weights for policy 1, policy_version 81220 (0.0008) -[2023-10-14 16:50:24,715][75950] Updated weights for policy 1, policy_version 81230 (0.0010) -[2023-10-14 16:50:24,928][75949] Updated weights for policy 0, policy_version 81441 (0.0010) -[2023-10-14 16:50:25,080][75950] Updated weights for policy 1, policy_version 81240 (0.0009) -[2023-10-14 16:50:25,285][75949] Updated weights for policy 0, policy_version 81451 (0.0010) -[2023-10-14 16:50:25,656][75949] Updated weights for policy 0, policy_version 81461 (0.0008) -[2023-10-14 16:50:26,033][75949] Updated weights for policy 0, policy_version 81471 (0.0008) -[2023-10-14 16:50:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 166625280. Throughput: 0: 1687.3, 1: 1670.8. Samples: 41667452. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) -[2023-10-14 16:50:28,164][74987] Avg episode reward: [(0, '28.260'), (1, '36.750')] -[2023-10-14 16:50:29,318][75950] Updated weights for policy 1, policy_version 81250 (0.0011) -[2023-10-14 16:50:29,675][75950] Updated weights for policy 1, policy_version 81260 (0.0010) -[2023-10-14 16:50:30,013][75949] Updated weights for policy 0, policy_version 81481 (0.0009) -[2023-10-14 16:50:30,048][75950] Updated weights for policy 1, policy_version 81270 (0.0009) -[2023-10-14 16:50:30,383][75949] Updated weights for policy 0, policy_version 81491 (0.0008) -[2023-10-14 16:50:30,404][75950] Updated weights for policy 1, policy_version 81280 (0.0007) -[2023-10-14 16:50:30,749][75949] Updated weights for policy 0, policy_version 81501 (0.0008) -[2023-10-14 16:50:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 166690816. Throughput: 0: 1672.8, 1: 1654.9. Samples: 41677010. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) -[2023-10-14 16:50:33,164][74987] Avg episode reward: [(0, '30.670'), (1, '32.880')] -[2023-10-14 16:50:34,454][75950] Updated weights for policy 1, policy_version 81290 (0.0007) -[2023-10-14 16:50:34,573][75949] Updated weights for policy 0, policy_version 81511 (0.0007) -[2023-10-14 16:50:34,822][75950] Updated weights for policy 1, policy_version 81300 (0.0008) -[2023-10-14 16:50:34,932][75949] Updated weights for policy 0, policy_version 81521 (0.0008) -[2023-10-14 16:50:35,193][75950] Updated weights for policy 1, policy_version 81310 (0.0009) -[2023-10-14 16:50:35,314][75949] Updated weights for policy 0, policy_version 81531 (0.0008) -[2023-10-14 16:50:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 166756352. Throughput: 0: 1686.1, 1: 1669.5. Samples: 41697504. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) -[2023-10-14 16:50:38,165][74987] Avg episode reward: [(0, '29.080'), (1, '33.450')] -[2023-10-14 16:50:39,346][75950] Updated weights for policy 1, policy_version 81320 (0.0009) -[2023-10-14 16:50:39,422][75949] Updated weights for policy 0, policy_version 81541 (0.0010) -[2023-10-14 16:50:39,722][75950] Updated weights for policy 1, policy_version 81330 (0.0009) -[2023-10-14 16:50:39,796][75949] Updated weights for policy 0, policy_version 81551 (0.0007) -[2023-10-14 16:50:40,087][75950] Updated weights for policy 1, policy_version 81340 (0.0007) -[2023-10-14 16:50:40,164][75949] Updated weights for policy 0, policy_version 81561 (0.0008) -[2023-10-14 16:50:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 166821888. Throughput: 0: 1693.3, 1: 1670.9. Samples: 41718186. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) -[2023-10-14 16:50:43,164][74987] Avg episode reward: [(0, '32.350'), (1, '33.980')] -[2023-10-14 16:50:43,171][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000081344_83296256.pth... -[2023-10-14 16:50:43,171][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000081568_83525632.pth... -[2023-10-14 16:50:43,211][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000080000_81920000.pth -[2023-10-14 16:50:43,214][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000079776_81690624.pth -[2023-10-14 16:50:43,217][75615] Saving new best policy, reward=32.350! -[2023-10-14 16:50:44,058][75950] Updated weights for policy 1, policy_version 81350 (0.0010) -[2023-10-14 16:50:44,429][75950] Updated weights for policy 1, policy_version 81360 (0.0010) -[2023-10-14 16:50:44,460][75949] Updated weights for policy 0, policy_version 81571 (0.0009) -[2023-10-14 16:50:44,789][75950] Updated weights for policy 1, policy_version 81370 (0.0007) -[2023-10-14 16:50:44,834][75949] Updated weights for policy 0, policy_version 81581 (0.0007) -[2023-10-14 16:50:45,195][75949] Updated weights for policy 0, policy_version 81591 (0.0008) -[2023-10-14 16:50:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 166887424. Throughput: 0: 1663.5, 1: 1668.1. Samples: 41727326. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) -[2023-10-14 16:50:48,164][74987] Avg episode reward: [(0, '29.510'), (1, '33.980')] -[2023-10-14 16:50:48,926][75950] Updated weights for policy 1, policy_version 81380 (0.0008) -[2023-10-14 16:50:49,292][75950] Updated weights for policy 1, policy_version 81390 (0.0009) -[2023-10-14 16:50:49,314][75949] Updated weights for policy 0, policy_version 81601 (0.0010) -[2023-10-14 16:50:49,669][75950] Updated weights for policy 1, policy_version 81400 (0.0007) -[2023-10-14 16:50:49,673][75949] Updated weights for policy 0, policy_version 81611 (0.0009) -[2023-10-14 16:50:50,050][75949] Updated weights for policy 0, policy_version 81621 (0.0008) -[2023-10-14 16:50:50,424][75949] Updated weights for policy 0, policy_version 81631 (0.0010) -[2023-10-14 16:50:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 166952960. Throughput: 0: 1684.0, 1: 1672.4. Samples: 41747712. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) -[2023-10-14 16:50:53,164][74987] Avg episode reward: [(0, '28.380'), (1, '32.490')] -[2023-10-14 16:50:53,736][75950] Updated weights for policy 1, policy_version 81410 (0.0007) -[2023-10-14 16:50:54,105][75950] Updated weights for policy 1, policy_version 81420 (0.0009) -[2023-10-14 16:50:54,481][75950] Updated weights for policy 1, policy_version 81430 (0.0009) -[2023-10-14 16:50:54,538][75949] Updated weights for policy 0, policy_version 81641 (0.0008) -[2023-10-14 16:50:54,836][75950] Updated weights for policy 1, policy_version 81440 (0.0009) -[2023-10-14 16:50:54,903][75949] Updated weights for policy 0, policy_version 81651 (0.0008) -[2023-10-14 16:50:55,265][75949] Updated weights for policy 0, policy_version 81661 (0.0007) -[2023-10-14 16:50:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 167018496. Throughput: 0: 1688.3, 1: 1672.8. Samples: 41768622. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) -[2023-10-14 16:50:58,165][74987] Avg episode reward: [(0, '28.350'), (1, '33.900')] -[2023-10-14 16:50:58,921][75950] Updated weights for policy 1, policy_version 81450 (0.0008) -[2023-10-14 16:50:59,286][75950] Updated weights for policy 1, policy_version 81460 (0.0008) -[2023-10-14 16:50:59,329][75949] Updated weights for policy 0, policy_version 81671 (0.0009) -[2023-10-14 16:50:59,657][75950] Updated weights for policy 1, policy_version 81470 (0.0008) -[2023-10-14 16:50:59,695][75949] Updated weights for policy 0, policy_version 81681 (0.0008) -[2023-10-14 16:51:00,070][75949] Updated weights for policy 0, policy_version 81691 (0.0010) -[2023-10-14 16:51:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 167084032. Throughput: 0: 1672.3, 1: 1672.5. Samples: 41777692. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) -[2023-10-14 16:51:03,165][74987] Avg episode reward: [(0, '27.540'), (1, '35.180')] -[2023-10-14 16:51:03,607][75950] Updated weights for policy 1, policy_version 81480 (0.0007) -[2023-10-14 16:51:03,982][75950] Updated weights for policy 1, policy_version 81490 (0.0008) -[2023-10-14 16:51:04,133][75949] Updated weights for policy 0, policy_version 81701 (0.0008) -[2023-10-14 16:51:04,350][75950] Updated weights for policy 1, policy_version 81500 (0.0008) -[2023-10-14 16:51:04,499][75949] Updated weights for policy 0, policy_version 81711 (0.0007) -[2023-10-14 16:51:04,883][75949] Updated weights for policy 0, policy_version 81721 (0.0009) -[2023-10-14 16:51:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 167149568. Throughput: 0: 1698.5, 1: 1674.3. Samples: 41798426. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) -[2023-10-14 16:51:08,165][74987] Avg episode reward: [(0, '28.780'), (1, '34.010')] -[2023-10-14 16:51:08,394][75950] Updated weights for policy 1, policy_version 81510 (0.0009) -[2023-10-14 16:51:08,762][75950] Updated weights for policy 1, policy_version 81520 (0.0009) -[2023-10-14 16:51:08,960][75949] Updated weights for policy 0, policy_version 81731 (0.0008) -[2023-10-14 16:51:09,132][75950] Updated weights for policy 1, policy_version 81530 (0.0010) -[2023-10-14 16:51:09,346][75949] Updated weights for policy 0, policy_version 81741 (0.0007) -[2023-10-14 16:51:09,715][75949] Updated weights for policy 0, policy_version 81751 (0.0009) -[2023-10-14 16:51:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167215104. Throughput: 0: 1692.4, 1: 1681.5. Samples: 41819276. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) -[2023-10-14 16:51:13,165][74987] Avg episode reward: [(0, '28.550'), (1, '32.810')] -[2023-10-14 16:51:13,216][75950] Updated weights for policy 1, policy_version 81540 (0.0007) -[2023-10-14 16:51:13,582][75950] Updated weights for policy 1, policy_version 81550 (0.0008) -[2023-10-14 16:51:13,695][75949] Updated weights for policy 0, policy_version 81761 (0.0007) -[2023-10-14 16:51:13,954][75950] Updated weights for policy 1, policy_version 81560 (0.0008) -[2023-10-14 16:51:14,062][75949] Updated weights for policy 0, policy_version 81771 (0.0008) -[2023-10-14 16:51:14,436][75949] Updated weights for policy 0, policy_version 81781 (0.0007) -[2023-10-14 16:51:14,808][75949] Updated weights for policy 0, policy_version 81791 (0.0009) -[2023-10-14 16:51:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 167280640. Throughput: 0: 1678.8, 1: 1684.9. Samples: 41828380. Policy #0 lag: (min: 31.0, avg: 31.9, max: 53.0) -[2023-10-14 16:51:18,165][74987] Avg episode reward: [(0, '28.990'), (1, '36.680')] -[2023-10-14 16:51:18,265][75950] Updated weights for policy 1, policy_version 81570 (0.0007) -[2023-10-14 16:51:18,633][75950] Updated weights for policy 1, policy_version 81580 (0.0008) -[2023-10-14 16:51:18,962][75949] Updated weights for policy 0, policy_version 81801 (0.0008) -[2023-10-14 16:51:19,002][75950] Updated weights for policy 1, policy_version 81590 (0.0007) -[2023-10-14 16:51:19,329][75949] Updated weights for policy 0, policy_version 81811 (0.0008) -[2023-10-14 16:51:19,366][75950] Updated weights for policy 1, policy_version 81600 (0.0008) -[2023-10-14 16:51:19,691][75949] Updated weights for policy 0, policy_version 81821 (0.0008) -[2023-10-14 16:51:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167346176. Throughput: 0: 1682.1, 1: 1680.9. Samples: 41848840. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 16:51:23,165][74987] Avg episode reward: [(0, '28.380'), (1, '35.540')] -[2023-10-14 16:51:23,403][75950] Updated weights for policy 1, policy_version 81610 (0.0007) -[2023-10-14 16:51:23,762][75950] Updated weights for policy 1, policy_version 81620 (0.0008) -[2023-10-14 16:51:23,832][75949] Updated weights for policy 0, policy_version 81831 (0.0007) -[2023-10-14 16:51:24,132][75950] Updated weights for policy 1, policy_version 81630 (0.0010) -[2023-10-14 16:51:24,209][75949] Updated weights for policy 0, policy_version 81841 (0.0010) -[2023-10-14 16:51:24,569][75949] Updated weights for policy 0, policy_version 81851 (0.0011) -[2023-10-14 16:51:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167411712. Throughput: 0: 1686.0, 1: 1676.9. Samples: 41869516. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 16:51:28,165][74987] Avg episode reward: [(0, '28.570'), (1, '35.040')] -[2023-10-14 16:51:28,259][75950] Updated weights for policy 1, policy_version 81640 (0.0007) -[2023-10-14 16:51:28,554][75949] Updated weights for policy 0, policy_version 81861 (0.0009) -[2023-10-14 16:51:28,626][75950] Updated weights for policy 1, policy_version 81650 (0.0008) -[2023-10-14 16:51:28,920][75949] Updated weights for policy 0, policy_version 81871 (0.0008) -[2023-10-14 16:51:28,997][75950] Updated weights for policy 1, policy_version 81660 (0.0008) -[2023-10-14 16:51:29,291][75949] Updated weights for policy 0, policy_version 81881 (0.0008) -[2023-10-14 16:51:33,095][75950] Updated weights for policy 1, policy_version 81670 (0.0009) -[2023-10-14 16:51:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167477248. Throughput: 0: 1685.7, 1: 1675.2. Samples: 41878568. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 16:51:33,165][74987] Avg episode reward: [(0, '27.040'), (1, '33.950')] -[2023-10-14 16:51:33,365][75949] Updated weights for policy 0, policy_version 81891 (0.0008) -[2023-10-14 16:51:33,464][75950] Updated weights for policy 1, policy_version 81680 (0.0008) -[2023-10-14 16:51:33,736][75949] Updated weights for policy 0, policy_version 81901 (0.0008) -[2023-10-14 16:51:33,825][75950] Updated weights for policy 1, policy_version 81690 (0.0008) -[2023-10-14 16:51:34,105][75949] Updated weights for policy 0, policy_version 81911 (0.0008) -[2023-10-14 16:51:37,937][75950] Updated weights for policy 1, policy_version 81700 (0.0008) -[2023-10-14 16:51:38,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167542784. Throughput: 0: 1684.1, 1: 1682.2. Samples: 41899196. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 16:51:38,164][74987] Avg episode reward: [(0, '26.800'), (1, '35.420')] -[2023-10-14 16:51:38,293][75950] Updated weights for policy 1, policy_version 81710 (0.0008) -[2023-10-14 16:51:38,298][75949] Updated weights for policy 0, policy_version 81921 (0.0009) -[2023-10-14 16:51:38,654][75950] Updated weights for policy 1, policy_version 81720 (0.0008) -[2023-10-14 16:51:38,663][75949] Updated weights for policy 0, policy_version 81931 (0.0008) -[2023-10-14 16:51:39,033][75949] Updated weights for policy 0, policy_version 81941 (0.0010) -[2023-10-14 16:51:39,406][75949] Updated weights for policy 0, policy_version 81951 (0.0012) -[2023-10-14 16:51:42,708][75950] Updated weights for policy 1, policy_version 81730 (0.0008) -[2023-10-14 16:51:43,070][75950] Updated weights for policy 1, policy_version 81740 (0.0007) -[2023-10-14 16:51:43,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167608320. Throughput: 0: 1677.7, 1: 1682.6. Samples: 41919838. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 16:51:43,164][74987] Avg episode reward: [(0, '26.790'), (1, '34.210')] -[2023-10-14 16:51:43,430][75950] Updated weights for policy 1, policy_version 81750 (0.0007) -[2023-10-14 16:51:43,536][75949] Updated weights for policy 0, policy_version 81961 (0.0008) -[2023-10-14 16:51:43,790][75950] Updated weights for policy 1, policy_version 81760 (0.0007) -[2023-10-14 16:51:43,906][75949] Updated weights for policy 0, policy_version 81971 (0.0008) -[2023-10-14 16:51:44,274][75949] Updated weights for policy 0, policy_version 81981 (0.0008) -[2023-10-14 16:51:48,156][75950] Updated weights for policy 1, policy_version 81770 (0.0010) -[2023-10-14 16:51:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 167673856. Throughput: 0: 1674.8, 1: 1681.6. Samples: 41928730. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 16:51:48,165][74987] Avg episode reward: [(0, '27.720'), (1, '33.470')] -[2023-10-14 16:51:48,428][75949] Updated weights for policy 0, policy_version 81991 (0.0009) -[2023-10-14 16:51:48,520][75950] Updated weights for policy 1, policy_version 81780 (0.0010) -[2023-10-14 16:51:48,809][75949] Updated weights for policy 0, policy_version 82001 (0.0008) -[2023-10-14 16:51:48,886][75950] Updated weights for policy 1, policy_version 81790 (0.0007) -[2023-10-14 16:51:49,172][75949] Updated weights for policy 0, policy_version 82011 (0.0009) -[2023-10-14 16:51:52,878][75950] Updated weights for policy 1, policy_version 81800 (0.0009) -[2023-10-14 16:51:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167739392. Throughput: 0: 1670.7, 1: 1680.3. Samples: 41949218. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 16:51:53,164][74987] Avg episode reward: [(0, '28.080'), (1, '33.080')] -[2023-10-14 16:51:53,244][75950] Updated weights for policy 1, policy_version 81810 (0.0007) -[2023-10-14 16:51:53,379][75949] Updated weights for policy 0, policy_version 82021 (0.0009) -[2023-10-14 16:51:53,614][75950] Updated weights for policy 1, policy_version 81820 (0.0008) -[2023-10-14 16:51:53,760][75949] Updated weights for policy 0, policy_version 82031 (0.0008) -[2023-10-14 16:51:54,130][75949] Updated weights for policy 0, policy_version 82041 (0.0009) -[2023-10-14 16:51:57,659][75950] Updated weights for policy 1, policy_version 81830 (0.0009) -[2023-10-14 16:51:58,029][75950] Updated weights for policy 1, policy_version 81840 (0.0010) -[2023-10-14 16:51:58,059][75949] Updated weights for policy 0, policy_version 82051 (0.0010) -[2023-10-14 16:51:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167804928. Throughput: 0: 1671.0, 1: 1668.3. Samples: 41969542. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 16:51:58,165][74987] Avg episode reward: [(0, '28.450'), (1, '34.200')] -[2023-10-14 16:51:58,393][75950] Updated weights for policy 1, policy_version 81850 (0.0008) -[2023-10-14 16:51:58,426][75949] Updated weights for policy 0, policy_version 82061 (0.0007) -[2023-10-14 16:51:58,792][75949] Updated weights for policy 0, policy_version 82071 (0.0009) -[2023-10-14 16:52:02,596][75950] Updated weights for policy 1, policy_version 81860 (0.0008) -[2023-10-14 16:52:02,900][75949] Updated weights for policy 0, policy_version 82081 (0.0007) -[2023-10-14 16:52:02,966][75950] Updated weights for policy 1, policy_version 81870 (0.0008) -[2023-10-14 16:52:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 167870464. Throughput: 0: 1671.1, 1: 1671.3. Samples: 41978790. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 16:52:03,165][74987] Avg episode reward: [(0, '28.290'), (1, '33.240')] -[2023-10-14 16:52:03,265][75949] Updated weights for policy 0, policy_version 82091 (0.0009) -[2023-10-14 16:52:03,330][75950] Updated weights for policy 1, policy_version 81880 (0.0009) -[2023-10-14 16:52:03,640][75949] Updated weights for policy 0, policy_version 82101 (0.0009) -[2023-10-14 16:52:04,007][75949] Updated weights for policy 0, policy_version 82111 (0.0008) -[2023-10-14 16:52:07,323][75950] Updated weights for policy 1, policy_version 81890 (0.0008) -[2023-10-14 16:52:07,691][75950] Updated weights for policy 1, policy_version 81900 (0.0010) -[2023-10-14 16:52:08,071][75950] Updated weights for policy 1, policy_version 81910 (0.0009) -[2023-10-14 16:52:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 167936000. Throughput: 0: 1668.2, 1: 1674.1. Samples: 41999244. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 16:52:08,164][74987] Avg episode reward: [(0, '28.020'), (1, '31.830')] -[2023-10-14 16:52:08,301][75949] Updated weights for policy 0, policy_version 82121 (0.0009) -[2023-10-14 16:52:08,435][75950] Updated weights for policy 1, policy_version 81920 (0.0009) -[2023-10-14 16:52:08,681][75949] Updated weights for policy 0, policy_version 82131 (0.0010) -[2023-10-14 16:52:09,051][75949] Updated weights for policy 0, policy_version 82141 (0.0011) -[2023-10-14 16:52:12,515][75950] Updated weights for policy 1, policy_version 81930 (0.0009) -[2023-10-14 16:52:12,882][75950] Updated weights for policy 1, policy_version 81940 (0.0008) -[2023-10-14 16:52:12,911][75949] Updated weights for policy 0, policy_version 82151 (0.0008) -[2023-10-14 16:52:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168001536. Throughput: 0: 1668.0, 1: 1668.4. Samples: 42019654. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) -[2023-10-14 16:52:13,165][74987] Avg episode reward: [(0, '27.950'), (1, '33.530')] -[2023-10-14 16:52:13,247][75950] Updated weights for policy 1, policy_version 81950 (0.0007) -[2023-10-14 16:52:13,285][75949] Updated weights for policy 0, policy_version 82161 (0.0009) -[2023-10-14 16:52:13,650][75949] Updated weights for policy 0, policy_version 82171 (0.0008) -[2023-10-14 16:52:17,294][75950] Updated weights for policy 1, policy_version 81960 (0.0010) -[2023-10-14 16:52:17,662][75950] Updated weights for policy 1, policy_version 81970 (0.0008) -[2023-10-14 16:52:17,746][75949] Updated weights for policy 0, policy_version 82181 (0.0008) -[2023-10-14 16:52:18,030][75950] Updated weights for policy 1, policy_version 81980 (0.0009) -[2023-10-14 16:52:18,114][75949] Updated weights for policy 0, policy_version 82191 (0.0009) -[2023-10-14 16:52:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 168067072. Throughput: 0: 1668.9, 1: 1680.0. Samples: 42029268. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:52:18,164][74987] Avg episode reward: [(0, '27.300'), (1, '36.970')] -[2023-10-14 16:52:18,475][75949] Updated weights for policy 0, policy_version 82201 (0.0008) -[2023-10-14 16:52:22,026][75950] Updated weights for policy 1, policy_version 81990 (0.0008) -[2023-10-14 16:52:22,396][75950] Updated weights for policy 1, policy_version 82000 (0.0008) -[2023-10-14 16:52:22,693][75949] Updated weights for policy 0, policy_version 82211 (0.0008) -[2023-10-14 16:52:22,758][75950] Updated weights for policy 1, policy_version 82010 (0.0008) -[2023-10-14 16:52:23,063][75949] Updated weights for policy 0, policy_version 82221 (0.0007) -[2023-10-14 16:52:23,164][74987] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 168165376. Throughput: 0: 1670.6, 1: 1677.6. Samples: 42049866. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:52:23,164][74987] Avg episode reward: [(0, '30.690'), (1, '34.900')] -[2023-10-14 16:52:23,436][75949] Updated weights for policy 0, policy_version 82231 (0.0007) -[2023-10-14 16:52:26,746][75950] Updated weights for policy 1, policy_version 82020 (0.0008) -[2023-10-14 16:52:27,116][75950] Updated weights for policy 1, policy_version 82030 (0.0009) -[2023-10-14 16:52:27,475][75949] Updated weights for policy 0, policy_version 82241 (0.0009) -[2023-10-14 16:52:27,482][75950] Updated weights for policy 1, policy_version 82040 (0.0009) -[2023-10-14 16:52:27,842][75949] Updated weights for policy 0, policy_version 82251 (0.0010) -[2023-10-14 16:52:28,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 168230912. Throughput: 0: 1664.7, 1: 1655.2. Samples: 42069238. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:52:28,165][74987] Avg episode reward: [(0, '28.570'), (1, '34.050')] -[2023-10-14 16:52:28,219][75949] Updated weights for policy 0, policy_version 82261 (0.0007) -[2023-10-14 16:52:28,591][75949] Updated weights for policy 0, policy_version 82271 (0.0007) -[2023-10-14 16:52:31,542][75950] Updated weights for policy 1, policy_version 82050 (0.0007) -[2023-10-14 16:52:31,913][75950] Updated weights for policy 1, policy_version 82060 (0.0009) -[2023-10-14 16:52:32,279][75950] Updated weights for policy 1, policy_version 82070 (0.0009) -[2023-10-14 16:52:32,649][75950] Updated weights for policy 1, policy_version 82080 (0.0008) -[2023-10-14 16:52:32,827][75949] Updated weights for policy 0, policy_version 82281 (0.0007) -[2023-10-14 16:52:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 168296448. Throughput: 0: 1673.1, 1: 1682.4. Samples: 42079726. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:52:33,164][74987] Avg episode reward: [(0, '29.430'), (1, '35.400')] -[2023-10-14 16:52:33,195][75949] Updated weights for policy 0, policy_version 82291 (0.0009) -[2023-10-14 16:52:33,571][75949] Updated weights for policy 0, policy_version 82301 (0.0009) -[2023-10-14 16:52:36,707][75950] Updated weights for policy 1, policy_version 82090 (0.0010) -[2023-10-14 16:52:37,079][75950] Updated weights for policy 1, policy_version 82100 (0.0009) -[2023-10-14 16:52:37,438][75950] Updated weights for policy 1, policy_version 82110 (0.0009) -[2023-10-14 16:52:37,600][75949] Updated weights for policy 0, policy_version 82311 (0.0009) -[2023-10-14 16:52:37,966][75949] Updated weights for policy 0, policy_version 82321 (0.0011) -[2023-10-14 16:52:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 168361984. Throughput: 0: 1676.0, 1: 1675.2. Samples: 42100024. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:52:38,164][74987] Avg episode reward: [(0, '28.120'), (1, '34.500')] -[2023-10-14 16:52:38,330][75949] Updated weights for policy 0, policy_version 82331 (0.0008) -[2023-10-14 16:52:41,416][75950] Updated weights for policy 1, policy_version 82120 (0.0010) -[2023-10-14 16:52:41,777][75950] Updated weights for policy 1, policy_version 82130 (0.0009) -[2023-10-14 16:52:42,143][75950] Updated weights for policy 1, policy_version 82140 (0.0009) -[2023-10-14 16:52:42,554][75949] Updated weights for policy 0, policy_version 82341 (0.0010) -[2023-10-14 16:52:42,953][75949] Updated weights for policy 0, policy_version 82351 (0.0010) -[2023-10-14 16:52:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 168427520. Throughput: 0: 1667.4, 1: 1659.1. Samples: 42119234. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:52:43,164][74987] Avg episode reward: [(0, '29.110'), (1, '33.590')] -[2023-10-14 16:52:43,172][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000082144_84115456.pth... -[2023-10-14 16:52:43,206][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000080576_82509824.pth -[2023-10-14 16:52:43,318][75949] Updated weights for policy 0, policy_version 82361 (0.0012) -[2023-10-14 16:52:43,577][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000082368_84344832.pth... -[2023-10-14 16:52:43,616][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000080800_82739200.pth -[2023-10-14 16:52:46,180][75950] Updated weights for policy 1, policy_version 82150 (0.0009) -[2023-10-14 16:52:46,548][75950] Updated weights for policy 1, policy_version 82160 (0.0011) -[2023-10-14 16:52:46,915][75950] Updated weights for policy 1, policy_version 82170 (0.0009) -[2023-10-14 16:52:47,420][75949] Updated weights for policy 0, policy_version 82371 (0.0010) -[2023-10-14 16:52:47,784][75949] Updated weights for policy 0, policy_version 82381 (0.0010) -[2023-10-14 16:52:48,157][75949] Updated weights for policy 0, policy_version 82391 (0.0010) -[2023-10-14 16:52:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 168493056. Throughput: 0: 1673.4, 1: 1684.3. Samples: 42129886. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:52:48,164][74987] Avg episode reward: [(0, '28.090'), (1, '35.950')] -[2023-10-14 16:52:51,208][75950] Updated weights for policy 1, policy_version 82180 (0.0011) -[2023-10-14 16:52:51,570][75950] Updated weights for policy 1, policy_version 82190 (0.0010) -[2023-10-14 16:52:51,933][75950] Updated weights for policy 1, policy_version 82200 (0.0009) -[2023-10-14 16:52:52,260][75949] Updated weights for policy 0, policy_version 82401 (0.0010) -[2023-10-14 16:52:52,618][75949] Updated weights for policy 0, policy_version 82411 (0.0007) -[2023-10-14 16:52:52,996][75949] Updated weights for policy 0, policy_version 82421 (0.0009) -[2023-10-14 16:52:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 168558592. Throughput: 0: 1674.7, 1: 1671.8. Samples: 42149834. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:52:53,165][74987] Avg episode reward: [(0, '30.430'), (1, '37.370')] -[2023-10-14 16:52:53,371][75949] Updated weights for policy 0, policy_version 82431 (0.0008) -[2023-10-14 16:52:55,956][75950] Updated weights for policy 1, policy_version 82210 (0.0008) -[2023-10-14 16:52:56,323][75950] Updated weights for policy 1, policy_version 82220 (0.0007) -[2023-10-14 16:52:56,683][75950] Updated weights for policy 1, policy_version 82230 (0.0009) -[2023-10-14 16:52:57,046][75950] Updated weights for policy 1, policy_version 82240 (0.0010) -[2023-10-14 16:52:57,341][75949] Updated weights for policy 0, policy_version 82441 (0.0007) -[2023-10-14 16:52:57,711][75949] Updated weights for policy 0, policy_version 82451 (0.0008) -[2023-10-14 16:52:58,081][75949] Updated weights for policy 0, policy_version 82461 (0.0007) -[2023-10-14 16:52:58,164][74987] Fps is (10 sec: 13106.6, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 168624128. Throughput: 0: 1656.5, 1: 1675.0. Samples: 42169570. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:52:58,165][74987] Avg episode reward: [(0, '28.620'), (1, '34.680')] -[2023-10-14 16:53:01,100][75950] Updated weights for policy 1, policy_version 82250 (0.0010) -[2023-10-14 16:53:01,469][75950] Updated weights for policy 1, policy_version 82260 (0.0008) -[2023-10-14 16:53:01,841][75950] Updated weights for policy 1, policy_version 82270 (0.0008) -[2023-10-14 16:53:02,159][75949] Updated weights for policy 0, policy_version 82471 (0.0008) -[2023-10-14 16:53:02,531][75949] Updated weights for policy 0, policy_version 82481 (0.0008) -[2023-10-14 16:53:02,895][75949] Updated weights for policy 0, policy_version 82491 (0.0010) -[2023-10-14 16:53:03,164][74987] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 168722432. Throughput: 0: 1671.8, 1: 1692.9. Samples: 42180678. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 16:53:03,165][74987] Avg episode reward: [(0, '30.410'), (1, '32.780')] -[2023-10-14 16:53:05,891][75950] Updated weights for policy 1, policy_version 82280 (0.0009) -[2023-10-14 16:53:06,258][75950] Updated weights for policy 1, policy_version 82290 (0.0007) -[2023-10-14 16:53:06,623][75950] Updated weights for policy 1, policy_version 82300 (0.0007) -[2023-10-14 16:53:06,957][75949] Updated weights for policy 0, policy_version 82501 (0.0009) -[2023-10-14 16:53:07,318][75949] Updated weights for policy 0, policy_version 82511 (0.0008) -[2023-10-14 16:53:07,686][75949] Updated weights for policy 0, policy_version 82521 (0.0008) -[2023-10-14 16:53:08,164][74987] Fps is (10 sec: 16384.4, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 168787968. Throughput: 0: 1674.5, 1: 1665.6. Samples: 42200170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:53:08,164][74987] Avg episode reward: [(0, '25.760'), (1, '35.880')] -[2023-10-14 16:53:10,561][75950] Updated weights for policy 1, policy_version 82310 (0.0007) -[2023-10-14 16:53:10,934][75950] Updated weights for policy 1, policy_version 82320 (0.0008) -[2023-10-14 16:53:11,311][75950] Updated weights for policy 1, policy_version 82330 (0.0010) -[2023-10-14 16:53:11,695][75949] Updated weights for policy 0, policy_version 82531 (0.0010) -[2023-10-14 16:53:12,068][75949] Updated weights for policy 0, policy_version 82541 (0.0009) -[2023-10-14 16:53:12,438][75949] Updated weights for policy 0, policy_version 82551 (0.0008) -[2023-10-14 16:53:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 168853504. Throughput: 0: 1654.7, 1: 1685.0. Samples: 42219522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:53:13,165][74987] Avg episode reward: [(0, '27.190'), (1, '34.200')] -[2023-10-14 16:53:15,360][75950] Updated weights for policy 1, policy_version 82340 (0.0008) -[2023-10-14 16:53:15,729][75950] Updated weights for policy 1, policy_version 82350 (0.0009) -[2023-10-14 16:53:16,097][75950] Updated weights for policy 1, policy_version 82360 (0.0010) -[2023-10-14 16:53:16,479][75949] Updated weights for policy 0, policy_version 82561 (0.0009) -[2023-10-14 16:53:16,854][75949] Updated weights for policy 0, policy_version 82571 (0.0009) -[2023-10-14 16:53:17,215][75949] Updated weights for policy 0, policy_version 82581 (0.0009) -[2023-10-14 16:53:17,590][75949] Updated weights for policy 0, policy_version 82591 (0.0010) -[2023-10-14 16:53:18,163][74987] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 168919040. Throughput: 0: 1679.6, 1: 1672.0. Samples: 42230544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:53:18,164][74987] Avg episode reward: [(0, '25.830'), (1, '32.800')] -[2023-10-14 16:53:20,311][75950] Updated weights for policy 1, policy_version 82370 (0.0009) -[2023-10-14 16:53:20,674][75950] Updated weights for policy 1, policy_version 82380 (0.0009) -[2023-10-14 16:53:21,050][75950] Updated weights for policy 1, policy_version 82390 (0.0009) -[2023-10-14 16:53:21,410][75950] Updated weights for policy 1, policy_version 82400 (0.0007) -[2023-10-14 16:53:21,648][75949] Updated weights for policy 0, policy_version 82601 (0.0009) -[2023-10-14 16:53:22,020][75949] Updated weights for policy 0, policy_version 82611 (0.0007) -[2023-10-14 16:53:22,390][75949] Updated weights for policy 0, policy_version 82621 (0.0008) -[2023-10-14 16:53:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 168984576. Throughput: 0: 1670.4, 1: 1666.9. Samples: 42250200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:53:23,164][74987] Avg episode reward: [(0, '25.250'), (1, '34.760')] -[2023-10-14 16:53:25,598][75950] Updated weights for policy 1, policy_version 82410 (0.0008) -[2023-10-14 16:53:25,968][75950] Updated weights for policy 1, policy_version 82420 (0.0007) -[2023-10-14 16:53:26,330][75950] Updated weights for policy 1, policy_version 82430 (0.0009) -[2023-10-14 16:53:26,391][75949] Updated weights for policy 0, policy_version 82631 (0.0008) -[2023-10-14 16:53:26,756][75949] Updated weights for policy 0, policy_version 82641 (0.0007) -[2023-10-14 16:53:27,124][75949] Updated weights for policy 0, policy_version 82651 (0.0008) -[2023-10-14 16:53:28,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 169050112. Throughput: 0: 1663.1, 1: 1687.8. Samples: 42270026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:53:28,165][74987] Avg episode reward: [(0, '28.720'), (1, '34.450')] -[2023-10-14 16:53:30,385][75950] Updated weights for policy 1, policy_version 82440 (0.0008) -[2023-10-14 16:53:30,757][75950] Updated weights for policy 1, policy_version 82450 (0.0010) -[2023-10-14 16:53:31,119][75950] Updated weights for policy 1, policy_version 82460 (0.0008) -[2023-10-14 16:53:31,331][75949] Updated weights for policy 0, policy_version 82661 (0.0008) -[2023-10-14 16:53:31,711][75949] Updated weights for policy 0, policy_version 82671 (0.0008) -[2023-10-14 16:53:32,083][75949] Updated weights for policy 0, policy_version 82681 (0.0008) -[2023-10-14 16:53:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 169115648. Throughput: 0: 1687.2, 1: 1674.1. Samples: 42281146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:53:33,165][74987] Avg episode reward: [(0, '25.740'), (1, '32.360')] -[2023-10-14 16:53:34,992][75950] Updated weights for policy 1, policy_version 82470 (0.0009) -[2023-10-14 16:53:35,364][75950] Updated weights for policy 1, policy_version 82480 (0.0010) -[2023-10-14 16:53:35,734][75950] Updated weights for policy 1, policy_version 82490 (0.0009) -[2023-10-14 16:53:36,129][75949] Updated weights for policy 0, policy_version 82691 (0.0008) -[2023-10-14 16:53:36,495][75949] Updated weights for policy 0, policy_version 82701 (0.0009) -[2023-10-14 16:53:36,865][75949] Updated weights for policy 0, policy_version 82711 (0.0007) -[2023-10-14 16:53:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 169181184. Throughput: 0: 1669.8, 1: 1678.8. Samples: 42300522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:53:38,164][74987] Avg episode reward: [(0, '31.720'), (1, '34.260')] -[2023-10-14 16:53:39,714][75950] Updated weights for policy 1, policy_version 82500 (0.0009) -[2023-10-14 16:53:40,083][75950] Updated weights for policy 1, policy_version 82510 (0.0009) -[2023-10-14 16:53:40,449][75950] Updated weights for policy 1, policy_version 82520 (0.0009) -[2023-10-14 16:53:40,915][75949] Updated weights for policy 0, policy_version 82721 (0.0007) -[2023-10-14 16:53:41,279][75949] Updated weights for policy 0, policy_version 82731 (0.0008) -[2023-10-14 16:53:41,655][75949] Updated weights for policy 0, policy_version 82741 (0.0008) -[2023-10-14 16:53:42,034][75949] Updated weights for policy 0, policy_version 82751 (0.0009) -[2023-10-14 16:53:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 169246720. Throughput: 0: 1673.4, 1: 1687.0. Samples: 42320786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:53:43,165][74987] Avg episode reward: [(0, '25.270'), (1, '34.240')] -[2023-10-14 16:53:44,620][75950] Updated weights for policy 1, policy_version 82530 (0.0008) -[2023-10-14 16:53:44,981][75950] Updated weights for policy 1, policy_version 82540 (0.0010) -[2023-10-14 16:53:45,356][75950] Updated weights for policy 1, policy_version 82550 (0.0010) -[2023-10-14 16:53:45,722][75950] Updated weights for policy 1, policy_version 82560 (0.0010) -[2023-10-14 16:53:46,058][75949] Updated weights for policy 0, policy_version 82761 (0.0009) -[2023-10-14 16:53:46,417][75949] Updated weights for policy 0, policy_version 82771 (0.0007) -[2023-10-14 16:53:46,781][75949] Updated weights for policy 0, policy_version 82781 (0.0008) -[2023-10-14 16:53:48,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 169312256. Throughput: 0: 1688.6, 1: 1660.4. Samples: 42331380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:53:48,164][74987] Avg episode reward: [(0, '31.410'), (1, '34.040')] -[2023-10-14 16:53:49,737][75950] Updated weights for policy 1, policy_version 82570 (0.0009) -[2023-10-14 16:53:50,102][75950] Updated weights for policy 1, policy_version 82580 (0.0009) -[2023-10-14 16:53:50,475][75950] Updated weights for policy 1, policy_version 82590 (0.0007) -[2023-10-14 16:53:50,756][75949] Updated weights for policy 0, policy_version 82791 (0.0009) -[2023-10-14 16:53:51,129][75949] Updated weights for policy 0, policy_version 82801 (0.0011) -[2023-10-14 16:53:51,501][75949] Updated weights for policy 0, policy_version 82811 (0.0009) -[2023-10-14 16:53:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 169377792. Throughput: 0: 1662.3, 1: 1683.4. Samples: 42350726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:53:53,164][74987] Avg episode reward: [(0, '25.270'), (1, '32.380')] -[2023-10-14 16:53:54,612][75950] Updated weights for policy 1, policy_version 82600 (0.0010) -[2023-10-14 16:53:54,983][75950] Updated weights for policy 1, policy_version 82610 (0.0009) -[2023-10-14 16:53:55,351][75950] Updated weights for policy 1, policy_version 82620 (0.0007) -[2023-10-14 16:53:55,585][75949] Updated weights for policy 0, policy_version 82821 (0.0009) -[2023-10-14 16:53:55,953][75949] Updated weights for policy 0, policy_version 82831 (0.0007) -[2023-10-14 16:53:56,327][75949] Updated weights for policy 0, policy_version 82841 (0.0009) -[2023-10-14 16:53:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 169443328. Throughput: 0: 1686.9, 1: 1685.3. Samples: 42371272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 16:53:58,164][74987] Avg episode reward: [(0, '32.140'), (1, '34.190')] -[2023-10-14 16:53:59,423][75950] Updated weights for policy 1, policy_version 82630 (0.0007) -[2023-10-14 16:53:59,789][75950] Updated weights for policy 1, policy_version 82640 (0.0009) -[2023-10-14 16:54:00,157][75950] Updated weights for policy 1, policy_version 82650 (0.0008) -[2023-10-14 16:54:00,397][75949] Updated weights for policy 0, policy_version 82851 (0.0008) -[2023-10-14 16:54:00,771][75949] Updated weights for policy 0, policy_version 82861 (0.0008) -[2023-10-14 16:54:01,140][75949] Updated weights for policy 0, policy_version 82871 (0.0008) -[2023-10-14 16:54:03,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 169508864. Throughput: 0: 1678.4, 1: 1672.5. Samples: 42381336. Policy #0 lag: (min: 4.0, avg: 5.0, max: 26.0) -[2023-10-14 16:54:03,165][74987] Avg episode reward: [(0, '26.150'), (1, '35.350')] -[2023-10-14 16:54:04,280][75950] Updated weights for policy 1, policy_version 82660 (0.0008) -[2023-10-14 16:54:04,641][75950] Updated weights for policy 1, policy_version 82670 (0.0008) -[2023-10-14 16:54:05,011][75950] Updated weights for policy 1, policy_version 82680 (0.0008) -[2023-10-14 16:54:05,232][75949] Updated weights for policy 0, policy_version 82881 (0.0009) -[2023-10-14 16:54:05,594][75949] Updated weights for policy 0, policy_version 82891 (0.0007) -[2023-10-14 16:54:05,964][75949] Updated weights for policy 0, policy_version 82901 (0.0007) -[2023-10-14 16:54:06,327][75949] Updated weights for policy 0, policy_version 82911 (0.0011) -[2023-10-14 16:54:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 169574400. Throughput: 0: 1664.2, 1: 1688.2. Samples: 42401060. Policy #0 lag: (min: 4.0, avg: 5.0, max: 26.0) -[2023-10-14 16:54:08,164][74987] Avg episode reward: [(0, '30.140'), (1, '34.110')] -[2023-10-14 16:54:09,101][75950] Updated weights for policy 1, policy_version 82690 (0.0009) -[2023-10-14 16:54:09,474][75950] Updated weights for policy 1, policy_version 82700 (0.0009) -[2023-10-14 16:54:09,843][75950] Updated weights for policy 1, policy_version 82710 (0.0010) -[2023-10-14 16:54:10,207][75950] Updated weights for policy 1, policy_version 82720 (0.0008) -[2023-10-14 16:54:10,457][75949] Updated weights for policy 0, policy_version 82921 (0.0007) -[2023-10-14 16:54:10,832][75949] Updated weights for policy 0, policy_version 82931 (0.0009) -[2023-10-14 16:54:11,192][75949] Updated weights for policy 0, policy_version 82941 (0.0010) -[2023-10-14 16:54:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 169639936. Throughput: 0: 1681.2, 1: 1693.6. Samples: 42421892. Policy #0 lag: (min: 4.0, avg: 5.0, max: 26.0) -[2023-10-14 16:54:13,165][74987] Avg episode reward: [(0, '25.110'), (1, '32.610')] -[2023-10-14 16:54:14,241][75950] Updated weights for policy 1, policy_version 82730 (0.0009) -[2023-10-14 16:54:14,620][75950] Updated weights for policy 1, policy_version 82740 (0.0007) -[2023-10-14 16:54:14,980][75950] Updated weights for policy 1, policy_version 82750 (0.0009) -[2023-10-14 16:54:15,124][75949] Updated weights for policy 0, policy_version 82951 (0.0010) -[2023-10-14 16:54:15,496][75949] Updated weights for policy 0, policy_version 82961 (0.0008) -[2023-10-14 16:54:15,865][75949] Updated weights for policy 0, policy_version 82971 (0.0007) -[2023-10-14 16:54:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169705472. Throughput: 0: 1662.8, 1: 1675.3. Samples: 42431362. Policy #0 lag: (min: 4.0, avg: 5.0, max: 26.0) -[2023-10-14 16:54:18,164][74987] Avg episode reward: [(0, '29.810'), (1, '35.240')] -[2023-10-14 16:54:19,166][75950] Updated weights for policy 1, policy_version 82760 (0.0011) -[2023-10-14 16:54:19,536][75950] Updated weights for policy 1, policy_version 82770 (0.0010) -[2023-10-14 16:54:19,888][75949] Updated weights for policy 0, policy_version 82981 (0.0008) -[2023-10-14 16:54:19,896][75950] Updated weights for policy 1, policy_version 82780 (0.0008) -[2023-10-14 16:54:20,262][75949] Updated weights for policy 0, policy_version 82991 (0.0007) -[2023-10-14 16:54:20,635][75949] Updated weights for policy 0, policy_version 83001 (0.0007) -[2023-10-14 16:54:23,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169771008. Throughput: 0: 1669.6, 1: 1685.6. Samples: 42451504. Policy #0 lag: (min: 4.0, avg: 5.0, max: 26.0) -[2023-10-14 16:54:23,164][74987] Avg episode reward: [(0, '26.370'), (1, '33.480')] -[2023-10-14 16:54:23,801][75950] Updated weights for policy 1, policy_version 82790 (0.0010) -[2023-10-14 16:54:24,159][75950] Updated weights for policy 1, policy_version 82800 (0.0009) -[2023-10-14 16:54:24,526][75950] Updated weights for policy 1, policy_version 82810 (0.0008) -[2023-10-14 16:54:24,662][75949] Updated weights for policy 0, policy_version 83011 (0.0009) -[2023-10-14 16:54:25,049][75949] Updated weights for policy 0, policy_version 83021 (0.0007) -[2023-10-14 16:54:25,422][75949] Updated weights for policy 0, policy_version 83031 (0.0008) -[2023-10-14 16:54:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169836544. Throughput: 0: 1683.2, 1: 1679.8. Samples: 42472120. Policy #0 lag: (min: 4.0, avg: 5.0, max: 26.0) -[2023-10-14 16:54:28,164][74987] Avg episode reward: [(0, '29.420'), (1, '30.680')] -[2023-10-14 16:54:28,814][75950] Updated weights for policy 1, policy_version 82820 (0.0008) -[2023-10-14 16:54:29,180][75950] Updated weights for policy 1, policy_version 82830 (0.0009) -[2023-10-14 16:54:29,492][75949] Updated weights for policy 0, policy_version 83041 (0.0008) -[2023-10-14 16:54:29,554][75950] Updated weights for policy 1, policy_version 82840 (0.0009) -[2023-10-14 16:54:29,854][75949] Updated weights for policy 0, policy_version 83051 (0.0010) -[2023-10-14 16:54:30,220][75949] Updated weights for policy 0, policy_version 83061 (0.0010) -[2023-10-14 16:54:30,592][75949] Updated weights for policy 0, policy_version 83071 (0.0010) -[2023-10-14 16:54:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169902080. Throughput: 0: 1654.9, 1: 1677.3. Samples: 42481328. Policy #0 lag: (min: 4.0, avg: 5.0, max: 26.0) -[2023-10-14 16:54:33,164][74987] Avg episode reward: [(0, '26.610'), (1, '32.150')] -[2023-10-14 16:54:33,584][75950] Updated weights for policy 1, policy_version 82850 (0.0009) -[2023-10-14 16:54:33,949][75950] Updated weights for policy 1, policy_version 82860 (0.0009) -[2023-10-14 16:54:34,308][75950] Updated weights for policy 1, policy_version 82870 (0.0007) -[2023-10-14 16:54:34,598][75949] Updated weights for policy 0, policy_version 83081 (0.0008) -[2023-10-14 16:54:34,682][75950] Updated weights for policy 1, policy_version 82880 (0.0007) -[2023-10-14 16:54:34,968][75949] Updated weights for policy 0, policy_version 83091 (0.0008) -[2023-10-14 16:54:35,346][75949] Updated weights for policy 0, policy_version 83101 (0.0010) -[2023-10-14 16:54:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169967616. Throughput: 0: 1686.8, 1: 1678.7. Samples: 42502176. Policy #0 lag: (min: 4.0, avg: 5.0, max: 26.0) -[2023-10-14 16:54:38,165][74987] Avg episode reward: [(0, '27.600'), (1, '35.760')] -[2023-10-14 16:54:38,821][75950] Updated weights for policy 1, policy_version 82890 (0.0008) -[2023-10-14 16:54:39,188][75950] Updated weights for policy 1, policy_version 82900 (0.0009) -[2023-10-14 16:54:39,356][75949] Updated weights for policy 0, policy_version 83111 (0.0009) -[2023-10-14 16:54:39,556][75950] Updated weights for policy 1, policy_version 82910 (0.0009) -[2023-10-14 16:54:39,725][75949] Updated weights for policy 0, policy_version 83121 (0.0009) -[2023-10-14 16:54:40,102][75949] Updated weights for policy 0, policy_version 83131 (0.0009) -[2023-10-14 16:54:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 170033152. Throughput: 0: 1694.9, 1: 1680.2. Samples: 42523152. Policy #0 lag: (min: 4.0, avg: 5.0, max: 26.0) -[2023-10-14 16:54:43,164][74987] Avg episode reward: [(0, '25.500'), (1, '31.600')] -[2023-10-14 16:54:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000083136_85131264.pth... -[2023-10-14 16:54:43,212][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000081568_83525632.pth -[2023-10-14 16:54:43,464][75950] Updated weights for policy 1, policy_version 82920 (0.0010) -[2023-10-14 16:54:43,846][75950] Updated weights for policy 1, policy_version 82930 (0.0007) -[2023-10-14 16:54:44,211][75950] Updated weights for policy 1, policy_version 82940 (0.0007) -[2023-10-14 16:54:44,263][75949] Updated weights for policy 0, policy_version 83141 (0.0008) -[2023-10-14 16:54:44,353][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000082944_84934656.pth... -[2023-10-14 16:54:44,393][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000081344_83296256.pth -[2023-10-14 16:54:44,637][75949] Updated weights for policy 0, policy_version 83151 (0.0008) -[2023-10-14 16:54:44,996][75949] Updated weights for policy 0, policy_version 83161 (0.0007) -[2023-10-14 16:54:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 170098688. Throughput: 0: 1672.8, 1: 1681.6. Samples: 42532282. Policy #0 lag: (min: 4.0, avg: 5.0, max: 26.0) -[2023-10-14 16:54:48,165][74987] Avg episode reward: [(0, '25.090'), (1, '33.620')] -[2023-10-14 16:54:48,363][75950] Updated weights for policy 1, policy_version 82950 (0.0009) -[2023-10-14 16:54:48,733][75950] Updated weights for policy 1, policy_version 82960 (0.0008) -[2023-10-14 16:54:49,091][75950] Updated weights for policy 1, policy_version 82970 (0.0008) -[2023-10-14 16:54:49,092][75949] Updated weights for policy 0, policy_version 83171 (0.0008) -[2023-10-14 16:54:49,450][75949] Updated weights for policy 0, policy_version 83181 (0.0008) -[2023-10-14 16:54:49,825][75949] Updated weights for policy 0, policy_version 83191 (0.0009) -[2023-10-14 16:54:52,975][75950] Updated weights for policy 1, policy_version 82980 (0.0008) -[2023-10-14 16:54:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170164224. Throughput: 0: 1686.4, 1: 1678.7. Samples: 42552490. Policy #0 lag: (min: 4.0, avg: 5.0, max: 26.0) -[2023-10-14 16:54:53,164][74987] Avg episode reward: [(0, '29.390'), (1, '35.660')] -[2023-10-14 16:54:53,335][75950] Updated weights for policy 1, policy_version 82990 (0.0008) -[2023-10-14 16:54:53,697][75950] Updated weights for policy 1, policy_version 83000 (0.0011) -[2023-10-14 16:54:53,893][75949] Updated weights for policy 0, policy_version 83201 (0.0011) -[2023-10-14 16:54:54,260][75949] Updated weights for policy 0, policy_version 83211 (0.0009) -[2023-10-14 16:54:54,627][75949] Updated weights for policy 0, policy_version 83221 (0.0008) -[2023-10-14 16:54:55,001][75949] Updated weights for policy 0, policy_version 83231 (0.0007) -[2023-10-14 16:54:57,770][75950] Updated weights for policy 1, policy_version 83010 (0.0010) -[2023-10-14 16:54:58,136][75950] Updated weights for policy 1, policy_version 83020 (0.0011) -[2023-10-14 16:54:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170229760. Throughput: 0: 1683.3, 1: 1680.6. Samples: 42573268. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 16:54:58,164][74987] Avg episode reward: [(0, '26.010'), (1, '34.350')] -[2023-10-14 16:54:58,508][75950] Updated weights for policy 1, policy_version 83030 (0.0008) -[2023-10-14 16:54:58,874][75950] Updated weights for policy 1, policy_version 83040 (0.0011) -[2023-10-14 16:54:59,142][75949] Updated weights for policy 0, policy_version 83241 (0.0009) -[2023-10-14 16:54:59,509][75949] Updated weights for policy 0, policy_version 83251 (0.0009) -[2023-10-14 16:54:59,879][75949] Updated weights for policy 0, policy_version 83261 (0.0009) -[2023-10-14 16:55:03,009][75950] Updated weights for policy 1, policy_version 83050 (0.0008) -[2023-10-14 16:55:03,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 170295296. Throughput: 0: 1672.2, 1: 1684.7. Samples: 42582424. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 16:55:03,164][74987] Avg episode reward: [(0, '30.820'), (1, '33.750')] -[2023-10-14 16:55:03,371][75950] Updated weights for policy 1, policy_version 83060 (0.0009) -[2023-10-14 16:55:03,743][75950] Updated weights for policy 1, policy_version 83070 (0.0007) -[2023-10-14 16:55:04,107][75949] Updated weights for policy 0, policy_version 83271 (0.0010) -[2023-10-14 16:55:04,482][75949] Updated weights for policy 0, policy_version 83281 (0.0008) -[2023-10-14 16:55:04,858][75949] Updated weights for policy 0, policy_version 83291 (0.0009) -[2023-10-14 16:55:07,878][75950] Updated weights for policy 1, policy_version 83080 (0.0008) -[2023-10-14 16:55:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170360832. Throughput: 0: 1680.8, 1: 1683.6. Samples: 42602906. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 16:55:08,164][74987] Avg episode reward: [(0, '26.380'), (1, '34.580')] -[2023-10-14 16:55:08,246][75950] Updated weights for policy 1, policy_version 83090 (0.0009) -[2023-10-14 16:55:08,613][75950] Updated weights for policy 1, policy_version 83100 (0.0009) -[2023-10-14 16:55:08,897][75949] Updated weights for policy 0, policy_version 83301 (0.0008) -[2023-10-14 16:55:09,267][75949] Updated weights for policy 0, policy_version 83311 (0.0010) -[2023-10-14 16:55:09,632][75949] Updated weights for policy 0, policy_version 83321 (0.0009) -[2023-10-14 16:55:12,691][75950] Updated weights for policy 1, policy_version 83110 (0.0007) -[2023-10-14 16:55:13,058][75950] Updated weights for policy 1, policy_version 83120 (0.0009) -[2023-10-14 16:55:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170426368. Throughput: 0: 1682.8, 1: 1687.1. Samples: 42623762. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 16:55:13,164][74987] Avg episode reward: [(0, '31.390'), (1, '34.920')] -[2023-10-14 16:55:13,421][75950] Updated weights for policy 1, policy_version 83130 (0.0011) -[2023-10-14 16:55:13,865][75949] Updated weights for policy 0, policy_version 83331 (0.0010) -[2023-10-14 16:55:14,259][75949] Updated weights for policy 0, policy_version 83341 (0.0009) -[2023-10-14 16:55:14,625][75949] Updated weights for policy 0, policy_version 83351 (0.0010) -[2023-10-14 16:55:17,484][75950] Updated weights for policy 1, policy_version 83140 (0.0010) -[2023-10-14 16:55:17,856][75950] Updated weights for policy 1, policy_version 83150 (0.0008) -[2023-10-14 16:55:18,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170491904. Throughput: 0: 1675.7, 1: 1692.9. Samples: 42632916. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 16:55:18,164][74987] Avg episode reward: [(0, '26.930'), (1, '32.790')] -[2023-10-14 16:55:18,222][75950] Updated weights for policy 1, policy_version 83160 (0.0010) -[2023-10-14 16:55:18,625][75949] Updated weights for policy 0, policy_version 83361 (0.0008) -[2023-10-14 16:55:18,988][75949] Updated weights for policy 0, policy_version 83371 (0.0011) -[2023-10-14 16:55:19,368][75949] Updated weights for policy 0, policy_version 83381 (0.0008) -[2023-10-14 16:55:19,742][75949] Updated weights for policy 0, policy_version 83391 (0.0009) -[2023-10-14 16:55:22,279][75950] Updated weights for policy 1, policy_version 83170 (0.0008) -[2023-10-14 16:55:22,643][75950] Updated weights for policy 1, policy_version 83180 (0.0007) -[2023-10-14 16:55:23,012][75950] Updated weights for policy 1, policy_version 83190 (0.0007) -[2023-10-14 16:55:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170557440. Throughput: 0: 1673.2, 1: 1691.6. Samples: 42653592. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 16:55:23,164][74987] Avg episode reward: [(0, '31.170'), (1, '31.960')] -[2023-10-14 16:55:23,381][75950] Updated weights for policy 1, policy_version 83200 (0.0007) -[2023-10-14 16:55:23,701][75949] Updated weights for policy 0, policy_version 83401 (0.0008) -[2023-10-14 16:55:24,074][75949] Updated weights for policy 0, policy_version 83411 (0.0008) -[2023-10-14 16:55:24,440][75949] Updated weights for policy 0, policy_version 83421 (0.0008) -[2023-10-14 16:55:27,383][75950] Updated weights for policy 1, policy_version 83210 (0.0008) -[2023-10-14 16:55:27,745][75950] Updated weights for policy 1, policy_version 83220 (0.0008) -[2023-10-14 16:55:28,117][75950] Updated weights for policy 1, policy_version 83230 (0.0009) -[2023-10-14 16:55:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170622976. Throughput: 0: 1671.3, 1: 1677.6. Samples: 42673850. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 16:55:28,164][74987] Avg episode reward: [(0, '26.640'), (1, '37.230')] -[2023-10-14 16:55:28,377][75949] Updated weights for policy 0, policy_version 83431 (0.0008) -[2023-10-14 16:55:28,750][75949] Updated weights for policy 0, policy_version 83441 (0.0008) -[2023-10-14 16:55:29,115][75949] Updated weights for policy 0, policy_version 83451 (0.0009) -[2023-10-14 16:55:31,963][75950] Updated weights for policy 1, policy_version 83240 (0.0010) -[2023-10-14 16:55:32,335][75950] Updated weights for policy 1, policy_version 83250 (0.0008) -[2023-10-14 16:55:32,702][75950] Updated weights for policy 1, policy_version 83260 (0.0008) -[2023-10-14 16:55:33,126][75949] Updated weights for policy 0, policy_version 83461 (0.0009) -[2023-10-14 16:55:33,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 170721280. Throughput: 0: 1671.6, 1: 1693.3. Samples: 42683702. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 16:55:33,164][74987] Avg episode reward: [(0, '28.960'), (1, '32.400')] -[2023-10-14 16:55:33,488][75949] Updated weights for policy 0, policy_version 83471 (0.0008) -[2023-10-14 16:55:33,864][75949] Updated weights for policy 0, policy_version 83481 (0.0008) -[2023-10-14 16:55:36,943][75950] Updated weights for policy 1, policy_version 83270 (0.0010) -[2023-10-14 16:55:37,315][75950] Updated weights for policy 1, policy_version 83280 (0.0009) -[2023-10-14 16:55:37,671][75950] Updated weights for policy 1, policy_version 83290 (0.0010) -[2023-10-14 16:55:37,899][75949] Updated weights for policy 0, policy_version 83491 (0.0010) -[2023-10-14 16:55:38,163][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 170786816. Throughput: 0: 1683.4, 1: 1691.1. Samples: 42704342. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 16:55:38,164][74987] Avg episode reward: [(0, '26.180'), (1, '32.770')] -[2023-10-14 16:55:38,271][75949] Updated weights for policy 0, policy_version 83501 (0.0007) -[2023-10-14 16:55:38,646][75949] Updated weights for policy 0, policy_version 83511 (0.0009) -[2023-10-14 16:55:41,779][75950] Updated weights for policy 1, policy_version 83300 (0.0009) -[2023-10-14 16:55:42,146][75950] Updated weights for policy 1, policy_version 83310 (0.0010) -[2023-10-14 16:55:42,527][75950] Updated weights for policy 1, policy_version 83320 (0.0009) -[2023-10-14 16:55:42,580][75949] Updated weights for policy 0, policy_version 83521 (0.0011) -[2023-10-14 16:55:42,947][75949] Updated weights for policy 0, policy_version 83531 (0.0009) -[2023-10-14 16:55:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 170852352. Throughput: 0: 1692.6, 1: 1661.0. Samples: 42724178. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 16:55:43,164][74987] Avg episode reward: [(0, '29.050'), (1, '34.230')] -[2023-10-14 16:55:43,311][75949] Updated weights for policy 0, policy_version 83541 (0.0010) -[2023-10-14 16:55:43,675][75949] Updated weights for policy 0, policy_version 83551 (0.0008) -[2023-10-14 16:55:46,380][75950] Updated weights for policy 1, policy_version 83330 (0.0009) -[2023-10-14 16:55:46,751][75950] Updated weights for policy 1, policy_version 83340 (0.0010) -[2023-10-14 16:55:47,118][75950] Updated weights for policy 1, policy_version 83350 (0.0008) -[2023-10-14 16:55:47,485][75950] Updated weights for policy 1, policy_version 83360 (0.0009) -[2023-10-14 16:55:47,713][75949] Updated weights for policy 0, policy_version 83561 (0.0008) -[2023-10-14 16:55:48,083][75949] Updated weights for policy 0, policy_version 83571 (0.0008) -[2023-10-14 16:55:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 170917888. Throughput: 0: 1695.0, 1: 1683.3. Samples: 42734446. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) -[2023-10-14 16:55:48,165][74987] Avg episode reward: [(0, '27.390'), (1, '31.710')] -[2023-10-14 16:55:48,461][75949] Updated weights for policy 0, policy_version 83581 (0.0011) -[2023-10-14 16:55:51,752][75950] Updated weights for policy 1, policy_version 83370 (0.0008) -[2023-10-14 16:55:52,116][75950] Updated weights for policy 1, policy_version 83380 (0.0007) -[2023-10-14 16:55:52,478][75950] Updated weights for policy 1, policy_version 83390 (0.0007) -[2023-10-14 16:55:52,684][75949] Updated weights for policy 0, policy_version 83591 (0.0008) -[2023-10-14 16:55:53,070][75949] Updated weights for policy 0, policy_version 83601 (0.0010) -[2023-10-14 16:55:53,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 170983424. Throughput: 0: 1695.6, 1: 1676.2. Samples: 42754638. Policy #0 lag: (min: 27.0, avg: 31.1, max: 59.0) -[2023-10-14 16:55:53,164][74987] Avg episode reward: [(0, '29.230'), (1, '31.470')] -[2023-10-14 16:55:53,439][75949] Updated weights for policy 0, policy_version 83611 (0.0009) -[2023-10-14 16:55:56,498][75950] Updated weights for policy 1, policy_version 83400 (0.0010) -[2023-10-14 16:55:56,859][75950] Updated weights for policy 1, policy_version 83410 (0.0011) -[2023-10-14 16:55:57,218][75950] Updated weights for policy 1, policy_version 83420 (0.0009) -[2023-10-14 16:55:57,489][75949] Updated weights for policy 0, policy_version 83621 (0.0008) -[2023-10-14 16:55:57,863][75949] Updated weights for policy 0, policy_version 83631 (0.0011) -[2023-10-14 16:55:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171048960. Throughput: 0: 1682.4, 1: 1653.4. Samples: 42773870. Policy #0 lag: (min: 27.0, avg: 31.1, max: 59.0) -[2023-10-14 16:55:58,165][74987] Avg episode reward: [(0, '28.670'), (1, '34.840')] -[2023-10-14 16:55:58,233][75949] Updated weights for policy 0, policy_version 83641 (0.0011) -[2023-10-14 16:56:01,364][75950] Updated weights for policy 1, policy_version 83430 (0.0010) -[2023-10-14 16:56:01,724][75950] Updated weights for policy 1, policy_version 83440 (0.0010) -[2023-10-14 16:56:02,096][75950] Updated weights for policy 1, policy_version 83450 (0.0010) -[2023-10-14 16:56:02,273][75949] Updated weights for policy 0, policy_version 83651 (0.0011) -[2023-10-14 16:56:02,655][75949] Updated weights for policy 0, policy_version 83661 (0.0010) -[2023-10-14 16:56:03,027][75949] Updated weights for policy 0, policy_version 83671 (0.0009) -[2023-10-14 16:56:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171114496. Throughput: 0: 1696.0, 1: 1677.2. Samples: 42784706. Policy #0 lag: (min: 27.0, avg: 31.1, max: 59.0) -[2023-10-14 16:56:03,164][74987] Avg episode reward: [(0, '28.560'), (1, '34.490')] -[2023-10-14 16:56:06,232][75950] Updated weights for policy 1, policy_version 83460 (0.0009) -[2023-10-14 16:56:06,600][75950] Updated weights for policy 1, policy_version 83470 (0.0009) -[2023-10-14 16:56:06,966][75950] Updated weights for policy 1, policy_version 83480 (0.0008) -[2023-10-14 16:56:07,113][75949] Updated weights for policy 0, policy_version 83681 (0.0009) -[2023-10-14 16:56:07,475][75949] Updated weights for policy 0, policy_version 83691 (0.0010) -[2023-10-14 16:56:07,841][75949] Updated weights for policy 0, policy_version 83701 (0.0010) -[2023-10-14 16:56:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171180032. Throughput: 0: 1689.7, 1: 1666.0. Samples: 42804596. Policy #0 lag: (min: 27.0, avg: 31.1, max: 59.0) -[2023-10-14 16:56:08,164][74987] Avg episode reward: [(0, '28.780'), (1, '34.360')] -[2023-10-14 16:56:08,217][75949] Updated weights for policy 0, policy_version 83711 (0.0011) -[2023-10-14 16:56:11,245][75950] Updated weights for policy 1, policy_version 83490 (0.0008) -[2023-10-14 16:56:11,612][75950] Updated weights for policy 1, policy_version 83500 (0.0007) -[2023-10-14 16:56:11,979][75950] Updated weights for policy 1, policy_version 83510 (0.0007) -[2023-10-14 16:56:12,350][75950] Updated weights for policy 1, policy_version 83520 (0.0009) -[2023-10-14 16:56:12,505][75949] Updated weights for policy 0, policy_version 83721 (0.0010) -[2023-10-14 16:56:12,872][75949] Updated weights for policy 0, policy_version 83731 (0.0008) -[2023-10-14 16:56:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171245568. Throughput: 0: 1671.4, 1: 1662.0. Samples: 42823852. Policy #0 lag: (min: 27.0, avg: 31.1, max: 59.0) -[2023-10-14 16:56:13,164][74987] Avg episode reward: [(0, '27.790'), (1, '34.930')] -[2023-10-14 16:56:13,243][75949] Updated weights for policy 0, policy_version 83741 (0.0009) -[2023-10-14 16:56:16,336][75950] Updated weights for policy 1, policy_version 83530 (0.0009) -[2023-10-14 16:56:16,698][75950] Updated weights for policy 1, policy_version 83540 (0.0010) -[2023-10-14 16:56:17,056][75950] Updated weights for policy 1, policy_version 83550 (0.0010) -[2023-10-14 16:56:17,303][75949] Updated weights for policy 0, policy_version 83751 (0.0009) -[2023-10-14 16:56:17,676][75949] Updated weights for policy 0, policy_version 83761 (0.0009) -[2023-10-14 16:56:18,039][75949] Updated weights for policy 0, policy_version 83771 (0.0008) -[2023-10-14 16:56:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171311104. Throughput: 0: 1685.2, 1: 1673.0. Samples: 42834822. Policy #0 lag: (min: 27.0, avg: 31.1, max: 59.0) -[2023-10-14 16:56:18,164][74987] Avg episode reward: [(0, '30.230'), (1, '34.600')] -[2023-10-14 16:56:21,154][75950] Updated weights for policy 1, policy_version 83560 (0.0011) -[2023-10-14 16:56:21,513][75950] Updated weights for policy 1, policy_version 83570 (0.0008) -[2023-10-14 16:56:21,884][75950] Updated weights for policy 1, policy_version 83580 (0.0008) -[2023-10-14 16:56:22,097][75949] Updated weights for policy 0, policy_version 83781 (0.0007) -[2023-10-14 16:56:22,465][75949] Updated weights for policy 0, policy_version 83791 (0.0007) -[2023-10-14 16:56:22,843][75949] Updated weights for policy 0, policy_version 83801 (0.0008) -[2023-10-14 16:56:23,163][74987] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 171409408. Throughput: 0: 1682.0, 1: 1660.7. Samples: 42854760. Policy #0 lag: (min: 27.0, avg: 31.1, max: 59.0) -[2023-10-14 16:56:23,164][74987] Avg episode reward: [(0, '27.620'), (1, '35.140')] -[2023-10-14 16:56:25,926][75950] Updated weights for policy 1, policy_version 83590 (0.0008) -[2023-10-14 16:56:26,292][75950] Updated weights for policy 1, policy_version 83600 (0.0009) -[2023-10-14 16:56:26,662][75950] Updated weights for policy 1, policy_version 83610 (0.0008) -[2023-10-14 16:56:26,778][75949] Updated weights for policy 0, policy_version 83811 (0.0008) -[2023-10-14 16:56:27,141][75949] Updated weights for policy 0, policy_version 83821 (0.0010) -[2023-10-14 16:56:27,516][75949] Updated weights for policy 0, policy_version 83831 (0.0009) -[2023-10-14 16:56:28,164][74987] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 171474944. Throughput: 0: 1654.2, 1: 1678.0. Samples: 42874128. Policy #0 lag: (min: 27.0, avg: 31.1, max: 59.0) -[2023-10-14 16:56:28,165][74987] Avg episode reward: [(0, '30.340'), (1, '34.470')] -[2023-10-14 16:56:30,655][75950] Updated weights for policy 1, policy_version 83620 (0.0008) -[2023-10-14 16:56:31,014][75950] Updated weights for policy 1, policy_version 83630 (0.0009) -[2023-10-14 16:56:31,382][75950] Updated weights for policy 1, policy_version 83640 (0.0011) -[2023-10-14 16:56:31,513][75949] Updated weights for policy 0, policy_version 83841 (0.0008) -[2023-10-14 16:56:31,885][75949] Updated weights for policy 0, policy_version 83851 (0.0007) -[2023-10-14 16:56:32,244][75949] Updated weights for policy 0, policy_version 83861 (0.0010) -[2023-10-14 16:56:32,619][75949] Updated weights for policy 0, policy_version 83871 (0.0010) -[2023-10-14 16:56:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 171540480. Throughput: 0: 1678.3, 1: 1682.9. Samples: 42885702. Policy #0 lag: (min: 27.0, avg: 31.1, max: 59.0) -[2023-10-14 16:56:33,165][74987] Avg episode reward: [(0, '28.030'), (1, '32.330')] -[2023-10-14 16:56:35,401][75950] Updated weights for policy 1, policy_version 83650 (0.0009) -[2023-10-14 16:56:35,822][75950] Updated weights for policy 1, policy_version 83660 (0.0011) -[2023-10-14 16:56:36,188][75950] Updated weights for policy 1, policy_version 83670 (0.0009) -[2023-10-14 16:56:36,560][75950] Updated weights for policy 1, policy_version 83680 (0.0008) -[2023-10-14 16:56:36,758][75949] Updated weights for policy 0, policy_version 83881 (0.0010) -[2023-10-14 16:56:37,134][75949] Updated weights for policy 0, policy_version 83891 (0.0010) -[2023-10-14 16:56:37,505][75949] Updated weights for policy 0, policy_version 83901 (0.0009) -[2023-10-14 16:56:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 171606016. Throughput: 0: 1674.4, 1: 1663.0. Samples: 42904820. Policy #0 lag: (min: 27.0, avg: 31.1, max: 59.0) -[2023-10-14 16:56:38,164][74987] Avg episode reward: [(0, '29.260'), (1, '35.540')] -[2023-10-14 16:56:40,620][75950] Updated weights for policy 1, policy_version 83690 (0.0010) -[2023-10-14 16:56:40,988][75950] Updated weights for policy 1, policy_version 83700 (0.0010) -[2023-10-14 16:56:41,350][75950] Updated weights for policy 1, policy_version 83710 (0.0009) -[2023-10-14 16:56:41,512][75949] Updated weights for policy 0, policy_version 83911 (0.0009) -[2023-10-14 16:56:41,889][75949] Updated weights for policy 0, policy_version 83921 (0.0009) -[2023-10-14 16:56:42,259][75949] Updated weights for policy 0, policy_version 83931 (0.0007) -[2023-10-14 16:56:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 171671552. Throughput: 0: 1659.0, 1: 1686.8. Samples: 42924432. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 16:56:43,165][74987] Avg episode reward: [(0, '28.040'), (1, '36.150')] -[2023-10-14 16:56:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000083936_85950464.pth... -[2023-10-14 16:56:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000083712_85721088.pth... -[2023-10-14 16:56:43,211][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000082368_84344832.pth -[2023-10-14 16:56:43,216][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000082144_84115456.pth -[2023-10-14 16:56:45,387][75950] Updated weights for policy 1, policy_version 83720 (0.0010) -[2023-10-14 16:56:45,750][75950] Updated weights for policy 1, policy_version 83730 (0.0007) -[2023-10-14 16:56:46,118][75950] Updated weights for policy 1, policy_version 83740 (0.0008) -[2023-10-14 16:56:46,228][75949] Updated weights for policy 0, policy_version 83941 (0.0008) -[2023-10-14 16:56:46,594][75949] Updated weights for policy 0, policy_version 83951 (0.0010) -[2023-10-14 16:56:46,965][75949] Updated weights for policy 0, policy_version 83961 (0.0008) -[2023-10-14 16:56:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 171737088. Throughput: 0: 1681.2, 1: 1673.6. Samples: 42935674. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 16:56:48,164][74987] Avg episode reward: [(0, '28.950'), (1, '33.040')] -[2023-10-14 16:56:50,269][75950] Updated weights for policy 1, policy_version 83750 (0.0008) -[2023-10-14 16:56:50,631][75950] Updated weights for policy 1, policy_version 83760 (0.0008) -[2023-10-14 16:56:51,001][75950] Updated weights for policy 1, policy_version 83770 (0.0008) -[2023-10-14 16:56:51,290][75949] Updated weights for policy 0, policy_version 83971 (0.0009) -[2023-10-14 16:56:51,684][75949] Updated weights for policy 0, policy_version 83981 (0.0009) -[2023-10-14 16:56:52,067][75949] Updated weights for policy 0, policy_version 83991 (0.0010) -[2023-10-14 16:56:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 171802624. Throughput: 0: 1670.2, 1: 1672.7. Samples: 42955024. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 16:56:53,165][74987] Avg episode reward: [(0, '26.830'), (1, '35.360')] -[2023-10-14 16:56:55,109][75950] Updated weights for policy 1, policy_version 83780 (0.0010) -[2023-10-14 16:56:55,484][75950] Updated weights for policy 1, policy_version 83790 (0.0010) -[2023-10-14 16:56:55,844][75950] Updated weights for policy 1, policy_version 83800 (0.0009) -[2023-10-14 16:56:56,097][75949] Updated weights for policy 0, policy_version 84001 (0.0009) -[2023-10-14 16:56:56,461][75949] Updated weights for policy 0, policy_version 84011 (0.0009) -[2023-10-14 16:56:56,837][75949] Updated weights for policy 0, policy_version 84021 (0.0007) -[2023-10-14 16:56:57,204][75949] Updated weights for policy 0, policy_version 84031 (0.0008) -[2023-10-14 16:56:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 171868160. Throughput: 0: 1668.2, 1: 1692.4. Samples: 42975080. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 16:56:58,165][74987] Avg episode reward: [(0, '28.810'), (1, '36.160')] -[2023-10-14 16:56:59,931][75950] Updated weights for policy 1, policy_version 83810 (0.0010) -[2023-10-14 16:57:00,303][75950] Updated weights for policy 1, policy_version 83820 (0.0009) -[2023-10-14 16:57:00,662][75950] Updated weights for policy 1, policy_version 83830 (0.0008) -[2023-10-14 16:57:01,027][75950] Updated weights for policy 1, policy_version 83840 (0.0007) -[2023-10-14 16:57:01,310][75949] Updated weights for policy 0, policy_version 84041 (0.0009) -[2023-10-14 16:57:01,679][75949] Updated weights for policy 0, policy_version 84051 (0.0008) -[2023-10-14 16:57:02,048][75949] Updated weights for policy 0, policy_version 84061 (0.0008) -[2023-10-14 16:57:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 171933696. Throughput: 0: 1683.7, 1: 1673.6. Samples: 42985900. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 16:57:03,164][74987] Avg episode reward: [(0, '27.870'), (1, '32.140')] -[2023-10-14 16:57:05,001][75950] Updated weights for policy 1, policy_version 83850 (0.0009) -[2023-10-14 16:57:05,383][75950] Updated weights for policy 1, policy_version 83860 (0.0008) -[2023-10-14 16:57:05,757][75950] Updated weights for policy 1, policy_version 83870 (0.0008) -[2023-10-14 16:57:06,091][75949] Updated weights for policy 0, policy_version 84071 (0.0007) -[2023-10-14 16:57:06,448][75949] Updated weights for policy 0, policy_version 84081 (0.0009) -[2023-10-14 16:57:06,828][75949] Updated weights for policy 0, policy_version 84091 (0.0009) -[2023-10-14 16:57:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 171999232. Throughput: 0: 1660.6, 1: 1680.4. Samples: 43005106. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 16:57:08,164][74987] Avg episode reward: [(0, '28.770'), (1, '34.320')] -[2023-10-14 16:57:09,860][75950] Updated weights for policy 1, policy_version 83880 (0.0011) -[2023-10-14 16:57:10,227][75950] Updated weights for policy 1, policy_version 83890 (0.0010) -[2023-10-14 16:57:10,602][75950] Updated weights for policy 1, policy_version 83900 (0.0011) -[2023-10-14 16:57:11,001][75949] Updated weights for policy 0, policy_version 84101 (0.0008) -[2023-10-14 16:57:11,361][75949] Updated weights for policy 0, policy_version 84111 (0.0007) -[2023-10-14 16:57:11,735][75949] Updated weights for policy 0, policy_version 84121 (0.0007) -[2023-10-14 16:57:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 172064768. Throughput: 0: 1673.8, 1: 1686.9. Samples: 43025362. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 16:57:13,164][74987] Avg episode reward: [(0, '26.610'), (1, '34.600')] -[2023-10-14 16:57:14,537][75950] Updated weights for policy 1, policy_version 83910 (0.0012) -[2023-10-14 16:57:14,908][75950] Updated weights for policy 1, policy_version 83920 (0.0010) -[2023-10-14 16:57:15,274][75950] Updated weights for policy 1, policy_version 83930 (0.0009) -[2023-10-14 16:57:15,933][75949] Updated weights for policy 0, policy_version 84131 (0.0009) -[2023-10-14 16:57:16,294][75949] Updated weights for policy 0, policy_version 84141 (0.0008) -[2023-10-14 16:57:16,674][75949] Updated weights for policy 0, policy_version 84151 (0.0010) -[2023-10-14 16:57:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 172130304. Throughput: 0: 1674.7, 1: 1659.1. Samples: 43035724. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 16:57:18,165][74987] Avg episode reward: [(0, '26.790'), (1, '32.050')] -[2023-10-14 16:57:19,433][75950] Updated weights for policy 1, policy_version 83940 (0.0008) -[2023-10-14 16:57:19,804][75950] Updated weights for policy 1, policy_version 83950 (0.0007) -[2023-10-14 16:57:20,159][75950] Updated weights for policy 1, policy_version 83960 (0.0008) -[2023-10-14 16:57:20,784][75949] Updated weights for policy 0, policy_version 84161 (0.0009) -[2023-10-14 16:57:21,139][75949] Updated weights for policy 0, policy_version 84171 (0.0011) -[2023-10-14 16:57:21,507][75949] Updated weights for policy 0, policy_version 84181 (0.0010) -[2023-10-14 16:57:21,881][75949] Updated weights for policy 0, policy_version 84191 (0.0010) -[2023-10-14 16:57:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 172195840. Throughput: 0: 1657.0, 1: 1683.2. Samples: 43055132. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 16:57:23,164][74987] Avg episode reward: [(0, '28.020'), (1, '31.450')] -[2023-10-14 16:57:24,447][75950] Updated weights for policy 1, policy_version 83970 (0.0007) -[2023-10-14 16:57:24,866][75950] Updated weights for policy 1, policy_version 83980 (0.0010) -[2023-10-14 16:57:25,235][75950] Updated weights for policy 1, policy_version 83990 (0.0010) -[2023-10-14 16:57:25,588][75950] Updated weights for policy 1, policy_version 84000 (0.0008) -[2023-10-14 16:57:25,820][75949] Updated weights for policy 0, policy_version 84201 (0.0008) -[2023-10-14 16:57:26,184][75949] Updated weights for policy 0, policy_version 84211 (0.0007) -[2023-10-14 16:57:26,552][75949] Updated weights for policy 0, policy_version 84221 (0.0009) -[2023-10-14 16:57:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 172261376. Throughput: 0: 1683.3, 1: 1676.1. Samples: 43075606. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 16:57:28,164][74987] Avg episode reward: [(0, '27.200'), (1, '31.600')] -[2023-10-14 16:57:29,629][75950] Updated weights for policy 1, policy_version 84010 (0.0007) -[2023-10-14 16:57:29,995][75950] Updated weights for policy 1, policy_version 84020 (0.0007) -[2023-10-14 16:57:30,367][75950] Updated weights for policy 1, policy_version 84030 (0.0008) -[2023-10-14 16:57:30,731][75949] Updated weights for policy 0, policy_version 84231 (0.0009) -[2023-10-14 16:57:31,105][75949] Updated weights for policy 0, policy_version 84241 (0.0008) -[2023-10-14 16:57:31,473][75949] Updated weights for policy 0, policy_version 84251 (0.0010) -[2023-10-14 16:57:33,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 172326912. Throughput: 0: 1673.4, 1: 1655.9. Samples: 43085490. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) -[2023-10-14 16:57:33,165][74987] Avg episode reward: [(0, '27.900'), (1, '31.440')] -[2023-10-14 16:57:34,459][75950] Updated weights for policy 1, policy_version 84040 (0.0009) -[2023-10-14 16:57:34,810][75950] Updated weights for policy 1, policy_version 84050 (0.0010) -[2023-10-14 16:57:35,177][75950] Updated weights for policy 1, policy_version 84060 (0.0008) -[2023-10-14 16:57:35,339][75949] Updated weights for policy 0, policy_version 84261 (0.0007) -[2023-10-14 16:57:35,710][75949] Updated weights for policy 0, policy_version 84271 (0.0008) -[2023-10-14 16:57:36,082][75949] Updated weights for policy 0, policy_version 84281 (0.0008) -[2023-10-14 16:57:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 172392448. Throughput: 0: 1669.0, 1: 1669.8. Samples: 43105272. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 16:57:38,164][74987] Avg episode reward: [(0, '27.230'), (1, '33.310')] -[2023-10-14 16:57:39,264][75950] Updated weights for policy 1, policy_version 84070 (0.0007) -[2023-10-14 16:57:39,631][75950] Updated weights for policy 1, policy_version 84080 (0.0007) -[2023-10-14 16:57:39,994][75950] Updated weights for policy 1, policy_version 84090 (0.0007) -[2023-10-14 16:57:40,197][75949] Updated weights for policy 0, policy_version 84291 (0.0008) -[2023-10-14 16:57:40,577][75949] Updated weights for policy 0, policy_version 84301 (0.0008) -[2023-10-14 16:57:40,953][75949] Updated weights for policy 0, policy_version 84311 (0.0009) -[2023-10-14 16:57:43,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 172457984. Throughput: 0: 1688.0, 1: 1673.1. Samples: 43126330. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 16:57:43,164][74987] Avg episode reward: [(0, '27.850'), (1, '32.930')] -[2023-10-14 16:57:44,100][75950] Updated weights for policy 1, policy_version 84100 (0.0009) -[2023-10-14 16:57:44,473][75950] Updated weights for policy 1, policy_version 84110 (0.0009) -[2023-10-14 16:57:44,745][75949] Updated weights for policy 0, policy_version 84321 (0.0011) -[2023-10-14 16:57:44,841][75950] Updated weights for policy 1, policy_version 84120 (0.0009) -[2023-10-14 16:57:45,105][75949] Updated weights for policy 0, policy_version 84331 (0.0008) -[2023-10-14 16:57:45,471][75949] Updated weights for policy 0, policy_version 84341 (0.0008) -[2023-10-14 16:57:45,829][75949] Updated weights for policy 0, policy_version 84351 (0.0011) -[2023-10-14 16:57:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 172523520. Throughput: 0: 1667.7, 1: 1660.4. Samples: 43135666. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 16:57:48,164][74987] Avg episode reward: [(0, '28.300'), (1, '32.610')] -[2023-10-14 16:57:48,971][75950] Updated weights for policy 1, policy_version 84130 (0.0009) -[2023-10-14 16:57:49,339][75950] Updated weights for policy 1, policy_version 84140 (0.0009) -[2023-10-14 16:57:49,700][75950] Updated weights for policy 1, policy_version 84150 (0.0008) -[2023-10-14 16:57:50,001][75949] Updated weights for policy 0, policy_version 84361 (0.0007) -[2023-10-14 16:57:50,063][75950] Updated weights for policy 1, policy_version 84160 (0.0008) -[2023-10-14 16:57:50,366][75949] Updated weights for policy 0, policy_version 84371 (0.0008) -[2023-10-14 16:57:50,736][75949] Updated weights for policy 0, policy_version 84381 (0.0008) -[2023-10-14 16:57:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 172589056. Throughput: 0: 1682.3, 1: 1669.6. Samples: 43155938. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 16:57:53,164][74987] Avg episode reward: [(0, '28.070'), (1, '34.450')] -[2023-10-14 16:57:54,265][75950] Updated weights for policy 1, policy_version 84170 (0.0010) -[2023-10-14 16:57:54,630][75950] Updated weights for policy 1, policy_version 84180 (0.0009) -[2023-10-14 16:57:54,783][75949] Updated weights for policy 0, policy_version 84391 (0.0007) -[2023-10-14 16:57:55,003][75950] Updated weights for policy 1, policy_version 84190 (0.0009) -[2023-10-14 16:57:55,147][75949] Updated weights for policy 0, policy_version 84401 (0.0009) -[2023-10-14 16:57:55,518][75949] Updated weights for policy 0, policy_version 84411 (0.0012) -[2023-10-14 16:57:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172654592. Throughput: 0: 1690.5, 1: 1668.0. Samples: 43176498. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 16:57:58,164][74987] Avg episode reward: [(0, '27.660'), (1, '32.130')] -[2023-10-14 16:57:58,941][75950] Updated weights for policy 1, policy_version 84200 (0.0009) -[2023-10-14 16:57:59,308][75950] Updated weights for policy 1, policy_version 84210 (0.0011) -[2023-10-14 16:57:59,580][75949] Updated weights for policy 0, policy_version 84421 (0.0008) -[2023-10-14 16:57:59,680][75950] Updated weights for policy 1, policy_version 84220 (0.0008) -[2023-10-14 16:57:59,950][75949] Updated weights for policy 0, policy_version 84431 (0.0007) -[2023-10-14 16:58:00,323][75949] Updated weights for policy 0, policy_version 84441 (0.0008) -[2023-10-14 16:58:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172720128. Throughput: 0: 1663.5, 1: 1669.2. Samples: 43185692. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 16:58:03,164][74987] Avg episode reward: [(0, '28.530'), (1, '34.420')] -[2023-10-14 16:58:03,814][75950] Updated weights for policy 1, policy_version 84230 (0.0008) -[2023-10-14 16:58:04,187][75950] Updated weights for policy 1, policy_version 84240 (0.0009) -[2023-10-14 16:58:04,320][75949] Updated weights for policy 0, policy_version 84451 (0.0009) -[2023-10-14 16:58:04,556][75950] Updated weights for policy 1, policy_version 84250 (0.0008) -[2023-10-14 16:58:04,692][75949] Updated weights for policy 0, policy_version 84461 (0.0008) -[2023-10-14 16:58:05,057][75949] Updated weights for policy 0, policy_version 84471 (0.0007) -[2023-10-14 16:58:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172785664. Throughput: 0: 1688.6, 1: 1673.0. Samples: 43206406. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 16:58:08,165][74987] Avg episode reward: [(0, '28.790'), (1, '36.480')] -[2023-10-14 16:58:08,761][75950] Updated weights for policy 1, policy_version 84260 (0.0008) -[2023-10-14 16:58:09,109][75949] Updated weights for policy 0, policy_version 84481 (0.0009) -[2023-10-14 16:58:09,154][75950] Updated weights for policy 1, policy_version 84270 (0.0007) -[2023-10-14 16:58:09,480][75949] Updated weights for policy 0, policy_version 84491 (0.0010) -[2023-10-14 16:58:09,523][75950] Updated weights for policy 1, policy_version 84280 (0.0008) -[2023-10-14 16:58:09,853][75949] Updated weights for policy 0, policy_version 84501 (0.0008) -[2023-10-14 16:58:10,218][75949] Updated weights for policy 0, policy_version 84511 (0.0008) -[2023-10-14 16:58:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 172851200. Throughput: 0: 1692.3, 1: 1673.7. Samples: 43227074. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 16:58:13,165][74987] Avg episode reward: [(0, '29.310'), (1, '34.700')] -[2023-10-14 16:58:13,625][75950] Updated weights for policy 1, policy_version 84290 (0.0007) -[2023-10-14 16:58:13,991][75950] Updated weights for policy 1, policy_version 84300 (0.0009) -[2023-10-14 16:58:14,245][75949] Updated weights for policy 0, policy_version 84521 (0.0007) -[2023-10-14 16:58:14,368][75950] Updated weights for policy 1, policy_version 84310 (0.0008) -[2023-10-14 16:58:14,622][75949] Updated weights for policy 0, policy_version 84531 (0.0008) -[2023-10-14 16:58:14,721][75950] Updated weights for policy 1, policy_version 84320 (0.0008) -[2023-10-14 16:58:14,988][75949] Updated weights for policy 0, policy_version 84541 (0.0008) -[2023-10-14 16:58:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 172916736. Throughput: 0: 1671.7, 1: 1674.0. Samples: 43236050. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 16:58:18,165][74987] Avg episode reward: [(0, '28.370'), (1, '35.570')] -[2023-10-14 16:58:18,829][75950] Updated weights for policy 1, policy_version 84330 (0.0009) -[2023-10-14 16:58:19,112][75949] Updated weights for policy 0, policy_version 84551 (0.0008) -[2023-10-14 16:58:19,206][75950] Updated weights for policy 1, policy_version 84340 (0.0008) -[2023-10-14 16:58:19,473][75949] Updated weights for policy 0, policy_version 84561 (0.0008) -[2023-10-14 16:58:19,568][75950] Updated weights for policy 1, policy_version 84350 (0.0008) -[2023-10-14 16:58:19,841][75949] Updated weights for policy 0, policy_version 84571 (0.0009) -[2023-10-14 16:58:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172982272. Throughput: 0: 1688.1, 1: 1673.1. Samples: 43256526. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 16:58:23,165][74987] Avg episode reward: [(0, '28.440'), (1, '33.760')] -[2023-10-14 16:58:23,642][75950] Updated weights for policy 1, policy_version 84360 (0.0010) -[2023-10-14 16:58:23,911][75949] Updated weights for policy 0, policy_version 84581 (0.0008) -[2023-10-14 16:58:24,003][75950] Updated weights for policy 1, policy_version 84370 (0.0010) -[2023-10-14 16:58:24,297][75949] Updated weights for policy 0, policy_version 84591 (0.0008) -[2023-10-14 16:58:24,362][75950] Updated weights for policy 1, policy_version 84380 (0.0009) -[2023-10-14 16:58:24,662][75949] Updated weights for policy 0, policy_version 84601 (0.0009) -[2023-10-14 16:58:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173047808. Throughput: 0: 1685.6, 1: 1669.1. Samples: 43277292. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) -[2023-10-14 16:58:28,165][74987] Avg episode reward: [(0, '28.510'), (1, '34.480')] -[2023-10-14 16:58:28,612][75950] Updated weights for policy 1, policy_version 84390 (0.0008) -[2023-10-14 16:58:28,927][75949] Updated weights for policy 0, policy_version 84611 (0.0009) -[2023-10-14 16:58:28,980][75950] Updated weights for policy 1, policy_version 84400 (0.0007) -[2023-10-14 16:58:29,313][75949] Updated weights for policy 0, policy_version 84621 (0.0009) -[2023-10-14 16:58:29,345][75950] Updated weights for policy 1, policy_version 84410 (0.0007) -[2023-10-14 16:58:29,687][75949] Updated weights for policy 0, policy_version 84631 (0.0010) -[2023-10-14 16:58:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173113344. Throughput: 0: 1673.9, 1: 1669.7. Samples: 43286128. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) -[2023-10-14 16:58:33,165][74987] Avg episode reward: [(0, '27.270'), (1, '36.190')] -[2023-10-14 16:58:33,345][75950] Updated weights for policy 1, policy_version 84420 (0.0008) -[2023-10-14 16:58:33,711][75950] Updated weights for policy 1, policy_version 84430 (0.0008) -[2023-10-14 16:58:33,780][75949] Updated weights for policy 0, policy_version 84641 (0.0009) -[2023-10-14 16:58:34,075][75950] Updated weights for policy 1, policy_version 84440 (0.0007) -[2023-10-14 16:58:34,147][75949] Updated weights for policy 0, policy_version 84651 (0.0007) -[2023-10-14 16:58:34,514][75949] Updated weights for policy 0, policy_version 84661 (0.0007) -[2023-10-14 16:58:34,881][75949] Updated weights for policy 0, policy_version 84671 (0.0010) -[2023-10-14 16:58:38,083][75950] Updated weights for policy 1, policy_version 84450 (0.0008) -[2023-10-14 16:58:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173178880. Throughput: 0: 1681.8, 1: 1673.1. Samples: 43306908. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) -[2023-10-14 16:58:38,165][74987] Avg episode reward: [(0, '27.350'), (1, '33.570')] -[2023-10-14 16:58:38,454][75950] Updated weights for policy 1, policy_version 84460 (0.0010) -[2023-10-14 16:58:38,818][75949] Updated weights for policy 0, policy_version 84681 (0.0009) -[2023-10-14 16:58:38,820][75950] Updated weights for policy 1, policy_version 84470 (0.0008) -[2023-10-14 16:58:39,181][75949] Updated weights for policy 0, policy_version 84691 (0.0009) -[2023-10-14 16:58:39,182][75950] Updated weights for policy 1, policy_version 84480 (0.0009) -[2023-10-14 16:58:39,559][75949] Updated weights for policy 0, policy_version 84701 (0.0009) -[2023-10-14 16:58:43,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173244416. Throughput: 0: 1688.0, 1: 1668.4. Samples: 43327536. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) -[2023-10-14 16:58:43,164][74987] Avg episode reward: [(0, '28.440'), (1, '33.220')] -[2023-10-14 16:58:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000084704_86736896.pth... -[2023-10-14 16:58:43,210][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000083136_85131264.pth -[2023-10-14 16:58:43,426][75950] Updated weights for policy 1, policy_version 84490 (0.0008) -[2023-10-14 16:58:43,659][75949] Updated weights for policy 0, policy_version 84711 (0.0009) -[2023-10-14 16:58:43,785][75950] Updated weights for policy 1, policy_version 84500 (0.0009) -[2023-10-14 16:58:44,019][75949] Updated weights for policy 0, policy_version 84721 (0.0008) -[2023-10-14 16:58:44,142][75950] Updated weights for policy 1, policy_version 84510 (0.0008) -[2023-10-14 16:58:44,214][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000084512_86540288.pth... -[2023-10-14 16:58:44,244][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000082944_84934656.pth -[2023-10-14 16:58:44,392][75949] Updated weights for policy 0, policy_version 84731 (0.0009) -[2023-10-14 16:58:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173309952. Throughput: 0: 1686.2, 1: 1665.3. Samples: 43336510. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) -[2023-10-14 16:58:48,164][74987] Avg episode reward: [(0, '29.140'), (1, '34.060')] -[2023-10-14 16:58:48,282][75950] Updated weights for policy 1, policy_version 84520 (0.0009) -[2023-10-14 16:58:48,553][75949] Updated weights for policy 0, policy_version 84741 (0.0010) -[2023-10-14 16:58:48,651][75950] Updated weights for policy 1, policy_version 84530 (0.0009) -[2023-10-14 16:58:48,922][75949] Updated weights for policy 0, policy_version 84751 (0.0009) -[2023-10-14 16:58:49,012][75950] Updated weights for policy 1, policy_version 84540 (0.0009) -[2023-10-14 16:58:49,295][75949] Updated weights for policy 0, policy_version 84761 (0.0008) -[2023-10-14 16:58:52,978][75950] Updated weights for policy 1, policy_version 84550 (0.0009) -[2023-10-14 16:58:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173375488. Throughput: 0: 1683.0, 1: 1663.2. Samples: 43356984. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) -[2023-10-14 16:58:53,164][74987] Avg episode reward: [(0, '28.670'), (1, '31.930')] -[2023-10-14 16:58:53,253][75949] Updated weights for policy 0, policy_version 84771 (0.0009) -[2023-10-14 16:58:53,349][75950] Updated weights for policy 1, policy_version 84560 (0.0009) -[2023-10-14 16:58:53,631][75949] Updated weights for policy 0, policy_version 84781 (0.0009) -[2023-10-14 16:58:53,700][75950] Updated weights for policy 1, policy_version 84570 (0.0008) -[2023-10-14 16:58:53,992][75949] Updated weights for policy 0, policy_version 84791 (0.0008) -[2023-10-14 16:58:57,947][75950] Updated weights for policy 1, policy_version 84580 (0.0008) -[2023-10-14 16:58:58,096][75949] Updated weights for policy 0, policy_version 84801 (0.0008) -[2023-10-14 16:58:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173441024. Throughput: 0: 1676.4, 1: 1667.9. Samples: 43377566. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) -[2023-10-14 16:58:58,164][74987] Avg episode reward: [(0, '28.640'), (1, '32.120')] -[2023-10-14 16:58:58,347][75950] Updated weights for policy 1, policy_version 84590 (0.0008) -[2023-10-14 16:58:58,459][75949] Updated weights for policy 0, policy_version 84811 (0.0007) -[2023-10-14 16:58:58,713][75950] Updated weights for policy 1, policy_version 84600 (0.0009) -[2023-10-14 16:58:58,832][75949] Updated weights for policy 0, policy_version 84821 (0.0007) -[2023-10-14 16:58:59,198][75949] Updated weights for policy 0, policy_version 84831 (0.0008) -[2023-10-14 16:59:02,766][75950] Updated weights for policy 1, policy_version 84610 (0.0009) -[2023-10-14 16:59:03,124][75950] Updated weights for policy 1, policy_version 84620 (0.0008) -[2023-10-14 16:59:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173506560. Throughput: 0: 1673.6, 1: 1668.8. Samples: 43386456. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) -[2023-10-14 16:59:03,164][74987] Avg episode reward: [(0, '27.760'), (1, '33.530')] -[2023-10-14 16:59:03,313][75949] Updated weights for policy 0, policy_version 84841 (0.0007) -[2023-10-14 16:59:03,491][75950] Updated weights for policy 1, policy_version 84630 (0.0008) -[2023-10-14 16:59:03,689][75949] Updated weights for policy 0, policy_version 84851 (0.0007) -[2023-10-14 16:59:03,851][75950] Updated weights for policy 1, policy_version 84640 (0.0007) -[2023-10-14 16:59:04,053][75949] Updated weights for policy 0, policy_version 84861 (0.0008) -[2023-10-14 16:59:08,008][75950] Updated weights for policy 1, policy_version 84650 (0.0007) -[2023-10-14 16:59:08,127][75949] Updated weights for policy 0, policy_version 84871 (0.0008) -[2023-10-14 16:59:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173572096. Throughput: 0: 1680.3, 1: 1668.2. Samples: 43407206. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) -[2023-10-14 16:59:08,164][74987] Avg episode reward: [(0, '28.090'), (1, '33.390')] -[2023-10-14 16:59:08,370][75950] Updated weights for policy 1, policy_version 84660 (0.0007) -[2023-10-14 16:59:08,488][75949] Updated weights for policy 0, policy_version 84881 (0.0007) -[2023-10-14 16:59:08,752][75950] Updated weights for policy 1, policy_version 84670 (0.0007) -[2023-10-14 16:59:08,860][75949] Updated weights for policy 0, policy_version 84891 (0.0008) -[2023-10-14 16:59:12,885][75950] Updated weights for policy 1, policy_version 84680 (0.0009) -[2023-10-14 16:59:12,993][75949] Updated weights for policy 0, policy_version 84901 (0.0008) -[2023-10-14 16:59:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173637632. Throughput: 0: 1678.4, 1: 1665.9. Samples: 43427786. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) -[2023-10-14 16:59:13,164][74987] Avg episode reward: [(0, '28.850'), (1, '32.680')] -[2023-10-14 16:59:13,243][75950] Updated weights for policy 1, policy_version 84690 (0.0009) -[2023-10-14 16:59:13,373][75949] Updated weights for policy 0, policy_version 84911 (0.0007) -[2023-10-14 16:59:13,621][75950] Updated weights for policy 1, policy_version 84700 (0.0009) -[2023-10-14 16:59:13,740][75949] Updated weights for policy 0, policy_version 84921 (0.0008) -[2023-10-14 16:59:17,771][75950] Updated weights for policy 1, policy_version 84710 (0.0009) -[2023-10-14 16:59:17,779][75949] Updated weights for policy 0, policy_version 84931 (0.0009) -[2023-10-14 16:59:18,146][75950] Updated weights for policy 1, policy_version 84720 (0.0009) -[2023-10-14 16:59:18,147][75949] Updated weights for policy 0, policy_version 84941 (0.0009) -[2023-10-14 16:59:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 173703168. Throughput: 0: 1681.6, 1: 1667.5. Samples: 43436836. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) -[2023-10-14 16:59:18,165][74987] Avg episode reward: [(0, '27.900'), (1, '34.030')] -[2023-10-14 16:59:18,513][75950] Updated weights for policy 1, policy_version 84730 (0.0008) -[2023-10-14 16:59:18,517][75949] Updated weights for policy 0, policy_version 84951 (0.0010) -[2023-10-14 16:59:22,549][75949] Updated weights for policy 0, policy_version 84961 (0.0007) -[2023-10-14 16:59:22,607][75950] Updated weights for policy 1, policy_version 84740 (0.0010) -[2023-10-14 16:59:22,916][75949] Updated weights for policy 0, policy_version 84971 (0.0007) -[2023-10-14 16:59:22,966][75950] Updated weights for policy 1, policy_version 84750 (0.0009) -[2023-10-14 16:59:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173768704. Throughput: 0: 1677.2, 1: 1666.6. Samples: 43457376. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) -[2023-10-14 16:59:23,164][74987] Avg episode reward: [(0, '28.910'), (1, '35.080')] -[2023-10-14 16:59:23,291][75949] Updated weights for policy 0, policy_version 84981 (0.0009) -[2023-10-14 16:59:23,339][75950] Updated weights for policy 1, policy_version 84760 (0.0009) -[2023-10-14 16:59:23,652][75949] Updated weights for policy 0, policy_version 84991 (0.0007) -[2023-10-14 16:59:27,459][75950] Updated weights for policy 1, policy_version 84770 (0.0008) -[2023-10-14 16:59:27,796][75949] Updated weights for policy 0, policy_version 85001 (0.0007) -[2023-10-14 16:59:27,826][75950] Updated weights for policy 1, policy_version 84780 (0.0008) -[2023-10-14 16:59:28,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173834240. Throughput: 0: 1667.6, 1: 1666.0. Samples: 43477550. Policy #0 lag: (min: 11.0, avg: 16.4, max: 43.0) -[2023-10-14 16:59:28,164][74987] Avg episode reward: [(0, '28.960'), (1, '34.220')] -[2023-10-14 16:59:28,169][75949] Updated weights for policy 0, policy_version 85011 (0.0009) -[2023-10-14 16:59:28,187][75950] Updated weights for policy 1, policy_version 84790 (0.0009) -[2023-10-14 16:59:28,540][75949] Updated weights for policy 0, policy_version 85021 (0.0009) -[2023-10-14 16:59:28,554][75950] Updated weights for policy 1, policy_version 84800 (0.0009) -[2023-10-14 16:59:32,610][75949] Updated weights for policy 0, policy_version 85031 (0.0007) -[2023-10-14 16:59:32,684][75950] Updated weights for policy 1, policy_version 84810 (0.0007) -[2023-10-14 16:59:32,969][75949] Updated weights for policy 0, policy_version 85041 (0.0009) -[2023-10-14 16:59:33,046][75950] Updated weights for policy 1, policy_version 84820 (0.0008) -[2023-10-14 16:59:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173899776. Throughput: 0: 1672.3, 1: 1673.4. Samples: 43487068. Policy #0 lag: (min: 11.0, avg: 16.4, max: 43.0) -[2023-10-14 16:59:33,165][74987] Avg episode reward: [(0, '28.660'), (1, '34.860')] -[2023-10-14 16:59:33,351][75949] Updated weights for policy 0, policy_version 85051 (0.0007) -[2023-10-14 16:59:33,416][75950] Updated weights for policy 1, policy_version 84830 (0.0008) -[2023-10-14 16:59:37,497][75949] Updated weights for policy 0, policy_version 85061 (0.0009) -[2023-10-14 16:59:37,512][75950] Updated weights for policy 1, policy_version 84840 (0.0007) -[2023-10-14 16:59:37,863][75949] Updated weights for policy 0, policy_version 85071 (0.0008) -[2023-10-14 16:59:37,871][75950] Updated weights for policy 1, policy_version 84850 (0.0007) -[2023-10-14 16:59:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173965312. Throughput: 0: 1669.1, 1: 1668.8. Samples: 43507186. Policy #0 lag: (min: 11.0, avg: 16.4, max: 43.0) -[2023-10-14 16:59:38,165][74987] Avg episode reward: [(0, '28.240'), (1, '36.040')] -[2023-10-14 16:59:38,242][75949] Updated weights for policy 0, policy_version 85081 (0.0007) -[2023-10-14 16:59:38,244][75950] Updated weights for policy 1, policy_version 84860 (0.0007) -[2023-10-14 16:59:42,162][75949] Updated weights for policy 0, policy_version 85091 (0.0007) -[2023-10-14 16:59:42,217][75950] Updated weights for policy 1, policy_version 84870 (0.0009) -[2023-10-14 16:59:42,533][75949] Updated weights for policy 0, policy_version 85101 (0.0009) -[2023-10-14 16:59:42,592][75950] Updated weights for policy 1, policy_version 84880 (0.0009) -[2023-10-14 16:59:42,889][75949] Updated weights for policy 0, policy_version 85111 (0.0009) -[2023-10-14 16:59:42,954][75950] Updated weights for policy 1, policy_version 84890 (0.0008) -[2023-10-14 16:59:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 174030848. Throughput: 0: 1662.5, 1: 1656.1. Samples: 43526906. Policy #0 lag: (min: 11.0, avg: 16.4, max: 43.0) -[2023-10-14 16:59:43,165][74987] Avg episode reward: [(0, '28.410'), (1, '35.160')] -[2023-10-14 16:59:47,086][75949] Updated weights for policy 0, policy_version 85121 (0.0007) -[2023-10-14 16:59:47,167][75950] Updated weights for policy 1, policy_version 84900 (0.0009) -[2023-10-14 16:59:47,440][75949] Updated weights for policy 0, policy_version 85131 (0.0008) -[2023-10-14 16:59:47,559][75950] Updated weights for policy 1, policy_version 84910 (0.0008) -[2023-10-14 16:59:47,814][75949] Updated weights for policy 0, policy_version 85141 (0.0009) -[2023-10-14 16:59:47,924][75950] Updated weights for policy 1, policy_version 84920 (0.0010) -[2023-10-14 16:59:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 174096384. Throughput: 0: 1677.3, 1: 1674.0. Samples: 43537268. Policy #0 lag: (min: 11.0, avg: 16.4, max: 43.0) -[2023-10-14 16:59:48,165][74987] Avg episode reward: [(0, '28.950'), (1, '33.240')] -[2023-10-14 16:59:48,178][75949] Updated weights for policy 0, policy_version 85151 (0.0007) -[2023-10-14 16:59:51,848][75950] Updated weights for policy 1, policy_version 84930 (0.0009) -[2023-10-14 16:59:52,218][75950] Updated weights for policy 1, policy_version 84940 (0.0008) -[2023-10-14 16:59:52,376][75949] Updated weights for policy 0, policy_version 85161 (0.0010) -[2023-10-14 16:59:52,574][75950] Updated weights for policy 1, policy_version 84950 (0.0007) -[2023-10-14 16:59:52,742][75949] Updated weights for policy 0, policy_version 85171 (0.0007) -[2023-10-14 16:59:52,947][75950] Updated weights for policy 1, policy_version 84960 (0.0007) -[2023-10-14 16:59:53,115][75949] Updated weights for policy 0, policy_version 85181 (0.0007) -[2023-10-14 16:59:53,163][74987] Fps is (10 sec: 16384.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 174194688. Throughput: 0: 1671.2, 1: 1672.7. Samples: 43557682. Policy #0 lag: (min: 11.0, avg: 16.4, max: 43.0) -[2023-10-14 16:59:53,164][74987] Avg episode reward: [(0, '30.360'), (1, '34.330')] -[2023-10-14 16:59:56,998][75950] Updated weights for policy 1, policy_version 84970 (0.0010) -[2023-10-14 16:59:57,210][75949] Updated weights for policy 0, policy_version 85191 (0.0008) -[2023-10-14 16:59:57,362][75950] Updated weights for policy 1, policy_version 84980 (0.0008) -[2023-10-14 16:59:57,576][75949] Updated weights for policy 0, policy_version 85201 (0.0007) -[2023-10-14 16:59:57,723][75950] Updated weights for policy 1, policy_version 84990 (0.0009) -[2023-10-14 16:59:57,948][75949] Updated weights for policy 0, policy_version 85211 (0.0008) -[2023-10-14 16:59:58,164][74987] Fps is (10 sec: 19661.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 174292992. Throughput: 0: 1655.9, 1: 1649.0. Samples: 43576506. Policy #0 lag: (min: 11.0, avg: 16.4, max: 43.0) -[2023-10-14 16:59:58,164][74987] Avg episode reward: [(0, '27.850'), (1, '35.110')] -[2023-10-14 17:00:01,883][75950] Updated weights for policy 1, policy_version 85000 (0.0008) -[2023-10-14 17:00:02,068][75949] Updated weights for policy 0, policy_version 85221 (0.0008) -[2023-10-14 17:00:02,247][75950] Updated weights for policy 1, policy_version 85010 (0.0008) -[2023-10-14 17:00:02,439][75949] Updated weights for policy 0, policy_version 85231 (0.0007) -[2023-10-14 17:00:02,615][75950] Updated weights for policy 1, policy_version 85020 (0.0011) -[2023-10-14 17:00:02,805][75949] Updated weights for policy 0, policy_version 85241 (0.0010) -[2023-10-14 17:00:03,163][74987] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 174358528. Throughput: 0: 1672.5, 1: 1673.7. Samples: 43587414. Policy #0 lag: (min: 11.0, avg: 16.4, max: 43.0) -[2023-10-14 17:00:03,164][74987] Avg episode reward: [(0, '30.090'), (1, '34.230')] -[2023-10-14 17:00:06,674][75950] Updated weights for policy 1, policy_version 85030 (0.0010) -[2023-10-14 17:00:06,870][75949] Updated weights for policy 0, policy_version 85251 (0.0009) -[2023-10-14 17:00:07,041][75950] Updated weights for policy 1, policy_version 85040 (0.0009) -[2023-10-14 17:00:07,253][75949] Updated weights for policy 0, policy_version 85261 (0.0008) -[2023-10-14 17:00:07,410][75950] Updated weights for policy 1, policy_version 85050 (0.0008) -[2023-10-14 17:00:07,613][75949] Updated weights for policy 0, policy_version 85271 (0.0008) -[2023-10-14 17:00:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 174424064. Throughput: 0: 1676.5, 1: 1667.5. Samples: 43607858. Policy #0 lag: (min: 11.0, avg: 16.4, max: 43.0) -[2023-10-14 17:00:08,165][74987] Avg episode reward: [(0, '26.770'), (1, '34.370')] -[2023-10-14 17:00:11,599][75950] Updated weights for policy 1, policy_version 85060 (0.0008) -[2023-10-14 17:00:11,699][75949] Updated weights for policy 0, policy_version 85281 (0.0009) -[2023-10-14 17:00:11,962][75950] Updated weights for policy 1, policy_version 85070 (0.0007) -[2023-10-14 17:00:12,063][75949] Updated weights for policy 0, policy_version 85291 (0.0008) -[2023-10-14 17:00:12,340][75950] Updated weights for policy 1, policy_version 85080 (0.0007) -[2023-10-14 17:00:12,436][75949] Updated weights for policy 0, policy_version 85301 (0.0008) -[2023-10-14 17:00:12,800][75949] Updated weights for policy 0, policy_version 85311 (0.0007) -[2023-10-14 17:00:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 174489600. Throughput: 0: 1653.6, 1: 1652.5. Samples: 43626324. Policy #0 lag: (min: 11.0, avg: 16.4, max: 43.0) -[2023-10-14 17:00:13,164][74987] Avg episode reward: [(0, '29.880'), (1, '35.720')] -[2023-10-14 17:00:16,338][75950] Updated weights for policy 1, policy_version 85090 (0.0008) -[2023-10-14 17:00:16,701][75950] Updated weights for policy 1, policy_version 85100 (0.0008) -[2023-10-14 17:00:16,888][75949] Updated weights for policy 0, policy_version 85321 (0.0008) -[2023-10-14 17:00:17,067][75950] Updated weights for policy 1, policy_version 85110 (0.0009) -[2023-10-14 17:00:17,263][75949] Updated weights for policy 0, policy_version 85331 (0.0008) -[2023-10-14 17:00:17,431][75950] Updated weights for policy 1, policy_version 85120 (0.0008) -[2023-10-14 17:00:17,636][75949] Updated weights for policy 0, policy_version 85341 (0.0009) -[2023-10-14 17:00:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 174555136. Throughput: 0: 1670.9, 1: 1677.0. Samples: 43637724. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 17:00:18,165][74987] Avg episode reward: [(0, '25.950'), (1, '34.330')] -[2023-10-14 17:00:21,587][75950] Updated weights for policy 1, policy_version 85130 (0.0010) -[2023-10-14 17:00:21,800][75949] Updated weights for policy 0, policy_version 85351 (0.0009) -[2023-10-14 17:00:21,945][75950] Updated weights for policy 1, policy_version 85140 (0.0010) -[2023-10-14 17:00:22,171][75949] Updated weights for policy 0, policy_version 85361 (0.0010) -[2023-10-14 17:00:22,309][75950] Updated weights for policy 1, policy_version 85150 (0.0009) -[2023-10-14 17:00:22,542][75949] Updated weights for policy 0, policy_version 85371 (0.0009) -[2023-10-14 17:00:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 174620672. Throughput: 0: 1670.3, 1: 1673.8. Samples: 43657670. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 17:00:23,165][74987] Avg episode reward: [(0, '27.240'), (1, '34.090')] -[2023-10-14 17:00:26,447][75950] Updated weights for policy 1, policy_version 85160 (0.0009) -[2023-10-14 17:00:26,811][75950] Updated weights for policy 1, policy_version 85170 (0.0008) -[2023-10-14 17:00:26,822][75949] Updated weights for policy 0, policy_version 85381 (0.0009) -[2023-10-14 17:00:27,180][75950] Updated weights for policy 1, policy_version 85180 (0.0008) -[2023-10-14 17:00:27,185][75949] Updated weights for policy 0, policy_version 85391 (0.0008) -[2023-10-14 17:00:27,560][75949] Updated weights for policy 0, policy_version 85401 (0.0009) -[2023-10-14 17:00:28,163][74987] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 174686208. Throughput: 0: 1652.8, 1: 1665.2. Samples: 43676216. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 17:00:28,164][74987] Avg episode reward: [(0, '26.430'), (1, '36.000')] -[2023-10-14 17:00:31,238][75950] Updated weights for policy 1, policy_version 85190 (0.0008) -[2023-10-14 17:00:31,573][75949] Updated weights for policy 0, policy_version 85411 (0.0008) -[2023-10-14 17:00:31,603][75950] Updated weights for policy 1, policy_version 85200 (0.0009) -[2023-10-14 17:00:31,938][75949] Updated weights for policy 0, policy_version 85421 (0.0009) -[2023-10-14 17:00:31,970][75950] Updated weights for policy 1, policy_version 85210 (0.0008) -[2023-10-14 17:00:32,301][75949] Updated weights for policy 0, policy_version 85431 (0.0009) -[2023-10-14 17:00:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 174751744. Throughput: 0: 1667.0, 1: 1677.2. Samples: 43687754. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 17:00:33,165][74987] Avg episode reward: [(0, '28.520'), (1, '33.530')] -[2023-10-14 17:00:36,211][75950] Updated weights for policy 1, policy_version 85220 (0.0008) -[2023-10-14 17:00:36,217][75949] Updated weights for policy 0, policy_version 85441 (0.0010) -[2023-10-14 17:00:36,586][75949] Updated weights for policy 0, policy_version 85451 (0.0007) -[2023-10-14 17:00:36,609][75950] Updated weights for policy 1, policy_version 85230 (0.0008) -[2023-10-14 17:00:36,949][75949] Updated weights for policy 0, policy_version 85461 (0.0008) -[2023-10-14 17:00:36,980][75950] Updated weights for policy 1, policy_version 85240 (0.0008) -[2023-10-14 17:00:37,318][75949] Updated weights for policy 0, policy_version 85471 (0.0008) -[2023-10-14 17:00:38,163][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 174817280. Throughput: 0: 1659.7, 1: 1661.6. Samples: 43707140. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 17:00:38,164][74987] Avg episode reward: [(0, '28.040'), (1, '31.770')] -[2023-10-14 17:00:41,079][75950] Updated weights for policy 1, policy_version 85250 (0.0008) -[2023-10-14 17:00:41,353][75949] Updated weights for policy 0, policy_version 85481 (0.0009) -[2023-10-14 17:00:41,447][75950] Updated weights for policy 1, policy_version 85260 (0.0007) -[2023-10-14 17:00:41,722][75949] Updated weights for policy 0, policy_version 85491 (0.0007) -[2023-10-14 17:00:41,820][75950] Updated weights for policy 1, policy_version 85270 (0.0007) -[2023-10-14 17:00:42,089][75949] Updated weights for policy 0, policy_version 85501 (0.0007) -[2023-10-14 17:00:42,182][75950] Updated weights for policy 1, policy_version 85280 (0.0007) -[2023-10-14 17:00:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 174882816. Throughput: 0: 1662.2, 1: 1666.7. Samples: 43726304. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 17:00:43,165][74987] Avg episode reward: [(0, '28.750'), (1, '35.060')] -[2023-10-14 17:00:43,177][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000085280_87326720.pth... -[2023-10-14 17:00:43,177][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000085504_87556096.pth... -[2023-10-14 17:00:43,207][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000083712_85721088.pth -[2023-10-14 17:00:43,211][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000083936_85950464.pth -[2023-10-14 17:00:46,118][75949] Updated weights for policy 0, policy_version 85511 (0.0009) -[2023-10-14 17:00:46,264][75950] Updated weights for policy 1, policy_version 85290 (0.0010) -[2023-10-14 17:00:46,477][75949] Updated weights for policy 0, policy_version 85521 (0.0009) -[2023-10-14 17:00:46,629][75950] Updated weights for policy 1, policy_version 85300 (0.0008) -[2023-10-14 17:00:46,845][75949] Updated weights for policy 0, policy_version 85531 (0.0009) -[2023-10-14 17:00:46,987][75950] Updated weights for policy 1, policy_version 85310 (0.0007) -[2023-10-14 17:00:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 174948352. Throughput: 0: 1676.8, 1: 1669.3. Samples: 43737992. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 17:00:48,165][74987] Avg episode reward: [(0, '26.250'), (1, '34.970')] -[2023-10-14 17:00:50,967][75949] Updated weights for policy 0, policy_version 85541 (0.0008) -[2023-10-14 17:00:51,111][75950] Updated weights for policy 1, policy_version 85320 (0.0009) -[2023-10-14 17:00:51,353][75949] Updated weights for policy 0, policy_version 85551 (0.0008) -[2023-10-14 17:00:51,466][75950] Updated weights for policy 1, policy_version 85330 (0.0008) -[2023-10-14 17:00:51,722][75949] Updated weights for policy 0, policy_version 85561 (0.0008) -[2023-10-14 17:00:51,843][75950] Updated weights for policy 1, policy_version 85340 (0.0009) -[2023-10-14 17:00:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 175013888. Throughput: 0: 1654.2, 1: 1652.8. Samples: 43756674. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 17:00:53,165][74987] Avg episode reward: [(0, '27.670'), (1, '31.280')] -[2023-10-14 17:00:55,843][75949] Updated weights for policy 0, policy_version 85571 (0.0008) -[2023-10-14 17:00:56,064][75950] Updated weights for policy 1, policy_version 85350 (0.0009) -[2023-10-14 17:00:56,202][75949] Updated weights for policy 0, policy_version 85581 (0.0009) -[2023-10-14 17:00:56,429][75950] Updated weights for policy 1, policy_version 85360 (0.0010) -[2023-10-14 17:00:56,584][75949] Updated weights for policy 0, policy_version 85591 (0.0009) -[2023-10-14 17:00:56,800][75950] Updated weights for policy 1, policy_version 85370 (0.0008) -[2023-10-14 17:00:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 175079424. Throughput: 0: 1670.2, 1: 1662.0. Samples: 43776274. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 17:00:58,164][74987] Avg episode reward: [(0, '28.880'), (1, '32.810')] -[2023-10-14 17:01:00,616][75949] Updated weights for policy 0, policy_version 85601 (0.0007) -[2023-10-14 17:01:00,953][75950] Updated weights for policy 1, policy_version 85380 (0.0008) -[2023-10-14 17:01:00,977][75949] Updated weights for policy 0, policy_version 85611 (0.0007) -[2023-10-14 17:01:01,320][75950] Updated weights for policy 1, policy_version 85390 (0.0007) -[2023-10-14 17:01:01,347][75949] Updated weights for policy 0, policy_version 85621 (0.0009) -[2023-10-14 17:01:01,674][75950] Updated weights for policy 1, policy_version 85400 (0.0007) -[2023-10-14 17:01:01,715][75949] Updated weights for policy 0, policy_version 85631 (0.0009) -[2023-10-14 17:01:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 175144960. Throughput: 0: 1673.7, 1: 1662.6. Samples: 43787858. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 17:01:03,164][74987] Avg episode reward: [(0, '27.370'), (1, '34.260')] -[2023-10-14 17:01:05,666][75949] Updated weights for policy 0, policy_version 85641 (0.0009) -[2023-10-14 17:01:05,791][75950] Updated weights for policy 1, policy_version 85410 (0.0010) -[2023-10-14 17:01:06,033][75949] Updated weights for policy 0, policy_version 85651 (0.0009) -[2023-10-14 17:01:06,152][75950] Updated weights for policy 1, policy_version 85420 (0.0010) -[2023-10-14 17:01:06,405][75949] Updated weights for policy 0, policy_version 85661 (0.0008) -[2023-10-14 17:01:06,518][75950] Updated weights for policy 1, policy_version 85430 (0.0008) -[2023-10-14 17:01:06,881][75950] Updated weights for policy 1, policy_version 85440 (0.0010) -[2023-10-14 17:01:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 175210496. Throughput: 0: 1657.9, 1: 1650.0. Samples: 43806528. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) -[2023-10-14 17:01:08,165][74987] Avg episode reward: [(0, '28.810'), (1, '30.640')] -[2023-10-14 17:01:10,597][75949] Updated weights for policy 0, policy_version 85671 (0.0008) -[2023-10-14 17:01:10,966][75949] Updated weights for policy 0, policy_version 85681 (0.0007) -[2023-10-14 17:01:10,975][75950] Updated weights for policy 1, policy_version 85450 (0.0007) -[2023-10-14 17:01:11,340][75950] Updated weights for policy 1, policy_version 85460 (0.0008) -[2023-10-14 17:01:11,341][75949] Updated weights for policy 0, policy_version 85691 (0.0008) -[2023-10-14 17:01:11,701][75950] Updated weights for policy 1, policy_version 85470 (0.0010) -[2023-10-14 17:01:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 175276032. Throughput: 0: 1681.9, 1: 1664.1. Samples: 43826786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:01:13,164][74987] Avg episode reward: [(0, '26.870'), (1, '34.090')] -[2023-10-14 17:01:15,490][75949] Updated weights for policy 0, policy_version 85701 (0.0008) -[2023-10-14 17:01:15,758][75950] Updated weights for policy 1, policy_version 85480 (0.0009) -[2023-10-14 17:01:15,852][75949] Updated weights for policy 0, policy_version 85711 (0.0007) -[2023-10-14 17:01:16,125][75950] Updated weights for policy 1, policy_version 85490 (0.0008) -[2023-10-14 17:01:16,216][75949] Updated weights for policy 0, policy_version 85721 (0.0008) -[2023-10-14 17:01:16,493][75950] Updated weights for policy 1, policy_version 85500 (0.0009) -[2023-10-14 17:01:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175341568. Throughput: 0: 1675.3, 1: 1661.3. Samples: 43837904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:01:18,164][74987] Avg episode reward: [(0, '28.730'), (1, '35.670')] -[2023-10-14 17:01:20,291][75949] Updated weights for policy 0, policy_version 85731 (0.0008) -[2023-10-14 17:01:20,599][75950] Updated weights for policy 1, policy_version 85510 (0.0008) -[2023-10-14 17:01:20,667][75949] Updated weights for policy 0, policy_version 85741 (0.0008) -[2023-10-14 17:01:20,965][75950] Updated weights for policy 1, policy_version 85520 (0.0009) -[2023-10-14 17:01:21,032][75949] Updated weights for policy 0, policy_version 85751 (0.0010) -[2023-10-14 17:01:21,323][75950] Updated weights for policy 1, policy_version 85530 (0.0009) -[2023-10-14 17:01:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175407104. Throughput: 0: 1665.9, 1: 1659.1. Samples: 43856762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:01:23,164][74987] Avg episode reward: [(0, '26.560'), (1, '31.970')] -[2023-10-14 17:01:25,075][75949] Updated weights for policy 0, policy_version 85761 (0.0009) -[2023-10-14 17:01:25,411][75950] Updated weights for policy 1, policy_version 85540 (0.0008) -[2023-10-14 17:01:25,446][75949] Updated weights for policy 0, policy_version 85771 (0.0009) -[2023-10-14 17:01:25,812][75949] Updated weights for policy 0, policy_version 85781 (0.0009) -[2023-10-14 17:01:25,817][75950] Updated weights for policy 1, policy_version 85550 (0.0008) -[2023-10-14 17:01:26,180][75950] Updated weights for policy 1, policy_version 85560 (0.0008) -[2023-10-14 17:01:26,184][75949] Updated weights for policy 0, policy_version 85791 (0.0008) -[2023-10-14 17:01:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175472640. Throughput: 0: 1683.6, 1: 1675.8. Samples: 43877478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:01:28,164][74987] Avg episode reward: [(0, '28.060'), (1, '33.900')] -[2023-10-14 17:01:30,236][75950] Updated weights for policy 1, policy_version 85570 (0.0008) -[2023-10-14 17:01:30,404][75949] Updated weights for policy 0, policy_version 85801 (0.0007) -[2023-10-14 17:01:30,603][75950] Updated weights for policy 1, policy_version 85580 (0.0007) -[2023-10-14 17:01:30,779][75949] Updated weights for policy 0, policy_version 85811 (0.0008) -[2023-10-14 17:01:30,969][75950] Updated weights for policy 1, policy_version 85590 (0.0008) -[2023-10-14 17:01:31,139][75949] Updated weights for policy 0, policy_version 85821 (0.0009) -[2023-10-14 17:01:31,336][75950] Updated weights for policy 1, policy_version 85600 (0.0010) -[2023-10-14 17:01:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175538176. Throughput: 0: 1666.0, 1: 1668.4. Samples: 43888040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:01:33,164][74987] Avg episode reward: [(0, '27.590'), (1, '35.040')] -[2023-10-14 17:01:35,172][75949] Updated weights for policy 0, policy_version 85831 (0.0008) -[2023-10-14 17:01:35,380][75950] Updated weights for policy 1, policy_version 85610 (0.0008) -[2023-10-14 17:01:35,542][75949] Updated weights for policy 0, policy_version 85841 (0.0008) -[2023-10-14 17:01:35,741][75950] Updated weights for policy 1, policy_version 85620 (0.0007) -[2023-10-14 17:01:35,896][75949] Updated weights for policy 0, policy_version 85851 (0.0009) -[2023-10-14 17:01:36,102][75950] Updated weights for policy 1, policy_version 85630 (0.0007) -[2023-10-14 17:01:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175603712. Throughput: 0: 1675.2, 1: 1672.3. Samples: 43907312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:01:38,165][74987] Avg episode reward: [(0, '29.220'), (1, '34.160')] -[2023-10-14 17:01:40,038][75949] Updated weights for policy 0, policy_version 85861 (0.0008) -[2023-10-14 17:01:40,340][75950] Updated weights for policy 1, policy_version 85640 (0.0008) -[2023-10-14 17:01:40,427][75949] Updated weights for policy 0, policy_version 85871 (0.0010) -[2023-10-14 17:01:40,694][75950] Updated weights for policy 1, policy_version 85650 (0.0008) -[2023-10-14 17:01:40,785][75949] Updated weights for policy 0, policy_version 85881 (0.0008) -[2023-10-14 17:01:41,064][75950] Updated weights for policy 1, policy_version 85660 (0.0008) -[2023-10-14 17:01:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 175669248. Throughput: 0: 1686.3, 1: 1678.3. Samples: 43927682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:01:43,164][74987] Avg episode reward: [(0, '26.490'), (1, '32.930')] -[2023-10-14 17:01:44,786][75949] Updated weights for policy 0, policy_version 85891 (0.0008) -[2023-10-14 17:01:45,122][75950] Updated weights for policy 1, policy_version 85670 (0.0008) -[2023-10-14 17:01:45,150][75949] Updated weights for policy 0, policy_version 85901 (0.0007) -[2023-10-14 17:01:45,484][75950] Updated weights for policy 1, policy_version 85680 (0.0008) -[2023-10-14 17:01:45,514][75949] Updated weights for policy 0, policy_version 85911 (0.0007) -[2023-10-14 17:01:45,859][75950] Updated weights for policy 1, policy_version 85690 (0.0009) -[2023-10-14 17:01:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175734784. Throughput: 0: 1664.5, 1: 1657.8. Samples: 43937360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:01:48,164][74987] Avg episode reward: [(0, '29.520'), (1, '35.000')] -[2023-10-14 17:01:49,470][75949] Updated weights for policy 0, policy_version 85921 (0.0007) -[2023-10-14 17:01:49,747][75950] Updated weights for policy 1, policy_version 85700 (0.0008) -[2023-10-14 17:01:49,828][75949] Updated weights for policy 0, policy_version 85931 (0.0007) -[2023-10-14 17:01:50,101][75950] Updated weights for policy 1, policy_version 85710 (0.0008) -[2023-10-14 17:01:50,206][75949] Updated weights for policy 0, policy_version 85941 (0.0008) -[2023-10-14 17:01:50,462][75950] Updated weights for policy 1, policy_version 85720 (0.0009) -[2023-10-14 17:01:50,572][75949] Updated weights for policy 0, policy_version 85951 (0.0008) -[2023-10-14 17:01:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175800320. Throughput: 0: 1679.5, 1: 1672.1. Samples: 43957348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:01:53,164][74987] Avg episode reward: [(0, '28.080'), (1, '34.180')] -[2023-10-14 17:01:54,513][75950] Updated weights for policy 1, policy_version 85730 (0.0007) -[2023-10-14 17:01:54,622][75949] Updated weights for policy 0, policy_version 85961 (0.0007) -[2023-10-14 17:01:54,878][75950] Updated weights for policy 1, policy_version 85740 (0.0008) -[2023-10-14 17:01:54,993][75949] Updated weights for policy 0, policy_version 85971 (0.0008) -[2023-10-14 17:01:55,252][75950] Updated weights for policy 1, policy_version 85750 (0.0008) -[2023-10-14 17:01:55,358][75949] Updated weights for policy 0, policy_version 85981 (0.0009) -[2023-10-14 17:01:55,609][75950] Updated weights for policy 1, policy_version 85760 (0.0007) -[2023-10-14 17:01:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 175865856. Throughput: 0: 1687.4, 1: 1676.9. Samples: 43978180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:01:58,165][74987] Avg episode reward: [(0, '27.950'), (1, '31.070')] -[2023-10-14 17:01:59,404][75949] Updated weights for policy 0, policy_version 85991 (0.0008) -[2023-10-14 17:01:59,715][75950] Updated weights for policy 1, policy_version 85770 (0.0008) -[2023-10-14 17:01:59,769][75949] Updated weights for policy 0, policy_version 86001 (0.0009) -[2023-10-14 17:02:00,087][75950] Updated weights for policy 1, policy_version 85780 (0.0008) -[2023-10-14 17:02:00,142][75949] Updated weights for policy 0, policy_version 86011 (0.0008) -[2023-10-14 17:02:00,438][75950] Updated weights for policy 1, policy_version 85790 (0.0010) -[2023-10-14 17:02:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175931392. Throughput: 0: 1666.2, 1: 1651.0. Samples: 43987178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:02:03,164][74987] Avg episode reward: [(0, '28.450'), (1, '32.550')] -[2023-10-14 17:02:04,129][75949] Updated weights for policy 0, policy_version 86021 (0.0007) -[2023-10-14 17:02:04,490][75949] Updated weights for policy 0, policy_version 86031 (0.0008) -[2023-10-14 17:02:04,638][75950] Updated weights for policy 1, policy_version 85800 (0.0008) -[2023-10-14 17:02:04,857][75949] Updated weights for policy 0, policy_version 86041 (0.0009) -[2023-10-14 17:02:05,008][75950] Updated weights for policy 1, policy_version 85810 (0.0009) -[2023-10-14 17:02:05,369][75950] Updated weights for policy 1, policy_version 85820 (0.0010) -[2023-10-14 17:02:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175996928. Throughput: 0: 1686.8, 1: 1670.8. Samples: 44007858. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) -[2023-10-14 17:02:08,164][74987] Avg episode reward: [(0, '26.660'), (1, '34.830')] -[2023-10-14 17:02:08,920][75949] Updated weights for policy 0, policy_version 86051 (0.0007) -[2023-10-14 17:02:09,298][75949] Updated weights for policy 0, policy_version 86061 (0.0009) -[2023-10-14 17:02:09,577][75950] Updated weights for policy 1, policy_version 85830 (0.0008) -[2023-10-14 17:02:09,669][75949] Updated weights for policy 0, policy_version 86071 (0.0008) -[2023-10-14 17:02:09,946][75950] Updated weights for policy 1, policy_version 85840 (0.0007) -[2023-10-14 17:02:10,310][75950] Updated weights for policy 1, policy_version 85850 (0.0008) -[2023-10-14 17:02:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176062464. Throughput: 0: 1688.9, 1: 1672.5. Samples: 44028742. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) -[2023-10-14 17:02:13,164][74987] Avg episode reward: [(0, '28.960'), (1, '33.540')] -[2023-10-14 17:02:13,713][75949] Updated weights for policy 0, policy_version 86081 (0.0009) -[2023-10-14 17:02:14,082][75949] Updated weights for policy 0, policy_version 86091 (0.0009) -[2023-10-14 17:02:14,221][75950] Updated weights for policy 1, policy_version 85860 (0.0008) -[2023-10-14 17:02:14,456][75949] Updated weights for policy 0, policy_version 86101 (0.0008) -[2023-10-14 17:02:14,591][75950] Updated weights for policy 1, policy_version 85870 (0.0008) -[2023-10-14 17:02:14,827][75949] Updated weights for policy 0, policy_version 86111 (0.0008) -[2023-10-14 17:02:14,959][75950] Updated weights for policy 1, policy_version 85880 (0.0009) -[2023-10-14 17:02:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 176128000. Throughput: 0: 1676.6, 1: 1655.1. Samples: 44037966. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) -[2023-10-14 17:02:18,165][74987] Avg episode reward: [(0, '26.560'), (1, '32.070')] -[2023-10-14 17:02:18,926][75949] Updated weights for policy 0, policy_version 86121 (0.0007) -[2023-10-14 17:02:19,255][75950] Updated weights for policy 1, policy_version 85890 (0.0009) -[2023-10-14 17:02:19,296][75949] Updated weights for policy 0, policy_version 86131 (0.0008) -[2023-10-14 17:02:19,612][75950] Updated weights for policy 1, policy_version 85900 (0.0008) -[2023-10-14 17:02:19,668][75949] Updated weights for policy 0, policy_version 86141 (0.0008) -[2023-10-14 17:02:19,994][75950] Updated weights for policy 1, policy_version 85910 (0.0008) -[2023-10-14 17:02:20,360][75950] Updated weights for policy 1, policy_version 85920 (0.0007) -[2023-10-14 17:02:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176193536. Throughput: 0: 1693.3, 1: 1667.9. Samples: 44058566. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) -[2023-10-14 17:02:23,164][74987] Avg episode reward: [(0, '29.450'), (1, '35.130')] -[2023-10-14 17:02:23,741][75949] Updated weights for policy 0, policy_version 86151 (0.0008) -[2023-10-14 17:02:24,101][75949] Updated weights for policy 0, policy_version 86161 (0.0007) -[2023-10-14 17:02:24,408][75950] Updated weights for policy 1, policy_version 85930 (0.0008) -[2023-10-14 17:02:24,473][75949] Updated weights for policy 0, policy_version 86171 (0.0008) -[2023-10-14 17:02:24,779][75950] Updated weights for policy 1, policy_version 85940 (0.0007) -[2023-10-14 17:02:25,149][75950] Updated weights for policy 1, policy_version 85950 (0.0008) -[2023-10-14 17:02:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176259072. Throughput: 0: 1695.0, 1: 1680.1. Samples: 44079564. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) -[2023-10-14 17:02:28,165][74987] Avg episode reward: [(0, '25.170'), (1, '34.470')] -[2023-10-14 17:02:28,614][75949] Updated weights for policy 0, policy_version 86181 (0.0010) -[2023-10-14 17:02:29,006][75949] Updated weights for policy 0, policy_version 86191 (0.0010) -[2023-10-14 17:02:29,212][75950] Updated weights for policy 1, policy_version 85960 (0.0008) -[2023-10-14 17:02:29,374][75949] Updated weights for policy 0, policy_version 86201 (0.0008) -[2023-10-14 17:02:29,583][75950] Updated weights for policy 1, policy_version 85970 (0.0008) -[2023-10-14 17:02:29,949][75950] Updated weights for policy 1, policy_version 85980 (0.0007) -[2023-10-14 17:02:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176324608. Throughput: 0: 1690.7, 1: 1670.6. Samples: 44088616. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) -[2023-10-14 17:02:33,164][74987] Avg episode reward: [(0, '28.970'), (1, '32.680')] -[2023-10-14 17:02:33,521][75949] Updated weights for policy 0, policy_version 86211 (0.0009) -[2023-10-14 17:02:33,877][75949] Updated weights for policy 0, policy_version 86221 (0.0010) -[2023-10-14 17:02:33,988][75950] Updated weights for policy 1, policy_version 85990 (0.0009) -[2023-10-14 17:02:34,246][75949] Updated weights for policy 0, policy_version 86231 (0.0008) -[2023-10-14 17:02:34,356][75950] Updated weights for policy 1, policy_version 86000 (0.0009) -[2023-10-14 17:02:34,714][75950] Updated weights for policy 1, policy_version 86010 (0.0008) -[2023-10-14 17:02:38,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176390144. Throughput: 0: 1695.9, 1: 1678.8. Samples: 44109208. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) -[2023-10-14 17:02:38,164][74987] Avg episode reward: [(0, '25.720'), (1, '32.790')] -[2023-10-14 17:02:38,351][75949] Updated weights for policy 0, policy_version 86241 (0.0008) -[2023-10-14 17:02:38,704][75950] Updated weights for policy 1, policy_version 86020 (0.0008) -[2023-10-14 17:02:38,721][75949] Updated weights for policy 0, policy_version 86251 (0.0007) -[2023-10-14 17:02:39,067][75950] Updated weights for policy 1, policy_version 86030 (0.0007) -[2023-10-14 17:02:39,098][75949] Updated weights for policy 0, policy_version 86261 (0.0007) -[2023-10-14 17:02:39,424][75950] Updated weights for policy 1, policy_version 86040 (0.0007) -[2023-10-14 17:02:39,459][75949] Updated weights for policy 0, policy_version 86271 (0.0007) -[2023-10-14 17:02:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176455680. Throughput: 0: 1690.0, 1: 1682.6. Samples: 44129946. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) -[2023-10-14 17:02:43,164][74987] Avg episode reward: [(0, '30.220'), (1, '37.530')] -[2023-10-14 17:02:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000086048_88113152.pth... -[2023-10-14 17:02:43,209][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000084512_86540288.pth -[2023-10-14 17:02:43,213][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000086048_88113152.pth -[2023-10-14 17:02:43,455][75950] Updated weights for policy 1, policy_version 86050 (0.0009) -[2023-10-14 17:02:43,529][75949] Updated weights for policy 0, policy_version 86281 (0.0009) -[2023-10-14 17:02:43,815][75950] Updated weights for policy 1, policy_version 86060 (0.0008) -[2023-10-14 17:02:43,901][75949] Updated weights for policy 0, policy_version 86291 (0.0009) -[2023-10-14 17:02:44,187][75950] Updated weights for policy 1, policy_version 86070 (0.0007) -[2023-10-14 17:02:44,268][75949] Updated weights for policy 0, policy_version 86301 (0.0007) -[2023-10-14 17:02:44,373][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000086304_88375296.pth... -[2023-10-14 17:02:44,402][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000084704_86736896.pth -[2023-10-14 17:02:44,406][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000086304_88375296.pth -[2023-10-14 17:02:44,554][75950] Updated weights for policy 1, policy_version 86080 (0.0008) -[2023-10-14 17:02:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176521216. Throughput: 0: 1688.7, 1: 1684.6. Samples: 44138974. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) -[2023-10-14 17:02:48,164][74987] Avg episode reward: [(0, '25.610'), (1, '35.130')] -[2023-10-14 17:02:48,406][75949] Updated weights for policy 0, policy_version 86311 (0.0009) -[2023-10-14 17:02:48,674][75950] Updated weights for policy 1, policy_version 86090 (0.0008) -[2023-10-14 17:02:48,779][75949] Updated weights for policy 0, policy_version 86321 (0.0010) -[2023-10-14 17:02:49,037][75950] Updated weights for policy 1, policy_version 86100 (0.0009) -[2023-10-14 17:02:49,142][75949] Updated weights for policy 0, policy_version 86331 (0.0007) -[2023-10-14 17:02:49,408][75950] Updated weights for policy 1, policy_version 86110 (0.0008) -[2023-10-14 17:02:53,161][75949] Updated weights for policy 0, policy_version 86341 (0.0010) -[2023-10-14 17:02:53,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176586752. Throughput: 0: 1686.8, 1: 1683.1. Samples: 44159504. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) -[2023-10-14 17:02:53,164][74987] Avg episode reward: [(0, '28.950'), (1, '32.360')] -[2023-10-14 17:02:53,527][75949] Updated weights for policy 0, policy_version 86351 (0.0008) -[2023-10-14 17:02:53,695][75950] Updated weights for policy 1, policy_version 86120 (0.0009) -[2023-10-14 17:02:53,901][75949] Updated weights for policy 0, policy_version 86361 (0.0007) -[2023-10-14 17:02:54,052][75950] Updated weights for policy 1, policy_version 86130 (0.0009) -[2023-10-14 17:02:54,427][75950] Updated weights for policy 1, policy_version 86140 (0.0009) -[2023-10-14 17:02:58,102][75949] Updated weights for policy 0, policy_version 86371 (0.0008) -[2023-10-14 17:02:58,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 176652288. Throughput: 0: 1675.2, 1: 1680.0. Samples: 44179728. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) -[2023-10-14 17:02:58,165][74987] Avg episode reward: [(0, '25.380'), (1, '34.500')] -[2023-10-14 17:02:58,469][75949] Updated weights for policy 0, policy_version 86381 (0.0008) -[2023-10-14 17:02:58,686][75950] Updated weights for policy 1, policy_version 86150 (0.0008) -[2023-10-14 17:02:58,833][75949] Updated weights for policy 0, policy_version 86391 (0.0008) -[2023-10-14 17:02:59,054][75950] Updated weights for policy 1, policy_version 86160 (0.0008) -[2023-10-14 17:02:59,418][75950] Updated weights for policy 1, policy_version 86170 (0.0008) -[2023-10-14 17:03:02,740][75949] Updated weights for policy 0, policy_version 86401 (0.0008) -[2023-10-14 17:03:03,099][75949] Updated weights for policy 0, policy_version 86411 (0.0010) -[2023-10-14 17:03:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176717824. Throughput: 0: 1675.9, 1: 1678.8. Samples: 44188926. Policy #0 lag: (min: 26.0, avg: 28.5, max: 58.0) -[2023-10-14 17:03:03,165][74987] Avg episode reward: [(0, '28.990'), (1, '33.760')] -[2023-10-14 17:03:03,474][75949] Updated weights for policy 0, policy_version 86421 (0.0008) -[2023-10-14 17:03:03,528][75950] Updated weights for policy 1, policy_version 86180 (0.0009) -[2023-10-14 17:03:03,835][75949] Updated weights for policy 0, policy_version 86431 (0.0009) -[2023-10-14 17:03:03,927][75950] Updated weights for policy 1, policy_version 86190 (0.0009) -[2023-10-14 17:03:04,290][75950] Updated weights for policy 1, policy_version 86200 (0.0009) -[2023-10-14 17:03:08,000][75949] Updated weights for policy 0, policy_version 86441 (0.0007) -[2023-10-14 17:03:08,147][75950] Updated weights for policy 1, policy_version 86210 (0.0009) -[2023-10-14 17:03:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176783360. Throughput: 0: 1672.1, 1: 1680.9. Samples: 44209450. Policy #0 lag: (min: 26.0, avg: 28.5, max: 58.0) -[2023-10-14 17:03:08,164][74987] Avg episode reward: [(0, '26.370'), (1, '34.310')] -[2023-10-14 17:03:08,370][75949] Updated weights for policy 0, policy_version 86451 (0.0009) -[2023-10-14 17:03:08,517][75950] Updated weights for policy 1, policy_version 86220 (0.0007) -[2023-10-14 17:03:08,750][75949] Updated weights for policy 0, policy_version 86461 (0.0008) -[2023-10-14 17:03:08,884][75950] Updated weights for policy 1, policy_version 86230 (0.0008) -[2023-10-14 17:03:09,250][75950] Updated weights for policy 1, policy_version 86240 (0.0008) -[2023-10-14 17:03:12,713][75949] Updated weights for policy 0, policy_version 86471 (0.0008) -[2023-10-14 17:03:13,078][75949] Updated weights for policy 0, policy_version 86481 (0.0008) -[2023-10-14 17:03:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176848896. Throughput: 0: 1667.9, 1: 1674.5. Samples: 44229968. Policy #0 lag: (min: 26.0, avg: 28.5, max: 58.0) -[2023-10-14 17:03:13,164][74987] Avg episode reward: [(0, '26.550'), (1, '32.250')] -[2023-10-14 17:03:13,333][75950] Updated weights for policy 1, policy_version 86250 (0.0008) -[2023-10-14 17:03:13,461][75949] Updated weights for policy 0, policy_version 86491 (0.0008) -[2023-10-14 17:03:13,699][75950] Updated weights for policy 1, policy_version 86260 (0.0008) -[2023-10-14 17:03:14,074][75950] Updated weights for policy 1, policy_version 86270 (0.0009) -[2023-10-14 17:03:17,860][75949] Updated weights for policy 0, policy_version 86501 (0.0007) -[2023-10-14 17:03:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176914432. Throughput: 0: 1674.7, 1: 1670.9. Samples: 44239166. Policy #0 lag: (min: 26.0, avg: 28.5, max: 58.0) -[2023-10-14 17:03:18,165][74987] Avg episode reward: [(0, '27.610'), (1, '34.280')] -[2023-10-14 17:03:18,243][75949] Updated weights for policy 0, policy_version 86511 (0.0007) -[2023-10-14 17:03:18,259][75950] Updated weights for policy 1, policy_version 86280 (0.0009) -[2023-10-14 17:03:18,600][75949] Updated weights for policy 0, policy_version 86521 (0.0008) -[2023-10-14 17:03:18,628][75950] Updated weights for policy 1, policy_version 86290 (0.0009) -[2023-10-14 17:03:18,995][75950] Updated weights for policy 1, policy_version 86300 (0.0009) -[2023-10-14 17:03:22,680][75949] Updated weights for policy 0, policy_version 86531 (0.0008) -[2023-10-14 17:03:23,027][75950] Updated weights for policy 1, policy_version 86310 (0.0008) -[2023-10-14 17:03:23,054][75949] Updated weights for policy 0, policy_version 86541 (0.0007) -[2023-10-14 17:03:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176979968. Throughput: 0: 1668.9, 1: 1666.2. Samples: 44259288. Policy #0 lag: (min: 26.0, avg: 28.5, max: 58.0) -[2023-10-14 17:03:23,164][74987] Avg episode reward: [(0, '25.100'), (1, '36.090')] -[2023-10-14 17:03:23,394][75950] Updated weights for policy 1, policy_version 86320 (0.0008) -[2023-10-14 17:03:23,426][75949] Updated weights for policy 0, policy_version 86551 (0.0007) -[2023-10-14 17:03:23,760][75950] Updated weights for policy 1, policy_version 86330 (0.0008) -[2023-10-14 17:03:27,526][75949] Updated weights for policy 0, policy_version 86561 (0.0008) -[2023-10-14 17:03:27,880][75950] Updated weights for policy 1, policy_version 86340 (0.0009) -[2023-10-14 17:03:27,889][75949] Updated weights for policy 0, policy_version 86571 (0.0009) -[2023-10-14 17:03:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 177045504. Throughput: 0: 1660.3, 1: 1663.9. Samples: 44279534. Policy #0 lag: (min: 26.0, avg: 28.5, max: 58.0) -[2023-10-14 17:03:28,165][74987] Avg episode reward: [(0, '28.870'), (1, '34.210')] -[2023-10-14 17:03:28,254][75950] Updated weights for policy 1, policy_version 86350 (0.0008) -[2023-10-14 17:03:28,260][75949] Updated weights for policy 0, policy_version 86581 (0.0009) -[2023-10-14 17:03:28,623][75950] Updated weights for policy 1, policy_version 86360 (0.0007) -[2023-10-14 17:03:28,630][75949] Updated weights for policy 0, policy_version 86591 (0.0008) -[2023-10-14 17:03:32,670][75949] Updated weights for policy 0, policy_version 86601 (0.0009) -[2023-10-14 17:03:32,690][75950] Updated weights for policy 1, policy_version 86370 (0.0008) -[2023-10-14 17:03:33,036][75949] Updated weights for policy 0, policy_version 86611 (0.0009) -[2023-10-14 17:03:33,048][75950] Updated weights for policy 1, policy_version 86380 (0.0007) -[2023-10-14 17:03:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 177111040. Throughput: 0: 1664.3, 1: 1662.5. Samples: 44288680. Policy #0 lag: (min: 26.0, avg: 28.5, max: 58.0) -[2023-10-14 17:03:33,164][74987] Avg episode reward: [(0, '25.480'), (1, '33.300')] -[2023-10-14 17:03:33,406][75949] Updated weights for policy 0, policy_version 86621 (0.0008) -[2023-10-14 17:03:33,409][75950] Updated weights for policy 1, policy_version 86390 (0.0009) -[2023-10-14 17:03:33,780][75950] Updated weights for policy 1, policy_version 86400 (0.0008) -[2023-10-14 17:03:37,732][75949] Updated weights for policy 0, policy_version 86631 (0.0008) -[2023-10-14 17:03:37,918][75950] Updated weights for policy 1, policy_version 86410 (0.0008) -[2023-10-14 17:03:38,102][75949] Updated weights for policy 0, policy_version 86641 (0.0009) -[2023-10-14 17:03:38,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 177176576. Throughput: 0: 1665.3, 1: 1663.8. Samples: 44309316. Policy #0 lag: (min: 26.0, avg: 28.5, max: 58.0) -[2023-10-14 17:03:38,164][74987] Avg episode reward: [(0, '29.240'), (1, '35.210')] -[2023-10-14 17:03:38,284][75950] Updated weights for policy 1, policy_version 86420 (0.0008) -[2023-10-14 17:03:38,467][75949] Updated weights for policy 0, policy_version 86651 (0.0009) -[2023-10-14 17:03:38,652][75950] Updated weights for policy 1, policy_version 86430 (0.0007) -[2023-10-14 17:03:42,427][75949] Updated weights for policy 0, policy_version 86661 (0.0009) -[2023-10-14 17:03:42,796][75949] Updated weights for policy 0, policy_version 86671 (0.0009) -[2023-10-14 17:03:42,845][75950] Updated weights for policy 1, policy_version 86440 (0.0008) -[2023-10-14 17:03:43,156][75949] Updated weights for policy 0, policy_version 86681 (0.0008) -[2023-10-14 17:03:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 177242112. Throughput: 0: 1665.3, 1: 1664.4. Samples: 44329564. Policy #0 lag: (min: 26.0, avg: 28.5, max: 58.0) -[2023-10-14 17:03:43,164][74987] Avg episode reward: [(0, '26.400'), (1, '34.070')] -[2023-10-14 17:03:43,206][75950] Updated weights for policy 1, policy_version 86450 (0.0008) -[2023-10-14 17:03:43,575][75950] Updated weights for policy 1, policy_version 86460 (0.0008) -[2023-10-14 17:03:47,037][75949] Updated weights for policy 0, policy_version 86691 (0.0008) -[2023-10-14 17:03:47,396][75949] Updated weights for policy 0, policy_version 86701 (0.0008) -[2023-10-14 17:03:47,601][75950] Updated weights for policy 1, policy_version 86470 (0.0009) -[2023-10-14 17:03:47,761][75949] Updated weights for policy 0, policy_version 86711 (0.0007) -[2023-10-14 17:03:47,966][75950] Updated weights for policy 1, policy_version 86480 (0.0010) -[2023-10-14 17:03:48,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 177340416. Throughput: 0: 1683.7, 1: 1665.7. Samples: 44339648. Policy #0 lag: (min: 26.0, avg: 28.5, max: 58.0) -[2023-10-14 17:03:48,164][74987] Avg episode reward: [(0, '29.520'), (1, '33.580')] -[2023-10-14 17:03:48,332][75950] Updated weights for policy 1, policy_version 86490 (0.0009) -[2023-10-14 17:03:51,676][75949] Updated weights for policy 0, policy_version 86721 (0.0007) -[2023-10-14 17:03:52,041][75949] Updated weights for policy 0, policy_version 86731 (0.0007) -[2023-10-14 17:03:52,413][75949] Updated weights for policy 0, policy_version 86741 (0.0007) -[2023-10-14 17:03:52,493][75950] Updated weights for policy 1, policy_version 86500 (0.0008) -[2023-10-14 17:03:52,777][75949] Updated weights for policy 0, policy_version 86751 (0.0007) -[2023-10-14 17:03:52,866][75950] Updated weights for policy 1, policy_version 86510 (0.0007) -[2023-10-14 17:03:53,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 177405952. Throughput: 0: 1686.6, 1: 1667.8. Samples: 44360396. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:03:53,164][74987] Avg episode reward: [(0, '25.960'), (1, '32.180')] -[2023-10-14 17:03:53,231][75950] Updated weights for policy 1, policy_version 86520 (0.0008) -[2023-10-14 17:03:56,833][75949] Updated weights for policy 0, policy_version 86761 (0.0011) -[2023-10-14 17:03:57,193][75949] Updated weights for policy 0, policy_version 86771 (0.0010) -[2023-10-14 17:03:57,222][75950] Updated weights for policy 1, policy_version 86530 (0.0007) -[2023-10-14 17:03:57,566][75949] Updated weights for policy 0, policy_version 86781 (0.0009) -[2023-10-14 17:03:57,592][75950] Updated weights for policy 1, policy_version 86540 (0.0008) -[2023-10-14 17:03:57,955][75950] Updated weights for policy 1, policy_version 86550 (0.0007) -[2023-10-14 17:03:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 177471488. Throughput: 0: 1661.1, 1: 1660.9. Samples: 44379456. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:03:58,164][74987] Avg episode reward: [(0, '28.040'), (1, '34.470')] -[2023-10-14 17:03:58,320][75950] Updated weights for policy 1, policy_version 86560 (0.0007) -[2023-10-14 17:04:01,604][75949] Updated weights for policy 0, policy_version 86791 (0.0010) -[2023-10-14 17:04:01,974][75949] Updated weights for policy 0, policy_version 86801 (0.0009) -[2023-10-14 17:04:02,310][75950] Updated weights for policy 1, policy_version 86570 (0.0007) -[2023-10-14 17:04:02,344][75949] Updated weights for policy 0, policy_version 86811 (0.0008) -[2023-10-14 17:04:02,676][75950] Updated weights for policy 1, policy_version 86580 (0.0009) -[2023-10-14 17:04:03,040][75950] Updated weights for policy 1, policy_version 86590 (0.0009) -[2023-10-14 17:04:03,164][74987] Fps is (10 sec: 16383.6, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 177569792. Throughput: 0: 1683.5, 1: 1677.5. Samples: 44390410. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:04:03,165][74987] Avg episode reward: [(0, '27.860'), (1, '33.110')] -[2023-10-14 17:04:06,584][75949] Updated weights for policy 0, policy_version 86821 (0.0010) -[2023-10-14 17:04:06,977][75949] Updated weights for policy 0, policy_version 86831 (0.0010) -[2023-10-14 17:04:07,070][75950] Updated weights for policy 1, policy_version 86600 (0.0008) -[2023-10-14 17:04:07,337][75949] Updated weights for policy 0, policy_version 86841 (0.0008) -[2023-10-14 17:04:07,433][75950] Updated weights for policy 1, policy_version 86610 (0.0008) -[2023-10-14 17:04:07,790][75950] Updated weights for policy 1, policy_version 86620 (0.0008) -[2023-10-14 17:04:08,164][74987] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 177635328. Throughput: 0: 1681.5, 1: 1681.5. Samples: 44410628. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:04:08,164][74987] Avg episode reward: [(0, '28.390'), (1, '32.120')] -[2023-10-14 17:04:11,256][75949] Updated weights for policy 0, policy_version 86851 (0.0008) -[2023-10-14 17:04:11,623][75949] Updated weights for policy 0, policy_version 86861 (0.0009) -[2023-10-14 17:04:11,985][75949] Updated weights for policy 0, policy_version 86871 (0.0009) -[2023-10-14 17:04:12,050][75950] Updated weights for policy 1, policy_version 86630 (0.0008) -[2023-10-14 17:04:12,414][75950] Updated weights for policy 1, policy_version 86640 (0.0009) -[2023-10-14 17:04:12,771][75950] Updated weights for policy 1, policy_version 86650 (0.0009) -[2023-10-14 17:04:13,163][74987] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 177700864. Throughput: 0: 1670.8, 1: 1658.8. Samples: 44429362. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:04:13,164][74987] Avg episode reward: [(0, '27.540'), (1, '33.990')] -[2023-10-14 17:04:15,993][75949] Updated weights for policy 0, policy_version 86881 (0.0009) -[2023-10-14 17:04:16,365][75949] Updated weights for policy 0, policy_version 86891 (0.0011) -[2023-10-14 17:04:16,734][75949] Updated weights for policy 0, policy_version 86901 (0.0008) -[2023-10-14 17:04:16,980][75950] Updated weights for policy 1, policy_version 86660 (0.0007) -[2023-10-14 17:04:17,097][75949] Updated weights for policy 0, policy_version 86911 (0.0008) -[2023-10-14 17:04:17,349][75950] Updated weights for policy 1, policy_version 86670 (0.0009) -[2023-10-14 17:04:17,710][75950] Updated weights for policy 1, policy_version 86680 (0.0010) -[2023-10-14 17:04:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 177766400. Throughput: 0: 1700.0, 1: 1674.2. Samples: 44440520. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:04:18,164][74987] Avg episode reward: [(0, '26.350'), (1, '34.780')] -[2023-10-14 17:04:21,261][75949] Updated weights for policy 0, policy_version 86921 (0.0010) -[2023-10-14 17:04:21,617][75950] Updated weights for policy 1, policy_version 86690 (0.0009) -[2023-10-14 17:04:21,632][75949] Updated weights for policy 0, policy_version 86931 (0.0010) -[2023-10-14 17:04:21,990][75950] Updated weights for policy 1, policy_version 86700 (0.0007) -[2023-10-14 17:04:22,006][75949] Updated weights for policy 0, policy_version 86941 (0.0007) -[2023-10-14 17:04:22,354][75950] Updated weights for policy 1, policy_version 86710 (0.0008) -[2023-10-14 17:04:22,708][75950] Updated weights for policy 1, policy_version 86720 (0.0008) -[2023-10-14 17:04:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 177831936. Throughput: 0: 1678.8, 1: 1680.0. Samples: 44460462. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:04:23,164][74987] Avg episode reward: [(0, '28.640'), (1, '32.280')] -[2023-10-14 17:04:25,961][75949] Updated weights for policy 0, policy_version 86951 (0.0008) -[2023-10-14 17:04:26,337][75949] Updated weights for policy 0, policy_version 86961 (0.0008) -[2023-10-14 17:04:26,701][75949] Updated weights for policy 0, policy_version 86971 (0.0009) -[2023-10-14 17:04:26,851][75950] Updated weights for policy 1, policy_version 86730 (0.0010) -[2023-10-14 17:04:27,210][75950] Updated weights for policy 1, policy_version 86740 (0.0010) -[2023-10-14 17:04:27,585][75950] Updated weights for policy 1, policy_version 86750 (0.0008) -[2023-10-14 17:04:28,163][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 177897472. Throughput: 0: 1677.6, 1: 1657.9. Samples: 44479662. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:04:28,164][74987] Avg episode reward: [(0, '26.850'), (1, '32.110')] -[2023-10-14 17:04:30,831][75949] Updated weights for policy 0, policy_version 86981 (0.0007) -[2023-10-14 17:04:31,195][75949] Updated weights for policy 0, policy_version 86991 (0.0007) -[2023-10-14 17:04:31,566][75949] Updated weights for policy 0, policy_version 87001 (0.0007) -[2023-10-14 17:04:31,693][75950] Updated weights for policy 1, policy_version 86760 (0.0007) -[2023-10-14 17:04:32,049][75950] Updated weights for policy 1, policy_version 86770 (0.0007) -[2023-10-14 17:04:32,419][75950] Updated weights for policy 1, policy_version 86780 (0.0008) -[2023-10-14 17:04:33,163][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 177963008. Throughput: 0: 1684.9, 1: 1682.3. Samples: 44491172. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:04:33,164][74987] Avg episode reward: [(0, '28.740'), (1, '35.160')] -[2023-10-14 17:04:35,594][75949] Updated weights for policy 0, policy_version 87011 (0.0008) -[2023-10-14 17:04:35,965][75949] Updated weights for policy 0, policy_version 87021 (0.0008) -[2023-10-14 17:04:36,333][75949] Updated weights for policy 0, policy_version 87031 (0.0008) -[2023-10-14 17:04:36,424][75950] Updated weights for policy 1, policy_version 86790 (0.0007) -[2023-10-14 17:04:36,791][75950] Updated weights for policy 1, policy_version 86800 (0.0009) -[2023-10-14 17:04:37,164][75950] Updated weights for policy 1, policy_version 86810 (0.0008) -[2023-10-14 17:04:38,164][74987] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 178028544. Throughput: 0: 1657.8, 1: 1677.3. Samples: 44510476. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:04:38,165][74987] Avg episode reward: [(0, '26.670'), (1, '36.610')] -[2023-10-14 17:04:40,365][75949] Updated weights for policy 0, policy_version 87041 (0.0009) -[2023-10-14 17:04:40,732][75949] Updated weights for policy 0, policy_version 87051 (0.0007) -[2023-10-14 17:04:41,099][75949] Updated weights for policy 0, policy_version 87061 (0.0009) -[2023-10-14 17:04:41,358][75950] Updated weights for policy 1, policy_version 86820 (0.0008) -[2023-10-14 17:04:41,467][75949] Updated weights for policy 0, policy_version 87071 (0.0010) -[2023-10-14 17:04:41,751][75950] Updated weights for policy 1, policy_version 86830 (0.0007) -[2023-10-14 17:04:42,124][75950] Updated weights for policy 1, policy_version 86840 (0.0007) -[2023-10-14 17:04:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 178094080. Throughput: 0: 1685.1, 1: 1663.4. Samples: 44530136. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:04:43,165][74987] Avg episode reward: [(0, '29.110'), (1, '32.520')] -[2023-10-14 17:04:43,178][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000087072_89161728.pth... -[2023-10-14 17:04:43,179][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000086848_88932352.pth... -[2023-10-14 17:04:43,209][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000085504_87556096.pth -[2023-10-14 17:04:43,213][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000085280_87326720.pth -[2023-10-14 17:04:45,597][75949] Updated weights for policy 0, policy_version 87081 (0.0010) -[2023-10-14 17:04:45,954][75950] Updated weights for policy 1, policy_version 86850 (0.0009) -[2023-10-14 17:04:45,965][75949] Updated weights for policy 0, policy_version 87091 (0.0009) -[2023-10-14 17:04:46,317][75950] Updated weights for policy 1, policy_version 86860 (0.0007) -[2023-10-14 17:04:46,328][75949] Updated weights for policy 0, policy_version 87101 (0.0009) -[2023-10-14 17:04:46,677][75950] Updated weights for policy 1, policy_version 86870 (0.0011) -[2023-10-14 17:04:47,043][75950] Updated weights for policy 1, policy_version 86880 (0.0011) -[2023-10-14 17:04:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 178159616. Throughput: 0: 1674.8, 1: 1681.1. Samples: 44541422. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) -[2023-10-14 17:04:48,164][74987] Avg episode reward: [(0, '26.840'), (1, '33.770')] -[2023-10-14 17:04:50,372][75949] Updated weights for policy 0, policy_version 87111 (0.0008) -[2023-10-14 17:04:50,738][75949] Updated weights for policy 0, policy_version 87121 (0.0008) -[2023-10-14 17:04:51,105][75949] Updated weights for policy 0, policy_version 87131 (0.0008) -[2023-10-14 17:04:51,338][75950] Updated weights for policy 1, policy_version 86890 (0.0008) -[2023-10-14 17:04:51,702][75950] Updated weights for policy 1, policy_version 86900 (0.0009) -[2023-10-14 17:04:52,066][75950] Updated weights for policy 1, policy_version 86910 (0.0007) -[2023-10-14 17:04:53,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 178225152. Throughput: 0: 1667.2, 1: 1664.0. Samples: 44560532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:04:53,164][74987] Avg episode reward: [(0, '29.410'), (1, '36.840')] -[2023-10-14 17:04:55,130][75949] Updated weights for policy 0, policy_version 87141 (0.0009) -[2023-10-14 17:04:55,511][75949] Updated weights for policy 0, policy_version 87151 (0.0008) -[2023-10-14 17:04:55,876][75949] Updated weights for policy 0, policy_version 87161 (0.0010) -[2023-10-14 17:04:56,211][75950] Updated weights for policy 1, policy_version 86920 (0.0008) -[2023-10-14 17:04:56,575][75950] Updated weights for policy 1, policy_version 86930 (0.0009) -[2023-10-14 17:04:56,954][75950] Updated weights for policy 1, policy_version 86940 (0.0011) -[2023-10-14 17:04:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 178290688. Throughput: 0: 1686.9, 1: 1673.5. Samples: 44580582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:04:58,165][74987] Avg episode reward: [(0, '27.780'), (1, '33.940')] -[2023-10-14 17:05:00,104][75949] Updated weights for policy 0, policy_version 87171 (0.0009) -[2023-10-14 17:05:00,476][75949] Updated weights for policy 0, policy_version 87181 (0.0008) -[2023-10-14 17:05:00,853][75949] Updated weights for policy 0, policy_version 87191 (0.0009) -[2023-10-14 17:05:01,064][75950] Updated weights for policy 1, policy_version 86950 (0.0010) -[2023-10-14 17:05:01,426][75950] Updated weights for policy 1, policy_version 86960 (0.0007) -[2023-10-14 17:05:01,789][75950] Updated weights for policy 1, policy_version 86970 (0.0009) -[2023-10-14 17:05:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178356224. Throughput: 0: 1669.3, 1: 1686.1. Samples: 44591514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:05:03,165][74987] Avg episode reward: [(0, '28.670'), (1, '33.730')] -[2023-10-14 17:05:04,967][75949] Updated weights for policy 0, policy_version 87201 (0.0009) -[2023-10-14 17:05:05,340][75949] Updated weights for policy 0, policy_version 87211 (0.0008) -[2023-10-14 17:05:05,711][75949] Updated weights for policy 0, policy_version 87221 (0.0008) -[2023-10-14 17:05:05,905][75950] Updated weights for policy 1, policy_version 86980 (0.0009) -[2023-10-14 17:05:06,083][75949] Updated weights for policy 0, policy_version 87231 (0.0009) -[2023-10-14 17:05:06,269][75950] Updated weights for policy 1, policy_version 86990 (0.0008) -[2023-10-14 17:05:06,628][75950] Updated weights for policy 1, policy_version 87000 (0.0008) -[2023-10-14 17:05:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178421760. Throughput: 0: 1679.5, 1: 1659.7. Samples: 44610726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:05:08,164][74987] Avg episode reward: [(0, '27.340'), (1, '34.950')] -[2023-10-14 17:05:09,991][75949] Updated weights for policy 0, policy_version 87241 (0.0008) -[2023-10-14 17:05:10,369][75949] Updated weights for policy 0, policy_version 87251 (0.0008) -[2023-10-14 17:05:10,604][75950] Updated weights for policy 1, policy_version 87010 (0.0008) -[2023-10-14 17:05:10,744][75949] Updated weights for policy 0, policy_version 87261 (0.0007) -[2023-10-14 17:05:10,970][75950] Updated weights for policy 1, policy_version 87020 (0.0009) -[2023-10-14 17:05:11,336][75950] Updated weights for policy 1, policy_version 87030 (0.0009) -[2023-10-14 17:05:11,702][75950] Updated weights for policy 1, policy_version 87040 (0.0010) -[2023-10-14 17:05:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178487296. Throughput: 0: 1688.7, 1: 1679.5. Samples: 44631230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:05:13,164][74987] Avg episode reward: [(0, '27.570'), (1, '35.380')] -[2023-10-14 17:05:14,831][75949] Updated weights for policy 0, policy_version 87271 (0.0010) -[2023-10-14 17:05:15,205][75949] Updated weights for policy 0, policy_version 87281 (0.0009) -[2023-10-14 17:05:15,572][75949] Updated weights for policy 0, policy_version 87291 (0.0007) -[2023-10-14 17:05:15,870][75950] Updated weights for policy 1, policy_version 87050 (0.0007) -[2023-10-14 17:05:16,233][75950] Updated weights for policy 1, policy_version 87060 (0.0008) -[2023-10-14 17:05:16,599][75950] Updated weights for policy 1, policy_version 87070 (0.0012) -[2023-10-14 17:05:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178552832. Throughput: 0: 1666.1, 1: 1676.9. Samples: 44641608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:05:18,165][74987] Avg episode reward: [(0, '27.190'), (1, '31.470')] -[2023-10-14 17:05:19,573][75949] Updated weights for policy 0, policy_version 87301 (0.0009) -[2023-10-14 17:05:19,946][75949] Updated weights for policy 0, policy_version 87311 (0.0011) -[2023-10-14 17:05:20,316][75949] Updated weights for policy 0, policy_version 87321 (0.0010) -[2023-10-14 17:05:20,631][75950] Updated weights for policy 1, policy_version 87080 (0.0009) -[2023-10-14 17:05:20,992][75950] Updated weights for policy 1, policy_version 87090 (0.0008) -[2023-10-14 17:05:21,351][75950] Updated weights for policy 1, policy_version 87100 (0.0009) -[2023-10-14 17:05:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178618368. Throughput: 0: 1690.3, 1: 1654.9. Samples: 44661010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:05:23,164][74987] Avg episode reward: [(0, '28.740'), (1, '33.420')] -[2023-10-14 17:05:24,350][75949] Updated weights for policy 0, policy_version 87331 (0.0007) -[2023-10-14 17:05:24,720][75949] Updated weights for policy 0, policy_version 87341 (0.0008) -[2023-10-14 17:05:25,085][75949] Updated weights for policy 0, policy_version 87351 (0.0010) -[2023-10-14 17:05:25,426][75950] Updated weights for policy 1, policy_version 87110 (0.0009) -[2023-10-14 17:05:25,792][75950] Updated weights for policy 1, policy_version 87120 (0.0008) -[2023-10-14 17:05:26,158][75950] Updated weights for policy 1, policy_version 87130 (0.0009) -[2023-10-14 17:05:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 178683904. Throughput: 0: 1688.2, 1: 1678.7. Samples: 44681646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:05:28,165][74987] Avg episode reward: [(0, '28.310'), (1, '35.300')] -[2023-10-14 17:05:29,277][75949] Updated weights for policy 0, policy_version 87361 (0.0009) -[2023-10-14 17:05:29,644][75949] Updated weights for policy 0, policy_version 87371 (0.0009) -[2023-10-14 17:05:30,012][75949] Updated weights for policy 0, policy_version 87381 (0.0010) -[2023-10-14 17:05:30,235][75950] Updated weights for policy 1, policy_version 87140 (0.0010) -[2023-10-14 17:05:30,386][75949] Updated weights for policy 0, policy_version 87391 (0.0007) -[2023-10-14 17:05:30,616][75950] Updated weights for policy 1, policy_version 87150 (0.0009) -[2023-10-14 17:05:30,984][75950] Updated weights for policy 1, policy_version 87160 (0.0008) -[2023-10-14 17:05:33,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 178749440. Throughput: 0: 1668.4, 1: 1664.8. Samples: 44691414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:05:33,165][74987] Avg episode reward: [(0, '30.370'), (1, '33.620')] -[2023-10-14 17:05:34,415][75949] Updated weights for policy 0, policy_version 87401 (0.0007) -[2023-10-14 17:05:34,788][75949] Updated weights for policy 0, policy_version 87411 (0.0007) -[2023-10-14 17:05:35,154][75949] Updated weights for policy 0, policy_version 87421 (0.0007) -[2023-10-14 17:05:35,172][75950] Updated weights for policy 1, policy_version 87170 (0.0008) -[2023-10-14 17:05:35,538][75950] Updated weights for policy 1, policy_version 87180 (0.0008) -[2023-10-14 17:05:35,901][75950] Updated weights for policy 1, policy_version 87190 (0.0009) -[2023-10-14 17:05:36,257][75950] Updated weights for policy 1, policy_version 87200 (0.0008) -[2023-10-14 17:05:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178814976. Throughput: 0: 1683.1, 1: 1662.3. Samples: 44711080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:05:38,165][74987] Avg episode reward: [(0, '27.220'), (1, '33.010')] -[2023-10-14 17:05:39,272][75949] Updated weights for policy 0, policy_version 87431 (0.0009) -[2023-10-14 17:05:39,645][75949] Updated weights for policy 0, policy_version 87441 (0.0008) -[2023-10-14 17:05:40,011][75949] Updated weights for policy 0, policy_version 87451 (0.0010) -[2023-10-14 17:05:40,392][75950] Updated weights for policy 1, policy_version 87210 (0.0008) -[2023-10-14 17:05:40,763][75950] Updated weights for policy 1, policy_version 87220 (0.0009) -[2023-10-14 17:05:41,123][75950] Updated weights for policy 1, policy_version 87230 (0.0010) -[2023-10-14 17:05:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178880512. Throughput: 0: 1687.4, 1: 1675.6. Samples: 44731918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:05:43,165][74987] Avg episode reward: [(0, '29.030'), (1, '35.150')] -[2023-10-14 17:05:44,157][75949] Updated weights for policy 0, policy_version 87461 (0.0008) -[2023-10-14 17:05:44,555][75949] Updated weights for policy 0, policy_version 87471 (0.0008) -[2023-10-14 17:05:44,926][75949] Updated weights for policy 0, policy_version 87481 (0.0009) -[2023-10-14 17:05:45,203][75950] Updated weights for policy 1, policy_version 87240 (0.0009) -[2023-10-14 17:05:45,582][75950] Updated weights for policy 1, policy_version 87250 (0.0009) -[2023-10-14 17:05:45,956][75950] Updated weights for policy 1, policy_version 87260 (0.0009) -[2023-10-14 17:05:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178946048. Throughput: 0: 1671.3, 1: 1662.2. Samples: 44741522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:05:48,164][74987] Avg episode reward: [(0, '26.190'), (1, '35.880')] -[2023-10-14 17:05:48,952][75949] Updated weights for policy 0, policy_version 87491 (0.0008) -[2023-10-14 17:05:49,320][75949] Updated weights for policy 0, policy_version 87501 (0.0008) -[2023-10-14 17:05:49,689][75949] Updated weights for policy 0, policy_version 87511 (0.0009) -[2023-10-14 17:05:50,003][75950] Updated weights for policy 1, policy_version 87270 (0.0008) -[2023-10-14 17:05:50,368][75950] Updated weights for policy 1, policy_version 87280 (0.0011) -[2023-10-14 17:05:50,738][75950] Updated weights for policy 1, policy_version 87290 (0.0009) -[2023-10-14 17:05:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179011584. Throughput: 0: 1680.2, 1: 1667.4. Samples: 44761370. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:05:53,164][74987] Avg episode reward: [(0, '26.890'), (1, '33.470')] -[2023-10-14 17:05:53,750][75949] Updated weights for policy 0, policy_version 87521 (0.0007) -[2023-10-14 17:05:54,121][75949] Updated weights for policy 0, policy_version 87531 (0.0011) -[2023-10-14 17:05:54,497][75949] Updated weights for policy 0, policy_version 87541 (0.0011) -[2023-10-14 17:05:54,864][75949] Updated weights for policy 0, policy_version 87551 (0.0009) -[2023-10-14 17:05:54,895][75950] Updated weights for policy 1, policy_version 87300 (0.0007) -[2023-10-14 17:05:55,259][75950] Updated weights for policy 1, policy_version 87310 (0.0009) -[2023-10-14 17:05:55,614][75950] Updated weights for policy 1, policy_version 87320 (0.0008) -[2023-10-14 17:05:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 179077120. Throughput: 0: 1680.0, 1: 1673.1. Samples: 44782120. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:05:58,165][74987] Avg episode reward: [(0, '26.090'), (1, '34.650')] -[2023-10-14 17:05:58,811][75949] Updated weights for policy 0, policy_version 87561 (0.0007) -[2023-10-14 17:05:59,180][75949] Updated weights for policy 0, policy_version 87571 (0.0008) -[2023-10-14 17:05:59,545][75949] Updated weights for policy 0, policy_version 87581 (0.0011) -[2023-10-14 17:05:59,793][75950] Updated weights for policy 1, policy_version 87330 (0.0008) -[2023-10-14 17:06:00,159][75950] Updated weights for policy 1, policy_version 87340 (0.0008) -[2023-10-14 17:06:00,518][75950] Updated weights for policy 1, policy_version 87350 (0.0007) -[2023-10-14 17:06:00,881][75950] Updated weights for policy 1, policy_version 87360 (0.0007) -[2023-10-14 17:06:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179142656. Throughput: 0: 1678.9, 1: 1658.2. Samples: 44791778. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:06:03,164][74987] Avg episode reward: [(0, '28.160'), (1, '36.860')] -[2023-10-14 17:06:03,544][75949] Updated weights for policy 0, policy_version 87591 (0.0009) -[2023-10-14 17:06:03,913][75949] Updated weights for policy 0, policy_version 87601 (0.0009) -[2023-10-14 17:06:04,280][75949] Updated weights for policy 0, policy_version 87611 (0.0009) -[2023-10-14 17:06:04,916][75950] Updated weights for policy 1, policy_version 87370 (0.0009) -[2023-10-14 17:06:05,287][75950] Updated weights for policy 1, policy_version 87380 (0.0009) -[2023-10-14 17:06:05,656][75950] Updated weights for policy 1, policy_version 87390 (0.0008) -[2023-10-14 17:06:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179208192. Throughput: 0: 1681.1, 1: 1677.0. Samples: 44812124. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:06:08,164][74987] Avg episode reward: [(0, '27.700'), (1, '32.180')] -[2023-10-14 17:06:08,284][75949] Updated weights for policy 0, policy_version 87621 (0.0010) -[2023-10-14 17:06:08,653][75949] Updated weights for policy 0, policy_version 87631 (0.0012) -[2023-10-14 17:06:09,018][75949] Updated weights for policy 0, policy_version 87641 (0.0007) -[2023-10-14 17:06:09,671][75950] Updated weights for policy 1, policy_version 87400 (0.0009) -[2023-10-14 17:06:10,041][75950] Updated weights for policy 1, policy_version 87410 (0.0008) -[2023-10-14 17:06:10,423][75950] Updated weights for policy 1, policy_version 87420 (0.0008) -[2023-10-14 17:06:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179273728. Throughput: 0: 1682.9, 1: 1679.9. Samples: 44832972. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:06:13,164][74987] Avg episode reward: [(0, '28.780'), (1, '33.060')] -[2023-10-14 17:06:13,333][75949] Updated weights for policy 0, policy_version 87651 (0.0008) -[2023-10-14 17:06:13,696][75949] Updated weights for policy 0, policy_version 87661 (0.0007) -[2023-10-14 17:06:14,066][75949] Updated weights for policy 0, policy_version 87671 (0.0007) -[2023-10-14 17:06:14,428][75950] Updated weights for policy 1, policy_version 87430 (0.0007) -[2023-10-14 17:06:14,785][75950] Updated weights for policy 1, policy_version 87440 (0.0009) -[2023-10-14 17:06:15,164][75950] Updated weights for policy 1, policy_version 87450 (0.0009) -[2023-10-14 17:06:18,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 179339264. Throughput: 0: 1687.3, 1: 1661.3. Samples: 44842102. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:06:18,164][74987] Avg episode reward: [(0, '26.690'), (1, '35.530')] -[2023-10-14 17:06:18,171][75949] Updated weights for policy 0, policy_version 87681 (0.0007) -[2023-10-14 17:06:18,538][75949] Updated weights for policy 0, policy_version 87691 (0.0007) -[2023-10-14 17:06:18,912][75949] Updated weights for policy 0, policy_version 87701 (0.0011) -[2023-10-14 17:06:19,276][75949] Updated weights for policy 0, policy_version 87711 (0.0009) -[2023-10-14 17:06:19,344][75950] Updated weights for policy 1, policy_version 87460 (0.0010) -[2023-10-14 17:06:19,711][75950] Updated weights for policy 1, policy_version 87470 (0.0009) -[2023-10-14 17:06:20,074][75950] Updated weights for policy 1, policy_version 87480 (0.0008) -[2023-10-14 17:06:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179404800. Throughput: 0: 1688.3, 1: 1680.1. Samples: 44862656. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:06:23,164][74987] Avg episode reward: [(0, '28.800'), (1, '36.180')] -[2023-10-14 17:06:23,342][75949] Updated weights for policy 0, policy_version 87721 (0.0009) -[2023-10-14 17:06:23,718][75949] Updated weights for policy 0, policy_version 87731 (0.0008) -[2023-10-14 17:06:24,088][75949] Updated weights for policy 0, policy_version 87741 (0.0011) -[2023-10-14 17:06:24,144][75950] Updated weights for policy 1, policy_version 87490 (0.0009) -[2023-10-14 17:06:24,561][75950] Updated weights for policy 1, policy_version 87500 (0.0007) -[2023-10-14 17:06:24,939][75950] Updated weights for policy 1, policy_version 87510 (0.0007) -[2023-10-14 17:06:25,304][75950] Updated weights for policy 1, policy_version 87520 (0.0010) -[2023-10-14 17:06:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 179470336. Throughput: 0: 1682.9, 1: 1682.2. Samples: 44883348. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:06:28,165][74987] Avg episode reward: [(0, '29.160'), (1, '35.220')] -[2023-10-14 17:06:28,210][75949] Updated weights for policy 0, policy_version 87751 (0.0009) -[2023-10-14 17:06:28,579][75949] Updated weights for policy 0, policy_version 87761 (0.0010) -[2023-10-14 17:06:28,952][75949] Updated weights for policy 0, policy_version 87771 (0.0008) -[2023-10-14 17:06:29,285][75950] Updated weights for policy 1, policy_version 87530 (0.0010) -[2023-10-14 17:06:29,658][75950] Updated weights for policy 1, policy_version 87540 (0.0011) -[2023-10-14 17:06:30,031][75950] Updated weights for policy 1, policy_version 87550 (0.0009) -[2023-10-14 17:06:32,846][75949] Updated weights for policy 0, policy_version 87781 (0.0009) -[2023-10-14 17:06:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179535872. Throughput: 0: 1687.8, 1: 1671.0. Samples: 44892668. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:06:33,164][74987] Avg episode reward: [(0, '30.370'), (1, '36.320')] -[2023-10-14 17:06:33,242][75949] Updated weights for policy 0, policy_version 87791 (0.0007) -[2023-10-14 17:06:33,609][75949] Updated weights for policy 0, policy_version 87801 (0.0009) -[2023-10-14 17:06:34,043][75950] Updated weights for policy 1, policy_version 87560 (0.0011) -[2023-10-14 17:06:34,408][75950] Updated weights for policy 1, policy_version 87570 (0.0010) -[2023-10-14 17:06:34,778][75950] Updated weights for policy 1, policy_version 87580 (0.0008) -[2023-10-14 17:06:37,600][75949] Updated weights for policy 0, policy_version 87811 (0.0009) -[2023-10-14 17:06:37,968][75949] Updated weights for policy 0, policy_version 87821 (0.0008) -[2023-10-14 17:06:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 179601408. Throughput: 0: 1689.4, 1: 1685.7. Samples: 44913250. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:06:38,164][74987] Avg episode reward: [(0, '29.430'), (1, '34.130')] -[2023-10-14 17:06:38,336][75949] Updated weights for policy 0, policy_version 87831 (0.0012) -[2023-10-14 17:06:38,765][75950] Updated weights for policy 1, policy_version 87590 (0.0008) -[2023-10-14 17:06:39,131][75950] Updated weights for policy 1, policy_version 87600 (0.0008) -[2023-10-14 17:06:39,499][75950] Updated weights for policy 1, policy_version 87610 (0.0009) -[2023-10-14 17:06:42,401][75949] Updated weights for policy 0, policy_version 87841 (0.0009) -[2023-10-14 17:06:42,765][75949] Updated weights for policy 0, policy_version 87851 (0.0010) -[2023-10-14 17:06:43,141][75949] Updated weights for policy 0, policy_version 87861 (0.0007) -[2023-10-14 17:06:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 179666944. Throughput: 0: 1679.5, 1: 1688.2. Samples: 44933666. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:06:43,165][74987] Avg episode reward: [(0, '30.580'), (1, '34.080')] -[2023-10-14 17:06:43,177][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000087616_89718784.pth... -[2023-10-14 17:06:43,208][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000086048_88113152.pth -[2023-10-14 17:06:43,512][75949] Updated weights for policy 0, policy_version 87871 (0.0007) -[2023-10-14 17:06:43,545][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000087872_89980928.pth... -[2023-10-14 17:06:43,546][75950] Updated weights for policy 1, policy_version 87620 (0.0009) -[2023-10-14 17:06:43,586][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000086304_88375296.pth -[2023-10-14 17:06:43,913][75950] Updated weights for policy 1, policy_version 87630 (0.0008) -[2023-10-14 17:06:44,280][75950] Updated weights for policy 1, policy_version 87640 (0.0008) -[2023-10-14 17:06:47,644][75949] Updated weights for policy 0, policy_version 87881 (0.0010) -[2023-10-14 17:06:48,022][75949] Updated weights for policy 0, policy_version 87891 (0.0008) -[2023-10-14 17:06:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179732480. Throughput: 0: 1685.5, 1: 1679.1. Samples: 44943186. Policy #0 lag: (min: 30.0, avg: 31.7, max: 58.0) -[2023-10-14 17:06:48,165][74987] Avg episode reward: [(0, '28.800'), (1, '33.180')] -[2023-10-14 17:06:48,380][75949] Updated weights for policy 0, policy_version 87901 (0.0010) -[2023-10-14 17:06:48,436][75950] Updated weights for policy 1, policy_version 87650 (0.0008) -[2023-10-14 17:06:48,802][75950] Updated weights for policy 1, policy_version 87660 (0.0011) -[2023-10-14 17:06:49,165][75950] Updated weights for policy 1, policy_version 87670 (0.0009) -[2023-10-14 17:06:49,529][75950] Updated weights for policy 1, policy_version 87680 (0.0010) -[2023-10-14 17:06:52,407][75949] Updated weights for policy 0, policy_version 87911 (0.0009) -[2023-10-14 17:06:52,785][75949] Updated weights for policy 0, policy_version 87921 (0.0011) -[2023-10-14 17:06:53,143][75949] Updated weights for policy 0, policy_version 87931 (0.0011) -[2023-10-14 17:06:53,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179798016. Throughput: 0: 1686.5, 1: 1685.3. Samples: 44963858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:06:53,164][74987] Avg episode reward: [(0, '29.040'), (1, '34.380')] -[2023-10-14 17:06:53,745][75950] Updated weights for policy 1, policy_version 87690 (0.0008) -[2023-10-14 17:06:54,104][75950] Updated weights for policy 1, policy_version 87700 (0.0009) -[2023-10-14 17:06:54,480][75950] Updated weights for policy 1, policy_version 87710 (0.0009) -[2023-10-14 17:06:57,161][75949] Updated weights for policy 0, policy_version 87941 (0.0008) -[2023-10-14 17:06:57,527][75949] Updated weights for policy 0, policy_version 87951 (0.0009) -[2023-10-14 17:06:57,901][75949] Updated weights for policy 0, policy_version 87961 (0.0008) -[2023-10-14 17:06:58,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 179896320. Throughput: 0: 1668.8, 1: 1683.8. Samples: 44983840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:06:58,164][74987] Avg episode reward: [(0, '27.410'), (1, '33.870')] -[2023-10-14 17:06:58,548][75950] Updated weights for policy 1, policy_version 87720 (0.0009) -[2023-10-14 17:06:58,922][75950] Updated weights for policy 1, policy_version 87730 (0.0008) -[2023-10-14 17:06:59,297][75950] Updated weights for policy 1, policy_version 87740 (0.0009) -[2023-10-14 17:07:01,915][75949] Updated weights for policy 0, policy_version 87971 (0.0008) -[2023-10-14 17:07:02,288][75949] Updated weights for policy 0, policy_version 87981 (0.0007) -[2023-10-14 17:07:02,647][75949] Updated weights for policy 0, policy_version 87991 (0.0008) -[2023-10-14 17:07:03,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 179961856. Throughput: 0: 1688.1, 1: 1681.6. Samples: 44993738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:07:03,164][74987] Avg episode reward: [(0, '28.610'), (1, '33.390')] -[2023-10-14 17:07:03,452][75950] Updated weights for policy 1, policy_version 87750 (0.0008) -[2023-10-14 17:07:03,805][75950] Updated weights for policy 1, policy_version 87760 (0.0009) -[2023-10-14 17:07:04,174][75950] Updated weights for policy 1, policy_version 87770 (0.0010) -[2023-10-14 17:07:06,743][75949] Updated weights for policy 0, policy_version 88001 (0.0007) -[2023-10-14 17:07:07,106][75949] Updated weights for policy 0, policy_version 88011 (0.0007) -[2023-10-14 17:07:07,483][75949] Updated weights for policy 0, policy_version 88021 (0.0008) -[2023-10-14 17:07:07,847][75949] Updated weights for policy 0, policy_version 88031 (0.0009) -[2023-10-14 17:07:07,950][75950] Updated weights for policy 1, policy_version 87780 (0.0010) -[2023-10-14 17:07:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 180027392. Throughput: 0: 1688.6, 1: 1687.8. Samples: 45014596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:07:08,164][74987] Avg episode reward: [(0, '28.000'), (1, '33.720')] -[2023-10-14 17:07:08,321][75950] Updated weights for policy 1, policy_version 87790 (0.0009) -[2023-10-14 17:07:08,688][75950] Updated weights for policy 1, policy_version 87800 (0.0009) -[2023-10-14 17:07:11,781][75949] Updated weights for policy 0, policy_version 88041 (0.0008) -[2023-10-14 17:07:12,156][75949] Updated weights for policy 0, policy_version 88051 (0.0007) -[2023-10-14 17:07:12,533][75949] Updated weights for policy 0, policy_version 88061 (0.0009) -[2023-10-14 17:07:12,873][75950] Updated weights for policy 1, policy_version 87810 (0.0009) -[2023-10-14 17:07:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 180092928. Throughput: 0: 1666.7, 1: 1686.8. Samples: 45034252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:07:13,164][74987] Avg episode reward: [(0, '28.010'), (1, '33.710')] -[2023-10-14 17:07:13,296][75950] Updated weights for policy 1, policy_version 87820 (0.0008) -[2023-10-14 17:07:13,666][75950] Updated weights for policy 1, policy_version 87830 (0.0007) -[2023-10-14 17:07:14,030][75950] Updated weights for policy 1, policy_version 87840 (0.0008) -[2023-10-14 17:07:16,373][75949] Updated weights for policy 0, policy_version 88071 (0.0009) -[2023-10-14 17:07:16,746][75949] Updated weights for policy 0, policy_version 88081 (0.0010) -[2023-10-14 17:07:17,113][75949] Updated weights for policy 0, policy_version 88091 (0.0008) -[2023-10-14 17:07:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 180158464. Throughput: 0: 1698.7, 1: 1677.7. Samples: 45044606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:07:18,164][74987] Avg episode reward: [(0, '27.900'), (1, '34.690')] -[2023-10-14 17:07:18,263][75950] Updated weights for policy 1, policy_version 87850 (0.0011) -[2023-10-14 17:07:18,623][75950] Updated weights for policy 1, policy_version 87860 (0.0008) -[2023-10-14 17:07:18,991][75950] Updated weights for policy 1, policy_version 87870 (0.0009) -[2023-10-14 17:07:21,111][75949] Updated weights for policy 0, policy_version 88101 (0.0009) -[2023-10-14 17:07:21,485][75949] Updated weights for policy 0, policy_version 88111 (0.0010) -[2023-10-14 17:07:21,856][75949] Updated weights for policy 0, policy_version 88121 (0.0007) -[2023-10-14 17:07:23,005][75950] Updated weights for policy 1, policy_version 87880 (0.0008) -[2023-10-14 17:07:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 180224000. Throughput: 0: 1686.5, 1: 1680.3. Samples: 45064756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:07:23,164][74987] Avg episode reward: [(0, '27.980'), (1, '36.000')] -[2023-10-14 17:07:23,376][75950] Updated weights for policy 1, policy_version 87890 (0.0008) -[2023-10-14 17:07:23,749][75950] Updated weights for policy 1, policy_version 87900 (0.0008) -[2023-10-14 17:07:25,910][75949] Updated weights for policy 0, policy_version 88131 (0.0009) -[2023-10-14 17:07:26,277][75949] Updated weights for policy 0, policy_version 88141 (0.0009) -[2023-10-14 17:07:26,647][75949] Updated weights for policy 0, policy_version 88151 (0.0010) -[2023-10-14 17:07:27,717][75950] Updated weights for policy 1, policy_version 87910 (0.0010) -[2023-10-14 17:07:28,079][75950] Updated weights for policy 1, policy_version 87920 (0.0009) -[2023-10-14 17:07:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 180289536. Throughput: 0: 1687.1, 1: 1676.9. Samples: 45085048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:07:28,164][74987] Avg episode reward: [(0, '28.630'), (1, '36.460')] -[2023-10-14 17:07:28,452][75950] Updated weights for policy 1, policy_version 87930 (0.0011) -[2023-10-14 17:07:30,559][75949] Updated weights for policy 0, policy_version 88161 (0.0008) -[2023-10-14 17:07:30,933][75949] Updated weights for policy 0, policy_version 88171 (0.0007) -[2023-10-14 17:07:31,298][75949] Updated weights for policy 0, policy_version 88181 (0.0008) -[2023-10-14 17:07:31,665][75949] Updated weights for policy 0, policy_version 88191 (0.0007) -[2023-10-14 17:07:32,464][75950] Updated weights for policy 1, policy_version 87940 (0.0009) -[2023-10-14 17:07:32,828][75950] Updated weights for policy 1, policy_version 87950 (0.0009) -[2023-10-14 17:07:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 180355072. Throughput: 0: 1704.1, 1: 1679.5. Samples: 45095448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:07:33,164][74987] Avg episode reward: [(0, '27.490'), (1, '36.010')] -[2023-10-14 17:07:33,202][75950] Updated weights for policy 1, policy_version 87960 (0.0012) -[2023-10-14 17:07:35,812][75949] Updated weights for policy 0, policy_version 88201 (0.0011) -[2023-10-14 17:07:36,177][75949] Updated weights for policy 0, policy_version 88211 (0.0009) -[2023-10-14 17:07:36,543][75949] Updated weights for policy 0, policy_version 88221 (0.0007) -[2023-10-14 17:07:37,327][75950] Updated weights for policy 1, policy_version 87970 (0.0010) -[2023-10-14 17:07:37,702][75950] Updated weights for policy 1, policy_version 87980 (0.0009) -[2023-10-14 17:07:38,068][75950] Updated weights for policy 1, policy_version 87990 (0.0009) -[2023-10-14 17:07:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 180420608. Throughput: 0: 1675.6, 1: 1680.6. Samples: 45114888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:07:38,164][74987] Avg episode reward: [(0, '29.460'), (1, '33.400')] -[2023-10-14 17:07:38,432][75950] Updated weights for policy 1, policy_version 88000 (0.0009) -[2023-10-14 17:07:40,586][75949] Updated weights for policy 0, policy_version 88231 (0.0009) -[2023-10-14 17:07:40,961][75949] Updated weights for policy 0, policy_version 88241 (0.0007) -[2023-10-14 17:07:41,325][75949] Updated weights for policy 0, policy_version 88251 (0.0008) -[2023-10-14 17:07:42,613][75950] Updated weights for policy 1, policy_version 88010 (0.0009) -[2023-10-14 17:07:42,971][75950] Updated weights for policy 1, policy_version 88020 (0.0007) -[2023-10-14 17:07:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 180486144. Throughput: 0: 1695.4, 1: 1668.6. Samples: 45135220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:07:43,165][74987] Avg episode reward: [(0, '28.510'), (1, '33.120')] -[2023-10-14 17:07:43,327][75950] Updated weights for policy 1, policy_version 88030 (0.0007) -[2023-10-14 17:07:45,323][75949] Updated weights for policy 0, policy_version 88261 (0.0008) -[2023-10-14 17:07:45,692][75949] Updated weights for policy 0, policy_version 88271 (0.0008) -[2023-10-14 17:07:46,059][75949] Updated weights for policy 0, policy_version 88281 (0.0008) -[2023-10-14 17:07:47,291][75950] Updated weights for policy 1, policy_version 88040 (0.0009) -[2023-10-14 17:07:47,655][75950] Updated weights for policy 1, policy_version 88050 (0.0009) -[2023-10-14 17:07:48,033][75950] Updated weights for policy 1, policy_version 88060 (0.0007) -[2023-10-14 17:07:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 180551680. Throughput: 0: 1688.6, 1: 1680.9. Samples: 45145368. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:07:48,164][74987] Avg episode reward: [(0, '29.050'), (1, '33.390')] -[2023-10-14 17:07:50,190][75949] Updated weights for policy 0, policy_version 88291 (0.0008) -[2023-10-14 17:07:50,565][75949] Updated weights for policy 0, policy_version 88301 (0.0007) -[2023-10-14 17:07:50,937][75949] Updated weights for policy 0, policy_version 88311 (0.0011) -[2023-10-14 17:07:51,923][75950] Updated weights for policy 1, policy_version 88070 (0.0008) -[2023-10-14 17:07:52,282][75950] Updated weights for policy 1, policy_version 88080 (0.0007) -[2023-10-14 17:07:52,651][75950] Updated weights for policy 1, policy_version 88090 (0.0007) -[2023-10-14 17:07:53,164][74987] Fps is (10 sec: 16384.2, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 180649984. Throughput: 0: 1670.4, 1: 1681.3. Samples: 45165424. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:07:53,165][74987] Avg episode reward: [(0, '26.980'), (1, '32.020')] -[2023-10-14 17:07:55,019][75949] Updated weights for policy 0, policy_version 88321 (0.0010) -[2023-10-14 17:07:55,384][75949] Updated weights for policy 0, policy_version 88331 (0.0008) -[2023-10-14 17:07:55,759][75949] Updated weights for policy 0, policy_version 88341 (0.0008) -[2023-10-14 17:07:56,133][75949] Updated weights for policy 0, policy_version 88351 (0.0009) -[2023-10-14 17:07:56,831][75950] Updated weights for policy 1, policy_version 88100 (0.0008) -[2023-10-14 17:07:57,192][75950] Updated weights for policy 1, policy_version 88110 (0.0010) -[2023-10-14 17:07:57,567][75950] Updated weights for policy 1, policy_version 88120 (0.0011) -[2023-10-14 17:07:58,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 180715520. Throughput: 0: 1696.1, 1: 1657.0. Samples: 45185142. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:07:58,165][74987] Avg episode reward: [(0, '26.550'), (1, '32.680')] -[2023-10-14 17:08:00,321][75949] Updated weights for policy 0, policy_version 88361 (0.0009) -[2023-10-14 17:08:00,690][75949] Updated weights for policy 0, policy_version 88371 (0.0009) -[2023-10-14 17:08:01,066][75949] Updated weights for policy 0, policy_version 88381 (0.0010) -[2023-10-14 17:08:01,935][75950] Updated weights for policy 1, policy_version 88130 (0.0009) -[2023-10-14 17:08:02,353][75950] Updated weights for policy 1, policy_version 88140 (0.0007) -[2023-10-14 17:08:02,716][75950] Updated weights for policy 1, policy_version 88150 (0.0007) -[2023-10-14 17:08:03,090][75950] Updated weights for policy 1, policy_version 88160 (0.0007) -[2023-10-14 17:08:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 180781056. Throughput: 0: 1671.9, 1: 1682.8. Samples: 45195566. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:08:03,164][74987] Avg episode reward: [(0, '26.100'), (1, '34.140')] -[2023-10-14 17:08:05,097][75949] Updated weights for policy 0, policy_version 88391 (0.0008) -[2023-10-14 17:08:05,483][75949] Updated weights for policy 0, policy_version 88401 (0.0009) -[2023-10-14 17:08:05,853][75949] Updated weights for policy 0, policy_version 88411 (0.0009) -[2023-10-14 17:08:07,238][75950] Updated weights for policy 1, policy_version 88170 (0.0008) -[2023-10-14 17:08:07,600][75950] Updated weights for policy 1, policy_version 88180 (0.0009) -[2023-10-14 17:08:07,958][75950] Updated weights for policy 1, policy_version 88190 (0.0007) -[2023-10-14 17:08:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 180846592. Throughput: 0: 1673.8, 1: 1678.4. Samples: 45215606. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:08:08,165][74987] Avg episode reward: [(0, '27.420'), (1, '34.850')] -[2023-10-14 17:08:10,000][75949] Updated weights for policy 0, policy_version 88421 (0.0010) -[2023-10-14 17:08:10,383][75949] Updated weights for policy 0, policy_version 88431 (0.0009) -[2023-10-14 17:08:10,753][75949] Updated weights for policy 0, policy_version 88441 (0.0007) -[2023-10-14 17:08:12,021][75950] Updated weights for policy 1, policy_version 88200 (0.0007) -[2023-10-14 17:08:12,389][75950] Updated weights for policy 1, policy_version 88210 (0.0008) -[2023-10-14 17:08:12,762][75950] Updated weights for policy 1, policy_version 88220 (0.0009) -[2023-10-14 17:08:13,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 180912128. Throughput: 0: 1683.8, 1: 1657.6. Samples: 45235410. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:08:13,164][74987] Avg episode reward: [(0, '25.580'), (1, '33.970')] -[2023-10-14 17:08:14,751][75949] Updated weights for policy 0, policy_version 88451 (0.0009) -[2023-10-14 17:08:15,122][75949] Updated weights for policy 0, policy_version 88461 (0.0008) -[2023-10-14 17:08:15,494][75949] Updated weights for policy 0, policy_version 88471 (0.0009) -[2023-10-14 17:08:16,839][75950] Updated weights for policy 1, policy_version 88230 (0.0008) -[2023-10-14 17:08:17,217][75950] Updated weights for policy 1, policy_version 88240 (0.0008) -[2023-10-14 17:08:17,586][75950] Updated weights for policy 1, policy_version 88250 (0.0007) -[2023-10-14 17:08:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 180977664. Throughput: 0: 1661.2, 1: 1678.7. Samples: 45245742. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:08:18,164][74987] Avg episode reward: [(0, '29.030'), (1, '33.530')] -[2023-10-14 17:08:19,650][75949] Updated weights for policy 0, policy_version 88481 (0.0009) -[2023-10-14 17:08:20,017][75949] Updated weights for policy 0, policy_version 88491 (0.0007) -[2023-10-14 17:08:20,388][75949] Updated weights for policy 0, policy_version 88501 (0.0008) -[2023-10-14 17:08:20,758][75949] Updated weights for policy 0, policy_version 88511 (0.0008) -[2023-10-14 17:08:21,577][75950] Updated weights for policy 1, policy_version 88260 (0.0009) -[2023-10-14 17:08:21,937][75950] Updated weights for policy 1, policy_version 88270 (0.0011) -[2023-10-14 17:08:22,312][75950] Updated weights for policy 1, policy_version 88280 (0.0009) -[2023-10-14 17:08:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 181043200. Throughput: 0: 1685.7, 1: 1673.7. Samples: 45266062. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:08:23,165][74987] Avg episode reward: [(0, '26.050'), (1, '35.360')] -[2023-10-14 17:08:24,528][75949] Updated weights for policy 0, policy_version 88521 (0.0008) -[2023-10-14 17:08:24,913][75949] Updated weights for policy 0, policy_version 88531 (0.0009) -[2023-10-14 17:08:25,275][75949] Updated weights for policy 0, policy_version 88541 (0.0008) -[2023-10-14 17:08:26,471][75950] Updated weights for policy 1, policy_version 88290 (0.0008) -[2023-10-14 17:08:26,838][75950] Updated weights for policy 1, policy_version 88300 (0.0008) -[2023-10-14 17:08:27,202][75950] Updated weights for policy 1, policy_version 88310 (0.0009) -[2023-10-14 17:08:27,569][75950] Updated weights for policy 1, policy_version 88320 (0.0008) -[2023-10-14 17:08:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 181108736. Throughput: 0: 1690.0, 1: 1658.2. Samples: 45285886. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:08:28,165][74987] Avg episode reward: [(0, '31.580'), (1, '33.230')] -[2023-10-14 17:08:29,294][75949] Updated weights for policy 0, policy_version 88551 (0.0009) -[2023-10-14 17:08:29,672][75949] Updated weights for policy 0, policy_version 88561 (0.0007) -[2023-10-14 17:08:30,037][75949] Updated weights for policy 0, policy_version 88571 (0.0010) -[2023-10-14 17:08:31,572][75950] Updated weights for policy 1, policy_version 88330 (0.0008) -[2023-10-14 17:08:31,948][75950] Updated weights for policy 1, policy_version 88340 (0.0009) -[2023-10-14 17:08:32,314][75950] Updated weights for policy 1, policy_version 88350 (0.0008) -[2023-10-14 17:08:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 181174272. Throughput: 0: 1673.7, 1: 1677.0. Samples: 45296150. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:08:33,164][74987] Avg episode reward: [(0, '26.300'), (1, '33.050')] -[2023-10-14 17:08:34,048][75949] Updated weights for policy 0, policy_version 88581 (0.0009) -[2023-10-14 17:08:34,419][75949] Updated weights for policy 0, policy_version 88591 (0.0010) -[2023-10-14 17:08:34,795][75949] Updated weights for policy 0, policy_version 88601 (0.0008) -[2023-10-14 17:08:36,504][75950] Updated weights for policy 1, policy_version 88360 (0.0009) -[2023-10-14 17:08:36,879][75950] Updated weights for policy 1, policy_version 88370 (0.0010) -[2023-10-14 17:08:37,260][75950] Updated weights for policy 1, policy_version 88380 (0.0010) -[2023-10-14 17:08:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 181239808. Throughput: 0: 1694.6, 1: 1664.2. Samples: 45316570. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:08:38,165][74987] Avg episode reward: [(0, '30.530'), (1, '36.940')] -[2023-10-14 17:08:38,759][75949] Updated weights for policy 0, policy_version 88611 (0.0008) -[2023-10-14 17:08:39,122][75949] Updated weights for policy 0, policy_version 88621 (0.0008) -[2023-10-14 17:08:39,485][75949] Updated weights for policy 0, policy_version 88631 (0.0008) -[2023-10-14 17:08:41,408][75950] Updated weights for policy 1, policy_version 88390 (0.0009) -[2023-10-14 17:08:41,783][75950] Updated weights for policy 1, policy_version 88400 (0.0009) -[2023-10-14 17:08:42,160][75950] Updated weights for policy 1, policy_version 88410 (0.0008) -[2023-10-14 17:08:43,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 181305344. Throughput: 0: 1695.0, 1: 1671.1. Samples: 45336616. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) -[2023-10-14 17:08:43,165][74987] Avg episode reward: [(0, '24.920'), (1, '34.350')] -[2023-10-14 17:08:43,176][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000088416_90537984.pth... -[2023-10-14 17:08:43,176][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000088640_90767360.pth... -[2023-10-14 17:08:43,205][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000086848_88932352.pth -[2023-10-14 17:08:43,212][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000087072_89161728.pth -[2023-10-14 17:08:43,599][75949] Updated weights for policy 0, policy_version 88641 (0.0008) -[2023-10-14 17:08:43,968][75949] Updated weights for policy 0, policy_version 88651 (0.0008) -[2023-10-14 17:08:44,341][75949] Updated weights for policy 0, policy_version 88661 (0.0008) -[2023-10-14 17:08:44,705][75949] Updated weights for policy 0, policy_version 88671 (0.0008) -[2023-10-14 17:08:46,061][75950] Updated weights for policy 1, policy_version 88420 (0.0008) -[2023-10-14 17:08:46,419][75950] Updated weights for policy 1, policy_version 88430 (0.0007) -[2023-10-14 17:08:46,782][75950] Updated weights for policy 1, policy_version 88440 (0.0009) -[2023-10-14 17:08:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 181370880. Throughput: 0: 1686.9, 1: 1682.0. Samples: 45347168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:08:48,165][74987] Avg episode reward: [(0, '29.630'), (1, '35.150')] -[2023-10-14 17:08:48,773][75949] Updated weights for policy 0, policy_version 88681 (0.0009) -[2023-10-14 17:08:49,140][75949] Updated weights for policy 0, policy_version 88691 (0.0010) -[2023-10-14 17:08:49,505][75949] Updated weights for policy 0, policy_version 88701 (0.0010) -[2023-10-14 17:08:50,887][75950] Updated weights for policy 1, policy_version 88450 (0.0008) -[2023-10-14 17:08:51,293][75950] Updated weights for policy 1, policy_version 88460 (0.0010) -[2023-10-14 17:08:51,654][75950] Updated weights for policy 1, policy_version 88470 (0.0009) -[2023-10-14 17:08:52,019][75950] Updated weights for policy 1, policy_version 88480 (0.0007) -[2023-10-14 17:08:53,163][74987] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 181436416. Throughput: 0: 1698.1, 1: 1667.9. Samples: 45367072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:08:53,164][74987] Avg episode reward: [(0, '27.100'), (1, '34.840')] -[2023-10-14 17:08:53,434][75949] Updated weights for policy 0, policy_version 88711 (0.0008) -[2023-10-14 17:08:53,799][75949] Updated weights for policy 0, policy_version 88721 (0.0009) -[2023-10-14 17:08:54,173][75949] Updated weights for policy 0, policy_version 88731 (0.0007) -[2023-10-14 17:08:55,882][75950] Updated weights for policy 1, policy_version 88490 (0.0008) -[2023-10-14 17:08:56,251][75950] Updated weights for policy 1, policy_version 88500 (0.0009) -[2023-10-14 17:08:56,613][75950] Updated weights for policy 1, policy_version 88510 (0.0008) -[2023-10-14 17:08:58,061][75949] Updated weights for policy 0, policy_version 88741 (0.0007) -[2023-10-14 17:08:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 181501952. Throughput: 0: 1703.7, 1: 1684.8. Samples: 45387894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:08:58,164][74987] Avg episode reward: [(0, '28.710'), (1, '34.060')] -[2023-10-14 17:08:58,441][75949] Updated weights for policy 0, policy_version 88751 (0.0008) -[2023-10-14 17:08:58,817][75949] Updated weights for policy 0, policy_version 88761 (0.0009) -[2023-10-14 17:09:00,707][75950] Updated weights for policy 1, policy_version 88520 (0.0008) -[2023-10-14 17:09:01,070][75950] Updated weights for policy 1, policy_version 88530 (0.0008) -[2023-10-14 17:09:01,442][75950] Updated weights for policy 1, policy_version 88540 (0.0009) -[2023-10-14 17:09:03,109][75949] Updated weights for policy 0, policy_version 88771 (0.0008) -[2023-10-14 17:09:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 181567488. Throughput: 0: 1697.3, 1: 1682.5. Samples: 45397836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:09:03,164][74987] Avg episode reward: [(0, '28.400'), (1, '34.810')] -[2023-10-14 17:09:03,480][75949] Updated weights for policy 0, policy_version 88781 (0.0010) -[2023-10-14 17:09:03,842][75949] Updated weights for policy 0, policy_version 88791 (0.0008) -[2023-10-14 17:09:05,330][75950] Updated weights for policy 1, policy_version 88550 (0.0007) -[2023-10-14 17:09:05,698][75950] Updated weights for policy 1, policy_version 88560 (0.0008) -[2023-10-14 17:09:06,063][75950] Updated weights for policy 1, policy_version 88570 (0.0008) -[2023-10-14 17:09:07,767][75949] Updated weights for policy 0, policy_version 88801 (0.0009) -[2023-10-14 17:09:08,134][75949] Updated weights for policy 0, policy_version 88811 (0.0007) -[2023-10-14 17:09:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 181633024. Throughput: 0: 1703.4, 1: 1666.2. Samples: 45417696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:09:08,165][74987] Avg episode reward: [(0, '26.630'), (1, '33.680')] -[2023-10-14 17:09:08,502][75949] Updated weights for policy 0, policy_version 88821 (0.0007) -[2023-10-14 17:09:08,875][75949] Updated weights for policy 0, policy_version 88831 (0.0008) -[2023-10-14 17:09:10,087][75950] Updated weights for policy 1, policy_version 88580 (0.0009) -[2023-10-14 17:09:10,466][75950] Updated weights for policy 1, policy_version 88590 (0.0009) -[2023-10-14 17:09:10,825][75950] Updated weights for policy 1, policy_version 88600 (0.0010) -[2023-10-14 17:09:13,010][75949] Updated weights for policy 0, policy_version 88841 (0.0010) -[2023-10-14 17:09:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 181698560. Throughput: 0: 1698.2, 1: 1694.2. Samples: 45438544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:09:13,165][74987] Avg episode reward: [(0, '30.420'), (1, '33.060')] -[2023-10-14 17:09:13,368][75949] Updated weights for policy 0, policy_version 88851 (0.0009) -[2023-10-14 17:09:13,737][75949] Updated weights for policy 0, policy_version 88861 (0.0008) -[2023-10-14 17:09:14,929][75950] Updated weights for policy 1, policy_version 88610 (0.0009) -[2023-10-14 17:09:15,285][75950] Updated weights for policy 1, policy_version 88620 (0.0011) -[2023-10-14 17:09:15,659][75950] Updated weights for policy 1, policy_version 88630 (0.0012) -[2023-10-14 17:09:16,019][75950] Updated weights for policy 1, policy_version 88640 (0.0009) -[2023-10-14 17:09:17,748][75949] Updated weights for policy 0, policy_version 88871 (0.0010) -[2023-10-14 17:09:18,105][75949] Updated weights for policy 0, policy_version 88881 (0.0009) -[2023-10-14 17:09:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 181764096. Throughput: 0: 1699.8, 1: 1676.5. Samples: 45448086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:09:18,164][74987] Avg episode reward: [(0, '27.670'), (1, '33.210')] -[2023-10-14 17:09:18,474][75949] Updated weights for policy 0, policy_version 88891 (0.0009) -[2023-10-14 17:09:20,188][75950] Updated weights for policy 1, policy_version 88650 (0.0007) -[2023-10-14 17:09:20,548][75950] Updated weights for policy 1, policy_version 88660 (0.0009) -[2023-10-14 17:09:20,914][75950] Updated weights for policy 1, policy_version 88670 (0.0010) -[2023-10-14 17:09:22,617][75949] Updated weights for policy 0, policy_version 88901 (0.0007) -[2023-10-14 17:09:22,989][75949] Updated weights for policy 0, policy_version 88911 (0.0007) -[2023-10-14 17:09:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 181829632. Throughput: 0: 1699.3, 1: 1671.6. Samples: 45468260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:09:23,164][74987] Avg episode reward: [(0, '28.680'), (1, '34.210')] -[2023-10-14 17:09:23,356][75949] Updated weights for policy 0, policy_version 88921 (0.0009) -[2023-10-14 17:09:25,055][75950] Updated weights for policy 1, policy_version 88680 (0.0009) -[2023-10-14 17:09:25,424][75950] Updated weights for policy 1, policy_version 88690 (0.0009) -[2023-10-14 17:09:25,779][75950] Updated weights for policy 1, policy_version 88700 (0.0010) -[2023-10-14 17:09:27,492][75949] Updated weights for policy 0, policy_version 88931 (0.0010) -[2023-10-14 17:09:27,862][75949] Updated weights for policy 0, policy_version 88941 (0.0008) -[2023-10-14 17:09:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 181895168. Throughput: 0: 1689.8, 1: 1693.9. Samples: 45488882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:09:28,164][74987] Avg episode reward: [(0, '27.210'), (1, '34.350')] -[2023-10-14 17:09:28,238][75949] Updated weights for policy 0, policy_version 88951 (0.0009) -[2023-10-14 17:09:29,759][75950] Updated weights for policy 1, policy_version 88710 (0.0009) -[2023-10-14 17:09:30,124][75950] Updated weights for policy 1, policy_version 88720 (0.0009) -[2023-10-14 17:09:30,484][75950] Updated weights for policy 1, policy_version 88730 (0.0009) -[2023-10-14 17:09:32,251][75949] Updated weights for policy 0, policy_version 88961 (0.0008) -[2023-10-14 17:09:32,619][75949] Updated weights for policy 0, policy_version 88971 (0.0007) -[2023-10-14 17:09:32,990][75949] Updated weights for policy 0, policy_version 88981 (0.0009) -[2023-10-14 17:09:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 181960704. Throughput: 0: 1696.8, 1: 1668.6. Samples: 45498612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:09:33,164][74987] Avg episode reward: [(0, '29.460'), (1, '32.750')] -[2023-10-14 17:09:33,353][75949] Updated weights for policy 0, policy_version 88991 (0.0008) -[2023-10-14 17:09:34,270][75950] Updated weights for policy 1, policy_version 88740 (0.0008) -[2023-10-14 17:09:34,645][75950] Updated weights for policy 1, policy_version 88750 (0.0008) -[2023-10-14 17:09:35,008][75950] Updated weights for policy 1, policy_version 88760 (0.0007) -[2023-10-14 17:09:37,354][75949] Updated weights for policy 0, policy_version 89001 (0.0007) -[2023-10-14 17:09:37,723][75949] Updated weights for policy 0, policy_version 89011 (0.0009) -[2023-10-14 17:09:38,102][75949] Updated weights for policy 0, policy_version 89021 (0.0009) -[2023-10-14 17:09:38,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182026240. Throughput: 0: 1696.0, 1: 1688.6. Samples: 45519382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:09:38,164][74987] Avg episode reward: [(0, '26.570'), (1, '33.710')] -[2023-10-14 17:09:39,220][75950] Updated weights for policy 1, policy_version 88770 (0.0007) -[2023-10-14 17:09:39,632][75950] Updated weights for policy 1, policy_version 88780 (0.0009) -[2023-10-14 17:09:39,995][75950] Updated weights for policy 1, policy_version 88790 (0.0008) -[2023-10-14 17:09:40,358][75950] Updated weights for policy 1, policy_version 88800 (0.0008) -[2023-10-14 17:09:42,127][75949] Updated weights for policy 0, policy_version 89031 (0.0007) -[2023-10-14 17:09:42,497][75949] Updated weights for policy 0, policy_version 89041 (0.0009) -[2023-10-14 17:09:42,858][75949] Updated weights for policy 0, policy_version 89051 (0.0009) -[2023-10-14 17:09:43,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 182124544. Throughput: 0: 1670.9, 1: 1693.7. Samples: 45539300. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:09:43,165][74987] Avg episode reward: [(0, '29.190'), (1, '35.030')] -[2023-10-14 17:09:44,474][75950] Updated weights for policy 1, policy_version 88810 (0.0009) -[2023-10-14 17:09:44,841][75950] Updated weights for policy 1, policy_version 88820 (0.0008) -[2023-10-14 17:09:45,204][75950] Updated weights for policy 1, policy_version 88830 (0.0007) -[2023-10-14 17:09:47,043][75949] Updated weights for policy 0, policy_version 89061 (0.0008) -[2023-10-14 17:09:47,442][75949] Updated weights for policy 0, policy_version 89071 (0.0009) -[2023-10-14 17:09:47,807][75949] Updated weights for policy 0, policy_version 89081 (0.0009) -[2023-10-14 17:09:48,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 182190080. Throughput: 0: 1693.8, 1: 1672.7. Samples: 45549330. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:09:48,165][74987] Avg episode reward: [(0, '27.600'), (1, '32.840')] -[2023-10-14 17:09:49,352][75950] Updated weights for policy 1, policy_version 88840 (0.0009) -[2023-10-14 17:09:49,714][75950] Updated weights for policy 1, policy_version 88850 (0.0009) -[2023-10-14 17:09:50,078][75950] Updated weights for policy 1, policy_version 88860 (0.0009) -[2023-10-14 17:09:51,797][75949] Updated weights for policy 0, policy_version 89091 (0.0007) -[2023-10-14 17:09:52,173][75949] Updated weights for policy 0, policy_version 89101 (0.0008) -[2023-10-14 17:09:52,539][75949] Updated weights for policy 0, policy_version 89111 (0.0009) -[2023-10-14 17:09:53,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 182255616. Throughput: 0: 1686.7, 1: 1689.3. Samples: 45569616. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:09:53,164][74987] Avg episode reward: [(0, '29.220'), (1, '34.560')] -[2023-10-14 17:09:54,197][75950] Updated weights for policy 1, policy_version 88870 (0.0008) -[2023-10-14 17:09:54,565][75950] Updated weights for policy 1, policy_version 88880 (0.0008) -[2023-10-14 17:09:54,922][75950] Updated weights for policy 1, policy_version 88890 (0.0008) -[2023-10-14 17:09:56,570][75949] Updated weights for policy 0, policy_version 89121 (0.0008) -[2023-10-14 17:09:56,945][75949] Updated weights for policy 0, policy_version 89131 (0.0009) -[2023-10-14 17:09:57,312][75949] Updated weights for policy 0, policy_version 89141 (0.0010) -[2023-10-14 17:09:57,690][75949] Updated weights for policy 0, policy_version 89151 (0.0011) -[2023-10-14 17:09:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 182321152. Throughput: 0: 1660.4, 1: 1689.1. Samples: 45589272. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:09:58,164][74987] Avg episode reward: [(0, '27.720'), (1, '38.760')] -[2023-10-14 17:09:58,176][75801] Saving new best policy, reward=38.760! -[2023-10-14 17:09:58,936][75950] Updated weights for policy 1, policy_version 88900 (0.0008) -[2023-10-14 17:09:59,300][75950] Updated weights for policy 1, policy_version 88910 (0.0011) -[2023-10-14 17:09:59,668][75950] Updated weights for policy 1, policy_version 88920 (0.0010) -[2023-10-14 17:10:01,691][75949] Updated weights for policy 0, policy_version 89161 (0.0011) -[2023-10-14 17:10:02,060][75949] Updated weights for policy 0, policy_version 89171 (0.0010) -[2023-10-14 17:10:02,421][75949] Updated weights for policy 0, policy_version 89181 (0.0007) -[2023-10-14 17:10:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 182386688. Throughput: 0: 1689.6, 1: 1678.9. Samples: 45599668. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:10:03,165][74987] Avg episode reward: [(0, '27.790'), (1, '35.810')] -[2023-10-14 17:10:03,523][75950] Updated weights for policy 1, policy_version 88930 (0.0010) -[2023-10-14 17:10:03,891][75950] Updated weights for policy 1, policy_version 88940 (0.0010) -[2023-10-14 17:10:04,257][75950] Updated weights for policy 1, policy_version 88950 (0.0010) -[2023-10-14 17:10:04,625][75950] Updated weights for policy 1, policy_version 88960 (0.0010) -[2023-10-14 17:10:06,530][75949] Updated weights for policy 0, policy_version 89191 (0.0007) -[2023-10-14 17:10:06,901][75949] Updated weights for policy 0, policy_version 89201 (0.0008) -[2023-10-14 17:10:07,264][75949] Updated weights for policy 0, policy_version 89211 (0.0008) -[2023-10-14 17:10:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 182452224. Throughput: 0: 1675.4, 1: 1695.3. Samples: 45619942. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:10:08,165][74987] Avg episode reward: [(0, '27.160'), (1, '34.490')] -[2023-10-14 17:10:08,682][75950] Updated weights for policy 1, policy_version 88970 (0.0009) -[2023-10-14 17:10:09,043][75950] Updated weights for policy 1, policy_version 88980 (0.0008) -[2023-10-14 17:10:09,412][75950] Updated weights for policy 1, policy_version 88990 (0.0007) -[2023-10-14 17:10:11,295][75949] Updated weights for policy 0, policy_version 89221 (0.0009) -[2023-10-14 17:10:11,673][75949] Updated weights for policy 0, policy_version 89231 (0.0008) -[2023-10-14 17:10:12,032][75949] Updated weights for policy 0, policy_version 89241 (0.0008) -[2023-10-14 17:10:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 182517760. Throughput: 0: 1663.5, 1: 1690.1. Samples: 45639794. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:10:13,164][74987] Avg episode reward: [(0, '26.130'), (1, '37.190')] -[2023-10-14 17:10:13,574][75950] Updated weights for policy 1, policy_version 89000 (0.0007) -[2023-10-14 17:10:13,944][75950] Updated weights for policy 1, policy_version 89010 (0.0008) -[2023-10-14 17:10:14,319][75950] Updated weights for policy 1, policy_version 89020 (0.0008) -[2023-10-14 17:10:16,037][75949] Updated weights for policy 0, policy_version 89251 (0.0007) -[2023-10-14 17:10:16,401][75949] Updated weights for policy 0, policy_version 89261 (0.0008) -[2023-10-14 17:10:16,769][75949] Updated weights for policy 0, policy_version 89271 (0.0008) -[2023-10-14 17:10:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 182583296. Throughput: 0: 1684.6, 1: 1681.4. Samples: 45650082. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:10:18,165][74987] Avg episode reward: [(0, '30.420'), (1, '37.050')] -[2023-10-14 17:10:18,398][75950] Updated weights for policy 1, policy_version 89030 (0.0008) -[2023-10-14 17:10:18,761][75950] Updated weights for policy 1, policy_version 89040 (0.0009) -[2023-10-14 17:10:19,127][75950] Updated weights for policy 1, policy_version 89050 (0.0009) -[2023-10-14 17:10:21,003][75949] Updated weights for policy 0, policy_version 89281 (0.0009) -[2023-10-14 17:10:21,375][75949] Updated weights for policy 0, policy_version 89291 (0.0007) -[2023-10-14 17:10:21,745][75949] Updated weights for policy 0, policy_version 89301 (0.0008) -[2023-10-14 17:10:22,121][75949] Updated weights for policy 0, policy_version 89311 (0.0008) -[2023-10-14 17:10:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 182648832. Throughput: 0: 1664.3, 1: 1674.2. Samples: 45669614. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:10:23,165][74987] Avg episode reward: [(0, '25.780'), (1, '35.020')] -[2023-10-14 17:10:23,323][75950] Updated weights for policy 1, policy_version 89060 (0.0008) -[2023-10-14 17:10:23,685][75950] Updated weights for policy 1, policy_version 89070 (0.0009) -[2023-10-14 17:10:24,045][75950] Updated weights for policy 1, policy_version 89080 (0.0009) -[2023-10-14 17:10:26,153][75949] Updated weights for policy 0, policy_version 89321 (0.0008) -[2023-10-14 17:10:26,518][75949] Updated weights for policy 0, policy_version 89331 (0.0007) -[2023-10-14 17:10:26,890][75949] Updated weights for policy 0, policy_version 89341 (0.0008) -[2023-10-14 17:10:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 182714368. Throughput: 0: 1668.4, 1: 1679.1. Samples: 45689934. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:10:28,165][74987] Avg episode reward: [(0, '31.100'), (1, '34.520')] -[2023-10-14 17:10:28,187][75950] Updated weights for policy 1, policy_version 89090 (0.0007) -[2023-10-14 17:10:28,600][75950] Updated weights for policy 1, policy_version 89100 (0.0007) -[2023-10-14 17:10:28,969][75950] Updated weights for policy 1, policy_version 89110 (0.0007) -[2023-10-14 17:10:29,339][75950] Updated weights for policy 1, policy_version 89120 (0.0007) -[2023-10-14 17:10:31,039][75949] Updated weights for policy 0, policy_version 89351 (0.0009) -[2023-10-14 17:10:31,414][75949] Updated weights for policy 0, policy_version 89361 (0.0007) -[2023-10-14 17:10:31,784][75949] Updated weights for policy 0, policy_version 89371 (0.0007) -[2023-10-14 17:10:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 182779904. Throughput: 0: 1676.5, 1: 1677.2. Samples: 45700248. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:10:33,164][74987] Avg episode reward: [(0, '27.660'), (1, '37.260')] -[2023-10-14 17:10:33,509][75950] Updated weights for policy 1, policy_version 89130 (0.0010) -[2023-10-14 17:10:33,870][75950] Updated weights for policy 1, policy_version 89140 (0.0010) -[2023-10-14 17:10:34,234][75950] Updated weights for policy 1, policy_version 89150 (0.0009) -[2023-10-14 17:10:35,942][75949] Updated weights for policy 0, policy_version 89381 (0.0008) -[2023-10-14 17:10:36,335][75949] Updated weights for policy 0, policy_version 89391 (0.0008) -[2023-10-14 17:10:36,703][75949] Updated weights for policy 0, policy_version 89401 (0.0009) -[2023-10-14 17:10:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 182845440. Throughput: 0: 1657.3, 1: 1681.1. Samples: 45719846. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) -[2023-10-14 17:10:38,165][74987] Avg episode reward: [(0, '31.060'), (1, '36.770')] -[2023-10-14 17:10:38,322][75950] Updated weights for policy 1, policy_version 89160 (0.0008) -[2023-10-14 17:10:38,680][75950] Updated weights for policy 1, policy_version 89170 (0.0010) -[2023-10-14 17:10:39,054][75950] Updated weights for policy 1, policy_version 89180 (0.0007) -[2023-10-14 17:10:40,767][75949] Updated weights for policy 0, policy_version 89411 (0.0010) -[2023-10-14 17:10:41,136][75949] Updated weights for policy 0, policy_version 89421 (0.0009) -[2023-10-14 17:10:41,503][75949] Updated weights for policy 0, policy_version 89431 (0.0011) -[2023-10-14 17:10:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 182910976. Throughput: 0: 1676.9, 1: 1676.0. Samples: 45740156. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:10:43,165][74987] Avg episode reward: [(0, '27.020'), (1, '34.830')] -[2023-10-14 17:10:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000089440_91586560.pth... -[2023-10-14 17:10:43,202][75950] Updated weights for policy 1, policy_version 89190 (0.0008) -[2023-10-14 17:10:43,210][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000087872_89980928.pth -[2023-10-14 17:10:43,576][75950] Updated weights for policy 1, policy_version 89200 (0.0008) -[2023-10-14 17:10:43,942][75950] Updated weights for policy 1, policy_version 89210 (0.0008) -[2023-10-14 17:10:44,154][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000089216_91357184.pth... -[2023-10-14 17:10:44,194][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000087616_89718784.pth -[2023-10-14 17:10:45,525][75949] Updated weights for policy 0, policy_version 89441 (0.0007) -[2023-10-14 17:10:45,888][75949] Updated weights for policy 0, policy_version 89451 (0.0007) -[2023-10-14 17:10:46,263][75949] Updated weights for policy 0, policy_version 89461 (0.0011) -[2023-10-14 17:10:46,632][75949] Updated weights for policy 0, policy_version 89471 (0.0010) -[2023-10-14 17:10:48,070][75950] Updated weights for policy 1, policy_version 89220 (0.0007) -[2023-10-14 17:10:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 182976512. Throughput: 0: 1671.5, 1: 1673.0. Samples: 45750170. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:10:48,165][74987] Avg episode reward: [(0, '30.050'), (1, '37.120')] -[2023-10-14 17:10:48,430][75950] Updated weights for policy 1, policy_version 89230 (0.0008) -[2023-10-14 17:10:48,809][75950] Updated weights for policy 1, policy_version 89240 (0.0011) -[2023-10-14 17:10:50,914][75949] Updated weights for policy 0, policy_version 89481 (0.0009) -[2023-10-14 17:10:51,288][75949] Updated weights for policy 0, policy_version 89491 (0.0009) -[2023-10-14 17:10:51,653][75949] Updated weights for policy 0, policy_version 89501 (0.0008) -[2023-10-14 17:10:52,737][75950] Updated weights for policy 1, policy_version 89250 (0.0010) -[2023-10-14 17:10:53,094][75950] Updated weights for policy 1, policy_version 89260 (0.0010) -[2023-10-14 17:10:53,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 183042048. Throughput: 0: 1660.6, 1: 1672.4. Samples: 45769926. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:10:53,164][74987] Avg episode reward: [(0, '26.800'), (1, '39.150')] -[2023-10-14 17:10:53,473][75950] Updated weights for policy 1, policy_version 89270 (0.0009) -[2023-10-14 17:10:53,826][75801] Saving new best policy, reward=39.150! -[2023-10-14 17:10:53,828][75950] Updated weights for policy 1, policy_version 89280 (0.0008) -[2023-10-14 17:10:55,740][75949] Updated weights for policy 0, policy_version 89511 (0.0008) -[2023-10-14 17:10:56,105][75949] Updated weights for policy 0, policy_version 89521 (0.0008) -[2023-10-14 17:10:56,484][75949] Updated weights for policy 0, policy_version 89531 (0.0011) -[2023-10-14 17:10:57,929][75950] Updated weights for policy 1, policy_version 89290 (0.0009) -[2023-10-14 17:10:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 183107584. Throughput: 0: 1678.0, 1: 1672.1. Samples: 45790550. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:10:58,164][74987] Avg episode reward: [(0, '27.490'), (1, '37.620')] -[2023-10-14 17:10:58,304][75950] Updated weights for policy 1, policy_version 89300 (0.0010) -[2023-10-14 17:10:58,678][75950] Updated weights for policy 1, policy_version 89310 (0.0009) -[2023-10-14 17:11:00,405][75949] Updated weights for policy 0, policy_version 89541 (0.0009) -[2023-10-14 17:11:00,773][75949] Updated weights for policy 0, policy_version 89551 (0.0011) -[2023-10-14 17:11:01,139][75949] Updated weights for policy 0, policy_version 89561 (0.0010) -[2023-10-14 17:11:02,611][75950] Updated weights for policy 1, policy_version 89320 (0.0009) -[2023-10-14 17:11:02,978][75950] Updated weights for policy 1, policy_version 89330 (0.0010) -[2023-10-14 17:11:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 183173120. Throughput: 0: 1667.7, 1: 1677.5. Samples: 45800614. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:11:03,165][74987] Avg episode reward: [(0, '27.460'), (1, '34.200')] -[2023-10-14 17:11:03,360][75950] Updated weights for policy 1, policy_version 89340 (0.0009) -[2023-10-14 17:11:05,310][75949] Updated weights for policy 0, policy_version 89571 (0.0010) -[2023-10-14 17:11:05,682][75949] Updated weights for policy 0, policy_version 89581 (0.0007) -[2023-10-14 17:11:06,056][75949] Updated weights for policy 0, policy_version 89591 (0.0009) -[2023-10-14 17:11:07,496][75950] Updated weights for policy 1, policy_version 89350 (0.0008) -[2023-10-14 17:11:07,859][75950] Updated weights for policy 1, policy_version 89360 (0.0009) -[2023-10-14 17:11:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 183238656. Throughput: 0: 1667.4, 1: 1686.6. Samples: 45820542. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:11:08,164][74987] Avg episode reward: [(0, '28.560'), (1, '34.690')] -[2023-10-14 17:11:08,221][75950] Updated weights for policy 1, policy_version 89370 (0.0007) -[2023-10-14 17:11:10,008][75949] Updated weights for policy 0, policy_version 89601 (0.0011) -[2023-10-14 17:11:10,385][75949] Updated weights for policy 0, policy_version 89611 (0.0007) -[2023-10-14 17:11:10,747][75949] Updated weights for policy 0, policy_version 89621 (0.0009) -[2023-10-14 17:11:11,118][75949] Updated weights for policy 0, policy_version 89631 (0.0008) -[2023-10-14 17:11:12,268][75950] Updated weights for policy 1, policy_version 89380 (0.0009) -[2023-10-14 17:11:12,644][75950] Updated weights for policy 1, policy_version 89390 (0.0010) -[2023-10-14 17:11:13,005][75950] Updated weights for policy 1, policy_version 89400 (0.0009) -[2023-10-14 17:11:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 183304192. Throughput: 0: 1684.4, 1: 1670.8. Samples: 45840918. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:11:13,165][74987] Avg episode reward: [(0, '28.750'), (1, '35.950')] -[2023-10-14 17:11:15,100][75949] Updated weights for policy 0, policy_version 89641 (0.0007) -[2023-10-14 17:11:15,470][75949] Updated weights for policy 0, policy_version 89651 (0.0009) -[2023-10-14 17:11:15,836][75949] Updated weights for policy 0, policy_version 89661 (0.0011) -[2023-10-14 17:11:17,218][75950] Updated weights for policy 1, policy_version 89410 (0.0010) -[2023-10-14 17:11:17,605][75950] Updated weights for policy 1, policy_version 89420 (0.0008) -[2023-10-14 17:11:17,970][75950] Updated weights for policy 1, policy_version 89430 (0.0010) -[2023-10-14 17:11:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 183369728. Throughput: 0: 1664.1, 1: 1681.6. Samples: 45850808. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:11:18,164][74987] Avg episode reward: [(0, '29.150'), (1, '35.260')] -[2023-10-14 17:11:18,341][75950] Updated weights for policy 1, policy_version 89440 (0.0010) -[2023-10-14 17:11:20,039][75949] Updated weights for policy 0, policy_version 89671 (0.0009) -[2023-10-14 17:11:20,413][75949] Updated weights for policy 0, policy_version 89681 (0.0007) -[2023-10-14 17:11:20,769][75949] Updated weights for policy 0, policy_version 89691 (0.0009) -[2023-10-14 17:11:22,224][75950] Updated weights for policy 1, policy_version 89450 (0.0009) -[2023-10-14 17:11:22,595][75950] Updated weights for policy 1, policy_version 89460 (0.0010) -[2023-10-14 17:11:22,968][75950] Updated weights for policy 1, policy_version 89470 (0.0010) -[2023-10-14 17:11:23,164][74987] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 183468032. Throughput: 0: 1674.5, 1: 1685.7. Samples: 45871058. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:11:23,165][74987] Avg episode reward: [(0, '27.840'), (1, '35.790')] -[2023-10-14 17:11:24,887][75949] Updated weights for policy 0, policy_version 89701 (0.0009) -[2023-10-14 17:11:25,280][75949] Updated weights for policy 0, policy_version 89711 (0.0007) -[2023-10-14 17:11:25,644][75949] Updated weights for policy 0, policy_version 89721 (0.0008) -[2023-10-14 17:11:27,058][75950] Updated weights for policy 1, policy_version 89480 (0.0010) -[2023-10-14 17:11:27,428][75950] Updated weights for policy 1, policy_version 89490 (0.0009) -[2023-10-14 17:11:27,792][75950] Updated weights for policy 1, policy_version 89500 (0.0008) -[2023-10-14 17:11:28,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 183533568. Throughput: 0: 1681.9, 1: 1665.7. Samples: 45890798. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:11:28,164][74987] Avg episode reward: [(0, '27.630'), (1, '36.240')] -[2023-10-14 17:11:29,638][75949] Updated weights for policy 0, policy_version 89731 (0.0008) -[2023-10-14 17:11:30,008][75949] Updated weights for policy 0, policy_version 89741 (0.0009) -[2023-10-14 17:11:30,371][75949] Updated weights for policy 0, policy_version 89751 (0.0009) -[2023-10-14 17:11:31,896][75950] Updated weights for policy 1, policy_version 89510 (0.0008) -[2023-10-14 17:11:32,259][75950] Updated weights for policy 1, policy_version 89520 (0.0010) -[2023-10-14 17:11:32,631][75950] Updated weights for policy 1, policy_version 89530 (0.0011) -[2023-10-14 17:11:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 183599104. Throughput: 0: 1660.6, 1: 1693.0. Samples: 45901084. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:11:33,165][74987] Avg episode reward: [(0, '29.860'), (1, '33.350')] -[2023-10-14 17:11:34,527][75949] Updated weights for policy 0, policy_version 89761 (0.0010) -[2023-10-14 17:11:34,894][75949] Updated weights for policy 0, policy_version 89771 (0.0008) -[2023-10-14 17:11:35,277][75949] Updated weights for policy 0, policy_version 89781 (0.0009) -[2023-10-14 17:11:35,649][75949] Updated weights for policy 0, policy_version 89791 (0.0009) -[2023-10-14 17:11:36,722][75950] Updated weights for policy 1, policy_version 89540 (0.0009) -[2023-10-14 17:11:37,088][75950] Updated weights for policy 1, policy_version 89550 (0.0008) -[2023-10-14 17:11:37,447][75950] Updated weights for policy 1, policy_version 89560 (0.0008) -[2023-10-14 17:11:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 183664640. Throughput: 0: 1684.5, 1: 1686.6. Samples: 45921626. Policy #0 lag: (min: 9.0, avg: 21.5, max: 41.0) -[2023-10-14 17:11:38,164][74987] Avg episode reward: [(0, '28.790'), (1, '34.350')] -[2023-10-14 17:11:39,398][75949] Updated weights for policy 0, policy_version 89801 (0.0008) -[2023-10-14 17:11:39,765][75949] Updated weights for policy 0, policy_version 89811 (0.0007) -[2023-10-14 17:11:40,132][75949] Updated weights for policy 0, policy_version 89821 (0.0007) -[2023-10-14 17:11:41,425][75950] Updated weights for policy 1, policy_version 89570 (0.0008) -[2023-10-14 17:11:41,793][75950] Updated weights for policy 1, policy_version 89580 (0.0007) -[2023-10-14 17:11:42,155][75950] Updated weights for policy 1, policy_version 89590 (0.0007) -[2023-10-14 17:11:42,519][75950] Updated weights for policy 1, policy_version 89600 (0.0007) -[2023-10-14 17:11:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 183730176. Throughput: 0: 1691.3, 1: 1660.0. Samples: 45941362. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:11:43,165][74987] Avg episode reward: [(0, '29.850'), (1, '33.550')] -[2023-10-14 17:11:44,266][75949] Updated weights for policy 0, policy_version 89831 (0.0009) -[2023-10-14 17:11:44,630][75949] Updated weights for policy 0, policy_version 89841 (0.0007) -[2023-10-14 17:11:45,005][75949] Updated weights for policy 0, policy_version 89851 (0.0008) -[2023-10-14 17:11:46,642][75950] Updated weights for policy 1, policy_version 89610 (0.0009) -[2023-10-14 17:11:47,011][75950] Updated weights for policy 1, policy_version 89620 (0.0008) -[2023-10-14 17:11:47,373][75950] Updated weights for policy 1, policy_version 89630 (0.0009) -[2023-10-14 17:11:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 183795712. Throughput: 0: 1669.3, 1: 1684.4. Samples: 45951532. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:11:48,164][74987] Avg episode reward: [(0, '26.560'), (1, '32.870')] -[2023-10-14 17:11:48,954][75949] Updated weights for policy 0, policy_version 89861 (0.0009) -[2023-10-14 17:11:49,324][75949] Updated weights for policy 0, policy_version 89871 (0.0008) -[2023-10-14 17:11:49,690][75949] Updated weights for policy 0, policy_version 89881 (0.0009) -[2023-10-14 17:11:51,483][75950] Updated weights for policy 1, policy_version 89640 (0.0009) -[2023-10-14 17:11:51,846][75950] Updated weights for policy 1, policy_version 89650 (0.0007) -[2023-10-14 17:11:52,221][75950] Updated weights for policy 1, policy_version 89660 (0.0009) -[2023-10-14 17:11:53,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 183861248. Throughput: 0: 1694.4, 1: 1665.5. Samples: 45971736. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:11:53,165][74987] Avg episode reward: [(0, '31.120'), (1, '34.080')] -[2023-10-14 17:11:53,689][75949] Updated weights for policy 0, policy_version 89891 (0.0007) -[2023-10-14 17:11:54,047][75949] Updated weights for policy 0, policy_version 89901 (0.0009) -[2023-10-14 17:11:54,423][75949] Updated weights for policy 0, policy_version 89911 (0.0008) -[2023-10-14 17:11:56,262][75950] Updated weights for policy 1, policy_version 89670 (0.0009) -[2023-10-14 17:11:56,627][75950] Updated weights for policy 1, policy_version 89680 (0.0010) -[2023-10-14 17:11:56,991][75950] Updated weights for policy 1, policy_version 89690 (0.0009) -[2023-10-14 17:11:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 183926784. Throughput: 0: 1696.4, 1: 1653.7. Samples: 45991674. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:11:58,165][74987] Avg episode reward: [(0, '29.190'), (1, '33.820')] -[2023-10-14 17:11:58,343][75949] Updated weights for policy 0, policy_version 89921 (0.0008) -[2023-10-14 17:11:58,707][75949] Updated weights for policy 0, policy_version 89931 (0.0008) -[2023-10-14 17:11:59,081][75949] Updated weights for policy 0, policy_version 89941 (0.0008) -[2023-10-14 17:11:59,447][75949] Updated weights for policy 0, policy_version 89951 (0.0008) -[2023-10-14 17:12:01,193][75950] Updated weights for policy 1, policy_version 89700 (0.0010) -[2023-10-14 17:12:01,563][75950] Updated weights for policy 1, policy_version 89710 (0.0010) -[2023-10-14 17:12:01,928][75950] Updated weights for policy 1, policy_version 89720 (0.0009) -[2023-10-14 17:12:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 183992320. Throughput: 0: 1691.2, 1: 1674.1. Samples: 46002248. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:12:03,165][74987] Avg episode reward: [(0, '30.710'), (1, '32.980')] -[2023-10-14 17:12:03,470][75949] Updated weights for policy 0, policy_version 89961 (0.0008) -[2023-10-14 17:12:03,844][75949] Updated weights for policy 0, policy_version 89971 (0.0007) -[2023-10-14 17:12:04,218][75949] Updated weights for policy 0, policy_version 89981 (0.0007) -[2023-10-14 17:12:06,248][75950] Updated weights for policy 1, policy_version 89730 (0.0009) -[2023-10-14 17:12:06,655][75950] Updated weights for policy 1, policy_version 89740 (0.0010) -[2023-10-14 17:12:07,016][75950] Updated weights for policy 1, policy_version 89750 (0.0009) -[2023-10-14 17:12:07,386][75950] Updated weights for policy 1, policy_version 89760 (0.0008) -[2023-10-14 17:12:08,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 184057856. Throughput: 0: 1707.9, 1: 1661.7. Samples: 46022690. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:12:08,164][74987] Avg episode reward: [(0, '26.520'), (1, '33.690')] -[2023-10-14 17:12:08,207][75949] Updated weights for policy 0, policy_version 89991 (0.0008) -[2023-10-14 17:12:08,574][75949] Updated weights for policy 0, policy_version 90001 (0.0009) -[2023-10-14 17:12:08,939][75949] Updated weights for policy 0, policy_version 90011 (0.0008) -[2023-10-14 17:12:11,403][75950] Updated weights for policy 1, policy_version 89770 (0.0010) -[2023-10-14 17:12:11,768][75950] Updated weights for policy 1, policy_version 89780 (0.0010) -[2023-10-14 17:12:12,135][75950] Updated weights for policy 1, policy_version 89790 (0.0008) -[2023-10-14 17:12:13,082][75949] Updated weights for policy 0, policy_version 90021 (0.0007) -[2023-10-14 17:12:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 184123392. Throughput: 0: 1709.5, 1: 1666.1. Samples: 46042700. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:12:13,165][74987] Avg episode reward: [(0, '30.470'), (1, '34.840')] -[2023-10-14 17:12:13,477][75949] Updated weights for policy 0, policy_version 90031 (0.0008) -[2023-10-14 17:12:13,845][75949] Updated weights for policy 0, policy_version 90041 (0.0009) -[2023-10-14 17:12:16,309][75950] Updated weights for policy 1, policy_version 89800 (0.0008) -[2023-10-14 17:12:16,675][75950] Updated weights for policy 1, policy_version 89810 (0.0010) -[2023-10-14 17:12:17,044][75950] Updated weights for policy 1, policy_version 89820 (0.0011) -[2023-10-14 17:12:17,979][75949] Updated weights for policy 0, policy_version 90051 (0.0008) -[2023-10-14 17:12:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 184188928. Throughput: 0: 1701.5, 1: 1668.5. Samples: 46052736. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:12:18,165][74987] Avg episode reward: [(0, '27.180'), (1, '37.160')] -[2023-10-14 17:12:18,358][75949] Updated weights for policy 0, policy_version 90061 (0.0008) -[2023-10-14 17:12:18,731][75949] Updated weights for policy 0, policy_version 90071 (0.0010) -[2023-10-14 17:12:21,033][75950] Updated weights for policy 1, policy_version 89830 (0.0008) -[2023-10-14 17:12:21,396][75950] Updated weights for policy 1, policy_version 89840 (0.0009) -[2023-10-14 17:12:21,771][75950] Updated weights for policy 1, policy_version 89850 (0.0007) -[2023-10-14 17:12:22,757][75949] Updated weights for policy 0, policy_version 90081 (0.0010) -[2023-10-14 17:12:23,131][75949] Updated weights for policy 0, policy_version 90091 (0.0007) -[2023-10-14 17:12:23,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 184254464. Throughput: 0: 1696.4, 1: 1655.6. Samples: 46072466. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:12:23,164][74987] Avg episode reward: [(0, '28.330'), (1, '35.730')] -[2023-10-14 17:12:23,502][75949] Updated weights for policy 0, policy_version 90101 (0.0008) -[2023-10-14 17:12:23,872][75949] Updated weights for policy 0, policy_version 90111 (0.0008) -[2023-10-14 17:12:25,815][75950] Updated weights for policy 1, policy_version 89860 (0.0008) -[2023-10-14 17:12:26,186][75950] Updated weights for policy 1, policy_version 89870 (0.0008) -[2023-10-14 17:12:26,541][75950] Updated weights for policy 1, policy_version 89880 (0.0011) -[2023-10-14 17:12:27,931][75949] Updated weights for policy 0, policy_version 90121 (0.0008) -[2023-10-14 17:12:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 184320000. Throughput: 0: 1694.4, 1: 1675.4. Samples: 46093006. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:12:28,165][74987] Avg episode reward: [(0, '28.150'), (1, '35.510')] -[2023-10-14 17:12:28,307][75949] Updated weights for policy 0, policy_version 90131 (0.0009) -[2023-10-14 17:12:28,676][75949] Updated weights for policy 0, policy_version 90141 (0.0007) -[2023-10-14 17:12:30,755][75950] Updated weights for policy 1, policy_version 89890 (0.0009) -[2023-10-14 17:12:31,123][75950] Updated weights for policy 1, policy_version 89900 (0.0008) -[2023-10-14 17:12:31,480][75950] Updated weights for policy 1, policy_version 89910 (0.0009) -[2023-10-14 17:12:31,843][75950] Updated weights for policy 1, policy_version 89920 (0.0007) -[2023-10-14 17:12:32,835][75949] Updated weights for policy 0, policy_version 90151 (0.0007) -[2023-10-14 17:12:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 184385536. Throughput: 0: 1697.4, 1: 1675.6. Samples: 46103318. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:12:33,165][74987] Avg episode reward: [(0, '29.010'), (1, '35.230')] -[2023-10-14 17:12:33,215][75949] Updated weights for policy 0, policy_version 90161 (0.0007) -[2023-10-14 17:12:33,584][75949] Updated weights for policy 0, policy_version 90171 (0.0007) -[2023-10-14 17:12:35,982][75950] Updated weights for policy 1, policy_version 89930 (0.0009) -[2023-10-14 17:12:36,351][75950] Updated weights for policy 1, policy_version 89940 (0.0008) -[2023-10-14 17:12:36,709][75950] Updated weights for policy 1, policy_version 89950 (0.0008) -[2023-10-14 17:12:37,657][75949] Updated weights for policy 0, policy_version 90181 (0.0008) -[2023-10-14 17:12:38,018][75949] Updated weights for policy 0, policy_version 90191 (0.0007) -[2023-10-14 17:12:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 184451072. Throughput: 0: 1692.6, 1: 1666.2. Samples: 46122882. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) -[2023-10-14 17:12:38,164][74987] Avg episode reward: [(0, '28.860'), (1, '34.750')] -[2023-10-14 17:12:38,394][75949] Updated weights for policy 0, policy_version 90201 (0.0008) -[2023-10-14 17:12:40,700][75950] Updated weights for policy 1, policy_version 89960 (0.0008) -[2023-10-14 17:12:41,069][75950] Updated weights for policy 1, policy_version 89970 (0.0009) -[2023-10-14 17:12:41,428][75950] Updated weights for policy 1, policy_version 89980 (0.0009) -[2023-10-14 17:12:42,471][75949] Updated weights for policy 0, policy_version 90211 (0.0008) -[2023-10-14 17:12:42,844][75949] Updated weights for policy 0, policy_version 90221 (0.0009) -[2023-10-14 17:12:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 184516608. Throughput: 0: 1681.6, 1: 1688.5. Samples: 46143330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:12:43,164][74987] Avg episode reward: [(0, '28.110'), (1, '32.600')] -[2023-10-14 17:12:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000089984_92143616.pth... -[2023-10-14 17:12:43,209][75949] Updated weights for policy 0, policy_version 90231 (0.0009) -[2023-10-14 17:12:43,212][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000088416_90537984.pth -[2023-10-14 17:12:43,552][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000090240_92405760.pth... -[2023-10-14 17:12:43,581][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000088640_90767360.pth -[2023-10-14 17:12:45,378][75950] Updated weights for policy 1, policy_version 89990 (0.0008) -[2023-10-14 17:12:45,745][75950] Updated weights for policy 1, policy_version 90000 (0.0010) -[2023-10-14 17:12:46,106][75950] Updated weights for policy 1, policy_version 90010 (0.0008) -[2023-10-14 17:12:47,359][75949] Updated weights for policy 0, policy_version 90241 (0.0008) -[2023-10-14 17:12:47,729][75949] Updated weights for policy 0, policy_version 90251 (0.0008) -[2023-10-14 17:12:48,099][75949] Updated weights for policy 0, policy_version 90261 (0.0008) -[2023-10-14 17:12:48,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 184582144. Throughput: 0: 1683.5, 1: 1676.5. Samples: 46153448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:12:48,164][74987] Avg episode reward: [(0, '27.310'), (1, '33.740')] -[2023-10-14 17:12:48,468][75949] Updated weights for policy 0, policy_version 90271 (0.0009) -[2023-10-14 17:12:50,150][75950] Updated weights for policy 1, policy_version 90020 (0.0008) -[2023-10-14 17:12:50,519][75950] Updated weights for policy 1, policy_version 90030 (0.0011) -[2023-10-14 17:12:50,884][75950] Updated weights for policy 1, policy_version 90040 (0.0011) -[2023-10-14 17:12:52,398][75949] Updated weights for policy 0, policy_version 90281 (0.0007) -[2023-10-14 17:12:52,773][75949] Updated weights for policy 0, policy_version 90291 (0.0008) -[2023-10-14 17:12:53,134][75949] Updated weights for policy 0, policy_version 90301 (0.0008) -[2023-10-14 17:12:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 184647680. Throughput: 0: 1683.3, 1: 1669.3. Samples: 46173556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:12:53,164][74987] Avg episode reward: [(0, '27.050'), (1, '34.280')] -[2023-10-14 17:12:54,853][75950] Updated weights for policy 1, policy_version 90050 (0.0010) -[2023-10-14 17:12:55,228][75950] Updated weights for policy 1, policy_version 90060 (0.0009) -[2023-10-14 17:12:55,590][75950] Updated weights for policy 1, policy_version 90070 (0.0010) -[2023-10-14 17:12:55,953][75950] Updated weights for policy 1, policy_version 90080 (0.0007) -[2023-10-14 17:12:57,198][75949] Updated weights for policy 0, policy_version 90311 (0.0009) -[2023-10-14 17:12:57,568][75949] Updated weights for policy 0, policy_version 90321 (0.0008) -[2023-10-14 17:12:57,940][75949] Updated weights for policy 0, policy_version 90331 (0.0007) -[2023-10-14 17:12:58,164][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 184745984. Throughput: 0: 1663.5, 1: 1687.8. Samples: 46193510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:12:58,164][74987] Avg episode reward: [(0, '26.820'), (1, '34.430')] -[2023-10-14 17:13:00,002][75950] Updated weights for policy 1, policy_version 90090 (0.0007) -[2023-10-14 17:13:00,366][75950] Updated weights for policy 1, policy_version 90100 (0.0008) -[2023-10-14 17:13:00,729][75950] Updated weights for policy 1, policy_version 90110 (0.0007) -[2023-10-14 17:13:02,116][75949] Updated weights for policy 0, policy_version 90341 (0.0009) -[2023-10-14 17:13:02,500][75949] Updated weights for policy 0, policy_version 90351 (0.0011) -[2023-10-14 17:13:02,870][75949] Updated weights for policy 0, policy_version 90361 (0.0011) -[2023-10-14 17:13:03,163][74987] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 184811520. Throughput: 0: 1686.4, 1: 1670.3. Samples: 46203786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:03,164][74987] Avg episode reward: [(0, '27.960'), (1, '34.040')] -[2023-10-14 17:13:04,886][75950] Updated weights for policy 1, policy_version 90120 (0.0007) -[2023-10-14 17:13:05,254][75950] Updated weights for policy 1, policy_version 90130 (0.0007) -[2023-10-14 17:13:05,634][75950] Updated weights for policy 1, policy_version 90140 (0.0008) -[2023-10-14 17:13:06,889][75949] Updated weights for policy 0, policy_version 90371 (0.0007) -[2023-10-14 17:13:07,260][75949] Updated weights for policy 0, policy_version 90381 (0.0008) -[2023-10-14 17:13:07,629][75949] Updated weights for policy 0, policy_version 90391 (0.0009) -[2023-10-14 17:13:08,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 184877056. Throughput: 0: 1689.8, 1: 1679.1. Samples: 46224064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:08,164][74987] Avg episode reward: [(0, '28.450'), (1, '35.750')] -[2023-10-14 17:13:09,585][75950] Updated weights for policy 1, policy_version 90150 (0.0009) -[2023-10-14 17:13:09,947][75950] Updated weights for policy 1, policy_version 90160 (0.0008) -[2023-10-14 17:13:10,323][75950] Updated weights for policy 1, policy_version 90170 (0.0007) -[2023-10-14 17:13:11,594][75949] Updated weights for policy 0, policy_version 90401 (0.0008) -[2023-10-14 17:13:11,971][75949] Updated weights for policy 0, policy_version 90411 (0.0009) -[2023-10-14 17:13:12,331][75949] Updated weights for policy 0, policy_version 90421 (0.0009) -[2023-10-14 17:13:12,700][75949] Updated weights for policy 0, policy_version 90431 (0.0007) -[2023-10-14 17:13:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 184942592. Throughput: 0: 1665.5, 1: 1687.3. Samples: 46243882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:13,165][74987] Avg episode reward: [(0, '28.630'), (1, '33.550')] -[2023-10-14 17:13:14,547][75950] Updated weights for policy 1, policy_version 90180 (0.0008) -[2023-10-14 17:13:14,905][75950] Updated weights for policy 1, policy_version 90190 (0.0009) -[2023-10-14 17:13:15,277][75950] Updated weights for policy 1, policy_version 90200 (0.0008) -[2023-10-14 17:13:16,576][75949] Updated weights for policy 0, policy_version 90441 (0.0008) -[2023-10-14 17:13:16,942][75949] Updated weights for policy 0, policy_version 90451 (0.0009) -[2023-10-14 17:13:17,307][75949] Updated weights for policy 0, policy_version 90461 (0.0009) -[2023-10-14 17:13:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 185008128. Throughput: 0: 1694.4, 1: 1658.5. Samples: 46254196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:18,164][74987] Avg episode reward: [(0, '27.530'), (1, '34.080')] -[2023-10-14 17:13:19,320][75950] Updated weights for policy 1, policy_version 90210 (0.0007) -[2023-10-14 17:13:19,684][75950] Updated weights for policy 1, policy_version 90220 (0.0008) -[2023-10-14 17:13:20,055][75950] Updated weights for policy 1, policy_version 90230 (0.0008) -[2023-10-14 17:13:20,413][75950] Updated weights for policy 1, policy_version 90240 (0.0009) -[2023-10-14 17:13:21,396][75949] Updated weights for policy 0, policy_version 90471 (0.0008) -[2023-10-14 17:13:21,763][75949] Updated weights for policy 0, policy_version 90481 (0.0010) -[2023-10-14 17:13:22,137][75949] Updated weights for policy 0, policy_version 90491 (0.0008) -[2023-10-14 17:13:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 185073664. Throughput: 0: 1683.3, 1: 1679.5. Samples: 46274208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:23,165][74987] Avg episode reward: [(0, '30.660'), (1, '36.420')] -[2023-10-14 17:13:24,631][75950] Updated weights for policy 1, policy_version 90250 (0.0008) -[2023-10-14 17:13:24,999][75950] Updated weights for policy 1, policy_version 90260 (0.0010) -[2023-10-14 17:13:25,366][75950] Updated weights for policy 1, policy_version 90270 (0.0007) -[2023-10-14 17:13:26,109][75949] Updated weights for policy 0, policy_version 90501 (0.0007) -[2023-10-14 17:13:26,480][75949] Updated weights for policy 0, policy_version 90511 (0.0007) -[2023-10-14 17:13:26,863][75949] Updated weights for policy 0, policy_version 90521 (0.0007) -[2023-10-14 17:13:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 185139200. Throughput: 0: 1676.1, 1: 1676.9. Samples: 46294216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:28,164][74987] Avg episode reward: [(0, '27.250'), (1, '34.840')] -[2023-10-14 17:13:29,516][75950] Updated weights for policy 1, policy_version 90280 (0.0008) -[2023-10-14 17:13:29,876][75950] Updated weights for policy 1, policy_version 90290 (0.0009) -[2023-10-14 17:13:30,244][75950] Updated weights for policy 1, policy_version 90300 (0.0008) -[2023-10-14 17:13:30,921][75949] Updated weights for policy 0, policy_version 90531 (0.0009) -[2023-10-14 17:13:31,278][75949] Updated weights for policy 0, policy_version 90541 (0.0010) -[2023-10-14 17:13:31,643][75949] Updated weights for policy 0, policy_version 90551 (0.0010) -[2023-10-14 17:13:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 185204736. Throughput: 0: 1702.2, 1: 1653.4. Samples: 46304450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:33,165][74987] Avg episode reward: [(0, '27.350'), (1, '33.440')] -[2023-10-14 17:13:34,303][75950] Updated weights for policy 1, policy_version 90310 (0.0007) -[2023-10-14 17:13:34,672][75950] Updated weights for policy 1, policy_version 90320 (0.0007) -[2023-10-14 17:13:35,038][75950] Updated weights for policy 1, policy_version 90330 (0.0008) -[2023-10-14 17:13:35,603][75949] Updated weights for policy 0, policy_version 90561 (0.0007) -[2023-10-14 17:13:35,967][75949] Updated weights for policy 0, policy_version 90571 (0.0008) -[2023-10-14 17:13:36,336][75949] Updated weights for policy 0, policy_version 90581 (0.0009) -[2023-10-14 17:13:36,703][75949] Updated weights for policy 0, policy_version 90591 (0.0010) -[2023-10-14 17:13:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 185270272. Throughput: 0: 1674.4, 1: 1671.1. Samples: 46324104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:38,165][74987] Avg episode reward: [(0, '28.180'), (1, '34.710')] -[2023-10-14 17:13:39,215][75950] Updated weights for policy 1, policy_version 90340 (0.0007) -[2023-10-14 17:13:39,573][75950] Updated weights for policy 1, policy_version 90350 (0.0008) -[2023-10-14 17:13:39,943][75950] Updated weights for policy 1, policy_version 90360 (0.0007) -[2023-10-14 17:13:40,761][75949] Updated weights for policy 0, policy_version 90601 (0.0008) -[2023-10-14 17:13:41,125][75949] Updated weights for policy 0, policy_version 90611 (0.0009) -[2023-10-14 17:13:41,494][75949] Updated weights for policy 0, policy_version 90621 (0.0009) -[2023-10-14 17:13:43,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 185335808. Throughput: 0: 1690.3, 1: 1671.9. Samples: 46344806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:43,164][74987] Avg episode reward: [(0, '26.790'), (1, '35.670')] -[2023-10-14 17:13:44,121][75950] Updated weights for policy 1, policy_version 90370 (0.0009) -[2023-10-14 17:13:44,545][75950] Updated weights for policy 1, policy_version 90380 (0.0008) -[2023-10-14 17:13:44,909][75950] Updated weights for policy 1, policy_version 90390 (0.0007) -[2023-10-14 17:13:45,272][75950] Updated weights for policy 1, policy_version 90400 (0.0008) -[2023-10-14 17:13:45,418][75949] Updated weights for policy 0, policy_version 90631 (0.0008) -[2023-10-14 17:13:45,792][75949] Updated weights for policy 0, policy_version 90641 (0.0007) -[2023-10-14 17:13:46,164][75949] Updated weights for policy 0, policy_version 90651 (0.0008) -[2023-10-14 17:13:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 185401344. Throughput: 0: 1693.0, 1: 1655.6. Samples: 46354472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:48,164][74987] Avg episode reward: [(0, '30.440'), (1, '33.810')] -[2023-10-14 17:13:49,270][75950] Updated weights for policy 1, policy_version 90410 (0.0010) -[2023-10-14 17:13:49,642][75950] Updated weights for policy 1, policy_version 90420 (0.0008) -[2023-10-14 17:13:50,006][75950] Updated weights for policy 1, policy_version 90430 (0.0007) -[2023-10-14 17:13:50,285][75949] Updated weights for policy 0, policy_version 90661 (0.0008) -[2023-10-14 17:13:50,654][75949] Updated weights for policy 0, policy_version 90671 (0.0007) -[2023-10-14 17:13:51,028][75949] Updated weights for policy 0, policy_version 90681 (0.0007) -[2023-10-14 17:13:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 185466880. Throughput: 0: 1675.5, 1: 1664.3. Samples: 46374358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:53,164][74987] Avg episode reward: [(0, '25.970'), (1, '36.030')] -[2023-10-14 17:13:54,052][75950] Updated weights for policy 1, policy_version 90440 (0.0009) -[2023-10-14 17:13:54,413][75950] Updated weights for policy 1, policy_version 90450 (0.0010) -[2023-10-14 17:13:54,778][75950] Updated weights for policy 1, policy_version 90460 (0.0011) -[2023-10-14 17:13:55,221][75949] Updated weights for policy 0, policy_version 90691 (0.0007) -[2023-10-14 17:13:55,638][75949] Updated weights for policy 0, policy_version 90701 (0.0007) -[2023-10-14 17:13:55,998][75949] Updated weights for policy 0, policy_version 90711 (0.0007) -[2023-10-14 17:13:58,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 185532416. Throughput: 0: 1699.0, 1: 1665.8. Samples: 46395298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:13:58,165][74987] Avg episode reward: [(0, '29.340'), (1, '36.440')] -[2023-10-14 17:13:58,808][75950] Updated weights for policy 1, policy_version 90470 (0.0009) -[2023-10-14 17:13:59,169][75950] Updated weights for policy 1, policy_version 90480 (0.0007) -[2023-10-14 17:13:59,538][75950] Updated weights for policy 1, policy_version 90490 (0.0009) -[2023-10-14 17:14:00,017][75949] Updated weights for policy 0, policy_version 90721 (0.0008) -[2023-10-14 17:14:00,393][75949] Updated weights for policy 0, policy_version 90731 (0.0010) -[2023-10-14 17:14:00,775][75949] Updated weights for policy 0, policy_version 90741 (0.0011) -[2023-10-14 17:14:01,139][75949] Updated weights for policy 0, policy_version 90751 (0.0009) -[2023-10-14 17:14:03,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 185597952. Throughput: 0: 1681.4, 1: 1669.3. Samples: 46404980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:14:03,164][74987] Avg episode reward: [(0, '25.630'), (1, '35.580')] -[2023-10-14 17:14:03,551][75950] Updated weights for policy 1, policy_version 90500 (0.0008) -[2023-10-14 17:14:03,914][75950] Updated weights for policy 1, policy_version 90510 (0.0008) -[2023-10-14 17:14:04,288][75950] Updated weights for policy 1, policy_version 90520 (0.0008) -[2023-10-14 17:14:05,274][75949] Updated weights for policy 0, policy_version 90761 (0.0008) -[2023-10-14 17:14:05,643][75949] Updated weights for policy 0, policy_version 90771 (0.0007) -[2023-10-14 17:14:06,000][75949] Updated weights for policy 0, policy_version 90781 (0.0008) -[2023-10-14 17:14:08,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 185663488. Throughput: 0: 1677.4, 1: 1671.8. Samples: 46424922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:14:08,165][74987] Avg episode reward: [(0, '29.040'), (1, '35.960')] -[2023-10-14 17:14:08,567][75950] Updated weights for policy 1, policy_version 90530 (0.0008) -[2023-10-14 17:14:08,937][75950] Updated weights for policy 1, policy_version 90540 (0.0007) -[2023-10-14 17:14:09,295][75950] Updated weights for policy 1, policy_version 90550 (0.0009) -[2023-10-14 17:14:09,656][75950] Updated weights for policy 1, policy_version 90560 (0.0010) -[2023-10-14 17:14:09,871][75949] Updated weights for policy 0, policy_version 90791 (0.0009) -[2023-10-14 17:14:10,236][75949] Updated weights for policy 0, policy_version 90801 (0.0008) -[2023-10-14 17:14:10,611][75949] Updated weights for policy 0, policy_version 90811 (0.0009) -[2023-10-14 17:14:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 185729024. Throughput: 0: 1693.5, 1: 1673.2. Samples: 46445716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:14:13,164][74987] Avg episode reward: [(0, '25.720'), (1, '35.890')] -[2023-10-14 17:14:13,792][75950] Updated weights for policy 1, policy_version 90570 (0.0007) -[2023-10-14 17:14:14,173][75950] Updated weights for policy 1, policy_version 90580 (0.0009) -[2023-10-14 17:14:14,533][75949] Updated weights for policy 0, policy_version 90821 (0.0009) -[2023-10-14 17:14:14,537][75950] Updated weights for policy 1, policy_version 90590 (0.0009) -[2023-10-14 17:14:14,897][75949] Updated weights for policy 0, policy_version 90831 (0.0009) -[2023-10-14 17:14:15,269][75949] Updated weights for policy 0, policy_version 90841 (0.0011) -[2023-10-14 17:14:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 185794560. Throughput: 0: 1665.6, 1: 1679.1. Samples: 46454964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:14:18,165][74987] Avg episode reward: [(0, '27.960'), (1, '36.810')] -[2023-10-14 17:14:18,379][75950] Updated weights for policy 1, policy_version 90600 (0.0009) -[2023-10-14 17:14:18,741][75950] Updated weights for policy 1, policy_version 90610 (0.0009) -[2023-10-14 17:14:19,113][75950] Updated weights for policy 1, policy_version 90620 (0.0008) -[2023-10-14 17:14:19,436][75949] Updated weights for policy 0, policy_version 90851 (0.0010) -[2023-10-14 17:14:19,813][75949] Updated weights for policy 0, policy_version 90861 (0.0007) -[2023-10-14 17:14:20,182][75949] Updated weights for policy 0, policy_version 90871 (0.0008) -[2023-10-14 17:14:23,137][75950] Updated weights for policy 1, policy_version 90630 (0.0008) -[2023-10-14 17:14:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 185860096. Throughput: 0: 1693.9, 1: 1679.1. Samples: 46475890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:14:23,165][74987] Avg episode reward: [(0, '28.800'), (1, '36.960')] -[2023-10-14 17:14:23,499][75950] Updated weights for policy 1, policy_version 90640 (0.0009) -[2023-10-14 17:14:23,870][75950] Updated weights for policy 1, policy_version 90650 (0.0008) -[2023-10-14 17:14:24,118][75949] Updated weights for policy 0, policy_version 90881 (0.0011) -[2023-10-14 17:14:24,490][75949] Updated weights for policy 0, policy_version 90891 (0.0011) -[2023-10-14 17:14:24,863][75949] Updated weights for policy 0, policy_version 90901 (0.0011) -[2023-10-14 17:14:25,233][75949] Updated weights for policy 0, policy_version 90911 (0.0010) -[2023-10-14 17:14:28,012][75950] Updated weights for policy 1, policy_version 90660 (0.0007) -[2023-10-14 17:14:28,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 185925632. Throughput: 0: 1701.3, 1: 1676.2. Samples: 46496796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:14:28,164][74987] Avg episode reward: [(0, '28.540'), (1, '34.420')] -[2023-10-14 17:14:28,372][75950] Updated weights for policy 1, policy_version 90670 (0.0008) -[2023-10-14 17:14:28,746][75950] Updated weights for policy 1, policy_version 90680 (0.0007) -[2023-10-14 17:14:29,243][75949] Updated weights for policy 0, policy_version 90921 (0.0007) -[2023-10-14 17:14:29,620][75949] Updated weights for policy 0, policy_version 90931 (0.0008) -[2023-10-14 17:14:29,985][75949] Updated weights for policy 0, policy_version 90941 (0.0010) -[2023-10-14 17:14:32,867][75950] Updated weights for policy 1, policy_version 90690 (0.0008) -[2023-10-14 17:14:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 185991168. Throughput: 0: 1680.7, 1: 1682.4. Samples: 46505814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:14:33,165][74987] Avg episode reward: [(0, '27.600'), (1, '35.310')] -[2023-10-14 17:14:33,287][75950] Updated weights for policy 1, policy_version 90700 (0.0009) -[2023-10-14 17:14:33,666][75950] Updated weights for policy 1, policy_version 90710 (0.0009) -[2023-10-14 17:14:34,027][75950] Updated weights for policy 1, policy_version 90720 (0.0008) -[2023-10-14 17:14:34,033][75949] Updated weights for policy 0, policy_version 90951 (0.0009) -[2023-10-14 17:14:34,398][75949] Updated weights for policy 0, policy_version 90961 (0.0009) -[2023-10-14 17:14:34,770][75949] Updated weights for policy 0, policy_version 90971 (0.0010) -[2023-10-14 17:14:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186056704. Throughput: 0: 1697.7, 1: 1676.0. Samples: 46526176. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:14:38,165][74987] Avg episode reward: [(0, '26.870'), (1, '35.640')] -[2023-10-14 17:14:38,283][75950] Updated weights for policy 1, policy_version 90730 (0.0008) -[2023-10-14 17:14:38,651][75950] Updated weights for policy 1, policy_version 90740 (0.0009) -[2023-10-14 17:14:38,769][75949] Updated weights for policy 0, policy_version 90981 (0.0010) -[2023-10-14 17:14:39,014][75950] Updated weights for policy 1, policy_version 90750 (0.0010) -[2023-10-14 17:14:39,137][75949] Updated weights for policy 0, policy_version 90991 (0.0008) -[2023-10-14 17:14:39,512][75949] Updated weights for policy 0, policy_version 91001 (0.0007) -[2023-10-14 17:14:43,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186122240. Throughput: 0: 1698.4, 1: 1671.5. Samples: 46546940. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:14:43,164][74987] Avg episode reward: [(0, '25.890'), (1, '32.510')] -[2023-10-14 17:14:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000091008_93192192.pth... -[2023-10-14 17:14:43,188][75950] Updated weights for policy 1, policy_version 90760 (0.0010) -[2023-10-14 17:14:43,209][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000089440_91586560.pth -[2023-10-14 17:14:43,557][75950] Updated weights for policy 1, policy_version 90770 (0.0007) -[2023-10-14 17:14:43,683][75949] Updated weights for policy 0, policy_version 91011 (0.0007) -[2023-10-14 17:14:43,914][75950] Updated weights for policy 1, policy_version 90780 (0.0008) -[2023-10-14 17:14:44,061][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000090784_92962816.pth... -[2023-10-14 17:14:44,076][75949] Updated weights for policy 0, policy_version 91021 (0.0010) -[2023-10-14 17:14:44,095][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000089216_91357184.pth -[2023-10-14 17:14:44,446][75949] Updated weights for policy 0, policy_version 91031 (0.0009) -[2023-10-14 17:14:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 186187776. Throughput: 0: 1682.0, 1: 1665.8. Samples: 46555634. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:14:48,165][74987] Avg episode reward: [(0, '25.990'), (1, '33.330')] -[2023-10-14 17:14:48,244][75950] Updated weights for policy 1, policy_version 90790 (0.0008) -[2023-10-14 17:14:48,487][75949] Updated weights for policy 0, policy_version 91041 (0.0009) -[2023-10-14 17:14:48,612][75950] Updated weights for policy 1, policy_version 90800 (0.0009) -[2023-10-14 17:14:48,857][75949] Updated weights for policy 0, policy_version 91051 (0.0011) -[2023-10-14 17:14:48,974][75950] Updated weights for policy 1, policy_version 90810 (0.0009) -[2023-10-14 17:14:49,232][75949] Updated weights for policy 0, policy_version 91061 (0.0007) -[2023-10-14 17:14:49,599][75949] Updated weights for policy 0, policy_version 91071 (0.0009) -[2023-10-14 17:14:53,018][75950] Updated weights for policy 1, policy_version 90820 (0.0008) -[2023-10-14 17:14:53,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186253312. Throughput: 0: 1698.1, 1: 1663.7. Samples: 46576204. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:14:53,164][74987] Avg episode reward: [(0, '28.100'), (1, '35.170')] -[2023-10-14 17:14:53,371][75950] Updated weights for policy 1, policy_version 90830 (0.0008) -[2023-10-14 17:14:53,665][75949] Updated weights for policy 0, policy_version 91081 (0.0009) -[2023-10-14 17:14:53,739][75950] Updated weights for policy 1, policy_version 90840 (0.0009) -[2023-10-14 17:14:54,027][75949] Updated weights for policy 0, policy_version 91091 (0.0009) -[2023-10-14 17:14:54,406][75949] Updated weights for policy 0, policy_version 91101 (0.0007) -[2023-10-14 17:14:57,887][75950] Updated weights for policy 1, policy_version 90850 (0.0008) -[2023-10-14 17:14:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 186318848. Throughput: 0: 1699.3, 1: 1662.2. Samples: 46596986. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:14:58,164][74987] Avg episode reward: [(0, '25.260'), (1, '34.360')] -[2023-10-14 17:14:58,242][75950] Updated weights for policy 1, policy_version 90860 (0.0008) -[2023-10-14 17:14:58,400][75949] Updated weights for policy 0, policy_version 91111 (0.0008) -[2023-10-14 17:14:58,611][75950] Updated weights for policy 1, policy_version 90870 (0.0007) -[2023-10-14 17:14:58,756][75949] Updated weights for policy 0, policy_version 91121 (0.0008) -[2023-10-14 17:14:58,969][75950] Updated weights for policy 1, policy_version 90880 (0.0008) -[2023-10-14 17:14:59,123][75949] Updated weights for policy 0, policy_version 91131 (0.0009) -[2023-10-14 17:15:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186384384. Throughput: 0: 1697.0, 1: 1661.5. Samples: 46606096. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:15:03,165][74987] Avg episode reward: [(0, '30.580'), (1, '32.900')] -[2023-10-14 17:15:03,167][75949] Updated weights for policy 0, policy_version 91141 (0.0010) -[2023-10-14 17:15:03,203][75950] Updated weights for policy 1, policy_version 90890 (0.0007) -[2023-10-14 17:15:03,532][75949] Updated weights for policy 0, policy_version 91151 (0.0010) -[2023-10-14 17:15:03,567][75950] Updated weights for policy 1, policy_version 90900 (0.0008) -[2023-10-14 17:15:03,908][75949] Updated weights for policy 0, policy_version 91161 (0.0007) -[2023-10-14 17:15:03,936][75950] Updated weights for policy 1, policy_version 90910 (0.0008) -[2023-10-14 17:15:08,042][75949] Updated weights for policy 0, policy_version 91171 (0.0009) -[2023-10-14 17:15:08,097][75950] Updated weights for policy 1, policy_version 90920 (0.0008) -[2023-10-14 17:15:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186449920. Throughput: 0: 1691.4, 1: 1653.1. Samples: 46626390. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:15:08,164][74987] Avg episode reward: [(0, '26.430'), (1, '34.880')] -[2023-10-14 17:15:08,407][75949] Updated weights for policy 0, policy_version 91181 (0.0007) -[2023-10-14 17:15:08,455][75950] Updated weights for policy 1, policy_version 90930 (0.0007) -[2023-10-14 17:15:08,783][75949] Updated weights for policy 0, policy_version 91191 (0.0007) -[2023-10-14 17:15:08,818][75950] Updated weights for policy 1, policy_version 90940 (0.0008) -[2023-10-14 17:15:12,956][75949] Updated weights for policy 0, policy_version 91201 (0.0007) -[2023-10-14 17:15:12,986][75950] Updated weights for policy 1, policy_version 90950 (0.0008) -[2023-10-14 17:15:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186515456. Throughput: 0: 1685.4, 1: 1652.9. Samples: 46647020. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:15:13,164][74987] Avg episode reward: [(0, '30.610'), (1, '36.730')] -[2023-10-14 17:15:13,316][75949] Updated weights for policy 0, policy_version 91211 (0.0008) -[2023-10-14 17:15:13,353][75950] Updated weights for policy 1, policy_version 90960 (0.0009) -[2023-10-14 17:15:13,686][75949] Updated weights for policy 0, policy_version 91221 (0.0008) -[2023-10-14 17:15:13,717][75950] Updated weights for policy 1, policy_version 90970 (0.0007) -[2023-10-14 17:15:14,061][75949] Updated weights for policy 0, policy_version 91231 (0.0007) -[2023-10-14 17:15:17,846][75950] Updated weights for policy 1, policy_version 90980 (0.0009) -[2023-10-14 17:15:18,036][75949] Updated weights for policy 0, policy_version 91241 (0.0008) -[2023-10-14 17:15:18,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186580992. Throughput: 0: 1687.7, 1: 1653.4. Samples: 46656164. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:15:18,164][74987] Avg episode reward: [(0, '25.080'), (1, '35.900')] -[2023-10-14 17:15:18,247][75950] Updated weights for policy 1, policy_version 90990 (0.0009) -[2023-10-14 17:15:18,393][75949] Updated weights for policy 0, policy_version 91251 (0.0007) -[2023-10-14 17:15:18,613][75950] Updated weights for policy 1, policy_version 91000 (0.0008) -[2023-10-14 17:15:18,772][75949] Updated weights for policy 0, policy_version 91261 (0.0008) -[2023-10-14 17:15:22,703][75950] Updated weights for policy 1, policy_version 91010 (0.0008) -[2023-10-14 17:15:22,765][75949] Updated weights for policy 0, policy_version 91271 (0.0008) -[2023-10-14 17:15:23,067][75950] Updated weights for policy 1, policy_version 91020 (0.0008) -[2023-10-14 17:15:23,145][75949] Updated weights for policy 0, policy_version 91281 (0.0007) -[2023-10-14 17:15:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186646528. Throughput: 0: 1685.6, 1: 1649.1. Samples: 46676236. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:15:23,165][74987] Avg episode reward: [(0, '28.960'), (1, '33.490')] -[2023-10-14 17:15:23,432][75950] Updated weights for policy 1, policy_version 91030 (0.0008) -[2023-10-14 17:15:23,508][75949] Updated weights for policy 0, policy_version 91291 (0.0009) -[2023-10-14 17:15:23,797][75950] Updated weights for policy 1, policy_version 91040 (0.0010) -[2023-10-14 17:15:27,665][75949] Updated weights for policy 0, policy_version 91301 (0.0007) -[2023-10-14 17:15:28,036][75949] Updated weights for policy 0, policy_version 91311 (0.0008) -[2023-10-14 17:15:28,106][75950] Updated weights for policy 1, policy_version 91050 (0.0009) -[2023-10-14 17:15:28,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186712064. Throughput: 0: 1678.8, 1: 1649.6. Samples: 46696714. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:15:28,164][74987] Avg episode reward: [(0, '26.720'), (1, '36.020')] -[2023-10-14 17:15:28,412][75949] Updated weights for policy 0, policy_version 91321 (0.0008) -[2023-10-14 17:15:28,464][75950] Updated weights for policy 1, policy_version 91060 (0.0009) -[2023-10-14 17:15:28,838][75950] Updated weights for policy 1, policy_version 91070 (0.0009) -[2023-10-14 17:15:32,586][75949] Updated weights for policy 0, policy_version 91331 (0.0010) -[2023-10-14 17:15:32,932][75950] Updated weights for policy 1, policy_version 91080 (0.0007) -[2023-10-14 17:15:32,978][75949] Updated weights for policy 0, policy_version 91341 (0.0008) -[2023-10-14 17:15:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186777600. Throughput: 0: 1685.5, 1: 1653.1. Samples: 46705870. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) -[2023-10-14 17:15:33,165][74987] Avg episode reward: [(0, '29.810'), (1, '35.340')] -[2023-10-14 17:15:33,301][75950] Updated weights for policy 1, policy_version 91090 (0.0009) -[2023-10-14 17:15:33,351][75949] Updated weights for policy 0, policy_version 91351 (0.0009) -[2023-10-14 17:15:33,665][75950] Updated weights for policy 1, policy_version 91100 (0.0008) -[2023-10-14 17:15:37,325][75949] Updated weights for policy 0, policy_version 91361 (0.0007) -[2023-10-14 17:15:37,684][75949] Updated weights for policy 0, policy_version 91371 (0.0009) -[2023-10-14 17:15:37,753][75950] Updated weights for policy 1, policy_version 91110 (0.0009) -[2023-10-14 17:15:38,049][75949] Updated weights for policy 0, policy_version 91381 (0.0009) -[2023-10-14 17:15:38,122][75950] Updated weights for policy 1, policy_version 91120 (0.0007) -[2023-10-14 17:15:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186843136. Throughput: 0: 1683.2, 1: 1652.8. Samples: 46726326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:15:38,164][74987] Avg episode reward: [(0, '27.300'), (1, '32.930')] -[2023-10-14 17:15:38,428][75949] Updated weights for policy 0, policy_version 91391 (0.0009) -[2023-10-14 17:15:38,480][75950] Updated weights for policy 1, policy_version 91130 (0.0009) -[2023-10-14 17:15:42,499][75949] Updated weights for policy 0, policy_version 91401 (0.0007) -[2023-10-14 17:15:42,582][75950] Updated weights for policy 1, policy_version 91140 (0.0010) -[2023-10-14 17:15:42,860][75949] Updated weights for policy 0, policy_version 91411 (0.0007) -[2023-10-14 17:15:42,951][75950] Updated weights for policy 1, policy_version 91150 (0.0008) -[2023-10-14 17:15:43,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186908672. Throughput: 0: 1664.6, 1: 1649.2. Samples: 46746110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:15:43,164][74987] Avg episode reward: [(0, '29.830'), (1, '33.730')] -[2023-10-14 17:15:43,224][75949] Updated weights for policy 0, policy_version 91421 (0.0007) -[2023-10-14 17:15:43,314][75950] Updated weights for policy 1, policy_version 91160 (0.0008) -[2023-10-14 17:15:47,395][75950] Updated weights for policy 1, policy_version 91170 (0.0008) -[2023-10-14 17:15:47,413][75949] Updated weights for policy 0, policy_version 91431 (0.0009) -[2023-10-14 17:15:47,746][75950] Updated weights for policy 1, policy_version 91180 (0.0008) -[2023-10-14 17:15:47,786][75949] Updated weights for policy 0, policy_version 91441 (0.0009) -[2023-10-14 17:15:48,116][75950] Updated weights for policy 1, policy_version 91190 (0.0010) -[2023-10-14 17:15:48,143][75949] Updated weights for policy 0, policy_version 91451 (0.0009) -[2023-10-14 17:15:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186974208. Throughput: 0: 1674.6, 1: 1658.2. Samples: 46756070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:15:48,164][74987] Avg episode reward: [(0, '27.260'), (1, '34.190')] -[2023-10-14 17:15:48,485][75950] Updated weights for policy 1, policy_version 91200 (0.0008) -[2023-10-14 17:15:52,334][75949] Updated weights for policy 0, policy_version 91461 (0.0008) -[2023-10-14 17:15:52,602][75950] Updated weights for policy 1, policy_version 91210 (0.0007) -[2023-10-14 17:15:52,705][75949] Updated weights for policy 0, policy_version 91471 (0.0008) -[2023-10-14 17:15:52,965][75950] Updated weights for policy 1, policy_version 91220 (0.0009) -[2023-10-14 17:15:53,078][75949] Updated weights for policy 0, policy_version 91481 (0.0008) -[2023-10-14 17:15:53,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 187039744. Throughput: 0: 1672.3, 1: 1666.5. Samples: 46776636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:15:53,164][74987] Avg episode reward: [(0, '29.220'), (1, '34.180')] -[2023-10-14 17:15:53,335][75950] Updated weights for policy 1, policy_version 91230 (0.0008) -[2023-10-14 17:15:57,208][75949] Updated weights for policy 0, policy_version 91491 (0.0009) -[2023-10-14 17:15:57,263][75950] Updated weights for policy 1, policy_version 91240 (0.0008) -[2023-10-14 17:15:57,577][75949] Updated weights for policy 0, policy_version 91501 (0.0008) -[2023-10-14 17:15:57,625][75950] Updated weights for policy 1, policy_version 91250 (0.0007) -[2023-10-14 17:15:57,937][75949] Updated weights for policy 0, policy_version 91511 (0.0008) -[2023-10-14 17:15:57,986][75950] Updated weights for policy 1, policy_version 91260 (0.0008) -[2023-10-14 17:15:58,164][74987] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 187138048. Throughput: 0: 1653.4, 1: 1654.6. Samples: 46795882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:15:58,165][74987] Avg episode reward: [(0, '30.090'), (1, '34.380')] -[2023-10-14 17:16:01,925][75949] Updated weights for policy 0, policy_version 91521 (0.0007) -[2023-10-14 17:16:02,174][75950] Updated weights for policy 1, policy_version 91270 (0.0008) -[2023-10-14 17:16:02,283][75949] Updated weights for policy 0, policy_version 91531 (0.0007) -[2023-10-14 17:16:02,549][75950] Updated weights for policy 1, policy_version 91280 (0.0009) -[2023-10-14 17:16:02,654][75949] Updated weights for policy 0, policy_version 91541 (0.0008) -[2023-10-14 17:16:02,912][75950] Updated weights for policy 1, policy_version 91290 (0.0008) -[2023-10-14 17:16:03,032][75949] Updated weights for policy 0, policy_version 91551 (0.0007) -[2023-10-14 17:16:03,164][74987] Fps is (10 sec: 19660.7, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 187236352. Throughput: 0: 1665.8, 1: 1669.5. Samples: 46806254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:16:03,164][74987] Avg episode reward: [(0, '28.080'), (1, '34.690')] -[2023-10-14 17:16:06,963][75950] Updated weights for policy 1, policy_version 91300 (0.0008) -[2023-10-14 17:16:07,131][75949] Updated weights for policy 0, policy_version 91561 (0.0007) -[2023-10-14 17:16:07,351][75950] Updated weights for policy 1, policy_version 91310 (0.0007) -[2023-10-14 17:16:07,507][75949] Updated weights for policy 0, policy_version 91571 (0.0007) -[2023-10-14 17:16:07,718][75950] Updated weights for policy 1, policy_version 91320 (0.0009) -[2023-10-14 17:16:07,870][75949] Updated weights for policy 0, policy_version 91581 (0.0008) -[2023-10-14 17:16:08,164][74987] Fps is (10 sec: 16384.1, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 187301888. Throughput: 0: 1667.1, 1: 1678.7. Samples: 46826798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:16:08,165][74987] Avg episode reward: [(0, '29.170'), (1, '35.590')] -[2023-10-14 17:16:11,815][75950] Updated weights for policy 1, policy_version 91330 (0.0008) -[2023-10-14 17:16:12,079][75949] Updated weights for policy 0, policy_version 91591 (0.0007) -[2023-10-14 17:16:12,181][75950] Updated weights for policy 1, policy_version 91340 (0.0007) -[2023-10-14 17:16:12,453][75949] Updated weights for policy 0, policy_version 91601 (0.0009) -[2023-10-14 17:16:12,536][75950] Updated weights for policy 1, policy_version 91350 (0.0008) -[2023-10-14 17:16:12,824][75949] Updated weights for policy 0, policy_version 91611 (0.0007) -[2023-10-14 17:16:12,903][75950] Updated weights for policy 1, policy_version 91360 (0.0008) -[2023-10-14 17:16:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 187367424. Throughput: 0: 1649.7, 1: 1656.1. Samples: 46845476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:16:13,164][74987] Avg episode reward: [(0, '28.770'), (1, '33.820')] -[2023-10-14 17:16:16,805][75949] Updated weights for policy 0, policy_version 91621 (0.0008) -[2023-10-14 17:16:17,048][75950] Updated weights for policy 1, policy_version 91370 (0.0007) -[2023-10-14 17:16:17,172][75949] Updated weights for policy 0, policy_version 91631 (0.0008) -[2023-10-14 17:16:17,413][75950] Updated weights for policy 1, policy_version 91380 (0.0008) -[2023-10-14 17:16:17,542][75949] Updated weights for policy 0, policy_version 91641 (0.0009) -[2023-10-14 17:16:17,784][75950] Updated weights for policy 1, policy_version 91390 (0.0008) -[2023-10-14 17:16:18,164][74987] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 187432960. Throughput: 0: 1669.1, 1: 1676.1. Samples: 46856404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:16:18,164][74987] Avg episode reward: [(0, '29.470'), (1, '34.640')] -[2023-10-14 17:16:21,689][75950] Updated weights for policy 1, policy_version 91400 (0.0008) -[2023-10-14 17:16:21,754][75949] Updated weights for policy 0, policy_version 91651 (0.0009) -[2023-10-14 17:16:22,046][75950] Updated weights for policy 1, policy_version 91410 (0.0008) -[2023-10-14 17:16:22,152][75949] Updated weights for policy 0, policy_version 91661 (0.0009) -[2023-10-14 17:16:22,420][75950] Updated weights for policy 1, policy_version 91420 (0.0008) -[2023-10-14 17:16:22,524][75949] Updated weights for policy 0, policy_version 91671 (0.0007) -[2023-10-14 17:16:23,163][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 187498496. Throughput: 0: 1669.2, 1: 1677.0. Samples: 46876906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:16:23,164][74987] Avg episode reward: [(0, '27.330'), (1, '35.060')] -[2023-10-14 17:16:26,440][75949] Updated weights for policy 0, policy_version 91681 (0.0007) -[2023-10-14 17:16:26,568][75950] Updated weights for policy 1, policy_version 91430 (0.0009) -[2023-10-14 17:16:26,804][75949] Updated weights for policy 0, policy_version 91691 (0.0008) -[2023-10-14 17:16:26,931][75950] Updated weights for policy 1, policy_version 91440 (0.0009) -[2023-10-14 17:16:27,171][75949] Updated weights for policy 0, policy_version 91701 (0.0007) -[2023-10-14 17:16:27,285][75950] Updated weights for policy 1, policy_version 91450 (0.0009) -[2023-10-14 17:16:27,543][75949] Updated weights for policy 0, policy_version 91711 (0.0007) -[2023-10-14 17:16:28,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 187564032. Throughput: 0: 1655.4, 1: 1662.4. Samples: 46895412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:16:28,165][74987] Avg episode reward: [(0, '28.850'), (1, '35.480')] -[2023-10-14 17:16:31,448][75950] Updated weights for policy 1, policy_version 91460 (0.0008) -[2023-10-14 17:16:31,696][75949] Updated weights for policy 0, policy_version 91721 (0.0007) -[2023-10-14 17:16:31,819][75950] Updated weights for policy 1, policy_version 91470 (0.0008) -[2023-10-14 17:16:32,070][75949] Updated weights for policy 0, policy_version 91731 (0.0007) -[2023-10-14 17:16:32,184][75950] Updated weights for policy 1, policy_version 91480 (0.0007) -[2023-10-14 17:16:32,441][75949] Updated weights for policy 0, policy_version 91741 (0.0009) -[2023-10-14 17:16:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 187629568. Throughput: 0: 1671.2, 1: 1682.7. Samples: 46906994. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:16:33,164][74987] Avg episode reward: [(0, '28.220'), (1, '36.190')] -[2023-10-14 17:16:36,234][75950] Updated weights for policy 1, policy_version 91490 (0.0007) -[2023-10-14 17:16:36,329][75949] Updated weights for policy 0, policy_version 91751 (0.0008) -[2023-10-14 17:16:36,602][75950] Updated weights for policy 1, policy_version 91500 (0.0009) -[2023-10-14 17:16:36,702][75949] Updated weights for policy 0, policy_version 91761 (0.0009) -[2023-10-14 17:16:36,970][75950] Updated weights for policy 1, policy_version 91510 (0.0008) -[2023-10-14 17:16:37,073][75949] Updated weights for policy 0, policy_version 91771 (0.0009) -[2023-10-14 17:16:37,332][75950] Updated weights for policy 1, policy_version 91520 (0.0009) -[2023-10-14 17:16:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 187695104. Throughput: 0: 1662.3, 1: 1671.1. Samples: 46926638. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:16:38,164][74987] Avg episode reward: [(0, '28.580'), (1, '35.310')] -[2023-10-14 17:16:41,084][75949] Updated weights for policy 0, policy_version 91781 (0.0009) -[2023-10-14 17:16:41,409][75950] Updated weights for policy 1, policy_version 91530 (0.0009) -[2023-10-14 17:16:41,459][75949] Updated weights for policy 0, policy_version 91791 (0.0010) -[2023-10-14 17:16:41,770][75950] Updated weights for policy 1, policy_version 91540 (0.0007) -[2023-10-14 17:16:41,831][75949] Updated weights for policy 0, policy_version 91801 (0.0009) -[2023-10-14 17:16:42,146][75950] Updated weights for policy 1, policy_version 91550 (0.0009) -[2023-10-14 17:16:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 187760640. Throughput: 0: 1667.2, 1: 1665.6. Samples: 46945856. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:16:43,164][74987] Avg episode reward: [(0, '26.740'), (1, '35.090')] -[2023-10-14 17:16:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000091552_93749248.pth... -[2023-10-14 17:16:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000091808_94011392.pth... -[2023-10-14 17:16:43,203][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000089984_92143616.pth -[2023-10-14 17:16:43,210][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000090240_92405760.pth -[2023-10-14 17:16:46,052][75949] Updated weights for policy 0, policy_version 91811 (0.0008) -[2023-10-14 17:16:46,157][75950] Updated weights for policy 1, policy_version 91560 (0.0008) -[2023-10-14 17:16:46,411][75949] Updated weights for policy 0, policy_version 91821 (0.0008) -[2023-10-14 17:16:46,514][75950] Updated weights for policy 1, policy_version 91570 (0.0008) -[2023-10-14 17:16:46,783][75949] Updated weights for policy 0, policy_version 91831 (0.0008) -[2023-10-14 17:16:46,882][75950] Updated weights for policy 1, policy_version 91580 (0.0008) -[2023-10-14 17:16:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 187826176. Throughput: 0: 1679.6, 1: 1676.3. Samples: 46957272. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:16:48,165][74987] Avg episode reward: [(0, '28.330'), (1, '36.140')] -[2023-10-14 17:16:50,835][75949] Updated weights for policy 0, policy_version 91841 (0.0007) -[2023-10-14 17:16:51,079][75950] Updated weights for policy 1, policy_version 91590 (0.0007) -[2023-10-14 17:16:51,207][75949] Updated weights for policy 0, policy_version 91851 (0.0010) -[2023-10-14 17:16:51,453][75950] Updated weights for policy 1, policy_version 91600 (0.0009) -[2023-10-14 17:16:51,574][75949] Updated weights for policy 0, policy_version 91861 (0.0009) -[2023-10-14 17:16:51,815][75950] Updated weights for policy 1, policy_version 91610 (0.0007) -[2023-10-14 17:16:51,937][75949] Updated weights for policy 0, policy_version 91871 (0.0008) -[2023-10-14 17:16:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 187891712. Throughput: 0: 1662.4, 1: 1654.8. Samples: 46976074. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:16:53,164][74987] Avg episode reward: [(0, '27.840'), (1, '35.840')] -[2023-10-14 17:16:55,941][75949] Updated weights for policy 0, policy_version 91881 (0.0008) -[2023-10-14 17:16:56,132][75950] Updated weights for policy 1, policy_version 91620 (0.0008) -[2023-10-14 17:16:56,312][75949] Updated weights for policy 0, policy_version 91891 (0.0007) -[2023-10-14 17:16:56,535][75950] Updated weights for policy 1, policy_version 91630 (0.0007) -[2023-10-14 17:16:56,688][75949] Updated weights for policy 0, policy_version 91901 (0.0008) -[2023-10-14 17:16:56,902][75950] Updated weights for policy 1, policy_version 91640 (0.0009) -[2023-10-14 17:16:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 187957248. Throughput: 0: 1681.1, 1: 1662.1. Samples: 46995918. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:16:58,164][74987] Avg episode reward: [(0, '28.410'), (1, '35.510')] -[2023-10-14 17:17:00,675][75949] Updated weights for policy 0, policy_version 91911 (0.0008) -[2023-10-14 17:17:00,946][75950] Updated weights for policy 1, policy_version 91650 (0.0009) -[2023-10-14 17:17:01,053][75949] Updated weights for policy 0, policy_version 91921 (0.0009) -[2023-10-14 17:17:01,315][75950] Updated weights for policy 1, policy_version 91660 (0.0008) -[2023-10-14 17:17:01,415][75949] Updated weights for policy 0, policy_version 91931 (0.0008) -[2023-10-14 17:17:01,677][75950] Updated weights for policy 1, policy_version 91670 (0.0009) -[2023-10-14 17:17:02,043][75950] Updated weights for policy 1, policy_version 91680 (0.0009) -[2023-10-14 17:17:03,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188022784. Throughput: 0: 1682.6, 1: 1671.4. Samples: 47007334. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:17:03,165][74987] Avg episode reward: [(0, '27.540'), (1, '34.860')] -[2023-10-14 17:17:05,489][75949] Updated weights for policy 0, policy_version 91941 (0.0009) -[2023-10-14 17:17:05,861][75949] Updated weights for policy 0, policy_version 91951 (0.0008) -[2023-10-14 17:17:05,986][75950] Updated weights for policy 1, policy_version 91690 (0.0009) -[2023-10-14 17:17:06,235][75949] Updated weights for policy 0, policy_version 91961 (0.0008) -[2023-10-14 17:17:06,346][75950] Updated weights for policy 1, policy_version 91700 (0.0008) -[2023-10-14 17:17:06,714][75950] Updated weights for policy 1, policy_version 91710 (0.0008) -[2023-10-14 17:17:08,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188088320. Throughput: 0: 1663.1, 1: 1655.5. Samples: 47026240. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:17:08,164][74987] Avg episode reward: [(0, '29.800'), (1, '36.300')] -[2023-10-14 17:17:10,358][75949] Updated weights for policy 0, policy_version 91971 (0.0009) -[2023-10-14 17:17:10,757][75949] Updated weights for policy 0, policy_version 91981 (0.0008) -[2023-10-14 17:17:10,852][75950] Updated weights for policy 1, policy_version 91720 (0.0008) -[2023-10-14 17:17:11,125][75949] Updated weights for policy 0, policy_version 91991 (0.0008) -[2023-10-14 17:17:11,218][75950] Updated weights for policy 1, policy_version 91730 (0.0008) -[2023-10-14 17:17:11,576][75950] Updated weights for policy 1, policy_version 91740 (0.0008) -[2023-10-14 17:17:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188153856. Throughput: 0: 1686.9, 1: 1669.6. Samples: 47046450. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:17:13,164][74987] Avg episode reward: [(0, '28.240'), (1, '36.400')] -[2023-10-14 17:17:15,435][75949] Updated weights for policy 0, policy_version 92001 (0.0008) -[2023-10-14 17:17:15,775][75950] Updated weights for policy 1, policy_version 91750 (0.0009) -[2023-10-14 17:17:15,802][75949] Updated weights for policy 0, policy_version 92011 (0.0007) -[2023-10-14 17:17:16,144][75950] Updated weights for policy 1, policy_version 91760 (0.0010) -[2023-10-14 17:17:16,169][75949] Updated weights for policy 0, policy_version 92021 (0.0007) -[2023-10-14 17:17:16,515][75950] Updated weights for policy 1, policy_version 91770 (0.0008) -[2023-10-14 17:17:16,534][75949] Updated weights for policy 0, policy_version 92031 (0.0007) -[2023-10-14 17:17:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188219392. Throughput: 0: 1679.6, 1: 1663.7. Samples: 47057446. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:17:18,165][74987] Avg episode reward: [(0, '29.260'), (1, '35.330')] -[2023-10-14 17:17:20,502][75950] Updated weights for policy 1, policy_version 91780 (0.0009) -[2023-10-14 17:17:20,577][75949] Updated weights for policy 0, policy_version 92041 (0.0007) -[2023-10-14 17:17:20,869][75950] Updated weights for policy 1, policy_version 91790 (0.0009) -[2023-10-14 17:17:20,952][75949] Updated weights for policy 0, policy_version 92051 (0.0008) -[2023-10-14 17:17:21,235][75950] Updated weights for policy 1, policy_version 91800 (0.0009) -[2023-10-14 17:17:21,320][75949] Updated weights for policy 0, policy_version 92061 (0.0009) -[2023-10-14 17:17:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188284928. Throughput: 0: 1666.6, 1: 1651.3. Samples: 47075946. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:17:23,165][74987] Avg episode reward: [(0, '27.960'), (1, '36.360')] -[2023-10-14 17:17:25,357][75950] Updated weights for policy 1, policy_version 91810 (0.0010) -[2023-10-14 17:17:25,440][75949] Updated weights for policy 0, policy_version 92071 (0.0008) -[2023-10-14 17:17:25,725][75950] Updated weights for policy 1, policy_version 91820 (0.0008) -[2023-10-14 17:17:25,802][75949] Updated weights for policy 0, policy_version 92081 (0.0009) -[2023-10-14 17:17:26,096][75950] Updated weights for policy 1, policy_version 91830 (0.0007) -[2023-10-14 17:17:26,165][75949] Updated weights for policy 0, policy_version 92091 (0.0010) -[2023-10-14 17:17:26,455][75950] Updated weights for policy 1, policy_version 91840 (0.0008) -[2023-10-14 17:17:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188350464. Throughput: 0: 1683.2, 1: 1666.7. Samples: 47096602. Policy #0 lag: (min: 10.0, avg: 12.5, max: 42.0) -[2023-10-14 17:17:28,165][74987] Avg episode reward: [(0, '27.170'), (1, '38.190')] -[2023-10-14 17:17:30,234][75949] Updated weights for policy 0, policy_version 92101 (0.0009) -[2023-10-14 17:17:30,603][75949] Updated weights for policy 0, policy_version 92111 (0.0007) -[2023-10-14 17:17:30,670][75950] Updated weights for policy 1, policy_version 91850 (0.0007) -[2023-10-14 17:17:30,980][75949] Updated weights for policy 0, policy_version 92121 (0.0008) -[2023-10-14 17:17:31,042][75950] Updated weights for policy 1, policy_version 91860 (0.0008) -[2023-10-14 17:17:31,401][75950] Updated weights for policy 1, policy_version 91870 (0.0008) -[2023-10-14 17:17:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188416000. Throughput: 0: 1669.2, 1: 1660.5. Samples: 47107106. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:17:33,165][74987] Avg episode reward: [(0, '29.850'), (1, '37.700')] -[2023-10-14 17:17:35,060][75949] Updated weights for policy 0, policy_version 92131 (0.0008) -[2023-10-14 17:17:35,435][75949] Updated weights for policy 0, policy_version 92141 (0.0007) -[2023-10-14 17:17:35,499][75950] Updated weights for policy 1, policy_version 91880 (0.0007) -[2023-10-14 17:17:35,786][75949] Updated weights for policy 0, policy_version 92151 (0.0008) -[2023-10-14 17:17:35,869][75950] Updated weights for policy 1, policy_version 91890 (0.0007) -[2023-10-14 17:17:36,236][75950] Updated weights for policy 1, policy_version 91900 (0.0008) -[2023-10-14 17:17:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188481536. Throughput: 0: 1674.5, 1: 1659.3. Samples: 47126096. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:17:38,165][74987] Avg episode reward: [(0, '27.860'), (1, '35.350')] -[2023-10-14 17:17:39,723][75949] Updated weights for policy 0, policy_version 92161 (0.0008) -[2023-10-14 17:17:40,099][75949] Updated weights for policy 0, policy_version 92171 (0.0009) -[2023-10-14 17:17:40,454][75950] Updated weights for policy 1, policy_version 91910 (0.0007) -[2023-10-14 17:17:40,467][75949] Updated weights for policy 0, policy_version 92181 (0.0008) -[2023-10-14 17:17:40,811][75950] Updated weights for policy 1, policy_version 91920 (0.0010) -[2023-10-14 17:17:40,843][75949] Updated weights for policy 0, policy_version 92191 (0.0009) -[2023-10-14 17:17:41,190][75950] Updated weights for policy 1, policy_version 91930 (0.0009) -[2023-10-14 17:17:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188547072. Throughput: 0: 1678.8, 1: 1672.6. Samples: 47146732. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:17:43,165][74987] Avg episode reward: [(0, '32.820'), (1, '36.820')] -[2023-10-14 17:17:43,175][75615] Saving new best policy, reward=32.820! -[2023-10-14 17:17:44,940][75949] Updated weights for policy 0, policy_version 92201 (0.0009) -[2023-10-14 17:17:45,301][75949] Updated weights for policy 0, policy_version 92211 (0.0008) -[2023-10-14 17:17:45,333][75950] Updated weights for policy 1, policy_version 91940 (0.0009) -[2023-10-14 17:17:45,669][75949] Updated weights for policy 0, policy_version 92221 (0.0007) -[2023-10-14 17:17:45,737][75950] Updated weights for policy 1, policy_version 91950 (0.0009) -[2023-10-14 17:17:46,096][75950] Updated weights for policy 1, policy_version 91960 (0.0009) -[2023-10-14 17:17:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188612608. Throughput: 0: 1657.3, 1: 1659.9. Samples: 47156608. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:17:48,165][74987] Avg episode reward: [(0, '27.430'), (1, '38.030')] -[2023-10-14 17:17:49,806][75949] Updated weights for policy 0, policy_version 92231 (0.0008) -[2023-10-14 17:17:50,174][75949] Updated weights for policy 0, policy_version 92241 (0.0008) -[2023-10-14 17:17:50,287][75950] Updated weights for policy 1, policy_version 91970 (0.0008) -[2023-10-14 17:17:50,543][75949] Updated weights for policy 0, policy_version 92251 (0.0010) -[2023-10-14 17:17:50,655][75950] Updated weights for policy 1, policy_version 91980 (0.0009) -[2023-10-14 17:17:51,021][75950] Updated weights for policy 1, policy_version 91990 (0.0008) -[2023-10-14 17:17:51,384][75950] Updated weights for policy 1, policy_version 92000 (0.0009) -[2023-10-14 17:17:53,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188678144. Throughput: 0: 1672.4, 1: 1654.7. Samples: 47175956. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:17:53,164][74987] Avg episode reward: [(0, '31.750'), (1, '34.920')] -[2023-10-14 17:17:54,530][75949] Updated weights for policy 0, policy_version 92261 (0.0007) -[2023-10-14 17:17:54,922][75949] Updated weights for policy 0, policy_version 92271 (0.0008) -[2023-10-14 17:17:55,299][75949] Updated weights for policy 0, policy_version 92281 (0.0008) -[2023-10-14 17:17:55,447][75950] Updated weights for policy 1, policy_version 92010 (0.0010) -[2023-10-14 17:17:55,823][75950] Updated weights for policy 1, policy_version 92020 (0.0009) -[2023-10-14 17:17:56,188][75950] Updated weights for policy 1, policy_version 92030 (0.0011) -[2023-10-14 17:17:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 188743680. Throughput: 0: 1677.5, 1: 1660.3. Samples: 47196654. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:17:58,165][74987] Avg episode reward: [(0, '26.550'), (1, '35.060')] -[2023-10-14 17:17:59,359][75949] Updated weights for policy 0, policy_version 92291 (0.0007) -[2023-10-14 17:17:59,732][75949] Updated weights for policy 0, policy_version 92301 (0.0009) -[2023-10-14 17:18:00,102][75949] Updated weights for policy 0, policy_version 92311 (0.0008) -[2023-10-14 17:18:00,265][75950] Updated weights for policy 1, policy_version 92040 (0.0010) -[2023-10-14 17:18:00,633][75950] Updated weights for policy 1, policy_version 92050 (0.0007) -[2023-10-14 17:18:01,004][75950] Updated weights for policy 1, policy_version 92060 (0.0009) -[2023-10-14 17:18:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188809216. Throughput: 0: 1654.8, 1: 1654.1. Samples: 47206344. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:18:03,164][74987] Avg episode reward: [(0, '28.810'), (1, '35.660')] -[2023-10-14 17:18:04,233][75949] Updated weights for policy 0, policy_version 92321 (0.0008) -[2023-10-14 17:18:04,598][75949] Updated weights for policy 0, policy_version 92331 (0.0007) -[2023-10-14 17:18:04,966][75949] Updated weights for policy 0, policy_version 92341 (0.0009) -[2023-10-14 17:18:05,027][75950] Updated weights for policy 1, policy_version 92070 (0.0007) -[2023-10-14 17:18:05,331][75949] Updated weights for policy 0, policy_version 92351 (0.0008) -[2023-10-14 17:18:05,396][75950] Updated weights for policy 1, policy_version 92080 (0.0007) -[2023-10-14 17:18:05,767][75950] Updated weights for policy 1, policy_version 92090 (0.0009) -[2023-10-14 17:18:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 188874752. Throughput: 0: 1676.0, 1: 1663.6. Samples: 47226228. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:18:08,165][74987] Avg episode reward: [(0, '26.770'), (1, '34.140')] -[2023-10-14 17:18:09,385][75949] Updated weights for policy 0, policy_version 92361 (0.0007) -[2023-10-14 17:18:09,756][75949] Updated weights for policy 0, policy_version 92371 (0.0009) -[2023-10-14 17:18:09,798][75950] Updated weights for policy 1, policy_version 92100 (0.0008) -[2023-10-14 17:18:10,115][75949] Updated weights for policy 0, policy_version 92381 (0.0007) -[2023-10-14 17:18:10,158][75950] Updated weights for policy 1, policy_version 92110 (0.0007) -[2023-10-14 17:18:10,532][75950] Updated weights for policy 1, policy_version 92120 (0.0007) -[2023-10-14 17:18:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188940288. Throughput: 0: 1677.1, 1: 1667.1. Samples: 47247092. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:18:13,164][74987] Avg episode reward: [(0, '28.160'), (1, '33.160')] -[2023-10-14 17:18:14,200][75949] Updated weights for policy 0, policy_version 92391 (0.0008) -[2023-10-14 17:18:14,561][75949] Updated weights for policy 0, policy_version 92401 (0.0009) -[2023-10-14 17:18:14,601][75950] Updated weights for policy 1, policy_version 92130 (0.0010) -[2023-10-14 17:18:14,932][75949] Updated weights for policy 0, policy_version 92411 (0.0009) -[2023-10-14 17:18:14,978][75950] Updated weights for policy 1, policy_version 92140 (0.0007) -[2023-10-14 17:18:15,334][75950] Updated weights for policy 1, policy_version 92150 (0.0007) -[2023-10-14 17:18:15,698][75950] Updated weights for policy 1, policy_version 92160 (0.0007) -[2023-10-14 17:18:18,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189005824. Throughput: 0: 1666.0, 1: 1654.2. Samples: 47256516. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:18:18,165][74987] Avg episode reward: [(0, '27.160'), (1, '34.620')] -[2023-10-14 17:18:19,029][75949] Updated weights for policy 0, policy_version 92421 (0.0010) -[2023-10-14 17:18:19,406][75949] Updated weights for policy 0, policy_version 92431 (0.0009) -[2023-10-14 17:18:19,771][75949] Updated weights for policy 0, policy_version 92441 (0.0009) -[2023-10-14 17:18:19,888][75950] Updated weights for policy 1, policy_version 92170 (0.0009) -[2023-10-14 17:18:20,245][75950] Updated weights for policy 1, policy_version 92180 (0.0009) -[2023-10-14 17:18:20,620][75950] Updated weights for policy 1, policy_version 92190 (0.0008) -[2023-10-14 17:18:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189071360. Throughput: 0: 1679.0, 1: 1667.8. Samples: 47276702. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:18:23,165][74987] Avg episode reward: [(0, '27.710'), (1, '34.970')] -[2023-10-14 17:18:23,789][75949] Updated weights for policy 0, policy_version 92451 (0.0009) -[2023-10-14 17:18:24,156][75949] Updated weights for policy 0, policy_version 92461 (0.0010) -[2023-10-14 17:18:24,521][75949] Updated weights for policy 0, policy_version 92471 (0.0008) -[2023-10-14 17:18:24,706][75950] Updated weights for policy 1, policy_version 92200 (0.0009) -[2023-10-14 17:18:25,068][75950] Updated weights for policy 1, policy_version 92210 (0.0008) -[2023-10-14 17:18:25,432][75950] Updated weights for policy 1, policy_version 92220 (0.0010) -[2023-10-14 17:18:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189136896. Throughput: 0: 1677.3, 1: 1668.8. Samples: 47297306. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) -[2023-10-14 17:18:28,164][74987] Avg episode reward: [(0, '27.670'), (1, '35.130')] -[2023-10-14 17:18:28,669][75949] Updated weights for policy 0, policy_version 92481 (0.0008) -[2023-10-14 17:18:29,023][75949] Updated weights for policy 0, policy_version 92491 (0.0008) -[2023-10-14 17:18:29,396][75949] Updated weights for policy 0, policy_version 92501 (0.0009) -[2023-10-14 17:18:29,634][75950] Updated weights for policy 1, policy_version 92230 (0.0009) -[2023-10-14 17:18:29,765][75949] Updated weights for policy 0, policy_version 92511 (0.0008) -[2023-10-14 17:18:29,997][75950] Updated weights for policy 1, policy_version 92240 (0.0009) -[2023-10-14 17:18:30,356][75950] Updated weights for policy 1, policy_version 92250 (0.0009) -[2023-10-14 17:18:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189202432. Throughput: 0: 1678.1, 1: 1652.5. Samples: 47306486. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:18:33,164][74987] Avg episode reward: [(0, '29.490'), (1, '35.170')] -[2023-10-14 17:18:33,850][75949] Updated weights for policy 0, policy_version 92521 (0.0011) -[2023-10-14 17:18:34,218][75949] Updated weights for policy 0, policy_version 92531 (0.0010) -[2023-10-14 17:18:34,338][75950] Updated weights for policy 1, policy_version 92260 (0.0008) -[2023-10-14 17:18:34,588][75949] Updated weights for policy 0, policy_version 92541 (0.0008) -[2023-10-14 17:18:34,722][75950] Updated weights for policy 1, policy_version 92270 (0.0007) -[2023-10-14 17:18:35,093][75950] Updated weights for policy 1, policy_version 92280 (0.0008) -[2023-10-14 17:18:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 189267968. Throughput: 0: 1676.2, 1: 1675.1. Samples: 47326764. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:18:38,165][74987] Avg episode reward: [(0, '28.190'), (1, '36.280')] -[2023-10-14 17:18:38,785][75949] Updated weights for policy 0, policy_version 92551 (0.0008) -[2023-10-14 17:18:39,147][75949] Updated weights for policy 0, policy_version 92561 (0.0008) -[2023-10-14 17:18:39,267][75950] Updated weights for policy 1, policy_version 92290 (0.0008) -[2023-10-14 17:18:39,515][75949] Updated weights for policy 0, policy_version 92571 (0.0009) -[2023-10-14 17:18:39,635][75950] Updated weights for policy 1, policy_version 92300 (0.0008) -[2023-10-14 17:18:40,003][75950] Updated weights for policy 1, policy_version 92310 (0.0009) -[2023-10-14 17:18:40,378][75950] Updated weights for policy 1, policy_version 92320 (0.0007) -[2023-10-14 17:18:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189333504. Throughput: 0: 1678.2, 1: 1673.5. Samples: 47347482. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:18:43,164][74987] Avg episode reward: [(0, '28.830'), (1, '35.470')] -[2023-10-14 17:18:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000092320_94535680.pth... -[2023-10-14 17:18:43,174][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000092576_94797824.pth... -[2023-10-14 17:18:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000090784_92962816.pth -[2023-10-14 17:18:43,219][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000091008_93192192.pth -[2023-10-14 17:18:43,609][75949] Updated weights for policy 0, policy_version 92581 (0.0008) -[2023-10-14 17:18:44,005][75949] Updated weights for policy 0, policy_version 92591 (0.0007) -[2023-10-14 17:18:44,380][75949] Updated weights for policy 0, policy_version 92601 (0.0008) -[2023-10-14 17:18:44,455][75950] Updated weights for policy 1, policy_version 92330 (0.0007) -[2023-10-14 17:18:44,809][75950] Updated weights for policy 1, policy_version 92340 (0.0009) -[2023-10-14 17:18:45,176][75950] Updated weights for policy 1, policy_version 92350 (0.0008) -[2023-10-14 17:18:48,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189399040. Throughput: 0: 1678.8, 1: 1656.9. Samples: 47356450. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:18:48,164][74987] Avg episode reward: [(0, '27.750'), (1, '33.850')] -[2023-10-14 17:18:48,468][75949] Updated weights for policy 0, policy_version 92611 (0.0008) -[2023-10-14 17:18:48,839][75949] Updated weights for policy 0, policy_version 92621 (0.0008) -[2023-10-14 17:18:49,212][75949] Updated weights for policy 0, policy_version 92631 (0.0008) -[2023-10-14 17:18:49,310][75950] Updated weights for policy 1, policy_version 92360 (0.0009) -[2023-10-14 17:18:49,671][75950] Updated weights for policy 1, policy_version 92370 (0.0009) -[2023-10-14 17:18:50,038][75950] Updated weights for policy 1, policy_version 92380 (0.0008) -[2023-10-14 17:18:53,141][75949] Updated weights for policy 0, policy_version 92641 (0.0008) -[2023-10-14 17:18:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189464576. Throughput: 0: 1682.4, 1: 1669.0. Samples: 47377038. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:18:53,164][74987] Avg episode reward: [(0, '28.780'), (1, '34.750')] -[2023-10-14 17:18:53,516][75949] Updated weights for policy 0, policy_version 92651 (0.0008) -[2023-10-14 17:18:53,892][75949] Updated weights for policy 0, policy_version 92661 (0.0009) -[2023-10-14 17:18:53,990][75950] Updated weights for policy 1, policy_version 92390 (0.0010) -[2023-10-14 17:18:54,255][75949] Updated weights for policy 0, policy_version 92671 (0.0009) -[2023-10-14 17:18:54,350][75950] Updated weights for policy 1, policy_version 92400 (0.0009) -[2023-10-14 17:18:54,717][75950] Updated weights for policy 1, policy_version 92410 (0.0007) -[2023-10-14 17:18:58,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 189530112. Throughput: 0: 1685.4, 1: 1668.3. Samples: 47398008. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:18:58,164][74987] Avg episode reward: [(0, '28.600'), (1, '36.060')] -[2023-10-14 17:18:58,248][75949] Updated weights for policy 0, policy_version 92681 (0.0009) -[2023-10-14 17:18:58,630][75949] Updated weights for policy 0, policy_version 92691 (0.0008) -[2023-10-14 17:18:58,894][75950] Updated weights for policy 1, policy_version 92420 (0.0008) -[2023-10-14 17:18:59,004][75949] Updated weights for policy 0, policy_version 92701 (0.0010) -[2023-10-14 17:18:59,266][75950] Updated weights for policy 1, policy_version 92430 (0.0007) -[2023-10-14 17:18:59,630][75950] Updated weights for policy 1, policy_version 92440 (0.0008) -[2023-10-14 17:19:03,111][75949] Updated weights for policy 0, policy_version 92711 (0.0009) -[2023-10-14 17:19:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189595648. Throughput: 0: 1683.6, 1: 1661.7. Samples: 47407052. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:19:03,164][74987] Avg episode reward: [(0, '28.050'), (1, '33.240')] -[2023-10-14 17:19:03,481][75949] Updated weights for policy 0, policy_version 92721 (0.0008) -[2023-10-14 17:19:03,785][75950] Updated weights for policy 1, policy_version 92450 (0.0009) -[2023-10-14 17:19:03,846][75949] Updated weights for policy 0, policy_version 92731 (0.0008) -[2023-10-14 17:19:04,157][75950] Updated weights for policy 1, policy_version 92460 (0.0009) -[2023-10-14 17:19:04,528][75950] Updated weights for policy 1, policy_version 92470 (0.0007) -[2023-10-14 17:19:04,899][75950] Updated weights for policy 1, policy_version 92480 (0.0009) -[2023-10-14 17:19:07,965][75949] Updated weights for policy 0, policy_version 92741 (0.0008) -[2023-10-14 17:19:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 189661184. Throughput: 0: 1682.8, 1: 1675.2. Samples: 47427810. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:19:08,164][74987] Avg episode reward: [(0, '30.100'), (1, '35.630')] -[2023-10-14 17:19:08,341][75949] Updated weights for policy 0, policy_version 92751 (0.0009) -[2023-10-14 17:19:08,706][75949] Updated weights for policy 0, policy_version 92761 (0.0008) -[2023-10-14 17:19:08,934][75950] Updated weights for policy 1, policy_version 92490 (0.0010) -[2023-10-14 17:19:09,298][75950] Updated weights for policy 1, policy_version 92500 (0.0008) -[2023-10-14 17:19:09,657][75950] Updated weights for policy 1, policy_version 92510 (0.0007) -[2023-10-14 17:19:12,860][75949] Updated weights for policy 0, policy_version 92771 (0.0008) -[2023-10-14 17:19:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189726720. Throughput: 0: 1678.6, 1: 1682.3. Samples: 47448544. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:19:13,164][74987] Avg episode reward: [(0, '27.380'), (1, '36.970')] -[2023-10-14 17:19:13,230][75949] Updated weights for policy 0, policy_version 92781 (0.0008) -[2023-10-14 17:19:13,605][75949] Updated weights for policy 0, policy_version 92791 (0.0007) -[2023-10-14 17:19:13,652][75950] Updated weights for policy 1, policy_version 92520 (0.0010) -[2023-10-14 17:19:14,009][75950] Updated weights for policy 1, policy_version 92530 (0.0009) -[2023-10-14 17:19:14,393][75950] Updated weights for policy 1, policy_version 92540 (0.0010) -[2023-10-14 17:19:17,549][75949] Updated weights for policy 0, policy_version 92801 (0.0008) -[2023-10-14 17:19:17,916][75949] Updated weights for policy 0, policy_version 92811 (0.0007) -[2023-10-14 17:19:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189792256. Throughput: 0: 1677.1, 1: 1680.6. Samples: 47457582. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:19:18,165][74987] Avg episode reward: [(0, '30.740'), (1, '33.650')] -[2023-10-14 17:19:18,295][75949] Updated weights for policy 0, policy_version 92821 (0.0008) -[2023-10-14 17:19:18,644][75950] Updated weights for policy 1, policy_version 92550 (0.0007) -[2023-10-14 17:19:18,655][75949] Updated weights for policy 0, policy_version 92831 (0.0008) -[2023-10-14 17:19:19,011][75950] Updated weights for policy 1, policy_version 92560 (0.0008) -[2023-10-14 17:19:19,387][75950] Updated weights for policy 1, policy_version 92570 (0.0010) -[2023-10-14 17:19:22,941][75949] Updated weights for policy 0, policy_version 92841 (0.0010) -[2023-10-14 17:19:23,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189857792. Throughput: 0: 1685.9, 1: 1675.2. Samples: 47478010. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:19:23,164][74987] Avg episode reward: [(0, '27.000'), (1, '34.790')] -[2023-10-14 17:19:23,311][75949] Updated weights for policy 0, policy_version 92851 (0.0010) -[2023-10-14 17:19:23,595][75950] Updated weights for policy 1, policy_version 92580 (0.0008) -[2023-10-14 17:19:23,676][75949] Updated weights for policy 0, policy_version 92861 (0.0007) -[2023-10-14 17:19:24,008][75950] Updated weights for policy 1, policy_version 92590 (0.0010) -[2023-10-14 17:19:24,385][75950] Updated weights for policy 1, policy_version 92600 (0.0010) -[2023-10-14 17:19:27,786][75949] Updated weights for policy 0, policy_version 92871 (0.0009) -[2023-10-14 17:19:28,149][75949] Updated weights for policy 0, policy_version 92881 (0.0012) -[2023-10-14 17:19:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189923328. Throughput: 0: 1673.7, 1: 1669.7. Samples: 47497936. Policy #0 lag: (min: 26.0, avg: 52.7, max: 56.0) -[2023-10-14 17:19:28,165][74987] Avg episode reward: [(0, '29.760'), (1, '33.740')] -[2023-10-14 17:19:28,511][75950] Updated weights for policy 1, policy_version 92610 (0.0010) -[2023-10-14 17:19:28,530][75949] Updated weights for policy 0, policy_version 92891 (0.0009) -[2023-10-14 17:19:28,872][75950] Updated weights for policy 1, policy_version 92620 (0.0007) -[2023-10-14 17:19:29,239][75950] Updated weights for policy 1, policy_version 92630 (0.0009) -[2023-10-14 17:19:29,602][75950] Updated weights for policy 1, policy_version 92640 (0.0009) -[2023-10-14 17:19:32,556][75949] Updated weights for policy 0, policy_version 92901 (0.0009) -[2023-10-14 17:19:32,940][75949] Updated weights for policy 0, policy_version 92911 (0.0009) -[2023-10-14 17:19:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189988864. Throughput: 0: 1680.9, 1: 1671.1. Samples: 47507290. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 17:19:33,164][74987] Avg episode reward: [(0, '25.750'), (1, '33.690')] -[2023-10-14 17:19:33,300][75949] Updated weights for policy 0, policy_version 92921 (0.0009) -[2023-10-14 17:19:33,609][75950] Updated weights for policy 1, policy_version 92650 (0.0009) -[2023-10-14 17:19:33,982][75950] Updated weights for policy 1, policy_version 92660 (0.0007) -[2023-10-14 17:19:34,338][75950] Updated weights for policy 1, policy_version 92670 (0.0008) -[2023-10-14 17:19:37,367][75949] Updated weights for policy 0, policy_version 92931 (0.0010) -[2023-10-14 17:19:37,730][75949] Updated weights for policy 0, policy_version 92941 (0.0011) -[2023-10-14 17:19:38,101][75949] Updated weights for policy 0, policy_version 92951 (0.0008) -[2023-10-14 17:19:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190054400. Throughput: 0: 1681.8, 1: 1673.8. Samples: 47528038. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 17:19:38,164][74987] Avg episode reward: [(0, '29.580'), (1, '33.300')] -[2023-10-14 17:19:38,588][75950] Updated weights for policy 1, policy_version 92680 (0.0010) -[2023-10-14 17:19:38,948][75950] Updated weights for policy 1, policy_version 92690 (0.0010) -[2023-10-14 17:19:39,311][75950] Updated weights for policy 1, policy_version 92700 (0.0010) -[2023-10-14 17:19:41,891][75949] Updated weights for policy 0, policy_version 92961 (0.0010) -[2023-10-14 17:19:42,264][75949] Updated weights for policy 0, policy_version 92971 (0.0008) -[2023-10-14 17:19:42,631][75949] Updated weights for policy 0, policy_version 92981 (0.0007) -[2023-10-14 17:19:43,006][75949] Updated weights for policy 0, policy_version 92991 (0.0007) -[2023-10-14 17:19:43,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190152704. Throughput: 0: 1658.4, 1: 1674.9. Samples: 47548008. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 17:19:43,164][74987] Avg episode reward: [(0, '25.290'), (1, '33.050')] -[2023-10-14 17:19:43,446][75950] Updated weights for policy 1, policy_version 92710 (0.0009) -[2023-10-14 17:19:43,807][75950] Updated weights for policy 1, policy_version 92720 (0.0007) -[2023-10-14 17:19:44,169][75950] Updated weights for policy 1, policy_version 92730 (0.0008) -[2023-10-14 17:19:47,251][75949] Updated weights for policy 0, policy_version 93001 (0.0011) -[2023-10-14 17:19:47,626][75949] Updated weights for policy 0, policy_version 93011 (0.0010) -[2023-10-14 17:19:47,999][75949] Updated weights for policy 0, policy_version 93021 (0.0009) -[2023-10-14 17:19:48,114][75950] Updated weights for policy 1, policy_version 92740 (0.0008) -[2023-10-14 17:19:48,164][74987] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190218240. Throughput: 0: 1679.0, 1: 1673.3. Samples: 47557904. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 17:19:48,165][74987] Avg episode reward: [(0, '29.540'), (1, '34.770')] -[2023-10-14 17:19:48,484][75950] Updated weights for policy 1, policy_version 92750 (0.0011) -[2023-10-14 17:19:48,846][75950] Updated weights for policy 1, policy_version 92760 (0.0007) -[2023-10-14 17:19:51,965][75949] Updated weights for policy 0, policy_version 93031 (0.0008) -[2023-10-14 17:19:52,337][75949] Updated weights for policy 0, policy_version 93041 (0.0009) -[2023-10-14 17:19:52,716][75949] Updated weights for policy 0, policy_version 93051 (0.0008) -[2023-10-14 17:19:52,867][75950] Updated weights for policy 1, policy_version 92770 (0.0009) -[2023-10-14 17:19:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190283776. Throughput: 0: 1677.8, 1: 1669.9. Samples: 47578454. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 17:19:53,164][74987] Avg episode reward: [(0, '26.350'), (1, '36.480')] -[2023-10-14 17:19:53,229][75950] Updated weights for policy 1, policy_version 92780 (0.0009) -[2023-10-14 17:19:53,598][75950] Updated weights for policy 1, policy_version 92790 (0.0008) -[2023-10-14 17:19:53,962][75950] Updated weights for policy 1, policy_version 92800 (0.0010) -[2023-10-14 17:19:56,688][75949] Updated weights for policy 0, policy_version 93061 (0.0009) -[2023-10-14 17:19:57,061][75949] Updated weights for policy 0, policy_version 93071 (0.0009) -[2023-10-14 17:19:57,429][75949] Updated weights for policy 0, policy_version 93081 (0.0009) -[2023-10-14 17:19:57,915][75950] Updated weights for policy 1, policy_version 92810 (0.0008) -[2023-10-14 17:19:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190349312. Throughput: 0: 1656.6, 1: 1666.7. Samples: 47598092. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 17:19:58,165][74987] Avg episode reward: [(0, '28.620'), (1, '34.880')] -[2023-10-14 17:19:58,274][75950] Updated weights for policy 1, policy_version 92820 (0.0010) -[2023-10-14 17:19:58,647][75950] Updated weights for policy 1, policy_version 92830 (0.0008) -[2023-10-14 17:20:01,600][75949] Updated weights for policy 0, policy_version 93091 (0.0009) -[2023-10-14 17:20:01,976][75949] Updated weights for policy 0, policy_version 93101 (0.0009) -[2023-10-14 17:20:02,340][75949] Updated weights for policy 0, policy_version 93111 (0.0008) -[2023-10-14 17:20:02,759][75950] Updated weights for policy 1, policy_version 92840 (0.0007) -[2023-10-14 17:20:03,122][75950] Updated weights for policy 1, policy_version 92850 (0.0008) -[2023-10-14 17:20:03,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190414848. Throughput: 0: 1682.4, 1: 1673.7. Samples: 47608606. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 17:20:03,165][74987] Avg episode reward: [(0, '26.300'), (1, '35.780')] -[2023-10-14 17:20:03,488][75950] Updated weights for policy 1, policy_version 92860 (0.0007) -[2023-10-14 17:20:06,615][75949] Updated weights for policy 0, policy_version 93121 (0.0009) -[2023-10-14 17:20:06,997][75949] Updated weights for policy 0, policy_version 93131 (0.0008) -[2023-10-14 17:20:07,359][75949] Updated weights for policy 0, policy_version 93141 (0.0009) -[2023-10-14 17:20:07,635][75950] Updated weights for policy 1, policy_version 92870 (0.0009) -[2023-10-14 17:20:07,726][75949] Updated weights for policy 0, policy_version 93151 (0.0008) -[2023-10-14 17:20:08,004][75950] Updated weights for policy 1, policy_version 92880 (0.0007) -[2023-10-14 17:20:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190480384. Throughput: 0: 1675.4, 1: 1681.6. Samples: 47629078. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 17:20:08,164][74987] Avg episode reward: [(0, '28.460'), (1, '39.930')] -[2023-10-14 17:20:08,370][75950] Updated weights for policy 1, policy_version 92890 (0.0008) -[2023-10-14 17:20:08,582][75801] Saving new best policy, reward=39.930! -[2023-10-14 17:20:11,682][75949] Updated weights for policy 0, policy_version 93161 (0.0008) -[2023-10-14 17:20:12,051][75949] Updated weights for policy 0, policy_version 93171 (0.0008) -[2023-10-14 17:20:12,429][75949] Updated weights for policy 0, policy_version 93181 (0.0009) -[2023-10-14 17:20:12,657][75950] Updated weights for policy 1, policy_version 92900 (0.0009) -[2023-10-14 17:20:13,069][75950] Updated weights for policy 1, policy_version 92910 (0.0008) -[2023-10-14 17:20:13,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190545920. Throughput: 0: 1661.1, 1: 1684.2. Samples: 47648472. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 17:20:13,165][74987] Avg episode reward: [(0, '27.040'), (1, '36.250')] -[2023-10-14 17:20:13,436][75950] Updated weights for policy 1, policy_version 92920 (0.0010) -[2023-10-14 17:20:16,500][75949] Updated weights for policy 0, policy_version 93191 (0.0010) -[2023-10-14 17:20:16,867][75949] Updated weights for policy 0, policy_version 93201 (0.0012) -[2023-10-14 17:20:17,232][75949] Updated weights for policy 0, policy_version 93211 (0.0008) -[2023-10-14 17:20:17,437][75950] Updated weights for policy 1, policy_version 92930 (0.0009) -[2023-10-14 17:20:17,801][75950] Updated weights for policy 1, policy_version 92940 (0.0010) -[2023-10-14 17:20:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 190611456. Throughput: 0: 1685.6, 1: 1684.2. Samples: 47658934. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 17:20:18,164][74987] Avg episode reward: [(0, '27.340'), (1, '34.590')] -[2023-10-14 17:20:18,168][75950] Updated weights for policy 1, policy_version 92950 (0.0010) -[2023-10-14 17:20:18,543][75950] Updated weights for policy 1, policy_version 92960 (0.0010) -[2023-10-14 17:20:21,321][75949] Updated weights for policy 0, policy_version 93221 (0.0008) -[2023-10-14 17:20:21,708][75949] Updated weights for policy 0, policy_version 93231 (0.0008) -[2023-10-14 17:20:22,081][75949] Updated weights for policy 0, policy_version 93241 (0.0010) -[2023-10-14 17:20:22,637][75950] Updated weights for policy 1, policy_version 92970 (0.0010) -[2023-10-14 17:20:23,002][75950] Updated weights for policy 1, policy_version 92980 (0.0007) -[2023-10-14 17:20:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190676992. Throughput: 0: 1669.1, 1: 1682.9. Samples: 47678878. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) -[2023-10-14 17:20:23,164][74987] Avg episode reward: [(0, '27.420'), (1, '36.170')] -[2023-10-14 17:20:23,367][75950] Updated weights for policy 1, policy_version 92990 (0.0010) -[2023-10-14 17:20:26,166][75949] Updated weights for policy 0, policy_version 93251 (0.0009) -[2023-10-14 17:20:26,537][75949] Updated weights for policy 0, policy_version 93261 (0.0009) -[2023-10-14 17:20:26,919][75949] Updated weights for policy 0, policy_version 93271 (0.0011) -[2023-10-14 17:20:27,278][75950] Updated weights for policy 1, policy_version 93000 (0.0008) -[2023-10-14 17:20:27,641][75950] Updated weights for policy 1, policy_version 93010 (0.0009) -[2023-10-14 17:20:28,016][75950] Updated weights for policy 1, policy_version 93020 (0.0009) -[2023-10-14 17:20:28,164][74987] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 190775296. Throughput: 0: 1667.1, 1: 1671.9. Samples: 47698264. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:20:28,164][74987] Avg episode reward: [(0, '26.820'), (1, '35.150')] -[2023-10-14 17:20:31,004][75949] Updated weights for policy 0, policy_version 93281 (0.0007) -[2023-10-14 17:20:31,369][75949] Updated weights for policy 0, policy_version 93291 (0.0011) -[2023-10-14 17:20:31,742][75949] Updated weights for policy 0, policy_version 93301 (0.0008) -[2023-10-14 17:20:32,114][75949] Updated weights for policy 0, policy_version 93311 (0.0009) -[2023-10-14 17:20:32,145][75950] Updated weights for policy 1, policy_version 93030 (0.0009) -[2023-10-14 17:20:32,510][75950] Updated weights for policy 1, policy_version 93040 (0.0009) -[2023-10-14 17:20:32,875][75950] Updated weights for policy 1, policy_version 93050 (0.0007) -[2023-10-14 17:20:33,164][74987] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 190840832. Throughput: 0: 1674.5, 1: 1686.6. Samples: 47709152. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:20:33,165][74987] Avg episode reward: [(0, '28.750'), (1, '33.990')] -[2023-10-14 17:20:36,099][75949] Updated weights for policy 0, policy_version 93321 (0.0009) -[2023-10-14 17:20:36,468][75949] Updated weights for policy 0, policy_version 93331 (0.0011) -[2023-10-14 17:20:36,832][75949] Updated weights for policy 0, policy_version 93341 (0.0009) -[2023-10-14 17:20:36,925][75950] Updated weights for policy 1, policy_version 93060 (0.0007) -[2023-10-14 17:20:37,287][75950] Updated weights for policy 1, policy_version 93070 (0.0009) -[2023-10-14 17:20:37,652][75950] Updated weights for policy 1, policy_version 93080 (0.0009) -[2023-10-14 17:20:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 190906368. Throughput: 0: 1660.1, 1: 1684.3. Samples: 47728954. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:20:38,165][74987] Avg episode reward: [(0, '26.310'), (1, '35.740')] -[2023-10-14 17:20:40,898][75949] Updated weights for policy 0, policy_version 93351 (0.0008) -[2023-10-14 17:20:41,266][75949] Updated weights for policy 0, policy_version 93361 (0.0008) -[2023-10-14 17:20:41,640][75949] Updated weights for policy 0, policy_version 93371 (0.0009) -[2023-10-14 17:20:41,928][75950] Updated weights for policy 1, policy_version 93090 (0.0008) -[2023-10-14 17:20:42,302][75950] Updated weights for policy 1, policy_version 93100 (0.0007) -[2023-10-14 17:20:42,681][75950] Updated weights for policy 1, policy_version 93110 (0.0008) -[2023-10-14 17:20:43,052][75950] Updated weights for policy 1, policy_version 93120 (0.0010) -[2023-10-14 17:20:43,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 190971904. Throughput: 0: 1683.2, 1: 1660.5. Samples: 47748560. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:20:43,164][74987] Avg episode reward: [(0, '28.150'), (1, '36.670')] -[2023-10-14 17:20:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000093120_95354880.pth... -[2023-10-14 17:20:43,174][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000093376_95617024.pth... -[2023-10-14 17:20:43,203][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000091808_94011392.pth -[2023-10-14 17:20:43,214][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000091552_93749248.pth -[2023-10-14 17:20:45,552][75949] Updated weights for policy 0, policy_version 93381 (0.0008) -[2023-10-14 17:20:45,921][75949] Updated weights for policy 0, policy_version 93391 (0.0008) -[2023-10-14 17:20:46,294][75949] Updated weights for policy 0, policy_version 93401 (0.0010) -[2023-10-14 17:20:47,191][75950] Updated weights for policy 1, policy_version 93130 (0.0007) -[2023-10-14 17:20:47,565][75950] Updated weights for policy 1, policy_version 93140 (0.0009) -[2023-10-14 17:20:47,936][75950] Updated weights for policy 1, policy_version 93150 (0.0010) -[2023-10-14 17:20:48,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 191037440. Throughput: 0: 1680.2, 1: 1669.7. Samples: 47759350. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:20:48,165][74987] Avg episode reward: [(0, '26.700'), (1, '35.020')] -[2023-10-14 17:20:50,364][75949] Updated weights for policy 0, policy_version 93411 (0.0009) -[2023-10-14 17:20:50,738][75949] Updated weights for policy 0, policy_version 93421 (0.0008) -[2023-10-14 17:20:51,106][75949] Updated weights for policy 0, policy_version 93431 (0.0010) -[2023-10-14 17:20:51,956][75950] Updated weights for policy 1, policy_version 93160 (0.0011) -[2023-10-14 17:20:52,324][75950] Updated weights for policy 1, policy_version 93170 (0.0008) -[2023-10-14 17:20:52,695][75950] Updated weights for policy 1, policy_version 93180 (0.0010) -[2023-10-14 17:20:53,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 191102976. Throughput: 0: 1662.2, 1: 1666.4. Samples: 47778868. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:20:53,165][74987] Avg episode reward: [(0, '28.460'), (1, '35.710')] -[2023-10-14 17:20:55,216][75949] Updated weights for policy 0, policy_version 93441 (0.0008) -[2023-10-14 17:20:55,594][75949] Updated weights for policy 0, policy_version 93451 (0.0008) -[2023-10-14 17:20:55,955][75949] Updated weights for policy 0, policy_version 93461 (0.0009) -[2023-10-14 17:20:56,319][75949] Updated weights for policy 0, policy_version 93471 (0.0008) -[2023-10-14 17:20:56,863][75950] Updated weights for policy 1, policy_version 93190 (0.0010) -[2023-10-14 17:20:57,226][75950] Updated weights for policy 1, policy_version 93200 (0.0009) -[2023-10-14 17:20:57,588][75950] Updated weights for policy 1, policy_version 93210 (0.0008) -[2023-10-14 17:20:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 191168512. Throughput: 0: 1687.0, 1: 1648.6. Samples: 47798574. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:20:58,164][74987] Avg episode reward: [(0, '24.960'), (1, '36.090')] -[2023-10-14 17:21:00,275][75949] Updated weights for policy 0, policy_version 93481 (0.0009) -[2023-10-14 17:21:00,645][75949] Updated weights for policy 0, policy_version 93491 (0.0009) -[2023-10-14 17:21:01,014][75949] Updated weights for policy 0, policy_version 93501 (0.0007) -[2023-10-14 17:21:01,875][75950] Updated weights for policy 1, policy_version 93220 (0.0010) -[2023-10-14 17:21:02,272][75950] Updated weights for policy 1, policy_version 93230 (0.0009) -[2023-10-14 17:21:02,646][75950] Updated weights for policy 1, policy_version 93240 (0.0009) -[2023-10-14 17:21:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 191234048. Throughput: 0: 1670.3, 1: 1669.8. Samples: 47809238. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:21:03,164][74987] Avg episode reward: [(0, '28.420'), (1, '36.890')] -[2023-10-14 17:21:04,831][75949] Updated weights for policy 0, policy_version 93511 (0.0009) -[2023-10-14 17:21:05,211][75949] Updated weights for policy 0, policy_version 93521 (0.0007) -[2023-10-14 17:21:05,570][75949] Updated weights for policy 0, policy_version 93531 (0.0010) -[2023-10-14 17:21:06,548][75950] Updated weights for policy 1, policy_version 93250 (0.0009) -[2023-10-14 17:21:06,910][75950] Updated weights for policy 1, policy_version 93260 (0.0007) -[2023-10-14 17:21:07,281][75950] Updated weights for policy 1, policy_version 93270 (0.0007) -[2023-10-14 17:21:07,642][75950] Updated weights for policy 1, policy_version 93280 (0.0010) -[2023-10-14 17:21:08,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 191299584. Throughput: 0: 1683.0, 1: 1667.0. Samples: 47829630. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:21:08,164][74987] Avg episode reward: [(0, '26.380'), (1, '34.690')] -[2023-10-14 17:21:09,628][75949] Updated weights for policy 0, policy_version 93541 (0.0008) -[2023-10-14 17:21:10,000][75949] Updated weights for policy 0, policy_version 93551 (0.0007) -[2023-10-14 17:21:10,377][75949] Updated weights for policy 0, policy_version 93561 (0.0007) -[2023-10-14 17:21:11,584][75950] Updated weights for policy 1, policy_version 93290 (0.0008) -[2023-10-14 17:21:11,940][75950] Updated weights for policy 1, policy_version 93300 (0.0007) -[2023-10-14 17:21:12,303][75950] Updated weights for policy 1, policy_version 93310 (0.0007) -[2023-10-14 17:21:13,164][74987] Fps is (10 sec: 13106.5, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 191365120. Throughput: 0: 1704.0, 1: 1654.9. Samples: 47849416. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:21:13,165][74987] Avg episode reward: [(0, '31.470'), (1, '33.450')] -[2023-10-14 17:21:14,503][75949] Updated weights for policy 0, policy_version 93571 (0.0009) -[2023-10-14 17:21:14,880][75949] Updated weights for policy 0, policy_version 93581 (0.0009) -[2023-10-14 17:21:15,260][75949] Updated weights for policy 0, policy_version 93591 (0.0008) -[2023-10-14 17:21:16,198][75950] Updated weights for policy 1, policy_version 93320 (0.0010) -[2023-10-14 17:21:16,561][75950] Updated weights for policy 1, policy_version 93330 (0.0010) -[2023-10-14 17:21:16,927][75950] Updated weights for policy 1, policy_version 93340 (0.0011) -[2023-10-14 17:21:18,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 191430656. Throughput: 0: 1674.6, 1: 1674.0. Samples: 47859842. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:21:18,165][74987] Avg episode reward: [(0, '26.470'), (1, '35.180')] -[2023-10-14 17:21:19,203][75949] Updated weights for policy 0, policy_version 93601 (0.0007) -[2023-10-14 17:21:19,574][75949] Updated weights for policy 0, policy_version 93611 (0.0007) -[2023-10-14 17:21:19,951][75949] Updated weights for policy 0, policy_version 93621 (0.0009) -[2023-10-14 17:21:20,329][75949] Updated weights for policy 0, policy_version 93631 (0.0010) -[2023-10-14 17:21:20,889][75950] Updated weights for policy 1, policy_version 93350 (0.0009) -[2023-10-14 17:21:21,256][75950] Updated weights for policy 1, policy_version 93360 (0.0008) -[2023-10-14 17:21:21,618][75950] Updated weights for policy 1, policy_version 93370 (0.0009) -[2023-10-14 17:21:23,163][74987] Fps is (10 sec: 13108.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 191496192. Throughput: 0: 1698.2, 1: 1653.7. Samples: 47879786. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) -[2023-10-14 17:21:23,164][74987] Avg episode reward: [(0, '29.080'), (1, '34.460')] -[2023-10-14 17:21:24,340][75949] Updated weights for policy 0, policy_version 93641 (0.0010) -[2023-10-14 17:21:24,701][75949] Updated weights for policy 0, policy_version 93651 (0.0008) -[2023-10-14 17:21:25,075][75949] Updated weights for policy 0, policy_version 93661 (0.0007) -[2023-10-14 17:21:25,826][75950] Updated weights for policy 1, policy_version 93380 (0.0007) -[2023-10-14 17:21:26,187][75950] Updated weights for policy 1, policy_version 93390 (0.0007) -[2023-10-14 17:21:26,553][75950] Updated weights for policy 1, policy_version 93400 (0.0009) -[2023-10-14 17:21:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 191561728. Throughput: 0: 1703.0, 1: 1666.8. Samples: 47900200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:21:28,165][74987] Avg episode reward: [(0, '25.080'), (1, '34.840')] -[2023-10-14 17:21:29,244][75949] Updated weights for policy 0, policy_version 93671 (0.0008) -[2023-10-14 17:21:29,607][75949] Updated weights for policy 0, policy_version 93681 (0.0009) -[2023-10-14 17:21:29,983][75949] Updated weights for policy 0, policy_version 93691 (0.0010) -[2023-10-14 17:21:30,781][75950] Updated weights for policy 1, policy_version 93410 (0.0008) -[2023-10-14 17:21:31,148][75950] Updated weights for policy 1, policy_version 93420 (0.0008) -[2023-10-14 17:21:31,511][75950] Updated weights for policy 1, policy_version 93430 (0.0009) -[2023-10-14 17:21:31,884][75950] Updated weights for policy 1, policy_version 93440 (0.0010) -[2023-10-14 17:21:33,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 191627264. Throughput: 0: 1680.0, 1: 1682.3. Samples: 47910654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:21:33,164][74987] Avg episode reward: [(0, '29.660'), (1, '37.570')] -[2023-10-14 17:21:33,930][75949] Updated weights for policy 0, policy_version 93701 (0.0009) -[2023-10-14 17:21:34,302][75949] Updated weights for policy 0, policy_version 93711 (0.0008) -[2023-10-14 17:21:34,672][75949] Updated weights for policy 0, policy_version 93721 (0.0007) -[2023-10-14 17:21:36,013][75950] Updated weights for policy 1, policy_version 93450 (0.0008) -[2023-10-14 17:21:36,383][75950] Updated weights for policy 1, policy_version 93460 (0.0010) -[2023-10-14 17:21:36,758][75950] Updated weights for policy 1, policy_version 93470 (0.0010) -[2023-10-14 17:21:38,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 191692800. Throughput: 0: 1707.3, 1: 1661.1. Samples: 47930444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:21:38,164][74987] Avg episode reward: [(0, '27.020'), (1, '36.590')] -[2023-10-14 17:21:38,863][75949] Updated weights for policy 0, policy_version 93731 (0.0010) -[2023-10-14 17:21:39,226][75949] Updated weights for policy 0, policy_version 93741 (0.0011) -[2023-10-14 17:21:39,602][75949] Updated weights for policy 0, policy_version 93751 (0.0007) -[2023-10-14 17:21:40,978][75950] Updated weights for policy 1, policy_version 93480 (0.0009) -[2023-10-14 17:21:41,347][75950] Updated weights for policy 1, policy_version 93490 (0.0009) -[2023-10-14 17:21:41,711][75950] Updated weights for policy 1, policy_version 93500 (0.0007) -[2023-10-14 17:21:43,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 191758336. Throughput: 0: 1701.2, 1: 1677.2. Samples: 47950600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:21:43,164][74987] Avg episode reward: [(0, '31.840'), (1, '36.000')] -[2023-10-14 17:21:43,721][75949] Updated weights for policy 0, policy_version 93761 (0.0007) -[2023-10-14 17:21:44,085][75949] Updated weights for policy 0, policy_version 93771 (0.0007) -[2023-10-14 17:21:44,456][75949] Updated weights for policy 0, policy_version 93781 (0.0010) -[2023-10-14 17:21:44,829][75949] Updated weights for policy 0, policy_version 93791 (0.0008) -[2023-10-14 17:21:45,863][75950] Updated weights for policy 1, policy_version 93510 (0.0011) -[2023-10-14 17:21:46,229][75950] Updated weights for policy 1, policy_version 93520 (0.0011) -[2023-10-14 17:21:46,594][75950] Updated weights for policy 1, policy_version 93530 (0.0008) -[2023-10-14 17:21:48,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 191823872. Throughput: 0: 1687.9, 1: 1686.0. Samples: 47961066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:21:48,165][74987] Avg episode reward: [(0, '25.690'), (1, '35.760')] -[2023-10-14 17:21:48,930][75949] Updated weights for policy 0, policy_version 93801 (0.0009) -[2023-10-14 17:21:49,309][75949] Updated weights for policy 0, policy_version 93811 (0.0009) -[2023-10-14 17:21:49,682][75949] Updated weights for policy 0, policy_version 93821 (0.0008) -[2023-10-14 17:21:50,688][75950] Updated weights for policy 1, policy_version 93540 (0.0011) -[2023-10-14 17:21:51,064][75950] Updated weights for policy 1, policy_version 93550 (0.0010) -[2023-10-14 17:21:51,426][75950] Updated weights for policy 1, policy_version 93560 (0.0008) -[2023-10-14 17:21:53,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 191889408. Throughput: 0: 1695.7, 1: 1662.2. Samples: 47980738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:21:53,164][74987] Avg episode reward: [(0, '31.390'), (1, '36.330')] -[2023-10-14 17:21:53,686][75949] Updated weights for policy 0, policy_version 93831 (0.0008) -[2023-10-14 17:21:54,060][75949] Updated weights for policy 0, policy_version 93841 (0.0009) -[2023-10-14 17:21:54,434][75949] Updated weights for policy 0, policy_version 93851 (0.0011) -[2023-10-14 17:21:55,355][75950] Updated weights for policy 1, policy_version 93570 (0.0009) -[2023-10-14 17:21:55,718][75950] Updated weights for policy 1, policy_version 93580 (0.0011) -[2023-10-14 17:21:56,083][75950] Updated weights for policy 1, policy_version 93590 (0.0010) -[2023-10-14 17:21:56,445][75950] Updated weights for policy 1, policy_version 93600 (0.0008) -[2023-10-14 17:21:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 191954944. Throughput: 0: 1693.2, 1: 1686.5. Samples: 48001500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:21:58,164][74987] Avg episode reward: [(0, '24.890'), (1, '34.490')] -[2023-10-14 17:21:58,429][75949] Updated weights for policy 0, policy_version 93861 (0.0009) -[2023-10-14 17:21:58,812][75949] Updated weights for policy 0, policy_version 93871 (0.0007) -[2023-10-14 17:21:59,186][75949] Updated weights for policy 0, policy_version 93881 (0.0007) -[2023-10-14 17:22:00,353][75950] Updated weights for policy 1, policy_version 93610 (0.0010) -[2023-10-14 17:22:00,723][75950] Updated weights for policy 1, policy_version 93620 (0.0009) -[2023-10-14 17:22:01,085][75950] Updated weights for policy 1, policy_version 93630 (0.0007) -[2023-10-14 17:22:03,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 192020480. Throughput: 0: 1693.2, 1: 1668.1. Samples: 48011102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:22:03,164][74987] Avg episode reward: [(0, '31.440'), (1, '34.790')] -[2023-10-14 17:22:03,183][75949] Updated weights for policy 0, policy_version 93891 (0.0007) -[2023-10-14 17:22:03,549][75949] Updated weights for policy 0, policy_version 93901 (0.0008) -[2023-10-14 17:22:03,923][75949] Updated weights for policy 0, policy_version 93911 (0.0008) -[2023-10-14 17:22:05,176][75950] Updated weights for policy 1, policy_version 93640 (0.0008) -[2023-10-14 17:22:05,548][75950] Updated weights for policy 1, policy_version 93650 (0.0008) -[2023-10-14 17:22:05,907][75950] Updated weights for policy 1, policy_version 93660 (0.0009) -[2023-10-14 17:22:08,037][75949] Updated weights for policy 0, policy_version 93921 (0.0009) -[2023-10-14 17:22:08,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 192086016. Throughput: 0: 1689.6, 1: 1675.8. Samples: 48031232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:22:08,164][74987] Avg episode reward: [(0, '24.150'), (1, '34.740')] -[2023-10-14 17:22:08,396][75949] Updated weights for policy 0, policy_version 93931 (0.0008) -[2023-10-14 17:22:08,782][75949] Updated weights for policy 0, policy_version 93941 (0.0009) -[2023-10-14 17:22:09,148][75949] Updated weights for policy 0, policy_version 93951 (0.0009) -[2023-10-14 17:22:09,840][75950] Updated weights for policy 1, policy_version 93670 (0.0009) -[2023-10-14 17:22:10,202][75950] Updated weights for policy 1, policy_version 93680 (0.0009) -[2023-10-14 17:22:10,569][75950] Updated weights for policy 1, policy_version 93690 (0.0009) -[2023-10-14 17:22:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 192151552. Throughput: 0: 1684.3, 1: 1691.2. Samples: 48052098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:22:13,165][74987] Avg episode reward: [(0, '29.920'), (1, '36.430')] -[2023-10-14 17:22:13,276][75949] Updated weights for policy 0, policy_version 93961 (0.0009) -[2023-10-14 17:22:13,649][75949] Updated weights for policy 0, policy_version 93971 (0.0011) -[2023-10-14 17:22:14,018][75949] Updated weights for policy 0, policy_version 93981 (0.0008) -[2023-10-14 17:22:14,544][75950] Updated weights for policy 1, policy_version 93700 (0.0008) -[2023-10-14 17:22:14,916][75950] Updated weights for policy 1, policy_version 93710 (0.0009) -[2023-10-14 17:22:15,280][75950] Updated weights for policy 1, policy_version 93720 (0.0009) -[2023-10-14 17:22:18,144][75949] Updated weights for policy 0, policy_version 93991 (0.0010) -[2023-10-14 17:22:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 192217088. Throughput: 0: 1682.8, 1: 1663.9. Samples: 48061254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:22:18,164][74987] Avg episode reward: [(0, '24.120'), (1, '34.230')] -[2023-10-14 17:22:18,524][75949] Updated weights for policy 0, policy_version 94001 (0.0008) -[2023-10-14 17:22:18,885][75949] Updated weights for policy 0, policy_version 94011 (0.0009) -[2023-10-14 17:22:19,412][75950] Updated weights for policy 1, policy_version 93730 (0.0007) -[2023-10-14 17:22:19,780][75950] Updated weights for policy 1, policy_version 93740 (0.0011) -[2023-10-14 17:22:20,158][75950] Updated weights for policy 1, policy_version 93750 (0.0010) -[2023-10-14 17:22:20,520][75950] Updated weights for policy 1, policy_version 93760 (0.0010) -[2023-10-14 17:22:22,749][75949] Updated weights for policy 0, policy_version 94021 (0.0008) -[2023-10-14 17:22:23,113][75949] Updated weights for policy 0, policy_version 94031 (0.0009) -[2023-10-14 17:22:23,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 192282624. Throughput: 0: 1683.3, 1: 1685.2. Samples: 48082028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:22:23,164][74987] Avg episode reward: [(0, '28.490'), (1, '36.090')] -[2023-10-14 17:22:23,475][75949] Updated weights for policy 0, policy_version 94041 (0.0009) -[2023-10-14 17:22:24,532][75950] Updated weights for policy 1, policy_version 93770 (0.0009) -[2023-10-14 17:22:24,886][75950] Updated weights for policy 1, policy_version 93780 (0.0008) -[2023-10-14 17:22:25,252][75950] Updated weights for policy 1, policy_version 93790 (0.0008) -[2023-10-14 17:22:27,372][75949] Updated weights for policy 0, policy_version 94051 (0.0008) -[2023-10-14 17:22:27,748][75949] Updated weights for policy 0, policy_version 94061 (0.0009) -[2023-10-14 17:22:28,118][75949] Updated weights for policy 0, policy_version 94071 (0.0011) -[2023-10-14 17:22:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 192348160. Throughput: 0: 1681.6, 1: 1694.6. Samples: 48102530. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) -[2023-10-14 17:22:28,165][74987] Avg episode reward: [(0, '26.130'), (1, '35.980')] -[2023-10-14 17:22:29,480][75950] Updated weights for policy 1, policy_version 93800 (0.0008) -[2023-10-14 17:22:29,857][75950] Updated weights for policy 1, policy_version 93810 (0.0008) -[2023-10-14 17:22:30,217][75950] Updated weights for policy 1, policy_version 93820 (0.0008) -[2023-10-14 17:22:32,260][75949] Updated weights for policy 0, policy_version 94081 (0.0010) -[2023-10-14 17:22:32,630][75949] Updated weights for policy 0, policy_version 94091 (0.0007) -[2023-10-14 17:22:33,004][75949] Updated weights for policy 0, policy_version 94101 (0.0007) -[2023-10-14 17:22:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 192413696. Throughput: 0: 1692.1, 1: 1661.8. Samples: 48111994. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) -[2023-10-14 17:22:33,164][74987] Avg episode reward: [(0, '27.730'), (1, '33.640')] -[2023-10-14 17:22:33,367][75949] Updated weights for policy 0, policy_version 94111 (0.0010) -[2023-10-14 17:22:34,267][75950] Updated weights for policy 1, policy_version 93830 (0.0009) -[2023-10-14 17:22:34,631][75950] Updated weights for policy 1, policy_version 93840 (0.0009) -[2023-10-14 17:22:34,993][75950] Updated weights for policy 1, policy_version 93850 (0.0008) -[2023-10-14 17:22:37,486][75949] Updated weights for policy 0, policy_version 94121 (0.0008) -[2023-10-14 17:22:37,862][75949] Updated weights for policy 0, policy_version 94131 (0.0010) -[2023-10-14 17:22:38,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 192479232. Throughput: 0: 1686.9, 1: 1690.6. Samples: 48132726. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) -[2023-10-14 17:22:38,164][74987] Avg episode reward: [(0, '27.160'), (1, '33.070')] -[2023-10-14 17:22:38,236][75949] Updated weights for policy 0, policy_version 94141 (0.0008) -[2023-10-14 17:22:39,235][75950] Updated weights for policy 1, policy_version 93860 (0.0009) -[2023-10-14 17:22:39,646][75950] Updated weights for policy 1, policy_version 93870 (0.0008) -[2023-10-14 17:22:40,012][75950] Updated weights for policy 1, policy_version 93880 (0.0007) -[2023-10-14 17:22:42,285][75949] Updated weights for policy 0, policy_version 94151 (0.0011) -[2023-10-14 17:22:42,671][75949] Updated weights for policy 0, policy_version 94161 (0.0009) -[2023-10-14 17:22:43,037][75949] Updated weights for policy 0, policy_version 94171 (0.0007) -[2023-10-14 17:22:43,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 192544768. Throughput: 0: 1677.0, 1: 1685.2. Samples: 48152798. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) -[2023-10-14 17:22:43,165][74987] Avg episode reward: [(0, '27.830'), (1, '34.690')] -[2023-10-14 17:22:43,174][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000093888_96141312.pth... -[2023-10-14 17:22:43,210][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000092320_94535680.pth -[2023-10-14 17:22:43,214][75801] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p1/milestones/checkpoint_000093888_96141312.pth -[2023-10-14 17:22:43,218][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000094176_96436224.pth... -[2023-10-14 17:22:43,247][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000092576_94797824.pth -[2023-10-14 17:22:43,251][75615] Saving a milestone ./train_atari/atari_riverraid_APPO/checkpoint_p0/milestones/checkpoint_000094176_96436224.pth -[2023-10-14 17:22:44,077][75950] Updated weights for policy 1, policy_version 93890 (0.0008) -[2023-10-14 17:22:44,447][75950] Updated weights for policy 1, policy_version 93900 (0.0007) -[2023-10-14 17:22:44,810][75950] Updated weights for policy 1, policy_version 93910 (0.0010) -[2023-10-14 17:22:45,180][75950] Updated weights for policy 1, policy_version 93920 (0.0010) -[2023-10-14 17:22:47,269][75949] Updated weights for policy 0, policy_version 94181 (0.0007) -[2023-10-14 17:22:47,659][75949] Updated weights for policy 0, policy_version 94191 (0.0007) -[2023-10-14 17:22:48,026][75949] Updated weights for policy 0, policy_version 94201 (0.0008) -[2023-10-14 17:22:48,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 192610304. Throughput: 0: 1693.6, 1: 1671.1. Samples: 48162512. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) -[2023-10-14 17:22:48,164][74987] Avg episode reward: [(0, '28.180'), (1, '34.630')] -[2023-10-14 17:22:49,260][75950] Updated weights for policy 1, policy_version 93930 (0.0008) -[2023-10-14 17:22:49,623][75950] Updated weights for policy 1, policy_version 93940 (0.0009) -[2023-10-14 17:22:49,995][75950] Updated weights for policy 1, policy_version 93950 (0.0007) -[2023-10-14 17:22:52,098][75949] Updated weights for policy 0, policy_version 94211 (0.0011) -[2023-10-14 17:22:52,464][75949] Updated weights for policy 0, policy_version 94221 (0.0008) -[2023-10-14 17:22:52,840][75949] Updated weights for policy 0, policy_version 94231 (0.0007) -[2023-10-14 17:22:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 192675840. Throughput: 0: 1686.0, 1: 1688.2. Samples: 48183072. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) -[2023-10-14 17:22:53,165][74987] Avg episode reward: [(0, '26.890'), (1, '34.760')] -[2023-10-14 17:22:53,896][75950] Updated weights for policy 1, policy_version 93960 (0.0007) -[2023-10-14 17:22:54,262][75950] Updated weights for policy 1, policy_version 93970 (0.0007) -[2023-10-14 17:22:54,629][75950] Updated weights for policy 1, policy_version 93980 (0.0009) -[2023-10-14 17:22:56,849][75949] Updated weights for policy 0, policy_version 94241 (0.0008) -[2023-10-14 17:22:57,220][75949] Updated weights for policy 0, policy_version 94251 (0.0009) -[2023-10-14 17:22:57,587][75949] Updated weights for policy 0, policy_version 94261 (0.0009) -[2023-10-14 17:22:57,962][75949] Updated weights for policy 0, policy_version 94271 (0.0009) -[2023-10-14 17:22:58,164][74987] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 192774144. Throughput: 0: 1672.8, 1: 1685.1. Samples: 48203204. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) -[2023-10-14 17:22:58,165][74987] Avg episode reward: [(0, '28.300'), (1, '34.470')] -[2023-10-14 17:22:58,624][75950] Updated weights for policy 1, policy_version 93990 (0.0008) -[2023-10-14 17:22:58,994][75950] Updated weights for policy 1, policy_version 94000 (0.0008) -[2023-10-14 17:22:59,367][75950] Updated weights for policy 1, policy_version 94010 (0.0008) -[2023-10-14 17:23:01,828][75949] Updated weights for policy 0, policy_version 94281 (0.0010) -[2023-10-14 17:23:02,187][75949] Updated weights for policy 0, policy_version 94291 (0.0010) -[2023-10-14 17:23:02,558][75949] Updated weights for policy 0, policy_version 94301 (0.0011) -[2023-10-14 17:23:03,164][74987] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 192839680. Throughput: 0: 1698.7, 1: 1683.4. Samples: 48213450. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) -[2023-10-14 17:23:03,165][74987] Avg episode reward: [(0, '26.370'), (1, '33.520')] -[2023-10-14 17:23:03,461][75950] Updated weights for policy 1, policy_version 94020 (0.0007) -[2023-10-14 17:23:03,819][75950] Updated weights for policy 1, policy_version 94030 (0.0010) -[2023-10-14 17:23:04,190][75950] Updated weights for policy 1, policy_version 94040 (0.0009) -[2023-10-14 17:23:06,491][75949] Updated weights for policy 0, policy_version 94311 (0.0010) -[2023-10-14 17:23:06,863][75949] Updated weights for policy 0, policy_version 94321 (0.0008) -[2023-10-14 17:23:07,235][75949] Updated weights for policy 0, policy_version 94331 (0.0008) -[2023-10-14 17:23:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 192905216. Throughput: 0: 1690.0, 1: 1685.8. Samples: 48233940. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) -[2023-10-14 17:23:08,164][74987] Avg episode reward: [(0, '28.640'), (1, '34.030')] -[2023-10-14 17:23:08,194][75950] Updated weights for policy 1, policy_version 94050 (0.0011) -[2023-10-14 17:23:08,562][75950] Updated weights for policy 1, policy_version 94060 (0.0011) -[2023-10-14 17:23:08,940][75950] Updated weights for policy 1, policy_version 94070 (0.0009) -[2023-10-14 17:23:09,308][75950] Updated weights for policy 1, policy_version 94080 (0.0008) -[2023-10-14 17:23:11,014][75949] Updated weights for policy 0, policy_version 94341 (0.0008) -[2023-10-14 17:23:11,385][75949] Updated weights for policy 0, policy_version 94351 (0.0009) -[2023-10-14 17:23:11,761][75949] Updated weights for policy 0, policy_version 94361 (0.0010) -[2023-10-14 17:23:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 192970752. Throughput: 0: 1684.0, 1: 1687.5. Samples: 48254248. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) -[2023-10-14 17:23:13,164][74987] Avg episode reward: [(0, '25.930'), (1, '33.460')] -[2023-10-14 17:23:13,520][75950] Updated weights for policy 1, policy_version 94090 (0.0010) -[2023-10-14 17:23:13,882][75950] Updated weights for policy 1, policy_version 94100 (0.0010) -[2023-10-14 17:23:14,253][75950] Updated weights for policy 1, policy_version 94110 (0.0008) -[2023-10-14 17:23:15,809][75949] Updated weights for policy 0, policy_version 94371 (0.0008) -[2023-10-14 17:23:16,192][75949] Updated weights for policy 0, policy_version 94381 (0.0010) -[2023-10-14 17:23:16,559][75949] Updated weights for policy 0, policy_version 94391 (0.0009) -[2023-10-14 17:23:18,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 193036288. Throughput: 0: 1702.3, 1: 1691.0. Samples: 48264692. Policy #0 lag: (min: 19.0, avg: 19.2, max: 29.0) -[2023-10-14 17:23:18,164][74987] Avg episode reward: [(0, '29.140'), (1, '35.770')] -[2023-10-14 17:23:18,363][75950] Updated weights for policy 1, policy_version 94120 (0.0009) -[2023-10-14 17:23:18,735][75950] Updated weights for policy 1, policy_version 94130 (0.0009) -[2023-10-14 17:23:19,110][75950] Updated weights for policy 1, policy_version 94140 (0.0008) -[2023-10-14 17:23:20,652][75949] Updated weights for policy 0, policy_version 94401 (0.0008) -[2023-10-14 17:23:21,015][75949] Updated weights for policy 0, policy_version 94411 (0.0008) -[2023-10-14 17:23:21,386][75949] Updated weights for policy 0, policy_version 94421 (0.0008) -[2023-10-14 17:23:21,745][75949] Updated weights for policy 0, policy_version 94431 (0.0011) -[2023-10-14 17:23:22,933][75950] Updated weights for policy 1, policy_version 94150 (0.0009) -[2023-10-14 17:23:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 193101824. Throughput: 0: 1677.8, 1: 1693.6. Samples: 48284438. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:23:23,165][74987] Avg episode reward: [(0, '25.800'), (1, '35.050')] -[2023-10-14 17:23:23,299][75950] Updated weights for policy 1, policy_version 94160 (0.0009) -[2023-10-14 17:23:23,667][75950] Updated weights for policy 1, policy_version 94170 (0.0007) -[2023-10-14 17:23:25,864][75949] Updated weights for policy 0, policy_version 94441 (0.0008) -[2023-10-14 17:23:26,233][75949] Updated weights for policy 0, policy_version 94451 (0.0008) -[2023-10-14 17:23:26,603][75949] Updated weights for policy 0, policy_version 94461 (0.0010) -[2023-10-14 17:23:27,618][75950] Updated weights for policy 1, policy_version 94180 (0.0007) -[2023-10-14 17:23:28,009][75950] Updated weights for policy 1, policy_version 94190 (0.0008) -[2023-10-14 17:23:28,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 193167360. Throughput: 0: 1683.7, 1: 1694.7. Samples: 48304828. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:23:28,164][74987] Avg episode reward: [(0, '29.250'), (1, '34.870')] -[2023-10-14 17:23:28,366][75950] Updated weights for policy 1, policy_version 94200 (0.0010) -[2023-10-14 17:23:30,830][75949] Updated weights for policy 0, policy_version 94471 (0.0010) -[2023-10-14 17:23:31,199][75949] Updated weights for policy 0, policy_version 94481 (0.0008) -[2023-10-14 17:23:31,571][75949] Updated weights for policy 0, policy_version 94491 (0.0008) -[2023-10-14 17:23:32,484][75950] Updated weights for policy 1, policy_version 94210 (0.0010) -[2023-10-14 17:23:32,855][75950] Updated weights for policy 1, policy_version 94220 (0.0008) -[2023-10-14 17:23:33,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 193232896. Throughput: 0: 1696.0, 1: 1698.8. Samples: 48315282. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:23:33,164][74987] Avg episode reward: [(0, '23.180'), (1, '35.980')] -[2023-10-14 17:23:33,215][75950] Updated weights for policy 1, policy_version 94230 (0.0008) -[2023-10-14 17:23:33,575][75950] Updated weights for policy 1, policy_version 94240 (0.0007) -[2023-10-14 17:23:35,734][75949] Updated weights for policy 0, policy_version 94501 (0.0010) -[2023-10-14 17:23:36,102][75949] Updated weights for policy 0, policy_version 94511 (0.0007) -[2023-10-14 17:23:36,464][75949] Updated weights for policy 0, policy_version 94521 (0.0008) -[2023-10-14 17:23:37,475][75950] Updated weights for policy 1, policy_version 94250 (0.0009) -[2023-10-14 17:23:37,850][75950] Updated weights for policy 1, policy_version 94260 (0.0010) -[2023-10-14 17:23:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 193298432. Throughput: 0: 1674.5, 1: 1700.9. Samples: 48334966. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:23:38,165][74987] Avg episode reward: [(0, '32.870'), (1, '35.850')] -[2023-10-14 17:23:38,166][75615] Saving new best policy, reward=32.870! -[2023-10-14 17:23:38,209][75950] Updated weights for policy 1, policy_version 94270 (0.0010) -[2023-10-14 17:23:40,494][75949] Updated weights for policy 0, policy_version 94531 (0.0008) -[2023-10-14 17:23:40,870][75949] Updated weights for policy 0, policy_version 94541 (0.0010) -[2023-10-14 17:23:41,231][75949] Updated weights for policy 0, policy_version 94551 (0.0011) -[2023-10-14 17:23:42,390][75950] Updated weights for policy 1, policy_version 94280 (0.0007) -[2023-10-14 17:23:42,762][75950] Updated weights for policy 1, policy_version 94290 (0.0008) -[2023-10-14 17:23:43,137][75950] Updated weights for policy 1, policy_version 94300 (0.0009) -[2023-10-14 17:23:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 193363968. Throughput: 0: 1690.9, 1: 1685.0. Samples: 48355118. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:23:43,165][74987] Avg episode reward: [(0, '23.910'), (1, '35.640')] -[2023-10-14 17:23:45,317][75949] Updated weights for policy 0, policy_version 94561 (0.0009) -[2023-10-14 17:23:45,683][75949] Updated weights for policy 0, policy_version 94571 (0.0008) -[2023-10-14 17:23:46,054][75949] Updated weights for policy 0, policy_version 94581 (0.0011) -[2023-10-14 17:23:46,414][75949] Updated weights for policy 0, policy_version 94591 (0.0009) -[2023-10-14 17:23:47,065][75950] Updated weights for policy 1, policy_version 94310 (0.0007) -[2023-10-14 17:23:47,435][75950] Updated weights for policy 1, policy_version 94320 (0.0009) -[2023-10-14 17:23:47,805][75950] Updated weights for policy 1, policy_version 94330 (0.0011) -[2023-10-14 17:23:48,164][74987] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 193462272. Throughput: 0: 1685.4, 1: 1698.1. Samples: 48365708. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:23:48,165][74987] Avg episode reward: [(0, '32.750'), (1, '36.840')] -[2023-10-14 17:23:50,651][75949] Updated weights for policy 0, policy_version 94601 (0.0010) -[2023-10-14 17:23:51,018][75949] Updated weights for policy 0, policy_version 94611 (0.0009) -[2023-10-14 17:23:51,388][75949] Updated weights for policy 0, policy_version 94621 (0.0009) -[2023-10-14 17:23:51,915][75950] Updated weights for policy 1, policy_version 94340 (0.0010) -[2023-10-14 17:23:52,295][75950] Updated weights for policy 1, policy_version 94350 (0.0008) -[2023-10-14 17:23:52,659][75950] Updated weights for policy 1, policy_version 94360 (0.0009) -[2023-10-14 17:23:53,163][74987] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 193527808. Throughput: 0: 1668.3, 1: 1702.5. Samples: 48385626. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:23:53,164][74987] Avg episode reward: [(0, '23.550'), (1, '37.640')] -[2023-10-14 17:23:55,338][75949] Updated weights for policy 0, policy_version 94631 (0.0009) -[2023-10-14 17:23:55,710][75949] Updated weights for policy 0, policy_version 94641 (0.0008) -[2023-10-14 17:23:56,079][75949] Updated weights for policy 0, policy_version 94651 (0.0008) -[2023-10-14 17:23:56,746][75950] Updated weights for policy 1, policy_version 94370 (0.0008) -[2023-10-14 17:23:57,113][75950] Updated weights for policy 1, policy_version 94380 (0.0007) -[2023-10-14 17:23:57,467][75950] Updated weights for policy 1, policy_version 94390 (0.0008) -[2023-10-14 17:23:57,833][75950] Updated weights for policy 1, policy_version 94400 (0.0007) -[2023-10-14 17:23:58,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 193593344. Throughput: 0: 1683.2, 1: 1677.4. Samples: 48405478. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:23:58,164][74987] Avg episode reward: [(0, '31.090'), (1, '36.170')] -[2023-10-14 17:24:00,114][75949] Updated weights for policy 0, policy_version 94661 (0.0009) -[2023-10-14 17:24:00,483][75949] Updated weights for policy 0, policy_version 94671 (0.0007) -[2023-10-14 17:24:00,850][75949] Updated weights for policy 0, policy_version 94681 (0.0007) -[2023-10-14 17:24:02,099][75950] Updated weights for policy 1, policy_version 94410 (0.0007) -[2023-10-14 17:24:02,467][75950] Updated weights for policy 1, policy_version 94420 (0.0009) -[2023-10-14 17:24:02,834][75950] Updated weights for policy 1, policy_version 94430 (0.0010) -[2023-10-14 17:24:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 193658880. Throughput: 0: 1666.0, 1: 1696.5. Samples: 48416004. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:24:03,164][74987] Avg episode reward: [(0, '23.950'), (1, '33.990')] -[2023-10-14 17:24:04,581][75949] Updated weights for policy 0, policy_version 94691 (0.0008) -[2023-10-14 17:24:04,933][75949] Updated weights for policy 0, policy_version 94701 (0.0009) -[2023-10-14 17:24:05,297][75949] Updated weights for policy 0, policy_version 94711 (0.0010) -[2023-10-14 17:24:06,979][75950] Updated weights for policy 1, policy_version 94440 (0.0007) -[2023-10-14 17:24:07,341][75950] Updated weights for policy 1, policy_version 94450 (0.0008) -[2023-10-14 17:24:07,700][75950] Updated weights for policy 1, policy_version 94460 (0.0008) -[2023-10-14 17:24:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 193724416. Throughput: 0: 1683.9, 1: 1690.2. Samples: 48436272. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:24:08,165][74987] Avg episode reward: [(0, '30.820'), (1, '36.000')] -[2023-10-14 17:24:09,440][75949] Updated weights for policy 0, policy_version 94721 (0.0010) -[2023-10-14 17:24:09,801][75949] Updated weights for policy 0, policy_version 94731 (0.0010) -[2023-10-14 17:24:10,176][75949] Updated weights for policy 0, policy_version 94741 (0.0010) -[2023-10-14 17:24:10,556][75949] Updated weights for policy 0, policy_version 94751 (0.0011) -[2023-10-14 17:24:11,829][75950] Updated weights for policy 1, policy_version 94470 (0.0010) -[2023-10-14 17:24:12,194][75950] Updated weights for policy 1, policy_version 94480 (0.0007) -[2023-10-14 17:24:12,572][75950] Updated weights for policy 1, policy_version 94490 (0.0009) -[2023-10-14 17:24:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 193789952. Throughput: 0: 1687.1, 1: 1664.8. Samples: 48455660. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:24:13,165][74987] Avg episode reward: [(0, '25.330'), (1, '32.080')] -[2023-10-14 17:24:14,685][75949] Updated weights for policy 0, policy_version 94761 (0.0009) -[2023-10-14 17:24:15,058][75949] Updated weights for policy 0, policy_version 94771 (0.0009) -[2023-10-14 17:24:15,435][75949] Updated weights for policy 0, policy_version 94781 (0.0009) -[2023-10-14 17:24:16,776][75950] Updated weights for policy 1, policy_version 94500 (0.0009) -[2023-10-14 17:24:17,162][75950] Updated weights for policy 1, policy_version 94510 (0.0010) -[2023-10-14 17:24:17,542][75950] Updated weights for policy 1, policy_version 94520 (0.0009) -[2023-10-14 17:24:18,164][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 193855488. Throughput: 0: 1659.6, 1: 1683.2. Samples: 48465706. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) -[2023-10-14 17:24:18,164][74987] Avg episode reward: [(0, '30.060'), (1, '33.550')] -[2023-10-14 17:24:19,529][75949] Updated weights for policy 0, policy_version 94791 (0.0009) -[2023-10-14 17:24:19,900][75949] Updated weights for policy 0, policy_version 94801 (0.0008) -[2023-10-14 17:24:20,270][75949] Updated weights for policy 0, policy_version 94811 (0.0007) -[2023-10-14 17:24:21,387][75950] Updated weights for policy 1, policy_version 94530 (0.0009) -[2023-10-14 17:24:21,751][75950] Updated weights for policy 1, policy_version 94540 (0.0010) -[2023-10-14 17:24:22,110][75950] Updated weights for policy 1, policy_version 94550 (0.0010) -[2023-10-14 17:24:22,477][75950] Updated weights for policy 1, policy_version 94560 (0.0009) -[2023-10-14 17:24:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 193921024. Throughput: 0: 1686.8, 1: 1667.6. Samples: 48485914. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:24:23,165][74987] Avg episode reward: [(0, '24.110'), (1, '35.470')] -[2023-10-14 17:24:24,484][75949] Updated weights for policy 0, policy_version 94821 (0.0008) -[2023-10-14 17:24:24,860][75949] Updated weights for policy 0, policy_version 94831 (0.0011) -[2023-10-14 17:24:25,238][75949] Updated weights for policy 0, policy_version 94841 (0.0010) -[2023-10-14 17:24:26,611][75950] Updated weights for policy 1, policy_version 94570 (0.0007) -[2023-10-14 17:24:26,984][75950] Updated weights for policy 1, policy_version 94580 (0.0009) -[2023-10-14 17:24:27,346][75950] Updated weights for policy 1, policy_version 94590 (0.0009) -[2023-10-14 17:24:28,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 193986560. Throughput: 0: 1681.5, 1: 1659.6. Samples: 48505466. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:24:28,165][74987] Avg episode reward: [(0, '30.740'), (1, '34.170')] -[2023-10-14 17:24:29,272][75949] Updated weights for policy 0, policy_version 94851 (0.0010) -[2023-10-14 17:24:29,645][75949] Updated weights for policy 0, policy_version 94861 (0.0008) -[2023-10-14 17:24:30,022][75949] Updated weights for policy 0, policy_version 94871 (0.0008) -[2023-10-14 17:24:31,171][75950] Updated weights for policy 1, policy_version 94600 (0.0008) -[2023-10-14 17:24:31,538][75950] Updated weights for policy 1, policy_version 94610 (0.0008) -[2023-10-14 17:24:31,911][75950] Updated weights for policy 1, policy_version 94620 (0.0009) -[2023-10-14 17:24:33,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 194052096. Throughput: 0: 1662.0, 1: 1678.1. Samples: 48516012. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:24:33,165][74987] Avg episode reward: [(0, '25.100'), (1, '34.290')] -[2023-10-14 17:24:33,997][75949] Updated weights for policy 0, policy_version 94881 (0.0009) -[2023-10-14 17:24:34,362][75949] Updated weights for policy 0, policy_version 94891 (0.0008) -[2023-10-14 17:24:34,733][75949] Updated weights for policy 0, policy_version 94901 (0.0007) -[2023-10-14 17:24:35,100][75949] Updated weights for policy 0, policy_version 94911 (0.0008) -[2023-10-14 17:24:36,230][75950] Updated weights for policy 1, policy_version 94630 (0.0009) -[2023-10-14 17:24:36,591][75950] Updated weights for policy 1, policy_version 94640 (0.0008) -[2023-10-14 17:24:36,958][75950] Updated weights for policy 1, policy_version 94650 (0.0010) -[2023-10-14 17:24:38,164][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 194117632. Throughput: 0: 1690.3, 1: 1650.6. Samples: 48535966. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:24:38,165][74987] Avg episode reward: [(0, '30.390'), (1, '34.560')] -[2023-10-14 17:24:39,101][75949] Updated weights for policy 0, policy_version 94921 (0.0007) -[2023-10-14 17:24:39,458][75949] Updated weights for policy 0, policy_version 94931 (0.0007) -[2023-10-14 17:24:39,833][75949] Updated weights for policy 0, policy_version 94941 (0.0008) -[2023-10-14 17:24:41,151][75950] Updated weights for policy 1, policy_version 94660 (0.0011) -[2023-10-14 17:24:41,509][75950] Updated weights for policy 1, policy_version 94670 (0.0011) -[2023-10-14 17:24:41,875][75950] Updated weights for policy 1, policy_version 94680 (0.0008) -[2023-10-14 17:24:43,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 194183168. Throughput: 0: 1688.9, 1: 1655.2. Samples: 48555964. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:24:43,164][74987] Avg episode reward: [(0, '25.430'), (1, '35.400')] -[2023-10-14 17:24:43,172][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000094688_96960512.pth... -[2023-10-14 17:24:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000094944_97222656.pth... -[2023-10-14 17:24:43,218][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000093376_95617024.pth -[2023-10-14 17:24:43,218][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000093120_95354880.pth -[2023-10-14 17:24:43,876][75949] Updated weights for policy 0, policy_version 94951 (0.0008) -[2023-10-14 17:24:44,238][75949] Updated weights for policy 0, policy_version 94961 (0.0007) -[2023-10-14 17:24:44,612][75949] Updated weights for policy 0, policy_version 94971 (0.0007) -[2023-10-14 17:24:45,928][75950] Updated weights for policy 1, policy_version 94690 (0.0010) -[2023-10-14 17:24:46,293][75950] Updated weights for policy 1, policy_version 94700 (0.0007) -[2023-10-14 17:24:46,660][75950] Updated weights for policy 1, policy_version 94710 (0.0007) -[2023-10-14 17:24:47,021][75950] Updated weights for policy 1, policy_version 94720 (0.0008) -[2023-10-14 17:24:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 194248704. Throughput: 0: 1677.2, 1: 1664.8. Samples: 48566392. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:24:48,164][74987] Avg episode reward: [(0, '28.510'), (1, '33.950')] -[2023-10-14 17:24:48,731][75949] Updated weights for policy 0, policy_version 94981 (0.0008) -[2023-10-14 17:24:49,105][75949] Updated weights for policy 0, policy_version 94991 (0.0007) -[2023-10-14 17:24:49,472][75949] Updated weights for policy 0, policy_version 95001 (0.0010) -[2023-10-14 17:24:50,882][75950] Updated weights for policy 1, policy_version 94730 (0.0008) -[2023-10-14 17:24:51,256][75950] Updated weights for policy 1, policy_version 94740 (0.0010) -[2023-10-14 17:24:51,626][75950] Updated weights for policy 1, policy_version 94750 (0.0008) -[2023-10-14 17:24:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 194314240. Throughput: 0: 1683.0, 1: 1647.1. Samples: 48586128. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:24:53,165][74987] Avg episode reward: [(0, '24.020'), (1, '33.710')] -[2023-10-14 17:24:53,641][75949] Updated weights for policy 0, policy_version 95011 (0.0007) -[2023-10-14 17:24:54,001][75949] Updated weights for policy 0, policy_version 95021 (0.0009) -[2023-10-14 17:24:54,371][75949] Updated weights for policy 0, policy_version 95031 (0.0008) -[2023-10-14 17:24:55,822][75950] Updated weights for policy 1, policy_version 94760 (0.0008) -[2023-10-14 17:24:56,183][75950] Updated weights for policy 1, policy_version 94770 (0.0010) -[2023-10-14 17:24:56,543][75950] Updated weights for policy 1, policy_version 94780 (0.0009) -[2023-10-14 17:24:58,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 194379776. Throughput: 0: 1686.9, 1: 1670.1. Samples: 48606724. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:24:58,164][74987] Avg episode reward: [(0, '27.400'), (1, '35.300')] -[2023-10-14 17:24:58,326][75949] Updated weights for policy 0, policy_version 95041 (0.0009) -[2023-10-14 17:24:58,693][75949] Updated weights for policy 0, policy_version 95051 (0.0008) -[2023-10-14 17:24:59,051][75949] Updated weights for policy 0, policy_version 95061 (0.0008) -[2023-10-14 17:24:59,431][75949] Updated weights for policy 0, policy_version 95071 (0.0008) -[2023-10-14 17:25:00,490][75950] Updated weights for policy 1, policy_version 94790 (0.0010) -[2023-10-14 17:25:00,850][75950] Updated weights for policy 1, policy_version 94800 (0.0011) -[2023-10-14 17:25:01,212][75950] Updated weights for policy 1, policy_version 94810 (0.0010) -[2023-10-14 17:25:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 194445312. Throughput: 0: 1686.6, 1: 1671.8. Samples: 48616836. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:25:03,164][74987] Avg episode reward: [(0, '23.480'), (1, '35.810')] -[2023-10-14 17:25:03,418][75949] Updated weights for policy 0, policy_version 95081 (0.0009) -[2023-10-14 17:25:03,796][75949] Updated weights for policy 0, policy_version 95091 (0.0008) -[2023-10-14 17:25:04,162][75949] Updated weights for policy 0, policy_version 95101 (0.0011) -[2023-10-14 17:25:05,370][75950] Updated weights for policy 1, policy_version 94820 (0.0009) -[2023-10-14 17:25:05,754][75950] Updated weights for policy 1, policy_version 94830 (0.0008) -[2023-10-14 17:25:06,111][75950] Updated weights for policy 1, policy_version 94840 (0.0010) -[2023-10-14 17:25:08,079][75949] Updated weights for policy 0, policy_version 95111 (0.0008) -[2023-10-14 17:25:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 194510848. Throughput: 0: 1692.8, 1: 1658.6. Samples: 48636728. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:25:08,165][74987] Avg episode reward: [(0, '25.570'), (1, '35.470')] -[2023-10-14 17:25:08,446][75949] Updated weights for policy 0, policy_version 95121 (0.0009) -[2023-10-14 17:25:08,813][75949] Updated weights for policy 0, policy_version 95131 (0.0008) -[2023-10-14 17:25:10,362][75950] Updated weights for policy 1, policy_version 94850 (0.0010) -[2023-10-14 17:25:10,733][75950] Updated weights for policy 1, policy_version 94860 (0.0009) -[2023-10-14 17:25:11,102][75950] Updated weights for policy 1, policy_version 94870 (0.0009) -[2023-10-14 17:25:11,467][75950] Updated weights for policy 1, policy_version 94880 (0.0008) -[2023-10-14 17:25:13,095][75949] Updated weights for policy 0, policy_version 95141 (0.0010) -[2023-10-14 17:25:13,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 194576384. Throughput: 0: 1698.4, 1: 1674.7. Samples: 48657254. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:25:13,165][74987] Avg episode reward: [(0, '26.550'), (1, '35.820')] -[2023-10-14 17:25:13,472][75949] Updated weights for policy 0, policy_version 95151 (0.0009) -[2023-10-14 17:25:13,849][75949] Updated weights for policy 0, policy_version 95161 (0.0009) -[2023-10-14 17:25:15,587][75950] Updated weights for policy 1, policy_version 94890 (0.0008) -[2023-10-14 17:25:15,954][75950] Updated weights for policy 1, policy_version 94900 (0.0008) -[2023-10-14 17:25:16,321][75950] Updated weights for policy 1, policy_version 94910 (0.0009) -[2023-10-14 17:25:17,840][75949] Updated weights for policy 0, policy_version 95171 (0.0010) -[2023-10-14 17:25:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 194641920. Throughput: 0: 1695.1, 1: 1663.1. Samples: 48667128. Policy #0 lag: (min: 2.0, avg: 9.3, max: 34.0) -[2023-10-14 17:25:18,165][74987] Avg episode reward: [(0, '23.890'), (1, '37.520')] -[2023-10-14 17:25:18,213][75949] Updated weights for policy 0, policy_version 95181 (0.0009) -[2023-10-14 17:25:18,574][75949] Updated weights for policy 0, policy_version 95191 (0.0008) -[2023-10-14 17:25:20,415][75950] Updated weights for policy 1, policy_version 94920 (0.0007) -[2023-10-14 17:25:20,785][75950] Updated weights for policy 1, policy_version 94930 (0.0008) -[2023-10-14 17:25:21,141][75950] Updated weights for policy 1, policy_version 94940 (0.0008) -[2023-10-14 17:25:22,701][75949] Updated weights for policy 0, policy_version 95201 (0.0008) -[2023-10-14 17:25:23,070][75949] Updated weights for policy 0, policy_version 95211 (0.0009) -[2023-10-14 17:25:23,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 194707456. Throughput: 0: 1690.8, 1: 1662.8. Samples: 48686882. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) -[2023-10-14 17:25:23,165][74987] Avg episode reward: [(0, '30.030'), (1, '36.490')] -[2023-10-14 17:25:23,443][75949] Updated weights for policy 0, policy_version 95221 (0.0008) -[2023-10-14 17:25:23,816][75949] Updated weights for policy 0, policy_version 95231 (0.0011) -[2023-10-14 17:25:25,388][75950] Updated weights for policy 1, policy_version 94950 (0.0009) -[2023-10-14 17:25:25,753][75950] Updated weights for policy 1, policy_version 94960 (0.0010) -[2023-10-14 17:25:26,119][75950] Updated weights for policy 1, policy_version 94970 (0.0011) -[2023-10-14 17:25:27,950][75949] Updated weights for policy 0, policy_version 95241 (0.0008) -[2023-10-14 17:25:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194772992. Throughput: 0: 1685.2, 1: 1682.5. Samples: 48707514. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) -[2023-10-14 17:25:28,165][74987] Avg episode reward: [(0, '23.290'), (1, '36.080')] -[2023-10-14 17:25:28,314][75949] Updated weights for policy 0, policy_version 95251 (0.0008) -[2023-10-14 17:25:28,676][75949] Updated weights for policy 0, policy_version 95261 (0.0008) -[2023-10-14 17:25:30,003][75950] Updated weights for policy 1, policy_version 94980 (0.0009) -[2023-10-14 17:25:30,363][75950] Updated weights for policy 1, policy_version 94990 (0.0008) -[2023-10-14 17:25:30,731][75950] Updated weights for policy 1, policy_version 95000 (0.0008) -[2023-10-14 17:25:32,611][75949] Updated weights for policy 0, policy_version 95271 (0.0007) -[2023-10-14 17:25:32,983][75949] Updated weights for policy 0, policy_version 95281 (0.0007) -[2023-10-14 17:25:33,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 194838528. Throughput: 0: 1689.9, 1: 1665.2. Samples: 48717372. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) -[2023-10-14 17:25:33,164][74987] Avg episode reward: [(0, '31.450'), (1, '35.310')] -[2023-10-14 17:25:33,349][75949] Updated weights for policy 0, policy_version 95291 (0.0008) -[2023-10-14 17:25:34,907][75950] Updated weights for policy 1, policy_version 95010 (0.0009) -[2023-10-14 17:25:35,284][75950] Updated weights for policy 1, policy_version 95020 (0.0011) -[2023-10-14 17:25:35,645][75950] Updated weights for policy 1, policy_version 95030 (0.0011) -[2023-10-14 17:25:36,012][75950] Updated weights for policy 1, policy_version 95040 (0.0009) -[2023-10-14 17:25:37,398][75949] Updated weights for policy 0, policy_version 95301 (0.0008) -[2023-10-14 17:25:37,762][75949] Updated weights for policy 0, policy_version 95311 (0.0008) -[2023-10-14 17:25:38,141][75949] Updated weights for policy 0, policy_version 95321 (0.0007) -[2023-10-14 17:25:38,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194904064. Throughput: 0: 1691.6, 1: 1668.0. Samples: 48737310. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) -[2023-10-14 17:25:38,164][74987] Avg episode reward: [(0, '26.130'), (1, '35.350')] -[2023-10-14 17:25:40,017][75950] Updated weights for policy 1, policy_version 95050 (0.0009) -[2023-10-14 17:25:40,394][75950] Updated weights for policy 1, policy_version 95060 (0.0011) -[2023-10-14 17:25:40,766][75950] Updated weights for policy 1, policy_version 95070 (0.0009) -[2023-10-14 17:25:42,116][75949] Updated weights for policy 0, policy_version 95331 (0.0008) -[2023-10-14 17:25:42,477][75949] Updated weights for policy 0, policy_version 95341 (0.0007) -[2023-10-14 17:25:42,845][75949] Updated weights for policy 0, policy_version 95351 (0.0010) -[2023-10-14 17:25:43,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194969600. Throughput: 0: 1676.7, 1: 1675.0. Samples: 48757552. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) -[2023-10-14 17:25:43,164][74987] Avg episode reward: [(0, '31.920'), (1, '35.670')] -[2023-10-14 17:25:44,807][75950] Updated weights for policy 1, policy_version 95080 (0.0010) -[2023-10-14 17:25:45,177][75950] Updated weights for policy 1, policy_version 95090 (0.0010) -[2023-10-14 17:25:45,534][75950] Updated weights for policy 1, policy_version 95100 (0.0008) -[2023-10-14 17:25:47,036][75949] Updated weights for policy 0, policy_version 95361 (0.0011) -[2023-10-14 17:25:47,393][75949] Updated weights for policy 0, policy_version 95371 (0.0008) -[2023-10-14 17:25:47,764][75949] Updated weights for policy 0, policy_version 95381 (0.0007) -[2023-10-14 17:25:48,121][75949] Updated weights for policy 0, policy_version 95391 (0.0007) -[2023-10-14 17:25:48,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195067904. Throughput: 0: 1693.2, 1: 1656.7. Samples: 48767580. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) -[2023-10-14 17:25:48,165][74987] Avg episode reward: [(0, '25.290'), (1, '33.270')] -[2023-10-14 17:25:49,688][75950] Updated weights for policy 1, policy_version 95110 (0.0008) -[2023-10-14 17:25:50,055][75950] Updated weights for policy 1, policy_version 95120 (0.0008) -[2023-10-14 17:25:50,433][75950] Updated weights for policy 1, policy_version 95130 (0.0009) -[2023-10-14 17:25:52,188][75949] Updated weights for policy 0, policy_version 95401 (0.0011) -[2023-10-14 17:25:52,562][75949] Updated weights for policy 0, policy_version 95411 (0.0009) -[2023-10-14 17:25:52,929][75949] Updated weights for policy 0, policy_version 95421 (0.0007) -[2023-10-14 17:25:53,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195133440. Throughput: 0: 1683.9, 1: 1677.2. Samples: 48787978. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) -[2023-10-14 17:25:53,165][74987] Avg episode reward: [(0, '30.470'), (1, '32.800')] -[2023-10-14 17:25:54,531][75950] Updated weights for policy 1, policy_version 95140 (0.0010) -[2023-10-14 17:25:54,936][75950] Updated weights for policy 1, policy_version 95150 (0.0010) -[2023-10-14 17:25:55,308][75950] Updated weights for policy 1, policy_version 95160 (0.0009) -[2023-10-14 17:25:57,036][75949] Updated weights for policy 0, policy_version 95431 (0.0008) -[2023-10-14 17:25:57,406][75949] Updated weights for policy 0, policy_version 95441 (0.0008) -[2023-10-14 17:25:57,762][75949] Updated weights for policy 0, policy_version 95451 (0.0011) -[2023-10-14 17:25:58,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195198976. Throughput: 0: 1664.1, 1: 1681.9. Samples: 48807822. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) -[2023-10-14 17:25:58,164][74987] Avg episode reward: [(0, '24.970'), (1, '34.940')] -[2023-10-14 17:25:59,134][75950] Updated weights for policy 1, policy_version 95170 (0.0008) -[2023-10-14 17:25:59,503][75950] Updated weights for policy 1, policy_version 95180 (0.0007) -[2023-10-14 17:25:59,869][75950] Updated weights for policy 1, policy_version 95190 (0.0008) -[2023-10-14 17:26:00,236][75950] Updated weights for policy 1, policy_version 95200 (0.0008) -[2023-10-14 17:26:01,914][75949] Updated weights for policy 0, policy_version 95461 (0.0011) -[2023-10-14 17:26:02,314][75949] Updated weights for policy 0, policy_version 95471 (0.0009) -[2023-10-14 17:26:02,675][75949] Updated weights for policy 0, policy_version 95481 (0.0009) -[2023-10-14 17:26:03,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195264512. Throughput: 0: 1688.5, 1: 1667.0. Samples: 48818128. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) -[2023-10-14 17:26:03,165][74987] Avg episode reward: [(0, '32.370'), (1, '35.290')] -[2023-10-14 17:26:04,263][75950] Updated weights for policy 1, policy_version 95210 (0.0008) -[2023-10-14 17:26:04,623][75950] Updated weights for policy 1, policy_version 95220 (0.0007) -[2023-10-14 17:26:04,986][75950] Updated weights for policy 1, policy_version 95230 (0.0007) -[2023-10-14 17:26:06,819][75949] Updated weights for policy 0, policy_version 95491 (0.0010) -[2023-10-14 17:26:07,182][75949] Updated weights for policy 0, policy_version 95501 (0.0011) -[2023-10-14 17:26:07,555][75949] Updated weights for policy 0, policy_version 95511 (0.0010) -[2023-10-14 17:26:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 195330048. Throughput: 0: 1676.1, 1: 1694.7. Samples: 48838568. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) -[2023-10-14 17:26:08,165][74987] Avg episode reward: [(0, '25.200'), (1, '33.740')] -[2023-10-14 17:26:09,166][75950] Updated weights for policy 1, policy_version 95240 (0.0008) -[2023-10-14 17:26:09,529][75950] Updated weights for policy 1, policy_version 95250 (0.0008) -[2023-10-14 17:26:09,901][75950] Updated weights for policy 1, policy_version 95260 (0.0009) -[2023-10-14 17:26:11,702][75949] Updated weights for policy 0, policy_version 95521 (0.0010) -[2023-10-14 17:26:12,076][75949] Updated weights for policy 0, policy_version 95531 (0.0011) -[2023-10-14 17:26:12,444][75949] Updated weights for policy 0, policy_version 95541 (0.0009) -[2023-10-14 17:26:12,822][75949] Updated weights for policy 0, policy_version 95551 (0.0009) -[2023-10-14 17:26:13,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 195395584. Throughput: 0: 1656.9, 1: 1693.9. Samples: 48858302. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) -[2023-10-14 17:26:13,164][74987] Avg episode reward: [(0, '30.680'), (1, '33.830')] -[2023-10-14 17:26:13,964][75950] Updated weights for policy 1, policy_version 95270 (0.0008) -[2023-10-14 17:26:14,333][75950] Updated weights for policy 1, policy_version 95280 (0.0008) -[2023-10-14 17:26:14,707][75950] Updated weights for policy 1, policy_version 95290 (0.0007) -[2023-10-14 17:26:16,701][75949] Updated weights for policy 0, policy_version 95561 (0.0010) -[2023-10-14 17:26:17,074][75949] Updated weights for policy 0, policy_version 95571 (0.0010) -[2023-10-14 17:26:17,437][75949] Updated weights for policy 0, policy_version 95581 (0.0010) -[2023-10-14 17:26:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195461120. Throughput: 0: 1677.8, 1: 1678.4. Samples: 48868404. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:26:18,165][74987] Avg episode reward: [(0, '24.920'), (1, '34.800')] -[2023-10-14 17:26:18,815][75950] Updated weights for policy 1, policy_version 95300 (0.0009) -[2023-10-14 17:26:19,179][75950] Updated weights for policy 1, policy_version 95310 (0.0007) -[2023-10-14 17:26:19,542][75950] Updated weights for policy 1, policy_version 95320 (0.0008) -[2023-10-14 17:26:21,627][75949] Updated weights for policy 0, policy_version 95591 (0.0011) -[2023-10-14 17:26:21,990][75949] Updated weights for policy 0, policy_version 95601 (0.0010) -[2023-10-14 17:26:22,362][75949] Updated weights for policy 0, policy_version 95611 (0.0009) -[2023-10-14 17:26:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195526656. Throughput: 0: 1667.1, 1: 1695.1. Samples: 48888606. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:26:23,165][74987] Avg episode reward: [(0, '30.150'), (1, '33.290')] -[2023-10-14 17:26:23,346][75950] Updated weights for policy 1, policy_version 95330 (0.0007) -[2023-10-14 17:26:23,721][75950] Updated weights for policy 1, policy_version 95340 (0.0007) -[2023-10-14 17:26:24,085][75950] Updated weights for policy 1, policy_version 95350 (0.0008) -[2023-10-14 17:26:24,447][75950] Updated weights for policy 1, policy_version 95360 (0.0008) -[2023-10-14 17:26:26,572][75949] Updated weights for policy 0, policy_version 95621 (0.0008) -[2023-10-14 17:26:26,941][75949] Updated weights for policy 0, policy_version 95631 (0.0007) -[2023-10-14 17:26:27,310][75949] Updated weights for policy 0, policy_version 95641 (0.0008) -[2023-10-14 17:26:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195592192. Throughput: 0: 1653.9, 1: 1703.8. Samples: 48908650. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:26:28,165][74987] Avg episode reward: [(0, '26.060'), (1, '33.460')] -[2023-10-14 17:26:28,518][75950] Updated weights for policy 1, policy_version 95370 (0.0009) -[2023-10-14 17:26:28,889][75950] Updated weights for policy 1, policy_version 95380 (0.0009) -[2023-10-14 17:26:29,251][75950] Updated weights for policy 1, policy_version 95390 (0.0009) -[2023-10-14 17:26:31,318][75949] Updated weights for policy 0, policy_version 95651 (0.0008) -[2023-10-14 17:26:31,686][75949] Updated weights for policy 0, policy_version 95661 (0.0009) -[2023-10-14 17:26:32,062][75949] Updated weights for policy 0, policy_version 95671 (0.0008) -[2023-10-14 17:26:33,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195657728. Throughput: 0: 1668.8, 1: 1700.0. Samples: 48919176. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:26:33,165][74987] Avg episode reward: [(0, '27.900'), (1, '34.560')] -[2023-10-14 17:26:33,183][75950] Updated weights for policy 1, policy_version 95400 (0.0011) -[2023-10-14 17:26:33,541][75950] Updated weights for policy 1, policy_version 95410 (0.0009) -[2023-10-14 17:26:33,911][75950] Updated weights for policy 1, policy_version 95420 (0.0009) -[2023-10-14 17:26:36,070][75949] Updated weights for policy 0, policy_version 95681 (0.0007) -[2023-10-14 17:26:36,441][75949] Updated weights for policy 0, policy_version 95691 (0.0009) -[2023-10-14 17:26:36,808][75949] Updated weights for policy 0, policy_version 95701 (0.0010) -[2023-10-14 17:26:37,167][75949] Updated weights for policy 0, policy_version 95711 (0.0009) -[2023-10-14 17:26:37,987][75950] Updated weights for policy 1, policy_version 95430 (0.0008) -[2023-10-14 17:26:38,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195723264. Throughput: 0: 1659.6, 1: 1710.0. Samples: 48939610. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:26:38,165][74987] Avg episode reward: [(0, '24.830'), (1, '37.180')] -[2023-10-14 17:26:38,358][75950] Updated weights for policy 1, policy_version 95440 (0.0008) -[2023-10-14 17:26:38,727][75950] Updated weights for policy 1, policy_version 95450 (0.0008) -[2023-10-14 17:26:41,298][75949] Updated weights for policy 0, policy_version 95721 (0.0009) -[2023-10-14 17:26:41,669][75949] Updated weights for policy 0, policy_version 95731 (0.0007) -[2023-10-14 17:26:42,052][75949] Updated weights for policy 0, policy_version 95741 (0.0010) -[2023-10-14 17:26:42,905][75950] Updated weights for policy 1, policy_version 95460 (0.0008) -[2023-10-14 17:26:43,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195788800. Throughput: 0: 1661.1, 1: 1710.8. Samples: 48959560. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:26:43,164][74987] Avg episode reward: [(0, '27.740'), (1, '35.700')] -[2023-10-14 17:26:43,175][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000095744_98041856.pth... -[2023-10-14 17:26:43,210][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000094176_96436224.pth -[2023-10-14 17:26:43,315][75950] Updated weights for policy 1, policy_version 95470 (0.0009) -[2023-10-14 17:26:43,672][75950] Updated weights for policy 1, policy_version 95480 (0.0009) -[2023-10-14 17:26:43,957][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000095488_97779712.pth... -[2023-10-14 17:26:43,997][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000093888_96141312.pth -[2023-10-14 17:26:46,012][75949] Updated weights for policy 0, policy_version 95751 (0.0010) -[2023-10-14 17:26:46,376][75949] Updated weights for policy 0, policy_version 95761 (0.0009) -[2023-10-14 17:26:46,748][75949] Updated weights for policy 0, policy_version 95771 (0.0010) -[2023-10-14 17:26:47,694][75950] Updated weights for policy 1, policy_version 95490 (0.0008) -[2023-10-14 17:26:48,060][75950] Updated weights for policy 1, policy_version 95500 (0.0011) -[2023-10-14 17:26:48,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 195854336. Throughput: 0: 1672.7, 1: 1698.4. Samples: 48969826. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:26:48,164][74987] Avg episode reward: [(0, '25.500'), (1, '34.870')] -[2023-10-14 17:26:48,431][75950] Updated weights for policy 1, policy_version 95510 (0.0011) -[2023-10-14 17:26:48,793][75950] Updated weights for policy 1, policy_version 95520 (0.0009) -[2023-10-14 17:26:50,871][75949] Updated weights for policy 0, policy_version 95781 (0.0008) -[2023-10-14 17:26:51,258][75949] Updated weights for policy 0, policy_version 95791 (0.0009) -[2023-10-14 17:26:51,619][75949] Updated weights for policy 0, policy_version 95801 (0.0008) -[2023-10-14 17:26:53,022][75950] Updated weights for policy 1, policy_version 95530 (0.0007) -[2023-10-14 17:26:53,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 195919872. Throughput: 0: 1660.4, 1: 1691.1. Samples: 48989386. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:26:53,164][74987] Avg episode reward: [(0, '27.880'), (1, '36.510')] -[2023-10-14 17:26:53,392][75950] Updated weights for policy 1, policy_version 95540 (0.0007) -[2023-10-14 17:26:53,745][75950] Updated weights for policy 1, policy_version 95550 (0.0008) -[2023-10-14 17:26:55,602][75949] Updated weights for policy 0, policy_version 95811 (0.0008) -[2023-10-14 17:26:55,980][75949] Updated weights for policy 0, policy_version 95821 (0.0008) -[2023-10-14 17:26:56,350][75949] Updated weights for policy 0, policy_version 95831 (0.0010) -[2023-10-14 17:26:57,779][75950] Updated weights for policy 1, policy_version 95560 (0.0007) -[2023-10-14 17:26:58,142][75950] Updated weights for policy 1, policy_version 95570 (0.0008) -[2023-10-14 17:26:58,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 195985408. Throughput: 0: 1675.9, 1: 1689.9. Samples: 49009762. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:26:58,164][74987] Avg episode reward: [(0, '28.480'), (1, '35.080')] -[2023-10-14 17:26:58,518][75950] Updated weights for policy 1, policy_version 95580 (0.0009) -[2023-10-14 17:27:00,596][75949] Updated weights for policy 0, policy_version 95841 (0.0009) -[2023-10-14 17:27:00,969][75949] Updated weights for policy 0, policy_version 95851 (0.0010) -[2023-10-14 17:27:01,332][75949] Updated weights for policy 0, policy_version 95861 (0.0008) -[2023-10-14 17:27:01,691][75949] Updated weights for policy 0, policy_version 95871 (0.0009) -[2023-10-14 17:27:02,520][75950] Updated weights for policy 1, policy_version 95590 (0.0009) -[2023-10-14 17:27:02,891][75950] Updated weights for policy 1, policy_version 95600 (0.0008) -[2023-10-14 17:27:03,163][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 196050944. Throughput: 0: 1674.9, 1: 1695.3. Samples: 49020062. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:27:03,164][74987] Avg episode reward: [(0, '27.870'), (1, '32.150')] -[2023-10-14 17:27:03,261][75950] Updated weights for policy 1, policy_version 95610 (0.0009) -[2023-10-14 17:27:05,668][75949] Updated weights for policy 0, policy_version 95881 (0.0008) -[2023-10-14 17:27:06,045][75949] Updated weights for policy 0, policy_version 95891 (0.0009) -[2023-10-14 17:27:06,421][75949] Updated weights for policy 0, policy_version 95901 (0.0010) -[2023-10-14 17:27:07,283][75950] Updated weights for policy 1, policy_version 95620 (0.0009) -[2023-10-14 17:27:07,650][75950] Updated weights for policy 1, policy_version 95630 (0.0008) -[2023-10-14 17:27:08,016][75950] Updated weights for policy 1, policy_version 95640 (0.0009) -[2023-10-14 17:27:08,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 196116480. Throughput: 0: 1658.1, 1: 1696.6. Samples: 49039568. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:27:08,165][74987] Avg episode reward: [(0, '29.470'), (1, '33.420')] -[2023-10-14 17:27:10,697][75949] Updated weights for policy 0, policy_version 95911 (0.0007) -[2023-10-14 17:27:11,071][75949] Updated weights for policy 0, policy_version 95921 (0.0007) -[2023-10-14 17:27:11,439][75949] Updated weights for policy 0, policy_version 95931 (0.0009) -[2023-10-14 17:27:12,156][75950] Updated weights for policy 1, policy_version 95650 (0.0007) -[2023-10-14 17:27:12,526][75950] Updated weights for policy 1, policy_version 95660 (0.0008) -[2023-10-14 17:27:12,891][75950] Updated weights for policy 1, policy_version 95670 (0.0010) -[2023-10-14 17:27:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 196182016. Throughput: 0: 1683.4, 1: 1671.8. Samples: 49059634. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-10-14 17:27:13,165][74987] Avg episode reward: [(0, '26.410'), (1, '35.620')] -[2023-10-14 17:27:13,251][75950] Updated weights for policy 1, policy_version 95680 (0.0007) -[2023-10-14 17:27:15,377][75949] Updated weights for policy 0, policy_version 95941 (0.0009) -[2023-10-14 17:27:15,746][75949] Updated weights for policy 0, policy_version 95951 (0.0007) -[2023-10-14 17:27:16,120][75949] Updated weights for policy 0, policy_version 95961 (0.0008) -[2023-10-14 17:27:17,436][75950] Updated weights for policy 1, policy_version 95690 (0.0008) -[2023-10-14 17:27:17,812][75950] Updated weights for policy 1, policy_version 95700 (0.0007) -[2023-10-14 17:27:18,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 196247552. Throughput: 0: 1672.0, 1: 1682.3. Samples: 49070118. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:27:18,164][74987] Avg episode reward: [(0, '27.720'), (1, '33.450')] -[2023-10-14 17:27:18,187][75950] Updated weights for policy 1, policy_version 95710 (0.0008) -[2023-10-14 17:27:20,154][75949] Updated weights for policy 0, policy_version 95971 (0.0010) -[2023-10-14 17:27:20,520][75949] Updated weights for policy 0, policy_version 95981 (0.0008) -[2023-10-14 17:27:20,904][75949] Updated weights for policy 0, policy_version 95991 (0.0008) -[2023-10-14 17:27:22,039][75950] Updated weights for policy 1, policy_version 95720 (0.0009) -[2023-10-14 17:27:22,405][75950] Updated weights for policy 1, policy_version 95730 (0.0008) -[2023-10-14 17:27:22,764][75950] Updated weights for policy 1, policy_version 95740 (0.0009) -[2023-10-14 17:27:23,163][74987] Fps is (10 sec: 16384.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 196345856. Throughput: 0: 1663.3, 1: 1676.7. Samples: 49089906. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:27:23,164][74987] Avg episode reward: [(0, '25.430'), (1, '33.220')] -[2023-10-14 17:27:25,100][75949] Updated weights for policy 0, policy_version 96001 (0.0011) -[2023-10-14 17:27:25,475][75949] Updated weights for policy 0, policy_version 96011 (0.0010) -[2023-10-14 17:27:25,829][75949] Updated weights for policy 0, policy_version 96021 (0.0008) -[2023-10-14 17:27:26,194][75949] Updated weights for policy 0, policy_version 96031 (0.0009) -[2023-10-14 17:27:26,958][75950] Updated weights for policy 1, policy_version 95750 (0.0009) -[2023-10-14 17:27:27,322][75950] Updated weights for policy 1, policy_version 95760 (0.0008) -[2023-10-14 17:27:27,690][75950] Updated weights for policy 1, policy_version 95770 (0.0008) -[2023-10-14 17:27:28,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 196411392. Throughput: 0: 1677.5, 1: 1654.5. Samples: 49109502. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:27:28,164][74987] Avg episode reward: [(0, '29.960'), (1, '36.580')] -[2023-10-14 17:27:30,253][75949] Updated weights for policy 0, policy_version 96041 (0.0010) -[2023-10-14 17:27:30,621][75949] Updated weights for policy 0, policy_version 96051 (0.0007) -[2023-10-14 17:27:30,990][75949] Updated weights for policy 0, policy_version 96061 (0.0007) -[2023-10-14 17:27:31,772][75950] Updated weights for policy 1, policy_version 95780 (0.0008) -[2023-10-14 17:27:32,158][75950] Updated weights for policy 1, policy_version 95790 (0.0010) -[2023-10-14 17:27:32,531][75950] Updated weights for policy 1, policy_version 95800 (0.0007) -[2023-10-14 17:27:33,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 196476928. Throughput: 0: 1657.0, 1: 1683.1. Samples: 49120132. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:27:33,164][74987] Avg episode reward: [(0, '24.320'), (1, '35.330')] -[2023-10-14 17:27:35,131][75949] Updated weights for policy 0, policy_version 96071 (0.0009) -[2023-10-14 17:27:35,502][75949] Updated weights for policy 0, policy_version 96081 (0.0008) -[2023-10-14 17:27:35,869][75949] Updated weights for policy 0, policy_version 96091 (0.0009) -[2023-10-14 17:27:36,387][75950] Updated weights for policy 1, policy_version 95810 (0.0007) -[2023-10-14 17:27:36,751][75950] Updated weights for policy 1, policy_version 95820 (0.0008) -[2023-10-14 17:27:37,114][75950] Updated weights for policy 1, policy_version 95830 (0.0010) -[2023-10-14 17:27:37,487][75950] Updated weights for policy 1, policy_version 95840 (0.0011) -[2023-10-14 17:27:38,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 196542464. Throughput: 0: 1666.7, 1: 1678.5. Samples: 49139918. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:27:38,165][74987] Avg episode reward: [(0, '30.410'), (1, '34.730')] -[2023-10-14 17:27:39,993][75949] Updated weights for policy 0, policy_version 96101 (0.0010) -[2023-10-14 17:27:40,377][75949] Updated weights for policy 0, policy_version 96111 (0.0010) -[2023-10-14 17:27:40,738][75949] Updated weights for policy 0, policy_version 96121 (0.0010) -[2023-10-14 17:27:41,724][75950] Updated weights for policy 1, policy_version 95850 (0.0008) -[2023-10-14 17:27:42,092][75950] Updated weights for policy 1, policy_version 95860 (0.0007) -[2023-10-14 17:27:42,463][75950] Updated weights for policy 1, policy_version 95870 (0.0010) -[2023-10-14 17:27:43,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 196608000. Throughput: 0: 1669.9, 1: 1657.8. Samples: 49159506. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:27:43,164][74987] Avg episode reward: [(0, '25.200'), (1, '34.780')] -[2023-10-14 17:27:44,859][75949] Updated weights for policy 0, policy_version 96131 (0.0008) -[2023-10-14 17:27:45,228][75949] Updated weights for policy 0, policy_version 96141 (0.0007) -[2023-10-14 17:27:45,606][75949] Updated weights for policy 0, policy_version 96151 (0.0007) -[2023-10-14 17:27:46,468][75950] Updated weights for policy 1, policy_version 95880 (0.0010) -[2023-10-14 17:27:46,840][75950] Updated weights for policy 1, policy_version 95890 (0.0010) -[2023-10-14 17:27:47,205][75950] Updated weights for policy 1, policy_version 95900 (0.0007) -[2023-10-14 17:27:48,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 196673536. Throughput: 0: 1651.7, 1: 1685.5. Samples: 49170236. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:27:48,164][74987] Avg episode reward: [(0, '31.500'), (1, '35.160')] -[2023-10-14 17:27:49,664][75949] Updated weights for policy 0, policy_version 96161 (0.0007) -[2023-10-14 17:27:50,038][75949] Updated weights for policy 0, policy_version 96171 (0.0007) -[2023-10-14 17:27:50,406][75949] Updated weights for policy 0, policy_version 96181 (0.0008) -[2023-10-14 17:27:50,774][75949] Updated weights for policy 0, policy_version 96191 (0.0009) -[2023-10-14 17:27:51,344][75950] Updated weights for policy 1, policy_version 95910 (0.0008) -[2023-10-14 17:27:51,708][75950] Updated weights for policy 1, policy_version 95920 (0.0008) -[2023-10-14 17:27:52,071][75950] Updated weights for policy 1, policy_version 95930 (0.0007) -[2023-10-14 17:27:53,164][74987] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 196739072. Throughput: 0: 1667.8, 1: 1672.3. Samples: 49189874. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:27:53,165][74987] Avg episode reward: [(0, '25.060'), (1, '34.510')] -[2023-10-14 17:27:54,829][75949] Updated weights for policy 0, policy_version 96201 (0.0009) -[2023-10-14 17:27:55,201][75949] Updated weights for policy 0, policy_version 96211 (0.0008) -[2023-10-14 17:27:55,577][75949] Updated weights for policy 0, policy_version 96221 (0.0007) -[2023-10-14 17:27:56,036][75950] Updated weights for policy 1, policy_version 95940 (0.0008) -[2023-10-14 17:27:56,401][75950] Updated weights for policy 1, policy_version 95950 (0.0008) -[2023-10-14 17:27:56,775][75950] Updated weights for policy 1, policy_version 95960 (0.0008) -[2023-10-14 17:27:58,163][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 196804608. Throughput: 0: 1669.7, 1: 1671.4. Samples: 49209980. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:27:58,164][74987] Avg episode reward: [(0, '32.640'), (1, '34.790')] -[2023-10-14 17:27:59,605][75949] Updated weights for policy 0, policy_version 96231 (0.0008) -[2023-10-14 17:27:59,983][75949] Updated weights for policy 0, policy_version 96241 (0.0010) -[2023-10-14 17:28:00,364][75949] Updated weights for policy 0, policy_version 96251 (0.0007) -[2023-10-14 17:28:00,780][75950] Updated weights for policy 1, policy_version 95970 (0.0007) -[2023-10-14 17:28:01,143][75950] Updated weights for policy 1, policy_version 95980 (0.0009) -[2023-10-14 17:28:01,509][75950] Updated weights for policy 1, policy_version 95990 (0.0010) -[2023-10-14 17:28:01,871][75950] Updated weights for policy 1, policy_version 96000 (0.0009) -[2023-10-14 17:28:03,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 196870144. Throughput: 0: 1652.7, 1: 1687.5. Samples: 49220428. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:28:03,165][74987] Avg episode reward: [(0, '24.260'), (1, '35.170')] -[2023-10-14 17:28:04,527][75949] Updated weights for policy 0, policy_version 96261 (0.0008) -[2023-10-14 17:28:04,903][75949] Updated weights for policy 0, policy_version 96271 (0.0011) -[2023-10-14 17:28:05,258][75949] Updated weights for policy 0, policy_version 96281 (0.0008) -[2023-10-14 17:28:05,907][75950] Updated weights for policy 1, policy_version 96010 (0.0009) -[2023-10-14 17:28:06,272][75950] Updated weights for policy 1, policy_version 96020 (0.0009) -[2023-10-14 17:28:06,630][75950] Updated weights for policy 1, policy_version 96030 (0.0008) -[2023-10-14 17:28:08,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 196935680. Throughput: 0: 1675.6, 1: 1659.9. Samples: 49240002. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:28:08,164][74987] Avg episode reward: [(0, '32.410'), (1, '35.070')] -[2023-10-14 17:28:09,339][75949] Updated weights for policy 0, policy_version 96291 (0.0008) -[2023-10-14 17:28:09,700][75949] Updated weights for policy 0, policy_version 96301 (0.0011) -[2023-10-14 17:28:10,067][75949] Updated weights for policy 0, policy_version 96311 (0.0010) -[2023-10-14 17:28:10,636][75950] Updated weights for policy 1, policy_version 96040 (0.0008) -[2023-10-14 17:28:10,999][75950] Updated weights for policy 1, policy_version 96050 (0.0007) -[2023-10-14 17:28:11,364][75950] Updated weights for policy 1, policy_version 96060 (0.0010) -[2023-10-14 17:28:13,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 197001216. Throughput: 0: 1677.1, 1: 1679.7. Samples: 49260556. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) -[2023-10-14 17:28:13,164][74987] Avg episode reward: [(0, '24.030'), (1, '35.500')] -[2023-10-14 17:28:14,087][75949] Updated weights for policy 0, policy_version 96321 (0.0010) -[2023-10-14 17:28:14,459][75949] Updated weights for policy 0, policy_version 96331 (0.0007) -[2023-10-14 17:28:14,824][75949] Updated weights for policy 0, policy_version 96341 (0.0011) -[2023-10-14 17:28:15,192][75949] Updated weights for policy 0, policy_version 96351 (0.0008) -[2023-10-14 17:28:15,454][75950] Updated weights for policy 1, policy_version 96070 (0.0010) -[2023-10-14 17:28:15,824][75950] Updated weights for policy 1, policy_version 96080 (0.0007) -[2023-10-14 17:28:16,186][75950] Updated weights for policy 1, policy_version 96090 (0.0008) -[2023-10-14 17:28:18,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 197066752. Throughput: 0: 1667.1, 1: 1678.7. Samples: 49270694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:28:18,165][74987] Avg episode reward: [(0, '31.340'), (1, '36.060')] -[2023-10-14 17:28:19,139][75949] Updated weights for policy 0, policy_version 96361 (0.0009) -[2023-10-14 17:28:19,507][75949] Updated weights for policy 0, policy_version 96371 (0.0010) -[2023-10-14 17:28:19,870][75949] Updated weights for policy 0, policy_version 96381 (0.0011) -[2023-10-14 17:28:20,192][75950] Updated weights for policy 1, policy_version 96100 (0.0012) -[2023-10-14 17:28:20,563][75950] Updated weights for policy 1, policy_version 96110 (0.0010) -[2023-10-14 17:28:20,927][75950] Updated weights for policy 1, policy_version 96120 (0.0009) -[2023-10-14 17:28:23,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 197132288. Throughput: 0: 1679.5, 1: 1665.3. Samples: 49290434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:28:23,165][74987] Avg episode reward: [(0, '23.980'), (1, '36.590')] -[2023-10-14 17:28:24,029][75949] Updated weights for policy 0, policy_version 96391 (0.0008) -[2023-10-14 17:28:24,401][75949] Updated weights for policy 0, policy_version 96401 (0.0007) -[2023-10-14 17:28:24,771][75949] Updated weights for policy 0, policy_version 96411 (0.0009) -[2023-10-14 17:28:25,168][75950] Updated weights for policy 1, policy_version 96130 (0.0010) -[2023-10-14 17:28:25,574][75950] Updated weights for policy 1, policy_version 96140 (0.0009) -[2023-10-14 17:28:25,940][75950] Updated weights for policy 1, policy_version 96150 (0.0008) -[2023-10-14 17:28:26,303][75950] Updated weights for policy 1, policy_version 96160 (0.0010) -[2023-10-14 17:28:28,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 197197824. Throughput: 0: 1687.3, 1: 1684.4. Samples: 49311234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:28:28,164][74987] Avg episode reward: [(0, '30.430'), (1, '36.230')] -[2023-10-14 17:28:28,796][75949] Updated weights for policy 0, policy_version 96421 (0.0010) -[2023-10-14 17:28:29,181][75949] Updated weights for policy 0, policy_version 96431 (0.0008) -[2023-10-14 17:28:29,547][75949] Updated weights for policy 0, policy_version 96441 (0.0009) -[2023-10-14 17:28:30,319][75950] Updated weights for policy 1, policy_version 96170 (0.0008) -[2023-10-14 17:28:30,687][75950] Updated weights for policy 1, policy_version 96180 (0.0007) -[2023-10-14 17:28:31,050][75950] Updated weights for policy 1, policy_version 96190 (0.0008) -[2023-10-14 17:28:33,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 197263360. Throughput: 0: 1679.3, 1: 1668.6. Samples: 49320890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:28:33,164][74987] Avg episode reward: [(0, '23.470'), (1, '35.040')] -[2023-10-14 17:28:33,630][75949] Updated weights for policy 0, policy_version 96451 (0.0007) -[2023-10-14 17:28:33,998][75949] Updated weights for policy 0, policy_version 96461 (0.0007) -[2023-10-14 17:28:34,372][75949] Updated weights for policy 0, policy_version 96471 (0.0009) -[2023-10-14 17:28:35,365][75950] Updated weights for policy 1, policy_version 96200 (0.0008) -[2023-10-14 17:28:35,728][75950] Updated weights for policy 1, policy_version 96210 (0.0007) -[2023-10-14 17:28:36,098][75950] Updated weights for policy 1, policy_version 96220 (0.0007) -[2023-10-14 17:28:38,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 197328896. Throughput: 0: 1689.6, 1: 1663.4. Samples: 49340756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:28:38,165][74987] Avg episode reward: [(0, '30.740'), (1, '35.090')] -[2023-10-14 17:28:38,437][75949] Updated weights for policy 0, policy_version 96481 (0.0007) -[2023-10-14 17:28:38,804][75949] Updated weights for policy 0, policy_version 96491 (0.0008) -[2023-10-14 17:28:39,172][75949] Updated weights for policy 0, policy_version 96501 (0.0010) -[2023-10-14 17:28:39,535][75949] Updated weights for policy 0, policy_version 96511 (0.0009) -[2023-10-14 17:28:40,254][75950] Updated weights for policy 1, policy_version 96230 (0.0007) -[2023-10-14 17:28:40,621][75950] Updated weights for policy 1, policy_version 96240 (0.0008) -[2023-10-14 17:28:40,998][75950] Updated weights for policy 1, policy_version 96250 (0.0009) -[2023-10-14 17:28:43,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197394432. Throughput: 0: 1686.8, 1: 1680.3. Samples: 49361502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:28:43,164][74987] Avg episode reward: [(0, '23.910'), (1, '37.300')] -[2023-10-14 17:28:43,173][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000096256_98566144.pth... -[2023-10-14 17:28:43,173][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000096512_98828288.pth... -[2023-10-14 17:28:43,208][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000094944_97222656.pth -[2023-10-14 17:28:43,213][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000094688_96960512.pth -[2023-10-14 17:28:43,675][75949] Updated weights for policy 0, policy_version 96521 (0.0008) -[2023-10-14 17:28:44,041][75949] Updated weights for policy 0, policy_version 96531 (0.0008) -[2023-10-14 17:28:44,407][75949] Updated weights for policy 0, policy_version 96541 (0.0011) -[2023-10-14 17:28:45,085][75950] Updated weights for policy 1, policy_version 96260 (0.0008) -[2023-10-14 17:28:45,455][75950] Updated weights for policy 1, policy_version 96270 (0.0007) -[2023-10-14 17:28:45,828][75950] Updated weights for policy 1, policy_version 96280 (0.0008) -[2023-10-14 17:28:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197459968. Throughput: 0: 1685.0, 1: 1665.5. Samples: 49371200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:28:48,164][74987] Avg episode reward: [(0, '27.020'), (1, '36.900')] -[2023-10-14 17:28:48,439][75949] Updated weights for policy 0, policy_version 96551 (0.0011) -[2023-10-14 17:28:48,802][75949] Updated weights for policy 0, policy_version 96561 (0.0008) -[2023-10-14 17:28:49,178][75949] Updated weights for policy 0, policy_version 96571 (0.0008) -[2023-10-14 17:28:49,884][75950] Updated weights for policy 1, policy_version 96290 (0.0009) -[2023-10-14 17:28:50,242][75950] Updated weights for policy 1, policy_version 96300 (0.0008) -[2023-10-14 17:28:50,622][75950] Updated weights for policy 1, policy_version 96310 (0.0008) -[2023-10-14 17:28:50,996][75950] Updated weights for policy 1, policy_version 96320 (0.0009) -[2023-10-14 17:28:53,050][75949] Updated weights for policy 0, policy_version 96581 (0.0010) -[2023-10-14 17:28:53,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197525504. Throughput: 0: 1684.6, 1: 1679.7. Samples: 49391394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:28:53,165][74987] Avg episode reward: [(0, '22.830'), (1, '36.190')] -[2023-10-14 17:28:53,423][75949] Updated weights for policy 0, policy_version 96591 (0.0009) -[2023-10-14 17:28:53,797][75949] Updated weights for policy 0, policy_version 96601 (0.0008) -[2023-10-14 17:28:54,980][75950] Updated weights for policy 1, policy_version 96330 (0.0008) -[2023-10-14 17:28:55,349][75950] Updated weights for policy 1, policy_version 96340 (0.0007) -[2023-10-14 17:28:55,711][75950] Updated weights for policy 1, policy_version 96350 (0.0007) -[2023-10-14 17:28:57,901][75949] Updated weights for policy 0, policy_version 96611 (0.0009) -[2023-10-14 17:28:58,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197591040. Throughput: 0: 1687.6, 1: 1682.2. Samples: 49412198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:28:58,164][74987] Avg episode reward: [(0, '27.490'), (1, '35.680')] -[2023-10-14 17:28:58,276][75949] Updated weights for policy 0, policy_version 96621 (0.0009) -[2023-10-14 17:28:58,650][75949] Updated weights for policy 0, policy_version 96631 (0.0009) -[2023-10-14 17:28:59,668][75950] Updated weights for policy 1, policy_version 96360 (0.0008) -[2023-10-14 17:29:00,038][75950] Updated weights for policy 1, policy_version 96370 (0.0007) -[2023-10-14 17:29:00,396][75950] Updated weights for policy 1, policy_version 96380 (0.0008) -[2023-10-14 17:29:02,819][75949] Updated weights for policy 0, policy_version 96641 (0.0009) -[2023-10-14 17:29:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197656576. Throughput: 0: 1685.9, 1: 1665.8. Samples: 49421520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:03,164][74987] Avg episode reward: [(0, '26.130'), (1, '36.450')] -[2023-10-14 17:29:03,178][75949] Updated weights for policy 0, policy_version 96651 (0.0008) -[2023-10-14 17:29:03,556][75949] Updated weights for policy 0, policy_version 96661 (0.0010) -[2023-10-14 17:29:03,930][75949] Updated weights for policy 0, policy_version 96671 (0.0008) -[2023-10-14 17:29:04,589][75950] Updated weights for policy 1, policy_version 96390 (0.0008) -[2023-10-14 17:29:04,952][75950] Updated weights for policy 1, policy_version 96400 (0.0007) -[2023-10-14 17:29:05,319][75950] Updated weights for policy 1, policy_version 96410 (0.0009) -[2023-10-14 17:29:08,090][75949] Updated weights for policy 0, policy_version 96681 (0.0008) -[2023-10-14 17:29:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197722112. Throughput: 0: 1686.1, 1: 1684.1. Samples: 49442092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:08,165][74987] Avg episode reward: [(0, '28.010'), (1, '34.270')] -[2023-10-14 17:29:08,447][75949] Updated weights for policy 0, policy_version 96691 (0.0007) -[2023-10-14 17:29:08,823][75949] Updated weights for policy 0, policy_version 96701 (0.0007) -[2023-10-14 17:29:09,267][75950] Updated weights for policy 1, policy_version 96420 (0.0009) -[2023-10-14 17:29:09,636][75950] Updated weights for policy 1, policy_version 96430 (0.0009) -[2023-10-14 17:29:09,996][75950] Updated weights for policy 1, policy_version 96440 (0.0009) -[2023-10-14 17:29:12,723][75949] Updated weights for policy 0, policy_version 96711 (0.0007) -[2023-10-14 17:29:13,084][75949] Updated weights for policy 0, policy_version 96721 (0.0007) -[2023-10-14 17:29:13,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197787648. Throughput: 0: 1682.3, 1: 1685.5. Samples: 49462782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:13,164][74987] Avg episode reward: [(0, '24.310'), (1, '35.280')] -[2023-10-14 17:29:13,457][75949] Updated weights for policy 0, policy_version 96731 (0.0009) -[2023-10-14 17:29:14,009][75950] Updated weights for policy 1, policy_version 96450 (0.0010) -[2023-10-14 17:29:14,430][75950] Updated weights for policy 1, policy_version 96460 (0.0008) -[2023-10-14 17:29:14,797][75950] Updated weights for policy 1, policy_version 96470 (0.0008) -[2023-10-14 17:29:15,163][75950] Updated weights for policy 1, policy_version 96480 (0.0009) -[2023-10-14 17:29:17,524][75949] Updated weights for policy 0, policy_version 96741 (0.0008) -[2023-10-14 17:29:17,900][75949] Updated weights for policy 0, policy_version 96751 (0.0008) -[2023-10-14 17:29:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197853184. Throughput: 0: 1692.5, 1: 1666.3. Samples: 49472038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:18,165][74987] Avg episode reward: [(0, '25.480'), (1, '37.830')] -[2023-10-14 17:29:18,271][75949] Updated weights for policy 0, policy_version 96761 (0.0007) -[2023-10-14 17:29:19,179][75950] Updated weights for policy 1, policy_version 96490 (0.0008) -[2023-10-14 17:29:19,543][75950] Updated weights for policy 1, policy_version 96500 (0.0009) -[2023-10-14 17:29:19,920][75950] Updated weights for policy 1, policy_version 96510 (0.0010) -[2023-10-14 17:29:22,285][75949] Updated weights for policy 0, policy_version 96771 (0.0009) -[2023-10-14 17:29:22,650][75949] Updated weights for policy 0, policy_version 96781 (0.0009) -[2023-10-14 17:29:23,017][75949] Updated weights for policy 0, policy_version 96791 (0.0010) -[2023-10-14 17:29:23,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 197918720. Throughput: 0: 1695.2, 1: 1685.9. Samples: 49492906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:23,165][74987] Avg episode reward: [(0, '26.130'), (1, '39.540')] -[2023-10-14 17:29:23,955][75950] Updated weights for policy 1, policy_version 96520 (0.0009) -[2023-10-14 17:29:24,320][75950] Updated weights for policy 1, policy_version 96530 (0.0007) -[2023-10-14 17:29:24,687][75950] Updated weights for policy 1, policy_version 96540 (0.0007) -[2023-10-14 17:29:27,114][75949] Updated weights for policy 0, policy_version 96801 (0.0008) -[2023-10-14 17:29:27,487][75949] Updated weights for policy 0, policy_version 96811 (0.0010) -[2023-10-14 17:29:27,851][75949] Updated weights for policy 0, policy_version 96821 (0.0009) -[2023-10-14 17:29:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 197984256. Throughput: 0: 1678.6, 1: 1686.4. Samples: 49512926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:28,165][74987] Avg episode reward: [(0, '26.040'), (1, '38.440')] -[2023-10-14 17:29:28,232][75949] Updated weights for policy 0, policy_version 96831 (0.0010) -[2023-10-14 17:29:28,830][75950] Updated weights for policy 1, policy_version 96550 (0.0008) -[2023-10-14 17:29:29,197][75950] Updated weights for policy 1, policy_version 96560 (0.0007) -[2023-10-14 17:29:29,560][75950] Updated weights for policy 1, policy_version 96570 (0.0007) -[2023-10-14 17:29:32,275][75949] Updated weights for policy 0, policy_version 96841 (0.0010) -[2023-10-14 17:29:32,643][75949] Updated weights for policy 0, policy_version 96851 (0.0009) -[2023-10-14 17:29:33,013][75949] Updated weights for policy 0, policy_version 96861 (0.0007) -[2023-10-14 17:29:33,164][74987] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198082560. Throughput: 0: 1691.9, 1: 1673.3. Samples: 49522634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:33,165][74987] Avg episode reward: [(0, '28.100'), (1, '36.730')] -[2023-10-14 17:29:33,809][75950] Updated weights for policy 1, policy_version 96580 (0.0008) -[2023-10-14 17:29:34,179][75950] Updated weights for policy 1, policy_version 96590 (0.0008) -[2023-10-14 17:29:34,550][75950] Updated weights for policy 1, policy_version 96600 (0.0010) -[2023-10-14 17:29:37,175][75949] Updated weights for policy 0, policy_version 96871 (0.0007) -[2023-10-14 17:29:37,548][75949] Updated weights for policy 0, policy_version 96881 (0.0008) -[2023-10-14 17:29:37,919][75949] Updated weights for policy 0, policy_version 96891 (0.0009) -[2023-10-14 17:29:38,164][74987] Fps is (10 sec: 16384.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 198148096. Throughput: 0: 1688.4, 1: 1686.0. Samples: 49543242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:38,165][74987] Avg episode reward: [(0, '25.430'), (1, '37.550')] -[2023-10-14 17:29:38,780][75950] Updated weights for policy 1, policy_version 96610 (0.0011) -[2023-10-14 17:29:39,152][75950] Updated weights for policy 1, policy_version 96620 (0.0010) -[2023-10-14 17:29:39,513][75950] Updated weights for policy 1, policy_version 96630 (0.0008) -[2023-10-14 17:29:39,877][75950] Updated weights for policy 1, policy_version 96640 (0.0008) -[2023-10-14 17:29:42,022][75949] Updated weights for policy 0, policy_version 96901 (0.0010) -[2023-10-14 17:29:42,393][75949] Updated weights for policy 0, policy_version 96911 (0.0008) -[2023-10-14 17:29:42,769][75949] Updated weights for policy 0, policy_version 96921 (0.0007) -[2023-10-14 17:29:43,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 198213632. Throughput: 0: 1663.8, 1: 1684.8. Samples: 49562888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:43,164][74987] Avg episode reward: [(0, '27.740'), (1, '35.870')] -[2023-10-14 17:29:44,042][75950] Updated weights for policy 1, policy_version 96650 (0.0011) -[2023-10-14 17:29:44,400][75950] Updated weights for policy 1, policy_version 96660 (0.0010) -[2023-10-14 17:29:44,775][75950] Updated weights for policy 1, policy_version 96670 (0.0007) -[2023-10-14 17:29:46,752][75949] Updated weights for policy 0, policy_version 96931 (0.0010) -[2023-10-14 17:29:47,133][75949] Updated weights for policy 0, policy_version 96941 (0.0007) -[2023-10-14 17:29:47,502][75949] Updated weights for policy 0, policy_version 96951 (0.0008) -[2023-10-14 17:29:48,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198279168. Throughput: 0: 1680.7, 1: 1679.6. Samples: 49572734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:48,164][74987] Avg episode reward: [(0, '24.190'), (1, '32.690')] -[2023-10-14 17:29:48,704][75950] Updated weights for policy 1, policy_version 96680 (0.0008) -[2023-10-14 17:29:49,075][75950] Updated weights for policy 1, policy_version 96690 (0.0008) -[2023-10-14 17:29:49,451][75950] Updated weights for policy 1, policy_version 96700 (0.0009) -[2023-10-14 17:29:51,681][75949] Updated weights for policy 0, policy_version 96961 (0.0008) -[2023-10-14 17:29:52,064][75949] Updated weights for policy 0, policy_version 96971 (0.0008) -[2023-10-14 17:29:52,437][75949] Updated weights for policy 0, policy_version 96981 (0.0009) -[2023-10-14 17:29:52,797][75949] Updated weights for policy 0, policy_version 96991 (0.0008) -[2023-10-14 17:29:53,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198344704. Throughput: 0: 1683.4, 1: 1681.2. Samples: 49593500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:53,165][74987] Avg episode reward: [(0, '28.550'), (1, '35.720')] -[2023-10-14 17:29:53,519][75950] Updated weights for policy 1, policy_version 96710 (0.0008) -[2023-10-14 17:29:53,880][75950] Updated weights for policy 1, policy_version 96720 (0.0009) -[2023-10-14 17:29:54,251][75950] Updated weights for policy 1, policy_version 96730 (0.0009) -[2023-10-14 17:29:56,828][75949] Updated weights for policy 0, policy_version 97001 (0.0008) -[2023-10-14 17:29:57,198][75949] Updated weights for policy 0, policy_version 97011 (0.0009) -[2023-10-14 17:29:57,575][75949] Updated weights for policy 0, policy_version 97021 (0.0008) -[2023-10-14 17:29:58,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198410240. Throughput: 0: 1655.4, 1: 1687.9. Samples: 49613230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:29:58,165][74987] Avg episode reward: [(0, '24.460'), (1, '36.300')] -[2023-10-14 17:29:58,350][75950] Updated weights for policy 1, policy_version 96740 (0.0008) -[2023-10-14 17:29:58,723][75950] Updated weights for policy 1, policy_version 96750 (0.0009) -[2023-10-14 17:29:59,096][75950] Updated weights for policy 1, policy_version 96760 (0.0008) -[2023-10-14 17:30:01,593][75949] Updated weights for policy 0, policy_version 97031 (0.0009) -[2023-10-14 17:30:01,972][75949] Updated weights for policy 0, policy_version 97041 (0.0008) -[2023-10-14 17:30:02,348][75949] Updated weights for policy 0, policy_version 97051 (0.0009) -[2023-10-14 17:30:03,031][75950] Updated weights for policy 1, policy_version 96770 (0.0009) -[2023-10-14 17:30:03,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198475776. Throughput: 0: 1677.3, 1: 1693.0. Samples: 49623698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:30:03,164][74987] Avg episode reward: [(0, '29.830'), (1, '35.670')] -[2023-10-14 17:30:03,448][75950] Updated weights for policy 1, policy_version 96780 (0.0010) -[2023-10-14 17:30:03,828][75950] Updated weights for policy 1, policy_version 96790 (0.0009) -[2023-10-14 17:30:04,191][75950] Updated weights for policy 1, policy_version 96800 (0.0009) -[2023-10-14 17:30:06,540][75949] Updated weights for policy 0, policy_version 97061 (0.0008) -[2023-10-14 17:30:06,913][75949] Updated weights for policy 0, policy_version 97071 (0.0009) -[2023-10-14 17:30:07,284][75949] Updated weights for policy 0, policy_version 97081 (0.0007) -[2023-10-14 17:30:08,058][75950] Updated weights for policy 1, policy_version 96810 (0.0009) -[2023-10-14 17:30:08,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 198541312. Throughput: 0: 1663.1, 1: 1690.3. Samples: 49643810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:30:08,164][74987] Avg episode reward: [(0, '24.350'), (1, '35.610')] -[2023-10-14 17:30:08,420][75950] Updated weights for policy 1, policy_version 96820 (0.0008) -[2023-10-14 17:30:08,786][75950] Updated weights for policy 1, policy_version 96830 (0.0007) -[2023-10-14 17:30:11,197][75949] Updated weights for policy 0, policy_version 97091 (0.0010) -[2023-10-14 17:30:11,562][75949] Updated weights for policy 0, policy_version 97101 (0.0011) -[2023-10-14 17:30:11,932][75949] Updated weights for policy 0, policy_version 97111 (0.0007) -[2023-10-14 17:30:12,885][75950] Updated weights for policy 1, policy_version 96840 (0.0009) -[2023-10-14 17:30:13,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198606848. Throughput: 0: 1662.0, 1: 1687.1. Samples: 49663636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:30:13,164][74987] Avg episode reward: [(0, '30.800'), (1, '38.810')] -[2023-10-14 17:30:13,245][75950] Updated weights for policy 1, policy_version 96850 (0.0009) -[2023-10-14 17:30:13,607][75950] Updated weights for policy 1, policy_version 96860 (0.0009) -[2023-10-14 17:30:16,108][75949] Updated weights for policy 0, policy_version 97121 (0.0007) -[2023-10-14 17:30:16,482][75949] Updated weights for policy 0, policy_version 97131 (0.0008) -[2023-10-14 17:30:16,847][75949] Updated weights for policy 0, policy_version 97141 (0.0009) -[2023-10-14 17:30:17,223][75949] Updated weights for policy 0, policy_version 97151 (0.0010) -[2023-10-14 17:30:17,712][75950] Updated weights for policy 1, policy_version 96870 (0.0009) -[2023-10-14 17:30:18,070][75950] Updated weights for policy 1, policy_version 96880 (0.0010) -[2023-10-14 17:30:18,163][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 198672384. Throughput: 0: 1677.2, 1: 1688.1. Samples: 49674070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) -[2023-10-14 17:30:18,164][74987] Avg episode reward: [(0, '24.970'), (1, '37.540')] -[2023-10-14 17:30:18,436][75950] Updated weights for policy 1, policy_version 96890 (0.0009) -[2023-10-14 17:30:21,442][75949] Updated weights for policy 0, policy_version 97161 (0.0008) -[2023-10-14 17:30:21,821][75949] Updated weights for policy 0, policy_version 97171 (0.0008) -[2023-10-14 17:30:22,185][75949] Updated weights for policy 0, policy_version 97181 (0.0007) -[2023-10-14 17:30:22,672][75950] Updated weights for policy 1, policy_version 96900 (0.0009) -[2023-10-14 17:30:23,043][75950] Updated weights for policy 1, policy_version 96910 (0.0008) -[2023-10-14 17:30:23,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 198737920. Throughput: 0: 1663.0, 1: 1682.8. Samples: 49693802. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:30:23,164][74987] Avg episode reward: [(0, '31.350'), (1, '36.650')] -[2023-10-14 17:30:23,408][75950] Updated weights for policy 1, policy_version 96920 (0.0008) -[2023-10-14 17:30:26,342][75949] Updated weights for policy 0, policy_version 97191 (0.0009) -[2023-10-14 17:30:26,705][75949] Updated weights for policy 0, policy_version 97201 (0.0009) -[2023-10-14 17:30:27,069][75949] Updated weights for policy 0, policy_version 97211 (0.0009) -[2023-10-14 17:30:27,493][75950] Updated weights for policy 1, policy_version 96930 (0.0007) -[2023-10-14 17:30:27,867][75950] Updated weights for policy 1, policy_version 96940 (0.0008) -[2023-10-14 17:30:28,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 198803456. Throughput: 0: 1667.9, 1: 1679.0. Samples: 49713500. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:30:28,164][74987] Avg episode reward: [(0, '25.780'), (1, '37.600')] -[2023-10-14 17:30:28,230][75950] Updated weights for policy 1, policy_version 96950 (0.0009) -[2023-10-14 17:30:28,588][75950] Updated weights for policy 1, policy_version 96960 (0.0009) -[2023-10-14 17:30:31,125][75949] Updated weights for policy 0, policy_version 97221 (0.0008) -[2023-10-14 17:30:31,496][75949] Updated weights for policy 0, policy_version 97231 (0.0007) -[2023-10-14 17:30:31,862][75949] Updated weights for policy 0, policy_version 97241 (0.0008) -[2023-10-14 17:30:32,679][75950] Updated weights for policy 1, policy_version 96970 (0.0008) -[2023-10-14 17:30:33,039][75950] Updated weights for policy 1, policy_version 96980 (0.0009) -[2023-10-14 17:30:33,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 198868992. Throughput: 0: 1680.5, 1: 1685.2. Samples: 49724190. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:30:33,165][74987] Avg episode reward: [(0, '30.580'), (1, '37.680')] -[2023-10-14 17:30:33,413][75950] Updated weights for policy 1, policy_version 96990 (0.0007) -[2023-10-14 17:30:35,907][75949] Updated weights for policy 0, policy_version 97251 (0.0009) -[2023-10-14 17:30:36,282][75949] Updated weights for policy 0, policy_version 97261 (0.0010) -[2023-10-14 17:30:36,643][75949] Updated weights for policy 0, policy_version 97271 (0.0010) -[2023-10-14 17:30:37,492][75950] Updated weights for policy 1, policy_version 97000 (0.0008) -[2023-10-14 17:30:37,857][75950] Updated weights for policy 1, policy_version 97010 (0.0009) -[2023-10-14 17:30:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 198934528. Throughput: 0: 1659.0, 1: 1684.7. Samples: 49743966. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:30:38,164][74987] Avg episode reward: [(0, '24.750'), (1, '37.540')] -[2023-10-14 17:30:38,230][75950] Updated weights for policy 1, policy_version 97020 (0.0009) -[2023-10-14 17:30:40,652][75949] Updated weights for policy 0, policy_version 97281 (0.0008) -[2023-10-14 17:30:41,010][75949] Updated weights for policy 0, policy_version 97291 (0.0010) -[2023-10-14 17:30:41,378][75949] Updated weights for policy 0, policy_version 97301 (0.0011) -[2023-10-14 17:30:41,751][75949] Updated weights for policy 0, policy_version 97311 (0.0008) -[2023-10-14 17:30:42,204][75950] Updated weights for policy 1, policy_version 97030 (0.0008) -[2023-10-14 17:30:42,584][75950] Updated weights for policy 1, policy_version 97040 (0.0007) -[2023-10-14 17:30:42,946][75950] Updated weights for policy 1, policy_version 97050 (0.0010) -[2023-10-14 17:30:43,163][74987] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 199032832. Throughput: 0: 1680.9, 1: 1667.2. Samples: 49763896. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:30:43,164][74987] Avg episode reward: [(0, '31.430'), (1, '33.780')] -[2023-10-14 17:30:43,172][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000097056_99385344.pth... -[2023-10-14 17:30:43,172][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000097312_99647488.pth... -[2023-10-14 17:30:43,207][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000095744_98041856.pth -[2023-10-14 17:30:43,212][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000095488_97779712.pth -[2023-10-14 17:30:46,005][75949] Updated weights for policy 0, policy_version 97321 (0.0010) -[2023-10-14 17:30:46,384][75949] Updated weights for policy 0, policy_version 97331 (0.0009) -[2023-10-14 17:30:46,762][75949] Updated weights for policy 0, policy_version 97341 (0.0009) -[2023-10-14 17:30:47,012][75950] Updated weights for policy 1, policy_version 97060 (0.0010) -[2023-10-14 17:30:47,382][75950] Updated weights for policy 1, policy_version 97070 (0.0009) -[2023-10-14 17:30:47,744][75950] Updated weights for policy 1, policy_version 97080 (0.0009) -[2023-10-14 17:30:48,164][74987] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 199098368. Throughput: 0: 1677.8, 1: 1681.2. Samples: 49774852. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:30:48,165][74987] Avg episode reward: [(0, '26.110'), (1, '33.450')] -[2023-10-14 17:30:50,759][75949] Updated weights for policy 0, policy_version 97351 (0.0010) -[2023-10-14 17:30:51,141][75949] Updated weights for policy 0, policy_version 97361 (0.0008) -[2023-10-14 17:30:51,512][75949] Updated weights for policy 0, policy_version 97371 (0.0008) -[2023-10-14 17:30:51,951][75950] Updated weights for policy 1, policy_version 97090 (0.0008) -[2023-10-14 17:30:52,378][75950] Updated weights for policy 1, policy_version 97100 (0.0010) -[2023-10-14 17:30:52,741][75950] Updated weights for policy 1, policy_version 97110 (0.0008) -[2023-10-14 17:30:53,106][75950] Updated weights for policy 1, policy_version 97120 (0.0007) -[2023-10-14 17:30:53,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 199163904. Throughput: 0: 1662.9, 1: 1678.3. Samples: 49794164. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:30:53,165][74987] Avg episode reward: [(0, '30.460'), (1, '35.960')] -[2023-10-14 17:30:55,601][75949] Updated weights for policy 0, policy_version 97381 (0.0010) -[2023-10-14 17:30:55,997][75949] Updated weights for policy 0, policy_version 97391 (0.0009) -[2023-10-14 17:30:56,368][75949] Updated weights for policy 0, policy_version 97401 (0.0009) -[2023-10-14 17:30:57,035][75950] Updated weights for policy 1, policy_version 97130 (0.0009) -[2023-10-14 17:30:57,398][75950] Updated weights for policy 1, policy_version 97140 (0.0008) -[2023-10-14 17:30:57,777][75950] Updated weights for policy 1, policy_version 97150 (0.0007) -[2023-10-14 17:30:58,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 199229440. Throughput: 0: 1680.5, 1: 1652.1. Samples: 49813606. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:30:58,164][74987] Avg episode reward: [(0, '26.500'), (1, '35.140')] -[2023-10-14 17:31:00,367][75949] Updated weights for policy 0, policy_version 97411 (0.0011) -[2023-10-14 17:31:00,744][75949] Updated weights for policy 0, policy_version 97421 (0.0009) -[2023-10-14 17:31:01,116][75949] Updated weights for policy 0, policy_version 97431 (0.0010) -[2023-10-14 17:31:01,765][75950] Updated weights for policy 1, policy_version 97160 (0.0008) -[2023-10-14 17:31:02,126][75950] Updated weights for policy 1, policy_version 97170 (0.0009) -[2023-10-14 17:31:02,499][75950] Updated weights for policy 1, policy_version 97180 (0.0008) -[2023-10-14 17:31:03,163][74987] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 199294976. Throughput: 0: 1669.3, 1: 1676.0. Samples: 49824608. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:31:03,164][74987] Avg episode reward: [(0, '27.880'), (1, '35.800')] -[2023-10-14 17:31:05,045][75949] Updated weights for policy 0, policy_version 97441 (0.0009) -[2023-10-14 17:31:05,421][75949] Updated weights for policy 0, policy_version 97451 (0.0008) -[2023-10-14 17:31:05,795][75949] Updated weights for policy 0, policy_version 97461 (0.0009) -[2023-10-14 17:31:06,160][75949] Updated weights for policy 0, policy_version 97471 (0.0008) -[2023-10-14 17:31:06,471][75950] Updated weights for policy 1, policy_version 97190 (0.0009) -[2023-10-14 17:31:06,836][75950] Updated weights for policy 1, policy_version 97200 (0.0009) -[2023-10-14 17:31:07,201][75950] Updated weights for policy 1, policy_version 97210 (0.0009) -[2023-10-14 17:31:08,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 199360512. Throughput: 0: 1668.7, 1: 1672.0. Samples: 49844136. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:31:08,165][74987] Avg episode reward: [(0, '26.190'), (1, '36.460')] -[2023-10-14 17:31:10,254][75949] Updated weights for policy 0, policy_version 97481 (0.0008) -[2023-10-14 17:31:10,639][75949] Updated weights for policy 0, policy_version 97491 (0.0008) -[2023-10-14 17:31:11,013][75949] Updated weights for policy 0, policy_version 97501 (0.0008) -[2023-10-14 17:31:11,319][75950] Updated weights for policy 1, policy_version 97220 (0.0008) -[2023-10-14 17:31:11,678][75950] Updated weights for policy 1, policy_version 97230 (0.0010) -[2023-10-14 17:31:12,050][75950] Updated weights for policy 1, policy_version 97240 (0.0009) -[2023-10-14 17:31:13,164][74987] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 199426048. Throughput: 0: 1690.3, 1: 1658.2. Samples: 49864180. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:31:13,165][74987] Avg episode reward: [(0, '29.840'), (1, '37.510')] -[2023-10-14 17:31:14,850][75949] Updated weights for policy 0, policy_version 97511 (0.0008) -[2023-10-14 17:31:15,212][75949] Updated weights for policy 0, policy_version 97521 (0.0009) -[2023-10-14 17:31:15,586][75949] Updated weights for policy 0, policy_version 97531 (0.0009) -[2023-10-14 17:31:16,314][75950] Updated weights for policy 1, policy_version 97250 (0.0008) -[2023-10-14 17:31:16,683][75950] Updated weights for policy 1, policy_version 97260 (0.0009) -[2023-10-14 17:31:17,051][75950] Updated weights for policy 1, policy_version 97270 (0.0008) -[2023-10-14 17:31:17,418][75950] Updated weights for policy 1, policy_version 97280 (0.0009) -[2023-10-14 17:31:18,163][74987] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 199491584. Throughput: 0: 1662.9, 1: 1679.0. Samples: 49874578. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:31:18,164][74987] Avg episode reward: [(0, '26.530'), (1, '36.590')] -[2023-10-14 17:31:19,708][75949] Updated weights for policy 0, policy_version 97541 (0.0009) -[2023-10-14 17:31:20,071][75949] Updated weights for policy 0, policy_version 97551 (0.0008) -[2023-10-14 17:31:20,443][75949] Updated weights for policy 0, policy_version 97561 (0.0009) -[2023-10-14 17:31:21,345][75950] Updated weights for policy 1, policy_version 97290 (0.0009) -[2023-10-14 17:31:21,725][75950] Updated weights for policy 1, policy_version 97300 (0.0010) -[2023-10-14 17:31:22,090][75950] Updated weights for policy 1, policy_version 97310 (0.0011) -[2023-10-14 17:31:23,164][74987] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 199557120. Throughput: 0: 1678.7, 1: 1669.2. Samples: 49894618. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-10-14 17:31:23,165][74987] Avg episode reward: [(0, '29.140'), (1, '35.010')] -[2023-10-14 17:31:24,475][75949] Updated weights for policy 0, policy_version 97571 (0.0008) -[2023-10-14 17:31:24,847][75949] Updated weights for policy 0, policy_version 97581 (0.0009) -[2023-10-14 17:31:25,219][75949] Updated weights for policy 0, policy_version 97591 (0.0009) -[2023-10-14 17:31:26,293][75950] Updated weights for policy 1, policy_version 97320 (0.0008) -[2023-10-14 17:31:26,658][75950] Updated weights for policy 1, policy_version 97330 (0.0007) -[2023-10-14 17:31:27,024][75950] Updated weights for policy 1, policy_version 97340 (0.0009) -[2023-10-14 17:31:28,164][74987] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 199622656. Throughput: 0: 1687.8, 1: 1662.2. Samples: 49914646. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 17:31:28,165][74987] Avg episode reward: [(0, '26.180'), (1, '35.250')] -[2023-10-14 17:31:29,115][75949] Updated weights for policy 0, policy_version 97601 (0.0009) -[2023-10-14 17:31:29,483][75949] Updated weights for policy 0, policy_version 97611 (0.0009) -[2023-10-14 17:31:29,858][75949] Updated weights for policy 0, policy_version 97621 (0.0011) -[2023-10-14 17:31:30,226][75949] Updated weights for policy 0, policy_version 97631 (0.0011) -[2023-10-14 17:31:31,084][75950] Updated weights for policy 1, policy_version 97350 (0.0008) -[2023-10-14 17:31:31,442][75950] Updated weights for policy 1, policy_version 97360 (0.0011) -[2023-10-14 17:31:31,813][75950] Updated weights for policy 1, policy_version 97370 (0.0008) -[2023-10-14 17:31:33,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 199688192. Throughput: 0: 1663.3, 1: 1679.6. Samples: 49925278. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 17:31:33,164][74987] Avg episode reward: [(0, '28.720'), (1, '37.120')] -[2023-10-14 17:31:34,217][75949] Updated weights for policy 0, policy_version 97641 (0.0010) -[2023-10-14 17:31:34,586][75949] Updated weights for policy 0, policy_version 97651 (0.0008) -[2023-10-14 17:31:34,953][75949] Updated weights for policy 0, policy_version 97661 (0.0007) -[2023-10-14 17:31:35,825][75950] Updated weights for policy 1, policy_version 97380 (0.0009) -[2023-10-14 17:31:36,184][75950] Updated weights for policy 1, policy_version 97390 (0.0008) -[2023-10-14 17:31:36,554][75950] Updated weights for policy 1, policy_version 97400 (0.0008) -[2023-10-14 17:31:38,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 199753728. Throughput: 0: 1690.0, 1: 1664.4. Samples: 49945116. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 17:31:38,165][74987] Avg episode reward: [(0, '25.610'), (1, '34.060')] -[2023-10-14 17:31:38,983][75949] Updated weights for policy 0, policy_version 97671 (0.0010) -[2023-10-14 17:31:39,356][75949] Updated weights for policy 0, policy_version 97681 (0.0009) -[2023-10-14 17:31:39,717][75949] Updated weights for policy 0, policy_version 97691 (0.0008) -[2023-10-14 17:31:40,714][75950] Updated weights for policy 1, policy_version 97410 (0.0007) -[2023-10-14 17:31:41,121][75950] Updated weights for policy 1, policy_version 97420 (0.0008) -[2023-10-14 17:31:41,477][75950] Updated weights for policy 1, policy_version 97430 (0.0009) -[2023-10-14 17:31:41,845][75950] Updated weights for policy 1, policy_version 97440 (0.0007) -[2023-10-14 17:31:43,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 199819264. Throughput: 0: 1693.1, 1: 1680.3. Samples: 49965408. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 17:31:43,164][74987] Avg episode reward: [(0, '26.970'), (1, '33.190')] -[2023-10-14 17:31:43,996][75949] Updated weights for policy 0, policy_version 97701 (0.0008) -[2023-10-14 17:31:44,383][75949] Updated weights for policy 0, policy_version 97711 (0.0007) -[2023-10-14 17:31:44,764][75949] Updated weights for policy 0, policy_version 97721 (0.0009) -[2023-10-14 17:31:45,987][75950] Updated weights for policy 1, policy_version 97450 (0.0008) -[2023-10-14 17:31:46,359][75950] Updated weights for policy 1, policy_version 97460 (0.0010) -[2023-10-14 17:31:46,724][75950] Updated weights for policy 1, policy_version 97470 (0.0008) -[2023-10-14 17:31:48,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 199884800. Throughput: 0: 1675.3, 1: 1682.6. Samples: 49975714. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 17:31:48,165][74987] Avg episode reward: [(0, '24.680'), (1, '36.490')] -[2023-10-14 17:31:48,756][75949] Updated weights for policy 0, policy_version 97731 (0.0008) -[2023-10-14 17:31:49,121][75949] Updated weights for policy 0, policy_version 97741 (0.0007) -[2023-10-14 17:31:49,484][75949] Updated weights for policy 0, policy_version 97751 (0.0007) -[2023-10-14 17:31:50,843][75950] Updated weights for policy 1, policy_version 97480 (0.0008) -[2023-10-14 17:31:51,209][75950] Updated weights for policy 1, policy_version 97490 (0.0009) -[2023-10-14 17:31:51,573][75950] Updated weights for policy 1, policy_version 97500 (0.0009) -[2023-10-14 17:31:53,163][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 199950336. Throughput: 0: 1698.7, 1: 1666.2. Samples: 49995556. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 17:31:53,164][74987] Avg episode reward: [(0, '29.460'), (1, '36.120')] -[2023-10-14 17:31:53,744][75949] Updated weights for policy 0, policy_version 97761 (0.0008) -[2023-10-14 17:31:54,105][75949] Updated weights for policy 0, policy_version 97771 (0.0010) -[2023-10-14 17:31:54,473][75949] Updated weights for policy 0, policy_version 97781 (0.0011) -[2023-10-14 17:31:54,839][75949] Updated weights for policy 0, policy_version 97791 (0.0009) -[2023-10-14 17:31:55,568][75950] Updated weights for policy 1, policy_version 97510 (0.0008) -[2023-10-14 17:31:55,936][75950] Updated weights for policy 1, policy_version 97520 (0.0008) -[2023-10-14 17:31:56,311][75950] Updated weights for policy 1, policy_version 97530 (0.0009) -[2023-10-14 17:31:58,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 200015872. Throughput: 0: 1690.4, 1: 1683.2. Samples: 50015992. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 17:31:58,165][74987] Avg episode reward: [(0, '29.470'), (1, '34.120')] -[2023-10-14 17:31:58,896][75949] Updated weights for policy 0, policy_version 97801 (0.0008) -[2023-10-14 17:31:59,263][75949] Updated weights for policy 0, policy_version 97811 (0.0010) -[2023-10-14 17:31:59,635][75949] Updated weights for policy 0, policy_version 97821 (0.0007) -[2023-10-14 17:32:00,299][75950] Updated weights for policy 1, policy_version 97540 (0.0008) -[2023-10-14 17:32:00,660][75950] Updated weights for policy 1, policy_version 97550 (0.0009) -[2023-10-14 17:32:01,031][75950] Updated weights for policy 1, policy_version 97560 (0.0010) -[2023-10-14 17:32:03,164][74987] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 200081408. Throughput: 0: 1686.6, 1: 1675.9. Samples: 50025890. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 17:32:03,164][74987] Avg episode reward: [(0, '27.490'), (1, '35.440')] -[2023-10-14 17:32:03,849][75949] Updated weights for policy 0, policy_version 97831 (0.0010) -[2023-10-14 17:32:04,210][75949] Updated weights for policy 0, policy_version 97841 (0.0011) -[2023-10-14 17:32:04,576][75949] Updated weights for policy 0, policy_version 97851 (0.0009) -[2023-10-14 17:32:05,104][75950] Updated weights for policy 1, policy_version 97570 (0.0010) -[2023-10-14 17:32:05,467][75950] Updated weights for policy 1, policy_version 97580 (0.0008) -[2023-10-14 17:32:05,830][75950] Updated weights for policy 1, policy_version 97590 (0.0010) -[2023-10-14 17:32:06,197][75950] Updated weights for policy 1, policy_version 97600 (0.0008) -[2023-10-14 17:32:08,164][74987] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 200146944. Throughput: 0: 1691.1, 1: 1670.8. Samples: 50045904. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 17:32:08,165][74987] Avg episode reward: [(0, '29.080'), (1, '38.250')] -[2023-10-14 17:32:08,651][75949] Updated weights for policy 0, policy_version 97861 (0.0009) -[2023-10-14 17:32:09,021][75949] Updated weights for policy 0, policy_version 97871 (0.0007) -[2023-10-14 17:32:09,395][75949] Updated weights for policy 0, policy_version 97881 (0.0007) -[2023-10-14 17:32:10,377][75950] Updated weights for policy 1, policy_version 97610 (0.0007) -[2023-10-14 17:32:10,740][75950] Updated weights for policy 1, policy_version 97620 (0.0009) -[2023-10-14 17:32:11,112][75950] Updated weights for policy 1, policy_version 97630 (0.0010) -[2023-10-14 17:32:13,164][74987] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 200212480. Throughput: 0: 1691.2, 1: 1684.0. Samples: 50066532. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 17:32:13,165][74987] Avg episode reward: [(0, '26.900'), (1, '37.890')] -[2023-10-14 17:32:13,310][75949] Updated weights for policy 0, policy_version 97891 (0.0008) -[2023-10-14 17:32:13,688][75949] Updated weights for policy 0, policy_version 97901 (0.0008) -[2023-10-14 17:32:14,046][75949] Updated weights for policy 0, policy_version 97911 (0.0011) -[2023-10-14 17:32:15,118][75950] Updated weights for policy 1, policy_version 97640 (0.0009) -[2023-10-14 17:32:15,486][75950] Updated weights for policy 1, policy_version 97650 (0.0009) -[2023-10-14 17:32:15,861][75950] Updated weights for policy 1, policy_version 97660 (0.0008) -[2023-10-14 17:32:18,097][75949] Updated weights for policy 0, policy_version 97921 (0.0009) -[2023-10-14 17:32:18,164][74987] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 200278016. Throughput: 0: 1688.6, 1: 1663.0. Samples: 50076102. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) -[2023-10-14 17:32:18,165][74987] Avg episode reward: [(0, '28.300'), (1, '36.830')] -[2023-10-14 17:32:18,457][75949] Updated weights for policy 0, policy_version 97931 (0.0010) -[2023-10-14 17:32:18,833][75949] Updated weights for policy 0, policy_version 97941 (0.0009) -[2023-10-14 17:32:19,201][75949] Updated weights for policy 0, policy_version 97951 (0.0010) -[2023-10-14 17:32:19,235][76011] Stopping RolloutWorker_w11... -[2023-10-14 17:32:19,235][75993] Stopping RolloutWorker_w5... -[2023-10-14 17:32:19,235][75984] Stopping RolloutWorker_w2... -[2023-10-14 17:32:19,235][75615] Stopping Batcher_0... -[2023-10-14 17:32:19,235][76627] Stopping RolloutWorker_w14... -[2023-10-14 17:32:19,235][75993] Loop rollout_proc5_evt_loop terminating... -[2023-10-14 17:32:19,235][76011] Loop rollout_proc11_evt_loop terminating... -[2023-10-14 17:32:19,235][74987] Component RolloutWorker_w11 stopped! -[2023-10-14 17:32:19,235][75984] Loop rollout_proc2_evt_loop terminating... -[2023-10-14 17:32:19,235][76627] Loop rollout_proc14_evt_loop terminating... -[2023-10-14 17:32:19,235][75801] Stopping Batcher_1... -[2023-10-14 17:32:19,235][76013] Stopping RolloutWorker_w13... -[2023-10-14 17:32:19,235][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000097952_100302848.pth... -[2023-10-14 17:32:19,235][75987] Stopping RolloutWorker_w1... -[2023-10-14 17:32:19,236][74987] Component RolloutWorker_w5 stopped! -[2023-10-14 17:32:19,236][76659] Stopping RolloutWorker_w15... -[2023-10-14 17:32:19,236][76013] Loop rollout_proc13_evt_loop terminating... -[2023-10-14 17:32:19,236][75801] Loop batcher_evt_loop terminating... -[2023-10-14 17:32:19,236][74987] Component Batcher_0 stopped! -[2023-10-14 17:32:19,236][75987] Loop rollout_proc1_evt_loop terminating... -[2023-10-14 17:32:19,236][76659] Loop rollout_proc15_evt_loop terminating... -[2023-10-14 17:32:19,236][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000097664_100007936.pth... -[2023-10-14 17:32:19,236][74987] Component RolloutWorker_w2 stopped! -[2023-10-14 17:32:19,237][74987] Component RolloutWorker_w14 stopped! -[2023-10-14 17:32:19,237][75996] Stopping RolloutWorker_w3... -[2023-10-14 17:32:19,237][74987] Component Batcher_1 stopped! -[2023-10-14 17:32:19,237][75996] Loop rollout_proc3_evt_loop terminating... -[2023-10-14 17:32:19,237][74987] Component RolloutWorker_w13 stopped! -[2023-10-14 17:32:19,238][74987] Component RolloutWorker_w1 stopped! -[2023-10-14 17:32:19,238][74987] Component RolloutWorker_w15 stopped! -[2023-10-14 17:32:19,238][74987] Component RolloutWorker_w3 stopped! -[2023-10-14 17:32:19,239][76005] Stopping RolloutWorker_w6... -[2023-10-14 17:32:19,239][74987] Component RolloutWorker_w6 stopped! -[2023-10-14 17:32:19,239][76005] Loop rollout_proc6_evt_loop terminating... -[2023-10-14 17:32:19,240][74987] Component RolloutWorker_w10 stopped! -[2023-10-14 17:32:19,240][76010] Stopping RolloutWorker_w10... -[2023-10-14 17:32:19,236][75615] Loop batcher_evt_loop terminating... -[2023-10-14 17:32:19,241][76010] Loop rollout_proc10_evt_loop terminating... -[2023-10-14 17:32:19,241][74987] Component RolloutWorker_w12 stopped! -[2023-10-14 17:32:19,241][76012] Stopping RolloutWorker_w12... -[2023-10-14 17:32:19,241][75994] Stopping RolloutWorker_w4... -[2023-10-14 17:32:19,241][74987] Component RolloutWorker_w4 stopped! -[2023-10-14 17:32:19,241][76007] Stopping RolloutWorker_w8... -[2023-10-14 17:32:19,241][75994] Loop rollout_proc4_evt_loop terminating... -[2023-10-14 17:32:19,241][76012] Loop rollout_proc12_evt_loop terminating... -[2023-10-14 17:32:19,241][75983] Stopping RolloutWorker_w0... -[2023-10-14 17:32:19,242][74987] Component RolloutWorker_w8 stopped! -[2023-10-14 17:32:19,242][76007] Loop rollout_proc8_evt_loop terminating... -[2023-10-14 17:32:19,242][75983] Loop rollout_proc0_evt_loop terminating... -[2023-10-14 17:32:19,242][74987] Component RolloutWorker_w0 stopped! -[2023-10-14 17:32:19,242][74987] Component RolloutWorker_w7 stopped! -[2023-10-14 17:32:19,242][76006] Stopping RolloutWorker_w7... -[2023-10-14 17:32:19,242][76008] Stopping RolloutWorker_w9... -[2023-10-14 17:32:19,242][74987] Component RolloutWorker_w9 stopped! -[2023-10-14 17:32:19,243][76006] Loop rollout_proc7_evt_loop terminating... -[2023-10-14 17:32:19,243][76008] Loop rollout_proc9_evt_loop terminating... -[2023-10-14 17:32:19,253][75950] Weights refcount: 2 0 -[2023-10-14 17:32:19,255][75950] Stopping InferenceWorker_p1-w0... -[2023-10-14 17:32:19,256][75950] Loop inference_proc1-0_evt_loop terminating... -[2023-10-14 17:32:19,255][74987] Component InferenceWorker_p1-w0 stopped! -[2023-10-14 17:32:19,261][75949] Weights refcount: 2 0 -[2023-10-14 17:32:19,262][75949] Stopping InferenceWorker_p0-w0... -[2023-10-14 17:32:19,262][75949] Loop inference_proc0-0_evt_loop terminating... -[2023-10-14 17:32:19,262][74987] Component InferenceWorker_p0-w0 stopped! -[2023-10-14 17:32:19,272][75801] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000096256_98566144.pth -[2023-10-14 17:32:19,277][75801] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p1/checkpoint_000097664_100007936.pth... -[2023-10-14 17:32:19,283][75615] Removing ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000096512_98828288.pth -[2023-10-14 17:32:19,289][75615] Saving ./train_atari/atari_riverraid_APPO/checkpoint_p0/checkpoint_000097952_100302848.pth... -[2023-10-14 17:32:19,319][75801] Stopping LearnerWorker_p1... -[2023-10-14 17:32:19,319][75801] Loop learner_proc1_evt_loop terminating... -[2023-10-14 17:32:19,319][74987] Component LearnerWorker_p1 stopped! -[2023-10-14 17:32:19,346][75615] Stopping LearnerWorker_p0... -[2023-10-14 17:32:19,346][75615] Loop learner_proc0_evt_loop terminating... -[2023-10-14 17:32:19,346][74987] Component LearnerWorker_p0 stopped! -[2023-10-14 17:32:19,347][74987] Waiting for process learner_proc0 to stop... -[2023-10-14 17:32:20,211][74987] Waiting for process learner_proc1 to stop... -[2023-10-14 17:32:20,212][74987] Waiting for process inference_proc0-0 to join... -[2023-10-14 17:32:20,212][74987] Waiting for process inference_proc1-0 to join... -[2023-10-14 17:32:20,212][74987] Waiting for process rollout_proc0 to join... -[2023-10-14 17:32:20,212][74987] Waiting for process rollout_proc1 to join... -[2023-10-14 17:32:20,213][74987] Waiting for process rollout_proc2 to join... -[2023-10-14 17:32:20,213][74987] Waiting for process rollout_proc3 to join... -[2023-10-14 17:32:20,213][74987] Waiting for process rollout_proc4 to join... -[2023-10-14 17:32:20,214][74987] Waiting for process rollout_proc5 to join... -[2023-10-14 17:32:20,214][74987] Waiting for process rollout_proc6 to join... -[2023-10-14 17:32:20,214][74987] Waiting for process rollout_proc7 to join... -[2023-10-14 17:32:20,215][74987] Waiting for process rollout_proc8 to join... -[2023-10-14 17:32:20,215][74987] Waiting for process rollout_proc9 to join... -[2023-10-14 17:32:20,215][74987] Waiting for process rollout_proc10 to join... -[2023-10-14 17:32:20,215][74987] Waiting for process rollout_proc11 to join... -[2023-10-14 17:32:20,216][74987] Waiting for process rollout_proc12 to join... -[2023-10-14 17:32:20,216][74987] Waiting for process rollout_proc13 to join... -[2023-10-14 17:32:20,216][74987] Waiting for process rollout_proc14 to join... -[2023-10-14 17:32:20,216][74987] Waiting for process rollout_proc15 to join... -[2023-10-14 17:32:20,217][74987] Batcher 0 profile tree view: -batching: 171.2740, releasing_batches: 0.0896 -[2023-10-14 17:32:20,217][74987] Batcher 1 profile tree view: -batching: 169.5530, releasing_batches: 0.0890 -[2023-10-14 17:32:20,217][74987] InferenceWorker_p0-w0 profile tree view: -wait_policy: 0.0018 - wait_policy_total: 2646.9537 -update_model: 207.3851 - weight_update: 0.0010 -one_step: 0.0024 - handle_policy_step: 11421.4793 - deserialize: 65.4969, stack: 195.6884, obs_to_device_normalize: 2536.0432, forward: 5168.2709, prepare_outputs: 2490.9608, send_messages: 461.9540 -[2023-10-14 17:32:20,217][74987] InferenceWorker_p1-w0 profile tree view: -wait_policy: 0.0000 - wait_policy_total: 2717.2924 -update_model: 209.6288 - weight_update: 0.0008 -one_step: 0.0018 - handle_policy_step: 11344.0963 - deserialize: 63.6917, stack: 193.5431, obs_to_device_normalize: 2540.9641, forward: 5133.1278, prepare_outputs: 2448.3307, send_messages: 464.4721 -[2023-10-14 17:32:20,218][74987] Learner 0 profile tree view: -misc: 0.0185, prepare_batch: 270.4280 -train: 3643.1207 - epoch_init: 0.1945, minibatch_init: 13.1679, losses_postprocess: 898.1378, kl_divergence: 32.5220, update: 384.9134, after_optimizer: 2127.2424 - calculate_losses: 170.1151 - losses_init: 0.3945, forward_head: 59.5935, bptt_initial: 1.4388, bptt: 2.1241, tail: 37.9400, advantages_returns: 11.2280, losses: 43.7496 -[2023-10-14 17:32:20,218][74987] Learner 1 profile tree view: -misc: 0.0177, prepare_batch: 269.4054 -train: 3602.6548 - epoch_init: 0.1925, minibatch_init: 13.0576, losses_postprocess: 890.6000, kl_divergence: 31.6279, update: 382.0150, after_optimizer: 2101.4878 - calculate_losses: 166.9076 - losses_init: 0.3792, forward_head: 55.9285, bptt_initial: 1.4529, bptt: 1.9535, tail: 38.3033, advantages_returns: 11.3416, losses: 43.7087 -[2023-10-14 17:32:20,218][74987] RolloutWorker_w0 profile tree view: -wait_for_trajectories: 1.2451, enqueue_policy_requests: 410.5651, process_policy_outputs: 191.6241, env_step: 7692.1190, finalize_trajectories: 3.5547, complete_rollouts: 3.0139 -post_env_step: 379.8936 - process_env_step: 87.1622 -[2023-10-14 17:32:20,218][74987] RolloutWorker_w15 profile tree view: -wait_for_trajectories: 1.2260, enqueue_policy_requests: 407.9171, process_policy_outputs: 191.9549, env_step: 7750.7702, finalize_trajectories: 3.4890, complete_rollouts: 2.9060 -post_env_step: 376.9935 - process_env_step: 85.1569 -[2023-10-14 17:32:20,219][74987] Loop Runner_EvtLoop terminating... -[2023-10-14 17:32:20,219][74987] Runner profile tree view: -main_loop: 14972.4774 -[2023-10-14 17:32:20,219][74987] Collected {0: 100302848, 1: 100007936}, FPS: 13378.6 +version https://git-lfs.github.com/spec/v1 +oid sha256:e8ab78572f2ca72ef44f2638add95cf2c584fcc5fdbbb1afcb1c9aaad2ea49ad +size 49655364