Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1700009657.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68c59090512835bf7ce877929d253f149d63e71ad90c78be2951a1db4b4837d0
+size 89944269

.summary/1/events.out.tfevents.1700009657.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70559069710c5193620eb7540dcce59443206e10d265fbd9dbb423a6215f87c1
+size 47250976

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_namethisgame
     metrics:
     - type: mean_reward
-      value: 14371.00 +/- 1723.86
+      value: 22607.00 +/- 2779.10
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_namethisgame** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_namethisgame
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_namethisgame** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_namethisgame
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001915904_490471424_reward_394.430.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b96e9e390108a3ba0db7682166381e546a958f1c75d2b3176bdab2cbe3b3d6f
+size 20721843

checkpoint_p0/checkpoint_001951872_499679232.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a98ebb308e3b1fb1f1a4700dabb49a17e090ad68cdfb2408b9960aaa52af1a8
+size 20722179

checkpoint_p0/checkpoint_001953120_500006912.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55c43dfcf4d9c128a85853dc76d54264041ccf44a0bdd66fec37ceeaa25f8531
+size 20722179

checkpoint_p0/milestones/checkpoint_000012576_3219456.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12d737786bfff197dd602d730a5a61a5a8cd3835797a3e351c143e0566651d56
+size 20723035

checkpoint_p0/milestones/checkpoint_000025504_6529024.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a04660d796ae83e8d1abce9a89a04cea92af73870af08d172d55396b51d13a1
+size 20723035

checkpoint_p0/milestones/checkpoint_000038368_9822208.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8315af314202d5e33a4ac0262fe37f03f89edcdce03f415e7c8a7bc15bd10fe
+size 20723035

checkpoint_p0/milestones/checkpoint_000051232_13115392.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:efa3a213221b253a23767e7ed1b9492b87b926808c0084a945cd145ab9741702
+size 20723091

checkpoint_p0/milestones/checkpoint_000064128_16416768.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:383e33fe3f2fbdb1174ddaf1be6194ea699beb2a6f4cc75568d410211a2016df
+size 20723091

checkpoint_p0/milestones/checkpoint_000076928_19693568.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34dd250c36d8ae54f36a5e92489d1400da4bfb84bd0f3d9a76cf032d082ac12b
+size 20723091

checkpoint_p0/milestones/checkpoint_000089824_22994944.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56f0fc09c49a2e63c940baaf39aba1b5c838519febf9373e2a906d4a28fa5f4b
+size 20723091

checkpoint_p0/milestones/checkpoint_000102624_26271744.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37181cc257b76cca8425f30c2cf26fcbc57fd1b5f3ea862c95ecb3d460ea530b
+size 20723091

checkpoint_p0/milestones/checkpoint_000115424_29548544.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa635bd4374e2b4a577c3a46a93936fe95e9e73dd00470b8853f4275b5235007
+size 20723091

checkpoint_p0/milestones/checkpoint_000128288_32841728.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7f46e7bda7468eb5e5f2890c63b2d97c470a62932e5fad472ca42dea098b2f6
+size 20723091

checkpoint_p0/milestones/checkpoint_000141088_36118528.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dffc9875daa5619f29fa5a760781db43d22898742df7451be3ed1731106ceb6a
+size 20723091

checkpoint_p0/milestones/checkpoint_000153920_39403520.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e181ba4cc0555e1763a217f19fad194d820e9c9ed8ab636b6344ff759d23b83c
+size 20723091

checkpoint_p0/milestones/checkpoint_000166752_42688512.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f223b7393785122f737bc9ad0f3b5718de08711cae7cd1652ad13a18ae47e1c
+size 20723091

checkpoint_p0/milestones/checkpoint_000179616_45981696.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20e82dddea1dae0abc85f0de91c561b920a1403f8bd5fd946d5b96287446eaf9
+size 20723091

checkpoint_p0/milestones/checkpoint_000192448_49266688.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e13c60d3011f2632c3e61755bd65ba1ded368d06923a974bc483146bd964b6f
+size 20723091

checkpoint_p0/milestones/checkpoint_000205312_52559872.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08ce50b15973d30b707eca1200f27b1b87feb055b77e86ac0f8ada8d9602ddc4
+size 20723091

checkpoint_p0/milestones/checkpoint_000218112_55836672.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7120d44f8eaac8942150ba2fcf6341dfdcd42d809681b6763b81e217498a6c3
+size 20723091

checkpoint_p0/milestones/checkpoint_000230912_59113472.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:145e14d652bda19c160db0c92f74fef271f3c4806ca6d361b2d0e9559c03fa66
+size 20723091

checkpoint_p0/milestones/checkpoint_000243744_62398464.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc295893e0ab04810949e4427d3cdd96e027323c6db24205038d50d575f51204
+size 20723091

checkpoint_p0/milestones/checkpoint_000256576_65683456.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:288e8bee08126ce8934be8461c6f4991c3b7c8ca0c7ecb302abc994157b2440a
+size 20723091

checkpoint_p0/milestones/checkpoint_000269440_68976640.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:677dc5c2a7b2b96ad8496fd01d2cb2d7d5f7229fe5e67cb553c68ff6cdeb41d6
+size 20723091

checkpoint_p0/milestones/checkpoint_000282240_72253440.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9118ad2322aaf06c34af409480f58a9f1daeb1dac0437b3dcc6edd9fa03719e
+size 20723091

checkpoint_p0/milestones/checkpoint_000294976_75513856.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9892ad9f3491adbc3bac0d75971159fbe1902072a38dcf662c32a3331aef4089
+size 20723091

checkpoint_p0/milestones/checkpoint_000307776_78790656.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8113310a96b0409058c409e5d81400f62ee6f90094b5bb532f02bf8ce1b2f447
+size 20723091

checkpoint_p0/milestones/checkpoint_000320576_82067456.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fb2ff560d11c6f0d5f1be677035631eaecbfdad0c0186bc600f24c4b2093d8f
+size 20723091

checkpoint_p0/milestones/checkpoint_000333408_85352448.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e66816fea52c5e36eb435fb03c8682e47932923b260516033bd23c479cb0570a
+size 20723091

checkpoint_p0/milestones/checkpoint_000346208_88629248.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1bf13dddeb67610905ab8ac834e053e22022888ba24238b4e684537eedb2d55c
+size 20723091

checkpoint_p0/milestones/checkpoint_000359104_91930624.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a2245c3c95e12fa6b0fb034a556e97284d3218aef6acaddbdd862b37453e595
+size 20723091

checkpoint_p0/milestones/checkpoint_000371872_95199232.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4416586c0beb84923b9be2a42425e2b96e392af1c8b0d419466c69da3f3d1646
+size 20723091

checkpoint_p0/milestones/checkpoint_000384672_98476032.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fec062709cdad7dc41c735c750861787ad6a89c3e5cb42ba98183810e849d025
+size 20723091

checkpoint_p0/milestones/checkpoint_000397472_101752832.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:159ae66ed215d1f5512b8b19c12858a594c6399dcd9957205643f628e985aa2e
+size 20723147

checkpoint_p0/milestones/checkpoint_000410304_105037824.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:451048e464158436ac85179924cccde12262b2370b24537c4aebb36a47e11d9f
+size 20723147

checkpoint_p0/milestones/checkpoint_000422976_108281856.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a214ecdaa8ecc31f9a9476a07e3a0d3ef1beddedee600c939ed612a3025e485e
+size 20723147

checkpoint_p0/milestones/checkpoint_000435680_111534080.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d223963d397ef4417c95d5b672d6ea7c44123b9a05776758ee18718c7fdb04c
+size 20723147

checkpoint_p0/milestones/checkpoint_000448416_114794496.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66611f9b65b9380eaa270e199a93741e538d5a0c15dc98f7462efb7bb013eedf
+size 20723147

checkpoint_p0/milestones/checkpoint_000461120_118046720.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9824fb4af73f00fd94ed33a3a36bbf502456460d8160ef29437afd68ddaf7333
+size 20723147

checkpoint_p0/milestones/checkpoint_000473856_121307136.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7bcff795db1888c9c05c8718c4988af2adf4cbc87a4b403ce6f177511bc7a52
+size 20723147

checkpoint_p0/milestones/checkpoint_000486496_124542976.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3bcd9fe4c0eedb4070a6bd773c36d200caa58fcd18a7c61ef04a9d5875c46872
+size 20723147

checkpoint_p0/milestones/checkpoint_000499232_127803392.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35dfda4d4d59000eedb60623a13ec411df66c038d5c9424ad9daa5266b601bf3
+size 20723147

checkpoint_p0/milestones/checkpoint_000511904_131047424.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:00f35bd3008fa05c7479a4139ce7fee159b5b28ed6fce077f6e8785852966e18
+size 20723147

checkpoint_p0/milestones/checkpoint_000524576_134291456.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d1c05feae4892911398de8ba04ecc7aa20ab2dae1b3f18434d5cfc69e28a413
+size 20723147

checkpoint_p0/milestones/checkpoint_000537376_137568256.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:816988fe9ef1572df5dc3d4a643e263840d8e22a2cf00ffdbf9d0eafc2c35cf9
+size 20723147

checkpoint_p0/milestones/checkpoint_000550048_140812288.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d58b1d3e0fb1cff6544935452144ea809963c7240a49ce86bb18da432af05250
+size 20723147