Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1700548189.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e72cab06f5d7c8398dc3e471abfb4f85262dbcb2bb67dbd6316a28a6b0f6f98d
+size 94086055

.summary/1/events.out.tfevents.1700548189.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:463554fb7a493ddf0288f93aa36ca9c4de99841f2fb0b845a92bef14956a409e
+size 49427605

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_riverraid
     metrics:
     - type: mean_reward
-      value: 8855.00 +/- 577.95
+      value: 52696.00 +/- 3378.58
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_riverraid** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_riverraid
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_riverraid** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_riverraid
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001864192_477233152_reward_85.330.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4485448b1a5308f68a76e512ad60b1e5654a0c9b65c3fc7e4abdfb781ec6ea52
+size 20795763

checkpoint_p0/checkpoint_001958784_502906880.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fff7997c09b6c75366cadd03249081f77c6c4c1e8da4bced153e1f1db5d5a634
+size 20796099

checkpoint_p0/checkpoint_001959392_503218176.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e76413845bbce6fadb76344fa9ad07635050ced44519afb00d15e19797d7f92a
+size 20796099

checkpoint_p0/milestones/checkpoint_000012000_3072000.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1abe2a1afbfdfd09a2bb2904713a8c8e6f61ec6726005f2ce04a5a3022b6ad4
+size 20796955

checkpoint_p0/milestones/checkpoint_000024256_6209536.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fbe252db6c521171dbadf6560c7e35b314a170d6b095cc6b327d1224c327b89
+size 20796955

checkpoint_p0/milestones/checkpoint_000036480_9338880.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19ef78042aff31e395bbb94dcb078e7ede07755c2383894cd8a73df524b10a67
+size 20796955

checkpoint_p0/milestones/checkpoint_000048768_12484608.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:650198cd88da818d8ef0293848571689ad5bfc2856e099e39c9906b144e7351e
+size 20797011

checkpoint_p0/milestones/checkpoint_000060864_15581184.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50548337ac536b129360727a833cfff2cce81a00fd21ab7d6cef85ce457af983
+size 20797011

checkpoint_p0/milestones/checkpoint_000073088_18710528.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6fd9d68767d9a67892bab7aa18b100782bd272d13dd0986bc9a48b6411e9644
+size 20797011

checkpoint_p0/milestones/checkpoint_000085312_21839872.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e3acba8a8288ffa88146f4b54a700591fe1c3ba868d7453c21727a37354778d
+size 20797011

checkpoint_p0/milestones/checkpoint_000097600_24985600.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:202bc691ca41712795ebc6bba6e7a91930c48b70b04fea0d7b3b113f4ea0dbab
+size 20797011

checkpoint_p0/milestones/checkpoint_000109824_28114944.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6857c9468540bc08e1cc06e5811d00052b4aa6373920e8c3de71bc9111e0bf82
+size 20797011

checkpoint_p0/milestones/checkpoint_000122112_31260672.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:084b9883bb0a35a67bac16a5a7a2297939d5a205ba2c22c492e4a972fedd6c0c
+size 20797011

checkpoint_p0/milestones/checkpoint_000134400_34406400.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b015636364ebc60ff3e77c11bb991d8402983bbca7e0a93c9f37fec15cdea715
+size 20797011

checkpoint_p0/milestones/checkpoint_000146624_37535744.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dbde70ce05ce40004b5696fbb7632b4bf4e33782858e4ba2b6648aaf61a3a9dd
+size 20797011

checkpoint_p0/milestones/checkpoint_000158944_40689664.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94dbc5af3139b16f42a1d8493c99eacbc25194270aad27eb12a5fc7836f91bf8
+size 20797011

checkpoint_p0/milestones/checkpoint_000171264_43843584.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88d2109a2bd33e912aa2ef0a927fb1d65d32c79f8f8db38e5bd41e278fe633bf
+size 20797011

checkpoint_p0/milestones/checkpoint_000183488_46972928.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1624dd5a55c4767b6ee35ddbe279dc4725d23e58bc157aed5d1184ee4f13ece3
+size 20797011

checkpoint_p0/milestones/checkpoint_000195808_50126848.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91ee8ddde9cf83d701b9ae77c073ee2c879947eec49cb770eaaa76a1be0981b0
+size 20797011

checkpoint_p0/milestones/checkpoint_000208096_53272576.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e83bdc85ac6428f123bd2d6c25ae6c6f73de0bc60e1a1cdb01073d1cdea211f
+size 20797011

checkpoint_p0/milestones/checkpoint_000220384_56418304.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:650077840031341b177e48b32b20fa6c5fa0facc848420fee729728e9583c081
+size 20797011

checkpoint_p0/milestones/checkpoint_000232704_59572224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24cd5414aae5583c11422de79bf41bf73aba87054b05af0538806435aaca5580
+size 20797011

checkpoint_p0/milestones/checkpoint_000244928_62701568.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce765e858f606267a302fd1bd6e7697dbac83f31fa66bf776c65e0d8babeb461
+size 20797011

checkpoint_p0/milestones/checkpoint_000257280_65863680.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:556afe03e444dba03a2808f19f10baf93db2cd6d16c4c4f0214c4dcbad4b088c
+size 20797011

checkpoint_p0/milestones/checkpoint_000269568_69009408.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cfb747082a13542653203b1f8a3c924a83ec4bd4b369ee4e1a1f118127c65624
+size 20797011

checkpoint_p0/milestones/checkpoint_000281824_72146944.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f6a8b8116e9bbe1b0dd21b370444b109b8dbdd1872b52a7ec051ac146282a5d
+size 20797011

checkpoint_p0/milestones/checkpoint_000294144_75300864.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b17a39dbfbd15ba7c4b68fc8d9d7b7e4de2185b977e70ad8ba103c3cdd83613
+size 20797011

checkpoint_p0/milestones/checkpoint_000306432_78446592.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33f9f92deb66b0dce7a2851c2a2eb1b843b136e4b4d9e8f84ff1107d47e05363
+size 20797011

checkpoint_p0/milestones/checkpoint_000318720_81592320.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db0dae8f7b4ae3c7b9b20a043b5605c5f62f9878ef1b8e890a7f2e6a704cd5f3
+size 20797011

checkpoint_p0/milestones/checkpoint_000331008_84738048.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:abedf384a2d87f168d8cb4030f12cfe0c985cb63638e44a0bcbafe42ff2ba9ea
+size 20797011

checkpoint_p0/milestones/checkpoint_000343264_87875584.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06355f1b0b012d4dd4ff9dc187abc2c0216540770b66a1a361a2decb667d0136
+size 20797011

checkpoint_p0/milestones/checkpoint_000355552_91021312.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1739124960fad362fd3a9d7d45a39fd6c0e2862d506243070702044747aafaf
+size 20797011

checkpoint_p0/milestones/checkpoint_000367904_94183424.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da6fcc98a6adc90ffb84ed0693c15391f6b17b63500b07d50760f47656fd341e
+size 20797011

checkpoint_p0/milestones/checkpoint_000380192_97329152.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b4225b53ec38fae9a63a9105f44a64c5d0d669bac0a688b1021c1e6e72e10e34
+size 20797011

checkpoint_p0/milestones/checkpoint_000392512_100483072.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0648e3c0e980ecdcce12d7e95a9f30a387daa2821f7ea362f5ee09d20e61816
+size 20797067

checkpoint_p0/milestones/checkpoint_000404800_103628800.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:022e5bda879c420632acdfa8ffbc89067c446118f702d9f7f08b0e0574f4ccb3
+size 20797067

checkpoint_p0/milestones/checkpoint_000417056_106766336.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86803d83c83bfd5a70d51a2421759b925aa6b85c5a5278ce03efcf371e8bcc97
+size 20797067

checkpoint_p0/milestones/checkpoint_000429408_109928448.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6317c3ef8dd4abbf01cd51cb0c2d845b21986252cfa97558c9b34eede7eb98c7
+size 20797067

checkpoint_p0/milestones/checkpoint_000441696_113074176.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:108626ca04c5f3d0b2af603bb7049b770732827b1408ec606e5f954b3e14a1c8
+size 20797067

checkpoint_p0/milestones/checkpoint_000453984_116219904.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8562a0b3366155bc220c88c371c9c07acf7dca1fc697b2247a4eb665e06475f8
+size 20797067

checkpoint_p0/milestones/checkpoint_000466240_119357440.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b41fe3769e3ce14804536ef17e31d75704257f49991bd2e014d1951755842e0
+size 20797067

checkpoint_p0/milestones/checkpoint_000478592_122519552.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c93cee50b0b2c222c12e7e4a8e7d3664458a1e61ebe63a0fef53bc7b506bdffb
+size 20797067

checkpoint_p0/milestones/checkpoint_000490912_125673472.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7a993ab358bb47b9e15db9afb84da980146c16449d97bc27342550fcfcf9395
+size 20797067

checkpoint_p0/milestones/checkpoint_000503264_128835584.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebb64401478897d6dfe06d89e6be3379987c55d843d15d1d7d3f1c3440b6af55
+size 20797067

checkpoint_p0/milestones/checkpoint_000515552_131981312.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af867e058610134ce75faa4769c7c9368991f873e0c5a905bf91e10eb1c47f97
+size 20797067

checkpoint_p0/milestones/checkpoint_000527840_135127040.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f38bfbde850cc2561740a7f036f5a8e145bab12315cc375f4055660505f820e3
+size 20797067