Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1698308277.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ffe4b5434adc721bbf3b69f4738c048704a55ece43fad8afc4009d1068750a5c
+size 84026378

.summary/1/events.out.tfevents.1698308277.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6379b998119ac5b1486637d122deaafa5329644f3f5a87cffd60d4c8826a0a5d
+size 44138315

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_breakout
     metrics:
     - type: mean_reward
-      value: 495.10 +/- 169.27
+      value: 863.40 +/- 1.28
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_breakout** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_breakout
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_breakout** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_breakout
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001856608_475291648_reward_62.190.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9e00f50c87f5be30a02fc47210d5c286f360b0592ef522607d11bac4103a98c
+size 20709555

checkpoint_p0/checkpoint_001952704_499892224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6269c9f2ef4170279d708ac468c6ad9ad892a9d730b2291e8faebac02e46a52
+size 20709891

checkpoint_p0/checkpoint_001953136_500015104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a9f53694a43bc9ce0dd0e8b980cb9d3fa0f25170a36dc08aba9f249f8822efb
+size 20709891

checkpoint_p0/milestones/checkpoint_000012864_3293184.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fab2d1291870d9fd429d0d2bee4ab9fe49ac8358fd9b3843f3395eb23245a5fb
+size 20710747

checkpoint_p0/milestones/checkpoint_000026240_6717440.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:231f687f80b63d3d597011156c4db23f1e86b183d2db55c619a67850c6632778
+size 20710747

checkpoint_p0/milestones/checkpoint_000039744_10174464.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d72fd91d40dfcfadf1872abe3a10f81d7050eb959d04476b64a324e4b0290b44
+size 20710803

checkpoint_p0/milestones/checkpoint_000053184_13615104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdecc7fcb7995f87984e5a89629e94703ad1c68c3512e47df8da32a5e578afd3
+size 20710803

checkpoint_p0/milestones/checkpoint_000066624_17055744.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f425cb0dd9a06af89e4932dfb0cfce0b09ba1304dd0d6a2f725f8963b9f2d0b
+size 20710803

checkpoint_p0/milestones/checkpoint_000080160_20520960.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af68fa513f2ec6fc852f9bbf2586f6d4f315b6d542ea051f673d95f6438b802d
+size 20710803

checkpoint_p0/milestones/checkpoint_000093696_23986176.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9f86b1ffbe0c852b737a68a32131f2c37b505fab6cee0db699cb87664b85a90
+size 20710803

checkpoint_p0/milestones/checkpoint_000107232_27451392.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d0ca8a1e07291511cc9ca5d5535fd5845f9cb078b82f017c10588cb5fbd309a
+size 20710803

checkpoint_p0/milestones/checkpoint_000120704_30900224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0370623d2badd9ed23bcc5452f5ef4d9fc7f1bf700cd08931cef7cfd5c231d0f
+size 20710803

checkpoint_p0/milestones/checkpoint_000134240_34365440.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:47869cd47805b442e2394a6c90eda1cacebce55d1d3166abb0cd59522069d332
+size 20710803

checkpoint_p0/milestones/checkpoint_000147616_37789696.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3d81996a5a683f93ab8f5eb89d3b7d54798ca592052b8c1e541e30dfb3698dc
+size 20710803

checkpoint_p0/milestones/checkpoint_000160896_41189376.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1916cad0b044b564f28aa058c0847c4dd4649d54e3110f1aeec27eed8ecc62a9
+size 20710803

checkpoint_p0/milestones/checkpoint_000174144_44580864.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84293ea2dd52c3ce9366505c4d899d63f0101e905ca35e422f2ae321a7a0292b
+size 20710803

checkpoint_p0/milestones/checkpoint_000187648_48037888.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c6527266d2c8ab3eaa8c3c20f84330f7bb2839ba8c2ad70551622e07d6e7b3f
+size 20710803

checkpoint_p0/milestones/checkpoint_000201280_51527680.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87c2e6905a15b044ff1eaf7d65b9118446f529d2493924b01634e2d1f474ddeb
+size 20710803

checkpoint_p0/milestones/checkpoint_000214784_54984704.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da0f642791c300bbc66c7813550af0b7c662abc2c1814992e930492bbdf75032
+size 20710803

checkpoint_p0/milestones/checkpoint_000228224_58425344.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20502cae03fc21fbabda4de9113abc43062945f8c1709c737e774d64e28035fc
+size 20710803

checkpoint_p0/milestones/checkpoint_000241728_61882368.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1b619afbe021f12e0ff2128ce574cf000f4765e28ac108a9ec003ac171d11ed
+size 20710803

checkpoint_p0/milestones/checkpoint_000255264_65347584.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d250765c7f656269d5f3ff06690bdbdbb4d1d47d07737a61df4f8b1ed840be87
+size 20710803

checkpoint_p0/milestones/checkpoint_000268704_68788224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6ba8630fc9b7305cd4752866079d5901b8e0de4b5e0042be4c7bed3a88ca762
+size 20710803

checkpoint_p0/milestones/checkpoint_000282304_72269824.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:721c2ab7fe8c8e9e2ea1f429b273246e38252f59a7d50860dd0264fe26570d55
+size 20710803

checkpoint_p0/milestones/checkpoint_000295840_75735040.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1e997a7a85aae71f47485cca27680f3172104bb2a4443a4173b66278ce5d8fb
+size 20710803

checkpoint_p0/milestones/checkpoint_000309408_79208448.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf424dc4de548ab11f1ad884a04b22a1c7de479ca30e925258872bf129225a01
+size 20710803

checkpoint_p0/milestones/checkpoint_000322976_82681856.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0399fe10531ca4ed4c33cb68277b5be437395004b9197335ef7bcaecf3df0f6f
+size 20710803

checkpoint_p0/milestones/checkpoint_000336512_86147072.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f04a9cc856674402943bac0153bfe1fd778f9c8e2457154800d57ac6c9bc56e7
+size 20710803

checkpoint_p0/milestones/checkpoint_000350016_89604096.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0cd614b4ff088df0e38dff74e16a1cb02b320e066a5e9fe9371e22506d9d2815
+size 20710803

checkpoint_p0/milestones/checkpoint_000363616_93085696.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ff06cee84b7157fab78b82ba3a889044071cd28b3ae7751fbf2f7d3bcc579f7
+size 20710803

checkpoint_p0/milestones/checkpoint_000377184_96559104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee4498d6295514e4bd3e1d77623310da23d726b8db7235333c4a69d1113bbffd
+size 20710803

checkpoint_p0/milestones/checkpoint_000390720_100024320.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:807d03fd5b6d2704746b3258ac88460f26694cd4581d1429e87d5f19f815e8a1
+size 20710859

checkpoint_p0/milestones/checkpoint_000404288_103497728.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:401aa32429b71ee3ffc2c906e62964e9abea4494b5384e1dab52b74ac33aa85f
+size 20710859

checkpoint_p0/milestones/checkpoint_000417824_106962944.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:139a5302fa38e38acc67cfe488fc23f4578d68bf3b5abb56ec39e7b829394645
+size 20710859

checkpoint_p0/milestones/checkpoint_000431360_110428160.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6ea0ea676dd443442b53ae8bbf86f7816cf53c1a9c17c95d6c07110aeb6b9dd
+size 20710859

checkpoint_p0/milestones/checkpoint_000444960_113909760.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2acd2d5ae33a41943d153a26cc441b03a40849aa887a253b1abe9f103d6ea9b2
+size 20710859

checkpoint_p0/milestones/checkpoint_000458464_117366784.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9bf4bd2dac6cb1dac6e700de31070e6287da1de04b883f643359bfdeda981e45
+size 20710859

checkpoint_p0/milestones/checkpoint_000472032_120840192.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2190ee9f9d9bbfbf01d657bbf167cb6594edd403f682e8c368a28bf8ecee1251
+size 20710859

checkpoint_p0/milestones/checkpoint_000485600_124313600.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:216693dc7cf4a3497e795d60e84a00c5cfcd96211e5e646751514f2d7fcc03f0
+size 20710859

checkpoint_p0/milestones/checkpoint_000499168_127787008.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf7d0741eeb8ebebcb6bcfef7785ca7926714b9da9ae87fe05ea88eec6a16cb3
+size 20710859

checkpoint_p0/milestones/checkpoint_000512704_131252224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2090013c26903cc7b232ac416bf5a526a3364770b460cb96c6f7dac4ff40969a
+size 20710859

checkpoint_p0/milestones/checkpoint_000526272_134725632.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c02e2657b214ba119b7cc1c6fbcbd9eb3cc93faf8dda614b702a6ed9a837ded6
+size 20710859

checkpoint_p0/milestones/checkpoint_000539872_138207232.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed11fabddfe76198d098115454f1348a658250061a1fcb59d59525b199e3e6c6
+size 20710859

checkpoint_p0/milestones/checkpoint_000553440_141680640.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc7bc923e4fa4b36c37daca50870debf435792d5c6a6fa743f853240113db8d4
+size 20710859

checkpoint_p0/milestones/checkpoint_000566976_145145856.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:376f96d15dfbe73e658cf20542f1aa487cda9d55c5872c4ad65894c0f6ce4fb2
+size 20710859

checkpoint_p0/milestones/checkpoint_000580512_148611072.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ceb67c20e5b4b661477f16d976211cfb0ef246f0ebff56f77d9a17d2e77f51ed
+size 20710859