Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1698653362.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b8ed7f6fd5862e56ad7c41807bd2888ad4bebe41e3819af3fc62200b82cd453
+size 49290166

.summary/0/events.out.tfevents.1698760576.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f881fcc2c6a758a26d99629d9fbf808494e9b47304510551acc8b7e9daa44820
+size 30416159

.summary/1/events.out.tfevents.1698653362.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32644dc5beb2bf6874817ae4ac4cc0be2568a085496219b51514159cbaee2981
+size 26079604

.summary/1/events.out.tfevents.1698760576.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73ae0722d4c0e370d67f191460f91e76b7ef3d4c69b367f6939c48681db79168
+size 15755067

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_defender
     metrics:
     - type: mean_reward
-      value: 59120.00 +/- 11022.48
+      value: 302420.00 +/- 89166.87
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_defender** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_defender
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_defender** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_defender
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001199552_307085312_reward_133.300.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d00656292f99ffa433dda9da279a4f17b2f039944c3ee3fdbf2592bd5783495
+size 20795955

checkpoint_p0/checkpoint_001952192_499761152.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b48983e20b1a58f131d52a27f4c81dcfb51f926c26fce6676329397b119c4f74
+size 20796291

checkpoint_p0/checkpoint_001953184_500015104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c374beee5d8282dc847caecb9d95d3e4fd90de607947f4b5e88c2a5118096663
+size 20796291

checkpoint_p0/milestones/checkpoint_000013440_3440640.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:171779875487b0744ee29ffdc011e1857e1e306c1b2360dc2b63e2a5b83d03da
+size 20796955

checkpoint_p0/milestones/checkpoint_000027776_7110656.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afcdac1cde6080b0344a6074a2f9ad0d6f3822b7d829c15a2d05f2c5bf2507fe
+size 20796955

checkpoint_p0/milestones/checkpoint_000042240_10813440.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74494ba968c48b1e469f33caffac89b83d3d8da3a5dd7d2548b7e4e6db27d3b4
+size 20797011

checkpoint_p0/milestones/checkpoint_000056640_14499840.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cbf91b393385c5b856ece8a9a9f8c524193b3e6e0d41fcf6500da2d02450beaf
+size 20797011

checkpoint_p0/milestones/checkpoint_000071200_18227200.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76146d4a91c174f8d99de057730a79cf67dd47493b42aea30cf5f8186ef48e8e
+size 20797011

checkpoint_p0/milestones/checkpoint_000085568_21905408.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:487ce530b9bf572c0ae46c5a6c07c366c6355c28ac8319cfd33ce96b27ff7dcf
+size 20797011

checkpoint_p0/milestones/checkpoint_000100032_25608192.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c33d0cfef08c25a95554c3e913498f9c61f50bd5c4154cc831853ae4b1bfd21a
+size 20797011

checkpoint_p0/milestones/checkpoint_000114560_29327360.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eebeae7e81b1622aef570a1a6e87d42df78360e0fcf0a0c96ecf1d4ab5742c70
+size 20797011

checkpoint_p0/milestones/checkpoint_000128928_33005568.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a2064826869f5202dadab94a30e5326a4a49e91e13b495221370987b1f8276b
+size 20797011

checkpoint_p0/milestones/checkpoint_000143392_36708352.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f35cddbcdfd23d51f56cf042d389af9de238c4353672c0f35d2a41ca04956c64
+size 20797011

checkpoint_p0/milestones/checkpoint_000157856_40411136.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:546a6e83ce911c4864340f8c0b4cd452f4cfa5c35f5ab28546e41ab7b1a59496
+size 20797011

checkpoint_p0/milestones/checkpoint_000172320_44113920.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3a6f399201371a8e927991411bd7097aa786b53e7651f7d165f827d9f7f21e0
+size 20797011

checkpoint_p0/milestones/checkpoint_000186816_47824896.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43947330c34e5d138f772d67f12c080c9e66781a33d7bccdaced71a23696279a
+size 20797011

checkpoint_p0/milestones/checkpoint_000201152_51494912.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cfaa706efc98bb500a5add6c8ddc31964b98ae014bfd6ae7d7dfbd6e8c7e8601
+size 20797011

checkpoint_p0/milestones/checkpoint_000215488_55164928.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0672a1d4bedc37866316f2919abbe72fb4b353639e3f7a017cd863877a550754
+size 20797011

checkpoint_p0/milestones/checkpoint_000229952_58867712.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb20114c49a6bed9f6129b1681af65b3ee70f3b8e5927ba4529f202bca6dee6e
+size 20797011

checkpoint_p0/milestones/checkpoint_000244544_62603264.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24d160dc235cc994e8ee2edaff5b0b6a9e6a4f5b37af03e2a0badafa5d445667
+size 20797011

checkpoint_p0/milestones/checkpoint_000259008_66306048.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cbaefee8382e9dadce5d6026b8a3684fae44df1ef0014c72366fde2d53d0f157
+size 20797011

checkpoint_p0/milestones/checkpoint_000273536_70025216.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55621c51819b997c8abd305125b9e8a1d9234e28958a675f3cd62f6d36bdad18
+size 20797011

checkpoint_p0/milestones/checkpoint_000287968_73719808.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f46bd619f1cf1f6085a9665ad9e0f35c549ce308ad8a5d3b668a0e80facd0b7
+size 20797011

checkpoint_p0/milestones/checkpoint_000302304_77389824.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a67f3d95bb741ac2c361f663ea7b7320be28c1d93ff1fb96732c3d44c3ef8151
+size 20797011

checkpoint_p0/milestones/checkpoint_000316576_81043456.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd4f0d438f6733ca2beb3c739417c60122dfac0d1b280420d9aaf1dc37a8fb70
+size 20797011

checkpoint_p0/milestones/checkpoint_000330816_84688896.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:acc7ff89e0887087bfe0000dcce5571b542e2e76a6fc5f7bd21c4ee30b1fa3bb
+size 20797011

checkpoint_p0/milestones/checkpoint_000345024_88326144.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:184346ddfe7fa34d6f66970127b5e95b1bbe235cccf9ab028a08ebb8c0f3eac5
+size 20797011

checkpoint_p0/milestones/checkpoint_000359328_91987968.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:284f1a2aa91a51e976a4ec2e43c628e1a7196ea7a4efac45dce5100a00b1a028
+size 20797011

checkpoint_p0/milestones/checkpoint_000373632_95649792.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:00704e0aacb7f4e459942189bff9df6c23f7a9b940ede4e9af17179a4524e865
+size 20797011

checkpoint_p0/milestones/checkpoint_000387968_99319808.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec936c2a9d639958eb96a6c022196402b5bcee427fc390ab796ffdd87db8b522
+size 20797011

checkpoint_p0/milestones/checkpoint_000402240_102973440.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d76536852a9f7f4b7608214d3a7cec4a873f7320ebb56af6a1b30c449c51f46
+size 20797067

checkpoint_p0/milestones/checkpoint_000416352_106586112.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c2d92b070ddabc18d22662494c6bdf912a044a9686761e6b14716f20f14b8d7
+size 20797067

checkpoint_p0/milestones/checkpoint_000430592_110231552.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b82af7b90d74780a30bcbe24c0bac19d29dc8ea8e0cfef0b51069022c7519787
+size 20797067

checkpoint_p0/milestones/checkpoint_000444832_113876992.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2b53ed34dca6dc3c1c8437fc505eafe7f3fae17fde4da3686b10a0eacdd426e
+size 20797067

checkpoint_p0/milestones/checkpoint_000459104_117530624.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a5774f1bda58155b7c9f6f43facb7b5247f0298e29329111b7f50fe773e0752
+size 20797067

checkpoint_p0/milestones/checkpoint_000473376_121184256.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7118f5307c6fb3e0bbf22d08c5a3dd0e47d29a0ace98c364e6ecc16dae8c57a9
+size 20797067

checkpoint_p0/milestones/checkpoint_000487680_124846080.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f5ddf4559c1f8887d53f389ad5da6a1a612d01ebf9be1d152c8ed98f82cd4b3
+size 20797067

checkpoint_p0/milestones/checkpoint_000501984_128507904.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8b71644f9c4b997bd4efad0a65ca2cb77ae9195753ae10757bd863019b36149
+size 20797067

checkpoint_p0/milestones/checkpoint_000516256_132161536.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:410449b25e86ae5220374bf428310bc74c439c351be9f7423847c501cc5e6f78
+size 20797067

checkpoint_p0/milestones/checkpoint_000530528_135815168.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:21e35bdf64aa7cdd0fdcaa9df205f1a7b8c0546e7953b529f11e1a68d66d65aa
+size 20797067

checkpoint_p0/milestones/checkpoint_000544864_139485184.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7631a4a3132b177e30d7a112687875eba8281cb35e55a5bedc048887638698ac
+size 20797067

checkpoint_p0/milestones/checkpoint_000559136_143138816.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:07a61cc3d3b166b9fed03544b3b0111230268a275a65e6c265d5a7819d249195
+size 20797067

checkpoint_p0/milestones/checkpoint_000573472_146808832.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc719db7dcda6850808925c5797ac74b251b877ad16390987a5c2de1a8c3a07b
+size 20797067

checkpoint_p0/milestones/checkpoint_000587808_150478848.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7c70189fac5b625c3dcf5e8e60a1c831af04587c3e369204871d35ab9c53a5b
+size 20797067