Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1700193818.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0e4b493f4491651cd3cb24732b8e42437e20dfd506852a13b3c56c8965dcb49
+size 83257098

.summary/1/events.out.tfevents.1700193818.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad23fcc5f9942abeaa243ed387f8b288e8a05084db5017d087db04b2eab9b667
+size 43735780

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_phoenix
     metrics:
     - type: mean_reward
-      value: 69115.00 +/- 36640.02
+      value: 552421.00 +/- 20266.68
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_phoenix** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_phoenix
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_phoenix** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_phoenix
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001961840_504471552_reward_1354.400.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fe91f8ce401cbd26367149b8b321637c79dc0acb99d1becb89e57d86977950f
+size 20734131

checkpoint_p0/checkpoint_001970096_508698624.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:383115df9ffea15c4f2fabd21a86b71cd2c70253b2df9f947d2fba3d4a2cf8d8
+size 20734467

checkpoint_p0/checkpoint_001970560_508936192.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:caead47430d7a52da35cb48027676db88efd7c3c2ce8799695fefc39042d00c9
+size 20734467

checkpoint_p0/milestones/checkpoint_000013696_3506176.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c67d8bbadd47b0f109f4541542c55fac32461c739cd31eacd5aab37a7f6ab00d
+size 20735323

checkpoint_p0/milestones/checkpoint_000027776_7110656.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce0815d3cb5ee26e10014602c31da002bd385f5ceecf4297efb67af21bc8a03a
+size 20735323

checkpoint_p0/milestones/checkpoint_000041824_10706944.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10d4c82e41404d7d7546dbdfe99900c289f6041a85c5310e6a300a853af236fd
+size 20735379

checkpoint_p0/milestones/checkpoint_000055904_14311424.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:802c3d6dffe4eedc37d2ee3a8aa243e39e4a37ae1c4526a7475b7475fc80e7d0
+size 20735379

checkpoint_p0/milestones/checkpoint_000069952_17907712.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83c44732208fd1c63a7239601a9b3cb93ef63bf59dad77f0be7ce9d71920291a
+size 20735379

checkpoint_p0/milestones/checkpoint_000084064_21520384.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1097a2681bdb7a65a57e86872f4b2cbf4387f6e1d010d6f98a3f886c83e00b5c
+size 20735379

checkpoint_p0/milestones/checkpoint_000098144_25124864.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:476a1f17cb1fb99e3e82cc1d4d6a528076dc1be0b9974a1554c576010bdd1fbc
+size 20735379

checkpoint_p0/milestones/checkpoint_000112224_28729344.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0c4ab9f3dfdea2333244359bc14e9b1b8385e650bdd92a0c450539f9e015325
+size 20735379

checkpoint_p0/milestones/checkpoint_000126240_32317440.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4f5b031c2b4a41a96c07baa99f0b12f02d4f9ccb01f580277a8186bd6c0ed68
+size 20735379

checkpoint_p0/milestones/checkpoint_000140320_35921920.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:53405737f9142937a48a742fc98e2021e9cf1ca59f7fba72fa20cb8dac70f32e
+size 20735379

checkpoint_p0/milestones/checkpoint_000154368_39518208.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a4389e5c58488d30a5e288cbe38f225a7ad3932117052311286118e095d7080
+size 20735379

checkpoint_p0/milestones/checkpoint_000168416_43114496.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dfaaf162ae6e51f48f687f80c0418c9e30e594c499a65f1bc218c05e92371a9c
+size 20735379

checkpoint_p0/milestones/checkpoint_000182432_46702592.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:334a6e001ace28afd8cb5f2105cd610ac2cce0d49eb018f562134526d04e6790
+size 20735379

checkpoint_p0/milestones/checkpoint_000196416_50282496.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da640eea2d5b511181c89c4ebac610bd7990b81babc569e1d79b229dd29e57d5
+size 20735379

checkpoint_p0/milestones/checkpoint_000210432_53870592.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8a9af9b1d6507f5a6e7c8b58b90311f1b93a5fb9e54a3e93e5a05f9f6216ea2
+size 20735379

checkpoint_p0/milestones/checkpoint_000224480_57466880.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3d1f7758afe1a3f35dd1b9789ece8e875e10c098e5da6586c9476c326ee7f97
+size 20735379

checkpoint_p0/milestones/checkpoint_000238496_61054976.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc72e0621e07e124572ef22f3e0f8dbdaa5cd01432803d975caf50b461cd8817
+size 20735379

checkpoint_p0/milestones/checkpoint_000252544_64651264.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8fccd22a68e6df607856b642715b0288b76ea293ee295ff3672f24c920d5d152
+size 20735379

checkpoint_p0/milestones/checkpoint_000266592_68247552.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c80158a87a51c8cf1c2fa91c32bb0ae94c2f07d01410d7ced7496834ab6f6c18
+size 20735379

checkpoint_p0/milestones/checkpoint_000280672_71852032.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79a7b331239741f9d51e7fde7c2c94368c461f45408e5fdcaad6b580826b7bde
+size 20735379

checkpoint_p0/milestones/checkpoint_000294720_75448320.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:656083a83b047c747bc11d5db4ff0f59cfd2593561d5e1fae3f9a52522607fdc
+size 20735379

checkpoint_p0/milestones/checkpoint_000308768_79044608.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5952ea8c3688467b895c621c8bbf3dc863225507dbc8686093330fb6e581397b
+size 20735379

checkpoint_p0/milestones/checkpoint_000322784_82632704.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a1389fefd82451df65cb896a4ceb52fb1907246adcfae7e68ea87489746e019
+size 20735379

checkpoint_p0/milestones/checkpoint_000336768_86212608.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdff36a0e974808b81be74187b531d95e618c643bd029cf05751358ca812b065
+size 20735379

checkpoint_p0/milestones/checkpoint_000350816_89808896.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecbceaedcd2af6cfedde214e9cc14eeba9fa6e05f50c1246a10dcd87bfef5b5b
+size 20735379

checkpoint_p0/milestones/checkpoint_000364864_93405184.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5f6e366cdca45a615809c4df213353bf861e30a692077189be7fed44433ce3f
+size 20735379

checkpoint_p0/milestones/checkpoint_000378912_97001472.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cfa2b03ebbe7ffdb4a088a969b7d76fb28376f4a27efdae3c27900d0ce8992e5
+size 20735379

checkpoint_p0/milestones/checkpoint_000392992_100605952.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7818056f3cf9fcb4dd555fd4a78925abbedd21a8d099be6dd15067b21e3a242
+size 20735435

checkpoint_p0/milestones/checkpoint_000407008_104194048.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9120176b76bd590151caa80ddfaaaabf90c1733ff79f2124925983bb03a7e501
+size 20735435

checkpoint_p0/milestones/checkpoint_000420992_107773952.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:171495ab0c44e248bff8de50d4308db0960068bf118b36024458316b6a85e4f1
+size 20735435

checkpoint_p0/milestones/checkpoint_000435040_111370240.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca7854d1f17822414c02f66f8c838ec3bb739c0eff3ee6a2ade6eef1cb2834f1
+size 20735435

checkpoint_p0/milestones/checkpoint_000449120_114974720.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90a3d09262d42f339c8f5774c9e237deab99d0e1d68b806d86ae895867db5732
+size 20735435

checkpoint_p0/milestones/checkpoint_000463136_118562816.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eaa7fbf5815881547ab961125eecdd39a4900b1f6a81c01ee52467750ccd3800
+size 20735435

checkpoint_p0/milestones/checkpoint_000477184_122159104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:561e869c265f65960ab8ea7f8ce1535bc84146a17261500a98233dea23981b70
+size 20735435

checkpoint_p0/milestones/checkpoint_000491200_125747200.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9d152b2a8725300b89c3e089bc4d46430c0831e99884f7dec7d06ca2314a2e7
+size 20735435

checkpoint_p0/milestones/checkpoint_000505184_129327104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78f4d75321493d2dde07cb1f5a0dce4c6a0f5f128e1ef48b224eefe51f631d21
+size 20735435

checkpoint_p0/milestones/checkpoint_000519264_132931584.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2b70ecd642837adf8764742d06eaf916fdb0b338f04471adfd92e92a88b7713
+size 20735435

checkpoint_p0/milestones/checkpoint_000533248_136511488.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:92968ea61799215facedc22a27ca93c285003d913d0bc3f55f023929aea74288
+size 20735435

checkpoint_p0/milestones/checkpoint_000547264_140099584.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81783383b205814546e77840560c275cb345d6b8df126ea3e5d9a0f8062b9168
+size 20735435

checkpoint_p0/milestones/checkpoint_000561216_143671296.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f012ca8270ca8b2ad3b5fa0d972600c763f2dceb3fadfedbe2adc50d0bb1fb9
+size 20735435

checkpoint_p0/milestones/checkpoint_000575264_147267584.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2230cdfb3d4f1403613a26e895b190b63cf0050ac5287a71ece60853b88647d9
+size 20735435

checkpoint_p0/milestones/checkpoint_000589312_150863872.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7632e654fa69619e4e17a63427690e265f9abd67be20f94ef084884e23a4f768
+size 20735435

checkpoint_p0/milestones/checkpoint_000603360_154460160.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b573d43673b4eff91af27d317dfaf34a868eeb06cbb3d6d9d62f27fd0c871554
+size 20735435