Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1697545916.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:739332f8a54de12ecb2e276ef86376e1952fa149ab37d8b9e0b5de4780f0840f
+size 1180

.summary/0/events.out.tfevents.1697643349.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:636017b4237c16c0d1a347ad47d74ade18c16b5b3c2a2cf68ce00e7ec4ca0357
+size 40

.summary/0/events.out.tfevents.1697718879.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36d664e354cc66e8b10eb1172ff5e5531c36bc4bf108be3085507b8ab41ee888
+size 2354845

.summary/0/events.out.tfevents.1697722839.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ad65f968afb1f5c29929ceeeea4896e1bc156f28587b3110dc0736a43d0199b
+size 821

.summary/0/events.out.tfevents.1697722878.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e257b37ac11c574f1f4c3788e68433cce06bfef113236255e9f24c2639b89005
+size 57595

.summary/0/events.out.tfevents.1697723034.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06d64b42be4da7b75c6842204a3e4ea84815b61fd2e50045465431f422b38432
+size 59732

.summary/0/events.out.tfevents.1697723216.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b6992e0b102159c3c4c7a3acfedc613aa51353f1c8753eb7722c6a4d9395bc2
+size 23983628

.summary/0/events.out.tfevents.1697818119.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c42911dac6f3ce48409841fb3d5c20bb47afca0c85ce804b816975f77269eb5c
+size 64005360

.summary/1/events.out.tfevents.1697545916.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b2ad207cb360eaf7826168045af6dc425777916b664b51d5b545c038965142c
+size 217

.summary/1/events.out.tfevents.1697643349.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4976d78a9aa31688c74db67d11d5ad265bc606dcd8159430cadcdfc9068ad9e0
+size 40

.summary/1/events.out.tfevents.1697718879.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24750301f35e252440f1cadc9c872bac4226423c16e6fceaf8e549923b47729a
+size 1465144

.summary/1/events.out.tfevents.1697722839.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d3222213af83de428f659cc17b506d7034d33b3ffc7a57f6e436390c890845e
+size 168

.summary/1/events.out.tfevents.1697722878.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59ad5f0621689fef8e7526874e2122491c011127875081791846737760166328
+size 29356

.summary/1/events.out.tfevents.1697723034.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5021592fde71c867f360c19a50a630fd67697a8dd245ef599d9e0ca2471de90
+size 31060

.summary/1/events.out.tfevents.1697723216.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9213ad22a2bc89ae760d4ba4892aad067e38bb6a338a65a0b01956709ad9e64b
+size 12713449

.summary/1/events.out.tfevents.1697818119.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6deb015ecf4a11dd917ed62d4ec789bf07aab9e41f120118c24ac6363de50a90
+size 33280131

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_battlezone
     metrics:
     - type: mean_reward
-      value: 78900.00 +/- 30044.80
+      value: 234600.00 +/- 114609.95
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_battlezone** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_battlezone
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_battlezone** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_battlezone
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_000562880_144097280_reward_21.206.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3413858d52aafdeb098940a4603cb7ef85970dcc10dc7b1833714fe37d131b5d
+size 20795955

checkpoint_p0/checkpoint_001967872_503775232.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1bb5605711bdc509eb7099e383293e541311745cdc3c4707cd60633c5d3e078
+size 20796291

checkpoint_p0/checkpoint_001968768_504004608.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3425c5951f2dab69b07de50dcd6ae56ac21d2f1b736d2af555e8583a45decb8e
+size 20796291

checkpoint_p0/milestones/checkpoint_000012640_3235840.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30c50fdb5eea3c638f0f923f2ca48b2ecfb1688a343da52b7639ec2b54464508
+size 20797083

checkpoint_p0/milestones/checkpoint_000025344_6488064.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d99cbf30c03fcab539e21b8085680be6724768a49fea02a56cab50f55a3c398c
+size 20797083

checkpoint_p0/milestones/checkpoint_000038016_9732096.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1bc35a8390caae151a2d8a48fc342343465c3173a89b6f0c8bec5298997382a8
+size 20797083

checkpoint_p0/milestones/checkpoint_000057280_14663680.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5925c73cadca62e21067418b7d0e3dff93d518e1b77b0edb87015a93965cdddd
+size 20797139

checkpoint_p0/milestones/checkpoint_000069984_17915904.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c4f3322b2ecd925e6da9a8d367611d8125dd3deb445772e0fa272af517ebe97
+size 20797139

checkpoint_p0/milestones/checkpoint_000082688_21168128.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:056ca4b261652dd0e0cf4844db187240ee9e151fbcedba26d220ae546ae2091d
+size 20797139

checkpoint_p0/milestones/checkpoint_000095392_24420352.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd93383e052b097bb31f6fd45bcdfe570c88352ee98cb050979a86ff44c9f7c0
+size 20797139

checkpoint_p0/milestones/checkpoint_000108128_27680768.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71f387a5a80ed135b7e6b49d02ac8b1584d41985b5348731d68b093ef2d02d15
+size 20797139

checkpoint_p0/milestones/checkpoint_000120800_30924800.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b865ca4d020feb0ef0ae8584bda4ab662440a771a2c5e0d016d4ffbc9ce3671
+size 20797139

checkpoint_p0/milestones/checkpoint_000133568_34193408.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa9200bf83bbb10d570f1ffbd6a2de1141062d90a0766d3bd2e0e191fa3273d6
+size 20797139

checkpoint_p0/milestones/checkpoint_000146272_37445632.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9f426069fe1acca44783aba3744e3d237aa2214d35a22c4f71aaca009012fd8
+size 20797139

checkpoint_p0/milestones/checkpoint_000159104_40730624.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b803f24a314c23674a470403c45b8f31dffbb60cd24569e1a8d4e64dc5d357bd
+size 20797139

checkpoint_p0/milestones/checkpoint_000171904_44007424.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0962b3dc19ac64c940f81b89a498faeac53adfe0c94a823ffdcb8625fb735aa
+size 20797139

checkpoint_p0/milestones/checkpoint_000184768_47300608.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:016afc0ed8e47bb7ddc04fd2f644ad85ad466652eebe64ec413d7f0c052dc1f4
+size 20797139

checkpoint_p0/milestones/checkpoint_000197472_50552832.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa365cd11171880aa4b126927a82eda698fe43e6e87be051cca65d32cc7caea8
+size 20797139

checkpoint_p0/milestones/checkpoint_000210208_53813248.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ae10edf8335d0081422dbc074a0785a7073d4bd1d10d2772dd8c5814a0551f5
+size 20797139

checkpoint_p0/milestones/checkpoint_000223040_57098240.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74e274cf9f8951e43d24e2da2278826b379b8d9f3d06a22aa70bb43f3f03bd01
+size 20797139

checkpoint_p0/milestones/checkpoint_000235712_60342272.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79107707a49be81b5d0e12fdaa5731d2b7f9660b40cb44e92a851cac965748d4
+size 20797139

checkpoint_p0/milestones/checkpoint_000248480_63610880.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5bd38eff0825c5fb40e048c708bd83ed82884aa44cd0d7da27b7d4e7cf32d81
+size 20797139

checkpoint_p0/milestones/checkpoint_000261312_66895872.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7470cae8c5c83e73df3a528d44290d29cecdffa14815c7e82780df50b268745c
+size 20797139

checkpoint_p0/milestones/checkpoint_000273984_70139904.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37e6842618fe3dcfaaa2eeacdbd8376e4fabf0f6580d926af2bd7068b9b43a9b
+size 20797139

checkpoint_p0/milestones/checkpoint_000286752_73408512.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6cd6371e2d0835ed53bb9aaed0e2058d56d35ce31a2e7fce515d9535041cd2e
+size 20797139

checkpoint_p0/milestones/checkpoint_000299488_76668928.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:508bc304e3ad3bef29850d9ae67e3f4358a4784d2eb79b6b9f1aa435f6a58e76
+size 20797139

checkpoint_p0/milestones/checkpoint_000312288_79945728.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04b295a58825bfd1659af7d677bc1e4d550705117642d49a35ed9cc640bce3d6
+size 20797139

checkpoint_p0/milestones/checkpoint_000325024_83206144.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d8213ff6a93ee5bb116974464bf3de68ce0691d1c80f62b59efeb1e14e793d30
+size 20797139

checkpoint_p0/milestones/checkpoint_000337792_86474752.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b4325d4a89fed069dfdd7ae11cd5ea0c0fc35ec2dc708418938a82efe090599
+size 20797139

checkpoint_p0/milestones/checkpoint_000350432_89710592.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:777cb4a6018e86065534096b3b718ae4089be50bf5ddfa2a3019931fa648531d
+size 20797139

checkpoint_p0/milestones/checkpoint_000363200_92979200.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db86969ea4ad6a842835eeddfd58c83122185294190c68b35a82afd1eb096b40
+size 20797139

checkpoint_p0/milestones/checkpoint_000375872_96223232.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8a98d6ecdf035693361e7265ac6281727cd409d58491c8811fae6a746e03976
+size 20797139