Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1700946348.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:932534f7a4d7e159fd3aebd12fc1ce6636cf80e746c288e241daf6a1148a5c41
+size 89936761

.summary/1/events.out.tfevents.1700946348.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a832c59073baa33ccc11e20859c293f148d24ba88f757b68a9cb70a31b086ee
+size 47233294

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_seaquest
     metrics:
     - type: mean_reward
-      value: 1856.00 +/- 28.00
+      value: 2874.00 +/- 12.81
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_seaquest** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_seaquest
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_seaquest** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_seaquest
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001913536_489865216_reward_36.080.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1ed1067b2bd0a31249b09c09290a2467c29eaa011b5378a8b01206bc82f5f6a
+size 20795763

checkpoint_p0/checkpoint_001957968_502497280.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38a58d0035c8380c345ff1dd49d6a9a6803a5f5016feba166b7b6d2df79de340
+size 20796099

checkpoint_p0/checkpoint_001958192_502611968.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c430a2a191e6bcee41ec5a57210c6aeabed572404b6663986199ad4cc611a57c
+size 20796099

checkpoint_p0/milestones/checkpoint_000012448_3186688.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:997c83f821c2ea2f4ed46dbaaaa1a7c2914c90783bdf576a2148d330820bbbe1
+size 20796955

checkpoint_p0/milestones/checkpoint_000025184_6447104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8aaa7e67441686e0976dc66b19217ca4152947357e287009b7e9742e97a6a36c
+size 20796955

checkpoint_p0/milestones/checkpoint_000037888_9699328.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a16cde45ed7e61871cb69314fcd751792c2739979b025b784375f2410d36e9ec
+size 20796955

checkpoint_p0/milestones/checkpoint_000050656_12967936.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:288bc5336cab62298208a2c4292088484ae19f5ce89892e2557ae4263034b2e6
+size 20797011

checkpoint_p0/milestones/checkpoint_000063456_16244736.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36d9691a94bb3a5d07fbb3edbaef92a8379c96bf933a4abb2c706e2ae9e6c3b5
+size 20797011

checkpoint_p0/milestones/checkpoint_000076224_19513344.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2787925f7c09a55ec7c3657362ee6f2d555223831d6d1cdc5c7f7bb5a91cd32
+size 20797011

checkpoint_p0/milestones/checkpoint_000088992_22781952.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de68cccaccca6f27e7a181790fd9df51e27c637c69eb23ace922b93777587f7a
+size 20797011

checkpoint_p0/milestones/checkpoint_000101792_26058752.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:624a2218f55d0356913b4c03ae610506c3fd450d1f690413c8c518baeaea766b
+size 20797011

checkpoint_p0/milestones/checkpoint_000114144_29220864.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb889dba2917d77e57cf7fcbb2869bd80a6e7a1369d0580c1727f3d9f3b1429d
+size 20797011

checkpoint_p0/milestones/checkpoint_000126720_32440320.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:80b194f4e46ca8c17447d22a45a7c723cd079e970b73e3bab870d8728c89ba30
+size 20797011

checkpoint_p0/milestones/checkpoint_000139584_35733504.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb48d7cbc4345938a02f9cf3ec14114e11d5a8a79264cb88e7a210c21e87af83
+size 20797011

checkpoint_p0/milestones/checkpoint_000152320_38993920.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca05d120a3da29960db5038cf015115eaa2c9d026b639e6c3883226384d61884
+size 20797011

checkpoint_p0/milestones/checkpoint_000165216_42295296.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:116ce2601aa92a69a393cb9bc8a7bb95c6cfd710c3e2d2eeba9484717031c607
+size 20797011

checkpoint_p0/milestones/checkpoint_000178112_45596672.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:480d35abd84b2a0587fa0927fa4e144d3b275de9db2627e3d261c7f21b9956e4
+size 20797011

checkpoint_p0/milestones/checkpoint_000191040_48906240.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89ba7eef1bcd95b2ff16b4f056a7f54df32a7907f3bfd4075ea254c065879914
+size 20797011

checkpoint_p0/milestones/checkpoint_000204000_52224000.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a7a41d371da2123e1a3ac43c8c9e846ac8114e8eeac49ea4a17e0c2b51bc64e
+size 20797011

checkpoint_p0/milestones/checkpoint_000216928_55533568.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04d99383c907ececbfccfc74c22649717a5a31eb094b3b15cda6c7caa14d8753
+size 20797011

checkpoint_p0/milestones/checkpoint_000229856_58843136.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6965e109e1397ca88c0bc99b6c35b80af6b7fc7d0d54c756e8dac73870628b3d
+size 20797011

checkpoint_p0/milestones/checkpoint_000242752_62144512.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6d440ac52d10354bf46486728d56f90c02316970a885c5093a759a812b5895e
+size 20797011

checkpoint_p0/milestones/checkpoint_000255616_65437696.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dce40f7056430a2555d2b66bf3c67a1d850165b59f6ba91252bf39d059f7f001
+size 20797011

checkpoint_p0/milestones/checkpoint_000268512_68739072.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8401f7d55f27ef840c35bb714b7d9f35660f90af989ed78509bcf4d41ea56f77
+size 20797011

checkpoint_p0/milestones/checkpoint_000281376_72032256.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02424c61e2cfc10ad851429d9cc344fce44f25d6bfea652c40e69efbfc94f72a
+size 20797011

checkpoint_p0/milestones/checkpoint_000294304_75341824.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0b1045f56ef76a7ab9e504bdc5179ea69c2336b4d11ad8959ff5fb133a9d9fe
+size 20797011

checkpoint_p0/milestones/checkpoint_000307168_78635008.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a00e7aeb5525dcf4aa6f5a3a6e5b6cad33bed787bd9b3805dcf7ea7a4e243190
+size 20797011

checkpoint_p0/milestones/checkpoint_000320064_81936384.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:545a24d96f68ecbbb458c30bd9a709c6162a04937b42f6e817f31887b5e356b1
+size 20797011

checkpoint_p0/milestones/checkpoint_000332928_85229568.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9fe2218212353e734d4e67363e1716e1abb79824a798334e9cba595eb16c567
+size 20797011

checkpoint_p0/milestones/checkpoint_000345792_88522752.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb747b859e71fe89af4268c65532d9799a349fdd1bab58ef2be256ff7e6ac2be
+size 20797011

checkpoint_p0/milestones/checkpoint_000358720_91832320.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:302ee1367aa9a4ea44b5e8d105edb93499400be56906204c2b0f52a587b01b63
+size 20797011

checkpoint_p0/milestones/checkpoint_000371616_95133696.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f25c06a684d779bf4933830644528b8f7fb2348dbeb2d4863a299a0e1e581224
+size 20797011

checkpoint_p0/milestones/checkpoint_000384448_98418688.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86bf2121ad320946f7dd79837203b92f572de940b709d40f58d50aa007727a50
+size 20797011

checkpoint_p0/milestones/checkpoint_000397280_101703680.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25ab7f733d9f20d565ad2806fdcb0a4ea3e3aa3605df01f6a03bbefc1f662af8
+size 20797067

checkpoint_p0/milestones/checkpoint_000410144_104996864.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d82aea8dde801cc633ccc51f43bb17a563f147c732c87a6d6317a138fee44ea4
+size 20797067

checkpoint_p0/milestones/checkpoint_000422976_108281856.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8973e3bbdbe08e206098cd6b30f1aefa2cb57c65e2cc7d789c2c7ac57bfbc7f
+size 20797067

checkpoint_p0/milestones/checkpoint_000435808_111566848.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6090a3a7e1eb7d0c276ef41ae8a0261f64dbdb7760723a75455b034b990efdc3
+size 20797067

checkpoint_p0/milestones/checkpoint_000448608_114843648.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f27ea6678f0033611a18a7f25ecbad366628e81f414b0e4d1dfa42be16e7afaf
+size 20797067

checkpoint_p0/milestones/checkpoint_000461280_118087680.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba2ad6b1deea74f65345754f31f546056393da13bc3fede41c268cd66753a555
+size 20797067

checkpoint_p0/milestones/checkpoint_000474080_121364480.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2260efd75e82a15a2c946514a997307def18ac7d2632e523b139d614cd8f4ff4
+size 20797067

checkpoint_p0/milestones/checkpoint_000486816_124624896.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:575c8b8b3fd40a13a9f5f48de8a53fd8965084dc4f50001b911db5dc7ebc1318
+size 20797067

checkpoint_p0/milestones/checkpoint_000499520_127877120.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a59c508a82ad3ddcd79b0f8321ed3cf54b470864711eec8c575445355c5ce3cc
+size 20797067

checkpoint_p0/milestones/checkpoint_000512288_131145728.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a72cc99c2e655803857c63a369e6da12dfe3c8d65b794b5f210718f90dfb178b
+size 20797067

checkpoint_p0/milestones/checkpoint_000525024_134406144.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fd945df16d7d2dd4fdf7a36863b84f36ece8dc6542ef689a16b98823d14262b
+size 20797067

checkpoint_p0/milestones/checkpoint_000537824_137682944.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42a5d7f96a8260bfe53f7897c615071292644931788d76269d067f10d18b77a1
+size 20797067

checkpoint_p0/milestones/checkpoint_000550624_140959744.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d22d26d10a6f83407d449c9e5116fa82ea1c911205d8be65f466cb99f6c00842
+size 20797067