Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1698824082.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46a9650243c3504948a11814a18f1389b23d100815904bc484c1c234e00e6edc
+size 89267843

.summary/1/events.out.tfevents.1698824082.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e58b84e4bf3e81d71e8f1743774ac095c78b9362005397cd24477866d5c4675
+size 46890053

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_doubledunk
     metrics:
     - type: mean_reward
-      value: -0.60 +/- 1.80
+      value: -0.20 +/- 0.60
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_doubledunk** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_doubledunk
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_doubledunk** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_doubledunk
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001536216_393273344_reward_0.460.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d77295abf4b7986952e38f47b4e13be1bbb1011df772fb165ba4a19f54ffa2dd
+size 20795763

checkpoint_p0/checkpoint_001951928_499695616.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd411de43d6812c7c557b0a74eb375228c5c2138c94382239761f52de561ec97
+size 20796099

checkpoint_p0/checkpoint_001953080_500006912.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43000c5814518eed059e252dfb47c6ff31fab2dd8ff879f853f493faeadf509c
+size 20796099

checkpoint_p0/milestones/checkpoint_000012384_3170304.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:debb76056cdf332a007842729591a75aef607ebad96427f31ab4e9f6a9ddc772
+size 20796955

checkpoint_p0/milestones/checkpoint_000025280_6471680.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6dff08274cc148d15aaed32494f5ddedd992d5187358d0e81709f1618a1f96d2
+size 20796955

checkpoint_p0/milestones/checkpoint_000038176_9773056.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ece6167d47566435eecb7815c0132be1d39d60da36c0f301d6b46034eab4a36c
+size 20796955

checkpoint_p0/milestones/checkpoint_000050976_13049856.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6d261989c2e7ba360e970648f3927af36548819a0187e58809ca6972551d73a
+size 20797011

checkpoint_p0/milestones/checkpoint_000063904_16359424.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8cff83324c588b469264c92315c2fec18710502d4ef1be2a4eb201382f41f13
+size 20797011

checkpoint_p0/milestones/checkpoint_000076800_19660800.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6e2a2189f74b84433e23f1e82d6b39972b9aae60763c1ca8f0e6fb4e947e0bd
+size 20797011

checkpoint_p0/milestones/checkpoint_000089664_22953984.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a35abf779719e4095e0eb376de99b78df6c3599acd1c3e1442fd2b014d64c8d2
+size 20797011

checkpoint_p0/milestones/checkpoint_000102592_26263552.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:edabee6277ae45e342c04e959627d76875fb3611be9d3528a59565cca54130b7
+size 20797011

checkpoint_p0/milestones/checkpoint_000115392_29540352.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0f960727b8afde31ecd02f791619da19f905ff1ea69428e84b077d3a8081682
+size 20797011

checkpoint_p0/milestones/checkpoint_000128160_32808960.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c2cc1b5d06571afd74027a98943b430e28d0462afaa396285dfeb0fd9d4d4d2
+size 20797011

checkpoint_p0/milestones/checkpoint_000141024_36102144.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c62de335b1131119baf8e3031594f2a2abf9cfcb1f210477daec952e9b707c22
+size 20797011

checkpoint_p0/milestones/checkpoint_000153888_39395328.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a69f93337475901a709b2704a3162a15d6254acc6d33c175525b03b14ae033af
+size 20797011

checkpoint_p0/milestones/checkpoint_000166560_42639360.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7236a51efb1dbfb273f178ce4b036ac24b21dc799055b1a43336cae780185d3c
+size 20797011

checkpoint_p0/milestones/checkpoint_000179392_45924352.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a18c33bdc166f3050d28013e18b9ed774910819650808f7c8fe63b3126589ff
+size 20797011

checkpoint_p0/milestones/checkpoint_000192224_49209344.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e465facda97ab3afb34740fe76af866f8751ac5053ffc2b5e49f67012fe0ed56
+size 20797011

checkpoint_p0/milestones/checkpoint_000204960_52469760.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e909a2815bdc2638572205f095437ff5611b25b4e251995b49a0443324f334b2
+size 20797011

checkpoint_p0/milestones/checkpoint_000217632_55713792.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1d03b5c527026c8502ee5660dc06e6bb2c796030fef399c6fdea826f10105f2
+size 20797011

checkpoint_p0/milestones/checkpoint_000230432_58990592.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d258e97d687f763e8dc76cca763b89ac121503d74cb584910b34f16aadd73286
+size 20797011

checkpoint_p0/milestones/checkpoint_000243232_62267392.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:967eea47e89c55a6dec336c1317c97a85b5e1034622fb0c48755a789baf099b8
+size 20797011

checkpoint_p0/milestones/checkpoint_000256064_65552384.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ae4ea7cd75c0f8040e9ea34e9ed7dd7a85fd073893af079b302c170fe826071
+size 20797011

checkpoint_p0/milestones/checkpoint_000268992_68861952.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4542c6d54740df221ff2e2ce4128a0b27adf4f3d9d09a75e01b0544bf25fc85b
+size 20797011

checkpoint_p0/milestones/checkpoint_000281824_72146944.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e67d797f7676478fae3862fca1d1d4a28d2f2f5f995c8489f274e7eec2c8a1d8
+size 20797011

checkpoint_p0/milestones/checkpoint_000294720_75448320.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83d4f35c105c110ccdb069d8620af5428d76024368014656b2ec5f69700339ae
+size 20797011

checkpoint_p0/milestones/checkpoint_000307680_78766080.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9d25ecf637495ac12d22e973f8be3a51358c6f115bf7ce4052ee4eeef40369b
+size 20797011

checkpoint_p0/milestones/checkpoint_000320416_82026496.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cb5423726a065eef11d85b4c6c6fc0fb52bb533722bbed007d1c895c13f0124
+size 20797011

checkpoint_p0/milestones/checkpoint_000332608_85147648.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcd4b7ff894760e31352e18d3d6376fe7811afbb77fd85624df0264f4dccea8f
+size 20797011

checkpoint_p0/milestones/checkpoint_000345312_88399872.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0259d418b4f00c6aae2e0ae3bab69955aabc7e2bc567e5d1e79de90e3b505c38
+size 20797011

checkpoint_p0/milestones/checkpoint_000358176_91693056.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6aeba675041972d80f85c376cf2ad416520ff568e713a111097085462b2767b2
+size 20797011

checkpoint_p0/milestones/checkpoint_000371072_94994432.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5339ac0d0cca46fdebbdf8e04480fa62d84725ab5fe96f5924591d2f4fac13cb
+size 20797011

checkpoint_p0/milestones/checkpoint_000383936_98287616.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd9f71064f4b0c28ad36784443f03c194af813a53e7e3d854e1a154f7889a510
+size 20797011

checkpoint_p0/milestones/checkpoint_000396832_101588992.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eebc7d7edd5017e6252a121fbd5344a50117713e21e9861ab70de7ebd5fd5ff1
+size 20797067

checkpoint_p0/milestones/checkpoint_000409728_104890368.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48b3f43f921e2908dfe78354574dc1c6c8278deacac894a9956953fdd3f997e6
+size 20797067

checkpoint_p0/milestones/checkpoint_000422592_108183552.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1249b783103506e3018603800707efa8b22cec039958fbaa1cf32f80fec2e4f
+size 20797067

checkpoint_p0/milestones/checkpoint_000435392_111460352.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf688786db384003e8f0759f5e2e5543f4b4678672b9a236f765f0de47ea8f29
+size 20797067

checkpoint_p0/milestones/checkpoint_000448256_114753536.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e19c8808e00180da3bcf910f2fda4fc191ae991a814c4e9fe99f019ead633fce
+size 20797067

checkpoint_p0/milestones/checkpoint_000461152_118054912.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7aaf328cbaf76488a1f2f3006879d81729cc8762aa1eab522fc21624650549b1
+size 20797067

checkpoint_p0/milestones/checkpoint_000473952_121331712.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b6cb566c54ad255a070e69cdd42868ecf4d9410963bb43430a196028fec2ed9
+size 20797067

checkpoint_p0/milestones/checkpoint_000486816_124624896.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72c7015186577d8d8ce4f4424a7dac6559afad4a1cee1f031412936543c8c811
+size 20797067

checkpoint_p0/milestones/checkpoint_000499584_127893504.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:346063ebc930d6c2d09a3ff5b88b62c840d82b7b54562352c201d59ed0712626
+size 20797067

checkpoint_p0/milestones/checkpoint_000512384_131170304.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3131df0bcf9ca3c88d703f5ab9cfa025304669fd59a85ea44d89ff06be52b26c
+size 20797067

checkpoint_p0/milestones/checkpoint_000525248_134463488.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66ec390938911291e15071bcb725b733cdf4b9d31f32866da0319bcfe4917a33
+size 20797067

checkpoint_p0/milestones/checkpoint_000538208_137781248.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bffa2a88e5e440062223e86bcca79c4c6c79d70b4421382507227709f093d114
+size 20797067

checkpoint_p0/milestones/checkpoint_000550976_141049856.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39154d96ca27e8f6134e3863d2dda5e8bb4f74aa61de27c071903f52c70176c8
+size 20797067