Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1701760474.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:925d123aaad6fa5bb101c4aa397fd3071ea82ebe675623fde7f80cdd6b730ec3
+size 82086057

.summary/1/events.out.tfevents.1701760474.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ac6593151bd2eca3bd76350dfd46bd772edfe0a2c1e2cd087e56380141e5c4a
+size 43116976

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_stargunner
     metrics:
     - type: mean_reward
-      value: 71800.00 +/- 8432.32
+      value: 217080.00 +/- 28388.30
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_stargunner** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_stargunner
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_stargunner** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_stargunner
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001938752_496320512_reward_51.210.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f92f881a9f54a693cc107baec7d28bb956ec6eab94abd3c9d8893fa8cc9a357
+size 20795763

checkpoint_p0/checkpoint_001953296_500097024.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a0a704d8cebd198f743fde05edb83415fea76b05e3bba87f5324484a5be000c
+size 20796099

checkpoint_p0/checkpoint_001953744_500326400.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:940de475617e6c397066e03469ff09860409c60e66b0736cef1be6b69ad79ae6
+size 20796099

checkpoint_p0/milestones/checkpoint_000013888_3555328.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7bf0db2246ee16ae530b00abfff74ffbc95a7c51e28a8271fd56bd1f61591a71
+size 20796955

checkpoint_p0/milestones/checkpoint_000027968_7159808.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45e717ec7cf8aeb9b1e88850882d94c573a37a2c332396273b366e30a8c5d504
+size 20796955

checkpoint_p0/milestones/checkpoint_000042016_10756096.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3bd557317154aa1f05bf21b2718c2d47973fa0491436413d90cc8eee0633ef81
+size 20797011

checkpoint_p0/milestones/checkpoint_000056064_14352384.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee76b97e138e0b4a53637f9cfdb13e2793d142b111891bfb942c363930529b07
+size 20797011

checkpoint_p0/milestones/checkpoint_000070144_17956864.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85492cf1f094ba0668a50d389c98508e4d0ce4315ea58be0458b860581c25024
+size 20797011

checkpoint_p0/milestones/checkpoint_000084224_21561344.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c8be9880d2a7a590d94a32e16e1c8ecf5faa2dca66b5b794678afb69b39aa88
+size 20797011

checkpoint_p0/milestones/checkpoint_000098208_25141248.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93ef03e2830192ab3c32a122a93c5cf898da082952b9ac1229ca98edef9ed725
+size 20797011

checkpoint_p0/milestones/checkpoint_000112288_28745728.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:62d3fef1da0909a975750ad5281c0859fa3232f2daf8042643f70ac0d542acb5
+size 20797011

checkpoint_p0/milestones/checkpoint_000125952_32243712.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ee77a84eeacdb65b2c6e557c3d88b10c632675126abecada7365b526d3da440
+size 20797011

checkpoint_p0/milestones/checkpoint_000139424_35692544.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f70ee64888a82d7a6a5bbe03e6396af419fca8fbe5df9fcd2eedc2e87af81d5
+size 20797011

checkpoint_p0/milestones/checkpoint_000153376_39264256.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:008e9fa6992bf3d305679f7618cb1552539073340bb8f26bef93678a28536684
+size 20797011

checkpoint_p0/milestones/checkpoint_000167392_42852352.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f239331aa4fb99f591aa1e029b77a97454edf4b4cf89a92bb6c6f4b018987fd
+size 20797011

checkpoint_p0/milestones/checkpoint_000181440_46448640.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d4bc08747b2dd124a21e220dfa8834f882f04f4c835f2557b0918944b944d78
+size 20797011

checkpoint_p0/milestones/checkpoint_000195552_50061312.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09bef6de45b9274d37ca9249ab5ceffe0dd90212aba121a6b350740b447d40e5
+size 20797011

checkpoint_p0/milestones/checkpoint_000209600_53657600.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b716d668ae786b0bc9c70d285f892fe2c0e8a7501d074f618f97985738bb019
+size 20797011

checkpoint_p0/milestones/checkpoint_000223520_57221120.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:991772859991738dcfcd929fb7df18c40b803d836aac44feb9f64637eaee3ee8
+size 20797011

checkpoint_p0/milestones/checkpoint_000237536_60809216.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:664d7d43d9d542a3050ea5a3af1447912b8c060b69f9e7dde9c00d32256b026b
+size 20797011

checkpoint_p0/milestones/checkpoint_000251616_64413696.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d1614d95516f551c96436b7a8d0bcfd8e045a046697ab5ae2217302c9fb163e5
+size 20797011

checkpoint_p0/milestones/checkpoint_000265664_68009984.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f3c80fdd316ed1e224a014115fccc28475c5e6b18a6261c1a9aaa80efcd34ff
+size 20797011

checkpoint_p0/milestones/checkpoint_000279744_71614464.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4274baab325beda6dd199e7502cb3a64c61db4171dbb0d8a399fcee5b3e232d6
+size 20797011

checkpoint_p0/milestones/checkpoint_000293792_75210752.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9585a5e36d4549ad68730f9c0e029cee371ff44a07404cba19d97e15f4065b4e
+size 20797011

checkpoint_p0/milestones/checkpoint_000307808_78798848.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46cd356534b97c7b1d308398d7b715c059edc45969811de5f8d0e36464ff66e5
+size 20797011

checkpoint_p0/milestones/checkpoint_000321856_82395136.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a76ba4f23a7ffc7bd19d579bbe25b3ae747ac8a42a65205602ccc10ec25eb286
+size 20797011

checkpoint_p0/milestones/checkpoint_000335872_85983232.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:266869ee8c412a126b6076a463f424b9c05f9c10f6d1d16965eb1bc478b3a8e7
+size 20797011

checkpoint_p0/milestones/checkpoint_000349856_89563136.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c45850f529c9666ab092ee4f302bad066c88c256dabebb9afcab6543486ea11
+size 20797011

checkpoint_p0/milestones/checkpoint_000363776_93126656.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8beb336968a79d259a46937c70cdae2dbcc56b85b47ca204f4b95d665f029b7e
+size 20797011

checkpoint_p0/milestones/checkpoint_000377664_96681984.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95ce45542d2d44fa83d91f0aeecdd3c2f45c87355131aa0e3338e0f00c2f6248
+size 20797011

checkpoint_p0/milestones/checkpoint_000391552_100237312.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2f5e961cc7bf2f8b25b0016b39db8428459bbfe78f284556c4f2b725fb95268
+size 20797067

checkpoint_p0/milestones/checkpoint_000405440_103792640.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f53da6e93e5e96fec0e2b036b47aceab7bdbda667fe2b4832f8020200e60cc54
+size 20797067

checkpoint_p0/milestones/checkpoint_000419360_107356160.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3a6e1042d2fdd72ca9d4a232c36d9c7dfb1e4de397a22094e2751e8726bcce3
+size 20797067

checkpoint_p0/milestones/checkpoint_000433216_110903296.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3c61eaa264e42feeee212ccbc971295e209412cdd42b47b52f181203bb19b27
+size 20797067

checkpoint_p0/milestones/checkpoint_000447200_114483200.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b3b4fe63c59b9e4b3b5e94b593304756039a0965ac912996e6e5911dc7c4c28
+size 20797067

checkpoint_p0/milestones/checkpoint_000461088_118038528.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3cdd6ed4f7df56cf37bcf3820421a40a5fdff208a65c6d579d63e170da984c4
+size 20797067

checkpoint_p0/milestones/checkpoint_000475040_121610240.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6aec2ad8c256165e543f5bc67857c4ad158f1d6c7a58f05dedb31597655a2379
+size 20797067

checkpoint_p0/milestones/checkpoint_000488928_125165568.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1eda7267552a30f5d3bee2c230b0a538fc61e69bc43df8d6c7c593cf91059b6
+size 20797067

checkpoint_p0/milestones/checkpoint_000502848_128729088.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:194d0a1a18c2722da18174ac7ad9cb8215d4934569e0f64530b83ab52ca06803
+size 20797067

checkpoint_p0/milestones/checkpoint_000516800_132300800.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:792aaf7a76d09d81f8b5ff13b729cc2d8cb720869b229341fcdf299e0f057965
+size 20797067

checkpoint_p0/milestones/checkpoint_000530720_135864320.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98b5fb04952a87fb160a983c17f3dc27bee94a9782f57ddce5b4f07110043abb
+size 20797067

checkpoint_p0/milestones/checkpoint_000544736_139452416.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d878f9f64d55a2170a0cce726aa4438996dc808686c4d1cdcf5e7cc81810e6e
+size 20797067

checkpoint_p0/milestones/checkpoint_000558784_143048704.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a00c22c7e396eebcc98149ca78ebe2d53bac5b8c7ad9c6f35f674ea198b8ae49
+size 20797067

checkpoint_p0/milestones/checkpoint_000572864_146653184.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b3f2a021d2b92ff8048301d35517e38c7410569ae956b9724748efb034effe0
+size 20797067

checkpoint_p0/milestones/checkpoint_000586880_150241280.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d76fb9855a2c31f7c0d2dd203e28a6df27cccc855989a861178c728a6f12a02
+size 20797067

checkpoint_p0/milestones/checkpoint_000600960_153845760.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fd2ca6a96b74563e39fe09e7f493f9c77854d265b7505e46051b57804585708
+size 20797067