Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1699827641.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:62b1d6329c0997458854151854ed93e738b1dfc329eb7834643835ec3931a5d4
+size 88956238

.summary/1/events.out.tfevents.1699827641.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a05f1c3c4ef64ea5b405d64e1cf6b73508b87b4c403129e64c3c36eae98a073
+size 46718867

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_mspacman
     metrics:
     - type: mean_reward
-      value: 4593.00 +/- 1156.44
+      value: 7131.00 +/- 1180.12
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_mspacman** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_mspacman
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_mspacman** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_mspacman
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001899968_486391808_reward_175.340.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2a4de01b014448e26b4c1b6a37acbc741cc6156d78713419abaf6a6745cb8c7
+size 20740275

checkpoint_p0/checkpoint_001952320_499793920.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5029f795f2f7d9c08268dd0831996bf57ebaf961b9dae3da25e4c763e5c24ed5
+size 20740611

checkpoint_p0/checkpoint_001953120_500006912.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db49bd9a42e69e67023bdc566ba220388a531430bac929ae500734250042f78f
+size 20740611

checkpoint_p0/milestones/checkpoint_000012704_3252224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a6aac241645edba1ac64ce44c5f8b1362d76d987781cfbb4c171138019674e5
+size 20741467

checkpoint_p0/milestones/checkpoint_000025728_6586368.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4162fa407bd3cf9fa3329ae507c00c6f26a098bde023e82c021eeaa72661a9b6
+size 20741467

checkpoint_p0/milestones/checkpoint_000038784_9928704.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c4484c8ea61b820e584a710fb1ee9b30804338bc45b1d64ffa0608579d76232
+size 20741467

checkpoint_p0/milestones/checkpoint_000051808_13262848.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b401c9564a391e4e84263546458d25018a447cb8f7ad25b7df5b781b1dc72b67
+size 20741523

checkpoint_p0/milestones/checkpoint_000064736_16572416.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca3867c9b99d774326cdd739c4f48351800e81100b7946c997a7ed4a24dd053a
+size 20741523

checkpoint_p0/milestones/checkpoint_000077344_19800064.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b04ad8a447a61a4311bf097e68acf930bff6243b4558e08c248126f40b79469b
+size 20741523

checkpoint_p0/milestones/checkpoint_000090336_23126016.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ad9ed5973fcd062cdd5d443b16f7c43a9e6efe6066f7f6e54da59b2e3d6eb61
+size 20741523

checkpoint_p0/milestones/checkpoint_000103360_26460160.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a49a103bad1a4175a26ca1d57780fc107c45d02b4120dda2bbc104cf15ff3cf
+size 20741523

checkpoint_p0/milestones/checkpoint_000116352_29786112.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e1235ee169839259aa23f17783a15d55e607d33ba79b6999a4c77749669e355
+size 20741523

checkpoint_p0/milestones/checkpoint_000129280_33095680.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b6dcbec2f2dda41ac0f9b72f89f2917f64d1bcec4922b66f5afec8eb464f2e9
+size 20741523

checkpoint_p0/milestones/checkpoint_000142304_36429824.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd9110b135e491b361ac63cacb491af4cb906e5c6e16630d1625c823cadb88ec
+size 20741523

checkpoint_p0/milestones/checkpoint_000155232_39739392.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9c7b3a58e071e7bfb3ae556ecd6261334ca2068770425c42a51bb0aad9e8c3c
+size 20741523

checkpoint_p0/milestones/checkpoint_000168224_43065344.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:40c23731cec9747ef3703bd3f6727bc895dfef28fb62e4c8efd9c76b9680873c
+size 20741523

checkpoint_p0/milestones/checkpoint_000181184_46383104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb77d503c3369e58364ee23910a07fe9cd5207300879c33e81fd9951e84fee9f
+size 20741523

checkpoint_p0/milestones/checkpoint_000194112_49692672.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c98c21c06df1ded4577ab2fd7ca9d104a1d25ea52549014106b9e9f2c5734ce7
+size 20741523

checkpoint_p0/milestones/checkpoint_000207040_53002240.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6dd1e5c74c371ad07c3ef1d146063dbf4da093863e5de31722d0fdbc7c5f65d0
+size 20741523

checkpoint_p0/milestones/checkpoint_000220032_56328192.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3794c40b08b3403d3f51a56f286a45451fa043db51f2d406893975bc05efe1a0
+size 20741523

checkpoint_p0/milestones/checkpoint_000233024_59654144.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa55eaad8f8c19fd6466691818a00cdafee9f54250a8ee3264cfe9047ba76289
+size 20741523

checkpoint_p0/milestones/checkpoint_000245984_62971904.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:854082421cd5f1f3fe30d545b643649f70c7d5eb1a2305c309f8de9d35f1ef48
+size 20741523

checkpoint_p0/milestones/checkpoint_000258912_66281472.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45598d8392dad90a04368fb35bba71611f5a29367c54c3b5b8735d6a2c095fff
+size 20741523

checkpoint_p0/milestones/checkpoint_000271840_69591040.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5944361bc7c4e3fd190b1ba8effc032e285ae87fabb88e8e4a20e1bf1e42438e
+size 20741523

checkpoint_p0/milestones/checkpoint_000284832_72916992.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b42eb7db19345fc799b6d9ee08be425729a13fae8eb6ba187eb962d573e65069
+size 20741523

checkpoint_p0/milestones/checkpoint_000297728_76218368.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:700d3db29b94a14badd9077cb6b763fb9770c3a7f903ec09e66c3c27fc4e7377
+size 20741523

checkpoint_p0/milestones/checkpoint_000310720_79544320.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20d96e88dfdf194b751a4ea55d379b463747e36f2be5b7ad757468effb611a10
+size 20741523

checkpoint_p0/milestones/checkpoint_000323648_82853888.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9584db63a082316eeb01801e64750be75226205d741e6aecd1d00a9d851444c4
+size 20741523

checkpoint_p0/milestones/checkpoint_000336544_86155264.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ef64ec826a25419544bd1a38d488c72513fdbf0dbd373c9b47628168649036f
+size 20741523

checkpoint_p0/milestones/checkpoint_000349472_89464832.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9530bbeea20d068f64b0adc4aa66031345c709d265f76a944f3e43fd1becdd9f
+size 20741523

checkpoint_p0/milestones/checkpoint_000362432_92782592.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:865ce3f7f8344311b0e25984f3cdb8c83b90892cf8b0b516cea1575b3a9c129f
+size 20741523

checkpoint_p0/milestones/checkpoint_000375360_96092160.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f20c9ac834d1472fdea0dd426240da72b7a702332b22927f51a584651672d0e
+size 20741523

checkpoint_p0/milestones/checkpoint_000388288_99401728.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6ff15d2ff0540d3f8d16788bb24b84ab286babc434e21639add89f0347af317
+size 20741523

checkpoint_p0/milestones/checkpoint_000401248_102719488.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24662038f57ba915066ddc1a02211e68b906463048114fda5d4538822c79f110
+size 20741579

checkpoint_p0/milestones/checkpoint_000414112_106012672.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d8f53b5909e9c42084b71e298f5a37352375575e3380362285f16cd9a4aa532
+size 20741579

checkpoint_p0/milestones/checkpoint_000426976_109305856.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71f0e66ac9e7b143bbfdd5a4e6e47a926277eb8772134c8e3541da3fcbffe239
+size 20741579

checkpoint_p0/milestones/checkpoint_000439872_112607232.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86306df0f015824fad9786322152538e944a427e1624b432438fb4f1923a0889
+size 20741579

checkpoint_p0/milestones/checkpoint_000452736_115900416.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8905bf0e4a9e0033a64b87d0629577497f9b1cf7435e5748483009f9ae214f47
+size 20741579

checkpoint_p0/milestones/checkpoint_000465568_119185408.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b466935bf5e90a0b7d518f716743fd2c72dc24a90fb6546e0ba00de36bbdc37
+size 20741579

checkpoint_p0/milestones/checkpoint_000478432_122478592.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce5f9e42ebd0b22d2b25e69b95701a57126ec82aecb14ac637fcb4e2772c253e
+size 20741579

checkpoint_p0/milestones/checkpoint_000491136_125730816.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6b42d10ad0bd7cb30f668d5da2e7880c6931b3cb66e591b521f6ce89c693322
+size 20741579

checkpoint_p0/milestones/checkpoint_000503936_129007616.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f94124797401b5113195e1242dd19a05e60e8c0ae98336b1553c7f1613c320f
+size 20741579

checkpoint_p0/milestones/checkpoint_000516832_132308992.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b3f869c9f4f82b2b562d36138a4f0b85914c854f16602e71417bfbc895e0bdb
+size 20741579

checkpoint_p0/milestones/checkpoint_000529632_135585792.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e00108c885be86175571474830adb434ed337fd2189dba63c8e34d522aa2080
+size 20741579

checkpoint_p0/milestones/checkpoint_000542368_138846208.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4bb78d7ff0f2cee3d2eecd435e88cdd2c39306ab5cf546665ae47802be143e16
+size 20741579

checkpoint_p0/milestones/checkpoint_000555200_142131200.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec76b7089a5b98826c7ebe0a8ac0b04af5d529b03cd6eab210ac16014e283741
+size 20741579