Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1702513428.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:459a32fa052fbc1b7ef7de5bfd0d4964b2e41cb89905f6754c1ea4d87211c9f7
+size 85718318

.summary/1/events.out.tfevents.1702513428.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3cb015724c9597fd2985e88f4dc8c002474f5ccf29d9b6d034cef34c9adba5f
+size 45030587

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_wizardofwor
     metrics:
     - type: mean_reward
-      value: 13670.00 +/- 8376.40
+      value: 53850.00 +/- 11044.21
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_wizardofwor** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_wizardofwor
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_wizardofwor** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_wizardofwor
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001860256_476225536_reward_33.130.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:221d059feb7e6f4e281fbceab368e5049575d74349c131dc8eaf4e0575692682
+size 20746419

checkpoint_p0/checkpoint_001951904_499687424.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f6fd168fc335099f5f6d25b767f508c29f390da25b76946bf112cc9b4b5da73
+size 20746755

checkpoint_p0/checkpoint_001953120_500006912.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cfa07a876fdada8729fd56f275128e8331267d64fbd43a0d89a1ef6b2f6d83b3
+size 20746755

checkpoint_p0/milestones/checkpoint_000013024_3334144.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54c40df58b4a04da0080041e041d17c6cfa72af94299486c16e33a4b729991df
+size 20747611

checkpoint_p0/milestones/checkpoint_000026400_6758400.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:105bffde17046ed650eb49864e32df323d9809ed893634206a2b7c6f68cce0ae
+size 20747611

checkpoint_p0/milestones/checkpoint_000039808_10190848.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3796f3aad7a94c8bb394f850b676cbc53d995a22ae9ddf966545c74eee8cfc9d
+size 20747667

checkpoint_p0/milestones/checkpoint_000053184_13615104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef878d5e8b1ff6b90ab2710d01c5e156a36c0377865f17643c64376802d973b0
+size 20747667

checkpoint_p0/milestones/checkpoint_000066592_17047552.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f958f2ceb363d1f2bcdef35bc8321aac6df7c1aa12f27da67fe97fa59dbd435b
+size 20747667

checkpoint_p0/milestones/checkpoint_000079968_20471808.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd0d4d230c7c9e340bab8adedca1b84cb42ff81d510c0dd4516a28ef46528451
+size 20747667

checkpoint_p0/milestones/checkpoint_000093440_23920640.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3a9350f1b8db7cec779604584a51bef81e73e45b2a6be9d7e01907ca566d77d
+size 20747667

checkpoint_p0/milestones/checkpoint_000106848_27353088.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d32cc2b94d4f9989f52cf9211ad90569b0c33d6600ea80da355e4e317288ae56
+size 20747667

checkpoint_p0/milestones/checkpoint_000120288_30793728.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3dad62e5aa6543d4450529c36b3b34d7a41064749a9e156c6aa4defcc24bbff5
+size 20747667

checkpoint_p0/milestones/checkpoint_000133728_34234368.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59e92648030ea364d8ad8cd2e1aa1387e49e4494e508fffd882d5203e0b0cf65
+size 20747667

checkpoint_p0/milestones/checkpoint_000147136_37666816.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:703dc18959731edc9038fb0c6a1e8ae6d4464c0e6e394b54b1e961add3a73680
+size 20747667

checkpoint_p0/milestones/checkpoint_000160576_41107456.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4718b0f7808c6c85ab80c9ef755c0bbbb03acc70657837840b294e5a4f431523
+size 20747667

checkpoint_p0/milestones/checkpoint_000174048_44556288.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:665abe9fc9f8b4a65c9be00895f75864df010183ee1856117d5e11da988d4428
+size 20747667

checkpoint_p0/milestones/checkpoint_000187488_47996928.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d80d656ed00b97b436d8fa3c9c4dc3394a15a09a23b8e0fe5bcf3747a78d3d70
+size 20747667

checkpoint_p0/milestones/checkpoint_000200896_51429376.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38067b0afd82828638b3646af7add02a69f0afb3d0ef5ea05c96ca014c4695f7
+size 20747667

checkpoint_p0/milestones/checkpoint_000214368_54878208.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2b2af0edad99deed7996a5c327744709f8dbdfbc21e631b3a9ad99eb69c68c3
+size 20747667

checkpoint_p0/milestones/checkpoint_000227872_58335232.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:662e873252108e0092faf2b16ec2c56473a076a9cee64dbbc430e166d601d109
+size 20747667

checkpoint_p0/milestones/checkpoint_000241120_61726720.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e23360b791bdc5b74453faabeccc9c7e1be34f8f54a64d33f55447b2acb17071
+size 20747667

checkpoint_p0/milestones/checkpoint_000254592_65175552.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30c91b3669817b78477509ce344dd69dfedb430f29ed6773d135b21f86fa4096
+size 20747667

checkpoint_p0/milestones/checkpoint_000268032_68616192.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e7430dffcaa4f0e79e123ca43f200949ac6809e182649167397d478f424c0a6
+size 20747667

checkpoint_p0/milestones/checkpoint_000281504_72065024.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3773b6169259501cf34cc57ed5ea2058de1d93282f4142532aecb23d0ca69647
+size 20747667

checkpoint_p0/milestones/checkpoint_000294880_75489280.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68838fb23d5f6c1c238939fc9be936a6c06d31d9e5c88ebe479419fe6fd4c2d6
+size 20747667

checkpoint_p0/milestones/checkpoint_000308320_78929920.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f17c766da2d3a2fd83792c473776a292ccd7d1eee8b7d1d775694aa45e6a4ece
+size 20747667

checkpoint_p0/milestones/checkpoint_000321760_82370560.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca35e64ba6a7c804baf135530b47239250113a393cfebd8d80476cbbfadbb8ba
+size 20747667

checkpoint_p0/milestones/checkpoint_000335200_85811200.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db14f0af8644a709f29d9a9a934e2238e0385f8514701a46e2f758fda6786c92
+size 20747667

checkpoint_p0/milestones/checkpoint_000348608_89243648.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a25ad8aa46aeaceb498b2df4de5b3289ae533bed242226bce8ac8b6cede1ffd5
+size 20747667

checkpoint_p0/milestones/checkpoint_000362016_92676096.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c71bf3489264ab439681d603a8feae7f9aa2da5cb2aa62530f6eb564522d7e7
+size 20747667

checkpoint_p0/milestones/checkpoint_000375424_96108544.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e747792dfe8b1dab18b84015ccc06a04ac62bcb1b6d70fd315c20787933aa05
+size 20747667

checkpoint_p0/milestones/checkpoint_000388736_99516416.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:314c2a84b369f7c5cfcc76c5439da2b8a163aa2ea95d9770f662d184311429ba
+size 20747667

checkpoint_p0/milestones/checkpoint_000401984_102907904.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c11fa4eb41b051f56126421327375d4e7a5330a0c3fef679cc64b02278689c4
+size 20747723

checkpoint_p0/milestones/checkpoint_000415328_106323968.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d749d442da69d454d6bce4459e716bc8c7ff5d7fd10ec4a3711af698c96ee5a
+size 20747723

checkpoint_p0/milestones/checkpoint_000428608_109723648.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2b2c3a1ca637a69b7d1637fc370c4701d10a296c7e69d5e864bb244d135a7e1
+size 20747723

checkpoint_p0/milestones/checkpoint_000441920_113131520.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c5e650db3ac3520f99e68619993c0a55203a64d77c9c7d14cc44d450dec6a6f
+size 20747723

checkpoint_p0/milestones/checkpoint_000455232_116539392.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b4ce30827ba88e3e85fb5ebe0f0ad169c009a317f60bcf85b3b3c4c69f4c46aa
+size 20747723

checkpoint_p0/milestones/checkpoint_000468512_119939072.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5be180855c31989a0de3b47650882c2700d9774bfc83180bbb4d8400eaf63815
+size 20747723

checkpoint_p0/milestones/checkpoint_000481792_123338752.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:667ef3dc27ae80f55b303a3e3f4a792827b6be7a2fef8d26d6611c4151738187
+size 20747723

checkpoint_p0/milestones/checkpoint_000495040_126730240.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67f483dd413e976dde015f0aa3de0641051ffb0519b7f2a72f9eb6c51571fe08
+size 20747723

checkpoint_p0/milestones/checkpoint_000508192_130097152.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c61cbc1373888a875f4d6b5139e0044513f4d0e1ad8274df5ab43bf6f7d74bfb
+size 20747723

checkpoint_p0/milestones/checkpoint_000521440_133488640.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02041a562535b52a7c6bc732d7ed87873cff145b6fafd415e069647ce2c9fecb
+size 20747723

checkpoint_p0/milestones/checkpoint_000534688_136880128.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2d74c5f9cb455fe6c6f034fba21383bc5ab910079688ea33df28a6d0f9dacde
+size 20747723

checkpoint_p0/milestones/checkpoint_000547968_140279808.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a52891dd3911becc872586d7db32651049d2412a05ba790dd2cb27c0ab05f4fc
+size 20747723

checkpoint_p0/milestones/checkpoint_000561248_143679488.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36ae46b83b4a9fa0c4c6aabe30d087ea063804636c9777ded31f8bfc2c872e90
+size 20747723

checkpoint_p0/milestones/checkpoint_000574560_147087360.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:602e61b63ea5351999248dc21dd83cac6d1baad746df293e3b266f663215550f
+size 20747723