Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1702334350.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2761d4eb5b02fd434f58a03f7738f44389b45c50ac76dca77f7e1c90d2b18348
+size 87624512

.summary/1/events.out.tfevents.1702334350.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:437def2d41d0d135000a97a82077c3c27a3cbd67b55500f05cc67908afbae75d
+size 46031050

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_venture
     metrics:
     - type: mean_reward
-      value: 130.00 +/- 190.00
+      value: 1840.00 +/- 120.00
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_venture** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_venture
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_venture** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_venture
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001897344_485720064_reward_3.840.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:acfdecb68b2070747950c88a53d35ddb8e542be5881ae2dbbcf786917ebf36d3
+size 20795763

checkpoint_p0/checkpoint_001953696_500301824.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d1ed1b704e8fbf23207fe3b6d666f3f30c1f08278d51a68a30d4f9b5e4fbc855
+size 20796099

checkpoint_p0/checkpoint_001953968_500441088.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd02e38db403d983f93f964e4386968671b1dca0ecae260751a5fe6f27f0ae28
+size 20796099

checkpoint_p0/milestones/checkpoint_000012800_3276800.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99214566f89d734f7d1caaa7a533f32dbebde8d01b7aa37e92a3bb77a20fdbba
+size 20796955

checkpoint_p0/milestones/checkpoint_000025856_6619136.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e3065f7d3f1c16e72559b5ae3d8fd83d062633c321bd758da4bffd15b85bf23
+size 20796955

checkpoint_p0/milestones/checkpoint_000038880_9953280.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64955560d594fce6010fe5b2805d4952f5a75f712c21838765d2228e5eed5f5d
+size 20796955

checkpoint_p0/milestones/checkpoint_000051968_13303808.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b693c02b2740c5776fbb795fc448464a4f41341bbefca1223897073a73db50f
+size 20797011

checkpoint_p0/milestones/checkpoint_000064640_16547840.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5cb230191995719da404bdc4b42d4e5385f97e286d9ac251611c393e19a56d5
+size 20797011

checkpoint_p0/milestones/checkpoint_000077568_19857408.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18cf453d98c6935ca813b752543dc833f35bf215684125d8eda9be471feb5bc5
+size 20797011

checkpoint_p0/milestones/checkpoint_000090624_23199744.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:673f1d5407e6b5b67d86441138f20b0b489a7a0a98c2a3f32dea459c90235631
+size 20797011

checkpoint_p0/milestones/checkpoint_000103776_26566656.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ccf74e251d4fa77f0b08ac5677c913e76fb260c877b65a270ccffe7672e1c76e
+size 20797011

checkpoint_p0/milestones/checkpoint_000116864_29917184.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84cb0237ae51c7bc169a492f37540c8bb845c1426210d44793b46f6d46b09531
+size 20797011

checkpoint_p0/milestones/checkpoint_000129984_33275904.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2758ab463d83797d717e4684b18a855062998d93f8a684045419369d947dd26d
+size 20797011

checkpoint_p0/milestones/checkpoint_000143072_36626432.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35d9e1bdd7e6a177c7267d15a0c0164e826e38901541c38b2abd4a52670075f7
+size 20797011

checkpoint_p0/milestones/checkpoint_000156160_39976960.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:326fd19f22807e4be145caa444f2b1084cb628e86e39e9702813ea281072dfb5
+size 20797011

checkpoint_p0/milestones/checkpoint_000169248_43327488.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:806e176f6135660862fecb39e0dd704263d2bcdf2bbcc72c6d94f06bc6786169
+size 20797011

checkpoint_p0/milestones/checkpoint_000182368_46686208.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5844f0a74f35e84676fde539686c7143bfe2139d33c21a3c1cf1773e4d411bab
+size 20797011

checkpoint_p0/milestones/checkpoint_000195424_50028544.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19a70d6824eea030d18e9cc374bddc899c6ce476c9aeb52873d4370045149985
+size 20797011

checkpoint_p0/milestones/checkpoint_000208576_53395456.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb3be314cb5a7af5b4a4ff739dc8962f801ccae454be5353843781791027579e
+size 20797011

checkpoint_p0/milestones/checkpoint_000221696_56754176.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64e395f6f90f660725db6815432fbb68abac570b082a03a6bcd080b0b55bd3a0
+size 20797011

checkpoint_p0/milestones/checkpoint_000234848_60121088.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee2d2265683efba15537a084cfb8c76b050b21fa8de5af0d2203016c98cef7ca
+size 20797011

checkpoint_p0/milestones/checkpoint_000248000_63488000.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cf4bc86cebda2220ab2825a94f2e6a31d8ced0625f6cafe591e027cbb73cbff
+size 20797011

checkpoint_p0/milestones/checkpoint_000261184_66863104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d609ad76f74d67d4dd8910962377286032e0b9e857dd2a5e92c89a811ff06397
+size 20797011

checkpoint_p0/milestones/checkpoint_000274336_70230016.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6c720e45a834bb891eb4ddc1b35c718f21f996687fd8abdf15aadd4b38ebfab
+size 20797011

checkpoint_p0/milestones/checkpoint_000287456_73588736.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:572a52321eb79fb3e065d15397d5975f02c96e10e8533c3b2dc2e9507d8fb771
+size 20797011

checkpoint_p0/milestones/checkpoint_000300640_76963840.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:011d8fe206d7f7fc92307143693dd21b82ff824a425912a7a9da5b28778dbb7c
+size 20797011

checkpoint_p0/milestones/checkpoint_000313792_80330752.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5246ab1b2e92f0e3a099365a53d8edadb6dfd1b6f4913fbbb7bde0165e2211cd
+size 20797011

checkpoint_p0/milestones/checkpoint_000326944_83697664.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:23de32c8221c63e43ae68b146fd1a647550d6f23dd6ddb7cac73ca23245ca7a5
+size 20797011

checkpoint_p0/milestones/checkpoint_000340064_87056384.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77b462e1d4128c9165109210311696aa75a09c3efa71b2052c0510cd45bfdfb2
+size 20797011

checkpoint_p0/milestones/checkpoint_000353216_90423296.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94a5fd963408e062f5ccc0494483e16a945d86f41d366b15e92fe5aff3556c99
+size 20797011

checkpoint_p0/milestones/checkpoint_000366368_93790208.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5030343bc1a4f068f14da146e91d86bba581559da0311c8bb66434a73e0af392
+size 20797011

checkpoint_p0/milestones/checkpoint_000379488_97148928.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcb8aa84cddc4c71d55d8b3ed4eeb78c3d1da71fba5d8c98544c9178575ad33b
+size 20797011

checkpoint_p0/milestones/checkpoint_000392640_100515840.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdb9aa284b276bedd3e13da1d2bdf3f629e4e47c14faaae6d72ae597eac767c0
+size 20797067

checkpoint_p0/milestones/checkpoint_000405600_103833600.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3525a30713f1e45f4bc39fd7d036b0466f5717d5390f264b8db335b13fccb7d8
+size 20797067

checkpoint_p0/milestones/checkpoint_000418720_107192320.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:661c1eb00d0705f4b607d86bce8c8b876195ceb1424eb7cb4fe33537e4d195cf
+size 20797067

checkpoint_p0/milestones/checkpoint_000431808_110542848.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff098e42746023a462b2a507e21150c8cea9edb07f38666f3c83c13826b67b19
+size 20797067

checkpoint_p0/milestones/checkpoint_000444992_113917952.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad3bc8413d5cc8d8236c9286d2ae3d2d5126bd7676e53ca451f4c2f57fa97d8c
+size 20797067

checkpoint_p0/milestones/checkpoint_000458112_117276672.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a59aad7d8ca64bfff20fb6e7380688cd8708518a8fb695f0ded8e44379f136e
+size 20797067

checkpoint_p0/milestones/checkpoint_000471232_120635392.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e173baaf7e2aa9e85816a23105c32dd4ec4aa60f73ae09ed14151b8678fa402
+size 20797067

checkpoint_p0/milestones/checkpoint_000484352_123994112.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45c27835a037f065ea1e3b9cca600c926f75bb6f31ad90c8190cbf997ca7ce36
+size 20797067

checkpoint_p0/milestones/checkpoint_000497472_127352832.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc83485f40bc8eb377c577752fd7f599b138e55e1478d8be614ad1a8b949a1d7
+size 20797067

checkpoint_p0/milestones/checkpoint_000510656_130727936.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc0a9ed865a405ce2976a63232c112de32c8d02fc783de2699002d6737946b7c
+size 20797067

checkpoint_p0/milestones/checkpoint_000523712_134070272.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73fa6a299d8830cb0cd5de5e11b9ec6af963c66063e516feee6fcd6a371f7eca
+size 20797067

checkpoint_p0/milestones/checkpoint_000536864_137437184.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c0b00508092ed21f272d0d12d2500002a1528435a8e249ac4266a4a7c5b2f16
+size 20797067

checkpoint_p0/milestones/checkpoint_000550016_140804096.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76f9ad9dc123db6765bb73236d8553b4dcbe67d23895f5da01cc075dbc40d28f
+size 20797067

checkpoint_p0/milestones/checkpoint_000563136_144162816.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd9be1966a21bcfbdf7c432e5964ffd06598f2b9f0fda661219501c66dc94c5f
+size 20797067