Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1698129108.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51c2957ebf722ad6311fa781c16b4d8f3c6500f7405434a3f70269b4acb28fd8
+size 87326717

.summary/1/events.out.tfevents.1698129108.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8597951e4d296a921251cb69251174a6a7837e00462ffb324b83b5ce0f1cdfc
+size 45889523

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_berzerk
     metrics:
     - type: mean_reward
-      value: 2047.00 +/- 696.72
+      value: 46256.00 +/- 17678.86
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_berzerk** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_berzerk
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_berzerk** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_berzerk
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001875680_480174080_reward_34.260.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e2d362a3a690f7fac67d079a964b931699ecbaa2e7d40d4492a13631db3238a
+size 20795763

checkpoint_p0/checkpoint_001953504_500203520.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3b0f7a27fced906be2b267a14c2389dc13721b20b97c5821b4f357da9fca67b
+size 20796099

checkpoint_p0/checkpoint_001953968_500441088.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d479b0c0ce66205ca397e081e4d03cb26c829d7f040a339805051d05cb1cd9b7
+size 20796099

checkpoint_p0/milestones/checkpoint_000013728_3514368.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a43d5c1519beefb839e7e6b993111e38096ed07b7c0707374b036880383fc746
+size 20796955

checkpoint_p0/milestones/checkpoint_000027712_7094272.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77c95cc3e6d92c683ddd3d083b03392fde907a31df1e75a31f13dea1788682f4
+size 20796955

checkpoint_p0/milestones/checkpoint_000041792_10698752.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:acb126c3b590a1542e468ea08dbfa1d3ee21c4a345c288d3cb74aa9603ae037a
+size 20797011

checkpoint_p0/milestones/checkpoint_000055968_14327808.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba903e3e849bc71a4c08f9baa19c95c03f89d1380d8e72acc04dd925f127c9c4
+size 20797011

checkpoint_p0/milestones/checkpoint_000070016_17924096.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:605af8d06ccd03b9d524aa2de2ed1fd0ee1c57cf66853ccb0ded2792845812a4
+size 20797011

checkpoint_p0/milestones/checkpoint_000084128_21536768.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:faed992ccc2747b5aeb8f904b38c0174633f75c743246bb501fed14f95ec7829
+size 20797011

checkpoint_p0/milestones/checkpoint_000098272_25157632.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdbe8f4c65533aa1ffa23a851a9a2651d265d750c7a145810e0702c860d945ba
+size 20797011

checkpoint_p0/milestones/checkpoint_000112448_28786688.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc59e190011eb9ab108d92a3ad54b086d8e37aa48a65a86628e43493deeb3231
+size 20797011

checkpoint_p0/milestones/checkpoint_000125408_32104448.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5408063c944bc1ad7e786c41d46366e67946917709beed0871be0f4eab2a5517
+size 20797011

checkpoint_p0/milestones/checkpoint_000139584_35733504.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a07dbc9dfb6e6d3df4a887316beda46998358c0ad31e8a6ba94b5f11ab91062
+size 20797011

checkpoint_p0/milestones/checkpoint_000153728_39354368.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8986503fb79039e812f896a7ca5fa79460282e2181da052683d471ecdb9774b4
+size 20797011

checkpoint_p0/milestones/checkpoint_000167776_42950656.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b8c12169e0dceb950e72ba9b0f6449fb0b8e92be7289b78404e56556fb639a5
+size 20797011

checkpoint_p0/milestones/checkpoint_000181888_46563328.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4743cc355386e77d96e61930516545fc1fd760e6081e1f8e86b5562dc9156817
+size 20797011

checkpoint_p0/milestones/checkpoint_000196032_50184192.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d70f547d23e097db69b7cb27843c69cd6f146359dd17598b2c3461452b9d970
+size 20797011

checkpoint_p0/milestones/checkpoint_000210176_53805056.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d6104bb2a35dece58c95335e0665d781c7052dd75aac1f4f7fbee00a1a3ed7b
+size 20797011

checkpoint_p0/milestones/checkpoint_000224352_57434112.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32dcb7469f6829de47258a477480471a2373fabb9a3d09fa5fb29846685f8b44
+size 20797011

checkpoint_p0/milestones/checkpoint_000238432_61038592.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:114139d9bdc412a7195666c7e6f5f5d4bae103708db4e355376d5664c7639f5c
+size 20797011

checkpoint_p0/milestones/checkpoint_000252544_64651264.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbe251508c8cff39ec51db58b4287d22386fbce83c8bb6a43f666b45a8f5abbc
+size 20797011

checkpoint_p0/milestones/checkpoint_000266624_68255744.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9ca0ec207c08567d7f9b6900e2ef50bea3dd1c8fefeb866c3987f91df1943a2
+size 20797011

checkpoint_p0/milestones/checkpoint_000280736_71868416.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2933d09a3f1ea241614d1b2948b194f1447c3ebde3475869d81e3d196afee6da
+size 20797011

checkpoint_p0/milestones/checkpoint_000294912_75497472.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b268886fc7be824a5c03cdd7f84c57b1f24e9a6535bd392ad9b0700239e39e36
+size 20797011

checkpoint_p0/milestones/checkpoint_000309056_79118336.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b41196570ff4dd679b861b7f40a00ba54d5a66dbaccf370098ead6e3f62d8dfc
+size 20797011

checkpoint_p0/milestones/checkpoint_000323392_82788352.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb6cad209d597d36622f8f580d724b7ace4f3211d63998e201e228dc5ee388b5
+size 20797011

checkpoint_p0/milestones/checkpoint_000337504_86401024.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51b148a24ead6664246bed9c7c81929a71e63c004b7f33f07db71862a1238728
+size 20797011

checkpoint_p0/milestones/checkpoint_000351520_89989120.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:402a726b0cd876de0380cf5440e7a40fff190d0e6f11084888635370d69e9c45
+size 20797011

checkpoint_p0/milestones/checkpoint_000364096_93208576.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e60f9c92e82588a132b20afb1744c200d81dc0ed1494fc5232d2b598b3cee64
+size 20797011

checkpoint_p0/milestones/checkpoint_000373248_95551488.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16519024f963440a02c6381cf791b36c7a600277256af0a30aa64abda7c58fd2
+size 20797011

checkpoint_p0/milestones/checkpoint_000382464_97910784.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:40b64510c678906d3182debfef6c18d1942a770efd4ab17c2da667d0e43486bb
+size 20797011

checkpoint_p0/milestones/checkpoint_000391680_100270080.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63a79ff047368bb253f544435d54a577a301594310171576fa28588d19d53e9f
+size 20797067

checkpoint_p0/milestones/checkpoint_000400928_102637568.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df6f3d6f739826898382c2af48dfebbe8940a1efbfa24e57dca8a05bfcb35f23
+size 20797067

checkpoint_p0/milestones/checkpoint_000410176_105005056.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19bf158eaf09ba8c6fd034c5664088c5597e8c2dcee0299a10f4788b4cace497
+size 20797067

checkpoint_p0/milestones/checkpoint_000419392_107364352.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27258069769dfd45a34429e15ab561b97af38d8bcced342052822770c169d7bc
+size 20797067

checkpoint_p0/milestones/checkpoint_000428608_109723648.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:698e87a95ad13132aa40b9e0625b86c097174a0414aa0068db622d4cc69f3949
+size 20797067

checkpoint_p0/milestones/checkpoint_000437824_112082944.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76489ebabea0c00919ee888fdd43279d57cfe4354a2ef323fd057bf133727c47
+size 20797067

checkpoint_p0/milestones/checkpoint_000447040_114442240.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13df87e8b6107d4bf20c8261fb1d706f06fd2d713246b25a7ed4878c4b5b3cfe
+size 20797067

checkpoint_p0/milestones/checkpoint_000457216_117047296.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cfe6c08dfffdd777c72d3c6b3634dea7937efd264781441a5d9d15f99fda45bf
+size 20797067

checkpoint_p0/milestones/checkpoint_000466560_119439360.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3db6a853d70ca154051032ac9c0b0542ee012cde6b9d22e8438aa0c12d251f5e
+size 20797067

checkpoint_p0/milestones/checkpoint_000475808_121806848.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e53c870fa62aa9ebcdbdb71bf63cacf02d4a842f2d54cdc38e1c249e43678ddc
+size 20797067

checkpoint_p0/milestones/checkpoint_000485056_124174336.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad82067259527cb47c1b6341e93940b1570c36380489d2ad071145dd45958ed5
+size 20797067

checkpoint_p0/milestones/checkpoint_000494272_126533632.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e57c7b2db296e2512e52d49c7fbd669ee0356146dfa092614731229a85173dc9
+size 20797067

checkpoint_p0/milestones/checkpoint_000503488_128892928.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:441ac4f2538bd860af372d213132ff30b2755bfadc79350e49148e36136b2a52
+size 20797067

checkpoint_p0/milestones/checkpoint_000512704_131252224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc120ad04c0b0396d208d32c6d63ec7a16d7bed43cae7d60ba8964c59f4e9b39
+size 20797067

checkpoint_p0/milestones/checkpoint_000521920_133611520.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29e0302f1c8864c2e772123fdc410c524987fedce0e4fa5393183c5d44c9ac08
+size 20797067