Upload folder using huggingface_hub

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 git.diff filter=lfs diff=lfs merge=lfs -text
 replay.mp4 filter=lfs diff=lfs merge=lfs -text
+sf_log.txt filter=lfs diff=lfs merge=lfs -text

.summary/0/events.out.tfevents.1701928702.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7088c596c3dc18b711e6cef7b429e1bbc990e624ad5a0814be4ebffb7589b1bf
+size 83625864

.summary/1/events.out.tfevents.1701928702.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59c18a79b5425fb0db0c08242bd53ad72b1da06d6daa3cabf9910766d97c7afc
+size 43926452

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_timepilot
     metrics:
     - type: mean_reward
-      value: 15050.00 +/- 1467.82
+      value: 148350.00 +/- 100627.70
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_timepilot** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_timepilot
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_timepilot** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_timepilot
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_001942784_497352704_reward_122.080.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecbacd52bcdfc27a7ffee884edc32efe3ee1715d4f02dd2503f9d5ed4443d709
+size 20746419

checkpoint_p0/checkpoint_001951968_499703808.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d079ed607345d689ed9ae63d66262c94667985db86b46ffee05b7822c06198b0
+size 20746755

checkpoint_p0/checkpoint_001953120_500006912.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:640abd465b472c8a5943c0b572a59c0034dcf10a44a3fc025b9f50a71379f4b7
+size 20746755

checkpoint_p0/milestones/checkpoint_000013248_3391488.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5d2eb666346be65fcc32f8e88d0afb412b1eb470823dcd6ad0c1585af6c2f60
+size 20747611

checkpoint_p0/milestones/checkpoint_000026816_6864896.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f461d7a1f414a8f1d31ca3930bbabc5bc2ccc722c303a7b0433eebd9162b55c0
+size 20747611

checkpoint_p0/milestones/checkpoint_000040448_10354688.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44cc7872aaa53c552acdacd946b96eab6038b32fcd20a11af3aaf363997feff8
+size 20747667

checkpoint_p0/milestones/checkpoint_000054080_13844480.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60fb3a50e473d8783a540478cc44b5132c7ab86cb243fc4e727f4fe65bfb9791
+size 20747667

checkpoint_p0/milestones/checkpoint_000067744_17342464.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9eb7f6daba2666160aa2f706b703de78e07d6faf3d5f1819b64057d358dc010a
+size 20747667

checkpoint_p0/milestones/checkpoint_000081376_20832256.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:782ce348f91d5a2734c540cdd36a1ea4fa5318c642edcc9a44a5369bb221a4ee
+size 20747667

checkpoint_p0/milestones/checkpoint_000094912_24297472.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a2c1b6852129956d1143503e2bea1fe1811bfd2d3c6607946457717ffbc4e02
+size 20747667

checkpoint_p0/milestones/checkpoint_000108544_27787264.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1f08e89da3126972bda7f1aab398271541f72e6c9f82da400e5fc76e0a80fe0
+size 20747667

checkpoint_p0/milestones/checkpoint_000122208_31285248.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:372f4ddda3eabe8469b90f9ac0ea6ede3f4a63723fdd9c42e2eba15035c03355
+size 20747667

checkpoint_p0/milestones/checkpoint_000135936_34799616.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f42cf6cc71e55696fe9d5a671f13ca2cf05f560acee2bfff4432bc938844974f
+size 20747667

checkpoint_p0/milestones/checkpoint_000149632_38305792.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e30f5316a6427180ac5a9e030a198e9aca966f09f0aa07b0de83e3e1c65385d1
+size 20747667

checkpoint_p0/milestones/checkpoint_000163360_41820160.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b96eb7f2be564a22c8b3af3f681b52ed3fe45883714ae9cc69d206602b556099
+size 20747667

checkpoint_p0/milestones/checkpoint_000177024_45318144.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ed5a62e862c382359b6073abda5e34d0f076ea669348fc43f3c94c13ac3b194
+size 20747667

checkpoint_p0/milestones/checkpoint_000190752_48832512.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8fc7ff5b1b75c27bb59a5288f47b9856ae2b1eca2cf93009e4e9f1378315086
+size 20747667

checkpoint_p0/milestones/checkpoint_000204480_52346880.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:705ec016adb3074313e589aa23f17620bb88b991c6c03d90aa637a8108d32503
+size 20747667

checkpoint_p0/milestones/checkpoint_000218144_55844864.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd1bced19d1ec301ba7c6452fe99d1bd6950c61fc29536be01638319fed88c8b
+size 20747667

checkpoint_p0/milestones/checkpoint_000231872_59359232.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9923c290f5b1e02a140b2a527a43ac54bbcda25c3712dc380704dd62d55ba9f9
+size 20747667

checkpoint_p0/milestones/checkpoint_000245568_62865408.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1911b8e86fd819a0a5fd509b310a46c664d061d6a686c044d213450ec7d5959f
+size 20747667

checkpoint_p0/milestones/checkpoint_000259232_66363392.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f108b922fcf8e4be7f2d21d321a6e82d0359a8f4ba47a589103bc64651e10f00
+size 20747667

checkpoint_p0/milestones/checkpoint_000272960_69877760.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:003f6a6aa637257a8492efa028227849ed513518ae87ddefc1caf1b3838cfaa0
+size 20747667

checkpoint_p0/milestones/checkpoint_000286624_73375744.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df960dfe7b48aa62b803e246265de55c7eadf84b56d3889af22328e1e57404b0
+size 20747667

checkpoint_p0/milestones/checkpoint_000300192_76849152.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ca0ed32727a11556454a50d1c63b0eb7528e90f227bf36e015ca5bc3bd89a44
+size 20747667

checkpoint_p0/milestones/checkpoint_000313792_80330752.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:678469d2842a548f9d0f8070fb8bd058c099e45cf5a3265f72e2ea3ecf293efa
+size 20747667

checkpoint_p0/milestones/checkpoint_000327424_83820544.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e4146b447bf6282db28f7987845c1b940dec9d2ff207d6ce246c18b3d0a320bb
+size 20747667

checkpoint_p0/milestones/checkpoint_000341024_87302144.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc59a59ee177370978618d7e7a7f947d98bbbe2c5d7a3ffe185850513dd7e26c
+size 20747667

checkpoint_p0/milestones/checkpoint_000354656_90791936.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb7fea5cbb1d4736d39f8ab5dd0ccbc306454303f9098f54fdbf2b2c2585b8a2
+size 20747667

checkpoint_p0/milestones/checkpoint_000368320_94289920.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f131a2f8d7b8ea3ef6504bf93d149f15bc2c17b57c870314a8fe0465b19d5ef
+size 20747667

checkpoint_p0/milestones/checkpoint_000381952_97779712.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:460979af9d31833728642cf62bdbb44f27052412dc8a22c6e79ac0ff7f26575b
+size 20747667

checkpoint_p0/milestones/checkpoint_000395584_101269504.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0eabbd34038c6bb17f586c8bba47f5b16b39437dea4610f8c3078f0921595565
+size 20747723

checkpoint_p0/milestones/checkpoint_000409184_104751104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5b57ccb611463682624a74b7bfd54d9fd5c47ac9b95293df8b5b1420c2cf7db
+size 20747723

checkpoint_p0/milestones/checkpoint_000422816_108240896.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99771f8679e6c114fe4b2390afd5f0c2603eaebf144c919569495bcd6ad384e9
+size 20747723

checkpoint_p0/milestones/checkpoint_000436384_111714304.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce009464070612d4f132989e66604e043d7afeb9a5b70e8641e3f774827303f3
+size 20747723

checkpoint_p0/milestones/checkpoint_000449984_115195904.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ada65927e4d67510e78145dcd7a63cea2002df297dd7d3dc05fc29a3105e771d
+size 20747723

checkpoint_p0/milestones/checkpoint_000463616_118685696.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9be42b7d4a3e9f034e549b939902effbd25c6dd89f507c70266b1125760188ae
+size 20747723

checkpoint_p0/milestones/checkpoint_000477184_122159104.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9a40bcf285756e1d0e1afa3a3742f8270fd3a1ff7d65261309e42f9ec6e152e
+size 20747723

checkpoint_p0/milestones/checkpoint_000490816_125648896.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:820d20bd73f975de3ab6e0942d81342b9ff3690fa8d4774c577614e0bdcd63b8
+size 20747723

checkpoint_p0/milestones/checkpoint_000504512_129155072.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9d4beca02110855a89fe5d1cc59f60ab5c2ef45eedbebb01b7ca0f204680694
+size 20747723

checkpoint_p0/milestones/checkpoint_000518144_132644864.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8660a9ca204638f21f81ed10b9c7ae3c23fe9a51cdd108a774133a23b69abf9e
+size 20747723

checkpoint_p0/milestones/checkpoint_000531776_136134656.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3716560536799fb54ecf6f9f7d38a88ed145720e52e637e18e3630dbd60d7f9b
+size 20747723

checkpoint_p0/milestones/checkpoint_000545504_139649024.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:11345418be405240f85d19bd9c32514b3841d5e1c25878354153aa346bec680f
+size 20747723

checkpoint_p0/milestones/checkpoint_000559296_143179776.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce15e90e4f44779652008b1d2f4772ab9bc53197b4b7158faf5e60ab55cc5e72
+size 20747723

checkpoint_p0/milestones/checkpoint_000572992_146685952.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f758e60574aa7afda3793480f52108c59d15fb2c3f34a790be67243d766efc0e
+size 20747723

checkpoint_p0/milestones/checkpoint_000586752_150208512.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:258cefb82d5eda2f43c34fdcccf6cd150205eaeac3007b158540fb9d94e92094
+size 20747723