Upload folder using huggingface_hub

.summary/0/events.out.tfevents.1701130338.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f795503ea46201827b18427a68691a7fd304be6fafb3adef5e329e45e581a754
+size 19422588

.summary/1/events.out.tfevents.1701130338.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7996f47cfc312feaf27259f535adc0d00cf242e81f49a9b42d907bb379f7622
+size 10545123

README.md

@@ -15,35 +15,39 @@ model-index:
       type: atari_skiing
     metrics:
     - type: mean_reward
-      value: -27467.60 +/- 3896.12
+      value: -8894.80 +/- 1235.78
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_skiing** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
+In terms of throughput I am managing to reach throughputs of 2,500 - 3,000 across both policies using sample factory using two Quadro P2200's (not particularly powerful GPUs) each loaded up about 60% (3GB). Previously using the stable baselines 3 (sb3) implementation of PPO it would take about a week to train an atari agent to 100 million timesteps synchronously. By comparison the sample factory async implementation takes only just over 2 hours to achieve the same result. That is about 84 times faster with only typically a 21 watt burn per GPU. I am thus very grateful to Alex Petrenko and all the sample factory team for their work on this.
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_skiing
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +145,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_skiing** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_skiing
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_000238688_61104128_reward_-487.520.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5db938ec789003d82fb83a5f79debe638758115d9a733cbf4c276c2fc33e4a4
+size 20703411

checkpoint_p0/checkpoint_000406208_103989248.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d3174f8056272efe8be3a0db47630b9c78f21b9a8c35426f7c219e71835a95d
+size 20703747

checkpoint_p0/checkpoint_000406944_104177664.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77537955f1d5de6be0269d486de68cc9326b15a9eea0792e6a8cdabbf3103aba
+size 20703747

checkpoint_p0/milestones/checkpoint_000012544_3211264.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:364b7cd78d440d90deacf9c7eab0de4e9d24ccc94acc69f7a19352a450a82398
+size 20704603

checkpoint_p0/milestones/checkpoint_000025280_6471680.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5269cbda536e8eb7a9390429ac5f742fd8089eddb26cecc4eb400b3cf1af4b60
+size 20704603

checkpoint_p0/milestones/checkpoint_000037984_9723904.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6ffb0930cf7ec9a5a0ce87482a9e8628a2904eb4a033451e9b5039dfffd5b7e
+size 20704603

checkpoint_p0/milestones/checkpoint_000050656_12967936.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c83055f0218852f35be03a51c58111918bcae95ffa79ba626ed5838265fe666d
+size 20704659

checkpoint_p0/milestones/checkpoint_000063360_16220160.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d73ce56810630647b27e1c24589745b6aa05fa6521a081c4e7f64c17233c84ad
+size 20704659

checkpoint_p0/milestones/checkpoint_000076096_19480576.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb3dd28ed60183aa3dcd13e71381e7a936f6bad68f9680c4c395eded34126cd5
+size 20704659

checkpoint_p0/milestones/checkpoint_000088832_22740992.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74e69d10b061b3e44b42eb6a89620ff54fb416de7c21a075d85eeaede39fc271
+size 20704659

checkpoint_p0/milestones/checkpoint_000101568_26001408.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19b7731feb884fc431a991f27e617a95a92cbcc6e1a7282a3918740b033b77f2
+size 20704659

checkpoint_p0/milestones/checkpoint_000114304_29261824.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fba01668a70d15cbce551031015d4fef5af03e777263b4138d049a5af4a5a53b
+size 20704659

checkpoint_p0/milestones/checkpoint_000127072_32530432.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05a879c908acbe987268f7b00614925385dfb9b40a5b1b0cd9a29efc99d8fabc
+size 20704659

checkpoint_p0/milestones/checkpoint_000139840_35799040.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ddc9b16e35b78766f11c49dc9bcc2f186511657ab89b6e741575a17fef6a38c
+size 20704659

checkpoint_p0/milestones/checkpoint_000152576_39059456.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72b145a27827479a9bd113021213cf55d3f45271745ada08cc5919e05f3e34d6
+size 20704659

checkpoint_p0/milestones/checkpoint_000165344_42328064.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:083b248a8a15389645fdef2993952a6595851ea093f8da71a079191351c98c2d
+size 20704659

checkpoint_p0/milestones/checkpoint_000178112_45596672.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7741227ea2955fe78b8260249bcf94addb6dbd0c89a4efdd649400845659e4a1
+size 20704659

checkpoint_p0/milestones/checkpoint_000190816_48848896.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b183985e7e45c36572d923b9d2c5b3ffcd15e63280685495f5ae19401d5f0c55
+size 20704659

checkpoint_p0/milestones/checkpoint_000203520_52101120.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6bb8d9bdd2297aa0e59459eab0dd45e90960cc32e57d1efc714780581652ef74
+size 20704659

checkpoint_p0/milestones/checkpoint_000216256_55361536.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a62c17e383377c0095f675e75b4380f6282c7d3178430e5376132b66316dadc0
+size 20704659

checkpoint_p0/milestones/checkpoint_000228960_58613760.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b517ea3fff1d0b2dd7c120c26fbdf3a731a9fa06513227cc031ff153c05d5ec
+size 20704659

checkpoint_p0/milestones/checkpoint_000241632_61857792.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa47c1b0de96b9ada994701654cc71eae16c07d2a99ca34405feeca8fa895f16
+size 20704659

checkpoint_p0/milestones/checkpoint_000254304_65101824.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76a858a84dcc10ca93e87ee8ebba7145f98904b73f8ceda62b6c260e29c1e798
+size 20704659

checkpoint_p0/milestones/checkpoint_000267040_68362240.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13e1684f34aceee0d44f4f485356ee32d128fb7fde8555b68fe92913eb23580a
+size 20704659

checkpoint_p0/milestones/checkpoint_000279776_71622656.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5942d99ed98e548c8c97b40b2b7306f2ebfa81bea26eec04d667284b4cbbec63
+size 20704659

checkpoint_p0/milestones/checkpoint_000292512_74883072.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f7bf26033f74058b9c065827f126e1149e834a24098dbea92639e7abe06e436
+size 20704659

checkpoint_p0/milestones/checkpoint_000305248_78143488.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a139a15221b19990a3fc66226e737b205f8e20a4ab0b588a65d15501a47e6a17
+size 20704659

checkpoint_p0/milestones/checkpoint_000317824_81362944.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a552cfac0cf7a02d419dc824808131f543511ff81b44e0ea9333a64589d16c1d
+size 20704659

checkpoint_p0/milestones/checkpoint_000330592_84631552.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:714c74131f9befa4b887d5bbb5448ccc8319d350fc2b8554dd6db335a6ba9f5d
+size 20704659

checkpoint_p0/milestones/checkpoint_000343232_87867392.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4bc4634227db46c75d87defb16fc59e1d3b25a3782b03e267134ac2fbf4921e
+size 20704659

checkpoint_p0/milestones/checkpoint_000355840_91095040.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7885960e76275ef20e7a7c6afa42498a6a1c548dba3a235294984b507144092
+size 20704659

checkpoint_p0/milestones/checkpoint_000368416_94314496.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:543cc827324cada8a052ab7a7b1fc4aa17ae6653c46687125b5fd88c5a90d8a6
+size 20704659

checkpoint_p0/milestones/checkpoint_000381024_97542144.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:681eaad61239c8939585d83a9356db2fd2296329155a41c9e724a1f4530b2bcd
+size 20704659

checkpoint_p0/milestones/checkpoint_000393632_100769792.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d120dd580a65525466ced454b949e2ec52b0d08a12f931917ac7d8a2358831d6
+size 20704715

checkpoint_p0/milestones/checkpoint_000406208_103989248.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:240957e27387716b2939d14a570f9097651923835df88dfec760b5d5c1ec8b6f
+size 20704715

checkpoint_p1/best_000128352_32858112_reward_-489.330.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1b9f59f9e1b6aa28a6bf1ce202385d41d9cac377292d4a6471892314461cc13
+size 20703411

checkpoint_p1/checkpoint_000405984_103931904.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43665ea1b87569ef5d8b62a254f10b783c84fd605acd01cdf9a9b348baa5f44a
+size 20703747

checkpoint_p1/checkpoint_000406688_104112128.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ff9d06ba02bf25243a8c64a92b0eb6ed2e7e9a0d0f83eed5068c1cdb948361b
+size 20703747

checkpoint_p1/milestones/checkpoint_000012480_3194880.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:112e3dd1224eaff9693f6acc227351accf7a24d0596f353f5a21ef175789a007
+size 20704603

checkpoint_p1/milestones/checkpoint_000025152_6438912.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8dd2865b382876a5cfc7a6b5222d26ade089826403f99c03d9dff9a19085a50c
+size 20704603

checkpoint_p1/milestones/checkpoint_000037824_9682944.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29076683bb3d7e03443a8a4a25654c126a071f5a65267ee2aadd8d8fa6fc7eb5
+size 20704603

checkpoint_p1/milestones/checkpoint_000050560_12943360.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb754f1f3989b9dfe0d9452b2112c00400bf849df0c5a64e1ab27d43be857a78
+size 20704659

checkpoint_p1/milestones/checkpoint_000063296_16203776.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b78d7d3b5031f1dc43a93d8c3281306e3bc1387972df48a77fc9bcbbfb9eaf59
+size 20704659

checkpoint_p1/milestones/checkpoint_000076000_19456000.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f67c777abaab87538784241e32e60104a23be5c994b4cf06ff01c2f0ff63d61
+size 20704659

checkpoint_p1/milestones/checkpoint_000088704_22708224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5661e84d5bbf3f310eb035042d29624995decf738c36a85d4bf158163f2aa251
+size 20704659

checkpoint_p1/milestones/checkpoint_000101344_25944064.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72ed6bb492fea63c43ce1a77ca4029fe3d31ac9b898ed675e72a0f630eafa27c
+size 20704659

checkpoint_p1/milestones/checkpoint_000114080_29204480.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:018d1d67ffb01c832d359f067c1e058a27893866b317977bd801435d5554f45c
+size 20704659