Upload folder using huggingface_hub

.summary/0/events.out.tfevents.1697542852.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ee1d5e49d893ff354bef06bf11e0ca35dbc184f6461d7355b0064e5f31d3d79
+size 28212

.summary/0/events.out.tfevents.1697545572.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:293d37db085e936373971881782e5d5df4e7de9b15b5453e9cf2c0e990bde9a0
+size 209301

.summary/1/events.out.tfevents.1697542852.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f95ee9fddd1c6ad6c44eab8c668c6e06936d0cd1de17fc07a53cab4a724ddd3
+size 19665

.summary/1/events.out.tfevents.1697545572.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f9cacb5ded5dc10b3cd59ef32cfb55dbe73b644a3fd6ad171b3c71df1b153f1
+size 151382

README.md

@@ -15,35 +15,38 @@ model-index:
       type: atari_asteroid
     metrics:
     - type: mean_reward
-      value: 2228.00 +/- 433.72
+      value: 1218.00 +/- 462.14
       name: mean_reward
       verified: false
 ---
 
-A(n) **APPO** model trained on the **atari_asteroid** environment.
+## About the Project
 
-This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
-Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+This project is an attempt to maximise performance of high sample throughput APPO RL models in Atari environments in as carbon efficient a manner as possible using a single, not particularly high performance single machine. It is about demonstrating the generalisability of on-policy algorithms to create good performance quickly (by sacrificing sample efficiency) while also proving that this route to RL production is accessible to even hobbyists like me (I am a gastroenterologist not a computer scientist). 
 
 
-## Downloading the model
+## Project Aims
 
-After installing Sample-Factory, download the model with:
-```
-python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_asteroid
-```
+This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it anywhere near sota performance. 
 
-    
-## About the Model
+I then re-trained the models with 100 million timesteps- at this point 2 environments maxed out at sota performance (Pong and Freeway) with four approaching sota performance - (atlantis, boxing, tennis and fishingderby.) =6/57 near sota. 
+
+The aim now is to try and reach state-of-the-art (SOTA) performance on a further block of atari environments using up to 1 billion training timesteps initially with appo. I will flag the models with SOTA when they reach at or near these levels. 
 
-This model as with all the others in the benchmarks was trained initially asynchronously un-seeded to 10 million steps for the purposes of setting a sample factory async baseline for this model on this environment but only 3/57 made it. 
+After this I will switch on V-Trace to see if the Impala variations perform any better with the same seed (I have seeded '1234')
 
-The aim is to reach state-of-the-art (SOTA) performance on each atari environment. I will flag the models with SOTA when they reach at or near these levels. 
 
-The hyperparameters used in the model are the ones I have pushed to my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his.
-I saved time and energy by using many of his tuned hyperparameters to maximise performance. However, he used 2 billion training steps. I have started as explained above at 10 million then moved to 100m to see how performance goes:
+## About the Model
+
+The hyperparameters used in the model are described in my shell script on my fork of sample-factory: https://github.com/MattStammers/sample-factory. Given that https://huggingface.co/edbeeching has kindly shared his parameters, I saved time and energy by using many of his tuned hyperparameters to reduce carbon inefficiency:
 ```
 hyperparameters =  {
+  "help": false,
+  "algo": "APPO",
+  "env": "atari_asteroid",
+  "experiment": "atari_asteroid_APPO",
+  "train_dir": "./train_atari",
+  "restart_behavior": "restart",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -141,12 +144,28 @@ hyperparameters =  {
   "env_gpu_observations": true,
   "env_frameskip": 4,
   "env_framestack": 4,
-  }
+  "pixel_format": "CHW"
+}
 
   ```
 
 
     
+A(n) **APPO** model trained on the **atari_asteroid** environment.
+
+This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory. Sample factory is a 
+high throughput on-policy RL framework. I have been using 
+Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
+
+
+## Downloading the model
+
+After installing Sample-Factory, download the model with:
+```
+python -m sample_factory.huggingface.load_from_hub -r MattStammers/APPO-atari_asteroid
+```
+
+    
 ## Using the model
 
 To run the model after download, use the `enjoy` script corresponding to this environment:

checkpoint_p0/best_000003328_851968_reward_6.150.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3b1d34866f93ce669b44c875e32eb2a08db9599671de2381e8ecdf818ab4dc6
+size 20771187

checkpoint_p0/checkpoint_000002976_761856.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:00756fbccc5f403b76ea255d4e1354ed13a7b17ab5606488857562bfd6cb6139
+size 20771651

checkpoint_p0/checkpoint_000003584_917504.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:996fbf05ca461b48108ff8cc3fd98d9b424dc4f5c2801c572895b6b938e1bb74
+size 20771651

checkpoint_p1/best_000001696_434176_reward_6.290.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98eda7c41fba1c02fdc6c7384f58c0a892323a3bb6465d07e28b47e0501a3146
+size 20771187

checkpoint_p1/checkpoint_000003040_778240.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af91ccea4ee73e81ec76e860944e3d40d0510d97fdc8aeb6252861c149391ff9
+size 20771651

checkpoint_p1/checkpoint_000003648_933888.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fea6fd6f01e693c30be5fc076a9d7e46e2cb8b37cfbfce53ead3b2dd094e30f7
+size 20771651

config.json

@@ -4,7 +4,7 @@
   "env": "atari_asteroid",
   "experiment": "atari_asteroid_APPO",
   "train_dir": "./train_atari",
-  "restart_behavior": "restart",
+  "restart_behavior": "resume",
   "device": "gpu",
   "seed": 1234,
   "num_policies": 2,
@@ -12,11 +12,11 @@
   "serial_mode": false,
   "batched_sampling": true,
   "num_batches_to_accumulate": 2,
-  "worker_num_splits": 1,
+  "worker_num_splits": 2,
   "policy_workers_per_policy": 1,
   "max_policy_lag": 1000,
   "num_workers": 16,
-  "num_envs_per_worker": 2,
+  "num_envs_per_worker": 8,
   "batch_size": 1024,
   "num_batches_per_epoch": 8,
   "num_epochs": 4,
@@ -64,10 +64,10 @@
   "experiment_summaries_interval": 3,
   "flush_summaries_interval": 30,
   "stats_avg": 100,
-  "summaries_use_frameskip": true,
+  "summaries_use_frameskip": false,
   "heartbeat_interval": 10,
   "heartbeat_reporting_interval": 60,
-  "train_for_env_steps": 100000000,
+  "train_for_env_steps": 500000000,
   "train_for_seconds": 10000000000,
   "save_every_sec": 120,
   "keep_checkpoints": 2,
@@ -124,25 +124,27 @@
   "pbt_target_objective": "true_objective",
   "pbt_perturb_min": 1.1,
   "pbt_perturb_max": 1.5,
-  "command_line": "--algo=APPO --env=atari_asteroid --experiment=atari_asteroid_APPO --num_policies=2 --restart_behavior=restart --train_dir=./train_atari --train_for_env_steps=100000000 --seed=1234 --num_workers=16 --num_envs_per_worker=2 --num_batches_per_epoch=8 --async_rl=true --batched_sampling=true --batch_size=1024 --max_grad_norm=0 --learning_rate=0.0003033891184 --heartbeat_interval=10 --heartbeat_reporting_interval=60 --save_milestones_sec=1200 --num_epochs=4 --exploration_loss_coeff=0.0004677351413 --with_wandb=true --wandb_user=matt-stammers --wandb_project=atari_APPO --wandb_group=atari_asteroid --wandb_job_type=SF --wandb_tags=atari",
+  "command_line": "--algo=APPO --env=atari_asteroid --experiment=atari_asteroid_APPO --num_policies=2 --restart_behavior=resume --train_dir=./train_atari --train_for_env_steps=100000000 --seed=1234 --num_workers=16 --num_envs_per_worker=8 --num_batches_per_epoch=8 --worker_num_splits=2 --async_rl=true --batched_sampling=true --batch_size=1024 --max_grad_norm=0 --learning_rate=0.0003033891184 --heartbeat_interval=10 --heartbeat_reporting_interval=60 --save_milestones_sec=1200 --num_epochs=4 --exploration_loss_coeff=0.0004677351413 --summaries_use_frameskip=False --with_wandb=true --wandb_user=matt-stammers --wandb_project=atari_APPO --wandb_group=atari_asteroid --wandb_job_type=SF --wandb_tags=atari",
   "cli_args": {
     "algo": "APPO",
     "env": "atari_asteroid",
     "experiment": "atari_asteroid_APPO",
     "train_dir": "./train_atari",
-    "restart_behavior": "restart",
+    "restart_behavior": "resume",
     "seed": 1234,
     "num_policies": 2,
     "async_rl": true,
     "batched_sampling": true,
+    "worker_num_splits": 2,
     "num_workers": 16,
-    "num_envs_per_worker": 2,
+    "num_envs_per_worker": 8,
     "batch_size": 1024,
     "num_batches_per_epoch": 8,
     "num_epochs": 4,
     "exploration_loss_coeff": 0.0004677351413,
     "max_grad_norm": 0.0,
     "learning_rate": 0.0003033891184,
+    "summaries_use_frameskip": false,
     "heartbeat_interval": 10,
     "heartbeat_reporting_interval": 60,
     "train_for_env_steps": 100000000,
@@ -158,5 +160,5 @@
   },
   "git_hash": "5fff97c2f535da5987d358cdbe6927cccd43621e",
   "git_repo_name": "not a git repository",
-  "wandb_unique_id": "atari_asteroid_APPO_20231008_114614_125146"
+  "wandb_unique_id": "atari_asteroid_APPO_20231017_124050_252182"
 }

git.diff

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3357904f421d3f4924836316b1741bf64d5dd0e807d5e80ac07059b4c52a7008
-size 14426734
+oid sha256:2464da1601e095c629f46a5ef1ef7322a64234d560931c130d8a8e640a96e217
+size 14449550