Hugging Face's logo

tsilva
/
Level1-2_stable-baselines3-ppo_91df110f

Reinforcement Learning
stable-baselines3
ppo
supermariobrosnes-turbo
rlab
SuperMarioBros-Nes-v0
Model card Files
xet
 Community
tsilva commited on
Commit
a0606eb
·
verified ·
1 Parent(s): 5645b41

Release Level1-2 PPO checkpoint

Browse files
Files changed (6) hide show
  1. .gitattributes +1 -0
  2. README.md +82 -0
  3. model.zip +3 -0
  4. model_metadata.json +328 -0
  5. release_manifest.json +0 -0
  6. replay.mp4 +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ replay.mp4 filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: stable-baselines3
3
+ pipeline_tag: reinforcement-learning
4
+ tags:
5
+ - reinforcement-learning
6
+ - stable-baselines3
7
+ - ppo
8
+ - stable-retro
9
+ - rlab
10
+ - super-mario-bros
11
+ - nes
12
+ - SuperMarioBros-Nes-v0
13
+ metrics:
14
+ - completion-rate
15
+ ---
16
+
17
+ # SuperMarioBros-NES Level1-2 PPO
18
+
19
+ PPO policy checkpoint for completing `SuperMarioBros-Nes-v0` `Level1-2` with `supermariobrosnes-turbo`, trained with [`rlab`](https://github.com/tsilva/rlab).
20
+
21
+ ## Quick Start
22
+
23
+ Install `rlab` once, import the ROM, then play or evaluate this checkpoint directly from Hugging Face:
24
+
25
+ ```bash
26
+ uv tool install --from git+https://github.com/tsilva/rlab rlab
27
+ rlab import-roms ~/roms --game SuperMarioBros-Nes-v0
28
+ rlab play hf://tsilva/SuperMarioBros-Nes-v0_Level1-2
29
+ rlab eval hf://tsilva/SuperMarioBros-Nes-v0_Level1-2
30
+ ```
31
+
32
+ ## Evaluation Results
33
+
34
+ | `eval_profile` | `episodes` | `seed_start` | `completion_rate` | `max_x_max` | `reward_mean` | `checkpoint_step` |
35
+ |---|---:|---:|---:|---:|---:|---:|
36
+ | `local release eval`, deterministic policy | 100 | 10000 | `100/100` episodes | 3129 | 3148.550 | 7500000 |
37
+
38
+ This release uses the `step-7500000` checkpoint from the W&B run below. The training run reached the Level1-2 early-stop gate at `7,506,208` timesteps; this staged release eval was run locally because the run's post-training checkpoint eval crashed after its first checkpoint during W&B summary bookkeeping.
39
+
40
+ ## Environment Details
41
+
42
+ | Setting | Value |
43
+ |---|---|
44
+ | `env_provider` | `supermariobrosnes-turbo` |
45
+ | `env_id` | `supermariobrosnes-turbo:SuperMarioBros-Nes-v0` |
46
+ | `game` | `SuperMarioBros-Nes-v0` |
47
+ | `state` | `Level1-2` |
48
+ | `preprocessing` | crop top `32` px, grayscale, resize to `84 x 84` |
49
+ | `frame_stack` | `4` |
50
+ | `frame_skip` | `4` |
51
+ | `max_pool_frames` | disabled |
52
+ | `policy_observation_layout` | channel-first `(4, 84, 84)` |
53
+ | `action_set` | `simple` |
54
+ | `reward_mode` | `score` |
55
+ | `reward_shaping` | score-progress reward, `terminal_reward=50`, `death_penalty=25`, `reward_scale=10` |
56
+ | `max_episode_steps` | `4500` |
57
+ | `train_done_on_events` | life loss, level change |
58
+ | `eval_done_on_events` | level change |
59
+
60
+ ## Training Recipe
61
+
62
+ | Setting | Value |
63
+ |---|---:|
64
+ | `goal_id` | `Level1-2` |
65
+ | `recipe_id` | `base` |
66
+ | `checkpoint_step` | 7500000 |
67
+ | `frame_skip` | 4 |
68
+ | `max_episode_steps` | 4500 |
69
+ | `reward_mode` | `score` |
70
+ | `reward_shaping` | score-progress reward, `terminal_reward=50`, `death_penalty=25`, `reward_scale=10` |
71
+ | `done_on_events` | life loss, level change |
72
+
73
+ ## Provenance
74
+
75
+ | Item | Value |
76
+ |---|---|
77
+ | `source_project` | [`rlab`](https://github.com/tsilva/rlab) |
78
+ | `goal_id` | `Level1-2` |
79
+ | `goal_title` | `Sample-efficient SuperMarioBros-NES Level1-2 completion` |
80
+ | `wandb_run` | [`Level1-2_base_s1_20260704T072752Z`](https://wandb.ai/tsilva/SuperMarioBros-Nes-v0/runs/dtpvaqq2) |
81
+ | `wandb_artifact` | `tsilva/SuperMarioBros-Nes-v0/Level1-2_base_s1_20260704T072752Z-checkpoint:step-7500000` |
82
+ | `eval_source` | `local release eval` |
model.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:031cad88ae2a0a12064e1459601bfa51fbe5e7aa69e0c39aefbd1ba0f4cfd483
3
+ size 21054227
model_metadata.json ADDED
@@ -0,0 +1,328 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "artifact_storage_uri": "s3://wandb/rlab/SuperMarioBros-Nes-v0/Level1-2_base_s1_20260704T072752Z/checkpoint/ppo_supermariobros-nes-v0_7500000_steps.zip",
3
+ "checkpoint_step": 7500000,
4
+ "env_config": {
5
+ "action_set": "simple",
6
+ "clip_rewards": false,
7
+ "completion_reward": 0.0,
8
+ "death_penalty": 25.0,
9
+ "done_on_events": [
10
+ "life_loss",
11
+ "level_change"
12
+ ],
13
+ "env_provider": "supermariobrosnes-turbo",
14
+ "env_threads": 4,
15
+ "env_wrappers": [
16
+ {
17
+ "id": "SuperMarioBrosNesProgressInfoWrapper",
18
+ "kwargs": {}
19
+ },
20
+ {
21
+ "id": "SuperMarioBrosNesRewardEnvWrapper",
22
+ "kwargs": {
23
+ "completion_reward": 0.0,
24
+ "death_penalty": 25.0,
25
+ "progress_reward_scale": 1.0,
26
+ "reward_mode": "score",
27
+ "reward_scale": 10.0,
28
+ "score_progress_clipped": false,
29
+ "terminal_reward": 50.0
30
+ }
31
+ }
32
+ ],
33
+ "frame_skip": 4,
34
+ "game": "SuperMarioBros-Nes-v0",
35
+ "hud_crop_top": -1,
36
+ "info_events": {},
37
+ "max_episode_steps": 4500,
38
+ "max_pool_frames": false,
39
+ "no_progress_min_delta": 0,
40
+ "no_progress_timeout_steps": 0,
41
+ "obs_crop": [
42
+ 32,
43
+ 0,
44
+ 0,
45
+ 0
46
+ ],
47
+ "obs_resize_algorithm": "area",
48
+ "observation_size": 84,
49
+ "progress_reward_cap": 30.0,
50
+ "progress_reward_scale": 1.0,
51
+ "reward_mode": "score",
52
+ "reward_scale": 10.0,
53
+ "score_progress_clipped": false,
54
+ "state": "Level1-2",
55
+ "state_distribution": [],
56
+ "state_probs": [],
57
+ "state_sampling_mode": "single",
58
+ "states": [],
59
+ "sticky_action_prob": 0.0,
60
+ "task_conditioning": false,
61
+ "task_conditioning_info_values": [],
62
+ "task_conditioning_info_vars": [],
63
+ "terminal_reward": 50.0,
64
+ "time_penalty": 0.0,
65
+ "use_retro_reward": false
66
+ },
67
+ "environment": {
68
+ "action": {
69
+ "action_set": "simple"
70
+ },
71
+ "env_id": "supermariobrosnes-turbo:SuperMarioBros-Nes-v0",
72
+ "preprocessing": {
73
+ "frame_skip": 4,
74
+ "frame_stack": 4,
75
+ "max_pool_frames": false,
76
+ "maxpool_last_two": false,
77
+ "obs_copy": "safe_view",
78
+ "obs_crop": [
79
+ 32,
80
+ 0,
81
+ 0,
82
+ 0
83
+ ],
84
+ "obs_grayscale": true,
85
+ "obs_resize": [
86
+ 84,
87
+ 84
88
+ ],
89
+ "obs_resize_algorithm": "area",
90
+ "pipeline": "supermariobrosnes_turbo_native_vec_env",
91
+ "policy_observation_layout": "channel_first",
92
+ "sticky_action_prob": 0.0
93
+ },
94
+ "reward": {
95
+ "clip_rewards": false,
96
+ "completion_reward": 0.0,
97
+ "death_penalty": 25.0,
98
+ "env_wrappers": [
99
+ {
100
+ "id": "SuperMarioBrosNesProgressInfoWrapper",
101
+ "kwargs": {}
102
+ },
103
+ {
104
+ "id": "SuperMarioBrosNesRewardEnvWrapper",
105
+ "kwargs": {
106
+ "completion_reward": 0.0,
107
+ "death_penalty": 25.0,
108
+ "progress_reward_scale": 1.0,
109
+ "reward_mode": "score",
110
+ "reward_scale": 10.0,
111
+ "score_progress_clipped": false,
112
+ "terminal_reward": 50.0
113
+ }
114
+ }
115
+ ],
116
+ "progress_reward_cap": 30.0,
117
+ "progress_reward_scale": 1.0,
118
+ "reward_mode": "score",
119
+ "reward_scale": 10.0,
120
+ "score_progress_clipped": false,
121
+ "terminal_reward": 50.0,
122
+ "time_penalty": 0.0,
123
+ "use_retro_reward": false
124
+ },
125
+ "schema_version": 1,
126
+ "state": "Level1-2",
127
+ "state_probs": [],
128
+ "states": [],
129
+ "task_conditioning": {
130
+ "task_conditioning": false,
131
+ "task_conditioning_info_values": [],
132
+ "task_conditioning_info_vars": []
133
+ },
134
+ "termination": {
135
+ "done_on_events": [
136
+ "life_loss",
137
+ "level_change"
138
+ ],
139
+ "info_events": {},
140
+ "max_episode_steps": 4500,
141
+ "no_progress_min_delta": 0,
142
+ "no_progress_timeout_steps": 0
143
+ }
144
+ },
145
+ "environment_hash": "sha256:24189df154a9b4e09521a674eb4147577412ef49d6652ec4683b523ed33a93a2",
146
+ "filename": "model.zip",
147
+ "goal_slug": "Level1-2",
148
+ "kind": "checkpoint",
149
+ "metadata_version": 2,
150
+ "queue_train_job_id": 4,
151
+ "recipe_path": "experiments/goals/SuperMarioBros-Nes-v0/Level1-2/recipes/base.yaml",
152
+ "recipe_slug": "base",
153
+ "run_description": "Seed 1 runs the Level1-2 transfer of the successful Level1-1 B55 low-KL late-decay recipe, created to test whether that policy recipe transfers without changing PPO or reward hyperparameters while overriding only Level1-2 identity and reporting metadata.",
154
+ "run_name": "Level1-2_base_s1_20260704T072752Z",
155
+ "training_metadata": {
156
+ "env_config": {
157
+ "action_set": "simple",
158
+ "clip_rewards": false,
159
+ "completion_reward": 0.0,
160
+ "death_penalty": 25.0,
161
+ "done_on_events": [
162
+ "life_loss",
163
+ "level_change"
164
+ ],
165
+ "env_provider": "supermariobrosnes-turbo",
166
+ "env_threads": 4,
167
+ "env_wrappers": [
168
+ {
169
+ "id": "SuperMarioBrosNesProgressInfoWrapper",
170
+ "kwargs": {}
171
+ },
172
+ {
173
+ "id": "SuperMarioBrosNesRewardEnvWrapper",
174
+ "kwargs": {
175
+ "completion_reward": 0.0,
176
+ "death_penalty": 25.0,
177
+ "progress_reward_scale": 1.0,
178
+ "reward_mode": "score",
179
+ "reward_scale": 10.0,
180
+ "score_progress_clipped": false,
181
+ "terminal_reward": 50.0
182
+ }
183
+ }
184
+ ],
185
+ "frame_skip": 4,
186
+ "game": "SuperMarioBros-Nes-v0",
187
+ "hud_crop_top": -1,
188
+ "info_events": {},
189
+ "max_episode_steps": 4500,
190
+ "max_pool_frames": false,
191
+ "no_progress_min_delta": 0,
192
+ "no_progress_timeout_steps": 0,
193
+ "obs_crop": [
194
+ 32,
195
+ 0,
196
+ 0,
197
+ 0
198
+ ],
199
+ "obs_resize_algorithm": "area",
200
+ "observation_size": 84,
201
+ "progress_reward_cap": 30.0,
202
+ "progress_reward_scale": 1.0,
203
+ "reward_mode": "score",
204
+ "reward_scale": 10.0,
205
+ "score_progress_clipped": false,
206
+ "state": "Level1-2",
207
+ "state_distribution": [],
208
+ "state_probs": [],
209
+ "state_sampling_mode": "single",
210
+ "states": [],
211
+ "sticky_action_prob": 0.0,
212
+ "task_conditioning": false,
213
+ "task_conditioning_info_values": [],
214
+ "task_conditioning_info_vars": [],
215
+ "terminal_reward": 50.0,
216
+ "time_penalty": 0.0,
217
+ "use_retro_reward": false
218
+ },
219
+ "environment": {
220
+ "action": {
221
+ "action_set": "simple"
222
+ },
223
+ "env_id": "supermariobrosnes-turbo:SuperMarioBros-Nes-v0",
224
+ "preprocessing": {
225
+ "frame_skip": 4,
226
+ "frame_stack": 4,
227
+ "max_pool_frames": false,
228
+ "maxpool_last_two": false,
229
+ "obs_copy": "safe_view",
230
+ "obs_crop": [
231
+ 32,
232
+ 0,
233
+ 0,
234
+ 0
235
+ ],
236
+ "obs_grayscale": true,
237
+ "obs_resize": [
238
+ 84,
239
+ 84
240
+ ],
241
+ "obs_resize_algorithm": "area",
242
+ "pipeline": "supermariobrosnes_turbo_native_vec_env",
243
+ "policy_observation_layout": "channel_first",
244
+ "sticky_action_prob": 0.0
245
+ },
246
+ "reward": {
247
+ "clip_rewards": false,
248
+ "completion_reward": 0.0,
249
+ "death_penalty": 25.0,
250
+ "env_wrappers": [
251
+ {
252
+ "id": "SuperMarioBrosNesProgressInfoWrapper",
253
+ "kwargs": {}
254
+ },
255
+ {
256
+ "id": "SuperMarioBrosNesRewardEnvWrapper",
257
+ "kwargs": {
258
+ "completion_reward": 0.0,
259
+ "death_penalty": 25.0,
260
+ "progress_reward_scale": 1.0,
261
+ "reward_mode": "score",
262
+ "reward_scale": 10.0,
263
+ "score_progress_clipped": false,
264
+ "terminal_reward": 50.0
265
+ }
266
+ }
267
+ ],
268
+ "progress_reward_cap": 30.0,
269
+ "progress_reward_scale": 1.0,
270
+ "reward_mode": "score",
271
+ "reward_scale": 10.0,
272
+ "score_progress_clipped": false,
273
+ "terminal_reward": 50.0,
274
+ "time_penalty": 0.0,
275
+ "use_retro_reward": false
276
+ },
277
+ "schema_version": 1,
278
+ "state": "Level1-2",
279
+ "state_probs": [],
280
+ "states": [],
281
+ "task_conditioning": {
282
+ "task_conditioning": false,
283
+ "task_conditioning_info_values": [],
284
+ "task_conditioning_info_vars": []
285
+ },
286
+ "termination": {
287
+ "done_on_events": [
288
+ "life_loss",
289
+ "level_change"
290
+ ],
291
+ "info_events": {},
292
+ "max_episode_steps": 4500,
293
+ "no_progress_min_delta": 0,
294
+ "no_progress_timeout_steps": 0
295
+ }
296
+ },
297
+ "environment_hash": "sha256:24189df154a9b4e09521a674eb4147577412ef49d6652ec4683b523ed33a93a2",
298
+ "preprocessing": {
299
+ "frame_skip": 4,
300
+ "frame_stack": 4,
301
+ "maxpool_last_two": false,
302
+ "obs_copy": "safe_view",
303
+ "obs_crop": [
304
+ 32,
305
+ 0,
306
+ 0,
307
+ 0
308
+ ],
309
+ "obs_grayscale": true,
310
+ "obs_resize": [
311
+ 84,
312
+ 84
313
+ ],
314
+ "obs_resize_algorithm": "area",
315
+ "pipeline": "supermariobrosnes_turbo_native_vec_env",
316
+ "policy_observation_layout": "channel_first",
317
+ "sticky_action_prob": 0.0
318
+ },
319
+ "versions": {
320
+ "stable_baselines3": "2.8.0",
321
+ "stable_retro_turbo": "1.0.1.post4",
322
+ "supermariobrosnes_turbo": "0.2.4"
323
+ }
324
+ },
325
+ "training_metadata_hash": "5f907ca4343b943bca056c4cce6603023449a5b2dec1dbf4683fcd1591bdf526",
326
+ "wandb_run_id": "dtpvaqq2",
327
+ "wandb_run_path": "tsilva/SuperMarioBros-Nes-v0/dtpvaqq2"
328
+ }
release_manifest.json ADDED
The diff for this file is too large to render. See raw diff
 
replay.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:94b3184669e8ef9297ec298c5136d3340f6fda762664cc4cd1dcf8e7fba42d0b
3
+ size 628219