jos0054/ppo-LunarLander-v2

PPO agent for LunarLander-v2

Trained from scratch for 1,015,808 environment steps across 16 environments as the Hugging Face Deep RL Course Unit 1 assignment.

Evaluation

The Hub evaluation is recorded in the model metadata and results.json. A separate deterministic evaluation over 10 episodes (seed 123) achieved 256.65 +/- 24.56, with mean minus standard deviation 232.09. The course requires at least 200.

Load the agent

pip install "gymnasium[box2d]==0.29.1" "stable-baselines3==2.7.0" huggingface-hub

import gymnasium as gym
from huggingface_hub import hf_hub_download
from stable_baselines3 import PPO

checkpoint = hf_hub_download("jos0054/ppo-LunarLander-v2", "ppo-LunarLander-v2.zip")
model = PPO.load(checkpoint, device="cpu")
env = gym.make("LunarLander-v2")
obs, info = env.reset(seed=321)
for _ in range(1000):
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        break
env.close()

Training notebook

Open the completed Colab notebook (Google Drive access permissions apply).

PPO uses an MLP policy, n_steps=1024, batch_size=64, n_epochs=4, gamma=0.999, gae_lambda=0.98, ent_coef=0.01, and seed=42. Training is stochastic and results can vary.