flydexo/ppo-from-scratch-LunarLander-v3

PPO Agent Playing LunarLander-v3

This is a trained model of a PPO agent playing LunarLander-v3, written from scratch in PyTorch (CleanRL-style implementation, separate actor and critic networks).

Evaluation

mean_reward = 217.95 +/- 51.27 over 10 episodes (greedy policy).

Files

• LunarLander-actor.pt — policy network state dict (8 -> 64 -> 64 -> 64 -> 4, Tanh)
• LunarLander-critic.pt — value network state dict (8 -> 64 -> 64 -> 64 -> 1, Tanh)
• replay.mp4 — replay video of the greedy agent
• results.json — evaluation results

Usage

policy = Policy(obs_dim=8, hidden_dim=64, act_dim=4, n_layers=2)
policy.load_state_dict(torch.load("LunarLander-actor.pt"))
action, _, _ = policy.act(torch.tensor(obs), evaluation=True)

Hyperparameters

env_id = LunarLander-v3
p_hidden_dim = 64
p_n_layers = 2
value_hidden_dim = 64
value_n_layers = 2
gamma = 0.999
total_timesteps = 2000000
beta = 0.01
num_envs = 16
num_steps = 1024
num_minibatch = 32
update_epochs = 4
learning_rate = 0.00025
gae_lambda = 0.98
eps = 0.2
v_coeff = 0.5
clip_value_loss = False