PPO LunarLander-v2

This repository contains my Proximal Policy Optimization (PPO) agent trained using PyTorch and Gymnasium on the LunarLander-v2 environment.

Algorithm

The agent uses Proximal Policy Optimization (PPO).

The implementation includes:

• Actor-Critic architecture
• Generalized Advantage Estimation (GAE)
• PPO clipped policy objective
• Clipped value loss
• Advantage normalization
• Entropy regularization
• Gradient clipping
• Learning-rate annealing

Training Details

Parameter	Value
Environment	LunarLander-v2
Algorithm	PPO
Total timesteps	100000
Learning rate	0.00025
Number of environments	8
Steps per rollout	128
Gamma	0.99
GAE Lambda	0.95
Minibatches	4
Update epochs	4
PPO clip coefficient	0.2
Entropy coefficient	0.01
Value coefficient	0.5
Seed	1

Evaluation

The trained agent was evaluated for 10 episodes.

• Mean reward: -167.32
• Standard deviation: 88.93

Model Architecture

The policy uses an Actor-Critic architecture.

The actor predicts action logits for the discrete LunarLander action space.

The critic estimates the value of the current state.

Both networks contain two fully connected hidden layers with 64 units and Tanh activation functions.

Files

• model.pt — trained PyTorch model
• hyperparameters.txt — PPO hyperparameters
• evaluation.txt — evaluation results
• README.md — model card

Frameworks

• PyTorch
• Gymnasium
• NumPy