Vladprime/ppo-custom-LunarLander-v2

Custom PyTorch PPO — LunarLander-v2

From-scratch implementation for Vlad Kolomiiets (Vladprime) of clipped PPO, generalized advantage estimation, clipped value loss, entropy regularization and gradient clipping from HF Deep RL Unit 8 Part I. No pretrained model was used. The trainer is in train.py; it does not use Stable Baselines as its PPO implementation.

Training seed 20261101; 149504 environment steps. Independent evaluation: 200 fixed seeds 1000000–1000199, stochastic categorical policy (sampling seeded separately for each evaluation episode), unmodified environment rewards. Mean reward -135.2229, standard deviation 83.1176, mean minus standard deviation -218.3405. This is an initial trained policy, with remaining room to improve. Reloading reproduced the first five evaluation episode returns.

Reproduce

pip install -r requirements.txt
python train.py