from-scratch/README.md

From-scratch PPO Agent (Unit 8) - LunarLander-v2

This folder contains a from-scratch CleanRL-style PPO implementation (Unit 8 of the Hugging Face Deep RL Course), uploaded alongside the existing Unit 1 SB3 model in this repo.

• mean_reward: 175.30 +/- 60.72 (20 episodes, deterministic/argmax)
• Algorithm: PPO implemented from scratch in PyTorch (GAE, clipped surrogate, value loss, entropy bonus, LR annealing, grad clipping)
• Training: 1,000,000 timesteps, 16 parallel envs

Full implementation: ppo_scratch.py