Unit 8 - PPO LunarLander

README.md

@@ -1,10 +1,10 @@
 ---
-library_name: stable-baselines3
 tags:
 - LunarLander-v2
-- deep-reinforcement-learning
+- ppo
 - reinforcement-learning
-- stable-baselines3
+- deep-rl-course
+- custom-implementation
 model-index:
 - name: PPO
   results:
@@ -16,22 +16,36 @@ model-index:
       type: LunarLander-v2
     metrics:
     - type: mean_reward
-      value: 267.49 +/- 17.43
+      value: 8.59 +/- 73.34
       name: mean_reward
       verified: false
 ---
+# PPO Agent Playing LunarLander-v2
 
-# **PPO** Agent playing **LunarLander-v2**
-This is a trained model of a **PPO** agent playing **LunarLander-v2**
-using the [stable-baselines3 library](https://github.com/DLR-RM/stable-baselines3).
+Trained with a PPO implementation from scratch (CleanRL-style) in PyTorch.
 
-## Usage (with Stable-baselines3)
-TODO: Add your code
+## Results
+- Mean reward: 8.59
+- Std reward: 73.34
 
-
-```python
-from stable_baselines3 import ...
-from huggingface_sb3 import load_from_hub
-
-...
-```
+## Hyperparameters
+- **exp_name**: ppo_from_scratch
+- **seed**: 1
+- **cuda**: 1
+- **env_id**: LunarLander-v2
+- **total_timesteps**: 400000
+- **learning_rate**: 0.00025
+- **num_envs**: 8
+- **num_steps**: 128
+- **anneal_lr**: 1
+- **gamma**: 0.99
+- **gae_lambda**: 0.95
+- **num_minibatches**: 4
+- **update_epochs**: 4
+- **clip_coef**: 0.2
+- **ent_coef**: 0.01
+- **vf_coef**: 0.5
+- **max_grad_norm**: 0.5
+- **repo_id**: CharithAnupama/ppo-LunarLander-v2
+- **batch_size**: 1024
+- **minibatch_size**: 256

logs/events.out.tfevents.1766052476.1d358b537c13.11731.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3a7a5cc171622cd93ebd16816656d99bc7b36863888d172b60290e6dfd006ec
+size 88

model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c5d98b6437421e6ef0e36154eb388164095d74cd2df9055ba48f366f02ae2b7
+size 43419

replay.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:133bde4452d3d5503c0195cd6ecf54235fdf271816c2e98fe11d077d6486f3ff
-size 145104
+oid sha256:288fca0eabde9af1426338462b92b1d7a86209b89f077d65e3c1d9916ca991bf
+size 217586

results.json

@@ -1 +1 @@
-{"mean_reward": 267.4945156, "std_reward": 17.43157326554414, "is_deterministic": true, "n_eval_episodes": 10, "eval_datetime": "2025-11-13T05:49:21.774743"}
+{"env_id": "LunarLander-v2", "mean_reward": 8.593038541298025, "std_reward": 73.3423435822931, "n_evaluation_episodes": 10, "eval_datetime": "2025-12-18T10:10:52.975563"}