| --- |
| license: apache-2.0 |
| tags: |
| - robotics |
| - reinforcement-learning |
| - isaac-lab |
| - forge |
| - rl-games |
| - ppo |
| --- |
| |
| # Forge PegInsert expert (PPO) |
|
|
| PPO policy for `Isaac-Forge-PegInsert-Direct-v0` ([Isaac Lab](https://github.com/isaac-sim/IsaacLab) FORGE suite), |
| trained with `rl_games`. Final return **345.5**; measured success **~96% over ~1,010 collection episodes**. |
|
|
| ## ⚠️ This policy is for **stock** PegInsert only |
|
|
| It was trained on 2026-06-30, before the task config in our Isaac Lab checkout was modified for a |
| separate study (spawn raised to 8 cm with ±5 cm lateral noise, and a ±10° tilt of the peg *inside |
| the gripper*). On stock dynamics it succeeds ~96% of the time; on that modified variant it scores |
| **0/12**. A policy trained from scratch on the modified variant plateaus at a return of 72.7 against |
| this one's 345.5 — the geometry explains why: with a 7.986 mm peg in an 8.100 mm hole, a peg cocked |
| beyond ~9.6° cannot enter at all, and the tilt is not observable in the policy's inputs. |
|
|
| On an unmodified Isaac Lab, run it as-is. |
|
|
| ## What is in the file |
|
|
| `rl_games` checkpoints hold more than the policy: the actor-critic weights, the optimizer state, |
| and — importantly for inference — the running observation normalisation statistics. That is why a |
| small MLP policy takes 204 MB. Load it through the Isaac Lab player rather than `torch.load` alone, |
| so the normaliser is restored with the weights: |
|
|
| ```bash |
| ./isaaclab.sh -p scripts/reinforcement_learning/rl_games/play.py \ |
| --task Isaac-Forge-PegInsert-Direct-v0 --checkpoint /path/to/forge_peginsert_2026-06-30.pth --num_envs 1 --headless |
| ``` |
|
|
| ## Training |
|
|
| Stock Isaac Lab `rl_games` PPO, unchanged hyper-parameters except the epoch count: |
| 128 environments, seed 0, **300 epochs** (the shipped config stops at 200, which cuts the run while |
| the return is still climbing). Snapshots exist at epochs 100 and 200; this repo ships epoch 300. |
|
|
| ## Related |
|
|
| - Demonstrations produced with this policy: [`angledusgar/forge-v1`](https://huggingface.co/datasets/angledusgar/forge-v1) |
| - Its failures: [`angledusgar/forge-failure-v1`](https://huggingface.co/datasets/angledusgar/forge-failure-v1) |
|
|
|
|
