Antonio Serrano Muñoz
commited on
Commit
·
59c7572
1
Parent(s):
c95d51d
Add files
Browse files- README.md +84 -1
- agent.pickle +3 -0
- agent.pt +3 -0
README.md
CHANGED
|
@@ -1,3 +1,86 @@
|
|
| 1 |
---
|
| 2 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
library_name: skrl
|
| 3 |
+
tags:
|
| 4 |
+
- deep-reinforcement-learning
|
| 5 |
+
- reinforcement-learning
|
| 6 |
+
- skrl
|
| 7 |
+
model-index:
|
| 8 |
+
- name: PPO
|
| 9 |
+
results:
|
| 10 |
+
- metrics:
|
| 11 |
+
- type: mean_reward
|
| 12 |
+
value: -8.89 +/- 10.3
|
| 13 |
+
name: Total reward (mean)
|
| 14 |
+
task:
|
| 15 |
+
type: reinforcement-learning
|
| 16 |
+
name: reinforcement-learning
|
| 17 |
+
dataset:
|
| 18 |
+
name: IsaacGymEnvs-FactoryTaskNutBoltScrew
|
| 19 |
+
type: IsaacGymEnvs-FactoryTaskNutBoltScrew
|
| 20 |
---
|
| 21 |
+
|
| 22 |
+
<!-- ---
|
| 23 |
+
torch: -21.51 +/- 14.99
|
| 24 |
+
jax: -35.77 +/- 0.39
|
| 25 |
+
numpy: -8.89 +/- 10.3
|
| 26 |
+
--- -->
|
| 27 |
+
|
| 28 |
+
# IsaacGymEnvs-FactoryTaskNutBoltScrew-PPO
|
| 29 |
+
|
| 30 |
+
Trained agent for [NVIDIA Isaac Gym Preview](https://github.com/NVIDIA-Omniverse/IsaacGymEnvs) environments.
|
| 31 |
+
|
| 32 |
+
- **Task:** FactoryTaskNutBoltScrew
|
| 33 |
+
- **Agent:** [PPO](https://skrl.readthedocs.io/en/latest/api/agents/ppo.html)
|
| 34 |
+
|
| 35 |
+
# Usage (with skrl)
|
| 36 |
+
|
| 37 |
+
Note: Visit the skrl [Examples](https://skrl.readthedocs.io/en/latest/intro/examples.html) section to access the scripts.
|
| 38 |
+
|
| 39 |
+
* PyTorch
|
| 40 |
+
|
| 41 |
+
```python
|
| 42 |
+
from skrl.utils.huggingface import download_model_from_huggingface
|
| 43 |
+
|
| 44 |
+
# assuming that there is an agent named `agent`
|
| 45 |
+
path = download_model_from_huggingface("skrl/IsaacGymEnvs-FactoryTaskNutBoltScrew-PPO", filename="agent.pt")
|
| 46 |
+
agent.load(path)
|
| 47 |
+
```
|
| 48 |
+
|
| 49 |
+
* JAX
|
| 50 |
+
|
| 51 |
+
```python
|
| 52 |
+
from skrl.utils.huggingface import download_model_from_huggingface
|
| 53 |
+
|
| 54 |
+
# assuming that there is an agent named `agent`
|
| 55 |
+
path = download_model_from_huggingface("skrl/IsaacGymEnvs-FactoryTaskNutBoltScrew-PPO", filename="agent.pickle")
|
| 56 |
+
agent.load(path)
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
# Hyperparameters
|
| 60 |
+
|
| 61 |
+
Note: Undefined parameters keep their values by default.
|
| 62 |
+
|
| 63 |
+
```python
|
| 64 |
+
# https://skrl.readthedocs.io/en/latest/api/agents/ppo.html#configuration-and-hyperparameters
|
| 65 |
+
cfg = PPO_DEFAULT_CONFIG.copy()
|
| 66 |
+
cfg["rollouts"] = 128 # memory_size
|
| 67 |
+
cfg["learning_epochs"] = 8
|
| 68 |
+
cfg["mini_batches"] = 32 # 128 * 128 / 512
|
| 69 |
+
cfg["discount_factor"] = 0.99
|
| 70 |
+
cfg["lambda"] = 0.95
|
| 71 |
+
cfg["learning_rate"] = 1e-4
|
| 72 |
+
cfg["random_timesteps"] = 0
|
| 73 |
+
cfg["learning_starts"] = 0
|
| 74 |
+
cfg["grad_norm_clip"] = 0
|
| 75 |
+
cfg["ratio_clip"] = 0.2
|
| 76 |
+
cfg["value_clip"] = 0.2
|
| 77 |
+
cfg["clip_predicted_values"] = True
|
| 78 |
+
cfg["entropy_loss_scale"] = 0.0
|
| 79 |
+
cfg["value_loss_scale"] = 1.0
|
| 80 |
+
cfg["kl_threshold"] = 0.016
|
| 81 |
+
cfg["rewards_shaper"] = None
|
| 82 |
+
cfg["state_preprocessor"] = RunningStandardScaler
|
| 83 |
+
cfg["state_preprocessor_kwargs"] = {"size": env.observation_space, "device": device}
|
| 84 |
+
cfg["value_preprocessor"] = RunningStandardScaler
|
| 85 |
+
cfg["value_preprocessor_kwargs"] = {"size": 1, "device": device}
|
| 86 |
+
```
|
agent.pickle
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b2ab85cd58e3e0e4df41dc0e319a9f368b7b33f44434d9d38de31e5cfb1aafde
|
| 3 |
+
size 1202993
|
agent.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9f60e87bef672a33f656d3531c9ef247367cc2a5831d93586599d7d18cadbbb1
|
| 3 |
+
size 622478
|