| --- |
| license: apache-2.0 |
| tags: |
| - robotics |
| - reinforcement-learning |
| - dexterous-manipulation |
| - mujoco |
| --- |
| |
| # Cyclops Reorient — Reward Weight Ablations |
|
|
| ## Default Reward Parameters |
|
|
| | Index | Name | Default Weight | |
| |-------|-----------------------|:--------------:| |
| | 0 | orientation_alignment | 30.0 | |
| | 1 | hand_pose | 0.2 | |
| | 2 | action_rate | -0.5 | |
| | 3 | torque | 0 | |
| | 4 | tip_slide | -0.3 | |
| | 5 | cage_escape | -100.0 | |
| | 6 | finger_collision | -1.0 | |
| | 7 | hold_escalation | 11.4 | |
| | 8 | palm_detach | 0.5 | |
| | 9 | contact_count | 0.0 | |
| |
| **Success threshold:** 0.2 rad (~11.5°) | **Hold steps:** 5 | **Goal switch delay:** 20 steps |
| |
| ## Ablation Runs |
| |
| ### Original Study |
| |
| `UNDESIRED_OBJECT_CONTACT_BODIES` modified to include `proximal_link_(center|left|right)`. `touch_site_fingertip` changed to a sphere on the sensor surface. Removed `proximal_link_(center|left|right)` and `distal_link_(center|left|right)` from `DR_CONTACT_GROUP_A_GEOMS`. Added `base_gripper_col` to `SELF_COLLISION_GEOMS`. Adjusted `config/cyclops_finger/rsl_rl/ppo.py`: `entropy_coef` 0.001 → 0.01, `min_std` 0.2 → 0.4, `init_std` 0.5 → 0.6. |
| |
| | Run | SLURM Job | orientation_alignment | hand_pose | action_rate | torque | tip_slide | cage_escape | finger_collision | hold_escalation | palm_detach | contact_count | |
| |-----|-----------|:---------------------:|:---------:|:-----------:|:------:|:---------:|:-----------:|:----------------:|:---------------:|:-----------:|:-------------:| |
| | 01 — default | 29274656 | 30.0 | 0.2 | -0.5 | 0 | -0.3 | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| | 02 — action_rate -0.3 | 29275079 | 30.0 | 0.2 | **-0.3** | 0 | -0.3 | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| | 03 — orientation_alignment 60.0 | 29275102 | **60.0** | 0.2 | -0.5 | 0 | -0.3 | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| | 04 — hold_escalation 20.0 | 29275119 | 30.0 | 0.2 | -0.5 | 0 | -0.3 | -100.0 | -1.0 | **20.0** | 0.5 | 0.0 | |
| | 05 — tip_slide -0.1 | 29275142 | 30.0 | 0.2 | -0.5 | 0 | **-0.1** | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| | 06 — cage_escape -50.0 | 29275352 | 30.0 | 0.2 | -0.5 | 0 | -0.3 | **-50.0** | -1.0 | 11.4 | 0.5 | 0.0 | |
| | 07 — cage_escape -20.0 orientation_alignment 100.0 (continued: 29374450) | 29274710 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-20.0** | -1.0 | 11.4 | 0.5 | 0.0 | |
| | 08 — cage_escape -50.0 orientation_alignment 60.0 | 29275361 | **60.0** | 0.2 | -0.5 | 0 | -0.3 | **-50.0** | -1.0 | 11.4 | 0.5 | 0.0 | |
| | 09 — cage_escape -10.0 orientation_alignment 100.0 | 29354503 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-10.0** | -1.0 | 11.4 | 0.5 | 0.0 | |
| | 10 — cage_escape -5.0 orientation_alignment 100.0 | 29354510 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-5.0** | -1.0 | 11.4 | 0.5 | 0.0 | |
| | 11 — cage_escape -20.0 orientation_alignment 150.0 | 29354667 | **150.0** | 0.2 | -0.5 | 0 | -0.3 | **-20.0** | -1.0 | 11.4 | 0.5 | 0.0 | |
| |
| ### New Study — Fingertip Contact Target Reward |
| |
| Added `Fingertip_Contact_Target_Reward`. Replaced `Distal_Link` with `Fingerpad_Sensor` in `TIP_Collision_Geoms`. |
|
|
| | Run | SLURM Job | orientation_alignment | hand_pose | action_rate | torque | tip_slide | cage_escape | finger_collision | hold_escalation | palm_detach | contact_count | |
| |-----|-----------|:---------------------:|:---------:|:-----------:|:------:|:---------:|:-----------:|:----------------:|:---------------:|:-----------:|:-------------:| |
| | 12 — cage_escape -20.0 orientation_alignment 100.0 | 29374461 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-20.0** | -1.0 | 11.4 | 0.5 | 0.0 | |
| | 13 — cage_escape -10.0 orientation_alignment 100.0 | 29374463 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-10.0** | -1.0 | 11.4 | 0.5 | 0.0 | |
| | 14 — cage_escape -10.0 orientation_alignment 100.0 contact_count 2 | 29374469 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-10.0** | -1.0 | 11.4 | 0.5 | **2.0** | |
|
|
|
|
