---
license: apache-2.0
tags:
  - robotics
  - reinforcement-learning
  - dexterous-manipulation
  - mujoco
---

# Cyclops Reorient — Reward Weight Ablations

## Default Reward Parameters

| Index | Name                  | Default Weight |
|-------|-----------------------|:--------------:|
| 0     | orientation_alignment | 30.0           |
| 1     | hand_pose             | 0.2            |
| 2     | action_rate           | -0.5           |
| 3     | torque                | 0              |
| 4     | tip_slide             | -0.3           |
| 5     | cage_escape           | -100.0         |
| 6     | finger_collision      | -1.0           |
| 7     | hold_escalation       | 11.4           |
| 8     | palm_detach           | 0.5            |
| 9     | contact_count         | 0.0            |

**Success threshold:** 0.2 rad (~11.5°) | **Hold steps:** 5 | **Goal switch delay:** 20 steps

## Ablation Runs

### Original Study

`UNDESIRED_OBJECT_CONTACT_BODIES` modified to include `proximal_link_(center|left|right)`. `touch_site_fingertip` changed to a sphere on the sensor surface. Removed `proximal_link_(center|left|right)` and `distal_link_(center|left|right)` from `DR_CONTACT_GROUP_A_GEOMS`. Added `base_gripper_col` to `SELF_COLLISION_GEOMS`. Adjusted `config/cyclops_finger/rsl_rl/ppo.py`: `entropy_coef` 0.001 → 0.01, `min_std` 0.2 → 0.4, `init_std` 0.5 → 0.6.

| Run | SLURM Job | orientation_alignment | hand_pose | action_rate | torque | tip_slide | cage_escape | finger_collision | hold_escalation | palm_detach | contact_count |
|-----|-----------|:---------------------:|:---------:|:-----------:|:------:|:---------:|:-----------:|:----------------:|:---------------:|:-----------:|:-------------:|
| 01 — default | 29274656 | 30.0 | 0.2 | -0.5 | 0 | -0.3 | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 |
| 02 — action_rate -0.3 | 29275079 | 30.0 | 0.2 | **-0.3** | 0 | -0.3 | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 |
| 03 — orientation_alignment 60.0 | 29275102 | **60.0** | 0.2 | -0.5 | 0 | -0.3 | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 |
| 04 — hold_escalation 20.0 | 29275119 | 30.0 | 0.2 | -0.5 | 0 | -0.3 | -100.0 | -1.0 | **20.0** | 0.5 | 0.0 |
| 05 — tip_slide -0.1 | 29275142 | 30.0 | 0.2 | -0.5 | 0 | **-0.1** | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 |
| 06 — cage_escape -50.0 | 29275352 | 30.0 | 0.2 | -0.5 | 0 | -0.3 | **-50.0** | -1.0 | 11.4 | 0.5 | 0.0 |
| 07 — cage_escape -20.0 orientation_alignment 100.0 (continued: 29374450) | 29274710 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-20.0** | -1.0 | 11.4 | 0.5 | 0.0 |
| 08 — cage_escape -50.0 orientation_alignment 60.0 | 29275361 | **60.0** | 0.2 | -0.5 | 0 | -0.3 | **-50.0** | -1.0 | 11.4 | 0.5 | 0.0 |
| 09 — cage_escape -10.0 orientation_alignment 100.0 | 29354503 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-10.0** | -1.0 | 11.4 | 0.5 | 0.0 |
| 10 — cage_escape -5.0 orientation_alignment 100.0 | 29354510 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-5.0** | -1.0 | 11.4 | 0.5 | 0.0 |
| 11 — cage_escape -20.0 orientation_alignment 150.0 | 29354667 | **150.0** | 0.2 | -0.5 | 0 | -0.3 | **-20.0** | -1.0 | 11.4 | 0.5 | 0.0 |

### New Study — Fingertip Contact Target Reward

Added `Fingertip_Contact_Target_Reward`. Replaced `Distal_Link` with `Fingerpad_Sensor` in `TIP_Collision_Geoms`.

| Run | SLURM Job | orientation_alignment | hand_pose | action_rate | torque | tip_slide | cage_escape | finger_collision | hold_escalation | palm_detach | contact_count |
|-----|-----------|:---------------------:|:---------:|:-----------:|:------:|:---------:|:-----------:|:----------------:|:---------------:|:-----------:|:-------------:|
| 12 — cage_escape -20.0 orientation_alignment 100.0 | 29374461 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-20.0** | -1.0 | 11.4 | 0.5 | 0.0 |
| 13 — cage_escape -10.0 orientation_alignment 100.0 | 29374463 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-10.0** | -1.0 | 11.4 | 0.5 | 0.0 |
| 14 — cage_escape -10.0 orientation_alignment 100.0 contact_count 2 | 29374469 | **100.0** | 0.2 | -0.5 | 0 | -0.3 | **-10.0** | -1.0 | 11.4 | 0.5 | **2.0** |