README.md

# Dropbear Locomotion

COM-guided humanoid locomotion for the Dropbear robot in NVIDIA Isaac Lab,
trained with RSL-RL PPO. This repository contains the robot USD, complete
training environment, reward and penalty stack, checkpoint adapters,
deterministic evaluator, interactive live viewer, and the validated v0.1.0
policy artifacts. The experimental v0.2.0 update adds terrain-height
observations, low-obstacle curriculum training, stronger bilateral gait
shaping, and real training-environment passthrough in the live viewer.

[![License: CC BY-NC-SA 4.0](https://img.shields.io/badge/license-CC--BY--NC--SA--4.0-blue)](LICENSE)
[![Hugging Face](https://img.shields.io/badge/model-Hugging%20Face-yellow)](https://huggingface.co/cudabenchmarktest/dropbear-locomotion)

![Actual vectorized training environment mirrored in the interactive viewer](media/dropbear_train_live_domain_randomization.png)

The frame above is an actual vectorized training environment mirrored from the
headless trainer, not an independent checkpoint rollout. `TRAIN LIVE • DR`
streams its complete robot state, command, terrain level, randomized mass, and
friction to the split-GPU renderer. Click the button to restore the saved
manual rollout and joystick control.

## Training lineage and credits

The original Dropbear policy weights through approximately training iteration
7000 come from [Hyperspawn Robotics](https://github.com/Hyperspawn). The newer
continuation layers—including COM-guided training, gait and contact shaping,
checkpoint adapters, deterministic evaluation, and live-view tooling—are
co-authored with [robit-man](https://github.com/robit-man). Checkpoint
iteration numbers preserve that continuous training lineage.

## Experimental v0.2.0 obstacle checkpoint

`model_10479.pt` is the current Stage 65b checkpoint. It expands the actor and
critic from 58/72 to 162/176 inputs by appending a 104-value terrain-height
scan. The new columns were zero-initialized when branching from
`model_10280.pt`, preserving the source policy exactly before obstacle
fine-tuning.

Stage 65b used 512 parallel environments, a fixed `+0.20 m/s` forward command,
disabled pushes, and a curriculum containing 35% flat retention tiles and 65%
anchored cross-path boxes. Full box height spans 3–14 cm; the terrain
curriculum promotes successful environments through the difficulty rows.

| Final training signal at iteration 10479 | Value |
|---|---:|
| Normal time-out fraction | 84.46% |
| Bad-orientation termination | 15.54% |
| Feet-overlap termination | 0.00% |
| Mean episode length | 736.2 / 1,000 steps |
| Mean terrain level | 2.57 |
| Planar velocity error | 0.258 m/s |

This checkpoint is published for continued terrain and gait training. It is
not marked converged: the last 20 updates had an 85.4% mean horizon and the
reward curve had not plateaued. The v0.1.0 checkpoint below remains the
deterministically validated flat-ground release.

![Stage 65b low-obstacle training metrics](media/stage65b_obstacle_training_metrics.png)

### Directional obstacle continuation

Stage 66 continues `model_10479.pt` across forward, backward, robot-left,
robot-right, turn-left, and turn-right box approaches. The 3–14 cm obstacle is
rotated and placed on the matching approach path; flat retention environments
cycle through the same six command modes. Knee and step-through shaping use
equal left/right weights. Mass and contact friction remain randomized, while
impulse disturbances are deferred until this first directional gate is
stable.

Reproduce the branch with:

```bash
DROPBEAR_TRAIN_GPU=0 bash scripts/train_stage66_directional_obstacles.sh
```

For synchronized three-A100 training, expose the three physical device indices
as a comma-separated list and set the worker count:

```bash
DROPBEAR_TRAIN_GPU=0,1,2 \
DROPBEAR_TRAIN_GPU_COUNT=3 \
bash scripts/train_stage66_directional_obstacles.sh
```

Boxes isolate single clearance and landing failures. The intended progression
after this gate is curb-style step-up/down, then repeated stairs; stairs are
not mixed into the initial branch so bilateral regressions remain attributable.

### v0.2.1 live-viewer fix

Manual control can remain pinned to the validated Stage 65b actor while the
upper-right metrics and `TRAIN LIVE` mode follow the active Stage 66 run. This
prevents a newly written training checkpoint from replacing the manual actor
mid-episode:

```bash
CUDA_VISIBLE_DEVICES=1 env_isaaclab/bin/python -u \
  scripts/play_dropbear_live.py \
  --actor-base-lin-vel \
  --viewer-com-control \
  --viewer-directional-obstacle-terrain \
  --viewer-disable-pushes \
  --task Isaac-Dropbear-Velocity-Play-v0 \
  --checkpoint checkpoints/dropbear_locomotion_v0.2.0/model_10479.pt \
  --follow-checkpoints logs/rsl_rl/dropbear_velocity/STAGE66_RUN \
  --pin-manual-policy \
  --training-live-state /tmp/dropbear_training_live.json \
  --viewer-command 0.20 \
  --viewer-fps 50 \
  --viewer-display-fps 50 \
  --device cuda:0 \
  --num_envs 1 \
  --viz none
```

The simulation and RTX render stay on the selected CUDA device. The desktop
window presents consecutive policy frames and uses optical flow only between
two genuine rendered frames; scene resets are hard cuts.

## v0.1.0 checkpoint

`model_8745.pt` is the frozen Stage 46 checkpoint. It uses a 58-dimensional
actor observation, a 72-dimensional privileged critic observation, and 14
joint-position actions. The actor includes measured base velocity for
closed-loop correction and two COM-reference error channels.

Deterministic evaluation used the full 20-second/1,000-step horizon, exact
commands, a plane, disabled pushes, nominal policy-joint reset, and strict
inter-foot gates of 0.16 m center distance and 0.10 m anatomical lateral
separation.

| Command | Trials | Falls | Commanded | Mean COM velocity | Mean COM error |
|---|---:|---:|---:|---:|---:|
| Forward | 32 | 0 | `(+0.20, 0.00)` | `(+0.209, +0.014)` | 0.094 m/s |
| Backward | 32 | 0 | `(-0.20, 0.00)` | `(-0.148, +0.005)` | 0.119 m/s |
| Left | 32 | 0 | `(0.00, +0.20)` | `(+0.038, +0.163)` | 0.097 m/s |
| Right | 32 | 0 | `(0.00, -0.20)` | `(+0.004, -0.119)` | 0.149 m/s |

All 128 cardinal trials timed out normally with zero falls. Forward tracking is
already close to the requested speed. Backward and lateral tracking are stable
but conservative, especially robot-right; these are honest targets for the
next multi-command consolidation pass. Full machine-readable results are in
[evaluation/cardinal_0.20_strict_128.json](evaluation/cardinal_0.20_strict_128.json).

![Stage 46 training metrics](media/stage46_training_metrics.png)

## Included artifacts

| Path | Purpose |
|---|---|
| `checkpoints/dropbear_locomotion_v0.2.0/model_10479.pt` | Experimental terrain-scan/low-obstacle RSL-RL checkpoint |
| `checkpoints/dropbear_locomotion_v0.2.0/terrain_scan_branch_parent_model_10280.pt` | Exact zero-initialized terrain-scan branch parent |
| `configs/release_v0.2.0/` | Resolved Stage 65b RSL-RL/environment configs |
| `training/stage65b/` | Stage 65b TensorBoard event history |
| `checkpoints/dropbear_locomotion_v0.1.0/model_8745.pt` | Full RSL-RL checkpoint: actor, critic, normalizers, optimizer state |
| `checkpoints/dropbear_locomotion_v0.1.0/policy.pt` | Exported TorchScript actor |
| `checkpoints/dropbear_locomotion_v0.1.0/policy.onnx*` | Exported ONNX actor and external weights |
| `checkpoints/dropbear_locomotion_v0.1.0/warm_start_model_8666.pt` | Exact Stage 45 parent used for the final continuation |
| `dropbear_walk/isaaclab_asset/dropbear.usd` | Robot articulation and visual/collision asset |
| `configs/release_v0.1.0/` | Resolved RSL-RL and environment configs |
| `training/stage46/` | TensorBoard event history |
| `evaluation/` | Deterministic validation reports |

Use `model_8745.pt` for Isaac Lab playback or continued training. Use
`policy.pt` or `policy.onnx` for deployment integration after reproducing the
same 58-value observation order, normalization, 14-action order, action scale,
and control rate described in [docs/CHECKPOINTS.md](docs/CHECKPOINTS.md).

## Quick start

The release was tested with Isaac Sim 6.0.0, the Isaac Lab repository at commit
`a4a7602f29e755e2673fe0022ea35566df6dd7d5`, Isaac Lab Python package 4.5.22,
RSL-RL 5.0.1, PyTorch 2.10.0+cu128, and Python 3.12.

```bash
git clone https://github.com/robit-man/dropbear-locomotion.git
cd dropbear-locomotion
git lfs install
git lfs pull

git clone https://github.com/isaac-sim/IsaacLab.git
git -C IsaacLab checkout a4a7602f29e755e2673fe0022ea35566df6dd7d5

# Install Isaac Lab/Isaac Sim according to the upstream instructions, then:
./IsaacLab/isaaclab.sh -p -m pip install -e ./dropbear_walk
```

Open the interactive live viewer:

```bash
./IsaacLab/isaaclab.sh -p scripts/play_dropbear_live.py \
  --actor-base-lin-vel \
  --viewer-com-control \
  --viewer-gait-period 0.55 \
  --viewer-com-stand-height 1.052 \
  --viewer-com-height-delta 0 \
  --viewer-com-height-error-scale 0.03 \
  --viewer-com-vertical-velocity-error-scale 0.08 \
  --viewer-plane \
  --viewer-disable-pushes \
  --viewer-reset-policy-joints-only \
  --viewer-reset-joint-position-range 1.0 1.0 \
  --viewer-reset-joint-velocity-range 0.0 0.0 \
  --task Isaac-Dropbear-Velocity-Play-v0 \
  --checkpoint checkpoints/dropbear_locomotion_v0.1.0/model_8745.pt \
  --viewer-command 0.0 \
  --viewer-lateral-speed 0.20 \
  --viewer-yaw-rate 0.0 \
  --viewer-joystick-max-speed 1.0 \
  --viewer-joystick-max-yaw-rate 1.0 \
  --device cuda:0 \
  --num_envs 1 \
  --viz none \
  --real-time
```

The first joystick movement takes ownership of the planar command. Up is robot
forward, left is robot-left, right is robot-right, down is robot-backward, and
release springs to zero. At the pad rim, the left/right sectors add yaw toward
the commanded side while retaining strafe; the top/bottom sectors remain
straight. Drag anywhere outside the pad to orbit the camera.

The viewer keeps native 960×540 renders at the 50 Hz policy cadence. The
display-attached GPU presents the completed RGB frames and desktop controls.
Simulation, render, and display hardware names are detected at runtime rather
than hard-coded.

For a live trainer/viewer split, add
`--live-preview-state /tmp/dropbear_training_live.json` to
`scripts/train_dropbear.py` and
`--training-live-state /tmp/dropbear_training_live.json` to the viewer. The
button beside the joystick then switches between manual control and a sampled
environment from the running vectorized trainer. See
[docs/LIVE_VIEWER.md](docs/LIVE_VIEWER.md) for the complete commands and state
contract.

## Documentation

- [Training and reproduction](docs/TRAINING.md)
- [COM, gait, arm, and foot reward design](docs/REWARD_DESIGN.md)
- [Checkpoint and deployment contract](docs/CHECKPOINTS.md)
- [Live viewer and split-GPU rendering](docs/LIVE_VIEWER.md)
- [Pose-transfer adapter and current boundaries](docs/POSE_TRANSFER.md)

## Scope and safety

This is a simulation research release. It is not a hardware safety controller.
The v0.1 checkpoint is validated only in the conditions reported above. The
v0.2 terrain checkpoint is explicitly experimental. Push robustness, 0.5+ m/s
motion, crouched locomotion, and ground-to-stand recovery require additional
training and verification before physical use.

## License

Code, policy weights, media, and the supplied USD asset are released under
[CC BY-NC-SA 4.0](LICENSE).