| --- |
| tags: |
| - microduck |
| - robotics |
| - reinforcement-learning |
| - onnx |
| - microduck-slot:walk |
| library_name: microduck |
| pipeline_tag: robotics |
| --- |
| |
| # walk-getup |
|
|
| Walk, and get back up after a fall, with +-1 deg of gear play in every servo (Mjlab-VelStand-Flat-Backlash-MicroDuck on Pollen's soft_carpet branch). Trained in simulation only with Pollen's own recipe from that branch: behaviour cloning from their alpha_stand get-up expert on fallen frames plus a walk anchor on upright frames, 3,000 iterations on top of our 6,000-iteration run, 4096 envs, 83 min on one RTX 5090. In 7 forced-spawn takes of 12 s with a shove every 1-2 s it went down 35 times and got back up 31 times. Deep (10 mm) soft carpet with the motor model still knocks it down. Not yet tested on a real Microduck. |
|
|
| A **perpetual** policy for the [microduck](https://github.com/pollen-robotics/microduck) (61-D observation, 14 actions, 50 Hz). Runs until told otherwise — a gait for the `walk` slot. |
|
|
| ## Run it on a robot |
|
|
| ```bash |
| sudo robotctl policy load walk witcheer/microduck-walk-getup |
| ``` |
|
|
| The observation normalizer is baked into `policy.onnx`; feed raw observations. |
| `manifest.json` follows schema 2 of the microduck policy manifest (`docs/policy-manifest.md` in the daemon repo). |
|
|
| ## Training |
|
|
| - **repo**: `pollen-robotics/microduck_rl` |
| - **branch**: `HEAD` |
| - **commit**: `695f3b751` |
| - exported from a checkout with uncommitted changes |
|
|
| ## Results in simulation (v1, 2026-10-12) |
|
|
| Simulation only. This policy has not been run on a real Microduck. |
|
|
| ### Where the get-up comes from |
|
|
| The get-up is Pollen's, not ours. Pollen ships `alpha_stand`, a stand expert that gets up from the back and the belly, and their public `soft_carpet` branch of microduck_rl (commit `695f3b7`, `tasks/distill.py`) teaches it to a walking policy: behaviour cloning from the expert on fallen frames, a walk anchor on upright frames, PPO on top. Their own note on that branch says the student cannot discover the get-up, so they distil it, and our own runs agree: ten get-up runs of ours (seven from scratch, three warm-started) all froze half-rolled on the back. What we did: rebuilt `alpha_stand.onnx` into a local checkpoint, ran that recipe on the gear-play walking task with our own level 1 walk as the anchor, and trained it. The manifest's `branch: HEAD` is the detached `soft_carpet` worktree at the commit above. |
| |
| ### Training |
| |
| - Task `Mjlab-VelStand-Flat-Backlash-MicroDuck` (the maker's task: a free hinge of ±1° in series with each of the 14 servo joints, the encoder reading through it as a real servo does). |
| - Warm start from level 11 of our skill tree (6,000 iterations of the same recipe on `Mjlab-VelStand-Flat-MicroDuck`, no gear play), then 6,000 -> 9,000 iterations here. 4096 parallel environments, 83 min across two blocks at about 1.68 s/iteration on one RTX 5090. |
| - Mean reward, 100-iteration windows (last full window before the block cut): -5.1 at 7,000, -4.1 at 8,000, -3.6 at 8,900. Flat from about 7,000. Recovery-success term 0.020 in each of those windows. |
| |
| ### Proof takes |
| |
| `headless_play` in `Mjlab-VelStand-Flat-Backlash-MicroDuck`, forced spawns, 12 s each (one episode), samples every 0.1 s, body-frame gravity and trunk height read from the simulator. The branch keeps its `topple_push` event active in play, so the duck is shoved over every 1 to 2 s and no take "holds" for 12 s: read the down and rise counts. Down = tilt over 60°, back up = body-frame gravity z below -0.9. |
| |
| - 7 takes (3 from flat on the back, 2 face down, 2 standing): **down 35 times (ground spawns included), back up 31 times**, upright 50% to 73% of each take, 59% on average. Zero environment resets inside any take (only the episode end at 12 s). |
| - from the back: 15 down, 14 rises. All three starts were truly flat on the back (gravity x -1.00 after reset). The first get-up that stayed up for at least 1 s came at 2.2, 2.7 and 2.9 s, the trunk climbing through intermediate heights (one take: 46, 77, 97, 104, 113 mm). |
| - face down: 9 down, 7 rises. standing: 11 down, 10 rises. Median time back up 0.8 to 1.0 s per spawn. |
| - For comparison, level 11 (no gear play) in its own sim on 2026-10-05: 34 down, 30 rises in 9 takes. Gear play costs the get-up nothing we can measure at this sample size. |
| |
| ### Soft carpet (Pollen's `infer_policy.py`, motor model on) |
|
|
| Pollen's inference sim, command 0.30 m/s forward, 12 s, deterministic (one take per case). |
|
|
| - Flat floor: upright for the whole take, achieved forward speed 0.11 to 0.13 m/s. |
| - 6 mm carpet: no fall (lowest 1 s trunk average 87 mm). |
| - 10 mm carpet: down by 3 s, back up by 4 s, down again at 11 s. Level 11 lay at about 40 mm from 3 s to the end on the same case. Less stuck, not fixed: deep soft carpet with the motor model still knocks it down. |
|
|
| ### Not tested |
|
|
| Rough ground (our rough-ground retrain of this policy, level 11d, did no better than this one and is not published), head commands, velocity tracking in the branch sim, and any real robot. |
|
|
| ### Local changes in the training checkout (`dirty: true`) |
|
|
| - `tasks/distill.py`: three lines so the expert and the walk anchor load from local checkpoints (`MICRODUCK_BC_EXPERT`, `MICRODUCK_BC_ANCHOR`) instead of Pollen's wandb runs. This one matters for the run. |
| - `tasks/mdp.py` and `microduck_velocity_env_cfg.py`: our terrain patches (ground-height reads, terrain ladder), ported for the rough level. The flat task's full config is identical with and without them. |
| - `microduck_velstand_env_cfg.py`: a deep-carpet switch (`MICRODUCK_CARPET_DEEP`) for another run, off for this one, so the carpet randomisation is at the maker's values. |
|
|
| ### Skill tree |
|
|
| Level 11c of a skill tree trained in simulation for a Microduck still on order. Every level, including the ones that did not work, is in the dataset [witcheer/microduck-skill-tree](https://huggingface.co/datasets/witcheer/microduck-skill-tree) (folder `level-11c-walk-getup-gear-play`: reward curve, queue item, manifest, carpet takes; the flat proof takes are the `w-*` files in `level-11d-walk-getup-rough-gear-play/proof-takes/proof-x-c-2026-10-11`). The other policies are under `witcheer/microduck-*`; [walk-recover](https://huggingface.co/witcheer/microduck-walk-recover) is the walk without the get-up. |
|
|
| Simulation only. Not yet tested on a real Microduck. |
|
|
| Pin this version: |
|
|
| ```bash |
| sudo robotctl policy load walk witcheer/microduck-walk-getup@v1 |
| ``` |
|
|
