README.md

---
license: cc-by-nc-4.0
tags:
  - emg
  - semg
  - biosignals
  - musculoskeletal
  - motor-control
  - robotics
  - pytorch-lightning
  - regression
library_name: pytorch
pipeline_tag: other
---

# emg2tendon — pretrained models

Seq2seq regression from surface EMG to **musculoskeletal tendon controls**:
**16-channel sEMG @ 2 kHz → 39-channel MyoHand tendon control ∈ [0, 1]**, over
2-second windows (`T = 4000`). Trained on the full **emg2pose** dataset
(25,253 recordings, 193 subjects, ~370 h), with tendon targets produced by a
QForce inverse-dynamics pipeline through the MyoSuite **MyoHand** model.

Reference implementation + eval code: **https://github.com/sagarverma/emg2tendon**
Project page: **https://emg2tendon.github.io**
Paper: *emg2tendon: From sEMG Signals to Tendon Control in Musculoskeletal Hands*, Sagar Verma, RSS 2025.

## Files

| File | Model | Params | val tendon RMSE | open-loop pose (deg) |
|---|---|--:|--:|--:|
| `tds.ckpt` | TDS (time-depth-separable conv) | 0.10 M | 0.310 | 15.1–16.2 |
| `sensingdynamics.ckpt` | SensingDynamics | 0.56 M | 0.308 | 15.1–16.2 |
| `neuropose.ckpt` | NeuroPose | 7.15 M | 0.308 | 15.1–16.2 |
| `cldm.ckpt` | Conditional Latent Diffusion (self-contained: both VAEs + U-Net) | 7.01 M | 0.440 | 16.8–17.6 |
| `emg_stats.npz` | per-channel EMG mean/std — **required for inference** | — | — | — |

Checkpoints are PyTorch-Lightning `.ckpt` files (`state_dict` +
`hyper_parameters`), loaded by the wrappers in the GitHub repo
(`RegressionModule` for the three baselines, `CLDMModule` for CLDM).

### `ablations/` — improvement-campaign checkpoints

Trained on an **8,000-recording subset** for speed, to ablate a modernized
training recipe (window-sampling fix, velocity + smoothness loss, EMG
augmentation, pose-sensitivity-weighted tendon loss, temporal architectures):

| File | Description | Params | val tendon RMSE | Δ vs reference |
|---|---|--:|--:|--:|
| `tds_old.ckpt` | TDS, original recipe (reference) | 0.10 M | 0.3105 | — |
| `tds_new.ckpt` | TDS, new recipe + sensitivity weighting | 0.10 M | 0.3090 | −0.5% |
| `tds_new_nosens.ckpt` | TDS, new recipe, no sensitivity weighting | 0.10 M | 0.3089 | −0.5% |
| `tcn_new.ckpt` | TCN (~2 s receptive field) | 1.57 M | 0.3057 | −1.5% |
| `gru_new.ckpt` | GRU velocity-decode (best) | 1.17 M | 0.3049 | −1.8% |

> **Use `ablations/emg_stats.npz` with these** — the normalization statistics
> were computed over the 8k subset and differ from the full-25k statistics at
> the root.

## Usage

```bash
git clone https://github.com/sagarverma/emg2tendon && cd emg2tendon
pip install torch pytorch-lightning hydra-core diffusers "numpy==1.26.4"

python - <<'PY'
from huggingface_hub import snapshot_download
print(snapshot_download("Micropilot/emg2tendon"))
PY
```

Then run the shipped eval / render entrypoints:

```bash
# pose-space evaluation (per-step + open-loop rollout through MyoHand)
python scripts/evaluate_pose.py --model tds --checkpoint tds.ckpt \
    --index index.json --stats_cache emg_stats.npz

# side-by-side MuJoCo video (reference vs achieved pose)
MUJOCO_GL=egl python scripts/render_model.py --model tds --checkpoint tds.ckpt \
    --emg emg/<base>.npy --pose pose/<base>.npy --index index.json \
    --stats_cache emg_stats.npz --out out.mp4
```

Inference contract: EMG is per-channel standardized with the shipped
`mean`/`std` (`(emg - mean) / std`, guarding `std < 1e-8`); tendon output stays
in native `[0, 1]` space and is never normalized. Tensors are channel-first
(`[N, C, T]`) at the module boundary — see `CONTRACT.md` in the code repo.

## Evaluation

Protocol: the three held-out emg2pose generalization conditions (unseen **user**,
unseen **stage**, **user+stage**), 200 recordings per condition. Predicted
tendon controls are forward-simulated through MyoHand and the achieved pose is
compared to the emg2pose ground truth, both **per-step** and in a **0.5 s
open-loop rollout**.

| Model | per-step (deg) | open-loop 0.5 s (deg) |
|---|--:|--:|
| *ground-truth tendon (ID ceiling)* | *0.11* | *~14.0* |
| TDS / SensingDynamics / NeuroPose | ~0.09 | **15.1–16.2** |
| CLDM | ~0.11 | 16.8–17.6 |

The inverse-dynamics step itself is near-exact (0.109° MAE per-step over all
25,253 recordings), so per-step numbers sit at the ID ceiling for every model.

### Limitations

- **The open-loop ceiling is ~14°**: even perfect tendon controls drift under
  MyoHand muscle dynamics. The baselines are already within ~1–2° of it, so
  there is little headroom for a better EMG→tendon model to improve open-loop
  pose — these baselines are near-optimal for this metric, not the bottleneck.
- **`cldm.ckpt` is undertrained** (diffusion stage cut short for compute
  budget) and underperforms the regression baselines here — the opposite of the
  paper's ranking. Treat it as a starting point, not a faithful CLDM result.
- Trained only on emg2pose (wrist-worn 16-ch sEMG band, right/left hand, seated
  desk-scale gestures). No claims outside that distribution.
- Ablation checkpoints use an 8k subset and are not directly comparable to the
  root full-25k checkpoints.

## License

`cc-by-nc-4.0`, inherited from the **emg2pose** dataset these models are trained
on (Meta, CC BY-NC 4.0). Non-commercial use only.

## Citation

```bibtex
@inproceedings{verma2025emg2tendon,
  title        = {{emg2tendon: From sEMG Signals to Tendon Control in Musculoskeletal Hands}},
  author       = {{Sagar Verma}},
  year         = 2025,
  booktitle    = {Robotics: Science and Systems}
}
```