NeuralActuator pretrained checkpoints

Pretrained checkpoints for the RSS 2026 paper NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception. The model is a Flax Transformer encoder (4 layers, hidden dim 192, 4 attention heads, ~1.4M parameters) over an 8-step history of commanded positions, proprioception and actuator telemetry, with three output heads: joint torque, 3-axis end-effector external force with a contact gate, and per-motor condition logits. The torque head is trained without torque labels by backpropagating a position loss through a differentiable simulator (MuJoCo MJX); the force and condition heads use direct supervision from the force sensor and the condition flags.

Two platforms are covered: the ROBOTIS OpenManipulator-X (Dynamixel XM430-W350, 5 motors), the SO-101 (Feetech STS3215, LeRobot ecosystem, 6 joints) and the 7-DoF Franka Panda. Architecture and feature normalization are identical across platforms; only the joint count and the input feature layout differ.

• Code, configs, dataset and full documentation: github.com/Frank-ZY-Dou/Dynamics-Modeling/tree/main/NeuralActuator
• Training data: the Neural Actuation Dataset (NAD), 450 trajectories across 45 tasks on the two platforms, shipped with the code release and mirrored on the Hub.
• Project page: frank-zy-dou.github.io/projects/NeuralActuator

Checkpoints

Eleven checkpoints, one per benchmark configuration. Verify downloads against these md5 sums:

File	Benchmark	md5
omx_no_load_with_gripper.pkl	Table 1, with-gripper tasks	8233a0ba1b9629b2b7dd5b2a81a2e536
omx_no_load_no_gripper.pkl	Table 1, no-gripper tasks	548929e3b54680a467b27aef48983d70
omx_force_sensor.pkl	Table 2, force sensor	486b4a7681d379825643d3f2eab3d659
omx_weight_all_ema.pkl	Table 3, all nine weight tasks (EMA weights, reported above)	c463eaabf653884158f51201097ea00c
omx_weight_all.pkl	Table 3, all nine weight tasks (raw weights of the same run)	60ecebb84c3e8d1b94d651e96ff355fc
omx_pick_place.pkl	Table 3, pick-and-place subset	8d9834930126459c906c3cf756e39859
omx_motor_condition.pkl	Table 4, motor condition	e15721cc63fe56bbfaf89d2eb1b0314d
so101_weight.pkl	SO-101 weight benchmark, paper protocol (six 300-500 g tasks)	95e9ed8b076f4011bfd3d6dc32876363
so101_weight_extended.pkl	SO-101 weight benchmark, extended ten-task data	6ab6f6ea031d04a5c50b76eabd1c2020
so101_weight_residual.pkl	SO-101 weight benchmark, residual parameterization	37027625f46837fc89fc66a2acb47574
franka_lift_hold.pkl	Franka Panda lift-and-hold benchmark	94d1e67fe6fd1969f660746ffd213f20

Table numbers refer to the results tables in the paper and the GitHub README, which also carries the evaluation numbers of each released checkpoint on the NAD test split. The Table 3 checkpoint ships as both the raw and the EMA weights of the same training run; the results table reports the EMA variant.

Each .pkl is a plain pickle of a dict with keys params (Flax parameter tree), ema_params (EMA copy of the same tree), and feature_mean / feature_std (normalization statistics), so evaluation needs no extra files.

How to load

import pickle
from huggingface_hub import hf_hub_download

path = hf_hub_download("frankzydou/NeuralActuator", "omx_weight_all_ema.pkl")
ckpt = pickle.load(open(path, "rb"))   # params, ema_params, feature_mean, feature_std

To run the checkpoints, use the inference and evaluation scripts from the GitHub release, which reconstruct the model around the parameter tree:

# single-trajectory rollout with predicted force
python infer_actuator.py --robot omx --checkpoint omx_weight_all_ema.pkl \
    --config configs/lift_hold.yaml \
    --csv data/weight/pick_place_object_500g/test/001.csv \
    --out outputs/pick_place_500g_pred.npz

# telemetry-only virtual force sensor (no simulator in the loop)
python infer_actuator.py --robot omx --checkpoint omx_weight_all_ema.pkl \
    --config configs/lift_hold.yaml \
    --csv data/weight/pick_place_object_500g/test/001.csv \
    --force_only --out outputs/pick_place_500g_force.npz

# full benchmark evaluation
bash scripts/eval_force_sensor.sh omx_force_sensor.pkl 0

The config only supplies the architecture and simulation settings, which all released checkpoints share, so any config for the same robot works. In the telemetry-only mode a forward pass takes about 2 ms per step on a plain CPU after JIT warmup, well inside the ~60 Hz servo telemetry rate.

License

MIT, same as the code release.

Citation

@inproceedings{dou2026neuralactuator,
  title     = {{NeuralActuator}: Neural Actuation Modeling for Robot Dynamics and External Force Perception},
  author    = {Dou, Zhiyang and Onyemelukwe, John U. and Zhang, Hangxing and Zhang, Heng and Guo, Minghao and Tian, Yunsheng and Lipiec, Michal Piotr and Jacob, Joshua and Liu, Chao and Chen, Peter Yichen and Ivanov, Yuri and Matusik, Wojciech},
  booktitle = {Proceedings of Robotics: Science and Systems (RSS)},
  year      = {2026}
}