Contact force from a GelSight image,
with no force sensor

React records tactile images and sensor pose. It has no load cell, so the newtons in it are estimated from the images alone — reconstructed to a depth map, then calibrated on presses of known load.

0.986best Spearman ρ vs ground-truth force
5force-labelled datasets, one protocol
0.925agreement between two estimators sharing no calibration
0frames of our own rig in any calibration
reconstruction panel
One row per press: raw frame, signed colour difference, depth from the lookup table, depth without any table, and both as surfaces. The calibration-free column uses no per-sensor calibration at all.

Two reconstructions, one question

The only step that needs a choice is image → surface gradient. A lookup table learns it from ~700 sphere presses per sensor; a linear photometric solve needs only the LED geometry. On markerless GelSight Mini presses the calibration-free solve leads by +0.21 ρ; on a marker gel it trails by -0.18, which is what the physics predicts.

Full comparison on the results page · how it works on method.