Method

frame − reference   →  dI, signed RGB difference
dI                  →  surface gradient        ← the only step that needs a choice
∇ integrate         →  depth
depth               →  5 features → newtons    ← fitted on presses of known load

Lookup table

A (90,90,90,2) array from difference colour to gradient, filled by pressing a sphere of unknown radius: a² = d(2R−d) recovers the radius and the depth datum from the data. ~700 frames per sensor.

Calibration-free

Three LEDs at known azimuths, so each channel reads the gradient projected on one direction and (gx, gy) is a 3×2 least-squares solve. No table, no sphere presses. This is the GelSight Wedge driver's approach. It recovers shape but not scale.

Fitting the newtons, and why the weights can lie

Five collinear contact features, then a monotone isotonic calibration. Isotonic clips outside its fitted range, so transfer ρ is scored on the linear projection: on the isotonic output a fully extrapolated target returns a constant, and a constant has no ranks. Least squares can also cancel large opposite-sign terms, a balance holding only at the ratios it was fitted on — that sends one row of the transfer matrix negative.

Stage 1 scored on its own — no force labels

Force estimation is image→depth then depth→newtons, and a ρ only ever scores the pair: a geometrically wrong depth that is monotone in contact size still ranks force well. Depth has no ground truth, so stage 1 is judged by eye on the panels below and by physical checks that need no labels.

dataset	n	flat-gel leak, LUT	leak, calib-free	peak [mm]	over the gel	truncated	LUT vs calib-free shape
cnc_mini_26	120	0.016	0.013	2.71	14%	90%	+0.796
cnc	120	0.014	0.013	1.66	1%	76%	+0.838
feats	120	0.064	0.041	1.12	0%	100%	+0.703
sparsh	89	0.016	0.011	0.93	0%	64%	+0.895
faf	117	0.045	0.041	1.69	8%	89%	+0.845

Leak is mean |depth| off-contact over peak — zero for a coherent surface. “Over the gel” counts peaks past the 4.25 mm elastomer, possible only where the contact runs off the sensor and the depth is extrapolated. “Truncated” is a fact about the capture, not the method, and bounds what any reconstruction can know. The last column is the two reconstructions agreeing with each other, which is evidence neither invents the shape — not that either is right.

One row needed a fix before it could be read at all — see sensors.

cnc_mini_26 — GelSight Mini, CNC presses, 0–20 N · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.

FoTa cnc_Mini — GelSight Mini, CNC probes · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.

FEATS — marker gel, FEA force labels · MARKER · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.

Sparsh / Meta — 10 gel pads · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.

FeelAnyForce — 42 captures · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.

Three claims made here and withdrawn (with their numbers)

“React's poor reconstruction is cross-sensor transfer” — the sensors were the other way round; the table's own capture is a GelSight Mini. “A DC-biased gradient integrates into a dome” — removing the DC moved the leak 0.0715 → 0.0713. “Calibration-free is 2–3× better” — three frames of noise; on 24 it is a tie. And the LED map was first chosen by a criterion that rewards a small reconstruction, which split a connector into two blobs while scoring best; it is now set by sphere presses reconstructing as circles (axis ratio 1.266 vs 1.798).