Spaces:

yxma
/

react-force-recovery

Running

App Files Files Community

yxma commited on 17 days ago

Commit

d47f18b

verified ·

1 Parent(s): 5d823e5

force recovery: methods, evaluation, debug log

Browse files

Files changed (12) hide show

.gitattributes +1 -0
assets/cross_dataset.png +2 -2
assets/panel_sparsh.png +2 -2
assets/panel_sparsh_zh.png +2 -2
assets/pred_vs_gt.png +2 -2
assets/sparsh_channel_fix.png +3 -0
gallery.html +4 -1
index.html +4 -1
method.html +7 -8
results.html +4 -1
sensors.html +101 -0
workbench.html +4 -1

.gitattributes CHANGED Viewed

@@ -136,3 +136,4 @@ assets/panel_sparsh_zh.png filter=lfs diff=lfs merge=lfs -text
 assets/react_normals.png filter=lfs diff=lfs merge=lfs -text
 assets/pred_vs_gt.png filter=lfs diff=lfs merge=lfs -text
 assets/cross_dataset.png filter=lfs diff=lfs merge=lfs -text

 assets/react_normals.png filter=lfs diff=lfs merge=lfs -text
 assets/pred_vs_gt.png filter=lfs diff=lfs merge=lfs -text
 assets/cross_dataset.png filter=lfs diff=lfs merge=lfs -text
+assets/sparsh_channel_fix.png filter=lfs diff=lfs merge=lfs -text

assets/cross_dataset.png CHANGED Viewed

Git LFS Details

SHA256: 11ec061b65ebbc408c14829bf64133a95ee4db84f48d77a8550687d808e90c24
Pointer size: 131 Bytes
Size of remote file: 130 kB

Git LFS Details

SHA256: 759c3944ca0870121e696c8045e36b99b779c1764292a78fc99a2cd1833896a2
Pointer size: 131 Bytes
Size of remote file: 128 kB

assets/panel_sparsh.png CHANGED Viewed

Git LFS Details

SHA256: 8343d672deff66ad4c960516ff415f95f9cd8f2a712e1a64f427e259d468ae4d
Pointer size: 132 Bytes
Size of remote file: 1.04 MB

Git LFS Details

SHA256: 8dd37db536126ed0466799359fd93d67780fc344a66be33ed6c9ecb29c4e5936
Pointer size: 131 Bytes
Size of remote file: 999 kB

assets/panel_sparsh_zh.png CHANGED Viewed

Git LFS Details

SHA256: 423fa4ebf42c5970a6ef64edb75195f8fd07161eaff490508f0f775db42ed23b
Pointer size: 132 Bytes
Size of remote file: 1.07 MB

Git LFS Details

SHA256: f94374c20716aa8cba6eb97a24fcff00f7454bf3e96f8cdd5e5585f7a84cec06
Pointer size: 132 Bytes
Size of remote file: 1.03 MB

assets/pred_vs_gt.png CHANGED Viewed

Git LFS Details

SHA256: c9f2afee78eecc8b4f15b9e45cc5f0ad52ac628eaf04ad94b1257074548415f4
Pointer size: 131 Bytes
Size of remote file: 247 kB

Git LFS Details

SHA256: 8e7b6b05646501ec2bd334ad9640966fe4d417256777b5d73a091b2428ef3202
Pointer size: 131 Bytes
Size of remote file: 245 kB

assets/sparsh_channel_fix.png ADDED Viewed

Git LFS Details

SHA256: 3369d0d839f143c5da2ad791e3b05233db3534a09a0b6c5d8e373043f774b006
Pointer size: 132 Bytes
Size of remote file: 1.21 MB

gallery.html CHANGED Viewed

@@ -52,7 +52,10 @@ padding:14px 16px;overflow-x:auto;font-size:var(--s1);max-width:100%}
 .tablewrap{overflow-x:auto;-webkit-overflow-scrolling:touch}
 .tablewrap table{min-width:640px}
 img{max-width:100%;height:auto}
-</style></head><body><div class="wrap"><nav><a href="index.html">overview</a><a href="method.html">method</a><a href="results.html">results</a><a href="gallery.html" aria-current="page">gallery</a><a href="workbench.html">3D workbench</a></nav>
 <h1>Gallery</h1>
 <figure><img src="assets/depth_validation_panel.png" alt="React contacts">
 <figcaption>React, motherboard episode 000 — the three strongest contacts.

 .tablewrap{overflow-x:auto;-webkit-overflow-scrolling:touch}
 .tablewrap table{min-width:640px}
 img{max-width:100%;height:auto}
+/* Subscripts default to a fraction of the parent and rendered at 11.7px —
+   off the --s0..--s5 scale the audit counts. Pinned to the smallest step. */
+sub,sup{font-size:var(--s0);line-height:0}
+</style></head><body><div class="wrap"><nav><a href="index.html">overview</a><a href="method.html">method</a><a href="results.html">results</a><a href="sensors.html">sensors</a><a href="gallery.html" aria-current="page">gallery</a><a href="workbench.html">3D workbench</a></nav>
 <h1>Gallery</h1>
 <figure><img src="assets/depth_validation_panel.png" alt="React contacts">
 <figcaption>React, motherboard episode 000 — the three strongest contacts.

index.html CHANGED Viewed

@@ -52,7 +52,10 @@ padding:14px 16px;overflow-x:auto;font-size:var(--s1);max-width:100%}
 .tablewrap{overflow-x:auto;-webkit-overflow-scrolling:touch}
 .tablewrap table{min-width:640px}
 img{max-width:100%;height:auto}
-</style></head><body><div class="wrap"><nav><a href="index.html" aria-current="page">overview</a><a href="method.html">method</a><a href="results.html">results</a><a href="gallery.html">gallery</a><a href="workbench.html">3D workbench</a></nav>
 <h1>Contact force from a GelSight image,<br>with no force sensor</h1>
 <p>React records tactile images and sensor pose. It has no load cell, so the
 newtons in it are estimated from the images alone — reconstructed to a depth

 .tablewrap{overflow-x:auto;-webkit-overflow-scrolling:touch}
 .tablewrap table{min-width:640px}
 img{max-width:100%;height:auto}
+/* Subscripts default to a fraction of the parent and rendered at 11.7px —
+   off the --s0..--s5 scale the audit counts. Pinned to the smallest step. */
+sub,sup{font-size:var(--s0);line-height:0}
+</style></head><body><div class="wrap"><nav><a href="index.html" aria-current="page">overview</a><a href="method.html">method</a><a href="results.html">results</a><a href="sensors.html">sensors</a><a href="gallery.html">gallery</a><a href="workbench.html">3D workbench</a></nav>
 <h1>Contact force from a GelSight image,<br>with no force sensor</h1>
 <p>React records tactile images and sensor pose. It has no load cell, so the
 newtons in it are estimated from the images alone — reconstructed to a depth

method.html CHANGED Viewed

@@ -52,7 +52,10 @@ padding:14px 16px;overflow-x:auto;font-size:var(--s1);max-width:100%}
 .tablewrap{overflow-x:auto;-webkit-overflow-scrolling:touch}
 .tablewrap table{min-width:640px}
 img{max-width:100%;height:auto}
-</style></head><body><div class="wrap"><nav><a href="index.html">overview</a><a href="method.html" aria-current="page">method</a><a href="results.html">results</a><a href="gallery.html">gallery</a><a href="workbench.html">3D workbench</a></nav>
 <h1>Method</h1>
 <pre class="card">frame − reference   →  dI, signed RGB difference
 dI                  →  surface gradient        ← the only step that needs a choice
@@ -80,7 +83,7 @@ panels below and by physical checks that need no labels.</p>
 <tr><td>cnc_mini_26</td><td>120</td><td>0.016</td><td>0.013</td><td>2.71</td><td>14%</td><td>90%</td><td>+0.796</td></tr>
 <tr><td>cnc</td><td>120</td><td>0.015</td><td>0.014</td><td>1.71</td><td>1%</td><td>68%</td><td>+0.829</td></tr>
 <tr><td>feats</td><td>120</td><td>0.052</td><td>0.046</td><td>1.08</td><td>0%</td><td>99%</td><td>+0.688</td></tr>
-<tr><td>sparsh</td><td>90</td><td>0.038</td><td>0.031</td><td>0.53</td><td>0%</td><td>59%</td><td style='color:var(--bad)'>-0.082</td></tr>
 <tr><td>faf</td><td>119</td><td>0.044</td><td>0.035</td><td>2.14</td><td>10%</td><td>96%</td><td>+0.777</td></tr></tbody></table></div>
 <p class="dim">Leak is mean |depth| off-contact over peak — zero for a coherent
@@ -91,12 +94,8 @@ reconstruction can know. The last column is the two reconstructions agreeing
 with each other, which is evidence neither invents the shape — not that either
 is right.</p>
-<p><b>Sparsh is the row that matters.</b> −0.082 means the two disagree
-entirely there, and the frames show why: both render a round sphere press
-wrongly, in orthogonal directions. The LED azimuths are per-sensor wiring, and
-these are ours — on Sparsh's own spheres a sphere comes out 3.3× elongated
-under our value and 1.5× under the one measured from its data. Its force ρ is
-0.93 either way, which is the whole reason this stage is scored separately.</p>
 <figure><img src="assets/panel_cnc_mini_26.png" alt="cnc_mini_26 — GelSight Mini, CNC presses, 0–20 N"><figcaption>cnc_mini_26 — GelSight Mini, CNC presses, 0–20 N · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.</figcaption></figure><figure><img src="assets/panel_fota_cnc.png" alt="FoTa cnc_Mini — GelSight Mini, CNC probes"><figcaption>FoTa cnc_Mini — GelSight Mini, CNC probes · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.</figcaption></figure><figure><img src="assets/panel_feats.png" alt="FEATS — marker gel, FEA force labels"><figcaption>FEATS — marker gel, FEA force labels · MARKER · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.</figcaption></figure><figure><img src="assets/panel_sparsh.png" alt="Sparsh / Meta — 10 gel pads"><figcaption>Sparsh / Meta — 10 gel pads · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.</figcaption></figure><figure><img src="assets/panel_faf.png" alt="FeelAnyForce — 42 captures"><figcaption>FeelAnyForce — 42 captures · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.</figcaption></figure>

 .tablewrap{overflow-x:auto;-webkit-overflow-scrolling:touch}
 .tablewrap table{min-width:640px}
 img{max-width:100%;height:auto}
+/* Subscripts default to a fraction of the parent and rendered at 11.7px —
+   off the --s0..--s5 scale the audit counts. Pinned to the smallest step. */
+sub,sup{font-size:var(--s0);line-height:0}
+</style></head><body><div class="wrap"><nav><a href="index.html">overview</a><a href="method.html" aria-current="page">method</a><a href="results.html">results</a><a href="sensors.html">sensors</a><a href="gallery.html">gallery</a><a href="workbench.html">3D workbench</a></nav>
 <h1>Method</h1>
 <pre class="card">frame − reference   →  dI, signed RGB difference
 dI                  →  surface gradient        ← the only step that needs a choice
 <tr><td>cnc_mini_26</td><td>120</td><td>0.016</td><td>0.013</td><td>2.71</td><td>14%</td><td>90%</td><td>+0.796</td></tr>
 <tr><td>cnc</td><td>120</td><td>0.015</td><td>0.014</td><td>1.71</td><td>1%</td><td>68%</td><td>+0.829</td></tr>
 <tr><td>feats</td><td>120</td><td>0.052</td><td>0.046</td><td>1.08</td><td>0%</td><td>99%</td><td>+0.688</td></tr>
+<tr><td>sparsh</td><td>90</td><td>0.016</td><td>0.012</td><td>0.93</td><td>0%</td><td>59%</td><td>+0.895</td></tr>
 <tr><td>faf</td><td>119</td><td>0.044</td><td>0.035</td><td>2.14</td><td>10%</td><td>96%</td><td>+0.777</td></tr></tbody></table></div>
 <p class="dim">Leak is mean |depth| off-contact over peak — zero for a coherent
 with each other, which is evidence neither invents the shape — not that either
 is right.</p>
+<p class="dim">One row needed a fix before it could be read at all — see
+<a href="sensors.html">sensors</a>.</p>
 <figure><img src="assets/panel_cnc_mini_26.png" alt="cnc_mini_26 — GelSight Mini, CNC presses, 0–20 N"><figcaption>cnc_mini_26 — GelSight Mini, CNC presses, 0–20 N · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.</figcaption></figure><figure><img src="assets/panel_fota_cnc.png" alt="FoTa cnc_Mini — GelSight Mini, CNC probes"><figcaption>FoTa cnc_Mini — GelSight Mini, CNC probes · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.</figcaption></figure><figure><img src="assets/panel_feats.png" alt="FEATS — marker gel, FEA force labels"><figcaption>FEATS — marker gel, FEA force labels · MARKER · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.</figcaption></figure><figure><img src="assets/panel_sparsh.png" alt="Sparsh / Meta — 10 gel pads"><figcaption>Sparsh / Meta — 10 gel pads · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.</figcaption></figure><figure><img src="assets/panel_faf.png" alt="FeelAnyForce — 42 captures"><figcaption>FeelAnyForce — 42 captures · markerless · 8 samples. Columns 5 and 6 are the same surface from each reconstruction; the calibration-free one is drawn with relative height because its scale is not recovered.</figcaption></figure>

results.html CHANGED Viewed

@@ -52,7 +52,10 @@ padding:14px 16px;overflow-x:auto;font-size:var(--s1);max-width:100%}
 .tablewrap{overflow-x:auto;-webkit-overflow-scrolling:touch}
 .tablewrap table{min-width:640px}
 img{max-width:100%;height:auto}
-</style></head><body><div class="wrap"><nav><a href="index.html">overview</a><a href="method.html">method</a><a href="results.html" aria-current="page">results</a><a href="gallery.html">gallery</a><a href="workbench.html">3D workbench</a></nav>
 <h1>Results</h1>
 <p>One protocol everywhere: half the frames in each group fit a 5-feature
 least squares with isotonic calibration, half are scored; pooled Spearman ρ,

 .tablewrap{overflow-x:auto;-webkit-overflow-scrolling:touch}
 .tablewrap table{min-width:640px}
 img{max-width:100%;height:auto}
+/* Subscripts default to a fraction of the parent and rendered at 11.7px —
+   off the --s0..--s5 scale the audit counts. Pinned to the smallest step. */
+sub,sup{font-size:var(--s0);line-height:0}
+</style></head><body><div class="wrap"><nav><a href="index.html">overview</a><a href="method.html">method</a><a href="results.html" aria-current="page">results</a><a href="sensors.html">sensors</a><a href="gallery.html">gallery</a><a href="workbench.html">3D workbench</a></nav>
 <h1>Results</h1>
 <p>One protocol everywhere: half the frames in each group fit a 5-feature
 least squares with isotonic calibration, half are scored; pooled Spearman ρ,

sensors.html ADDED Viewed

	@@ -0,0 +1,101 @@

+<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Sensors</title><style>
+:root{--bg:#0b1020;--fg:#e8eefb;--dim:#8ea0c2;--line:#1e2a45;--card:#111a2e;
+--accent:#ffc46b;--ok:#7be0a0;--bad:#ff8f7a;
+--s0:12px;--s1:14px;--s2:16px;--s3:20px;--s4:28px;--s5:40px}
+*{box-sizing:border-box}
+body{margin:0;background:var(--bg);color:var(--fg);font-size:var(--s2);
+font-family:'IBM Plex Sans',system-ui,sans-serif;line-height:1.6}
+.wrap{max-width:1100px;margin:0 auto;padding:0 20px 72px}
+a{color:var(--accent)}
+h1{font-size:var(--s5);line-height:1.15;margin:40px 0 8px;font-weight:650}
+h2{font-size:var(--s4);margin:44px 0 10px;font-weight:600}
+h3{font-size:var(--s3);margin:28px 0 6px;font-weight:600}
+p{margin:10px 0;max-width:70ch}
+.dim{color:var(--dim);font-size:var(--s1)}
+nav{display:flex;gap:8px;flex-wrap:wrap;margin:18px 0 4px}
+nav a{display:inline-block;padding:10px 16px;min-height:44px;line-height:24px;
+border:1px solid var(--line);border-radius:999px;text-decoration:none;
+background:var(--card)}
+nav a[aria-current]{border-color:var(--accent)}
+.cards{display:grid;grid-template-columns:repeat(auto-fit,minmax(200px,1fr));
+gap:12px;margin:20px 0}
+.card{background:var(--card);border:1px solid var(--line);border-radius:12px;
+padding:16px}
+.card b{display:block;font-size:var(--s4);color:var(--accent);font-weight:650}
+.card span{color:var(--dim);font-size:var(--s1)}
+figure{margin:24px 0}
+figure img{width:100%;border-radius:10px;border:1px solid var(--line);
+background:#fff}
+figcaption{color:var(--dim);font-size:var(--s1);margin-top:8px;max-width:80ch}
+table{border-collapse:collapse;width:100%;margin:14px 0;font-size:var(--s1)}
+th,td{border-bottom:1px solid var(--line);padding:9px 10px;text-align:right}
+th:first-child,td:first-child{text-align:left}
+th{color:var(--dim);font-weight:500}
+td b{color:var(--ok)}
+details{background:var(--card);border:1px solid var(--line);border-radius:10px;
+padding:12px 16px;margin:16px 0}
+summary{cursor:pointer;color:var(--dim);font-size:var(--s1);min-height:44px;
+display:flex;align-items:center}
+code{background:#0d1526;padding:2px 6px;border-radius:5px;font-size:var(--s1)}
+/* Inline links in prose measured 47x16 and 54x16 at 375 px — a tap target,
+   because a finger does not know it is "only prose". Padding alone would
+   break the line box, so the height comes from an inline-block with the
+   line-height carrying it. */
+p a,figcaption a{display:inline-block;min-height:44px;line-height:44px;
+padding:0 2px}
+/* The <pre> pipeline diagram and the results table are the two things wider
+   than a phone. Let each scroll inside its own box rather than pushing the
+   document sideways — a horizontally scrolling PAGE hides content with no
+   affordance, a scrolling code block is a known idiom. */
+pre{background:var(--card);border:1px solid var(--line);border-radius:10px;
+padding:14px 16px;overflow-x:auto;font-size:var(--s1);max-width:100%}
+.tablewrap{overflow-x:auto;-webkit-overflow-scrolling:touch}
+.tablewrap table{min-width:640px}
+img{max-width:100%;height:auto}
+/* Subscripts default to a fraction of the parent and rendered at 11.7px —
+   off the --s0..--s5 scale the audit counts. Pinned to the smallest step. */
+sub,sup{font-size:var(--s0);line-height:0}
+</style></head><body><div class="wrap"><nav><a href="index.html">overview</a><a href="method.html">method</a><a href="results.html">results</a><a href="sensors.html" aria-current="page">sensors</a><a href="gallery.html">gallery</a><a href="workbench.html">3D workbench</a></nav>
+<h1>A second sensor is not a second dataset</h1>
+<p>The calibration-free solve reads each <b>colour channel</b> as one LED
+direction — <code>dI<sub>k</sub>/I<sub>ref</sub> ≈ g<sub>x</sub>cos θ<sub>k</sub>
++ g<sub>y</sub>sin θ<sub>k</sub></code>. So a channel permutation does not tint
+an image, it <b>rotates the whole recovered gradient field</b>.</p>
+<p>Sparsh's frames reach us with R and B exchanged. It was found by scoring the
+depth stage on its own: its two reconstructions of the same contact correlated
+at <b>−0.13</b>, against 0.69–0.83 on every other dataset, and both rendered a
+round sphere press wrongly in orthogonal directions.</p>
+<h2>Measured, not searched</h2>
+<p>For a sphere the surface gradient points radially outward, so the dipole
+direction of each channel's difference <em>is</em> that channel's LED azimuth.
+Thirty sphere presses per sensor:</p>
+<div class="tablewrap"><table><thead><tr><th></th><th>rest hue</th>
+<th>R</th><th>G</th><th>B</th></tr></thead><tbody>
+<tr><td>our Mini</td><td>172.1°</td><td>259.2°</td><td>5.1°</td><td>51.1°</td></tr>
+<tr><td>Sparsh, as-is</td><td>42.1°</td><td>75.7°</td><td>4.3°</td><td>259.8°</td></tr>
+<tr><td>Sparsh, R↔B</td><td>197.9°</td><td><b>259.8°</b></td><td><b>4.3°</b></td><td>75.7°</td></tr>
+</tbody></table></div>
+<p class="dim">Swapped, R and G land within 1° of ours. A different gel tint
+cannot align LED azimuths; a channel-order difference does exactly that.</p>
+<figure><img src="assets/sparsh_channel_fix.png" alt="Sparsh channel fix">
+<figcaption>The same frames before and after, at every stage. Sphere axis ratio
+3.62&nbsp;→&nbsp;1.53 (1.0 is a circle), agreement with the LUT
+−0.125&nbsp;→&nbsp;+0.920, flat-gel leak 0.0272&nbsp;→&nbsp;0.0071. Across the
+dataset, agreement −0.082&nbsp;→&nbsp;+0.895.
+</figcaption></figure>
+<h2>Why the force numbers never showed it</h2>
+<p>Force ρ on Sparsh was <b>0.909 before</b> the fix and 0.894 after; the LUT
+gained 0.822&nbsp;→&nbsp;0.894. Contact size tracks force whatever the shape
+does, so a ρ cannot see a geometry this wrong. That is the argument for scoring
+image→depth separately from depth→newtons.</p>
+<p class="dim">A correction of ours: a six-permutation search first put Sparsh's
+best at (90,330,210) and that was written up as “LED wiring differs per
+sensor”. It is the same fact said uselessly — (90,330,210) is (210,330,90) with
+R and B exchanged. The azimuths never differed.</p>
+</div></body></html>

workbench.html CHANGED Viewed

@@ -52,7 +52,10 @@ padding:14px 16px;overflow-x:auto;font-size:var(--s1);max-width:100%}
 .tablewrap{overflow-x:auto;-webkit-overflow-scrolling:touch}
 .tablewrap table{min-width:640px}
 img{max-width:100%;height:auto}
-</style></head><body><div class="wrap"><nav><a href="index.html">overview</a><a href="method.html">method</a><a href="results.html">results</a><a href="gallery.html">gallery</a><a href="workbench.html" aria-current="page">3D workbench</a></nav>
 <h1>3D workbench</h1>
 <p>Every stage of the reconstruction on one frame, with its knobs: difference
 image, gradient field, integrated depth, surface.</p>

 .tablewrap{overflow-x:auto;-webkit-overflow-scrolling:touch}
 .tablewrap table{min-width:640px}
 img{max-width:100%;height:auto}
+/* Subscripts default to a fraction of the parent and rendered at 11.7px —
+   off the --s0..--s5 scale the audit counts. Pinned to the smallest step. */
+sub,sup{font-size:var(--s0);line-height:0}
+</style></head><body><div class="wrap"><nav><a href="index.html">overview</a><a href="method.html">method</a><a href="results.html">results</a><a href="sensors.html">sensors</a><a href="gallery.html">gallery</a><a href="workbench.html" aria-current="page">3D workbench</a></nav>
 <h1>3D workbench</h1>
 <p>Every stage of the reconstruction on one frame, with its knobs: difference
 image, gradient field, integrated depth, surface.</p>