Spaces:
Running
Running
| <html class="bg-white dark:bg-zinc-900" lang="en" data-theme="light"> <head><meta charset="UTF-8"><meta name="description" content="TacForcing streams robot action generation and incorporates fresh tactile feedback during execution for contact-rich manipulation."><meta name="viewport" content="width=device-width"><meta name="generator" content="Astro v6.3.5"><link rel="icon" type="image/png" href="/favicon.png"><meta property="og:title" content="TacForcing: Streaming Action Generation with Execution-Time Tactile Feedback"><meta property="og:description" content="TacForcing streams robot action generation and incorporates fresh tactile feedback during execution for contact-rich manipulation."><meta property="og:type" content="website"><meta property="og:image" content="/og.png"><title>TacForcing: Streaming Action Generation with Execution-Time Tactile Feedback</title><link rel="stylesheet" href="/_astro/index.DEeHd8u9.css"></head> <body class="prose prose-lg prose-zinc dark:prose-invert prose-a:text-blue-700 prose-a:hover:text-blue-900 prose-a:dark:text-blue-200 prose-a:dark:hover:text-blue-100 prose-a:underline prose-a:decoration-dashed prose-a:underline-offset-4 prose-a:hover:decoration-solid prose-a:font-normal prose-code:bg-zinc-200 prose-code:dark:bg-zinc-800 prose-code:text-zinc-800 prose-code:dark:text-zinc-200 prose-code:px-1.5 prose-code:py-0.5 prose-code:rounded prose-code:font-medium prose-code:before:content-none prose-code:after:content-none max-w-none pt-16 pb-6"> <main> <header class="full-bleed relative mb-12 -mt-16 pb-28 pt-36" data-astro-cid-3ef6ksr2> <div class="cover absolute inset-0 z-0 overflow-hidden" data-astro-cid-3ef6ksr2> <video class="aspect-video h-auto w-full rounded-lg" src="/_astro/transfer-blue-side.DkfxKWSZ.mp4" loop playsinline muted autoplay></video> <div class="absolute inset-0 bg-[linear-gradient(180deg,rgba(7,15,20,.48),rgba(7,15,20,.74))]" data-astro-cid-3ef6ksr2></div> </div> <div class="not-prose absolute top-5 left-5 z-20 flex max-w-[calc(100%-2.5rem)] items-center gap-3 rounded-2xl bg-white px-4 py-3 shadow-lg text-shadow-none sm:top-7 sm:left-8" data-astro-cid-3ef6ksr2> <img class="m-0 block bg-white object-contain h-12 w-12 sm:h-14 sm:w-14" src="/_astro/sjtu-logo.BRg-E74n.png" alt="Shanghai Jiao Tong University" loading="eager" decoding="async" data-astro-cid-3ef6ksr2> <span class="h-10 w-px bg-zinc-200" data-astro-cid-3ef6ksr2></span><img class="m-0 block bg-white object-contain h-12 w-auto max-w-[11rem] sm:h-14 sm:max-w-[13rem]" src="/_astro/deng-lab-logo.BP1POFwN.png" alt="DENG Lab" loading="eager" decoding="async" data-astro-cid-3ef6ksr2> </div> <div class="relative z-10 mx-auto flex max-w-[50rem] flex-col items-center px-6 text-center prose-headings:text-white gap-10 text-white [text-shadow:0_0_8px_rgba(0,0,0,0.6)]" data-astro-cid-3ef6ksr2> <h1 class="my-0 text-4xl leading-[1.08] font-semibold tracking-[-0.035em] sm:text-5xl" data-astro-cid-3ef6ksr2> TacForcing: Streaming Action Generation with Execution-Time Tactile Feedback </h1> <div class="flex flex-row flex-wrap justify-center gap-x-6 gap-y-3" data-astro-cid-3ef6ksr2> <div class="flex flex-col items-center text-center" data-astro-cid-3ef6ksr2> <div class="flex flex-row text-[1.05rem] font-medium" data-astro-cid-3ef6ksr2> Jianbo Zhou </div> <div class="text-xs opacity-80" data-astro-cid-3ef6ksr2>SJTU · SCUT</div> </div><div class="flex flex-col items-center text-center" data-astro-cid-3ef6ksr2> <div class="flex flex-row text-[1.05rem] font-medium" data-astro-cid-3ef6ksr2> Boyuan Zhao </div> <div class="text-xs opacity-80" data-astro-cid-3ef6ksr2>ECUST</div> </div><div class="flex flex-col items-center text-center" data-astro-cid-3ef6ksr2> <div class="flex flex-row text-[1.05rem] font-medium" data-astro-cid-3ef6ksr2> Yuzheng Zhang </div> <div class="text-xs opacity-80" data-astro-cid-3ef6ksr2>ECNU</div> </div><div class="flex flex-col items-center text-center" data-astro-cid-3ef6ksr2> <div class="flex flex-row text-[1.05rem] font-medium" data-astro-cid-3ef6ksr2> Yiyang Chen </div> <div class="text-xs opacity-80" data-astro-cid-3ef6ksr2>SJTU</div> </div><div class="flex flex-col items-center text-center" data-astro-cid-3ef6ksr2> <div class="flex flex-row text-[1.05rem] font-medium" data-astro-cid-3ef6ksr2> Wenxin Chen </div> <div class="text-xs opacity-80" data-astro-cid-3ef6ksr2>SJTU</div> </div><div class="flex flex-col items-center text-center" data-astro-cid-3ef6ksr2> <div class="flex flex-row text-[1.05rem] font-medium" data-astro-cid-3ef6ksr2> Qiuyue Li </div> <div class="text-xs opacity-80" data-astro-cid-3ef6ksr2>PolyU</div> </div><div class="flex flex-col items-center text-center" data-astro-cid-3ef6ksr2> <div class="flex flex-row text-[1.05rem] font-medium" data-astro-cid-3ef6ksr2> Xiangyang Gu </div> <div class="text-xs opacity-80" data-astro-cid-3ef6ksr2>CCUT</div> </div><div class="flex flex-col items-center text-center" data-astro-cid-3ef6ksr2> <div class="flex flex-row text-[1.05rem] font-medium" data-astro-cid-3ef6ksr2> Yuhan Cao </div> <div class="text-xs opacity-80" data-astro-cid-3ef6ksr2>UESTC</div> </div><div class="flex flex-col items-center text-center" data-astro-cid-3ef6ksr2> <div class="flex flex-row text-[1.05rem] font-medium" data-astro-cid-3ef6ksr2> Xiao Xia </div> <div class="text-xs opacity-80" data-astro-cid-3ef6ksr2>SJTU</div> </div><div class="flex flex-col items-center text-center" data-astro-cid-3ef6ksr2> <div class="flex flex-row text-[1.05rem] font-medium" data-astro-cid-3ef6ksr2> Yanzhe Hu </div> <div class="text-xs opacity-80" data-astro-cid-3ef6ksr2>SJTU · HUST</div> </div><div class="flex flex-col items-center text-center" data-astro-cid-3ef6ksr2> <div class="flex flex-row text-[1.05rem] font-medium" data-astro-cid-3ef6ksr2> Zhijie Deng <sup class="text-sm" data-astro-cid-3ef6ksr2> † </sup> </div> <div class="text-xs opacity-80" data-astro-cid-3ef6ksr2>SJTU</div> </div> </div> <div class="rounded-full border border-white/35 bg-black/20 px-4 py-1 text-sm font-semibold tracking-wide backdrop-blur-sm" data-astro-cid-3ef6ksr2> Preprint · 2026 </div> <div class="text-center text-xs opacity-85" data-astro-cid-3ef6ksr2> <span data-astro-cid-3ef6ksr2> <sup data-astro-cid-3ef6ksr2>†</sup> Corresponding author </span> </div> <div class="not-prose flex flex-row flex-wrap justify-center gap-2" data-astro-cid-3ef6ksr2> <a href="https://arxiv.org/pdf/2608.25798" class="flex flex-row items-center gap-2 rounded-full px-5 py-2 text-lg hover:no-underline bg-white text-zinc-800 shadow-[0_0_8px_rgba(0,0,0,0.6)] text-shadow-none hover:bg-zinc-200" data-astro-cid-3ef6ksr2> <svg width="0.88em" height="1em" data-astro-cid-3ef6ksr2="true" data-icon="academicons:arxiv"> <symbol id="ai:academicons:arxiv" viewBox="0 0 448 512"><path fill="currentColor" d="M62.258 8.006a22.22 22.22 0 0 0-20.929 13.448c-3.404 8.169-.96 13.898 6.506 24.59c10.935 16.09 122.178 149.673 122.178 149.673l-24.619 23.038c-20.74 20.735-21.632 48.566-2.34 67.852l28.663 27.3l-79.976 98.235c-6.21 6.614-10.053 18.221-6.585 26.552a22.7 22.7 0 0 0 21.21 14.06a20.23 20.23 0 0 0 15.249-7.536l95.122-88.437L363.33 496.39a27.14 27.14 0 0 0 18.418 7.61a25.3 25.3 0 0 0 7.335-1.108a27.66 27.66 0 0 0 18.4-18.99a25.6 25.6 0 0 0-6.481-23.69L272.219 305.195l23.062-21.443c17.198-15.504 17.29-42.455.197-58.076l-25.257-24.228L357.417 98.46l.115-.133l.103-.14c7.793-10.123 11.52-17.92 7.502-27.806a36.17 36.17 0 0 0-23.647-18.37a24 24 0 0 0-3.166-.212l-.006.018a28.52 28.52 0 0 0-18.252 8.123l-.203.166l-.19.173L218.6 151.925L79.261 18.253S70.995 8.213 62.258 8.006m276.06 51.214q1.115.004 2.22.148a29.3 29.3 0 0 1 17.719 13.81c2.246 5.523 1.554 10.01-6.506 20.484L264.861 196.3l-40.882-39.22l100.68-91.304a21.77 21.77 0 0 1 13.66-6.536zM175.077 201.127L395.19 464.872c4.32 5.408 7.02 10.818 5.18 16.914a20.25 20.25 0 0 1-13.463 14.037a17.6 17.6 0 0 1-5.17.784a19.8 19.8 0 0 1-13.293-5.56l-220.15-209.694c-17.317-17.316-14.698-40.33 2.158-57.186z"/></symbol><use href="#ai:academicons:arxiv"></use> </svg> <span data-astro-cid-3ef6ksr2>Paper</span> </a><span class="flex cursor-default flex-row items-center gap-2 rounded-full px-5 py-2 text-lg border border-white/35 bg-white/20 text-white backdrop-blur-sm text-shadow-none" aria-disabled="true" data-astro-cid-3ef6ksr2> <svg width="1em" height="1em" data-astro-cid-3ef6ksr2="true" data-icon="ri:github-line"> <symbol id="ai:ri:github-line" viewBox="0 0 24 24"><path fill="currentColor" d="M5.884 18.653c-.3-.2-.558-.455-.86-.816a51 51 0 0 1-.466-.579c-.463-.575-.755-.841-1.056-.95a1 1 0 1 1 .675-1.882c.752.27 1.261.735 1.947 1.588c-.094-.117.34.427.433.539c.19.227.33.365.44.438c.204.137.588.196 1.15.14c.024-.382.094-.753.202-1.095c-2.968-.726-4.648-2.64-4.648-6.396c0-1.24.37-2.356 1.058-3.292c-.218-.894-.185-1.975.302-3.192a1 1 0 0 1 .63-.582c.081-.024.127-.035.208-.047c.803-.124 1.937.17 3.415 1.096a11.7 11.7 0 0 1 2.687-.308c.912 0 1.819.104 2.684.308c1.477-.933 2.614-1.227 3.422-1.096q.128.02.218.05a1 1 0 0 1 .616.58c.487 1.216.52 2.296.302 3.19c.691.936 1.058 2.045 1.058 3.293c0 3.757-1.674 5.665-4.642 6.392c.125.415.19.878.19 1.38c0 .665-.002 1.299-.007 2.01c0 .19-.002.394-.005.706a1 1 0 0 1-.018 1.958c-1.14.227-1.984-.532-1.984-1.525l.002-.447l.005-.705c.005-.707.008-1.337.008-1.997c0-.697-.184-1.152-.426-1.361c-.661-.57-.326-1.654.541-1.751c2.966-.333 4.336-1.482 4.336-4.66c0-.955-.312-1.744-.913-2.404A1 1 0 0 1 17.2 6.19c.166-.414.236-.957.095-1.614l-.01.003c-.491.139-1.11.44-1.858.949a1 1 0 0 1-.833.135a9.6 9.6 0 0 0-2.592-.349c-.89 0-1.772.118-2.592.35a1 1 0 0 1-.829-.134c-.753-.507-1.374-.807-1.87-.947c-.143.653-.072 1.194.093 1.607a1 1 0 0 1-.189 1.045c-.597.655-.913 1.458-.913 2.404c0 3.172 1.371 4.328 4.322 4.66c.865.097 1.202 1.177.545 1.748c-.193.168-.43.732-.43 1.364v3.15c0 .985-.834 1.725-1.96 1.528a1 1 0 0 1-.04-1.962v-.99c-.91.061-1.661-.088-2.254-.485"/></symbol><use href="#ai:ri:github-line"></use> </svg> <span data-astro-cid-3ef6ksr2>Code · Coming soon</span> </span><a href="#bibtex" class="flex flex-row items-center gap-2 rounded-full px-5 py-2 text-lg hover:no-underline bg-white text-zinc-800 shadow-[0_0_8px_rgba(0,0,0,0.6)] text-shadow-none hover:bg-zinc-200" data-astro-cid-3ef6ksr2> <svg width="1em" height="1em" data-astro-cid-3ef6ksr2="true" data-icon="ri:double-quotes-l"> <symbol id="ai:ri:double-quotes-l" viewBox="0 0 24 24"><path fill="currentColor" d="M4.583 17.321C3.553 16.227 3 15 3 13.011c0-3.5 2.457-6.637 6.03-8.188l.893 1.378c-3.335 1.804-3.987 4.145-4.247 5.621c.537-.278 1.24-.375 1.929-.311c1.804.167 3.226 1.648 3.226 3.489a3.5 3.5 0 0 1-3.5 3.5a3.87 3.87 0 0 1-2.748-1.179m10 0C13.553 16.227 13 15 13 13.011c0-3.5 2.457-6.637 6.03-8.188l.893 1.378c-3.335 1.804-3.987 4.145-4.247 5.621c.537-.278 1.24-.375 1.929-.311c1.804.167 3.226 1.648 3.226 3.489a3.5 3.5 0 0 1-3.5 3.5a3.87 3.87 0 0 1-2.748-1.179"/></symbol><use href="#ai:ri:double-quotes-l"></use> </svg> <span data-astro-cid-3ef6ksr2>BibTeX</span> </a> </div> </div> </header> | |
| <div class="full-bleed bg-zinc-200 py-8 dark:bg-zinc-800"> <div class="mx-[var(--actual-inline-margin)] [&>:first-child]:mt-0 [&>:last-child]:mb-0"> <h2 id="abstract">Abstract</h2><p>Contact-rich manipulation requires adapting to contact states that can evolve substantially within an action horizon. However, chunk-based vision-language-action models predict complete action chunks from observations collected before execution, leaving tactile conditioning stale during execution. Existing tactile-reactive approaches typically rely on separate high-frequency controllers, which increase both architectural and training complexity. In this paper, we introduce <strong>TacForcing</strong>, a streaming action-generation framework that effectively incorporates execution-time tactile feedback. Instead of employing a separate reactive controller, TacForcing replaces the standard action expert with a streaming action expert to generate actions conditioned on the evolving tactile observations acquired during execution. TacForcing also introduces Execution-Aware Tactile Attention (EATA), which restricts tactile conditioning to actions nearing execution, thereby reducing the temporal mismatch between tactile acquisition and action execution. Across six simulated UniVTAC tasks and three real-world contact-rich manipulation tasks, TacForcing achieves average success rates of 65% and 69%, respectively, outperforming strong baselines in both settings.</p> </div> </div> | |
| <div class="not-prose my-10 grid grid-cols-2 gap-3 mx-auto max-w-2xl"> <div class="stat-card relative overflow-hidden rounded-2xl border border-zinc-200 bg-white p-5 shadow-sm dark:border-zinc-700 dark:bg-zinc-900"> <div class="mb-3 h-1 w-9 rounded-full bg-[var(--tac-orange)]"></div> <div class="text-4xl font-semibold tracking-tight text-zinc-900 dark:text-white"> 65% </div> <div class="mt-2 text-sm font-semibold text-zinc-800 dark:text-zinc-100"> Simulation success </div> <div class="mt-1 text-xs leading-relaxed text-zinc-500 dark:text-zinc-400"> Average over six UniVTAC tasks </div> <div class="absolute -top-5 -right-5 h-16 w-16 rounded-full border border-[var(--tac-orange-soft)]" aria-hidden="true"></div> </div><div class="stat-card relative overflow-hidden rounded-2xl border border-zinc-200 bg-white p-5 shadow-sm dark:border-zinc-700 dark:bg-zinc-900"> <div class="mb-3 h-1 w-9 rounded-full bg-[var(--tac-orange)]"></div> <div class="text-4xl font-semibold tracking-tight text-zinc-900 dark:text-white"> 69% </div> <div class="mt-2 text-sm font-semibold text-zinc-800 dark:text-zinc-100"> Real-world success </div> <div class="mt-1 text-xs leading-relaxed text-zinc-500 dark:text-zinc-400"> Average over three contact-rich tasks </div> <div class="absolute -top-5 -right-5 h-16 w-16 rounded-full border border-[var(--tac-orange-soft)]" aria-hidden="true"></div> </div> </div> | |
| <div class="mx-[calc(max(var(--minimum-inline-margin),(100cqw-100rem)/2)-var(--actual-inline-margin))]"> <figure> <div class="flex w-full justify-center **:my-0"> <picture> <source srcset="/_astro/tacforcing-framework.pdf_ZYeBSm.avif 640w, /_astro/tacforcing-framework.pdf_Z23zuYc.avif 750w, /_astro/tacforcing-framework.pdf_1P7g9.avif 828w, /_astro/tacforcing-framework.pdf_Z1HVp1Q.avif 1080w, /_astro/tacforcing-framework.pdf_1GnQJN.avif 1280w, /_astro/tacforcing-framework.pdf_2wwDPl.avif 1668w, /_astro/tacforcing-framework.pdf_Z2vurWc.avif 1752w" type="image/avif"> <img src="/_astro/tacforcing-framework.pdf_zM9BE.png" srcset="/_astro/tacforcing-framework.pdf_1eVVwD.png 640w, /_astro/tacforcing-framework.pdf_aB3qN.png 750w, /_astro/tacforcing-framework.pdf_2g1FG9.png 828w, /_astro/tacforcing-framework.pdf_1nlcx0.png 1080w, /_astro/tacforcing-framework.pdf_ZhvEuh.png 1280w, /_astro/tacforcing-framework.pdf_xC7Ag.png 1668w, /_astro/tacforcing-framework.pdf_zM9BE.png 1752w" alt="TacForcing framework: a VLM and Streaming Action Expert connected to an execution-time tactile feedback loop" loading="eager" decoding="sync" fetchpriority="high" sizes="(min-width: 1752px) 1752px, 100vw" data-astro-image="constrained" data-astro-image-fit="contain" data-astro-image-pos="center" width="1752" height="702" class="rounded-lg max-h-[calc(100svh-3rem)] max-w-full w-min mx-auto "> </picture> </div> <figcaption class="text-center"> TacForcing progressively completes, executes, and tactilely refines action blocks inside one streaming generation process. </figcaption> </figure> </div> | |
| <h2 id="why-execution-time-touch">Why execution-time touch?</h2> | |
| <p>Vision-language-action policies commonly predict an entire action chunk from observations captured before execution. That works well when the scene changes slowly, but contact states can shift sharply within the same horizon: grasp force changes, slip begins, or a fingertip meets an occluded surface while the visual scene remains nearly unchanged.</p> | |
| <p>Conditioning the complete chunk on one initial tactile frame therefore creates a growing <strong>temporal mismatch</strong>. TacForcing addresses that mismatch without adding a separate high-frequency reactive controller.</p> | |
| <figure> <div class="flex w-full justify-center **:my-0"> <picture> <source srcset="/_astro/tactile-observations.pdf_Z1yg9Ej.avif 640w, /_astro/tactile-observations.pdf_Q3E98.avif 750w, /_astro/tactile-observations.pdf_Z1YTVRt.avif 828w, /_astro/tactile-observations.pdf_ZOurpw.avif 1080w, /_astro/tactile-observations.pdf_29VGF5.avif 1280w, /_astro/tactile-observations.pdf_IgQYV.avif 1668w, /_astro/tactile-observations.pdf_27yQFf.avif 1846w" type="image/avif"> <img src="/_astro/tactile-observations.pdf_8Ekqa.png" srcset="/_astro/tactile-observations.pdf_EUoKG.png 640w, /_astro/tactile-observations.pdf_Z1YVUeN.png 750w, /_astro/tactile-observations.pdf_egBxw.png 828w, /_astro/tactile-observations.pdf_2gMa9k.png 1080w, /_astro/tactile-observations.pdf_b2aq0.png 1280w, /_astro/tactile-observations.pdf_Z1fCEf9.png 1668w, /_astro/tactile-observations.pdf_8Ekqa.png 1846w" alt="Comparison showing visual observations changing slowly while tactile observations evolve during action execution" loading="lazy" decoding="async" sizes="(min-width: 1846px) 1846px, 100vw" data-astro-image="constrained" data-astro-image-fit="contain" data-astro-image-pos="center" width="1846" height="793" class="rounded-lg max-h-[calc(100svh-3rem)] max-w-full w-min mx-auto "> </picture> </div> <figcaption class="text-center"> Tactile representations can evolve substantially inside a single action chunk even when visual observations remain similar. </figcaption> </figure> | |
| <h2 id="streaming-generation-with-fresh-tactile-feedback">Streaming generation with fresh tactile feedback</h2> | |
| <p>TacForcing divides an action horizon of <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>H</mi><mo>=</mo><mi>K</mi><mi>B</mi></mrow><annotation encoding="application/x-tex">H=KB</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6833em"></span><span class="mord mathnormal" style="margin-right:0.0813em">H</span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em"></span></span><span class="base"><span class="strut" style="height:0.6833em"></span><span class="mord mathnormal" style="margin-right:0.0715em">K</span><span class="mord mathnormal" style="margin-right:0.0502em">B</span></span></span></span> actions into <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>K</mi></mrow><annotation encoding="application/x-tex">K</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6833em"></span><span class="mord mathnormal" style="margin-right:0.0715em">K</span></span></span></span> consecutive blocks, each containing <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>B</mi></mrow><annotation encoding="application/x-tex">B</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6833em"></span><span class="mord mathnormal" style="margin-right:0.0502em">B</span></span></span></span> actions. Instead of assigning one flow time to the entire chunk, block <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>k</mi></mrow><annotation encoding="application/x-tex">k</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em"></span><span class="mord mathnormal" style="margin-right:0.0315em">k</span></span></span></span> follows its own schedule:</p> | |
| <span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msubsup><mi>λ</mi><mi>k</mi><mrow><mo stretchy="false">(</mo><mi>n</mi><mo stretchy="false">)</mo></mrow></msubsup><mo>=</mo><mi>min</mi><mo></mo><mrow><mo fence="true">(</mo><mfrac><mi>n</mi><mrow><mi>k</mi><mi>S</mi></mrow></mfrac><mo separator="true">,</mo><mn>1</mn><mo fence="true">)</mo></mrow><mi mathvariant="normal">.</mi></mrow><annotation encoding="application/x-tex">\lambda_k^{(n)} = \min\left(\frac{n}{kS}, 1\right).</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.3461em;vertical-align:-0.3013em"></span><span class="mord"><span class="mord mathnormal">λ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.0448em"><span style="top:-2.3987em;margin-left:0em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.0315em">k</span></span></span><span style="top:-3.2198em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mathnormal mtight">n</span><span class="mclose mtight">)</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.3013em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em"></span></span><span class="base"><span class="strut" style="height:1.836em;vertical-align:-0.686em"></span><span class="mop">min</span><span class="mspace" style="margin-right:0.1667em"></span><span class="minner"><span class="mopen delimcenter" style="top:0em"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.1076em"><span style="top:-2.314em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.0315em">k</span><span class="mord mathnormal" style="margin-right:0.0576em">S</span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:0.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord mathnormal">n</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.686em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em"></span><span class="mord">1</span><span class="mclose delimcenter" style="top:0em"><span class="delimsizing size2">)</span></span></span><span class="mspace" style="margin-right:0.1667em"></span><span class="mord">.</span></span></span></span></span> | |
| <p>Near-term blocks become ready first. Once a block is executed, the latest fingertip observations are encoded and generation resumes from the retained intermediate states of every unfinished block.</p> | |
| <div class="not-prose my-10 grid gap-3 md:grid-cols-[1fr_auto_1fr_auto_1fr] md:items-stretch" data-astro-cid-jv27ffxo> <div class="method-step" data-astro-cid-jv27ffxo> <span data-astro-cid-jv27ffxo>01</span> <h3 data-astro-cid-jv27ffxo>Generate a block</h3> <p data-astro-cid-jv27ffxo> | |
| Near-term actions reach clean states first while later blocks retain their | |
| intermediate states. | |
| </p> </div> <div class="method-arrow" aria-hidden="true" data-astro-cid-jv27ffxo>→</div> <div class="method-step" data-astro-cid-jv27ffxo> <span data-astro-cid-jv27ffxo>02</span> <h3 data-astro-cid-jv27ffxo>Execute & sense</h3> <p data-astro-cid-jv27ffxo> | |
| The ready block is executed, then fresh tactile observations are encoded | |
| from the fingertips. | |
| </p> </div> <div class="method-arrow" aria-hidden="true" data-astro-cid-jv27ffxo>→</div> <div class="method-step" data-astro-cid-jv27ffxo> <span data-astro-cid-jv27ffxo>03</span> <h3 data-astro-cid-jv27ffxo>Resume with touch</h3> <p data-astro-cid-jv27ffxo> | |
| EATA routes the latest tactile tokens only to the action block scheduled | |
| to execute next. | |
| </p> </div> </div> | |
| <h3 id="execution-aware-tactile-attention">Execution-Aware Tactile Attention</h3> | |
| <p>The newest tactile observation is most relevant to the action block that will execute next; applying it to distant blocks may make those actions depend on contact information that will be stale by the time they run. EATA uses a simple visibility mask so that only the next block can directly attend to the current tactile tokens:</p> | |
| <span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msubsup><mi mathvariant="script">M</mi><mrow><mi>i</mi><mo separator="true">,</mo><mi>m</mi></mrow><mrow><mo stretchy="false">(</mo><mi>k</mi><mo stretchy="false">)</mo></mrow></msubsup><mo>=</mo><mrow><mo fence="true">{</mo><mtable rowspacing="0.36em" columnalign="left left" columnspacing="1em"><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><mrow><mn>0</mn><mo separator="true">,</mo></mrow></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><mrow><mi>b</mi><mo stretchy="false">(</mo><mi>i</mi><mo stretchy="false">)</mo><mo>=</mo><mi>k</mi><mo separator="true">,</mo></mrow></mstyle></mtd></mtr><mtr><mtd><mstyle scriptlevel="0" displaystyle="false"><mrow><mo>−</mo><mi mathvariant="normal">∞</mi><mo separator="true">,</mo></mrow></mstyle></mtd><mtd><mstyle scriptlevel="0" displaystyle="false"><mrow><mtext>otherwise</mtext><mi mathvariant="normal">.</mi></mrow></mstyle></mtd></mtr></mtable></mrow></mrow><annotation encoding="application/x-tex">\mathcal{M}_{i,m}^{(k)} = | |
| \begin{cases} | |
| 0, & b(i)=k,\\ | |
| -\infty, & \text{otherwise}. | |
| \end{cases}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.4578em;vertical-align:-0.413em"></span><span class="mord"><span class="mord mathcal">M</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.0448em"><span style="top:-2.4231em;margin-left:0em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mpunct mtight">,</span><span class="mord mathnormal mtight">m</span></span></span></span><span style="top:-3.2198em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mopen mtight">(</span><span class="mord mathnormal mtight" style="margin-right:0.0315em">k</span><span class="mclose mtight">)</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.413em"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em"></span></span><span class="base"><span class="strut" style="height:3em;vertical-align:-1.25em"></span><span class="minner"><span class="mopen delimcenter" style="top:0em"><span class="delimsizing size4">{</span></span><span class="mord"><span class="mtable"><span class="col-align-l"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.69em"><span style="top:-3.69em"><span class="pstrut" style="height:3.008em"></span><span class="mord"><span class="mord">0</span><span class="mpunct">,</span></span></span><span style="top:-2.25em"><span class="pstrut" style="height:3.008em"></span><span class="mord"><span class="mord">−</span><span class="mord">∞</span><span class="mpunct">,</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.19em"><span></span></span></span></span></span><span class="arraycolsep" style="width:1em"></span><span class="col-align-l"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.69em"><span style="top:-3.69em"><span class="pstrut" style="height:3.008em"></span><span class="mord"><span class="mord mathnormal">b</span><span class="mopen">(</span><span class="mord mathnormal">i</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em"></span><span class="mord mathnormal" style="margin-right:0.0315em">k</span><span class="mpunct">,</span></span></span><span style="top:-2.25em"><span class="pstrut" style="height:3.008em"></span><span class="mord"><span class="mord text"><span class="mord">otherwise</span></span><span class="mord">.</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.19em"><span></span></span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span></span></span></span></span> | |
| <p>The same temporal structure is used during training and inference. This keeps the model aligned with the tactile information that will actually be available at each execution stage.</p> | |
| <h2 id="real-world-demonstrations">Real-world demonstrations</h2> | |
| <p>All videos below are encoded and displayed at <strong>5× speed</strong>. Front and side views play together when both recordings are available.</p> | |
| <h3 id="transfer-liquid">Transfer Liquid</h3> | |
| <p>The robot manipulates a transparent dropper to draw liquid from a flask and dispense it into a beaker, requiring precise grasp and contact regulation despite partial visual occlusion.</p> | |
| <div class="mx-[calc(max(var(--minimum-inline-margin),(100cqw-100rem)/2)-var(--actual-inline-margin))]"> <article class="demo-card not-prose my-6 overflow-hidden rounded-2xl border border-zinc-200 bg-white shadow-sm dark:border-zinc-700 dark:bg-zinc-900" data-astro-cid-vaqhqpsm> <div class="demo-grid has-secondary" data-astro-cid-vaqhqpsm> <div class="video-frame front-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/transfer-blue-front.BZ7jwMqA.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Transfer Liquid · Blue, Front view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Front view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> <div class="video-frame side-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/transfer-blue-side.DkfxKWSZ.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Transfer Liquid · Blue, Side view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Side view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> </div> <div class="px-5 py-4" data-astro-cid-vaqhqpsm> <h3 class="m-0 text-lg font-semibold whitespace-nowrap text-zinc-900 dark:text-white" data-astro-cid-vaqhqpsm> Transfer Liquid · Blue </h3> </div> </article> <script type="module">const t=document.querySelectorAll(".demo-video");if("IntersectionObserver"in window){const s=new IntersectionObserver(e=>{e.forEach(o=>{const r=o.target;o.isIntersecting?r.play().catch(()=>{}):r.pause()})},{rootMargin:"180px 0px",threshold:.15});t.forEach(e=>s.observe(e))}</script><article class="demo-card not-prose my-6 overflow-hidden rounded-2xl border border-zinc-200 bg-white shadow-sm dark:border-zinc-700 dark:bg-zinc-900" data-astro-cid-vaqhqpsm> <div class="demo-grid has-secondary" data-astro-cid-vaqhqpsm> <div class="video-frame front-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/transfer-gold-front.Dp1jq5vF.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Transfer Liquid · Gold, Front view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Front view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> <div class="video-frame side-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/transfer-gold-side.Dz_yOqC5.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Transfer Liquid · Gold, Side view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Side view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> </div> <div class="px-5 py-4" data-astro-cid-vaqhqpsm> <h3 class="m-0 text-lg font-semibold whitespace-nowrap text-zinc-900 dark:text-white" data-astro-cid-vaqhqpsm> Transfer Liquid · Gold </h3> </div> </article> </div> | |
| <h3 id="wipe-board">Wipe Board</h3> | |
| <p>The robot moves an eraser across a marked whiteboard while maintaining sufficient surface contact to remove the marks.</p> | |
| <div class="mx-[calc(max(var(--minimum-inline-margin),(100cqw-100rem)/2)-var(--actual-inline-margin))]"> <article class="demo-card not-prose my-6 overflow-hidden rounded-2xl border border-zinc-200 bg-white shadow-sm dark:border-zinc-700 dark:bg-zinc-900" data-astro-cid-vaqhqpsm> <div class="demo-grid has-secondary" data-astro-cid-vaqhqpsm> <div class="video-frame front-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/wipe-tacforcing-front.DlS-UnU_.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Wipe Board · TacForcing, Front view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Front view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> <div class="video-frame side-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/wipe-tacforcing-side.BLPT7ZV6.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Wipe Board · TacForcing, Side view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Side view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> </div> <div class="px-5 py-4" data-astro-cid-vaqhqpsm> <h3 class="m-0 text-lg font-semibold whitespace-nowrap text-zinc-900 dark:text-white" data-astro-cid-vaqhqpsm> Wipe Board · TacForcing </h3> </div> </article> <article class="demo-card not-prose my-6 overflow-hidden rounded-2xl border border-zinc-200 bg-white shadow-sm dark:border-zinc-700 dark:bg-zinc-900" data-astro-cid-vaqhqpsm> <div class="demo-grid has-secondary" data-astro-cid-vaqhqpsm> <div class="video-frame front-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/wipe-face-front.Bv1EJbBD.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Wipe Board · Face, Front view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Front view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> <div class="video-frame side-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/wipe-face-side.C7cfXtD_.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Wipe Board · Face, Side view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Side view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> </div> <div class="px-5 py-4" data-astro-cid-vaqhqpsm> <h3 class="m-0 text-lg font-semibold whitespace-nowrap text-zinc-900 dark:text-white" data-astro-cid-vaqhqpsm> Wipe Board · Face </h3> </div> </article> <article class="demo-card not-prose my-6 overflow-hidden rounded-2xl border border-zinc-200 bg-white shadow-sm dark:border-zinc-700 dark:bg-zinc-900" data-astro-cid-vaqhqpsm> <div class="demo-grid has-secondary" data-astro-cid-vaqhqpsm> <div class="video-frame front-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/wipe-black-front.6ISu1yv6.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Wipe Board · Patch, Front view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Front view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> <div class="video-frame side-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/wipe-black-side.CbZkd_BR.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Wipe Board · Patch, Side view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Side view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> </div> <div class="px-5 py-4" data-astro-cid-vaqhqpsm> <h3 class="m-0 text-lg font-semibold whitespace-nowrap text-zinc-900 dark:text-white" data-astro-cid-vaqhqpsm> Wipe Board · Patch </h3> </div> </article> </div> | |
| <h3 id="stand-bottle">Stand Bottle</h3> | |
| <p>The robot grasps a bottle lying horizontally on the table, reorients it in hand, and places it upright while maintaining a stable grasp.</p> | |
| <div class="mx-[calc(max(var(--minimum-inline-margin),(100cqw-100rem)/2)-var(--actual-inline-margin))]"> <article class="demo-card not-prose my-6 overflow-hidden rounded-2xl border border-zinc-200 bg-white shadow-sm dark:border-zinc-700 dark:bg-zinc-900" data-astro-cid-vaqhqpsm> <div class="demo-grid has-secondary" data-astro-cid-vaqhqpsm> <div class="video-frame front-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/stand-bottle-front.Czdq02zM.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Stand Bottle, Front view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Front view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> <div class="video-frame side-view" data-astro-cid-vaqhqpsm> <video class="demo-video" src="/_astro/stand-bottle-side.D8ucrG5n.mp4" muted loop playsinline autoplay preload="metadata" aria-label="Stand Bottle, Side view" data-astro-cid-vaqhqpsm></video> <span class="view-label" data-astro-cid-vaqhqpsm>Side view</span> <span class="speed-label" data-astro-cid-vaqhqpsm>5× SPEED</span> </div> </div> <div class="px-5 py-4" data-astro-cid-vaqhqpsm> <h3 class="m-0 text-lg font-semibold whitespace-nowrap text-zinc-900 dark:text-white" data-astro-cid-vaqhqpsm> Stand Bottle </h3> </div> </article> </div> | |
| <h2 id="experiments">Experiments</h2> | |
| <h3 id="simulation-results">Simulation results</h3> | |
| <p>TacForcing achieves the best average success rate and the highest or tied-highest score on five of the six tasks.</p> | |
| <div class="mx-[calc(max(var(--minimum-inline-margin),(100cqw-100rem)/2)-var(--actual-inline-margin))]"> <figure> <div class="flex w-full justify-center **:my-0"> <picture> <source srcset="/_astro/univtac-tasks.pdf_Z14OV3n.avif 640w, /_astro/univtac-tasks.pdf_2rAlfB.avif 750w, /_astro/univtac-tasks.pdf_22y2fQ.avif 828w, /_astro/univtac-tasks.pdf_Zr0Qpu.avif 1080w, /_astro/univtac-tasks.pdf_ZJgl8O.avif 1280w, /_astro/univtac-tasks.pdf_1S0Gdw.avif 1668w, /_astro/univtac-tasks.pdf_Z2ib6g5.avif 1840w" type="image/avif"> <img src="/_astro/univtac-tasks.pdf_JUB1S.png" srcset="/_astro/univtac-tasks.pdf_Z1Wxm2H.png 640w, /_astro/univtac-tasks.pdf_1yRUgh.png 750w, /_astro/univtac-tasks.pdf_19PBgw.png 828w, /_astro/univtac-tasks.pdf_Z2j5kXY.png 1080w, /_astro/univtac-tasks.pdf_2sQj6C.png 1280w, /_astro/univtac-tasks.pdf_Z94Jir.png 1668w, /_astro/univtac-tasks.pdf_JUB1S.png 1840w" alt="The six UniVTAC simulation tasks used to evaluate TacForcing" loading="lazy" decoding="async" sizes="(min-width: 1840px) 1840px, 100vw" data-astro-image="constrained" data-astro-image-fit="contain" data-astro-image-pos="center" width="1840" height="1410" class="rounded-lg max-h-[calc(100svh-3rem)] max-w-full w-min mx-auto "> </picture> </div> <figcaption class="text-center"> Six contact-rich manipulation tasks from the UniVTAC benchmark. </figcaption> </figure> </div> | |
| <div class="full-bleed overflow-auto px-6"> <table class="mx-auto w-auto"> <thead><tr><th style="text-align:center">Method</th><th style="text-align:center">Lift Bottle</th><th style="text-align:center">Pull-out Key</th><th style="text-align:center">Lift Can</th><th style="text-align:center">Put Bottle in Shelf</th><th style="text-align:center">Insert Hole</th><th style="text-align:center">Insert Tube</th><th style="text-align:center"><strong>Avg.</strong></th></tr></thead><tbody><tr><td style="text-align:center">π₀.₅</td><td style="text-align:center">88</td><td style="text-align:center">43</td><td style="text-align:center">46</td><td style="text-align:center"><strong>43</strong></td><td style="text-align:center">39</td><td style="text-align:center">48</td><td style="text-align:center">51</td></tr><tr><td style="text-align:center">UniVTAC-ACT</td><td style="text-align:center">59</td><td style="text-align:center">41</td><td style="text-align:center">24</td><td style="text-align:center">6</td><td style="text-align:center">36</td><td style="text-align:center">58</td><td style="text-align:center">37</td></tr><tr><td style="text-align:center">RDP</td><td style="text-align:center">84</td><td style="text-align:center">18</td><td style="text-align:center">12</td><td style="text-align:center">41</td><td style="text-align:center">23</td><td style="text-align:center">75</td><td style="text-align:center">42</td></tr><tr><td style="text-align:center">FTP-1</td><td style="text-align:center">89</td><td style="text-align:center">35</td><td style="text-align:center"><strong>66</strong></td><td style="text-align:center">23</td><td style="text-align:center">62</td><td style="text-align:center">76</td><td style="text-align:center">59</td></tr><tr><td style="text-align:center"><strong>TacForcing</strong></td><td style="text-align:center"><strong>90</strong></td><td style="text-align:center"><strong>48</strong></td><td style="text-align:center">63</td><td style="text-align:center"><strong>43</strong></td><td style="text-align:center"><strong>69</strong></td><td style="text-align:center"><strong>79</strong></td><td style="text-align:center"><strong>65</strong></td></tr></tbody> </table> </div> | |
| <h3 id="real-world-results">Real-world results</h3> | |
| <p>TacForcing reaches a <strong>69% average success rate</strong> across the three real-world tasks.</p> | |
| <figure> <div class="flex w-full justify-center **:my-0"> <picture> <source srcset="/_astro/real-world-results.pdf_RNGSr.avif 640w, /_astro/real-world-results.pdf_1eXr6z.avif 750w, /_astro/real-world-results.pdf_35EtR.avif 828w, /_astro/real-world-results.pdf_ZRaDo7.avif 1080w, /_astro/real-world-results.pdf_Sn4TY.avif 1280w, /_astro/real-world-results.pdf_Z2ipTAU.avif 1668w, /_astro/real-world-results.pdf_Zuny6e.avif 1919w" type="image/avif"> <img src="/_astro/real-world-results.pdf_Z2ti5lj.png" srcset="/_astro/real-world-results.pdf_ZFQdVf.png 640w, /_astro/real-world-results.pdf_ZjGtI7.png 750w, /_astro/real-world-results.pdf_Z1vzgkO.png 828w, /_astro/real-world-results.pdf_1m0U1S.png 1080w, /_astro/real-world-results.pdf_Z1WCusW.png 1280w, /_astro/real-world-results.pdf_Z4elaU.png 1668w, /_astro/real-world-results.pdf_Z2ti5lj.png 1919w" alt="Real-world Stand Bottle, Transfer Liquid, and Wipe Board tasks with success-rate comparison" loading="lazy" decoding="async" sizes="(min-width: 1919px) 1919px, 100vw" data-astro-image="constrained" data-astro-image-fit="contain" data-astro-image-pos="center" width="1919" height="1080" class="rounded-lg max-h-[calc(100svh-3rem)] max-w-full w-min mx-auto "> </picture> </div> <figcaption class="text-center"> Representative real-world task snapshots and success rates across all three tasks. </figcaption> </figure> | |
| <h2 id="what-matters-ablating-execution-time-touch">What matters? Ablating execution-time touch</h2> | |
| <p>A single fixed tactile observation does not consistently improve performance. Streaming the tactile updates helps, and aligning those updates with the next block through EATA yields the strongest results on every evaluated ablation task.</p> | |
| <div class="not-prose my-8 rounded-2xl border border-zinc-200 bg-white p-5 shadow-sm dark:border-zinc-700 dark:bg-zinc-900" data-astro-cid-cvmid3nv> <div class="mb-5 flex flex-wrap items-center justify-between gap-3" data-astro-cid-cvmid3nv> <div data-astro-cid-cvmid3nv> <div class="text-sm font-semibold text-zinc-900 dark:text-white" data-astro-cid-cvmid3nv> | |
| Average success rate | |
| </div> <div class="text-xs text-zinc-500 dark:text-zinc-400" data-astro-cid-cvmid3nv> | |
| Three simulation tasks / three real-world tasks | |
| </div> </div> <div class="flex gap-4 text-xs font-medium text-zinc-600 dark:text-zinc-300" data-astro-cid-cvmid3nv> <span class="legend simulation" data-astro-cid-cvmid3nv>Simulation</span> <span class="legend real" data-astro-cid-cvmid3nv>Real world</span> </div> </div> <div class="space-y-5" data-astro-cid-cvmid3nv> <div class="bar-row" data-astro-cid-cvmid3nv> <div class="mb-2 flex items-center justify-between gap-4" data-astro-cid-cvmid3nv> <span class="text-sm font-semibold text-zinc-800 dark:text-zinc-100" data-astro-cid-cvmid3nv> Base </span> </div> <div class="grid grid-cols-[4.5rem_1fr] items-center gap-3 text-xs" data-astro-cid-cvmid3nv> <span class="text-zinc-500 dark:text-zinc-400" data-astro-cid-cvmid3nv>Simulation</span> <div class="track" data-astro-cid-cvmid3nv> <span class="bar simulation" style="width:43%" data-astro-cid-cvmid3nv></span> <b data-astro-cid-cvmid3nv>43%</b> </div> <span class="text-zinc-500 dark:text-zinc-400" data-astro-cid-cvmid3nv>Real world</span> <div class="track" data-astro-cid-cvmid3nv> <span class="bar real" style="width:42%" data-astro-cid-cvmid3nv></span> <b data-astro-cid-cvmid3nv>42%</b> </div> </div> </div><div class="bar-row" data-astro-cid-cvmid3nv> <div class="mb-2 flex items-center justify-between gap-4" data-astro-cid-cvmid3nv> <span class="text-sm font-semibold text-zinc-800 dark:text-zinc-100" data-astro-cid-cvmid3nv> Fixed tactile </span> </div> <div class="grid grid-cols-[4.5rem_1fr] items-center gap-3 text-xs" data-astro-cid-cvmid3nv> <span class="text-zinc-500 dark:text-zinc-400" data-astro-cid-cvmid3nv>Simulation</span> <div class="track" data-astro-cid-cvmid3nv> <span class="bar simulation" style="width:42%" data-astro-cid-cvmid3nv></span> <b data-astro-cid-cvmid3nv>42%</b> </div> <span class="text-zinc-500 dark:text-zinc-400" data-astro-cid-cvmid3nv>Real world</span> <div class="track" data-astro-cid-cvmid3nv> <span class="bar real" style="width:31%" data-astro-cid-cvmid3nv></span> <b data-astro-cid-cvmid3nv>31%</b> </div> </div> </div><div class="bar-row" data-astro-cid-cvmid3nv> <div class="mb-2 flex items-center justify-between gap-4" data-astro-cid-cvmid3nv> <span class="text-sm font-semibold text-zinc-800 dark:text-zinc-100" data-astro-cid-cvmid3nv> TacForcing w/o EATA </span> </div> <div class="grid grid-cols-[4.5rem_1fr] items-center gap-3 text-xs" data-astro-cid-cvmid3nv> <span class="text-zinc-500 dark:text-zinc-400" data-astro-cid-cvmid3nv>Simulation</span> <div class="track" data-astro-cid-cvmid3nv> <span class="bar simulation" style="width:51%" data-astro-cid-cvmid3nv></span> <b data-astro-cid-cvmid3nv>51%</b> </div> <span class="text-zinc-500 dark:text-zinc-400" data-astro-cid-cvmid3nv>Real world</span> <div class="track" data-astro-cid-cvmid3nv> <span class="bar real" style="width:48%" data-astro-cid-cvmid3nv></span> <b data-astro-cid-cvmid3nv>48%</b> </div> </div> </div><div class="bar-row ours" data-astro-cid-cvmid3nv> <div class="mb-2 flex items-center justify-between gap-4" data-astro-cid-cvmid3nv> <span class="text-sm font-semibold text-zinc-800 dark:text-zinc-100" data-astro-cid-cvmid3nv> TacForcing </span> <span class="rounded-full bg-[var(--tac-orange-soft)] px-2 py-0.5 text-[0.65rem] font-bold tracking-wide text-amber-900" data-astro-cid-cvmid3nv> | |
| FULL MODEL | |
| </span> </div> <div class="grid grid-cols-[4.5rem_1fr] items-center gap-3 text-xs" data-astro-cid-cvmid3nv> <span class="text-zinc-500 dark:text-zinc-400" data-astro-cid-cvmid3nv>Simulation</span> <div class="track" data-astro-cid-cvmid3nv> <span class="bar simulation" style="width:60%" data-astro-cid-cvmid3nv></span> <b data-astro-cid-cvmid3nv>60%</b> </div> <span class="text-zinc-500 dark:text-zinc-400" data-astro-cid-cvmid3nv>Real world</span> <div class="track" data-astro-cid-cvmid3nv> <span class="bar real" style="width:69%" data-astro-cid-cvmid3nv></span> <b data-astro-cid-cvmid3nv>69%</b> </div> </div> </div> </div> </div> | |
| <p>Adding EATA to streaming tactile updates raises average success from <strong>51% to 60%</strong> in simulation and from <strong>48% to 69%</strong> in the real world. Relative to fixed tactile conditioning, the complete method gains 18 and 38 percentage points, respectively.</p> | |
| <h2 id="conclusion">Conclusion</h2> | |
| <p>TacForcing turns action generation itself into the feedback loop. It progressively completes action blocks, executes them, refreshes tactile observations, and resumes generation from the unfinished states. EATA then ensures each tactile update affects the actions close enough to execution to use it. The result is a simpler path to tactile reactivity—and consistent gains across both simulated and real contact-rich manipulation.</p> | |
| <h2 id="bibtex">BibTeX</h2> | |
| <div class="expressive-code"><link rel="stylesheet" href="/_astro/ec.0wf8d.css"/><script type="module" src="/_astro/ec.0vx5m.js"></script><figure class="frame"><figcaption class="header"></figcaption><pre data-language="bibtex"><code><div class="ec-line"><div class="code"><span style="--0:#F97583;--1:#BF3441">@misc</span><span style="--0:#E1E4E8;--1:#24292E">{</span><span style="--0:#B392F0;--1:#6F42C1">zhou2026tacforcing</span><span style="--0:#E1E4E8;--1:#24292E">,</span></div></div><div class="ec-line"><div class="code"><span class="indent"> </span><span style="--0:#79B8FF;--1:#005CC5">title</span><span style="--0:#E1E4E8;--1:#24292E"> = </span><span style="--0:#9ECBFF;--1:#032F62">{</span><span style="--0:#E1E4E8;--1:#24292E">TacForcing: Streaming Action Generation with</span></div></div><div class="ec-line"><div class="code"><span class="indent"><span style="--0:#E1E4E8;--1:#24292E"> </span></span><span style="--0:#E1E4E8;--1:#24292E">Execution-Time Tactile Feedback</span><span style="--0:#9ECBFF;--1:#032F62">}</span><span style="--0:#E1E4E8;--1:#24292E">,</span></div></div><div class="ec-line"><div class="code"><span class="indent"> </span><span style="--0:#79B8FF;--1:#005CC5">author</span><span style="--0:#E1E4E8;--1:#24292E"> = </span><span style="--0:#9ECBFF;--1:#032F62">{</span><span style="--0:#E1E4E8;--1:#24292E">Zhou, Jianbo and Zhao, Boyuan and Zhang, Yuzheng and</span></div></div><div class="ec-line"><div class="code"><span class="indent"><span style="--0:#E1E4E8;--1:#24292E"> </span></span><span style="--0:#E1E4E8;--1:#24292E">Chen, Yiyang and Chen, Wenxin and Li, Qiuyue and</span></div></div><div class="ec-line"><div class="code"><span class="indent"><span style="--0:#E1E4E8;--1:#24292E"> </span></span><span style="--0:#E1E4E8;--1:#24292E">Gu, Xiangyang and Cao, Yuhan and Xia, Xiao and</span></div></div><div class="ec-line"><div class="code"><span class="indent"><span style="--0:#E1E4E8;--1:#24292E"> </span></span><span style="--0:#E1E4E8;--1:#24292E">Hu, Yanzhe and Deng, Zhijie</span><span style="--0:#9ECBFF;--1:#032F62">}</span><span style="--0:#E1E4E8;--1:#24292E">,</span></div></div><div class="ec-line"><div class="code"><span class="indent"> </span><span style="--0:#79B8FF;--1:#005CC5">year</span><span style="--0:#E1E4E8;--1:#24292E"> = </span><span style="--0:#9ECBFF;--1:#032F62">{</span><span style="--0:#E1E4E8;--1:#24292E">2026</span><span style="--0:#9ECBFF;--1:#032F62">}</span><span style="--0:#E1E4E8;--1:#24292E">,</span></div></div><div class="ec-line"><div class="code"><span class="indent"> </span><span style="--0:#79B8FF;--1:#005CC5">eprint</span><span style="--0:#E1E4E8;--1:#24292E"> = </span><span style="--0:#9ECBFF;--1:#032F62">{</span><span style="--0:#E1E4E8;--1:#24292E">2608.25798</span><span style="--0:#9ECBFF;--1:#032F62">}</span><span style="--0:#E1E4E8;--1:#24292E">,</span></div></div><div class="ec-line"><div class="code"><span class="indent"> </span><span style="--0:#79B8FF;--1:#005CC5">archivePrefix</span><span style="--0:#E1E4E8;--1:#24292E"> = </span><span style="--0:#9ECBFF;--1:#032F62">{</span><span style="--0:#E1E4E8;--1:#24292E">arXiv</span><span style="--0:#9ECBFF;--1:#032F62">}</span><span style="--0:#E1E4E8;--1:#24292E">,</span></div></div><div class="ec-line"><div class="code"><span class="indent"> </span><span style="--0:#79B8FF;--1:#005CC5">primaryClass</span><span style="--0:#E1E4E8;--1:#24292E"> = </span><span style="--0:#9ECBFF;--1:#032F62">{</span><span style="--0:#E1E4E8;--1:#24292E">cs.RO</span><span style="--0:#9ECBFF;--1:#032F62">}</span><span style="--0:#E1E4E8;--1:#24292E">,</span></div></div><div class="ec-line"><div class="code"><span class="indent"> </span><span style="--0:#79B8FF;--1:#005CC5">doi</span><span style="--0:#E1E4E8;--1:#24292E"> = </span><span style="--0:#9ECBFF;--1:#032F62">{</span><span style="--0:#E1E4E8;--1:#24292E">10.48550/arXiv.2608.25798</span><span style="--0:#9ECBFF;--1:#032F62">}</span><span style="--0:#E1E4E8;--1:#24292E">,</span></div></div><div class="ec-line"><div class="code"><span class="indent"> </span><span style="--0:#79B8FF;--1:#005CC5">url</span><span style="--0:#E1E4E8;--1:#24292E"> = </span><span style="--0:#9ECBFF;--1:#032F62">{</span><span style="--0:#E1E4E8;--1:#24292E">https://arxiv.org/abs/2608.25798</span><span style="--0:#9ECBFF;--1:#032F62">}</span></div></div><div class="ec-line"><div class="code"><span style="--0:#E1E4E8;--1:#24292E">}</span></div></div></code></pre><div class="copy"><div aria-live="polite"></div><button title="Copy to clipboard" data-copied="Copied!" data-code="@misc{zhou2026tacforcing, title = {TacForcing: Streaming Action Generation with Execution-Time Tactile Feedback}, author = {Zhou, Jianbo and Zhao, Boyuan and Zhang, Yuzheng and Chen, Yiyang and Chen, Wenxin and Li, Qiuyue and Gu, Xiangyang and Cao, Yuhan and Xia, Xiao and Hu, Yanzhe and Deng, Zhijie}, year = {2026}, eprint = {2608.25798}, archivePrefix = {arXiv}, primaryClass = {cs.RO}, doi = {10.48550/arXiv.2608.25798}, url = {https://arxiv.org/abs/2608.25798}}"><div></div></button></div></figure></div> </main> <footer class="mx-auto max-w-[50rem] px-6 text-center"> <p class="prose-sm text-center"> | |
| Built with <a href="https://research-template.roman.technology" rel="nofollow">Roman Hauksson-Neill's project page template</a> </p> </footer> </body></html> |