CarryOn: A Dual-Loop Harness for Personalized and Reliable Robot Assistance
Abstract
CarryOn enhances frozen vision-language-action policies with persistent personalized planning and supervised execution with recovery skills to improve safety and adaptability.
While generalist vision-language-action (VLA) models have enabled diverse open-world manipulation, real-world robotic assistance requires two critical capabilities they currently lack: personalization and reliable execution. First, executing an instruction like "make me a sandwich" requires grounding subtask decomposition in persistent user preferences rather than relying on a single generic solution. Second, reliable deployment demands explainable safety assurance that proactively prevents hazards and gracefully recovers from occasional physical failures. To address these gaps, we present CarryOn, a dual-loop harness designed to augment a frozen VLA policy. The planning loop maintains persistent user-level and scene-level context that is continually grounded against new observations and revised through interaction, enabling personalized task selection and execution grounding across conversations and changing scenes. The execution loop actively supervises policy rollouts, shielding actions from predicted hazardous contact. When execution fails, the system invokes reusable control-based recovery skills that restore task progress and are refined through reflection on earlier failures. On a real robot and in simulation, CarryOn dispatches preference-consistent subtasks across users and scenes, reduces the violation rate of an already safety-finetuned VLA from 52.6% to 6.1%, and increases recovery success from 13% to 80% when refined skills are reused.
Community
CarryOn combines personalized planning, runtime protection, and offline recovery refinement around a frozen VLA.
The planning loop maintains persistent, revisable user and scene context to guide personalized task decisions. The execution loop combines proactive geometric shielding with reactive recovery, whose recovery skills improve through offline reflection.
Two highlights from our evaluations:
- On LIBERO-Safety, CarryOn further reduces the safety-finetuned policy’s violation rate from 52.6% to 6.1%.
- On perturbed LIBERO-Object tasks, success rate increases from 13% with VLA alone to 80.0% after two successive reflection-and-retry rounds, with each round refining and retrying the remaining failures.
Please also check our real-robot demo to see how remembered preferences, new observations, shielding, and recovery work together within an assistance task 😊
Get this paper in your agent:
hf papers read 2610.21453 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper
