abc-grasp-eval / README.md
tianxiaoshen's picture
Replace old benchmark with agent and world-model evaluation galleries
9d71edb verified
|
Raw History Blame Contribute Delete
973 Bytes
metadata
title: ABC Grasp Evaluation
emoji: 🤖
colorFrom: blue
colorTo: green
sdk: static
app_file: index.html
fullWidth: true
short_description: Agent predictions and world-model grasp rollouts

ABC Grasp Evaluation

Current human-confirmed grasp benchmark: 37 successes and 37 failures.

  • Agents: initial images and recorded action targets; 70 scored attempts, excluding four labeled examples. Success threshold: 0.5.
  • World Models: generated videos alongside recorded videos; 74 attempts. Generated outcomes are AI-reviewed with user corrections, not all human-confirmed.

These evaluation sets differ by four examples. Fable5p1 Extra disclosed reading outside its allowed inputs; the gallery marks this protocol deviation.

The static site contains display predictions, recorded outcomes, and videos. No inference runs here. Private input packets and detailed review records are not included. This replaces the previous motion-proxy benchmark display.