spikefly/PACE-LIBERO

PACE · LIBERO

✍️ Fei Ni, Zhuo Chen, Xianze Yao, Zerui Chen, Changrui Chen, Yifu Yuan, Shan Luo, Jianye Hao, Jiankang Deng
💻 Code: github.com/Fei-Ni/PACE

PACE is a vision-language-action policy that internalizes task progress: it predicts a foresight anchor (an embedding of the finished task), reads its current progress and a forward commitment against it, and conditions its action decoder on them. This repository holds one checkpoint trained jointly on the four LIBERO suites.

Result

LIBERO success rate (%), 10 tasks × 50 trials per suite, seed 7, evaluated with examples/LIBERO/eval_libero.sh from the code repository.

Spatial	Object	Goal	Long	Average
99.0	99.8	99.4	96.2	98.6

The paper (Table 1) reports 99.0 / 99.8 / 97.8 / 98.0, average 98.7. This checkpoint was retrained with the released code; the recipe is described in the code repository's docs/TRAINING.md.

Failure alarm

Every response of the policy server also carries a failure alarm (audit_flag, with audit_score and audit_stall): it fires when the progress the policy commits to keeps going undelivered, or when its progress reading stops rising. With the thresholds in config.yaml (a shared 0.4 for unmet commitments; the stall limit calibrated on other checkpoints, never on this one), it flags 92.9% of the 28 failed episodes of the evaluation above and 7.2% of the 1,972 successful ones, with a median of half the episode still left. It only reports and never changes the actions. Details and a calibration script: docs/PROGRESS.md in the code repository.

Architecture

Component	Choice
Vision-language backbone	Qwen3-VL-4B-Instruct
Action head	flow-matching DiT-B (16 layers), chunk of 8 actions, 4 denoising steps
Progress encoder	DINOv2 ViT-B/14, frozen (CLS, 768-d)
Progress interface	4 goal-query tokens → foresight anchor; pcur, Δpcur readouts; Δpprev input token; anchor + pcur + Δpcur prepended to the DiT
Inputs	third-person + wrist images (224 × 224) at t and at the latest query at or before t − 7 (t − 8 with the LIBERO client), instruction; no proprioception
Actions	7-D end-effector delta (xyz, rpy, gripper), normalized with dataset_statistics.json (key franka)

Training data

The four *_no_noops LIBERO datasets (Spatial, Object, Goal, 10) in LeRobot format, 30K steps at batch 64.

Files

PACE-LIBERO/
├── README.md
├── config.yaml               # model config (base_vlm = Qwen/Qwen3-VL-4B-Instruct)
├── dataset_statistics.json   # action normalization statistics
└── pytorch_model.pt          # full model state dict

How to load

huggingface-cli download spikefly/PACE-LIBERO --local-dir ./PACE-LIBERO

from pace.model.framework.base_framework import baseframework

policy = baseframework.from_pretrained("./PACE-LIBERO").cuda().eval()   # or "spikefly/PACE-LIBERO"

Qwen3-VL-4B-Instruct (processor and architecture) and the DINOv2 ViT-B/14 weights are fetched on first use; set BASE_VLM / DINOV2_WEIGHTS to local copies to skip the downloads. For evaluation, see examples/LIBERO/ in the code repository.

Sibling checkpoint

Checkpoint	Benchmark
spikefly/PACE-SimplerEnv	SimplerEnv WidowX

Citation

@misc{ni2026pace,
  title  = {{PACE}: Progress Actively-internalized Conditioning Execution for Long-horizon Manipulation},
  author = {Ni, Fei and Chen, Zhuo and Yao, Xianze and Chen, Zerui and Chen, Changrui and Yuan, Yifu and Luo, Shan and Hao, Jianye and Deng, Jiankang},
  year   = {2026}
}

License

MIT for the PACE weights and code. The backbone weights derive from Qwen3-VL-4B-Instruct (Apache-2.0) and the progress encoder is DINOv2 (Apache-2.0); LIBERO data follows its own license.