README.md

---
license: mit
tags:
- robotics
- vla
- vision-language-action
- libero
- model-compression
pipeline_tag: robotics
---

# VLADrop-OpenVLA-OFT-LIBERO-DTR30

Checkpoint for [Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?](https://arxiv.org/abs/2606.27755).

DTR (Drop-Then-Recovery) removes transformer blocks from a pretrained VLA model and
recovery-fine-tunes the smaller dense model. Code: https://github.com/s1ghhh/VLADrop

## This checkpoint

| | |
|---|---|
| Paper row | Table 1 'Keep 2 Language' / Table 5 'DTR-30' (2.60x task speedup) |
| Dropped blocks | Llama-2 decoder blocks [1..30] (keep blocks 0 and 31) |
| Recovery training | LoRA rank 32 (merged), batch size 16, 50K steps, lr 5e-4 |
| LIBERO success rate | Spatial 97.2 / Object 99.0 / Goal 95.4 / Long 88.6 / Avg 95.1; LIBERO-Goal speed benchmark: SR 90.0, 2.94x action speedup, 3.06 GB |

## Usage

Merged full-weight OpenVLA-OFT checkpoint. The drop lists are baked into
`config.json` (`text_config.drop_attn_list` / `drop_mlp_list`), so evaluation
reconstructs the pruned graph automatically. Requires the VLADrop modified
transformers (`transformers-openvla-oft-dropped`) from https://github.com/s1ghhh/VLADrop

```bash
python -m experiments.robot.libero.run_libero_eval_random_dropped \
    --pretrained_checkpoint <this_repo_local_path> \
    --task_suite_name libero_spatial \
    --num_trials_per_task 50 --center_crop True
```

Includes `action_head--*.pt` and `proprio_projector--*.pt` (L1 regression head, 2 input images + proprio).

## Citation

```bibtex
@article{sun2026vladrop,
  title={Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?},
  author={Sun, Guoheng and Feng, Kaixi and He, Shwai and Gong, Xiaochuan and He, Yexiao and Wang, Ziyao and Shen, Zheyu and Ye, Wanghao and Kompella, Ramana Rao and Liu, Gaowen and Li, Ang},
  journal={arXiv preprint arXiv:2606.27755},
  year={2026}
}
```