Looking once versus repeatedly. AdaLoop turns a span of the frozen vision encoder into a question-conditioned perception loop that updates its visual state over T iterations; the language model answers from the resulting sequence.
## Checkpoints Scores are macro accuracy on VisRO-Core6 (3,600 items per split), as *measured from this checkpoint / reported in the paper*. Variants A, B and C use `p=1` stopping (stop once two consecutive answers agree) with at most 3 iterations. | Backbone | Setting | Path | ID | OOD | | --- | --- | --- | --- | --- | | Qwen2.5-VL-3B | SFT | [`qwen/sft`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/qwen/sft) | 49.9 / 49.9 | 23.6 / 23.9 | | Qwen2.5-VL-3B | AdaLN | [`qwen/adaln`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/qwen/adaln) | 59.9 / 60.0 | 30.2 / 30.2 | | Qwen2.5-VL-3B | AdaLoop-A | [`qwen/variant_a`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/qwen/variant_a) | 64.6 / 64.6 | 32.5 / 32.4 | | Qwen2.5-VL-3B | AdaLoop-B | [`qwen/variant_b`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/qwen/variant_b) | 62.1 / 62.1 | 31.1 / 31.1 | | Qwen2.5-VL-3B | AdaLoop-C | [`qwen/variant_c`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/qwen/variant_c) | 63.0 / 63.0 | 32.0 / 32.0 | | PaliGemma2-3B | SFT | [`paligemma/sft`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/paligemma/sft) | 62.7 / 62.6 | 31.5 / 31.5 | | PaliGemma2-3B | AdaLN | [`paligemma/adaln`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/paligemma/adaln) | 72.4 / 72.4 | 40.0 / 40.0 | | PaliGemma2-3B | AdaLoop-A | [`paligemma/variant_a`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/paligemma/variant_a) | 75.2 / 75.2 | 43.9 / 43.9 | | PaliGemma2-3B | AdaLoop-B | [`paligemma/variant_b`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/paligemma/variant_b) | 74.0 / – | 43.7 / – | | PaliGemma2-3B | AdaLoop-C | [`paligemma/variant_c`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/paligemma/variant_c) | 74.1 / – | 41.1 / – | | LLaVA-1.5-7B | SFT | [`llava/sft`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/llava/sft) | 50.4 / 50.4 | 24.8 / 24.8 | | LLaVA-1.5-7B | AdaLN | [`llava/adaln`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/llava/adaln) | 57.7 / 57.7 | 29.6 / 29.6 | | LLaVA-1.5-7B | AdaLoop-A | [`llava/variant_a`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/llava/variant_a) | 61.9 / 61.8 | 31.2 / 31.1 | | LLaVA-1.5-7B | AdaLoop-B | [`llava/variant_b`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/llava/variant_b) | 59.3 / – | 29.7 / – | | LLaVA-1.5-7B | AdaLoop-C | [`llava/variant_c`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/llava/variant_c) | 60.0 / – | 30.6 / – | Slight differences may arise from different hardware and numeric precision. The Qwen SFT checkpoint and the PaliGemma and LLaVA Variant B and C checkpoints were trained by us with the same recipes (the paper reports Variants B and C for Qwen only); all others are the paper's runs. ## Download Download one checkpoint, or everything: ```bash hf download Lyy0725/AdaLoop --include "qwen/variant_a/*" --local-dir checkpoints hf download Lyy0725/AdaLoop --local-dir checkpoints ``` Then evaluate it on VisRO with the AdaLoop code, for example: ```bash adaloop-qwen-eval --checkpoint checkpoints/qwen/variant_a \ --visro-root data/VisRO --tasks core6 --splits eval,ood \ --output-dir outputs/eval/qwen_variant_a ``` ## Base Models Each checkpoint holds only the trained weights (LoRA and the AdaLoop modules) and is loaded on top of its base model, which its `adaloop_manifest.json` pins: | Backbone | Base model | | --- | --- | | Qwen2.5-VL-3B | [`Qwen/Qwen2.5-VL-3B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct), fetched from the Hub at the pinned revision | | PaliGemma2-3B | the Big Vision weights `paligemma2-3b-mix-448.b16.npz` from the gated [`google/paligemma2-3b-mix-448-jax`](https://huggingface.co/google/paligemma2-3b-mix-448-jax), and the PaliGemma SentencePiece tokenizer | | LLaVA-1.5-7B | our stage-2 LLaVA-1.5-7B at 448 px, [`llava/base/llava-v1.5-7b-448-s2`](https://huggingface.co/Lyy0725/AdaLoop/tree/main/llava/base) in this repository (Vicuna-7B-v1.5 trained with the LLaVA-1.5 recipe at 448 px), and the CLIP tower [`openai/clip-vit-large-patch14-336`](https://huggingface.co/openai/clip-vit-large-patch14-336) | ## Files Every checkpoint directory holds `adaloop_config.yaml` (the training recipe), `adaloop_manifest.json` (the base model and every file, pinned by size and SHA-256), `adaloop_runtime.json` and `SHA256SUMS`, plus its weights: - Qwen: `adapter_config.json`, `adapter_model.safetensors` (LoRA), `non_lora_trainables.safetensors` (merger and AdaLoop modules) and `processor/`; - PaliGemma: `trainable.params.msgpack`; - LLaVA: `adapter_config.json`, `adapter_model.safetensors` (LoRA), `non_lora_trainables.safetensors` (projector and AdaLoop modules) and `config.json`. The loaders check every file against the manifest before use. `SHA256SUMS` at the root lists every checkpoint file. ## License The checkpoints are derived from Qwen2.5-VL, PaliGemma 2 and LLaVA-1.5 (Vicuna, Llama 2, CLIP), and each is subject to the terms of its base model: the [Qwen Research License](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE) of Qwen2.5-VL-3B-Instruct, the Gemma Terms of Use, and the Llama 2 Community License. The AdaLoop code is under the Apache License 2.0, and VisRO under CC BY-SA 4.0. ## Citation If you find this work useful in your research, please consider citing our paper :) ```bib @article{adaloop2026, title={AdaLoop: Thinking with the Vision Encoder}, author={Han, Xinran Nicole and Lu, Yiyang and Gan, Katelyn and Zickler, Todd}, journal={arXiv preprint arXiv:2610.21323}, year={2026} } ```