Beyond Zooming: Learning Multi-Tool Visual Reasoning for Ultra-High-Resolution Remote Sensing

Fengxiang Wang1,*, Jiangnan Huang2,*, Mingshuo Chen1, Yueying Li1,
Yang Shi1, Junwei Luo2, Haoyu Wang3,†, Yansheng Li2, Jing Zhang2,
Haiyan Zhao3,†, Wenjing Yang1

1 National University of Defense Technology   2 Wuhan University   3 Tsinghua University
* Equal contribution   † Corresponding authors

📃 Paper | 🤗 GeoLens Model | 🤗 GeoMTVR Dataset | 🏆 XLRS-Bench

--- # 📚 Contents - [📚 Contents](#-contents) - [🔥News](#news) - [📜Abstract](#abstract) - [🔍Method](#method) - [Supervised Fine-Tuning (SFT)](#supervised-fine-tuning-sft) - [Tool-Attention-Focused Reinforcement Learning (RTAL)](#tool-attention-focused-reinforcement-learning-rtal) - [🚀Reproduction](#reproduction) - [Environment](#environment) - [SFT](#sft) - [RTAL](#rtal) - [XLRS-Bench](#xlrs-bench) - [RSHR](#rshr) - [LRS-GRO-eval](#lrs-gro-eval) - [📊Results](#results) - [🔗Citation](#citation) - [🤝Acknowledgement](#acknowledgement) # 🔥News * **[2026.07]** Paper released on [arXiv](https://arxiv.org/abs/2607.25993). * **[2026.06]** Model weights and code released. * **[2026.06]** GeoMTVR dataset released. # 📜Abstract Ultra-high-resolution (UHR) remote-sensing (RS) imagery provides fine-grained Earth-observation evidence over city-scale scenes, but poses a fundamental challenge for multimodal large language models (MLLMs): task-relevant evidence is often sparse, local, and spatially dispersed across extremely large visual contexts. A natural solution is to equip MLLMs with zoom-in tools for active local inspection. However, through a pilot study on XLRS-Bench, we find that zoom-in is only partially effective: it resolves easy and medium-level tasks with locally recoverable evidence, but saturates on hard cases requiring global search, multi-region comparison, path planning, or dispersed-evidence reasoning. Motivated by this finding, we move beyond single-tool zoom-in and introduce **GeoMTVR**, a large-scale Geospatial Multi-Tool Visual Reasoning dataset built from wide-area satellite imagery. GeoMTVR contains **13K** UHR VQA samples with interleaved reasoning trajectories, diverse visual tool calls, and returned visual observations, enabling models to learn question decomposition, tool selection, regional inspection, object-level grounding, auxiliary visual reasoning, and cross-tool evidence integration. Beyond supervised fine-tuning, we propose a **tool-attention-focused reinforcement learning** algorithm (RTAL) that concentrates optimization on critical tool-use decisions, including when to invoke tools, which tool to select, where to apply it, and how to interpret tool outputs. By combining SFT on GeoMTVR with our RL algorithm, we develop **GeoLens**, a multi-tool visual reasoning MLLM for UHR RS. Experiments show that GeoLens consistently outperforms direct reasoning and single-tool zoom-in baselines, achieving stronger accuracy, better evidence grounding, and more efficient tool-use trajectories. ## Model Weights | Model | Description | Link | |-------|-------------|------| | **GeoLens** | Final multi-tool visual reasoning model (SFT + RTAL) | [🤗 NaOHjiang/GeoLens](https://huggingface.co/NaOHjiang/GeoLens) | | **GeoMTVR** | SFT training dataset (13K UHR VQA samples with tool trajectories) | [🤗 NaOHjiang/GeoMTVR](https://huggingface.co/datasets/NaOHjiang/GeoMTVR) | ## Repository Structure ``` GeoLens/ sft/ # SFT configs and data-building scripts rtal/RTAL_dl_final/ # RTAL RL training code (dual JSD loss) evaluation/ EVAL_by_myself/ # XLRS-Bench eval & score recomputation RSHR_eval/ # RSHR benchmark evaluation LRSGRO_eval/ # LRS-GRO-eval evaluation scripts/ # Reproduction wrappers (SFT, RTAL, eval) docs/ # Source manifest and notes ``` # 🔍Method ## Supervised Fine-Tuning (SFT) We fine-tune Qwen2.5-VL-7B-Instruct on the GeoMTVR dataset, which contains **13K** UHR VQA samples with interleaved tool-use trajectories. The SFT stage teaches the model to follow tool-use instructions, perform regional inspections, and integrate multi-tool evidence. ## Tool-Attention-Focused Reinforcement Learning (RTAL) We propose **RTAL** (Reinforced Tool-Attention Learning), a novel RL algorithm that applies a dual JSD (Jensen-Shannon Divergence) loss with **key-mask** and **valid-mask** to concentrate optimization on critical tool-use decisions: - **When to invoke tools**: The model learns to decide whether current visual evidence is sufficient or additional tool calls are needed. - **Which tool to select**: The model learns to choose the appropriate tool (zoom-in, region inspection, object grounding, etc.) based on the task requirement. - **Where to apply it**: The model learns to localize the correct spatial region for tool application. - **How to interpret tool outputs**: The model learns to integrate tool-returned visual observations into its reasoning chain. # 🚀Reproduction ## Environment ```bash git clone https://github.com/NaOH3021/GeoLens.git cd GeoLens python -m venv .venv source .venv/bin/activate pip install --upgrade pip pip install -r requirements.txt pip install -e rtal/RTAL_dl_final/verl ``` ## SFT The SFT stage fine-tunes Qwen2.5-VL-7B-Instruct on the GeoMTVR dataset to obtain the SFT checkpoint. ### Step 1: Download Dependencies | Dependency | Source | Notes | |------------|--------|-------| | **Qwen2.5-VL-7B-Instruct** | [🤗 Qwen/Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) | Base model | | **GeoMTVR dataset** | [🤗 NaOHjiang/GeoMTVR](https://huggingface.co/datasets/NaOHjiang/GeoMTVR) | Multi-tool visual reasoning dataset (13K samples) | | **LLaMA-Factory** | `pip install llamafactory` | Training framework used by the SFT script | Place the base model at a local path, e.g., `/path/to/models/Qwen2.5-VL-7B-Instruct`. ### Step 2: Download and Prepare Dataset Download the GeoMTVR dataset from [HuggingFace](https://huggingface.co/datasets/NaOHjiang/GeoMTVR). The repository contains annotation files (`dataset_info.json` + 3 JSONL files) and image chunks: ```bash # Download the dataset repository git lfs clone https://huggingface.co/datasets/NaOHjiang/GeoMTVR # Or download files manually via the web interface (see "Files" tab) # Merge and extract the image archive cd GeoMTVR cat images/geomtvr_images_chunk_*.tar.part > geomtvr_images.tar tar xf geomtvr_images.tar ``` Organize the extracted data into a single dataset directory (LLaMA-Factory format): ``` /path/to/geomtvr-data/ cold_start/ super_accepted_or_route.3img_summary.jsonl zoom_results.visual.qa.with_aux.full.strict_superstyle.jsonl route_visual_accepted_388.superstyle_sft.jsonl images/ aux_images/... vqa_results/... dataset_info.json ``` Refer to [LLaMA-Factory dataset preparation](https://github.com/hiyouga/LLaMA-Factory/blob/main/data/README.md) for the exact format. ### Step 3: Run SFT ```bash BASE_MODEL=/path/to/models/Qwen2.5-VL-7B-Instruct \ DATASET_DIR=/path/to/geomtvr-data \ OUTPUT_DIR=/path/to/output/my_model_SFT_13078 \ CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \ bash scripts/run_sft.sh ``` The script automatically injects your paths into the config template at `sft/cold_start_qwen25vl_8gpu/config_cold_start_super_plus_lrs_8gpu_20260408.yaml`. If using a different number of GPUs or DeepSpeed config, modify the following in the template: - `model_name_or_path` → your `BASE_MODEL` path - `dataset_dir` → your `DATASET_DIR` path - `output_dir` → your `OUTPUT_DIR` path - `deepspeed` → path to your DeepSpeed config (ZeRO-3 recommended for full-parameter tuning) > **Note**: We do not release the SFT checkpoint. The SFT step must be run locally before proceeding to RTAL. Training on 8×A100-80GB takes approximately 2 days. ## RTAL RTAL (Reinforced Tool-Attention Learning) starts from the SFT checkpoint and uses key-mask + valid-mask dual JSD loss with rollout `n=8`. The training data consists of parquet files with the following schema: prompt (with tool-call format), response, and reward scores. ### Step 1: Prepare Data Convert the GeoMTVR dataset into the parquet format expected by the [verl](https://github.com/volcengine/verl) framework: ``` /path/to/rtal_data/ train.parquet # Training set with tool-interleaved prompts and responses val.parquet # Validation set ``` Each parquet row must contain at least `prompt` and `response` columns. Refer to the data bundle structure in `rtal/RTAL_dl_final/repro_supervqa_rl748/data_bundles/` for the exact schema. ### Step 2: Run RTAL ```bash BASE_MODEL=/path/to/output/my_model_SFT_13078 \ TRAIN_PARQUET=/path/to/rtal_data/train.parquet \ VAL_PARQUET=/path/to/rtal_data/val.parquet \ OUTPUT_DIR=/path/to/rtal_outputs \ CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \ PYTHON_BIN=/path/to/python \ bash scripts/run_rtal.sh ``` Key hyper-parameters (config: `rtal/RTAL_dl_final/repro_supervqa_rl748/config_supervqa_rl748_sft13078_rtal_dualloss_washed_super8066_lrsgro1913.yaml`): - `attention_jsd_loss_coef_key_mask=0.01` - `attention_jsd_loss_coef_valid_mask=0.01` - `actor_rollout_ref.rollout.n=8` - `max_prompt_length=8192` - `max_response_length=1024` > **Note**: If you prefer to skip SFT and directly run RTAL, you can use our released GeoLens checkpoint as a starting point: set `BASE_MODEL` to the downloaded GeoLens model path. ## Evaluation ### XLRS-Bench ```bash DATASET_GLOB='/path/to/XLRS-Bench-lite_part*.jsonl' \ PYTHON_BIN=/path/to/python \ bash scripts/run_eval.sh /path/to/hf_model ``` Default generation uses: ``` max_new_tokens=8, temperature=0, do_sample=false, choice_constrained=true, choice_allow_space=true ``` Score recomputation: ```bash bash scripts/recompute_xlrs_scores.sh \ --samples_jsonl /path/to/*_samples_xlrs-lite-local-custom.jsonl \ --results_json /path/to/*_results.json \ --model_name MODEL_NAME \ --summary_only ``` ### RSHR Evaluation code is in `evaluation/RSHR_eval/`. Download the benchmark images first: ```bash cd evaluation/RSHR_eval python download_xhrbench_images.py --save_dir /path/to/rshr_images ``` Run evaluation on a HuggingFace model: ```bash python eval_qwen25vl_xhrbench.py \ --model_path /path/to/hf_model \ --save_dir /path/to/rshr_images \ --output_dir results/ ``` This evaluates MCQ, Open-Ended VQA, and Image Captioning tasks. See `evaluation/RSHR_eval/README.md` for details. ### LRS-GRO-eval Evaluation code is in `evaluation/LRSGRO_eval/`. First download the ZoomEarth-3B monitor: ```bash cd evaluation/LRSGRO_eval bash download_zoomearth3b_with_monitor.sh ``` Generate inference results: ```bash bash infer.sh /path/to/hf_model /path/to/output/inference.jsonl ``` Then compute accuracy: ```bash bash eval.sh /path/to/output/inference.jsonl ``` See `evaluation/LRSGRO_eval/run_scripts/` for batch evaluation scripts on multiple models. # 📊Results GeoLens achieves state-of-the-art performance on three benchmarks: | Benchmark | Metric | GeoLens | |-----------|--------|---------| | **XLRS-Bench** | Macro-13 Avg | **54.2%** | | **GRO-eval** | Accuracy | **60.7%** | | **RSHR** | Accuracy | **42.54%** | Results demonstrate that GeoLens consistently outperforms direct reasoning and single-tool zoom-in baselines, with stronger accuracy, better evidence grounding, and more efficient tool-use trajectories. # 🔗Citation If you find our work helpful, please consider citing: ```latex @article{wang2026geolens, title={Beyond Zooming: Learning Multi-Tool Visual Reasoning for Ultra-High-Resolution Remote Sensing}, author={Fengxiang Wang and Jiangnan Huang and Mingshuo Chen and Yueying Li and Yang Shi and Junwei Luo and Haoyu Wang and Yansheng Li and Jing Zhang and Haiyan Zhao and Wenjing Yang}, journal={arXiv preprint arXiv:2607.25993}, year={2026} } ``` # 🤝Acknowledgement * [Qwen2.5-VL](https://github.com/QwenLM/Qwen2.5-VL): The base model for GeoLens. * [XLRS-Bench](https://github.com/AI9Stars/XLRS-Bench): Evaluation benchmark for ultra-high-resolution remote sensing. * [verl](https://github.com/volcengine/verl): The RL training framework used for RTAL. * [GeoLLaVA-8K](https://github.com/initiavcms/GeoLLaVA-8K): Prior work on UHR RS MLLMs.