Add metadata, paper and code links to model card

README.md

@@ -1,3 +1,14 @@
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: image-text-to-text
+tags:
+- multimodal
+- vision-language
+- agent
+- deep-research
+---
+
 # WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent
 
 <p align="center">
@@ -5,88 +16,69 @@
 </p>
 
 ![version](https://img.shields.io/badge/version-1.0.0-blue)
+[![Paper](https://img.shields.io/badge/Paper-2508.05748-red)](https://huggingface.co/papers/2508.05748)
+[![GitHub](https://img.shields.io/badge/GitHub-Repo-green)](https://github.com/Alibaba-NLP/WebAgent)
+[![Project Page](https://img.shields.io/badge/Project-Page-blue)](https://tongyi-agent.github.io/blog/introducing-tongyi-deep-research/)
 
 ## 🥇 Introduction
 In this paper, we introduce **WebWatcher**, a multimodal agent for deep research that possesses enhanced visual-language reasoning capabilities. Our work presents a unified framework that combines complex vision-language reasoning with multi-tool interaction.
 
+This model was presented in the paper [WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent](https://huggingface.co/papers/2508.05748).
+
 <p align="center">
   <img src="./assets/webwatcher_main.png" alt="logo" width="80%"/>
 </p>
 
-
 Key features of our approach include:
 
 <p align="center">
   <img src="./assets/distribution_level.png" alt="logo" width="80%"/>
 </p>
 
-- BrowseComp-VL Benchmark: We propose a new benchmark, BrowseComp-VL, to evaluate the capabilities of multimodal agents. This challenging dataset is designed for in-depth multimodal reasoning and strategic planning, mirroring the complexity of BrowseComp but extending it into the visual domain. It emphasizes tasks that require both visual perception and advanced information-gathering abilities.
+- **BrowseComp-VL Benchmark**: We propose a new benchmark, BrowseComp-VL, to evaluate the capabilities of multimodal agents. This challenging dataset is designed for in-depth multimodal reasoning and strategic planning, mirroring the complexity of BrowseComp but extending it into the visual domain. It emphasizes tasks that require both visual perception and advanced information-gathering abilities.
 
 <p align="center">
   <img src="./assets/data_pipelines.png" alt="logo" width="80%"/>
 </p>
 
-- Automated Trajectory Generation: To provide robust tool-use capabilities, we developed an automated pipeline to generate high-quality, multi-step reasoning trajectories. These trajectories, which are grounded in actual tool-use behavior and reflect procedural decision-making, are used for efficient cold-start training and further optimization via reinforcement learning. The agent is equipped with several tools, including Web Image Search, Web Text Search, Webpage Visit, Code Interpreter, and an internal OCR tool.
+- **Automated Trajectory Generation**: To provide robust tool-use capabilities, we developed an automated pipeline to generate high-quality, multi-step reasoning trajectories. These trajectories, which are grounded in actual tool-use behavior and reflect procedural decision-making, are used for efficient cold-start training and further optimization via reinforcement learning. The agent is equipped with several tools, including Web Image Search, Web Text Search, Webpage Visit, Code Interpreter, and an internal OCR tool.
 
-- Superior Performance: WebWatcher significantly outperforms proprietary baselines, RAG workflows, and other open-source agents across four challenging VQA benchmarks: Humanity's Last Exam (HLE)-VL, BrowseComp-VL, LiveVQA, and MMSearch. The WebWatcher-32B model, in particular, achieves an average score of 18.2% on HLE, surpassing the GPT-4o-based OmniSearch baseline. It also achieves top-tier performance on LiveVQA (58.7%) and MMSearch (55.3%), demonstrating stable and superior results on demanding, real-world visual search benchmarks.
+- **Superior Performance**: WebWatcher significantly outperforms proprietary baselines, RAG workflows, and other open-source agents across four challenging VQA benchmarks: Humanity's Last Exam (HLE)-VL, BrowseComp-VL, LiveVQA, and MMSearch. The WebWatcher-32B model achieves top-tier performance, demonstrating stable and superior results on demanding, real-world visual search benchmarks.
 
 ## 🚀 Performance Highlights
 <p align="center">
   <img src="./assets/webwatcher_performance_general.png" alt="logo" width="80%"/>
 </p>
 
-1. Complex Reasoning (HLE-VL): On the Human Life Exam (HLE-VL), a benchmark for multi-step complex reasoning, WebWatcher achieved a commanding lead with a Pass@1 score of 13.6%, substantially outperforming representative models including GPT-4o (9.8%), Gemini2.5-flash (9.2%), and Qwen2.5-VL-72B (8.6%).
+1. **Complex Reasoning (HLE-VL)**: On the Human Life Exam (HLE-VL), WebWatcher achieved a Pass@1 score of 13.6%, substantially outperforming representative models including GPT-4o (9.8%) and Qwen2.5-VL-72B (8.6%).
 
-2. Information Retrieval (MMSearch): In the MMSearch evaluation, WebWatcher demonstrated exceptional retrieval accuracy with a Pass@1 score of 55.3%, significantly surpassing Gemini2.5-flash (43.9%) and GPT-4o (24.1%), showcasing superior precision in retrieval tasks and robust information aggregation capabilities in complex scenarios.
+2. **Information Retrieval (MMSearch)**: WebWatcher demonstrated exceptional retrieval accuracy with a Pass@1 score of 55.3%, significantly surpassing Gemini2.5-flash (43.9%) and GPT-4o (24.1%).
 
-3. Knowledge-Retrieval Integration (LiveVQA): On the LiveVQA benchmark, WebWatcher achieved a Pass@1 score of 58.7%, outperforming Gemini2.5-flash (41.3%), Qwen2.5-VL-72B (35.7%), and GPT-4o (34.0%).
-
-4. Information Optimization and Aggregation (BrowseComp-VL): On BrowseComp-VL, the most comprehensively challenging benchmark, WebWatcher dominated with an average score of 27.0%, more than doubling the performance of mainstream models including GPT-4o (13.4%), Gemini2.5-flash (13.0%), and Claude-3.7 (11.2%).
+3. **Knowledge-Retrieval Integration (LiveVQA)**: On the LiveVQA benchmark, WebWatcher achieved a Pass@1 score of 58.7%, outperforming Gemini2.5-flash (41.3%) and GPT-4o (34.0%).
 
+4. **Information Optimization and Aggregation (BrowseComp-VL)**: On BrowseComp-VL, WebWatcher dominated with an average score of 27.0%, more than doubling the performance of mainstream models including GPT-4o (13.4%).
 
 ## 🔧 Quick Start
 
-### Step 1: Download the WebWatcher model
-
-You can download WebWatcher via Hugging Face [🤗 HuggingFace](https://huggingface.co/Alibaba-NLP/).
-
-### Step 2: Data Preparation
-
-Before running inference, test set images need to be downloaded to the `infer/scripts_eval/images` folder. This can be accomplished by running `infer/scripts_eval/download_image.py`. If you encounter issues downloading images from our provided OSS URLs, please obtain the images from the original dataset source and place them in the corresponding `infer/scripts_eval/images` folder.
-
-### Step 3: Inference
-
-Run `infer/scripts_eval/scripts/eval.sh` with the following required parameters:
-
-- **benchmark**: Name of the dataset to test. Available options: `'hle'`, `'gaia'`, `'livevqa'`, `'mmsearch'`, `'simplevqa'`, `'bc_vl_v1'`, `'bc_vl_v2'`. These test sets should be pre-stored in `infer/vl_search_r1/eval_data` with naming convention like `hle.jsonl`. We have provided format examples for some datasets in `infer/vl_search_r1/eval_data`. If extending to new datasets, please ensure consistent formatting.
-- **EXPERIMENT_NAME**: Name for this experiment (user-defined)
-- **MODEL_PATH**: Path to the trained model
-- **DASHSCOPE_API_KEY**: GPT API key
-- **IMG_SEARCH_KEY**: Google SerpApi key for image search
-- **JINA_API_KEY**: Jina API key
-- **SCRAPERAPI_KEY**: Scraper API key
-- **QWEN_SEARCH_KEY**: Google SerpApi key for text search
-
-**Note**: For image search tools, if you need to upload searched images to OSS, the following are required:
-- **ALIBABA_CLOUD_ACCESS_KEY_ID**: Alibaba Cloud OSS access key ID
-- **ALIBABA_CLOUD_ACCESS_KEY_SECRET**: Alibaba Cloud OSS access key secret
-
-### Step 4: Evaluation
+The model can be used with the `transformers` library. For detailed instructions on running the agent with its associated tools (web search, OCR, etc.), please refer to the [official GitHub repository](https://github.com/Alibaba-NLP/WebAgent).
 
-Run `infer/vl_search_r1/pass3.sh` to use LLM-as-judge for evaluating Pass@3 and Pass@1 metrics. Parameters:
+### Inference
 
-- **DIRECTORY**: Path to the folder containing JSONL files generated from inference
-- **DASHSCOPE_API_KEY**: GPT API key
+Running inference typically involves using the provided scripts in the repository:
 
+1. **Download the WebWatcher model** via Hugging Face.
+2. **Data Preparation**: Download test set images to the `infer/scripts_eval/images` folder.
+3. **Run Inference**: Use `infer/scripts_eval/scripts/eval.sh` with the required parameters (MODEL_PATH, API keys for tools, etc.).
 
 ## 📑 Citation
 
 If this work is helpful, please kindly cite as:
 
-```bigquery
+```bibtex
 @article{geng2025webwatcher,
   title={WebWatcher: Breaking New Frontiers of Vision-Language Deep Research Agent},
   author={Geng, Xinyu and Xia, Peng and Zhang, Zhen and Wang, Xinyu and Wang, Qiuchen and Ding, Ruixue and Wang, Chenxi and Wu, Jialong and Zhao, Yida and Li, Kuan and others},
   journal={arXiv preprint arXiv:2508.05748},
   year={2025}
 }
+```