Add model card and metadata
#1
by nielsr HF Staff - opened
README.md
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: transformers
|
| 3 |
+
pipeline_tag: image-text-to-text
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# UniDoc-RL-7B
|
| 7 |
+
|
| 8 |
+
[UniDoc-RL](https://huggingface.co/papers/2604.14967) is a unified reinforcement learning framework for **visual document Retrieval-Augmented Generation (RAG)**, where a Large Vision-Language Model (LVLM) agent jointly performs retrieval, reranking, active visual perception, and reasoning within a single decision process.
|
| 9 |
+
|
| 10 |
+
This repository contains the 7B model checkpoint, based on the **Qwen2.5-VL** architecture.
|
| 11 |
+
|
| 12 |
+
## Overview
|
| 13 |
+
|
| 14 |
+
UniDoc-RL formulates visual information acquisition as a sequential decision-making problem with a hierarchical action space. Specifically, it progressively refines visual evidence from coarse-grained document retrieval to fine-grained image selection and active region cropping, allowing the model to suppress irrelevant content and attend to information-dense regions (such as charts, tables, and dense text).
|
| 15 |
+
|
| 16 |
+
Key features include:
|
| 17 |
+
- **Hierarchical Action Space**: The model uses structured actions such as `<search>`, `<select>`, `<bbox>`, and `<answer>`.
|
| 18 |
+
- **Progressive Evidence Acquisition**: Refines evidence from page-level retrieval to fine-grained region inspection.
|
| 19 |
+
- **RL Alignment**: Trained using Group Relative Policy Optimization (GRPO) with a dense multi-reward scheme to align behavior across multiple objectives.
|
| 20 |
+
|
| 21 |
+
## Resources
|
| 22 |
+
|
| 23 |
+
- **Paper**: [UniDoc-RL: Coarse-to-Fine Visual RAG with Hierarchical Actions and Dense Rewards](https://huggingface.co/papers/2604.14967)
|
| 24 |
+
- **Code**: [GitHub - deepglint/UniDoc-RL](https://github.com/deepglint/UniDoc-RL)
|
| 25 |
+
- **Dataset**: [DeepGlint-AI/UniDoc-RL](https://huggingface.co/datasets/DeepGlint-AI/UniDoc-RL)
|
| 26 |
+
|
| 27 |
+
## Citation
|
| 28 |
+
|
| 29 |
+
```bibtex
|
| 30 |
+
@misc{unidocrl2026,
|
| 31 |
+
title={UniDoc-RL: Coarse-to-Fine Visual RAG with Hierarchical Actions and Dense Rewards},
|
| 32 |
+
author={Jun Wang and Shuo Tan and Zelong Sun and Tiancheng Gu and Yongle Zhao and Ziyong Feng and Kaicheng Yang and Cewu Lu},
|
| 33 |
+
year={2026},
|
| 34 |
+
note={Project page and paper link: https://huggingface.co/papers/2604.14967}
|
| 35 |
+
}
|
| 36 |
+
```
|