Add pipeline tag, library name, and paper link to model card
Browse filesHi! I'm Niels, part of the community science team at Hugging Face.
This pull request improves the model card for this repository:
- Added the `image-text-to-text` pipeline tag for better discoverability on the Hub.
- Added `library_name: transformers` to enable automated code snippets.
- Added a link to the research paper on Hugging Face Papers.
- Included the citation information and relevant links to the GitHub repository and project page.
The model architecture (`Qwen3VLForConditionalGeneration`) indicates compatibility with the Transformers library.
README.md
CHANGED
|
@@ -1,12 +1,14 @@
|
|
| 1 |
---
|
| 2 |
-
|
|
|
|
| 3 |
datasets:
|
| 4 |
- GUI-Libra/GUI-Libra-81K-RL
|
| 5 |
- GUI-Libra/GUI-Libra-81K-SFT
|
| 6 |
language:
|
| 7 |
- en
|
| 8 |
-
|
| 9 |
-
|
|
|
|
| 10 |
tags:
|
| 11 |
- VLM
|
| 12 |
- GUI
|
|
@@ -15,12 +17,12 @@ tags:
|
|
| 15 |
|
| 16 |
# Introduction
|
| 17 |
|
| 18 |
-
|
| 19 |
-
|
| 20 |
|
| 21 |
-
**GitHub:** https://github.com/GUI-Libra/GUI-Libra
|
| 22 |
**Website:** https://GUI-Libra.github.io
|
| 23 |
|
|
|
|
| 24 |
|
| 25 |
# Usage
|
| 26 |
## 1) Start an OpenAI-compatible vLLM server
|
|
@@ -28,7 +30,7 @@ The models from paper "GUI-Libra: Training Native GUI Agents to Reason and Act w
|
|
| 28 |
```bash
|
| 29 |
pip install -U vllm
|
| 30 |
vllm serve GUI-Libra/GUI-Libra-4B --port 8000 --api-key token-abc123
|
| 31 |
-
```
|
| 32 |
|
| 33 |
* Endpoint: `http://localhost:8000/v1`
|
| 34 |
* The `api_key` here must match `--api-key`.
|
|
@@ -79,11 +81,17 @@ action_type: Scroll, action_target: None, value: "up" | "down" | "left" | "right
|
|
| 79 |
|
| 80 |
task_desc = 'Go to Amazon.com and buy a math book'
|
| 81 |
prev_txt = ''
|
| 82 |
-
question_description = '''Please generate the next move according to the UI screenshot {}, instruction and previous actions.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
img_size_string = '(original image size {}x{})'.format(img_size[0], img_size[1])
|
| 84 |
query = question_description.format(img_size_string, task_desc, prev_txt)
|
| 85 |
|
| 86 |
-
query = query + '
|
|
|
|
| 87 |
<thinking>Your step-by-step thought process here...</thinking>
|
| 88 |
<answer>
|
| 89 |
{
|
|
@@ -125,5 +133,16 @@ python minimal_infer.py
|
|
| 125 |
* If you hit OOM or slowdowns, reduce image size or run fewer concurrent requests.
|
| 126 |
* The example assumes your vLLM server is running locally on port `8000`.
|
| 127 |
|
| 128 |
-
|
| 129 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model:
|
| 3 |
+
- Qwen/Qwen3-VL-4B-Instruct
|
| 4 |
datasets:
|
| 5 |
- GUI-Libra/GUI-Libra-81K-RL
|
| 6 |
- GUI-Libra/GUI-Libra-81K-SFT
|
| 7 |
language:
|
| 8 |
- en
|
| 9 |
+
license: apache-2.0
|
| 10 |
+
pipeline_tag: image-text-to-text
|
| 11 |
+
library_name: transformers
|
| 12 |
tags:
|
| 13 |
- VLM
|
| 14 |
- GUI
|
|
|
|
| 17 |
|
| 18 |
# Introduction
|
| 19 |
|
| 20 |
+
This repository contains the weights for **GUI-Libra-4B**, a native GUI agent model presented in the paper [GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL](https://huggingface.co/papers/2602.22190).
|
|
|
|
| 21 |
|
| 22 |
+
**GitHub:** https://github.com/GUI-Libra/GUI-Libra
|
| 23 |
**Website:** https://GUI-Libra.github.io
|
| 24 |
|
| 25 |
+
GUI-Libra is a post-training framework that turns open-source VLMs into strong native GUI agents—models that see a screenshot, think step-by-step, and output an executable action, all within a single forward pass.
|
| 26 |
|
| 27 |
# Usage
|
| 28 |
## 1) Start an OpenAI-compatible vLLM server
|
|
|
|
| 30 |
```bash
|
| 31 |
pip install -U vllm
|
| 32 |
vllm serve GUI-Libra/GUI-Libra-4B --port 8000 --api-key token-abc123
|
| 33 |
+
```
|
| 34 |
|
| 35 |
* Endpoint: `http://localhost:8000/v1`
|
| 36 |
* The `api_key` here must match `--api-key`.
|
|
|
|
| 81 |
|
| 82 |
task_desc = 'Go to Amazon.com and buy a math book'
|
| 83 |
prev_txt = ''
|
| 84 |
+
question_description = '''Please generate the next move according to the UI screenshot {}, instruction and previous actions.
|
| 85 |
+
|
| 86 |
+
Instruction: {}
|
| 87 |
+
|
| 88 |
+
Interaction History: {}
|
| 89 |
+
'''
|
| 90 |
img_size_string = '(original image size {}x{})'.format(img_size[0], img_size[1])
|
| 91 |
query = question_description.format(img_size_string, task_desc, prev_txt)
|
| 92 |
|
| 93 |
+
query = query + '
|
| 94 |
+
' + '''The response should be structured in the following format:
|
| 95 |
<thinking>Your step-by-step thought process here...</thinking>
|
| 96 |
<answer>
|
| 97 |
{
|
|
|
|
| 133 |
* If you hit OOM or slowdowns, reduce image size or run fewer concurrent requests.
|
| 134 |
* The example assumes your vLLM server is running locally on port `8000`.
|
| 135 |
|
| 136 |
+
## Citation
|
| 137 |
+
|
| 138 |
+
```bibtex
|
| 139 |
+
@misc{yang2026guilibratrainingnativegui,
|
| 140 |
+
title={GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL},
|
| 141 |
+
author={Rui Yang and Qianhui Wu and Zhaoyang Wang and Hanyang Chen and Ke Yang and Hao Cheng and Huaxiu Yao and Baoling Peng and Huan Zhang and Jianfeng Gao and Tong Zhang},
|
| 142 |
+
year={2026},
|
| 143 |
+
eprint={2602.22190},
|
| 144 |
+
archivePrefix={arXiv},
|
| 145 |
+
primaryClass={cs.LG},
|
| 146 |
+
url={https://arxiv.org/abs/2602.22190},
|
| 147 |
+
}
|
| 148 |
+
```
|