README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-VL-3B-Instruct
language:
- en
- zh
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- qwen2.5-vl
- gui-agent
- swipe-synthesis
- mobile-agent
- reinforcement-learning
---

# GUISwiper 3B (RL)

GUISwiper is an RL-aligned GUI agent model for **human-like swipe execution**, introduced in the paper
[SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe Synthesis](https://arxiv.org/abs/2601.18305)
(ACM MM 2026 Oral).

This repository hosts the **final RL-aligned 3B checkpoint** (bfloat16), fine-tuned from
[`Qwen/Qwen2.5-VL-3B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) and
evaluated on [SwipeBench](https://huggingface.co/datasets/TSKGHS17/SwipeBench).

## Model Details

| Property        | Value                                        |
|-----------------|----------------------------------------------|
| Base model      | Qwen/Qwen2.5-VL-3B-Instruct                  |
| Parameters      | 3B (bfloat16, ~7 GB)                         |
| Training        | RL alignment   |
| Evaluation      | SwipeBench                                   |
| Input           | GUI screenshots / screen videos + instruction |
| Output          | Human-like swipe action (trajectory)        |
| Hardware        | NVIDIA GPUs (see paper for details)          |

## Usage

```python
import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor

repo_id = "drunksu/GUISwiper"

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    repo_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(repo_id)

# image (GUI screenshot) + instruction -> swipe trajectory
# (follow the prompt format in the SwipeGen repo for the full inference pipeline)
image = load_your_gui_screenshot()  # PIL.Image
messages = [{"role": "user", "content": [
    {"type": "image", "image": image},
    {"type": "text", "text": "Describe the swipe to perform here."},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=256)
print(processor.batch_decode(output, skip_special_tokens=True)[0])
```

> Requires `transformers >= 4.49.0`. See the [SwipeGen GitHub repo](https://github.com/TSKGHS17/SwipeGen)
> for the complete inference and evaluation pipeline.

## Download a Single File

```python
from huggingface_hub import hf_hub_download

path = hf_hub_download("drunksu/GUISwiper", "model-00001-of-00002.safetensors")
```

## Citation

```bibtex
@misc{swipegen2026,
  title  = {SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe Synthesis},
  author = {SwipeGen Team},
  journal = {arXiv preprint arXiv:2601.18305},
  year   = {2026},
  note   = {Code and models: \url{https://github.com/TSKGHS17/SwipeGen}}
}
```

If you use GUISwiper, please also reference the official repository:
<https://github.com/TSKGHS17/SwipeGen>.

## Links

- Paper: https://arxiv.org/abs/2601.18305
- Project / code: https://github.com/TSKGHS17/SwipeGen


## License & Disclaimer

The model weights are released under Apache-2.0, consistent with the base model
`Qwen2.5-VL-3B-Instruct`. Users should comply with the original license terms of Qwen2.5-VL
and use the model responsibly; outputs are generated by AI and may contain errors.