README.md

---
title: GUI-Perturbed Baseline Results Viewer
emoji: 🔍
colorFrom: blue
colorTo: gray
sdk: docker
app_port: 8501
tags:
- GUI-Perturbed
- GUI-DR
- Domaine Randomization
pinned: false
short_description: Baseline results from 3 GUI grounding models using GUI-DR. 

---

# GUI-Perturbed: Baseline Results Viewer

We evaluated three 7B grounding models that share a base checkpoint but differ in their post-training: Qwen2.5-VL-7B, UI-TARS-1.5-7B, and GTA1-7B. Using a common base lets us attribute the differences to the training recipe rather than the architecture.

Each model ran against GUI-Perturbed, a dataset built by applying domain randomization (controlled, systematic variation) to real web pages preserved as MHTML archives. The variation runs along two independent axes:

- **Appearance:** original, precision (70% browser zoom), style, and text-shrink.
- **Instruction phrasing:** direct ("click the submit button") and relational ("click the button above the search bar").

In total the viewer holds 6,240 results per model. Our analysis on the results can be found in the technical write-up [here](https://www.fig.inc/blog/measuring-brittleness-in-gui-grounding-models-using-gui-perturbed/?utm_source=huggingface&utm_medium=readme&utm_campaign=guip-p2).

## How to read it

Filter by model, perturbation type (visual/instructional), outcome, and whether reasoning was applied. Each result shows the original webpage, the perturbed variant, the results of each model's click, and whether it landed on the correct element.

## The rest of the work

- [Technical Report](http://www.fig.inc/blog/gui-pertubed-breaking-browser-use-models/?utm_source=huggingface&utm_medium=readme&utm_campaign=guip-p2)
- [GUI-DR Augmentation Pipeline](https://github.com/ManifoldRG/GUI-DR)
- [GUI-Perturbed Dataset](https://huggingface.co/datasets/figai/GUI-Perturbed)

## Citation

```bibtex
@online{measuring_gui_models_robustness_technical_report_2026,
  title   = {Measuring Brittleness in GUI Grounding Models using GUI-Perturbed},
  author  = {Yangyue Wang and Harshvardhan Sikka and Yash Mathur and Tony Zhou and Jinu Nyachhyon and Pranav Guruprasad},
  year    = {2026},
  url     = {www.fig.inc/blog/measuring-brittleness-in-gui-grounding-models-using-gui-perturbed/},
  note    = {Part 2: Baseline evaluation}
}
```

```bibtex
@misc{wang2026guiperturbeddomainrandomizationreveals,
      title={GUI-Perturbed: Domain Randomization Reveals Systematic Brittleness in GUI Grounding Models}, 
      author={Yangyue Wang and Harshvardhan Sikka and Yash Mathur and Tony Zhou and Jinu Nyachhyon and Pranav Guruprasad},
      year={2026},
      eprint={2604.14262},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2604.14262}, 
}
```