Instructions to use ruotian/SelectGround-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use ruotian/SelectGround-8B with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-VL-8B-Instruct") model = PeftModel.from_pretrained(base_model, "ruotian/SelectGround-8B") - Notebooks
- Google Colab
- Kaggle
Add files using upload-large-folder tool
Browse files- README.md +27 -0
- adapter_config.json +15 -0
- adapter_model.safetensors +3 -0
- selection_head.pt +3 -0
- visual_merger.pt +3 -0
README.md
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: peft
|
| 4 |
+
base_model: Qwen/Qwen3-VL-8B-Instruct
|
| 5 |
+
pipeline_tag: image-text-to-text
|
| 6 |
+
tags:
|
| 7 |
+
- gui-grounding
|
| 8 |
+
- computer-use
|
| 9 |
+
- qwen3-vl
|
| 10 |
+
datasets:
|
| 11 |
+
- ruotian/SelectGround-8B-Data
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# SelectGround-8B
|
| 15 |
+
|
| 16 |
+
SelectGround-8B is a GUI grounding model built from Qwen3-VL-8B-Instruct with standard SFT and a target–distractor selection loss. It returns one click point as `[x, y]` in normalized `[0, 1000]` coordinates.
|
| 17 |
+
|
| 18 |
+
This repository contains the LoRA checkpoint and the visual-merger state used for exact reproduction. `selection_head.pt` is the training-only layer/head aggregation state; inference does not add a router or residual module.
|
| 19 |
+
|
| 20 |
+
| Method | ScreenSpot-Pro | UI-Vision | OSWorld-G |
|
| 21 |
+
|---|---:|---:|---:|
|
| 22 |
+
| SelectGround-8B | 64.96 | 38.68 | 70.00 |
|
| 23 |
+
| + ConGround | 72.17 | 44.39 | 70.00 |
|
| 24 |
+
|
| 25 |
+
OSWorld-G excludes refusal cases. UI-Vision is the equal-weight mean over basic, functional, and spatial element grounding.
|
| 26 |
+
|
| 27 |
+
Code and usage are in the [SelectGround repository](https://github.com/zhangruotian/SelectGround). Training data are in [SelectGround-8B-Data](https://huggingface.co/datasets/ruotian/SelectGround-8B-Data).
|
adapter_config.json
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"base_model_name_or_path": "Qwen/Qwen3-VL-8B-Instruct",
|
| 3 |
+
"bias": "none",
|
| 4 |
+
"inference_mode": true,
|
| 5 |
+
"init_lora_weights": true,
|
| 6 |
+
"lora_alpha": 128,
|
| 7 |
+
"lora_dropout": 0.05,
|
| 8 |
+
"peft_type": "LORA",
|
| 9 |
+
"r": 64,
|
| 10 |
+
"revision": "0c351dd01ed87e9c1b53cbc748cba10e6187ff3b",
|
| 11 |
+
"target_modules": ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
|
| 12 |
+
"task_type": "CAUSAL_LM",
|
| 13 |
+
"use_dora": false,
|
| 14 |
+
"use_rslora": false
|
| 15 |
+
}
|
adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aa33306498c323fea5405798a8feded09f4e5dfda386dd412c48cf174d1581cd
|
| 3 |
+
size 349251816
|
selection_head.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:456e706f68b76c64f3a7db24ee2e220711d7b16adffee1f4e7b02b938c3f9d77
|
| 3 |
+
size 2586
|
visual_merger.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:125b9b37b243724cee09eb064650a6e14d23f186097738b5144170e870da2fab
|
| 3 |
+
size 80241309
|