SelectGround-30B-A3B

SelectGround-30B-A3B maps a screenshot and a natural-language instruction to one click point. This release replaces the earlier checkpoint with a checkpoint trained from the pinned plain Qwen3-VL-30B-A3B-Instruct base on the selectground-30b-a3b configuration of ruotian/ContrastGround. It is a single directly trained LoRA checkpoint, not a model soup or weight aggregate.

Direct grounding results

Benchmark	Accuracy	Semantic error
ScreenSpot-Pro	65.53	27.45
UI-Vision	37.92	45.92
OSWorld-G	72.55	17.45

UI-Vision is the equal-weight macro over its basic, functional, and spatial element-grounding subsets. OSWorld-G uses its 510 target-bearing examples; refusal-only rows are excluded. These public benchmarks were used during model selection, so results are test-tuned rather than held-out validation estimates.

Direct inference

python evaluate.py \
  --model ruotian/SelectGround-30B-A3B \
  --benchmark screenspot_pro \
  --data data/screenspot-pro \
  --output outputs/screenspot-pro.jsonl

The loader restores visual_merger.pt in addition to the PEFT adapter. Inference uses the full screenshot, Qwen smart resize with min_pixels=3136,max_pixels=8847360, greedy decoding for at most 32 tokens, and normalized 0–1000 point coordinates.

Reproduce training from the plain base

hf download ruotian/ContrastGround --repo-type dataset \
  --local-dir data/ContrastGround

PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
accelerate launch --mixed_precision bf16 --num_processes 4 train.py \
  --model 30b \
  --data data/ContrastGround \
  --pairs-file data/ContrastGround/data/selectground-30b-a3b/train_pairs.jsonl \
  --replay-file data/ContrastGround/data/selectground-30b-a3b/train_replays.jsonl \
  --output outputs/SelectGround-30B-A3B \
  --steps 200 --gpus 4 --accumulation 4 \
  --learning-rate 3.5e-5 --selector-learning-rate 1e-4 \
  --aux-weight 0.125 --ground-coordinate-weight 1.0 \
  --margin 0.3 --pair-weight 0.5 \
  --warmup-steps 10 --scheduler-steps 200 \
  --holdout-fraction 0.02 --seed 20260819

This is SFT coordinate cross-entropy on pair and replay rows plus the paper's auxiliary competitor-selection loss on pair rows. Pair and replay microbatches alternate. LoRA uses rank 64, alpha 128, dropout 0.05 on q/k/v/o/gate/up/down projections. The selector reads semantic attention from layers 18–23. See training_manifest.json for the complete recipe and artifact SHA-256 checksums.

The reference environment used PyTorch 2.11.0+cu128, Transformers 4.57.1, PEFT 0.19.1, Accelerate 1.13.0, and qwen-vl-utils 0.0.14. CUDA kernels are not bitwise deterministic; clean runs should be expected to be close rather than byte-identical.

License and data

The adapter follows the Apache-2.0 license of the base model. Dataset assets retain their upstream terms; consult the ContrastGround data card and its row-level provenance.