Image-to-Text
Transformers
English
gui-grounding
screen-understanding
vision-language-model
icon-detection
screenspot
visual-search
Instructions to use luisf-mc/gui-g2-3b-ccf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use luisf-mc/gui-g2-3b-ccf with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("image-to-text", model="luisf-mc/gui-g2-3b-ccf")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("luisf-mc/gui-g2-3b-ccf", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add side-by-side comparison image to model card
Browse files- .gitattributes +1 -0
- README.md +4 -0
- comparison.png +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
comparison.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -22,6 +22,10 @@ metrics:
|
|
| 22 |
|
| 23 |
A drop-in inference wrapper that improves GUI-G2-3B's icon-grounding accuracy by **+2.2pp on ScreenSpot-v2** at zero training cost. The base weights are unchanged; everything is in the inference pipeline.
|
| 24 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
## What this is
|
| 26 |
|
| 27 |
[GUI-G2-3B](https://huggingface.co/inclusionAI/GUI-G2-3B) (89.2% on ScreenSpot-v2) is a strong open-source 3B grounding model. Its main weakness is on small icons, where it lands at 80.5% vs 96.0% on text. We add a single inference-time technique -- Cursor-Centric Focusing (CCF) -- that wraps the base model with a coarse-then-refined prediction loop:
|
|
|
|
| 22 |
|
| 23 |
A drop-in inference wrapper that improves GUI-G2-3B's icon-grounding accuracy by **+2.2pp on ScreenSpot-v2** at zero training cost. The base weights are unchanged; everything is in the inference pipeline.
|
| 24 |
|
| 25 |
+

|
| 26 |
+
|
| 27 |
+
*Real samples from ScreenSpot-v2. Each tile shows the same instruction predicted by GUI-G2-3B alone (red X) and GUI-G2-3B + CCF (green check, inside the ground-truth bbox). The full per-sample run backing these picks is in [`benchmarks/demo_candidates.jsonl`](https://github.com/LufeMC/gui-g2-3b-ccf/blob/main/benchmarks/demo_candidates.jsonl) on the GitHub repo, so the picks are verifiable.*
|
| 28 |
+
|
| 29 |
## What this is
|
| 30 |
|
| 31 |
[GUI-G2-3B](https://huggingface.co/inclusionAI/GUI-G2-3B) (89.2% on ScreenSpot-v2) is a strong open-source 3B grounding model. Its main weakness is on small icons, where it lands at 80.5% vs 96.0% on text. We add a single inference-time technique -- Cursor-Centric Focusing (CCF) -- that wraps the base model with a coarse-then-refined prediction loop:
|
comparison.png
ADDED
|
Git LFS Details
|