|
Download eval/tables.md from KeytechSRL/TernaryPilot-v2-Computer-Browser-Use-LoRA: direct link, hf CLI and curl.
- Browser
- Download file 3.15 kB
-
https://huggingface.co/KeytechSRL/TernaryPilot-v2-Computer-Browser-Use-LoRA/resolve/main/eval/tables.md
- Command line
-
hf download hf://KeytechSRL/TernaryPilot-v2-Computer-Browser-Use-LoRA/eval/tables.md
-
curl -L -o tables.md https://huggingface.co/KeytechSRL/TernaryPilot-v2-Computer-Browser-Use-LoRA/resolve/main/eval/tables.md
3.15 kB
v2 glance
| benchmark | cases | base model | TernaryPilot v2 | gain |
|---|---|---|---|---|
| ScreenSpot-v2 | 1,272 | 46.5% | 60.7% | +14.2 |
| ScreenSpot-v2, desktop + web only | 771 | 37.0% | 71.5% | +34.5 |
| ScreenSpot-Pro | 423 | 1.7% | 11.1% | +9.4 |
| GUI-360 grounding | 2,000 | 16.9% | 66.4% | +49.5 |
| GUI-360 actions | 1,192 | 0.5% | 42.2% | +41.7 |
v1 page
| benchmark | cases | base model | v1 | v2 |
|---|---|---|---|---|
| ScreenSpot-v2 | 1,272 | 46.5% | 80.5% | 60.7% |
| ScreenSpot-Pro | 423 | 1.7% | 6.4% | 11.1% |
| GUI-360 grounding | 2,000 | 16.9% | 42.2% | 66.4% |
| GUI-360 actions | 1,192 | 0.5% | 17.3% | 42.2% |
ss2 by platform and type
| platform / type | cases | base | v2 |
|---|---|---|---|
| mobile-text | 290 | 68.3% | 52.8% |
| web-text | 234 | 40.6% | 82.9% |
| mobile-icon | 211 | 51.7% | 32.2% |
| web-icon | 203 | 30.5% | 66.5% |
| desktop-text | 194 | 39.2% | 73.7% |
| desktop-icon | 140 | 37.1% | 56.4% |
sspro by application group
| application group | cases | base | v2 |
|---|---|---|---|
| Creative | 195 | 1.0% | 9.7% |
| Dev | 80 | 1.2% | 7.5% |
| Office | 64 | 1.6% | 14.1% |
| Scientific | 50 | 6.0% | 26.0% |
| CAD | 34 | 0.0% | 0.0% |
sspro by target type
| target | cases | base | v2 |
|---|---|---|---|
| text | 269 | 2.2% | 14.9% |
| icon | 154 | 0.6% | 4.5% |
g360g by app
| app | cases | base | v2 |
|---|---|---|---|
| word | 848 | 24.4% | 71.0% |
| excel | 620 | 9.7% | 63.2% |
| ppt | 532 | 13.2% | 62.8% |
g360a by action
| action | cases | base | v2 |
|---|---|---|---|
| click | 927 | 0.0% | 43.4% |
| type | 243 | 0.0% | 37.9% |
| wheel_mouse_input | 20 | 30.0% | 45.0% |
| drag | 2 | 0.0% | 0.0% |
g360a by app
| app | cases | base | v2 |
|---|---|---|---|
| word | 491 | 0.2% | 44.2% |
| excel | 371 | 0.3% | 38.8% |
| ppt | 330 | 1.2% | 43.0% |
decider public
| set | language | questions | base | TernaryDecider v1 | gain |
|---|---|---|---|---|---|
| BoolQ (validation) | en | 1,000 | 83.2% | 89.8% | +6.6 |
| Civil Comments, toxic? (test) | en | 1,000 | 80.6% | 91.3% | +10.7 |
| MASSIVE intent, 8 options (test) | en | 1,000 | 91.2% | 97.5% | +6.3 |
| MASSIVE intent, 8 options (test) | it | 1,000 | 88.5% | 96.7% | +8.2 |
| PAWS-X (test) | de | 1,000 | 73.1% | 89.8% | +16.7 |
| PAWS-X (test) | en | 1,000 | 78.9% | 93.5% | +14.6 |
| PAWS-X (test) | es | 1,000 | 74.8% | 89.6% | +14.8 |
| PAWS-X (test) | fr | 1,000 | 73.4% | 89.8% | +16.4 |
| SQuAD 2.0, answerable? (validation) | en | 1,000 | 83.1% | 90.2% | +7.1 |
| XNLI (test) | de | 1,000 | 72.2% | 82.3% | +10.1 |
| XNLI (test) | en | 1,000 | 79.5% | 88.3% | +8.8 |
| XNLI (test) | es | 1,000 | 74.8% | 83.6% | +8.8 |
| XNLI (test) | fr | 1,000 | 71.3% | 82.5% | +11.2 |
| all 13 sets | 5 languages | 13,000 | 78.8% | 89.6% | +10.8 |
public overall base 78.8 lora 89.6 n 13000