Post
483
MK-SH-V: A 260M Visual Decision Model
I've released MK-SH-V, an open-source visual decision model built on H Company's NeoMME-260M.
Instead of generating text, it scores candidate answers from screenshots and documents, returning probabilities, a decision or an abstention.
Key results:
82.6% GUI accuracy, compared with 73.1% for a 4B zero-shot VLM.
+29.4 percentage points in real-image accuracy after incorporating real training data.
86% coverage* with a 95% selective-risk upper bound below 5% on held-out synthetic data.
The limitations are documented too. Receipt accuracy remains weak, and calibrated abstention does not yet generalize reliably to real images.
The entire experimental process, code and benchmarks are available under Apache 2.0.
Model: Gjusev/MK-SH-V
GitHub: https://github.com/Gjusev/mk-sh-v
Feedback and contributions are welcome!
I've released MK-SH-V, an open-source visual decision model built on H Company's NeoMME-260M.
Instead of generating text, it scores candidate answers from screenshots and documents, returning probabilities, a decision or an abstention.
Key results:
82.6% GUI accuracy, compared with 73.1% for a 4B zero-shot VLM.
+29.4 percentage points in real-image accuracy after incorporating real training data.
86% coverage* with a 95% selective-risk upper bound below 5% on held-out synthetic data.
The limitations are documented too. Receipt accuracy remains weak, and calibrated abstention does not yet generalize reliably to real images.
The entire experimental process, code and benchmarks are available under Apache 2.0.
Model: Gjusev/MK-SH-V
GitHub: https://github.com/Gjusev/mk-sh-v
Feedback and contributions are welcome!