Building your own AI? Join the AI Owner's Club on Skool โ a community that owns its models instead of renting them.
Amaara Decide Vision v4 (GGUF)
This is a decision model, not a chatbot. It never generates text. It is given an image, a state, and a set of typed questions, and returns an answer plus calibrated probabilities for each option. Because it scores a fixed option set instead of sampling tokens, the probability it returns is a real probability you can threshold on, not a softmax over a vocabulary.
Served through llama-server on the /v1/systemone endpoint. /v1/chat/completions returns
HTTP 400 by design.
What changed in v4
v3 had an inverted result on geometry questions: giving it the correct image made accuracy worse than giving it a random image (โ0.125). A vision model that does worse with the right pixels is not using the pixels. v4 was trained specifically to fix that, and did:
| source | n | accuracy | chance | lift | wrong-image accuracy | vision delta | v3 delta |
|---|---|---|---|---|---|---|---|
| synthetic/charts | 150 | 0.927 | 0.245 | +0.681 | 0.273 | +0.653 | +0.582 |
| vision/ai2d | 150 | 0.907 | 0.250 | +0.657 | 0.680 | +0.227 | +0.250 |
| vision/intergps | 150 | 0.507 | 0.250 | +0.257 | 0.387 | +0.120 | โ0.125 |
| vision/mapqa/yesno | 150 | 0.880 | 0.500 | +0.380 | 0.553 | +0.327 | +0.275 |
| vision/rico-widget/element | 141 | 0.716 | 0.096 | +0.621 | 0.314 | +0.402 | +0.583 |
Overall accuracy: 584/741 = 0.788. Mean vision delta: +0.3457.
How these numbers were produced
The vision delta column is the measurement that matters. Each row is run twice over the same 150 rows and the same questions:
- once with the real image, and
- once with the image replaced by a different image from the same source.
A model that ignores the image scores identically in both conditions, so its delta is 0. The delta is therefore a direct measurement of how much the model actually uses the pixels, and it cannot be inflated by guessing the majority class.
Calibration moves the right way too: on charts the expected calibration error is 0.018 with real images and 0.328 with wrong images. Without the image the model is confidently wrong, which is the correct behaviour for a calibrated model.
Evaluation: 150 rows per source, drawn with a fixed seed, scored on held-out rows.
Honest limitations
- Geometry (intergps) is still weak at 0.507 accuracy, only +0.257 above chance. It is now directionally correct (positive delta) but not reliable. Do not use this model for geometry reasoning.
- UI element grounding regressed vs v3: delta 0.402 here versus 0.583 on v3, and accuracy 0.716 versus 0.773. If your workload is UI element localisation, v3 is the better model; v4 traded some of that away to fix geometry and mapqa.
ai2dis flat (0.907 accuracy, delta +0.227 vs +0.250 on v3) โ within noise at n=150.- The wrong-image ablation uses an image from the same source, so it controls for domain but not for every possible shortcut.
- No comparison against a reference vision decision model (Clef or otherwise) has been run. These numbers are absolute, not relative. The ablation above is a measurement of visual grounding, which is a different question from "is this better than model X".
This repo: full weights
Merged safetensors, no GGUF. Load with FastDecisionModel.from_pretrained(...).
For llama.cpp use the sibling amaara-decide-vision-GGUF repo (the mmproj there is mandatory).
The mmproj is a separate file. Without --mmproj the server loads only the language tower and
every image request fails; the failure surfaces as an HTTP 501 that many clients report as "no
answers".
Usage
llama-server \
-m amaara-decide-vision-v4-Q8_0.gguf \
--mmproj mmproj-vision-v4-F16.gguf \
-c 4096 -b 8192 -ub 4096 \
--image-min-tokens 1024 \
--port 8903 --host 127.0.0.1 -ngl 99
Two flags are not optional:
-b 8192โ an image plus prompt is ~2,955 tokens, which exceeds the default 2,048 physical batch. With the default the request fails with "input is too large to process".--image-min-tokens 1024โ Qwen-VL models need at least 1024 image tokens for grounding; below that accuracy degrades.
Then:
curl -s http://127.0.0.1:8903/v1/systemone \
-H 'Content-Type: application/json' \
-d '{
"state": "user is looking at a dashboard",
"images": ["data:image/png;base64,<...>"],
"questions": {
"q1": {"type": "choice",
"instructions": "Which chart type is shown?",
"criteria": {"bar": "vertical bars", "line": "a line series"}}
}
}'
Response:
{"answers": {"q1": {"type": "choice", "choice": "bar",
"probabilities": {"bar": 0.91, "line": 0.09}}}}
Training
Fine-tuned from unsloth/Qwen3.5-0.8B with a joint decision head, LoRA r=16, lr 2e-4, 3,000
steps over 18,668 decision items built from 20,341 rows (1,673 skipped). GPU: a single 16 GB
consumer card, 5.5 GB peak. Total training time 1 h 32 m.
The vision tower was frozen โ the v4 mmproj is 198 MB, byte-for-byte the same size as v3's, because only the language tower and joint head were adapted.
Licence
Apache-2.0. Base weights: Qwen3.5-0.8B (Apache-2.0).
Join the AI Owner's Club
We build these models in the open with a community of people who want to own their AI instead of renting it. If you're training your own models, running them on your own hardware, and refusing to accept that you have to pay per token forever, you'll fit right in.
- Downloads last month
- 201
