mlx-community/Fara1.5-9B-8bit

microsoft/Fara1.5-9B converted to MLX and quantized to 8-bit, for inference on Apple Silicon.

Fara1.5 is a computer-use / web-agent vision-language model built on Qwen3.5 — it reads screenshots and acts on interfaces. The vision path is preserved in this conversion, because for this model class it is the point.

Quantization

Requested bits	8
Group size	64
Mode	affine
Effective bits per weight	8.86
On-disk size	9.8 GB
Shards	2

Effective bits exceed the requested value because mlx-vlm quantizes only the language model and leaves the vision tower in bf16 by design — 333 vision tensors, none of them quantized. The vision encoder is a small share of the weights but disproportionately sensitive to quantization error.

language_model : quantized (8-bit, group size 64)
vision_tower   : 333 tensors, bf16    <- unquantized

Measured results

All figures measured against the unquantized bf16 source, greedy decoding, on an M2 Pro / 32 GB. At 9B the bf16 control fits in memory, so this is a behavioural comparison, not a weight-level proxy.

Metric	bf16 (source)	8-bit
Perplexity	3.2339	3.2150
Perplexity ratio	1.000	0.994
Top-1 agreement	—	1.000
KL(bf16 ‖ quant)	0	0.000593 nats/tok
Task accuracy	8/8	8/8
Decode tok/s	10.2	19.7
Peak RAM	18.97 GB	11.91 GB

Top-1 agreement is 1.000 over 204 teacher-forced tokens — the 8-bit model picked the same argmax token as bf16 on every one. KL divergence is 5.9e-4 nats/token. At this fidelity, quantization is not the limiting factor in output quality.

At 8-bit the model runs 1.9x faster using 1.6x less memory than bf16.

Generation agreement (reported, but not a quality metric)

Metric	vs bf16
BLEU	62.05
chrF	66.9
ROUGE-1 / ROUGE-L	0.736 / 0.731
Exact match	4/6

BLEU against bf16 is not a quality measure. It treats bf16 output as ground truth, so it penalises valid paraphrase. The divergences here are exactly that — both models answer correctly, then differ in trailing filler:

prompt: "Name the capital of Japan in one word."
bf16 : "Tokyo ... I will answer the question directly without any extra content."
8-bit: "Tokyo ... The user asked for the capital of Japan in one word."

Same answer, different tail. That is why task accuracy is measured separately — and it is 8/8 for both.

Vision path

Verified working, not merely present. Given a synthetic UI screenshot with two buttons and a total, the 8-bit model returned:

"two large rectangular buttons side by side: On the left: a "SubmitOrder" button. On the right: a "Cancel" button. Below them, the total amount displayed is 42.50 USD."

Both button labels and the amount read correctly.

What was not measured

No standard benchmarks — no MMLU, GSM8K, ScreenSpot, WebArena, or any agentic evaluation. The accuracy layer is 8 short verifiable prompts, not a benchmark. No blind LLM-judge quality grading was run for this variant. The vision path was verified for correctness on a single synthetic screenshot, not scored on a dataset. If your use case is the full computer-use loop, evaluate on your own tasks.

Usage

pip install mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("mlx-community/Fara1.5-9B-8bit")

prompt = apply_chat_template(
    processor, model.config,
    "Describe this screenshot. What buttons do you see?",
    num_images=1,
)
out = generate(model, processor, prompt, image=["screenshot.png"], max_tokens=256)
print(out.text)

Text-only works too — pass num_images=0 and omit image.

Note that stock mlx-lm loads the text path only; use mlx-vlm for image input.

Credits

All credit for the model belongs to Microsoft. This is a format conversion and quantization; no training or fine-tuning was performed. Licensed MIT, as the original. See the original card for intended use and limitations.