README.md

---
license: mit
library_name: mlx
pipeline_tag: image-text-to-text
language:
- en
base_model: microsoft/Fara1.5-9B
tags:
- computer-use
- cua
- web-agent
- multimodal
- vision-language
- agent
- browser-automation
- magentic
- fara
- mlx
---

# mlx-community/Fara1.5-9B-8bit

[microsoft/Fara1.5-9B](https://huggingface.co/microsoft/Fara1.5-9B) converted to
MLX and quantized to **8-bit**, for inference on Apple Silicon.

Fara1.5 is a computer-use / web-agent vision-language model built on Qwen3.5 — it
reads screenshots and acts on interfaces. **The vision path is preserved in this
conversion**, because for this model class it is the point.

## Quantization

| | |
|---|---|
| Requested bits | 8 |
| Group size | 64 |
| Mode | affine |
| **Effective bits per weight** | **8.86** |
| On-disk size | 9.8 GB |
| Shards | 2 |

Effective bits exceed the requested value because `mlx-vlm` quantizes only the
language model and leaves the **vision tower in bf16** by design — 333 vision
tensors, none of them quantized. The vision encoder is a small share of the
weights but disproportionately sensitive to quantization error.

```
language_model : quantized (8-bit, group size 64)
vision_tower   : 333 tensors, bf16    <- unquantized
```

## Measured results

All figures measured against the **unquantized bf16 source**, greedy decoding, on
an M2 Pro / 32 GB. At 9B the bf16 control fits in memory, so this is a
*behavioural* comparison, not a weight-level proxy.

| Metric | bf16 (source) | 8-bit |
|---|---|---|
| Perplexity | 3.2339 | 3.2150 |
| Perplexity ratio | 1.000 | **0.994** |
| Top-1 agreement | — | **1.000** |
| KL(bf16 ‖ quant) | 0 | **0.000593** nats/tok |
| Task accuracy | 8/8 | **8/8** |
| Decode tok/s | 10.2 | **19.7** |
| Peak RAM | 18.97 GB | **11.91 GB** |

**Top-1 agreement is 1.000 over 204 teacher-forced tokens** — the 8-bit model
picked the same argmax token as bf16 on every one. KL divergence is 5.9e-4
nats/token. At this fidelity, quantization is not the limiting factor in output
quality.

At 8-bit the model runs **1.9x faster** using **1.6x less memory** than bf16.

### Generation agreement (reported, but not a quality metric)

| Metric | vs bf16 |
|---|---|
| BLEU | 62.05 |
| chrF | 66.9 |
| ROUGE-1 / ROUGE-L | 0.736 / 0.731 |
| Exact match | 4/6 |

**BLEU against bf16 is not a quality measure.** It treats bf16 output as ground
truth, so it penalises valid paraphrase. The divergences here are exactly that —
both models answer correctly, then differ in trailing filler:

```
prompt: "Name the capital of Japan in one word."
bf16 : "Tokyo ... I will answer the question directly without any extra content."
8-bit: "Tokyo ... The user asked for the capital of Japan in one word."
```

Same answer, different tail. That is why task accuracy is measured separately —
and it is 8/8 for both.

### Vision path

Verified working, not merely present. Given a synthetic UI screenshot with two
buttons and a total, the 8-bit model returned:

> *"two large rectangular buttons side by side: On the left: a "SubmitOrder"
> button. On the right: a "Cancel" button. Below them, the total amount displayed
> is 42.50 USD."*

Both button labels and the amount read correctly.

## What was not measured

No standard benchmarks — no MMLU, GSM8K, ScreenSpot, WebArena, or any agentic
evaluation. The accuracy layer is 8 short verifiable prompts, not a benchmark. No
blind LLM-judge quality grading was run for this variant. The vision path was
verified for correctness on a single synthetic screenshot, not scored on a
dataset. **If your use case is the full computer-use loop, evaluate on your own
tasks.**

## Usage

```bash
pip install mlx-vlm
```

```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("mlx-community/Fara1.5-9B-8bit")

prompt = apply_chat_template(
    processor, model.config,
    "Describe this screenshot. What buttons do you see?",
    num_images=1,
)
out = generate(model, processor, prompt, image=["screenshot.png"], max_tokens=256)
print(out.text)
```

Text-only works too — pass `num_images=0` and omit `image`.

Note that stock `mlx-lm` loads the **text path only**; use `mlx-vlm` for image
input.

## Credits

All credit for the model belongs to Microsoft. This is a format conversion and
quantization; no training or fine-tuning was performed. Licensed MIT, as the
original. See the [original card](https://huggingface.co/microsoft/Fara1.5-9B)
for intended use and limitations.