README.md

---
license: mit
library_name: mlx
pipeline_tag: image-text-to-text
tags:
- mlx
- computer-use
- cua
- web-agent
- multimodal
- vision-language
- agent
base_model: microsoft/Fara1.5-4B
---

# Fara1.5-4B-8bit

8-bit MLX quantization of
[`microsoft/Fara1.5-4B`](https://huggingface.co/microsoft/Fara1.5-4B), a 4.5B
computer-use / web-agent VLM built on Qwen3.5-4B. It reads screenshots and acts on
interfaces. Converted with `mlx-vlm` 0.6.8 (`mlx` 0.32.0) for inference on Apple
Silicon.

Every prior MLX conversion of this model is 4-bit, so **8-bit was the open gap**.
Fidelity was measured against the unquantized bf16 source, not assumed.

```bash
pip install mlx-vlm
python -m mlx_vlm generate \
    --model mlx-community/Fara1.5-4B-8bit \
    --image screenshot.png \
    --prompt "Describe the buttons and the total shown in this screenshot." \
    --max-tokens 256
```

Image input needs `mlx-vlm`. Stock `mlx-lm` loads the text path only, which for a
computer-use model is not a smaller model but a broken one.

## Quantization

| | |
|---|---|
| Bits | 8 |
| Group size | 64 |
| Mode | affine |
| Effective bits/weight | **9.053** |
| Size on disk | 4.8 GB |
| Quantized tensors | 249 (language model) |
| Unquantized tensors | 297 (vision tower, bf16) |

Effective bits per weight exceeds the nominal 8 because `mlx-vlm` quantizes only
the language model and leaves the vision tower in bf16 by design. **The vision
path is preserved**, which for a computer-use model is the whole point: verified
at tensor level, of 297 vision tensors **zero** carry a `.scales` entry.

### A conversion trap worth knowing about

This model declares `dtype: bfloat16` at the top level of its config but
**`float32` in both `text_config` and `vision_config`**. `mlx_vlm.convert` honours
the sub-configs, so converting without an explicit `--dtype bfloat16` silently
keeps the scales, biases and the entire vision tower in fp32:

| | default | `--dtype bfloat16` |
|---|---|---|
| Effective bpw | 10.695 | **9.053** |
| Size on disk | 5.7 GB | **4.8 GB** |
| Peak RAM | 7.44 GB | **6.23 GB** |
| Weight SNR vs bf16 | 45.22 dB | **42.70 dB** |

This repo is the `--dtype bfloat16` build. The higher SNR of the fp32 build is
**not** better fidelity: it is a float32 reconstruction being compared against a
bf16 source, bought with 0.9 GB of precision that has no business in an 8-bit
repo. Rounding those scales to bf16 accounts for the entire 2.5 dB difference.

## Measured fidelity

Against the **unquantized bf16 source**, greedy decoding, on an M2 Pro / 32 GB.

| Metric | bf16 (source) | 8-bit |
|---|---|---|
| Weight SNR | n/a | **42.70 dB** |
| Relative L2 | n/a | 0.73% |
| Cosine similarity | n/a | 0.999973 |
| Worst single element | n/a | 0.003906 |
| Perplexity | 3.4974 | 3.4775 |
| Top-1 agreement | n/a | **0.9853** |
| KL (nats/token) | 0 | **0.00086** |
| Task accuracy | 7/8 | **7/8** |
| BLEU vs bf16 | n/a | 81.30 |
| Decode tok/s | 18.5 | **35.4** |
| Peak RAM | 9.23 GB | **6.15 GB** |

Weight-level metrics stream the source tensor by tensor from disk over all
4,204,789,760 quantized parameters. No prompts, no sampling, hardware-independent.

**42.70 dB is the expected value, and that is the point.** Six 8-bit affine
conversions at group size 64 across six different architectures (AREX-Turbo,
Fara-7B, Fara1.5-4B, GLM-4.1V, MagenticBrain, OvisOCR2) all land on
**42.66-42.70 dB**. A reading outside that band is a defect signal, which is
exactly how the fp32-scales issue above was caught.

Highest-error tensors, all in the linear-attention input projections:

```
rel_l2=0.00841  snr=41.50 dB  language_model.model.layers.25.linear_attn.in_proj_a
rel_l2=0.00839  snr=41.53 dB  language_model.model.layers.28.linear_attn.in_proj_a
rel_l2=0.00834  snr=41.57 dB  language_model.model.layers.26.linear_attn.in_proj_a
```

**1.9x faster, 1.5x less memory** than bf16.

### Vision path, verified working rather than merely present

Given a synthetic UI screenshot, the 8-bit model read both button labels
(`SubmitOrder`, `Cancel`) and the total (`42.50 USD`) correctly.

### On the perplexity aggregate

The harness reports aggregate perplexity as `exp(sum_nll/sum_tokens)` over five
passages, and flags it as unreliable here: the bf16 source itself spans **10.3x**
across those passages, so a single out-of-domain passage can dominate. The
per-passage ratios are reported instead, and they agree with the aggregate rather
than contradict it, all within 1.8% of the bf16 reference:

| Passage | bf16 PPL | 8-bit PPL | Ratio |
|---|---|---|---|
| prose | 4.66 | 4.69 | 1.007 |
| python-code | 1.23 | 1.23 | 0.999 |
| legal | 8.82 | 8.70 | 0.986 |
| biology | 6.62 | 6.64 | 1.002 |
| meta | 12.74 | 12.51 | 0.982 |

BLEU of 81.30 measures **drift, not quality**: it treats the bf16 output as ground
truth, so it penalises valid paraphrase and would penalise a better answer.

## What was not measured

- **No judged quality.** In the open-ended set 11/18 (bf16) and 14/18 (8-bit)
  answers hit the 640-token cap, so grading them would have measured truncation
  rather than quality. This model emits a `<think>` block, which consumes budget.
- **No agentic evaluation.** No WebArena, no ScreenSpot, no WebVoyager, no
  end-to-end computer-use task. This model's actual job is multi-step browser
  automation and none of that is scored here.
- **No standard benchmarks** (MMLU, GSM8K). The accuracy layer is 8 short
  verifiable prompts, not a benchmark.
- **The vision check is one screenshot**, not a dataset, and it was scored for
  correctness by reading it, not by a metric.
- Throughput figures are for one M2 Pro and do not transfer to other chips.

## Related conversions

| Model | Variant | Note |
|---|---|---|
| [Fara1.5-9B-8bit](https://huggingface.co/mlx-community/Fara1.5-9B-8bit) | 8.86 bpw | larger sibling, top-1 agreement 1.000 |
| [Fara-7B-8bit](https://huggingface.co/mlx-community/Fara-7B-8bit) | 9.11 bpw | previous generation |
| [Fara-7B-4bit](https://huggingface.co/mlx-community/Fara-7B-4bit) | 5.44 bpw | previous generation |

## Credits

All credit for the underlying model belongs to
[Microsoft](https://huggingface.co/microsoft). This is a format conversion and
quantization; no training or fine-tuning was performed. See the
[source model card](https://huggingface.co/microsoft/Fara1.5-4B) for license,
intended use and limitations.