license: mit
library_name: mlx
pipeline_tag: image-text-to-text
tags:
- mlx
- computer-use
- cua
- web-agent
- multimodal
- vision-language
- agent
base_model: microsoft/Fara1.5-4B
Fara1.5-4B-8bit
8-bit MLX quantization of
microsoft/Fara1.5-4B, a 4.5B
computer-use / web-agent VLM built on Qwen3.5-4B. It reads screenshots and acts on
interfaces. Converted with mlx-vlm 0.6.8 (mlx 0.32.0) for inference on Apple
Silicon.
Every prior MLX conversion of this model is 4-bit, so 8-bit was the open gap. Fidelity was measured against the unquantized bf16 source, not assumed.
pip install mlx-vlm
python -m mlx_vlm generate \
--model mlx-community/Fara1.5-4B-8bit \
--image screenshot.png \
--prompt "Describe the buttons and the total shown in this screenshot." \
--max-tokens 256
Image input needs mlx-vlm. Stock mlx-lm loads the text path only, which for a
computer-use model is not a smaller model but a broken one.
Quantization
| Bits | 8 |
| Group size | 64 |
| Mode | affine |
| Effective bits/weight | 9.053 |
| Size on disk | 4.8 GB |
| Quantized tensors | 249 (language model) |
| Unquantized tensors | 297 (vision tower, bf16) |
Effective bits per weight exceeds the nominal 8 because mlx-vlm quantizes only
the language model and leaves the vision tower in bf16 by design. The vision
path is preserved, which for a computer-use model is the whole point: verified
at tensor level, of 297 vision tensors zero carry a .scales entry.
A conversion trap worth knowing about
This model declares dtype: bfloat16 at the top level of its config but
float32 in both text_config and vision_config. mlx_vlm.convert honours
the sub-configs, so converting without an explicit --dtype bfloat16 silently
keeps the scales, biases and the entire vision tower in fp32:
| default | --dtype bfloat16 |
|
|---|---|---|
| Effective bpw | 10.695 | 9.053 |
| Size on disk | 5.7 GB | 4.8 GB |
| Peak RAM | 7.44 GB | 6.23 GB |
| Weight SNR vs bf16 | 45.22 dB | 42.70 dB |
This repo is the --dtype bfloat16 build. The higher SNR of the fp32 build is
not better fidelity: it is a float32 reconstruction being compared against a
bf16 source, bought with 0.9 GB of precision that has no business in an 8-bit
repo. Rounding those scales to bf16 accounts for the entire 2.5 dB difference.
Measured fidelity
Against the unquantized bf16 source, greedy decoding, on an M2 Pro / 32 GB.
| Metric | bf16 (source) | 8-bit |
|---|---|---|
| Weight SNR | n/a | 42.70 dB |
| Relative L2 | n/a | 0.73% |
| Cosine similarity | n/a | 0.999973 |
| Worst single element | n/a | 0.003906 |
| Perplexity | 3.4974 | 3.4775 |
| Top-1 agreement | n/a | 0.9853 |
| KL (nats/token) | 0 | 0.00086 |
| Task accuracy | 7/8 | 7/8 |
| BLEU vs bf16 | n/a | 81.30 |
| Decode tok/s | 18.5 | 35.4 |
| Peak RAM | 9.23 GB | 6.15 GB |
Weight-level metrics stream the source tensor by tensor from disk over all 4,204,789,760 quantized parameters. No prompts, no sampling, hardware-independent.
42.70 dB is the expected value, and that is the point. Six 8-bit affine conversions at group size 64 across six different architectures (AREX-Turbo, Fara-7B, Fara1.5-4B, GLM-4.1V, MagenticBrain, OvisOCR2) all land on 42.66-42.70 dB. A reading outside that band is a defect signal, which is exactly how the fp32-scales issue above was caught.
Highest-error tensors, all in the linear-attention input projections:
rel_l2=0.00841 snr=41.50 dB language_model.model.layers.25.linear_attn.in_proj_a
rel_l2=0.00839 snr=41.53 dB language_model.model.layers.28.linear_attn.in_proj_a
rel_l2=0.00834 snr=41.57 dB language_model.model.layers.26.linear_attn.in_proj_a
1.9x faster, 1.5x less memory than bf16.
Vision path, verified working rather than merely present
Given a synthetic UI screenshot, the 8-bit model read both button labels
(SubmitOrder, Cancel) and the total (42.50 USD) correctly.
On the perplexity aggregate
The harness reports aggregate perplexity as exp(sum_nll/sum_tokens) over five
passages, and flags it as unreliable here: the bf16 source itself spans 10.3x
across those passages, so a single out-of-domain passage can dominate. The
per-passage ratios are reported instead, and they agree with the aggregate rather
than contradict it, all within 1.8% of the bf16 reference:
| Passage | bf16 PPL | 8-bit PPL | Ratio |
|---|---|---|---|
| prose | 4.66 | 4.69 | 1.007 |
| python-code | 1.23 | 1.23 | 0.999 |
| legal | 8.82 | 8.70 | 0.986 |
| biology | 6.62 | 6.64 | 1.002 |
| meta | 12.74 | 12.51 | 0.982 |
BLEU of 81.30 measures drift, not quality: it treats the bf16 output as ground truth, so it penalises valid paraphrase and would penalise a better answer.
What was not measured
- No judged quality. In the open-ended set 11/18 (bf16) and 14/18 (8-bit)
answers hit the 640-token cap, so grading them would have measured truncation
rather than quality. This model emits a
<think>block, which consumes budget. - No agentic evaluation. No WebArena, no ScreenSpot, no WebVoyager, no end-to-end computer-use task. This model's actual job is multi-step browser automation and none of that is scored here.
- No standard benchmarks (MMLU, GSM8K). The accuracy layer is 8 short verifiable prompts, not a benchmark.
- The vision check is one screenshot, not a dataset, and it was scored for correctness by reading it, not by a metric.
- Throughput figures are for one M2 Pro and do not transfer to other chips.
Related conversions
| Model | Variant | Note |
|---|---|---|
| Fara1.5-9B-8bit | 8.86 bpw | larger sibling, top-1 agreement 1.000 |
| Fara-7B-8bit | 9.11 bpw | previous generation |
| Fara-7B-4bit | 5.44 bpw | previous generation |
Credits
All credit for the underlying model belongs to Microsoft. This is a format conversion and quantization; no training or fine-tuning was performed. See the source model card for license, intended use and limitations.