---
license: apache-2.0
library_name: mlx
pipeline_tag: image-text-to-text
base_model: mlx-community/Qwen3.5-0.8B-OptiQ-4bit
tags:
- mlx
- optiq
- vision
- vlm
- lora
- chartqa
- apple-silicon
---

# chartreader-0.8B-OptiQ-4bit

**A 0.8B vision-language model that reads charts, fine-tuned on a Mac.**

This is the OptiQ 4-bit quant of `Qwen3.5-0.8B` bundled with **ChartReader**, an
image+text LoRA trained on ChartQA. The vision tower stays frozen; the LoRA
adapts the language tower to answer chart questions the way ChartQA wants:
short, exact, just the value. The base + sidecar keep full image+text
inference, so the same repo works for general VLM tasks too.

## Results

80 held-out ChartQA questions, base vs. base + ChartReader, images letterboxed
to a 512 px canvas.

| Metric | Base 0.8B | + ChartReader | Δ |
|---|---|---|---|
| Relaxed accuracy | 50.0% | 55.0% | +5.0 pp |
| **Exact match** | 26.2% | **40.0%** | **+13.8 pp** |
| Output similarity | 0.385 | 0.598 | +0.21 |

The base reads charts loosely and verbosely ("There are 10 food items shown in
the bar graph"); ChartReader answers concisely ("3"). The big win is format and
exact-match; the relaxed-accuracy gain is smaller but real.

## Files

```
config.json, model.safetensors        the base Qwen3.5-0.8B OptiQ-4bit quant
optiq_vision.safetensors              vision sidecar (full image+text inference)
mtp.safetensors                       multi-token-prediction draft head
adapters/chartreader/                 the ChartQA LoRA (adapters.safetensors)
```

The LoRA is **not merged into the weights** — it rides alongside and applies at
serve time, so you keep the plain base quant plus the ChartReader behavior.

## Use

```bash
pip install mlx-optiq
huggingface-cli download mlx-community/chartreader-0.8B-OptiQ-4bit --local-dir ./chartreader

optiq serve --model ./chartreader --adapter ./chartreader/adapters/chartreader
```

Then send an image + a chart question to the OpenAI-compatible endpoint on
`localhost:8080`. Without the `--adapter`, the same repo serves as the plain
image+text base model.

## How it was made

Image+text LoRA on the language tower, vision tower frozen:

```bash
optiq lora train mlx-community/Qwen3.5-0.8B-OptiQ-4bit \
    --vision --data ./chartqa/train.jsonl \
    --rank 8 --iters 800 --learning-rate 5e-5 --output ./chartreader
```

Trained on a 24 GB Apple Silicon Mac. Every image is letterboxed to a uniform
512 px canvas (uniform shape keeps training memory bounded), gradient
checkpointing fits the hybrid gated-delta backward, and gradient clipping +
a 5e-5 learning rate prevent the mode collapse short targets otherwise cause.
The whole flow — build the dataset, run the LoRA — is also available in the
OptiQ Lab.

Full write-up: [Fine-tuning a vision model on a Mac](https://mlx-optiq.com/blog/vision-lora-on-a-mac).

---

Built with [OptiQ](https://mlx-optiq.com) — mixed-precision quantization,
LoRA, and an OpenAI-compatible server for LLMs and VLMs on Apple Silicon.
`pip install mlx-optiq`.