---
license: mit
library_name: mlx
tags:
- mlx
- florence2
- omniparser
- ui
- apple-silicon
base_model: microsoft/OmniParser-v2.0
pipeline_tag: image-text-to-text
---

# OmniParser icon_caption — MLX

**MLX (bfloat16) conversion of [microsoft/OmniParser-v2.0](https://huggingface.co/microsoft/OmniParser-v2.0)'s `icon_caption`** — a Florence-2 fine-tuned on UI elements, for captioning interactive icons in screenshots. Runs on Apple Silicon via `mlx_vlm` with **no PyTorch**.

License: **MIT** (© Microsoft Corporation) — see `LICENSE`. This repo redistributes the original MIT-licensed weights converted to MLX format.

## Usage

Needs a small no-torch patch on transformers 5.x (register `florence2_language`, route the image processor to `CLIPImageProcessorPil`). See the conversion recipe at the bottom.

```python
# apply the florence2 no-torch patch first (see recipe), then:
from mlx_vlm import load, generate
model, processor = load("PlusMinus1/omniparser-icon-caption-mlx")
out = generate(model, processor, "<CAPTION>", image=["icon_crop.png"], max_tokens=20)
```

## Provenance
- Base: microsoft/Florence-2-base (MIT)
- Fine-tune: microsoft/OmniParser-v2.0 `icon_caption` (MIT)
- Conversion: `mlx_vlm.convert` (bfloat16) + a transformers-5.x no-torch compatibility patch.