README.md

---
license: mit
base_model: egeorcun/lucida
base_model_relation: quantized
library_name: mlx
tags:
  - mlx
  - mlx-swift
  - image-segmentation
  - background-removal
  - image-matting
  - birefnet
pipeline_tag: image-segmentation
---

# Lucida-fp16 (MLX)

[egeorcun/lucida](https://huggingface.co/egeorcun/lucida) converted for **MLX / MLX-Swift** on Apple
Silicon. Lucida is a fine-tune of [ZhengPeng7/BiRefNet_HR](https://huggingface.co/ZhengPeng7/BiRefNet_HR)
for foreground matting / background removal, tuned toward the cases most open matting models handle
worst: camouflaged subjects, transparency, text and logos, print design, and illustration.

Single tier at **1024×1024**, the resolution Lucida was trained at.

## What was converted

`model.safetensors` — 687 tensors, **fp16** (~440 MB). Produced from the upstream fp32 checkpoint with
the `birefnet-convert` tool in
[xocialize/mlx-birefnet-swift](https://github.com/xocialize/mlx-birefnet-swift):

- checkpoint prefixes stripped; PyTorch-only buffers dropped (`relative_position_index`,
  `num_batches_tracked` — 67 tensors)
- `decoder.conv_out1.0.{weight,bias}` → `decoder.conv_out1.{weight,bias}` (upstream wraps the final
  conv in an `nn.Sequential`; the Swift port uses a bare conv)
- every 4-D conv weight transposed from PyTorch NCHW `(O, I, kH, kW)` to MLX NHWC `(O, kH, kW, I)`
- cast to fp16

The **architecture is unchanged** from BiRefNet_HR — the converted key set is byte-identical to both
upstream BiRefNet bases (754 tensors in → 687 out, zero keys or shapes differing), so this loads
through the stock BiRefNet graph (`swin_v1_l`, `mul_scl_ipt=cat`, `cxt_num=3`, `ASPPDeformable`).

## Verification

Gated against a PyTorch oracle at 1024², with the oracle's exact normalized input tensor injected on
both sides (so the comparison measures the model, not preprocessing):

| | |
|---|---|
| encoder stages (CPU stream) | maxAbs ≤ 8.3e-5 relative to range, cosine 1.0000000 |
| e2e logits | cosine **0.99998** |
| e2e sigmoid matte | cosine **0.99977** |
| foreground fraction | **0.0292**, matching the oracle's 0.0292 exactly |

**fp16 is the right dtype here, and bf16 is not.** At the same gate bf16 fails (sigmoid cosine
0.9958, foreground fraction drifting to 0.0272) — its 8-bit mantissa is too coarse for this graph.
No bf16 variant is published for that reason.

The fp32 conversion is bit-exact against the source, so any residual difference above is Apple-GPU
fp32 accumulation, not conversion loss.

## Usage (MLX-Swift)

```swift
import MLXServeCore
import MLXBiRefNet

let engine = MLXServeEngine()
let id = try await engine.register(Lucida.registration, configuration: .lucida())
let resp = try await engine.run(MattingRequest(image: photo), package: id) as! MattingResponse
// resp.matte — soft-alpha grayscale PNG at the input's dimensions
```

Measured on an M-series Max: ~0.38 s per 1024² matte, ~0.6 GB resident + ~14 GB peak transient.

## Choosing between Lucida and BiRefNet

Lucida is **not** a drop-in replacement — the two lead on different material. On the author's own
203-image benchmark (MAE, lower is better):

| Lucida is better at | BiRefNet-HR is better at |
|---|---|
| camouflage 0.0270 vs 0.0752 · print/design 0.0235 vs 0.0544 · text & logos 0.0091 vs 0.0207 · illustration 0.0092 vs 0.0157 · transparency 0.0358 vs 0.0687 | hair 0.0048 vs 0.0093 · thin structures 0.0196 vs 0.0322 · complex edges 0.0385 vs 0.0484 |

Overall on that set: 0.0257 vs 0.0346. So route photographic portrait/hair work to BiRefNet and
non-photographic, hard-edge, text or illustration work here.

> Those figures are the **author's own** — 203 self-selected images, 9 self-defined categories, MAE
> only (which flatters soft-alpha smoothness), and the published table names "Lucida-v7" while the
> source repo ships one unversioned checkpoint. Treat as directional, not decisive.

Also note the author's **colour-decontamination and edge-refinement passes are not part of the
checkpoint** — they live in [their GitHub repo](https://github.com/egeorcun/lucida). This conversion
provides the network only, emitting raw sigmoid alpha.

## License & provenance

MIT, following upstream's declaration for both the weights and the code.

One thing to be aware of before commercial use: the upstream card states the training data
"mix[es] MIT-licensed and research-only datasets" — P3M-10k, COD10K and DIS5K, alongside the
CC-BY-4.0 [ToonOut](https://huggingface.co/datasets/joelseytre/toonout) set. That is a question about
the data the model was derived from rather than about the MIT grant we received, but it is worth your
own review rather than inheriting the label uncritically.

## Credits

- **Lucida** — [egeorcun](https://huggingface.co/egeorcun) ([code + benchmark](https://github.com/egeorcun/lucida))
- **BiRefNet** — [ZhengPeng7](https://huggingface.co/ZhengPeng7) ([paper](https://arxiv.org/abs/2401.03407), [code](https://github.com/ZhengPeng7/BiRefNet))
- **MLX-Swift port** — [xocialize/mlx-birefnet-swift](https://github.com/xocialize/mlx-birefnet-swift), whose vendored core derives from [mnmly/mlx-swift-BiRefNet](https://github.com/mnmly/mlx-swift-BiRefNet) (MIT)