Lucida-fp16 (MLX)

egeorcun/lucida converted for MLX / MLX-Swift on Apple Silicon. Lucida is a fine-tune of ZhengPeng7/BiRefNet_HR for foreground matting / background removal, tuned toward the cases most open matting models handle worst: camouflaged subjects, transparency, text and logos, print design, and illustration.

Single tier at 1024×1024, the resolution Lucida was trained at.

What was converted

model.safetensors — 687 tensors, fp16 (~440 MB). Produced from the upstream fp32 checkpoint with the birefnet-convert tool in xocialize/mlx-birefnet-swift:

• checkpoint prefixes stripped; PyTorch-only buffers dropped (relative_position_index, num_batches_tracked — 67 tensors)
• decoder.conv_out1.0.{weight,bias} → decoder.conv_out1.{weight,bias} (upstream wraps the final conv in an nn.Sequential; the Swift port uses a bare conv)
• every 4-D conv weight transposed from PyTorch NCHW (O, I, kH, kW) to MLX NHWC (O, kH, kW, I)
• cast to fp16

The architecture is unchanged from BiRefNet_HR — the converted key set is byte-identical to both upstream BiRefNet bases (754 tensors in → 687 out, zero keys or shapes differing), so this loads through the stock BiRefNet graph (swin_v1_l, mul_scl_ipt=cat, cxt_num=3, ASPPDeformable).

Verification

Gated against a PyTorch oracle at 1024², with the oracle's exact normalized input tensor injected on both sides (so the comparison measures the model, not preprocessing):

encoder stages (CPU stream)	maxAbs ≤ 8.3e-5 relative to range, cosine 1.0000000
e2e logits	cosine 0.99998
e2e sigmoid matte	cosine 0.99977
foreground fraction	0.0292, matching the oracle's 0.0292 exactly

fp16 is the right dtype here, and bf16 is not. At the same gate bf16 fails (sigmoid cosine 0.9958, foreground fraction drifting to 0.0272) — its 8-bit mantissa is too coarse for this graph. No bf16 variant is published for that reason.

The fp32 conversion is bit-exact against the source, so any residual difference above is Apple-GPU fp32 accumulation, not conversion loss.

Usage (MLX-Swift)

import MLXServeCore
import MLXBiRefNet

let engine = MLXServeEngine()
let id = try await engine.register(Lucida.registration, configuration: .lucida())
let resp = try await engine.run(MattingRequest(image: photo), package: id) as! MattingResponse
// resp.matte — soft-alpha grayscale PNG at the input's dimensions

Measured on an M-series Max: ~0.38 s per 1024² matte, ~0.6 GB resident + ~14 GB peak transient.

Choosing between Lucida and BiRefNet

Lucida is not a drop-in replacement — the two lead on different material. On the author's own 203-image benchmark (MAE, lower is better):

Lucida is better at	BiRefNet-HR is better at
camouflage 0.0270 vs 0.0752 · print/design 0.0235 vs 0.0544 · text & logos 0.0091 vs 0.0207 · illustration 0.0092 vs 0.0157 · transparency 0.0358 vs 0.0687	hair 0.0048 vs 0.0093 · thin structures 0.0196 vs 0.0322 · complex edges 0.0385 vs 0.0484

Overall on that set: 0.0257 vs 0.0346. So route photographic portrait/hair work to BiRefNet and non-photographic, hard-edge, text or illustration work here.

Those figures are the author's own — 203 self-selected images, 9 self-defined categories, MAE only (which flatters soft-alpha smoothness), and the published table names "Lucida-v7" while the source repo ships one unversioned checkpoint. Treat as directional, not decisive.

Also note the author's colour-decontamination and edge-refinement passes are not part of the checkpoint — they live in their GitHub repo. This conversion provides the network only, emitting raw sigmoid alpha.

License & provenance

MIT, following upstream's declaration for both the weights and the code.

One thing to be aware of before commercial use: the upstream card states the training data "mix[es] MIT-licensed and research-only datasets" — P3M-10k, COD10K and DIS5K, alongside the CC-BY-4.0 ToonOut set. That is a question about the data the model was derived from rather than about the MIT grant we received, but it is worth your own review rather than inheriting the label uncritically.

Credits

• Lucida — egeorcun (code + benchmark)
• BiRefNet — ZhengPeng7 (paper, code)
• MLX-Swift port — xocialize/mlx-birefnet-swift, whose vendored core derives from mnmly/mlx-swift-BiRefNet (MIT)