Commit ·
d29925c
1
Parent(s): 67b44c8
Update production Core ML model card
Browse files
README.md
CHANGED
|
@@ -7,64 +7,92 @@ tags:
|
|
| 7 |
- coreml
|
| 8 |
- ios
|
| 9 |
- inpainting
|
| 10 |
-
- lora
|
| 11 |
- stable-diffusion
|
| 12 |
---
|
| 13 |
|
| 14 |
-
# Clover Image Tiny Inpaint
|
| 15 |
|
| 16 |
-
Core ML
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
|
|
|
| 20 |
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
state in FP16. Shared tokenizer, text encoder, and VAE decoder resources are
|
| 25 |
-
hard-linked from the required main Clover installation instead of downloaded
|
| 26 |
-
again. `hq-v4-int8/adapter-schema.json` maps each small `.safetensors` style
|
| 27 |
-
into one of three independent state slots.
|
| 28 |
|
| 29 |
-
##
|
| 30 |
-
|
| 31 |
-
PyTorch/Core ML parity on deterministic inputs:
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
|
| 38 |
-
|
| 39 |
-
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
-
|
|
|
|
|
|
|
| 42 |
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
| Masked target MAE | 0.2510 | **0.2231** |
|
| 47 |
-
| Changed pixels outside the mask | 0 | **0** |
|
| 48 |
|
| 49 |
## Runtime contract
|
| 50 |
|
| 51 |
- Minimum OS: iOS 18
|
| 52 |
-
- Resolution: 512×512
|
| 53 |
-
- U-Net input:
|
| 54 |
-
-
|
| 55 |
-
-
|
| 56 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
## Citation
|
| 59 |
|
| 60 |
```bibtex
|
| 61 |
@software{lozadaperez2026cloverimagetinyinpaintcoreml,
|
| 62 |
author = {Lukas Lozada Perez},
|
| 63 |
-
title = {Clover Image Tiny Inpaint
|
| 64 |
year = {2026},
|
| 65 |
url = {https://huggingface.co/neonforestmist/Clover-Image-Tiny-Inpaint-CoreML}
|
| 66 |
}
|
| 67 |
```
|
| 68 |
|
| 69 |
-
Designed and developed independently by Lukas Lozada Perez.
|
| 70 |
-
the CreativeML Open RAIL-M license
|
|
|
|
|
|
| 7 |
- coreml
|
| 8 |
- ios
|
| 9 |
- inpainting
|
|
|
|
| 10 |
- stable-diffusion
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# Clover Image Tiny Inpaint — Core ML
|
| 14 |
|
| 15 |
+
The production Core ML conversion of
|
| 16 |
+
[`neonforestmist/Clover-Image-Tiny-Inpaint`](https://huggingface.co/neonforestmist/Clover-Image-Tiny-Inpaint)
|
| 17 |
+
for fully local 512 × 512 inpainting on iPhone and iPad. It uses the same
|
| 18 |
+
trained nine-channel inpainting checkpoint as the Diffusers release; Core ML
|
| 19 |
+
changes the execution format, not the learned weights.
|
| 20 |
|
| 21 |
+
Use this model with
|
| 22 |
+
[`neonforestmist/Clover-Image-Tiny-iOS`](https://github.com/neonforestmist/Clover-Image-Tiny-iOS).
|
| 23 |
+
The app downloads, verifies, and installs the resources automatically.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
+
## Recommended release
|
|
|
|
|
|
|
| 26 |
|
| 27 |
+
The production entry point is [`manifest-pipeline.json`](manifest-pipeline.json).
|
| 28 |
+
It describes a **1.79 GB**, two-stage FP16 Core ML pipeline for iOS 18 or newer.
|
| 29 |
+
Core ML owns the handoff between the stages, which reduces the lifetime of
|
| 30 |
+
large intermediate tensors without changing the U-Net calculation.
|
| 31 |
|
| 32 |
+
| Resource | Purpose | Size |
|
| 33 |
+
|---|---|---:|
|
| 34 |
+
| `pipeline-v1/UnetPipeline.mlmodelc` | Full nine-channel FP16 inpainting U-Net | 1.72 GB |
|
| 35 |
+
| `VAEEncoder.mlmodelc` | Encodes the masked source image | 68.5 MB |
|
| 36 |
+
| `manifest-pipeline.json` | Pinned paths, sizes, and SHA-256 checksums | — |
|
| 37 |
|
| 38 |
+
The separate Clover installation is still required. Its approximately
|
| 39 |
+
**994.9 MB** runtime provides the tokenizer, text encoder, and VAE decoder, so
|
| 40 |
+
this repository does not download duplicate copies of those components.
|
| 41 |
|
| 42 |
+
The repository also retains compressed and stateful research artifacts for
|
| 43 |
+
comparison. The shipping iOS app uses `manifest-pipeline.json`, not the older
|
| 44 |
+
`manifest.json` entry point.
|
|
|
|
|
|
|
| 45 |
|
| 46 |
## Runtime contract
|
| 47 |
|
| 48 |
- Minimum OS: iOS 18
|
| 49 |
+
- Resolution: 512 × 512
|
| 50 |
+
- U-Net sample input: Float16 `[1, 9, 64, 64]`
|
| 51 |
+
- Timestep input: Float16 `[1]`
|
| 52 |
+
- Text input: Float16 `[1, 768, 1, 77]`
|
| 53 |
+
- Noise prediction output: Float32 `[1, 4, 64, 64]`
|
| 54 |
+
- Channels: noisy latent (4) + mask (1) + masked-image latent (4)
|
| 55 |
+
- Mask semantics: white regenerates, black preserves
|
| 56 |
+
- Recommended scheduler: DPM-Solver++ multistep
|
| 57 |
+
- Recommended settings: 20 steps, guidance scale 6.0
|
| 58 |
+
|
| 59 |
+
The selected source region is replaced by neutral gray before VAE encoding,
|
| 60 |
+
which becomes zero after normalization to `[-1, 1]`. For a small mask, the iOS
|
| 61 |
+
runtime uses a focused 512 × 512 crop with surrounding source context and then
|
| 62 |
+
composites the result through the exact user mask. Pixels outside the mask are
|
| 63 |
+
copied directly from the source image.
|
| 64 |
+
|
| 65 |
+
## LoRA compatibility
|
| 66 |
+
|
| 67 |
+
LoRAs made for the regular Clover Create model target its four-channel U-Net
|
| 68 |
+
and cannot be applied directly to this nine-channel inpainting U-Net. To ship
|
| 69 |
+
an inpainting style, train or adapt it for the inpainting checkpoint and fuse
|
| 70 |
+
it before Core ML conversion.
|
| 71 |
+
|
| 72 |
+
## Validation
|
| 73 |
+
|
| 74 |
+
The production bundle is checked in three ways:
|
| 75 |
+
|
| 76 |
+
- every file is verified against the byte count and SHA-256 in the schema-v3 manifest;
|
| 77 |
+
- the compiled pipeline passes a native macOS Core ML prediction smoke test with finite output and the expected tensor shape;
|
| 78 |
+
- the complete app flow was exercised on a physical iPhone 15 running iOS 26.6 using a 30-step masked edit.
|
| 79 |
+
|
| 80 |
+
The physical-device run completed successfully in about 73 seconds in the
|
| 81 |
+
XCTest/debug validation environment. That timing is a release smoke test, not
|
| 82 |
+
a formal performance benchmark; first-run Core ML compilation and device
|
| 83 |
+
thermal state can materially change latency.
|
| 84 |
|
| 85 |
## Citation
|
| 86 |
|
| 87 |
```bibtex
|
| 88 |
@software{lozadaperez2026cloverimagetinyinpaintcoreml,
|
| 89 |
author = {Lukas Lozada Perez},
|
| 90 |
+
title = {Clover Image Tiny Inpaint Core ML},
|
| 91 |
year = {2026},
|
| 92 |
url = {https://huggingface.co/neonforestmist/Clover-Image-Tiny-Inpaint-CoreML}
|
| 93 |
}
|
| 94 |
```
|
| 95 |
|
| 96 |
+
Designed and developed independently by Lukas Lozada Perez. Model weights are
|
| 97 |
+
available under the CreativeML Open RAIL-M license. Inference runs completely
|
| 98 |
+
on device after installation.
|