neonforestmist commited on
Commit
d29925c
·
1 Parent(s): 67b44c8

Update production Core ML model card

Browse files
Files changed (1) hide show
  1. README.md +64 -36
README.md CHANGED
@@ -7,64 +7,92 @@ tags:
7
  - coreml
8
  - ios
9
  - inpainting
10
- - lora
11
  - stable-diffusion
12
  ---
13
 
14
- # Clover Image Tiny Inpaint HQ — Core ML
15
 
16
- Core ML resources for the high-quality Clover Image Tiny inpainting pipeline.
17
- The iOS 18 batch-one U-Net accepts `[1, 9, 64, 64]`, runs classifier-free
18
- guidance as two serial passes, and exposes 144 mutable Core ML states for exact
19
- runtime composition of up to three Clover styles.
 
20
 
21
- The production U-Net is under `hq-v4-int8/Unet.mlmodelc`. Its immutable
22
- weights use per-channel symmetric int8 compression, reducing the complete
23
- inpainting add-on from about 1.79 GB to **931 MB** while leaving mutable LoRA
24
- state in FP16. Shared tokenizer, text encoder, and VAE decoder resources are
25
- hard-linked from the required main Clover installation instead of downloaded
26
- again. `hq-v4-int8/adapter-schema.json` maps each small `.safetensors` style
27
- into one of three independent state slots.
28
 
29
- ## Validation
30
-
31
- PyTorch/Core ML parity on deterministic inputs:
32
 
33
- | Configuration | FP16 | Int8 production |
34
- |---|---:|---:|
35
- | Base HQ inpainting U-Net | 78.56 dB | **59.32 dB** |
36
- | Monet 0.70 + Pointillism 0.45 + Watercolor Anime 1.10 | 78.47 dB | **59.30 dB** |
37
 
38
- The release gate is 35 dB. The three-style result validates the exact
39
- block-concatenated LoRA sum rather than a UI-only approximation.
 
 
 
40
 
41
- Held-out 24-case inpainting quality relative to the previous Diffusers release:
 
 
42
 
43
- | Metric | Previous | HQ |
44
- |---|---:|---:|
45
- | Masked prompt CLIP similarity | 0.2642 | **0.2768** |
46
- | Masked target MAE | 0.2510 | **0.2231** |
47
- | Changed pixels outside the mask | 0 | **0** |
48
 
49
  ## Runtime contract
50
 
51
  - Minimum OS: iOS 18
52
- - Resolution: 512×512
53
- - U-Net input: noisy latent (4) + mask (1) + masked image latent (4)
54
- - Mask: white regenerates, black preserves
55
- - Maximum simultaneous styles: 3
56
- - Safety checker: not constructed by the Clover iOS pipeline
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
 
58
  ## Citation
59
 
60
  ```bibtex
61
  @software{lozadaperez2026cloverimagetinyinpaintcoreml,
62
  author = {Lukas Lozada Perez},
63
- title = {Clover Image Tiny Inpaint HQ Core ML},
64
  year = {2026},
65
  url = {https://huggingface.co/neonforestmist/Clover-Image-Tiny-Inpaint-CoreML}
66
  }
67
  ```
68
 
69
- Designed and developed independently by Lukas Lozada Perez. Open weights under
70
- the CreativeML Open RAIL-M license; inference runs completely on device.
 
 
7
  - coreml
8
  - ios
9
  - inpainting
 
10
  - stable-diffusion
11
  ---
12
 
13
+ # Clover Image Tiny Inpaint — Core ML
14
 
15
+ The production Core ML conversion of
16
+ [`neonforestmist/Clover-Image-Tiny-Inpaint`](https://huggingface.co/neonforestmist/Clover-Image-Tiny-Inpaint)
17
+ for fully local 512 × 512 inpainting on iPhone and iPad. It uses the same
18
+ trained nine-channel inpainting checkpoint as the Diffusers release; Core ML
19
+ changes the execution format, not the learned weights.
20
 
21
+ Use this model with
22
+ [`neonforestmist/Clover-Image-Tiny-iOS`](https://github.com/neonforestmist/Clover-Image-Tiny-iOS).
23
+ The app downloads, verifies, and installs the resources automatically.
 
 
 
 
24
 
25
+ ## Recommended release
 
 
26
 
27
+ The production entry point is [`manifest-pipeline.json`](manifest-pipeline.json).
28
+ It describes a **1.79 GB**, two-stage FP16 Core ML pipeline for iOS 18 or newer.
29
+ Core ML owns the handoff between the stages, which reduces the lifetime of
30
+ large intermediate tensors without changing the U-Net calculation.
31
 
32
+ | Resource | Purpose | Size |
33
+ |---|---|---:|
34
+ | `pipeline-v1/UnetPipeline.mlmodelc` | Full nine-channel FP16 inpainting U-Net | 1.72 GB |
35
+ | `VAEEncoder.mlmodelc` | Encodes the masked source image | 68.5 MB |
36
+ | `manifest-pipeline.json` | Pinned paths, sizes, and SHA-256 checksums | — |
37
 
38
+ The separate Clover installation is still required. Its approximately
39
+ **994.9 MB** runtime provides the tokenizer, text encoder, and VAE decoder, so
40
+ this repository does not download duplicate copies of those components.
41
 
42
+ The repository also retains compressed and stateful research artifacts for
43
+ comparison. The shipping iOS app uses `manifest-pipeline.json`, not the older
44
+ `manifest.json` entry point.
 
 
45
 
46
  ## Runtime contract
47
 
48
  - Minimum OS: iOS 18
49
+ - Resolution: 512 × 512
50
+ - U-Net sample input: Float16 `[1, 9, 64, 64]`
51
+ - Timestep input: Float16 `[1]`
52
+ - Text input: Float16 `[1, 768, 1, 77]`
53
+ - Noise prediction output: Float32 `[1, 4, 64, 64]`
54
+ - Channels: noisy latent (4) + mask (1) + masked-image latent (4)
55
+ - Mask semantics: white regenerates, black preserves
56
+ - Recommended scheduler: DPM-Solver++ multistep
57
+ - Recommended settings: 20 steps, guidance scale 6.0
58
+
59
+ The selected source region is replaced by neutral gray before VAE encoding,
60
+ which becomes zero after normalization to `[-1, 1]`. For a small mask, the iOS
61
+ runtime uses a focused 512 × 512 crop with surrounding source context and then
62
+ composites the result through the exact user mask. Pixels outside the mask are
63
+ copied directly from the source image.
64
+
65
+ ## LoRA compatibility
66
+
67
+ LoRAs made for the regular Clover Create model target its four-channel U-Net
68
+ and cannot be applied directly to this nine-channel inpainting U-Net. To ship
69
+ an inpainting style, train or adapt it for the inpainting checkpoint and fuse
70
+ it before Core ML conversion.
71
+
72
+ ## Validation
73
+
74
+ The production bundle is checked in three ways:
75
+
76
+ - every file is verified against the byte count and SHA-256 in the schema-v3 manifest;
77
+ - the compiled pipeline passes a native macOS Core ML prediction smoke test with finite output and the expected tensor shape;
78
+ - the complete app flow was exercised on a physical iPhone 15 running iOS 26.6 using a 30-step masked edit.
79
+
80
+ The physical-device run completed successfully in about 73 seconds in the
81
+ XCTest/debug validation environment. That timing is a release smoke test, not
82
+ a formal performance benchmark; first-run Core ML compilation and device
83
+ thermal state can materially change latency.
84
 
85
  ## Citation
86
 
87
  ```bibtex
88
  @software{lozadaperez2026cloverimagetinyinpaintcoreml,
89
  author = {Lukas Lozada Perez},
90
+ title = {Clover Image Tiny Inpaint Core ML},
91
  year = {2026},
92
  url = {https://huggingface.co/neonforestmist/Clover-Image-Tiny-Inpaint-CoreML}
93
  }
94
  ```
95
 
96
+ Designed and developed independently by Lukas Lozada Perez. Model weights are
97
+ available under the CreativeML Open RAIL-M license. Inference runs completely
98
+ on device after installation.