| --- |
| license: other |
| license_name: mixed |
| license_link: LICENSE.md |
| library_name: executorch |
| tags: |
| - executorch |
| - on-device |
| - image-restoration |
| - inpainting |
| - colorization |
| - super-resolution |
| --- |
| |
| # Yesteryear models |
|
|
| On-device ExecuTorch builds of the models used by **Yesteryear**, a private photo restoration |
| app. Photos are processed on the phone; these files are downloaded by the app the first time |
| a feature is used. |
|
|
| Each model is a re-export of a published open model, converted for |
| [ExecuTorch](https://github.com/pytorch/executorch) 1.4 with no retraining. `*_xnnpack.pte` |
| files run on the CPU (XNNPACK, float32) on Android and iOS; `*_coreml.pte` files run on iOS |
| (Core ML, float16). LaMa and DDColor ship as XNNPACK only: their float16 Core ML builds were |
| not accurate enough, so iOS uses the XNNPACK files for them. |
|
|
| | File | Model | Purpose | Upstream license | |
| |---|---|---|---| |
| | `realesrgan_x4v3_t256_{xnnpack,coreml}.pte` | [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) general x4v3 | 4x upscaling, 256 px tiles | BSD-3-Clause | |
| | `bopbtl_scratch_t384_{xnnpack,xnnpack_int8,coreml}.pte` | Scratch detection from [Bringing Old Photos Back to Life](https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life) | Finds scratches and tears, 384 px | MIT | |
| | `lama_big_t512_xnnpack.pte` | [LaMa](https://github.com/advimman/lama) (big-lama) | Inpaints the damage, 512 px tiles | Apache-2.0 | |
| | `ddcolor_paper_tiny_t512_xnnpack.pte` | [DDColor](https://github.com/piddnad/DDColor) tiny | Colorization, 512 px | Apache-2.0 | |
|
|
| Full license texts and copyright notices are in [`licenses/`](licenses). See |
| [`LICENSE.md`](LICENSE.md) for details and changes made. |
|
|
| ## Changes from upstream |
|
|
| The networks and weights are unchanged. To run under ExecuTorch, some operations are rewritten |
| as mathematically equivalent forms (verified against the upstream PyTorch models): |
|
|
| - LaMa: the Fourier transform is computed as matrix multiplications against fixed DFT tables. |
| - DDColor: layer norm, attention and pooling are written with operations XNNPACK supports; |
| spectral normalization is folded into the weights. |
| - All: reflection padding is built from slices; inputs and outputs are wrapped for the app |
| (for example, LaMa takes RGB plus a mask channel and composites its own output). |
|
|
| ## Notes |
|
|
| - `bopbtl_scratch_t384_xnnpack_int8.pte` uses 8-bit weights with dynamically quantized |
| activations (38 MB instead of 150 MB). Its scratch masks match the float model with IoU 0.95 |
| to 0.98 on test photos; the app uses it on Android. |
|
|
| - The LaMa and DDColor weights were trained on Places365 and ImageNet respectively; those |
| datasets have their own terms of use for the images themselves. |
| - The Bringing Old Photos Back to Life repository states that its code and pretrained models |
| are MIT licensed. |
|
|
