| --- |
| language: |
| - en |
| license: other |
| pipeline_tag: image-to-image |
| tags: |
| - super-resolution |
| - image-upscaling |
| - image-restoration |
| - qnn |
| - onnx |
| - snapdragon |
| - qualcomm |
| - mobile |
| - npu |
| - android |
| - generative-ai |
| library_name: custom |
| --- |
| |
| # PiSA-Lite |
|
|
| **PiSA-Lite is a lightweight, mobile-optimized version of PiSA-SR for Snapdragon-powered smartphones. It is designed to preserve high-quality textures and semantic image details while running through Qualcomm's NPU.** |
|
|
| > PiSA-Lite is an unofficial optimization based on PiSA-SR. It is not affiliated with or endorsed by the original PiSA-SR authors. |
|
|
| ## Overview |
|
|
| PiSA-Lite keeps the original PiSA-SR architecture and its semantic image-restoration behavior while preparing the model for mobile deployment. |
|
|
| Unlike small super-resolution models that mainly sharpen edges, PiSA-Lite aims to preserve PiSA-SR's ability to reconstruct material-aware details such as: |
|
|
| - wood grain |
| - grass and vegetation |
| - metal reflections |
| - fabric textures |
| - hair and fine surface details |
| - building and object structure |
|
|
| The current release includes: |
|
|
| - precompiled Qualcomm QNN Context Binaries for Snapdragon 8 Gen 3 |
| - ONNX source models for compiling separate builds for other supported Snapdragon chips |
| - a fixed 4× super-resolution pipeline |
| - an FP16/W8A16 quality configuration |
|
|
| ## Model Details |
|
|
| | Property | Value | |
| |---|---| |
| | Base project | PiSA-SR | |
| | Task | Generative image super-resolution | |
| | Input | 128 × 128 RGB image | |
| | Output | 512 × 512 RGB image | |
| | Upscale factor | 4× | |
| | Latent shape | `1 × 4 × 64 × 64` | |
| | Target runtime | Qualcomm QNN / HTP NPU | |
| | Current target SoC | Snapdragon 8 Gen 3 / SM8650 | |
| | Current target device family | Samsung Galaxy S24 Family | |
| | Deployment format | QNN Context Binary | |
| | Source export format | ONNX | |
|
|
| ## Files |
|
|
| ### Snapdragon 8 Gen 3 QNN Models |
|
|
| The included QNN binaries were compiled specifically for Snapdragon 8 Gen 3 / SM8650: |
|
|
| ```text |
| pisa_encoder_quality.bin |
| pisa_denoiser_quality.bin |
| pisa_decoder_quality.bin |
| ``` |
|
|
| | File | Purpose | Precision | Approximate size | |
| |---|---|---:|---:| |
| | `pisa_encoder_quality.bin` | Converts the image into latent space | FP16 | 74 MiB | |
| | `pisa_denoiser_quality.bin` | Restores PiSA textures and semantic details | W8A16 | 791 MiB | |
| | `pisa_decoder_quality.bin` | Converts the restored latent into an image | FP16 | 104 MiB | |
|
|
| Total package size is approximately **970 MiB**. |
|
|
| ### ONNX Models |
|
|
| The ONNX files are source models for creating separate QNN builds for other supported Snapdragon chips: |
|
|
| ```text |
| encoder.onnx |
| denoiser.onnx |
| decoder.onnx |
| ``` |
|
|
| The ONNX files are **not** pre-optimized universal mobile models. They must be compiled for the intended Snapdragon target using Qualcomm AI Hub, QAIRT, or another compatible Qualcomm QNN toolchain. |
|
|
| ## Hardware Compatibility |
|
|
| The supplied `.bin` files are compiled for: |
|
|
| ```text |
| Qualcomm Snapdragon 8 Gen 3 |
| SoC: SM8650 |
| Samsung Galaxy S24 Family |
| Android 14 |
| ``` |
|
|
| QNN Context Binaries are hardware-specific. |
|
|
| Do not assume that the supplied Snapdragon 8 Gen 3 binaries will work on: |
|
|
| - Snapdragon 8 Gen 2 |
| - Snapdragon 8 Elite |
| - Snapdragon 7-series devices |
| - Exynos devices |
| - MediaTek devices |
| - desktop CPUs or GPUs |
|
|
| For another supported Snapdragon chip, use the ONNX models to compile a separate QNN package for that target. |
|
|
| ## Pipeline |
|
|
| ```text |
| 128 × 128 input image |
| ↓ |
| Resize to 512 × 512 |
| ↓ |
| PiSA VAE Encoder |
| ↓ |
| Latent sampling |
| ↓ |
| PiSA Denoiser |
| ↓ |
| PiSA VAE Decoder |
| ↓ |
| Color correction |
| ↓ |
| 512 × 512 output image |
| ``` |
|
|
| All three model components must be executed in order. |
|
|
| ## Precision Configuration |
|
|
| The current quality release uses: |
|
|
| ```text |
| Encoder: FP16 |
| Denoiser: W8A16 |
| Decoder: FP16 |
| ``` |
|
|
| This reduces the size of the largest PiSA component while keeping the texture-sensitive VAE encoder and decoder in FP16. |
|
|
| ## Android Integration |
|
|
| The QNN files are not standalone applications and cannot be opened directly. |
|
|
| An Android application must load them through Qualcomm QAIRT/QNN, typically through a native C++ layer: |
|
|
| ```text |
| Kotlin / Java UI |
| ↓ |
| JNI |
| ↓ |
| C++ QNN runner |
| ↓ |
| QNN HTP backend |
| ↓ |
| Encoder → Denoiser → Decoder |
| ``` |
|
|
| Recommended private storage layout: |
|
|
| ```text |
| /data/user/0/<application-id>/files/models/pisa_sm8650/ |
| ├── pisa_encoder_quality.bin |
| ├── pisa_denoiser_quality.bin |
| └── pisa_decoder_quality.bin |
| ``` |
|
|
| Because the complete model package is large, downloading the files after installation is generally preferable to embedding them directly inside the APK. |
|
|
| ## Compiling for Another Snapdragon Chip |
|
|
| Use the ONNX models as source graphs and compile each component for the selected target device: |
|
|
| ```text |
| encoder.onnx |
| denoiser.onnx |
| decoder.onnx |
| ↓ |
| Qualcomm AI Hub / QAIRT / QNN compiler |
| ↓ |
| target-specific QNN Context Binaries |
| ``` |
|
|
| A separate set of binaries should be generated for each supported Snapdragon family. |
|
|
| The application should detect the device SoC before downloading or loading a model package. |
|
|
| ```text |
| SM8650 / Snapdragon 8 Gen 3 |
| → Load the included SM8650 package |
| |
| Another supported Snapdragon chip |
| → Download a separately compiled package |
| |
| Unsupported hardware |
| → Use a smaller GPU or CPU fallback model |
| ``` |
|
|
| ## Intended Use |
|
|
| PiSA-Lite is intended for: |
|
|
| - low-resolution photo restoration |
| - experimental mobile photography |
| - restoring vegetation and environmental details |
| - improving material textures |
| - enhancing compressed images |
| - improving game screenshots |
| - research into mobile generative super-resolution |
|
|
| ## Out-of-Scope Use |
|
|
| PiSA-Lite is not recommended for: |
|
|
| - forensic image analysis |
| - identity verification |
| - medical imaging |
| - document or evidence recovery |
| - exact text reconstruction |
| - license-plate recovery |
| - recovering factual details that are not visible in the source image |
|
|
| ## Limitations |
|
|
| PiSA-Lite is a generative super-resolution model and may create visually plausible details that were not present in the original low-resolution input. |
|
|
| Possible failure cases include: |
|
|
| - invented textures |
| - incorrect small text |
| - altered faces |
| - changed logos or symbols |
| - inaccurate fine patterns |
| - unstable results on heavily degraded inputs |
| - high memory use compared with small CNN upscalers |
| - slower inference than models such as SPAN |
| - hardware-specific deployment requirements |
|
|
| Generated output should not be treated as factual evidence. |
|
|
| ## Current Status |
|
|
| - [x] PiSA-SR quality preserved in local testing |
| - [x] Weight-optimized PiSA-Lite package created |
| - [x] ONNX models exported |
| - [x] QNN Context Binaries compiled |
| - [x] Snapdragon 8 Gen 3 NPU inference completed |
| - [ ] Public Android runtime example |
| - [ ] On-device speed and memory benchmarks |
| - [ ] Additional Snapdragon targets |
| - [ ] Larger calibration dataset |
| - [ ] Hugging Face demo Space |
|
|
| ## Comparison |
|
|
| | Model | Sharpness | Semantic texture reconstruction | Mobile suitability | |
| |---|---:|---:|---:| |
| | SPAN | Good | Limited | High | |
| | TinySR | Very good | Medium | Medium | |
| | PiSA-SR | Very good | Very high | Low | |
| | PiSA-Lite | Very good | Very high in current tests | Targeted at Snapdragon NPU | |
|
|
| The PiSA-Lite quality claim is based on local visual testing and should be validated on a larger public benchmark set. |
|
|
| ## Credits |
|
|
| PiSA-Lite is based on the original **PiSA-SR** project and research. |
|
|
| All credit for the original architecture, training method, pretrained model, and research belongs to the original PiSA-SR authors. |
|
|
| PiSA-Lite focuses on: |
|
|
| - mobile deployment |
| - weight optimization |
| - fixed-shape inference |
| - ONNX export |
| - Qualcomm QNN compilation |
| - Snapdragon NPU execution |
|
|
| ## License and Redistribution |
|
|
| The metadata uses `license: other` because redistribution rights may depend on multiple upstream components. |
|
|
| Before redistributing model weights or binaries, review and comply with: |
|
|
| - the original PiSA-SR license |
| - the Stable Diffusion 2.1 base-model license |
| - all pretrained-model licenses |
| - Qualcomm AI Hub and QNN terms |
| - any checkpoint or dataset restrictions |
|
|
| Uploading this repository does not automatically grant rights beyond the relevant upstream licenses. |
|
|
| ## Disclaimer |
|
|
| This project is experimental and provided without warranty. |
|
|
| The maintainers are not responsible for: |
|
|
| - hallucinated or inaccurate reconstructed details |
| - unsupported-device crashes |
| - excessive memory usage |
| - incorrect Android integration |
| - redistribution outside upstream license terms |
| - damage or data loss caused by use of the model |
|
|
| Use PiSA-Lite at your own risk. |
|
|
| ## Repository |
|
|
| GitHub: |
|
|
| ```text |
| https://github.com/LoewolfERSTELLER/PiSA-Lite |
| ``` |
|
|
| ## Short Description |
|
|
| > PiSA-Lite is an unofficial, mobile-optimized PiSA-SR upscaler for Snapdragon smartphones, designed to preserve high-quality textures and semantic image details through Qualcomm's NPU. |
