--- language: - en license: other pipeline_tag: image-to-image tags: - super-resolution - image-upscaling - image-restoration - qnn - onnx - snapdragon - qualcomm - mobile - npu - android - generative-ai library_name: custom --- # PiSA-Lite **PiSA-Lite is a lightweight, mobile-optimized version of PiSA-SR for Snapdragon-powered smartphones. It is designed to preserve high-quality textures and semantic image details while running through Qualcomm's NPU.** > PiSA-Lite is an unofficial optimization based on PiSA-SR. It is not affiliated with or endorsed by the original PiSA-SR authors. ## Overview PiSA-Lite keeps the original PiSA-SR architecture and its semantic image-restoration behavior while preparing the model for mobile deployment. Unlike small super-resolution models that mainly sharpen edges, PiSA-Lite aims to preserve PiSA-SR's ability to reconstruct material-aware details such as: - wood grain - grass and vegetation - metal reflections - fabric textures - hair and fine surface details - building and object structure The current release includes: - precompiled Qualcomm QNN Context Binaries for Snapdragon 8 Gen 3 - ONNX source models for compiling separate builds for other supported Snapdragon chips - a fixed 4× super-resolution pipeline - an FP16/W8A16 quality configuration ## Model Details | Property | Value | |---|---| | Base project | PiSA-SR | | Task | Generative image super-resolution | | Input | 128 × 128 RGB image | | Output | 512 × 512 RGB image | | Upscale factor | 4× | | Latent shape | `1 × 4 × 64 × 64` | | Target runtime | Qualcomm QNN / HTP NPU | | Current target SoC | Snapdragon 8 Gen 3 / SM8650 | | Current target device family | Samsung Galaxy S24 Family | | Deployment format | QNN Context Binary | | Source export format | ONNX | ## Files ### Snapdragon 8 Gen 3 QNN Models The included QNN binaries were compiled specifically for Snapdragon 8 Gen 3 / SM8650: ```text pisa_encoder_quality.bin pisa_denoiser_quality.bin pisa_decoder_quality.bin ``` | File | Purpose | Precision | Approximate size | |---|---|---:|---:| | `pisa_encoder_quality.bin` | Converts the image into latent space | FP16 | 74 MiB | | `pisa_denoiser_quality.bin` | Restores PiSA textures and semantic details | W8A16 | 791 MiB | | `pisa_decoder_quality.bin` | Converts the restored latent into an image | FP16 | 104 MiB | Total package size is approximately **970 MiB**. ### ONNX Models The ONNX files are source models for creating separate QNN builds for other supported Snapdragon chips: ```text encoder.onnx denoiser.onnx decoder.onnx ``` The ONNX files are **not** pre-optimized universal mobile models. They must be compiled for the intended Snapdragon target using Qualcomm AI Hub, QAIRT, or another compatible Qualcomm QNN toolchain. ## Hardware Compatibility The supplied `.bin` files are compiled for: ```text Qualcomm Snapdragon 8 Gen 3 SoC: SM8650 Samsung Galaxy S24 Family Android 14 ``` QNN Context Binaries are hardware-specific. Do not assume that the supplied Snapdragon 8 Gen 3 binaries will work on: - Snapdragon 8 Gen 2 - Snapdragon 8 Elite - Snapdragon 7-series devices - Exynos devices - MediaTek devices - desktop CPUs or GPUs For another supported Snapdragon chip, use the ONNX models to compile a separate QNN package for that target. ## Pipeline ```text 128 × 128 input image ↓ Resize to 512 × 512 ↓ PiSA VAE Encoder ↓ Latent sampling ↓ PiSA Denoiser ↓ PiSA VAE Decoder ↓ Color correction ↓ 512 × 512 output image ``` All three model components must be executed in order. ## Precision Configuration The current quality release uses: ```text Encoder: FP16 Denoiser: W8A16 Decoder: FP16 ``` This reduces the size of the largest PiSA component while keeping the texture-sensitive VAE encoder and decoder in FP16. ## Android Integration The QNN files are not standalone applications and cannot be opened directly. An Android application must load them through Qualcomm QAIRT/QNN, typically through a native C++ layer: ```text Kotlin / Java UI ↓ JNI ↓ C++ QNN runner ↓ QNN HTP backend ↓ Encoder → Denoiser → Decoder ``` Recommended private storage layout: ```text /data/user/0//files/models/pisa_sm8650/ ├── pisa_encoder_quality.bin ├── pisa_denoiser_quality.bin └── pisa_decoder_quality.bin ``` Because the complete model package is large, downloading the files after installation is generally preferable to embedding them directly inside the APK. ## Compiling for Another Snapdragon Chip Use the ONNX models as source graphs and compile each component for the selected target device: ```text encoder.onnx denoiser.onnx decoder.onnx ↓ Qualcomm AI Hub / QAIRT / QNN compiler ↓ target-specific QNN Context Binaries ``` A separate set of binaries should be generated for each supported Snapdragon family. The application should detect the device SoC before downloading or loading a model package. ```text SM8650 / Snapdragon 8 Gen 3 → Load the included SM8650 package Another supported Snapdragon chip → Download a separately compiled package Unsupported hardware → Use a smaller GPU or CPU fallback model ``` ## Intended Use PiSA-Lite is intended for: - low-resolution photo restoration - experimental mobile photography - restoring vegetation and environmental details - improving material textures - enhancing compressed images - improving game screenshots - research into mobile generative super-resolution ## Out-of-Scope Use PiSA-Lite is not recommended for: - forensic image analysis - identity verification - medical imaging - document or evidence recovery - exact text reconstruction - license-plate recovery - recovering factual details that are not visible in the source image ## Limitations PiSA-Lite is a generative super-resolution model and may create visually plausible details that were not present in the original low-resolution input. Possible failure cases include: - invented textures - incorrect small text - altered faces - changed logos or symbols - inaccurate fine patterns - unstable results on heavily degraded inputs - high memory use compared with small CNN upscalers - slower inference than models such as SPAN - hardware-specific deployment requirements Generated output should not be treated as factual evidence. ## Current Status - [x] PiSA-SR quality preserved in local testing - [x] Weight-optimized PiSA-Lite package created - [x] ONNX models exported - [x] QNN Context Binaries compiled - [x] Snapdragon 8 Gen 3 NPU inference completed - [ ] Public Android runtime example - [ ] On-device speed and memory benchmarks - [ ] Additional Snapdragon targets - [ ] Larger calibration dataset - [ ] Hugging Face demo Space ## Comparison | Model | Sharpness | Semantic texture reconstruction | Mobile suitability | |---|---:|---:|---:| | SPAN | Good | Limited | High | | TinySR | Very good | Medium | Medium | | PiSA-SR | Very good | Very high | Low | | PiSA-Lite | Very good | Very high in current tests | Targeted at Snapdragon NPU | The PiSA-Lite quality claim is based on local visual testing and should be validated on a larger public benchmark set. ## Credits PiSA-Lite is based on the original **PiSA-SR** project and research. All credit for the original architecture, training method, pretrained model, and research belongs to the original PiSA-SR authors. PiSA-Lite focuses on: - mobile deployment - weight optimization - fixed-shape inference - ONNX export - Qualcomm QNN compilation - Snapdragon NPU execution ## License and Redistribution The metadata uses `license: other` because redistribution rights may depend on multiple upstream components. Before redistributing model weights or binaries, review and comply with: - the original PiSA-SR license - the Stable Diffusion 2.1 base-model license - all pretrained-model licenses - Qualcomm AI Hub and QNN terms - any checkpoint or dataset restrictions Uploading this repository does not automatically grant rights beyond the relevant upstream licenses. ## Disclaimer This project is experimental and provided without warranty. The maintainers are not responsible for: - hallucinated or inaccurate reconstructed details - unsupported-device crashes - excessive memory usage - incorrect Android integration - redistribution outside upstream license terms - damage or data loss caused by use of the model Use PiSA-Lite at your own risk. ## Repository GitHub: ```text https://github.com/LoewolfERSTELLER/PiSA-Lite ``` ## Short Description > PiSA-Lite is an unofficial, mobile-optimized PiSA-SR upscaler for Snapdragon smartphones, designed to preserve high-quality textures and semantic image details through Qualcomm's NPU.