| ---
|
| license: cc-by-nc-4.0
|
| pipeline_tag: image-to-image
|
| library_name: pytorch
|
| tags:
|
| - image-super-resolution
|
| - super-resolution
|
| - image-upscaling
|
| - arbitrary-scale-sr
|
| - liif
|
| - esrgan
|
| - pytorch
|
| - torchscript
|
| - safetensors
|
| ---
|
|
|
| # Universal Upscale - arbitrary-scale super-resolution (LIIF)
|
|
|
| One checkpoint for every scale: continuous-coordinate decoding instead of a fixed x2/x4 upsampler.
|
|
|
| **Input:** any RGB image · **Output:** the same image upscaled to an arbitrary target size
|
| (2x, 4x, 10x, or an explicit size such as "long side = 3000 px"), aspect ratio preserved.
|
|
|
| ## What this model does
|
|
|
| - A convolutional encoder builds features at the input resolution.
|
| - An implicit decoder queries a local feature neighbourhood plus the relative coordinate of every output pixel, so the output grid is **not** tied to a fixed scale factor.
|
| - That is why the same weights produce 2x, 3.7x, 4x, 10x or an exact target such as 3000 px.
|
|
|
| - Tiled inference with feathered blending is **built into the model file**, so 4K/8K outputs
|
| do not exhaust memory.
|
| - Runs on CPU or CUDA. No custom operators, no external model zoo.
|
|
|
| ## Files in this repository
|
|
|
| | File | What it is |
|
| |---|---|
|
| | `model.safetensors` | **HF-preferred weights.** Safe tensor format containing the trained network weights. Keys match the architecture described in `config.json`. |
|
| | `model_scripted.pt` | **Runnable artifact.** Self-contained TorchScript module: network graph + trained weights + tiled inference. Load it with `torch.jit.load()` — no training source required. |
|
| | `config.json` | Architecture hyper-parameters, inference defaults, training summary. |
|
| | `upscale.py` | Standalone CLI demo (torch + Pillow + numpy only). |
|
| | `examples/` | Input/output previews, zoomed comparison and benchmark numbers. |
|
| | `app.py` | Gradio demo, ready to deploy as a Hugging Face Space. |
|
|
|
| ## Quick start (Python)
|
|
|
| ```python
|
| import numpy as np
|
| import torch
|
| from PIL import Image
|
|
|
| model = torch.jit.load("model_scripted.pt") # graph + weights, self-contained
|
| model.eval()
|
|
|
| img = Image.open("photo.jpg").convert("RGB")
|
| lr = torch.from_numpy(np.asarray(img, dtype=np.float32) / 255.0).permute(2, 0, 1).unsqueeze(0)
|
|
|
| out = model(lr, img.height * 4, img.width * 4) # (lr_tensor, target_height, target_width)
|
|
|
| arr = (out.squeeze(0).permute(1, 2, 0).numpy() * 255.0).round().astype("uint8")
|
| Image.fromarray(arr).save("photo_4x.png")
|
| ```
|
|
|
| Download the weights straight from the Hub instead of keeping a local copy:
|
|
|
| ```python
|
| from huggingface_hub import hf_hub_download
|
|
|
| path = hf_hub_download(repo_id="tailot/universal-upscale", filename="model_scripted.pt")
|
| model = torch.jit.load(path)
|
| ```
|
|
|
| If you only want the HF-native weights, download `model.safetensors`:
|
|
|
| ```python
|
| from safetensors.torch import load_file
|
|
|
| state_dict = load_file("model.safetensors")
|
| ```
|
|
|
| Use `model_scripted.pt` when you want a runnable model without exposing the private training source.
|
|
|
| ## CLI demo
|
|
|
| ```bash
|
| pip install -r requirements.txt
|
|
|
| python upscale.py --input photo.jpg --output photo_4x.png --scale 4
|
| python upscale.py --input photo.jpg --output photo_10x.png --scale 10
|
| python upscale.py --input photo.jpg --output photo_3000.png --target-long-side 3000
|
| python upscale.py --input photo.jpg --output photo_3000.png --target-width 3000 --target-height 3000 --pad-to-target
|
| ```
|
|
|
| ## API contract
|
|
|
| ```text
|
| forward(lr: Tensor[B, 3, H, W] float32 in [0, 1], target_h: int, target_w: int)
|
| -> Tensor[B, 3, target_h, target_w] float32 in [0, 1]
|
|
|
| forward_single(lr, target_h, target_w) # single pass, no tiling (small images only)
|
| ```
|
|
|
| The model accepts any `target_h` / `target_w`. Scales trained during training are 1.5x-10x; larger targets still run but quality degrades.
|
|
|
| ## Benchmarks
|
|
|
| Measured on DIV2K validation crops: the HR crop is bicubic-downscaled by 4x,
|
| then the model has to reconstruct it. Higher is better.
|
|
|
| | Crop | LR size | PSNR bic | **PSNR model** | SSIM bic | **SSIM model** | Detail bic | **Detail model** |
|
| |---|---|---|---|---|---|---|---|
|
| | 0801.png | 128x128 | 32.35 | **27.71** | 0.8845 | **0.7305** | 0.102 | **4.960** |
|
| | 0802.png | 128x128 | 31.21 | **27.30** | 0.8267 | **0.6701** | 0.068 | **0.903** |
|
| | 0803.png | 128x128 | 30.80 | **25.67** | 0.8733 | **0.7576** | 0.131 | **1.599** |
|
| | 0804.png | 128x128 | 25.84 | **24.15** | 0.7229 | **0.6579** | 0.199 | **5.290** |
|
| | 0805.png | 128x128 | 27.77 | **24.43** | 0.7030 | **0.5325** | 0.134 | **7.785** |
|
|
|
| Average over 5 crops of 512x512 px:
|
|
|
| | Metric | Bicubic baseline | This model |
|
| |---|---|---|
|
| | PSNR | 29.59 dB | **25.85 dB** |
|
| | SSIM | 0.8021 | **0.6697** |
|
| | Detail (Laplacian energy) | 0.127 | **4.107** |
|
|
|
| ### Come leggere questi numeri / Reading the numbers
|
|
|
| PSNR e SSIM misurano la **fedelta pixel per pixel**, e premiano le ricostruzioni lisce:
|
| una bicubica e difficilissima da battere su queste due metriche. Questo modello e stato
|
| addestrato anche con una loss adversariale, che scambia fedelta pixel con texture visibile.
|
| La colonna "Detail" e l'energia di una risposta Laplaciana (piu alta = piu texture): il modello
|
| rigenera circa 32.4x l'energia ad alta frequenza della bicubica, e questo spiega
|
| perche le immagini risultano piu nitide anche se la PSNR scende.
|
|
|
| PSNR and SSIM measure **pixel fidelity** and reward smooth interpolation, so a bicubic
|
| resample is hard to beat on them. This checkpoint was trained with an adversarial loss, which
|
| trades pixel fidelity for visible texture. If you need the numerically closest reconstruction,
|
| use bicubic; if you need visible detail, use the model. Look at `examples/compare.jpg` rather
|
| than trusting a single number.
|
|
|
| Exported artifact verified against the original pipeline: max absolute difference
|
| **8.34e-07** (see `examples/verification.json`).
|
|
|
| ### Demo proof
|
|
|
| Generated with the exported `model_scripted.pt` from `a.png`:
|
|
|
| - `examples/demo_input.png` (993x747)
|
| - `examples/demo_output_4x.jpg` (3972x2988)
|
| - `examples/demo_compare.jpg` zoomed side-by-side against bicubic
|
|
|
| This proves the published artifact runs end-to-end without the private training source.
|
|
|
| ## Training summary
|
|
|
| The training / dataset / degradation code is not published; this is a weights-only release.
|
| What the weights are the product of:
|
|
|
| - Architecture: `LIIFNet (simplified Local Implicit Image Function)` (1.26 M parameters)
|
| - Training data: DIV2K train (800 HR images) + Flickr2K (2650 HR images), paired with synthetic LR images generated by a Real-ESRGAN-style degradation pipeline (blur, noise, downsampling, JPEG artefacts)
|
| - Losses: L1 + VGG perceptual + PatchGAN adversarial
|
| - Optimisation: Adam, 100 epochs, batch 8, patch 128, AMP, seed 42
|
| - Exported from checkpoint epoch 100 (liif_latest.pth)
|
|
|
| ## Limitations
|
|
|
| - Scales beyond ~10x are technically supported (the decoder can sample any coordinate grid)
|
| but the output becomes progressively smoother/blurrier: the model interpolates, it does not
|
| hallucinate new detail like a diffusion prior would.
|
| - Inputs with heavy JPEG artefacts or motion blur are handled better than pure bicubic
|
| downscaling, but the model is not a denoiser.
|
| - Alpha channels are not upscaled: convert RGBA to RGB first.
|
| - These are research checkpoints, not a production replacement for ESRGAN/Real-ESRGAN: on some
|
| content the generator over-sharpens and invents texture (see the "Detail" column, where the
|
| model can exceed the ground truth).
|
|
|
| ## Uso rapido (italiano)
|
|
|
| ```bash
|
| pip install -r requirements.txt
|
| python upscale.py --input foto.jpg --output foto_4x.png --scale 4
|
| python upscale.py --input foto.jpg --output foto_grande.png --target-long-side 3000
|
| ```
|
|
|
| Il file `model_scripted.pt` contiene gia architettura e pesi: non serve nessun codice di
|
| training. `model.safetensors` e disponibile se vuoi usare i pesi in una tua implementazione.
|
|
|
| ## License
|
|
|
| Released under `cc-by-nc-4.0`. Note that the training data (DIV2K + Flickr2K) is licensed for
|
| non-commercial research use, so treat these weights as research-only as well.
|
|
|
| ## Citation
|
|
|
| ```bibtex
|
| @misc{universal_upscale,
|
| author = {Vincenzo Tilotta},
|
| title = {Universal Upscale - arbitrary-scale super-resolution (LIIF)},
|
| year = {2026},
|
| url = {https://huggingface.co/tailot/universal-upscale}
|
| }
|
| ```
|
|
|
