Add comprehensive model card for GSASR

This PR adds a comprehensive model card for the GSASR model.
It includes:
- The paper link: [Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution](https://huggingface.co/papers/2501.06838)
- A link to the project page: https://mt-cly.github.io/GSASR.github.io/
- A link to the GitHub repository for code: https://github.com/ChrisDud0257/GSASR
- The `pipeline_tag: image-to-image`, ensuring the model appears in relevant searches.
- The `library_name: pytorch`, reflecting its compatibility.
- The `license: apache-2.0`.
- Detailed installation and sample usage instructions.
- Performance tables and pre-trained model links.
- Citation information and acknowledgements.

README.md

@@ -0,0 +1,282 @@
+---
+license: apache-2.0
+pipeline_tag: image-to-image
+library_name: pytorch
+tags:
+  - super-resolution
+  - image-enhancement
+  - generative-modeling
+---
+
+# Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution (GSASR)
+
+This repository contains the official models and code for **GSASR**, a novel method for arbitrary-scale super-resolution, presented in the paper [Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution](https://huggingface.co/papers/2501.06838).
+
+📚 [Paper](https://huggingface.co/papers/2501.06838) | 🌐 [Project Page](https://mt-cly.github.io/GSASR.github.io/) | 💻 [Code](https://github.com/ChrisDud0257/GSASR) | 🚀 [Gradio Demo](https://huggingface.co/spaces/mutou0308/GSASR)
+
+## Abstract
+Implicit Neural Representations (INR) have been successfully employed for Arbitrary-scale Super-Resolution (ASR). However, INR-based models need to query the multi-layer perceptron module numerous times and render a pixel in each query, resulting in insufficient representation capability and low computational efficiency. Recently, Gaussian Splatting (GS) has shown its advantages over INR in both visual quality and rendering speed in 3D tasks, which motivates us to explore whether GS can be employed for the ASR task. However, directly applying GS to ASR is exceptionally challenging because the original GS is an optimization-based method through overfitting each single scene, while in ASR we aim to learn a single model that can generalize to different images and scaling factors. We overcome these challenges by developing two novel techniques. Firstly, to generalize GS for ASR, we elaborately design an architecture to predict the corresponding image-conditioned Gaussians of the input low-resolution image in a feed-forward manner. Each Gaussian can fit the shape and direction of an area of complex textures, showing powerful representation capability. Secondly, we implement an efficient differentiable 2D GPU/CUDA-based scale-aware rasterization to render super-resolved images by sampling discrete RGB values from the predicted continuous Gaussians. Via end-to-end training, our optimized network, namely GSASR, can perform ASR for any image and unseen scaling factors. Extensive experiments validate the effectiveness of our proposed method.
+
+## Overview
+
+GSASR achieves state-of-the-art in arbitrary-scale super-resolution by representing given low-resolution images as millions of continuous 2D Gaussians.
+
+![Fast Rasterization](https://github.com/ChrisDud0257/GSASR/raw/main/assets/sampling.png)
+
+## Performance
+
+**Comparisons with representative/SoTA ASR models (PSNR/SSIM are tested on Y channel of Ycbcr space).**
+
+<div style="overflow-x:auto; font-size:10px;">
+<table>
+  <tr>
+    <th rowspan="2">Encoder Backbone</th>
+    <th rowspan="2">Methods</th>
+    <th rowspan="2">Version</th>
+    <th rowspan="2">Training Dataset</th>
+    <th colspan="3" align="center">PSNR/SSIM/LPIPS/DIST (x4 scaling factor)</th>
+  </tr>
+  <tr>
+    <td align="center">DIV2K</td>
+    <td align="center">LSDIR</td>
+    <td align="center">Urban100</td>
+  </tr>
+
+  <!-- EDSR Backbone -->
+  <tr>
+    <td rowspan="6">EDSR</td>
+    <td>LIIF</td>
+    <td>Paper</td>
+    <td>DIV2K</td>
+    <td align="center">30.43/0.8388/0.2662/0.1403</td>
+    <td align="center">26.21/0.7614/0.2978/0.1678</td>
+    <td align="center">26.14/0.7885/0.2271/0.1738</td>
+  </tr>
+  <tr>
+    <td>GaussianSR</td>
+    <td>Paper</td>
+    <td>DIV2K</td>
+    <td align="center">30.46/0.8389/0.2684/0.1406</td>
+    <td align="center">26.23/0.7615/0.3007/0.1679</td>
+    <td align="center">26.19/0.7893/0.2283/0.1730</td>
+  </tr>
+  <tr>
+    <td>CiaoSR</td>
+    <td>Paper</td>
+    <td>DIV2K</td>
+    <td align="center">30.67/0.8431/0.2585/0.1370</td>
+    <td align="center">26.42/0.7681/0.2865/0.1631</td>
+    <td align="center">26.69/0.8091/0.2078/0.1659</td>
+  </tr>
+  <tr>
+    <td>GSASR</td>
+    <td>Paper Reported</td>
+    <td>DIV2K</td>
+    <td align="center">30.89/0.8486/0.2518/0.1301</td>
+    <td align="center">26.65/0.7774/0.2777/0.1554</td>
+    <td align="center">27.01/0.8142/0.1987/0.1552</td>
+  </tr>
+  <tr>
+    <td>GSASR</td>
+    <td>Enhanced</td>
+    <td>DIV2K</td>
+    <td align="center">31.01/0.8509/0.2508/0.1306</td>
+    <td align="center">26.78/0.7813/0.2962/0.1543</td>
+    <td align="center">27.34/0.8230/0.1920/0.1515</td>
+  </tr>
+  <tr>
+    <td>GSASR</td>
+    <td>Enhanced</td>
+    <td>DF2K</td>
+    <td align="center">31.04/0.8515/0.2512/0.1307</td>
+    <td align="center">26.82/0.7827/0.2751/0.1540</td>
+    <td align="center">27.45/0.8256/0.1902/0.1507</td>
+  </tr>
+
+  <!-- RDN Backbone -->
+  <tr>
+    <td rowspan="6">RDN</td>
+    <td>LIIF</td>
+    <td>Paper</td>
+    <td>DIV2K</td>
+    <td align="center">30.71/0.8449/0.2566/0.1354</td>
+    <td align="center">26.48/0.7714/0.2838/0.1603</td>
+    <td align="center">26.71/0.8055/0.2062/0.1562</td>
+  </tr>
+  <tr>
+    <td>GaussianSR</td>
+    <td>Paper</td>
+    <td>DIV2K</td>
+    <td align="center">30.76/0.8457/0.2570/0.1347</td>
+    <td align="center">26.53/0.7727/0.2837/0.1595</td>
+    <td align="center">26.77/0.8064/0.2069/0.1610</td>
+  </tr>
+  <tr>
+    <td>CiaoSR</td>
+    <td>Paper</td>
+    <td>DIV2K</td>
+    <td align="center">30.91/0.8481/0.2525/0.1327</td>
+    <td align="center">26.66/0.7770/0.2768/0.1563</td>
+    <td align="center">27.10/0.8142/0.1966/0.1559</td>
+  </tr>
+  <tr>
+    <td>GSASR</td>
+    <td>Paper Reported</td>
+    <td>DIV2K</td>
+    <td align="center">30.96/0.8500/0.2505/0.1288</td>
+    <td align="center">26.73/0.7801/0.2752/0.1533</td>
+    <td align="center">27.15/0.8177/0.1953/0.1515</td>
+  </tr>
+  <tr>
+    <td>GSASR</td>
+    <td>Enhanced</td>
+    <td>DIV2K</td>
+    <td align="center">31.03/0.8513/0.2499/0.1306</td>
+    <td align="center">26.79/0.7819/0.2740/0.1543</td>
+    <td align="center">27.37/0.8238/0.1898/0.1511</td>
+  </tr>
+  <tr>
+    <td>GSASR</td>
+    <td>Enhanced</td>
+    <td>DF2K</td>
+    <td align="center">31.10/0.8525/0.2482/0.1296</td>
+    <td align="center">26.88/0.7848/0.2709/0.1527</td>
+    <td align="center">27.58/0.8289/0.1849/0.1500</td>
+  </tr>
+
+  <!-- SWIN Backbone -->
+  <tr>
+    <td rowspan="4">SWIN</td>
+    <td>CiaoSR</td>
+    <td>Paper</td>
+    <td>DIV2K</td>
+    <td align="center">31.05/0.8511/0.2487/0.1316</td>
+    <td align="center">26.80/0.7812/0.2724/0.1552</td>
+    <td align="center">27.40/0.8231/0.1869/0.1535</td>
+  </tr>
+  <tr>
+    <td>GSASR</td>
+    <td>Paper (not Reported)</td>
+    <td>DIV2K</td>
+    <td align="center">31.06/0.8521/0.2487/0.1270</td>
+    <td align="center">26.84/0.7837/0.2719/0.1503</td>
+    <td align="center">27.39/0.8247/0.1913/0.1466</td>
+  </tr>
+  <tr>
+    <td>GSASR</td>
+    <td>Enhanced</td>
+    <td>DIV2K</td>
+    <td align="center">31.10/0.8530/0.2463/0.1285</td>
+    <td align="center">26.88/0.7849/0.2690/0.1517</td>
+    <td align="center">27.55/0.8280/0.1850/0.1475</td>
+  </tr>
+  <tr>
+    <td>GSASR</td>
+    <td>Enhanced</td>
+    <td>DF2K</td>
+    <td align="center">31.17/0.8541/0.2456/0.1288</td>
+    <td align="center">26.96/0.7876/0.2665/0.1513</td>
+    <td align="center">27.81/0.8343/0.1781/0.1465</td>
+  </tr>
+
+  <!-- HATL Backbone -->
+  <tr>
+    <td>HATL</td>
+    <td>GSASR</td>
+    <td>Ultra Performance</td>
+    <td>SA1B</td>
+    <td align="center">31.31/0.8570/0.2381/0.1268</td>
+    <td align="center">27.17/0.7948/0.2548/0.1470</td>
+    <td align="center">28.44/0.8493/0.1580/0.1394</td>
+  </tr>
+</table>
+</div>
+
+**Comparisons with representative/SoTA ASR models (PSNR/SSIM are tested on Y channel of Ycbcr space).**
+
+We provide three versions of GSASR:
+ - Paper: the results we reported in our paper. (not reported) means results are not shown in our paper due to limited pages.
+ - Enhanced: we introduce [Rotary Position Embedding (ROPE)](https://github.com/naver-ai/rope-vit) with Flash Attention, and utilize Automatic Mixed Precision (AMP) strategy during training/inference to to reduce memory and time cost.
+ - Ultra Performance: based on `Enhanced` settings, we explore the performance upper bound of GSASR by introducing  [HAT-L](https://github.com/XPixelGroup/HAT) encoder and [SA1B](https://ai.meta.com/datasets/segment-anything/) dataset.
+
+## Pre-trained Models (Enhanced and Ultra Performance Version)
+
+| Model Backbone | Training Dataset | Download                                                                                                                                      | Version           |
+| :--------------- | :--------------- | :-------------------------------------------------------------------------------------------------------------------------------------------- | :---------------- |
+| EDSR           | DIV2K            | [Google Drive](https://drive.google.com/drive/folders/1R6ZCdAd6t_2CCpjCK67F9nag9jitMhI6?usp=sharing), [Hugging Face](https://huggingface.co/mutou0308/GSASR/tree/main/GSASR_enhenced_ultra/EDSR_DIV2K) | Enhanced          |
+| EDSR           | DF2K             | [Google Drive](https://drive.google.com/drive/folders/16TV2yJt_lfNqJnATtJnEkHV1KoBuW8ww?usp=sharing), [Hugging Face](https://huggingface.co/mutou0308/GSASR/tree/main/GSASR_enhenced_ultra/EDSR_DF2K)   | Enhanced          |
+| RDN            | DIV2K            | [Google Drive](https://drive.google.com/drive/folders/1guSg28c8gvrTkCvTmNbzqf9vWJfLv58Q?usp=sharing), [Hugging Face](https://huggingface.co/mutou0308/GSASR/tree/main/GSASR_enhenced_ultra/RDN_DIV2K) | Enhanced          |
+| RDN            | DF2K             | [Google Drive](https://drive.google.com/drive/folders/1vkBvsiiNqTFKmPtNjPlqMn_mh_ClUrKE?usp=sharing), [Hugging Face](https://huggingface.co/mutou0308/GSASR/tree/main/GSASR_enhenced_ultra/RDN_DF2K)   | Enhanced          |
+| SWIN           | DIV2K            | [Google Drive](https://drive.google.com/drive/folders/1kVLkOs4KrXlXsPsh0oqvey2dvT6TxqH-?usp=sharing), [Hugging Face](https://huggingface.co/mutou0308/GSASR/tree/main/GSASR_enhenced_ultra/SWIN_DIV2K) | Enhanced          |
+| SWIN           | DF2K             | [Google Drive](https://drive.google.com/drive/folders/1ql6dktVUlQFIoPSJkEuvvMPz9TlacMdy?usp=sharing), [Hugging Face](https://huggingface.co/mutou0308/GSASR/tree/main/GSASR_enhenced_ultra/SWIN_DF2K)   | Enhanced          |
+| HATL           | SA1B             | [Google Drive](https://drive.google.com/drive/folders/1Pn-4JWvlMj50CulmAcBI1Hssiu-6nSYI?usp=sharing), [Hugging Face](https://huggingface.co/mutou0308/GSASR/tree/main/GSASR_enhenced_ultra/HATL-SA1B)   | Ultra Performance |
+
+## Usage
+
+### Preparation
+- Pytorch == 2.0 (PyTorch Version must >= 2.0)
+- Anaconda
+- CUDA Toolkit (necessary)
+
+Firstly, please make sure you have installed [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit-archive)! Since we have hand-crafted CUDA operators, you need to compile them when you run GSASR.
+
+```bash
+git clone https://github.com/ChrisDud0257/GSASR
+cd GSASR
+conda create --name gsasr python=3.10
+conda activate gsasr
+export CUDA_HOME=${path_to_CUDA} ### specify the path to cuda-11.8
+pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
+python setup_gscuda.py install # gscuda
+cd TrainTestGSASR
+pip install -r requirements.txt
+BASICSR_EXT=True python setup.py develop # basicsr
+```
+
+We have tested that the versions of CUDA from 11.0 to 12.4 are all OK.
+
+### Running Inference
+
+You need to properly authenticate with Hugging Face to download our model weights. Once set up, our code will handle it automatically at your first run. You can authenticate by running:
+
+```bash
+# This will prompt you to enter your Hugging Face credentials.
+huggingface-cli login
+```
+
+You can try GSASR easily by launching the Gradio demo or running in command.
+
+### 🚀 Gradio demo
+```bash
+python demo_gr.py
+```
+
+### 💻 CLI
+```bash
+python inference_enhenced.py \
+    --input_img_path <path_to_img> \
+    --save_sr_path <path_to_saved_folder> \
+    --model <{EDSR_DIV2K, EDSR_DF2K, RDN_DIV2K, RDN_DF2K, SWIN_DIV2K,SWIN_DF2K, HATL_SA1B}> \
+    --scale <scale> [--tile_process] [--AMP_test]
+```
+If it fails to access Huggingface, try to manually download pretrained models and specify local path with `--model_path <path_to_model_weight>`. 
+
+Using `--tile_process` and `--AMP_test` if memory is limited.
+
+## Citation
+
+If you find this research helpful for you, please cite our paper.
+```bibtex
+@article{chen2025generalized,
+  title={Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution},
+  author={Chen, Du and Chen, Liyi and Zhang, Zhengqiang and Zhang, Lei},
+  journal={arXiv preprint arXiv:2501.06838},
+  year={2025}
+}
+```
+
+## Acknowledgement
+
+This project is built mainly based on the excellent [BasicSR](https://github.com/XPixelGroup/BasicSR), [HAT](https://github.com/XPixelGroup/HAT) and [ROPE-ViT](https://github.com/naver-ai/rope-vit) codeframe. We appreciate it a lot for their developers.
+
+We sincerely thank [Mr.Zhengqiang Zhang](https://github.com/xtudbxk) for his support in the CUDA operator of rasterization.