Improve model card: Add pipeline tag, library name, paper, code, and detailed content

README.md

@@ -1,3 +1,115 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+pipeline_tag: image-to-image
+library_name: diffusers
+---
+
+# ODTSR: One-Step Diffusion Transformer for Controllable Real-World Image Super-Resolution
+
+This repository contains the official implementation of **ODTSR**, a model presented in the paper:
+[**One-Step Diffusion Transformer for Controllable Real-World Image Super-Resolution**](https://huggingface.co/papers/2511.17138)
+
+**Authors**: Yushun Fang, Yuxiang Chen, Shibo Yin, Qiang Hu, Jiangchao Yao, Ya Zhang, Xiaoyun Zhang, Yanfeng Wang
+**Affiliations**: Shanghai Jiao Tong University, Xiaohongshu Inc
+
+**Code**: [https://github.com/RedMediaTech/ODTSR](https://github.com/RedMediaTech/ODTSR)
+
+<div align="center">
+  <img src="https://github.com/RedMediaTech/ODTSR/raw/main/static/1.png" alt="ODTSR Overview Framework" width="80%">
+</div>
+
+## Overview
+Recent advances in diffusion-based real-world image super-resolution (Real-ISR) have demonstrated remarkable perceptual quality, yet the balance between fidelity and controllability remains a problem: multi-step diffusion-based methods suffer from generative diversity and randomness, resulting in low fidelity, while one-step methods lose control flexibility due to fidelity-specific finetuning.
+
+**ODTSR** addresses this by presenting a one-step diffusion transformer based on Qwen-Image that performs Real-ISR considering fidelity and controllability simultaneously. It introduces a newly designed **Noise-hybrid Visual Stream (NVS)** that receives low-quality images with adjustable noise (Control Noise) and consistent noise (Prior Noise). Furthermore, **Fidelity-aware Adversarial Training (FAA)** is employed to enhance controllability and achieve one-step inference. ODTSR not only achieves state-of-the-art (SOTA) performance on generic Real-ISR, but also enables prompt controllability on challenging scenarios such as real-world scene text image super-resolution (STISR) of Chinese characters without training on specific datasets.
+
+## Key Features
+*   **One-Step Super-Resolution**: Based on Qwen-Image, ODTSR trains a single-step SR model using LoRA, with model parameters reaching 20B.
+*   **Controllability**: With our proposed Noise-hybrid Visual Stream and Fidelity-aware Adversarial Training, the SR process can be jointly controlled by prompts as well as a Fidelity Weight $f$.
+*   **Multilingual Support**: English and Chinese prompts are supported.
+*   **Versatile Performance**: The model demonstrates strong performance in text images, fine-grained textures, and face images.
+
+## Visual Results
+
+### Results with fixed prompts & high fidelity
+<div align="center">
+  <img src="https://github.com/RedMediaTech/ODTSR/raw/main/static/4.jpeg" alt="Results with fixed prompts & high fidelity" width="80%">
+</div>
+Under the high-fidelity setting with a fixed prompt, our model produces restorations that adhere more closely to the LQ input while remaining natural, significantly reducing the sense of AI processing.
+
+### Text Real-ISR Results
+<div align="center">
+  <img src="https://github.com/RedMediaTech/ODTSR/raw/main/static/2.jpeg" alt="Text Real-ISR Results" width="80%">
+</div>
+In text scenarios, when the prompt specifies the text to be restored, the model automatically matches the LQ text and performs the restoration.
+
+### Controllable Real-ISR Results
+<div align="center">
+  <img src="https://github.com/RedMediaTech/ODTSR/raw/main/static/3.jpeg" alt="Controllable Real-ISR Results" width="80%">
+</div>
+Qualitative results of controllable SR with prompt and adjustable Fidelity Weight (denoted as $f$) on Div2k-val dataset. As $f$ decreases from 1 to 0, detail generation and prompt adherence gradually strengthen.
+
+## Dependencies and Installation
+
+1.  Prepare conda env:
+    ```bash
+    conda create -n yourenv python=3.11
+    ```
+2.  Install `pytorch` (we recommend `torch==2.6.0`):
+    ```bash
+    pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0  -f https://mirrors.aliyun.com/pytorch-wheels/cu124/
+    ```
+3.  Install this repo (based on [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio/tree/main)). The required packages will be automatically installed:
+    ```bash
+    cd xxxx/ODTSR # Replace xxxx with your path
+    pip3 install -e . -v  -i https://mirrors.cloud.tencent.com/pypi/simple
+    ```
+4.  (For training) Install `basicsr`:
+    ```bash
+    pip install basicsr
+    ```
+    Note:
+    You can apply the the following command to fix a bug in `basicsr`. Make sure to replace `/opt/conda` with the path to your own conda environment:
+    ```bash
+    sed -i '8s/from torchvision.transforms.functional_tensor import rgb_to_grayscale/from torchvision.transforms._functional_tensor import rgb_to_grayscale/' /opt/conda/lib/python3.11/site-packages/basicsr/data/degradations.py
+    ```
+5.  Download base model to your disk: [Qwen-Image](https://huggingface.co/Qwen/Qwen-Image/tree/main)
+6.  (For training) Download base model to your disk: [Wan2.1-T2V-1.3B](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B/tree/main)
+7.  (For inference) Download the trained ODTSR model weight: [huggingface](https://huggingface.co/double8fun/ODTSR/tree/main)
+
+## Inference with Script
+Note: you need at least 40GB GPU memory to infer. We will support CPU offload to reduce GPU memory usage soon.
+We now supports tile-based processing (tile size: 512×512), enabling input of arbitrary resolutions and SR at any scale factor.
+
+Please replace `experiments/qwen_one_step_gan/${EXP_DATE}/checkpoints/net_gen_iter_10001.pth` with the trained ODTSR model weight.
+```bash
+sh examples/qwen_image/test_gan.sh
+```
+<div align="center">
+  <img src="https://github.com/RedMediaTech/ODTSR/raw/main/static/infer.png" alt="Inference Workflow" width="70%">
+</div>
+
+## Inference with Gradio
+```bash
+sh examples/qwen_image/test_gradio.sh
+```
+<img src="https://github.com/RedMediaTech/ODTSR/raw/main/static/gradio.jpeg" alt="Gradio Demo" >
+
+## License
+This project is released under the [Apache 2.0 license](https://github.com/RedMediaTech/ODTSR/blob/main/LICENSE).
+
+## Acknowledgement
+This project is based on [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio/tree/main).
+We also leveraged some of [PiSA-SR](https://github.com/csslc/PiSA-SR/tree/main)'s code in dataloader part.
+Thanks for the awesome work!
+
+## Citation
+If ODTSR is helpful to you, please consider citing our paper:
+```bibtex
+@article{fang2025onestep,
+      title={One-Step Diffusion Transformer for Controllable Real-World Image Super-Resolution},
+      author={Fang, Yushun and Chen, Yuxiang and Yin, Shibo and Hu, Qiang and Yao, Jiangchao and Zhang, Ya and Zhang, Xiaoyun and Wang, Yanfeng},
+      journal={arXiv preprint arXiv:2511.17138},
+      year={2025}
+}
+```