|
Download README.md from Yesianrohn/TextRIRO: direct link, hf CLI and curl.
- Browser
- Download file 5.75 kB
-
https://huggingface.co/Yesianrohn/TextRIRO/resolve/main/README.md
- Command line
-
hf download hf://Yesianrohn/TextRIRO/README.md
-
curl -L -o README.md https://huggingface.co/Yesianrohn/TextRIRO/resolve/main/README.md
5.75 kB
| language: | |
| - en | |
| pipeline_tag: image-to-image | |
| base_model: stable-diffusion-v1-5/stable-diffusion-v1-5 | |
| datasets: | |
| - topdu/OpenOCR-Data | |
| tags: | |
| - scene-text-editing | |
| - image-editing | |
| - conditional-diffusion | |
| - pytorch | |
| - arxiv:2610.27592 | |
| # TextRIRO | |
| **Official model checkpoint for [Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?](https://huggingface.co/papers/2610.27592) (NeurIPS 2026).** | |
| Xingsong Ye, Yongkun Du, Jiaxin Zhang, Chong Sun, and Zhineng Chen | |
| [HF Paper](https://huggingface.co/papers/2610.27592) 路 [arXiv](https://arxiv.org/abs/2610.27592) 路 [NeurIPS 2026](https://neurips.cc/virtual/2026/loc/sydney/poster/153296) 路 [Code](https://github.com/YesianRohn/TextRIRO) 路 [Model](https://huggingface.co/Yesianrohn/TextRIRO) 路 [Demo](https://huggingface.co/spaces/Yesianrohn/TextRIRO-Demo) 路 [TextRIRO-3M](https://huggingface.co/datasets/Yesianrohn/TextRIRO-3M) 路 [RIROCommon](https://huggingface.co/datasets/Yesianrohn/RIROCommon) | |
| ## Overview | |
| TextRIRO uses conditional diffusion to replace text in a scene-text crop while preserving the reference appearance. **Shuffle&Crop** creates self-paired supervision by disrupting text semantics in real images while retaining style cues. A multi-scale style encoder and an STR-pretrained text encoder condition generation on appearance and target characters. Training requires no synthetic before-and-after editing pairs or character segmentation. | |
| Here, **real-only training** refers to editing supervision and STR-based text-encoder pre-training. The generative backbone uses pretrained Stable Diffusion components. | |
|  | |
| ## Checkpoint and architecture | |
| | Item | Details | | |
| | --- | --- | | |
| | Released weights | [`textriro.pth`](https://huggingface.co/Yesianrohn/TextRIRO/blob/main/textriro.pth) | | |
| | Inputs | A cropped scene-text image and the desired replacement text | | |
| | Output | An edited crop with the requested text | | |
| | Backbone components | Stable Diffusion 1.5 U-Net and scheduler; Stable Diffusion 2.1 VAE | | |
| | Text conditioning | STR-pretrained character embeddings provided in the code repository | | |
| | Generation resolution | 64 脳 256 pixels; the inference script restores the source dimensions | | |
| | Training data | U14M-Filter, the filtered real-image subset of Union14M, distributed through [OpenOCR-Data](https://huggingface.co/datasets/topdu/OpenOCR-Data/tree/main/Union14M-L-LMDB-Filtered) | | |
| This repository distributes a PyTorch checkpoint for the project's custom implementation. Use the loading and inference scripts in the [code repository](https://github.com/YesianRohn/TextRIRO). | |
| ## Quick start | |
| ### 1. Prepare the implementation | |
| ```bash | |
| git clone https://github.com/YesianRohn/TextRIRO.git | |
| cd TextRIRO | |
| ``` | |
| Follow the repository's [installation and checkpoint preparation instructions](https://github.com/YesianRohn/TextRIRO#installation). The inference script requires CUDA. Prepare the SD 1.5 U-Net/scheduler, SD 2.1 VAE, and STR-pretrained text encoder at the paths specified in [`configs/train.yaml`](https://github.com/YesianRohn/TextRIRO/blob/main/configs/train.yaml). | |
| ### 2. Download the checkpoint | |
| Run this from the cloned repository root: | |
| ```python | |
| from huggingface_hub import hf_hub_download | |
| hf_hub_download( | |
| repo_id="Yesianrohn/TextRIRO", | |
| filename="textriro.pth", | |
| local_dir=".", | |
| ) | |
| ``` | |
| ### 3. Prepare images and replacement text | |
| Put source crops in `example/i_s/`. Create `example/i_t.txt` with one image filename and replacement string per line: | |
| ```text | |
| sample_001.png FLASH | |
| sample_002.jpg Real In Real Out | |
| ``` | |
| Separate the filename and target with one space. Spaces inside the target are supported. Every source image must have a matching entry. | |
| ### 4. Generate edited crops | |
| ```bash | |
| python inference.py \ | |
| --ckpt_path textriro.pth \ | |
| --dataset_dir example \ | |
| --output_dir example_result \ | |
| --num_inference_steps 20 \ | |
| --guidance_scale 2.2 \ | |
| --seed 42 | |
| ``` | |
| Outputs are written to `example_result/` using the source filenames. To try the model interactively, visit the [online demo](https://huggingface.co/spaces/Yesianrohn/TextRIRO-Demo). | |
| ## Scope and limitations | |
| - The released inference script processes text crops. Full-image text detection and compositing require additional processing. | |
| - The current encoder accepts up to **25 target characters**; longer strings are truncated. | |
| - Supported characters are defined by [`EN_symbol_dict.txt`](https://github.com/YesianRohn/TextRIRO/blob/main/EN_symbol_dict.txt) plus the space character. The release should not be treated as a general multilingual editor. | |
| - Output text accuracy and visual consistency should be checked for each application. Detailed evaluations and comparisons are available in the [paper](https://huggingface.co/papers/2610.27592). | |
| ## Related resources | |
| - [TextRIRO-3M](https://huggingface.co/datasets/Yesianrohn/TextRIRO-3M): a synthetic long-text recognition dataset created with TextRIRO. | |
| - [RIROCommon](https://huggingface.co/datasets/Yesianrohn/RIROCommon): a benchmark covering equal-length and variable-length scene text editing. | |
| - [Source code](https://github.com/YesianRohn/TextRIRO): training, inference, configurations, and evaluation guidance. | |
| ## Citation | |
| If you use TextRIRO in your research, please cite: | |
| ```bibtex | |
| @inproceedings{ye2026textriro, | |
| title = {Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?}, | |
| author = {Ye, Xingsong and Du, Yongkun and Zhang, Jiaxin and Sun, Chong and Chen, Zhineng}, | |
| booktitle = {Advances in Neural Information Processing Systems}, | |
| year = {2026}, | |
| url = {https://neurips.cc/virtual/2026/loc/sydney/poster/153296} | |
| } | |
| ``` | |