--- language: - en pipeline_tag: image-to-image base_model: stable-diffusion-v1-5/stable-diffusion-v1-5 datasets: - topdu/OpenOCR-Data tags: - scene-text-editing - image-editing - conditional-diffusion - pytorch - arxiv:2610.27592 --- # TextRIRO **Official model checkpoint for [Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?](https://huggingface.co/papers/2610.27592) (NeurIPS 2026).** Xingsong Ye, Yongkun Du, Jiaxin Zhang, Chong Sun, and Zhineng Chen [HF Paper](https://huggingface.co/papers/2610.27592) · [arXiv](https://arxiv.org/abs/2610.27592) · [NeurIPS 2026](https://neurips.cc/virtual/2026/loc/sydney/poster/153296) · [Code](https://github.com/YesianRohn/TextRIRO) · [Model](https://huggingface.co/Yesianrohn/TextRIRO) · [Demo](https://huggingface.co/spaces/Yesianrohn/TextRIRO-Demo) · [TextRIRO-3M](https://huggingface.co/datasets/Yesianrohn/TextRIRO-3M) · [RIROCommon](https://huggingface.co/datasets/Yesianrohn/RIROCommon) ## Overview TextRIRO uses conditional diffusion to replace text in a scene-text crop while preserving the reference appearance. **Shuffle&Crop** creates self-paired supervision by disrupting text semantics in real images while retaining style cues. A multi-scale style encoder and an STR-pretrained text encoder condition generation on appearance and target characters. Training requires no synthetic before-and-after editing pairs or character segmentation. Here, **real-only training** refers to editing supervision and STR-based text-encoder pre-training. The generative backbone uses pretrained Stable Diffusion components. ![TextRIRO framework](https://raw.githubusercontent.com/YesianRohn/TextRIRO/main/assets/framework.png) ## Checkpoint and architecture | Item | Details | | --- | --- | | Released weights | [`textriro.pth`](https://huggingface.co/Yesianrohn/TextRIRO/blob/main/textriro.pth) | | Inputs | A cropped scene-text image and the desired replacement text | | Output | An edited crop with the requested text | | Backbone components | Stable Diffusion 1.5 U-Net and scheduler; Stable Diffusion 2.1 VAE | | Text conditioning | STR-pretrained character embeddings provided in the code repository | | Generation resolution | 64 × 256 pixels; the inference script restores the source dimensions | | Training data | U14M-Filter, the filtered real-image subset of Union14M, distributed through [OpenOCR-Data](https://huggingface.co/datasets/topdu/OpenOCR-Data/tree/main/Union14M-L-LMDB-Filtered) | This repository distributes a PyTorch checkpoint for the project's custom implementation. Use the loading and inference scripts in the [code repository](https://github.com/YesianRohn/TextRIRO). ## Quick start ### 1. Prepare the implementation ```bash git clone https://github.com/YesianRohn/TextRIRO.git cd TextRIRO ``` Follow the repository's [installation and checkpoint preparation instructions](https://github.com/YesianRohn/TextRIRO#installation). The inference script requires CUDA. Prepare the SD 1.5 U-Net/scheduler, SD 2.1 VAE, and STR-pretrained text encoder at the paths specified in [`configs/train.yaml`](https://github.com/YesianRohn/TextRIRO/blob/main/configs/train.yaml). ### 2. Download the checkpoint Run this from the cloned repository root: ```python from huggingface_hub import hf_hub_download hf_hub_download( repo_id="Yesianrohn/TextRIRO", filename="textriro.pth", local_dir=".", ) ``` ### 3. Prepare images and replacement text Put source crops in `example/i_s/`. Create `example/i_t.txt` with one image filename and replacement string per line: ```text sample_001.png FLASH sample_002.jpg Real In Real Out ``` Separate the filename and target with one space. Spaces inside the target are supported. Every source image must have a matching entry. ### 4. Generate edited crops ```bash python inference.py \ --ckpt_path textriro.pth \ --dataset_dir example \ --output_dir example_result \ --num_inference_steps 20 \ --guidance_scale 2.2 \ --seed 42 ``` Outputs are written to `example_result/` using the source filenames. To try the model interactively, visit the [online demo](https://huggingface.co/spaces/Yesianrohn/TextRIRO-Demo). ## Scope and limitations - The released inference script processes text crops. Full-image text detection and compositing require additional processing. - The current encoder accepts up to **25 target characters**; longer strings are truncated. - Supported characters are defined by [`EN_symbol_dict.txt`](https://github.com/YesianRohn/TextRIRO/blob/main/EN_symbol_dict.txt) plus the space character. The release should not be treated as a general multilingual editor. - Output text accuracy and visual consistency should be checked for each application. Detailed evaluations and comparisons are available in the [paper](https://huggingface.co/papers/2610.27592). ## Related resources - [TextRIRO-3M](https://huggingface.co/datasets/Yesianrohn/TextRIRO-3M): a synthetic long-text recognition dataset created with TextRIRO. - [RIROCommon](https://huggingface.co/datasets/Yesianrohn/RIROCommon): a benchmark covering equal-length and variable-length scene text editing. - [Source code](https://github.com/YesianRohn/TextRIRO): training, inference, configurations, and evaluation guidance. ## Citation If you use TextRIRO in your research, please cite: ```bibtex @inproceedings{ye2026textriro, title = {Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?}, author = {Ye, Xingsong and Du, Yongkun and Zhang, Jiaxin and Sun, Chong and Chen, Zhineng}, booktitle = {Advances in Neural Information Processing Systems}, year = {2026}, url = {https://neurips.cc/virtual/2026/loc/sydney/poster/153296} } ```