Download README.md from Yesianrohn/TextRIRO: direct link, hf CLI and curl.
- Browser
- Download file 5.75 kB
-
https://huggingface.co/Yesianrohn/TextRIRO/resolve/main/README.md
- Command line
-
hf download hf://Yesianrohn/TextRIRO/README.md
-
curl -L -o README.md https://huggingface.co/Yesianrohn/TextRIRO/resolve/main/README.md
language:
- en
pipeline_tag: image-to-image
base_model: stable-diffusion-v1-5/stable-diffusion-v1-5
datasets:
- topdu/OpenOCR-Data
tags:
- scene-text-editing
- image-editing
- conditional-diffusion
- pytorch
- arxiv:2610.27592
TextRIRO
Official model checkpoint for Real In, Real Out: What If We Only Use Real Data for Scene Text Editing? (NeurIPS 2026).
Xingsong Ye, Yongkun Du, Jiaxin Zhang, Chong Sun, and Zhineng Chen
HF Paper 路 arXiv 路 NeurIPS 2026 路 Code 路 Model 路 Demo 路 TextRIRO-3M 路 RIROCommon
Overview
TextRIRO uses conditional diffusion to replace text in a scene-text crop while preserving the reference appearance. Shuffle&Crop creates self-paired supervision by disrupting text semantics in real images while retaining style cues. A multi-scale style encoder and an STR-pretrained text encoder condition generation on appearance and target characters. Training requires no synthetic before-and-after editing pairs or character segmentation.
Here, real-only training refers to editing supervision and STR-based text-encoder pre-training. The generative backbone uses pretrained Stable Diffusion components.
Checkpoint and architecture
| Item | Details |
|---|---|
| Released weights | textriro.pth |
| Inputs | A cropped scene-text image and the desired replacement text |
| Output | An edited crop with the requested text |
| Backbone components | Stable Diffusion 1.5 U-Net and scheduler; Stable Diffusion 2.1 VAE |
| Text conditioning | STR-pretrained character embeddings provided in the code repository |
| Generation resolution | 64 脳 256 pixels; the inference script restores the source dimensions |
| Training data | U14M-Filter, the filtered real-image subset of Union14M, distributed through OpenOCR-Data |
This repository distributes a PyTorch checkpoint for the project's custom implementation. Use the loading and inference scripts in the code repository.
Quick start
1. Prepare the implementation
git clone https://github.com/YesianRohn/TextRIRO.git
cd TextRIRO
Follow the repository's installation and checkpoint preparation instructions. The inference script requires CUDA. Prepare the SD 1.5 U-Net/scheduler, SD 2.1 VAE, and STR-pretrained text encoder at the paths specified in configs/train.yaml.
2. Download the checkpoint
Run this from the cloned repository root:
from huggingface_hub import hf_hub_download
hf_hub_download(
repo_id="Yesianrohn/TextRIRO",
filename="textriro.pth",
local_dir=".",
)
3. Prepare images and replacement text
Put source crops in example/i_s/. Create example/i_t.txt with one image filename and replacement string per line:
sample_001.png FLASH
sample_002.jpg Real In Real Out
Separate the filename and target with one space. Spaces inside the target are supported. Every source image must have a matching entry.
4. Generate edited crops
python inference.py \
--ckpt_path textriro.pth \
--dataset_dir example \
--output_dir example_result \
--num_inference_steps 20 \
--guidance_scale 2.2 \
--seed 42
Outputs are written to example_result/ using the source filenames. To try the model interactively, visit the online demo.
Scope and limitations
- The released inference script processes text crops. Full-image text detection and compositing require additional processing.
- The current encoder accepts up to 25 target characters; longer strings are truncated.
- Supported characters are defined by
EN_symbol_dict.txtplus the space character. The release should not be treated as a general multilingual editor. - Output text accuracy and visual consistency should be checked for each application. Detailed evaluations and comparisons are available in the paper.
Related resources
- TextRIRO-3M: a synthetic long-text recognition dataset created with TextRIRO.
- RIROCommon: a benchmark covering equal-length and variable-length scene text editing.
- Source code: training, inference, configurations, and evaluation guidance.
Citation
If you use TextRIRO in your research, please cite:
@inproceedings{ye2026textriro,
title = {Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?},
author = {Ye, Xingsong and Du, Yongkun and Zhang, Jiaxin and Sun, Chong and Chen, Zhineng},
booktitle = {Advances in Neural Information Processing Systems},
year = {2026},
url = {https://neurips.cc/virtual/2026/loc/sydney/poster/153296}
}
