README.md

metadata

license: mit
library_name: pytorch
pipeline_tag: image-to-image
tags:
  - handwriting-synthesis
  - vision-transformer
  - generative-adversarial-network
  - pytorch
  - computer-vision
  - iam
  - vietnamese-handwriting

WriteViT: Handwritten Text Generation with Vision Transformers

WriteViT is a one-shot handwritten text synthesis framework that learns a writer's style from a small set of reference images and generates handwritten text in that style. The model combines a ViT-based writer encoder, a multi-scale Transformer generator with conditional positional encoding, and a lightweight ViT recognizer.

Model Summary

Field	Description
Task	Handwritten text image generation
Framework	PyTorch
Architecture	ViT writer encoder, Transformer generator, ViT recognizer
Supported Data	IAM English handwriting and Vietnamese handwriting data
Image Height	32px
Checkpoints	English and Vietnamese checkpoints included
Intended Use	Research, reproducibility, handwriting synthesis, and data augmentation experiments

Highlights

• Learns writer style from a small set of reference handwriting images.
• Generates handwritten text conditioned on target text and writer style.
• Supports English IAM-style handwriting and Vietnamese handwriting generation.
• Includes prepared dataset pickles, lexicons, font templates, and released checkpoints.
• Provides training code and qualitative result figures in one reproducible PyTorch release.

Qualitative Results

Handwriting Generation

Handwriting Reconstruction

Released Artifacts

File	Purpose
File/eng_ckpt.pth	Released English/IAM checkpoint
File/vn_ckpt.pth	Released Vietnamese checkpoint
File/vgg19.pth	VGG19 backbone checkpoint/resource used by the project
File/IAM.pickle	Prepared IAM handwriting dataset pickle
File/VN.pickle	Prepared Vietnamese handwriting dataset pickle
File/unifont.pickle	Font/template data for query rendering
File/english_words.txt	English lexicon
File/vn_words.txt	Vietnamese lexicon

Repository Structure

.
|-- data/                 # Dataset loading and preparation utilities
|-- Figures/              # Architecture and qualitative result figures
|-- File/                 # Datasets, checkpoints, lexicons, and font resources
|-- models/               # Generator, discriminators, recognizer, and writer encoder
|-- util/                 # Shared model and training utilities
|-- params.py             # Experiment and dataset configuration
|-- train.py              # Training entry point
`-- requirements.txt

Installation

Python 3.7 or newer and a CUDA-capable GPU are recommended for training.

Install PyTorch for your CUDA version first, then install the project dependencies:

pip install -r requirements.txt

Configuration

Experiment settings are defined in params.py.

The default configuration uses IAM:

DATASET = 'IAM'
DATASET_PATHS = './File/IAM.pickle'
NUM_WRITERS = 339
WORDS_PATH = './File/english_words.txt'

To train or evaluate with Vietnamese data, switch to:

DATASET = 'VNDB'
DATASET_PATHS = './File/VN.pickle'
NUM_WRITERS = 106
WORDS_PATH = './File/vn_words.txt'

The available recognizer backbones are resnet18, vgg11, and vgg19.

Training

Review params.py, especially DATASET, DATASET_PATHS, NUM_WRITERS, WORDS_PATH, BACKBONE, learning rates, batch size, and RESUME.

Start training:

CUDA_VISIBLE_DEVICES=0 python train.py

Outputs are written to:

saved_models/<EXP_NAME>/
saved_images/<EXP_NAME>/

When RESUME = True, the training script loads:

saved_models/<EXP_NAME>/model.pth

Data Format

Prepared dataset pickle files contain writer-split handwriting samples:

{
    "train": {
        "writer_id": [
            {"img": PIL.Image.Image, "label": "handwritten text"}
        ]
    },
    "test": {
        "writer_id": [
            {"img": PIL.Image.Image, "label": "handwritten text"}
        ]
    }
}

Resources

• Paper: https://arxiv.org/abs/2505.13235
• Interactive demo: https://colab.research.google.com/drive/15Lswqr-aQwI-fF6yRoGYt-2pxSlC2L-R
• Original datasets/checkpoints folder: https://drive.google.com/drive/folders/1ZgYS6-6l6fjKY75RJipONBByujIgf-uE

Citation

If you use WriteViT in your research, please cite:

@article{nam2025writevit,
  title   = {WriteViT: Handwritten Text Generation with Vision Transformer},
  author  = {Dang Hoai Nam and Huynh Tong Dang Khoa and Vo Nguyen Le Duy},
  journal = {arXiv preprint arXiv:2505.13235},
  year    = {2025}
}

Acknowledgements

This repository builds on Handwriting Transformers by Ankan Kumar Bhunia et al. We thank the authors for making their work publicly available.