Recapture: Create realistic scans and photos from digital documents to diversify AI training sets
GitHub repository: https://github.com/PSPDFKit-labs/recapture
One of the key problems when training VLM models is what's called the dataset mixture problem, or how to combine datasets to achieve the best possible model performance overall. For instance, a document model trained only on clean digital-born pages will perform well on those but not necessarily in domains where people scan documents with variable quality, like receipts or government IDs.
Most public document datasets, like DocLayNet or PubLayNet, contain lots of clean digital-born documents, but real-world photographed or scanned documents are usually not available in sufficient quantities to train specialized models. Moreover, those kinds of scanned documents are difficult to produce in high quantities, because that typically requires printing and then scanning or photographing millions of digital-born documents.
In this article, we introduce a new tool to help scale that process. Recapture is an open-source tool by Nutrient that helps data engineers scale the labeling process. It takes a clean page image and its layout boxes and then renders the page as a realistic phone photo or flatbed scan. You just need to label the page once, and Recapture produces as many degraded copies as you want, each one with the corresponding labeling already applied.
How Recapture Works
The usual way to make synthetic document photos is by means of 2D augmentation: Starting from a flat document, you apply some perspective, blur, Gaussian noise, or changes in brightness. While 2D augmentation usually works well for scans, it's not a very good technique for photograph generation. For instance, paper in a photo has some creases, folds, and shadows that 2D augmentation cannot replicate.
Recapture combines 2D augmentation (scan mode) and 3D augmentation (photo mode).
In scan mode, Recapture adds printer marks (like ink bleed, low toner, and toner streaks) and then scanner artifacts (like skew, dust, streaks, dark edges, and gray or black-and-white output). Scan mode is very fast and doesn’t need a GPU.
Photo mode is a more interesting feature. Recapture implements this algorithm:
- Divides the page into a grid with 1,024 points.
- Deforms the mesh to add crumples, folds, curls, and lifted edges. The crumples are produced by height maps of real scanned documents, so they are not simply a texture.
- Places the page on one of 35 possible desk surfaces, with optional desk items such as a keyboard, a mug, or sticky notes that cast shadows.
- Lights the scene with one lamp, room light, and optional hand and phone shadows.
- Takes the picture with a virtual phone camera, which adds focus blur, sensor noise, white balance, tone mapping, sharpening, and JPEG. You can choose quality from 1, a good camera, to 0, a cheap phone.
You can keep settings fixed or instruct your agent to use random ranges to produce the variability that training sets need. Recapture settings are in JSON format, so a human or agent can edit them easily.
If you are a human and like GUI's, Recapture help you capture the document environment intuitively:
Creating a Dataset with Recapture
Creating a dataset with Recapture is very easy. You just need clean page images and layout labels. Recapture accepts spatial layout boxes in a JSON format like this one:
{
"boxes": [
{
"id": "p1",
"label": "paragraph",
"bbox": [120, 340, 980, 610]
},
{
"id": "f1",
"label": "figure",
"quad": [[100, 700], [900, 700], [900, 1200], [100, 1200]]
}
]
}
You can use labels from an existing dataset, an annotation tool, or another document parser. Save them in the JSON format above, next to each page image: invoice.png and invoice.json. You may need to convert labels from formats such as COCO first.
If you don't have labels yet, you need a layout detector or a document parser to create them. Recapture also reads the spatial output of Nutrient Parse, our document parsing API, without a conversion step. We built that reader because we use the two tools together.
Install Recapture with Python 3.12 and uv, then download the texture assets (about 105 MB):
git clone https://github.com/PSPDFKit-labs/recapture.git
cd recapture
uv venv .venv --python 3.12
uv pip install --python .venv/bin/python -e '.[editor]' requests
.venv/bin/recapture fetch-assets --yes
source .venv/bin/activate
Put your page images in a new pages/ folder. The repo already has a docs/ folder with its own images. If you have a NUTRIENT_API_KEY set, this script sends each PNG to Nutrient Parse and saves the labels beside it:
import json
import os
from pathlib import Path
import requests
for page in sorted(Path("pages").glob("*.png")):
with page.open("rb") as f:
response = requests.post(
"https://api.nutrient.io/extraction/parse",
headers={"Authorization": f"Bearer {os.environ['NUTRIENT_API_KEY']}"},
files={"file": f},
data={"instructions": json.dumps({
"mode": "understand",
"output": {"format": "spatial"},
})},
)
response.raise_for_status()
page.with_suffix(".json").write_text(response.text)
The result from this script is a bunch of pairs such as invoice.png and invoice.json. Now you need to create a config in Recapture, check a few samples, then render and export:
recapture init my.json --from mix
recapture preview my.json -i pages/ -n 6
recapture render -i pages/ -c my.json -o dataset/ -n 10
recapture export dataset/ -o export/ --format coco --val 0.1 --test 0.1
The mix preset generates roughly 70% photos and 30% scans. You can adjust this to match the mixture of documents you need. Run recapture edit my.json --doc pages/invoice.png to change the settings. In this example, Recapture makes 10 variants per page and adjusts the labels accordingly.
The export splits by page, so all variants of a page land in the same split. With --val 0.1 --test 0.1, you need at least 10 pages to get a validation and a test split. With fewer pages, the export warns and writes only a training split.
Let Your Coding Agent Build the Dataset
You can also ask a coding agent to make the dataset for you. Recapture offers a convenient Claude Code plugin:
/plugin marketplace add PSPDFKit-labs/recapture
/plugin install recapture@recapture
If you use Codex, you can install the same plugin with codex plugin marketplace add PSPDFKit-labs/recapture and codex plugin add recapture@recapture.
Then open your agent in the folder with your pages and just describe the dataset you want in natural language:
Degrade the pages in pages/, 30% scans, 3 images per page, COCO export.
Your agent will ask before installing the CLI and downloading the textures. It'll then write a config, show you a preview, and wait for your approval. At the end, it reports the images and boxes in each split and the class list.
Your agent will use the label files already in the folder, so run the Nutrient Parse script first if your pages don't have labels yet.
Conclusion
Training document models takes more than clean digital pages. Receipts, IDs, and scanned forms come with folds, shadows, blur and uneven lighting. Recapture helps you add those variations to your training data without printing and photographing every page by hand.
You simply start with a few labeled pages, try different settings, and check the results. Once you find a mix that fits your use case, you can generate more examples, and the labels will already be in place, which saves you a lot of time.
Recapture is open source, and we'd like to hear how it works on your documents. Share your results, report bugs, or suggest features on GitHub.
