Research release · trained from scratch研究发布 · 从零训练

Write Chinese into manga, not over it. 把中文,
写进漫画里。

A 119.3M-parameter diffusion transformer. Give it a page and the translated lines: it redraws the lettering inside the balloon and leaves the artwork untouched. 一个 1.193 亿参数的扩散 Transformer。给它漫画页和译文,它在气泡里重新写出文字,画面其余部分原样保留。

119.3Mparams参数 MITcode + weights代码与权重
The same panel with the balloon lettered 早上好 in Chinese by the v3 checkpoint. A gardener greets a cat; the balloon says おはよう in Japanese.
erase
box
Japanese日文原页 v3 outputv3 输出
Drag to compare拖动对比
01 · METHOD方法

From page to lettering in four moves.原页到嵌字,
分四步。

The Chinese arrives as a glyph image, the flow starts from the source page itself, and only the edited region is pasted back. The art outside it is never regenerated.中文以字形图输入,流从原页本身出发,只把被编辑的区域贴回原图,区域外的画面不会被重新生成。

H×W×3 → h×w×16
01

Encode编码

A frozen 16-channel VAE compresses the page to a latent 8× smaller.冻结的 16 通道 VAE 把页面压成 8 倍小的隐变量。

早上好64 channels
02

Typeset a glyph map排字形图

The text is set in a plain font inside your box and folded into 64 channels, bypassing the VAE so strokes stay sharp.译文用标准字体排进你给的框,折叠成 64 个通道,绕开 VAE,笔画不会被糊掉。

source + 0.5·εt: 1 → 0
03

Flow from the source从原页出发

A 16-layer DiT starts near the source and predicts velocity over overlapping 512 px tiles.16 层 DiT 从原页附近出发,在重叠的 512 像素分块上预测速度。

soft mask α
04

Composite合成

Decode, then blend through a feathered mask. Pixels outside the region come straight from the input.解码后用羽化蒙版混合,区域外的像素直接取自输入。

MangaTrans-Diffusion architecture from source page and glyph condition through VAE, DiT rectified flow, tiled sampling and local compositing.
Released v3 inference path公开 v3 推理路径Open full size ↗打开大图 ↗
02 · CLOSE-UP细节

Look closely.放大看。

Four times the size, straight from the model with no retouching: vertical lettering set cleanly inside the balloon.放大四倍,模型直出、没有修饰:竖排文字干净地排进气泡。

Zoomed Japanese balloon
おはようinput · ×4输入 · ×4
Zoomed Chinese balloon
早上好v3 output · ×4v3 输出 · ×4

Render this exact panel.复现这一格。

Needs a CUDA GPU and the separately licensed FLUX.1-dev VAE. Setup is in the README. Input file ↗需要 CUDA 显卡和单独授权的 FLUX.1-dev VAE。配置见 中文说明。输入文件 ↗

python translate_page.py showcase/example/source.png \
  showcase/example/regions.json out.png \
  --weights weights/mangatrans-v3 \
  --width 0 --steps 30 --seed 0 --margin 12