license: apache-2.0
base_model: stabilityai/stable-diffusion-3.5-medium
library_name: peft
tags:
- lora
- on-policy-distillation
Text-rendering distillation — PDM student (λ = 2)
The Positive-Direction Matching student from the text-rendering half of our study of classifier-free guidance in on-policy distillation: a single OCR teacher distilled into a student along the student's own denoising trajectories, on SD3.5-Medium.
pdm_lambda2.safetensors 71.7 MB LoRA over a frozen stabilityai/stable-diffusion-3.5-medium
adapter_config.json r=32, alpha=64, attention q/k/v + output projections
What this setting is for
It is the benign half of the paper's "when does NBA emerge?" comparison, and the result is that PDM and the naive objective perform the same.
Teacher and student here share one base model, one LoRA shape and one guidance scale, and the
teacher's negative branch is the plain unconditional branch — nothing the student cannot
represent (c⁻_T = c⁻_S = ∅). Under that shared negative conditioning both branch errors fall
together for every objective, so naive's under-identification never gets travelled. The
contrasting privileged-conditioning setting is where the two objectives separate.
Results
OCR reward (%, ↑) on the 1 018-prompt held-out set, 40 denoising steps, trained at γ = 4.5:
| Method | γ=1 | γ=2 | γ=3 | γ=4 | γ=4.5 |
|---|---|---|---|---|---|
| Teacher | 75.24 | 92.91 | 94.16 | 94.43 | 94.09 |
| OPD (Naive) | 73.87 | 92.72 | 94.52 | 94.43 | 94.11 |
| OPD (PDM, this) | 74.48 | 93.73 | 93.99 | 94.02 | 94.38 |
Both students track the teacher across the sweep, including the drop at γ = 1 — that drop is inherited from the teacher rather than introduced by distillation.
Reward is R_OCR = 1 − min(d, |s|) / |s|, where s is the target string, ŝ is what
PaddleOCR reads back, both lowercased with spaces stripped, and d = 0 if s occurs inside ŝ
and Lev(ŝ, s) otherwise. Note the substring short-circuit: rendering the target inside otherwise
unrelated text still scores 1.0, so this is a legibility score rather than a plain edit distance.
Training
| objective | PDM, train.opd_loss = "pdm", train.pdm_lambda = 2.0 |
| guidance | γ = 4.5, student and teacher |
| sampler | flow (ODE), noise_level = 0 ⇒ mean matching; 10 steps train / 40 eval |
| partial trajectory | timestep_fraction = 0.99 ⇒ K = 9 of 10 transitions supervised |
| optimiser | AdamW, constant lr 3e-4, grad clip 1.0, EMA decay 0.9 every 8 steps |
| batch | 8 GPUs × 3 prompts × 3 accumulated ⇒ 72 trajectories per update |
| steps | 1000 updates, seed 42 |
| teacher | upstream's released OCR teacher, quanhaol/DiffusionOPD |
The teacher is not included here — fetch it from upstream. (The PEFT checkpoint this was extracted from carried a copy of it; we verified it is byte-identical to the released one and dropped it.)
Use
The code lives in the text-rendering repo, a fork of
ali-vilab/DiffusionOPD:
torchrun --nproc_per_node=8 scripts/evaluation.py \
--checkpoint_path <dir containing lora/> \
--model_type sd3 --dataset ocr \
--guidance_scale 4.5 --num_inference_steps 40 --mixed_precision fp16 \
--output_dir evaluation_output/pdm_lambda2
evaluation.py expects the adapter at <checkpoint_path>/lora/adapter_model.safetensors, so
place pdm_lambda2.safetensors and adapter_config.json in a lora/ subdirectory, renaming the
weights to adapter_model.safetensors.
Licence
Apache-2.0. SD3.5-Medium and the upstream teacher carry their own terms.