| --- |
| license: apache-2.0 |
| base_model: OpenMOSS-Team/MOSS-Image-Base |
| base_model_relation: adapter |
| library_name: peft |
| pipeline_tag: text-to-image |
| language: |
| - sv |
| tags: |
| - lora |
| - peft |
| - moss-image |
| - handwriting |
| - historical-documents |
| - archives |
| - htr |
| - synthetic-data |
| widget: |
| - text: "A flat, evenly lit archival scan of a handwritten page from the Bergskollegium records of Stora Kopparbergs bergslag." |
| output: |
| url: images/example.png |
| --- |
| |
| # diffu-moss-hand-eight |
|
|
| A LoRA adapter for [MOSS-Image-Base](https://huggingface.co/OpenMOSS-Team/MOSS-Image-Base) that writes in |
| eight archival hands in one adapter, each chosen by naming its collection in the prompt. It is part of [diffu](https://github.com/Borg93/diffu2), which draws synthetic archive pages with exact |
| ALTO and PAGE labels, for training handwriting and page-reading models. |
|
|
|  |
|
|
| *A page window drawn by MOSS-Image-Base with this adapter; not a scan.* |
|
|
| ## Files |
|
|
| | File | What | |
| |---|---| |
| | `adapter_config.json` | a PEFT `LoraConfig`: rank 32, alpha 32, on 265 Linear layers of the DiT | |
| | `adapter_model.safetensors` | the LoRA weights (bfloat16), PEFT's key layout | |
|
|
| The adapter covers the DiT's 42 joint blocks (`q`, `k`, `v`, `o`, `ffn_up_gate`, `ffn_down`), its text refiner, its two |
| embedders and its final layer. The module names are MOSS-Image's own, so it applies to any implementation that keeps |
| them; diffu's `MossTransformer` does. |
|
|
| ## Use |
|
|
| With [diffu](https://github.com/Borg93/diffu2), loaded onto MOSS-Image-Base's own DiT at draw time (no merged model |
| needed; diffu's `docs/MOSS.md`, "Hand LoRAs", and `docs/INFERENCE.md`): |
|
|
| ```bash |
| hf download OpenMOSS-Team/MOSS-Image-Base --revision 85975793d8b3d432f1b6e27f352abaa4c2a4c0d3 --local-dir models/MOSS-Image-Base |
| hf download Gabriel/diffu-moss-hand-eight --local-dir models/adapters/eight |
| uv run diffu-moss-init --base models/MOSS-Image-Base --out models/moss-base |
| uv run diffu-run --jobs run.jsonl --draw-settings draw.json --model models/moss-base \ |
| --lora hand=models/adapters/eight --out out/eight --gpus 1 |
| ``` |
|
|
| `"fuse_loras": true` in `draw.json` merges the adapter into the DiT's weights at load (needed with MXFP8 or compiled |
| blocks). |
|
|
| Or with PEFT alone, merged into the DiT, which diffu then draws with like any MOSS-Image model: |
|
|
| ```python |
| from peft import PeftModel |
| |
| from diffu_moss.transformer import MossTransformer |
| |
| dit = MossTransformer.from_pretrained("<MOSS-Image-Base snapshot>/transformer") |
| dit = PeftModel.from_pretrained(dit, "Gabriel/diffu-moss-hand-eight").merge_and_unload() |
| dit.save_pretrained("dit-eight") |
| ``` |
|
|
| ```bash |
| diffu-moss-init --base <MOSS-Image-Base snapshot> --transformer dit-eight --out moss-eight |
| ``` |
|
|
| Base snapshot: `OpenMOSS-Team/MOSS-Image-Base` at revision `85975793d8b3d432f1b6e27f352abaa4c2a4c0d3`. The adapter takes |
| no sample page: the hand comes from the adapter and the prompt. |
|
|
| ### Prompting |
|
|
| The hand is chosen by the collection the prompt's caption names, as in training (160 windows of each): |
|
|
| | Hand | Caption names it as | |
| |---|---| |
| | Göteborg register | "…a handwritten page from a Göteborg register, dated 1790–1794…" | |
| | Jämtlands domsaga | "…a handwritten page from the court records of Jämtlands domsaga…" | |
| | Stora Kopparbergs bergslag | "…a handwritten page from the Bergskollegium records of Stora Kopparbergs bergslag…" | |
| | Landskapshandlingar | "…a handwritten page from the landskapshandlingar provincial accounts…" | |
| | Överståthållarämbetet | "…a handwritten page from the records of Överståthållarämbetet…" | |
| | Påhlman's travel journals | "…a handwritten page from Carl Fredrik Påhlman's travel journals, dated 1824–1825…" | |
| | Riksregistraturet | "…a handwritten page from Riksregistraturet, the royal chancery register…" | |
| | Trolldomskommissionen | "…a handwritten page from the records of the witchcraft commissions (trolldomskommissionen), dated 1676…" | |
|
|
| Draw as MOSS-Image is drawn: about 1 MP, 50 steps, guidance 3, shift 5, block 22 skipped in the unconditional pass. |
|
|
| ## Training |
|
|
| - Data: 1,280 windows of about 1 MP cut from pages of Swedish archival handwriting ground truth, each |
| captioned as a MOSS Layout prompt (the collection named, every line's exact text and box). |
| - Trainer: KohakuMossTrainer, a LoRA (LyCORIS, algo `lora`), rank 32, alpha 32; AdamW at 1e-4, |
| 2,000 steps; MOSS-Image's own noise-level sampling; text encoder and VAE frozen. |
| - Format: converted from the trainer's LyCORIS export to PEFT's layout. The conversion was checked layer by layer: every |
| key lands on a layer PEFT builds from this config, none is missing or left over, and scaling x B @ A equals the |
| trained (alpha / r) x up @ down exactly. |
|
|
| ## Limitations |
|
|
| - Eight hands only, each from one collection; other hands |
| and kinds of page are untested. |
| - The drawn text must be checked: diffu reads every drawn line back and tags a line it cannot confirm as unclear in |
| the labels. |
|
|
| ## Licence |
|
|
| Apache-2.0, as MOSS-Image-Base. |
|
|
