---
license: apache-2.0
base_model: OpenMOSS-Team/MOSS-Image-Base
base_model_relation: adapter
library_name: peft
pipeline_tag: text-to-image
language:
  - sv
tags:
  - lora
  - peft
  - moss-image
  - handwriting
  - historical-documents
  - archives
  - htr
  - synthetic-data
widget:
  - text: "A flat, evenly lit archival scan of a handwritten page from the Bergskollegium records of Stora Kopparbergs bergslag."
    output:
      url: images/example.png
---

# diffu-moss-hand-eight

A LoRA adapter for [MOSS-Image-Base](https://huggingface.co/OpenMOSS-Team/MOSS-Image-Base) that writes in
eight archival hands in one adapter, each chosen by naming its collection in the prompt. It is part of [diffu](https://github.com/Borg93/diffu2), which draws synthetic archive pages with exact
ALTO and PAGE labels, for training handwriting and page-reading models.

![A window drawn with this adapter](images/example.png)

*A page window drawn by MOSS-Image-Base with this adapter; not a scan.*

## Files

| File | What |
|---|---|
| `adapter_config.json` | a PEFT `LoraConfig`: rank 32, alpha 32, on 265 Linear layers of the DiT |
| `adapter_model.safetensors` | the LoRA weights (bfloat16), PEFT's key layout |

The adapter covers the DiT's 42 joint blocks (`q`, `k`, `v`, `o`, `ffn_up_gate`, `ffn_down`), its text refiner, its two
embedders and its final layer. The module names are MOSS-Image's own, so it applies to any implementation that keeps
them; diffu's `MossTransformer` does.

## Use

With [diffu](https://github.com/Borg93/diffu2), loaded onto MOSS-Image-Base's own DiT at draw time (no merged model
needed; diffu's `docs/MOSS.md`, "Hand LoRAs", and `docs/INFERENCE.md`):

```bash
hf download OpenMOSS-Team/MOSS-Image-Base --revision 85975793d8b3d432f1b6e27f352abaa4c2a4c0d3 --local-dir models/MOSS-Image-Base
hf download Gabriel/diffu-moss-hand-eight --local-dir models/adapters/eight
uv run diffu-moss-init --base models/MOSS-Image-Base --out models/moss-base
uv run diffu-run --jobs run.jsonl --draw-settings draw.json --model models/moss-base \
    --lora hand=models/adapters/eight --out out/eight --gpus 1
```

`"fuse_loras": true` in `draw.json` merges the adapter into the DiT's weights at load (needed with MXFP8 or compiled
blocks).

Or with PEFT alone, merged into the DiT, which diffu then draws with like any MOSS-Image model:

```python
from peft import PeftModel

from diffu_moss.transformer import MossTransformer

dit = MossTransformer.from_pretrained("<MOSS-Image-Base snapshot>/transformer")
dit = PeftModel.from_pretrained(dit, "Gabriel/diffu-moss-hand-eight").merge_and_unload()
dit.save_pretrained("dit-eight")
```

```bash
diffu-moss-init --base <MOSS-Image-Base snapshot> --transformer dit-eight --out moss-eight
```

Base snapshot: `OpenMOSS-Team/MOSS-Image-Base` at revision `85975793d8b3d432f1b6e27f352abaa4c2a4c0d3`. The adapter takes
no sample page: the hand comes from the adapter and the prompt.

### Prompting

The hand is chosen by the collection the prompt's caption names, as in training (160 windows of each):

| Hand | Caption names it as |
|---|---|
| Göteborg register | "…a handwritten page from a Göteborg register, dated 1790–1794…" |
| Jämtlands domsaga | "…a handwritten page from the court records of Jämtlands domsaga…" |
| Stora Kopparbergs bergslag | "…a handwritten page from the Bergskollegium records of Stora Kopparbergs bergslag…" |
| Landskapshandlingar | "…a handwritten page from the landskapshandlingar provincial accounts…" |
| Överståthållarämbetet | "…a handwritten page from the records of Överståthållarämbetet…" |
| Påhlman's travel journals | "…a handwritten page from Carl Fredrik Påhlman's travel journals, dated 1824–1825…" |
| Riksregistraturet | "…a handwritten page from Riksregistraturet, the royal chancery register…" |
| Trolldomskommissionen | "…a handwritten page from the records of the witchcraft commissions (trolldomskommissionen), dated 1676…" |

Draw as MOSS-Image is drawn: about 1 MP, 50 steps, guidance 3, shift 5, block 22 skipped in the unconditional pass.

## Training

- Data: 1,280 windows of about 1 MP cut from pages of Swedish archival handwriting ground truth, each
  captioned as a MOSS Layout prompt (the collection named, every line's exact text and box).
- Trainer: KohakuMossTrainer, a LoRA (LyCORIS, algo `lora`), rank 32, alpha 32; AdamW at 1e-4,
  2,000 steps; MOSS-Image's own noise-level sampling; text encoder and VAE frozen.
- Format: converted from the trainer's LyCORIS export to PEFT's layout. The conversion was checked layer by layer: every
  key lands on a layer PEFT builds from this config, none is missing or left over, and scaling x B @ A equals the
  trained (alpha / r) x up @ down exactly.

## Limitations

- Eight hands only, each from one collection; other hands
  and kinds of page are untested.
- The drawn text must be checked: diffu reads every drawn line back and tags a line it cannot confirm as unclear in
  the labels.

## Licence

Apache-2.0, as MOSS-Image-Base.