File size: 4,891 Bytes
4ee05de 07fd252 554fc9f 07fd252 554fc9f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 | ---
language:
- ar
pipeline_tag: image-to-image
tags:
- arabic
- manuscripts
- historical-documents
- document-restoration
- image-restoration
- unet
- pytorch
datasets:
- Archatext/AraMS-28k-HTR
- Archatext/AraMS-Restore
---
# AraMS-Restore β restoration generators
Two U-Net generators that take a **degraded Arabic manuscript line** and return a
readable one. Same architecture, same data, same schedule β they differ only in
what they optimize:
| | `unimodal` (Level 1) | `recaware` (Level 2) |
|---|---|---|
| Loss | L1 + ResNet50 perceptual | the same **+ a frozen HATFormer OCR critic** |
| Optimizes for | visual fidelity | **legibility** |
| Text | not used | used as a loss target, never as an input |
The Level-2 critic reads the restored image and is scored against the true
transcription, so gradients flow into the pixels that matter for reading. Code:
[github.com/ArchaText/AraMS-Restore](https://github.com/ArchaText/AraMS-Restore).
## Results
On **real** damaged lines from a held-out manuscript β the honest test β the
image-only model does not help, while the recognition-aware one does:
| Input to the reader | CER % | WER % |
|---|---|---|
| Degraded (untouched) | 29.81 | 59.98 |
| Level 1 restored | 30.15 | 59.98 |
| **Level 2 restored** | **28.15** | **58.44** |
The ranking *inverts* between the two ways of measuring β Level 1 wins every pixel
metric it is optimized for, and loses on reading:
| | PSNR β | SSIM β | CER β |
|---|---|---|---|
| Degraded | 22.76 | 0.9498 | 35.11 |
| Level 1 | **34.66** | **0.9886** | 28.12 |
| Level 2 | 30.66 | 0.9781 | **27.13** |
*(Synthetic test set. CER read by the frozen HATFormer reader.)*
## Files
```
unimodal/best.pt 124 MB Level 1 generator (plain state_dict)
recaware/best.pt 124 MB Level 2 generator (plain state_dict)
```
Each is a `state_dict` for the generator built by
`restoration.models.unet_pix2pix.build_generator`: 31.04 M parameters, float32,
RGB in/out, `base=64`, `depth=4`, no discriminator (pix2pix off).
## Usage
```bash
hf download Archatext/Restoration --local-dir checkpoints/restore
```
That is exactly the layout the configs resolve, so the repo's scripts work
unchanged:
```bash
python scripts/restore_real.py --score # CER before vs. after, both models
python app/server.py # drag-and-drop web demo
```
Or load one directly:
```python
import torch, yaml
from restoration.models.unet_pix2pix import build_generator
cfg = yaml.safe_load(open("configs/restore_recaware.yaml"))
gen = build_generator(cfg["model"])
gen.load_state_dict(torch.load("checkpoints/restore/recaware/best.pt", map_location="cpu"))
gen.eval()
```
### Input representation
A line is resized to height 64 keeping aspect and right-padded to width 1152
(`build_strip`), RGB in [0, 1], padding pure black and excluded by a mask. 1152 =
3 Γ 384 tiles cleanly into the HATFormer 384 Γ 384 RTL-flipped canvas, which is
what makes the Level-2 recognition loss exact. Feeding a differently-shaped or
grayscale image will degrade output quality.
## Training
Both models: 20 epochs, AdamW `lr=2e-4` (Ξ²=0.5, 0.999), batch 8, seed 1337, and 4
distinct degraded variants per clean line per epoch. Damage is synthesized **on the
fly** from clean `train`-split lines only β never val/test β by the localized
degradation engine (erasure, tears, worm holes, bleed-through, ink blots, ink
feathering), calibrated against real manuscript decay.
| | weights |
|---|---|
| `unimodal` | L1 1.0, perceptual 0.02 |
| `recaware` | image 1.0 (L1 1.0 + perceptual 0.02), recognition 0.5 |
Level 2's critic is [`Archatext/hatformer-arams28k`](https://huggingface.co/Archatext/hatformer-arams28k),
frozen throughout.
Training data: [AraMS-28k-HTR](https://huggingface.co/datasets/Archatext/AraMS-28k-HTR),
`train` split β 20,103 lines from 9 manuscripts.
## Leakage
Splits are manuscript-level disjoint: the evaluation manuscripts (book_03, 05, 09)
are held out of restoration training entirely, **and** the OCR critic that scores
the output never trained on them either. Both conditions are needed β a critic that
had seen the test books would contaminate the training signal and the metric at
once.
## Limitations
- Trained on **synthetic** degradation; real decay is a different distribution, and
the gap shows β the CER gain on real damage (1.7 points) is smaller than on the
synthetic test set.
- Single script tradition and a single corpus of 14 manuscripts; no claim of
generalization to other hands, papers or languages.
- Level 1 makes real damage marginally *harder* to read (30.15 vs 29.81 CER) while
scoring far better on PSNR/SSIM β do not select a restoration model on pixel
metrics alone.
- Line-level only. Page segmentation is upstream and not part of this release.
## Citation
<!-- TODO: thesis / paper reference -->
|