Diaazg commited on
Commit
07fd252
Β·
verified Β·
1 Parent(s): 4ee05de

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +124 -3
README.md CHANGED
@@ -15,8 +15,129 @@ datasets:
15
  - Archatext/AraMS-Restore
16
  ---
17
 
18
- AraMS-Restore restores degraded lines from historical Arabic manuscripts. A U-Net takes a damaged handwritten line and returns a readable one, trained on synthetic damage (tears, ink blots, erasure, bleed-through, worm holes) calibrated against real manuscript decay.
19
 
20
- The contribution is the ablation between two objectives on the same generator: Level 1 optimizes visual fidelity (L1 + perceptual + optional PatchGAN), while Level 2 adds a frozen HATFormer OCR critic that reads the restored image and scores it against the true transcription β€” so gradients flow into the pixels that matter for reading. Both are evaluated on held-out manuscripts by CER before vs. after restoration, including on real damaged scans rather than only synthetic damage.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
- Trained on AraMS-28k-HTR (28,595 lines, 14 manuscripts, book-level disjoint splits). Includes the damage engine, evaluation scripts, and a drag-and-drop before/after web demo
 
15
  - Archatext/AraMS-Restore
16
  ---
17
 
18
+ # AraMS-Restore β€” restoration generators
19
 
20
+ Two U-Net generators that take a **degraded Arabic manuscript line** and return a
21
+ readable one. Same architecture, same data, same schedule β€” they differ only in
22
+ what they optimize:
23
+
24
+ | | `unimodal` (Level 1) | `recaware` (Level 2) |
25
+ |---|---|---|
26
+ | Loss | L1 + ResNet50 perceptual | the same **+ a frozen HATFormer OCR critic** |
27
+ | Optimizes for | visual fidelity | **legibility** |
28
+ | Text | not used | used as a loss target, never as an input |
29
+
30
+ The Level-2 critic reads the restored image and is scored against the true
31
+ transcription, so gradients flow into the pixels that matter for reading. Code:
32
+ [github.com/ArchaText/AraMS-Restore](https://github.com/ArchaText/AraMS-Restore).
33
+
34
+ ## Results
35
+
36
+ On **real** damaged lines from a held-out manuscript β€” the honest test β€” the
37
+ image-only model does not help, while the recognition-aware one does:
38
+
39
+ | Input to the reader | CER % | WER % |
40
+ |---|---|---|
41
+ | Degraded (untouched) | 29.81 | 59.98 |
42
+ | Level 1 restored | 30.15 | 59.98 |
43
+ | **Level 2 restored** | **28.15** | **58.44** |
44
+
45
+ The ranking *inverts* between the two ways of measuring β€” Level 1 wins every pixel
46
+ metric it is optimized for, and loses on reading:
47
+
48
+ | | PSNR ↑ | SSIM ↑ | CER ↓ |
49
+ |---|---|---|---|
50
+ | Degraded | 22.76 | 0.9498 | 35.11 |
51
+ | Level 1 | **34.66** | **0.9886** | 28.12 |
52
+ | Level 2 | 30.66 | 0.9781 | **27.13** |
53
+
54
+ *(Synthetic test set. CER read by the frozen HATFormer reader.)*
55
+
56
+ ## Files
57
+
58
+ ```
59
+ unimodal/best.pt 124 MB Level 1 generator (plain state_dict)
60
+ recaware/best.pt 124 MB Level 2 generator (plain state_dict)
61
+ ```
62
+
63
+ Each is a `state_dict` for the generator built by
64
+ `restoration.models.unet_pix2pix.build_generator`: 31.04 M parameters, float32,
65
+ RGB in/out, `base=64`, `depth=4`, no discriminator (pix2pix off).
66
+
67
+ ## Usage
68
+
69
+ ```bash
70
+ hf download Archatext/Restoration --local-dir checkpoints/restore
71
+ ```
72
+
73
+ That is exactly the layout the configs resolve, so the repo's scripts work
74
+ unchanged:
75
+
76
+ ```bash
77
+ python scripts/restore_real.py --score # CER before vs. after, both models
78
+ python app/server.py # drag-and-drop web demo
79
+ ```
80
+
81
+ Or load one directly:
82
+
83
+ ```python
84
+ import torch, yaml
85
+ from restoration.models.unet_pix2pix import build_generator
86
+
87
+ cfg = yaml.safe_load(open("configs/restore_recaware.yaml"))
88
+ gen = build_generator(cfg["model"])
89
+ gen.load_state_dict(torch.load("checkpoints/restore/recaware/best.pt", map_location="cpu"))
90
+ gen.eval()
91
+ ```
92
+
93
+ ### Input representation
94
+
95
+ A line is resized to height 64 keeping aspect and right-padded to width 1152
96
+ (`build_strip`), RGB in [0, 1], padding pure black and excluded by a mask. 1152 =
97
+ 3 Γ— 384 tiles cleanly into the HATFormer 384 Γ— 384 RTL-flipped canvas, which is
98
+ what makes the Level-2 recognition loss exact. Feeding a differently-shaped or
99
+ grayscale image will degrade output quality.
100
+
101
+ ## Training
102
+
103
+ Both models: 20 epochs, AdamW `lr=2e-4` (Ξ²=0.5, 0.999), batch 8, seed 1337, and 4
104
+ distinct degraded variants per clean line per epoch. Damage is synthesized **on the
105
+ fly** from clean `train`-split lines only β€” never val/test β€” by the localized
106
+ degradation engine (erasure, tears, worm holes, bleed-through, ink blots, ink
107
+ feathering), calibrated against real manuscript decay.
108
+
109
+ | | weights |
110
+ |---|---|
111
+ | `unimodal` | L1 1.0, perceptual 0.02 |
112
+ | `recaware` | image 1.0 (L1 1.0 + perceptual 0.02), recognition 0.5 |
113
+
114
+ Level 2's critic is [`Archatext/hatformer-arams28k`](https://huggingface.co/Archatext/hatformer-arams28k),
115
+ frozen throughout.
116
+
117
+ Training data: [AraMS-28k-HTR](https://huggingface.co/datasets/Archatext/AraMS-28k-HTR),
118
+ `train` split β€” 20,103 lines from 9 manuscripts.
119
+
120
+ ## Leakage
121
+
122
+ Splits are manuscript-level disjoint: the evaluation manuscripts (book_03, 05, 09)
123
+ are held out of restoration training entirely, **and** the OCR critic that scores
124
+ the output never trained on them either. Both conditions are needed β€” a critic that
125
+ had seen the test books would contaminate the training signal and the metric at
126
+ once.
127
+
128
+ ## Limitations
129
+
130
+ - Trained on **synthetic** degradation; real decay is a different distribution, and
131
+ the gap shows β€” the CER gain on real damage (1.7 points) is smaller than on the
132
+ synthetic test set.
133
+ - Single script tradition and a single corpus of 14 manuscripts; no claim of
134
+ generalization to other hands, papers or languages.
135
+ - Level 1 makes real damage marginally *harder* to read (30.15 vs 29.81 CER) while
136
+ scoring far better on PSNR/SSIM β€” do not select a restoration model on pixel
137
+ metrics alone.
138
+ - Line-level only. Page segmentation is upstream and not part of this release.
139
+
140
+ ## Citation
141
+
142
+ <!-- TODO: thesis / paper reference -->
143