--- license: cc-by-nc-4.0 tags: - vesuvius-challenge - herculaneum - papyrology - image-classification - pytorch library_name: pytorch pipeline_tag: image-classification --- # Herculaneum legibility proxy (proxy_v4 / proxy_s2s3_v1) A small ResNet-18 binary classifier that scores ~1 cm windows of Vesuvius Challenge **ink-probability maps** for *legible-text likelihood*: does this window contain connected Greek letterforms, or just fiber texture / noise / damage? It is a **triage tool, not an ink detector**. It runs *downstream* of the official ink-detection models, on their output maps, and answers a different question: "of all this detected ink, where should a human look first?" Built and used to produce a full legibility index of Scroll 1 (PHerc. Paris 4): **125,298 windows scored across all 78 of Scroll 1's currently-published official ink maps** (complete coverage, verified live against the S3 bucket 2026-07-08 — see "Coverage" below for how this grew from an initial 16-panel pass). 5,678 high-confidence text windows on the deduplicated wrap-series union (w010–w129); classic-lineage segments add 2,460 more on 18 physical surfaces (kept separate — their overlap with the series is unresolved), and 7,248 further gold windows come from redundant re-renders, counted only as consistency checks (see Coverage below). The same checkpoint, unmodified, was then run on three more scrolls with no fine-tuning in between: | Scroll | Windows scored | Segments/maps used | Coverage of official maps | Gold (≥0.9) | Independent check | |---|---|---|---|---|---| | Scroll 1 (PHerc. Paris 4) | 125,298 | 78 official segments (16 original panels + 62 added 2026-07-08) | 78 of 78 — complete | 5,678 on the deduplicated wrap-series union w010–w129 (+2,460 on classic-lineage segments, kept separate; +7,248 redundant re-renders — see Coverage) | 3 held-out panels, AUROC 0.985; wrap w028-037 independently gold-confirmed in 2 unrelated renders (50.9% and 40.0% gold density) | | PHerc 0139 | 4,905 | 38 | 38 of 38 — complete | 63 | title segment ranked 2nd of 38 by gold density; 63/63 human-reviewed (single reviewer): 54 clear-text / 9 possible / 0 rejected | | Scroll 5 (PHerc 172) | 24,528 | 53 × 2 ink-detector models | 53 of 53 — complete | 3 (by `july_retreat`-model score; 0 by the more conservative `november19` model — see cross-render caveat below) | max score 0.938 on a window with plainly visible, large uppercase Greek letters (segment w066) | | Scroll 4 (PHerc 1667) | 4,761 | 19 official maps | 19 of 19 segments that carry an official ink map (1 of the 20 public segments has none) — complete | 430 | per-wrap gold fraction correlates at Spearman r = 0.843 (p < 0.0001, n = 18) with the count of Greek letters *already transcribed* in that wrap — ground truth external to this model | ### Coverage — how the Scroll 1 index reached 78/78 The index originally covered only 16 of Scroll 1's 81 public segments (one continuous panel series, `w010`–`w100`, uploaded 2026-06-23) — 29,853 windows, 4,491 gold. A live S3 check on 2026-07-08 found the scroll had grown to 78 populated official maps (3 of the 81 public segments still have none). The remaining 62 were scored with the identical unmodified pipeline (same `proxy_v4.pt`, same WIN=512/STRIDE=256 grid, same ds8-resolution preprocessing — verified byte-for-byte SHA256-identical against the official pre-made `ink-detection/downsampled/*-ds8.jpg` product before trusting it on new segments), split into four groups: | Group | Segments | Windows | Gold (≥0.9) | Territory | |---|---|---|---|---| | 23 "classic"-lineage panels (name-level dedup: 18 physical surfaces) | 23 | 8,978 | 2,684 (2,460 on representatives + 224 on re-renders) | **separate bucket** — overlap with the panel series unresolved, so never summed into the wrap-series denominator | | 11 more panels from the same June-2026 series, `w101`–`w129` | 11 | 14,998 | 1,056 | **distinct** — extends past the original `w100` boundary | | 2 alternate-processing panels, `w046-052_jordi` / `w053-058_jordi` | 2 | 3,929 | 739 | **overlaps** — re-renders of `w046-052`/`w053-058`, already in the original 16 panels (confirmed by near-identical gold-density-by-row fingerprint against the originals) | | 26 panels from a second complete series uploaded 2026-07-01, covering `w010`–`w129` | 26 | 67,540 | 6,416 | **one panel is unique territory**: the June `w104-106` segment has no published ink map; the July `w104-106` panel (2,648 windows, 131 gold) covers that range and joins the deduplicated union. The other 25 panels (6,285 gold) re-cover indexed wraps and stay redundant | **Correction (2026-07-08, caught in adversarial review):** the `_jordi` panels were originally folded into the "distinct" count because their segment IDs looked like part of the `w101`–`w129` extension. They aren't — they cover wrap ranges already present in the original 16-panel index. Moved to the overlap bucket; the headline distinct-territory number dropped from an incorrect 8,970 to a correct 8,231 as a result. The July-2026 series is confirmed *not* a byte-identical duplicate of the June one (e.g. the `w010-027` file is 682 MB in June vs 1.07 GB in July, same recipe tag), but it covers the same physical wraps, so — like the `_jordi` panels — its gold windows are reported separately rather than folded into the "distinct territory" total, to avoid double-counting the same underlying text. [Update, 2026-07-11: with one exception found later — the July `w104-106` panel is unique territory; see the correction below.] Both overlap groups are useful anyway as independent consistency checks: wrap range `w028-037` (June vs July) scores gold at 50.9%/40.0%, and `w046-052`/`w053-058` (original vs `_jordi`) show matching gold-density fingerprints band-by-band — unrelated renders of the same regions agreeing on where the text is. **Correction (2026-07-11, peer review):** an earlier version of this card reported "8,231 gold on distinct territory" by summing the wrap-series panels with the classic segments. Two problems, both fixed here: the June `w104-106` segment has no published ink map, so the July panel covering those wraps is unique territory (131 gold) wrongly excluded as redundant; and the classic bucket contained re-renders of the same physical surfaces (a `_copy`, a `_v14`, a `_v2_flatboi`, a `_v8`, an offset-0 re-render — 23 panels are 18 surfaces) and its physical overlap with the panel series is unresolved. **Canonical accounting: deduplicated wrap-series union (June w010–w129 + July w104-106) = 5,678 gold windows; classic-lineage representatives = 2,460 gold on 18 surfaces, reported separately; redundant re-renders = 7,248 gold (224 classic + 739 `_jordi` + 6,285 July), used only as consistency checks.** Cross-check invariant: 5,678 + 2,460 + 224 + 739 + 6,285 = 15,386 gold rows in the full 125,298-window index — **shipped as `full_index_complete.json` in the companion dataset repo** (SHA256 `4d393d70ce886ed62b7e73e365f1d01cbe7f6efa37168fb3f27ade2b89d6e7a8`; every bucket and denominator above is recomputed with asserts by its `summarize_index.py`). Note the score key in that file is named `v3` for legacy reasons; every score in it was produced by `proxy_v4.pt`. ## Files | File | What it is | MD5 (first 12) | |---|---|---| | `proxy_v4.pt` | main checkpoint, trained on official Scroll 1 panel maps | `c869ce189f2a` | | `proxy_s2s3_v1.pt` | domain-calibrated variant: same S1 knowledge + 192 windows of self-rendered Scroll 2/3 maps as explicit negatives | `95b6839c58c8` | Both are plain `state_dict`s for `torchvision.models.resnet18()` with `fc = nn.Linear(512, 1)` (122 tensors). ### Which one to use — the split is by *rendering pipeline*, not by scroll `proxy_v4` generalizes across **scrolls**, as long as the map came out of the official/community rendering pipeline (the texture statistics of an "official-style" ink map are consistent regardless of which scroll it's of). See the table above for the four-scroll validation: one native, one blind positive control (0139), one cross-render-calibration caveat that's about the ink detector, not this classifier (Scroll 5), and one check against fully independent, non-visual ground truth — transcribed-letter counts, not just human eyeballing (Scroll 4; project-internal validation note — the correlation, n, and p reported above are the complete result). `proxy_s2s3_v1` exists because that assumption breaks on maps rendered by a **custom / non-official pipeline** — which for us happened to be Scrolls 2 and 3, only because no official ink maps existed yet for the regions we needed and we rendered our own. On those self-rendered mosaics, `proxy_v4` alone gives an inflated background-noise floor (~0.04); the 192 self-rendered-domain negatives used to fine-tune `proxy_s2s3_v1` drop that to ~0.003 without hurting Scroll 1 performance (AUROC 0.984 vs 0.985). It was never evaluated on official maps of other scrolls (0139, Scroll 5) — there's no reason to prefer it over `proxy_v4` there, since `proxy_v4` already has evidence of working on that domain and `proxy_s2s3_v1` doesn't. | Map came from | Use | |---|---| | Official / community rendering pipeline (any scroll) | `proxy_v4.pt` | | Your own custom rendering pipeline | `proxy_s2s3_v1.pt`, or better: fine-tune your own domain-negatives the same way (see Training summary) | ## How to use The preprocessing is part of the model. Deviate from it and the scores shift silently — this exact recipe is what the training used: ```python import numpy as np import torch, torch.nn as nn, torchvision from PIL import Image m = torchvision.models.resnet18() m.fc = nn.Linear(512, 1) m.load_state_dict(torch.load("proxy_v4.pt", map_location="cpu")) m.eval() def score_window(win_u8): # win_u8: 2D uint8 crop of an ink map c = win_u8.astype(np.float32) active = c[c > 10] if len(active) > 50: # percentile stretch over active pixels lo, hi = np.percentile(active, [2, 99.5]) c = np.clip((c - lo) / max(hi - lo, 1e-6), 0, 1) else: c = c / 255.0 img = Image.fromarray((c * 255).astype(np.uint8)).resize((160, 160), Image.BILINEAR) t = torch.from_numpy(np.array(img, np.float32) / 255.0)[None].repeat(3, 1, 1) with torch.no_grad(): return torch.sigmoid(m(t[None])).item() ``` ### The window must be ~1 cm physical — this is the #1 usage error The model was trained on 512 px windows of ds8 panels at ≈19.2 µm/px ≈ **9,830 µm ≈ 1 cm** per window. On any other map, compute the window size from the segment's real physical scale, e.g.: ```python px_um = (area_cm2 * 1e8 / (H * W)) ** 0.5 # from the mesh meta.json area WIN = round(9830 / px_um) ``` Two failure modes we hit ourselves, so you don't have to: 1. **Do not assume 512 px.** On a 7.91 µm-scan ds8 map the right window is ~155 px; on a 2.4 µm-scan ds8 map it's ~512–560 px. 2. **Do not stretch undersized crops.** If a fragment is smaller than WIN, resizing what you have up to 160×160 changes the fiber texture's apparent frequency and *inflates scores* (we measured score anti-correlating with true window size, r ≈ −0.45, before fixing this). Pad onto a black 160×160 canvas at the training scale (61.4 µm per classifier pixel) instead, and treat such windows with extra suspicion. ### Thresholds we used - **≥ 0.9** — "gold": on Scroll 1, round-2 human review confirmed 120/120 of the model's top-ranked proposals. On PHerc 0139, all 63 gold windows were human-reviewed (single reviewer): 54 clear text (85.7%), 9 possible, 0 rejected as noise/artifact. These are one reviewer's visual verdicts on model-selected windows, not population precision estimates. - **0.35–0.70** — genuinely ambiguous band; useful for uncertainty sampling if you're fine-tuning. - Scores are *not* calibrated probabilities, and they do **not transfer across ink-model renders**: on Scroll 5, the same three candidate windows (segment w066 and neighbors) scored 0.938 / 0.914 / 0.903 on `july_retreat`-model maps but 0.266 / 0.029 / 0.334 on `november19`-model maps of the identical windows — pixel correlation between the two maps in those windows is 0.86–0.89 (i.e. both detectors render essentially the same visible letterforms; only the score disagrees). Rank within one map family; never compare raw scores across families, and never take an AND-style agreement filter (`min(scoreA, scoreB) ≥ 0.9`) at face value without checking whether one model is just systematically deflated — we lost 3 true positives to exactly that before catching it. ## Training summary - **Architecture:** torchvision ResNet-18 (ImageNet init), `fc → 1`, BCE-with-logits, AdamW. - **Data (proxy_v4):** 541 positive windows from three rounds of human-in-the-loop review of official Scroll 1 panel maps (393 browse-all + 120 model-proposed/human-confirmed + 28 uncertainty-sampled), negatives from 50 GPU-verified fiber windows (shipped in the companion dataset repo as `fiber_negatives_50.jsonl`; 12 of the 50 fall in the validation panels), background sampling, and 87 human-implied negatives at weight 0.7; 22 human-"unsure" windows excluded. Spatial validation: 3 full held-out panels → AUROC 0.985. (Corrected 2026-07-08: the browse-all round produced 398 raw rows, but 5 of those were tagged "unsure," not positive — 393 + 120 + 28 = 541. The full, independently re-verified breakdown ships as training *data*, not just this prose summary — see the companion dataset repo: https://huggingface.co/datasets/LimeGS/herculaneum-legibility-proxy-labels.) - **proxy_s2s3_v1:** initialized from proxy_v4, fine-tuned at LR 1e-4 with self-rendered Scroll 2/3 negatives: 23 human-reviewed noise windows at weight 1.0, plus ~169 additional implicit negatives at weight 0.7 from the same rendering pass. The 7 human-flagged candidate windows were excluded from training entirely. **Reproducibility caveat:** only the 23 explicitly-reviewed windows are traceable in this release — the ~169 implicit negatives were read at training time from a session-scratchpad file that no longer exists, so they can't currently be regenerated bit-for-bit. This doesn't change what was trained (the weights are final), it means the companion dataset repo can reproduce 23 of the ~192 S2/S3 negatives, not all of them — noted there in full, not glossed over here. - Trains in minutes on a laptop (Apple MPS, CUDA, or CPU); this is deliberately a small, reproducible model. ## Training data release The human-labeled training coordinates (not images — just `{panel, y, x, label, weight}`, since the underlying maps are already public) are released alongside this model at https://huggingface.co/datasets/LimeGS/herculaneum-legibility-proxy-labels, with a crop-regeneration script, a consolidated training script, pinned dependency versions, and a self-test. Scope of reproducibility, stated precisely: the full `proxy_v4` recipe (labels + 50 fiber negatives + procedural background negatives → crops → training with the original spatial holdout) can be re-run end to end from public data; expect a checkpoint that matches in architecture and closely in behavior, not bit-for-bit (GPU/RNG/environment nondeterminism; a retrained checkpoint's AUROC will vary around 0.985, while the PUBLISHED checkpoint's validation AUROC is exactly reproducible: the dataset repo's `eval_checkpoint.py` deterministically rebuilds the held-out split, 81 positives / 123 negatives, and yields 0.9853457794). `proxy_s2s3_v1` additionally has the documented 169-negative gap. The scoring INDEX built with the model ships in the same dataset repo (`full_index_complete.json` + `summarize_index.py`); the sweep script that produced it is project-internal, but its recipe is fully specified above (the `score_window()` preprocessing on a WIN=512 / stride-256 grid). ## Limitations / what this does not claim - **Not an ink detector.** Input is an ink-probability map produced by the official/community detectors, never raw CT. - **Finds letterform-like structure, not meaning.** A high score means "connected strokes at letter scale," which is what a human reviewer should look at — it is not a transcription and not proof of novel text. - **Domain is defined by rendering pipeline, not by scroll identity** — see "Which one to use" above. `proxy_v4` is validated on official-style maps from four scrolls (S1, PHerc 0139, Scroll 5, Scroll 4); a fifth scroll's official maps haven't been tried, though the pattern across four independent scrolls (including one quantitative correlation against transcribed-text ground truth, not just spot checks) makes transfer the expected outcome, not a hopeful one. - Cross-render score shifts are real even within `proxy_v4`'s validated domain: two different official ink-detection models of the *same* scroll region can disagree by 0.1+ (see Thresholds) — that's the ink detector's output changing, not this classifier's domain breaking. - Windows flagged "gold" on public maps are *locations worth expert attention*, on data anyone can download — whether the text there is actually untranscribed must be checked against the papyrological literature, which is a moving target. ## Provenance & license Trained exclusively on publicly released Vesuvius Challenge data (CC BY-NC 4.0); the weights are released under the same CC BY-NC 4.0 to match. Human labels by the project's own reviewer. No official-team code or checkpoints are redistributed here.