Spaces:
Running on Zero
Running on Zero
One page scroll bar and American English
Browse files- README.md +2 -2
- VENDORED.md +9 -4
- app.py +22 -18
- assets/ag896-composite.webp.json +1 -1
- assets/strip-inked.webp.json +1 -1
- assets/strip-plain.webp.json +1 -1
- assets/w00-composite.webp.json +1 -1
- assets/w00-ct.webp.json +1 -1
- assets/w00-overlay.webp.json +1 -1
- crackle.py +6 -6
- crackle_vc3d.py +1 -1
- engine.py +6 -6
- figures.py +12 -12
- koine_engine.py +4 -4
- link.py +3 -3
- onescroll.html +62 -0
- samples.json +125 -13
- upload.py +2 -2
README.md
CHANGED
|
@@ -40,7 +40,7 @@ SHA-256), so they need no GPU quota; your own layers and From a segment always r
|
|
| 40 |
|
| 41 |
## For everyone
|
| 42 |
|
| 43 |
-
- **Example**: public,
|
| 44 |
- **Your layers**: a ZIP of layer images (`00.tif`, `01.tif`, ...) or a multi-page TIFF, 17 to 65 layers of one
|
| 45 |
region, about 320 to 1024 px across at about 9 um.
|
| 46 |
- **From a segment**: only have a screenshot of a released segment? Give the segment (`SCROLL/segments/SEGMENT`)
|
|
@@ -49,7 +49,7 @@ SHA-256), so they need no GPU quota; your own layers and From a segment always r
|
|
| 49 |
- **What is crackle?**: a short guide to segments, layers, how to make the upload and how to read the answer.
|
| 50 |
|
| 51 |
A single image or screenshot uploaded as a picture is refused with the reason: the models read 17 layers at once,
|
| 52 |
-
and on public
|
| 53 |
failed their preregistered tests (tile AUC 0.57, and 0.48 on two unseen scrolls, against a bar of 0.80).
|
| 54 |
|
| 55 |
## For researchers
|
|
|
|
| 40 |
|
| 41 |
## For everyone
|
| 42 |
|
| 43 |
+
- **Example**: public, labeled PHerc0841 crops from the Vesuvius Challenge open data, one click, no upload.
|
| 44 |
- **Your layers**: a ZIP of layer images (`00.tif`, `01.tif`, ...) or a multi-page TIFF, 17 to 65 layers of one
|
| 45 |
region, about 320 to 1024 px across at about 9 um.
|
| 46 |
- **From a segment**: only have a screenshot of a released segment? Give the segment (`SCROLL/segments/SEGMENT`)
|
|
|
|
| 49 |
- **What is crackle?**: a short guide to segments, layers, how to make the upload and how to read the answer.
|
| 50 |
|
| 51 |
A single image or screenshot uploaded as a picture is refused with the reason: the models read 17 layers at once,
|
| 52 |
+
and on public labeled crops they scored at chance from one layer. Two small models trained on single layers also
|
| 53 |
failed their preregistered tests (tile AUC 0.57, and 0.48 on two unseen scrolls, against a bar of 0.80).
|
| 54 |
|
| 55 |
## For researchers
|
VENDORED.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
# Copied files
|
| 2 |
|
| 3 |
-
These files are Ink Lens's, copied unchanged from ChaseHendrick/Scrolls `spaces/ink-lens/` at commit
|
| 4 |
-
(
|
| 5 |
`tests/test_crackle_vc3d.py` checks every hash below.
|
| 6 |
|
| 7 |
`engine.py` and `gate.py` run the readers and the control gate. `inklens.py`, `vc3dout.py`, `volio.py` and
|
|
@@ -12,10 +12,15 @@ files and reach the VC3D agent bridge. `samples.json` lists Ink Lens's public op
|
|
| 12 |
|
| 13 |
| File | SHA-256 |
|
| 14 |
| --- | --- |
|
| 15 |
-
| engine.py |
|
| 16 |
| gate.py | 1036c20467c0aa1d2e6b4970cdd4be8126aebeb19b70dee497332d29b41ba31b |
|
| 17 |
| inklens.py | df8abad85bde87e49a681499a5a6b911a182dc449b9c08174e475182740ca0e5 |
|
| 18 |
| vc3dout.py | 6af807cb2e601332efe6bbf122692852aa9e36e3627dd554ec79754e2e2988bd |
|
| 19 |
| volio.py | 881c7c1a50a81b93d673b87121abd747ebce1c29a64ca3bf1a527b8e81b8cd1a |
|
| 20 |
| bridge.py | 6e1ce10a90d524baf20a861173dab8c7d4cfcf481f9f693f3a54a7cc19840aa0 |
|
| 21 |
-
| samples.json |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# Copied files
|
| 2 |
|
| 3 |
+
These files are Ink Lens's, copied unchanged from ChaseHendrick/Scrolls `spaces/ink-lens/` at commit 11c0272
|
| 4 |
+
(27 Oct 2026, PR #253). Copy them again, never edit them here, so both tools run the same engine, gate and VC3D writer.
|
| 5 |
`tests/test_crackle_vc3d.py` checks every hash below.
|
| 6 |
|
| 7 |
`engine.py` and `gate.py` run the readers and the control gate. `inklens.py`, `vc3dout.py`, `volio.py` and
|
|
|
|
| 12 |
|
| 13 |
| File | SHA-256 |
|
| 14 |
| --- | --- |
|
| 15 |
+
| engine.py | 280309c7bf86bd2d8c55bb64feb52168cbde230d02912fec72d1288260ccffa7 |
|
| 16 |
| gate.py | 1036c20467c0aa1d2e6b4970cdd4be8126aebeb19b70dee497332d29b41ba31b |
|
| 17 |
| inklens.py | df8abad85bde87e49a681499a5a6b911a182dc449b9c08174e475182740ca0e5 |
|
| 18 |
| vc3dout.py | 6af807cb2e601332efe6bbf122692852aa9e36e3627dd554ec79754e2e2988bd |
|
| 19 |
| volio.py | 881c7c1a50a81b93d673b87121abd747ebce1c29a64ca3bf1a527b8e81b8cd1a |
|
| 20 |
| bridge.py | 6e1ce10a90d524baf20a861173dab8c7d4cfcf481f9f693f3a54a7cc19840aa0 |
|
| 21 |
+
| samples.json | 80dec75266c1095e9cf77e81bea8c0c3c6e8662be5b7ca481a77adb4c94dd161 |
|
| 22 |
+
|
| 23 |
+
`connect.json`, `connect.html` and `CONNECT.md` (the Connect your tool tab) and `onescroll.html` (the one page scroll
|
| 24 |
+
bar fragment that `app.py` adds to the page head) are copied from ChaseHendrick/Scrolls `spaces/connect/crackle-check/`,
|
| 25 |
+
where public kit's `spaceconnect render` makes them; change them there and copy them again. They are not in the table
|
| 26 |
+
because they change whenever the shared panel does.
|
app.py
CHANGED
|
@@ -215,8 +215,8 @@ def check_file(path, um_per_px=None, flip=False, reader=None):
|
|
| 215 |
"""Crackle check of one uploaded region of surface layers.
|
| 216 |
|
| 217 |
path: a ZIP of layer images, a multi-page TIFF, or an .npz with array 'volume' (layers first, 17 to 65 layers
|
| 218 |
-
|
| 219 |
-
is not enough). um_per_px: the pixel size of the upload in
|
| 220 |
flip: reverse the layer order first. reader: readerv3 (Reader V3, the default), d9v2 or both (maps averaged).
|
| 221 |
Returns (answer as Markdown, result dict, overlay preview, maps .npz with float32 forward and reverse and uint8
|
| 222 |
kept). Model output, not a reading."""
|
|
@@ -293,7 +293,7 @@ def _example(sample_id, reader):
|
|
| 293 |
|
| 294 |
|
| 295 |
def check_example(sample_id="0841-w00", reader=None):
|
| 296 |
-
"""Crackle check of a public example: a 640 px crop of a
|
| 297 |
from the Vesuvius Challenge open-data bucket. sample_id: 0841-w00, 0841-ag896 or 0841-ag405. reader: readerv3
|
| 298 |
(default), d9v2 or both. Same returns as /check. Results are cached per sample and reader. Model output."""
|
| 299 |
out = _example(sample_id, reader)
|
|
@@ -324,7 +324,7 @@ def check_segment(segment, volume="", x=0, y=0, size=384, reader=None):
|
|
| 324 |
"""Crackle check of a spot on a released segment, for people who have only a screenshot: the Space reads that
|
| 325 |
region's layers from the Vesuvius Challenge open-data bucket itself. segment: SCROLL/segments/SEGMENT (or its
|
| 326 |
open-data URL). volume: one of the segment's surface volumes (blank: the cheapest usable one, see
|
| 327 |
-
/list_volumes). x, y: the
|
| 328 |
9.4 um grid. reader: readerv3 (default), d9v2 or both. Same returns as /check. Model output."""
|
| 329 |
out = _segment(segment, volume, x, y, size, reader)
|
| 330 |
return out["markdown"], out["result"], out["preview"], out["npz"]
|
|
@@ -338,7 +338,7 @@ def list_volumes(segment):
|
|
| 338 |
|
| 339 |
|
| 340 |
def list_readers():
|
| 341 |
-
"""The readers this Space offers, with
|
| 342 |
info = {}
|
| 343 |
for n in READERS:
|
| 344 |
r = engine.READERS.get(n) or koine_engine.KOINE_READERS[n]
|
|
@@ -406,6 +406,10 @@ CSS = """
|
|
| 406 |
# The page's outer padding sits outside the block Gradio scopes css= to, so the phone gutter goes in <head>.
|
| 407 |
HEAD = """<style>@media (max-width: 600px) { .gradio-container { padding-left: 0 !important; padding-right: 0 !important; }
|
| 408 |
.gradio-container > .main { padding-left: 16px !important; padding-right: 16px !important; } }</style>"""
|
|
|
|
|
|
|
|
|
|
|
|
|
| 409 |
|
| 410 |
INTRO = """<div class="cc-hero">
|
| 411 |
<h1>Crackle Check</h1>
|
|
@@ -456,7 +460,7 @@ def idle_html():
|
|
| 456 |
ct, ov = _asset_uri("w00-ct.webp"), _asset_uri("w00-overlay.webp")
|
| 457 |
if not (ex and ct and ov):
|
| 458 |
return IDLE
|
| 459 |
-
pair = (_figure(ct, "
|
| 460 |
+ _figure(ov, "The same CT with amber patches", "Ink signal above the control, in amber"))
|
| 461 |
return (f'<div class="cc-verdict cc-idle"><div class="cc-kicker">What an answer looks like</div>'
|
| 462 |
f'<div class="word">{ex["answer"]}</div><div class="sub">{ex["title"]}, a public example read by '
|
|
@@ -473,10 +477,10 @@ def learn_figures():
|
|
| 473 |
inked, plain = _asset_uri("strip-inked.webp"), _asset_uri("strip-plain.webp")
|
| 474 |
if inked and plain:
|
| 475 |
out["crackle"] = ('<div class="cc-figs">'
|
| 476 |
-
+ _figure(inked, "Five
|
| 477 |
"<b>Where the model sees ink</b> (Reader V3) on PHerc0841 w00, one spot at five depths "
|
| 478 |
"(layers 4 to 24 of 28)", "cc-wide")
|
| 479 |
-
+ _figure(plain, "Five
|
| 480 |
"<b>Plain papyrus</b> nearby, where it sees none, at the same depths", "cc-wide")
|
| 481 |
+ '<div class="cc-cap">Hard to tell apart by eye in any one layer. The texture changes '
|
| 482 |
'through the depth, and the models compare all the layers at once, which is why one '
|
|
@@ -487,7 +491,7 @@ def learn_figures():
|
|
| 487 |
uri, ex = _asset_uri(f"{short}-composite.webp"), _examples_info().get(sid)
|
| 488 |
if uri and ex:
|
| 489 |
tiles.append(_figure(uri, "Four square papyrus panels: CT, CT with amber patches, a score map and its "
|
| 490 |
-
"nearly
|
| 491 |
f"<b>{ex['title']}</b>: {ex['answer']}, {ex['excess']:.1f} x the control. Left to "
|
| 492 |
"right: CT, kept signal, your order, control", "cc-wide"))
|
| 493 |
out["answer"] = (f'<div class="cc-figs">{figures.gauge_svg()}' + "".join(tiles)
|
|
@@ -561,7 +565,7 @@ The scans are 3D volumes. A **segment** is one sheet of papyrus traced through t
|
|
| 561 |
`64.tif`). Ink sits in depth, so the models read **17 layers at once**: one screenshot is not enough.
|
| 562 |
|
| 563 |
1. Pick a region about 3 to 10 mm across (about 320 to 1024 px at 9 um) on a segment.
|
| 564 |
-
2. Take the same crop from at least 17 layers
|
| 565 |
layers 22 to 42 work, or send all of them.
|
| 566 |
3. ZIP the images, keeping their numbered names, or save them as one multi-page TIFF. Upload it on the Check tab.
|
| 567 |
|
|
@@ -585,7 +589,7 @@ scale.
|
|
| 585 |
|
| 586 |
LEARN_FAQ = """## Questions people ask
|
| 587 |
|
| 588 |
-
**Can I upload a screenshot?** Not as a picture. On public
|
| 589 |
layer. Two small models trained on single layers also failed their preregistered tests: tile AUC 0.57, and 0.48 on two
|
| 590 |
scrolls the model never saw, both against a bar of 0.80. If the screenshot is from a released segment, use **From a segment** instead: give the segment and the spot,
|
| 591 |
and the Space reads the real layers there from the open data.
|
|
@@ -611,8 +615,8 @@ def researchers_md():
|
|
| 611 |
|
| 612 |
Each reader runs on your stack in the order sent and with the layers reversed, in one GPU call. d9v2 runs on
|
| 613 |
[Ink Lens](https://github.com/ChaseHendrick/Scrolls/blob/main/docs/tools/ink-lens.md)'s engine (128 px patches,
|
| 614 |
-
three 17-layer depth windows when 21 or more layers are sent); Reader V3 runs the team's koine inference (
|
| 615 |
-
layers, 384 px patches, 50 % overlap, floored Hann blend, robust
|
| 616 |
match koine's own CLI, correlation 1.000). "Both" averages the readers' maps. The gate (`{gate.RULE_ID}`): papyrus
|
| 617 |
is the middle layer above its 5th percentile; T is the 99th percentile of the reversed map on papyrus; excess is the
|
| 618 |
share of papyrus your order puts above T divided by 1 %. Yes from 3, Maybe from 1.5, Too small under 100,000
|
|
@@ -625,11 +629,11 @@ preregistered ranking (0.890 against 0.862 for Reader v2 and 0.861 for d9v2, 21
|
|
| 625 |
ranking's paired rule it was level with Reader v2, and its author benchmarked on PHerc0841 before release, so the
|
| 626 |
crops are not unseen by its selection.
|
| 627 |
|
| 628 |
-
| Id | Reader |
|
| 629 |
| --- | --- | --- | --- | --- |
|
| 630 |
{readers}
|
| 631 |
|
| 632 |
-
To add a reader, add an entry with a fixed URL, SHA-256 and
|
| 633 |
readers) or `koine_engine.KOINE_READERS` (koine readers) and its name to `CRACKLE_READERS`.
|
| 634 |
|
| 635 |
## Measured (Model output, development measurement on public PHerc0841, 21 Oct 2026)
|
|
@@ -755,12 +759,12 @@ def build_demo():
|
|
| 755 |
mode = gr.State("example")
|
| 756 |
with gr.Tabs():
|
| 757 |
with gr.Tab("Example") as tab_ex:
|
| 758 |
-
gr.Markdown("A 640 px crop of a public,
|
| 759 |
"Vesuvius Challenge open data. No upload needed.", elem_classes="cc-note")
|
| 760 |
sid = gr.Dropdown(samples, value=EXAMPLES[0], label="Example")
|
| 761 |
with gr.Tab("Your layers") as tab_up:
|
| 762 |
gr.Markdown("A ZIP of layer images, one multi-page TIFF, or an .npz with a `volume` "
|
| 763 |
-
"array: 17 to 65 layers
|
| 764 |
elem_classes="cc-note")
|
| 765 |
f = gr.File(label="Layers", show_label=False,
|
| 766 |
file_types=[".zip", ".tif", ".tiff", ".npz", ".png", ".jpg", ".jpeg"],
|
|
@@ -781,7 +785,7 @@ def build_demo():
|
|
| 781 |
with gr.Row():
|
| 782 |
sx = gr.Number(value=None, label="x (column)", precision=0, minimum=0)
|
| 783 |
sy = gr.Number(value=None, label="y (row)", precision=0, minimum=0)
|
| 784 |
-
gr.Markdown("The
|
| 785 |
"has about four times the pixels of a 9 um one).", elem_classes="cc-caption")
|
| 786 |
with gr.Accordion("Options", open=False, elem_classes="cc-acc"):
|
| 787 |
size = gr.Slider(link.MIN_SIDE, link.MAX_SIDE, value=link.SIDE, step=32,
|
|
|
|
| 215 |
"""Crackle check of one uploaded region of surface layers.
|
| 216 |
|
| 217 |
path: a ZIP of layer images, a multi-page TIFF, or an .npz with array 'volume' (layers first, 17 to 65 layers
|
| 218 |
+
centered on the papyrus surface, 128 to 2048 px a side). A single image is refused with the reason (one layer
|
| 219 |
+
is not enough). um_per_px: the pixel size of the upload in micrometers (blank or 0: about 9 um, used as it is).
|
| 220 |
flip: reverse the layer order first. reader: readerv3 (Reader V3, the default), d9v2 or both (maps averaged).
|
| 221 |
Returns (answer as Markdown, result dict, overlay preview, maps .npz with float32 forward and reverse and uint8
|
| 222 |
kept). Model output, not a reading."""
|
|
|
|
| 293 |
|
| 294 |
|
| 295 |
def check_example(sample_id="0841-w00", reader=None):
|
| 296 |
+
"""Crackle check of a public example: a 640 px crop of a labeled PHerc0841 segment (not prize-eligible), read
|
| 297 |
from the Vesuvius Challenge open-data bucket. sample_id: 0841-w00, 0841-ag896 or 0841-ag405. reader: readerv3
|
| 298 |
(default), d9v2 or both. Same returns as /check. Results are cached per sample and reader. Model output."""
|
| 299 |
out = _example(sample_id, reader)
|
|
|
|
| 324 |
"""Crackle check of a spot on a released segment, for people who have only a screenshot: the Space reads that
|
| 325 |
region's layers from the Vesuvius Challenge open-data bucket itself. segment: SCROLL/segments/SEGMENT (or its
|
| 326 |
open-data URL). volume: one of the segment's surface volumes (blank: the cheapest usable one, see
|
| 327 |
+
/list_volumes). x, y: the center, in that volume's pixels (column, row). size: 320 to 640 px on the readers'
|
| 328 |
9.4 um grid. reader: readerv3 (default), d9v2 or both. Same returns as /check. Model output."""
|
| 329 |
out = _segment(segment, volume, x, y, size, reader)
|
| 330 |
return out["markdown"], out["result"], out["preview"], out["npz"]
|
|
|
|
| 338 |
|
| 339 |
|
| 340 |
def list_readers():
|
| 341 |
+
"""The readers this Space offers, with license, source and what each was held out from, and the examples."""
|
| 342 |
info = {}
|
| 343 |
for n in READERS:
|
| 344 |
r = engine.READERS.get(n) or koine_engine.KOINE_READERS[n]
|
|
|
|
| 406 |
# The page's outer padding sits outside the block Gradio scopes css= to, so the phone gutter goes in <head>.
|
| 407 |
HEAD = """<style>@media (max-width: 600px) { .gradio-container { padding-left: 0 !important; padding-right: 0 !important; }
|
| 408 |
.gradio-container > .main { padding-left: 16px !important; padding-right: 16px !important; } }</style>"""
|
| 409 |
+
# One page scroll bar: frames fit their content and the page scrolls (copied from public Scrolls'
|
| 410 |
+
# spaces/connect/crackle-check/, made by kit spaceconnect render; see VENDORED.md).
|
| 411 |
+
ONESCROLL = HERE / "onescroll.html"
|
| 412 |
+
HEAD = HEAD + (ONESCROLL.read_text() if ONESCROLL.exists() else "")
|
| 413 |
|
| 414 |
INTRO = """<div class="cc-hero">
|
| 415 |
<h1>Crackle Check</h1>
|
|
|
|
| 460 |
ct, ov = _asset_uri("w00-ct.webp"), _asset_uri("w00-overlay.webp")
|
| 461 |
if not (ex and ct and ov):
|
| 462 |
return IDLE
|
| 463 |
+
pair = (_figure(ct, "Gray papyrus CT texture", "Middle CT layer")
|
| 464 |
+ _figure(ov, "The same CT with amber patches", "Ink signal above the control, in amber"))
|
| 465 |
return (f'<div class="cc-verdict cc-idle"><div class="cc-kicker">What an answer looks like</div>'
|
| 466 |
f'<div class="word">{ex["answer"]}</div><div class="sub">{ex["title"]}, a public example read by '
|
|
|
|
| 477 |
inked, plain = _asset_uri("strip-inked.webp"), _asset_uri("strip-plain.webp")
|
| 478 |
if inked and plain:
|
| 479 |
out["crackle"] = ('<div class="cc-figs">'
|
| 480 |
+
+ _figure(inked, "Five gray CT tiles of papyrus at increasing depth",
|
| 481 |
"<b>Where the model sees ink</b> (Reader V3) on PHerc0841 w00, one spot at five depths "
|
| 482 |
"(layers 4 to 24 of 28)", "cc-wide")
|
| 483 |
+
+ _figure(plain, "Five gray CT tiles of papyrus at increasing depth",
|
| 484 |
"<b>Plain papyrus</b> nearby, where it sees none, at the same depths", "cc-wide")
|
| 485 |
+ '<div class="cc-cap">Hard to tell apart by eye in any one layer. The texture changes '
|
| 486 |
'through the depth, and the models compare all the layers at once, which is why one '
|
|
|
|
| 491 |
uri, ex = _asset_uri(f"{short}-composite.webp"), _examples_info().get(sid)
|
| 492 |
if uri and ex:
|
| 493 |
tiles.append(_figure(uri, "Four square papyrus panels: CT, CT with amber patches, a score map and its "
|
| 494 |
+
"nearly gray control",
|
| 495 |
f"<b>{ex['title']}</b>: {ex['answer']}, {ex['excess']:.1f} x the control. Left to "
|
| 496 |
"right: CT, kept signal, your order, control", "cc-wide"))
|
| 497 |
out["answer"] = (f'<div class="cc-figs">{figures.gauge_svg()}' + "".join(tiles)
|
|
|
|
| 565 |
`64.tif`). Ink sits in depth, so the models read **17 layers at once**: one screenshot is not enough.
|
| 566 |
|
| 567 |
1. Pick a region about 3 to 10 mm across (about 320 to 1024 px at 9 um) on a segment.
|
| 568 |
+
2. Take the same crop from at least 17 layers centered on the surface. From a segment's `layers` folder of 65,
|
| 569 |
layers 22 to 42 work, or send all of them.
|
| 570 |
3. ZIP the images, keeping their numbered names, or save them as one multi-page TIFF. Upload it on the Check tab.
|
| 571 |
|
|
|
|
| 589 |
|
| 590 |
LEARN_FAQ = """## Questions people ask
|
| 591 |
|
| 592 |
+
**Can I upload a screenshot?** Not as a picture. On public labeled crops both models scored at chance from one
|
| 593 |
layer. Two small models trained on single layers also failed their preregistered tests: tile AUC 0.57, and 0.48 on two
|
| 594 |
scrolls the model never saw, both against a bar of 0.80. If the screenshot is from a released segment, use **From a segment** instead: give the segment and the spot,
|
| 595 |
and the Space reads the real layers there from the open data.
|
|
|
|
| 615 |
|
| 616 |
Each reader runs on your stack in the order sent and with the layers reversed, in one GPU call. d9v2 runs on
|
| 617 |
[Ink Lens](https://github.com/ChaseHendrick/Scrolls/blob/main/docs/tools/ink-lens.md)'s engine (128 px patches,
|
| 618 |
+
three 17-layer depth windows when 21 or more layers are sent); Reader V3 runs the team's koine inference (center 17
|
| 619 |
+
layers, 384 px patches, 50 % overlap, floored Hann blend, robust normalization), reproduced patch for patch (maps
|
| 620 |
match koine's own CLI, correlation 1.000). "Both" averages the readers' maps. The gate (`{gate.RULE_ID}`): papyrus
|
| 621 |
is the middle layer above its 5th percentile; T is the 99th percentile of the reversed map on papyrus; excess is the
|
| 622 |
share of papyrus your order puts above T divided by 1 %. Yes from 3, Maybe from 1.5, Too small under 100,000
|
|
|
|
| 629 |
ranking's paired rule it was level with Reader v2, and its author benchmarked on PHerc0841 before release, so the
|
| 630 |
crops are not unseen by its selection.
|
| 631 |
|
| 632 |
+
| Id | Reader | License | Held out | SHA-256 |
|
| 633 |
| --- | --- | --- | --- | --- |
|
| 634 |
{readers}
|
| 635 |
|
| 636 |
+
To add a reader, add an entry with a fixed URL, SHA-256 and license to Ink Lens's `engine.READERS` (128 px villa
|
| 637 |
readers) or `koine_engine.KOINE_READERS` (koine readers) and its name to `CRACKLE_READERS`.
|
| 638 |
|
| 639 |
## Measured (Model output, development measurement on public PHerc0841, 21 Oct 2026)
|
|
|
|
| 759 |
mode = gr.State("example")
|
| 760 |
with gr.Tabs():
|
| 761 |
with gr.Tab("Example") as tab_ex:
|
| 762 |
+
gr.Markdown("A 640 px crop of a public, labeled PHerc0841 segment from the "
|
| 763 |
"Vesuvius Challenge open data. No upload needed.", elem_classes="cc-note")
|
| 764 |
sid = gr.Dropdown(samples, value=EXAMPLES[0], label="Example")
|
| 765 |
with gr.Tab("Your layers") as tab_up:
|
| 766 |
gr.Markdown("A ZIP of layer images, one multi-page TIFF, or an .npz with a `volume` "
|
| 767 |
+
"array: 17 to 65 layers centered on the papyrus surface.",
|
| 768 |
elem_classes="cc-note")
|
| 769 |
f = gr.File(label="Layers", show_label=False,
|
| 770 |
file_types=[".zip", ".tif", ".tiff", ".npz", ".png", ".jpg", ".jpeg"],
|
|
|
|
| 785 |
with gr.Row():
|
| 786 |
sx = gr.Number(value=None, label="x (column)", precision=0, minimum=0)
|
| 787 |
sy = gr.Number(value=None, label="y (row)", precision=0, minimum=0)
|
| 788 |
+
gr.Markdown("The center of the spot, in that surface volume's pixels (a 2.4 um volume "
|
| 789 |
"has about four times the pixels of a 9 um one).", elem_classes="cc-caption")
|
| 790 |
with gr.Accordion("Options", open=False, elem_classes="cc-acc"):
|
| 791 |
size = gr.Slider(link.MIN_SIDE, link.MAX_SIDE, value=link.SIDE, step=32,
|
assets/ag896-composite.webp.json
CHANGED
|
@@ -5,7 +5,7 @@
|
|
| 5 |
"tool": "crackle-check",
|
| 6 |
"kind": "figure",
|
| 7 |
"caption": "PHerc0841 auto-grown 896, a 640 px public crop, left to right: the middle CT layer; the kept ink signal in amber over it; the model score in the stored layer order; the same with the layers reversed (the control), on one scale, amber above the control's 99th percentile, read by Reader V3. Answer Yes, 28.4 times the control. Model output, not a reading. PHerc0841, segment 20260220214732-auto_grown_20260220144552896, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 8 |
-
"alt": "Four square papyrus CT panels: plain
|
| 9 |
"sources": [
|
| 10 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220214732-auto_grown_20260220144552896/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2496, 3136, 1600, 2240])",
|
| 11 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
|
|
|
| 5 |
"tool": "crackle-check",
|
| 6 |
"kind": "figure",
|
| 7 |
"caption": "PHerc0841 auto-grown 896, a 640 px public crop, left to right: the middle CT layer; the kept ink signal in amber over it; the model score in the stored layer order; the same with the layers reversed (the control), on one scale, amber above the control's 99th percentile, read by Reader V3. Answer Yes, 28.4 times the control. Model output, not a reading. PHerc0841, segment 20260220214732-auto_grown_20260220144552896, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 8 |
+
"alt": "Four square papyrus CT panels: plain gray, then with amber patches, then a gray and amber score map, then its nearly all gray control.",
|
| 9 |
"sources": [
|
| 10 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220214732-auto_grown_20260220144552896/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2496, 3136, 1600, 2240])",
|
| 11 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
assets/strip-inked.webp.json
CHANGED
|
@@ -4,7 +4,7 @@
|
|
| 4 |
"tool": "crackle-check",
|
| 5 |
"kind": "still",
|
| 6 |
"caption": "PHerc0841 w00, the window where the models keep the most ink (192 px at rows 192, columns 64 of the example crop), at layers 4, 9, 14, 19, 24 of 28. The window was chosen from the models' kept map; the picture itself is CT only. PHerc0841, segment 20260220213127-w00, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 7 |
-
"alt": "Five
|
| 8 |
"sources": [
|
| 9 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220213127-w00/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2624, 3264, 2688, 3328])",
|
| 10 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
|
|
|
| 4 |
"tool": "crackle-check",
|
| 5 |
"kind": "still",
|
| 6 |
"caption": "PHerc0841 w00, the window where the models keep the most ink (192 px at rows 192, columns 64 of the example crop), at layers 4, 9, 14, 19, 24 of 28. The window was chosen from the models' kept map; the picture itself is CT only. PHerc0841, segment 20260220213127-w00, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 7 |
+
"alt": "Five gray CT tiles of papyrus side by side at increasing depth.",
|
| 8 |
"sources": [
|
| 9 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220213127-w00/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2624, 3264, 2688, 3328])",
|
| 10 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
assets/strip-plain.webp.json
CHANGED
|
@@ -4,7 +4,7 @@
|
|
| 4 |
"tool": "crackle-check",
|
| 5 |
"kind": "still",
|
| 6 |
"caption": "PHerc0841 w00, a window of plain papyrus where they keep none (192 px at rows 272, columns 288 of the example crop), at layers 4, 9, 14, 19, 24 of 28. The window was chosen from the models' kept map; the picture itself is CT only. PHerc0841, segment 20260220213127-w00, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 7 |
-
"alt": "Five
|
| 8 |
"sources": [
|
| 9 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220213127-w00/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2624, 3264, 2688, 3328])",
|
| 10 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
|
|
|
| 4 |
"tool": "crackle-check",
|
| 5 |
"kind": "still",
|
| 6 |
"caption": "PHerc0841 w00, a window of plain papyrus where they keep none (192 px at rows 272, columns 288 of the example crop), at layers 4, 9, 14, 19, 24 of 28. The window was chosen from the models' kept map; the picture itself is CT only. PHerc0841, segment 20260220213127-w00, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 7 |
+
"alt": "Five gray CT tiles of papyrus side by side at increasing depth.",
|
| 8 |
"sources": [
|
| 9 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220213127-w00/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2624, 3264, 2688, 3328])",
|
| 10 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
assets/w00-composite.webp.json
CHANGED
|
@@ -5,7 +5,7 @@
|
|
| 5 |
"tool": "crackle-check",
|
| 6 |
"kind": "figure",
|
| 7 |
"caption": "PHerc0841 w00, a 640 px public crop, left to right: the middle CT layer; the kept ink signal in amber over it; the model score in the stored layer order; the same with the layers reversed (the control), on one scale, amber above the control's 99th percentile, read by Reader V3. Answer Yes, 18.2 times the control. Model output, not a reading. PHerc0841, segment 20260220213127-w00, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 8 |
-
"alt": "Four square papyrus CT panels: plain
|
| 9 |
"sources": [
|
| 10 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220213127-w00/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2624, 3264, 2688, 3328])",
|
| 11 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
|
|
|
| 5 |
"tool": "crackle-check",
|
| 6 |
"kind": "figure",
|
| 7 |
"caption": "PHerc0841 w00, a 640 px public crop, left to right: the middle CT layer; the kept ink signal in amber over it; the model score in the stored layer order; the same with the layers reversed (the control), on one scale, amber above the control's 99th percentile, read by Reader V3. Answer Yes, 18.2 times the control. Model output, not a reading. PHerc0841, segment 20260220213127-w00, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 8 |
+
"alt": "Four square papyrus CT panels: plain gray, then with amber patches, then a gray and amber score map, then its nearly all gray control.",
|
| 9 |
"sources": [
|
| 10 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220213127-w00/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2624, 3264, 2688, 3328])",
|
| 11 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
assets/w00-ct.webp.json
CHANGED
|
@@ -4,7 +4,7 @@
|
|
| 4 |
"tool": "crackle-check",
|
| 5 |
"kind": "still",
|
| 6 |
"caption": "The middle CT layer of the PHerc0841 w00 example crop. PHerc0841, segment 20260220213127-w00, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 7 |
-
"alt": "
|
| 8 |
"sources": [
|
| 9 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220213127-w00/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2624, 3264, 2688, 3328])",
|
| 10 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
|
|
|
| 4 |
"tool": "crackle-check",
|
| 5 |
"kind": "still",
|
| 6 |
"caption": "The middle CT layer of the PHerc0841 w00 example crop. PHerc0841, segment 20260220213127-w00, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 7 |
+
"alt": "Gray papyrus CT texture.",
|
| 8 |
"sources": [
|
| 9 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220213127-w00/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2624, 3264, 2688, 3328])",
|
| 10 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
assets/w00-overlay.webp.json
CHANGED
|
@@ -5,7 +5,7 @@
|
|
| 5 |
"tool": "crackle-check",
|
| 6 |
"kind": "still",
|
| 7 |
"caption": "The middle CT layer of the PHerc0841 w00 example crop with the kept ink signal in amber (Reader V3). Model output, not a reading. PHerc0841, segment 20260220213127-w00, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 8 |
-
"alt": "
|
| 9 |
"sources": [
|
| 10 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220213127-w00/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2624, 3264, 2688, 3328])",
|
| 11 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
|
|
|
| 5 |
"tool": "crackle-check",
|
| 6 |
"kind": "still",
|
| 7 |
"caption": "The middle CT layer of the PHerc0841 w00 example crop with the kept ink signal in amber (Reader V3). Model output, not a reading. PHerc0841, segment 20260220213127-w00, 9.366 um voxels. Data: Vesuvius Challenge, CC BY-NC 4.0.",
|
| 8 |
+
"alt": "Gray papyrus CT texture with amber patches.",
|
| 9 |
"sources": [
|
| 10 |
"https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc0841/segments/20260220213127-w00/surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr (crop [2624, 3264, 2688, 3328])",
|
| 11 |
"https://huggingface.co/spaces/SharpMeow/crackle-check"
|
crackle.py
CHANGED
|
@@ -37,10 +37,10 @@ ANSWERS = {
|
|
| 37 |
|
| 38 |
SINGLE_IMAGE = (
|
| 39 |
"**One image is not enough to tell.** Crackle is a 3D texture: the ink models read a stack of layers through the "
|
| 40 |
-
"papyrus surface. Given a single layer, like a screenshot, they score at chance (measured on public
|
| 41 |
"crops), so this tool will not guess. Send the same region as a stack of layers instead: a ZIP of the layer "
|
| 42 |
"images (for example 00.tif to 64.tif from a segment's `layers` folder, or from VC3D or `vc_render_tifxyz`), or "
|
| 43 |
-
"one multi-page TIFF, at least 17 layers
|
| 44 |
"From a segment tab (API /check_segment) reads its layers from the open data for you: give the segment and the "
|
| 45 |
"spot.")
|
| 46 |
|
|
@@ -72,7 +72,7 @@ def prepare(array, kind, um_per_px=None, flip=False):
|
|
| 72 |
raise CheckError(f"{a.shape[0]} layers: the model reads {engine.DEPTH} layers at once, so send at least "
|
| 73 |
f"{engine.DEPTH} (21 or more lets it average three depth windows)")
|
| 74 |
if a.shape[0] > engine.MAX_LAYERS:
|
| 75 |
-
raise CheckError(f"{a.shape[0]} layers: send at most {engine.MAX_LAYERS},
|
| 76 |
notes = []
|
| 77 |
if flip:
|
| 78 |
a = a[::-1]
|
|
@@ -102,7 +102,7 @@ def judge(F, R, volume):
|
|
| 102 |
|
| 103 |
|
| 104 |
def preview(volume, kept):
|
| 105 |
-
"""The middle layer in
|
| 106 |
ct = volume[volume.shape[0] // 2].astype(np.float32)
|
| 107 |
lo, hi = np.percentile(ct, [1, 99])
|
| 108 |
g = np.clip((ct - lo) / max(hi - lo, 1e-6), 0, 1)
|
|
@@ -133,7 +133,7 @@ def check(array, kind, run, um_per_px=None, flip=False, notes=()):
|
|
| 133 |
return out["markdown"], out["result"], out["preview"]
|
| 134 |
|
| 135 |
|
| 136 |
-
def
|
| 137 |
"""The middle layer, stretched to its 1st to 99th percentile, as RGB uint8 (the overlay's background)."""
|
| 138 |
return preview(volume, np.zeros(volume.shape[1:], np.float32))
|
| 139 |
|
|
@@ -159,7 +159,7 @@ def run_check(array, kind, run, um_per_px=None, flip=False, notes=()):
|
|
| 159 |
g = gate.decide(m["forward"], m["reverse"], ct)
|
| 160 |
result["per_reader"][name] = {"answer": ANSWERS[g["status"]][0], "excess": g["excess"]}
|
| 161 |
kept_u8 = gate.to_u8(kept)
|
| 162 |
-
return {"markdown": answer(d, hint, notes), "result": result, "preview": preview(stack, kept), "middle":
|
| 163 |
"forward": out["forward"], "reverse": out["reverse"], "kept": kept_u8,
|
| 164 |
"figures": figures.compare_html(stack, out["forward"], out["reverse"], d),
|
| 165 |
"panels": figures.composite(stack, out["forward"], out["reverse"], kept_u8, d) if d.get("threshold") else None}
|
|
|
|
| 37 |
|
| 38 |
SINGLE_IMAGE = (
|
| 39 |
"**One image is not enough to tell.** Crackle is a 3D texture: the ink models read a stack of layers through the "
|
| 40 |
+
"papyrus surface. Given a single layer, like a screenshot, they score at chance (measured on public labeled "
|
| 41 |
"crops), so this tool will not guess. Send the same region as a stack of layers instead: a ZIP of the layer "
|
| 42 |
"images (for example 00.tif to 64.tif from a segment's `layers` folder, or from VC3D or `vc_render_tifxyz`), or "
|
| 43 |
+
"one multi-page TIFF, at least 17 layers centered on the surface. If the picture is from a released segment, the "
|
| 44 |
"From a segment tab (API /check_segment) reads its layers from the open data for you: give the segment and the "
|
| 45 |
"spot.")
|
| 46 |
|
|
|
|
| 72 |
raise CheckError(f"{a.shape[0]} layers: the model reads {engine.DEPTH} layers at once, so send at least "
|
| 73 |
f"{engine.DEPTH} (21 or more lets it average three depth windows)")
|
| 74 |
if a.shape[0] > engine.MAX_LAYERS:
|
| 75 |
+
raise CheckError(f"{a.shape[0]} layers: send at most {engine.MAX_LAYERS}, centered on the papyrus surface")
|
| 76 |
notes = []
|
| 77 |
if flip:
|
| 78 |
a = a[::-1]
|
|
|
|
| 102 |
|
| 103 |
|
| 104 |
def preview(volume, kept):
|
| 105 |
+
"""The middle layer in gray with kept ink in amber, RGB uint8."""
|
| 106 |
ct = volume[volume.shape[0] // 2].astype(np.float32)
|
| 107 |
lo, hi = np.percentile(ct, [1, 99])
|
| 108 |
g = np.clip((ct - lo) / max(hi - lo, 1e-6), 0, 1)
|
|
|
|
| 133 |
return out["markdown"], out["result"], out["preview"]
|
| 134 |
|
| 135 |
|
| 136 |
+
def gray(volume):
|
| 137 |
"""The middle layer, stretched to its 1st to 99th percentile, as RGB uint8 (the overlay's background)."""
|
| 138 |
return preview(volume, np.zeros(volume.shape[1:], np.float32))
|
| 139 |
|
|
|
|
| 159 |
g = gate.decide(m["forward"], m["reverse"], ct)
|
| 160 |
result["per_reader"][name] = {"answer": ANSWERS[g["status"]][0], "excess": g["excess"]}
|
| 161 |
kept_u8 = gate.to_u8(kept)
|
| 162 |
+
return {"markdown": answer(d, hint, notes), "result": result, "preview": preview(stack, kept), "middle": gray(stack),
|
| 163 |
"forward": out["forward"], "reverse": out["reverse"], "kept": kept_u8,
|
| 164 |
"figures": figures.compare_html(stack, out["forward"], out["reverse"], d),
|
| 165 |
"panels": figures.composite(stack, out["forward"], out["reverse"], kept_u8, d) if d.get("threshold") else None}
|
crackle_vc3d.py
CHANGED
|
@@ -169,7 +169,7 @@ def recheck(volume, um_per_px, maps, result):
|
|
| 169 |
# ---------------------------------------------------------------- VC3D files
|
| 170 |
|
| 171 |
def resize_nearest(a, shape):
|
| 172 |
-
"""a resampled to shape by nearest
|
| 173 |
pooled render's grid back)."""
|
| 174 |
import numpy as np
|
| 175 |
if a.shape == tuple(shape):
|
|
|
|
| 169 |
# ---------------------------------------------------------------- VC3D files
|
| 170 |
|
| 171 |
def resize_nearest(a, shape):
|
| 172 |
+
"""a resampled to shape by nearest neighbor (the run rescales regions far from 9.4 um; the overlay needs the
|
| 173 |
pooled render's grid back)."""
|
| 174 |
import numpy as np
|
| 175 |
if a.shape == tuple(shape):
|
engine.py
CHANGED
|
@@ -3,13 +3,13 @@
|
|
| 3 |
The model is built by villa's own flat ink inference (vesuvius.ink_detection.inference.infer.configure_model, at the
|
| 4 |
commit pinned in VILLA_COMMIT) from a checkpoint downloaded by URL and refused unless its SHA-256 matches. The patch
|
| 5 |
schedule (128 px patches, stride 64, the last flush with the end), the layer choice (center crop, reversed for the
|
| 6 |
-
control), villa's per-patch robust
|
| 7 |
command line, both to fit one ZeroGPU call:
|
| 8 |
|
| 9 |
-
1. the robust
|
| 10 |
(robust_np is a line-for-line copy of villa's normalize_robust for a patch with no mask; tests/test_inklens.py
|
| 11 |
compares them, and compares robust_np with villa's own function when villa is importable);
|
| 12 |
-
2. the maps stay float (0 to 1) instead of villa's uint8 encoding, so the control-set threshold is not
|
| 13 |
|
| 14 |
run_reference() is the same algorithm in numpy with any predict function: the tests run it with a stand-in model,
|
| 15 |
and the torch path is checked against it.
|
|
@@ -72,7 +72,7 @@ BATCH = 64
|
|
| 72 |
MIN_SIDE, MAX_SIDE = PATCH, 2048
|
| 73 |
MIN_LAYERS, MAX_LAYERS = DEPTH, 65
|
| 74 |
# One call's duration: a fixed start plus a per-patch allowance, capped. The per-patch figure is an estimate (3 to
|
| 75 |
-
# 6 ms per patch with batched fp16 and
|
| 76 |
# seconds so the first calls calibrate it.
|
| 77 |
GPU_START_S, GPU_PER_PATCH_S, GPU_MAX_S = 4.0, 0.012, 40
|
| 78 |
|
|
@@ -175,7 +175,7 @@ def passes(windows):
|
|
| 175 |
return [(o, d) for o in WINDOW_OFFSETS[windows] for d in ("forward", "reverse")]
|
| 176 |
|
| 177 |
|
| 178 |
-
# ---------------------------------------------------------------- villa's
|
| 179 |
|
| 180 |
def robust_np(patch):
|
| 181 |
"""villa's normalize_robust (vesuvius/image_proc/intensity/normalization.py at VILLA_COMMIT) with no mask:
|
|
@@ -469,7 +469,7 @@ class Engine:
|
|
| 469 |
for b0 in range(0, len(coords), batch):
|
| 470 |
chunk = coords[b0:b0 + batch]
|
| 471 |
raw = torch.stack([sub[:, y:y + PATCH, x:x + PATCH] for y, x in chunk]) # (B, 17, 128, 128)
|
| 472 |
-
keep = raw.reshape(len(chunk), -1).any(dim=1) # villa skips all-zero patches, before
|
| 473 |
if not bool(keep.any()):
|
| 474 |
continue
|
| 475 |
if not bool(keep.all()):
|
|
|
|
| 3 |
The model is built by villa's own flat ink inference (vesuvius.ink_detection.inference.infer.configure_model, at the
|
| 4 |
commit pinned in VILLA_COMMIT) from a checkpoint downloaded by URL and refused unless its SHA-256 matches. The patch
|
| 5 |
schedule (128 px patches, stride 64, the last flush with the end), the layer choice (center crop, reversed for the
|
| 6 |
+
control), villa's per-patch robust normalization and the Hann blend are villa's; two things differ from villa's
|
| 7 |
command line, both to fit one ZeroGPU call:
|
| 8 |
|
| 9 |
+
1. the robust normalization runs batched on the device (robust_torch), checked against villa's CPU function
|
| 10 |
(robust_np is a line-for-line copy of villa's normalize_robust for a patch with no mask; tests/test_inklens.py
|
| 11 |
compares them, and compares robust_np with villa's own function when villa is importable);
|
| 12 |
+
2. the maps stay float (0 to 1) instead of villa's uint8 encoding, so the control-set threshold is not quantized.
|
| 13 |
|
| 14 |
run_reference() is the same algorithm in numpy with any predict function: the tests run it with a stand-in model,
|
| 15 |
and the torch path is checked against it.
|
|
|
|
| 72 |
MIN_SIDE, MAX_SIDE = PATCH, 2048
|
| 73 |
MIN_LAYERS, MAX_LAYERS = DEPTH, 65
|
| 74 |
# One call's duration: a fixed start plus a per-patch allowance, capped. The per-patch figure is an estimate (3 to
|
| 75 |
+
# 6 ms per patch with batched fp16 and normalization on the GPU) with a margin; every receipt records the measured
|
| 76 |
# seconds so the first calls calibrate it.
|
| 77 |
GPU_START_S, GPU_PER_PATCH_S, GPU_MAX_S = 4.0, 0.012, 40
|
| 78 |
|
|
|
|
| 175 |
return [(o, d) for o in WINDOW_OFFSETS[windows] for d in ("forward", "reverse")]
|
| 176 |
|
| 177 |
|
| 178 |
+
# ---------------------------------------------------------------- villa's normalization, numpy and torch
|
| 179 |
|
| 180 |
def robust_np(patch):
|
| 181 |
"""villa's normalize_robust (vesuvius/image_proc/intensity/normalization.py at VILLA_COMMIT) with no mask:
|
|
|
|
| 469 |
for b0 in range(0, len(coords), batch):
|
| 470 |
chunk = coords[b0:b0 + batch]
|
| 471 |
raw = torch.stack([sub[:, y:y + PATCH, x:x + PATCH] for y, x in chunk]) # (B, 17, 128, 128)
|
| 472 |
+
keep = raw.reshape(len(chunk), -1).any(dim=1) # villa skips all-zero patches, before normalizing
|
| 473 |
if not bool(keep.any()):
|
| 474 |
continue
|
| 475 |
if not bool(keep.all()):
|
figures.py
CHANGED
|
@@ -1,8 +1,8 @@
|
|
| 1 |
"""Crackle Check pictures: the maps beside their control, where the scores fall, and the answer scale (numpy, Pillow).
|
| 2 |
|
| 3 |
Everything here draws what the gate already measured; nothing changes the answer. Map tiles are WebP data URIs and
|
| 4 |
-
the charts are inline SVG that take the page's text
|
| 5 |
-
follow the overlay:
|
| 6 |
so amber always means "passes the control's 99th percentile". Everything drawn is model output, not a reading.
|
| 7 |
"""
|
| 8 |
|
|
@@ -29,7 +29,7 @@ def ramp(t):
|
|
| 29 |
return np.stack([np.interp(t, xs, [c[i] / 255.0 for _, c in RAMP]) for i in range(3)], -1)
|
| 30 |
|
| 31 |
|
| 32 |
-
def
|
| 33 |
"""A CT layer stretched to its 1st to 99th percentile, float 0 to 1."""
|
| 34 |
ct = np.asarray(ct, np.float32)
|
| 35 |
lo, hi = np.percentile(ct, [1, 99])
|
|
@@ -37,7 +37,7 @@ def ct_grey(ct):
|
|
| 37 |
|
| 38 |
|
| 39 |
def heat(m, threshold, top, papyrus_mask):
|
| 40 |
-
"""One map as RGB uint8: under T a dim
|
| 41 |
|
| 42 |
The same T and top are used for a map and its control, so the two tiles are on one scale."""
|
| 43 |
m = np.asarray(m, np.float32)
|
|
@@ -77,8 +77,8 @@ def gauge_svg(excess=None, width=440):
|
|
| 77 |
f'aria-label="Answer scale: No under 1.5, Maybe 1.5 to 3, Yes from 3 times the control'
|
| 78 |
+ (f'; this region {_f(excess)} times' if excess is not None else '') + '" '
|
| 79 |
'style="width:100%;max-width:560px;height:auto;font:12px ui-sans-serif,system-ui,sans-serif">']
|
| 80 |
-
for a, b,
|
| 81 |
-
parts.append(f'<rect x="{x(a):.1f}" y="{top}" width="{x(b) - x(a):.1f}" height="{bar}" fill="{
|
| 82 |
f'opacity=".8"/>')
|
| 83 |
parts.append(f'<text x="{(x(a) + x(b)) / 2:.1f}" y="{top - 7}" text-anchor="middle" fill="currentColor" '
|
| 84 |
f'opacity=".85">{label}</text>')
|
|
@@ -102,7 +102,7 @@ def gauge_svg(excess=None, width=440):
|
|
| 102 |
|
| 103 |
|
| 104 |
def histogram_svg(F, R, mask, threshold, width=440, height=170):
|
| 105 |
-
"""Where the scores fall on the papyrus: the control's (reversed) scores in
|
| 106 |
|
| 107 |
Bar heights are the share of papyrus pixels per bin on a square-root axis, so the 1 % tail stays visible."""
|
| 108 |
f, r = np.asarray(F, np.float32)[mask], np.asarray(R, np.float32)[mask]
|
|
@@ -118,14 +118,14 @@ def histogram_svg(F, R, mask, threshold, width=440, height=170):
|
|
| 118 |
X = lambda v: left + pw * (v - lo) / max(hi - lo, 1e-9) # noqa: E731
|
| 119 |
Y = lambda share: top + ph * (1 - np.sqrt(share) / peak) # noqa: E731
|
| 120 |
|
| 121 |
-
def bars(h,
|
| 122 |
out = []
|
| 123 |
for i, s in enumerate(h):
|
| 124 |
if s <= 0:
|
| 125 |
continue
|
| 126 |
y = Y(s)
|
| 127 |
out.append(f'<rect x="{left + i * bw + inset:.1f}" y="{y:.1f}" width="{max(bw - 2 * inset, 0.5):.1f}" '
|
| 128 |
-
f'height="{top + ph - y:.1f}" fill="{
|
| 129 |
return "".join(out)
|
| 130 |
|
| 131 |
tx = X(threshold)
|
|
@@ -171,10 +171,10 @@ def compare_html(volume, F, R, decision):
|
|
| 171 |
f'{(share or 0):.1%} of papyrus above T</figcaption></figure>')
|
| 172 |
return (f'<div class="cc-figs"><div class="cc-h">The map and its control, on one scale</div>'
|
| 173 |
f'<div class="cc-pair">{"".join(tiles)}</div>'
|
| 174 |
-
'<div class="cc-cap">Amber: model score above T, the level only 1 % of the control passes.
|
| 175 |
'Black: not papyrus.</div>'
|
| 176 |
f'<div class="cc-h">Where the scores fall</div>{histogram_svg(F, R, mask, T)}'
|
| 177 |
-
'<div class="cc-cap">Share of papyrus pixels at each score (square-root scale): the control in
|
| 178 |
'order in amber. The answer is your share above T divided by the control\'s 1 %. Model output, not a '
|
| 179 |
'reading.</div></div>')
|
| 180 |
|
|
@@ -183,7 +183,7 @@ def composite(volume, F, R, kept, decision, side=480, gap=12, background=(23, 26
|
|
| 183 |
"""CT, the overlay and the two maps in one RGB strip (for the README and the downloads), no text baked in."""
|
| 184 |
from PIL import Image
|
| 185 |
ct = np.asarray(volume)[np.asarray(volume).shape[0] // 2]
|
| 186 |
-
g =
|
| 187 |
mask = gate.papyrus(ct)
|
| 188 |
T = decision.get("threshold") or 1.0
|
| 189 |
top = float(np.percentile(np.concatenate([F[mask], R[mask]]), 99.8)) if mask.any() else 1.0
|
|
|
|
| 1 |
"""Crackle Check pictures: the maps beside their control, where the scores fall, and the answer scale (numpy, Pillow).
|
| 2 |
|
| 3 |
Everything here draws what the gate already measured; nothing changes the answer. Map tiles are WebP data URIs and
|
| 4 |
+
the charts are inline SVG that take the page's text color, so they read in light and dark themes alike. The colors
|
| 5 |
+
follow the overlay: gray for CT and for map values under the control's threshold T, copper to amber to ivory above T,
|
| 6 |
so amber always means "passes the control's 99th percentile". Everything drawn is model output, not a reading.
|
| 7 |
"""
|
| 8 |
|
|
|
|
| 29 |
return np.stack([np.interp(t, xs, [c[i] / 255.0 for _, c in RAMP]) for i in range(3)], -1)
|
| 30 |
|
| 31 |
|
| 32 |
+
def ct_gray(ct):
|
| 33 |
"""A CT layer stretched to its 1st to 99th percentile, float 0 to 1."""
|
| 34 |
ct = np.asarray(ct, np.float32)
|
| 35 |
lo, hi = np.percentile(ct, [1, 99])
|
|
|
|
| 37 |
|
| 38 |
|
| 39 |
def heat(m, threshold, top, papyrus_mask):
|
| 40 |
+
"""One map as RGB uint8: under T a dim gray that rises with the value, at or above T the copper ramp.
|
| 41 |
|
| 42 |
The same T and top are used for a map and its control, so the two tiles are on one scale."""
|
| 43 |
m = np.asarray(m, np.float32)
|
|
|
|
| 77 |
f'aria-label="Answer scale: No under 1.5, Maybe 1.5 to 3, Yes from 3 times the control'
|
| 78 |
+ (f'; this region {_f(excess)} times' if excess is not None else '') + '" '
|
| 79 |
'style="width:100%;max-width:560px;height:auto;font:12px ui-sans-serif,system-ui,sans-serif">']
|
| 80 |
+
for a, b, color, label in zones:
|
| 81 |
+
parts.append(f'<rect x="{x(a):.1f}" y="{top}" width="{x(b) - x(a):.1f}" height="{bar}" fill="{color}" '
|
| 82 |
f'opacity=".8"/>')
|
| 83 |
parts.append(f'<text x="{(x(a) + x(b)) / 2:.1f}" y="{top - 7}" text-anchor="middle" fill="currentColor" '
|
| 84 |
f'opacity=".85">{label}</text>')
|
|
|
|
| 102 |
|
| 103 |
|
| 104 |
def histogram_svg(F, R, mask, threshold, width=440, height=170):
|
| 105 |
+
"""Where the scores fall on the papyrus: the control's (reversed) scores in gray, yours in amber, T dashed.
|
| 106 |
|
| 107 |
Bar heights are the share of papyrus pixels per bin on a square-root axis, so the 1 % tail stays visible."""
|
| 108 |
f, r = np.asarray(F, np.float32)[mask], np.asarray(R, np.float32)[mask]
|
|
|
|
| 118 |
X = lambda v: left + pw * (v - lo) / max(hi - lo, 1e-9) # noqa: E731
|
| 119 |
Y = lambda share: top + ph * (1 - np.sqrt(share) / peak) # noqa: E731
|
| 120 |
|
| 121 |
+
def bars(h, color, opacity, inset):
|
| 122 |
out = []
|
| 123 |
for i, s in enumerate(h):
|
| 124 |
if s <= 0:
|
| 125 |
continue
|
| 126 |
y = Y(s)
|
| 127 |
out.append(f'<rect x="{left + i * bw + inset:.1f}" y="{y:.1f}" width="{max(bw - 2 * inset, 0.5):.1f}" '
|
| 128 |
+
f'height="{top + ph - y:.1f}" fill="{color}" opacity="{opacity}"/>')
|
| 129 |
return "".join(out)
|
| 130 |
|
| 131 |
tx = X(threshold)
|
|
|
|
| 171 |
f'{(share or 0):.1%} of papyrus above T</figcaption></figure>')
|
| 172 |
return (f'<div class="cc-figs"><div class="cc-h">The map and its control, on one scale</div>'
|
| 173 |
f'<div class="cc-pair">{"".join(tiles)}</div>'
|
| 174 |
+
'<div class="cc-cap">Amber: model score above T, the level only 1 % of the control passes. Gray: below T. '
|
| 175 |
'Black: not papyrus.</div>'
|
| 176 |
f'<div class="cc-h">Where the scores fall</div>{histogram_svg(F, R, mask, T)}'
|
| 177 |
+
'<div class="cc-cap">Share of papyrus pixels at each score (square-root scale): the control in gray, your '
|
| 178 |
'order in amber. The answer is your share above T divided by the control\'s 1 %. Model output, not a '
|
| 179 |
'reading.</div></div>')
|
| 180 |
|
|
|
|
| 183 |
"""CT, the overlay and the two maps in one RGB strip (for the README and the downloads), no text baked in."""
|
| 184 |
from PIL import Image
|
| 185 |
ct = np.asarray(volume)[np.asarray(volume).shape[0] // 2]
|
| 186 |
+
g = ct_gray(ct)
|
| 187 |
mask = gate.papyrus(ct)
|
| 188 |
T = decision.get("threshold") or 1.0
|
| 189 |
top = float(np.percentile(np.concatenate([F[mask], R[mask]]), 99.8)) if mask.any() else 1.0
|
koine_engine.py
CHANGED
|
@@ -3,11 +3,11 @@
|
|
| 3 |
Ink Lens's engine (engine.py, a copied file this folder never edits) runs 128 px villa-flat readers. Koine readers use
|
| 4 |
larger patches (Reader V3: 17 x 384 x 384), so they get this small engine with the same contract: run(volume) returns
|
| 5 |
{"forward", "reverse", ...} float32 maps in 0 to 1, the reverse map from the same layers in reversed depth order. It
|
| 6 |
-
follows koine's own inference (koine_machines/inference/infer.py at the pinned commit): the
|
| 7 |
or /255 preprocessing per patch, sigmoid of the model's logits, patches at 50 % overlap blended with koine's floored
|
| 8 |
Hann map, all-zero patches skipped. The checkpoint is SHA-256 pinned and loaded with torch.load(weights_only=True).
|
| 9 |
|
| 10 |
-
To add a reader, add an entry to KOINE_READERS (a fixed URL, its SHA-256 and
|
| 11 |
"""
|
| 12 |
|
| 13 |
from __future__ import annotations
|
|
@@ -78,7 +78,7 @@ def _tarball(tmp):
|
|
| 78 |
|
| 79 |
def ensure_koine(cache=engine.CACHE):
|
| 80 |
"""koine_machines on sys.path (KOINE_SRC, a local ink-detection folder, when set), plus villa's vesuvius package
|
| 81 |
-
from engine.ensure_villa for the robust
|
| 82 |
engine.ensure_villa(cache)
|
| 83 |
if os.environ.get("KOINE_SRC"):
|
| 84 |
src = Path(os.environ["KOINE_SRC"]).expanduser().resolve()
|
|
@@ -167,7 +167,7 @@ class KoineReader:
|
|
| 167 |
depth, patch = self.info["depth"], self.info["patch"]
|
| 168 |
if L < depth:
|
| 169 |
raise engine.EngineError(f"{L} layers: {self.name} reads {depth}")
|
| 170 |
-
start = L // 2 - depth // 2 # koine's
|
| 171 |
sub = vol[start:start + depth]
|
| 172 |
ph, pw = max(H, patch), max(W, patch) # a region smaller than one patch is padded with zeros
|
| 173 |
padded = np.zeros((depth, ph, pw), np.uint8)
|
|
|
|
| 3 |
Ink Lens's engine (engine.py, a copied file this folder never edits) runs 128 px villa-flat readers. Koine readers use
|
| 4 |
larger patches (Reader V3: 17 x 384 x 384), so they get this small engine with the same contract: run(volume) returns
|
| 5 |
{"forward", "reverse", ...} float32 maps in 0 to 1, the reverse map from the same layers in reversed depth order. It
|
| 6 |
+
follows koine's own inference (koine_machines/inference/infer.py at the pinned commit): the center 17 layers, robust
|
| 7 |
or /255 preprocessing per patch, sigmoid of the model's logits, patches at 50 % overlap blended with koine's floored
|
| 8 |
Hann map, all-zero patches skipped. The checkpoint is SHA-256 pinned and loaded with torch.load(weights_only=True).
|
| 9 |
|
| 10 |
+
To add a reader, add an entry to KOINE_READERS (a fixed URL, its SHA-256 and license); app.py lists it.
|
| 11 |
"""
|
| 12 |
|
| 13 |
from __future__ import annotations
|
|
|
|
| 78 |
|
| 79 |
def ensure_koine(cache=engine.CACHE):
|
| 80 |
"""koine_machines on sys.path (KOINE_SRC, a local ink-detection folder, when set), plus villa's vesuvius package
|
| 81 |
+
from engine.ensure_villa for the robust normalization koine imports."""
|
| 82 |
engine.ensure_villa(cache)
|
| 83 |
if os.environ.get("KOINE_SRC"):
|
| 84 |
src = Path(os.environ["KOINE_SRC"]).expanduser().resolve()
|
|
|
|
| 167 |
depth, patch = self.info["depth"], self.info["patch"]
|
| 168 |
if L < depth:
|
| 169 |
raise engine.EngineError(f"{L} layers: {self.name} reads {depth}")
|
| 170 |
+
start = L // 2 - depth // 2 # koine's center crop (upper center when uneven)
|
| 171 |
sub = vol[start:start + depth]
|
| 172 |
ph, pw = max(H, patch), max(W, patch) # a region smaller than one patch is padded with zeros
|
| 173 |
padded = np.zeros((depth, ph, pw), np.uint8)
|
link.py
CHANGED
|
@@ -7,7 +7,7 @@ them toward the readers' 9 um grid and hands the stack to the usual check. Nothi
|
|
| 7 |
|
| 8 |
scroll, segment = parse("PHerc0841/segments/20260220213127-w00") # or a bucket URL, or "PHerc0841 SEGMENT"
|
| 9 |
vols = volumes(scroll, segment) # its surface volumes, cheapest first
|
| 10 |
-
stack, um, note = region(scroll, segment, vols[0]["name"], x, y) # x, y: the
|
| 11 |
"""
|
| 12 |
|
| 13 |
from __future__ import annotations
|
|
@@ -137,7 +137,7 @@ def label(v):
|
|
| 137 |
|
| 138 |
def region(scroll, segment, name, x, y, side=SIDE, fetch=volio._get, opener=volio.open_zarr):
|
| 139 |
"""(stack uint8 layers x rows x columns on about the 9.4 um grid, um per pixel after pooling, note) for the
|
| 140 |
-
side x side region
|
| 141 |
past an edge is moved inside it and the note says so."""
|
| 142 |
side = int(side)
|
| 143 |
if not MIN_SIDE <= side <= MAX_SIDE:
|
|
@@ -177,7 +177,7 @@ def region(scroll, segment, name, x, y, side=SIDE, fetch=volio._get, opener=voli
|
|
| 177 |
c = stack.shape[0] // 2
|
| 178 |
stack = stack[c - engine.MAX_LAYERS // 2:c - engine.MAX_LAYERS // 2 + engine.MAX_LAYERS]
|
| 179 |
moved = (x0 + src // 2, y0 + src // 2) != (x, y)
|
| 180 |
-
note = (f"open data {scroll}/segments/{segment}/surface-volumes/{name},
|
| 181 |
f"y {y0 + src // 2}, {src} px at {um} um" + (f", pooled {pool} x {pool} x {pool}" if pool > 1 else "")
|
| 182 |
+ (" (moved inside the volume's edge)" if moved else ""))
|
| 183 |
return np.ascontiguousarray(stack), round(um * pool, 4), note
|
|
|
|
| 7 |
|
| 8 |
scroll, segment = parse("PHerc0841/segments/20260220213127-w00") # or a bucket URL, or "PHerc0841 SEGMENT"
|
| 9 |
vols = volumes(scroll, segment) # its surface volumes, cheapest first
|
| 10 |
+
stack, um, note = region(scroll, segment, vols[0]["name"], x, y) # x, y: the center, in that volume's pixels
|
| 11 |
"""
|
| 12 |
|
| 13 |
from __future__ import annotations
|
|
|
|
| 137 |
|
| 138 |
def region(scroll, segment, name, x, y, side=SIDE, fetch=volio._get, opener=volio.open_zarr):
|
| 139 |
"""(stack uint8 layers x rows x columns on about the 9.4 um grid, um per pixel after pooling, note) for the
|
| 140 |
+
side x side region centered on column x, row y of the named surface volume (its own pixels). A region that runs
|
| 141 |
past an edge is moved inside it and the note says so."""
|
| 142 |
side = int(side)
|
| 143 |
if not MIN_SIDE <= side <= MAX_SIDE:
|
|
|
|
| 177 |
c = stack.shape[0] // 2
|
| 178 |
stack = stack[c - engine.MAX_LAYERS // 2:c - engine.MAX_LAYERS // 2 + engine.MAX_LAYERS]
|
| 179 |
moved = (x0 + src // 2, y0 + src // 2) != (x, y)
|
| 180 |
+
note = (f"open data {scroll}/segments/{segment}/surface-volumes/{name}, center x {x0 + src // 2}, "
|
| 181 |
f"y {y0 + src // 2}, {src} px at {um} um" + (f", pooled {pool} x {pool} x {pool}" if pool > 1 else "")
|
| 182 |
+ (" (moved inside the volume's edge)" if moved else ""))
|
| 183 |
return np.ascontiguousarray(stack), round(um * pool, 4), note
|
onescroll.html
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<!-- space-onescroll:start -->
|
| 2 |
+
<!-- one page scroll bar: written by python -m kit spaceconnect render; do not edit -->
|
| 3 |
+
<style>iframe[srcdoc]{display:block;max-height:none!important}</style>
|
| 4 |
+
<script>
|
| 5 |
+
(function () {
|
| 6 |
+
if (window.__oneScroll) return;
|
| 7 |
+
window.__oneScroll = true;
|
| 8 |
+
var CSS = 'html,body{height:auto!important;min-height:0!important}html{overflow:hidden!important}';
|
| 9 |
+
function size(f) {
|
| 10 |
+
var d, st = f.__os;
|
| 11 |
+
try { d = f.contentDocument; } catch (e) { return; }
|
| 12 |
+
if (!d || !d.body || st.off) return;
|
| 13 |
+
var de = d.documentElement, r = Math.ceil(de.getBoundingClientRect().height), over = de.scrollHeight;
|
| 14 |
+
var ch = f.clientHeight, t = r;
|
| 15 |
+
if (over > ch + 1) t = Math.max(r, over);
|
| 16 |
+
else if (r >= st.r) t = Math.max(r, ch);
|
| 17 |
+
st.r = r;
|
| 18 |
+
if (Math.abs(t - ch) < 2) return;
|
| 19 |
+
var now = Date.now();
|
| 20 |
+
st.n = now - st.t0 < 4000 ? st.n + 1 : 1;
|
| 21 |
+
if (st.n === 1) st.t0 = now;
|
| 22 |
+
if (st.n > 40) { st.off = true; f.removeAttribute('scrolling'); var g = d.getElementById('space-onescroll'); if (g) g.remove(); return; }
|
| 23 |
+
f.style.setProperty('height', t + 'px', 'important');
|
| 24 |
+
}
|
| 25 |
+
function attach(f) {
|
| 26 |
+
var d, st = f.__os;
|
| 27 |
+
try { d = f.contentDocument; } catch (e) { return; }
|
| 28 |
+
if (!d || !d.documentElement || st.doc === d) return;
|
| 29 |
+
st.doc = d; st.r = Infinity; st.n = 0; st.off = false;
|
| 30 |
+
if (!d.getElementById('space-onescroll')) {
|
| 31 |
+
var s = d.createElement('style'); s.id = 'space-onescroll'; s.textContent = CSS;
|
| 32 |
+
(d.head || d.documentElement).appendChild(s);
|
| 33 |
+
}
|
| 34 |
+
f.setAttribute('scrolling', 'no');
|
| 35 |
+
f.style.setProperty('min-height', '0', 'important');
|
| 36 |
+
f.style.setProperty('max-height', 'none', 'important');
|
| 37 |
+
var w = d.defaultView, go = function () { size(f); };
|
| 38 |
+
if (w && w.ResizeObserver) { var ro = new w.ResizeObserver(go); ro.observe(d.documentElement); if (d.body) ro.observe(d.body); }
|
| 39 |
+
d.addEventListener('load', go, true);
|
| 40 |
+
go();
|
| 41 |
+
}
|
| 42 |
+
function bind(f) {
|
| 43 |
+
if (f.__os || f.getAttribute('data-scroll') === 'inner') return;
|
| 44 |
+
try { if (!f.contentDocument) return; } catch (e) { return; }
|
| 45 |
+
f.__os = {t0: 0, n: 0, r: 0};
|
| 46 |
+
f.addEventListener('load', function () { attach(f); });
|
| 47 |
+
if (f.contentDocument.readyState !== 'loading') attach(f);
|
| 48 |
+
else f.contentDocument.addEventListener('DOMContentLoaded', function () { attach(f); });
|
| 49 |
+
}
|
| 50 |
+
function scan() { var fs = document.getElementsByTagName('iframe'); for (var i = 0; i < fs.length; i++) bind(fs[i]); }
|
| 51 |
+
function start() {
|
| 52 |
+
scan();
|
| 53 |
+
new MutationObserver(scan).observe(document.documentElement, {childList: true, subtree: true});
|
| 54 |
+
window.addEventListener('resize', function () {
|
| 55 |
+
var fs = document.getElementsByTagName('iframe');
|
| 56 |
+
for (var i = 0; i < fs.length; i++) if (fs[i].__os) size(fs[i]);
|
| 57 |
+
});
|
| 58 |
+
}
|
| 59 |
+
if (document.readyState === 'loading') document.addEventListener('DOMContentLoaded', start); else start();
|
| 60 |
+
})();
|
| 61 |
+
</script>
|
| 62 |
+
<!-- space-onescroll:end -->
|
samples.json
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
{
|
| 2 |
-
"note": "Public crops of released segments that are not prize-eligible. Ink Lens reads them anonymously from the open-data bucket when someone runs one; nothing is bundled. Crops are (y0, y1, x0, x1) on the segment's 9 um surface volume, the 640 px windows of densest labeled ink used in docs/results.json. A window chosen with the labels gives a development number, not a blind test.",
|
| 3 |
"bucket": "https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com",
|
| 4 |
"samples": [
|
| 5 |
{
|
|
@@ -11,9 +11,37 @@
|
|
| 11 |
"labels": "ink-labels/2.403um-volume-20260319124803/20260918",
|
| 12 |
"tifxyz": "mesh/20260220213127-on-20250821151531-9.366um.tifxyz",
|
| 13 |
"um_per_px": 9.366,
|
| 14 |
-
"surface_shape": [
|
| 15 |
-
|
| 16 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
},
|
| 18 |
{
|
| 19 |
"id": "0841-ag896",
|
|
@@ -23,9 +51,37 @@
|
|
| 23 |
"surface_volume": "surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr",
|
| 24 |
"labels": "ink-labels/2.403um-volume-20260319124803/20260918",
|
| 25 |
"um_per_px": 9.366,
|
| 26 |
-
"surface_shape": [
|
| 27 |
-
|
| 28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
},
|
| 30 |
{
|
| 31 |
"id": "0841-ag405",
|
|
@@ -35,9 +91,37 @@
|
|
| 35 |
"surface_volume": "surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr",
|
| 36 |
"labels": "ink-labels/2.403um-volume-20260319124803/20260918",
|
| 37 |
"um_per_px": 9.366,
|
| 38 |
-
"surface_shape": [
|
| 39 |
-
|
| 40 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
},
|
| 42 |
{
|
| 43 |
"id": "0139-w035",
|
|
@@ -47,9 +131,37 @@
|
|
| 47 |
"surface_volume": "surface-volumes/9.362um-1.2m-113keV-volume-20250728140407.zarr",
|
| 48 |
"labels": "ink-labels/2.399um-volume-20260102150214/20260918",
|
| 49 |
"um_per_px": 9.362,
|
| 50 |
-
"surface_shape": [
|
| 51 |
-
|
| 52 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
}
|
| 54 |
]
|
| 55 |
}
|
|
|
|
| 1 |
{
|
| 2 |
+
"note": "Public crops of released segments that are not prize-eligible. Ink Lens reads them anonymously from the open-data bucket when someone runs one; nothing is bundled. Crops are (y0, y1, x0, x1) on the segment's 9 um surface volume, the 640 px windows of densest labeled ink used in docs/results.json. A window chosen with the labels gives a development number, not a blind test. scan24: the same segment on its finer scan (about 2.4 um), where every model of the comparison runs: its crop is on the 9.6 um grid (the labels' level 2, the 2.4 um render pooled 4x), centered where the 9 um crop is, by the two canvases' proportions; ct is the scan the 3D models read.",
|
| 3 |
"bucket": "https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com",
|
| 4 |
"samples": [
|
| 5 |
{
|
|
|
|
| 11 |
"labels": "ink-labels/2.403um-volume-20260319124803/20260918",
|
| 12 |
"tifxyz": "mesh/20260220213127-on-20250821151531-9.366um.tifxyz",
|
| 13 |
"um_per_px": 9.366,
|
| 14 |
+
"surface_shape": [
|
| 15 |
+
4220,
|
| 16 |
+
4760
|
| 17 |
+
],
|
| 18 |
+
"crop": [
|
| 19 |
+
2624,
|
| 20 |
+
3264,
|
| 21 |
+
2688,
|
| 22 |
+
3328
|
| 23 |
+
],
|
| 24 |
+
"role": "development",
|
| 25 |
+
"scan24": {
|
| 26 |
+
"surface_volume": "surface-volumes/2.403um-0.22m-77keV-volume-20260319124803.zarr",
|
| 27 |
+
"um_per_px": 2.403,
|
| 28 |
+
"level0_shape": [
|
| 29 |
+
16460,
|
| 30 |
+
18560
|
| 31 |
+
],
|
| 32 |
+
"grid_shape": [
|
| 33 |
+
4115,
|
| 34 |
+
4640
|
| 35 |
+
],
|
| 36 |
+
"crop": [
|
| 37 |
+
2551,
|
| 38 |
+
3191,
|
| 39 |
+
2612,
|
| 40 |
+
3252
|
| 41 |
+
],
|
| 42 |
+
"tifxyz": "mesh/20260220213127-on-20260319124803-2.403um.tifxyz",
|
| 43 |
+
"ct": "PHerc0841/volumes/20260319124803-2.403um-0.2m-77keV-masked.zarr"
|
| 44 |
+
}
|
| 45 |
},
|
| 46 |
{
|
| 47 |
"id": "0841-ag896",
|
|
|
|
| 51 |
"surface_volume": "surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr",
|
| 52 |
"labels": "ink-labels/2.403um-volume-20260319124803/20260918",
|
| 53 |
"um_per_px": 9.366,
|
| 54 |
+
"surface_shape": [
|
| 55 |
+
4640,
|
| 56 |
+
4720
|
| 57 |
+
],
|
| 58 |
+
"crop": [
|
| 59 |
+
2496,
|
| 60 |
+
3136,
|
| 61 |
+
1600,
|
| 62 |
+
2240
|
| 63 |
+
],
|
| 64 |
+
"role": "development",
|
| 65 |
+
"scan24": {
|
| 66 |
+
"surface_volume": "surface-volumes/2.403um-0.22m-77keV-volume-20260319124803.zarr",
|
| 67 |
+
"um_per_px": 2.403,
|
| 68 |
+
"level0_shape": [
|
| 69 |
+
18080,
|
| 70 |
+
18400
|
| 71 |
+
],
|
| 72 |
+
"grid_shape": [
|
| 73 |
+
4520,
|
| 74 |
+
4600
|
| 75 |
+
],
|
| 76 |
+
"crop": [
|
| 77 |
+
2423,
|
| 78 |
+
3063,
|
| 79 |
+
1551,
|
| 80 |
+
2191
|
| 81 |
+
],
|
| 82 |
+
"tifxyz": "mesh/20260220214732-on-20260319124803-2.403um.tifxyz",
|
| 83 |
+
"ct": "PHerc0841/volumes/20260319124803-2.403um-0.2m-77keV-masked.zarr"
|
| 84 |
+
}
|
| 85 |
},
|
| 86 |
{
|
| 87 |
"id": "0841-ag405",
|
|
|
|
| 91 |
"surface_volume": "surface-volumes/9.366um-1.2m-113keV-volume-20250821151531.zarr",
|
| 92 |
"labels": "ink-labels/2.403um-volume-20260319124803/20260918",
|
| 93 |
"um_per_px": 9.366,
|
| 94 |
+
"surface_shape": [
|
| 95 |
+
3760,
|
| 96 |
+
4900
|
| 97 |
+
],
|
| 98 |
+
"crop": [
|
| 99 |
+
1024,
|
| 100 |
+
1664,
|
| 101 |
+
2496,
|
| 102 |
+
3136
|
| 103 |
+
],
|
| 104 |
+
"role": "development",
|
| 105 |
+
"scan24": {
|
| 106 |
+
"surface_volume": "surface-volumes/2.403um-0.22m-77keV-volume-20260319124803.zarr",
|
| 107 |
+
"um_per_px": 2.403,
|
| 108 |
+
"level0_shape": [
|
| 109 |
+
14660,
|
| 110 |
+
19100
|
| 111 |
+
],
|
| 112 |
+
"grid_shape": [
|
| 113 |
+
3665,
|
| 114 |
+
4775
|
| 115 |
+
],
|
| 116 |
+
"crop": [
|
| 117 |
+
990,
|
| 118 |
+
1630,
|
| 119 |
+
2424,
|
| 120 |
+
3064
|
| 121 |
+
],
|
| 122 |
+
"tifxyz": "mesh/20260221022814-on-20260319124803-2.403um.tifxyz",
|
| 123 |
+
"ct": "PHerc0841/volumes/20260319124803-2.403um-0.2m-77keV-masked.zarr"
|
| 124 |
+
}
|
| 125 |
},
|
| 126 |
{
|
| 127 |
"id": "0139-w035",
|
|
|
|
| 131 |
"surface_volume": "surface-volumes/9.362um-1.2m-113keV-volume-20250728140407.zarr",
|
| 132 |
"labels": "ink-labels/2.399um-volume-20260102150214/20260918",
|
| 133 |
"um_per_px": 9.362,
|
| 134 |
+
"surface_shape": [
|
| 135 |
+
5820,
|
| 136 |
+
5240
|
| 137 |
+
],
|
| 138 |
+
"crop": [
|
| 139 |
+
1920,
|
| 140 |
+
2560,
|
| 141 |
+
2304,
|
| 142 |
+
2944
|
| 143 |
+
],
|
| 144 |
+
"role": "training",
|
| 145 |
+
"scan24": {
|
| 146 |
+
"surface_volume": "surface-volumes/2.399um-0.22m-78keV-volume-20260102150214.zarr",
|
| 147 |
+
"um_per_px": 2.399,
|
| 148 |
+
"level0_shape": [
|
| 149 |
+
22640,
|
| 150 |
+
20400
|
| 151 |
+
],
|
| 152 |
+
"grid_shape": [
|
| 153 |
+
5660,
|
| 154 |
+
5100
|
| 155 |
+
],
|
| 156 |
+
"crop": [
|
| 157 |
+
1858,
|
| 158 |
+
2498,
|
| 159 |
+
2234,
|
| 160 |
+
2874
|
| 161 |
+
],
|
| 162 |
+
"tifxyz": "mesh/20260317000000-on-20260102150214-2.399um.tifxyz",
|
| 163 |
+
"ct": "PHerc0139/volumes/20260102150214-2.399um-0.2m-78keV-masked.zarr"
|
| 164 |
+
}
|
| 165 |
}
|
| 166 |
]
|
| 167 |
}
|
upload.py
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |
Accepted: a single image (PNG, JPEG, WebP, BMP, one-page TIFF), a multi-page TIFF of surface layers, a ZIP of layer
|
| 4 |
images (ordered by the numbers in their names, as villa and VC3D name them: 00.tif, 01.tif, ...), or an .npz with an
|
| 5 |
-
array "volume" (Ink Lens's region format).
|
| 6 |
"""
|
| 7 |
|
| 8 |
from __future__ import annotations
|
|
@@ -25,7 +25,7 @@ class UploadError(ValueError):
|
|
| 25 |
|
| 26 |
|
| 27 |
def to_u8(a):
|
| 28 |
-
"""
|
| 29 |
a = np.asarray(a)
|
| 30 |
if a.ndim == 3 and a.shape[-1] in (3, 4):
|
| 31 |
rgb_u8 = a.dtype == np.uint8
|
|
|
|
| 2 |
|
| 3 |
Accepted: a single image (PNG, JPEG, WebP, BMP, one-page TIFF), a multi-page TIFF of surface layers, a ZIP of layer
|
| 4 |
images (ordered by the numbers in their names, as villa and VC3D name them: 00.tif, 01.tif, ...), or an .npz with an
|
| 5 |
+
array "volume" (Ink Lens's region format). Color is turned to gray, 16-bit is scaled to 8-bit. Nothing is kept.
|
| 6 |
"""
|
| 7 |
|
| 8 |
from __future__ import annotations
|
|
|
|
| 25 |
|
| 26 |
|
| 27 |
def to_u8(a):
|
| 28 |
+
"""Gray uint8 from a 2D or (H, W, C) array of any integer or float type."""
|
| 29 |
a = np.asarray(a)
|
| 30 |
if a.ndim == 3 and a.shape[-1] in (3, 4):
|
| 31 |
rgb_u8 = a.dtype == np.uint8
|