Ink Bench: examples on all 37 windings, researcher takeaway (rank by AP, report the X-ray floor), ink_9um credit, untrack pycache, sturdier leaderboard test
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- README.md +5 -1
- data/bench/index.json +124 -0
- data/bench/w023_gpu.png +3 -0
- data/bench/w024_gpu.png +3 -0
- data/bench/w025_gpu.png +3 -0
- data/bench/w026_gpu.png +3 -0
- data/bench/w027_gpu.png +3 -0
- data/bench/w028_gpu.png +3 -0
- data/bench/w029_gpu.png +3 -0
- data/bench/w030_gpu.png +3 -0
- data/bench/w031_gpu.png +3 -0
- data/bench/w032_gpu.png +3 -0
- data/bench/w033_gpu.png +3 -0
- data/bench/w034_gpu.png +3 -0
- data/bench/w035_gpu.png +3 -0
- data/bench/w036_gpu.png +3 -0
- data/bench/w037_gpu.png +3 -0
- data/bench/w038_gpu.png +3 -0
- data/bench/w039_gpu.png +3 -0
- data/bench/w040_gpu.png +3 -0
- data/bench/w041_gpu.png +3 -0
- data/bench/w042_gpu.png +3 -0
- data/bench/w043_gpu.png +3 -0
- data/bench/w044_gpu.png +3 -0
- data/bench/w045_gpu.png +3 -0
- data/bench/w052_gpu.png +3 -0
- data/bench/w053_gpu.png +3 -0
- data/bench/w054_gpu.png +3 -0
- data/bench/w055_gpu.png +3 -0
- data/bench/w056_gpu.png +3 -0
- data/bench/w057_gpu.png +3 -0
- data/bench/w058_gpu.png +3 -0
- data/bench/w059_gpu.png +3 -0
- docs/BENCH.md +7 -2
- docs/examples/bench/w023-ct-baseline.json +334 -0
- docs/examples/bench/w023-ink9um.json +335 -0
- docs/examples/bench/w023-noise-baseline.json +335 -0
- docs/examples/bench/w024-ct-baseline.json +334 -0
- docs/examples/bench/w024-ink9um.json +335 -0
- docs/examples/bench/w024-noise-baseline.json +335 -0
- docs/examples/bench/w025-ct-baseline.json +334 -0
- docs/examples/bench/w025-ink9um.json +335 -0
- docs/examples/bench/w025-noise-baseline.json +335 -0
- docs/examples/bench/w026-ct-baseline.json +334 -0
- docs/examples/bench/w026-ink9um.json +335 -0
- docs/examples/bench/w026-noise-baseline.json +335 -0
- docs/examples/bench/w027-ct-baseline.json +334 -0
- docs/examples/bench/w027-ink9um.json +335 -0
- docs/examples/bench/w027-noise-baseline.json +335 -0
- docs/examples/bench/w028-ct-baseline.json +334 -0
README.md
CHANGED
|
@@ -26,7 +26,7 @@ PHerc. 139 is a papyrus roll carbonized when Vesuvius buried Herculaneum in 79 C
|
|
| 26 |
- **Shapes**: the 9.362 µm tifxyz meshes of the 37 traced windings (`w023` to `w059`), decimated to 579,450 vertices.
|
| 27 |
- **X-ray layer**: the CT intensity on each surface, from the public 9.362 µm surface volumes.
|
| 28 |
- **Published ink (2.4 µm), amber**: the Vesuvius Challenge's public ink detection on the 2.4 µm scan. In 3D it is drawn through a local-contrast filter so letter-sized strokes stand out. The close-up view shows the published preview without filtering, downscaled to 2,400 px on its long side and re-saved as JPEG.
|
| 29 |
-
- **Our ink model (9 µm), cyan**: inked by running the public [`scrollprize/ink_9um`](https://huggingface.co/scrollprize/ink_9um) model (seed 42, step 75000) on the 9.362 µm scan.
|
| 30 |
|
| 31 |
Unrolling lays the windings end to end, outermost first. Each traced winding runs a little past one full turn, so roughly a tenth of the 2.8 m strip repeats surface from its neighbour.
|
| 32 |
|
|
@@ -57,3 +57,7 @@ Data: Giorgio Angelotti, Stephen Parsons, Sean Johnson, Elian Rafael Dal Prà, J
|
|
| 57 |
## Ink Bench
|
| 58 |
|
| 59 |
Score your own ink prediction for any winding against the published ink, in the browser (nothing is uploaded): automatic UV alignment, side by side, swipe and difference views, Pearson r, IoU, Dice and a precision and recall curve (agreement with another model's output, not ground truth), letter boxes marked "candidate for expert review", and a one-click reproducible bundle. `tools/inkbench.py` gives identical results in Python and on the command line. See [docs/BENCH.md](docs/BENCH.md) and the open bundle format [docs/BENCH_FORMAT.md](docs/BENCH_FORMAT.md). The Bench code and format are Apache-2.0 (`LICENSE-APACHE`, `NOTICE`, by Chase Hendrick); the data stays CC BY-NC 4.0.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
- **Shapes**: the 9.362 µm tifxyz meshes of the 37 traced windings (`w023` to `w059`), decimated to 579,450 vertices.
|
| 27 |
- **X-ray layer**: the CT intensity on each surface, from the public 9.362 µm surface volumes.
|
| 28 |
- **Published ink (2.4 µm), amber**: the Vesuvius Challenge's public ink detection on the 2.4 µm scan. In 3D it is drawn through a local-contrast filter so letter-sized strokes stand out. The close-up view shows the published preview without filtering, downscaled to 2,400 px on its long side and re-saved as JPEG.
|
| 29 |
+
- **Our ink model (9 µm), cyan**: inked by running the public [`scrollprize/ink_9um`](https://huggingface.co/scrollprize/ink_9um) model (seed 42, step 75000) on the 9.362 µm scan. The ink_9um model was trained by the Vesuvius Challenge and run by Chase Hendrick, on Hugging Face ZeroGPU and Modal, over all 37 windings (`w023` to `w059`). The first batch was 37,141 patches in 57 s of measured GPU time. The page reads the covered windings from the manifest, so its text updates as windings are added. The 9.362 µm scan is about four times coarser than the 2.4 µm one, so this layer is noisier. `w047` and `w050` ran with patch overlap 0.5; others ran with 0.25 or 0.2, and their 128 px patch grid shows as faint square tiles. Per-run records (checkpoint sha256, villa commit, requested and measured GPU seconds) are in `data/gpu/receipts/`; `data/gpu.json` is the summary the page reads.
|
| 30 |
|
| 31 |
Unrolling lays the windings end to end, outermost first. Each traced winding runs a little past one full turn, so roughly a tenth of the 2.8 m strip repeats surface from its neighbour.
|
| 32 |
|
|
|
|
| 57 |
## Ink Bench
|
| 58 |
|
| 59 |
Score your own ink prediction for any winding against the published ink, in the browser (nothing is uploaded): automatic UV alignment, side by side, swipe and difference views, Pearson r, IoU, Dice and a precision and recall curve (agreement with another model's output, not ground truth), letter boxes marked "candidate for expert review", and a one-click reproducible bundle. `tools/inkbench.py` gives identical results in Python and on the command line. See [docs/BENCH.md](docs/BENCH.md) and the open bundle format [docs/BENCH_FORMAT.md](docs/BENCH_FORMAT.md). The Bench code and format are Apache-2.0 (`LICENSE-APACHE`, `NOTICE`, by Chase Hendrick); the data stays CC BY-NC 4.0.
|
| 60 |
+
|
| 61 |
+
### Researcher takeaway
|
| 62 |
+
|
| 63 |
+
On the public ink_9um run over all 37 windings (`docs/examples/bench/`), against the published 2.4 µm ink: median AP 0.319 for ink_9um, 0.117 for raw X-ray intensity and 0.089 for uniform noise, and ink_9um beats X-ray intensity on AP on every winding. On Pearson r the medians are 0.422, 0.266 and 0.000, but X-ray intensity outranks ink_9um on `w039`, `w053`, `w054` and `w057`. So: rank ink models by AP (threshold-free and robust to brightness), always report the X-ray floor next to your model, and look at windings where the floor wins before trusting a correlation. These are agreement scores with another model's output, not ground truth.
|
data/bench/index.json
CHANGED
|
@@ -17,6 +17,10 @@
|
|
| 17 |
"ct": {
|
| 18 |
"file": "data/bench/w023_ct.png",
|
| 19 |
"sha256": "9e871edfb98d9c5455e3e8b6dae5bc2127b4b6c0ab2399ea5f80c9b9a38b1d2b"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
}
|
| 21 |
}
|
| 22 |
},
|
|
@@ -34,6 +38,10 @@
|
|
| 34 |
"ct": {
|
| 35 |
"file": "data/bench/w024_ct.png",
|
| 36 |
"sha256": "7f21087fc1b5a1a5812e02d24134004524373408d5fe533939f12003f90c6517"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
}
|
| 38 |
}
|
| 39 |
},
|
|
@@ -51,6 +59,10 @@
|
|
| 51 |
"ct": {
|
| 52 |
"file": "data/bench/w025_ct.png",
|
| 53 |
"sha256": "e30f050d29ccc83eaf9b29beafe66f0a47a35ac2a7f30dbd525f462c9ea325fa"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
}
|
| 55 |
}
|
| 56 |
},
|
|
@@ -68,6 +80,10 @@
|
|
| 68 |
"ct": {
|
| 69 |
"file": "data/bench/w026_ct.png",
|
| 70 |
"sha256": "d005b08b88e31ad0cc24bfbbff91041e8abad6275d39cbb243dddfe78b46f3f0"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
}
|
| 72 |
}
|
| 73 |
},
|
|
@@ -85,6 +101,10 @@
|
|
| 85 |
"ct": {
|
| 86 |
"file": "data/bench/w027_ct.png",
|
| 87 |
"sha256": "bf5b5d9a5d8574c03dba35ac6b502319aa6ebee01032cff01cc5f7d600ea22b6"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
}
|
| 89 |
}
|
| 90 |
},
|
|
@@ -102,6 +122,10 @@
|
|
| 102 |
"ct": {
|
| 103 |
"file": "data/bench/w028_ct.png",
|
| 104 |
"sha256": "02e88630f83b2fea27ec4473e8bf04d43a5c771d8b62e8900fd8f56dc4cc3004"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
}
|
| 106 |
}
|
| 107 |
},
|
|
@@ -119,6 +143,10 @@
|
|
| 119 |
"ct": {
|
| 120 |
"file": "data/bench/w029_ct.png",
|
| 121 |
"sha256": "2217cff5ce56140fec1f2f34a055b94ac04a9071d42f20f7d3e9887db23e9025"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 122 |
}
|
| 123 |
}
|
| 124 |
},
|
|
@@ -136,6 +164,10 @@
|
|
| 136 |
"ct": {
|
| 137 |
"file": "data/bench/w030_ct.png",
|
| 138 |
"sha256": "1e739695dddf66e512678c896f56e9fbb5dbb7ac80248db08a5432892f253cd6"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
}
|
| 140 |
}
|
| 141 |
},
|
|
@@ -153,6 +185,10 @@
|
|
| 153 |
"ct": {
|
| 154 |
"file": "data/bench/w031_ct.png",
|
| 155 |
"sha256": "371e1428275d44a00eb5bddf4f87ce93ddb6803cc134537e3304823ee2412be3"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 156 |
}
|
| 157 |
}
|
| 158 |
},
|
|
@@ -170,6 +206,10 @@
|
|
| 170 |
"ct": {
|
| 171 |
"file": "data/bench/w032_ct.png",
|
| 172 |
"sha256": "83a374f9ad2d93d0fcb30d1fcc6e632830fbd9b6d2bb6aa920c8ae2562674fd1"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 173 |
}
|
| 174 |
}
|
| 175 |
},
|
|
@@ -187,6 +227,10 @@
|
|
| 187 |
"ct": {
|
| 188 |
"file": "data/bench/w033_ct.png",
|
| 189 |
"sha256": "3ccb04488d2c8f5e8eea00f6c690508d5eb49c3c75ba8b1dfada08fb6038139c"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 190 |
}
|
| 191 |
}
|
| 192 |
},
|
|
@@ -204,6 +248,10 @@
|
|
| 204 |
"ct": {
|
| 205 |
"file": "data/bench/w034_ct.png",
|
| 206 |
"sha256": "0b0726cbc9d4fbf6a3851b580b4711606a59c7d221efbefebfe0af350cf74a5d"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 207 |
}
|
| 208 |
}
|
| 209 |
},
|
|
@@ -221,6 +269,10 @@
|
|
| 221 |
"ct": {
|
| 222 |
"file": "data/bench/w035_ct.png",
|
| 223 |
"sha256": "65b1c06363cf835d4b008c7155d3e4140c47489c9d1c905ad20baebeb5a18987"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 224 |
}
|
| 225 |
}
|
| 226 |
},
|
|
@@ -238,6 +290,10 @@
|
|
| 238 |
"ct": {
|
| 239 |
"file": "data/bench/w036_ct.png",
|
| 240 |
"sha256": "fe43a9e3eb5bed08041cc55a43134b63fb23a85d543f4e89d838350c7de55f0d"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 241 |
}
|
| 242 |
}
|
| 243 |
},
|
|
@@ -255,6 +311,10 @@
|
|
| 255 |
"ct": {
|
| 256 |
"file": "data/bench/w037_ct.png",
|
| 257 |
"sha256": "2656acecedda3f2647058e84185a9d9a5f691272ed0fa99e1c007ab2590a6733"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 258 |
}
|
| 259 |
}
|
| 260 |
},
|
|
@@ -272,6 +332,10 @@
|
|
| 272 |
"ct": {
|
| 273 |
"file": "data/bench/w038_ct.png",
|
| 274 |
"sha256": "3a7dc45d62f9da8ab65a46e3f388813749b058d1e93dfc6bed78d70cde7a76ee"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 275 |
}
|
| 276 |
}
|
| 277 |
},
|
|
@@ -289,6 +353,10 @@
|
|
| 289 |
"ct": {
|
| 290 |
"file": "data/bench/w039_ct.png",
|
| 291 |
"sha256": "927048bd37ec6e3838b6e957fc0dfc4c569f096688b088d48fca0a0dda8c2dbc"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 292 |
}
|
| 293 |
}
|
| 294 |
},
|
|
@@ -306,6 +374,10 @@
|
|
| 306 |
"ct": {
|
| 307 |
"file": "data/bench/w040_ct.png",
|
| 308 |
"sha256": "df430900b97f8459ea94ad284960bf61051731aca00b48db73ca07264d216e76"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 309 |
}
|
| 310 |
}
|
| 311 |
},
|
|
@@ -323,6 +395,10 @@
|
|
| 323 |
"ct": {
|
| 324 |
"file": "data/bench/w041_ct.png",
|
| 325 |
"sha256": "b4e58718a814135c14e2f7d6a2c556c259e9eaac0f72c4eb0abe52b8a288940f"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 326 |
}
|
| 327 |
}
|
| 328 |
},
|
|
@@ -340,6 +416,10 @@
|
|
| 340 |
"ct": {
|
| 341 |
"file": "data/bench/w042_ct.png",
|
| 342 |
"sha256": "0231a5f2b89abba26b3983869727145b140a3d3f0f04b86911c736249d0fb097"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 343 |
}
|
| 344 |
}
|
| 345 |
},
|
|
@@ -357,6 +437,10 @@
|
|
| 357 |
"ct": {
|
| 358 |
"file": "data/bench/w043_ct.png",
|
| 359 |
"sha256": "8fe83037cdab7f905bcd55d583950e71956d8729576151cd6e93b9b5610a8a36"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 360 |
}
|
| 361 |
}
|
| 362 |
},
|
|
@@ -374,6 +458,10 @@
|
|
| 374 |
"ct": {
|
| 375 |
"file": "data/bench/w044_ct.png",
|
| 376 |
"sha256": "b537b3e4505f07bd871dfc6a680471ba88de93c354e0ef8c6d825c009039290c"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 377 |
}
|
| 378 |
}
|
| 379 |
},
|
|
@@ -391,6 +479,10 @@
|
|
| 391 |
"ct": {
|
| 392 |
"file": "data/bench/w045_ct.png",
|
| 393 |
"sha256": "0d991e625d4abb1ffce31f904a3e47ed012cdfe291f02ceecd5fdf733fc4662b"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 394 |
}
|
| 395 |
}
|
| 396 |
},
|
|
@@ -534,6 +626,10 @@
|
|
| 534 |
"ct": {
|
| 535 |
"file": "data/bench/w052_ct.png",
|
| 536 |
"sha256": "29b475c37e05f7a8078e2eadedbbf1863899ca48e39caec2e7fb76d91a508154"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 537 |
}
|
| 538 |
}
|
| 539 |
},
|
|
@@ -551,6 +647,10 @@
|
|
| 551 |
"ct": {
|
| 552 |
"file": "data/bench/w053_ct.png",
|
| 553 |
"sha256": "bb50da6580261cef973e4276ec7936d114d55845c8efac9088ea2c5a91f76e5d"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 554 |
}
|
| 555 |
}
|
| 556 |
},
|
|
@@ -568,6 +668,10 @@
|
|
| 568 |
"ct": {
|
| 569 |
"file": "data/bench/w054_ct.png",
|
| 570 |
"sha256": "a6c5afcbeb31180ebfc9f286802fd07f01ae517a3db502e3972d73b68d6dd0b6"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 571 |
}
|
| 572 |
}
|
| 573 |
},
|
|
@@ -585,6 +689,10 @@
|
|
| 585 |
"ct": {
|
| 586 |
"file": "data/bench/w055_ct.png",
|
| 587 |
"sha256": "dab0bba53acdd7d658ef4a2bf147d8bcf81016faae4d96402e1b1593a2aea178"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 588 |
}
|
| 589 |
}
|
| 590 |
},
|
|
@@ -602,6 +710,10 @@
|
|
| 602 |
"ct": {
|
| 603 |
"file": "data/bench/w056_ct.png",
|
| 604 |
"sha256": "156992b325cb9b192ed76c07a80d904b34b4a49139b20e5e81b662b2387c3798"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 605 |
}
|
| 606 |
}
|
| 607 |
},
|
|
@@ -619,6 +731,10 @@
|
|
| 619 |
"ct": {
|
| 620 |
"file": "data/bench/w057_ct.png",
|
| 621 |
"sha256": "14ef32a30434a86f24c3395c29d496d3602b336756304f5e32bc9c18148e1ac2"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 622 |
}
|
| 623 |
}
|
| 624 |
},
|
|
@@ -636,6 +752,10 @@
|
|
| 636 |
"ct": {
|
| 637 |
"file": "data/bench/w058_ct.png",
|
| 638 |
"sha256": "9a2d6eb4f994a44a02ccb7fe4b0134768b5b636da2faf21f6a1234ac24e79848"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 639 |
}
|
| 640 |
}
|
| 641 |
},
|
|
@@ -653,6 +773,10 @@
|
|
| 653 |
"ct": {
|
| 654 |
"file": "data/bench/w059_ct.png",
|
| 655 |
"sha256": "fead2cd274c88bd50672df8d0537bc3eca86c27387b0ceb21312864c4d1622b7"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 656 |
}
|
| 657 |
}
|
| 658 |
}
|
|
|
|
| 17 |
"ct": {
|
| 18 |
"file": "data/bench/w023_ct.png",
|
| 19 |
"sha256": "9e871edfb98d9c5455e3e8b6dae5bc2127b4b6c0ab2399ea5f80c9b9a38b1d2b"
|
| 20 |
+
},
|
| 21 |
+
"gpu": {
|
| 22 |
+
"file": "data/bench/w023_gpu.png",
|
| 23 |
+
"sha256": "14bedeb6a276f50d4783caec9774ccc4cf5899ddd67a0d57bf75dc6d04372d0c"
|
| 24 |
}
|
| 25 |
}
|
| 26 |
},
|
|
|
|
| 38 |
"ct": {
|
| 39 |
"file": "data/bench/w024_ct.png",
|
| 40 |
"sha256": "7f21087fc1b5a1a5812e02d24134004524373408d5fe533939f12003f90c6517"
|
| 41 |
+
},
|
| 42 |
+
"gpu": {
|
| 43 |
+
"file": "data/bench/w024_gpu.png",
|
| 44 |
+
"sha256": "cc11fb74a30bc3fb48bd652946e3e9124d1a2d90947f613f715da37671a85efd"
|
| 45 |
}
|
| 46 |
}
|
| 47 |
},
|
|
|
|
| 59 |
"ct": {
|
| 60 |
"file": "data/bench/w025_ct.png",
|
| 61 |
"sha256": "e30f050d29ccc83eaf9b29beafe66f0a47a35ac2a7f30dbd525f462c9ea325fa"
|
| 62 |
+
},
|
| 63 |
+
"gpu": {
|
| 64 |
+
"file": "data/bench/w025_gpu.png",
|
| 65 |
+
"sha256": "c129d596d1fe215b2a5c671eff5143e938bd4c4d34ac1a6c7b9843d1f263f226"
|
| 66 |
}
|
| 67 |
}
|
| 68 |
},
|
|
|
|
| 80 |
"ct": {
|
| 81 |
"file": "data/bench/w026_ct.png",
|
| 82 |
"sha256": "d005b08b88e31ad0cc24bfbbff91041e8abad6275d39cbb243dddfe78b46f3f0"
|
| 83 |
+
},
|
| 84 |
+
"gpu": {
|
| 85 |
+
"file": "data/bench/w026_gpu.png",
|
| 86 |
+
"sha256": "262b1fb5329f95649d5d69da657ac3b111b9ba05c2151859c167efd85c700079"
|
| 87 |
}
|
| 88 |
}
|
| 89 |
},
|
|
|
|
| 101 |
"ct": {
|
| 102 |
"file": "data/bench/w027_ct.png",
|
| 103 |
"sha256": "bf5b5d9a5d8574c03dba35ac6b502319aa6ebee01032cff01cc5f7d600ea22b6"
|
| 104 |
+
},
|
| 105 |
+
"gpu": {
|
| 106 |
+
"file": "data/bench/w027_gpu.png",
|
| 107 |
+
"sha256": "89395044a5bf932b7adaeb8522e2d36c87e684f852b6c8ba2e0d30509376e650"
|
| 108 |
}
|
| 109 |
}
|
| 110 |
},
|
|
|
|
| 122 |
"ct": {
|
| 123 |
"file": "data/bench/w028_ct.png",
|
| 124 |
"sha256": "02e88630f83b2fea27ec4473e8bf04d43a5c771d8b62e8900fd8f56dc4cc3004"
|
| 125 |
+
},
|
| 126 |
+
"gpu": {
|
| 127 |
+
"file": "data/bench/w028_gpu.png",
|
| 128 |
+
"sha256": "f5b8b736eafc7bd21fd847360d4b9f676d5d94152eb3a5b0b551b5dda437d48e"
|
| 129 |
}
|
| 130 |
}
|
| 131 |
},
|
|
|
|
| 143 |
"ct": {
|
| 144 |
"file": "data/bench/w029_ct.png",
|
| 145 |
"sha256": "2217cff5ce56140fec1f2f34a055b94ac04a9071d42f20f7d3e9887db23e9025"
|
| 146 |
+
},
|
| 147 |
+
"gpu": {
|
| 148 |
+
"file": "data/bench/w029_gpu.png",
|
| 149 |
+
"sha256": "2c3c563b8d544fd58025674d718c5aa3393127c710bbe8eb636b699bea6226bb"
|
| 150 |
}
|
| 151 |
}
|
| 152 |
},
|
|
|
|
| 164 |
"ct": {
|
| 165 |
"file": "data/bench/w030_ct.png",
|
| 166 |
"sha256": "1e739695dddf66e512678c896f56e9fbb5dbb7ac80248db08a5432892f253cd6"
|
| 167 |
+
},
|
| 168 |
+
"gpu": {
|
| 169 |
+
"file": "data/bench/w030_gpu.png",
|
| 170 |
+
"sha256": "d0bfda06404540de9f895716b7cecbb7b87d8281625bdc77b0c21f0e58bab99b"
|
| 171 |
}
|
| 172 |
}
|
| 173 |
},
|
|
|
|
| 185 |
"ct": {
|
| 186 |
"file": "data/bench/w031_ct.png",
|
| 187 |
"sha256": "371e1428275d44a00eb5bddf4f87ce93ddb6803cc134537e3304823ee2412be3"
|
| 188 |
+
},
|
| 189 |
+
"gpu": {
|
| 190 |
+
"file": "data/bench/w031_gpu.png",
|
| 191 |
+
"sha256": "93fc5e9939c05dadc090dd0bb77090fe4016b9d3b45cc5bb0469beaa746a208c"
|
| 192 |
}
|
| 193 |
}
|
| 194 |
},
|
|
|
|
| 206 |
"ct": {
|
| 207 |
"file": "data/bench/w032_ct.png",
|
| 208 |
"sha256": "83a374f9ad2d93d0fcb30d1fcc6e632830fbd9b6d2bb6aa920c8ae2562674fd1"
|
| 209 |
+
},
|
| 210 |
+
"gpu": {
|
| 211 |
+
"file": "data/bench/w032_gpu.png",
|
| 212 |
+
"sha256": "ec48088615f9f7a3f80bfe61d130362a50d59d7de3d9bf2565f21f12212efbc5"
|
| 213 |
}
|
| 214 |
}
|
| 215 |
},
|
|
|
|
| 227 |
"ct": {
|
| 228 |
"file": "data/bench/w033_ct.png",
|
| 229 |
"sha256": "3ccb04488d2c8f5e8eea00f6c690508d5eb49c3c75ba8b1dfada08fb6038139c"
|
| 230 |
+
},
|
| 231 |
+
"gpu": {
|
| 232 |
+
"file": "data/bench/w033_gpu.png",
|
| 233 |
+
"sha256": "da90168078510f68b84b28617e270f0a33a85d374b9b0db0c565de2365511b1c"
|
| 234 |
}
|
| 235 |
}
|
| 236 |
},
|
|
|
|
| 248 |
"ct": {
|
| 249 |
"file": "data/bench/w034_ct.png",
|
| 250 |
"sha256": "0b0726cbc9d4fbf6a3851b580b4711606a59c7d221efbefebfe0af350cf74a5d"
|
| 251 |
+
},
|
| 252 |
+
"gpu": {
|
| 253 |
+
"file": "data/bench/w034_gpu.png",
|
| 254 |
+
"sha256": "015c6a53edb0e9c97c6bacd0f41a38b3ce42af3e1031a7debbdece3771e83dd8"
|
| 255 |
}
|
| 256 |
}
|
| 257 |
},
|
|
|
|
| 269 |
"ct": {
|
| 270 |
"file": "data/bench/w035_ct.png",
|
| 271 |
"sha256": "65b1c06363cf835d4b008c7155d3e4140c47489c9d1c905ad20baebeb5a18987"
|
| 272 |
+
},
|
| 273 |
+
"gpu": {
|
| 274 |
+
"file": "data/bench/w035_gpu.png",
|
| 275 |
+
"sha256": "64fe35009ee7787f8d3afa7b41e2a62936fa8a1a00b80376d5013ad7229175da"
|
| 276 |
}
|
| 277 |
}
|
| 278 |
},
|
|
|
|
| 290 |
"ct": {
|
| 291 |
"file": "data/bench/w036_ct.png",
|
| 292 |
"sha256": "fe43a9e3eb5bed08041cc55a43134b63fb23a85d543f4e89d838350c7de55f0d"
|
| 293 |
+
},
|
| 294 |
+
"gpu": {
|
| 295 |
+
"file": "data/bench/w036_gpu.png",
|
| 296 |
+
"sha256": "f326573cb16b362511bbbd8e11d51d9cae638c572134f5bf9acd83f983a7799e"
|
| 297 |
}
|
| 298 |
}
|
| 299 |
},
|
|
|
|
| 311 |
"ct": {
|
| 312 |
"file": "data/bench/w037_ct.png",
|
| 313 |
"sha256": "2656acecedda3f2647058e84185a9d9a5f691272ed0fa99e1c007ab2590a6733"
|
| 314 |
+
},
|
| 315 |
+
"gpu": {
|
| 316 |
+
"file": "data/bench/w037_gpu.png",
|
| 317 |
+
"sha256": "4b1f3efc88ebc02f44ade6d426371c7ceaa0f45a12d68828cae8a2a0948bcb33"
|
| 318 |
}
|
| 319 |
}
|
| 320 |
},
|
|
|
|
| 332 |
"ct": {
|
| 333 |
"file": "data/bench/w038_ct.png",
|
| 334 |
"sha256": "3a7dc45d62f9da8ab65a46e3f388813749b058d1e93dfc6bed78d70cde7a76ee"
|
| 335 |
+
},
|
| 336 |
+
"gpu": {
|
| 337 |
+
"file": "data/bench/w038_gpu.png",
|
| 338 |
+
"sha256": "a9e4b402fa9bc449ef35f0eb3edd3758ad1afafe4ac0178d05cd4245eaa74201"
|
| 339 |
}
|
| 340 |
}
|
| 341 |
},
|
|
|
|
| 353 |
"ct": {
|
| 354 |
"file": "data/bench/w039_ct.png",
|
| 355 |
"sha256": "927048bd37ec6e3838b6e957fc0dfc4c569f096688b088d48fca0a0dda8c2dbc"
|
| 356 |
+
},
|
| 357 |
+
"gpu": {
|
| 358 |
+
"file": "data/bench/w039_gpu.png",
|
| 359 |
+
"sha256": "373ddb854a2af7236703c3e4ebb58b148c335230fed2bb3e1498e8136c2c2870"
|
| 360 |
}
|
| 361 |
}
|
| 362 |
},
|
|
|
|
| 374 |
"ct": {
|
| 375 |
"file": "data/bench/w040_ct.png",
|
| 376 |
"sha256": "df430900b97f8459ea94ad284960bf61051731aca00b48db73ca07264d216e76"
|
| 377 |
+
},
|
| 378 |
+
"gpu": {
|
| 379 |
+
"file": "data/bench/w040_gpu.png",
|
| 380 |
+
"sha256": "5114fe17084c037b12c0dd883d93e65b287bdabce02e19d22fc1bfa0da0fd518"
|
| 381 |
}
|
| 382 |
}
|
| 383 |
},
|
|
|
|
| 395 |
"ct": {
|
| 396 |
"file": "data/bench/w041_ct.png",
|
| 397 |
"sha256": "b4e58718a814135c14e2f7d6a2c556c259e9eaac0f72c4eb0abe52b8a288940f"
|
| 398 |
+
},
|
| 399 |
+
"gpu": {
|
| 400 |
+
"file": "data/bench/w041_gpu.png",
|
| 401 |
+
"sha256": "d57c948b18da6c3c15569b858ad0ee34af03c97bb25bf51d2959e123115127c5"
|
| 402 |
}
|
| 403 |
}
|
| 404 |
},
|
|
|
|
| 416 |
"ct": {
|
| 417 |
"file": "data/bench/w042_ct.png",
|
| 418 |
"sha256": "0231a5f2b89abba26b3983869727145b140a3d3f0f04b86911c736249d0fb097"
|
| 419 |
+
},
|
| 420 |
+
"gpu": {
|
| 421 |
+
"file": "data/bench/w042_gpu.png",
|
| 422 |
+
"sha256": "726d4bc78e3256dc7076c9eae37d342f3253a436ee49a0a8993372f6c9746d3b"
|
| 423 |
}
|
| 424 |
}
|
| 425 |
},
|
|
|
|
| 437 |
"ct": {
|
| 438 |
"file": "data/bench/w043_ct.png",
|
| 439 |
"sha256": "8fe83037cdab7f905bcd55d583950e71956d8729576151cd6e93b9b5610a8a36"
|
| 440 |
+
},
|
| 441 |
+
"gpu": {
|
| 442 |
+
"file": "data/bench/w043_gpu.png",
|
| 443 |
+
"sha256": "b92f6007d57a05b7c08958c60b5070e70f895a445a12f740f2ec8165d4be42ea"
|
| 444 |
}
|
| 445 |
}
|
| 446 |
},
|
|
|
|
| 458 |
"ct": {
|
| 459 |
"file": "data/bench/w044_ct.png",
|
| 460 |
"sha256": "b537b3e4505f07bd871dfc6a680471ba88de93c354e0ef8c6d825c009039290c"
|
| 461 |
+
},
|
| 462 |
+
"gpu": {
|
| 463 |
+
"file": "data/bench/w044_gpu.png",
|
| 464 |
+
"sha256": "b5148db74ef0a5a4c705b7a6928656e36cbfaee0af8f42ed84bedb8df85199ae"
|
| 465 |
}
|
| 466 |
}
|
| 467 |
},
|
|
|
|
| 479 |
"ct": {
|
| 480 |
"file": "data/bench/w045_ct.png",
|
| 481 |
"sha256": "0d991e625d4abb1ffce31f904a3e47ed012cdfe291f02ceecd5fdf733fc4662b"
|
| 482 |
+
},
|
| 483 |
+
"gpu": {
|
| 484 |
+
"file": "data/bench/w045_gpu.png",
|
| 485 |
+
"sha256": "fe8cd07604368288da0eb2d4da667ca69d0349b7646814a6db77438d9d0c7bf9"
|
| 486 |
}
|
| 487 |
}
|
| 488 |
},
|
|
|
|
| 626 |
"ct": {
|
| 627 |
"file": "data/bench/w052_ct.png",
|
| 628 |
"sha256": "29b475c37e05f7a8078e2eadedbbf1863899ca48e39caec2e7fb76d91a508154"
|
| 629 |
+
},
|
| 630 |
+
"gpu": {
|
| 631 |
+
"file": "data/bench/w052_gpu.png",
|
| 632 |
+
"sha256": "c79565130088f5a8d0dd423ab64bebcf604e871a7a0960632578227e21f72278"
|
| 633 |
}
|
| 634 |
}
|
| 635 |
},
|
|
|
|
| 647 |
"ct": {
|
| 648 |
"file": "data/bench/w053_ct.png",
|
| 649 |
"sha256": "bb50da6580261cef973e4276ec7936d114d55845c8efac9088ea2c5a91f76e5d"
|
| 650 |
+
},
|
| 651 |
+
"gpu": {
|
| 652 |
+
"file": "data/bench/w053_gpu.png",
|
| 653 |
+
"sha256": "09c8e76abc3c5f1d4d979709bee06692946fd8b9b1054d262234ab41717ad319"
|
| 654 |
}
|
| 655 |
}
|
| 656 |
},
|
|
|
|
| 668 |
"ct": {
|
| 669 |
"file": "data/bench/w054_ct.png",
|
| 670 |
"sha256": "a6c5afcbeb31180ebfc9f286802fd07f01ae517a3db502e3972d73b68d6dd0b6"
|
| 671 |
+
},
|
| 672 |
+
"gpu": {
|
| 673 |
+
"file": "data/bench/w054_gpu.png",
|
| 674 |
+
"sha256": "c3b3cf2331601645db43d56b402d49285a6ded4984bfcbcbc06582780dc05a80"
|
| 675 |
}
|
| 676 |
}
|
| 677 |
},
|
|
|
|
| 689 |
"ct": {
|
| 690 |
"file": "data/bench/w055_ct.png",
|
| 691 |
"sha256": "dab0bba53acdd7d658ef4a2bf147d8bcf81016faae4d96402e1b1593a2aea178"
|
| 692 |
+
},
|
| 693 |
+
"gpu": {
|
| 694 |
+
"file": "data/bench/w055_gpu.png",
|
| 695 |
+
"sha256": "dc8f4c5ba397db806b852cb0075a23c008df3bdd2c8b554eee477d2c9dbaf86a"
|
| 696 |
}
|
| 697 |
}
|
| 698 |
},
|
|
|
|
| 710 |
"ct": {
|
| 711 |
"file": "data/bench/w056_ct.png",
|
| 712 |
"sha256": "156992b325cb9b192ed76c07a80d904b34b4a49139b20e5e81b662b2387c3798"
|
| 713 |
+
},
|
| 714 |
+
"gpu": {
|
| 715 |
+
"file": "data/bench/w056_gpu.png",
|
| 716 |
+
"sha256": "c2a59ccce65176ac716e507dd584b59128174330f1df1f285013921c87142539"
|
| 717 |
}
|
| 718 |
}
|
| 719 |
},
|
|
|
|
| 731 |
"ct": {
|
| 732 |
"file": "data/bench/w057_ct.png",
|
| 733 |
"sha256": "14ef32a30434a86f24c3395c29d496d3602b336756304f5e32bc9c18148e1ac2"
|
| 734 |
+
},
|
| 735 |
+
"gpu": {
|
| 736 |
+
"file": "data/bench/w057_gpu.png",
|
| 737 |
+
"sha256": "0baae2f99ec2b4d643ddcb471b2605e9543a2e95e9dd37772c18f4abf513c263"
|
| 738 |
}
|
| 739 |
}
|
| 740 |
},
|
|
|
|
| 752 |
"ct": {
|
| 753 |
"file": "data/bench/w058_ct.png",
|
| 754 |
"sha256": "9a2d6eb4f994a44a02ccb7fe4b0134768b5b636da2faf21f6a1234ac24e79848"
|
| 755 |
+
},
|
| 756 |
+
"gpu": {
|
| 757 |
+
"file": "data/bench/w058_gpu.png",
|
| 758 |
+
"sha256": "c1547ad9c0c3909c8d8fc1a25150f3133235b84ee065cb55efe860719bc14164"
|
| 759 |
}
|
| 760 |
}
|
| 761 |
},
|
|
|
|
| 773 |
"ct": {
|
| 774 |
"file": "data/bench/w059_ct.png",
|
| 775 |
"sha256": "fead2cd274c88bd50672df8d0537bc3eca86c27387b0ceb21312864c4d1622b7"
|
| 776 |
+
},
|
| 777 |
+
"gpu": {
|
| 778 |
+
"file": "data/bench/w059_gpu.png",
|
| 779 |
+
"sha256": "f70d59d0ffde0448a8b1905418f972e0850b61416f105d7003d987bd64c6165d"
|
| 780 |
}
|
| 781 |
}
|
| 782 |
}
|
data/bench/w023_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w024_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w025_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w026_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w027_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w028_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w029_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w030_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w031_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w032_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w033_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w034_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w035_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w036_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w037_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w038_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w039_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w040_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w041_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w042_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w043_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w044_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w045_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w052_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w053_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w054_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w055_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w056_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w057_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w058_gpu.png
ADDED
|
Git LFS Details
|
data/bench/w059_gpu.png
ADDED
|
Git LFS Details
|
docs/BENCH.md
CHANGED
|
@@ -16,9 +16,14 @@ Open it from the bar chart button in the [Unroll Space](https://huggingface.co/s
|
|
| 16 |
|
| 17 |
> Every score is **agreement with another model's output**, the Vesuvius Challenge published ink detection on the 2.4 µm scan. It is not ground truth. A model can score lower because it is different, or even better. Use the numbers to find where detectors agree and disagree, then send the interesting regions to people who read Greek papyri.
|
| 18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
## Reference grid
|
| 20 |
|
| 21 |
-
Each winding has a reference grid of 640 px on its long side (`data/bench/<id>_ink.png`, `<id>_ct.png`, and `<id>_gpu.png`
|
| 22 |
|
| 23 |
## Python and CLI
|
| 24 |
|
|
@@ -88,7 +93,7 @@ B.core //
|
|
| 88 |
|
| 89 |
## Leaderboard
|
| 90 |
|
| 91 |
-
The Leaderboard tab loads any number of bundles from URLs (one per line, CORS) or files and ranks them per winding by Pearson r, AP, best F1, IoU or Dice, entirely in the browser. `?board=url1,url2` opens it preloaded. "Load examples" shows the public ink_9um
|
| 92 |
|
| 93 |
## How it is tested
|
| 94 |
|
|
|
|
| 16 |
|
| 17 |
> Every score is **agreement with another model's output**, the Vesuvius Challenge published ink detection on the 2.4 µm scan. It is not ground truth. A model can score lower because it is different, or even better. Use the numbers to find where detectors agree and disagree, then send the interesting regions to people who read Greek papyri.
|
| 18 |
|
| 19 |
+
## Researcher takeaway
|
| 20 |
+
On the public ink_9um run over all 37 windings (`docs/examples/bench/`), against the published 2.4 µm ink: median AP 0.319 for ink_9um, 0.117 for raw X-ray intensity and 0.089 for uniform noise, and ink_9um beats X-ray intensity on AP on every winding. On Pearson r the medians are 0.422, 0.266 and 0.000, but X-ray intensity outranks ink_9um on `w039`, `w053`, `w054` and `w057`. So: rank ink models by AP (threshold-free and robust to brightness), always report the X-ray floor next to your model, and look at windings where the floor wins before trusting a correlation. These are agreement scores with another model's output, not ground truth.
|
| 21 |
+
|
| 22 |
+
The ink_9um model was trained by the Vesuvius Challenge and run by Chase Hendrick.
|
| 23 |
+
|
| 24 |
## Reference grid
|
| 25 |
|
| 26 |
+
Each winding has a reference grid of 640 px on its long side (`data/bench/<id>_ink.png`, `<id>_ct.png`, and `<id>_gpu.png`, the public ink_9um run), lossless 8 bit PNG, with checksums and pixels per mm in `data/bench/index.json` (about 8.3 px per mm, so 1 px is about 120 µm on the papyrus). They are built by `tools/make_bench_refs.py` from the page's own layers. All metrics are computed on this grid, after your prediction is resampled onto it (bilinear, 2x2 mean pooled first while it is more than twice the grid).
|
| 27 |
|
| 28 |
## Python and CLI
|
| 29 |
|
|
|
|
| 93 |
|
| 94 |
## Leaderboard
|
| 95 |
|
| 96 |
+
The Leaderboard tab loads any number of bundles from URLs (one per line, CORS) or files and ranks them per winding by Pearson r, AP, best F1, IoU or Dice, entirely in the browser. `?board=url1,url2` opens it preloaded. "Load examples" shows the public ink_9um run on all 37 windings next to two floors any ink model should beat: the X-ray intensity itself and uniform noise.
|
| 97 |
|
| 98 |
## How it is tested
|
| 99 |
|
docs/examples/bench/w023-ct-baseline.json
ADDED
|
@@ -0,0 +1,334 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:47:44Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w023",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w023_ink.png",
|
| 15 |
+
"sha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d",
|
| 16 |
+
"width": 589,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.861,
|
| 19 |
+
"alignTargetFile": "data/bench/w023_ink.png",
|
| 20 |
+
"alignTargetSha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "w023_ct.png",
|
| 24 |
+
"sha256": "9e871edfb98d9c5455e3e8b6dae5bc2127b4b6c0ab2399ea5f80c9b9a38b1d2b",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 589,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "baseline: X-ray intensity (same grid, no alignment needed)"
|
| 32 |
+
}
|
| 33 |
+
},
|
| 34 |
+
"alignment": {
|
| 35 |
+
"method": "manual",
|
| 36 |
+
"target": "ink",
|
| 37 |
+
"tx": 0.0,
|
| 38 |
+
"ty": 0.0,
|
| 39 |
+
"sx": 1.0,
|
| 40 |
+
"sy": 1.0,
|
| 41 |
+
"ncc": 0.342535,
|
| 42 |
+
"coarseNcc": null,
|
| 43 |
+
"evals": 0,
|
| 44 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 45 |
+
},
|
| 46 |
+
"metrics": {
|
| 47 |
+
"nValid": 376960,
|
| 48 |
+
"validFraction": 1.0,
|
| 49 |
+
"pearson": 0.342535,
|
| 50 |
+
"blurSigmaPx": 6.0,
|
| 51 |
+
"blurSigmaUM": 608.5,
|
| 52 |
+
"pearsonMatchedBlur": 0.464838,
|
| 53 |
+
"thresholds": {
|
| 54 |
+
"ref": 0.5,
|
| 55 |
+
"pred": 0.5
|
| 56 |
+
},
|
| 57 |
+
"iou": 0.267945,
|
| 58 |
+
"dice": 0.422644,
|
| 59 |
+
"precision": 0.366091,
|
| 60 |
+
"recall": 0.499862,
|
| 61 |
+
"refPositiveFraction": 0.297854,
|
| 62 |
+
"predPositiveFraction": 0.40669,
|
| 63 |
+
"ap": 0.367097,
|
| 64 |
+
"bestF1": {
|
| 65 |
+
"f1": 0.528033,
|
| 66 |
+
"threshold": 0.04
|
| 67 |
+
},
|
| 68 |
+
"pr": [
|
| 69 |
+
{
|
| 70 |
+
"t": 1.0,
|
| 71 |
+
"precision": 0.413174,
|
| 72 |
+
"recall": 0.000615
|
| 73 |
+
},
|
| 74 |
+
{
|
| 75 |
+
"t": 0.98,
|
| 76 |
+
"precision": 0.419048,
|
| 77 |
+
"recall": 0.001568
|
| 78 |
+
},
|
| 79 |
+
{
|
| 80 |
+
"t": 0.96,
|
| 81 |
+
"precision": 0.393643,
|
| 82 |
+
"recall": 0.002868
|
| 83 |
+
},
|
| 84 |
+
{
|
| 85 |
+
"t": 0.94,
|
| 86 |
+
"precision": 0.371141,
|
| 87 |
+
"recall": 0.004605
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"t": 0.92,
|
| 91 |
+
"precision": 0.381218,
|
| 92 |
+
"recall": 0.007303
|
| 93 |
+
},
|
| 94 |
+
{
|
| 95 |
+
"t": 0.9,
|
| 96 |
+
"precision": 0.379342,
|
| 97 |
+
"recall": 0.011186
|
| 98 |
+
},
|
| 99 |
+
{
|
| 100 |
+
"t": 0.88,
|
| 101 |
+
"precision": 0.369996,
|
| 102 |
+
"recall": 0.015969
|
| 103 |
+
},
|
| 104 |
+
{
|
| 105 |
+
"t": 0.86,
|
| 106 |
+
"precision": 0.372707,
|
| 107 |
+
"recall": 0.022622
|
| 108 |
+
},
|
| 109 |
+
{
|
| 110 |
+
"t": 0.84,
|
| 111 |
+
"precision": 0.374667,
|
| 112 |
+
"recall": 0.03135
|
| 113 |
+
},
|
| 114 |
+
{
|
| 115 |
+
"t": 0.82,
|
| 116 |
+
"precision": 0.376418,
|
| 117 |
+
"recall": 0.042849
|
| 118 |
+
},
|
| 119 |
+
{
|
| 120 |
+
"t": 0.8,
|
| 121 |
+
"precision": 0.374539,
|
| 122 |
+
"recall": 0.058827
|
| 123 |
+
},
|
| 124 |
+
{
|
| 125 |
+
"t": 0.78,
|
| 126 |
+
"precision": 0.374795,
|
| 127 |
+
"recall": 0.07541
|
| 128 |
+
},
|
| 129 |
+
{
|
| 130 |
+
"t": 0.76,
|
| 131 |
+
"precision": 0.376146,
|
| 132 |
+
"recall": 0.095022
|
| 133 |
+
},
|
| 134 |
+
{
|
| 135 |
+
"t": 0.74,
|
| 136 |
+
"precision": 0.37541,
|
| 137 |
+
"recall": 0.117128
|
| 138 |
+
},
|
| 139 |
+
{
|
| 140 |
+
"t": 0.72,
|
| 141 |
+
"precision": 0.37474,
|
| 142 |
+
"recall": 0.141246
|
| 143 |
+
},
|
| 144 |
+
{
|
| 145 |
+
"t": 0.7,
|
| 146 |
+
"precision": 0.37296,
|
| 147 |
+
"recall": 0.167342
|
| 148 |
+
},
|
| 149 |
+
{
|
| 150 |
+
"t": 0.68,
|
| 151 |
+
"precision": 0.372363,
|
| 152 |
+
"recall": 0.195745
|
| 153 |
+
},
|
| 154 |
+
{
|
| 155 |
+
"t": 0.66,
|
| 156 |
+
"precision": 0.371237,
|
| 157 |
+
"recall": 0.225608
|
| 158 |
+
},
|
| 159 |
+
{
|
| 160 |
+
"t": 0.64,
|
| 161 |
+
"precision": 0.37039,
|
| 162 |
+
"recall": 0.256531
|
| 163 |
+
},
|
| 164 |
+
{
|
| 165 |
+
"t": 0.62,
|
| 166 |
+
"precision": 0.369873,
|
| 167 |
+
"recall": 0.28952
|
| 168 |
+
},
|
| 169 |
+
{
|
| 170 |
+
"t": 0.6,
|
| 171 |
+
"precision": 0.369541,
|
| 172 |
+
"recall": 0.32951
|
| 173 |
+
},
|
| 174 |
+
{
|
| 175 |
+
"t": 0.58,
|
| 176 |
+
"precision": 0.368185,
|
| 177 |
+
"recall": 0.362721
|
| 178 |
+
},
|
| 179 |
+
{
|
| 180 |
+
"t": 0.56,
|
| 181 |
+
"precision": 0.367791,
|
| 182 |
+
"recall": 0.397118
|
| 183 |
+
},
|
| 184 |
+
{
|
| 185 |
+
"t": 0.54,
|
| 186 |
+
"precision": 0.36718,
|
| 187 |
+
"recall": 0.431336
|
| 188 |
+
},
|
| 189 |
+
{
|
| 190 |
+
"t": 0.52,
|
| 191 |
+
"precision": 0.366306,
|
| 192 |
+
"recall": 0.465439
|
| 193 |
+
},
|
| 194 |
+
{
|
| 195 |
+
"t": 0.5,
|
| 196 |
+
"precision": 0.366091,
|
| 197 |
+
"recall": 0.499862
|
| 198 |
+
},
|
| 199 |
+
{
|
| 200 |
+
"t": 0.48,
|
| 201 |
+
"precision": 0.366352,
|
| 202 |
+
"recall": 0.534802
|
| 203 |
+
},
|
| 204 |
+
{
|
| 205 |
+
"t": 0.46,
|
| 206 |
+
"precision": 0.366383,
|
| 207 |
+
"recall": 0.568842
|
| 208 |
+
},
|
| 209 |
+
{
|
| 210 |
+
"t": 0.44,
|
| 211 |
+
"precision": 0.366473,
|
| 212 |
+
"recall": 0.602437
|
| 213 |
+
},
|
| 214 |
+
{
|
| 215 |
+
"t": 0.42,
|
| 216 |
+
"precision": 0.365986,
|
| 217 |
+
"recall": 0.634233
|
| 218 |
+
},
|
| 219 |
+
{
|
| 220 |
+
"t": 0.4,
|
| 221 |
+
"precision": 0.365797,
|
| 222 |
+
"recall": 0.671889
|
| 223 |
+
},
|
| 224 |
+
{
|
| 225 |
+
"t": 0.38,
|
| 226 |
+
"precision": 0.366051,
|
| 227 |
+
"recall": 0.702411
|
| 228 |
+
},
|
| 229 |
+
{
|
| 230 |
+
"t": 0.36,
|
| 231 |
+
"precision": 0.365761,
|
| 232 |
+
"recall": 0.731054
|
| 233 |
+
},
|
| 234 |
+
{
|
| 235 |
+
"t": 0.34,
|
| 236 |
+
"precision": 0.365854,
|
| 237 |
+
"recall": 0.758984
|
| 238 |
+
},
|
| 239 |
+
{
|
| 240 |
+
"t": 0.32,
|
| 241 |
+
"precision": 0.365758,
|
| 242 |
+
"recall": 0.785
|
| 243 |
+
},
|
| 244 |
+
{
|
| 245 |
+
"t": 0.3,
|
| 246 |
+
"precision": 0.365498,
|
| 247 |
+
"recall": 0.808985
|
| 248 |
+
},
|
| 249 |
+
{
|
| 250 |
+
"t": 0.28,
|
| 251 |
+
"precision": 0.365094,
|
| 252 |
+
"recall": 0.830636
|
| 253 |
+
},
|
| 254 |
+
{
|
| 255 |
+
"t": 0.26,
|
| 256 |
+
"precision": 0.364846,
|
| 257 |
+
"recall": 0.851059
|
| 258 |
+
},
|
| 259 |
+
{
|
| 260 |
+
"t": 0.24,
|
| 261 |
+
"precision": 0.364451,
|
| 262 |
+
"recall": 0.869121
|
| 263 |
+
},
|
| 264 |
+
{
|
| 265 |
+
"t": 0.22,
|
| 266 |
+
"precision": 0.363978,
|
| 267 |
+
"recall": 0.885562
|
| 268 |
+
},
|
| 269 |
+
{
|
| 270 |
+
"t": 0.2,
|
| 271 |
+
"precision": 0.36361,
|
| 272 |
+
"recall": 0.903909
|
| 273 |
+
},
|
| 274 |
+
{
|
| 275 |
+
"t": 0.18,
|
| 276 |
+
"precision": 0.363437,
|
| 277 |
+
"recall": 0.917981
|
| 278 |
+
},
|
| 279 |
+
{
|
| 280 |
+
"t": 0.16,
|
| 281 |
+
"precision": 0.363229,
|
| 282 |
+
"recall": 0.930503
|
| 283 |
+
},
|
| 284 |
+
{
|
| 285 |
+
"t": 0.14,
|
| 286 |
+
"precision": 0.362969,
|
| 287 |
+
"recall": 0.941458
|
| 288 |
+
},
|
| 289 |
+
{
|
| 290 |
+
"t": 0.12,
|
| 291 |
+
"precision": 0.362743,
|
| 292 |
+
"recall": 0.951104
|
| 293 |
+
},
|
| 294 |
+
{
|
| 295 |
+
"t": 0.1,
|
| 296 |
+
"precision": 0.362424,
|
| 297 |
+
"recall": 0.959289
|
| 298 |
+
},
|
| 299 |
+
{
|
| 300 |
+
"t": 0.08,
|
| 301 |
+
"precision": 0.362018,
|
| 302 |
+
"recall": 0.966904
|
| 303 |
+
},
|
| 304 |
+
{
|
| 305 |
+
"t": 0.06,
|
| 306 |
+
"precision": 0.361708,
|
| 307 |
+
"recall": 0.973949
|
| 308 |
+
},
|
| 309 |
+
{
|
| 310 |
+
"t": 0.04,
|
| 311 |
+
"precision": 0.36137,
|
| 312 |
+
"recall": 0.980014
|
| 313 |
+
},
|
| 314 |
+
{
|
| 315 |
+
"t": 0.02,
|
| 316 |
+
"precision": 0.360408,
|
| 317 |
+
"recall": 0.985866
|
| 318 |
+
},
|
| 319 |
+
{
|
| 320 |
+
"t": 0.0,
|
| 321 |
+
"precision": 0.297854,
|
| 322 |
+
"recall": 1.0
|
| 323 |
+
}
|
| 324 |
+
]
|
| 325 |
+
},
|
| 326 |
+
"annotations": [],
|
| 327 |
+
"credits": {
|
| 328 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 329 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 330 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 331 |
+
"codeLicense": "Apache-2.0"
|
| 332 |
+
},
|
| 333 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 334 |
+
}
|
docs/examples/bench/w023-ink9um.json
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:47:43Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w023",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w023_ink.png",
|
| 15 |
+
"sha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d",
|
| 16 |
+
"width": 589,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.861,
|
| 19 |
+
"alignTargetFile": "data/bench/w023_ink.png",
|
| 20 |
+
"alignTargetSha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "w023_gpu.png",
|
| 24 |
+
"sha256": "14bedeb6a276f50d4783caec9774ccc4cf5899ddd67a0d57bf75dc6d04372d0c",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 589,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan",
|
| 32 |
+
"url": "https://huggingface.co/scrollprize/ink_9um"
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"alignment": {
|
| 36 |
+
"method": "auto",
|
| 37 |
+
"target": "ink",
|
| 38 |
+
"tx": 0.563,
|
| 39 |
+
"ty": 0.5,
|
| 40 |
+
"sx": 1.0004,
|
| 41 |
+
"sy": 1.00157,
|
| 42 |
+
"ncc": 0.496932,
|
| 43 |
+
"coarseNcc": 0.676478,
|
| 44 |
+
"evals": 38395,
|
| 45 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 46 |
+
},
|
| 47 |
+
"metrics": {
|
| 48 |
+
"nValid": 376320,
|
| 49 |
+
"validFraction": 0.998302,
|
| 50 |
+
"pearson": 0.496932,
|
| 51 |
+
"blurSigmaPx": 2.5,
|
| 52 |
+
"blurSigmaUM": 253.5,
|
| 53 |
+
"pearsonMatchedBlur": 0.537678,
|
| 54 |
+
"thresholds": {
|
| 55 |
+
"ref": 0.5,
|
| 56 |
+
"pred": 0.5
|
| 57 |
+
},
|
| 58 |
+
"iou": 0.322425,
|
| 59 |
+
"dice": 0.487627,
|
| 60 |
+
"precision": 0.611897,
|
| 61 |
+
"recall": 0.405312,
|
| 62 |
+
"refPositiveFraction": 0.29836,
|
| 63 |
+
"predPositiveFraction": 0.19763,
|
| 64 |
+
"ap": 0.572829,
|
| 65 |
+
"bestF1": {
|
| 66 |
+
"f1": 0.58117,
|
| 67 |
+
"threshold": 0.12
|
| 68 |
+
},
|
| 69 |
+
"pr": [
|
| 70 |
+
{
|
| 71 |
+
"t": 1.0,
|
| 72 |
+
"precision": 0.891379,
|
| 73 |
+
"recall": 0.004605
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"t": 0.98,
|
| 77 |
+
"precision": 0.809298,
|
| 78 |
+
"recall": 0.069773
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"t": 0.96,
|
| 82 |
+
"precision": 0.788433,
|
| 83 |
+
"recall": 0.091308
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"t": 0.94,
|
| 87 |
+
"precision": 0.77342,
|
| 88 |
+
"recall": 0.109415
|
| 89 |
+
},
|
| 90 |
+
{
|
| 91 |
+
"t": 0.92,
|
| 92 |
+
"precision": 0.760949,
|
| 93 |
+
"recall": 0.12534
|
| 94 |
+
},
|
| 95 |
+
{
|
| 96 |
+
"t": 0.9,
|
| 97 |
+
"precision": 0.751885,
|
| 98 |
+
"recall": 0.140293
|
| 99 |
+
},
|
| 100 |
+
{
|
| 101 |
+
"t": 0.88,
|
| 102 |
+
"precision": 0.743538,
|
| 103 |
+
"recall": 0.154232
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"t": 0.86,
|
| 107 |
+
"precision": 0.736346,
|
| 108 |
+
"recall": 0.168473
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"t": 0.84,
|
| 112 |
+
"precision": 0.728813,
|
| 113 |
+
"recall": 0.181984
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"t": 0.82,
|
| 117 |
+
"precision": 0.720644,
|
| 118 |
+
"recall": 0.195246
|
| 119 |
+
},
|
| 120 |
+
{
|
| 121 |
+
"t": 0.8,
|
| 122 |
+
"precision": 0.713955,
|
| 123 |
+
"recall": 0.209051
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"t": 0.78,
|
| 127 |
+
"precision": 0.706978,
|
| 128 |
+
"recall": 0.222686
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"t": 0.76,
|
| 132 |
+
"precision": 0.699595,
|
| 133 |
+
"recall": 0.235583
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"t": 0.74,
|
| 137 |
+
"precision": 0.691579,
|
| 138 |
+
"recall": 0.248319
|
| 139 |
+
},
|
| 140 |
+
{
|
| 141 |
+
"t": 0.72,
|
| 142 |
+
"precision": 0.683979,
|
| 143 |
+
"recall": 0.260734
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"t": 0.7,
|
| 147 |
+
"precision": 0.677081,
|
| 148 |
+
"recall": 0.273488
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"t": 0.68,
|
| 152 |
+
"precision": 0.669728,
|
| 153 |
+
"recall": 0.28577
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"t": 0.66,
|
| 157 |
+
"precision": 0.663324,
|
| 158 |
+
"recall": 0.298729
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"t": 0.64,
|
| 162 |
+
"precision": 0.655696,
|
| 163 |
+
"recall": 0.311225
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"t": 0.62,
|
| 167 |
+
"precision": 0.650146,
|
| 168 |
+
"recall": 0.325261
|
| 169 |
+
},
|
| 170 |
+
{
|
| 171 |
+
"t": 0.6,
|
| 172 |
+
"precision": 0.643842,
|
| 173 |
+
"recall": 0.338656
|
| 174 |
+
},
|
| 175 |
+
{
|
| 176 |
+
"t": 0.58,
|
| 177 |
+
"precision": 0.636991,
|
| 178 |
+
"recall": 0.351063
|
| 179 |
+
},
|
| 180 |
+
{
|
| 181 |
+
"t": 0.56,
|
| 182 |
+
"precision": 0.630731,
|
| 183 |
+
"recall": 0.364387
|
| 184 |
+
},
|
| 185 |
+
{
|
| 186 |
+
"t": 0.54,
|
| 187 |
+
"precision": 0.624137,
|
| 188 |
+
"recall": 0.378299
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"t": 0.52,
|
| 192 |
+
"precision": 0.617467,
|
| 193 |
+
"recall": 0.392041
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"t": 0.5,
|
| 197 |
+
"precision": 0.611897,
|
| 198 |
+
"recall": 0.405312
|
| 199 |
+
},
|
| 200 |
+
{
|
| 201 |
+
"t": 0.48,
|
| 202 |
+
"precision": 0.606205,
|
| 203 |
+
"recall": 0.419936
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"t": 0.46,
|
| 207 |
+
"precision": 0.6,
|
| 208 |
+
"recall": 0.434747
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"t": 0.44,
|
| 212 |
+
"precision": 0.594035,
|
| 213 |
+
"recall": 0.449149
|
| 214 |
+
},
|
| 215 |
+
{
|
| 216 |
+
"t": 0.42,
|
| 217 |
+
"precision": 0.587628,
|
| 218 |
+
"recall": 0.46404
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
"t": 0.4,
|
| 222 |
+
"precision": 0.581228,
|
| 223 |
+
"recall": 0.478896
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"t": 0.38,
|
| 227 |
+
"precision": 0.57507,
|
| 228 |
+
"recall": 0.494269
|
| 229 |
+
},
|
| 230 |
+
{
|
| 231 |
+
"t": 0.36,
|
| 232 |
+
"precision": 0.56844,
|
| 233 |
+
"recall": 0.510122
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"t": 0.34,
|
| 237 |
+
"precision": 0.561731,
|
| 238 |
+
"recall": 0.526269
|
| 239 |
+
},
|
| 240 |
+
{
|
| 241 |
+
"t": 0.32,
|
| 242 |
+
"precision": 0.554684,
|
| 243 |
+
"recall": 0.543405
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"t": 0.3,
|
| 247 |
+
"precision": 0.548081,
|
| 248 |
+
"recall": 0.561841
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"t": 0.28,
|
| 252 |
+
"precision": 0.540665,
|
| 253 |
+
"recall": 0.579645
|
| 254 |
+
},
|
| 255 |
+
{
|
| 256 |
+
"t": 0.26,
|
| 257 |
+
"precision": 0.53327,
|
| 258 |
+
"recall": 0.597868
|
| 259 |
+
},
|
| 260 |
+
{
|
| 261 |
+
"t": 0.24,
|
| 262 |
+
"precision": 0.525033,
|
| 263 |
+
"recall": 0.617177
|
| 264 |
+
},
|
| 265 |
+
{
|
| 266 |
+
"t": 0.22,
|
| 267 |
+
"precision": 0.517306,
|
| 268 |
+
"recall": 0.63816
|
| 269 |
+
},
|
| 270 |
+
{
|
| 271 |
+
"t": 0.2,
|
| 272 |
+
"precision": 0.509064,
|
| 273 |
+
"recall": 0.6595
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"t": 0.18,
|
| 277 |
+
"precision": 0.500298,
|
| 278 |
+
"recall": 0.681232
|
| 279 |
+
},
|
| 280 |
+
{
|
| 281 |
+
"t": 0.16,
|
| 282 |
+
"precision": 0.49167,
|
| 283 |
+
"recall": 0.704442
|
| 284 |
+
},
|
| 285 |
+
{
|
| 286 |
+
"t": 0.14,
|
| 287 |
+
"precision": 0.482454,
|
| 288 |
+
"recall": 0.728907
|
| 289 |
+
},
|
| 290 |
+
{
|
| 291 |
+
"t": 0.12,
|
| 292 |
+
"precision": 0.472721,
|
| 293 |
+
"recall": 0.754193
|
| 294 |
+
},
|
| 295 |
+
{
|
| 296 |
+
"t": 0.1,
|
| 297 |
+
"precision": 0.461733,
|
| 298 |
+
"recall": 0.781384
|
| 299 |
+
},
|
| 300 |
+
{
|
| 301 |
+
"t": 0.08,
|
| 302 |
+
"precision": 0.449505,
|
| 303 |
+
"recall": 0.810169
|
| 304 |
+
},
|
| 305 |
+
{
|
| 306 |
+
"t": 0.06,
|
| 307 |
+
"precision": 0.435419,
|
| 308 |
+
"recall": 0.842161
|
| 309 |
+
},
|
| 310 |
+
{
|
| 311 |
+
"t": 0.04,
|
| 312 |
+
"precision": 0.418136,
|
| 313 |
+
"recall": 0.878882
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"t": 0.02,
|
| 317 |
+
"precision": 0.394628,
|
| 318 |
+
"recall": 0.922425
|
| 319 |
+
},
|
| 320 |
+
{
|
| 321 |
+
"t": 0.0,
|
| 322 |
+
"precision": 0.29836,
|
| 323 |
+
"recall": 1.0
|
| 324 |
+
}
|
| 325 |
+
]
|
| 326 |
+
},
|
| 327 |
+
"annotations": [],
|
| 328 |
+
"credits": {
|
| 329 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 330 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 331 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 332 |
+
"codeLicense": "Apache-2.0"
|
| 333 |
+
},
|
| 334 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 335 |
+
}
|
docs/examples/bench/w023-noise-baseline.json
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:47:45Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w023",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w023_ink.png",
|
| 15 |
+
"sha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d",
|
| 16 |
+
"width": 589,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.861,
|
| 19 |
+
"alignTargetFile": "data/bench/w023_ink.png",
|
| 20 |
+
"alignTargetSha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "noise-seed23.png",
|
| 24 |
+
"sha256": "f66295cef27e849450ae58b39bb65cbfc7af90af40f3faad11e838e28f4e08f9",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 589,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "baseline: uniform noise",
|
| 32 |
+
"notes": "numpy default_rng(23).random, 8 bit"
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"alignment": {
|
| 36 |
+
"method": "manual",
|
| 37 |
+
"target": "ink",
|
| 38 |
+
"tx": 0.0,
|
| 39 |
+
"ty": 0.0,
|
| 40 |
+
"sx": 1.0,
|
| 41 |
+
"sy": 1.0,
|
| 42 |
+
"ncc": 0.001026,
|
| 43 |
+
"coarseNcc": null,
|
| 44 |
+
"evals": 0,
|
| 45 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 46 |
+
},
|
| 47 |
+
"metrics": {
|
| 48 |
+
"nValid": 376960,
|
| 49 |
+
"validFraction": 1.0,
|
| 50 |
+
"pearson": 0.001026,
|
| 51 |
+
"blurSigmaPx": 0.5,
|
| 52 |
+
"blurSigmaUM": 50.7,
|
| 53 |
+
"pearsonMatchedBlur": 0.001057,
|
| 54 |
+
"thresholds": {
|
| 55 |
+
"ref": 0.5,
|
| 56 |
+
"pred": 0.5
|
| 57 |
+
},
|
| 58 |
+
"iou": 0.229363,
|
| 59 |
+
"dice": 0.373141,
|
| 60 |
+
"precision": 0.298139,
|
| 61 |
+
"recall": 0.498562,
|
| 62 |
+
"refPositiveFraction": 0.297854,
|
| 63 |
+
"predPositiveFraction": 0.498085,
|
| 64 |
+
"ap": 0.298344,
|
| 65 |
+
"bestF1": {
|
| 66 |
+
"f1": 0.458994,
|
| 67 |
+
"threshold": 0.0
|
| 68 |
+
},
|
| 69 |
+
"pr": [
|
| 70 |
+
{
|
| 71 |
+
"t": 1.0,
|
| 72 |
+
"precision": 1.0,
|
| 73 |
+
"recall": 0.0
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"t": 0.98,
|
| 77 |
+
"precision": 0.296047,
|
| 78 |
+
"recall": 0.019612
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"t": 0.96,
|
| 82 |
+
"precision": 0.296167,
|
| 83 |
+
"recall": 0.039437
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"t": 0.94,
|
| 87 |
+
"precision": 0.298428,
|
| 88 |
+
"recall": 0.059521
|
| 89 |
+
},
|
| 90 |
+
{
|
| 91 |
+
"t": 0.92,
|
| 92 |
+
"precision": 0.300548,
|
| 93 |
+
"recall": 0.079579
|
| 94 |
+
},
|
| 95 |
+
{
|
| 96 |
+
"t": 0.9,
|
| 97 |
+
"precision": 0.300801,
|
| 98 |
+
"recall": 0.099645
|
| 99 |
+
},
|
| 100 |
+
{
|
| 101 |
+
"t": 0.88,
|
| 102 |
+
"precision": 0.298906,
|
| 103 |
+
"recall": 0.118749
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"t": 0.86,
|
| 107 |
+
"precision": 0.299743,
|
| 108 |
+
"recall": 0.138289
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"t": 0.84,
|
| 112 |
+
"precision": 0.299664,
|
| 113 |
+
"recall": 0.157875
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"t": 0.82,
|
| 117 |
+
"precision": 0.299698,
|
| 118 |
+
"recall": 0.177602
|
| 119 |
+
},
|
| 120 |
+
{
|
| 121 |
+
"t": 0.8,
|
| 122 |
+
"precision": 0.300253,
|
| 123 |
+
"recall": 0.201881
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"t": 0.78,
|
| 127 |
+
"precision": 0.299585,
|
| 128 |
+
"recall": 0.221003
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"t": 0.76,
|
| 132 |
+
"precision": 0.298898,
|
| 133 |
+
"recall": 0.240116
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"t": 0.74,
|
| 137 |
+
"precision": 0.299281,
|
| 138 |
+
"recall": 0.26053
|
| 139 |
+
},
|
| 140 |
+
{
|
| 141 |
+
"t": 0.72,
|
| 142 |
+
"precision": 0.299505,
|
| 143 |
+
"recall": 0.280275
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"t": 0.7,
|
| 147 |
+
"precision": 0.299145,
|
| 148 |
+
"recall": 0.299566
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"t": 0.68,
|
| 152 |
+
"precision": 0.29831,
|
| 153 |
+
"recall": 0.318813
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"t": 0.66,
|
| 157 |
+
"precision": 0.297711,
|
| 158 |
+
"recall": 0.33773
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"t": 0.64,
|
| 162 |
+
"precision": 0.297573,
|
| 163 |
+
"recall": 0.356825
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"t": 0.62,
|
| 167 |
+
"precision": 0.29771,
|
| 168 |
+
"recall": 0.376526
|
| 169 |
+
},
|
| 170 |
+
{
|
| 171 |
+
"t": 0.6,
|
| 172 |
+
"precision": 0.298141,
|
| 173 |
+
"recall": 0.4002
|
| 174 |
+
},
|
| 175 |
+
{
|
| 176 |
+
"t": 0.58,
|
| 177 |
+
"precision": 0.298043,
|
| 178 |
+
"recall": 0.419517
|
| 179 |
+
},
|
| 180 |
+
{
|
| 181 |
+
"t": 0.56,
|
| 182 |
+
"precision": 0.297849,
|
| 183 |
+
"recall": 0.439201
|
| 184 |
+
},
|
| 185 |
+
{
|
| 186 |
+
"t": 0.54,
|
| 187 |
+
"precision": 0.297855,
|
| 188 |
+
"recall": 0.458973
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"t": 0.52,
|
| 192 |
+
"precision": 0.297817,
|
| 193 |
+
"recall": 0.478398
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"t": 0.5,
|
| 197 |
+
"precision": 0.298139,
|
| 198 |
+
"recall": 0.498562
|
| 199 |
+
},
|
| 200 |
+
{
|
| 201 |
+
"t": 0.48,
|
| 202 |
+
"precision": 0.297952,
|
| 203 |
+
"recall": 0.517612
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"t": 0.46,
|
| 207 |
+
"precision": 0.29801,
|
| 208 |
+
"recall": 0.537171
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"t": 0.44,
|
| 212 |
+
"precision": 0.298175,
|
| 213 |
+
"recall": 0.55664
|
| 214 |
+
},
|
| 215 |
+
{
|
| 216 |
+
"t": 0.42,
|
| 217 |
+
"precision": 0.29786,
|
| 218 |
+
"recall": 0.57545
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
"t": 0.4,
|
| 222 |
+
"precision": 0.298317,
|
| 223 |
+
"recall": 0.600059
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"t": 0.38,
|
| 227 |
+
"precision": 0.298162,
|
| 228 |
+
"recall": 0.619359
|
| 229 |
+
},
|
| 230 |
+
{
|
| 231 |
+
"t": 0.36,
|
| 232 |
+
"precision": 0.298334,
|
| 233 |
+
"recall": 0.639487
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"t": 0.34,
|
| 237 |
+
"precision": 0.298307,
|
| 238 |
+
"recall": 0.658814
|
| 239 |
+
},
|
| 240 |
+
{
|
| 241 |
+
"t": 0.32,
|
| 242 |
+
"precision": 0.298448,
|
| 243 |
+
"recall": 0.679103
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"t": 0.3,
|
| 247 |
+
"precision": 0.29852,
|
| 248 |
+
"recall": 0.699009
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"t": 0.28,
|
| 252 |
+
"precision": 0.298225,
|
| 253 |
+
"recall": 0.718211
|
| 254 |
+
},
|
| 255 |
+
{
|
| 256 |
+
"t": 0.26,
|
| 257 |
+
"precision": 0.298215,
|
| 258 |
+
"recall": 0.737698
|
| 259 |
+
},
|
| 260 |
+
{
|
| 261 |
+
"t": 0.24,
|
| 262 |
+
"precision": 0.298211,
|
| 263 |
+
"recall": 0.757354
|
| 264 |
+
},
|
| 265 |
+
{
|
| 266 |
+
"t": 0.22,
|
| 267 |
+
"precision": 0.298169,
|
| 268 |
+
"recall": 0.77669
|
| 269 |
+
},
|
| 270 |
+
{
|
| 271 |
+
"t": 0.2,
|
| 272 |
+
"precision": 0.298193,
|
| 273 |
+
"recall": 0.800123
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"t": 0.18,
|
| 277 |
+
"precision": 0.298001,
|
| 278 |
+
"recall": 0.819209
|
| 279 |
+
},
|
| 280 |
+
{
|
| 281 |
+
"t": 0.16,
|
| 282 |
+
"precision": 0.297842,
|
| 283 |
+
"recall": 0.838536
|
| 284 |
+
},
|
| 285 |
+
{
|
| 286 |
+
"t": 0.14,
|
| 287 |
+
"precision": 0.297808,
|
| 288 |
+
"recall": 0.858308
|
| 289 |
+
},
|
| 290 |
+
{
|
| 291 |
+
"t": 0.12,
|
| 292 |
+
"precision": 0.297846,
|
| 293 |
+
"recall": 0.877893
|
| 294 |
+
},
|
| 295 |
+
{
|
| 296 |
+
"t": 0.1,
|
| 297 |
+
"precision": 0.297635,
|
| 298 |
+
"recall": 0.89722
|
| 299 |
+
},
|
| 300 |
+
{
|
| 301 |
+
"t": 0.08,
|
| 302 |
+
"precision": 0.297763,
|
| 303 |
+
"recall": 0.91734
|
| 304 |
+
},
|
| 305 |
+
{
|
| 306 |
+
"t": 0.06,
|
| 307 |
+
"precision": 0.297714,
|
| 308 |
+
"recall": 0.936551
|
| 309 |
+
},
|
| 310 |
+
{
|
| 311 |
+
"t": 0.04,
|
| 312 |
+
"precision": 0.297641,
|
| 313 |
+
"recall": 0.95635
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"t": 0.02,
|
| 317 |
+
"precision": 0.29774,
|
| 318 |
+
"recall": 0.97622
|
| 319 |
+
},
|
| 320 |
+
{
|
| 321 |
+
"t": 0.0,
|
| 322 |
+
"precision": 0.297854,
|
| 323 |
+
"recall": 1.0
|
| 324 |
+
}
|
| 325 |
+
]
|
| 326 |
+
},
|
| 327 |
+
"annotations": [],
|
| 328 |
+
"credits": {
|
| 329 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 330 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 331 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 332 |
+
"codeLicense": "Apache-2.0"
|
| 333 |
+
},
|
| 334 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 335 |
+
}
|
docs/examples/bench/w024-ct-baseline.json
ADDED
|
@@ -0,0 +1,334 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:47:56Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w024",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w024_ink.png",
|
| 15 |
+
"sha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327",
|
| 16 |
+
"width": 597,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.6321,
|
| 19 |
+
"alignTargetFile": "data/bench/w024_ink.png",
|
| 20 |
+
"alignTargetSha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "w024_ct.png",
|
| 24 |
+
"sha256": "7f21087fc1b5a1a5812e02d24134004524373408d5fe533939f12003f90c6517",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 597,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "baseline: X-ray intensity (same grid, no alignment needed)"
|
| 32 |
+
}
|
| 33 |
+
},
|
| 34 |
+
"alignment": {
|
| 35 |
+
"method": "manual",
|
| 36 |
+
"target": "ink",
|
| 37 |
+
"tx": 0.0,
|
| 38 |
+
"ty": 0.0,
|
| 39 |
+
"sx": 1.0,
|
| 40 |
+
"sy": 1.0,
|
| 41 |
+
"ncc": 0.361809,
|
| 42 |
+
"coarseNcc": null,
|
| 43 |
+
"evals": 0,
|
| 44 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 45 |
+
},
|
| 46 |
+
"metrics": {
|
| 47 |
+
"nValid": 382080,
|
| 48 |
+
"validFraction": 1.0,
|
| 49 |
+
"pearson": 0.361809,
|
| 50 |
+
"blurSigmaPx": 6.0,
|
| 51 |
+
"blurSigmaUM": 622.9,
|
| 52 |
+
"pearsonMatchedBlur": 0.464513,
|
| 53 |
+
"thresholds": {
|
| 54 |
+
"ref": 0.5,
|
| 55 |
+
"pred": 0.5
|
| 56 |
+
},
|
| 57 |
+
"iou": 0.288116,
|
| 58 |
+
"dice": 0.447345,
|
| 59 |
+
"precision": 0.418449,
|
| 60 |
+
"recall": 0.480526,
|
| 61 |
+
"refPositiveFraction": 0.333917,
|
| 62 |
+
"predPositiveFraction": 0.383454,
|
| 63 |
+
"ap": 0.41647,
|
| 64 |
+
"bestF1": {
|
| 65 |
+
"f1": 0.574501,
|
| 66 |
+
"threshold": 0.02
|
| 67 |
+
},
|
| 68 |
+
"pr": [
|
| 69 |
+
{
|
| 70 |
+
"t": 1.0,
|
| 71 |
+
"precision": 0.382353,
|
| 72 |
+
"recall": 0.000509
|
| 73 |
+
},
|
| 74 |
+
{
|
| 75 |
+
"t": 0.98,
|
| 76 |
+
"precision": 0.38412,
|
| 77 |
+
"recall": 0.001403
|
| 78 |
+
},
|
| 79 |
+
{
|
| 80 |
+
"t": 0.96,
|
| 81 |
+
"precision": 0.376357,
|
| 82 |
+
"recall": 0.002445
|
| 83 |
+
},
|
| 84 |
+
{
|
| 85 |
+
"t": 0.94,
|
| 86 |
+
"precision": 0.380223,
|
| 87 |
+
"recall": 0.00428
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"t": 0.92,
|
| 91 |
+
"precision": 0.392257,
|
| 92 |
+
"recall": 0.006592
|
| 93 |
+
},
|
| 94 |
+
{
|
| 95 |
+
"t": 0.9,
|
| 96 |
+
"precision": 0.396662,
|
| 97 |
+
"recall": 0.009688
|
| 98 |
+
},
|
| 99 |
+
{
|
| 100 |
+
"t": 0.88,
|
| 101 |
+
"precision": 0.401505,
|
| 102 |
+
"recall": 0.013803
|
| 103 |
+
},
|
| 104 |
+
{
|
| 105 |
+
"t": 0.86,
|
| 106 |
+
"precision": 0.403503,
|
| 107 |
+
"recall": 0.01914
|
| 108 |
+
},
|
| 109 |
+
{
|
| 110 |
+
"t": 0.84,
|
| 111 |
+
"precision": 0.411504,
|
| 112 |
+
"recall": 0.026242
|
| 113 |
+
},
|
| 114 |
+
{
|
| 115 |
+
"t": 0.82,
|
| 116 |
+
"precision": 0.416875,
|
| 117 |
+
"recall": 0.03524
|
| 118 |
+
},
|
| 119 |
+
{
|
| 120 |
+
"t": 0.8,
|
| 121 |
+
"precision": 0.419535,
|
| 122 |
+
"recall": 0.048815
|
| 123 |
+
},
|
| 124 |
+
{
|
| 125 |
+
"t": 0.78,
|
| 126 |
+
"precision": 0.424343,
|
| 127 |
+
"recall": 0.063261
|
| 128 |
+
},
|
| 129 |
+
{
|
| 130 |
+
"t": 0.76,
|
| 131 |
+
"precision": 0.425433,
|
| 132 |
+
"recall": 0.080136
|
| 133 |
+
},
|
| 134 |
+
{
|
| 135 |
+
"t": 0.74,
|
| 136 |
+
"precision": 0.42628,
|
| 137 |
+
"recall": 0.099574
|
| 138 |
+
},
|
| 139 |
+
{
|
| 140 |
+
"t": 0.72,
|
| 141 |
+
"precision": 0.428473,
|
| 142 |
+
"recall": 0.122171
|
| 143 |
+
},
|
| 144 |
+
{
|
| 145 |
+
"t": 0.7,
|
| 146 |
+
"precision": 0.427396,
|
| 147 |
+
"recall": 0.14661
|
| 148 |
+
},
|
| 149 |
+
{
|
| 150 |
+
"t": 0.68,
|
| 151 |
+
"precision": 0.426788,
|
| 152 |
+
"recall": 0.173056
|
| 153 |
+
},
|
| 154 |
+
{
|
| 155 |
+
"t": 0.66,
|
| 156 |
+
"precision": 0.426028,
|
| 157 |
+
"recall": 0.202033
|
| 158 |
+
},
|
| 159 |
+
{
|
| 160 |
+
"t": 0.64,
|
| 161 |
+
"precision": 0.42663,
|
| 162 |
+
"recall": 0.233558
|
| 163 |
+
},
|
| 164 |
+
{
|
| 165 |
+
"t": 0.62,
|
| 166 |
+
"precision": 0.425587,
|
| 167 |
+
"recall": 0.265247
|
| 168 |
+
},
|
| 169 |
+
{
|
| 170 |
+
"t": 0.6,
|
| 171 |
+
"precision": 0.423207,
|
| 172 |
+
"recall": 0.305135
|
| 173 |
+
},
|
| 174 |
+
{
|
| 175 |
+
"t": 0.58,
|
| 176 |
+
"precision": 0.422423,
|
| 177 |
+
"recall": 0.339881
|
| 178 |
+
},
|
| 179 |
+
{
|
| 180 |
+
"t": 0.56,
|
| 181 |
+
"precision": 0.421306,
|
| 182 |
+
"recall": 0.374979
|
| 183 |
+
},
|
| 184 |
+
{
|
| 185 |
+
"t": 0.54,
|
| 186 |
+
"precision": 0.420412,
|
| 187 |
+
"recall": 0.409827
|
| 188 |
+
},
|
| 189 |
+
{
|
| 190 |
+
"t": 0.52,
|
| 191 |
+
"precision": 0.419469,
|
| 192 |
+
"recall": 0.445318
|
| 193 |
+
},
|
| 194 |
+
{
|
| 195 |
+
"t": 0.5,
|
| 196 |
+
"precision": 0.418449,
|
| 197 |
+
"recall": 0.480526
|
| 198 |
+
},
|
| 199 |
+
{
|
| 200 |
+
"t": 0.48,
|
| 201 |
+
"precision": 0.41824,
|
| 202 |
+
"recall": 0.516323
|
| 203 |
+
},
|
| 204 |
+
{
|
| 205 |
+
"t": 0.46,
|
| 206 |
+
"precision": 0.417336,
|
| 207 |
+
"recall": 0.550254
|
| 208 |
+
},
|
| 209 |
+
{
|
| 210 |
+
"t": 0.44,
|
| 211 |
+
"precision": 0.417034,
|
| 212 |
+
"recall": 0.584278
|
| 213 |
+
},
|
| 214 |
+
{
|
| 215 |
+
"t": 0.42,
|
| 216 |
+
"precision": 0.416464,
|
| 217 |
+
"recall": 0.617206
|
| 218 |
+
},
|
| 219 |
+
{
|
| 220 |
+
"t": 0.4,
|
| 221 |
+
"precision": 0.415636,
|
| 222 |
+
"recall": 0.655268
|
| 223 |
+
},
|
| 224 |
+
{
|
| 225 |
+
"t": 0.38,
|
| 226 |
+
"precision": 0.415486,
|
| 227 |
+
"recall": 0.68593
|
| 228 |
+
},
|
| 229 |
+
{
|
| 230 |
+
"t": 0.36,
|
| 231 |
+
"precision": 0.415521,
|
| 232 |
+
"recall": 0.715417
|
| 233 |
+
},
|
| 234 |
+
{
|
| 235 |
+
"t": 0.34,
|
| 236 |
+
"precision": 0.415132,
|
| 237 |
+
"recall": 0.742732
|
| 238 |
+
},
|
| 239 |
+
{
|
| 240 |
+
"t": 0.32,
|
| 241 |
+
"precision": 0.414668,
|
| 242 |
+
"recall": 0.768935
|
| 243 |
+
},
|
| 244 |
+
{
|
| 245 |
+
"t": 0.3,
|
| 246 |
+
"precision": 0.414072,
|
| 247 |
+
"recall": 0.793272
|
| 248 |
+
},
|
| 249 |
+
{
|
| 250 |
+
"t": 0.28,
|
| 251 |
+
"precision": 0.413718,
|
| 252 |
+
"recall": 0.816018
|
| 253 |
+
},
|
| 254 |
+
{
|
| 255 |
+
"t": 0.26,
|
| 256 |
+
"precision": 0.413501,
|
| 257 |
+
"recall": 0.837298
|
| 258 |
+
},
|
| 259 |
+
{
|
| 260 |
+
"t": 0.24,
|
| 261 |
+
"precision": 0.41271,
|
| 262 |
+
"recall": 0.856149
|
| 263 |
+
},
|
| 264 |
+
{
|
| 265 |
+
"t": 0.22,
|
| 266 |
+
"precision": 0.412062,
|
| 267 |
+
"recall": 0.873102
|
| 268 |
+
},
|
| 269 |
+
{
|
| 270 |
+
"t": 0.2,
|
| 271 |
+
"precision": 0.411404,
|
| 272 |
+
"recall": 0.891506
|
| 273 |
+
},
|
| 274 |
+
{
|
| 275 |
+
"t": 0.18,
|
| 276 |
+
"precision": 0.410778,
|
| 277 |
+
"recall": 0.904682
|
| 278 |
+
},
|
| 279 |
+
{
|
| 280 |
+
"t": 0.16,
|
| 281 |
+
"precision": 0.410245,
|
| 282 |
+
"recall": 0.917528
|
| 283 |
+
},
|
| 284 |
+
{
|
| 285 |
+
"t": 0.14,
|
| 286 |
+
"precision": 0.409727,
|
| 287 |
+
"recall": 0.928776
|
| 288 |
+
},
|
| 289 |
+
{
|
| 290 |
+
"t": 0.12,
|
| 291 |
+
"precision": 0.40926,
|
| 292 |
+
"recall": 0.939028
|
| 293 |
+
},
|
| 294 |
+
{
|
| 295 |
+
"t": 0.1,
|
| 296 |
+
"precision": 0.408687,
|
| 297 |
+
"recall": 0.948434
|
| 298 |
+
},
|
| 299 |
+
{
|
| 300 |
+
"t": 0.08,
|
| 301 |
+
"precision": 0.408117,
|
| 302 |
+
"recall": 0.95722
|
| 303 |
+
},
|
| 304 |
+
{
|
| 305 |
+
"t": 0.06,
|
| 306 |
+
"precision": 0.407705,
|
| 307 |
+
"recall": 0.965481
|
| 308 |
+
},
|
| 309 |
+
{
|
| 310 |
+
"t": 0.04,
|
| 311 |
+
"precision": 0.40722,
|
| 312 |
+
"recall": 0.97328
|
| 313 |
+
},
|
| 314 |
+
{
|
| 315 |
+
"t": 0.02,
|
| 316 |
+
"precision": 0.406165,
|
| 317 |
+
"recall": 0.981134
|
| 318 |
+
},
|
| 319 |
+
{
|
| 320 |
+
"t": 0.0,
|
| 321 |
+
"precision": 0.333917,
|
| 322 |
+
"recall": 1.0
|
| 323 |
+
}
|
| 324 |
+
]
|
| 325 |
+
},
|
| 326 |
+
"annotations": [],
|
| 327 |
+
"credits": {
|
| 328 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 329 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 330 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 331 |
+
"codeLicense": "Apache-2.0"
|
| 332 |
+
},
|
| 333 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 334 |
+
}
|
docs/examples/bench/w024-ink9um.json
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:47:56Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w024",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w024_ink.png",
|
| 15 |
+
"sha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327",
|
| 16 |
+
"width": 597,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.6321,
|
| 19 |
+
"alignTargetFile": "data/bench/w024_ink.png",
|
| 20 |
+
"alignTargetSha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "w024_gpu.png",
|
| 24 |
+
"sha256": "cc11fb74a30bc3fb48bd652946e3e9124d1a2d90947f613f715da37671a85efd",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 597,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan",
|
| 32 |
+
"url": "https://huggingface.co/scrollprize/ink_9um"
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"alignment": {
|
| 36 |
+
"method": "auto",
|
| 37 |
+
"target": "ink",
|
| 38 |
+
"tx": 0.531,
|
| 39 |
+
"ty": 0.5,
|
| 40 |
+
"sx": 1.00018,
|
| 41 |
+
"sy": 1.00157,
|
| 42 |
+
"ncc": 0.572218,
|
| 43 |
+
"coarseNcc": 0.724818,
|
| 44 |
+
"evals": 40355,
|
| 45 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 46 |
+
},
|
| 47 |
+
"metrics": {
|
| 48 |
+
"nValid": 381440,
|
| 49 |
+
"validFraction": 0.998325,
|
| 50 |
+
"pearson": 0.572218,
|
| 51 |
+
"blurSigmaPx": 2.0,
|
| 52 |
+
"blurSigmaUM": 207.6,
|
| 53 |
+
"pearsonMatchedBlur": 0.608043,
|
| 54 |
+
"thresholds": {
|
| 55 |
+
"ref": 0.5,
|
| 56 |
+
"pred": 0.5
|
| 57 |
+
},
|
| 58 |
+
"iou": 0.396013,
|
| 59 |
+
"dice": 0.567349,
|
| 60 |
+
"precision": 0.706089,
|
| 61 |
+
"recall": 0.474178,
|
| 62 |
+
"refPositiveFraction": 0.334477,
|
| 63 |
+
"predPositiveFraction": 0.22462,
|
| 64 |
+
"ap": 0.670837,
|
| 65 |
+
"bestF1": {
|
| 66 |
+
"f1": 0.644796,
|
| 67 |
+
"threshold": 0.14
|
| 68 |
+
},
|
| 69 |
+
"pr": [
|
| 70 |
+
{
|
| 71 |
+
"t": 1.0,
|
| 72 |
+
"precision": 0.932029,
|
| 73 |
+
"recall": 0.007093
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"t": 0.98,
|
| 77 |
+
"precision": 0.874482,
|
| 78 |
+
"recall": 0.097638
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"t": 0.96,
|
| 82 |
+
"precision": 0.86084,
|
| 83 |
+
"recall": 0.125675
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"t": 0.94,
|
| 87 |
+
"precision": 0.851634,
|
| 88 |
+
"recall": 0.14748
|
| 89 |
+
},
|
| 90 |
+
{
|
| 91 |
+
"t": 0.92,
|
| 92 |
+
"precision": 0.843096,
|
| 93 |
+
"recall": 0.166401
|
| 94 |
+
},
|
| 95 |
+
{
|
| 96 |
+
"t": 0.9,
|
| 97 |
+
"precision": 0.83502,
|
| 98 |
+
"recall": 0.183677
|
| 99 |
+
},
|
| 100 |
+
{
|
| 101 |
+
"t": 0.88,
|
| 102 |
+
"precision": 0.827887,
|
| 103 |
+
"recall": 0.199972
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"t": 0.86,
|
| 107 |
+
"precision": 0.820642,
|
| 108 |
+
"recall": 0.21582
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"t": 0.84,
|
| 112 |
+
"precision": 0.813773,
|
| 113 |
+
"recall": 0.230814
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"t": 0.82,
|
| 117 |
+
"precision": 0.807815,
|
| 118 |
+
"recall": 0.246796
|
| 119 |
+
},
|
| 120 |
+
{
|
| 121 |
+
"t": 0.8,
|
| 122 |
+
"precision": 0.801974,
|
| 123 |
+
"recall": 0.26172
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"t": 0.78,
|
| 127 |
+
"precision": 0.795671,
|
| 128 |
+
"recall": 0.277169
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"t": 0.76,
|
| 132 |
+
"precision": 0.790217,
|
| 133 |
+
"recall": 0.292382
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"t": 0.74,
|
| 137 |
+
"precision": 0.783991,
|
| 138 |
+
"recall": 0.306836
|
| 139 |
+
},
|
| 140 |
+
{
|
| 141 |
+
"t": 0.72,
|
| 142 |
+
"precision": 0.777302,
|
| 143 |
+
"recall": 0.320333
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"t": 0.7,
|
| 147 |
+
"precision": 0.771313,
|
| 148 |
+
"recall": 0.334496
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"t": 0.68,
|
| 152 |
+
"precision": 0.764752,
|
| 153 |
+
"recall": 0.348824
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"t": 0.66,
|
| 157 |
+
"precision": 0.757991,
|
| 158 |
+
"recall": 0.36225
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"t": 0.64,
|
| 162 |
+
"precision": 0.751048,
|
| 163 |
+
"recall": 0.376328
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"t": 0.62,
|
| 167 |
+
"precision": 0.744027,
|
| 168 |
+
"recall": 0.39006
|
| 169 |
+
},
|
| 170 |
+
{
|
| 171 |
+
"t": 0.6,
|
| 172 |
+
"precision": 0.737464,
|
| 173 |
+
"recall": 0.403925
|
| 174 |
+
},
|
| 175 |
+
{
|
| 176 |
+
"t": 0.58,
|
| 177 |
+
"precision": 0.730956,
|
| 178 |
+
"recall": 0.418167
|
| 179 |
+
},
|
| 180 |
+
{
|
| 181 |
+
"t": 0.56,
|
| 182 |
+
"precision": 0.724791,
|
| 183 |
+
"recall": 0.432228
|
| 184 |
+
},
|
| 185 |
+
{
|
| 186 |
+
"t": 0.54,
|
| 187 |
+
"precision": 0.719011,
|
| 188 |
+
"recall": 0.446415
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"t": 0.52,
|
| 192 |
+
"precision": 0.71274,
|
| 193 |
+
"recall": 0.460516
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"t": 0.5,
|
| 197 |
+
"precision": 0.706089,
|
| 198 |
+
"recall": 0.474178
|
| 199 |
+
},
|
| 200 |
+
{
|
| 201 |
+
"t": 0.48,
|
| 202 |
+
"precision": 0.699017,
|
| 203 |
+
"recall": 0.487761
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"t": 0.46,
|
| 207 |
+
"precision": 0.69215,
|
| 208 |
+
"recall": 0.502065
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"t": 0.44,
|
| 212 |
+
"precision": 0.684481,
|
| 213 |
+
"recall": 0.516605
|
| 214 |
+
},
|
| 215 |
+
{
|
| 216 |
+
"t": 0.42,
|
| 217 |
+
"precision": 0.676919,
|
| 218 |
+
"recall": 0.530619
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
"t": 0.4,
|
| 222 |
+
"precision": 0.669169,
|
| 223 |
+
"recall": 0.545026
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"t": 0.38,
|
| 227 |
+
"precision": 0.66149,
|
| 228 |
+
"recall": 0.560216
|
| 229 |
+
},
|
| 230 |
+
{
|
| 231 |
+
"t": 0.36,
|
| 232 |
+
"precision": 0.654187,
|
| 233 |
+
"recall": 0.575751
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"t": 0.34,
|
| 237 |
+
"precision": 0.646507,
|
| 238 |
+
"recall": 0.591192
|
| 239 |
+
},
|
| 240 |
+
{
|
| 241 |
+
"t": 0.32,
|
| 242 |
+
"precision": 0.638707,
|
| 243 |
+
"recall": 0.607032
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"t": 0.3,
|
| 247 |
+
"precision": 0.630614,
|
| 248 |
+
"recall": 0.62321
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"t": 0.28,
|
| 252 |
+
"precision": 0.622001,
|
| 253 |
+
"recall": 0.640015
|
| 254 |
+
},
|
| 255 |
+
{
|
| 256 |
+
"t": 0.26,
|
| 257 |
+
"precision": 0.613529,
|
| 258 |
+
"recall": 0.657313
|
| 259 |
+
},
|
| 260 |
+
{
|
| 261 |
+
"t": 0.24,
|
| 262 |
+
"precision": 0.604511,
|
| 263 |
+
"recall": 0.674471
|
| 264 |
+
},
|
| 265 |
+
{
|
| 266 |
+
"t": 0.22,
|
| 267 |
+
"precision": 0.594999,
|
| 268 |
+
"recall": 0.692655
|
| 269 |
+
},
|
| 270 |
+
{
|
| 271 |
+
"t": 0.2,
|
| 272 |
+
"precision": 0.585037,
|
| 273 |
+
"recall": 0.711153
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"t": 0.18,
|
| 277 |
+
"precision": 0.57511,
|
| 278 |
+
"recall": 0.73005
|
| 279 |
+
},
|
| 280 |
+
{
|
| 281 |
+
"t": 0.16,
|
| 282 |
+
"precision": 0.564667,
|
| 283 |
+
"recall": 0.750288
|
| 284 |
+
},
|
| 285 |
+
{
|
| 286 |
+
"t": 0.14,
|
| 287 |
+
"precision": 0.553916,
|
| 288 |
+
"recall": 0.771349
|
| 289 |
+
},
|
| 290 |
+
{
|
| 291 |
+
"t": 0.12,
|
| 292 |
+
"precision": 0.542048,
|
| 293 |
+
"recall": 0.792927
|
| 294 |
+
},
|
| 295 |
+
{
|
| 296 |
+
"t": 0.1,
|
| 297 |
+
"precision": 0.528424,
|
| 298 |
+
"recall": 0.815281
|
| 299 |
+
},
|
| 300 |
+
{
|
| 301 |
+
"t": 0.08,
|
| 302 |
+
"precision": 0.513703,
|
| 303 |
+
"recall": 0.839924
|
| 304 |
+
},
|
| 305 |
+
{
|
| 306 |
+
"t": 0.06,
|
| 307 |
+
"precision": 0.496954,
|
| 308 |
+
"recall": 0.866887
|
| 309 |
+
},
|
| 310 |
+
{
|
| 311 |
+
"t": 0.04,
|
| 312 |
+
"precision": 0.476629,
|
| 313 |
+
"recall": 0.897479
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"t": 0.02,
|
| 317 |
+
"precision": 0.448327,
|
| 318 |
+
"recall": 0.934451
|
| 319 |
+
},
|
| 320 |
+
{
|
| 321 |
+
"t": 0.0,
|
| 322 |
+
"precision": 0.334477,
|
| 323 |
+
"recall": 1.0
|
| 324 |
+
}
|
| 325 |
+
]
|
| 326 |
+
},
|
| 327 |
+
"annotations": [],
|
| 328 |
+
"credits": {
|
| 329 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 330 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 331 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 332 |
+
"codeLicense": "Apache-2.0"
|
| 333 |
+
},
|
| 334 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 335 |
+
}
|
docs/examples/bench/w024-noise-baseline.json
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:47:58Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w024",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w024_ink.png",
|
| 15 |
+
"sha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327",
|
| 16 |
+
"width": 597,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.6321,
|
| 19 |
+
"alignTargetFile": "data/bench/w024_ink.png",
|
| 20 |
+
"alignTargetSha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "noise-seed24.png",
|
| 24 |
+
"sha256": "0d3e3dbf7db544bf85db22e4904c2170188eff607aa99dba92f106800bd7cdc4",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 597,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "baseline: uniform noise",
|
| 32 |
+
"notes": "numpy default_rng(24).random, 8 bit"
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"alignment": {
|
| 36 |
+
"method": "manual",
|
| 37 |
+
"target": "ink",
|
| 38 |
+
"tx": 0.0,
|
| 39 |
+
"ty": 0.0,
|
| 40 |
+
"sx": 1.0,
|
| 41 |
+
"sy": 1.0,
|
| 42 |
+
"ncc": -0.000154,
|
| 43 |
+
"coarseNcc": null,
|
| 44 |
+
"evals": 0,
|
| 45 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 46 |
+
},
|
| 47 |
+
"metrics": {
|
| 48 |
+
"nValid": 382080,
|
| 49 |
+
"validFraction": 1.0,
|
| 50 |
+
"pearson": -0.000154,
|
| 51 |
+
"blurSigmaPx": 6.0,
|
| 52 |
+
"blurSigmaUM": 622.9,
|
| 53 |
+
"pearsonMatchedBlur": 0.000533,
|
| 54 |
+
"thresholds": {
|
| 55 |
+
"ref": 0.5,
|
| 56 |
+
"pred": 0.5
|
| 57 |
+
},
|
| 58 |
+
"iou": 0.249425,
|
| 59 |
+
"dice": 0.399263,
|
| 60 |
+
"precision": 0.333244,
|
| 61 |
+
"recall": 0.497903,
|
| 62 |
+
"refPositiveFraction": 0.333917,
|
| 63 |
+
"predPositiveFraction": 0.498909,
|
| 64 |
+
"ap": 0.334221,
|
| 65 |
+
"bestF1": {
|
| 66 |
+
"f1": 0.500656,
|
| 67 |
+
"threshold": 0.0
|
| 68 |
+
},
|
| 69 |
+
"pr": [
|
| 70 |
+
{
|
| 71 |
+
"t": 1.0,
|
| 72 |
+
"precision": 1.0,
|
| 73 |
+
"recall": 0.0
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"t": 0.98,
|
| 77 |
+
"precision": 0.329879,
|
| 78 |
+
"recall": 0.019211
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"t": 0.96,
|
| 82 |
+
"precision": 0.335803,
|
| 83 |
+
"recall": 0.039425
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"t": 0.94,
|
| 87 |
+
"precision": 0.334846,
|
| 88 |
+
"recall": 0.058989
|
| 89 |
+
},
|
| 90 |
+
{
|
| 91 |
+
"t": 0.92,
|
| 92 |
+
"precision": 0.336679,
|
| 93 |
+
"recall": 0.079141
|
| 94 |
+
},
|
| 95 |
+
{
|
| 96 |
+
"t": 0.9,
|
| 97 |
+
"precision": 0.336394,
|
| 98 |
+
"recall": 0.098501
|
| 99 |
+
},
|
| 100 |
+
{
|
| 101 |
+
"t": 0.88,
|
| 102 |
+
"precision": 0.335897,
|
| 103 |
+
"recall": 0.118448
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"t": 0.86,
|
| 107 |
+
"precision": 0.337719,
|
| 108 |
+
"recall": 0.139211
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"t": 0.84,
|
| 112 |
+
"precision": 0.33856,
|
| 113 |
+
"recall": 0.159598
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"t": 0.82,
|
| 117 |
+
"precision": 0.337444,
|
| 118 |
+
"recall": 0.179107
|
| 119 |
+
},
|
| 120 |
+
{
|
| 121 |
+
"t": 0.8,
|
| 122 |
+
"precision": 0.335787,
|
| 123 |
+
"recall": 0.202339
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"t": 0.78,
|
| 127 |
+
"precision": 0.335545,
|
| 128 |
+
"recall": 0.22155
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"t": 0.76,
|
| 132 |
+
"precision": 0.335695,
|
| 133 |
+
"recall": 0.241411
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"t": 0.74,
|
| 137 |
+
"precision": 0.335913,
|
| 138 |
+
"recall": 0.261273
|
| 139 |
+
},
|
| 140 |
+
{
|
| 141 |
+
"t": 0.72,
|
| 142 |
+
"precision": 0.335316,
|
| 143 |
+
"recall": 0.280531
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"t": 0.7,
|
| 147 |
+
"precision": 0.33509,
|
| 148 |
+
"recall": 0.299954
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"t": 0.68,
|
| 152 |
+
"precision": 0.334654,
|
| 153 |
+
"recall": 0.31922
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"t": 0.66,
|
| 157 |
+
"precision": 0.334034,
|
| 158 |
+
"recall": 0.338713
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"t": 0.64,
|
| 162 |
+
"precision": 0.334197,
|
| 163 |
+
"recall": 0.3579
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"t": 0.62,
|
| 167 |
+
"precision": 0.333655,
|
| 168 |
+
"recall": 0.377229
|
| 169 |
+
},
|
| 170 |
+
{
|
| 171 |
+
"t": 0.6,
|
| 172 |
+
"precision": 0.333989,
|
| 173 |
+
"recall": 0.400618
|
| 174 |
+
},
|
| 175 |
+
{
|
| 176 |
+
"t": 0.58,
|
| 177 |
+
"precision": 0.333871,
|
| 178 |
+
"recall": 0.420079
|
| 179 |
+
},
|
| 180 |
+
{
|
| 181 |
+
"t": 0.56,
|
| 182 |
+
"precision": 0.3337,
|
| 183 |
+
"recall": 0.439651
|
| 184 |
+
},
|
| 185 |
+
{
|
| 186 |
+
"t": 0.54,
|
| 187 |
+
"precision": 0.333426,
|
| 188 |
+
"recall": 0.459089
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"t": 0.52,
|
| 192 |
+
"precision": 0.33325,
|
| 193 |
+
"recall": 0.478434
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"t": 0.5,
|
| 197 |
+
"precision": 0.333244,
|
| 198 |
+
"recall": 0.497903
|
| 199 |
+
},
|
| 200 |
+
{
|
| 201 |
+
"t": 0.48,
|
| 202 |
+
"precision": 0.332977,
|
| 203 |
+
"recall": 0.517052
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"t": 0.46,
|
| 207 |
+
"precision": 0.333027,
|
| 208 |
+
"recall": 0.5366
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"t": 0.44,
|
| 212 |
+
"precision": 0.333189,
|
| 213 |
+
"recall": 0.556649
|
| 214 |
+
},
|
| 215 |
+
{
|
| 216 |
+
"t": 0.42,
|
| 217 |
+
"precision": 0.333277,
|
| 218 |
+
"recall": 0.576519
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
"t": 0.4,
|
| 222 |
+
"precision": 0.333066,
|
| 223 |
+
"recall": 0.599375
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"t": 0.38,
|
| 227 |
+
"precision": 0.333098,
|
| 228 |
+
"recall": 0.618695
|
| 229 |
+
},
|
| 230 |
+
{
|
| 231 |
+
"t": 0.36,
|
| 232 |
+
"precision": 0.333016,
|
| 233 |
+
"recall": 0.638126
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"t": 0.34,
|
| 237 |
+
"precision": 0.333275,
|
| 238 |
+
"recall": 0.658701
|
| 239 |
+
},
|
| 240 |
+
{
|
| 241 |
+
"t": 0.32,
|
| 242 |
+
"precision": 0.333309,
|
| 243 |
+
"recall": 0.678264
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"t": 0.3,
|
| 247 |
+
"precision": 0.333411,
|
| 248 |
+
"recall": 0.697867
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"t": 0.28,
|
| 252 |
+
"precision": 0.333264,
|
| 253 |
+
"recall": 0.71689
|
| 254 |
+
},
|
| 255 |
+
{
|
| 256 |
+
"t": 0.26,
|
| 257 |
+
"precision": 0.333387,
|
| 258 |
+
"recall": 0.736462
|
| 259 |
+
},
|
| 260 |
+
{
|
| 261 |
+
"t": 0.24,
|
| 262 |
+
"precision": 0.333415,
|
| 263 |
+
"recall": 0.756378
|
| 264 |
+
},
|
| 265 |
+
{
|
| 266 |
+
"t": 0.22,
|
| 267 |
+
"precision": 0.33365,
|
| 268 |
+
"recall": 0.776412
|
| 269 |
+
},
|
| 270 |
+
{
|
| 271 |
+
"t": 0.2,
|
| 272 |
+
"precision": 0.333478,
|
| 273 |
+
"recall": 0.799393
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"t": 0.18,
|
| 277 |
+
"precision": 0.33341,
|
| 278 |
+
"recall": 0.818667
|
| 279 |
+
},
|
| 280 |
+
{
|
| 281 |
+
"t": 0.16,
|
| 282 |
+
"precision": 0.333651,
|
| 283 |
+
"recall": 0.83856
|
| 284 |
+
},
|
| 285 |
+
{
|
| 286 |
+
"t": 0.14,
|
| 287 |
+
"precision": 0.333609,
|
| 288 |
+
"recall": 0.858179
|
| 289 |
+
},
|
| 290 |
+
{
|
| 291 |
+
"t": 0.12,
|
| 292 |
+
"precision": 0.333629,
|
| 293 |
+
"recall": 0.877554
|
| 294 |
+
},
|
| 295 |
+
{
|
| 296 |
+
"t": 0.1,
|
| 297 |
+
"precision": 0.333554,
|
| 298 |
+
"recall": 0.897188
|
| 299 |
+
},
|
| 300 |
+
{
|
| 301 |
+
"t": 0.08,
|
| 302 |
+
"precision": 0.333709,
|
| 303 |
+
"recall": 0.917552
|
| 304 |
+
},
|
| 305 |
+
{
|
| 306 |
+
"t": 0.06,
|
| 307 |
+
"precision": 0.333833,
|
| 308 |
+
"recall": 0.937225
|
| 309 |
+
},
|
| 310 |
+
{
|
| 311 |
+
"t": 0.04,
|
| 312 |
+
"precision": 0.333941,
|
| 313 |
+
"recall": 0.957063
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"t": 0.02,
|
| 317 |
+
"precision": 0.333927,
|
| 318 |
+
"recall": 0.976752
|
| 319 |
+
},
|
| 320 |
+
{
|
| 321 |
+
"t": 0.0,
|
| 322 |
+
"precision": 0.333917,
|
| 323 |
+
"recall": 1.0
|
| 324 |
+
}
|
| 325 |
+
]
|
| 326 |
+
},
|
| 327 |
+
"annotations": [],
|
| 328 |
+
"credits": {
|
| 329 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 330 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 331 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 332 |
+
"codeLicense": "Apache-2.0"
|
| 333 |
+
},
|
| 334 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 335 |
+
}
|
docs/examples/bench/w025-ct-baseline.json
ADDED
|
@@ -0,0 +1,334 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:48:10Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w025",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w025_ink.png",
|
| 15 |
+
"sha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8",
|
| 16 |
+
"width": 598,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.3937,
|
| 19 |
+
"alignTargetFile": "data/bench/w025_ink.png",
|
| 20 |
+
"alignTargetSha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "w025_ct.png",
|
| 24 |
+
"sha256": "e30f050d29ccc83eaf9b29beafe66f0a47a35ac2a7f30dbd525f462c9ea325fa",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 598,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "baseline: X-ray intensity (same grid, no alignment needed)"
|
| 32 |
+
}
|
| 33 |
+
},
|
| 34 |
+
"alignment": {
|
| 35 |
+
"method": "manual",
|
| 36 |
+
"target": "ink",
|
| 37 |
+
"tx": 0.0,
|
| 38 |
+
"ty": 0.0,
|
| 39 |
+
"sx": 1.0,
|
| 40 |
+
"sy": 1.0,
|
| 41 |
+
"ncc": 0.359953,
|
| 42 |
+
"coarseNcc": null,
|
| 43 |
+
"evals": 0,
|
| 44 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 45 |
+
},
|
| 46 |
+
"metrics": {
|
| 47 |
+
"nValid": 382720,
|
| 48 |
+
"validFraction": 1.0,
|
| 49 |
+
"pearson": 0.359953,
|
| 50 |
+
"blurSigmaPx": 6.0,
|
| 51 |
+
"blurSigmaUM": 638.7,
|
| 52 |
+
"pearsonMatchedBlur": 0.437292,
|
| 53 |
+
"thresholds": {
|
| 54 |
+
"ref": 0.5,
|
| 55 |
+
"pred": 0.5
|
| 56 |
+
},
|
| 57 |
+
"iou": 0.204451,
|
| 58 |
+
"dice": 0.339493,
|
| 59 |
+
"precision": 0.236587,
|
| 60 |
+
"recall": 0.600833,
|
| 61 |
+
"refPositiveFraction": 0.155516,
|
| 62 |
+
"predPositiveFraction": 0.394947,
|
| 63 |
+
"ap": 0.242417,
|
| 64 |
+
"bestF1": {
|
| 65 |
+
"f1": 0.343571,
|
| 66 |
+
"threshold": 0.44
|
| 67 |
+
},
|
| 68 |
+
"pr": [
|
| 69 |
+
{
|
| 70 |
+
"t": 1.0,
|
| 71 |
+
"precision": 0.259459,
|
| 72 |
+
"recall": 0.000806
|
| 73 |
+
},
|
| 74 |
+
{
|
| 75 |
+
"t": 0.98,
|
| 76 |
+
"precision": 0.320594,
|
| 77 |
+
"recall": 0.002537
|
| 78 |
+
},
|
| 79 |
+
{
|
| 80 |
+
"t": 0.96,
|
| 81 |
+
"precision": 0.293318,
|
| 82 |
+
"recall": 0.004352
|
| 83 |
+
},
|
| 84 |
+
{
|
| 85 |
+
"t": 0.94,
|
| 86 |
+
"precision": 0.300469,
|
| 87 |
+
"recall": 0.007527
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"t": 0.92,
|
| 91 |
+
"precision": 0.283003,
|
| 92 |
+
"recall": 0.011526
|
| 93 |
+
},
|
| 94 |
+
{
|
| 95 |
+
"t": 0.9,
|
| 96 |
+
"precision": 0.287168,
|
| 97 |
+
"recall": 0.018011
|
| 98 |
+
},
|
| 99 |
+
{
|
| 100 |
+
"t": 0.88,
|
| 101 |
+
"precision": 0.287614,
|
| 102 |
+
"recall": 0.026529
|
| 103 |
+
},
|
| 104 |
+
{
|
| 105 |
+
"t": 0.86,
|
| 106 |
+
"precision": 0.286919,
|
| 107 |
+
"recall": 0.037148
|
| 108 |
+
},
|
| 109 |
+
{
|
| 110 |
+
"t": 0.84,
|
| 111 |
+
"precision": 0.287639,
|
| 112 |
+
"recall": 0.050942
|
| 113 |
+
},
|
| 114 |
+
{
|
| 115 |
+
"t": 0.82,
|
| 116 |
+
"precision": 0.287651,
|
| 117 |
+
"recall": 0.068449
|
| 118 |
+
},
|
| 119 |
+
{
|
| 120 |
+
"t": 0.8,
|
| 121 |
+
"precision": 0.282144,
|
| 122 |
+
"recall": 0.091433
|
| 123 |
+
},
|
| 124 |
+
{
|
| 125 |
+
"t": 0.78,
|
| 126 |
+
"precision": 0.27808,
|
| 127 |
+
"recall": 0.114266
|
| 128 |
+
},
|
| 129 |
+
{
|
| 130 |
+
"t": 0.76,
|
| 131 |
+
"precision": 0.275642,
|
| 132 |
+
"recall": 0.141283
|
| 133 |
+
},
|
| 134 |
+
{
|
| 135 |
+
"t": 0.74,
|
| 136 |
+
"precision": 0.273569,
|
| 137 |
+
"recall": 0.170349
|
| 138 |
+
},
|
| 139 |
+
{
|
| 140 |
+
"t": 0.72,
|
| 141 |
+
"precision": 0.271296,
|
| 142 |
+
"recall": 0.20296
|
| 143 |
+
},
|
| 144 |
+
{
|
| 145 |
+
"t": 0.7,
|
| 146 |
+
"precision": 0.268313,
|
| 147 |
+
"recall": 0.236563
|
| 148 |
+
},
|
| 149 |
+
{
|
| 150 |
+
"t": 0.68,
|
| 151 |
+
"precision": 0.265416,
|
| 152 |
+
"recall": 0.272266
|
| 153 |
+
},
|
| 154 |
+
{
|
| 155 |
+
"t": 0.66,
|
| 156 |
+
"precision": 0.26319,
|
| 157 |
+
"recall": 0.309935
|
| 158 |
+
},
|
| 159 |
+
{
|
| 160 |
+
"t": 0.64,
|
| 161 |
+
"precision": 0.259919,
|
| 162 |
+
"recall": 0.34725
|
| 163 |
+
},
|
| 164 |
+
{
|
| 165 |
+
"t": 0.62,
|
| 166 |
+
"precision": 0.257365,
|
| 167 |
+
"recall": 0.38544
|
| 168 |
+
},
|
| 169 |
+
{
|
| 170 |
+
"t": 0.6,
|
| 171 |
+
"precision": 0.253624,
|
| 172 |
+
"recall": 0.430652
|
| 173 |
+
},
|
| 174 |
+
{
|
| 175 |
+
"t": 0.58,
|
| 176 |
+
"precision": 0.249869,
|
| 177 |
+
"recall": 0.466422
|
| 178 |
+
},
|
| 179 |
+
{
|
| 180 |
+
"t": 0.56,
|
| 181 |
+
"precision": 0.24668,
|
| 182 |
+
"recall": 0.502159
|
| 183 |
+
},
|
| 184 |
+
{
|
| 185 |
+
"t": 0.54,
|
| 186 |
+
"precision": 0.243306,
|
| 187 |
+
"recall": 0.536501
|
| 188 |
+
},
|
| 189 |
+
{
|
| 190 |
+
"t": 0.52,
|
| 191 |
+
"precision": 0.239897,
|
| 192 |
+
"recall": 0.569129
|
| 193 |
+
},
|
| 194 |
+
{
|
| 195 |
+
"t": 0.5,
|
| 196 |
+
"precision": 0.236587,
|
| 197 |
+
"recall": 0.600833
|
| 198 |
+
},
|
| 199 |
+
{
|
| 200 |
+
"t": 0.48,
|
| 201 |
+
"precision": 0.233961,
|
| 202 |
+
"recall": 0.632302
|
| 203 |
+
},
|
| 204 |
+
{
|
| 205 |
+
"t": 0.46,
|
| 206 |
+
"precision": 0.231197,
|
| 207 |
+
"recall": 0.662108
|
| 208 |
+
},
|
| 209 |
+
{
|
| 210 |
+
"t": 0.44,
|
| 211 |
+
"precision": 0.228708,
|
| 212 |
+
"recall": 0.690217
|
| 213 |
+
},
|
| 214 |
+
{
|
| 215 |
+
"t": 0.42,
|
| 216 |
+
"precision": 0.225623,
|
| 217 |
+
"recall": 0.715251
|
| 218 |
+
},
|
| 219 |
+
{
|
| 220 |
+
"t": 0.4,
|
| 221 |
+
"precision": 0.222726,
|
| 222 |
+
"recall": 0.74472
|
| 223 |
+
},
|
| 224 |
+
{
|
| 225 |
+
"t": 0.38,
|
| 226 |
+
"precision": 0.220147,
|
| 227 |
+
"recall": 0.767167
|
| 228 |
+
},
|
| 229 |
+
{
|
| 230 |
+
"t": 0.36,
|
| 231 |
+
"precision": 0.217627,
|
| 232 |
+
"recall": 0.787631
|
| 233 |
+
},
|
| 234 |
+
{
|
| 235 |
+
"t": 0.34,
|
| 236 |
+
"precision": 0.215409,
|
| 237 |
+
"recall": 0.807238
|
| 238 |
+
},
|
| 239 |
+
{
|
| 240 |
+
"t": 0.32,
|
| 241 |
+
"precision": 0.213241,
|
| 242 |
+
"recall": 0.825467
|
| 243 |
+
},
|
| 244 |
+
{
|
| 245 |
+
"t": 0.3,
|
| 246 |
+
"precision": 0.211177,
|
| 247 |
+
"recall": 0.842319
|
| 248 |
+
},
|
| 249 |
+
{
|
| 250 |
+
"t": 0.28,
|
| 251 |
+
"precision": 0.209103,
|
| 252 |
+
"recall": 0.857172
|
| 253 |
+
},
|
| 254 |
+
{
|
| 255 |
+
"t": 0.26,
|
| 256 |
+
"precision": 0.20715,
|
| 257 |
+
"recall": 0.870781
|
| 258 |
+
},
|
| 259 |
+
{
|
| 260 |
+
"t": 0.24,
|
| 261 |
+
"precision": 0.20542,
|
| 262 |
+
"recall": 0.883936
|
| 263 |
+
},
|
| 264 |
+
{
|
| 265 |
+
"t": 0.22,
|
| 266 |
+
"precision": 0.203667,
|
| 267 |
+
"recall": 0.895193
|
| 268 |
+
},
|
| 269 |
+
{
|
| 270 |
+
"t": 0.2,
|
| 271 |
+
"precision": 0.201809,
|
| 272 |
+
"recall": 0.908701
|
| 273 |
+
},
|
| 274 |
+
{
|
| 275 |
+
"t": 0.18,
|
| 276 |
+
"precision": 0.200306,
|
| 277 |
+
"recall": 0.919017
|
| 278 |
+
},
|
| 279 |
+
{
|
| 280 |
+
"t": 0.16,
|
| 281 |
+
"precision": 0.198907,
|
| 282 |
+
"recall": 0.928443
|
| 283 |
+
},
|
| 284 |
+
{
|
| 285 |
+
"t": 0.14,
|
| 286 |
+
"precision": 0.197644,
|
| 287 |
+
"recall": 0.937953
|
| 288 |
+
},
|
| 289 |
+
{
|
| 290 |
+
"t": 0.12,
|
| 291 |
+
"precision": 0.196428,
|
| 292 |
+
"recall": 0.946588
|
| 293 |
+
},
|
| 294 |
+
{
|
| 295 |
+
"t": 0.1,
|
| 296 |
+
"precision": 0.195035,
|
| 297 |
+
"recall": 0.954065
|
| 298 |
+
},
|
| 299 |
+
{
|
| 300 |
+
"t": 0.08,
|
| 301 |
+
"precision": 0.193604,
|
| 302 |
+
"recall": 0.961038
|
| 303 |
+
},
|
| 304 |
+
{
|
| 305 |
+
"t": 0.06,
|
| 306 |
+
"precision": 0.192206,
|
| 307 |
+
"recall": 0.968111
|
| 308 |
+
},
|
| 309 |
+
{
|
| 310 |
+
"t": 0.04,
|
| 311 |
+
"precision": 0.191009,
|
| 312 |
+
"recall": 0.975554
|
| 313 |
+
},
|
| 314 |
+
{
|
| 315 |
+
"t": 0.02,
|
| 316 |
+
"precision": 0.189517,
|
| 317 |
+
"recall": 0.982661
|
| 318 |
+
},
|
| 319 |
+
{
|
| 320 |
+
"t": 0.0,
|
| 321 |
+
"precision": 0.155516,
|
| 322 |
+
"recall": 1.0
|
| 323 |
+
}
|
| 324 |
+
]
|
| 325 |
+
},
|
| 326 |
+
"annotations": [],
|
| 327 |
+
"credits": {
|
| 328 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 329 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 330 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 331 |
+
"codeLicense": "Apache-2.0"
|
| 332 |
+
},
|
| 333 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 334 |
+
}
|
docs/examples/bench/w025-ink9um.json
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:48:09Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w025",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w025_ink.png",
|
| 15 |
+
"sha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8",
|
| 16 |
+
"width": 598,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.3937,
|
| 19 |
+
"alignTargetFile": "data/bench/w025_ink.png",
|
| 20 |
+
"alignTargetSha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "w025_gpu.png",
|
| 24 |
+
"sha256": "c129d596d1fe215b2a5c671eff5143e938bd4c4d34ac1a6c7b9843d1f263f226",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 598,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan",
|
| 32 |
+
"url": "https://huggingface.co/scrollprize/ink_9um"
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"alignment": {
|
| 36 |
+
"method": "auto",
|
| 37 |
+
"target": "ink",
|
| 38 |
+
"tx": 0.531,
|
| 39 |
+
"ty": 0.594,
|
| 40 |
+
"sx": 1.00403,
|
| 41 |
+
"sy": 1.00007,
|
| 42 |
+
"ncc": 0.592898,
|
| 43 |
+
"coarseNcc": 0.749308,
|
| 44 |
+
"evals": 40350,
|
| 45 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 46 |
+
},
|
| 47 |
+
"metrics": {
|
| 48 |
+
"nValid": 382122,
|
| 49 |
+
"validFraction": 0.998438,
|
| 50 |
+
"pearson": 0.592898,
|
| 51 |
+
"blurSigmaPx": 2.0,
|
| 52 |
+
"blurSigmaUM": 212.9,
|
| 53 |
+
"pearsonMatchedBlur": 0.623003,
|
| 54 |
+
"thresholds": {
|
| 55 |
+
"ref": 0.5,
|
| 56 |
+
"pred": 0.5
|
| 57 |
+
},
|
| 58 |
+
"iou": 0.36052,
|
| 59 |
+
"dice": 0.529974,
|
| 60 |
+
"precision": 0.61375,
|
| 61 |
+
"recall": 0.466322,
|
| 62 |
+
"refPositiveFraction": 0.155759,
|
| 63 |
+
"predPositiveFraction": 0.118344,
|
| 64 |
+
"ap": 0.562828,
|
| 65 |
+
"bestF1": {
|
| 66 |
+
"f1": 0.547703,
|
| 67 |
+
"threshold": 0.34
|
| 68 |
+
},
|
| 69 |
+
"pr": [
|
| 70 |
+
{
|
| 71 |
+
"t": 1.0,
|
| 72 |
+
"precision": 0.968254,
|
| 73 |
+
"recall": 0.006149
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"t": 0.98,
|
| 77 |
+
"precision": 0.85798,
|
| 78 |
+
"recall": 0.110637
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"t": 0.96,
|
| 82 |
+
"precision": 0.843237,
|
| 83 |
+
"recall": 0.13972
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"t": 0.94,
|
| 87 |
+
"precision": 0.829066,
|
| 88 |
+
"recall": 0.161595
|
| 89 |
+
},
|
| 90 |
+
{
|
| 91 |
+
"t": 0.92,
|
| 92 |
+
"precision": 0.816653,
|
| 93 |
+
"recall": 0.180279
|
| 94 |
+
},
|
| 95 |
+
{
|
| 96 |
+
"t": 0.9,
|
| 97 |
+
"precision": 0.806396,
|
| 98 |
+
"recall": 0.197836
|
| 99 |
+
},
|
| 100 |
+
{
|
| 101 |
+
"t": 0.88,
|
| 102 |
+
"precision": 0.794865,
|
| 103 |
+
"recall": 0.214318
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"t": 0.86,
|
| 107 |
+
"precision": 0.783196,
|
| 108 |
+
"recall": 0.22991
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"t": 0.84,
|
| 112 |
+
"precision": 0.773132,
|
| 113 |
+
"recall": 0.243855
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"t": 0.82,
|
| 117 |
+
"precision": 0.763133,
|
| 118 |
+
"recall": 0.258472
|
| 119 |
+
},
|
| 120 |
+
{
|
| 121 |
+
"t": 0.8,
|
| 122 |
+
"precision": 0.755407,
|
| 123 |
+
"recall": 0.272888
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"t": 0.78,
|
| 127 |
+
"precision": 0.746371,
|
| 128 |
+
"recall": 0.285976
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"t": 0.76,
|
| 132 |
+
"precision": 0.737928,
|
| 133 |
+
"recall": 0.300408
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"t": 0.74,
|
| 137 |
+
"precision": 0.728903,
|
| 138 |
+
"recall": 0.314185
|
| 139 |
+
},
|
| 140 |
+
{
|
| 141 |
+
"t": 0.72,
|
| 142 |
+
"precision": 0.720131,
|
| 143 |
+
"recall": 0.327912
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"t": 0.7,
|
| 147 |
+
"precision": 0.71035,
|
| 148 |
+
"recall": 0.341336
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"t": 0.68,
|
| 152 |
+
"precision": 0.701014,
|
| 153 |
+
"recall": 0.354223
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"t": 0.66,
|
| 157 |
+
"precision": 0.691092,
|
| 158 |
+
"recall": 0.367311
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"t": 0.64,
|
| 162 |
+
"precision": 0.682133,
|
| 163 |
+
"recall": 0.380131
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"t": 0.62,
|
| 167 |
+
"precision": 0.671493,
|
| 168 |
+
"recall": 0.392715
|
| 169 |
+
},
|
| 170 |
+
{
|
| 171 |
+
"t": 0.6,
|
| 172 |
+
"precision": 0.661459,
|
| 173 |
+
"recall": 0.405484
|
| 174 |
+
},
|
| 175 |
+
{
|
| 176 |
+
"t": 0.58,
|
| 177 |
+
"precision": 0.652231,
|
| 178 |
+
"recall": 0.417262
|
| 179 |
+
},
|
| 180 |
+
{
|
| 181 |
+
"t": 0.56,
|
| 182 |
+
"precision": 0.64251,
|
| 183 |
+
"recall": 0.429157
|
| 184 |
+
},
|
| 185 |
+
{
|
| 186 |
+
"t": 0.54,
|
| 187 |
+
"precision": 0.632915,
|
| 188 |
+
"recall": 0.440985
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"t": 0.52,
|
| 192 |
+
"precision": 0.623577,
|
| 193 |
+
"recall": 0.45362
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"t": 0.5,
|
| 197 |
+
"precision": 0.61375,
|
| 198 |
+
"recall": 0.466322
|
| 199 |
+
},
|
| 200 |
+
{
|
| 201 |
+
"t": 0.48,
|
| 202 |
+
"precision": 0.604075,
|
| 203 |
+
"recall": 0.479696
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"t": 0.46,
|
| 207 |
+
"precision": 0.593219,
|
| 208 |
+
"recall": 0.492683
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"t": 0.44,
|
| 212 |
+
"precision": 0.581058,
|
| 213 |
+
"recall": 0.504763
|
| 214 |
+
},
|
| 215 |
+
{
|
| 216 |
+
"t": 0.42,
|
| 217 |
+
"precision": 0.570364,
|
| 218 |
+
"recall": 0.518271
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
"t": 0.4,
|
| 222 |
+
"precision": 0.560299,
|
| 223 |
+
"recall": 0.531427
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"t": 0.38,
|
| 227 |
+
"precision": 0.548314,
|
| 228 |
+
"recall": 0.544095
|
| 229 |
+
},
|
| 230 |
+
{
|
| 231 |
+
"t": 0.36,
|
| 232 |
+
"precision": 0.536368,
|
| 233 |
+
"recall": 0.557654
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"t": 0.34,
|
| 237 |
+
"precision": 0.525257,
|
| 238 |
+
"recall": 0.572153
|
| 239 |
+
},
|
| 240 |
+
{
|
| 241 |
+
"t": 0.32,
|
| 242 |
+
"precision": 0.513583,
|
| 243 |
+
"recall": 0.586351
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"t": 0.3,
|
| 247 |
+
"precision": 0.50199,
|
| 248 |
+
"recall": 0.601892
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"t": 0.28,
|
| 252 |
+
"precision": 0.490008,
|
| 253 |
+
"recall": 0.617131
|
| 254 |
+
},
|
| 255 |
+
{
|
| 256 |
+
"t": 0.26,
|
| 257 |
+
"precision": 0.477762,
|
| 258 |
+
"recall": 0.632773
|
| 259 |
+
},
|
| 260 |
+
{
|
| 261 |
+
"t": 0.24,
|
| 262 |
+
"precision": 0.465209,
|
| 263 |
+
"recall": 0.649944
|
| 264 |
+
},
|
| 265 |
+
{
|
| 266 |
+
"t": 0.22,
|
| 267 |
+
"precision": 0.451807,
|
| 268 |
+
"recall": 0.666426
|
| 269 |
+
},
|
| 270 |
+
{
|
| 271 |
+
"t": 0.2,
|
| 272 |
+
"precision": 0.438308,
|
| 273 |
+
"recall": 0.684823
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"t": 0.18,
|
| 277 |
+
"precision": 0.424135,
|
| 278 |
+
"recall": 0.702549
|
| 279 |
+
},
|
| 280 |
+
{
|
| 281 |
+
"t": 0.16,
|
| 282 |
+
"precision": 0.40955,
|
| 283 |
+
"recall": 0.722072
|
| 284 |
+
},
|
| 285 |
+
{
|
| 286 |
+
"t": 0.14,
|
| 287 |
+
"precision": 0.393667,
|
| 288 |
+
"recall": 0.743191
|
| 289 |
+
},
|
| 290 |
+
{
|
| 291 |
+
"t": 0.12,
|
| 292 |
+
"precision": 0.376616,
|
| 293 |
+
"recall": 0.764596
|
| 294 |
+
},
|
| 295 |
+
{
|
| 296 |
+
"t": 0.1,
|
| 297 |
+
"precision": 0.358264,
|
| 298 |
+
"recall": 0.7879
|
| 299 |
+
},
|
| 300 |
+
{
|
| 301 |
+
"t": 0.08,
|
| 302 |
+
"precision": 0.338383,
|
| 303 |
+
"recall": 0.815168
|
| 304 |
+
},
|
| 305 |
+
{
|
| 306 |
+
"t": 0.06,
|
| 307 |
+
"precision": 0.315728,
|
| 308 |
+
"recall": 0.844184
|
| 309 |
+
},
|
| 310 |
+
{
|
| 311 |
+
"t": 0.04,
|
| 312 |
+
"precision": 0.288123,
|
| 313 |
+
"recall": 0.879181
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"t": 0.02,
|
| 317 |
+
"precision": 0.252037,
|
| 318 |
+
"recall": 0.922294
|
| 319 |
+
},
|
| 320 |
+
{
|
| 321 |
+
"t": 0.0,
|
| 322 |
+
"precision": 0.155759,
|
| 323 |
+
"recall": 1.0
|
| 324 |
+
}
|
| 325 |
+
]
|
| 326 |
+
},
|
| 327 |
+
"annotations": [],
|
| 328 |
+
"credits": {
|
| 329 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 330 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 331 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 332 |
+
"codeLicense": "Apache-2.0"
|
| 333 |
+
},
|
| 334 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 335 |
+
}
|
docs/examples/bench/w025-noise-baseline.json
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:48:11Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w025",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w025_ink.png",
|
| 15 |
+
"sha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8",
|
| 16 |
+
"width": 598,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.3937,
|
| 19 |
+
"alignTargetFile": "data/bench/w025_ink.png",
|
| 20 |
+
"alignTargetSha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "noise-seed25.png",
|
| 24 |
+
"sha256": "cca11cae5b62ce76bbf1dc053614ab189878680be8021c962628d229ffceb639",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 598,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "baseline: uniform noise",
|
| 32 |
+
"notes": "numpy default_rng(25).random, 8 bit"
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"alignment": {
|
| 36 |
+
"method": "manual",
|
| 37 |
+
"target": "ink",
|
| 38 |
+
"tx": 0.0,
|
| 39 |
+
"ty": 0.0,
|
| 40 |
+
"sx": 1.0,
|
| 41 |
+
"sy": 1.0,
|
| 42 |
+
"ncc": 0.000355,
|
| 43 |
+
"coarseNcc": null,
|
| 44 |
+
"evals": 0,
|
| 45 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 46 |
+
},
|
| 47 |
+
"metrics": {
|
| 48 |
+
"nValid": 382720,
|
| 49 |
+
"validFraction": 1.0,
|
| 50 |
+
"pearson": 0.000355,
|
| 51 |
+
"blurSigmaPx": 6.0,
|
| 52 |
+
"blurSigmaUM": 638.7,
|
| 53 |
+
"pearsonMatchedBlur": 0.000806,
|
| 54 |
+
"thresholds": {
|
| 55 |
+
"ref": 0.5,
|
| 56 |
+
"pred": 0.5
|
| 57 |
+
},
|
| 58 |
+
"iou": 0.134334,
|
| 59 |
+
"dice": 0.236851,
|
| 60 |
+
"precision": 0.155441,
|
| 61 |
+
"recall": 0.497303,
|
| 62 |
+
"refPositiveFraction": 0.155516,
|
| 63 |
+
"predPositiveFraction": 0.497541,
|
| 64 |
+
"ap": 0.156205,
|
| 65 |
+
"bestF1": {
|
| 66 |
+
"f1": 0.269171,
|
| 67 |
+
"threshold": 0.0
|
| 68 |
+
},
|
| 69 |
+
"pr": [
|
| 70 |
+
{
|
| 71 |
+
"t": 1.0,
|
| 72 |
+
"precision": 1.0,
|
| 73 |
+
"recall": 0.0
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"t": 0.98,
|
| 77 |
+
"precision": 0.167365,
|
| 78 |
+
"recall": 0.021489
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"t": 0.96,
|
| 82 |
+
"precision": 0.161275,
|
| 83 |
+
"recall": 0.040794
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"t": 0.94,
|
| 87 |
+
"precision": 0.159883,
|
| 88 |
+
"recall": 0.060451
|
| 89 |
+
},
|
| 90 |
+
{
|
| 91 |
+
"t": 0.92,
|
| 92 |
+
"precision": 0.159164,
|
| 93 |
+
"recall": 0.080378
|
| 94 |
+
},
|
| 95 |
+
{
|
| 96 |
+
"t": 0.9,
|
| 97 |
+
"precision": 0.159305,
|
| 98 |
+
"recall": 0.10022
|
| 99 |
+
},
|
| 100 |
+
{
|
| 101 |
+
"t": 0.88,
|
| 102 |
+
"precision": 0.158096,
|
| 103 |
+
"recall": 0.119323
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"t": 0.86,
|
| 107 |
+
"precision": 0.157868,
|
| 108 |
+
"recall": 0.138964
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"t": 0.84,
|
| 112 |
+
"precision": 0.157373,
|
| 113 |
+
"recall": 0.15847
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"t": 0.82,
|
| 117 |
+
"precision": 0.156813,
|
| 118 |
+
"recall": 0.177893
|
| 119 |
+
},
|
| 120 |
+
{
|
| 121 |
+
"t": 0.8,
|
| 122 |
+
"precision": 0.155958,
|
| 123 |
+
"recall": 0.200272
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"t": 0.78,
|
| 127 |
+
"precision": 0.155983,
|
| 128 |
+
"recall": 0.219829
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"t": 0.76,
|
| 132 |
+
"precision": 0.155989,
|
| 133 |
+
"recall": 0.239436
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"t": 0.74,
|
| 137 |
+
"precision": 0.155477,
|
| 138 |
+
"recall": 0.258186
|
| 139 |
+
},
|
| 140 |
+
{
|
| 141 |
+
"t": 0.72,
|
| 142 |
+
"precision": 0.155227,
|
| 143 |
+
"recall": 0.277407
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"t": 0.7,
|
| 147 |
+
"precision": 0.15537,
|
| 148 |
+
"recall": 0.297233
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"t": 0.68,
|
| 152 |
+
"precision": 0.155284,
|
| 153 |
+
"recall": 0.316605
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"t": 0.66,
|
| 157 |
+
"precision": 0.154942,
|
| 158 |
+
"recall": 0.335607
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"t": 0.64,
|
| 162 |
+
"precision": 0.155212,
|
| 163 |
+
"recall": 0.355769
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"t": 0.62,
|
| 167 |
+
"precision": 0.155133,
|
| 168 |
+
"recall": 0.374603
|
| 169 |
+
},
|
| 170 |
+
{
|
| 171 |
+
"t": 0.6,
|
| 172 |
+
"precision": 0.155409,
|
| 173 |
+
"recall": 0.399099
|
| 174 |
+
},
|
| 175 |
+
{
|
| 176 |
+
"t": 0.58,
|
| 177 |
+
"precision": 0.155281,
|
| 178 |
+
"recall": 0.418505
|
| 179 |
+
},
|
| 180 |
+
{
|
| 181 |
+
"t": 0.56,
|
| 182 |
+
"precision": 0.155247,
|
| 183 |
+
"recall": 0.438028
|
| 184 |
+
},
|
| 185 |
+
{
|
| 186 |
+
"t": 0.54,
|
| 187 |
+
"precision": 0.155428,
|
| 188 |
+
"recall": 0.458055
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"t": 0.52,
|
| 192 |
+
"precision": 0.155539,
|
| 193 |
+
"recall": 0.478167
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"t": 0.5,
|
| 197 |
+
"precision": 0.155441,
|
| 198 |
+
"recall": 0.497303
|
| 199 |
+
},
|
| 200 |
+
{
|
| 201 |
+
"t": 0.48,
|
| 202 |
+
"precision": 0.15554,
|
| 203 |
+
"recall": 0.517146
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"t": 0.46,
|
| 207 |
+
"precision": 0.155535,
|
| 208 |
+
"recall": 0.536887
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"t": 0.44,
|
| 212 |
+
"precision": 0.155541,
|
| 213 |
+
"recall": 0.556579
|
| 214 |
+
},
|
| 215 |
+
{
|
| 216 |
+
"t": 0.42,
|
| 217 |
+
"precision": 0.155464,
|
| 218 |
+
"recall": 0.575883
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
"t": 0.4,
|
| 222 |
+
"precision": 0.155432,
|
| 223 |
+
"recall": 0.59912
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"t": 0.38,
|
| 227 |
+
"precision": 0.155575,
|
| 228 |
+
"recall": 0.6195
|
| 229 |
+
},
|
| 230 |
+
{
|
| 231 |
+
"t": 0.36,
|
| 232 |
+
"precision": 0.155318,
|
| 233 |
+
"recall": 0.638065
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"t": 0.34,
|
| 237 |
+
"precision": 0.155339,
|
| 238 |
+
"recall": 0.657286
|
| 239 |
+
},
|
| 240 |
+
{
|
| 241 |
+
"t": 0.32,
|
| 242 |
+
"precision": 0.155334,
|
| 243 |
+
"recall": 0.676742
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"t": 0.3,
|
| 247 |
+
"precision": 0.155558,
|
| 248 |
+
"recall": 0.697475
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"t": 0.28,
|
| 252 |
+
"precision": 0.155358,
|
| 253 |
+
"recall": 0.716275
|
| 254 |
+
},
|
| 255 |
+
{
|
| 256 |
+
"t": 0.26,
|
| 257 |
+
"precision": 0.155151,
|
| 258 |
+
"recall": 0.734908
|
| 259 |
+
},
|
| 260 |
+
{
|
| 261 |
+
"t": 0.24,
|
| 262 |
+
"precision": 0.155166,
|
| 263 |
+
"recall": 0.754633
|
| 264 |
+
},
|
| 265 |
+
{
|
| 266 |
+
"t": 0.22,
|
| 267 |
+
"precision": 0.155302,
|
| 268 |
+
"recall": 0.77466
|
| 269 |
+
},
|
| 270 |
+
{
|
| 271 |
+
"t": 0.2,
|
| 272 |
+
"precision": 0.155375,
|
| 273 |
+
"recall": 0.798501
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"t": 0.18,
|
| 277 |
+
"precision": 0.155477,
|
| 278 |
+
"recall": 0.818982
|
| 279 |
+
},
|
| 280 |
+
{
|
| 281 |
+
"t": 0.16,
|
| 282 |
+
"precision": 0.155276,
|
| 283 |
+
"recall": 0.837296
|
| 284 |
+
},
|
| 285 |
+
{
|
| 286 |
+
"t": 0.14,
|
| 287 |
+
"precision": 0.155432,
|
| 288 |
+
"recall": 0.857693
|
| 289 |
+
},
|
| 290 |
+
{
|
| 291 |
+
"t": 0.12,
|
| 292 |
+
"precision": 0.15544,
|
| 293 |
+
"recall": 0.877468
|
| 294 |
+
},
|
| 295 |
+
{
|
| 296 |
+
"t": 0.1,
|
| 297 |
+
"precision": 0.15553,
|
| 298 |
+
"recall": 0.897327
|
| 299 |
+
},
|
| 300 |
+
{
|
| 301 |
+
"t": 0.08,
|
| 302 |
+
"precision": 0.155506,
|
| 303 |
+
"recall": 0.917321
|
| 304 |
+
},
|
| 305 |
+
{
|
| 306 |
+
"t": 0.06,
|
| 307 |
+
"precision": 0.155471,
|
| 308 |
+
"recall": 0.936961
|
| 309 |
+
},
|
| 310 |
+
{
|
| 311 |
+
"t": 0.04,
|
| 312 |
+
"precision": 0.155464,
|
| 313 |
+
"recall": 0.956232
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"t": 0.02,
|
| 317 |
+
"precision": 0.155598,
|
| 318 |
+
"recall": 0.976512
|
| 319 |
+
},
|
| 320 |
+
{
|
| 321 |
+
"t": 0.0,
|
| 322 |
+
"precision": 0.155516,
|
| 323 |
+
"recall": 1.0
|
| 324 |
+
}
|
| 325 |
+
]
|
| 326 |
+
},
|
| 327 |
+
"annotations": [],
|
| 328 |
+
"credits": {
|
| 329 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 330 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 331 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 332 |
+
"codeLicense": "Apache-2.0"
|
| 333 |
+
},
|
| 334 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 335 |
+
}
|
docs/examples/bench/w026-ct-baseline.json
ADDED
|
@@ -0,0 +1,334 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:48:29Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w026",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w026_ink.png",
|
| 15 |
+
"sha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791",
|
| 16 |
+
"width": 629,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.1785,
|
| 19 |
+
"alignTargetFile": "data/bench/w026_ink.png",
|
| 20 |
+
"alignTargetSha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "w026_ct.png",
|
| 24 |
+
"sha256": "d005b08b88e31ad0cc24bfbbff91041e8abad6275d39cbb243dddfe78b46f3f0",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 629,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "baseline: X-ray intensity (same grid, no alignment needed)"
|
| 32 |
+
}
|
| 33 |
+
},
|
| 34 |
+
"alignment": {
|
| 35 |
+
"method": "manual",
|
| 36 |
+
"target": "ink",
|
| 37 |
+
"tx": 0.0,
|
| 38 |
+
"ty": 0.0,
|
| 39 |
+
"sx": 1.0,
|
| 40 |
+
"sy": 1.0,
|
| 41 |
+
"ncc": 0.278906,
|
| 42 |
+
"coarseNcc": null,
|
| 43 |
+
"evals": 0,
|
| 44 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 45 |
+
},
|
| 46 |
+
"metrics": {
|
| 47 |
+
"nValid": 402560,
|
| 48 |
+
"validFraction": 1.0,
|
| 49 |
+
"pearson": 0.278906,
|
| 50 |
+
"blurSigmaPx": 6.0,
|
| 51 |
+
"blurSigmaUM": 653.7,
|
| 52 |
+
"pearsonMatchedBlur": 0.376576,
|
| 53 |
+
"thresholds": {
|
| 54 |
+
"ref": 0.5,
|
| 55 |
+
"pred": 0.5
|
| 56 |
+
},
|
| 57 |
+
"iou": 0.064599,
|
| 58 |
+
"dice": 0.121359,
|
| 59 |
+
"precision": 0.068329,
|
| 60 |
+
"recall": 0.542037,
|
| 61 |
+
"refPositiveFraction": 0.048634,
|
| 62 |
+
"predPositiveFraction": 0.385801,
|
| 63 |
+
"ap": 0.072337,
|
| 64 |
+
"bestF1": {
|
| 65 |
+
"f1": 0.122104,
|
| 66 |
+
"threshold": 0.6
|
| 67 |
+
},
|
| 68 |
+
"pr": [
|
| 69 |
+
{
|
| 70 |
+
"t": 1.0,
|
| 71 |
+
"precision": 0.225806,
|
| 72 |
+
"recall": 0.002503
|
| 73 |
+
},
|
| 74 |
+
{
|
| 75 |
+
"t": 0.98,
|
| 76 |
+
"precision": 0.170956,
|
| 77 |
+
"recall": 0.00475
|
| 78 |
+
},
|
| 79 |
+
{
|
| 80 |
+
"t": 0.96,
|
| 81 |
+
"precision": 0.152941,
|
| 82 |
+
"recall": 0.007968
|
| 83 |
+
},
|
| 84 |
+
{
|
| 85 |
+
"t": 0.94,
|
| 86 |
+
"precision": 0.142022,
|
| 87 |
+
"recall": 0.012412
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"t": 0.92,
|
| 91 |
+
"precision": 0.133232,
|
| 92 |
+
"recall": 0.017979
|
| 93 |
+
},
|
| 94 |
+
{
|
| 95 |
+
"t": 0.9,
|
| 96 |
+
"precision": 0.125635,
|
| 97 |
+
"recall": 0.025283
|
| 98 |
+
},
|
| 99 |
+
{
|
| 100 |
+
"t": 0.88,
|
| 101 |
+
"precision": 0.112322,
|
| 102 |
+
"recall": 0.032639
|
| 103 |
+
},
|
| 104 |
+
{
|
| 105 |
+
"t": 0.86,
|
| 106 |
+
"precision": 0.106697,
|
| 107 |
+
"recall": 0.04362
|
| 108 |
+
},
|
| 109 |
+
{
|
| 110 |
+
"t": 0.84,
|
| 111 |
+
"precision": 0.101162,
|
| 112 |
+
"recall": 0.056901
|
| 113 |
+
},
|
| 114 |
+
{
|
| 115 |
+
"t": 0.82,
|
| 116 |
+
"precision": 0.095673,
|
| 117 |
+
"recall": 0.072173
|
| 118 |
+
},
|
| 119 |
+
{
|
| 120 |
+
"t": 0.8,
|
| 121 |
+
"precision": 0.090627,
|
| 122 |
+
"recall": 0.094034
|
| 123 |
+
},
|
| 124 |
+
{
|
| 125 |
+
"t": 0.78,
|
| 126 |
+
"precision": 0.086661,
|
| 127 |
+
"recall": 0.113393
|
| 128 |
+
},
|
| 129 |
+
{
|
| 130 |
+
"t": 0.76,
|
| 131 |
+
"precision": 0.084308,
|
| 132 |
+
"recall": 0.13699
|
| 133 |
+
},
|
| 134 |
+
{
|
| 135 |
+
"t": 0.74,
|
| 136 |
+
"precision": 0.082699,
|
| 137 |
+
"recall": 0.163142
|
| 138 |
+
},
|
| 139 |
+
{
|
| 140 |
+
"t": 0.72,
|
| 141 |
+
"precision": 0.080023,
|
| 142 |
+
"recall": 0.188477
|
| 143 |
+
},
|
| 144 |
+
{
|
| 145 |
+
"t": 0.7,
|
| 146 |
+
"precision": 0.079188,
|
| 147 |
+
"recall": 0.218868
|
| 148 |
+
},
|
| 149 |
+
{
|
| 150 |
+
"t": 0.68,
|
| 151 |
+
"precision": 0.07806,
|
| 152 |
+
"recall": 0.251149
|
| 153 |
+
},
|
| 154 |
+
{
|
| 155 |
+
"t": 0.66,
|
| 156 |
+
"precision": 0.076513,
|
| 157 |
+
"recall": 0.281285
|
| 158 |
+
},
|
| 159 |
+
{
|
| 160 |
+
"t": 0.64,
|
| 161 |
+
"precision": 0.075215,
|
| 162 |
+
"recall": 0.313004
|
| 163 |
+
},
|
| 164 |
+
{
|
| 165 |
+
"t": 0.62,
|
| 166 |
+
"precision": 0.073916,
|
| 167 |
+
"recall": 0.343855
|
| 168 |
+
},
|
| 169 |
+
{
|
| 170 |
+
"t": 0.6,
|
| 171 |
+
"precision": 0.07264,
|
| 172 |
+
"recall": 0.382726
|
| 173 |
+
},
|
| 174 |
+
{
|
| 175 |
+
"t": 0.58,
|
| 176 |
+
"precision": 0.071338,
|
| 177 |
+
"recall": 0.413525
|
| 178 |
+
},
|
| 179 |
+
{
|
| 180 |
+
"t": 0.56,
|
| 181 |
+
"precision": 0.070653,
|
| 182 |
+
"recall": 0.44693
|
| 183 |
+
},
|
| 184 |
+
{
|
| 185 |
+
"t": 0.54,
|
| 186 |
+
"precision": 0.069849,
|
| 187 |
+
"recall": 0.479416
|
| 188 |
+
},
|
| 189 |
+
{
|
| 190 |
+
"t": 0.52,
|
| 191 |
+
"precision": 0.069073,
|
| 192 |
+
"recall": 0.510982
|
| 193 |
+
},
|
| 194 |
+
{
|
| 195 |
+
"t": 0.5,
|
| 196 |
+
"precision": 0.068329,
|
| 197 |
+
"recall": 0.542037
|
| 198 |
+
},
|
| 199 |
+
{
|
| 200 |
+
"t": 0.48,
|
| 201 |
+
"precision": 0.067272,
|
| 202 |
+
"recall": 0.569517
|
| 203 |
+
},
|
| 204 |
+
{
|
| 205 |
+
"t": 0.46,
|
| 206 |
+
"precision": 0.066605,
|
| 207 |
+
"recall": 0.598325
|
| 208 |
+
},
|
| 209 |
+
{
|
| 210 |
+
"t": 0.44,
|
| 211 |
+
"precision": 0.065942,
|
| 212 |
+
"recall": 0.625396
|
| 213 |
+
},
|
| 214 |
+
{
|
| 215 |
+
"t": 0.42,
|
| 216 |
+
"precision": 0.065164,
|
| 217 |
+
"recall": 0.650577
|
| 218 |
+
},
|
| 219 |
+
{
|
| 220 |
+
"t": 0.4,
|
| 221 |
+
"precision": 0.064333,
|
| 222 |
+
"recall": 0.679436
|
| 223 |
+
},
|
| 224 |
+
{
|
| 225 |
+
"t": 0.38,
|
| 226 |
+
"precision": 0.063741,
|
| 227 |
+
"recall": 0.703136
|
| 228 |
+
},
|
| 229 |
+
{
|
| 230 |
+
"t": 0.36,
|
| 231 |
+
"precision": 0.063055,
|
| 232 |
+
"recall": 0.723669
|
| 233 |
+
},
|
| 234 |
+
{
|
| 235 |
+
"t": 0.34,
|
| 236 |
+
"precision": 0.062479,
|
| 237 |
+
"recall": 0.744305
|
| 238 |
+
},
|
| 239 |
+
{
|
| 240 |
+
"t": 0.32,
|
| 241 |
+
"precision": 0.062059,
|
| 242 |
+
"recall": 0.765451
|
| 243 |
+
},
|
| 244 |
+
{
|
| 245 |
+
"t": 0.3,
|
| 246 |
+
"precision": 0.061548,
|
| 247 |
+
"recall": 0.783686
|
| 248 |
+
},
|
| 249 |
+
{
|
| 250 |
+
"t": 0.28,
|
| 251 |
+
"precision": 0.06097,
|
| 252 |
+
"recall": 0.800184
|
| 253 |
+
},
|
| 254 |
+
{
|
| 255 |
+
"t": 0.26,
|
| 256 |
+
"precision": 0.060372,
|
| 257 |
+
"recall": 0.814588
|
| 258 |
+
},
|
| 259 |
+
{
|
| 260 |
+
"t": 0.24,
|
| 261 |
+
"precision": 0.059949,
|
| 262 |
+
"recall": 0.830064
|
| 263 |
+
},
|
| 264 |
+
{
|
| 265 |
+
"t": 0.22,
|
| 266 |
+
"precision": 0.059607,
|
| 267 |
+
"recall": 0.845541
|
| 268 |
+
},
|
| 269 |
+
{
|
| 270 |
+
"t": 0.2,
|
| 271 |
+
"precision": 0.059204,
|
| 272 |
+
"recall": 0.862192
|
| 273 |
+
},
|
| 274 |
+
{
|
| 275 |
+
"t": 0.18,
|
| 276 |
+
"precision": 0.058856,
|
| 277 |
+
"recall": 0.874962
|
| 278 |
+
},
|
| 279 |
+
{
|
| 280 |
+
"t": 0.16,
|
| 281 |
+
"precision": 0.058651,
|
| 282 |
+
"recall": 0.888855
|
| 283 |
+
},
|
| 284 |
+
{
|
| 285 |
+
"t": 0.14,
|
| 286 |
+
"precision": 0.058371,
|
| 287 |
+
"recall": 0.900756
|
| 288 |
+
},
|
| 289 |
+
{
|
| 290 |
+
"t": 0.12,
|
| 291 |
+
"precision": 0.058127,
|
| 292 |
+
"recall": 0.913474
|
| 293 |
+
},
|
| 294 |
+
{
|
| 295 |
+
"t": 0.1,
|
| 296 |
+
"precision": 0.05797,
|
| 297 |
+
"recall": 0.927725
|
| 298 |
+
},
|
| 299 |
+
{
|
| 300 |
+
"t": 0.08,
|
| 301 |
+
"precision": 0.057776,
|
| 302 |
+
"recall": 0.941618
|
| 303 |
+
},
|
| 304 |
+
{
|
| 305 |
+
"t": 0.06,
|
| 306 |
+
"precision": 0.057623,
|
| 307 |
+
"recall": 0.955562
|
| 308 |
+
},
|
| 309 |
+
{
|
| 310 |
+
"t": 0.04,
|
| 311 |
+
"precision": 0.057498,
|
| 312 |
+
"recall": 0.969609
|
| 313 |
+
},
|
| 314 |
+
{
|
| 315 |
+
"t": 0.02,
|
| 316 |
+
"precision": 0.057247,
|
| 317 |
+
"recall": 0.981714
|
| 318 |
+
},
|
| 319 |
+
{
|
| 320 |
+
"t": 0.0,
|
| 321 |
+
"precision": 0.048634,
|
| 322 |
+
"recall": 1.0
|
| 323 |
+
}
|
| 324 |
+
]
|
| 325 |
+
},
|
| 326 |
+
"annotations": [],
|
| 327 |
+
"credits": {
|
| 328 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 329 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 330 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 331 |
+
"codeLicense": "Apache-2.0"
|
| 332 |
+
},
|
| 333 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 334 |
+
}
|
docs/examples/bench/w026-ink9um.json
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:48:28Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w026",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w026_ink.png",
|
| 15 |
+
"sha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791",
|
| 16 |
+
"width": 629,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.1785,
|
| 19 |
+
"alignTargetFile": "data/bench/w026_ink.png",
|
| 20 |
+
"alignTargetSha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "w026_gpu.png",
|
| 24 |
+
"sha256": "262b1fb5329f95649d5d69da657ac3b111b9ba05c2151859c167efd85c700079",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 629,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan",
|
| 32 |
+
"url": "https://huggingface.co/scrollprize/ink_9um"
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"alignment": {
|
| 36 |
+
"method": "auto",
|
| 37 |
+
"target": "ink",
|
| 38 |
+
"tx": 0.531,
|
| 39 |
+
"ty": 0.969,
|
| 40 |
+
"sx": 0.99987,
|
| 41 |
+
"sy": 1.00178,
|
| 42 |
+
"ncc": 0.29816,
|
| 43 |
+
"coarseNcc": 0.426747,
|
| 44 |
+
"evals": 42527,
|
| 45 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 46 |
+
},
|
| 47 |
+
"metrics": {
|
| 48 |
+
"nValid": 401292,
|
| 49 |
+
"validFraction": 0.99685,
|
| 50 |
+
"pearson": 0.29816,
|
| 51 |
+
"blurSigmaPx": 2.0,
|
| 52 |
+
"blurSigmaUM": 217.9,
|
| 53 |
+
"pearsonMatchedBlur": 0.324342,
|
| 54 |
+
"thresholds": {
|
| 55 |
+
"ref": 0.5,
|
| 56 |
+
"pred": 0.5
|
| 57 |
+
},
|
| 58 |
+
"iou": 0.117089,
|
| 59 |
+
"dice": 0.209633,
|
| 60 |
+
"precision": 0.207984,
|
| 61 |
+
"recall": 0.211309,
|
| 62 |
+
"refPositiveFraction": 0.048787,
|
| 63 |
+
"predPositiveFraction": 0.049567,
|
| 64 |
+
"ap": 0.151895,
|
| 65 |
+
"bestF1": {
|
| 66 |
+
"f1": 0.22795,
|
| 67 |
+
"threshold": 0.32
|
| 68 |
+
},
|
| 69 |
+
"pr": [
|
| 70 |
+
{
|
| 71 |
+
"t": 1.0,
|
| 72 |
+
"precision": 0.5,
|
| 73 |
+
"recall": 0.000306
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"t": 0.98,
|
| 77 |
+
"precision": 0.409463,
|
| 78 |
+
"recall": 0.022985
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"t": 0.96,
|
| 82 |
+
"precision": 0.379817,
|
| 83 |
+
"recall": 0.031719
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"t": 0.94,
|
| 87 |
+
"precision": 0.374391,
|
| 88 |
+
"recall": 0.039279
|
| 89 |
+
},
|
| 90 |
+
{
|
| 91 |
+
"t": 0.92,
|
| 92 |
+
"precision": 0.361761,
|
| 93 |
+
"recall": 0.046583
|
| 94 |
+
},
|
| 95 |
+
{
|
| 96 |
+
"t": 0.9,
|
| 97 |
+
"precision": 0.346892,
|
| 98 |
+
"recall": 0.053581
|
| 99 |
+
},
|
| 100 |
+
{
|
| 101 |
+
"t": 0.88,
|
| 102 |
+
"precision": 0.337329,
|
| 103 |
+
"recall": 0.060374
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"t": 0.86,
|
| 107 |
+
"precision": 0.32753,
|
| 108 |
+
"recall": 0.067269
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"t": 0.84,
|
| 112 |
+
"precision": 0.31892,
|
| 113 |
+
"recall": 0.074216
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"t": 0.82,
|
| 117 |
+
"precision": 0.311369,
|
| 118 |
+
"recall": 0.080856
|
| 119 |
+
},
|
| 120 |
+
{
|
| 121 |
+
"t": 0.8,
|
| 122 |
+
"precision": 0.303041,
|
| 123 |
+
"recall": 0.087547
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"t": 0.78,
|
| 127 |
+
"precision": 0.297678,
|
| 128 |
+
"recall": 0.094954
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"t": 0.76,
|
| 132 |
+
"precision": 0.290994,
|
| 133 |
+
"recall": 0.102155
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"t": 0.74,
|
| 137 |
+
"precision": 0.280074,
|
| 138 |
+
"recall": 0.107978
|
| 139 |
+
},
|
| 140 |
+
{
|
| 141 |
+
"t": 0.72,
|
| 142 |
+
"precision": 0.272837,
|
| 143 |
+
"recall": 0.11564
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"t": 0.7,
|
| 147 |
+
"precision": 0.267072,
|
| 148 |
+
"recall": 0.123251
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"t": 0.68,
|
| 152 |
+
"precision": 0.258982,
|
| 153 |
+
"recall": 0.130708
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"t": 0.66,
|
| 157 |
+
"precision": 0.252205,
|
| 158 |
+
"recall": 0.138778
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"t": 0.64,
|
| 162 |
+
"precision": 0.242978,
|
| 163 |
+
"recall": 0.145367
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"t": 0.62,
|
| 167 |
+
"precision": 0.235898,
|
| 168 |
+
"recall": 0.153795
|
| 169 |
+
},
|
| 170 |
+
{
|
| 171 |
+
"t": 0.6,
|
| 172 |
+
"precision": 0.23039,
|
| 173 |
+
"recall": 0.162172
|
| 174 |
+
},
|
| 175 |
+
{
|
| 176 |
+
"t": 0.58,
|
| 177 |
+
"precision": 0.224914,
|
| 178 |
+
"recall": 0.170702
|
| 179 |
+
},
|
| 180 |
+
{
|
| 181 |
+
"t": 0.56,
|
| 182 |
+
"precision": 0.221309,
|
| 183 |
+
"recall": 0.180253
|
| 184 |
+
},
|
| 185 |
+
{
|
| 186 |
+
"t": 0.54,
|
| 187 |
+
"precision": 0.216931,
|
| 188 |
+
"recall": 0.190571
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"t": 0.52,
|
| 192 |
+
"precision": 0.212882,
|
| 193 |
+
"recall": 0.200889
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"t": 0.5,
|
| 197 |
+
"precision": 0.207984,
|
| 198 |
+
"recall": 0.211309
|
| 199 |
+
},
|
| 200 |
+
{
|
| 201 |
+
"t": 0.48,
|
| 202 |
+
"precision": 0.203289,
|
| 203 |
+
"recall": 0.221626
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"t": 0.46,
|
| 207 |
+
"precision": 0.19894,
|
| 208 |
+
"recall": 0.233987
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"t": 0.44,
|
| 212 |
+
"precision": 0.194885,
|
| 213 |
+
"recall": 0.247931
|
| 214 |
+
},
|
| 215 |
+
{
|
| 216 |
+
"t": 0.42,
|
| 217 |
+
"precision": 0.190327,
|
| 218 |
+
"recall": 0.260701
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
"t": 0.4,
|
| 222 |
+
"precision": 0.185865,
|
| 223 |
+
"recall": 0.274287
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"t": 0.38,
|
| 227 |
+
"precision": 0.182261,
|
| 228 |
+
"recall": 0.288845
|
| 229 |
+
},
|
| 230 |
+
{
|
| 231 |
+
"t": 0.36,
|
| 232 |
+
"precision": 0.17886,
|
| 233 |
+
"recall": 0.305445
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"t": 0.34,
|
| 237 |
+
"precision": 0.175594,
|
| 238 |
+
"recall": 0.323169
|
| 239 |
+
},
|
| 240 |
+
{
|
| 241 |
+
"t": 0.32,
|
| 242 |
+
"precision": 0.171351,
|
| 243 |
+
"recall": 0.340382
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"t": 0.3,
|
| 247 |
+
"precision": 0.166512,
|
| 248 |
+
"recall": 0.358157
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"t": 0.28,
|
| 252 |
+
"precision": 0.162272,
|
| 253 |
+
"recall": 0.377873
|
| 254 |
+
},
|
| 255 |
+
{
|
| 256 |
+
"t": 0.26,
|
| 257 |
+
"precision": 0.157717,
|
| 258 |
+
"recall": 0.398764
|
| 259 |
+
},
|
| 260 |
+
{
|
| 261 |
+
"t": 0.24,
|
| 262 |
+
"precision": 0.153892,
|
| 263 |
+
"recall": 0.422617
|
| 264 |
+
},
|
| 265 |
+
{
|
| 266 |
+
"t": 0.22,
|
| 267 |
+
"precision": 0.147826,
|
| 268 |
+
"recall": 0.444376
|
| 269 |
+
},
|
| 270 |
+
{
|
| 271 |
+
"t": 0.2,
|
| 272 |
+
"precision": 0.142145,
|
| 273 |
+
"recall": 0.467617
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"t": 0.18,
|
| 277 |
+
"precision": 0.136718,
|
| 278 |
+
"recall": 0.493871
|
| 279 |
+
},
|
| 280 |
+
{
|
| 281 |
+
"t": 0.16,
|
| 282 |
+
"precision": 0.130124,
|
| 283 |
+
"recall": 0.517877
|
| 284 |
+
},
|
| 285 |
+
{
|
| 286 |
+
"t": 0.14,
|
| 287 |
+
"precision": 0.123865,
|
| 288 |
+
"recall": 0.546276
|
| 289 |
+
},
|
| 290 |
+
{
|
| 291 |
+
"t": 0.12,
|
| 292 |
+
"precision": 0.117527,
|
| 293 |
+
"recall": 0.580243
|
| 294 |
+
},
|
| 295 |
+
{
|
| 296 |
+
"t": 0.1,
|
| 297 |
+
"precision": 0.111137,
|
| 298 |
+
"recall": 0.619318
|
| 299 |
+
},
|
| 300 |
+
{
|
| 301 |
+
"t": 0.08,
|
| 302 |
+
"precision": 0.104007,
|
| 303 |
+
"recall": 0.663449
|
| 304 |
+
},
|
| 305 |
+
{
|
| 306 |
+
"t": 0.06,
|
| 307 |
+
"precision": 0.095877,
|
| 308 |
+
"recall": 0.713709
|
| 309 |
+
},
|
| 310 |
+
{
|
| 311 |
+
"t": 0.04,
|
| 312 |
+
"precision": 0.086453,
|
| 313 |
+
"recall": 0.776995
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"t": 0.02,
|
| 317 |
+
"precision": 0.074829,
|
| 318 |
+
"recall": 0.866074
|
| 319 |
+
},
|
| 320 |
+
{
|
| 321 |
+
"t": 0.0,
|
| 322 |
+
"precision": 0.048787,
|
| 323 |
+
"recall": 1.0
|
| 324 |
+
}
|
| 325 |
+
]
|
| 326 |
+
},
|
| 327 |
+
"annotations": [],
|
| 328 |
+
"credits": {
|
| 329 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 330 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 331 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 332 |
+
"codeLicense": "Apache-2.0"
|
| 333 |
+
},
|
| 334 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 335 |
+
}
|
docs/examples/bench/w026-noise-baseline.json
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:48:30Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w026",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w026_ink.png",
|
| 15 |
+
"sha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791",
|
| 16 |
+
"width": 629,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.1785,
|
| 19 |
+
"alignTargetFile": "data/bench/w026_ink.png",
|
| 20 |
+
"alignTargetSha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "noise-seed26.png",
|
| 24 |
+
"sha256": "220a8b63d482620f4a98536fc97a971e7f69bebaafff14199d47b186ae43ffc4",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 629,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "baseline: uniform noise",
|
| 32 |
+
"notes": "numpy default_rng(26).random, 8 bit"
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"alignment": {
|
| 36 |
+
"method": "manual",
|
| 37 |
+
"target": "ink",
|
| 38 |
+
"tx": 0.0,
|
| 39 |
+
"ty": 0.0,
|
| 40 |
+
"sx": 1.0,
|
| 41 |
+
"sy": 1.0,
|
| 42 |
+
"ncc": -0.002983,
|
| 43 |
+
"coarseNcc": null,
|
| 44 |
+
"evals": 0,
|
| 45 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 46 |
+
},
|
| 47 |
+
"metrics": {
|
| 48 |
+
"nValid": 402560,
|
| 49 |
+
"validFraction": 1.0,
|
| 50 |
+
"pearson": -0.002983,
|
| 51 |
+
"blurSigmaPx": 6.0,
|
| 52 |
+
"blurSigmaUM": 653.7,
|
| 53 |
+
"pearsonMatchedBlur": -0.002789,
|
| 54 |
+
"thresholds": {
|
| 55 |
+
"ref": 0.5,
|
| 56 |
+
"pred": 0.5
|
| 57 |
+
},
|
| 58 |
+
"iou": 0.045741,
|
| 59 |
+
"dice": 0.087481,
|
| 60 |
+
"precision": 0.04801,
|
| 61 |
+
"recall": 0.491879,
|
| 62 |
+
"refPositiveFraction": 0.048634,
|
| 63 |
+
"predPositiveFraction": 0.498271,
|
| 64 |
+
"ap": 0.047854,
|
| 65 |
+
"bestF1": {
|
| 66 |
+
"f1": 0.092756,
|
| 67 |
+
"threshold": 0.0
|
| 68 |
+
},
|
| 69 |
+
"pr": [
|
| 70 |
+
{
|
| 71 |
+
"t": 1.0,
|
| 72 |
+
"precision": 1.0,
|
| 73 |
+
"recall": 0.0
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"t": 0.98,
|
| 77 |
+
"precision": 0.045315,
|
| 78 |
+
"recall": 0.018133
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"t": 0.96,
|
| 82 |
+
"precision": 0.046628,
|
| 83 |
+
"recall": 0.03744
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"t": 0.94,
|
| 87 |
+
"precision": 0.046986,
|
| 88 |
+
"recall": 0.056288
|
| 89 |
+
},
|
| 90 |
+
{
|
| 91 |
+
"t": 0.92,
|
| 92 |
+
"precision": 0.047393,
|
| 93 |
+
"recall": 0.07585
|
| 94 |
+
},
|
| 95 |
+
{
|
| 96 |
+
"t": 0.9,
|
| 97 |
+
"precision": 0.047743,
|
| 98 |
+
"recall": 0.095566
|
| 99 |
+
},
|
| 100 |
+
{
|
| 101 |
+
"t": 0.88,
|
| 102 |
+
"precision": 0.048053,
|
| 103 |
+
"recall": 0.115691
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"t": 0.86,
|
| 107 |
+
"precision": 0.048044,
|
| 108 |
+
"recall": 0.134692
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"t": 0.84,
|
| 112 |
+
"precision": 0.047887,
|
| 113 |
+
"recall": 0.153386
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"t": 0.82,
|
| 117 |
+
"precision": 0.047932,
|
| 118 |
+
"recall": 0.172898
|
| 119 |
+
},
|
| 120 |
+
{
|
| 121 |
+
"t": 0.8,
|
| 122 |
+
"precision": 0.047496,
|
| 123 |
+
"recall": 0.194555
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"t": 0.78,
|
| 127 |
+
"precision": 0.047122,
|
| 128 |
+
"recall": 0.21187
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"t": 0.76,
|
| 132 |
+
"precision": 0.046874,
|
| 133 |
+
"recall": 0.229697
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"t": 0.74,
|
| 137 |
+
"precision": 0.047309,
|
| 138 |
+
"recall": 0.250536
|
| 139 |
+
},
|
| 140 |
+
{
|
| 141 |
+
"t": 0.72,
|
| 142 |
+
"precision": 0.047345,
|
| 143 |
+
"recall": 0.270201
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"t": 0.7,
|
| 147 |
+
"precision": 0.047221,
|
| 148 |
+
"recall": 0.288691
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"t": 0.68,
|
| 152 |
+
"precision": 0.047153,
|
| 153 |
+
"recall": 0.307437
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"t": 0.66,
|
| 157 |
+
"precision": 0.047378,
|
| 158 |
+
"recall": 0.328174
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"t": 0.64,
|
| 162 |
+
"precision": 0.047424,
|
| 163 |
+
"recall": 0.34789
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"t": 0.62,
|
| 167 |
+
"precision": 0.047463,
|
| 168 |
+
"recall": 0.368117
|
| 169 |
+
},
|
| 170 |
+
{
|
| 171 |
+
"t": 0.6,
|
| 172 |
+
"precision": 0.047711,
|
| 173 |
+
"recall": 0.393094
|
| 174 |
+
},
|
| 175 |
+
{
|
| 176 |
+
"t": 0.58,
|
| 177 |
+
"precision": 0.047698,
|
| 178 |
+
"recall": 0.411942
|
| 179 |
+
},
|
| 180 |
+
{
|
| 181 |
+
"t": 0.56,
|
| 182 |
+
"precision": 0.047593,
|
| 183 |
+
"recall": 0.430381
|
| 184 |
+
},
|
| 185 |
+
{
|
| 186 |
+
"t": 0.54,
|
| 187 |
+
"precision": 0.047695,
|
| 188 |
+
"recall": 0.450046
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"t": 0.52,
|
| 192 |
+
"precision": 0.047728,
|
| 193 |
+
"recall": 0.469711
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"t": 0.5,
|
| 197 |
+
"precision": 0.04801,
|
| 198 |
+
"recall": 0.491879
|
| 199 |
+
},
|
| 200 |
+
{
|
| 201 |
+
"t": 0.48,
|
| 202 |
+
"precision": 0.048052,
|
| 203 |
+
"recall": 0.511595
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"t": 0.46,
|
| 207 |
+
"precision": 0.048076,
|
| 208 |
+
"recall": 0.53126
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"t": 0.44,
|
| 212 |
+
"precision": 0.048,
|
| 213 |
+
"recall": 0.54975
|
| 214 |
+
},
|
| 215 |
+
{
|
| 216 |
+
"t": 0.42,
|
| 217 |
+
"precision": 0.048023,
|
| 218 |
+
"recall": 0.569159
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
"t": 0.4,
|
| 222 |
+
"precision": 0.048048,
|
| 223 |
+
"recall": 0.592859
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"t": 0.38,
|
| 227 |
+
"precision": 0.047966,
|
| 228 |
+
"recall": 0.611196
|
| 229 |
+
},
|
| 230 |
+
{
|
| 231 |
+
"t": 0.36,
|
| 232 |
+
"precision": 0.048075,
|
| 233 |
+
"recall": 0.63224
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"t": 0.34,
|
| 237 |
+
"precision": 0.048121,
|
| 238 |
+
"recall": 0.652518
|
| 239 |
+
},
|
| 240 |
+
{
|
| 241 |
+
"t": 0.32,
|
| 242 |
+
"precision": 0.048089,
|
| 243 |
+
"recall": 0.671366
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"t": 0.3,
|
| 247 |
+
"precision": 0.048168,
|
| 248 |
+
"recall": 0.692001
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"t": 0.28,
|
| 252 |
+
"precision": 0.048165,
|
| 253 |
+
"recall": 0.711104
|
| 254 |
+
},
|
| 255 |
+
{
|
| 256 |
+
"t": 0.26,
|
| 257 |
+
"precision": 0.048116,
|
| 258 |
+
"recall": 0.729594
|
| 259 |
+
},
|
| 260 |
+
{
|
| 261 |
+
"t": 0.24,
|
| 262 |
+
"precision": 0.048122,
|
| 263 |
+
"recall": 0.748646
|
| 264 |
+
},
|
| 265 |
+
{
|
| 266 |
+
"t": 0.22,
|
| 267 |
+
"precision": 0.048233,
|
| 268 |
+
"recall": 0.769588
|
| 269 |
+
},
|
| 270 |
+
{
|
| 271 |
+
"t": 0.2,
|
| 272 |
+
"precision": 0.0483,
|
| 273 |
+
"recall": 0.794055
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"t": 0.18,
|
| 277 |
+
"precision": 0.048363,
|
| 278 |
+
"recall": 0.814588
|
| 279 |
+
},
|
| 280 |
+
{
|
| 281 |
+
"t": 0.16,
|
| 282 |
+
"precision": 0.048424,
|
| 283 |
+
"recall": 0.835325
|
| 284 |
+
},
|
| 285 |
+
{
|
| 286 |
+
"t": 0.14,
|
| 287 |
+
"precision": 0.048475,
|
| 288 |
+
"recall": 0.85591
|
| 289 |
+
},
|
| 290 |
+
{
|
| 291 |
+
"t": 0.12,
|
| 292 |
+
"precision": 0.048506,
|
| 293 |
+
"recall": 0.875677
|
| 294 |
+
},
|
| 295 |
+
{
|
| 296 |
+
"t": 0.1,
|
| 297 |
+
"precision": 0.048505,
|
| 298 |
+
"recall": 0.895342
|
| 299 |
+
},
|
| 300 |
+
{
|
| 301 |
+
"t": 0.08,
|
| 302 |
+
"precision": 0.048483,
|
| 303 |
+
"recall": 0.914547
|
| 304 |
+
},
|
| 305 |
+
{
|
| 306 |
+
"t": 0.06,
|
| 307 |
+
"precision": 0.048563,
|
| 308 |
+
"recall": 0.935489
|
| 309 |
+
},
|
| 310 |
+
{
|
| 311 |
+
"t": 0.04,
|
| 312 |
+
"precision": 0.048558,
|
| 313 |
+
"recall": 0.955052
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"t": 0.02,
|
| 317 |
+
"precision": 0.048564,
|
| 318 |
+
"recall": 0.975023
|
| 319 |
+
},
|
| 320 |
+
{
|
| 321 |
+
"t": 0.0,
|
| 322 |
+
"precision": 0.048634,
|
| 323 |
+
"recall": 1.0
|
| 324 |
+
}
|
| 325 |
+
]
|
| 326 |
+
},
|
| 327 |
+
"annotations": [],
|
| 328 |
+
"credits": {
|
| 329 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 330 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 331 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 332 |
+
"codeLicense": "Apache-2.0"
|
| 333 |
+
},
|
| 334 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 335 |
+
}
|
docs/examples/bench/w027-ct-baseline.json
ADDED
|
@@ -0,0 +1,334 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:50:58Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w027",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w027_ink.png",
|
| 15 |
+
"sha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2",
|
| 16 |
+
"width": 638,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.3357,
|
| 19 |
+
"alignTargetFile": "data/bench/w027_ink.png",
|
| 20 |
+
"alignTargetSha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "w027_ct.png",
|
| 24 |
+
"sha256": "bf5b5d9a5d8574c03dba35ac6b502319aa6ebee01032cff01cc5f7d600ea22b6",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 638,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "baseline: X-ray intensity (same grid, no alignment needed)"
|
| 32 |
+
}
|
| 33 |
+
},
|
| 34 |
+
"alignment": {
|
| 35 |
+
"method": "manual",
|
| 36 |
+
"target": "ink",
|
| 37 |
+
"tx": 0.0,
|
| 38 |
+
"ty": 0.0,
|
| 39 |
+
"sx": 1.0,
|
| 40 |
+
"sy": 1.0,
|
| 41 |
+
"ncc": 0.299446,
|
| 42 |
+
"coarseNcc": null,
|
| 43 |
+
"evals": 0,
|
| 44 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 45 |
+
},
|
| 46 |
+
"metrics": {
|
| 47 |
+
"nValid": 408320,
|
| 48 |
+
"validFraction": 1.0,
|
| 49 |
+
"pearson": 0.299446,
|
| 50 |
+
"blurSigmaPx": 6.0,
|
| 51 |
+
"blurSigmaUM": 642.7,
|
| 52 |
+
"pearsonMatchedBlur": 0.355799,
|
| 53 |
+
"thresholds": {
|
| 54 |
+
"ref": 0.5,
|
| 55 |
+
"pred": 0.5
|
| 56 |
+
},
|
| 57 |
+
"iou": 0.121584,
|
| 58 |
+
"dice": 0.216807,
|
| 59 |
+
"precision": 0.134544,
|
| 60 |
+
"recall": 0.557949,
|
| 61 |
+
"refPositiveFraction": 0.093674,
|
| 62 |
+
"predPositiveFraction": 0.388462,
|
| 63 |
+
"ap": 0.145828,
|
| 64 |
+
"bestF1": {
|
| 65 |
+
"f1": 0.218206,
|
| 66 |
+
"threshold": 0.44
|
| 67 |
+
},
|
| 68 |
+
"pr": [
|
| 69 |
+
{
|
| 70 |
+
"t": 1.0,
|
| 71 |
+
"precision": 0.355932,
|
| 72 |
+
"recall": 0.002196
|
| 73 |
+
},
|
| 74 |
+
{
|
| 75 |
+
"t": 0.98,
|
| 76 |
+
"precision": 0.365217,
|
| 77 |
+
"recall": 0.00549
|
| 78 |
+
},
|
| 79 |
+
{
|
| 80 |
+
"t": 0.96,
|
| 81 |
+
"precision": 0.359572,
|
| 82 |
+
"recall": 0.009673
|
| 83 |
+
},
|
| 84 |
+
{
|
| 85 |
+
"t": 0.94,
|
| 86 |
+
"precision": 0.336556,
|
| 87 |
+
"recall": 0.014562
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"t": 0.92,
|
| 91 |
+
"precision": 0.304087,
|
| 92 |
+
"recall": 0.019844
|
| 93 |
+
},
|
| 94 |
+
{
|
| 95 |
+
"t": 0.9,
|
| 96 |
+
"precision": 0.27519,
|
| 97 |
+
"recall": 0.026563
|
| 98 |
+
},
|
| 99 |
+
{
|
| 100 |
+
"t": 0.88,
|
| 101 |
+
"precision": 0.262881,
|
| 102 |
+
"recall": 0.03655
|
| 103 |
+
},
|
| 104 |
+
{
|
| 105 |
+
"t": 0.86,
|
| 106 |
+
"precision": 0.237259,
|
| 107 |
+
"recall": 0.04625
|
| 108 |
+
},
|
| 109 |
+
{
|
| 110 |
+
"t": 0.84,
|
| 111 |
+
"precision": 0.218432,
|
| 112 |
+
"recall": 0.05893
|
| 113 |
+
},
|
| 114 |
+
{
|
| 115 |
+
"t": 0.82,
|
| 116 |
+
"precision": 0.202996,
|
| 117 |
+
"recall": 0.074041
|
| 118 |
+
},
|
| 119 |
+
{
|
| 120 |
+
"t": 0.8,
|
| 121 |
+
"precision": 0.188728,
|
| 122 |
+
"recall": 0.095166
|
| 123 |
+
},
|
| 124 |
+
{
|
| 125 |
+
"t": 0.78,
|
| 126 |
+
"precision": 0.178574,
|
| 127 |
+
"recall": 0.11501
|
| 128 |
+
},
|
| 129 |
+
{
|
| 130 |
+
"t": 0.76,
|
| 131 |
+
"precision": 0.168964,
|
| 132 |
+
"recall": 0.136213
|
| 133 |
+
},
|
| 134 |
+
{
|
| 135 |
+
"t": 0.74,
|
| 136 |
+
"precision": 0.162469,
|
| 137 |
+
"recall": 0.161076
|
| 138 |
+
},
|
| 139 |
+
{
|
| 140 |
+
"t": 0.72,
|
| 141 |
+
"precision": 0.157527,
|
| 142 |
+
"recall": 0.188136
|
| 143 |
+
},
|
| 144 |
+
{
|
| 145 |
+
"t": 0.7,
|
| 146 |
+
"precision": 0.153249,
|
| 147 |
+
"recall": 0.217339
|
| 148 |
+
},
|
| 149 |
+
{
|
| 150 |
+
"t": 0.68,
|
| 151 |
+
"precision": 0.150783,
|
| 152 |
+
"recall": 0.249837
|
| 153 |
+
},
|
| 154 |
+
{
|
| 155 |
+
"t": 0.66,
|
| 156 |
+
"precision": 0.148056,
|
| 157 |
+
"recall": 0.282596
|
| 158 |
+
},
|
| 159 |
+
{
|
| 160 |
+
"t": 0.64,
|
| 161 |
+
"precision": 0.146126,
|
| 162 |
+
"recall": 0.316531
|
| 163 |
+
},
|
| 164 |
+
{
|
| 165 |
+
"t": 0.62,
|
| 166 |
+
"precision": 0.144255,
|
| 167 |
+
"recall": 0.350833
|
| 168 |
+
},
|
| 169 |
+
{
|
| 170 |
+
"t": 0.6,
|
| 171 |
+
"precision": 0.142247,
|
| 172 |
+
"recall": 0.392533
|
| 173 |
+
},
|
| 174 |
+
{
|
| 175 |
+
"t": 0.58,
|
| 176 |
+
"precision": 0.140191,
|
| 177 |
+
"recall": 0.42605
|
| 178 |
+
},
|
| 179 |
+
{
|
| 180 |
+
"t": 0.56,
|
| 181 |
+
"precision": 0.138673,
|
| 182 |
+
"recall": 0.45962
|
| 183 |
+
},
|
| 184 |
+
{
|
| 185 |
+
"t": 0.54,
|
| 186 |
+
"precision": 0.137246,
|
| 187 |
+
"recall": 0.493111
|
| 188 |
+
},
|
| 189 |
+
{
|
| 190 |
+
"t": 0.52,
|
| 191 |
+
"precision": 0.135788,
|
| 192 |
+
"recall": 0.525661
|
| 193 |
+
},
|
| 194 |
+
{
|
| 195 |
+
"t": 0.5,
|
| 196 |
+
"precision": 0.134544,
|
| 197 |
+
"recall": 0.557949
|
| 198 |
+
},
|
| 199 |
+
{
|
| 200 |
+
"t": 0.48,
|
| 201 |
+
"precision": 0.133308,
|
| 202 |
+
"recall": 0.589113
|
| 203 |
+
},
|
| 204 |
+
{
|
| 205 |
+
"t": 0.46,
|
| 206 |
+
"precision": 0.132051,
|
| 207 |
+
"recall": 0.618578
|
| 208 |
+
},
|
| 209 |
+
{
|
| 210 |
+
"t": 0.44,
|
| 211 |
+
"precision": 0.13119,
|
| 212 |
+
"recall": 0.648043
|
| 213 |
+
},
|
| 214 |
+
{
|
| 215 |
+
"t": 0.42,
|
| 216 |
+
"precision": 0.129787,
|
| 217 |
+
"recall": 0.674161
|
| 218 |
+
},
|
| 219 |
+
{
|
| 220 |
+
"t": 0.4,
|
| 221 |
+
"precision": 0.128471,
|
| 222 |
+
"recall": 0.705666
|
| 223 |
+
},
|
| 224 |
+
{
|
| 225 |
+
"t": 0.38,
|
| 226 |
+
"precision": 0.127202,
|
| 227 |
+
"recall": 0.728673
|
| 228 |
+
},
|
| 229 |
+
{
|
| 230 |
+
"t": 0.36,
|
| 231 |
+
"precision": 0.126083,
|
| 232 |
+
"recall": 0.751627
|
| 233 |
+
},
|
| 234 |
+
{
|
| 235 |
+
"t": 0.34,
|
| 236 |
+
"precision": 0.125017,
|
| 237 |
+
"recall": 0.772569
|
| 238 |
+
},
|
| 239 |
+
{
|
| 240 |
+
"t": 0.32,
|
| 241 |
+
"precision": 0.123818,
|
| 242 |
+
"recall": 0.791576
|
| 243 |
+
},
|
| 244 |
+
{
|
| 245 |
+
"t": 0.3,
|
| 246 |
+
"precision": 0.122866,
|
| 247 |
+
"recall": 0.810557
|
| 248 |
+
},
|
| 249 |
+
{
|
| 250 |
+
"t": 0.28,
|
| 251 |
+
"precision": 0.121995,
|
| 252 |
+
"recall": 0.827891
|
| 253 |
+
},
|
| 254 |
+
{
|
| 255 |
+
"t": 0.26,
|
| 256 |
+
"precision": 0.12106,
|
| 257 |
+
"recall": 0.843368
|
| 258 |
+
},
|
| 259 |
+
{
|
| 260 |
+
"t": 0.24,
|
| 261 |
+
"precision": 0.120205,
|
| 262 |
+
"recall": 0.857722
|
| 263 |
+
},
|
| 264 |
+
{
|
| 265 |
+
"t": 0.22,
|
| 266 |
+
"precision": 0.119499,
|
| 267 |
+
"recall": 0.87231
|
| 268 |
+
},
|
| 269 |
+
{
|
| 270 |
+
"t": 0.2,
|
| 271 |
+
"precision": 0.118561,
|
| 272 |
+
"recall": 0.886977
|
| 273 |
+
},
|
| 274 |
+
{
|
| 275 |
+
"t": 0.18,
|
| 276 |
+
"precision": 0.117871,
|
| 277 |
+
"recall": 0.899004
|
| 278 |
+
},
|
| 279 |
+
{
|
| 280 |
+
"t": 0.16,
|
| 281 |
+
"precision": 0.117224,
|
| 282 |
+
"recall": 0.910246
|
| 283 |
+
},
|
| 284 |
+
{
|
| 285 |
+
"t": 0.14,
|
| 286 |
+
"precision": 0.116586,
|
| 287 |
+
"recall": 0.921253
|
| 288 |
+
},
|
| 289 |
+
{
|
| 290 |
+
"t": 0.12,
|
| 291 |
+
"precision": 0.115769,
|
| 292 |
+
"recall": 0.930377
|
| 293 |
+
},
|
| 294 |
+
{
|
| 295 |
+
"t": 0.1,
|
| 296 |
+
"precision": 0.115116,
|
| 297 |
+
"recall": 0.941018
|
| 298 |
+
},
|
| 299 |
+
{
|
| 300 |
+
"t": 0.08,
|
| 301 |
+
"precision": 0.114402,
|
| 302 |
+
"recall": 0.951319
|
| 303 |
+
},
|
| 304 |
+
{
|
| 305 |
+
"t": 0.06,
|
| 306 |
+
"precision": 0.113659,
|
| 307 |
+
"recall": 0.961149
|
| 308 |
+
},
|
| 309 |
+
{
|
| 310 |
+
"t": 0.04,
|
| 311 |
+
"precision": 0.112997,
|
| 312 |
+
"recall": 0.971084
|
| 313 |
+
},
|
| 314 |
+
{
|
| 315 |
+
"t": 0.02,
|
| 316 |
+
"precision": 0.112282,
|
| 317 |
+
"recall": 0.981019
|
| 318 |
+
},
|
| 319 |
+
{
|
| 320 |
+
"t": 0.0,
|
| 321 |
+
"precision": 0.093674,
|
| 322 |
+
"recall": 1.0
|
| 323 |
+
}
|
| 324 |
+
]
|
| 325 |
+
},
|
| 326 |
+
"annotations": [],
|
| 327 |
+
"credits": {
|
| 328 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 329 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 330 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 331 |
+
"codeLicense": "Apache-2.0"
|
| 332 |
+
},
|
| 333 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 334 |
+
}
|
docs/examples/bench/w027-ink9um.json
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:50:57Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w027",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w027_ink.png",
|
| 15 |
+
"sha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2",
|
| 16 |
+
"width": 638,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.3357,
|
| 19 |
+
"alignTargetFile": "data/bench/w027_ink.png",
|
| 20 |
+
"alignTargetSha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "w027_gpu.png",
|
| 24 |
+
"sha256": "89395044a5bf932b7adaeb8522e2d36c87e684f852b6c8ba2e0d30509376e650",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 638,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan",
|
| 32 |
+
"url": "https://huggingface.co/scrollprize/ink_9um"
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"alignment": {
|
| 36 |
+
"method": "auto",
|
| 37 |
+
"target": "ink",
|
| 38 |
+
"tx": 1.0,
|
| 39 |
+
"ty": 0.75,
|
| 40 |
+
"sx": 1.00556,
|
| 41 |
+
"sy": 1.00236,
|
| 42 |
+
"ncc": 0.49414,
|
| 43 |
+
"coarseNcc": 0.702908,
|
| 44 |
+
"evals": 42312,
|
| 45 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 46 |
+
},
|
| 47 |
+
"metrics": {
|
| 48 |
+
"nValid": 408320,
|
| 49 |
+
"validFraction": 1.0,
|
| 50 |
+
"pearson": 0.49414,
|
| 51 |
+
"blurSigmaPx": 3.0,
|
| 52 |
+
"blurSigmaUM": 321.3,
|
| 53 |
+
"pearsonMatchedBlur": 0.535652,
|
| 54 |
+
"thresholds": {
|
| 55 |
+
"ref": 0.5,
|
| 56 |
+
"pred": 0.5
|
| 57 |
+
},
|
| 58 |
+
"iou": 0.266987,
|
| 59 |
+
"dice": 0.421452,
|
| 60 |
+
"precision": 0.442537,
|
| 61 |
+
"recall": 0.402285,
|
| 62 |
+
"refPositiveFraction": 0.093674,
|
| 63 |
+
"predPositiveFraction": 0.085154,
|
| 64 |
+
"ap": 0.383928,
|
| 65 |
+
"bestF1": {
|
| 66 |
+
"f1": 0.433898,
|
| 67 |
+
"threshold": 0.38
|
| 68 |
+
},
|
| 69 |
+
"pr": [
|
| 70 |
+
{
|
| 71 |
+
"t": 1.0,
|
| 72 |
+
"precision": 0.839416,
|
| 73 |
+
"recall": 0.003007
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"t": 0.98,
|
| 77 |
+
"precision": 0.671879,
|
| 78 |
+
"recall": 0.066276
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"t": 0.96,
|
| 82 |
+
"precision": 0.653496,
|
| 83 |
+
"recall": 0.088211
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"t": 0.94,
|
| 87 |
+
"precision": 0.640202,
|
| 88 |
+
"recall": 0.105833
|
| 89 |
+
},
|
| 90 |
+
{
|
| 91 |
+
"t": 0.92,
|
| 92 |
+
"precision": 0.626016,
|
| 93 |
+
"recall": 0.120787
|
| 94 |
+
},
|
| 95 |
+
{
|
| 96 |
+
"t": 0.9,
|
| 97 |
+
"precision": 0.614154,
|
| 98 |
+
"recall": 0.134539
|
| 99 |
+
},
|
| 100 |
+
{
|
| 101 |
+
"t": 0.88,
|
| 102 |
+
"precision": 0.602493,
|
| 103 |
+
"recall": 0.147847
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"t": 0.86,
|
| 107 |
+
"precision": 0.590695,
|
| 108 |
+
"recall": 0.160318
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"t": 0.84,
|
| 112 |
+
"precision": 0.582909,
|
| 113 |
+
"recall": 0.173704
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"t": 0.82,
|
| 117 |
+
"precision": 0.573791,
|
| 118 |
+
"recall": 0.187743
|
| 119 |
+
},
|
| 120 |
+
{
|
| 121 |
+
"t": 0.8,
|
| 122 |
+
"precision": 0.567528,
|
| 123 |
+
"recall": 0.201051
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"t": 0.78,
|
| 127 |
+
"precision": 0.559718,
|
| 128 |
+
"recall": 0.214045
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"t": 0.76,
|
| 132 |
+
"precision": 0.551387,
|
| 133 |
+
"recall": 0.227091
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"t": 0.74,
|
| 137 |
+
"precision": 0.54232,
|
| 138 |
+
"recall": 0.239719
|
| 139 |
+
},
|
| 140 |
+
{
|
| 141 |
+
"t": 0.72,
|
| 142 |
+
"precision": 0.533201,
|
| 143 |
+
"recall": 0.252765
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"t": 0.7,
|
| 147 |
+
"precision": 0.525344,
|
| 148 |
+
"recall": 0.265549
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"t": 0.68,
|
| 152 |
+
"precision": 0.518348,
|
| 153 |
+
"recall": 0.279929
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"t": 0.66,
|
| 157 |
+
"precision": 0.51035,
|
| 158 |
+
"recall": 0.293289
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"t": 0.64,
|
| 162 |
+
"precision": 0.502141,
|
| 163 |
+
"recall": 0.306544
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"t": 0.62,
|
| 167 |
+
"precision": 0.492882,
|
| 168 |
+
"recall": 0.319538
|
| 169 |
+
},
|
| 170 |
+
{
|
| 171 |
+
"t": 0.6,
|
| 172 |
+
"precision": 0.484261,
|
| 173 |
+
"recall": 0.333028
|
| 174 |
+
},
|
| 175 |
+
{
|
| 176 |
+
"t": 0.58,
|
| 177 |
+
"precision": 0.476978,
|
| 178 |
+
"recall": 0.347486
|
| 179 |
+
},
|
| 180 |
+
{
|
| 181 |
+
"t": 0.56,
|
| 182 |
+
"precision": 0.468635,
|
| 183 |
+
"recall": 0.361526
|
| 184 |
+
},
|
| 185 |
+
{
|
| 186 |
+
"t": 0.54,
|
| 187 |
+
"precision": 0.460308,
|
| 188 |
+
"recall": 0.375513
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"t": 0.52,
|
| 192 |
+
"precision": 0.451121,
|
| 193 |
+
"recall": 0.388245
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"t": 0.5,
|
| 197 |
+
"precision": 0.442537,
|
| 198 |
+
"recall": 0.402285
|
| 199 |
+
},
|
| 200 |
+
{
|
| 201 |
+
"t": 0.48,
|
| 202 |
+
"precision": 0.433741,
|
| 203 |
+
"recall": 0.416481
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"t": 0.46,
|
| 207 |
+
"precision": 0.424769,
|
| 208 |
+
"recall": 0.43141
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"t": 0.44,
|
| 212 |
+
"precision": 0.416711,
|
| 213 |
+
"recall": 0.447489
|
| 214 |
+
},
|
| 215 |
+
{
|
| 216 |
+
"t": 0.42,
|
| 217 |
+
"precision": 0.40637,
|
| 218 |
+
"recall": 0.462679
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
"t": 0.4,
|
| 222 |
+
"precision": 0.396898,
|
| 223 |
+
"recall": 0.477712
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"t": 0.38,
|
| 227 |
+
"precision": 0.386946,
|
| 228 |
+
"recall": 0.493817
|
| 229 |
+
},
|
| 230 |
+
{
|
| 231 |
+
"t": 0.36,
|
| 232 |
+
"precision": 0.377051,
|
| 233 |
+
"recall": 0.509373
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"t": 0.34,
|
| 237 |
+
"precision": 0.367086,
|
| 238 |
+
"recall": 0.526602
|
| 239 |
+
},
|
| 240 |
+
{
|
| 241 |
+
"t": 0.32,
|
| 242 |
+
"precision": 0.357072,
|
| 243 |
+
"recall": 0.544406
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"t": 0.3,
|
| 247 |
+
"precision": 0.346406,
|
| 248 |
+
"recall": 0.561636
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"t": 0.28,
|
| 252 |
+
"precision": 0.336663,
|
| 253 |
+
"recall": 0.581584
|
| 254 |
+
},
|
| 255 |
+
{
|
| 256 |
+
"t": 0.26,
|
| 257 |
+
"precision": 0.325605,
|
| 258 |
+
"recall": 0.599963
|
| 259 |
+
},
|
| 260 |
+
{
|
| 261 |
+
"t": 0.24,
|
| 262 |
+
"precision": 0.314132,
|
| 263 |
+
"recall": 0.620199
|
| 264 |
+
},
|
| 265 |
+
{
|
| 266 |
+
"t": 0.22,
|
| 267 |
+
"precision": 0.30291,
|
| 268 |
+
"recall": 0.641219
|
| 269 |
+
},
|
| 270 |
+
{
|
| 271 |
+
"t": 0.2,
|
| 272 |
+
"precision": 0.291763,
|
| 273 |
+
"recall": 0.663024
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"t": 0.18,
|
| 277 |
+
"precision": 0.279066,
|
| 278 |
+
"recall": 0.684201
|
| 279 |
+
},
|
| 280 |
+
{
|
| 281 |
+
"t": 0.16,
|
| 282 |
+
"precision": 0.266522,
|
| 283 |
+
"recall": 0.706424
|
| 284 |
+
},
|
| 285 |
+
{
|
| 286 |
+
"t": 0.14,
|
| 287 |
+
"precision": 0.253303,
|
| 288 |
+
"recall": 0.729875
|
| 289 |
+
},
|
| 290 |
+
{
|
| 291 |
+
"t": 0.12,
|
| 292 |
+
"precision": 0.239785,
|
| 293 |
+
"recall": 0.754085
|
| 294 |
+
},
|
| 295 |
+
{
|
| 296 |
+
"t": 0.1,
|
| 297 |
+
"precision": 0.225313,
|
| 298 |
+
"recall": 0.77921
|
| 299 |
+
},
|
| 300 |
+
{
|
| 301 |
+
"t": 0.08,
|
| 302 |
+
"precision": 0.210039,
|
| 303 |
+
"recall": 0.807341
|
| 304 |
+
},
|
| 305 |
+
{
|
| 306 |
+
"t": 0.06,
|
| 307 |
+
"precision": 0.193162,
|
| 308 |
+
"recall": 0.84044
|
| 309 |
+
},
|
| 310 |
+
{
|
| 311 |
+
"t": 0.04,
|
| 312 |
+
"precision": 0.173651,
|
| 313 |
+
"recall": 0.879422
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"t": 0.02,
|
| 317 |
+
"precision": 0.148893,
|
| 318 |
+
"recall": 0.924259
|
| 319 |
+
},
|
| 320 |
+
{
|
| 321 |
+
"t": 0.0,
|
| 322 |
+
"precision": 0.093674,
|
| 323 |
+
"recall": 1.0
|
| 324 |
+
}
|
| 325 |
+
]
|
| 326 |
+
},
|
| 327 |
+
"annotations": [],
|
| 328 |
+
"credits": {
|
| 329 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 330 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 331 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 332 |
+
"codeLicense": "Apache-2.0"
|
| 333 |
+
},
|
| 334 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 335 |
+
}
|
docs/examples/bench/w027-noise-baseline.json
ADDED
|
@@ -0,0 +1,335 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:51:00Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w027",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w027_ink.png",
|
| 15 |
+
"sha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2",
|
| 16 |
+
"width": 638,
|
| 17 |
+
"height": 640,
|
| 18 |
+
"pxPerMM": 9.3357,
|
| 19 |
+
"alignTargetFile": "data/bench/w027_ink.png",
|
| 20 |
+
"alignTargetSha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "noise-seed27.png",
|
| 24 |
+
"sha256": "3f35963811cc3ba350851a574a55123ebed8a5c217434dc46b650e037f3f1f17",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 638,
|
| 27 |
+
"height": 640,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "baseline: uniform noise",
|
| 32 |
+
"notes": "numpy default_rng(27).random, 8 bit"
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"alignment": {
|
| 36 |
+
"method": "manual",
|
| 37 |
+
"target": "ink",
|
| 38 |
+
"tx": 0.0,
|
| 39 |
+
"ty": 0.0,
|
| 40 |
+
"sx": 1.0,
|
| 41 |
+
"sy": 1.0,
|
| 42 |
+
"ncc": -0.002544,
|
| 43 |
+
"coarseNcc": null,
|
| 44 |
+
"evals": 0,
|
| 45 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 46 |
+
},
|
| 47 |
+
"metrics": {
|
| 48 |
+
"nValid": 408320,
|
| 49 |
+
"validFraction": 1.0,
|
| 50 |
+
"pearson": -0.002544,
|
| 51 |
+
"blurSigmaPx": 6.0,
|
| 52 |
+
"blurSigmaUM": 642.7,
|
| 53 |
+
"pearsonMatchedBlur": -0.002194,
|
| 54 |
+
"thresholds": {
|
| 55 |
+
"ref": 0.5,
|
| 56 |
+
"pred": 0.5
|
| 57 |
+
},
|
| 58 |
+
"iou": 0.08474,
|
| 59 |
+
"dice": 0.15624,
|
| 60 |
+
"precision": 0.092732,
|
| 61 |
+
"recall": 0.495778,
|
| 62 |
+
"refPositiveFraction": 0.093674,
|
| 63 |
+
"predPositiveFraction": 0.500816,
|
| 64 |
+
"ap": 0.093275,
|
| 65 |
+
"bestF1": {
|
| 66 |
+
"f1": 0.171302,
|
| 67 |
+
"threshold": 0.0
|
| 68 |
+
},
|
| 69 |
+
"pr": [
|
| 70 |
+
{
|
| 71 |
+
"t": 1.0,
|
| 72 |
+
"precision": 1.0,
|
| 73 |
+
"recall": 0.0
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"t": 0.98,
|
| 77 |
+
"precision": 0.094288,
|
| 78 |
+
"recall": 0.019896
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"t": 0.96,
|
| 82 |
+
"precision": 0.094086,
|
| 83 |
+
"recall": 0.039557
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"t": 0.94,
|
| 87 |
+
"precision": 0.093964,
|
| 88 |
+
"recall": 0.059139
|
| 89 |
+
},
|
| 90 |
+
{
|
| 91 |
+
"t": 0.92,
|
| 92 |
+
"precision": 0.092801,
|
| 93 |
+
"recall": 0.078329
|
| 94 |
+
},
|
| 95 |
+
{
|
| 96 |
+
"t": 0.9,
|
| 97 |
+
"precision": 0.09358,
|
| 98 |
+
"recall": 0.098695
|
| 99 |
+
},
|
| 100 |
+
{
|
| 101 |
+
"t": 0.88,
|
| 102 |
+
"precision": 0.093374,
|
| 103 |
+
"recall": 0.118199
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"t": 0.86,
|
| 107 |
+
"precision": 0.093263,
|
| 108 |
+
"recall": 0.137651
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"t": 0.84,
|
| 112 |
+
"precision": 0.09294,
|
| 113 |
+
"recall": 0.15671
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"t": 0.82,
|
| 117 |
+
"precision": 0.093395,
|
| 118 |
+
"recall": 0.177181
|
| 119 |
+
},
|
| 120 |
+
{
|
| 121 |
+
"t": 0.8,
|
| 122 |
+
"precision": 0.093183,
|
| 123 |
+
"recall": 0.200345
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"t": 0.78,
|
| 127 |
+
"precision": 0.093003,
|
| 128 |
+
"recall": 0.21964
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"t": 0.76,
|
| 132 |
+
"precision": 0.093011,
|
| 133 |
+
"recall": 0.239196
|
| 134 |
+
},
|
| 135 |
+
{
|
| 136 |
+
"t": 0.74,
|
| 137 |
+
"precision": 0.092967,
|
| 138 |
+
"recall": 0.2587
|
| 139 |
+
},
|
| 140 |
+
{
|
| 141 |
+
"t": 0.72,
|
| 142 |
+
"precision": 0.092936,
|
| 143 |
+
"recall": 0.278282
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"t": 0.7,
|
| 147 |
+
"precision": 0.092995,
|
| 148 |
+
"recall": 0.298282
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"t": 0.68,
|
| 152 |
+
"precision": 0.092958,
|
| 153 |
+
"recall": 0.317682
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"t": 0.66,
|
| 157 |
+
"precision": 0.092902,
|
| 158 |
+
"recall": 0.337211
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"t": 0.64,
|
| 162 |
+
"precision": 0.092796,
|
| 163 |
+
"recall": 0.356558
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"t": 0.62,
|
| 167 |
+
"precision": 0.093038,
|
| 168 |
+
"recall": 0.377056
|
| 169 |
+
},
|
| 170 |
+
{
|
| 171 |
+
"t": 0.6,
|
| 172 |
+
"precision": 0.092839,
|
| 173 |
+
"recall": 0.399514
|
| 174 |
+
},
|
| 175 |
+
{
|
| 176 |
+
"t": 0.58,
|
| 177 |
+
"precision": 0.092667,
|
| 178 |
+
"recall": 0.418076
|
| 179 |
+
},
|
| 180 |
+
{
|
| 181 |
+
"t": 0.56,
|
| 182 |
+
"precision": 0.092578,
|
| 183 |
+
"recall": 0.436848
|
| 184 |
+
},
|
| 185 |
+
{
|
| 186 |
+
"t": 0.54,
|
| 187 |
+
"precision": 0.092709,
|
| 188 |
+
"recall": 0.45677
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"t": 0.52,
|
| 192 |
+
"precision": 0.092773,
|
| 193 |
+
"recall": 0.476692
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"t": 0.5,
|
| 197 |
+
"precision": 0.092732,
|
| 198 |
+
"recall": 0.495778
|
| 199 |
+
},
|
| 200 |
+
{
|
| 201 |
+
"t": 0.48,
|
| 202 |
+
"precision": 0.092931,
|
| 203 |
+
"recall": 0.515987
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"t": 0.46,
|
| 207 |
+
"precision": 0.093119,
|
| 208 |
+
"recall": 0.536511
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"t": 0.44,
|
| 212 |
+
"precision": 0.093081,
|
| 213 |
+
"recall": 0.556198
|
| 214 |
+
},
|
| 215 |
+
{
|
| 216 |
+
"t": 0.42,
|
| 217 |
+
"precision": 0.093068,
|
| 218 |
+
"recall": 0.575335
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
"t": 0.4,
|
| 222 |
+
"precision": 0.093103,
|
| 223 |
+
"recall": 0.599074
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"t": 0.38,
|
| 227 |
+
"precision": 0.093244,
|
| 228 |
+
"recall": 0.619859
|
| 229 |
+
},
|
| 230 |
+
{
|
| 231 |
+
"t": 0.36,
|
| 232 |
+
"precision": 0.093374,
|
| 233 |
+
"recall": 0.639625
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"t": 0.34,
|
| 237 |
+
"precision": 0.093282,
|
| 238 |
+
"recall": 0.658449
|
| 239 |
+
},
|
| 240 |
+
{
|
| 241 |
+
"t": 0.32,
|
| 242 |
+
"precision": 0.093285,
|
| 243 |
+
"recall": 0.677952
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"t": 0.3,
|
| 247 |
+
"precision": 0.09349,
|
| 248 |
+
"recall": 0.698868
|
| 249 |
+
},
|
| 250 |
+
{
|
| 251 |
+
"t": 0.28,
|
| 252 |
+
"precision": 0.093568,
|
| 253 |
+
"recall": 0.718633
|
| 254 |
+
},
|
| 255 |
+
{
|
| 256 |
+
"t": 0.26,
|
| 257 |
+
"precision": 0.093552,
|
| 258 |
+
"recall": 0.737745
|
| 259 |
+
},
|
| 260 |
+
{
|
| 261 |
+
"t": 0.24,
|
| 262 |
+
"precision": 0.093576,
|
| 263 |
+
"recall": 0.757039
|
| 264 |
+
},
|
| 265 |
+
{
|
| 266 |
+
"t": 0.22,
|
| 267 |
+
"precision": 0.093437,
|
| 268 |
+
"recall": 0.775733
|
| 269 |
+
},
|
| 270 |
+
{
|
| 271 |
+
"t": 0.2,
|
| 272 |
+
"precision": 0.093393,
|
| 273 |
+
"recall": 0.798792
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"t": 0.18,
|
| 277 |
+
"precision": 0.093454,
|
| 278 |
+
"recall": 0.818793
|
| 279 |
+
},
|
| 280 |
+
{
|
| 281 |
+
"t": 0.16,
|
| 282 |
+
"precision": 0.093603,
|
| 283 |
+
"recall": 0.839813
|
| 284 |
+
},
|
| 285 |
+
{
|
| 286 |
+
"t": 0.14,
|
| 287 |
+
"precision": 0.093634,
|
| 288 |
+
"recall": 0.859787
|
| 289 |
+
},
|
| 290 |
+
{
|
| 291 |
+
"t": 0.12,
|
| 292 |
+
"precision": 0.093685,
|
| 293 |
+
"recall": 0.879683
|
| 294 |
+
},
|
| 295 |
+
{
|
| 296 |
+
"t": 0.1,
|
| 297 |
+
"precision": 0.093637,
|
| 298 |
+
"recall": 0.898821
|
| 299 |
+
},
|
| 300 |
+
{
|
| 301 |
+
"t": 0.08,
|
| 302 |
+
"precision": 0.093631,
|
| 303 |
+
"recall": 0.918063
|
| 304 |
+
},
|
| 305 |
+
{
|
| 306 |
+
"t": 0.06,
|
| 307 |
+
"precision": 0.093711,
|
| 308 |
+
"recall": 0.938142
|
| 309 |
+
},
|
| 310 |
+
{
|
| 311 |
+
"t": 0.04,
|
| 312 |
+
"precision": 0.093628,
|
| 313 |
+
"recall": 0.956835
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"t": 0.02,
|
| 317 |
+
"precision": 0.093714,
|
| 318 |
+
"recall": 0.97681
|
| 319 |
+
},
|
| 320 |
+
{
|
| 321 |
+
"t": 0.0,
|
| 322 |
+
"precision": 0.093674,
|
| 323 |
+
"recall": 1.0
|
| 324 |
+
}
|
| 325 |
+
]
|
| 326 |
+
},
|
| 327 |
+
"annotations": [],
|
| 328 |
+
"credits": {
|
| 329 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 330 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 331 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 332 |
+
"codeLicense": "Apache-2.0"
|
| 333 |
+
},
|
| 334 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 335 |
+
}
|
docs/examples/bench/w028-ct-baseline.json
ADDED
|
@@ -0,0 +1,334 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "unroll-inkbench/1",
|
| 3 |
+
"tool": {
|
| 4 |
+
"name": "inkbench",
|
| 5 |
+
"version": "1.0.0",
|
| 6 |
+
"impl": "python"
|
| 7 |
+
},
|
| 8 |
+
"created": "2026-10-16T03:51:10Z",
|
| 9 |
+
"scroll": "PHerc0139",
|
| 10 |
+
"segment": "w028",
|
| 11 |
+
"reference": {
|
| 12 |
+
"layer": "ink",
|
| 13 |
+
"label": "Published ink (2.4 um), Vesuvius Challenge",
|
| 14 |
+
"file": "data/bench/w028_ink.png",
|
| 15 |
+
"sha256": "fb51ea9e23deda1cb6583280fb7a2d700979cd5f6f61c6e1dc3652339976a380",
|
| 16 |
+
"width": 640,
|
| 17 |
+
"height": 631,
|
| 18 |
+
"pxPerMM": 9.2888,
|
| 19 |
+
"alignTargetFile": "data/bench/w028_ink.png",
|
| 20 |
+
"alignTargetSha256": "fb51ea9e23deda1cb6583280fb7a2d700979cd5f6f61c6e1dc3652339976a380"
|
| 21 |
+
},
|
| 22 |
+
"prediction": {
|
| 23 |
+
"name": "w028_ct.png",
|
| 24 |
+
"sha256": "02e88630f83b2fea27ec4473e8bf04d43a5c771d8b62e8900fd8f56dc4cc3004",
|
| 25 |
+
"format": "png",
|
| 26 |
+
"width": 640,
|
| 27 |
+
"height": 631,
|
| 28 |
+
"normalization": "max255",
|
| 29 |
+
"invert": false,
|
| 30 |
+
"model": {
|
| 31 |
+
"name": "baseline: X-ray intensity (same grid, no alignment needed)"
|
| 32 |
+
}
|
| 33 |
+
},
|
| 34 |
+
"alignment": {
|
| 35 |
+
"method": "manual",
|
| 36 |
+
"target": "ink",
|
| 37 |
+
"tx": 0.0,
|
| 38 |
+
"ty": 0.0,
|
| 39 |
+
"sx": 1.0,
|
| 40 |
+
"sy": 1.0,
|
| 41 |
+
"ncc": 0.29201,
|
| 42 |
+
"coarseNcc": null,
|
| 43 |
+
"evals": 0,
|
| 44 |
+
"model": "bilinear, scale about grid centre, then shift; units are reference px"
|
| 45 |
+
},
|
| 46 |
+
"metrics": {
|
| 47 |
+
"nValid": 403840,
|
| 48 |
+
"validFraction": 1.0,
|
| 49 |
+
"pearson": 0.29201,
|
| 50 |
+
"blurSigmaPx": 6.0,
|
| 51 |
+
"blurSigmaUM": 645.9,
|
| 52 |
+
"pearsonMatchedBlur": 0.418986,
|
| 53 |
+
"thresholds": {
|
| 54 |
+
"ref": 0.5,
|
| 55 |
+
"pred": 0.5
|
| 56 |
+
},
|
| 57 |
+
"iou": 0.097583,
|
| 58 |
+
"dice": 0.177814,
|
| 59 |
+
"precision": 0.105548,
|
| 60 |
+
"recall": 0.563921,
|
| 61 |
+
"refPositiveFraction": 0.077419,
|
| 62 |
+
"predPositiveFraction": 0.413637,
|
| 63 |
+
"ap": 0.11115,
|
| 64 |
+
"bestF1": {
|
| 65 |
+
"f1": 0.177941,
|
| 66 |
+
"threshold": 0.48
|
| 67 |
+
},
|
| 68 |
+
"pr": [
|
| 69 |
+
{
|
| 70 |
+
"t": 1.0,
|
| 71 |
+
"precision": 0.159794,
|
| 72 |
+
"recall": 0.000992
|
| 73 |
+
},
|
| 74 |
+
{
|
| 75 |
+
"t": 0.98,
|
| 76 |
+
"precision": 0.20075,
|
| 77 |
+
"recall": 0.003422
|
| 78 |
+
},
|
| 79 |
+
{
|
| 80 |
+
"t": 0.96,
|
| 81 |
+
"precision": 0.174542,
|
| 82 |
+
"recall": 0.005789
|
| 83 |
+
},
|
| 84 |
+
{
|
| 85 |
+
"t": 0.94,
|
| 86 |
+
"precision": 0.168952,
|
| 87 |
+
"recall": 0.009851
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"t": 0.92,
|
| 91 |
+
"precision": 0.163784,
|
| 92 |
+
"recall": 0.015449
|
| 93 |
+
},
|
| 94 |
+
{
|
| 95 |
+
"t": 0.9,
|
| 96 |
+
"precision": 0.158648,
|
| 97 |
+
"recall": 0.022677
|
| 98 |
+
},
|
| 99 |
+
{
|
| 100 |
+
"t": 0.88,
|
| 101 |
+
"precision": 0.153033,
|
| 102 |
+
"recall": 0.031953
|
| 103 |
+
},
|
| 104 |
+
{
|
| 105 |
+
"t": 0.86,
|
| 106 |
+
"precision": 0.149486,
|
| 107 |
+
"recall": 0.044619
|
| 108 |
+
},
|
| 109 |
+
{
|
| 110 |
+
"t": 0.84,
|
| 111 |
+
"precision": 0.145116,
|
| 112 |
+
"recall": 0.059587
|
| 113 |
+
},
|
| 114 |
+
{
|
| 115 |
+
"t": 0.82,
|
| 116 |
+
"precision": 0.141404,
|
| 117 |
+
"recall": 0.076923
|
| 118 |
+
},
|
| 119 |
+
{
|
| 120 |
+
"t": 0.8,
|
| 121 |
+
"precision": 0.137073,
|
| 122 |
+
"recall": 0.101359
|
| 123 |
+
},
|
| 124 |
+
{
|
| 125 |
+
"t": 0.78,
|
| 126 |
+
"precision": 0.132283,
|
| 127 |
+
"recall": 0.123589
|
| 128 |
+
},
|
| 129 |
+
{
|
| 130 |
+
"t": 0.76,
|
| 131 |
+
"precision": 0.128693,
|
| 132 |
+
"recall": 0.148089
|
| 133 |
+
},
|
| 134 |
+
{
|
| 135 |
+
"t": 0.74,
|
| 136 |
+
"precision": 0.126437,
|
| 137 |
+
"recall": 0.175884
|
| 138 |
+
},
|
| 139 |
+
{
|
| 140 |
+
"t": 0.72,
|
| 141 |
+
"precision": 0.123469,
|
| 142 |
+
"recall": 0.204702
|
| 143 |
+
},
|
| 144 |
+
{
|
| 145 |
+
"t": 0.7,
|
| 146 |
+
"precision": 0.121475,
|
| 147 |
+
"recall": 0.235887
|
| 148 |
+
},
|
| 149 |
+
{
|
| 150 |
+
"t": 0.68,
|
| 151 |
+
"precision": 0.118464,
|
| 152 |
+
"recall": 0.265377
|
| 153 |
+
},
|
| 154 |
+
{
|
| 155 |
+
"t": 0.66,
|
| 156 |
+
"precision": 0.116752,
|
| 157 |
+
"recall": 0.298065
|
| 158 |
+
},
|
| 159 |
+
{
|
| 160 |
+
"t": 0.64,
|
| 161 |
+
"precision": 0.11447,
|
| 162 |
+
"recall": 0.329538
|
| 163 |
+
},
|
| 164 |
+
{
|
| 165 |
+
"t": 0.62,
|
| 166 |
+
"precision": 0.112809,
|
| 167 |
+
"recall": 0.36197
|
| 168 |
+
},
|
| 169 |
+
{
|
| 170 |
+
"t": 0.6,
|
| 171 |
+
"precision": 0.110745,
|
| 172 |
+
"recall": 0.400448
|
| 173 |
+
},
|
| 174 |
+
{
|
| 175 |
+
"t": 0.58,
|
| 176 |
+
"precision": 0.109919,
|
| 177 |
+
"recall": 0.435343
|
| 178 |
+
},
|
| 179 |
+
{
|
| 180 |
+
"t": 0.56,
|
| 181 |
+
"precision": 0.108943,
|
| 182 |
+
"recall": 0.468959
|
| 183 |
+
},
|
| 184 |
+
{
|
| 185 |
+
"t": 0.54,
|
| 186 |
+
"precision": 0.107364,
|
| 187 |
+
"recall": 0.498992
|
| 188 |
+
},
|
| 189 |
+
{
|
| 190 |
+
"t": 0.52,
|
| 191 |
+
"precision": 0.106305,
|
| 192 |
+
"recall": 0.530945
|
| 193 |
+
},
|
| 194 |
+
{
|
| 195 |
+
"t": 0.5,
|
| 196 |
+
"precision": 0.105548,
|
| 197 |
+
"recall": 0.563921
|
| 198 |
+
},
|
| 199 |
+
{
|
| 200 |
+
"t": 0.48,
|
| 201 |
+
"precision": 0.104655,
|
| 202 |
+
"recall": 0.593667
|
| 203 |
+
},
|
| 204 |
+
{
|
| 205 |
+
"t": 0.46,
|
| 206 |
+
"precision": 0.103688,
|
| 207 |
+
"recall": 0.622901
|
| 208 |
+
},
|
| 209 |
+
{
|
| 210 |
+
"t": 0.44,
|
| 211 |
+
"precision": 0.10299,
|
| 212 |
+
"recall": 0.650952
|
| 213 |
+
},
|
| 214 |
+
{
|
| 215 |
+
"t": 0.42,
|
| 216 |
+
"precision": 0.10223,
|
| 217 |
+
"recall": 0.677595
|
| 218 |
+
},
|
| 219 |
+
{
|
| 220 |
+
"t": 0.4,
|
| 221 |
+
"precision": 0.101256,
|
| 222 |
+
"recall": 0.706413
|
| 223 |
+
},
|
| 224 |
+
{
|
| 225 |
+
"t": 0.38,
|
| 226 |
+
"precision": 0.100472,
|
| 227 |
+
"recall": 0.729698
|
| 228 |
+
},
|
| 229 |
+
{
|
| 230 |
+
"t": 0.36,
|
| 231 |
+
"precision": 0.099899,
|
| 232 |
+
"recall": 0.753174
|
| 233 |
+
},
|
| 234 |
+
{
|
| 235 |
+
"t": 0.34,
|
| 236 |
+
"precision": 0.099222,
|
| 237 |
+
"recall": 0.774828
|
| 238 |
+
},
|
| 239 |
+
{
|
| 240 |
+
"t": 0.32,
|
| 241 |
+
"precision": 0.098499,
|
| 242 |
+
"recall": 0.793923
|
| 243 |
+
},
|
| 244 |
+
{
|
| 245 |
+
"t": 0.3,
|
| 246 |
+
"precision": 0.097848,
|
| 247 |
+
"recall": 0.811099
|
| 248 |
+
},
|
| 249 |
+
{
|
| 250 |
+
"t": 0.28,
|
| 251 |
+
"precision": 0.09724,
|
| 252 |
+
"recall": 0.826963
|
| 253 |
+
},
|
| 254 |
+
{
|
| 255 |
+
"t": 0.26,
|
| 256 |
+
"precision": 0.096755,
|
| 257 |
+
"recall": 0.842763
|
| 258 |
+
},
|
| 259 |
+
{
|
| 260 |
+
"t": 0.24,
|
| 261 |
+
"precision": 0.096275,
|
| 262 |
+
"recall": 0.857157
|
| 263 |
+
},
|
| 264 |
+
{
|
| 265 |
+
"t": 0.22,
|
| 266 |
+
"precision": 0.095782,
|
| 267 |
+
"recall": 0.870302
|
| 268 |
+
},
|
| 269 |
+
{
|
| 270 |
+
"t": 0.2,
|
| 271 |
+
"precision": 0.095437,
|
| 272 |
+
"recall": 0.88655
|
| 273 |
+
},
|
| 274 |
+
{
|
| 275 |
+
"t": 0.18,
|
| 276 |
+
"precision": 0.095042,
|
| 277 |
+
"recall": 0.898225
|
| 278 |
+
},
|
| 279 |
+
{
|
| 280 |
+
"t": 0.16,
|
| 281 |
+
"precision": 0.094757,
|
| 282 |
+
"recall": 0.910251
|
| 283 |
+
},
|
| 284 |
+
{
|
| 285 |
+
"t": 0.14,
|
| 286 |
+
"precision": 0.094484,
|
| 287 |
+
"recall": 0.921542
|
| 288 |
+
},
|
| 289 |
+
{
|
| 290 |
+
"t": 0.12,
|
| 291 |
+
"precision": 0.094234,
|
| 292 |
+
"recall": 0.93312
|
| 293 |
+
},
|
| 294 |
+
{
|
| 295 |
+
"t": 0.1,
|
| 296 |
+
"precision": 0.093859,
|
| 297 |
+
"recall": 0.943355
|
| 298 |
+
},
|
| 299 |
+
{
|
| 300 |
+
"t": 0.08,
|
| 301 |
+
"precision": 0.093421,
|
| 302 |
+
"recall": 0.952279
|
| 303 |
+
},
|
| 304 |
+
{
|
| 305 |
+
"t": 0.06,
|
| 306 |
+
"precision": 0.093045,
|
| 307 |
+
"recall": 0.962226
|
| 308 |
+
},
|
| 309 |
+
{
|
| 310 |
+
"t": 0.04,
|
| 311 |
+
"precision": 0.09265,
|
| 312 |
+
"recall": 0.971534
|
| 313 |
+
},
|
| 314 |
+
{
|
| 315 |
+
"t": 0.02,
|
| 316 |
+
"precision": 0.092188,
|
| 317 |
+
"recall": 0.980937
|
| 318 |
+
},
|
| 319 |
+
{
|
| 320 |
+
"t": 0.0,
|
| 321 |
+
"precision": 0.077419,
|
| 322 |
+
"recall": 1.0
|
| 323 |
+
}
|
| 324 |
+
]
|
| 325 |
+
},
|
| 326 |
+
"annotations": [],
|
| 327 |
+
"credits": {
|
| 328 |
+
"data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.",
|
| 329 |
+
"dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)",
|
| 330 |
+
"code": "Unroll Ink Bench by Chase Hendrick",
|
| 331 |
+
"codeLicense": "Apache-2.0"
|
| 332 |
+
},
|
| 333 |
+
"disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings."
|
| 334 |
+
}
|