diff --git a/README.md b/README.md index 76d0d605d636caa862e6086a3ee9d5202489f2d5..c90fb1e0cd357c60b95a027b67b054d463a95cd4 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ PHerc. 139 is a papyrus roll carbonized when Vesuvius buried Herculaneum in 79 C - **Shapes**: the 9.362 µm tifxyz meshes of the 37 traced windings (`w023` to `w059`), decimated to 579,450 vertices. - **X-ray layer**: the CT intensity on each surface, from the public 9.362 µm surface volumes. - **Published ink (2.4 µm), amber**: the Vesuvius Challenge's public ink detection on the 2.4 µm scan. In 3D it is drawn through a local-contrast filter so letter-sized strokes stand out. The close-up view shows the published preview without filtering, downscaled to 2,400 px on its long side and re-saved as JPEG. -- **Our ink model (9 µm), cyan**: inked by running the public [`scrollprize/ink_9um`](https://huggingface.co/scrollprize/ink_9um) model (seed 42, step 75000) on the 9.362 µm scan. So far we have run the public `scrollprize/ink_9um` model on Hugging Face ZeroGPU over windings `w046` to `w051`. More windings are added as daily free GPU runs allow, so this layer is a work in progress and will grow. The first batch was 37,141 patches in 57 s of measured GPU time. The page reads the covered windings from the manifest, so its text updates as windings are added. The 9.362 µm scan is about four times coarser than the 2.4 µm one, so this layer is noisier. `w047` and `w050` ran with patch overlap 0.5; the other four ran with 0.25 or 0.2, and their 128 px patch grid shows as faint square tiles. Per-run records (checkpoint sha256, villa commit, requested and measured GPU seconds) are in `data/gpu/receipts/`; `data/gpu.json` is the summary the page reads. +- **Our ink model (9 µm), cyan**: inked by running the public [`scrollprize/ink_9um`](https://huggingface.co/scrollprize/ink_9um) model (seed 42, step 75000) on the 9.362 µm scan. The ink_9um model was trained by the Vesuvius Challenge and run by Chase Hendrick, on Hugging Face ZeroGPU and Modal, over all 37 windings (`w023` to `w059`). The first batch was 37,141 patches in 57 s of measured GPU time. The page reads the covered windings from the manifest, so its text updates as windings are added. The 9.362 µm scan is about four times coarser than the 2.4 µm one, so this layer is noisier. `w047` and `w050` ran with patch overlap 0.5; others ran with 0.25 or 0.2, and their 128 px patch grid shows as faint square tiles. Per-run records (checkpoint sha256, villa commit, requested and measured GPU seconds) are in `data/gpu/receipts/`; `data/gpu.json` is the summary the page reads. Unrolling lays the windings end to end, outermost first. Each traced winding runs a little past one full turn, so roughly a tenth of the 2.8 m strip repeats surface from its neighbour. @@ -57,3 +57,7 @@ Data: Giorgio Angelotti, Stephen Parsons, Sean Johnson, Elian Rafael Dal Prà, J ## Ink Bench Score your own ink prediction for any winding against the published ink, in the browser (nothing is uploaded): automatic UV alignment, side by side, swipe and difference views, Pearson r, IoU, Dice and a precision and recall curve (agreement with another model's output, not ground truth), letter boxes marked "candidate for expert review", and a one-click reproducible bundle. `tools/inkbench.py` gives identical results in Python and on the command line. See [docs/BENCH.md](docs/BENCH.md) and the open bundle format [docs/BENCH_FORMAT.md](docs/BENCH_FORMAT.md). The Bench code and format are Apache-2.0 (`LICENSE-APACHE`, `NOTICE`, by Chase Hendrick); the data stays CC BY-NC 4.0. + +### Researcher takeaway + +On the public ink_9um run over all 37 windings (`docs/examples/bench/`), against the published 2.4 µm ink: median AP 0.319 for ink_9um, 0.117 for raw X-ray intensity and 0.089 for uniform noise, and ink_9um beats X-ray intensity on AP on every winding. On Pearson r the medians are 0.422, 0.266 and 0.000, but X-ray intensity outranks ink_9um on `w039`, `w053`, `w054` and `w057`. So: rank ink models by AP (threshold-free and robust to brightness), always report the X-ray floor next to your model, and look at windings where the floor wins before trusting a correlation. These are agreement scores with another model's output, not ground truth. diff --git a/data/bench/index.json b/data/bench/index.json index d7b7ec3bf12cb8765a941577f1b8b2ee555f9c73..69ab56ea7aeaf5aa305919a683c03b9fe5922c20 100644 --- a/data/bench/index.json +++ b/data/bench/index.json @@ -17,6 +17,10 @@ "ct": { "file": "data/bench/w023_ct.png", "sha256": "9e871edfb98d9c5455e3e8b6dae5bc2127b4b6c0ab2399ea5f80c9b9a38b1d2b" + }, + "gpu": { + "file": "data/bench/w023_gpu.png", + "sha256": "14bedeb6a276f50d4783caec9774ccc4cf5899ddd67a0d57bf75dc6d04372d0c" } } }, @@ -34,6 +38,10 @@ "ct": { "file": "data/bench/w024_ct.png", "sha256": "7f21087fc1b5a1a5812e02d24134004524373408d5fe533939f12003f90c6517" + }, + "gpu": { + "file": "data/bench/w024_gpu.png", + "sha256": "cc11fb74a30bc3fb48bd652946e3e9124d1a2d90947f613f715da37671a85efd" } } }, @@ -51,6 +59,10 @@ "ct": { "file": "data/bench/w025_ct.png", "sha256": "e30f050d29ccc83eaf9b29beafe66f0a47a35ac2a7f30dbd525f462c9ea325fa" + }, + "gpu": { + "file": "data/bench/w025_gpu.png", + "sha256": "c129d596d1fe215b2a5c671eff5143e938bd4c4d34ac1a6c7b9843d1f263f226" } } }, @@ -68,6 +80,10 @@ "ct": { "file": "data/bench/w026_ct.png", "sha256": "d005b08b88e31ad0cc24bfbbff91041e8abad6275d39cbb243dddfe78b46f3f0" + }, + "gpu": { + "file": "data/bench/w026_gpu.png", + "sha256": "262b1fb5329f95649d5d69da657ac3b111b9ba05c2151859c167efd85c700079" } } }, @@ -85,6 +101,10 @@ "ct": { "file": "data/bench/w027_ct.png", "sha256": "bf5b5d9a5d8574c03dba35ac6b502319aa6ebee01032cff01cc5f7d600ea22b6" + }, + "gpu": { + "file": "data/bench/w027_gpu.png", + "sha256": "89395044a5bf932b7adaeb8522e2d36c87e684f852b6c8ba2e0d30509376e650" } } }, @@ -102,6 +122,10 @@ "ct": { "file": "data/bench/w028_ct.png", "sha256": "02e88630f83b2fea27ec4473e8bf04d43a5c771d8b62e8900fd8f56dc4cc3004" + }, + "gpu": { + "file": "data/bench/w028_gpu.png", + "sha256": "f5b8b736eafc7bd21fd847360d4b9f676d5d94152eb3a5b0b551b5dda437d48e" } } }, @@ -119,6 +143,10 @@ "ct": { "file": "data/bench/w029_ct.png", "sha256": "2217cff5ce56140fec1f2f34a055b94ac04a9071d42f20f7d3e9887db23e9025" + }, + "gpu": { + "file": "data/bench/w029_gpu.png", + "sha256": "2c3c563b8d544fd58025674d718c5aa3393127c710bbe8eb636b699bea6226bb" } } }, @@ -136,6 +164,10 @@ "ct": { "file": "data/bench/w030_ct.png", "sha256": "1e739695dddf66e512678c896f56e9fbb5dbb7ac80248db08a5432892f253cd6" + }, + "gpu": { + "file": "data/bench/w030_gpu.png", + "sha256": "d0bfda06404540de9f895716b7cecbb7b87d8281625bdc77b0c21f0e58bab99b" } } }, @@ -153,6 +185,10 @@ "ct": { "file": "data/bench/w031_ct.png", "sha256": "371e1428275d44a00eb5bddf4f87ce93ddb6803cc134537e3304823ee2412be3" + }, + "gpu": { + "file": "data/bench/w031_gpu.png", + "sha256": "93fc5e9939c05dadc090dd0bb77090fe4016b9d3b45cc5bb0469beaa746a208c" } } }, @@ -170,6 +206,10 @@ "ct": { "file": "data/bench/w032_ct.png", "sha256": "83a374f9ad2d93d0fcb30d1fcc6e632830fbd9b6d2bb6aa920c8ae2562674fd1" + }, + "gpu": { + "file": "data/bench/w032_gpu.png", + "sha256": "ec48088615f9f7a3f80bfe61d130362a50d59d7de3d9bf2565f21f12212efbc5" } } }, @@ -187,6 +227,10 @@ "ct": { "file": "data/bench/w033_ct.png", "sha256": "3ccb04488d2c8f5e8eea00f6c690508d5eb49c3c75ba8b1dfada08fb6038139c" + }, + "gpu": { + "file": "data/bench/w033_gpu.png", + "sha256": "da90168078510f68b84b28617e270f0a33a85d374b9b0db0c565de2365511b1c" } } }, @@ -204,6 +248,10 @@ "ct": { "file": "data/bench/w034_ct.png", "sha256": "0b0726cbc9d4fbf6a3851b580b4711606a59c7d221efbefebfe0af350cf74a5d" + }, + "gpu": { + "file": "data/bench/w034_gpu.png", + "sha256": "015c6a53edb0e9c97c6bacd0f41a38b3ce42af3e1031a7debbdece3771e83dd8" } } }, @@ -221,6 +269,10 @@ "ct": { "file": "data/bench/w035_ct.png", "sha256": "65b1c06363cf835d4b008c7155d3e4140c47489c9d1c905ad20baebeb5a18987" + }, + "gpu": { + "file": "data/bench/w035_gpu.png", + "sha256": "64fe35009ee7787f8d3afa7b41e2a62936fa8a1a00b80376d5013ad7229175da" } } }, @@ -238,6 +290,10 @@ "ct": { "file": "data/bench/w036_ct.png", "sha256": "fe43a9e3eb5bed08041cc55a43134b63fb23a85d543f4e89d838350c7de55f0d" + }, + "gpu": { + "file": "data/bench/w036_gpu.png", + "sha256": "f326573cb16b362511bbbd8e11d51d9cae638c572134f5bf9acd83f983a7799e" } } }, @@ -255,6 +311,10 @@ "ct": { "file": "data/bench/w037_ct.png", "sha256": "2656acecedda3f2647058e84185a9d9a5f691272ed0fa99e1c007ab2590a6733" + }, + "gpu": { + "file": "data/bench/w037_gpu.png", + "sha256": "4b1f3efc88ebc02f44ade6d426371c7ceaa0f45a12d68828cae8a2a0948bcb33" } } }, @@ -272,6 +332,10 @@ "ct": { "file": "data/bench/w038_ct.png", "sha256": "3a7dc45d62f9da8ab65a46e3f388813749b058d1e93dfc6bed78d70cde7a76ee" + }, + "gpu": { + "file": "data/bench/w038_gpu.png", + "sha256": "a9e4b402fa9bc449ef35f0eb3edd3758ad1afafe4ac0178d05cd4245eaa74201" } } }, @@ -289,6 +353,10 @@ "ct": { "file": "data/bench/w039_ct.png", "sha256": "927048bd37ec6e3838b6e957fc0dfc4c569f096688b088d48fca0a0dda8c2dbc" + }, + "gpu": { + "file": "data/bench/w039_gpu.png", + "sha256": "373ddb854a2af7236703c3e4ebb58b148c335230fed2bb3e1498e8136c2c2870" } } }, @@ -306,6 +374,10 @@ "ct": { "file": "data/bench/w040_ct.png", "sha256": "df430900b97f8459ea94ad284960bf61051731aca00b48db73ca07264d216e76" + }, + "gpu": { + "file": "data/bench/w040_gpu.png", + "sha256": "5114fe17084c037b12c0dd883d93e65b287bdabce02e19d22fc1bfa0da0fd518" } } }, @@ -323,6 +395,10 @@ "ct": { "file": "data/bench/w041_ct.png", "sha256": "b4e58718a814135c14e2f7d6a2c556c259e9eaac0f72c4eb0abe52b8a288940f" + }, + "gpu": { + "file": "data/bench/w041_gpu.png", + "sha256": "d57c948b18da6c3c15569b858ad0ee34af03c97bb25bf51d2959e123115127c5" } } }, @@ -340,6 +416,10 @@ "ct": { "file": "data/bench/w042_ct.png", "sha256": "0231a5f2b89abba26b3983869727145b140a3d3f0f04b86911c736249d0fb097" + }, + "gpu": { + "file": "data/bench/w042_gpu.png", + "sha256": "726d4bc78e3256dc7076c9eae37d342f3253a436ee49a0a8993372f6c9746d3b" } } }, @@ -357,6 +437,10 @@ "ct": { "file": "data/bench/w043_ct.png", "sha256": "8fe83037cdab7f905bcd55d583950e71956d8729576151cd6e93b9b5610a8a36" + }, + "gpu": { + "file": "data/bench/w043_gpu.png", + "sha256": "b92f6007d57a05b7c08958c60b5070e70f895a445a12f740f2ec8165d4be42ea" } } }, @@ -374,6 +458,10 @@ "ct": { "file": "data/bench/w044_ct.png", "sha256": "b537b3e4505f07bd871dfc6a680471ba88de93c354e0ef8c6d825c009039290c" + }, + "gpu": { + "file": "data/bench/w044_gpu.png", + "sha256": "b5148db74ef0a5a4c705b7a6928656e36cbfaee0af8f42ed84bedb8df85199ae" } } }, @@ -391,6 +479,10 @@ "ct": { "file": "data/bench/w045_ct.png", "sha256": "0d991e625d4abb1ffce31f904a3e47ed012cdfe291f02ceecd5fdf733fc4662b" + }, + "gpu": { + "file": "data/bench/w045_gpu.png", + "sha256": "fe8cd07604368288da0eb2d4da667ca69d0349b7646814a6db77438d9d0c7bf9" } } }, @@ -534,6 +626,10 @@ "ct": { "file": "data/bench/w052_ct.png", "sha256": "29b475c37e05f7a8078e2eadedbbf1863899ca48e39caec2e7fb76d91a508154" + }, + "gpu": { + "file": "data/bench/w052_gpu.png", + "sha256": "c79565130088f5a8d0dd423ab64bebcf604e871a7a0960632578227e21f72278" } } }, @@ -551,6 +647,10 @@ "ct": { "file": "data/bench/w053_ct.png", "sha256": "bb50da6580261cef973e4276ec7936d114d55845c8efac9088ea2c5a91f76e5d" + }, + "gpu": { + "file": "data/bench/w053_gpu.png", + "sha256": "09c8e76abc3c5f1d4d979709bee06692946fd8b9b1054d262234ab41717ad319" } } }, @@ -568,6 +668,10 @@ "ct": { "file": "data/bench/w054_ct.png", "sha256": "a6c5afcbeb31180ebfc9f286802fd07f01ae517a3db502e3972d73b68d6dd0b6" + }, + "gpu": { + "file": "data/bench/w054_gpu.png", + "sha256": "c3b3cf2331601645db43d56b402d49285a6ded4984bfcbcbc06582780dc05a80" } } }, @@ -585,6 +689,10 @@ "ct": { "file": "data/bench/w055_ct.png", "sha256": "dab0bba53acdd7d658ef4a2bf147d8bcf81016faae4d96402e1b1593a2aea178" + }, + "gpu": { + "file": "data/bench/w055_gpu.png", + "sha256": "dc8f4c5ba397db806b852cb0075a23c008df3bdd2c8b554eee477d2c9dbaf86a" } } }, @@ -602,6 +710,10 @@ "ct": { "file": "data/bench/w056_ct.png", "sha256": "156992b325cb9b192ed76c07a80d904b34b4a49139b20e5e81b662b2387c3798" + }, + "gpu": { + "file": "data/bench/w056_gpu.png", + "sha256": "c2a59ccce65176ac716e507dd584b59128174330f1df1f285013921c87142539" } } }, @@ -619,6 +731,10 @@ "ct": { "file": "data/bench/w057_ct.png", "sha256": "14ef32a30434a86f24c3395c29d496d3602b336756304f5e32bc9c18148e1ac2" + }, + "gpu": { + "file": "data/bench/w057_gpu.png", + "sha256": "0baae2f99ec2b4d643ddcb471b2605e9543a2e95e9dd37772c18f4abf513c263" } } }, @@ -636,6 +752,10 @@ "ct": { "file": "data/bench/w058_ct.png", "sha256": "9a2d6eb4f994a44a02ccb7fe4b0134768b5b636da2faf21f6a1234ac24e79848" + }, + "gpu": { + "file": "data/bench/w058_gpu.png", + "sha256": "c1547ad9c0c3909c8d8fc1a25150f3133235b84ee065cb55efe860719bc14164" } } }, @@ -653,6 +773,10 @@ "ct": { "file": "data/bench/w059_ct.png", "sha256": "fead2cd274c88bd50672df8d0537bc3eca86c27387b0ceb21312864c4d1622b7" + }, + "gpu": { + "file": "data/bench/w059_gpu.png", + "sha256": "f70d59d0ffde0448a8b1905418f972e0850b61416f105d7003d987bd64c6165d" } } } diff --git a/data/bench/w023_gpu.png b/data/bench/w023_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..449222f1d3d5c32772c10beafbb23cf890e2ab28 --- /dev/null +++ b/data/bench/w023_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:14bedeb6a276f50d4783caec9774ccc4cf5899ddd67a0d57bf75dc6d04372d0c +size 237354 diff --git a/data/bench/w024_gpu.png b/data/bench/w024_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..e45d5fb7b7380c084583a08ec43a701b04ee8b0a --- /dev/null +++ b/data/bench/w024_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cc11fb74a30bc3fb48bd652946e3e9124d1a2d90947f613f715da37671a85efd +size 240618 diff --git a/data/bench/w025_gpu.png b/data/bench/w025_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..fbd709bd8cfc3d2e9cc782957abb5fa8ab72e2db --- /dev/null +++ b/data/bench/w025_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c129d596d1fe215b2a5c671eff5143e938bd4c4d34ac1a6c7b9843d1f263f226 +size 216849 diff --git a/data/bench/w026_gpu.png b/data/bench/w026_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..b55f5d4661ce3aeac64abe6e42b0f06e12b50ef2 --- /dev/null +++ b/data/bench/w026_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:262b1fb5329f95649d5d69da657ac3b111b9ba05c2151859c167efd85c700079 +size 224221 diff --git a/data/bench/w027_gpu.png b/data/bench/w027_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..2ef4e947ec68d6d7538cd084ed5f229f970ba4ff --- /dev/null +++ b/data/bench/w027_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89395044a5bf932b7adaeb8522e2d36c87e684f852b6c8ba2e0d30509376e650 +size 231202 diff --git a/data/bench/w028_gpu.png b/data/bench/w028_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..69c03f46526b6c83dc164d33611a34eb61449ff8 --- /dev/null +++ b/data/bench/w028_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5b8b736eafc7bd21fd847360d4b9f676d5d94152eb3a5b0b551b5dda437d48e +size 236215 diff --git a/data/bench/w029_gpu.png b/data/bench/w029_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..ef4e886ba828448dc330dec5c59ac37ed9c77131 --- /dev/null +++ b/data/bench/w029_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2c3c563b8d544fd58025674d718c5aa3393127c710bbe8eb636b699bea6226bb +size 222188 diff --git a/data/bench/w030_gpu.png b/data/bench/w030_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..f6bb0b60abc7577abe7e759685a89e47e720376c --- /dev/null +++ b/data/bench/w030_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d0bfda06404540de9f895716b7cecbb7b87d8281625bdc77b0c21f0e58bab99b +size 231716 diff --git a/data/bench/w031_gpu.png b/data/bench/w031_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..59ad218fb5c12675612a585e1c6fd8923eff6133 --- /dev/null +++ b/data/bench/w031_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:93fc5e9939c05dadc090dd0bb77090fe4016b9d3b45cc5bb0469beaa746a208c +size 229022 diff --git a/data/bench/w032_gpu.png b/data/bench/w032_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..44a95bbef49dbe304a904588899dc3295e10b4dd --- /dev/null +++ b/data/bench/w032_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ec48088615f9f7a3f80bfe61d130362a50d59d7de3d9bf2565f21f12212efbc5 +size 235507 diff --git a/data/bench/w033_gpu.png b/data/bench/w033_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..82f48a855310580b6c90540c46697ce8cf4b06ed --- /dev/null +++ b/data/bench/w033_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da90168078510f68b84b28617e270f0a33a85d374b9b0db0c565de2365511b1c +size 240148 diff --git a/data/bench/w034_gpu.png b/data/bench/w034_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..df0f4a721acdbf1558c59a1ad8a2bd52e19bdf7e --- /dev/null +++ b/data/bench/w034_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:015c6a53edb0e9c97c6bacd0f41a38b3ce42af3e1031a7debbdece3771e83dd8 +size 225942 diff --git a/data/bench/w035_gpu.png b/data/bench/w035_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..339e8a13bc0143952d42eaf7673bd8e9cecbef60 --- /dev/null +++ b/data/bench/w035_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:64fe35009ee7787f8d3afa7b41e2a62936fa8a1a00b80376d5013ad7229175da +size 193830 diff --git a/data/bench/w036_gpu.png b/data/bench/w036_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..a9759b1942578442b15f0b78815ffecfe74192dc --- /dev/null +++ b/data/bench/w036_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f326573cb16b362511bbbd8e11d51d9cae638c572134f5bf9acd83f983a7799e +size 204656 diff --git a/data/bench/w037_gpu.png b/data/bench/w037_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..cf8b3406725f87e23bdce291ea464bfb712e6d80 --- /dev/null +++ b/data/bench/w037_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4b1f3efc88ebc02f44ade6d426371c7ceaa0f45a12d68828cae8a2a0948bcb33 +size 214412 diff --git a/data/bench/w038_gpu.png b/data/bench/w038_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..116083e52987010c5c5ce0108fc0dca116b39172 --- /dev/null +++ b/data/bench/w038_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9e4b402fa9bc449ef35f0eb3edd3758ad1afafe4ac0178d05cd4245eaa74201 +size 206825 diff --git a/data/bench/w039_gpu.png b/data/bench/w039_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..e38719e041622c4c3a49a238a6409a639e737c9e --- /dev/null +++ b/data/bench/w039_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:373ddb854a2af7236703c3e4ebb58b148c335230fed2bb3e1498e8136c2c2870 +size 166449 diff --git a/data/bench/w040_gpu.png b/data/bench/w040_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..e1b9c5553854a4c35159ec89a922a76cda518aba --- /dev/null +++ b/data/bench/w040_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5114fe17084c037b12c0dd883d93e65b287bdabce02e19d22fc1bfa0da0fd518 +size 202731 diff --git a/data/bench/w041_gpu.png b/data/bench/w041_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..f5f16d68b22a427698408dcd030c4587443b2b5e --- /dev/null +++ b/data/bench/w041_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d57c948b18da6c3c15569b858ad0ee34af03c97bb25bf51d2959e123115127c5 +size 192739 diff --git a/data/bench/w042_gpu.png b/data/bench/w042_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..636a40f713c58e385df234d7dbc989466b296d40 --- /dev/null +++ b/data/bench/w042_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:726d4bc78e3256dc7076c9eae37d342f3253a436ee49a0a8993372f6c9746d3b +size 192407 diff --git a/data/bench/w043_gpu.png b/data/bench/w043_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..ebd348d242ce6d35f34de5a56ade1de173d20a3d --- /dev/null +++ b/data/bench/w043_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b92f6007d57a05b7c08958c60b5070e70f895a445a12f740f2ec8165d4be42ea +size 202658 diff --git a/data/bench/w044_gpu.png b/data/bench/w044_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..eb6fdd56923c4bcd4137f634e7a59e42068cea70 --- /dev/null +++ b/data/bench/w044_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5148db74ef0a5a4c705b7a6928656e36cbfaee0af8f42ed84bedb8df85199ae +size 199969 diff --git a/data/bench/w045_gpu.png b/data/bench/w045_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..d1c079c6f7aec0361f5cc2831c192e8cec4a9ff6 --- /dev/null +++ b/data/bench/w045_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fe8cd07604368288da0eb2d4da667ca69d0349b7646814a6db77438d9d0c7bf9 +size 187961 diff --git a/data/bench/w052_gpu.png b/data/bench/w052_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..0022dd85e64ffc1320ed09bedb5d1f0a8dc7a367 --- /dev/null +++ b/data/bench/w052_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c79565130088f5a8d0dd423ab64bebcf604e871a7a0960632578227e21f72278 +size 206686 diff --git a/data/bench/w053_gpu.png b/data/bench/w053_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..dd820d0ca88949895713fe48195752e78dfffcf8 --- /dev/null +++ b/data/bench/w053_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09c8e76abc3c5f1d4d979709bee06692946fd8b9b1054d262234ab41717ad319 +size 178692 diff --git a/data/bench/w054_gpu.png b/data/bench/w054_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..821254b7d9600a514b1fd018d9a4512ae7b91e2b --- /dev/null +++ b/data/bench/w054_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c3b3cf2331601645db43d56b402d49285a6ded4984bfcbcbc06582780dc05a80 +size 171613 diff --git a/data/bench/w055_gpu.png b/data/bench/w055_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..3e27e37c8752aec9666463680b924baec49a2e00 --- /dev/null +++ b/data/bench/w055_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc8f4c5ba397db806b852cb0075a23c008df3bdd2c8b554eee477d2c9dbaf86a +size 185577 diff --git a/data/bench/w056_gpu.png b/data/bench/w056_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..0a1f5b8738e145a6033c9b331e9d95d3e64aa3cf --- /dev/null +++ b/data/bench/w056_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2a59ccce65176ac716e507dd584b59128174330f1df1f285013921c87142539 +size 195101 diff --git a/data/bench/w057_gpu.png b/data/bench/w057_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..61192c13498724ff06d995a7c135605b7448449a --- /dev/null +++ b/data/bench/w057_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0baae2f99ec2b4d643ddcb471b2605e9543a2e95e9dd37772c18f4abf513c263 +size 206842 diff --git a/data/bench/w058_gpu.png b/data/bench/w058_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..6541b09dcc132c6097f43d24b3d47860bc7e4eb3 --- /dev/null +++ b/data/bench/w058_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1547ad9c0c3909c8d8fc1a25150f3133235b84ee065cb55efe860719bc14164 +size 203488 diff --git a/data/bench/w059_gpu.png b/data/bench/w059_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..e449e8cfcdccb96c110eb66da77906add09990e6 --- /dev/null +++ b/data/bench/w059_gpu.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f70d59d0ffde0448a8b1905418f972e0850b61416f105d7003d987bd64c6165d +size 189596 diff --git a/docs/BENCH.md b/docs/BENCH.md index 8f78ebdc054ece5b6fbda0c789d9830ee3643280..a7b3578a8787d5ed629496adfc9d0ea07954f5a5 100644 --- a/docs/BENCH.md +++ b/docs/BENCH.md @@ -16,9 +16,14 @@ Open it from the bar chart button in the [Unroll Space](https://huggingface.co/s > Every score is **agreement with another model's output**, the Vesuvius Challenge published ink detection on the 2.4 µm scan. It is not ground truth. A model can score lower because it is different, or even better. Use the numbers to find where detectors agree and disagree, then send the interesting regions to people who read Greek papyri. +## Researcher takeaway +On the public ink_9um run over all 37 windings (`docs/examples/bench/`), against the published 2.4 µm ink: median AP 0.319 for ink_9um, 0.117 for raw X-ray intensity and 0.089 for uniform noise, and ink_9um beats X-ray intensity on AP on every winding. On Pearson r the medians are 0.422, 0.266 and 0.000, but X-ray intensity outranks ink_9um on `w039`, `w053`, `w054` and `w057`. So: rank ink models by AP (threshold-free and robust to brightness), always report the X-ray floor next to your model, and look at windings where the floor wins before trusting a correlation. These are agreement scores with another model's output, not ground truth. + +The ink_9um model was trained by the Vesuvius Challenge and run by Chase Hendrick. + ## Reference grid -Each winding has a reference grid of 640 px on its long side (`data/bench/_ink.png`, `_ct.png`, and `_gpu.png` for w046 to w051), lossless 8 bit PNG, with checksums and pixels per mm in `data/bench/index.json` (about 8.3 px per mm, so 1 px is about 120 µm on the papyrus). They are built by `tools/make_bench_refs.py` from the page's own layers. All metrics are computed on this grid, after your prediction is resampled onto it (bilinear, 2x2 mean pooled first while it is more than twice the grid). +Each winding has a reference grid of 640 px on its long side (`data/bench/_ink.png`, `_ct.png`, and `_gpu.png`, the public ink_9um run), lossless 8 bit PNG, with checksums and pixels per mm in `data/bench/index.json` (about 8.3 px per mm, so 1 px is about 120 µm on the papyrus). They are built by `tools/make_bench_refs.py` from the page's own layers. All metrics are computed on this grid, after your prediction is resampled onto it (bilinear, 2x2 mean pooled first while it is more than twice the grid). ## Python and CLI @@ -88,7 +93,7 @@ B.core // ## Leaderboard -The Leaderboard tab loads any number of bundles from URLs (one per line, CORS) or files and ranks them per winding by Pearson r, AP, best F1, IoU or Dice, entirely in the browser. `?board=url1,url2` opens it preloaded. "Load examples" shows the public ink_9um runs next to two floors any ink model should beat: the X-ray intensity itself and uniform noise. +The Leaderboard tab loads any number of bundles from URLs (one per line, CORS) or files and ranks them per winding by Pearson r, AP, best F1, IoU or Dice, entirely in the browser. `?board=url1,url2` opens it preloaded. "Load examples" shows the public ink_9um run on all 37 windings next to two floors any ink model should beat: the X-ray intensity itself and uniform noise. ## How it is tested diff --git a/docs/examples/bench/w023-ct-baseline.json b/docs/examples/bench/w023-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..eea02f8c256f82f8883fd1546acd02cd5e3fc2bb --- /dev/null +++ b/docs/examples/bench/w023-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:47:44Z", + "scroll": "PHerc0139", + "segment": "w023", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w023_ink.png", + "sha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d", + "width": 589, + "height": 640, + "pxPerMM": 9.861, + "alignTargetFile": "data/bench/w023_ink.png", + "alignTargetSha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d" + }, + "prediction": { + "name": "w023_ct.png", + "sha256": "9e871edfb98d9c5455e3e8b6dae5bc2127b4b6c0ab2399ea5f80c9b9a38b1d2b", + "format": "png", + "width": 589, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.342535, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 376960, + "validFraction": 1.0, + "pearson": 0.342535, + "blurSigmaPx": 6.0, + "blurSigmaUM": 608.5, + "pearsonMatchedBlur": 0.464838, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.267945, + "dice": 0.422644, + "precision": 0.366091, + "recall": 0.499862, + "refPositiveFraction": 0.297854, + "predPositiveFraction": 0.40669, + "ap": 0.367097, + "bestF1": { + "f1": 0.528033, + "threshold": 0.04 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.413174, + "recall": 0.000615 + }, + { + "t": 0.98, + "precision": 0.419048, + "recall": 0.001568 + }, + { + "t": 0.96, + "precision": 0.393643, + "recall": 0.002868 + }, + { + "t": 0.94, + "precision": 0.371141, + "recall": 0.004605 + }, + { + "t": 0.92, + "precision": 0.381218, + "recall": 0.007303 + }, + { + "t": 0.9, + "precision": 0.379342, + "recall": 0.011186 + }, + { + "t": 0.88, + "precision": 0.369996, + "recall": 0.015969 + }, + { + "t": 0.86, + "precision": 0.372707, + "recall": 0.022622 + }, + { + "t": 0.84, + "precision": 0.374667, + "recall": 0.03135 + }, + { + "t": 0.82, + "precision": 0.376418, + "recall": 0.042849 + }, + { + "t": 0.8, + "precision": 0.374539, + "recall": 0.058827 + }, + { + "t": 0.78, + "precision": 0.374795, + "recall": 0.07541 + }, + { + "t": 0.76, + "precision": 0.376146, + "recall": 0.095022 + }, + { + "t": 0.74, + "precision": 0.37541, + "recall": 0.117128 + }, + { + "t": 0.72, + "precision": 0.37474, + "recall": 0.141246 + }, + { + "t": 0.7, + "precision": 0.37296, + "recall": 0.167342 + }, + { + "t": 0.68, + "precision": 0.372363, + "recall": 0.195745 + }, + { + "t": 0.66, + "precision": 0.371237, + "recall": 0.225608 + }, + { + "t": 0.64, + "precision": 0.37039, + "recall": 0.256531 + }, + { + "t": 0.62, + "precision": 0.369873, + "recall": 0.28952 + }, + { + "t": 0.6, + "precision": 0.369541, + "recall": 0.32951 + }, + { + "t": 0.58, + "precision": 0.368185, + "recall": 0.362721 + }, + { + "t": 0.56, + "precision": 0.367791, + "recall": 0.397118 + }, + { + "t": 0.54, + "precision": 0.36718, + "recall": 0.431336 + }, + { + "t": 0.52, + "precision": 0.366306, + "recall": 0.465439 + }, + { + "t": 0.5, + "precision": 0.366091, + "recall": 0.499862 + }, + { + "t": 0.48, + "precision": 0.366352, + "recall": 0.534802 + }, + { + "t": 0.46, + "precision": 0.366383, + "recall": 0.568842 + }, + { + "t": 0.44, + "precision": 0.366473, + "recall": 0.602437 + }, + { + "t": 0.42, + "precision": 0.365986, + "recall": 0.634233 + }, + { + "t": 0.4, + "precision": 0.365797, + "recall": 0.671889 + }, + { + "t": 0.38, + "precision": 0.366051, + "recall": 0.702411 + }, + { + "t": 0.36, + "precision": 0.365761, + "recall": 0.731054 + }, + { + "t": 0.34, + "precision": 0.365854, + "recall": 0.758984 + }, + { + "t": 0.32, + "precision": 0.365758, + "recall": 0.785 + }, + { + "t": 0.3, + "precision": 0.365498, + "recall": 0.808985 + }, + { + "t": 0.28, + "precision": 0.365094, + "recall": 0.830636 + }, + { + "t": 0.26, + "precision": 0.364846, + "recall": 0.851059 + }, + { + "t": 0.24, + "precision": 0.364451, + "recall": 0.869121 + }, + { + "t": 0.22, + "precision": 0.363978, + "recall": 0.885562 + }, + { + "t": 0.2, + "precision": 0.36361, + "recall": 0.903909 + }, + { + "t": 0.18, + "precision": 0.363437, + "recall": 0.917981 + }, + { + "t": 0.16, + "precision": 0.363229, + "recall": 0.930503 + }, + { + "t": 0.14, + "precision": 0.362969, + "recall": 0.941458 + }, + { + "t": 0.12, + "precision": 0.362743, + "recall": 0.951104 + }, + { + "t": 0.1, + "precision": 0.362424, + "recall": 0.959289 + }, + { + "t": 0.08, + "precision": 0.362018, + "recall": 0.966904 + }, + { + "t": 0.06, + "precision": 0.361708, + "recall": 0.973949 + }, + { + "t": 0.04, + "precision": 0.36137, + "recall": 0.980014 + }, + { + "t": 0.02, + "precision": 0.360408, + "recall": 0.985866 + }, + { + "t": 0.0, + "precision": 0.297854, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w023-ink9um.json b/docs/examples/bench/w023-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..d1c6fbe7f80a209800f1fe348073cc1281509580 --- /dev/null +++ b/docs/examples/bench/w023-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:47:43Z", + "scroll": "PHerc0139", + "segment": "w023", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w023_ink.png", + "sha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d", + "width": 589, + "height": 640, + "pxPerMM": 9.861, + "alignTargetFile": "data/bench/w023_ink.png", + "alignTargetSha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d" + }, + "prediction": { + "name": "w023_gpu.png", + "sha256": "14bedeb6a276f50d4783caec9774ccc4cf5899ddd67a0d57bf75dc6d04372d0c", + "format": "png", + "width": 589, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.563, + "ty": 0.5, + "sx": 1.0004, + "sy": 1.00157, + "ncc": 0.496932, + "coarseNcc": 0.676478, + "evals": 38395, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 376320, + "validFraction": 0.998302, + "pearson": 0.496932, + "blurSigmaPx": 2.5, + "blurSigmaUM": 253.5, + "pearsonMatchedBlur": 0.537678, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.322425, + "dice": 0.487627, + "precision": 0.611897, + "recall": 0.405312, + "refPositiveFraction": 0.29836, + "predPositiveFraction": 0.19763, + "ap": 0.572829, + "bestF1": { + "f1": 0.58117, + "threshold": 0.12 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.891379, + "recall": 0.004605 + }, + { + "t": 0.98, + "precision": 0.809298, + "recall": 0.069773 + }, + { + "t": 0.96, + "precision": 0.788433, + "recall": 0.091308 + }, + { + "t": 0.94, + "precision": 0.77342, + "recall": 0.109415 + }, + { + "t": 0.92, + "precision": 0.760949, + "recall": 0.12534 + }, + { + "t": 0.9, + "precision": 0.751885, + "recall": 0.140293 + }, + { + "t": 0.88, + "precision": 0.743538, + "recall": 0.154232 + }, + { + "t": 0.86, + "precision": 0.736346, + "recall": 0.168473 + }, + { + "t": 0.84, + "precision": 0.728813, + "recall": 0.181984 + }, + { + "t": 0.82, + "precision": 0.720644, + "recall": 0.195246 + }, + { + "t": 0.8, + "precision": 0.713955, + "recall": 0.209051 + }, + { + "t": 0.78, + "precision": 0.706978, + "recall": 0.222686 + }, + { + "t": 0.76, + "precision": 0.699595, + "recall": 0.235583 + }, + { + "t": 0.74, + "precision": 0.691579, + "recall": 0.248319 + }, + { + "t": 0.72, + "precision": 0.683979, + "recall": 0.260734 + }, + { + "t": 0.7, + "precision": 0.677081, + "recall": 0.273488 + }, + { + "t": 0.68, + "precision": 0.669728, + "recall": 0.28577 + }, + { + "t": 0.66, + "precision": 0.663324, + "recall": 0.298729 + }, + { + "t": 0.64, + "precision": 0.655696, + "recall": 0.311225 + }, + { + "t": 0.62, + "precision": 0.650146, + "recall": 0.325261 + }, + { + "t": 0.6, + "precision": 0.643842, + "recall": 0.338656 + }, + { + "t": 0.58, + "precision": 0.636991, + "recall": 0.351063 + }, + { + "t": 0.56, + "precision": 0.630731, + "recall": 0.364387 + }, + { + "t": 0.54, + "precision": 0.624137, + "recall": 0.378299 + }, + { + "t": 0.52, + "precision": 0.617467, + "recall": 0.392041 + }, + { + "t": 0.5, + "precision": 0.611897, + "recall": 0.405312 + }, + { + "t": 0.48, + "precision": 0.606205, + "recall": 0.419936 + }, + { + "t": 0.46, + "precision": 0.6, + "recall": 0.434747 + }, + { + "t": 0.44, + "precision": 0.594035, + "recall": 0.449149 + }, + { + "t": 0.42, + "precision": 0.587628, + "recall": 0.46404 + }, + { + "t": 0.4, + "precision": 0.581228, + "recall": 0.478896 + }, + { + "t": 0.38, + "precision": 0.57507, + "recall": 0.494269 + }, + { + "t": 0.36, + "precision": 0.56844, + "recall": 0.510122 + }, + { + "t": 0.34, + "precision": 0.561731, + "recall": 0.526269 + }, + { + "t": 0.32, + "precision": 0.554684, + "recall": 0.543405 + }, + { + "t": 0.3, + "precision": 0.548081, + "recall": 0.561841 + }, + { + "t": 0.28, + "precision": 0.540665, + "recall": 0.579645 + }, + { + "t": 0.26, + "precision": 0.53327, + "recall": 0.597868 + }, + { + "t": 0.24, + "precision": 0.525033, + "recall": 0.617177 + }, + { + "t": 0.22, + "precision": 0.517306, + "recall": 0.63816 + }, + { + "t": 0.2, + "precision": 0.509064, + "recall": 0.6595 + }, + { + "t": 0.18, + "precision": 0.500298, + "recall": 0.681232 + }, + { + "t": 0.16, + "precision": 0.49167, + "recall": 0.704442 + }, + { + "t": 0.14, + "precision": 0.482454, + "recall": 0.728907 + }, + { + "t": 0.12, + "precision": 0.472721, + "recall": 0.754193 + }, + { + "t": 0.1, + "precision": 0.461733, + "recall": 0.781384 + }, + { + "t": 0.08, + "precision": 0.449505, + "recall": 0.810169 + }, + { + "t": 0.06, + "precision": 0.435419, + "recall": 0.842161 + }, + { + "t": 0.04, + "precision": 0.418136, + "recall": 0.878882 + }, + { + "t": 0.02, + "precision": 0.394628, + "recall": 0.922425 + }, + { + "t": 0.0, + "precision": 0.29836, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w023-noise-baseline.json b/docs/examples/bench/w023-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..b5f89416d225575f504d00ba6208986163776b27 --- /dev/null +++ b/docs/examples/bench/w023-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:47:45Z", + "scroll": "PHerc0139", + "segment": "w023", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w023_ink.png", + "sha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d", + "width": 589, + "height": 640, + "pxPerMM": 9.861, + "alignTargetFile": "data/bench/w023_ink.png", + "alignTargetSha256": "6e5a5d962eab758fe30776ac7db2ef8ea539c16c99938cc04bbaee5fa729839d" + }, + "prediction": { + "name": "noise-seed23.png", + "sha256": "f66295cef27e849450ae58b39bb65cbfc7af90af40f3faad11e838e28f4e08f9", + "format": "png", + "width": 589, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(23).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.001026, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 376960, + "validFraction": 1.0, + "pearson": 0.001026, + "blurSigmaPx": 0.5, + "blurSigmaUM": 50.7, + "pearsonMatchedBlur": 0.001057, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.229363, + "dice": 0.373141, + "precision": 0.298139, + "recall": 0.498562, + "refPositiveFraction": 0.297854, + "predPositiveFraction": 0.498085, + "ap": 0.298344, + "bestF1": { + "f1": 0.458994, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.296047, + "recall": 0.019612 + }, + { + "t": 0.96, + "precision": 0.296167, + "recall": 0.039437 + }, + { + "t": 0.94, + "precision": 0.298428, + "recall": 0.059521 + }, + { + "t": 0.92, + "precision": 0.300548, + "recall": 0.079579 + }, + { + "t": 0.9, + "precision": 0.300801, + "recall": 0.099645 + }, + { + "t": 0.88, + "precision": 0.298906, + "recall": 0.118749 + }, + { + "t": 0.86, + "precision": 0.299743, + "recall": 0.138289 + }, + { + "t": 0.84, + "precision": 0.299664, + "recall": 0.157875 + }, + { + "t": 0.82, + "precision": 0.299698, + "recall": 0.177602 + }, + { + "t": 0.8, + "precision": 0.300253, + "recall": 0.201881 + }, + { + "t": 0.78, + "precision": 0.299585, + "recall": 0.221003 + }, + { + "t": 0.76, + "precision": 0.298898, + "recall": 0.240116 + }, + { + "t": 0.74, + "precision": 0.299281, + "recall": 0.26053 + }, + { + "t": 0.72, + "precision": 0.299505, + "recall": 0.280275 + }, + { + "t": 0.7, + "precision": 0.299145, + "recall": 0.299566 + }, + { + "t": 0.68, + "precision": 0.29831, + "recall": 0.318813 + }, + { + "t": 0.66, + "precision": 0.297711, + "recall": 0.33773 + }, + { + "t": 0.64, + "precision": 0.297573, + "recall": 0.356825 + }, + { + "t": 0.62, + "precision": 0.29771, + "recall": 0.376526 + }, + { + "t": 0.6, + "precision": 0.298141, + "recall": 0.4002 + }, + { + "t": 0.58, + "precision": 0.298043, + "recall": 0.419517 + }, + { + "t": 0.56, + "precision": 0.297849, + "recall": 0.439201 + }, + { + "t": 0.54, + "precision": 0.297855, + "recall": 0.458973 + }, + { + "t": 0.52, + "precision": 0.297817, + "recall": 0.478398 + }, + { + "t": 0.5, + "precision": 0.298139, + "recall": 0.498562 + }, + { + "t": 0.48, + "precision": 0.297952, + "recall": 0.517612 + }, + { + "t": 0.46, + "precision": 0.29801, + "recall": 0.537171 + }, + { + "t": 0.44, + "precision": 0.298175, + "recall": 0.55664 + }, + { + "t": 0.42, + "precision": 0.29786, + "recall": 0.57545 + }, + { + "t": 0.4, + "precision": 0.298317, + "recall": 0.600059 + }, + { + "t": 0.38, + "precision": 0.298162, + "recall": 0.619359 + }, + { + "t": 0.36, + "precision": 0.298334, + "recall": 0.639487 + }, + { + "t": 0.34, + "precision": 0.298307, + "recall": 0.658814 + }, + { + "t": 0.32, + "precision": 0.298448, + "recall": 0.679103 + }, + { + "t": 0.3, + "precision": 0.29852, + "recall": 0.699009 + }, + { + "t": 0.28, + "precision": 0.298225, + "recall": 0.718211 + }, + { + "t": 0.26, + "precision": 0.298215, + "recall": 0.737698 + }, + { + "t": 0.24, + "precision": 0.298211, + "recall": 0.757354 + }, + { + "t": 0.22, + "precision": 0.298169, + "recall": 0.77669 + }, + { + "t": 0.2, + "precision": 0.298193, + "recall": 0.800123 + }, + { + "t": 0.18, + "precision": 0.298001, + "recall": 0.819209 + }, + { + "t": 0.16, + "precision": 0.297842, + "recall": 0.838536 + }, + { + "t": 0.14, + "precision": 0.297808, + "recall": 0.858308 + }, + { + "t": 0.12, + "precision": 0.297846, + "recall": 0.877893 + }, + { + "t": 0.1, + "precision": 0.297635, + "recall": 0.89722 + }, + { + "t": 0.08, + "precision": 0.297763, + "recall": 0.91734 + }, + { + "t": 0.06, + "precision": 0.297714, + "recall": 0.936551 + }, + { + "t": 0.04, + "precision": 0.297641, + "recall": 0.95635 + }, + { + "t": 0.02, + "precision": 0.29774, + "recall": 0.97622 + }, + { + "t": 0.0, + "precision": 0.297854, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w024-ct-baseline.json b/docs/examples/bench/w024-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..72e5531fddebe32046604bbd687aef212ff07c21 --- /dev/null +++ b/docs/examples/bench/w024-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:47:56Z", + "scroll": "PHerc0139", + "segment": "w024", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w024_ink.png", + "sha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327", + "width": 597, + "height": 640, + "pxPerMM": 9.6321, + "alignTargetFile": "data/bench/w024_ink.png", + "alignTargetSha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327" + }, + "prediction": { + "name": "w024_ct.png", + "sha256": "7f21087fc1b5a1a5812e02d24134004524373408d5fe533939f12003f90c6517", + "format": "png", + "width": 597, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.361809, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 382080, + "validFraction": 1.0, + "pearson": 0.361809, + "blurSigmaPx": 6.0, + "blurSigmaUM": 622.9, + "pearsonMatchedBlur": 0.464513, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.288116, + "dice": 0.447345, + "precision": 0.418449, + "recall": 0.480526, + "refPositiveFraction": 0.333917, + "predPositiveFraction": 0.383454, + "ap": 0.41647, + "bestF1": { + "f1": 0.574501, + "threshold": 0.02 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.382353, + "recall": 0.000509 + }, + { + "t": 0.98, + "precision": 0.38412, + "recall": 0.001403 + }, + { + "t": 0.96, + "precision": 0.376357, + "recall": 0.002445 + }, + { + "t": 0.94, + "precision": 0.380223, + "recall": 0.00428 + }, + { + "t": 0.92, + "precision": 0.392257, + "recall": 0.006592 + }, + { + "t": 0.9, + "precision": 0.396662, + "recall": 0.009688 + }, + { + "t": 0.88, + "precision": 0.401505, + "recall": 0.013803 + }, + { + "t": 0.86, + "precision": 0.403503, + "recall": 0.01914 + }, + { + "t": 0.84, + "precision": 0.411504, + "recall": 0.026242 + }, + { + "t": 0.82, + "precision": 0.416875, + "recall": 0.03524 + }, + { + "t": 0.8, + "precision": 0.419535, + "recall": 0.048815 + }, + { + "t": 0.78, + "precision": 0.424343, + "recall": 0.063261 + }, + { + "t": 0.76, + "precision": 0.425433, + "recall": 0.080136 + }, + { + "t": 0.74, + "precision": 0.42628, + "recall": 0.099574 + }, + { + "t": 0.72, + "precision": 0.428473, + "recall": 0.122171 + }, + { + "t": 0.7, + "precision": 0.427396, + "recall": 0.14661 + }, + { + "t": 0.68, + "precision": 0.426788, + "recall": 0.173056 + }, + { + "t": 0.66, + "precision": 0.426028, + "recall": 0.202033 + }, + { + "t": 0.64, + "precision": 0.42663, + "recall": 0.233558 + }, + { + "t": 0.62, + "precision": 0.425587, + "recall": 0.265247 + }, + { + "t": 0.6, + "precision": 0.423207, + "recall": 0.305135 + }, + { + "t": 0.58, + "precision": 0.422423, + "recall": 0.339881 + }, + { + "t": 0.56, + "precision": 0.421306, + "recall": 0.374979 + }, + { + "t": 0.54, + "precision": 0.420412, + "recall": 0.409827 + }, + { + "t": 0.52, + "precision": 0.419469, + "recall": 0.445318 + }, + { + "t": 0.5, + "precision": 0.418449, + "recall": 0.480526 + }, + { + "t": 0.48, + "precision": 0.41824, + "recall": 0.516323 + }, + { + "t": 0.46, + "precision": 0.417336, + "recall": 0.550254 + }, + { + "t": 0.44, + "precision": 0.417034, + "recall": 0.584278 + }, + { + "t": 0.42, + "precision": 0.416464, + "recall": 0.617206 + }, + { + "t": 0.4, + "precision": 0.415636, + "recall": 0.655268 + }, + { + "t": 0.38, + "precision": 0.415486, + "recall": 0.68593 + }, + { + "t": 0.36, + "precision": 0.415521, + "recall": 0.715417 + }, + { + "t": 0.34, + "precision": 0.415132, + "recall": 0.742732 + }, + { + "t": 0.32, + "precision": 0.414668, + "recall": 0.768935 + }, + { + "t": 0.3, + "precision": 0.414072, + "recall": 0.793272 + }, + { + "t": 0.28, + "precision": 0.413718, + "recall": 0.816018 + }, + { + "t": 0.26, + "precision": 0.413501, + "recall": 0.837298 + }, + { + "t": 0.24, + "precision": 0.41271, + "recall": 0.856149 + }, + { + "t": 0.22, + "precision": 0.412062, + "recall": 0.873102 + }, + { + "t": 0.2, + "precision": 0.411404, + "recall": 0.891506 + }, + { + "t": 0.18, + "precision": 0.410778, + "recall": 0.904682 + }, + { + "t": 0.16, + "precision": 0.410245, + "recall": 0.917528 + }, + { + "t": 0.14, + "precision": 0.409727, + "recall": 0.928776 + }, + { + "t": 0.12, + "precision": 0.40926, + "recall": 0.939028 + }, + { + "t": 0.1, + "precision": 0.408687, + "recall": 0.948434 + }, + { + "t": 0.08, + "precision": 0.408117, + "recall": 0.95722 + }, + { + "t": 0.06, + "precision": 0.407705, + "recall": 0.965481 + }, + { + "t": 0.04, + "precision": 0.40722, + "recall": 0.97328 + }, + { + "t": 0.02, + "precision": 0.406165, + "recall": 0.981134 + }, + { + "t": 0.0, + "precision": 0.333917, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w024-ink9um.json b/docs/examples/bench/w024-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..334c9993105567ceda4d0388dc462e824b8002d1 --- /dev/null +++ b/docs/examples/bench/w024-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:47:56Z", + "scroll": "PHerc0139", + "segment": "w024", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w024_ink.png", + "sha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327", + "width": 597, + "height": 640, + "pxPerMM": 9.6321, + "alignTargetFile": "data/bench/w024_ink.png", + "alignTargetSha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327" + }, + "prediction": { + "name": "w024_gpu.png", + "sha256": "cc11fb74a30bc3fb48bd652946e3e9124d1a2d90947f613f715da37671a85efd", + "format": "png", + "width": 597, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.531, + "ty": 0.5, + "sx": 1.00018, + "sy": 1.00157, + "ncc": 0.572218, + "coarseNcc": 0.724818, + "evals": 40355, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 381440, + "validFraction": 0.998325, + "pearson": 0.572218, + "blurSigmaPx": 2.0, + "blurSigmaUM": 207.6, + "pearsonMatchedBlur": 0.608043, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.396013, + "dice": 0.567349, + "precision": 0.706089, + "recall": 0.474178, + "refPositiveFraction": 0.334477, + "predPositiveFraction": 0.22462, + "ap": 0.670837, + "bestF1": { + "f1": 0.644796, + "threshold": 0.14 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.932029, + "recall": 0.007093 + }, + { + "t": 0.98, + "precision": 0.874482, + "recall": 0.097638 + }, + { + "t": 0.96, + "precision": 0.86084, + "recall": 0.125675 + }, + { + "t": 0.94, + "precision": 0.851634, + "recall": 0.14748 + }, + { + "t": 0.92, + "precision": 0.843096, + "recall": 0.166401 + }, + { + "t": 0.9, + "precision": 0.83502, + "recall": 0.183677 + }, + { + "t": 0.88, + "precision": 0.827887, + "recall": 0.199972 + }, + { + "t": 0.86, + "precision": 0.820642, + "recall": 0.21582 + }, + { + "t": 0.84, + "precision": 0.813773, + "recall": 0.230814 + }, + { + "t": 0.82, + "precision": 0.807815, + "recall": 0.246796 + }, + { + "t": 0.8, + "precision": 0.801974, + "recall": 0.26172 + }, + { + "t": 0.78, + "precision": 0.795671, + "recall": 0.277169 + }, + { + "t": 0.76, + "precision": 0.790217, + "recall": 0.292382 + }, + { + "t": 0.74, + "precision": 0.783991, + "recall": 0.306836 + }, + { + "t": 0.72, + "precision": 0.777302, + "recall": 0.320333 + }, + { + "t": 0.7, + "precision": 0.771313, + "recall": 0.334496 + }, + { + "t": 0.68, + "precision": 0.764752, + "recall": 0.348824 + }, + { + "t": 0.66, + "precision": 0.757991, + "recall": 0.36225 + }, + { + "t": 0.64, + "precision": 0.751048, + "recall": 0.376328 + }, + { + "t": 0.62, + "precision": 0.744027, + "recall": 0.39006 + }, + { + "t": 0.6, + "precision": 0.737464, + "recall": 0.403925 + }, + { + "t": 0.58, + "precision": 0.730956, + "recall": 0.418167 + }, + { + "t": 0.56, + "precision": 0.724791, + "recall": 0.432228 + }, + { + "t": 0.54, + "precision": 0.719011, + "recall": 0.446415 + }, + { + "t": 0.52, + "precision": 0.71274, + "recall": 0.460516 + }, + { + "t": 0.5, + "precision": 0.706089, + "recall": 0.474178 + }, + { + "t": 0.48, + "precision": 0.699017, + "recall": 0.487761 + }, + { + "t": 0.46, + "precision": 0.69215, + "recall": 0.502065 + }, + { + "t": 0.44, + "precision": 0.684481, + "recall": 0.516605 + }, + { + "t": 0.42, + "precision": 0.676919, + "recall": 0.530619 + }, + { + "t": 0.4, + "precision": 0.669169, + "recall": 0.545026 + }, + { + "t": 0.38, + "precision": 0.66149, + "recall": 0.560216 + }, + { + "t": 0.36, + "precision": 0.654187, + "recall": 0.575751 + }, + { + "t": 0.34, + "precision": 0.646507, + "recall": 0.591192 + }, + { + "t": 0.32, + "precision": 0.638707, + "recall": 0.607032 + }, + { + "t": 0.3, + "precision": 0.630614, + "recall": 0.62321 + }, + { + "t": 0.28, + "precision": 0.622001, + "recall": 0.640015 + }, + { + "t": 0.26, + "precision": 0.613529, + "recall": 0.657313 + }, + { + "t": 0.24, + "precision": 0.604511, + "recall": 0.674471 + }, + { + "t": 0.22, + "precision": 0.594999, + "recall": 0.692655 + }, + { + "t": 0.2, + "precision": 0.585037, + "recall": 0.711153 + }, + { + "t": 0.18, + "precision": 0.57511, + "recall": 0.73005 + }, + { + "t": 0.16, + "precision": 0.564667, + "recall": 0.750288 + }, + { + "t": 0.14, + "precision": 0.553916, + "recall": 0.771349 + }, + { + "t": 0.12, + "precision": 0.542048, + "recall": 0.792927 + }, + { + "t": 0.1, + "precision": 0.528424, + "recall": 0.815281 + }, + { + "t": 0.08, + "precision": 0.513703, + "recall": 0.839924 + }, + { + "t": 0.06, + "precision": 0.496954, + "recall": 0.866887 + }, + { + "t": 0.04, + "precision": 0.476629, + "recall": 0.897479 + }, + { + "t": 0.02, + "precision": 0.448327, + "recall": 0.934451 + }, + { + "t": 0.0, + "precision": 0.334477, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w024-noise-baseline.json b/docs/examples/bench/w024-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..e9effecc9998a7f71f970e9552a8fe93d6af6e97 --- /dev/null +++ b/docs/examples/bench/w024-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:47:58Z", + "scroll": "PHerc0139", + "segment": "w024", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w024_ink.png", + "sha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327", + "width": 597, + "height": 640, + "pxPerMM": 9.6321, + "alignTargetFile": "data/bench/w024_ink.png", + "alignTargetSha256": "496166c98593362e8fb6d3e10f489397028271d4f51714a05b7435aab685f327" + }, + "prediction": { + "name": "noise-seed24.png", + "sha256": "0d3e3dbf7db544bf85db22e4904c2170188eff607aa99dba92f106800bd7cdc4", + "format": "png", + "width": 597, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(24).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": -0.000154, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 382080, + "validFraction": 1.0, + "pearson": -0.000154, + "blurSigmaPx": 6.0, + "blurSigmaUM": 622.9, + "pearsonMatchedBlur": 0.000533, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.249425, + "dice": 0.399263, + "precision": 0.333244, + "recall": 0.497903, + "refPositiveFraction": 0.333917, + "predPositiveFraction": 0.498909, + "ap": 0.334221, + "bestF1": { + "f1": 0.500656, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.329879, + "recall": 0.019211 + }, + { + "t": 0.96, + "precision": 0.335803, + "recall": 0.039425 + }, + { + "t": 0.94, + "precision": 0.334846, + "recall": 0.058989 + }, + { + "t": 0.92, + "precision": 0.336679, + "recall": 0.079141 + }, + { + "t": 0.9, + "precision": 0.336394, + "recall": 0.098501 + }, + { + "t": 0.88, + "precision": 0.335897, + "recall": 0.118448 + }, + { + "t": 0.86, + "precision": 0.337719, + "recall": 0.139211 + }, + { + "t": 0.84, + "precision": 0.33856, + "recall": 0.159598 + }, + { + "t": 0.82, + "precision": 0.337444, + "recall": 0.179107 + }, + { + "t": 0.8, + "precision": 0.335787, + "recall": 0.202339 + }, + { + "t": 0.78, + "precision": 0.335545, + "recall": 0.22155 + }, + { + "t": 0.76, + "precision": 0.335695, + "recall": 0.241411 + }, + { + "t": 0.74, + "precision": 0.335913, + "recall": 0.261273 + }, + { + "t": 0.72, + "precision": 0.335316, + "recall": 0.280531 + }, + { + "t": 0.7, + "precision": 0.33509, + "recall": 0.299954 + }, + { + "t": 0.68, + "precision": 0.334654, + "recall": 0.31922 + }, + { + "t": 0.66, + "precision": 0.334034, + "recall": 0.338713 + }, + { + "t": 0.64, + "precision": 0.334197, + "recall": 0.3579 + }, + { + "t": 0.62, + "precision": 0.333655, + "recall": 0.377229 + }, + { + "t": 0.6, + "precision": 0.333989, + "recall": 0.400618 + }, + { + "t": 0.58, + "precision": 0.333871, + "recall": 0.420079 + }, + { + "t": 0.56, + "precision": 0.3337, + "recall": 0.439651 + }, + { + "t": 0.54, + "precision": 0.333426, + "recall": 0.459089 + }, + { + "t": 0.52, + "precision": 0.33325, + "recall": 0.478434 + }, + { + "t": 0.5, + "precision": 0.333244, + "recall": 0.497903 + }, + { + "t": 0.48, + "precision": 0.332977, + "recall": 0.517052 + }, + { + "t": 0.46, + "precision": 0.333027, + "recall": 0.5366 + }, + { + "t": 0.44, + "precision": 0.333189, + "recall": 0.556649 + }, + { + "t": 0.42, + "precision": 0.333277, + "recall": 0.576519 + }, + { + "t": 0.4, + "precision": 0.333066, + "recall": 0.599375 + }, + { + "t": 0.38, + "precision": 0.333098, + "recall": 0.618695 + }, + { + "t": 0.36, + "precision": 0.333016, + "recall": 0.638126 + }, + { + "t": 0.34, + "precision": 0.333275, + "recall": 0.658701 + }, + { + "t": 0.32, + "precision": 0.333309, + "recall": 0.678264 + }, + { + "t": 0.3, + "precision": 0.333411, + "recall": 0.697867 + }, + { + "t": 0.28, + "precision": 0.333264, + "recall": 0.71689 + }, + { + "t": 0.26, + "precision": 0.333387, + "recall": 0.736462 + }, + { + "t": 0.24, + "precision": 0.333415, + "recall": 0.756378 + }, + { + "t": 0.22, + "precision": 0.33365, + "recall": 0.776412 + }, + { + "t": 0.2, + "precision": 0.333478, + "recall": 0.799393 + }, + { + "t": 0.18, + "precision": 0.33341, + "recall": 0.818667 + }, + { + "t": 0.16, + "precision": 0.333651, + "recall": 0.83856 + }, + { + "t": 0.14, + "precision": 0.333609, + "recall": 0.858179 + }, + { + "t": 0.12, + "precision": 0.333629, + "recall": 0.877554 + }, + { + "t": 0.1, + "precision": 0.333554, + "recall": 0.897188 + }, + { + "t": 0.08, + "precision": 0.333709, + "recall": 0.917552 + }, + { + "t": 0.06, + "precision": 0.333833, + "recall": 0.937225 + }, + { + "t": 0.04, + "precision": 0.333941, + "recall": 0.957063 + }, + { + "t": 0.02, + "precision": 0.333927, + "recall": 0.976752 + }, + { + "t": 0.0, + "precision": 0.333917, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w025-ct-baseline.json b/docs/examples/bench/w025-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..651f44a316ebe1006810ef6b252496e6c4e2a7fa --- /dev/null +++ b/docs/examples/bench/w025-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:48:10Z", + "scroll": "PHerc0139", + "segment": "w025", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w025_ink.png", + "sha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8", + "width": 598, + "height": 640, + "pxPerMM": 9.3937, + "alignTargetFile": "data/bench/w025_ink.png", + "alignTargetSha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8" + }, + "prediction": { + "name": "w025_ct.png", + "sha256": "e30f050d29ccc83eaf9b29beafe66f0a47a35ac2a7f30dbd525f462c9ea325fa", + "format": "png", + "width": 598, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.359953, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 382720, + "validFraction": 1.0, + "pearson": 0.359953, + "blurSigmaPx": 6.0, + "blurSigmaUM": 638.7, + "pearsonMatchedBlur": 0.437292, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.204451, + "dice": 0.339493, + "precision": 0.236587, + "recall": 0.600833, + "refPositiveFraction": 0.155516, + "predPositiveFraction": 0.394947, + "ap": 0.242417, + "bestF1": { + "f1": 0.343571, + "threshold": 0.44 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.259459, + "recall": 0.000806 + }, + { + "t": 0.98, + "precision": 0.320594, + "recall": 0.002537 + }, + { + "t": 0.96, + "precision": 0.293318, + "recall": 0.004352 + }, + { + "t": 0.94, + "precision": 0.300469, + "recall": 0.007527 + }, + { + "t": 0.92, + "precision": 0.283003, + "recall": 0.011526 + }, + { + "t": 0.9, + "precision": 0.287168, + "recall": 0.018011 + }, + { + "t": 0.88, + "precision": 0.287614, + "recall": 0.026529 + }, + { + "t": 0.86, + "precision": 0.286919, + "recall": 0.037148 + }, + { + "t": 0.84, + "precision": 0.287639, + "recall": 0.050942 + }, + { + "t": 0.82, + "precision": 0.287651, + "recall": 0.068449 + }, + { + "t": 0.8, + "precision": 0.282144, + "recall": 0.091433 + }, + { + "t": 0.78, + "precision": 0.27808, + "recall": 0.114266 + }, + { + "t": 0.76, + "precision": 0.275642, + "recall": 0.141283 + }, + { + "t": 0.74, + "precision": 0.273569, + "recall": 0.170349 + }, + { + "t": 0.72, + "precision": 0.271296, + "recall": 0.20296 + }, + { + "t": 0.7, + "precision": 0.268313, + "recall": 0.236563 + }, + { + "t": 0.68, + "precision": 0.265416, + "recall": 0.272266 + }, + { + "t": 0.66, + "precision": 0.26319, + "recall": 0.309935 + }, + { + "t": 0.64, + "precision": 0.259919, + "recall": 0.34725 + }, + { + "t": 0.62, + "precision": 0.257365, + "recall": 0.38544 + }, + { + "t": 0.6, + "precision": 0.253624, + "recall": 0.430652 + }, + { + "t": 0.58, + "precision": 0.249869, + "recall": 0.466422 + }, + { + "t": 0.56, + "precision": 0.24668, + "recall": 0.502159 + }, + { + "t": 0.54, + "precision": 0.243306, + "recall": 0.536501 + }, + { + "t": 0.52, + "precision": 0.239897, + "recall": 0.569129 + }, + { + "t": 0.5, + "precision": 0.236587, + "recall": 0.600833 + }, + { + "t": 0.48, + "precision": 0.233961, + "recall": 0.632302 + }, + { + "t": 0.46, + "precision": 0.231197, + "recall": 0.662108 + }, + { + "t": 0.44, + "precision": 0.228708, + "recall": 0.690217 + }, + { + "t": 0.42, + "precision": 0.225623, + "recall": 0.715251 + }, + { + "t": 0.4, + "precision": 0.222726, + "recall": 0.74472 + }, + { + "t": 0.38, + "precision": 0.220147, + "recall": 0.767167 + }, + { + "t": 0.36, + "precision": 0.217627, + "recall": 0.787631 + }, + { + "t": 0.34, + "precision": 0.215409, + "recall": 0.807238 + }, + { + "t": 0.32, + "precision": 0.213241, + "recall": 0.825467 + }, + { + "t": 0.3, + "precision": 0.211177, + "recall": 0.842319 + }, + { + "t": 0.28, + "precision": 0.209103, + "recall": 0.857172 + }, + { + "t": 0.26, + "precision": 0.20715, + "recall": 0.870781 + }, + { + "t": 0.24, + "precision": 0.20542, + "recall": 0.883936 + }, + { + "t": 0.22, + "precision": 0.203667, + "recall": 0.895193 + }, + { + "t": 0.2, + "precision": 0.201809, + "recall": 0.908701 + }, + { + "t": 0.18, + "precision": 0.200306, + "recall": 0.919017 + }, + { + "t": 0.16, + "precision": 0.198907, + "recall": 0.928443 + }, + { + "t": 0.14, + "precision": 0.197644, + "recall": 0.937953 + }, + { + "t": 0.12, + "precision": 0.196428, + "recall": 0.946588 + }, + { + "t": 0.1, + "precision": 0.195035, + "recall": 0.954065 + }, + { + "t": 0.08, + "precision": 0.193604, + "recall": 0.961038 + }, + { + "t": 0.06, + "precision": 0.192206, + "recall": 0.968111 + }, + { + "t": 0.04, + "precision": 0.191009, + "recall": 0.975554 + }, + { + "t": 0.02, + "precision": 0.189517, + "recall": 0.982661 + }, + { + "t": 0.0, + "precision": 0.155516, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w025-ink9um.json b/docs/examples/bench/w025-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..00fbc3772d97ff9b71394fa79e70154a27a60829 --- /dev/null +++ b/docs/examples/bench/w025-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:48:09Z", + "scroll": "PHerc0139", + "segment": "w025", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w025_ink.png", + "sha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8", + "width": 598, + "height": 640, + "pxPerMM": 9.3937, + "alignTargetFile": "data/bench/w025_ink.png", + "alignTargetSha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8" + }, + "prediction": { + "name": "w025_gpu.png", + "sha256": "c129d596d1fe215b2a5c671eff5143e938bd4c4d34ac1a6c7b9843d1f263f226", + "format": "png", + "width": 598, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.531, + "ty": 0.594, + "sx": 1.00403, + "sy": 1.00007, + "ncc": 0.592898, + "coarseNcc": 0.749308, + "evals": 40350, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 382122, + "validFraction": 0.998438, + "pearson": 0.592898, + "blurSigmaPx": 2.0, + "blurSigmaUM": 212.9, + "pearsonMatchedBlur": 0.623003, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.36052, + "dice": 0.529974, + "precision": 0.61375, + "recall": 0.466322, + "refPositiveFraction": 0.155759, + "predPositiveFraction": 0.118344, + "ap": 0.562828, + "bestF1": { + "f1": 0.547703, + "threshold": 0.34 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.968254, + "recall": 0.006149 + }, + { + "t": 0.98, + "precision": 0.85798, + "recall": 0.110637 + }, + { + "t": 0.96, + "precision": 0.843237, + "recall": 0.13972 + }, + { + "t": 0.94, + "precision": 0.829066, + "recall": 0.161595 + }, + { + "t": 0.92, + "precision": 0.816653, + "recall": 0.180279 + }, + { + "t": 0.9, + "precision": 0.806396, + "recall": 0.197836 + }, + { + "t": 0.88, + "precision": 0.794865, + "recall": 0.214318 + }, + { + "t": 0.86, + "precision": 0.783196, + "recall": 0.22991 + }, + { + "t": 0.84, + "precision": 0.773132, + "recall": 0.243855 + }, + { + "t": 0.82, + "precision": 0.763133, + "recall": 0.258472 + }, + { + "t": 0.8, + "precision": 0.755407, + "recall": 0.272888 + }, + { + "t": 0.78, + "precision": 0.746371, + "recall": 0.285976 + }, + { + "t": 0.76, + "precision": 0.737928, + "recall": 0.300408 + }, + { + "t": 0.74, + "precision": 0.728903, + "recall": 0.314185 + }, + { + "t": 0.72, + "precision": 0.720131, + "recall": 0.327912 + }, + { + "t": 0.7, + "precision": 0.71035, + "recall": 0.341336 + }, + { + "t": 0.68, + "precision": 0.701014, + "recall": 0.354223 + }, + { + "t": 0.66, + "precision": 0.691092, + "recall": 0.367311 + }, + { + "t": 0.64, + "precision": 0.682133, + "recall": 0.380131 + }, + { + "t": 0.62, + "precision": 0.671493, + "recall": 0.392715 + }, + { + "t": 0.6, + "precision": 0.661459, + "recall": 0.405484 + }, + { + "t": 0.58, + "precision": 0.652231, + "recall": 0.417262 + }, + { + "t": 0.56, + "precision": 0.64251, + "recall": 0.429157 + }, + { + "t": 0.54, + "precision": 0.632915, + "recall": 0.440985 + }, + { + "t": 0.52, + "precision": 0.623577, + "recall": 0.45362 + }, + { + "t": 0.5, + "precision": 0.61375, + "recall": 0.466322 + }, + { + "t": 0.48, + "precision": 0.604075, + "recall": 0.479696 + }, + { + "t": 0.46, + "precision": 0.593219, + "recall": 0.492683 + }, + { + "t": 0.44, + "precision": 0.581058, + "recall": 0.504763 + }, + { + "t": 0.42, + "precision": 0.570364, + "recall": 0.518271 + }, + { + "t": 0.4, + "precision": 0.560299, + "recall": 0.531427 + }, + { + "t": 0.38, + "precision": 0.548314, + "recall": 0.544095 + }, + { + "t": 0.36, + "precision": 0.536368, + "recall": 0.557654 + }, + { + "t": 0.34, + "precision": 0.525257, + "recall": 0.572153 + }, + { + "t": 0.32, + "precision": 0.513583, + "recall": 0.586351 + }, + { + "t": 0.3, + "precision": 0.50199, + "recall": 0.601892 + }, + { + "t": 0.28, + "precision": 0.490008, + "recall": 0.617131 + }, + { + "t": 0.26, + "precision": 0.477762, + "recall": 0.632773 + }, + { + "t": 0.24, + "precision": 0.465209, + "recall": 0.649944 + }, + { + "t": 0.22, + "precision": 0.451807, + "recall": 0.666426 + }, + { + "t": 0.2, + "precision": 0.438308, + "recall": 0.684823 + }, + { + "t": 0.18, + "precision": 0.424135, + "recall": 0.702549 + }, + { + "t": 0.16, + "precision": 0.40955, + "recall": 0.722072 + }, + { + "t": 0.14, + "precision": 0.393667, + "recall": 0.743191 + }, + { + "t": 0.12, + "precision": 0.376616, + "recall": 0.764596 + }, + { + "t": 0.1, + "precision": 0.358264, + "recall": 0.7879 + }, + { + "t": 0.08, + "precision": 0.338383, + "recall": 0.815168 + }, + { + "t": 0.06, + "precision": 0.315728, + "recall": 0.844184 + }, + { + "t": 0.04, + "precision": 0.288123, + "recall": 0.879181 + }, + { + "t": 0.02, + "precision": 0.252037, + "recall": 0.922294 + }, + { + "t": 0.0, + "precision": 0.155759, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w025-noise-baseline.json b/docs/examples/bench/w025-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..1b51d24afa354f4fe6460845f1a5d2ba97621e8e --- /dev/null +++ b/docs/examples/bench/w025-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:48:11Z", + "scroll": "PHerc0139", + "segment": "w025", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w025_ink.png", + "sha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8", + "width": 598, + "height": 640, + "pxPerMM": 9.3937, + "alignTargetFile": "data/bench/w025_ink.png", + "alignTargetSha256": "298aee5988b1120978977f573824bbb1d550fd877a8b233dabea7724378758a8" + }, + "prediction": { + "name": "noise-seed25.png", + "sha256": "cca11cae5b62ce76bbf1dc053614ab189878680be8021c962628d229ffceb639", + "format": "png", + "width": 598, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(25).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.000355, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 382720, + "validFraction": 1.0, + "pearson": 0.000355, + "blurSigmaPx": 6.0, + "blurSigmaUM": 638.7, + "pearsonMatchedBlur": 0.000806, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.134334, + "dice": 0.236851, + "precision": 0.155441, + "recall": 0.497303, + "refPositiveFraction": 0.155516, + "predPositiveFraction": 0.497541, + "ap": 0.156205, + "bestF1": { + "f1": 0.269171, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.167365, + "recall": 0.021489 + }, + { + "t": 0.96, + "precision": 0.161275, + "recall": 0.040794 + }, + { + "t": 0.94, + "precision": 0.159883, + "recall": 0.060451 + }, + { + "t": 0.92, + "precision": 0.159164, + "recall": 0.080378 + }, + { + "t": 0.9, + "precision": 0.159305, + "recall": 0.10022 + }, + { + "t": 0.88, + "precision": 0.158096, + "recall": 0.119323 + }, + { + "t": 0.86, + "precision": 0.157868, + "recall": 0.138964 + }, + { + "t": 0.84, + "precision": 0.157373, + "recall": 0.15847 + }, + { + "t": 0.82, + "precision": 0.156813, + "recall": 0.177893 + }, + { + "t": 0.8, + "precision": 0.155958, + "recall": 0.200272 + }, + { + "t": 0.78, + "precision": 0.155983, + "recall": 0.219829 + }, + { + "t": 0.76, + "precision": 0.155989, + "recall": 0.239436 + }, + { + "t": 0.74, + "precision": 0.155477, + "recall": 0.258186 + }, + { + "t": 0.72, + "precision": 0.155227, + "recall": 0.277407 + }, + { + "t": 0.7, + "precision": 0.15537, + "recall": 0.297233 + }, + { + "t": 0.68, + "precision": 0.155284, + "recall": 0.316605 + }, + { + "t": 0.66, + "precision": 0.154942, + "recall": 0.335607 + }, + { + "t": 0.64, + "precision": 0.155212, + "recall": 0.355769 + }, + { + "t": 0.62, + "precision": 0.155133, + "recall": 0.374603 + }, + { + "t": 0.6, + "precision": 0.155409, + "recall": 0.399099 + }, + { + "t": 0.58, + "precision": 0.155281, + "recall": 0.418505 + }, + { + "t": 0.56, + "precision": 0.155247, + "recall": 0.438028 + }, + { + "t": 0.54, + "precision": 0.155428, + "recall": 0.458055 + }, + { + "t": 0.52, + "precision": 0.155539, + "recall": 0.478167 + }, + { + "t": 0.5, + "precision": 0.155441, + "recall": 0.497303 + }, + { + "t": 0.48, + "precision": 0.15554, + "recall": 0.517146 + }, + { + "t": 0.46, + "precision": 0.155535, + "recall": 0.536887 + }, + { + "t": 0.44, + "precision": 0.155541, + "recall": 0.556579 + }, + { + "t": 0.42, + "precision": 0.155464, + "recall": 0.575883 + }, + { + "t": 0.4, + "precision": 0.155432, + "recall": 0.59912 + }, + { + "t": 0.38, + "precision": 0.155575, + "recall": 0.6195 + }, + { + "t": 0.36, + "precision": 0.155318, + "recall": 0.638065 + }, + { + "t": 0.34, + "precision": 0.155339, + "recall": 0.657286 + }, + { + "t": 0.32, + "precision": 0.155334, + "recall": 0.676742 + }, + { + "t": 0.3, + "precision": 0.155558, + "recall": 0.697475 + }, + { + "t": 0.28, + "precision": 0.155358, + "recall": 0.716275 + }, + { + "t": 0.26, + "precision": 0.155151, + "recall": 0.734908 + }, + { + "t": 0.24, + "precision": 0.155166, + "recall": 0.754633 + }, + { + "t": 0.22, + "precision": 0.155302, + "recall": 0.77466 + }, + { + "t": 0.2, + "precision": 0.155375, + "recall": 0.798501 + }, + { + "t": 0.18, + "precision": 0.155477, + "recall": 0.818982 + }, + { + "t": 0.16, + "precision": 0.155276, + "recall": 0.837296 + }, + { + "t": 0.14, + "precision": 0.155432, + "recall": 0.857693 + }, + { + "t": 0.12, + "precision": 0.15544, + "recall": 0.877468 + }, + { + "t": 0.1, + "precision": 0.15553, + "recall": 0.897327 + }, + { + "t": 0.08, + "precision": 0.155506, + "recall": 0.917321 + }, + { + "t": 0.06, + "precision": 0.155471, + "recall": 0.936961 + }, + { + "t": 0.04, + "precision": 0.155464, + "recall": 0.956232 + }, + { + "t": 0.02, + "precision": 0.155598, + "recall": 0.976512 + }, + { + "t": 0.0, + "precision": 0.155516, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w026-ct-baseline.json b/docs/examples/bench/w026-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..0e3ce5fe7499ed3bfa53f79dc4a9e10f7551d63f --- /dev/null +++ b/docs/examples/bench/w026-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:48:29Z", + "scroll": "PHerc0139", + "segment": "w026", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w026_ink.png", + "sha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791", + "width": 629, + "height": 640, + "pxPerMM": 9.1785, + "alignTargetFile": "data/bench/w026_ink.png", + "alignTargetSha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791" + }, + "prediction": { + "name": "w026_ct.png", + "sha256": "d005b08b88e31ad0cc24bfbbff91041e8abad6275d39cbb243dddfe78b46f3f0", + "format": "png", + "width": 629, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.278906, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 402560, + "validFraction": 1.0, + "pearson": 0.278906, + "blurSigmaPx": 6.0, + "blurSigmaUM": 653.7, + "pearsonMatchedBlur": 0.376576, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.064599, + "dice": 0.121359, + "precision": 0.068329, + "recall": 0.542037, + "refPositiveFraction": 0.048634, + "predPositiveFraction": 0.385801, + "ap": 0.072337, + "bestF1": { + "f1": 0.122104, + "threshold": 0.6 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.225806, + "recall": 0.002503 + }, + { + "t": 0.98, + "precision": 0.170956, + "recall": 0.00475 + }, + { + "t": 0.96, + "precision": 0.152941, + "recall": 0.007968 + }, + { + "t": 0.94, + "precision": 0.142022, + "recall": 0.012412 + }, + { + "t": 0.92, + "precision": 0.133232, + "recall": 0.017979 + }, + { + "t": 0.9, + "precision": 0.125635, + "recall": 0.025283 + }, + { + "t": 0.88, + "precision": 0.112322, + "recall": 0.032639 + }, + { + "t": 0.86, + "precision": 0.106697, + "recall": 0.04362 + }, + { + "t": 0.84, + "precision": 0.101162, + "recall": 0.056901 + }, + { + "t": 0.82, + "precision": 0.095673, + "recall": 0.072173 + }, + { + "t": 0.8, + "precision": 0.090627, + "recall": 0.094034 + }, + { + "t": 0.78, + "precision": 0.086661, + "recall": 0.113393 + }, + { + "t": 0.76, + "precision": 0.084308, + "recall": 0.13699 + }, + { + "t": 0.74, + "precision": 0.082699, + "recall": 0.163142 + }, + { + "t": 0.72, + "precision": 0.080023, + "recall": 0.188477 + }, + { + "t": 0.7, + "precision": 0.079188, + "recall": 0.218868 + }, + { + "t": 0.68, + "precision": 0.07806, + "recall": 0.251149 + }, + { + "t": 0.66, + "precision": 0.076513, + "recall": 0.281285 + }, + { + "t": 0.64, + "precision": 0.075215, + "recall": 0.313004 + }, + { + "t": 0.62, + "precision": 0.073916, + "recall": 0.343855 + }, + { + "t": 0.6, + "precision": 0.07264, + "recall": 0.382726 + }, + { + "t": 0.58, + "precision": 0.071338, + "recall": 0.413525 + }, + { + "t": 0.56, + "precision": 0.070653, + "recall": 0.44693 + }, + { + "t": 0.54, + "precision": 0.069849, + "recall": 0.479416 + }, + { + "t": 0.52, + "precision": 0.069073, + "recall": 0.510982 + }, + { + "t": 0.5, + "precision": 0.068329, + "recall": 0.542037 + }, + { + "t": 0.48, + "precision": 0.067272, + "recall": 0.569517 + }, + { + "t": 0.46, + "precision": 0.066605, + "recall": 0.598325 + }, + { + "t": 0.44, + "precision": 0.065942, + "recall": 0.625396 + }, + { + "t": 0.42, + "precision": 0.065164, + "recall": 0.650577 + }, + { + "t": 0.4, + "precision": 0.064333, + "recall": 0.679436 + }, + { + "t": 0.38, + "precision": 0.063741, + "recall": 0.703136 + }, + { + "t": 0.36, + "precision": 0.063055, + "recall": 0.723669 + }, + { + "t": 0.34, + "precision": 0.062479, + "recall": 0.744305 + }, + { + "t": 0.32, + "precision": 0.062059, + "recall": 0.765451 + }, + { + "t": 0.3, + "precision": 0.061548, + "recall": 0.783686 + }, + { + "t": 0.28, + "precision": 0.06097, + "recall": 0.800184 + }, + { + "t": 0.26, + "precision": 0.060372, + "recall": 0.814588 + }, + { + "t": 0.24, + "precision": 0.059949, + "recall": 0.830064 + }, + { + "t": 0.22, + "precision": 0.059607, + "recall": 0.845541 + }, + { + "t": 0.2, + "precision": 0.059204, + "recall": 0.862192 + }, + { + "t": 0.18, + "precision": 0.058856, + "recall": 0.874962 + }, + { + "t": 0.16, + "precision": 0.058651, + "recall": 0.888855 + }, + { + "t": 0.14, + "precision": 0.058371, + "recall": 0.900756 + }, + { + "t": 0.12, + "precision": 0.058127, + "recall": 0.913474 + }, + { + "t": 0.1, + "precision": 0.05797, + "recall": 0.927725 + }, + { + "t": 0.08, + "precision": 0.057776, + "recall": 0.941618 + }, + { + "t": 0.06, + "precision": 0.057623, + "recall": 0.955562 + }, + { + "t": 0.04, + "precision": 0.057498, + "recall": 0.969609 + }, + { + "t": 0.02, + "precision": 0.057247, + "recall": 0.981714 + }, + { + "t": 0.0, + "precision": 0.048634, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w026-ink9um.json b/docs/examples/bench/w026-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..476f9c93322f99bb0de1e721b55bfc7d23919278 --- /dev/null +++ b/docs/examples/bench/w026-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:48:28Z", + "scroll": "PHerc0139", + "segment": "w026", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w026_ink.png", + "sha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791", + "width": 629, + "height": 640, + "pxPerMM": 9.1785, + "alignTargetFile": "data/bench/w026_ink.png", + "alignTargetSha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791" + }, + "prediction": { + "name": "w026_gpu.png", + "sha256": "262b1fb5329f95649d5d69da657ac3b111b9ba05c2151859c167efd85c700079", + "format": "png", + "width": 629, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.531, + "ty": 0.969, + "sx": 0.99987, + "sy": 1.00178, + "ncc": 0.29816, + "coarseNcc": 0.426747, + "evals": 42527, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 401292, + "validFraction": 0.99685, + "pearson": 0.29816, + "blurSigmaPx": 2.0, + "blurSigmaUM": 217.9, + "pearsonMatchedBlur": 0.324342, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.117089, + "dice": 0.209633, + "precision": 0.207984, + "recall": 0.211309, + "refPositiveFraction": 0.048787, + "predPositiveFraction": 0.049567, + "ap": 0.151895, + "bestF1": { + "f1": 0.22795, + "threshold": 0.32 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.5, + "recall": 0.000306 + }, + { + "t": 0.98, + "precision": 0.409463, + "recall": 0.022985 + }, + { + "t": 0.96, + "precision": 0.379817, + "recall": 0.031719 + }, + { + "t": 0.94, + "precision": 0.374391, + "recall": 0.039279 + }, + { + "t": 0.92, + "precision": 0.361761, + "recall": 0.046583 + }, + { + "t": 0.9, + "precision": 0.346892, + "recall": 0.053581 + }, + { + "t": 0.88, + "precision": 0.337329, + "recall": 0.060374 + }, + { + "t": 0.86, + "precision": 0.32753, + "recall": 0.067269 + }, + { + "t": 0.84, + "precision": 0.31892, + "recall": 0.074216 + }, + { + "t": 0.82, + "precision": 0.311369, + "recall": 0.080856 + }, + { + "t": 0.8, + "precision": 0.303041, + "recall": 0.087547 + }, + { + "t": 0.78, + "precision": 0.297678, + "recall": 0.094954 + }, + { + "t": 0.76, + "precision": 0.290994, + "recall": 0.102155 + }, + { + "t": 0.74, + "precision": 0.280074, + "recall": 0.107978 + }, + { + "t": 0.72, + "precision": 0.272837, + "recall": 0.11564 + }, + { + "t": 0.7, + "precision": 0.267072, + "recall": 0.123251 + }, + { + "t": 0.68, + "precision": 0.258982, + "recall": 0.130708 + }, + { + "t": 0.66, + "precision": 0.252205, + "recall": 0.138778 + }, + { + "t": 0.64, + "precision": 0.242978, + "recall": 0.145367 + }, + { + "t": 0.62, + "precision": 0.235898, + "recall": 0.153795 + }, + { + "t": 0.6, + "precision": 0.23039, + "recall": 0.162172 + }, + { + "t": 0.58, + "precision": 0.224914, + "recall": 0.170702 + }, + { + "t": 0.56, + "precision": 0.221309, + "recall": 0.180253 + }, + { + "t": 0.54, + "precision": 0.216931, + "recall": 0.190571 + }, + { + "t": 0.52, + "precision": 0.212882, + "recall": 0.200889 + }, + { + "t": 0.5, + "precision": 0.207984, + "recall": 0.211309 + }, + { + "t": 0.48, + "precision": 0.203289, + "recall": 0.221626 + }, + { + "t": 0.46, + "precision": 0.19894, + "recall": 0.233987 + }, + { + "t": 0.44, + "precision": 0.194885, + "recall": 0.247931 + }, + { + "t": 0.42, + "precision": 0.190327, + "recall": 0.260701 + }, + { + "t": 0.4, + "precision": 0.185865, + "recall": 0.274287 + }, + { + "t": 0.38, + "precision": 0.182261, + "recall": 0.288845 + }, + { + "t": 0.36, + "precision": 0.17886, + "recall": 0.305445 + }, + { + "t": 0.34, + "precision": 0.175594, + "recall": 0.323169 + }, + { + "t": 0.32, + "precision": 0.171351, + "recall": 0.340382 + }, + { + "t": 0.3, + "precision": 0.166512, + "recall": 0.358157 + }, + { + "t": 0.28, + "precision": 0.162272, + "recall": 0.377873 + }, + { + "t": 0.26, + "precision": 0.157717, + "recall": 0.398764 + }, + { + "t": 0.24, + "precision": 0.153892, + "recall": 0.422617 + }, + { + "t": 0.22, + "precision": 0.147826, + "recall": 0.444376 + }, + { + "t": 0.2, + "precision": 0.142145, + "recall": 0.467617 + }, + { + "t": 0.18, + "precision": 0.136718, + "recall": 0.493871 + }, + { + "t": 0.16, + "precision": 0.130124, + "recall": 0.517877 + }, + { + "t": 0.14, + "precision": 0.123865, + "recall": 0.546276 + }, + { + "t": 0.12, + "precision": 0.117527, + "recall": 0.580243 + }, + { + "t": 0.1, + "precision": 0.111137, + "recall": 0.619318 + }, + { + "t": 0.08, + "precision": 0.104007, + "recall": 0.663449 + }, + { + "t": 0.06, + "precision": 0.095877, + "recall": 0.713709 + }, + { + "t": 0.04, + "precision": 0.086453, + "recall": 0.776995 + }, + { + "t": 0.02, + "precision": 0.074829, + "recall": 0.866074 + }, + { + "t": 0.0, + "precision": 0.048787, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w026-noise-baseline.json b/docs/examples/bench/w026-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..cca9427420b7dc8a03e712cf4b52beb3c5d2035a --- /dev/null +++ b/docs/examples/bench/w026-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:48:30Z", + "scroll": "PHerc0139", + "segment": "w026", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w026_ink.png", + "sha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791", + "width": 629, + "height": 640, + "pxPerMM": 9.1785, + "alignTargetFile": "data/bench/w026_ink.png", + "alignTargetSha256": "436cb4c1d102dbf7bfca42b3700f1cf268c29e58a4d5a6f90d130feb9da8e791" + }, + "prediction": { + "name": "noise-seed26.png", + "sha256": "220a8b63d482620f4a98536fc97a971e7f69bebaafff14199d47b186ae43ffc4", + "format": "png", + "width": 629, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(26).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": -0.002983, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 402560, + "validFraction": 1.0, + "pearson": -0.002983, + "blurSigmaPx": 6.0, + "blurSigmaUM": 653.7, + "pearsonMatchedBlur": -0.002789, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.045741, + "dice": 0.087481, + "precision": 0.04801, + "recall": 0.491879, + "refPositiveFraction": 0.048634, + "predPositiveFraction": 0.498271, + "ap": 0.047854, + "bestF1": { + "f1": 0.092756, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.045315, + "recall": 0.018133 + }, + { + "t": 0.96, + "precision": 0.046628, + "recall": 0.03744 + }, + { + "t": 0.94, + "precision": 0.046986, + "recall": 0.056288 + }, + { + "t": 0.92, + "precision": 0.047393, + "recall": 0.07585 + }, + { + "t": 0.9, + "precision": 0.047743, + "recall": 0.095566 + }, + { + "t": 0.88, + "precision": 0.048053, + "recall": 0.115691 + }, + { + "t": 0.86, + "precision": 0.048044, + "recall": 0.134692 + }, + { + "t": 0.84, + "precision": 0.047887, + "recall": 0.153386 + }, + { + "t": 0.82, + "precision": 0.047932, + "recall": 0.172898 + }, + { + "t": 0.8, + "precision": 0.047496, + "recall": 0.194555 + }, + { + "t": 0.78, + "precision": 0.047122, + "recall": 0.21187 + }, + { + "t": 0.76, + "precision": 0.046874, + "recall": 0.229697 + }, + { + "t": 0.74, + "precision": 0.047309, + "recall": 0.250536 + }, + { + "t": 0.72, + "precision": 0.047345, + "recall": 0.270201 + }, + { + "t": 0.7, + "precision": 0.047221, + "recall": 0.288691 + }, + { + "t": 0.68, + "precision": 0.047153, + "recall": 0.307437 + }, + { + "t": 0.66, + "precision": 0.047378, + "recall": 0.328174 + }, + { + "t": 0.64, + "precision": 0.047424, + "recall": 0.34789 + }, + { + "t": 0.62, + "precision": 0.047463, + "recall": 0.368117 + }, + { + "t": 0.6, + "precision": 0.047711, + "recall": 0.393094 + }, + { + "t": 0.58, + "precision": 0.047698, + "recall": 0.411942 + }, + { + "t": 0.56, + "precision": 0.047593, + "recall": 0.430381 + }, + { + "t": 0.54, + "precision": 0.047695, + "recall": 0.450046 + }, + { + "t": 0.52, + "precision": 0.047728, + "recall": 0.469711 + }, + { + "t": 0.5, + "precision": 0.04801, + "recall": 0.491879 + }, + { + "t": 0.48, + "precision": 0.048052, + "recall": 0.511595 + }, + { + "t": 0.46, + "precision": 0.048076, + "recall": 0.53126 + }, + { + "t": 0.44, + "precision": 0.048, + "recall": 0.54975 + }, + { + "t": 0.42, + "precision": 0.048023, + "recall": 0.569159 + }, + { + "t": 0.4, + "precision": 0.048048, + "recall": 0.592859 + }, + { + "t": 0.38, + "precision": 0.047966, + "recall": 0.611196 + }, + { + "t": 0.36, + "precision": 0.048075, + "recall": 0.63224 + }, + { + "t": 0.34, + "precision": 0.048121, + "recall": 0.652518 + }, + { + "t": 0.32, + "precision": 0.048089, + "recall": 0.671366 + }, + { + "t": 0.3, + "precision": 0.048168, + "recall": 0.692001 + }, + { + "t": 0.28, + "precision": 0.048165, + "recall": 0.711104 + }, + { + "t": 0.26, + "precision": 0.048116, + "recall": 0.729594 + }, + { + "t": 0.24, + "precision": 0.048122, + "recall": 0.748646 + }, + { + "t": 0.22, + "precision": 0.048233, + "recall": 0.769588 + }, + { + "t": 0.2, + "precision": 0.0483, + "recall": 0.794055 + }, + { + "t": 0.18, + "precision": 0.048363, + "recall": 0.814588 + }, + { + "t": 0.16, + "precision": 0.048424, + "recall": 0.835325 + }, + { + "t": 0.14, + "precision": 0.048475, + "recall": 0.85591 + }, + { + "t": 0.12, + "precision": 0.048506, + "recall": 0.875677 + }, + { + "t": 0.1, + "precision": 0.048505, + "recall": 0.895342 + }, + { + "t": 0.08, + "precision": 0.048483, + "recall": 0.914547 + }, + { + "t": 0.06, + "precision": 0.048563, + "recall": 0.935489 + }, + { + "t": 0.04, + "precision": 0.048558, + "recall": 0.955052 + }, + { + "t": 0.02, + "precision": 0.048564, + "recall": 0.975023 + }, + { + "t": 0.0, + "precision": 0.048634, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w027-ct-baseline.json b/docs/examples/bench/w027-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..360f62b2233d705bb1f5b5ab294f29583238c79b --- /dev/null +++ b/docs/examples/bench/w027-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:50:58Z", + "scroll": "PHerc0139", + "segment": "w027", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w027_ink.png", + "sha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2", + "width": 638, + "height": 640, + "pxPerMM": 9.3357, + "alignTargetFile": "data/bench/w027_ink.png", + "alignTargetSha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2" + }, + "prediction": { + "name": "w027_ct.png", + "sha256": "bf5b5d9a5d8574c03dba35ac6b502319aa6ebee01032cff01cc5f7d600ea22b6", + "format": "png", + "width": 638, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.299446, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 408320, + "validFraction": 1.0, + "pearson": 0.299446, + "blurSigmaPx": 6.0, + "blurSigmaUM": 642.7, + "pearsonMatchedBlur": 0.355799, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.121584, + "dice": 0.216807, + "precision": 0.134544, + "recall": 0.557949, + "refPositiveFraction": 0.093674, + "predPositiveFraction": 0.388462, + "ap": 0.145828, + "bestF1": { + "f1": 0.218206, + "threshold": 0.44 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.355932, + "recall": 0.002196 + }, + { + "t": 0.98, + "precision": 0.365217, + "recall": 0.00549 + }, + { + "t": 0.96, + "precision": 0.359572, + "recall": 0.009673 + }, + { + "t": 0.94, + "precision": 0.336556, + "recall": 0.014562 + }, + { + "t": 0.92, + "precision": 0.304087, + "recall": 0.019844 + }, + { + "t": 0.9, + "precision": 0.27519, + "recall": 0.026563 + }, + { + "t": 0.88, + "precision": 0.262881, + "recall": 0.03655 + }, + { + "t": 0.86, + "precision": 0.237259, + "recall": 0.04625 + }, + { + "t": 0.84, + "precision": 0.218432, + "recall": 0.05893 + }, + { + "t": 0.82, + "precision": 0.202996, + "recall": 0.074041 + }, + { + "t": 0.8, + "precision": 0.188728, + "recall": 0.095166 + }, + { + "t": 0.78, + "precision": 0.178574, + "recall": 0.11501 + }, + { + "t": 0.76, + "precision": 0.168964, + "recall": 0.136213 + }, + { + "t": 0.74, + "precision": 0.162469, + "recall": 0.161076 + }, + { + "t": 0.72, + "precision": 0.157527, + "recall": 0.188136 + }, + { + "t": 0.7, + "precision": 0.153249, + "recall": 0.217339 + }, + { + "t": 0.68, + "precision": 0.150783, + "recall": 0.249837 + }, + { + "t": 0.66, + "precision": 0.148056, + "recall": 0.282596 + }, + { + "t": 0.64, + "precision": 0.146126, + "recall": 0.316531 + }, + { + "t": 0.62, + "precision": 0.144255, + "recall": 0.350833 + }, + { + "t": 0.6, + "precision": 0.142247, + "recall": 0.392533 + }, + { + "t": 0.58, + "precision": 0.140191, + "recall": 0.42605 + }, + { + "t": 0.56, + "precision": 0.138673, + "recall": 0.45962 + }, + { + "t": 0.54, + "precision": 0.137246, + "recall": 0.493111 + }, + { + "t": 0.52, + "precision": 0.135788, + "recall": 0.525661 + }, + { + "t": 0.5, + "precision": 0.134544, + "recall": 0.557949 + }, + { + "t": 0.48, + "precision": 0.133308, + "recall": 0.589113 + }, + { + "t": 0.46, + "precision": 0.132051, + "recall": 0.618578 + }, + { + "t": 0.44, + "precision": 0.13119, + "recall": 0.648043 + }, + { + "t": 0.42, + "precision": 0.129787, + "recall": 0.674161 + }, + { + "t": 0.4, + "precision": 0.128471, + "recall": 0.705666 + }, + { + "t": 0.38, + "precision": 0.127202, + "recall": 0.728673 + }, + { + "t": 0.36, + "precision": 0.126083, + "recall": 0.751627 + }, + { + "t": 0.34, + "precision": 0.125017, + "recall": 0.772569 + }, + { + "t": 0.32, + "precision": 0.123818, + "recall": 0.791576 + }, + { + "t": 0.3, + "precision": 0.122866, + "recall": 0.810557 + }, + { + "t": 0.28, + "precision": 0.121995, + "recall": 0.827891 + }, + { + "t": 0.26, + "precision": 0.12106, + "recall": 0.843368 + }, + { + "t": 0.24, + "precision": 0.120205, + "recall": 0.857722 + }, + { + "t": 0.22, + "precision": 0.119499, + "recall": 0.87231 + }, + { + "t": 0.2, + "precision": 0.118561, + "recall": 0.886977 + }, + { + "t": 0.18, + "precision": 0.117871, + "recall": 0.899004 + }, + { + "t": 0.16, + "precision": 0.117224, + "recall": 0.910246 + }, + { + "t": 0.14, + "precision": 0.116586, + "recall": 0.921253 + }, + { + "t": 0.12, + "precision": 0.115769, + "recall": 0.930377 + }, + { + "t": 0.1, + "precision": 0.115116, + "recall": 0.941018 + }, + { + "t": 0.08, + "precision": 0.114402, + "recall": 0.951319 + }, + { + "t": 0.06, + "precision": 0.113659, + "recall": 0.961149 + }, + { + "t": 0.04, + "precision": 0.112997, + "recall": 0.971084 + }, + { + "t": 0.02, + "precision": 0.112282, + "recall": 0.981019 + }, + { + "t": 0.0, + "precision": 0.093674, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w027-ink9um.json b/docs/examples/bench/w027-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..c3abdb981c54a95064246a81bc5d33ee9fc05da7 --- /dev/null +++ b/docs/examples/bench/w027-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:50:57Z", + "scroll": "PHerc0139", + "segment": "w027", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w027_ink.png", + "sha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2", + "width": 638, + "height": 640, + "pxPerMM": 9.3357, + "alignTargetFile": "data/bench/w027_ink.png", + "alignTargetSha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2" + }, + "prediction": { + "name": "w027_gpu.png", + "sha256": "89395044a5bf932b7adaeb8522e2d36c87e684f852b6c8ba2e0d30509376e650", + "format": "png", + "width": 638, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 1.0, + "ty": 0.75, + "sx": 1.00556, + "sy": 1.00236, + "ncc": 0.49414, + "coarseNcc": 0.702908, + "evals": 42312, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 408320, + "validFraction": 1.0, + "pearson": 0.49414, + "blurSigmaPx": 3.0, + "blurSigmaUM": 321.3, + "pearsonMatchedBlur": 0.535652, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.266987, + "dice": 0.421452, + "precision": 0.442537, + "recall": 0.402285, + "refPositiveFraction": 0.093674, + "predPositiveFraction": 0.085154, + "ap": 0.383928, + "bestF1": { + "f1": 0.433898, + "threshold": 0.38 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.839416, + "recall": 0.003007 + }, + { + "t": 0.98, + "precision": 0.671879, + "recall": 0.066276 + }, + { + "t": 0.96, + "precision": 0.653496, + "recall": 0.088211 + }, + { + "t": 0.94, + "precision": 0.640202, + "recall": 0.105833 + }, + { + "t": 0.92, + "precision": 0.626016, + "recall": 0.120787 + }, + { + "t": 0.9, + "precision": 0.614154, + "recall": 0.134539 + }, + { + "t": 0.88, + "precision": 0.602493, + "recall": 0.147847 + }, + { + "t": 0.86, + "precision": 0.590695, + "recall": 0.160318 + }, + { + "t": 0.84, + "precision": 0.582909, + "recall": 0.173704 + }, + { + "t": 0.82, + "precision": 0.573791, + "recall": 0.187743 + }, + { + "t": 0.8, + "precision": 0.567528, + "recall": 0.201051 + }, + { + "t": 0.78, + "precision": 0.559718, + "recall": 0.214045 + }, + { + "t": 0.76, + "precision": 0.551387, + "recall": 0.227091 + }, + { + "t": 0.74, + "precision": 0.54232, + "recall": 0.239719 + }, + { + "t": 0.72, + "precision": 0.533201, + "recall": 0.252765 + }, + { + "t": 0.7, + "precision": 0.525344, + "recall": 0.265549 + }, + { + "t": 0.68, + "precision": 0.518348, + "recall": 0.279929 + }, + { + "t": 0.66, + "precision": 0.51035, + "recall": 0.293289 + }, + { + "t": 0.64, + "precision": 0.502141, + "recall": 0.306544 + }, + { + "t": 0.62, + "precision": 0.492882, + "recall": 0.319538 + }, + { + "t": 0.6, + "precision": 0.484261, + "recall": 0.333028 + }, + { + "t": 0.58, + "precision": 0.476978, + "recall": 0.347486 + }, + { + "t": 0.56, + "precision": 0.468635, + "recall": 0.361526 + }, + { + "t": 0.54, + "precision": 0.460308, + "recall": 0.375513 + }, + { + "t": 0.52, + "precision": 0.451121, + "recall": 0.388245 + }, + { + "t": 0.5, + "precision": 0.442537, + "recall": 0.402285 + }, + { + "t": 0.48, + "precision": 0.433741, + "recall": 0.416481 + }, + { + "t": 0.46, + "precision": 0.424769, + "recall": 0.43141 + }, + { + "t": 0.44, + "precision": 0.416711, + "recall": 0.447489 + }, + { + "t": 0.42, + "precision": 0.40637, + "recall": 0.462679 + }, + { + "t": 0.4, + "precision": 0.396898, + "recall": 0.477712 + }, + { + "t": 0.38, + "precision": 0.386946, + "recall": 0.493817 + }, + { + "t": 0.36, + "precision": 0.377051, + "recall": 0.509373 + }, + { + "t": 0.34, + "precision": 0.367086, + "recall": 0.526602 + }, + { + "t": 0.32, + "precision": 0.357072, + "recall": 0.544406 + }, + { + "t": 0.3, + "precision": 0.346406, + "recall": 0.561636 + }, + { + "t": 0.28, + "precision": 0.336663, + "recall": 0.581584 + }, + { + "t": 0.26, + "precision": 0.325605, + "recall": 0.599963 + }, + { + "t": 0.24, + "precision": 0.314132, + "recall": 0.620199 + }, + { + "t": 0.22, + "precision": 0.30291, + "recall": 0.641219 + }, + { + "t": 0.2, + "precision": 0.291763, + "recall": 0.663024 + }, + { + "t": 0.18, + "precision": 0.279066, + "recall": 0.684201 + }, + { + "t": 0.16, + "precision": 0.266522, + "recall": 0.706424 + }, + { + "t": 0.14, + "precision": 0.253303, + "recall": 0.729875 + }, + { + "t": 0.12, + "precision": 0.239785, + "recall": 0.754085 + }, + { + "t": 0.1, + "precision": 0.225313, + "recall": 0.77921 + }, + { + "t": 0.08, + "precision": 0.210039, + "recall": 0.807341 + }, + { + "t": 0.06, + "precision": 0.193162, + "recall": 0.84044 + }, + { + "t": 0.04, + "precision": 0.173651, + "recall": 0.879422 + }, + { + "t": 0.02, + "precision": 0.148893, + "recall": 0.924259 + }, + { + "t": 0.0, + "precision": 0.093674, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w027-noise-baseline.json b/docs/examples/bench/w027-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..e2388e8c0dcac2399edff0ac76cb2c567c6f5991 --- /dev/null +++ b/docs/examples/bench/w027-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:00Z", + "scroll": "PHerc0139", + "segment": "w027", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w027_ink.png", + "sha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2", + "width": 638, + "height": 640, + "pxPerMM": 9.3357, + "alignTargetFile": "data/bench/w027_ink.png", + "alignTargetSha256": "0eda2b76e7922df9d8474b1cbfa09e83ed5d303d474cd0c0dd790fa7b0b820c2" + }, + "prediction": { + "name": "noise-seed27.png", + "sha256": "3f35963811cc3ba350851a574a55123ebed8a5c217434dc46b650e037f3f1f17", + "format": "png", + "width": 638, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(27).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": -0.002544, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 408320, + "validFraction": 1.0, + "pearson": -0.002544, + "blurSigmaPx": 6.0, + "blurSigmaUM": 642.7, + "pearsonMatchedBlur": -0.002194, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.08474, + "dice": 0.15624, + "precision": 0.092732, + "recall": 0.495778, + "refPositiveFraction": 0.093674, + "predPositiveFraction": 0.500816, + "ap": 0.093275, + "bestF1": { + "f1": 0.171302, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.094288, + "recall": 0.019896 + }, + { + "t": 0.96, + "precision": 0.094086, + "recall": 0.039557 + }, + { + "t": 0.94, + "precision": 0.093964, + "recall": 0.059139 + }, + { + "t": 0.92, + "precision": 0.092801, + "recall": 0.078329 + }, + { + "t": 0.9, + "precision": 0.09358, + "recall": 0.098695 + }, + { + "t": 0.88, + "precision": 0.093374, + "recall": 0.118199 + }, + { + "t": 0.86, + "precision": 0.093263, + "recall": 0.137651 + }, + { + "t": 0.84, + "precision": 0.09294, + "recall": 0.15671 + }, + { + "t": 0.82, + "precision": 0.093395, + "recall": 0.177181 + }, + { + "t": 0.8, + "precision": 0.093183, + "recall": 0.200345 + }, + { + "t": 0.78, + "precision": 0.093003, + "recall": 0.21964 + }, + { + "t": 0.76, + "precision": 0.093011, + "recall": 0.239196 + }, + { + "t": 0.74, + "precision": 0.092967, + "recall": 0.2587 + }, + { + "t": 0.72, + "precision": 0.092936, + "recall": 0.278282 + }, + { + "t": 0.7, + "precision": 0.092995, + "recall": 0.298282 + }, + { + "t": 0.68, + "precision": 0.092958, + "recall": 0.317682 + }, + { + "t": 0.66, + "precision": 0.092902, + "recall": 0.337211 + }, + { + "t": 0.64, + "precision": 0.092796, + "recall": 0.356558 + }, + { + "t": 0.62, + "precision": 0.093038, + "recall": 0.377056 + }, + { + "t": 0.6, + "precision": 0.092839, + "recall": 0.399514 + }, + { + "t": 0.58, + "precision": 0.092667, + "recall": 0.418076 + }, + { + "t": 0.56, + "precision": 0.092578, + "recall": 0.436848 + }, + { + "t": 0.54, + "precision": 0.092709, + "recall": 0.45677 + }, + { + "t": 0.52, + "precision": 0.092773, + "recall": 0.476692 + }, + { + "t": 0.5, + "precision": 0.092732, + "recall": 0.495778 + }, + { + "t": 0.48, + "precision": 0.092931, + "recall": 0.515987 + }, + { + "t": 0.46, + "precision": 0.093119, + "recall": 0.536511 + }, + { + "t": 0.44, + "precision": 0.093081, + "recall": 0.556198 + }, + { + "t": 0.42, + "precision": 0.093068, + "recall": 0.575335 + }, + { + "t": 0.4, + "precision": 0.093103, + "recall": 0.599074 + }, + { + "t": 0.38, + "precision": 0.093244, + "recall": 0.619859 + }, + { + "t": 0.36, + "precision": 0.093374, + "recall": 0.639625 + }, + { + "t": 0.34, + "precision": 0.093282, + "recall": 0.658449 + }, + { + "t": 0.32, + "precision": 0.093285, + "recall": 0.677952 + }, + { + "t": 0.3, + "precision": 0.09349, + "recall": 0.698868 + }, + { + "t": 0.28, + "precision": 0.093568, + "recall": 0.718633 + }, + { + "t": 0.26, + "precision": 0.093552, + "recall": 0.737745 + }, + { + "t": 0.24, + "precision": 0.093576, + "recall": 0.757039 + }, + { + "t": 0.22, + "precision": 0.093437, + "recall": 0.775733 + }, + { + "t": 0.2, + "precision": 0.093393, + "recall": 0.798792 + }, + { + "t": 0.18, + "precision": 0.093454, + "recall": 0.818793 + }, + { + "t": 0.16, + "precision": 0.093603, + "recall": 0.839813 + }, + { + "t": 0.14, + "precision": 0.093634, + "recall": 0.859787 + }, + { + "t": 0.12, + "precision": 0.093685, + "recall": 0.879683 + }, + { + "t": 0.1, + "precision": 0.093637, + "recall": 0.898821 + }, + { + "t": 0.08, + "precision": 0.093631, + "recall": 0.918063 + }, + { + "t": 0.06, + "precision": 0.093711, + "recall": 0.938142 + }, + { + "t": 0.04, + "precision": 0.093628, + "recall": 0.956835 + }, + { + "t": 0.02, + "precision": 0.093714, + "recall": 0.97681 + }, + { + "t": 0.0, + "precision": 0.093674, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w028-ct-baseline.json b/docs/examples/bench/w028-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..82e8170dfccfb4c4f66a8e2b78745f57157e6eac --- /dev/null +++ b/docs/examples/bench/w028-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:10Z", + "scroll": "PHerc0139", + "segment": "w028", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w028_ink.png", + "sha256": "fb51ea9e23deda1cb6583280fb7a2d700979cd5f6f61c6e1dc3652339976a380", + "width": 640, + "height": 631, + "pxPerMM": 9.2888, + "alignTargetFile": "data/bench/w028_ink.png", + "alignTargetSha256": "fb51ea9e23deda1cb6583280fb7a2d700979cd5f6f61c6e1dc3652339976a380" + }, + "prediction": { + "name": "w028_ct.png", + "sha256": "02e88630f83b2fea27ec4473e8bf04d43a5c771d8b62e8900fd8f56dc4cc3004", + "format": "png", + "width": 640, + "height": 631, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.29201, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 403840, + "validFraction": 1.0, + "pearson": 0.29201, + "blurSigmaPx": 6.0, + "blurSigmaUM": 645.9, + "pearsonMatchedBlur": 0.418986, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.097583, + "dice": 0.177814, + "precision": 0.105548, + "recall": 0.563921, + "refPositiveFraction": 0.077419, + "predPositiveFraction": 0.413637, + "ap": 0.11115, + "bestF1": { + "f1": 0.177941, + "threshold": 0.48 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.159794, + "recall": 0.000992 + }, + { + "t": 0.98, + "precision": 0.20075, + "recall": 0.003422 + }, + { + "t": 0.96, + "precision": 0.174542, + "recall": 0.005789 + }, + { + "t": 0.94, + "precision": 0.168952, + "recall": 0.009851 + }, + { + "t": 0.92, + "precision": 0.163784, + "recall": 0.015449 + }, + { + "t": 0.9, + "precision": 0.158648, + "recall": 0.022677 + }, + { + "t": 0.88, + "precision": 0.153033, + "recall": 0.031953 + }, + { + "t": 0.86, + "precision": 0.149486, + "recall": 0.044619 + }, + { + "t": 0.84, + "precision": 0.145116, + "recall": 0.059587 + }, + { + "t": 0.82, + "precision": 0.141404, + "recall": 0.076923 + }, + { + "t": 0.8, + "precision": 0.137073, + "recall": 0.101359 + }, + { + "t": 0.78, + "precision": 0.132283, + "recall": 0.123589 + }, + { + "t": 0.76, + "precision": 0.128693, + "recall": 0.148089 + }, + { + "t": 0.74, + "precision": 0.126437, + "recall": 0.175884 + }, + { + "t": 0.72, + "precision": 0.123469, + "recall": 0.204702 + }, + { + "t": 0.7, + "precision": 0.121475, + "recall": 0.235887 + }, + { + "t": 0.68, + "precision": 0.118464, + "recall": 0.265377 + }, + { + "t": 0.66, + "precision": 0.116752, + "recall": 0.298065 + }, + { + "t": 0.64, + "precision": 0.11447, + "recall": 0.329538 + }, + { + "t": 0.62, + "precision": 0.112809, + "recall": 0.36197 + }, + { + "t": 0.6, + "precision": 0.110745, + "recall": 0.400448 + }, + { + "t": 0.58, + "precision": 0.109919, + "recall": 0.435343 + }, + { + "t": 0.56, + "precision": 0.108943, + "recall": 0.468959 + }, + { + "t": 0.54, + "precision": 0.107364, + "recall": 0.498992 + }, + { + "t": 0.52, + "precision": 0.106305, + "recall": 0.530945 + }, + { + "t": 0.5, + "precision": 0.105548, + "recall": 0.563921 + }, + { + "t": 0.48, + "precision": 0.104655, + "recall": 0.593667 + }, + { + "t": 0.46, + "precision": 0.103688, + "recall": 0.622901 + }, + { + "t": 0.44, + "precision": 0.10299, + "recall": 0.650952 + }, + { + "t": 0.42, + "precision": 0.10223, + "recall": 0.677595 + }, + { + "t": 0.4, + "precision": 0.101256, + "recall": 0.706413 + }, + { + "t": 0.38, + "precision": 0.100472, + "recall": 0.729698 + }, + { + "t": 0.36, + "precision": 0.099899, + "recall": 0.753174 + }, + { + "t": 0.34, + "precision": 0.099222, + "recall": 0.774828 + }, + { + "t": 0.32, + "precision": 0.098499, + "recall": 0.793923 + }, + { + "t": 0.3, + "precision": 0.097848, + "recall": 0.811099 + }, + { + "t": 0.28, + "precision": 0.09724, + "recall": 0.826963 + }, + { + "t": 0.26, + "precision": 0.096755, + "recall": 0.842763 + }, + { + "t": 0.24, + "precision": 0.096275, + "recall": 0.857157 + }, + { + "t": 0.22, + "precision": 0.095782, + "recall": 0.870302 + }, + { + "t": 0.2, + "precision": 0.095437, + "recall": 0.88655 + }, + { + "t": 0.18, + "precision": 0.095042, + "recall": 0.898225 + }, + { + "t": 0.16, + "precision": 0.094757, + "recall": 0.910251 + }, + { + "t": 0.14, + "precision": 0.094484, + "recall": 0.921542 + }, + { + "t": 0.12, + "precision": 0.094234, + "recall": 0.93312 + }, + { + "t": 0.1, + "precision": 0.093859, + "recall": 0.943355 + }, + { + "t": 0.08, + "precision": 0.093421, + "recall": 0.952279 + }, + { + "t": 0.06, + "precision": 0.093045, + "recall": 0.962226 + }, + { + "t": 0.04, + "precision": 0.09265, + "recall": 0.971534 + }, + { + "t": 0.02, + "precision": 0.092188, + "recall": 0.980937 + }, + { + "t": 0.0, + "precision": 0.077419, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w028-ink9um.json b/docs/examples/bench/w028-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..5f454c6506cc2aa66863f6bb8c05208655953bff --- /dev/null +++ b/docs/examples/bench/w028-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:10Z", + "scroll": "PHerc0139", + "segment": "w028", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w028_ink.png", + "sha256": "fb51ea9e23deda1cb6583280fb7a2d700979cd5f6f61c6e1dc3652339976a380", + "width": 640, + "height": 631, + "pxPerMM": 9.2888, + "alignTargetFile": "data/bench/w028_ink.png", + "alignTargetSha256": "fb51ea9e23deda1cb6583280fb7a2d700979cd5f6f61c6e1dc3652339976a380" + }, + "prediction": { + "name": "w028_gpu.png", + "sha256": "f5b8b736eafc7bd21fd847360d4b9f676d5d94152eb3a5b0b551b5dda437d48e", + "format": "png", + "width": 640, + "height": 631, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.938, + "ty": 0.5, + "sx": 1.00211, + "sy": 0.99995, + "ncc": 0.454851, + "coarseNcc": 0.62305, + "evals": 42441, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 402570, + "validFraction": 0.996855, + "pearson": 0.454851, + "blurSigmaPx": 2.5, + "blurSigmaUM": 269.1, + "pearsonMatchedBlur": 0.50526, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.229218, + "dice": 0.372949, + "precision": 0.324688, + "recall": 0.438062, + "refPositiveFraction": 0.077664, + "predPositiveFraction": 0.104782, + "ap": 0.311547, + "bestF1": { + "f1": 0.373131, + "threshold": 0.48 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.722581, + "recall": 0.003582 + }, + { + "t": 0.98, + "precision": 0.581903, + "recall": 0.083512 + }, + { + "t": 0.96, + "precision": 0.554958, + "recall": 0.108844 + }, + { + "t": 0.94, + "precision": 0.538256, + "recall": 0.12893 + }, + { + "t": 0.92, + "precision": 0.522748, + "recall": 0.147001 + }, + { + "t": 0.9, + "precision": 0.509022, + "recall": 0.162418 + }, + { + "t": 0.88, + "precision": 0.496896, + "recall": 0.17921 + }, + { + "t": 0.86, + "precision": 0.486022, + "recall": 0.19517 + }, + { + "t": 0.84, + "precision": 0.474874, + "recall": 0.210363 + }, + { + "t": 0.82, + "precision": 0.462352, + "recall": 0.223701 + }, + { + "t": 0.8, + "precision": 0.449633, + "recall": 0.236846 + }, + { + "t": 0.78, + "precision": 0.438222, + "recall": 0.251047 + }, + { + "t": 0.76, + "precision": 0.428497, + "recall": 0.263649 + }, + { + "t": 0.74, + "precision": 0.42001, + "recall": 0.278618 + }, + { + "t": 0.72, + "precision": 0.410449, + "recall": 0.291732 + }, + { + "t": 0.7, + "precision": 0.401238, + "recall": 0.304846 + }, + { + "t": 0.68, + "precision": 0.392148, + "recall": 0.317895 + }, + { + "t": 0.66, + "precision": 0.383971, + "recall": 0.331457 + }, + { + "t": 0.64, + "precision": 0.376351, + "recall": 0.34521 + }, + { + "t": 0.62, + "precision": 0.367907, + "recall": 0.35794 + }, + { + "t": 0.6, + "precision": 0.35962, + "recall": 0.370478 + }, + { + "t": 0.58, + "precision": 0.353114, + "recall": 0.384455 + }, + { + "t": 0.56, + "precision": 0.346775, + "recall": 0.398561 + }, + { + "t": 0.54, + "precision": 0.339168, + "recall": 0.411514 + }, + { + "t": 0.52, + "precision": 0.331484, + "recall": 0.424308 + }, + { + "t": 0.5, + "precision": 0.324688, + "recall": 0.438062 + }, + { + "t": 0.48, + "precision": 0.317835, + "recall": 0.451719 + }, + { + "t": 0.46, + "precision": 0.309635, + "recall": 0.464289 + }, + { + "t": 0.44, + "precision": 0.302711, + "recall": 0.479642 + }, + { + "t": 0.42, + "precision": 0.294915, + "recall": 0.492276 + }, + { + "t": 0.4, + "precision": 0.287874, + "recall": 0.506637 + }, + { + "t": 0.38, + "precision": 0.280828, + "recall": 0.520966 + }, + { + "t": 0.36, + "precision": 0.274038, + "recall": 0.536638 + }, + { + "t": 0.34, + "precision": 0.266562, + "recall": 0.551863 + }, + { + "t": 0.32, + "precision": 0.259365, + "recall": 0.568047 + }, + { + "t": 0.3, + "precision": 0.25203, + "recall": 0.584775 + }, + { + "t": 0.28, + "precision": 0.244223, + "recall": 0.601375 + }, + { + "t": 0.26, + "precision": 0.236662, + "recall": 0.619447 + }, + { + "t": 0.24, + "precision": 0.229034, + "recall": 0.637326 + }, + { + "t": 0.22, + "precision": 0.221437, + "recall": 0.656613 + }, + { + "t": 0.2, + "precision": 0.213502, + "recall": 0.676411 + }, + { + "t": 0.18, + "precision": 0.206019, + "recall": 0.697841 + }, + { + "t": 0.16, + "precision": 0.198145, + "recall": 0.719463 + }, + { + "t": 0.14, + "precision": 0.18966, + "recall": 0.742492 + }, + { + "t": 0.12, + "precision": 0.180929, + "recall": 0.76632 + }, + { + "t": 0.1, + "precision": 0.171084, + "recall": 0.790149 + }, + { + "t": 0.08, + "precision": 0.161114, + "recall": 0.818807 + }, + { + "t": 0.06, + "precision": 0.14995, + "recall": 0.849896 + }, + { + "t": 0.04, + "precision": 0.136902, + "recall": 0.884312 + }, + { + "t": 0.02, + "precision": 0.12068, + "recall": 0.926883 + }, + { + "t": 0.0, + "precision": 0.077664, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w028-noise-baseline.json b/docs/examples/bench/w028-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..a4ef841af02ca9bbc98cc2526a25e8aea79d0971 --- /dev/null +++ b/docs/examples/bench/w028-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:11Z", + "scroll": "PHerc0139", + "segment": "w028", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w028_ink.png", + "sha256": "fb51ea9e23deda1cb6583280fb7a2d700979cd5f6f61c6e1dc3652339976a380", + "width": 640, + "height": 631, + "pxPerMM": 9.2888, + "alignTargetFile": "data/bench/w028_ink.png", + "alignTargetSha256": "fb51ea9e23deda1cb6583280fb7a2d700979cd5f6f61c6e1dc3652339976a380" + }, + "prediction": { + "name": "noise-seed28.png", + "sha256": "52eb94e6380c7cfc73c5ce51ac4313625827156c3dabaf3cfa4da7079a59ed65", + "format": "png", + "width": 640, + "height": 631, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(28).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.000344, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 403840, + "validFraction": 1.0, + "pearson": 0.000344, + "blurSigmaPx": 2.5, + "blurSigmaUM": 269.1, + "pearsonMatchedBlur": 0.000888, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.072096, + "dice": 0.134496, + "precision": 0.077698, + "recall": 0.500016, + "refPositiveFraction": 0.077419, + "predPositiveFraction": 0.498225, + "ap": 0.07761, + "bestF1": { + "f1": 0.143712, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.079095, + "recall": 0.019894 + }, + { + "t": 0.96, + "precision": 0.079659, + "recall": 0.040333 + }, + { + "t": 0.94, + "precision": 0.079411, + "recall": 0.060707 + }, + { + "t": 0.92, + "precision": 0.078498, + "recall": 0.08009 + }, + { + "t": 0.9, + "precision": 0.078083, + "recall": 0.099344 + }, + { + "t": 0.88, + "precision": 0.078175, + "recall": 0.119399 + }, + { + "t": 0.86, + "precision": 0.077512, + "recall": 0.138014 + }, + { + "t": 0.84, + "precision": 0.077998, + "recall": 0.158292 + }, + { + "t": 0.82, + "precision": 0.077531, + "recall": 0.176907 + }, + { + "t": 0.8, + "precision": 0.077421, + "recall": 0.200096 + }, + { + "t": 0.78, + "precision": 0.076636, + "recall": 0.217336 + }, + { + "t": 0.76, + "precision": 0.076497, + "recall": 0.236271 + }, + { + "t": 0.74, + "precision": 0.076852, + "recall": 0.257061 + }, + { + "t": 0.72, + "precision": 0.077035, + "recall": 0.277243 + }, + { + "t": 0.7, + "precision": 0.077077, + "recall": 0.296562 + }, + { + "t": 0.68, + "precision": 0.077222, + "recall": 0.316648 + }, + { + "t": 0.66, + "precision": 0.077509, + "recall": 0.337406 + }, + { + "t": 0.64, + "precision": 0.077438, + "recall": 0.356661 + }, + { + "t": 0.62, + "precision": 0.07753, + "recall": 0.376587 + }, + { + "t": 0.6, + "precision": 0.077666, + "recall": 0.400864 + }, + { + "t": 0.58, + "precision": 0.077724, + "recall": 0.421014 + }, + { + "t": 0.56, + "precision": 0.077799, + "recall": 0.441196 + }, + { + "t": 0.54, + "precision": 0.077688, + "recall": 0.460195 + }, + { + "t": 0.52, + "precision": 0.077634, + "recall": 0.47977 + }, + { + "t": 0.5, + "precision": 0.077698, + "recall": 0.500016 + }, + { + "t": 0.48, + "precision": 0.077633, + "recall": 0.519111 + }, + { + "t": 0.46, + "precision": 0.077789, + "recall": 0.539549 + }, + { + "t": 0.44, + "precision": 0.077672, + "recall": 0.558132 + }, + { + "t": 0.42, + "precision": 0.077751, + "recall": 0.578346 + }, + { + "t": 0.4, + "precision": 0.077645, + "recall": 0.60096 + }, + { + "t": 0.38, + "precision": 0.077629, + "recall": 0.620438 + }, + { + "t": 0.36, + "precision": 0.077544, + "recall": 0.639405 + }, + { + "t": 0.34, + "precision": 0.077549, + "recall": 0.65898 + }, + { + "t": 0.32, + "precision": 0.077555, + "recall": 0.678906 + }, + { + "t": 0.3, + "precision": 0.077546, + "recall": 0.698513 + }, + { + "t": 0.28, + "precision": 0.077486, + "recall": 0.717384 + }, + { + "t": 0.26, + "precision": 0.077462, + "recall": 0.736383 + }, + { + "t": 0.24, + "precision": 0.077391, + "recall": 0.755573 + }, + { + "t": 0.22, + "precision": 0.077379, + "recall": 0.774956 + }, + { + "t": 0.2, + "precision": 0.077204, + "recall": 0.796866 + }, + { + "t": 0.18, + "precision": 0.07719, + "recall": 0.816696 + }, + { + "t": 0.16, + "precision": 0.077266, + "recall": 0.837134 + }, + { + "t": 0.14, + "precision": 0.07729, + "recall": 0.856965 + }, + { + "t": 0.12, + "precision": 0.077491, + "recall": 0.878874 + }, + { + "t": 0.1, + "precision": 0.077416, + "recall": 0.897681 + }, + { + "t": 0.08, + "precision": 0.077444, + "recall": 0.917448 + }, + { + "t": 0.06, + "precision": 0.077327, + "recall": 0.935807 + }, + { + "t": 0.04, + "precision": 0.077358, + "recall": 0.955733 + }, + { + "t": 0.02, + "precision": 0.077461, + "recall": 0.976875 + }, + { + "t": 0.0, + "precision": 0.077419, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w029-ct-baseline.json b/docs/examples/bench/w029-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..e31d20b92a21867982c48199b127b3a874ee49f6 --- /dev/null +++ b/docs/examples/bench/w029-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:21Z", + "scroll": "PHerc0139", + "segment": "w029", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w029_ink.png", + "sha256": "5adf2a2346cc2b84cca0da0a927bc20812e7ed9f98b103211bb5836d823bf488", + "width": 640, + "height": 622, + "pxPerMM": 9.2126, + "alignTargetFile": "data/bench/w029_ink.png", + "alignTargetSha256": "5adf2a2346cc2b84cca0da0a927bc20812e7ed9f98b103211bb5836d823bf488" + }, + "prediction": { + "name": "w029_ct.png", + "sha256": "2217cff5ce56140fec1f2f34a055b94ac04a9071d42f20f7d3e9887db23e9025", + "format": "png", + "width": 640, + "height": 622, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.283609, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 398080, + "validFraction": 1.0, + "pearson": 0.283609, + "blurSigmaPx": 6.0, + "blurSigmaUM": 651.3, + "pearsonMatchedBlur": 0.41466, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.078051, + "dice": 0.144799, + "precision": 0.083946, + "recall": 0.526358, + "refPositiveFraction": 0.06252, + "predPositiveFraction": 0.392012, + "ap": 0.086866, + "bestF1": { + "f1": 0.144892, + "threshold": 0.46 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.184211, + "recall": 0.001688 + }, + { + "t": 0.98, + "precision": 0.160202, + "recall": 0.003817 + }, + { + "t": 0.96, + "precision": 0.153089, + "recall": 0.006871 + }, + { + "t": 0.94, + "precision": 0.147043, + "recall": 0.010889 + }, + { + "t": 0.92, + "precision": 0.133929, + "recall": 0.015068 + }, + { + "t": 0.9, + "precision": 0.127913, + "recall": 0.021175 + }, + { + "t": 0.88, + "precision": 0.120947, + "recall": 0.02893 + }, + { + "t": 0.86, + "precision": 0.114778, + "recall": 0.038292 + }, + { + "t": 0.84, + "precision": 0.110983, + "recall": 0.050305 + }, + { + "t": 0.82, + "precision": 0.10978, + "recall": 0.066297 + }, + { + "t": 0.8, + "precision": 0.104855, + "recall": 0.086347 + }, + { + "t": 0.78, + "precision": 0.100019, + "recall": 0.103946 + }, + { + "t": 0.76, + "precision": 0.097965, + "recall": 0.125121 + }, + { + "t": 0.74, + "precision": 0.095875, + "recall": 0.148224 + }, + { + "t": 0.72, + "precision": 0.093547, + "recall": 0.172774 + }, + { + "t": 0.7, + "precision": 0.092072, + "recall": 0.199815 + }, + { + "t": 0.68, + "precision": 0.09104, + "recall": 0.228825 + }, + { + "t": 0.66, + "precision": 0.089999, + "recall": 0.259322 + }, + { + "t": 0.64, + "precision": 0.089866, + "recall": 0.292872 + }, + { + "t": 0.62, + "precision": 0.088723, + "recall": 0.324012 + }, + { + "t": 0.6, + "precision": 0.087942, + "recall": 0.364674 + }, + { + "t": 0.58, + "precision": 0.087039, + "recall": 0.39734 + }, + { + "t": 0.56, + "precision": 0.08616, + "recall": 0.430006 + }, + { + "t": 0.54, + "precision": 0.085653, + "recall": 0.463436 + }, + { + "t": 0.52, + "precision": 0.08476, + "recall": 0.495178 + }, + { + "t": 0.5, + "precision": 0.083946, + "recall": 0.526358 + }, + { + "t": 0.48, + "precision": 0.083115, + "recall": 0.55589 + }, + { + "t": 0.46, + "precision": 0.082649, + "recall": 0.586829 + }, + { + "t": 0.44, + "precision": 0.081922, + "recall": 0.615156 + }, + { + "t": 0.42, + "precision": 0.081261, + "recall": 0.64284 + }, + { + "t": 0.4, + "precision": 0.08063, + "recall": 0.676189 + }, + { + "t": 0.38, + "precision": 0.0801, + "recall": 0.702387 + }, + { + "t": 0.36, + "precision": 0.079831, + "recall": 0.729267 + }, + { + "t": 0.34, + "precision": 0.079261, + "recall": 0.751567 + }, + { + "t": 0.32, + "precision": 0.079149, + "recall": 0.776117 + }, + { + "t": 0.3, + "precision": 0.078749, + "recall": 0.797051 + }, + { + "t": 0.28, + "precision": 0.078379, + "recall": 0.816498 + }, + { + "t": 0.26, + "precision": 0.078043, + "recall": 0.834699 + }, + { + "t": 0.24, + "precision": 0.077597, + "recall": 0.850651 + }, + { + "t": 0.22, + "precision": 0.077341, + "recall": 0.866884 + }, + { + "t": 0.2, + "precision": 0.07699, + "recall": 0.884241 + }, + { + "t": 0.18, + "precision": 0.076758, + "recall": 0.898505 + }, + { + "t": 0.16, + "precision": 0.076666, + "recall": 0.912769 + }, + { + "t": 0.14, + "precision": 0.076467, + "recall": 0.925024 + }, + { + "t": 0.12, + "precision": 0.076332, + "recall": 0.93744 + }, + { + "t": 0.1, + "precision": 0.076009, + "recall": 0.947244 + }, + { + "t": 0.08, + "precision": 0.075709, + "recall": 0.956967 + }, + { + "t": 0.06, + "precision": 0.075405, + "recall": 0.96641 + }, + { + "t": 0.04, + "precision": 0.07503, + "recall": 0.974124 + }, + { + "t": 0.02, + "precision": 0.074568, + "recall": 0.98204 + }, + { + "t": 0.0, + "precision": 0.06252, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w029-ink9um.json b/docs/examples/bench/w029-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..8e411a0f54d440498ed64acc9335e089fb7f5660 --- /dev/null +++ b/docs/examples/bench/w029-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:20Z", + "scroll": "PHerc0139", + "segment": "w029", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w029_ink.png", + "sha256": "5adf2a2346cc2b84cca0da0a927bc20812e7ed9f98b103211bb5836d823bf488", + "width": 640, + "height": 622, + "pxPerMM": 9.2126, + "alignTargetFile": "data/bench/w029_ink.png", + "alignTargetSha256": "5adf2a2346cc2b84cca0da0a927bc20812e7ed9f98b103211bb5836d823bf488" + }, + "prediction": { + "name": "w029_gpu.png", + "sha256": "2c3c563b8d544fd58025674d718c5aa3393127c710bbe8eb636b699bea6226bb", + "format": "png", + "width": 640, + "height": 622, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.531, + "ty": 0.563, + "sx": 1.00017, + "sy": 1.00003, + "ncc": 0.414241, + "coarseNcc": 0.563504, + "evals": 40466, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 396819, + "validFraction": 0.996832, + "pearson": 0.414241, + "blurSigmaPx": 2.5, + "blurSigmaUM": 271.4, + "pearsonMatchedBlur": 0.453936, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.195123, + "dice": 0.326532, + "precision": 0.294203, + "recall": 0.366843, + "refPositiveFraction": 0.062719, + "predPositiveFraction": 0.078204, + "ap": 0.260422, + "bestF1": { + "f1": 0.326532, + "threshold": 0.5 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.695122, + "recall": 0.00229 + }, + { + "t": 0.98, + "precision": 0.544421, + "recall": 0.063525 + }, + { + "t": 0.96, + "precision": 0.526801, + "recall": 0.0849 + }, + { + "t": 0.94, + "precision": 0.505219, + "recall": 0.101133 + }, + { + "t": 0.92, + "precision": 0.492611, + "recall": 0.116522 + }, + { + "t": 0.9, + "precision": 0.478453, + "recall": 0.130706 + }, + { + "t": 0.88, + "precision": 0.465384, + "recall": 0.14288 + }, + { + "t": 0.86, + "precision": 0.450117, + "recall": 0.154452 + }, + { + "t": 0.84, + "precision": 0.438036, + "recall": 0.166305 + }, + { + "t": 0.82, + "precision": 0.425267, + "recall": 0.177314 + }, + { + "t": 0.8, + "precision": 0.417616, + "recall": 0.189931 + }, + { + "t": 0.78, + "precision": 0.408876, + "recall": 0.201382 + }, + { + "t": 0.76, + "precision": 0.400634, + "recall": 0.213356 + }, + { + "t": 0.74, + "precision": 0.389325, + "recall": 0.22332 + }, + { + "t": 0.72, + "precision": 0.38004, + "recall": 0.23441 + }, + { + "t": 0.7, + "precision": 0.372298, + "recall": 0.246344 + }, + { + "t": 0.68, + "precision": 0.363827, + "recall": 0.258237 + }, + { + "t": 0.66, + "precision": 0.354988, + "recall": 0.268965 + }, + { + "t": 0.64, + "precision": 0.346864, + "recall": 0.280858 + }, + { + "t": 0.62, + "precision": 0.338663, + "recall": 0.293595 + }, + { + "t": 0.6, + "precision": 0.331113, + "recall": 0.305529 + }, + { + "t": 0.58, + "precision": 0.323816, + "recall": 0.318065 + }, + { + "t": 0.56, + "precision": 0.316114, + "recall": 0.329476 + }, + { + "t": 0.54, + "precision": 0.308565, + "recall": 0.341892 + }, + { + "t": 0.52, + "precision": 0.301938, + "recall": 0.354347 + }, + { + "t": 0.5, + "precision": 0.294203, + "recall": 0.366843 + }, + { + "t": 0.48, + "precision": 0.286195, + "recall": 0.379661 + }, + { + "t": 0.46, + "precision": 0.279366, + "recall": 0.39276 + }, + { + "t": 0.44, + "precision": 0.272033, + "recall": 0.406662 + }, + { + "t": 0.42, + "precision": 0.266019, + "recall": 0.420524 + }, + { + "t": 0.4, + "precision": 0.259994, + "recall": 0.434828 + }, + { + "t": 0.38, + "precision": 0.252852, + "recall": 0.448851 + }, + { + "t": 0.36, + "precision": 0.246982, + "recall": 0.465204 + }, + { + "t": 0.34, + "precision": 0.240776, + "recall": 0.481156 + }, + { + "t": 0.32, + "precision": 0.234066, + "recall": 0.496545 + }, + { + "t": 0.3, + "precision": 0.227627, + "recall": 0.513661 + }, + { + "t": 0.28, + "precision": 0.220426, + "recall": 0.530456 + }, + { + "t": 0.26, + "precision": 0.213104, + "recall": 0.547694 + }, + { + "t": 0.24, + "precision": 0.206376, + "recall": 0.567261 + }, + { + "t": 0.22, + "precision": 0.199176, + "recall": 0.586387 + }, + { + "t": 0.2, + "precision": 0.191306, + "recall": 0.605995 + }, + { + "t": 0.18, + "precision": 0.183499, + "recall": 0.626607 + }, + { + "t": 0.16, + "precision": 0.175531, + "recall": 0.648987 + }, + { + "t": 0.14, + "precision": 0.167725, + "recall": 0.673859 + }, + { + "t": 0.12, + "precision": 0.159448, + "recall": 0.701744 + }, + { + "t": 0.1, + "precision": 0.150783, + "recall": 0.730312 + }, + { + "t": 0.08, + "precision": 0.141554, + "recall": 0.764224 + }, + { + "t": 0.06, + "precision": 0.130862, + "recall": 0.801471 + }, + { + "t": 0.04, + "precision": 0.118619, + "recall": 0.847477 + }, + { + "t": 0.02, + "precision": 0.103374, + "recall": 0.907064 + }, + { + "t": 0.0, + "precision": 0.062719, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w029-noise-baseline.json b/docs/examples/bench/w029-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..f1029f6b48d77ae1b000eecaa5e4da7bed8848ae --- /dev/null +++ b/docs/examples/bench/w029-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:22Z", + "scroll": "PHerc0139", + "segment": "w029", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w029_ink.png", + "sha256": "5adf2a2346cc2b84cca0da0a927bc20812e7ed9f98b103211bb5836d823bf488", + "width": 640, + "height": 622, + "pxPerMM": 9.2126, + "alignTargetFile": "data/bench/w029_ink.png", + "alignTargetSha256": "5adf2a2346cc2b84cca0da0a927bc20812e7ed9f98b103211bb5836d823bf488" + }, + "prediction": { + "name": "noise-seed29.png", + "sha256": "48db4a12e3db6590fed6fbf987ce00c6070a5aff0e5e09bdc947bb14e3b7be60", + "format": "png", + "width": 640, + "height": 622, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(29).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": -0.001994, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 398080, + "validFraction": 1.0, + "pearson": -0.001994, + "blurSigmaPx": 3.5, + "blurSigmaUM": 379.9, + "pearsonMatchedBlur": -0.001284, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.058041, + "dice": 0.109713, + "precision": 0.061748, + "recall": 0.491522, + "refPositiveFraction": 0.06252, + "predPositiveFraction": 0.497666, + "ap": 0.062168, + "bestF1": { + "f1": 0.117683, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.064376, + "recall": 0.020251 + }, + { + "t": 0.96, + "precision": 0.063656, + "recall": 0.0401 + }, + { + "t": 0.94, + "precision": 0.06245, + "recall": 0.059225 + }, + { + "t": 0.92, + "precision": 0.062434, + "recall": 0.078672 + }, + { + "t": 0.9, + "precision": 0.061954, + "recall": 0.097396 + }, + { + "t": 0.88, + "precision": 0.0621, + "recall": 0.117245 + }, + { + "t": 0.86, + "precision": 0.062183, + "recall": 0.137014 + }, + { + "t": 0.84, + "precision": 0.062487, + "recall": 0.157224 + }, + { + "t": 0.82, + "precision": 0.062123, + "recall": 0.175466 + }, + { + "t": 0.8, + "precision": 0.06181, + "recall": 0.197887 + }, + { + "t": 0.78, + "precision": 0.061724, + "recall": 0.216369 + }, + { + "t": 0.76, + "precision": 0.061539, + "recall": 0.235013 + }, + { + "t": 0.74, + "precision": 0.061694, + "recall": 0.255384 + }, + { + "t": 0.72, + "precision": 0.061609, + "recall": 0.273746 + }, + { + "t": 0.7, + "precision": 0.06165, + "recall": 0.293314 + }, + { + "t": 0.68, + "precision": 0.06173, + "recall": 0.313002 + }, + { + "t": 0.66, + "precision": 0.061634, + "recall": 0.331726 + }, + { + "t": 0.64, + "precision": 0.061731, + "recall": 0.351736 + }, + { + "t": 0.62, + "precision": 0.061608, + "recall": 0.370098 + }, + { + "t": 0.6, + "precision": 0.061803, + "recall": 0.394527 + }, + { + "t": 0.58, + "precision": 0.061678, + "recall": 0.413091 + }, + { + "t": 0.56, + "precision": 0.061741, + "recall": 0.433261 + }, + { + "t": 0.54, + "precision": 0.061875, + "recall": 0.454034 + }, + { + "t": 0.52, + "precision": 0.061881, + "recall": 0.473361 + }, + { + "t": 0.5, + "precision": 0.061748, + "recall": 0.491522 + }, + { + "t": 0.48, + "precision": 0.061792, + "recall": 0.510929 + }, + { + "t": 0.46, + "precision": 0.061963, + "recall": 0.531943 + }, + { + "t": 0.44, + "precision": 0.061893, + "recall": 0.550667 + }, + { + "t": 0.42, + "precision": 0.061975, + "recall": 0.570918 + }, + { + "t": 0.4, + "precision": 0.062104, + "recall": 0.595307 + }, + { + "t": 0.38, + "precision": 0.062187, + "recall": 0.615718 + }, + { + "t": 0.36, + "precision": 0.06226, + "recall": 0.636451 + }, + { + "t": 0.34, + "precision": 0.062399, + "recall": 0.657707 + }, + { + "t": 0.32, + "precision": 0.062329, + "recall": 0.676511 + }, + { + "t": 0.3, + "precision": 0.06245, + "recall": 0.697083 + }, + { + "t": 0.28, + "precision": 0.062294, + "recall": 0.714642 + }, + { + "t": 0.26, + "precision": 0.062247, + "recall": 0.733727 + }, + { + "t": 0.24, + "precision": 0.062274, + "recall": 0.753656 + }, + { + "t": 0.22, + "precision": 0.062265, + "recall": 0.773063 + }, + { + "t": 0.2, + "precision": 0.062305, + "recall": 0.797131 + }, + { + "t": 0.18, + "precision": 0.062348, + "recall": 0.817302 + }, + { + "t": 0.16, + "precision": 0.062347, + "recall": 0.836789 + }, + { + "t": 0.14, + "precision": 0.062351, + "recall": 0.856397 + }, + { + "t": 0.12, + "precision": 0.062369, + "recall": 0.876567 + }, + { + "t": 0.1, + "precision": 0.062409, + "recall": 0.896657 + }, + { + "t": 0.08, + "precision": 0.062465, + "recall": 0.916908 + }, + { + "t": 0.06, + "precision": 0.062464, + "recall": 0.936234 + }, + { + "t": 0.04, + "precision": 0.062448, + "recall": 0.95544 + }, + { + "t": 0.02, + "precision": 0.06247, + "recall": 0.975329 + }, + { + "t": 0.0, + "precision": 0.06252, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w030-ct-baseline.json b/docs/examples/bench/w030-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..dac99b6b501636ec2330919acdc6ec2bfa029b1c --- /dev/null +++ b/docs/examples/bench/w030-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:32Z", + "scroll": "PHerc0139", + "segment": "w030", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w030_ink.png", + "sha256": "82b45b7f71d9611d22a29638f84c1f98c26e270b72d1002fec8bb90e56af5d50", + "width": 640, + "height": 612, + "pxPerMM": 9.1638, + "alignTargetFile": "data/bench/w030_ink.png", + "alignTargetSha256": "82b45b7f71d9611d22a29638f84c1f98c26e270b72d1002fec8bb90e56af5d50" + }, + "prediction": { + "name": "w030_ct.png", + "sha256": "1e739695dddf66e512678c896f56e9fbb5dbb7ac80248db08a5432892f253cd6", + "format": "png", + "width": 640, + "height": 612, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.311842, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 391680, + "validFraction": 1.0, + "pearson": 0.311842, + "blurSigmaPx": 6.0, + "blurSigmaUM": 654.8, + "pearsonMatchedBlur": 0.439538, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.158673, + "dice": 0.273887, + "precision": 0.183883, + "recall": 0.536466, + "refPositiveFraction": 0.132361, + "predPositiveFraction": 0.386152, + "ap": 0.198853, + "bestF1": { + "f1": 0.280938, + "threshold": 0.34 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.325893, + "recall": 0.001408 + }, + { + "t": 0.98, + "precision": 0.311712, + "recall": 0.003337 + }, + { + "t": 0.96, + "precision": 0.333683, + "recall": 0.006134 + }, + { + "t": 0.94, + "precision": 0.325133, + "recall": 0.009432 + }, + { + "t": 0.92, + "precision": 0.33424, + "recall": 0.014216 + }, + { + "t": 0.9, + "precision": 0.332388, + "recall": 0.02035 + }, + { + "t": 0.88, + "precision": 0.33093, + "recall": 0.028336 + }, + { + "t": 0.86, + "precision": 0.321173, + "recall": 0.038038 + }, + { + "t": 0.84, + "precision": 0.307655, + "recall": 0.049226 + }, + { + "t": 0.82, + "precision": 0.296034, + "recall": 0.063634 + }, + { + "t": 0.8, + "precision": 0.278727, + "recall": 0.083425 + }, + { + "t": 0.78, + "precision": 0.265194, + "recall": 0.101672 + }, + { + "t": 0.76, + "precision": 0.25197, + "recall": 0.123334 + }, + { + "t": 0.74, + "precision": 0.240592, + "recall": 0.146384 + }, + { + "t": 0.72, + "precision": 0.230158, + "recall": 0.171441 + }, + { + "t": 0.7, + "precision": 0.223939, + "recall": 0.200143 + }, + { + "t": 0.68, + "precision": 0.21702, + "recall": 0.229867 + }, + { + "t": 0.66, + "precision": 0.211227, + "recall": 0.261212 + }, + { + "t": 0.64, + "precision": 0.206611, + "recall": 0.294177 + }, + { + "t": 0.62, + "precision": 0.201697, + "recall": 0.327315 + }, + { + "t": 0.6, + "precision": 0.197452, + "recall": 0.368574 + }, + { + "t": 0.58, + "precision": 0.193351, + "recall": 0.400961 + }, + { + "t": 0.56, + "precision": 0.191206, + "recall": 0.436915 + }, + { + "t": 0.54, + "precision": 0.188596, + "recall": 0.470536 + }, + { + "t": 0.52, + "precision": 0.186273, + "recall": 0.504408 + }, + { + "t": 0.5, + "precision": 0.183883, + "recall": 0.536466 + }, + { + "t": 0.48, + "precision": 0.182038, + "recall": 0.568505 + }, + { + "t": 0.46, + "precision": 0.180597, + "recall": 0.600949 + }, + { + "t": 0.44, + "precision": 0.178873, + "recall": 0.631117 + }, + { + "t": 0.42, + "precision": 0.177197, + "recall": 0.65899 + }, + { + "t": 0.4, + "precision": 0.175829, + "recall": 0.693556 + }, + { + "t": 0.38, + "precision": 0.17421, + "recall": 0.71813 + }, + { + "t": 0.36, + "precision": 0.173037, + "recall": 0.742299 + }, + { + "t": 0.34, + "precision": 0.172001, + "recall": 0.766237 + }, + { + "t": 0.32, + "precision": 0.170719, + "recall": 0.78703 + }, + { + "t": 0.3, + "precision": 0.169575, + "recall": 0.806493 + }, + { + "t": 0.28, + "precision": 0.168692, + "recall": 0.825492 + }, + { + "t": 0.26, + "precision": 0.167892, + "recall": 0.843258 + }, + { + "t": 0.24, + "precision": 0.166977, + "recall": 0.859267 + }, + { + "t": 0.22, + "precision": 0.166084, + "recall": 0.873696 + }, + { + "t": 0.2, + "precision": 0.165141, + "recall": 0.889686 + }, + { + "t": 0.18, + "precision": 0.164302, + "recall": 0.901858 + }, + { + "t": 0.16, + "precision": 0.163566, + "recall": 0.913392 + }, + { + "t": 0.14, + "precision": 0.162846, + "recall": 0.924078 + }, + { + "t": 0.12, + "precision": 0.162059, + "recall": 0.934495 + }, + { + "t": 0.1, + "precision": 0.161115, + "recall": 0.944544 + }, + { + "t": 0.08, + "precision": 0.160134, + "recall": 0.954401 + }, + { + "t": 0.06, + "precision": 0.159243, + "recall": 0.964392 + }, + { + "t": 0.04, + "precision": 0.158352, + "recall": 0.974346 + }, + { + "t": 0.02, + "precision": 0.157194, + "recall": 0.98345 + }, + { + "t": 0.0, + "precision": 0.132361, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w030-ink9um.json b/docs/examples/bench/w030-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..d71f6219d6c6149792c1c7942a0ac5b0abefbdf1 --- /dev/null +++ b/docs/examples/bench/w030-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:31Z", + "scroll": "PHerc0139", + "segment": "w030", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w030_ink.png", + "sha256": "82b45b7f71d9611d22a29638f84c1f98c26e270b72d1002fec8bb90e56af5d50", + "width": 640, + "height": 612, + "pxPerMM": 9.1638, + "alignTargetFile": "data/bench/w030_ink.png", + "alignTargetSha256": "82b45b7f71d9611d22a29638f84c1f98c26e270b72d1002fec8bb90e56af5d50" + }, + "prediction": { + "name": "w030_gpu.png", + "sha256": "d0bfda06404540de9f895716b7cecbb7b87d8281625bdc77b0c21f0e58bab99b", + "format": "png", + "width": 640, + "height": 612, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.5, + "ty": 0.531, + "sx": 1.00015, + "sy": 1.00007, + "ncc": 0.509836, + "coarseNcc": 0.680604, + "evals": 40443, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 390429, + "validFraction": 0.996806, + "pearson": 0.509836, + "blurSigmaPx": 2.5, + "blurSigmaUM": 272.8, + "pearsonMatchedBlur": 0.551876, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.282712, + "dice": 0.440803, + "precision": 0.483893, + "recall": 0.404761, + "refPositiveFraction": 0.132785, + "predPositiveFraction": 0.11107, + "ap": 0.439811, + "bestF1": { + "f1": 0.456501, + "threshold": 0.34 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.891051, + "recall": 0.004417 + }, + { + "t": 0.98, + "precision": 0.765072, + "recall": 0.080532 + }, + { + "t": 0.96, + "precision": 0.742837, + "recall": 0.103023 + }, + { + "t": 0.94, + "precision": 0.727546, + "recall": 0.121405 + }, + { + "t": 0.92, + "precision": 0.710745, + "recall": 0.137164 + }, + { + "t": 0.9, + "precision": 0.696499, + "recall": 0.152364 + }, + { + "t": 0.88, + "precision": 0.685556, + "recall": 0.167081 + }, + { + "t": 0.86, + "precision": 0.671335, + "recall": 0.180294 + }, + { + "t": 0.84, + "precision": 0.660474, + "recall": 0.193392 + }, + { + "t": 0.82, + "precision": 0.646934, + "recall": 0.206373 + }, + { + "t": 0.8, + "precision": 0.637385, + "recall": 0.218757 + }, + { + "t": 0.78, + "precision": 0.625929, + "recall": 0.230774 + }, + { + "t": 0.76, + "precision": 0.614878, + "recall": 0.242984 + }, + { + "t": 0.74, + "precision": 0.604353, + "recall": 0.254924 + }, + { + "t": 0.72, + "precision": 0.593529, + "recall": 0.267153 + }, + { + "t": 0.7, + "precision": 0.584373, + "recall": 0.279286 + }, + { + "t": 0.68, + "precision": 0.574546, + "recall": 0.292055 + }, + { + "t": 0.66, + "precision": 0.564559, + "recall": 0.304381 + }, + { + "t": 0.64, + "precision": 0.553399, + "recall": 0.316243 + }, + { + "t": 0.62, + "precision": 0.542278, + "recall": 0.328685 + }, + { + "t": 0.6, + "precision": 0.531161, + "recall": 0.340142 + }, + { + "t": 0.58, + "precision": 0.521102, + "recall": 0.351774 + }, + { + "t": 0.56, + "precision": 0.511918, + "recall": 0.364543 + }, + { + "t": 0.54, + "precision": 0.502386, + "recall": 0.377756 + }, + { + "t": 0.52, + "precision": 0.493838, + "recall": 0.391123 + }, + { + "t": 0.5, + "precision": 0.483893, + "recall": 0.404761 + }, + { + "t": 0.48, + "precision": 0.474153, + "recall": 0.419131 + }, + { + "t": 0.46, + "precision": 0.46636, + "recall": 0.433598 + }, + { + "t": 0.44, + "precision": 0.455274, + "recall": 0.44629 + }, + { + "t": 0.42, + "precision": 0.445297, + "recall": 0.460062 + }, + { + "t": 0.4, + "precision": 0.435995, + "recall": 0.474336 + }, + { + "t": 0.38, + "precision": 0.426831, + "recall": 0.488513 + }, + { + "t": 0.36, + "precision": 0.416954, + "recall": 0.503501 + }, + { + "t": 0.34, + "precision": 0.407359, + "recall": 0.519125 + }, + { + "t": 0.32, + "precision": 0.397635, + "recall": 0.534576 + }, + { + "t": 0.3, + "precision": 0.388287, + "recall": 0.551569 + }, + { + "t": 0.28, + "precision": 0.377765, + "recall": 0.567598 + }, + { + "t": 0.26, + "precision": 0.36847, + "recall": 0.58681 + }, + { + "t": 0.24, + "precision": 0.357958, + "recall": 0.604344 + }, + { + "t": 0.22, + "precision": 0.347285, + "recall": 0.623556 + }, + { + "t": 0.2, + "precision": 0.336748, + "recall": 0.644311 + }, + { + "t": 0.18, + "precision": 0.325588, + "recall": 0.664892 + }, + { + "t": 0.16, + "precision": 0.314757, + "recall": 0.68854 + }, + { + "t": 0.14, + "precision": 0.303059, + "recall": 0.713404 + }, + { + "t": 0.12, + "precision": 0.290698, + "recall": 0.739 + }, + { + "t": 0.1, + "precision": 0.277313, + "recall": 0.765947 + }, + { + "t": 0.08, + "precision": 0.263115, + "recall": 0.798256 + }, + { + "t": 0.06, + "precision": 0.24665, + "recall": 0.832957 + }, + { + "t": 0.04, + "precision": 0.227545, + "recall": 0.873927 + }, + { + "t": 0.02, + "precision": 0.202597, + "recall": 0.920606 + }, + { + "t": 0.0, + "precision": 0.132785, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w030-noise-baseline.json b/docs/examples/bench/w030-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..22288a16ee90d844e308a2d1ca02c5170b58899e --- /dev/null +++ b/docs/examples/bench/w030-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:33Z", + "scroll": "PHerc0139", + "segment": "w030", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w030_ink.png", + "sha256": "82b45b7f71d9611d22a29638f84c1f98c26e270b72d1002fec8bb90e56af5d50", + "width": 640, + "height": 612, + "pxPerMM": 9.1638, + "alignTargetFile": "data/bench/w030_ink.png", + "alignTargetSha256": "82b45b7f71d9611d22a29638f84c1f98c26e270b72d1002fec8bb90e56af5d50" + }, + "prediction": { + "name": "noise-seed30.png", + "sha256": "37cc6690943a2d186506c1c5625cd4ac399ab20bec91dd7cd283533a70056a7d", + "format": "png", + "width": 640, + "height": 612, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(30).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.000863, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 391680, + "validFraction": 1.0, + "pearson": 0.000863, + "blurSigmaPx": 6.0, + "blurSigmaUM": 654.8, + "pearsonMatchedBlur": 0.002347, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.116669, + "dice": 0.208958, + "precision": 0.1322, + "recall": 0.498254, + "refPositiveFraction": 0.132361, + "predPositiveFraction": 0.498859, + "ap": 0.132479, + "bestF1": { + "f1": 0.233778, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.13094, + "recall": 0.019559 + }, + { + "t": 0.96, + "precision": 0.132352, + "recall": 0.039542 + }, + { + "t": 0.94, + "precision": 0.129942, + "recall": 0.05833 + }, + { + "t": 0.92, + "precision": 0.130153, + "recall": 0.077773 + }, + { + "t": 0.9, + "precision": 0.131841, + "recall": 0.098046 + }, + { + "t": 0.88, + "precision": 0.132784, + "recall": 0.118203 + }, + { + "t": 0.86, + "precision": 0.13311, + "recall": 0.138553 + }, + { + "t": 0.84, + "precision": 0.133434, + "recall": 0.158768 + }, + { + "t": 0.82, + "precision": 0.13349, + "recall": 0.17852 + }, + { + "t": 0.8, + "precision": 0.133653, + "recall": 0.20238 + }, + { + "t": 0.78, + "precision": 0.133447, + "recall": 0.221573 + }, + { + "t": 0.76, + "precision": 0.133609, + "recall": 0.241633 + }, + { + "t": 0.74, + "precision": 0.13323, + "recall": 0.261154 + }, + { + "t": 0.72, + "precision": 0.133052, + "recall": 0.280616 + }, + { + "t": 0.7, + "precision": 0.132781, + "recall": 0.299732 + }, + { + "t": 0.68, + "precision": 0.132898, + "recall": 0.319445 + }, + { + "t": 0.66, + "precision": 0.132388, + "recall": 0.338233 + }, + { + "t": 0.64, + "precision": 0.13212, + "recall": 0.357078 + }, + { + "t": 0.62, + "precision": 0.132021, + "recall": 0.376753 + }, + { + "t": 0.6, + "precision": 0.132093, + "recall": 0.400247 + }, + { + "t": 0.58, + "precision": 0.132148, + "recall": 0.41996 + }, + { + "t": 0.56, + "precision": 0.132108, + "recall": 0.439134 + }, + { + "t": 0.54, + "precision": 0.132086, + "recall": 0.458673 + }, + { + "t": 0.52, + "precision": 0.132006, + "recall": 0.478039 + }, + { + "t": 0.5, + "precision": 0.1322, + "recall": 0.498254 + }, + { + "t": 0.48, + "precision": 0.13227, + "recall": 0.518045 + }, + { + "t": 0.46, + "precision": 0.132287, + "recall": 0.537623 + }, + { + "t": 0.44, + "precision": 0.132327, + "recall": 0.557163 + }, + { + "t": 0.42, + "precision": 0.132235, + "recall": 0.576433 + }, + { + "t": 0.4, + "precision": 0.132542, + "recall": 0.601277 + }, + { + "t": 0.38, + "precision": 0.132543, + "recall": 0.620894 + }, + { + "t": 0.36, + "precision": 0.132549, + "recall": 0.640665 + }, + { + "t": 0.34, + "precision": 0.132794, + "recall": 0.661574 + }, + { + "t": 0.32, + "precision": 0.13298, + "recall": 0.682445 + }, + { + "t": 0.3, + "precision": 0.132765, + "recall": 0.701098 + }, + { + "t": 0.28, + "precision": 0.132728, + "recall": 0.720444 + }, + { + "t": 0.26, + "precision": 0.132764, + "recall": 0.740563 + }, + { + "t": 0.24, + "precision": 0.132822, + "recall": 0.760874 + }, + { + "t": 0.22, + "precision": 0.13296, + "recall": 0.781398 + }, + { + "t": 0.2, + "precision": 0.132749, + "recall": 0.803561 + }, + { + "t": 0.18, + "precision": 0.132663, + "recall": 0.822464 + }, + { + "t": 0.16, + "precision": 0.132642, + "recall": 0.841946 + }, + { + "t": 0.14, + "precision": 0.132575, + "recall": 0.86083 + }, + { + "t": 0.12, + "precision": 0.132495, + "recall": 0.880119 + }, + { + "t": 0.1, + "precision": 0.132438, + "recall": 0.899389 + }, + { + "t": 0.08, + "precision": 0.132169, + "recall": 0.917057 + }, + { + "t": 0.06, + "precision": 0.132176, + "recall": 0.936366 + }, + { + "t": 0.04, + "precision": 0.132374, + "recall": 0.957313 + }, + { + "t": 0.02, + "precision": 0.132359, + "recall": 0.976525 + }, + { + "t": 0.0, + "precision": 0.132361, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w031-ct-baseline.json b/docs/examples/bench/w031-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..044e12f2515959b4c50d23064090e189a5f5d466 --- /dev/null +++ b/docs/examples/bench/w031-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:44Z", + "scroll": "PHerc0139", + "segment": "w031", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w031_ink.png", + "sha256": "13fe38e344fb4b870204aa52c3422b8863af749e6a39d20823a0c9bff8f92e43", + "width": 640, + "height": 603, + "pxPerMM": 9.1142, + "alignTargetFile": "data/bench/w031_ink.png", + "alignTargetSha256": "13fe38e344fb4b870204aa52c3422b8863af749e6a39d20823a0c9bff8f92e43" + }, + "prediction": { + "name": "w031_ct.png", + "sha256": "371e1428275d44a00eb5bddf4f87ce93ddb6803cc134537e3304823ee2412be3", + "format": "png", + "width": 640, + "height": 603, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.325053, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 385920, + "validFraction": 1.0, + "pearson": 0.325053, + "blurSigmaPx": 6.0, + "blurSigmaUM": 658.3, + "pearsonMatchedBlur": 0.445986, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.15125, + "dice": 0.262758, + "precision": 0.170754, + "recall": 0.569747, + "refPositiveFraction": 0.110748, + "predPositiveFraction": 0.36953, + "ap": 0.184266, + "bestF1": { + "f1": 0.263073, + "threshold": 0.52 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.284264, + "recall": 0.00131 + }, + { + "t": 0.98, + "precision": 0.273245, + "recall": 0.003369 + }, + { + "t": 0.96, + "precision": 0.294118, + "recall": 0.006785 + }, + { + "t": 0.94, + "precision": 0.295918, + "recall": 0.011535 + }, + { + "t": 0.92, + "precision": 0.293058, + "recall": 0.018273 + }, + { + "t": 0.9, + "precision": 0.290805, + "recall": 0.027305 + }, + { + "t": 0.88, + "precision": 0.279916, + "recall": 0.037436 + }, + { + "t": 0.86, + "precision": 0.269447, + "recall": 0.050491 + }, + { + "t": 0.84, + "precision": 0.261241, + "recall": 0.065793 + }, + { + "t": 0.82, + "precision": 0.253741, + "recall": 0.084113 + }, + { + "t": 0.8, + "precision": 0.247418, + "recall": 0.110997 + }, + { + "t": 0.78, + "precision": 0.239764, + "recall": 0.134815 + }, + { + "t": 0.76, + "precision": 0.230999, + "recall": 0.159359 + }, + { + "t": 0.74, + "precision": 0.223611, + "recall": 0.186313 + }, + { + "t": 0.72, + "precision": 0.215607, + "recall": 0.214296 + }, + { + "t": 0.7, + "precision": 0.210328, + "recall": 0.244806 + }, + { + "t": 0.68, + "precision": 0.204605, + "recall": 0.275456 + }, + { + "t": 0.66, + "precision": 0.199201, + "recall": 0.306926 + }, + { + "t": 0.64, + "precision": 0.194113, + "recall": 0.338699 + }, + { + "t": 0.62, + "precision": 0.19115, + "recall": 0.373467 + }, + { + "t": 0.6, + "precision": 0.187216, + "recall": 0.414109 + }, + { + "t": 0.58, + "precision": 0.183323, + "recall": 0.445882 + }, + { + "t": 0.56, + "precision": 0.179394, + "recall": 0.476883 + }, + { + "t": 0.54, + "precision": 0.176383, + "recall": 0.508259 + }, + { + "t": 0.52, + "precision": 0.173848, + "recall": 0.540454 + }, + { + "t": 0.5, + "precision": 0.170754, + "recall": 0.569747 + }, + { + "t": 0.48, + "precision": 0.168146, + "recall": 0.599204 + }, + { + "t": 0.46, + "precision": 0.165524, + "recall": 0.627188 + }, + { + "t": 0.44, + "precision": 0.163312, + "recall": 0.655498 + }, + { + "t": 0.42, + "precision": 0.160805, + "recall": 0.681352 + }, + { + "t": 0.4, + "precision": 0.158184, + "recall": 0.711675 + }, + { + "t": 0.38, + "precision": 0.155904, + "recall": 0.735236 + }, + { + "t": 0.36, + "precision": 0.154151, + "recall": 0.758774 + }, + { + "t": 0.34, + "precision": 0.152202, + "recall": 0.779738 + }, + { + "t": 0.32, + "precision": 0.150455, + "recall": 0.8 + }, + { + "t": 0.3, + "precision": 0.148552, + "recall": 0.817548 + }, + { + "t": 0.28, + "precision": 0.146804, + "recall": 0.833482 + }, + { + "t": 0.26, + "precision": 0.145207, + "recall": 0.8489 + }, + { + "t": 0.24, + "precision": 0.143765, + "recall": 0.863687 + }, + { + "t": 0.22, + "precision": 0.142307, + "recall": 0.87679 + }, + { + "t": 0.2, + "precision": 0.14094, + "recall": 0.893262 + }, + { + "t": 0.18, + "precision": 0.139691, + "recall": 0.904609 + }, + { + "t": 0.16, + "precision": 0.138648, + "recall": 0.91598 + }, + { + "t": 0.14, + "precision": 0.137666, + "recall": 0.926673 + }, + { + "t": 0.12, + "precision": 0.136708, + "recall": 0.937295 + }, + { + "t": 0.1, + "precision": 0.135699, + "recall": 0.947146 + }, + { + "t": 0.08, + "precision": 0.134638, + "recall": 0.957417 + }, + { + "t": 0.06, + "precision": 0.133532, + "recall": 0.967501 + }, + { + "t": 0.04, + "precision": 0.132381, + "recall": 0.97693 + }, + { + "t": 0.02, + "precision": 0.131123, + "recall": 0.985447 + }, + { + "t": 0.0, + "precision": 0.110748, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w031-ink9um.json b/docs/examples/bench/w031-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..1432d910177faf1499374832a01b2d9fbf7bda0f --- /dev/null +++ b/docs/examples/bench/w031-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:43Z", + "scroll": "PHerc0139", + "segment": "w031", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w031_ink.png", + "sha256": "13fe38e344fb4b870204aa52c3422b8863af749e6a39d20823a0c9bff8f92e43", + "width": 640, + "height": 603, + "pxPerMM": 9.1142, + "alignTargetFile": "data/bench/w031_ink.png", + "alignTargetSha256": "13fe38e344fb4b870204aa52c3422b8863af749e6a39d20823a0c9bff8f92e43" + }, + "prediction": { + "name": "w031_gpu.png", + "sha256": "93fc5e9939c05dadc090dd0bb77090fe4016b9d3b45cc5bb0469beaa746a208c", + "format": "png", + "width": 640, + "height": 603, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.5, + "ty": 0.75, + "sx": 1.00021, + "sy": 1.0025, + "ncc": 0.445668, + "coarseNcc": 0.62556, + "evals": 40365, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 385317, + "validFraction": 0.998438, + "pearson": 0.445668, + "blurSigmaPx": 2.5, + "blurSigmaUM": 274.3, + "pearsonMatchedBlur": 0.488539, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.227832, + "dice": 0.371112, + "precision": 0.416562, + "recall": 0.334605, + "refPositiveFraction": 0.110922, + "predPositiveFraction": 0.089098, + "ap": 0.356774, + "bestF1": { + "f1": 0.396494, + "threshold": 0.34 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.910112, + "recall": 0.001895 + }, + { + "t": 0.98, + "precision": 0.71521, + "recall": 0.046537 + }, + { + "t": 0.96, + "precision": 0.6886, + "recall": 0.063173 + }, + { + "t": 0.94, + "precision": 0.662513, + "recall": 0.075877 + }, + { + "t": 0.92, + "precision": 0.651123, + "recall": 0.088863 + }, + { + "t": 0.9, + "precision": 0.634473, + "recall": 0.099906 + }, + { + "t": 0.88, + "precision": 0.620415, + "recall": 0.111207 + }, + { + "t": 0.86, + "precision": 0.604896, + "recall": 0.122578 + }, + { + "t": 0.84, + "precision": 0.588661, + "recall": 0.133131 + }, + { + "t": 0.82, + "precision": 0.574822, + "recall": 0.1438 + }, + { + "t": 0.8, + "precision": 0.56422, + "recall": 0.155405 + }, + { + "t": 0.78, + "precision": 0.554205, + "recall": 0.166378 + }, + { + "t": 0.76, + "precision": 0.540862, + "recall": 0.176369 + }, + { + "t": 0.74, + "precision": 0.529081, + "recall": 0.187085 + }, + { + "t": 0.72, + "precision": 0.518096, + "recall": 0.19862 + }, + { + "t": 0.7, + "precision": 0.509376, + "recall": 0.210997 + }, + { + "t": 0.68, + "precision": 0.499632, + "recall": 0.222508 + }, + { + "t": 0.66, + "precision": 0.488251, + "recall": 0.233832 + }, + { + "t": 0.64, + "precision": 0.478449, + "recall": 0.245952 + }, + { + "t": 0.62, + "precision": 0.469367, + "recall": 0.257581 + }, + { + "t": 0.6, + "precision": 0.459789, + "recall": 0.269139 + }, + { + "t": 0.58, + "precision": 0.450442, + "recall": 0.281563 + }, + { + "t": 0.56, + "precision": 0.441258, + "recall": 0.294478 + }, + { + "t": 0.54, + "precision": 0.433578, + "recall": 0.307674 + }, + { + "t": 0.52, + "precision": 0.425189, + "recall": 0.320613 + }, + { + "t": 0.5, + "precision": 0.416562, + "recall": 0.334605 + }, + { + "t": 0.48, + "precision": 0.408345, + "recall": 0.347801 + }, + { + "t": 0.46, + "precision": 0.400722, + "recall": 0.363594 + }, + { + "t": 0.44, + "precision": 0.392574, + "recall": 0.379457 + }, + { + "t": 0.42, + "precision": 0.384589, + "recall": 0.394712 + }, + { + "t": 0.4, + "precision": 0.375289, + "recall": 0.40985 + }, + { + "t": 0.38, + "precision": 0.366767, + "recall": 0.426673 + }, + { + "t": 0.36, + "precision": 0.35835, + "recall": 0.442513 + }, + { + "t": 0.34, + "precision": 0.349049, + "recall": 0.458868 + }, + { + "t": 0.32, + "precision": 0.338891, + "recall": 0.475082 + }, + { + "t": 0.3, + "precision": 0.330286, + "recall": 0.494525 + }, + { + "t": 0.28, + "precision": 0.321549, + "recall": 0.513945 + }, + { + "t": 0.26, + "precision": 0.312245, + "recall": 0.534417 + }, + { + "t": 0.24, + "precision": 0.302714, + "recall": 0.555686 + }, + { + "t": 0.22, + "precision": 0.293601, + "recall": 0.578498 + }, + { + "t": 0.2, + "precision": 0.284338, + "recall": 0.602971 + }, + { + "t": 0.18, + "precision": 0.274541, + "recall": 0.628124 + }, + { + "t": 0.16, + "precision": 0.263496, + "recall": 0.654282 + }, + { + "t": 0.14, + "precision": 0.252551, + "recall": 0.682218 + }, + { + "t": 0.12, + "precision": 0.240962, + "recall": 0.713313 + }, + { + "t": 0.1, + "precision": 0.229145, + "recall": 0.746701 + }, + { + "t": 0.08, + "precision": 0.216478, + "recall": 0.782358 + }, + { + "t": 0.06, + "precision": 0.20191, + "recall": 0.822602 + }, + { + "t": 0.04, + "precision": 0.184894, + "recall": 0.867688 + }, + { + "t": 0.02, + "precision": 0.163075, + "recall": 0.921783 + }, + { + "t": 0.0, + "precision": 0.110922, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w031-noise-baseline.json b/docs/examples/bench/w031-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..4bb5c4b038d2c04513c821ffbb6d95cf83f12060 --- /dev/null +++ b/docs/examples/bench/w031-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:45Z", + "scroll": "PHerc0139", + "segment": "w031", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w031_ink.png", + "sha256": "13fe38e344fb4b870204aa52c3422b8863af749e6a39d20823a0c9bff8f92e43", + "width": 640, + "height": 603, + "pxPerMM": 9.1142, + "alignTargetFile": "data/bench/w031_ink.png", + "alignTargetSha256": "13fe38e344fb4b870204aa52c3422b8863af749e6a39d20823a0c9bff8f92e43" + }, + "prediction": { + "name": "noise-seed31.png", + "sha256": "958a4396de210f297413e4fc513979e3cd80267b4f419094966c1fb4e406a53f", + "format": "png", + "width": 640, + "height": 603, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(31).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.000543, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 385920, + "validFraction": 1.0, + "pearson": 0.000543, + "blurSigmaPx": 2.0, + "blurSigmaUM": 219.4, + "pearsonMatchedBlur": 0.000557, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.099456, + "dice": 0.180919, + "precision": 0.110552, + "recall": 0.49773, + "refPositiveFraction": 0.110748, + "predPositiveFraction": 0.498616, + "ap": 0.11112, + "bestF1": { + "f1": 0.199412, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.113523, + "recall": 0.020192 + }, + { + "t": 0.96, + "precision": 0.113784, + "recall": 0.040056 + }, + { + "t": 0.94, + "precision": 0.112007, + "recall": 0.059172 + }, + { + "t": 0.92, + "precision": 0.112063, + "recall": 0.07927 + }, + { + "t": 0.9, + "precision": 0.111667, + "recall": 0.098643 + }, + { + "t": 0.88, + "precision": 0.112132, + "recall": 0.119045 + }, + { + "t": 0.86, + "precision": 0.11229, + "recall": 0.139144 + }, + { + "t": 0.84, + "precision": 0.112019, + "recall": 0.159055 + }, + { + "t": 0.82, + "precision": 0.111819, + "recall": 0.178662 + }, + { + "t": 0.8, + "precision": 0.111171, + "recall": 0.201053 + }, + { + "t": 0.78, + "precision": 0.111025, + "recall": 0.220707 + }, + { + "t": 0.76, + "precision": 0.110696, + "recall": 0.239658 + }, + { + "t": 0.74, + "precision": 0.1109, + "recall": 0.25985 + }, + { + "t": 0.72, + "precision": 0.111048, + "recall": 0.279878 + }, + { + "t": 0.7, + "precision": 0.110951, + "recall": 0.299275 + }, + { + "t": 0.68, + "precision": 0.111199, + "recall": 0.319724 + }, + { + "t": 0.66, + "precision": 0.110775, + "recall": 0.33788 + }, + { + "t": 0.64, + "precision": 0.110835, + "recall": 0.357698 + }, + { + "t": 0.62, + "precision": 0.11079, + "recall": 0.377211 + }, + { + "t": 0.6, + "precision": 0.110561, + "recall": 0.4 + }, + { + "t": 0.58, + "precision": 0.11073, + "recall": 0.420473 + }, + { + "t": 0.56, + "precision": 0.110832, + "recall": 0.440337 + }, + { + "t": 0.54, + "precision": 0.11076, + "recall": 0.459663 + }, + { + "t": 0.52, + "precision": 0.110727, + "recall": 0.478966 + }, + { + "t": 0.5, + "precision": 0.110552, + "recall": 0.49773 + }, + { + "t": 0.48, + "precision": 0.110789, + "recall": 0.518414 + }, + { + "t": 0.46, + "precision": 0.110867, + "recall": 0.538676 + }, + { + "t": 0.44, + "precision": 0.110981, + "recall": 0.55868 + }, + { + "t": 0.42, + "precision": 0.110803, + "recall": 0.577398 + }, + { + "t": 0.4, + "precision": 0.110766, + "recall": 0.600608 + }, + { + "t": 0.38, + "precision": 0.110715, + "recall": 0.619958 + }, + { + "t": 0.36, + "precision": 0.110881, + "recall": 0.640267 + }, + { + "t": 0.34, + "precision": 0.110905, + "recall": 0.660154 + }, + { + "t": 0.32, + "precision": 0.110912, + "recall": 0.679925 + }, + { + "t": 0.3, + "precision": 0.11098, + "recall": 0.700164 + }, + { + "t": 0.28, + "precision": 0.110777, + "recall": 0.718905 + }, + { + "t": 0.26, + "precision": 0.11079, + "recall": 0.738699 + }, + { + "t": 0.24, + "precision": 0.110857, + "recall": 0.758704 + }, + { + "t": 0.22, + "precision": 0.110915, + "recall": 0.778966 + }, + { + "t": 0.2, + "precision": 0.110983, + "recall": 0.802761 + }, + { + "t": 0.18, + "precision": 0.110912, + "recall": 0.821993 + }, + { + "t": 0.16, + "precision": 0.11092, + "recall": 0.841717 + }, + { + "t": 0.14, + "precision": 0.110863, + "recall": 0.861067 + }, + { + "t": 0.12, + "precision": 0.110863, + "recall": 0.880393 + }, + { + "t": 0.1, + "precision": 0.110912, + "recall": 0.900117 + }, + { + "t": 0.08, + "precision": 0.110881, + "recall": 0.91942 + }, + { + "t": 0.06, + "precision": 0.110793, + "recall": 0.937974 + }, + { + "t": 0.04, + "precision": 0.110669, + "recall": 0.956177 + }, + { + "t": 0.02, + "precision": 0.110647, + "recall": 0.97548 + }, + { + "t": 0.0, + "precision": 0.110748, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w032-ct-baseline.json b/docs/examples/bench/w032-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..f4c07c9c371cbea1735f1589fbb228fe8d58f5ef --- /dev/null +++ b/docs/examples/bench/w032-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:54Z", + "scroll": "PHerc0139", + "segment": "w032", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w032_ink.png", + "sha256": "1f595b4067a0317f874e5e2d5d229828408c4c3cd198ef41999c768570e38c50", + "width": 640, + "height": 597, + "pxPerMM": 8.9951, + "alignTargetFile": "data/bench/w032_ink.png", + "alignTargetSha256": "1f595b4067a0317f874e5e2d5d229828408c4c3cd198ef41999c768570e38c50" + }, + "prediction": { + "name": "w032_ct.png", + "sha256": "83a374f9ad2d93d0fcb30d1fcc6e632830fbd9b6d2bb6aa920c8ae2562674fd1", + "format": "png", + "width": 640, + "height": 597, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.319898, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 382080, + "validFraction": 1.0, + "pearson": 0.319898, + "blurSigmaPx": 6.0, + "blurSigmaUM": 667.0, + "pearsonMatchedBlur": 0.426249, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.202316, + "dice": 0.336544, + "precision": 0.238425, + "recall": 0.571893, + "refPositiveFraction": 0.166716, + "predPositiveFraction": 0.39989, + "ap": 0.243543, + "bestF1": { + "f1": 0.342067, + "threshold": 0.42 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.356098, + "recall": 0.001146 + }, + { + "t": 0.98, + "precision": 0.356322, + "recall": 0.00292 + }, + { + "t": 0.96, + "precision": 0.327468, + "recall": 0.005259 + }, + { + "t": 0.94, + "precision": 0.33022, + "recall": 0.009435 + }, + { + "t": 0.92, + "precision": 0.32116, + "recall": 0.014773 + }, + { + "t": 0.9, + "precision": 0.315043, + "recall": 0.021633 + }, + { + "t": 0.88, + "precision": 0.315534, + "recall": 0.031633 + }, + { + "t": 0.86, + "precision": 0.304522, + "recall": 0.043031 + }, + { + "t": 0.84, + "precision": 0.297891, + "recall": 0.057662 + }, + { + "t": 0.82, + "precision": 0.29225, + "recall": 0.074711 + }, + { + "t": 0.8, + "precision": 0.289017, + "recall": 0.099515 + }, + { + "t": 0.78, + "precision": 0.283238, + "recall": 0.122341 + }, + { + "t": 0.76, + "precision": 0.277728, + "recall": 0.146941 + }, + { + "t": 0.74, + "precision": 0.274317, + "recall": 0.174571 + }, + { + "t": 0.72, + "precision": 0.270102, + "recall": 0.203771 + }, + { + "t": 0.7, + "precision": 0.26599, + "recall": 0.234242 + }, + { + "t": 0.68, + "precision": 0.263485, + "recall": 0.266331 + }, + { + "t": 0.66, + "precision": 0.259646, + "recall": 0.298027 + }, + { + "t": 0.64, + "precision": 0.257347, + "recall": 0.331842 + }, + { + "t": 0.62, + "precision": 0.254168, + "recall": 0.364762 + }, + { + "t": 0.6, + "precision": 0.251704, + "recall": 0.406961 + }, + { + "t": 0.58, + "precision": 0.249071, + "recall": 0.440698 + }, + { + "t": 0.56, + "precision": 0.246817, + "recall": 0.475062 + }, + { + "t": 0.54, + "precision": 0.244458, + "recall": 0.50894 + }, + { + "t": 0.52, + "precision": 0.241645, + "recall": 0.541359 + }, + { + "t": 0.5, + "precision": 0.238425, + "recall": 0.571893 + }, + { + "t": 0.48, + "precision": 0.2359, + "recall": 0.603039 + }, + { + "t": 0.46, + "precision": 0.232979, + "recall": 0.632459 + }, + { + "t": 0.44, + "precision": 0.230169, + "recall": 0.660277 + }, + { + "t": 0.42, + "precision": 0.227663, + "recall": 0.687593 + }, + { + "t": 0.4, + "precision": 0.224375, + "recall": 0.717107 + }, + { + "t": 0.38, + "precision": 0.221774, + "recall": 0.739808 + }, + { + "t": 0.36, + "precision": 0.219399, + "recall": 0.762618 + }, + { + "t": 0.34, + "precision": 0.217018, + "recall": 0.783168 + }, + { + "t": 0.32, + "precision": 0.214903, + "recall": 0.802807 + }, + { + "t": 0.3, + "precision": 0.212843, + "recall": 0.820829 + }, + { + "t": 0.28, + "precision": 0.211033, + "recall": 0.837548 + }, + { + "t": 0.26, + "precision": 0.209257, + "recall": 0.852588 + }, + { + "t": 0.24, + "precision": 0.207627, + "recall": 0.866733 + }, + { + "t": 0.22, + "precision": 0.206104, + "recall": 0.879998 + }, + { + "t": 0.2, + "precision": 0.204467, + "recall": 0.894614 + }, + { + "t": 0.18, + "precision": 0.203269, + "recall": 0.906184 + }, + { + "t": 0.16, + "precision": 0.202148, + "recall": 0.916419 + }, + { + "t": 0.14, + "precision": 0.201058, + "recall": 0.925933 + }, + { + "t": 0.12, + "precision": 0.200219, + "recall": 0.935823 + }, + { + "t": 0.1, + "precision": 0.199423, + "recall": 0.945635 + }, + { + "t": 0.08, + "precision": 0.198581, + "recall": 0.954834 + }, + { + "t": 0.06, + "precision": 0.197806, + "recall": 0.964945 + }, + { + "t": 0.04, + "precision": 0.196959, + "recall": 0.974128 + }, + { + "t": 0.02, + "precision": 0.195799, + "recall": 0.983296 + }, + { + "t": 0.0, + "precision": 0.166716, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w032-ink9um.json b/docs/examples/bench/w032-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..7709fb9745044218bdf3d67adc1b3f7a2e35d34c --- /dev/null +++ b/docs/examples/bench/w032-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:53Z", + "scroll": "PHerc0139", + "segment": "w032", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w032_ink.png", + "sha256": "1f595b4067a0317f874e5e2d5d229828408c4c3cd198ef41999c768570e38c50", + "width": 640, + "height": 597, + "pxPerMM": 8.9951, + "alignTargetFile": "data/bench/w032_ink.png", + "alignTargetSha256": "1f595b4067a0317f874e5e2d5d229828408c4c3cd198ef41999c768570e38c50" + }, + "prediction": { + "name": "w032_gpu.png", + "sha256": "ec48088615f9f7a3f80bfe61d130362a50d59d7de3d9bf2565f21f12212efbc5", + "format": "png", + "width": 640, + "height": 597, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.563, + "ty": 0.75, + "sx": 1.00321, + "sy": 1.00253, + "ncc": 0.534102, + "coarseNcc": 0.711904, + "evals": 40313, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 382080, + "validFraction": 1.0, + "pearson": 0.534102, + "blurSigmaPx": 2.5, + "blurSigmaUM": 277.9, + "pearsonMatchedBlur": 0.577377, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.313862, + "dice": 0.47777, + "precision": 0.539843, + "recall": 0.4285, + "refPositiveFraction": 0.166716, + "predPositiveFraction": 0.132331, + "ap": 0.505747, + "bestF1": { + "f1": 0.501774, + "threshold": 0.3 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.930599, + "recall": 0.004631 + }, + { + "t": 0.98, + "precision": 0.819542, + "recall": 0.085983 + }, + { + "t": 0.96, + "precision": 0.798672, + "recall": 0.111415 + }, + { + "t": 0.94, + "precision": 0.782983, + "recall": 0.131462 + }, + { + "t": 0.92, + "precision": 0.769312, + "recall": 0.149155 + }, + { + "t": 0.9, + "precision": 0.754449, + "recall": 0.165058 + }, + { + "t": 0.88, + "precision": 0.741861, + "recall": 0.180646 + }, + { + "t": 0.86, + "precision": 0.729671, + "recall": 0.195388 + }, + { + "t": 0.84, + "precision": 0.716058, + "recall": 0.209313 + }, + { + "t": 0.82, + "precision": 0.704569, + "recall": 0.222468 + }, + { + "t": 0.8, + "precision": 0.694264, + "recall": 0.236173 + }, + { + "t": 0.78, + "precision": 0.682869, + "recall": 0.249439 + }, + { + "t": 0.76, + "precision": 0.674028, + "recall": 0.263065 + }, + { + "t": 0.74, + "precision": 0.663458, + "recall": 0.275907 + }, + { + "t": 0.72, + "precision": 0.653425, + "recall": 0.288702 + }, + { + "t": 0.7, + "precision": 0.642405, + "recall": 0.301371 + }, + { + "t": 0.68, + "precision": 0.632082, + "recall": 0.313506 + }, + { + "t": 0.66, + "precision": 0.620718, + "recall": 0.325672 + }, + { + "t": 0.64, + "precision": 0.610115, + "recall": 0.338624 + }, + { + "t": 0.62, + "precision": 0.59977, + "recall": 0.351308 + }, + { + "t": 0.6, + "precision": 0.590261, + "recall": 0.363836 + }, + { + "t": 0.58, + "precision": 0.58068, + "recall": 0.376819 + }, + { + "t": 0.56, + "precision": 0.571268, + "recall": 0.390163 + }, + { + "t": 0.54, + "precision": 0.560983, + "recall": 0.403491 + }, + { + "t": 0.52, + "precision": 0.550263, + "recall": 0.415831 + }, + { + "t": 0.5, + "precision": 0.539843, + "recall": 0.4285 + }, + { + "t": 0.48, + "precision": 0.530358, + "recall": 0.442111 + }, + { + "t": 0.46, + "precision": 0.521121, + "recall": 0.455517 + }, + { + "t": 0.44, + "precision": 0.511918, + "recall": 0.469992 + }, + { + "t": 0.42, + "precision": 0.501898, + "recall": 0.483744 + }, + { + "t": 0.4, + "precision": 0.49263, + "recall": 0.498972 + }, + { + "t": 0.38, + "precision": 0.482427, + "recall": 0.514796 + }, + { + "t": 0.36, + "precision": 0.472673, + "recall": 0.529098 + }, + { + "t": 0.34, + "precision": 0.462471, + "recall": 0.544106 + }, + { + "t": 0.32, + "precision": 0.452766, + "recall": 0.561296 + }, + { + "t": 0.3, + "precision": 0.443164, + "recall": 0.578251 + }, + { + "t": 0.28, + "precision": 0.432809, + "recall": 0.596226 + }, + { + "t": 0.26, + "precision": 0.422342, + "recall": 0.615237 + }, + { + "t": 0.24, + "precision": 0.411363, + "recall": 0.634594 + }, + { + "t": 0.22, + "precision": 0.40014, + "recall": 0.654877 + }, + { + "t": 0.2, + "precision": 0.388741, + "recall": 0.675693 + }, + { + "t": 0.18, + "precision": 0.37711, + "recall": 0.697185 + }, + { + "t": 0.16, + "precision": 0.364912, + "recall": 0.72166 + }, + { + "t": 0.14, + "precision": 0.352026, + "recall": 0.746495 + }, + { + "t": 0.12, + "precision": 0.338549, + "recall": 0.773403 + }, + { + "t": 0.1, + "precision": 0.32417, + "recall": 0.800829 + }, + { + "t": 0.08, + "precision": 0.308528, + "recall": 0.831552 + }, + { + "t": 0.06, + "precision": 0.289941, + "recall": 0.862981 + }, + { + "t": 0.04, + "precision": 0.267605, + "recall": 0.898586 + }, + { + "t": 0.02, + "precision": 0.238919, + "recall": 0.939858 + }, + { + "t": 0.0, + "precision": 0.166716, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w032-noise-baseline.json b/docs/examples/bench/w032-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..c0eaa11e9130de9460e95a21bc53c2c57f7434de --- /dev/null +++ b/docs/examples/bench/w032-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:51:55Z", + "scroll": "PHerc0139", + "segment": "w032", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w032_ink.png", + "sha256": "1f595b4067a0317f874e5e2d5d229828408c4c3cd198ef41999c768570e38c50", + "width": 640, + "height": 597, + "pxPerMM": 8.9951, + "alignTargetFile": "data/bench/w032_ink.png", + "alignTargetSha256": "1f595b4067a0317f874e5e2d5d229828408c4c3cd198ef41999c768570e38c50" + }, + "prediction": { + "name": "noise-seed32.png", + "sha256": "e578d6339f4855743eea066564d06f3f888d0c98f010ed4ab9045baa07939118", + "format": "png", + "width": 640, + "height": 597, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(32).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": -0.000693, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 382080, + "validFraction": 1.0, + "pearson": -0.000693, + "blurSigmaPx": 6.0, + "blurSigmaUM": 667.0, + "pearsonMatchedBlur": 0.001611, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.14269, + "dice": 0.249743, + "precision": 0.166623, + "recall": 0.498344, + "refPositiveFraction": 0.166716, + "predPositiveFraction": 0.498623, + "ap": 0.166748, + "bestF1": { + "f1": 0.285787, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.166978, + "recall": 0.019655 + }, + { + "t": 0.96, + "precision": 0.168676, + "recall": 0.039749 + }, + { + "t": 0.94, + "precision": 0.166898, + "recall": 0.058619 + }, + { + "t": 0.92, + "precision": 0.167938, + "recall": 0.078777 + }, + { + "t": 0.9, + "precision": 0.168599, + "recall": 0.099091 + }, + { + "t": 0.88, + "precision": 0.167741, + "recall": 0.11851 + }, + { + "t": 0.86, + "precision": 0.167714, + "recall": 0.13826 + }, + { + "t": 0.84, + "precision": 0.167508, + "recall": 0.157915 + }, + { + "t": 0.82, + "precision": 0.167551, + "recall": 0.177946 + }, + { + "t": 0.8, + "precision": 0.166638, + "recall": 0.200945 + }, + { + "t": 0.78, + "precision": 0.166469, + "recall": 0.220145 + }, + { + "t": 0.76, + "precision": 0.166266, + "recall": 0.239753 + }, + { + "t": 0.74, + "precision": 0.166233, + "recall": 0.259486 + }, + { + "t": 0.72, + "precision": 0.166004, + "recall": 0.27845 + }, + { + "t": 0.7, + "precision": 0.165601, + "recall": 0.297163 + }, + { + "t": 0.68, + "precision": 0.1658, + "recall": 0.316787 + }, + { + "t": 0.66, + "precision": 0.166054, + "recall": 0.336771 + }, + { + "t": 0.64, + "precision": 0.166197, + "recall": 0.356364 + }, + { + "t": 0.62, + "precision": 0.166306, + "recall": 0.375924 + }, + { + "t": 0.6, + "precision": 0.16632, + "recall": 0.399739 + }, + { + "t": 0.58, + "precision": 0.166612, + "recall": 0.420273 + }, + { + "t": 0.56, + "precision": 0.166595, + "recall": 0.439724 + }, + { + "t": 0.54, + "precision": 0.166896, + "recall": 0.460054 + }, + { + "t": 0.52, + "precision": 0.166719, + "recall": 0.479003 + }, + { + "t": 0.5, + "precision": 0.166623, + "recall": 0.498344 + }, + { + "t": 0.48, + "precision": 0.166775, + "recall": 0.518423 + }, + { + "t": 0.46, + "precision": 0.166761, + "recall": 0.537873 + }, + { + "t": 0.44, + "precision": 0.166442, + "recall": 0.556853 + }, + { + "t": 0.42, + "precision": 0.166275, + "recall": 0.575786 + }, + { + "t": 0.4, + "precision": 0.166291, + "recall": 0.599334 + }, + { + "t": 0.38, + "precision": 0.1665, + "recall": 0.619727 + }, + { + "t": 0.36, + "precision": 0.16656, + "recall": 0.639555 + }, + { + "t": 0.34, + "precision": 0.166597, + "recall": 0.659194 + }, + { + "t": 0.32, + "precision": 0.166639, + "recall": 0.67888 + }, + { + "t": 0.3, + "precision": 0.166597, + "recall": 0.698363 + }, + { + "t": 0.28, + "precision": 0.166399, + "recall": 0.717091 + }, + { + "t": 0.26, + "precision": 0.166467, + "recall": 0.736966 + }, + { + "t": 0.24, + "precision": 0.166522, + "recall": 0.756778 + }, + { + "t": 0.22, + "precision": 0.166491, + "recall": 0.776135 + }, + { + "t": 0.2, + "precision": 0.166728, + "recall": 0.800421 + }, + { + "t": 0.18, + "precision": 0.166886, + "recall": 0.820578 + }, + { + "t": 0.16, + "precision": 0.167061, + "recall": 0.841002 + }, + { + "t": 0.14, + "precision": 0.167054, + "recall": 0.860375 + }, + { + "t": 0.12, + "precision": 0.1668, + "recall": 0.878256 + }, + { + "t": 0.1, + "precision": 0.166781, + "recall": 0.898036 + }, + { + "t": 0.08, + "precision": 0.166737, + "recall": 0.917503 + }, + { + "t": 0.06, + "precision": 0.166662, + "recall": 0.936278 + }, + { + "t": 0.04, + "precision": 0.166885, + "recall": 0.957472 + }, + { + "t": 0.02, + "precision": 0.16676, + "recall": 0.976295 + }, + { + "t": 0.0, + "precision": 0.166716, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w033-ct-baseline.json b/docs/examples/bench/w033-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..fd254b7460a5edb2f7b4aefd7cc25d89a57834e7 --- /dev/null +++ b/docs/examples/bench/w033-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:03Z", + "scroll": "PHerc0139", + "segment": "w033", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w033_ink.png", + "sha256": "bc1efedf282ae996e5fc38b6405f67bc3fa45700a25788f6507a0463a0a1761f", + "width": 640, + "height": 592, + "pxPerMM": 8.9013, + "alignTargetFile": "data/bench/w033_ink.png", + "alignTargetSha256": "bc1efedf282ae996e5fc38b6405f67bc3fa45700a25788f6507a0463a0a1761f" + }, + "prediction": { + "name": "w033_ct.png", + "sha256": "3ccb04488d2c8f5e8eea00f6c690508d5eb49c3c75ba8b1dfada08fb6038139c", + "format": "png", + "width": 640, + "height": 592, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.283667, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 378880, + "validFraction": 1.0, + "pearson": 0.283667, + "blurSigmaPx": 6.0, + "blurSigmaUM": 674.1, + "pearsonMatchedBlur": 0.407929, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.195534, + "dice": 0.327107, + "precision": 0.23632, + "recall": 0.531167, + "refPositiveFraction": 0.182704, + "predPositiveFraction": 0.410658, + "ap": 0.240607, + "bestF1": { + "f1": 0.350826, + "threshold": 0.16 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.333333, + "recall": 0.000925 + }, + { + "t": 0.98, + "precision": 0.334615, + "recall": 0.002514 + }, + { + "t": 0.96, + "precision": 0.341117, + "recall": 0.004854 + }, + { + "t": 0.94, + "precision": 0.335315, + "recall": 0.008148 + }, + { + "t": 0.92, + "precision": 0.330579, + "recall": 0.012713 + }, + { + "t": 0.9, + "precision": 0.321128, + "recall": 0.018751 + }, + { + "t": 0.88, + "precision": 0.311626, + "recall": 0.026407 + }, + { + "t": 0.86, + "precision": 0.297706, + "recall": 0.035248 + }, + { + "t": 0.84, + "precision": 0.289107, + "recall": 0.046776 + }, + { + "t": 0.82, + "precision": 0.283392, + "recall": 0.060688 + }, + { + "t": 0.8, + "precision": 0.276466, + "recall": 0.080508 + }, + { + "t": 0.78, + "precision": 0.271008, + "recall": 0.099981 + }, + { + "t": 0.76, + "precision": 0.267043, + "recall": 0.121491 + }, + { + "t": 0.74, + "precision": 0.26239, + "recall": 0.14501 + }, + { + "t": 0.72, + "precision": 0.259431, + "recall": 0.170781 + }, + { + "t": 0.7, + "precision": 0.257272, + "recall": 0.199457 + }, + { + "t": 0.68, + "precision": 0.253799, + "recall": 0.227988 + }, + { + "t": 0.66, + "precision": 0.251526, + "recall": 0.258339 + }, + { + "t": 0.64, + "precision": 0.2488, + "recall": 0.28973 + }, + { + "t": 0.62, + "precision": 0.246784, + "recall": 0.322595 + }, + { + "t": 0.6, + "precision": 0.244189, + "recall": 0.362871 + }, + { + "t": 0.58, + "precision": 0.242346, + "recall": 0.396588 + }, + { + "t": 0.56, + "precision": 0.241004, + "recall": 0.431056 + }, + { + "t": 0.54, + "precision": 0.239232, + "recall": 0.464643 + }, + { + "t": 0.52, + "precision": 0.237827, + "recall": 0.498144 + }, + { + "t": 0.5, + "precision": 0.23632, + "recall": 0.531167 + }, + { + "t": 0.48, + "precision": 0.234767, + "recall": 0.562905 + }, + { + "t": 0.46, + "precision": 0.233218, + "recall": 0.593589 + }, + { + "t": 0.44, + "precision": 0.231592, + "recall": 0.623174 + }, + { + "t": 0.42, + "precision": 0.230572, + "recall": 0.652919 + }, + { + "t": 0.4, + "precision": 0.229171, + "recall": 0.686159 + }, + { + "t": 0.38, + "precision": 0.22791, + "recall": 0.711917 + }, + { + "t": 0.36, + "precision": 0.22664, + "recall": 0.736663 + }, + { + "t": 0.34, + "precision": 0.225285, + "recall": 0.75852 + }, + { + "t": 0.32, + "precision": 0.224028, + "recall": 0.77964 + }, + { + "t": 0.3, + "precision": 0.222939, + "recall": 0.800384 + }, + { + "t": 0.28, + "precision": 0.221877, + "recall": 0.818991 + }, + { + "t": 0.26, + "precision": 0.221075, + "recall": 0.837019 + }, + { + "t": 0.24, + "precision": 0.22019, + "recall": 0.853011 + }, + { + "t": 0.22, + "precision": 0.219357, + "recall": 0.867891 + }, + { + "t": 0.2, + "precision": 0.218652, + "recall": 0.885428 + }, + { + "t": 0.18, + "precision": 0.217881, + "recall": 0.898386 + }, + { + "t": 0.16, + "precision": 0.217282, + "recall": 0.910319 + }, + { + "t": 0.14, + "precision": 0.216576, + "recall": 0.921096 + }, + { + "t": 0.12, + "precision": 0.215916, + "recall": 0.931237 + }, + { + "t": 0.1, + "precision": 0.215402, + "recall": 0.941826 + }, + { + "t": 0.08, + "precision": 0.21485, + "recall": 0.95175 + }, + { + "t": 0.06, + "precision": 0.214087, + "recall": 0.96153 + }, + { + "t": 0.04, + "precision": 0.213484, + "recall": 0.971093 + }, + { + "t": 0.02, + "precision": 0.212555, + "recall": 0.98031 + }, + { + "t": 0.0, + "precision": 0.182704, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w033-ink9um.json b/docs/examples/bench/w033-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..2129ed75c89ce4aa84092d270c5d532b173f02d4 --- /dev/null +++ b/docs/examples/bench/w033-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:02Z", + "scroll": "PHerc0139", + "segment": "w033", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w033_ink.png", + "sha256": "bc1efedf282ae996e5fc38b6405f67bc3fa45700a25788f6507a0463a0a1761f", + "width": 640, + "height": 592, + "pxPerMM": 8.9013, + "alignTargetFile": "data/bench/w033_ink.png", + "alignTargetSha256": "bc1efedf282ae996e5fc38b6405f67bc3fa45700a25788f6507a0463a0a1761f" + }, + "prediction": { + "name": "w033_gpu.png", + "sha256": "da90168078510f68b84b28617e270f0a33a85d374b9b0db0c565de2365511b1c", + "format": "png", + "width": 640, + "height": 592, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.563, + "ty": 0.5, + "sx": 1.0002, + "sy": 1.0017, + "ncc": 0.506509, + "coarseNcc": 0.673459, + "evals": 40301, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 378288, + "validFraction": 0.998438, + "pearson": 0.506509, + "blurSigmaPx": 2.0, + "blurSigmaUM": 224.7, + "pearsonMatchedBlur": 0.549785, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.303942, + "dice": 0.46619, + "precision": 0.523427, + "recall": 0.420236, + "refPositiveFraction": 0.18299, + "predPositiveFraction": 0.146915, + "ap": 0.491435, + "bestF1": { + "f1": 0.492677, + "threshold": 0.26 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.934694, + "recall": 0.003308 + }, + { + "t": 0.98, + "precision": 0.811571, + "recall": 0.087543 + }, + { + "t": 0.96, + "precision": 0.785243, + "recall": 0.111769 + }, + { + "t": 0.94, + "precision": 0.76647, + "recall": 0.13143 + }, + { + "t": 0.92, + "precision": 0.751489, + "recall": 0.149488 + }, + { + "t": 0.9, + "precision": 0.736937, + "recall": 0.16483 + }, + { + "t": 0.88, + "precision": 0.7228, + "recall": 0.179565 + }, + { + "t": 0.86, + "precision": 0.708561, + "recall": 0.194646 + }, + { + "t": 0.84, + "precision": 0.695914, + "recall": 0.208413 + }, + { + "t": 0.82, + "precision": 0.683521, + "recall": 0.221458 + }, + { + "t": 0.8, + "precision": 0.672872, + "recall": 0.234951 + }, + { + "t": 0.78, + "precision": 0.662255, + "recall": 0.247967 + }, + { + "t": 0.76, + "precision": 0.650553, + "recall": 0.260116 + }, + { + "t": 0.74, + "precision": 0.639758, + "recall": 0.271788 + }, + { + "t": 0.72, + "precision": 0.627998, + "recall": 0.283721 + }, + { + "t": 0.7, + "precision": 0.617825, + "recall": 0.295913 + }, + { + "t": 0.68, + "precision": 0.607726, + "recall": 0.307485 + }, + { + "t": 0.66, + "precision": 0.597542, + "recall": 0.320226 + }, + { + "t": 0.64, + "precision": 0.586767, + "recall": 0.332202 + }, + { + "t": 0.62, + "precision": 0.577158, + "recall": 0.344495 + }, + { + "t": 0.6, + "precision": 0.567942, + "recall": 0.356659 + }, + { + "t": 0.58, + "precision": 0.558567, + "recall": 0.368548 + }, + { + "t": 0.56, + "precision": 0.549984, + "recall": 0.381564 + }, + { + "t": 0.54, + "precision": 0.541113, + "recall": 0.393858 + }, + { + "t": 0.52, + "precision": 0.532656, + "recall": 0.407524 + }, + { + "t": 0.5, + "precision": 0.523427, + "recall": 0.420236 + }, + { + "t": 0.48, + "precision": 0.514184, + "recall": 0.432833 + }, + { + "t": 0.46, + "precision": 0.505973, + "recall": 0.446629 + }, + { + "t": 0.44, + "precision": 0.497317, + "recall": 0.460483 + }, + { + "t": 0.42, + "precision": 0.488787, + "recall": 0.474828 + }, + { + "t": 0.4, + "precision": 0.480341, + "recall": 0.489563 + }, + { + "t": 0.38, + "precision": 0.471093, + "recall": 0.503937 + }, + { + "t": 0.36, + "precision": 0.462465, + "recall": 0.519105 + }, + { + "t": 0.34, + "precision": 0.452624, + "recall": 0.533912 + }, + { + "t": 0.32, + "precision": 0.44359, + "recall": 0.550164 + }, + { + "t": 0.3, + "precision": 0.434383, + "recall": 0.566531 + }, + { + "t": 0.28, + "precision": 0.425499, + "recall": 0.584098 + }, + { + "t": 0.26, + "precision": 0.416461, + "recall": 0.603037 + }, + { + "t": 0.24, + "precision": 0.405721, + "recall": 0.621239 + }, + { + "t": 0.22, + "precision": 0.395611, + "recall": 0.640596 + }, + { + "t": 0.2, + "precision": 0.385203, + "recall": 0.661572 + }, + { + "t": 0.18, + "precision": 0.374176, + "recall": 0.682692 + }, + { + "t": 0.16, + "precision": 0.362204, + "recall": 0.70387 + }, + { + "t": 0.14, + "precision": 0.35017, + "recall": 0.7272 + }, + { + "t": 0.12, + "precision": 0.337623, + "recall": 0.751499 + }, + { + "t": 0.1, + "precision": 0.324307, + "recall": 0.778759 + }, + { + "t": 0.08, + "precision": 0.309751, + "recall": 0.808647 + }, + { + "t": 0.06, + "precision": 0.293367, + "recall": 0.841628 + }, + { + "t": 0.04, + "precision": 0.274276, + "recall": 0.879968 + }, + { + "t": 0.02, + "precision": 0.249794, + "recall": 0.926224 + }, + { + "t": 0.0, + "precision": 0.18299, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w033-noise-baseline.json b/docs/examples/bench/w033-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..9f64f9e3283c9c4c9d0635b9a1daffeecd6bc402 --- /dev/null +++ b/docs/examples/bench/w033-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:05Z", + "scroll": "PHerc0139", + "segment": "w033", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w033_ink.png", + "sha256": "bc1efedf282ae996e5fc38b6405f67bc3fa45700a25788f6507a0463a0a1761f", + "width": 640, + "height": 592, + "pxPerMM": 8.9013, + "alignTargetFile": "data/bench/w033_ink.png", + "alignTargetSha256": "bc1efedf282ae996e5fc38b6405f67bc3fa45700a25788f6507a0463a0a1761f" + }, + "prediction": { + "name": "noise-seed33.png", + "sha256": "6860a36278cd746657b082aaecdc500cbc371b75b2dc8c7ab80eae89c2c1f71b", + "format": "png", + "width": 640, + "height": 592, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(33).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.001135, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 378880, + "validFraction": 1.0, + "pearson": 0.001135, + "blurSigmaPx": 1.5, + "blurSigmaUM": 168.5, + "pearsonMatchedBlur": 0.001473, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.1546, + "dice": 0.267798, + "precision": 0.18315, + "recall": 0.497927, + "refPositiveFraction": 0.182704, + "predPositiveFraction": 0.496714, + "ap": 0.182948, + "bestF1": { + "f1": 0.30896, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.183078, + "recall": 0.019661 + }, + { + "t": 0.96, + "precision": 0.185531, + "recall": 0.03964 + }, + { + "t": 0.94, + "precision": 0.183601, + "recall": 0.058969 + }, + { + "t": 0.92, + "precision": 0.181803, + "recall": 0.077994 + }, + { + "t": 0.9, + "precision": 0.181015, + "recall": 0.096774 + }, + { + "t": 0.88, + "precision": 0.182647, + "recall": 0.117143 + }, + { + "t": 0.86, + "precision": 0.182291, + "recall": 0.136212 + }, + { + "t": 0.84, + "precision": 0.181474, + "recall": 0.155064 + }, + { + "t": 0.82, + "precision": 0.181825, + "recall": 0.174494 + }, + { + "t": 0.8, + "precision": 0.182266, + "recall": 0.198388 + }, + { + "t": 0.78, + "precision": 0.182568, + "recall": 0.217962 + }, + { + "t": 0.76, + "precision": 0.182684, + "recall": 0.23797 + }, + { + "t": 0.74, + "precision": 0.183287, + "recall": 0.258556 + }, + { + "t": 0.72, + "precision": 0.183152, + "recall": 0.277899 + }, + { + "t": 0.7, + "precision": 0.182803, + "recall": 0.296737 + }, + { + "t": 0.68, + "precision": 0.18264, + "recall": 0.316152 + }, + { + "t": 0.66, + "precision": 0.183071, + "recall": 0.33681 + }, + { + "t": 0.64, + "precision": 0.183382, + "recall": 0.356919 + }, + { + "t": 0.62, + "precision": 0.18347, + "recall": 0.376985 + }, + { + "t": 0.6, + "precision": 0.183691, + "recall": 0.400647 + }, + { + "t": 0.58, + "precision": 0.183495, + "recall": 0.419629 + }, + { + "t": 0.56, + "precision": 0.183679, + "recall": 0.440056 + }, + { + "t": 0.54, + "precision": 0.1831, + "recall": 0.458764 + }, + { + "t": 0.52, + "precision": 0.183045, + "recall": 0.477977 + }, + { + "t": 0.5, + "precision": 0.18315, + "recall": 0.497927 + }, + { + "t": 0.48, + "precision": 0.182985, + "recall": 0.517227 + }, + { + "t": 0.46, + "precision": 0.183167, + "recall": 0.537293 + }, + { + "t": 0.44, + "precision": 0.183105, + "recall": 0.556694 + }, + { + "t": 0.42, + "precision": 0.183236, + "recall": 0.576745 + }, + { + "t": 0.4, + "precision": 0.183273, + "recall": 0.600725 + }, + { + "t": 0.38, + "precision": 0.183165, + "recall": 0.620198 + }, + { + "t": 0.36, + "precision": 0.183153, + "recall": 0.639527 + }, + { + "t": 0.34, + "precision": 0.182944, + "recall": 0.658437 + }, + { + "t": 0.32, + "precision": 0.182785, + "recall": 0.67752 + }, + { + "t": 0.3, + "precision": 0.182899, + "recall": 0.697788 + }, + { + "t": 0.28, + "precision": 0.182963, + "recall": 0.717623 + }, + { + "t": 0.26, + "precision": 0.182922, + "recall": 0.737399 + }, + { + "t": 0.24, + "precision": 0.182884, + "recall": 0.757263 + }, + { + "t": 0.22, + "precision": 0.18305, + "recall": 0.777632 + }, + { + "t": 0.2, + "precision": 0.183159, + "recall": 0.801627 + }, + { + "t": 0.18, + "precision": 0.183088, + "recall": 0.820551 + }, + { + "t": 0.16, + "precision": 0.183051, + "recall": 0.84027 + }, + { + "t": 0.14, + "precision": 0.182864, + "recall": 0.859396 + }, + { + "t": 0.12, + "precision": 0.182778, + "recall": 0.878855 + }, + { + "t": 0.1, + "precision": 0.18281, + "recall": 0.898487 + }, + { + "t": 0.08, + "precision": 0.182778, + "recall": 0.917845 + }, + { + "t": 0.06, + "precision": 0.182788, + "recall": 0.937492 + }, + { + "t": 0.04, + "precision": 0.182899, + "recall": 0.957962 + }, + { + "t": 0.02, + "precision": 0.182813, + "recall": 0.977045 + }, + { + "t": 0.0, + "precision": 0.182704, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w034-ct-baseline.json b/docs/examples/bench/w034-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..4f8b4f152fb46561cf050066640fb28e69a0f0de --- /dev/null +++ b/docs/examples/bench/w034-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:15Z", + "scroll": "PHerc0139", + "segment": "w034", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w034_ink.png", + "sha256": "2a61bfe01f4967d373162c0e284aabd786d9a95b91a4f8cc93bfd17fb74056fd", + "width": 640, + "height": 584, + "pxPerMM": 8.8557, + "alignTargetFile": "data/bench/w034_ink.png", + "alignTargetSha256": "2a61bfe01f4967d373162c0e284aabd786d9a95b91a4f8cc93bfd17fb74056fd" + }, + "prediction": { + "name": "w034_ct.png", + "sha256": "0b0726cbc9d4fbf6a3851b580b4711606a59c7d221efbefebfe0af350cf74a5d", + "format": "png", + "width": 640, + "height": 584, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.249872, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 373760, + "validFraction": 1.0, + "pearson": 0.249872, + "blurSigmaPx": 6.0, + "blurSigmaUM": 677.5, + "pearsonMatchedBlur": 0.389858, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.116884, + "dice": 0.209304, + "precision": 0.131412, + "recall": 0.513912, + "refPositiveFraction": 0.104332, + "predPositiveFraction": 0.408008, + "ap": 0.132244, + "bestF1": { + "f1": 0.217903, + "threshold": 0.3 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.16129, + "recall": 0.001026 + }, + { + "t": 0.98, + "precision": 0.159935, + "recall": 0.002539 + }, + { + "t": 0.96, + "precision": 0.154594, + "recall": 0.004488 + }, + { + "t": 0.94, + "precision": 0.153558, + "recall": 0.00736 + }, + { + "t": 0.92, + "precision": 0.147904, + "recall": 0.01095 + }, + { + "t": 0.9, + "precision": 0.143329, + "recall": 0.015592 + }, + { + "t": 0.88, + "precision": 0.146208, + "recall": 0.022695 + }, + { + "t": 0.86, + "precision": 0.144049, + "recall": 0.031004 + }, + { + "t": 0.84, + "precision": 0.143821, + "recall": 0.04098 + }, + { + "t": 0.82, + "precision": 0.145795, + "recall": 0.054366 + }, + { + "t": 0.8, + "precision": 0.144449, + "recall": 0.072137 + }, + { + "t": 0.78, + "precision": 0.144598, + "recall": 0.090063 + }, + { + "t": 0.76, + "precision": 0.144437, + "recall": 0.11086 + }, + { + "t": 0.74, + "precision": 0.142284, + "recall": 0.132761 + }, + { + "t": 0.72, + "precision": 0.141238, + "recall": 0.157251 + }, + { + "t": 0.7, + "precision": 0.140527, + "recall": 0.184229 + }, + { + "t": 0.68, + "precision": 0.140753, + "recall": 0.214156 + }, + { + "t": 0.66, + "precision": 0.139792, + "recall": 0.243749 + }, + { + "t": 0.64, + "precision": 0.137469, + "recall": 0.272548 + }, + { + "t": 0.62, + "precision": 0.136178, + "recall": 0.303706 + }, + { + "t": 0.6, + "precision": 0.135287, + "recall": 0.343275 + }, + { + "t": 0.58, + "precision": 0.134255, + "recall": 0.376612 + }, + { + "t": 0.56, + "precision": 0.132951, + "recall": 0.409309 + }, + { + "t": 0.54, + "precision": 0.132657, + "recall": 0.444852 + }, + { + "t": 0.52, + "precision": 0.131598, + "recall": 0.47819 + }, + { + "t": 0.5, + "precision": 0.131412, + "recall": 0.513912 + }, + { + "t": 0.48, + "precision": 0.130797, + "recall": 0.547814 + }, + { + "t": 0.46, + "precision": 0.130436, + "recall": 0.581869 + }, + { + "t": 0.44, + "precision": 0.129948, + "recall": 0.614361 + }, + { + "t": 0.42, + "precision": 0.129312, + "recall": 0.645262 + }, + { + "t": 0.4, + "precision": 0.128835, + "recall": 0.682446 + }, + { + "t": 0.38, + "precision": 0.128263, + "recall": 0.710501 + }, + { + "t": 0.36, + "precision": 0.127567, + "recall": 0.735838 + }, + { + "t": 0.34, + "precision": 0.126961, + "recall": 0.759892 + }, + { + "t": 0.32, + "precision": 0.126405, + "recall": 0.7831 + }, + { + "t": 0.3, + "precision": 0.126008, + "recall": 0.804898 + }, + { + "t": 0.28, + "precision": 0.125383, + "recall": 0.823618 + }, + { + "t": 0.26, + "precision": 0.12502, + "recall": 0.842313 + }, + { + "t": 0.24, + "precision": 0.124622, + "recall": 0.859213 + }, + { + "t": 0.22, + "precision": 0.12428, + "recall": 0.874522 + }, + { + "t": 0.2, + "precision": 0.123948, + "recall": 0.892037 + }, + { + "t": 0.18, + "precision": 0.123694, + "recall": 0.904937 + }, + { + "t": 0.16, + "precision": 0.123499, + "recall": 0.917579 + }, + { + "t": 0.14, + "precision": 0.123163, + "recall": 0.927632 + }, + { + "t": 0.12, + "precision": 0.122894, + "recall": 0.937607 + }, + { + "t": 0.1, + "precision": 0.122567, + "recall": 0.946762 + }, + { + "t": 0.08, + "precision": 0.122257, + "recall": 0.955994 + }, + { + "t": 0.06, + "precision": 0.121928, + "recall": 0.964457 + }, + { + "t": 0.04, + "precision": 0.121627, + "recall": 0.973279 + }, + { + "t": 0.02, + "precision": 0.121251, + "recall": 0.981844 + }, + { + "t": 0.0, + "precision": 0.104332, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w034-ink9um.json b/docs/examples/bench/w034-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..3fbae97df4af70d00d419d72065be4313e46f560 --- /dev/null +++ b/docs/examples/bench/w034-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:14Z", + "scroll": "PHerc0139", + "segment": "w034", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w034_ink.png", + "sha256": "2a61bfe01f4967d373162c0e284aabd786d9a95b91a4f8cc93bfd17fb74056fd", + "width": 640, + "height": 584, + "pxPerMM": 8.8557, + "alignTargetFile": "data/bench/w034_ink.png", + "alignTargetSha256": "2a61bfe01f4967d373162c0e284aabd786d9a95b91a4f8cc93bfd17fb74056fd" + }, + "prediction": { + "name": "w034_gpu.png", + "sha256": "015c6a53edb0e9c97c6bacd0f41a38b3ce42af3e1031a7debbdece3771e83dd8", + "format": "png", + "width": 640, + "height": 584, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.563, + "ty": 0.563, + "sx": 1.00014, + "sy": 1.00046, + "ncc": 0.49161, + "coarseNcc": 0.60212, + "evals": 38340, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 372537, + "validFraction": 0.996728, + "pearson": 0.49161, + "blurSigmaPx": 2.0, + "blurSigmaUM": 225.8, + "pearsonMatchedBlur": 0.528335, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.272219, + "dice": 0.427943, + "precision": 0.431718, + "recall": 0.424234, + "refPositiveFraction": 0.104674, + "predPositiveFraction": 0.10286, + "ap": 0.4076, + "bestF1": { + "f1": 0.430574, + "threshold": 0.44 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.92233, + "recall": 0.002436 + }, + { + "t": 0.98, + "precision": 0.784598, + "recall": 0.09014 + }, + { + "t": 0.96, + "precision": 0.743957, + "recall": 0.114451 + }, + { + "t": 0.94, + "precision": 0.722645, + "recall": 0.133966 + }, + { + "t": 0.92, + "precision": 0.701308, + "recall": 0.149865 + }, + { + "t": 0.9, + "precision": 0.68314, + "recall": 0.166252 + }, + { + "t": 0.88, + "precision": 0.664882, + "recall": 0.181536 + }, + { + "t": 0.86, + "precision": 0.645399, + "recall": 0.195333 + }, + { + "t": 0.84, + "precision": 0.629647, + "recall": 0.209796 + }, + { + "t": 0.82, + "precision": 0.614731, + "recall": 0.222798 + }, + { + "t": 0.8, + "precision": 0.600667, + "recall": 0.235569 + }, + { + "t": 0.78, + "precision": 0.586607, + "recall": 0.248904 + }, + { + "t": 0.76, + "precision": 0.573765, + "recall": 0.261803 + }, + { + "t": 0.74, + "precision": 0.562428, + "recall": 0.275394 + }, + { + "t": 0.72, + "precision": 0.550311, + "recall": 0.287934 + }, + { + "t": 0.7, + "precision": 0.537729, + "recall": 0.299705 + }, + { + "t": 0.68, + "precision": 0.525934, + "recall": 0.31204 + }, + { + "t": 0.66, + "precision": 0.516023, + "recall": 0.32499 + }, + { + "t": 0.64, + "precision": 0.503147, + "recall": 0.336197 + }, + { + "t": 0.62, + "precision": 0.493005, + "recall": 0.349712 + }, + { + "t": 0.6, + "precision": 0.483106, + "recall": 0.362995 + }, + { + "t": 0.58, + "precision": 0.472315, + "recall": 0.374715 + }, + { + "t": 0.56, + "precision": 0.46156, + "recall": 0.387511 + }, + { + "t": 0.54, + "precision": 0.451393, + "recall": 0.399615 + }, + { + "t": 0.52, + "precision": 0.441548, + "recall": 0.411745 + }, + { + "t": 0.5, + "precision": 0.431718, + "recall": 0.424234 + }, + { + "t": 0.48, + "precision": 0.422417, + "recall": 0.436748 + }, + { + "t": 0.46, + "precision": 0.412899, + "recall": 0.449186 + }, + { + "t": 0.44, + "precision": 0.403224, + "recall": 0.461905 + }, + { + "t": 0.42, + "precision": 0.392144, + "recall": 0.47365 + }, + { + "t": 0.4, + "precision": 0.38273, + "recall": 0.488652 + }, + { + "t": 0.38, + "precision": 0.373047, + "recall": 0.50209 + }, + { + "t": 0.36, + "precision": 0.363433, + "recall": 0.515964 + }, + { + "t": 0.34, + "precision": 0.353155, + "recall": 0.529991 + }, + { + "t": 0.32, + "precision": 0.343257, + "recall": 0.54407 + }, + { + "t": 0.3, + "precision": 0.333619, + "recall": 0.559969 + }, + { + "t": 0.28, + "precision": 0.323713, + "recall": 0.575561 + }, + { + "t": 0.26, + "precision": 0.313631, + "recall": 0.592871 + }, + { + "t": 0.24, + "precision": 0.303079, + "recall": 0.610386 + }, + { + "t": 0.22, + "precision": 0.293042, + "recall": 0.630235 + }, + { + "t": 0.2, + "precision": 0.282153, + "recall": 0.650135 + }, + { + "t": 0.18, + "precision": 0.271075, + "recall": 0.67024 + }, + { + "t": 0.16, + "precision": 0.259886, + "recall": 0.691986 + }, + { + "t": 0.14, + "precision": 0.247616, + "recall": 0.713963 + }, + { + "t": 0.12, + "precision": 0.235305, + "recall": 0.73812 + }, + { + "t": 0.1, + "precision": 0.222481, + "recall": 0.765714 + }, + { + "t": 0.08, + "precision": 0.208656, + "recall": 0.794794 + }, + { + "t": 0.06, + "precision": 0.193955, + "recall": 0.827645 + }, + { + "t": 0.04, + "precision": 0.177068, + "recall": 0.866983 + }, + { + "t": 0.02, + "precision": 0.156467, + "recall": 0.916707 + }, + { + "t": 0.0, + "precision": 0.104674, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w034-noise-baseline.json b/docs/examples/bench/w034-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..22998d3b5bcb1ec431dfa09a234e6f2262308824 --- /dev/null +++ b/docs/examples/bench/w034-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:17Z", + "scroll": "PHerc0139", + "segment": "w034", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w034_ink.png", + "sha256": "2a61bfe01f4967d373162c0e284aabd786d9a95b91a4f8cc93bfd17fb74056fd", + "width": 640, + "height": 584, + "pxPerMM": 8.8557, + "alignTargetFile": "data/bench/w034_ink.png", + "alignTargetSha256": "2a61bfe01f4967d373162c0e284aabd786d9a95b91a4f8cc93bfd17fb74056fd" + }, + "prediction": { + "name": "noise-seed34.png", + "sha256": "0a0306543dab38319b0458ca2ac03ccfaa197cec405a0169584f163fc813580c", + "format": "png", + "width": 640, + "height": 584, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(34).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.00104, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 373760, + "validFraction": 1.0, + "pearson": 0.00104, + "blurSigmaPx": 6.0, + "blurSigmaUM": 677.5, + "pearsonMatchedBlur": 0.001237, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.094518, + "dice": 0.172712, + "precision": 0.104484, + "recall": 0.49773, + "refPositiveFraction": 0.104332, + "predPositiveFraction": 0.497006, + "ap": 0.104611, + "bestF1": { + "f1": 0.18895, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.108142, + "recall": 0.020028 + }, + { + "t": 0.96, + "precision": 0.106548, + "recall": 0.040056 + }, + { + "t": 0.94, + "precision": 0.10574, + "recall": 0.05952 + }, + { + "t": 0.92, + "precision": 0.10617, + "recall": 0.079831 + }, + { + "t": 0.9, + "precision": 0.106033, + "recall": 0.099474 + }, + { + "t": 0.88, + "precision": 0.105027, + "recall": 0.118195 + }, + { + "t": 0.86, + "precision": 0.104695, + "recall": 0.13771 + }, + { + "t": 0.84, + "precision": 0.10462, + "recall": 0.157071 + }, + { + "t": 0.82, + "precision": 0.104739, + "recall": 0.176843 + }, + { + "t": 0.8, + "precision": 0.104275, + "recall": 0.199718 + }, + { + "t": 0.78, + "precision": 0.104344, + "recall": 0.219515 + }, + { + "t": 0.76, + "precision": 0.104546, + "recall": 0.239672 + }, + { + "t": 0.74, + "precision": 0.10434, + "recall": 0.258751 + }, + { + "t": 0.72, + "precision": 0.104405, + "recall": 0.278395 + }, + { + "t": 0.7, + "precision": 0.104288, + "recall": 0.2975 + }, + { + "t": 0.68, + "precision": 0.104231, + "recall": 0.317118 + }, + { + "t": 0.66, + "precision": 0.104163, + "recall": 0.336582 + }, + { + "t": 0.64, + "precision": 0.104045, + "recall": 0.355584 + }, + { + "t": 0.62, + "precision": 0.104209, + "recall": 0.375715 + }, + { + "t": 0.6, + "precision": 0.104228, + "recall": 0.399051 + }, + { + "t": 0.58, + "precision": 0.104328, + "recall": 0.419208 + }, + { + "t": 0.56, + "precision": 0.104302, + "recall": 0.438364 + }, + { + "t": 0.54, + "precision": 0.104448, + "recall": 0.458572 + }, + { + "t": 0.52, + "precision": 0.104592, + "recall": 0.4786 + }, + { + "t": 0.5, + "precision": 0.104484, + "recall": 0.49773 + }, + { + "t": 0.48, + "precision": 0.104373, + "recall": 0.51681 + }, + { + "t": 0.46, + "precision": 0.104175, + "recall": 0.535299 + }, + { + "t": 0.44, + "precision": 0.104209, + "recall": 0.554969 + }, + { + "t": 0.42, + "precision": 0.104247, + "recall": 0.574689 + }, + { + "t": 0.4, + "precision": 0.104249, + "recall": 0.598333 + }, + { + "t": 0.38, + "precision": 0.104226, + "recall": 0.617797 + }, + { + "t": 0.36, + "precision": 0.104399, + "recall": 0.638543 + }, + { + "t": 0.34, + "precision": 0.104423, + "recall": 0.65852 + }, + { + "t": 0.32, + "precision": 0.104466, + "recall": 0.678241 + }, + { + "t": 0.3, + "precision": 0.10448, + "recall": 0.698192 + }, + { + "t": 0.28, + "precision": 0.104437, + "recall": 0.718041 + }, + { + "t": 0.26, + "precision": 0.10443, + "recall": 0.737633 + }, + { + "t": 0.24, + "precision": 0.104533, + "recall": 0.757636 + }, + { + "t": 0.22, + "precision": 0.104518, + "recall": 0.777689 + }, + { + "t": 0.2, + "precision": 0.10459, + "recall": 0.802 + }, + { + "t": 0.18, + "precision": 0.104532, + "recall": 0.821541 + }, + { + "t": 0.16, + "precision": 0.104435, + "recall": 0.840467 + }, + { + "t": 0.14, + "precision": 0.104436, + "recall": 0.860059 + }, + { + "t": 0.12, + "precision": 0.10445, + "recall": 0.879703 + }, + { + "t": 0.1, + "precision": 0.10444, + "recall": 0.899526 + }, + { + "t": 0.08, + "precision": 0.104327, + "recall": 0.918118 + }, + { + "t": 0.06, + "precision": 0.104249, + "recall": 0.937146 + }, + { + "t": 0.04, + "precision": 0.10432, + "recall": 0.957379 + }, + { + "t": 0.02, + "precision": 0.104401, + "recall": 0.977202 + }, + { + "t": 0.0, + "precision": 0.104332, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w035-ct-baseline.json b/docs/examples/bench/w035-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..0b71d5e399181aa44576ee6fe33ba47ea5abf640 --- /dev/null +++ b/docs/examples/bench/w035-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:30Z", + "scroll": "PHerc0139", + "segment": "w035", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w035_ink.png", + "sha256": "03bfea34b06f8db6464e9a0b4e024653eb75d5e4872d7f91484a80945dc1e67c", + "width": 577, + "height": 640, + "pxPerMM": 11.7611, + "alignTargetFile": "data/bench/w035_ink.png", + "alignTargetSha256": "03bfea34b06f8db6464e9a0b4e024653eb75d5e4872d7f91484a80945dc1e67c" + }, + "prediction": { + "name": "w035_ct.png", + "sha256": "65b1c06363cf835d4b008c7155d3e4140c47489c9d1c905ad20baebeb5a18987", + "format": "png", + "width": 577, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.265032, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 369280, + "validFraction": 1.0, + "pearson": 0.265032, + "blurSigmaPx": 6.0, + "blurSigmaUM": 510.2, + "pearsonMatchedBlur": 0.3956, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.101898, + "dice": 0.18495, + "precision": 0.116697, + "recall": 0.445524, + "refPositiveFraction": 0.089501, + "predPositiveFraction": 0.341695, + "ap": 0.116624, + "bestF1": { + "f1": 0.195925, + "threshold": 0.26 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.178899, + "recall": 0.00118 + }, + { + "t": 0.98, + "precision": 0.180162, + "recall": 0.002693 + }, + { + "t": 0.96, + "precision": 0.183957, + "recall": 0.005204 + }, + { + "t": 0.94, + "precision": 0.165251, + "recall": 0.007655 + }, + { + "t": 0.92, + "precision": 0.149459, + "recall": 0.010862 + }, + { + "t": 0.9, + "precision": 0.144585, + "recall": 0.015552 + }, + { + "t": 0.88, + "precision": 0.135888, + "recall": 0.020816 + }, + { + "t": 0.86, + "precision": 0.13254, + "recall": 0.028653 + }, + { + "t": 0.84, + "precision": 0.130093, + "recall": 0.038062 + }, + { + "t": 0.82, + "precision": 0.130843, + "recall": 0.050135 + }, + { + "t": 0.8, + "precision": 0.127623, + "recall": 0.065505 + }, + { + "t": 0.78, + "precision": 0.127537, + "recall": 0.082327 + }, + { + "t": 0.76, + "precision": 0.125543, + "recall": 0.098817 + }, + { + "t": 0.74, + "precision": 0.124513, + "recall": 0.117909 + }, + { + "t": 0.72, + "precision": 0.122744, + "recall": 0.137484 + }, + { + "t": 0.7, + "precision": 0.122866, + "recall": 0.160449 + }, + { + "t": 0.68, + "precision": 0.122072, + "recall": 0.183837 + }, + { + "t": 0.66, + "precision": 0.121174, + "recall": 0.207921 + }, + { + "t": 0.64, + "precision": 0.120935, + "recall": 0.234395 + }, + { + "t": 0.62, + "precision": 0.119836, + "recall": 0.260749 + }, + { + "t": 0.6, + "precision": 0.119107, + "recall": 0.294605 + }, + { + "t": 0.58, + "precision": 0.11865, + "recall": 0.32344 + }, + { + "t": 0.56, + "precision": 0.118159, + "recall": 0.35297 + }, + { + "t": 0.54, + "precision": 0.117387, + "recall": 0.381985 + }, + { + "t": 0.52, + "precision": 0.117141, + "recall": 0.413936 + }, + { + "t": 0.5, + "precision": 0.116697, + "recall": 0.445524 + }, + { + "t": 0.48, + "precision": 0.115894, + "recall": 0.475205 + }, + { + "t": 0.46, + "precision": 0.115647, + "recall": 0.506914 + }, + { + "t": 0.44, + "precision": 0.114928, + "recall": 0.536837 + }, + { + "t": 0.42, + "precision": 0.114902, + "recall": 0.56909 + }, + { + "t": 0.4, + "precision": 0.114397, + "recall": 0.60443 + }, + { + "t": 0.38, + "precision": 0.113983, + "recall": 0.633022 + }, + { + "t": 0.36, + "precision": 0.113729, + "recall": 0.662431 + }, + { + "t": 0.34, + "precision": 0.113189, + "recall": 0.689147 + }, + { + "t": 0.32, + "precision": 0.11274, + "recall": 0.714683 + }, + { + "t": 0.3, + "precision": 0.11244, + "recall": 0.74028 + }, + { + "t": 0.28, + "precision": 0.112109, + "recall": 0.764001 + }, + { + "t": 0.26, + "precision": 0.111886, + "recall": 0.787208 + }, + { + "t": 0.24, + "precision": 0.111364, + "recall": 0.806632 + }, + { + "t": 0.22, + "precision": 0.111055, + "recall": 0.826571 + }, + { + "t": 0.2, + "precision": 0.110676, + "recall": 0.84884 + }, + { + "t": 0.18, + "precision": 0.110394, + "recall": 0.865662 + }, + { + "t": 0.16, + "precision": 0.110008, + "recall": 0.880488 + }, + { + "t": 0.14, + "precision": 0.109729, + "recall": 0.895404 + }, + { + "t": 0.12, + "precision": 0.109486, + "recall": 0.909867 + }, + { + "t": 0.1, + "precision": 0.109306, + "recall": 0.925781 + }, + { + "t": 0.08, + "precision": 0.109008, + "recall": 0.94094 + }, + { + "t": 0.06, + "precision": 0.108737, + "recall": 0.955281 + }, + { + "t": 0.04, + "precision": 0.108468, + "recall": 0.967807 + }, + { + "t": 0.02, + "precision": 0.107936, + "recall": 0.978185 + }, + { + "t": 0.0, + "precision": 0.089501, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w035-ink9um.json b/docs/examples/bench/w035-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..b55221f75e296c3127bcbfb215651f8321291efa --- /dev/null +++ b/docs/examples/bench/w035-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:29Z", + "scroll": "PHerc0139", + "segment": "w035", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w035_ink.png", + "sha256": "03bfea34b06f8db6464e9a0b4e024653eb75d5e4872d7f91484a80945dc1e67c", + "width": 577, + "height": 640, + "pxPerMM": 11.7611, + "alignTargetFile": "data/bench/w035_ink.png", + "alignTargetSha256": "03bfea34b06f8db6464e9a0b4e024653eb75d5e4872d7f91484a80945dc1e67c" + }, + "prediction": { + "name": "w035_gpu.png", + "sha256": "64fe35009ee7787f8d3afa7b41e2a62936fa8a1a00b80376d5013ad7229175da", + "format": "png", + "width": 577, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.875, + "ty": 0.719, + "sx": 1.00198, + "sy": 1.00092, + "ncc": 0.482965, + "coarseNcc": 0.513263, + "evals": 38474, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 368064, + "validFraction": 0.996707, + "pearson": 0.482965, + "blurSigmaPx": 2.0, + "blurSigmaUM": 170.1, + "pearsonMatchedBlur": 0.509776, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.26025, + "dice": 0.413013, + "precision": 0.41446, + "recall": 0.411576, + "refPositiveFraction": 0.089797, + "predPositiveFraction": 0.089172, + "ap": 0.376242, + "bestF1": { + "f1": 0.413852, + "threshold": 0.46 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.7397, + "recall": 0.023902 + }, + { + "t": 0.98, + "precision": 0.671945, + "recall": 0.141236 + }, + { + "t": 0.96, + "precision": 0.658825, + "recall": 0.163535 + }, + { + "t": 0.94, + "precision": 0.646236, + "recall": 0.180237 + }, + { + "t": 0.92, + "precision": 0.634221, + "recall": 0.193791 + }, + { + "t": 0.9, + "precision": 0.622771, + "recall": 0.206045 + }, + { + "t": 0.88, + "precision": 0.609127, + "recall": 0.21727 + }, + { + "t": 0.86, + "precision": 0.598087, + "recall": 0.228858 + }, + { + "t": 0.84, + "precision": 0.587297, + "recall": 0.240598 + }, + { + "t": 0.82, + "precision": 0.575774, + "recall": 0.251974 + }, + { + "t": 0.8, + "precision": 0.564506, + "recall": 0.261868 + }, + { + "t": 0.78, + "precision": 0.552844, + "recall": 0.272064 + }, + { + "t": 0.76, + "precision": 0.544814, + "recall": 0.283229 + }, + { + "t": 0.74, + "precision": 0.533341, + "recall": 0.293062 + }, + { + "t": 0.72, + "precision": 0.523147, + "recall": 0.302593 + }, + { + "t": 0.7, + "precision": 0.513037, + "recall": 0.312547 + }, + { + "t": 0.68, + "precision": 0.50255, + "recall": 0.321957 + }, + { + "t": 0.66, + "precision": 0.492163, + "recall": 0.331548 + }, + { + "t": 0.64, + "precision": 0.481618, + "recall": 0.340867 + }, + { + "t": 0.62, + "precision": 0.471257, + "recall": 0.350973 + }, + { + "t": 0.6, + "precision": 0.460715, + "recall": 0.360685 + }, + { + "t": 0.58, + "precision": 0.450429, + "recall": 0.370185 + }, + { + "t": 0.56, + "precision": 0.441013, + "recall": 0.379807 + }, + { + "t": 0.54, + "precision": 0.431974, + "recall": 0.390699 + }, + { + "t": 0.52, + "precision": 0.423422, + "recall": 0.400926 + }, + { + "t": 0.5, + "precision": 0.41446, + "recall": 0.411576 + }, + { + "t": 0.48, + "precision": 0.405153, + "recall": 0.421984 + }, + { + "t": 0.46, + "precision": 0.396024, + "recall": 0.433361 + }, + { + "t": 0.44, + "precision": 0.385449, + "recall": 0.443224 + }, + { + "t": 0.42, + "precision": 0.377499, + "recall": 0.455841 + }, + { + "t": 0.4, + "precision": 0.368791, + "recall": 0.468155 + }, + { + "t": 0.38, + "precision": 0.359463, + "recall": 0.479501 + }, + { + "t": 0.36, + "precision": 0.350753, + "recall": 0.493056 + }, + { + "t": 0.34, + "precision": 0.341335, + "recall": 0.506399 + }, + { + "t": 0.32, + "precision": 0.332585, + "recall": 0.520226 + }, + { + "t": 0.3, + "precision": 0.323486, + "recall": 0.535052 + }, + { + "t": 0.28, + "precision": 0.313863, + "recall": 0.55015 + }, + { + "t": 0.26, + "precision": 0.303458, + "recall": 0.565066 + }, + { + "t": 0.24, + "precision": 0.293395, + "recall": 0.582342 + }, + { + "t": 0.22, + "precision": 0.283476, + "recall": 0.599346 + }, + { + "t": 0.2, + "precision": 0.273359, + "recall": 0.61865 + }, + { + "t": 0.18, + "precision": 0.263671, + "recall": 0.637802 + }, + { + "t": 0.16, + "precision": 0.252814, + "recall": 0.657711 + }, + { + "t": 0.14, + "precision": 0.241866, + "recall": 0.679253 + }, + { + "t": 0.12, + "precision": 0.230052, + "recall": 0.703549 + }, + { + "t": 0.1, + "precision": 0.217528, + "recall": 0.729358 + }, + { + "t": 0.08, + "precision": 0.20401, + "recall": 0.75798 + }, + { + "t": 0.06, + "precision": 0.189095, + "recall": 0.790657 + }, + { + "t": 0.04, + "precision": 0.171967, + "recall": 0.830323 + }, + { + "t": 0.02, + "precision": 0.150858, + "recall": 0.885873 + }, + { + "t": 0.0, + "precision": 0.089797, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w035-noise-baseline.json b/docs/examples/bench/w035-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..e6bf899bc561423938d74ffb1f8b354bbfa67079 --- /dev/null +++ b/docs/examples/bench/w035-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:31Z", + "scroll": "PHerc0139", + "segment": "w035", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w035_ink.png", + "sha256": "03bfea34b06f8db6464e9a0b4e024653eb75d5e4872d7f91484a80945dc1e67c", + "width": 577, + "height": 640, + "pxPerMM": 11.7611, + "alignTargetFile": "data/bench/w035_ink.png", + "alignTargetSha256": "03bfea34b06f8db6464e9a0b4e024653eb75d5e4872d7f91484a80945dc1e67c" + }, + "prediction": { + "name": "noise-seed35.png", + "sha256": "f2b4fba2e6b29ba557e1ec3e0f400bd063c3acde28368e614401d3d2855f2b2c", + "format": "png", + "width": 577, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(35).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": -0.000707, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 369280, + "validFraction": 1.0, + "pearson": -0.000707, + "blurSigmaPx": 0.5, + "blurSigmaUM": 42.5, + "pearsonMatchedBlur": -0.000704, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.082064, + "dice": 0.15168, + "precision": 0.089455, + "recall": 0.498291, + "refPositiveFraction": 0.089501, + "predPositiveFraction": 0.498549, + "ap": 0.088983, + "bestF1": { + "f1": 0.164298, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.085021, + "recall": 0.018668 + }, + { + "t": 0.96, + "precision": 0.086586, + "recall": 0.037457 + }, + { + "t": 0.94, + "precision": 0.086426, + "recall": 0.056579 + }, + { + "t": 0.92, + "precision": 0.087192, + "recall": 0.076064 + }, + { + "t": 0.9, + "precision": 0.088234, + "recall": 0.096245 + }, + { + "t": 0.88, + "precision": 0.088119, + "recall": 0.115518 + }, + { + "t": 0.86, + "precision": 0.088212, + "recall": 0.134671 + }, + { + "t": 0.84, + "precision": 0.088067, + "recall": 0.154004 + }, + { + "t": 0.82, + "precision": 0.087865, + "recall": 0.172915 + }, + { + "t": 0.8, + "precision": 0.087918, + "recall": 0.195909 + }, + { + "t": 0.78, + "precision": 0.088077, + "recall": 0.215727 + }, + { + "t": 0.76, + "precision": 0.088561, + "recall": 0.236665 + }, + { + "t": 0.74, + "precision": 0.088937, + "recall": 0.256815 + }, + { + "t": 0.72, + "precision": 0.089129, + "recall": 0.277178 + }, + { + "t": 0.7, + "precision": 0.089214, + "recall": 0.296693 + }, + { + "t": 0.68, + "precision": 0.089238, + "recall": 0.316692 + }, + { + "t": 0.66, + "precision": 0.089258, + "recall": 0.336056 + }, + { + "t": 0.64, + "precision": 0.089338, + "recall": 0.356268 + }, + { + "t": 0.62, + "precision": 0.089474, + "recall": 0.375964 + }, + { + "t": 0.6, + "precision": 0.089375, + "recall": 0.398929 + }, + { + "t": 0.58, + "precision": 0.089193, + "recall": 0.417658 + }, + { + "t": 0.56, + "precision": 0.08941, + "recall": 0.438807 + }, + { + "t": 0.54, + "precision": 0.089616, + "recall": 0.459502 + }, + { + "t": 0.52, + "precision": 0.089762, + "recall": 0.480318 + }, + { + "t": 0.5, + "precision": 0.089455, + "recall": 0.498291 + }, + { + "t": 0.48, + "precision": 0.089519, + "recall": 0.517927 + }, + { + "t": 0.46, + "precision": 0.08955, + "recall": 0.537926 + }, + { + "t": 0.44, + "precision": 0.089539, + "recall": 0.557169 + }, + { + "t": 0.42, + "precision": 0.089542, + "recall": 0.576745 + }, + { + "t": 0.4, + "precision": 0.089542, + "recall": 0.600647 + }, + { + "t": 0.38, + "precision": 0.089511, + "recall": 0.6198 + }, + { + "t": 0.36, + "precision": 0.089422, + "recall": 0.639194 + }, + { + "t": 0.34, + "precision": 0.089418, + "recall": 0.6588 + }, + { + "t": 0.32, + "precision": 0.089405, + "recall": 0.678134 + }, + { + "t": 0.3, + "precision": 0.089374, + "recall": 0.697347 + }, + { + "t": 0.28, + "precision": 0.089361, + "recall": 0.717134 + }, + { + "t": 0.26, + "precision": 0.089352, + "recall": 0.73677 + }, + { + "t": 0.24, + "precision": 0.089381, + "recall": 0.75677 + }, + { + "t": 0.22, + "precision": 0.089408, + "recall": 0.775983 + }, + { + "t": 0.2, + "precision": 0.089509, + "recall": 0.800127 + }, + { + "t": 0.18, + "precision": 0.089564, + "recall": 0.820732 + }, + { + "t": 0.16, + "precision": 0.089583, + "recall": 0.840307 + }, + { + "t": 0.14, + "precision": 0.089531, + "recall": 0.859702 + }, + { + "t": 0.12, + "precision": 0.089463, + "recall": 0.878461 + }, + { + "t": 0.1, + "precision": 0.089464, + "recall": 0.89843 + }, + { + "t": 0.08, + "precision": 0.089413, + "recall": 0.917612 + }, + { + "t": 0.06, + "precision": 0.089551, + "recall": 0.93858 + }, + { + "t": 0.04, + "precision": 0.089442, + "recall": 0.956794 + }, + { + "t": 0.02, + "precision": 0.089458, + "recall": 0.976128 + }, + { + "t": 0.0, + "precision": 0.089501, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w036-ct-baseline.json b/docs/examples/bench/w036-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..169df6512ac1db381f265d0f8894488f4db55d72 --- /dev/null +++ b/docs/examples/bench/w036-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:42Z", + "scroll": "PHerc0139", + "segment": "w036", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w036_ink.png", + "sha256": "eff577394fe80d4cd8e9731203fc67fdf260b7b5d1be9d6187343613bed85a27", + "width": 640, + "height": 549, + "pxPerMM": 9.7934, + "alignTargetFile": "data/bench/w036_ink.png", + "alignTargetSha256": "eff577394fe80d4cd8e9731203fc67fdf260b7b5d1be9d6187343613bed85a27" + }, + "prediction": { + "name": "w036_ct.png", + "sha256": "fe43a9e3eb5bed08041cc55a43134b63fb23a85d543f4e89d838350c7de55f0d", + "format": "png", + "width": 640, + "height": 549, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.269443, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 351360, + "validFraction": 1.0, + "pearson": 0.269443, + "blurSigmaPx": 6.0, + "blurSigmaUM": 612.7, + "pearsonMatchedBlur": 0.420759, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.091271, + "dice": 0.167274, + "precision": 0.09967, + "recall": 0.51994, + "refPositiveFraction": 0.081358, + "predPositiveFraction": 0.424414, + "ap": 0.101863, + "bestF1": { + "f1": 0.176456, + "threshold": 0.06 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.148148, + "recall": 0.001119 + }, + { + "t": 0.98, + "precision": 0.155844, + "recall": 0.002939 + }, + { + "t": 0.96, + "precision": 0.14738, + "recall": 0.004723 + }, + { + "t": 0.94, + "precision": 0.139551, + "recall": 0.007171 + }, + { + "t": 0.92, + "precision": 0.134404, + "recall": 0.011124 + }, + { + "t": 0.9, + "precision": 0.129748, + "recall": 0.016372 + }, + { + "t": 0.88, + "precision": 0.122445, + "recall": 0.022843 + }, + { + "t": 0.86, + "precision": 0.118891, + "recall": 0.031659 + }, + { + "t": 0.84, + "precision": 0.114861, + "recall": 0.041909 + }, + { + "t": 0.82, + "precision": 0.112818, + "recall": 0.054992 + }, + { + "t": 0.8, + "precision": 0.110748, + "recall": 0.073602 + }, + { + "t": 0.78, + "precision": 0.109317, + "recall": 0.092143 + }, + { + "t": 0.76, + "precision": 0.107489, + "recall": 0.112223 + }, + { + "t": 0.74, + "precision": 0.107185, + "recall": 0.137095 + }, + { + "t": 0.72, + "precision": 0.106004, + "recall": 0.163227 + }, + { + "t": 0.7, + "precision": 0.105661, + "recall": 0.192682 + }, + { + "t": 0.68, + "precision": 0.105214, + "recall": 0.223221 + }, + { + "t": 0.66, + "precision": 0.104407, + "recall": 0.253621 + }, + { + "t": 0.64, + "precision": 0.103573, + "recall": 0.28514 + }, + { + "t": 0.62, + "precision": 0.10279, + "recall": 0.317708 + }, + { + "t": 0.6, + "precision": 0.102455, + "recall": 0.358392 + }, + { + "t": 0.58, + "precision": 0.102032, + "recall": 0.39201 + }, + { + "t": 0.56, + "precision": 0.10115, + "recall": 0.423214 + }, + { + "t": 0.54, + "precision": 0.100668, + "recall": 0.456307 + }, + { + "t": 0.52, + "precision": 0.100042, + "recall": 0.487966 + }, + { + "t": 0.5, + "precision": 0.09967, + "recall": 0.51994 + }, + { + "t": 0.48, + "precision": 0.099415, + "recall": 0.551319 + }, + { + "t": 0.46, + "precision": 0.098933, + "recall": 0.580179 + }, + { + "t": 0.44, + "precision": 0.099157, + "recall": 0.612957 + }, + { + "t": 0.42, + "precision": 0.09911, + "recall": 0.643357 + }, + { + "t": 0.4, + "precision": 0.098939, + "recall": 0.677569 + }, + { + "t": 0.38, + "precision": 0.098674, + "recall": 0.704156 + }, + { + "t": 0.36, + "precision": 0.098431, + "recall": 0.729413 + }, + { + "t": 0.34, + "precision": 0.098131, + "recall": 0.753026 + }, + { + "t": 0.32, + "precision": 0.098249, + "recall": 0.778423 + }, + { + "t": 0.3, + "precision": 0.098096, + "recall": 0.799797 + }, + { + "t": 0.28, + "precision": 0.098019, + "recall": 0.820402 + }, + { + "t": 0.26, + "precision": 0.098011, + "recall": 0.840691 + }, + { + "t": 0.24, + "precision": 0.097972, + "recall": 0.858497 + }, + { + "t": 0.22, + "precision": 0.09792, + "recall": 0.874729 + }, + { + "t": 0.2, + "precision": 0.097858, + "recall": 0.893269 + }, + { + "t": 0.18, + "precision": 0.097705, + "recall": 0.906143 + }, + { + "t": 0.16, + "precision": 0.097484, + "recall": 0.917197 + }, + { + "t": 0.14, + "precision": 0.097336, + "recall": 0.927552 + }, + { + "t": 0.12, + "precision": 0.097323, + "recall": 0.938326 + }, + { + "t": 0.1, + "precision": 0.097234, + "recall": 0.947702 + }, + { + "t": 0.08, + "precision": 0.097135, + "recall": 0.956237 + }, + { + "t": 0.06, + "precision": 0.097102, + "recall": 0.965403 + }, + { + "t": 0.04, + "precision": 0.096971, + "recall": 0.972924 + }, + { + "t": 0.02, + "precision": 0.096587, + "recall": 0.98055 + }, + { + "t": 0.0, + "precision": 0.081358, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w036-ink9um.json b/docs/examples/bench/w036-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..afea9d5005fc7dbdff14524d89edec77b70ebfc9 --- /dev/null +++ b/docs/examples/bench/w036-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:41Z", + "scroll": "PHerc0139", + "segment": "w036", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w036_ink.png", + "sha256": "eff577394fe80d4cd8e9731203fc67fdf260b7b5d1be9d6187343613bed85a27", + "width": 640, + "height": 549, + "pxPerMM": 9.7934, + "alignTargetFile": "data/bench/w036_ink.png", + "alignTargetSha256": "eff577394fe80d4cd8e9731203fc67fdf260b7b5d1be9d6187343613bed85a27" + }, + "prediction": { + "name": "w036_gpu.png", + "sha256": "f326573cb16b362511bbbd8e11d51d9cae638c572134f5bf9acd83f983a7799e", + "format": "png", + "width": 640, + "height": 549, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.531, + "ty": 0.625, + "sx": 1.00042, + "sy": 1.00029, + "ncc": 0.374841, + "coarseNcc": 0.480638, + "evals": 36246, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 350172, + "validFraction": 0.996619, + "pearson": 0.374841, + "blurSigmaPx": 2.5, + "blurSigmaUM": 255.3, + "pearsonMatchedBlur": 0.412028, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.179865, + "dice": 0.304891, + "precision": 0.299252, + "recall": 0.310747, + "refPositiveFraction": 0.081634, + "predPositiveFraction": 0.08477, + "ap": 0.254952, + "bestF1": { + "f1": 0.311938, + "threshold": 0.42 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.813333, + "recall": 0.002134 + }, + { + "t": 0.98, + "precision": 0.636771, + "recall": 0.044707 + }, + { + "t": 0.96, + "precision": 0.589206, + "recall": 0.059959 + }, + { + "t": 0.94, + "precision": 0.557089, + "recall": 0.072028 + }, + { + "t": 0.92, + "precision": 0.535754, + "recall": 0.083607 + }, + { + "t": 0.9, + "precision": 0.513544, + "recall": 0.094172 + }, + { + "t": 0.88, + "precision": 0.49521, + "recall": 0.104877 + }, + { + "t": 0.86, + "precision": 0.478966, + "recall": 0.114706 + }, + { + "t": 0.84, + "precision": 0.464807, + "recall": 0.124746 + }, + { + "t": 0.82, + "precision": 0.447666, + "recall": 0.134506 + }, + { + "t": 0.8, + "precision": 0.436411, + "recall": 0.145491 + }, + { + "t": 0.78, + "precision": 0.427317, + "recall": 0.156615 + }, + { + "t": 0.76, + "precision": 0.414568, + "recall": 0.166445 + }, + { + "t": 0.74, + "precision": 0.40292, + "recall": 0.176695 + }, + { + "t": 0.72, + "precision": 0.391978, + "recall": 0.187679 + }, + { + "t": 0.7, + "precision": 0.380353, + "recall": 0.197614 + }, + { + "t": 0.68, + "precision": 0.370161, + "recall": 0.208389 + }, + { + "t": 0.66, + "precision": 0.361519, + "recall": 0.219863 + }, + { + "t": 0.64, + "precision": 0.352746, + "recall": 0.230952 + }, + { + "t": 0.62, + "precision": 0.344522, + "recall": 0.242356 + }, + { + "t": 0.6, + "precision": 0.337157, + "recall": 0.254985 + }, + { + "t": 0.58, + "precision": 0.329071, + "recall": 0.265585 + }, + { + "t": 0.56, + "precision": 0.32161, + "recall": 0.276709 + }, + { + "t": 0.54, + "precision": 0.314113, + "recall": 0.287378 + }, + { + "t": 0.52, + "precision": 0.306762, + "recall": 0.299937 + }, + { + "t": 0.5, + "precision": 0.299252, + "recall": 0.310747 + }, + { + "t": 0.48, + "precision": 0.292248, + "recall": 0.323235 + }, + { + "t": 0.46, + "precision": 0.286385, + "recall": 0.336948 + }, + { + "t": 0.44, + "precision": 0.280022, + "recall": 0.350591 + }, + { + "t": 0.42, + "precision": 0.27305, + "recall": 0.363744 + }, + { + "t": 0.4, + "precision": 0.265792, + "recall": 0.377108 + }, + { + "t": 0.38, + "precision": 0.258663, + "recall": 0.390926 + }, + { + "t": 0.36, + "precision": 0.252232, + "recall": 0.406213 + }, + { + "t": 0.34, + "precision": 0.244672, + "recall": 0.419646 + }, + { + "t": 0.32, + "precision": 0.23745, + "recall": 0.434338 + }, + { + "t": 0.3, + "precision": 0.230145, + "recall": 0.449171 + }, + { + "t": 0.28, + "precision": 0.22362, + "recall": 0.466312 + }, + { + "t": 0.26, + "precision": 0.216905, + "recall": 0.483768 + }, + { + "t": 0.24, + "precision": 0.210145, + "recall": 0.501889 + }, + { + "t": 0.22, + "precision": 0.203535, + "recall": 0.522494 + }, + { + "t": 0.2, + "precision": 0.196501, + "recall": 0.542608 + }, + { + "t": 0.18, + "precision": 0.190042, + "recall": 0.566921 + }, + { + "t": 0.16, + "precision": 0.183092, + "recall": 0.592598 + }, + { + "t": 0.14, + "precision": 0.176032, + "recall": 0.620024 + }, + { + "t": 0.12, + "precision": 0.168484, + "recall": 0.649829 + }, + { + "t": 0.1, + "precision": 0.160349, + "recall": 0.681103 + }, + { + "t": 0.08, + "precision": 0.15213, + "recall": 0.718149 + }, + { + "t": 0.06, + "precision": 0.142438, + "recall": 0.759778 + }, + { + "t": 0.04, + "precision": 0.13153, + "recall": 0.812251 + }, + { + "t": 0.02, + "precision": 0.118008, + "recall": 0.879137 + }, + { + "t": 0.0, + "precision": 0.081634, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w036-noise-baseline.json b/docs/examples/bench/w036-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..f8f68899954a6e835abbe3401a3d756e94e6c3a3 --- /dev/null +++ b/docs/examples/bench/w036-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:43Z", + "scroll": "PHerc0139", + "segment": "w036", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w036_ink.png", + "sha256": "eff577394fe80d4cd8e9731203fc67fdf260b7b5d1be9d6187343613bed85a27", + "width": 640, + "height": 549, + "pxPerMM": 9.7934, + "alignTargetFile": "data/bench/w036_ink.png", + "alignTargetSha256": "eff577394fe80d4cd8e9731203fc67fdf260b7b5d1be9d6187343613bed85a27" + }, + "prediction": { + "name": "noise-seed36.png", + "sha256": "eb3fc68dd8289f7dcc5191e2549d601d5d88f380f3dbceef2e28b94e0c152f54", + "format": "png", + "width": 640, + "height": 549, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(36).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.000319, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 351360, + "validFraction": 1.0, + "pearson": 0.000319, + "blurSigmaPx": 0.0, + "blurSigmaUM": 0.0, + "pearsonMatchedBlur": 0.000319, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.075106, + "dice": 0.139718, + "precision": 0.081313, + "recall": 0.495942, + "refPositiveFraction": 0.081358, + "predPositiveFraction": 0.49622, + "ap": 0.08169, + "bestF1": { + "f1": 0.150474, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.079298, + "recall": 0.01896 + }, + { + "t": 0.96, + "precision": 0.083128, + "recall": 0.040125 + }, + { + "t": 0.94, + "precision": 0.083317, + "recall": 0.060169 + }, + { + "t": 0.92, + "precision": 0.082394, + "recall": 0.079235 + }, + { + "t": 0.9, + "precision": 0.082502, + "recall": 0.099209 + }, + { + "t": 0.88, + "precision": 0.08342, + "recall": 0.120409 + }, + { + "t": 0.86, + "precision": 0.082839, + "recall": 0.139614 + }, + { + "t": 0.84, + "precision": 0.082262, + "recall": 0.158259 + }, + { + "t": 0.82, + "precision": 0.082508, + "recall": 0.178164 + }, + { + "t": 0.8, + "precision": 0.082415, + "recall": 0.202162 + }, + { + "t": 0.78, + "precision": 0.082461, + "recall": 0.222172 + }, + { + "t": 0.76, + "precision": 0.08242, + "recall": 0.241622 + }, + { + "t": 0.74, + "precision": 0.082277, + "recall": 0.260897 + }, + { + "t": 0.72, + "precision": 0.082062, + "recall": 0.279927 + }, + { + "t": 0.7, + "precision": 0.081859, + "recall": 0.298713 + }, + { + "t": 0.68, + "precision": 0.081751, + "recall": 0.317568 + }, + { + "t": 0.66, + "precision": 0.081792, + "recall": 0.337683 + }, + { + "t": 0.64, + "precision": 0.081889, + "recall": 0.357588 + }, + { + "t": 0.62, + "precision": 0.081602, + "recall": 0.376093 + }, + { + "t": 0.6, + "precision": 0.081623, + "recall": 0.400126 + }, + { + "t": 0.58, + "precision": 0.081587, + "recall": 0.419331 + }, + { + "t": 0.56, + "precision": 0.081632, + "recall": 0.439061 + }, + { + "t": 0.54, + "precision": 0.081529, + "recall": 0.457881 + }, + { + "t": 0.52, + "precision": 0.081362, + "recall": 0.476527 + }, + { + "t": 0.5, + "precision": 0.081313, + "recall": 0.495942 + }, + { + "t": 0.48, + "precision": 0.081452, + "recall": 0.516477 + }, + { + "t": 0.46, + "precision": 0.081377, + "recall": 0.535647 + }, + { + "t": 0.44, + "precision": 0.081447, + "recall": 0.555482 + }, + { + "t": 0.42, + "precision": 0.081452, + "recall": 0.574967 + }, + { + "t": 0.4, + "precision": 0.081523, + "recall": 0.598895 + }, + { + "t": 0.38, + "precision": 0.081456, + "recall": 0.61824 + }, + { + "t": 0.36, + "precision": 0.081395, + "recall": 0.637445 + }, + { + "t": 0.34, + "precision": 0.0813, + "recall": 0.656265 + }, + { + "t": 0.32, + "precision": 0.081387, + "recall": 0.67687 + }, + { + "t": 0.3, + "precision": 0.08141, + "recall": 0.69667 + }, + { + "t": 0.28, + "precision": 0.081222, + "recall": 0.714686 + }, + { + "t": 0.26, + "precision": 0.081111, + "recall": 0.733156 + }, + { + "t": 0.24, + "precision": 0.081152, + "recall": 0.752921 + }, + { + "t": 0.22, + "precision": 0.081134, + "recall": 0.772371 + }, + { + "t": 0.2, + "precision": 0.081164, + "recall": 0.796019 + }, + { + "t": 0.18, + "precision": 0.081161, + "recall": 0.815889 + }, + { + "t": 0.16, + "precision": 0.081138, + "recall": 0.834744 + }, + { + "t": 0.14, + "precision": 0.081241, + "recall": 0.855314 + }, + { + "t": 0.12, + "precision": 0.081308, + "recall": 0.875603 + }, + { + "t": 0.1, + "precision": 0.081342, + "recall": 0.895893 + }, + { + "t": 0.08, + "precision": 0.081333, + "recall": 0.915553 + }, + { + "t": 0.06, + "precision": 0.081309, + "recall": 0.935423 + }, + { + "t": 0.04, + "precision": 0.081386, + "recall": 0.956517 + }, + { + "t": 0.02, + "precision": 0.081413, + "recall": 0.976737 + }, + { + "t": 0.0, + "precision": 0.081358, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w037-ct-baseline.json b/docs/examples/bench/w037-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..b10a0d5f568a74caff241d204bcc605a8576af8d --- /dev/null +++ b/docs/examples/bench/w037-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:52Z", + "scroll": "PHerc0139", + "segment": "w037", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w037_ink.png", + "sha256": "ccc197e70cacdda6a8f55b7aa084b38967378da0a063f6c591bb2dd9f801f4d1", + "width": 640, + "height": 545, + "pxPerMM": 9.4941, + "alignTargetFile": "data/bench/w037_ink.png", + "alignTargetSha256": "ccc197e70cacdda6a8f55b7aa084b38967378da0a063f6c591bb2dd9f801f4d1" + }, + "prediction": { + "name": "w037_ct.png", + "sha256": "2656acecedda3f2647058e84185a9d9a5f691272ed0fa99e1c007ab2590a6733", + "format": "png", + "width": 640, + "height": 545, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.269574, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 348800, + "validFraction": 1.0, + "pearson": 0.269574, + "blurSigmaPx": 6.0, + "blurSigmaUM": 632.0, + "pearsonMatchedBlur": 0.436115, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.089405, + "dice": 0.164136, + "precision": 0.097383, + "recall": 0.521844, + "refPositiveFraction": 0.078288, + "predPositiveFraction": 0.419524, + "ap": 0.096985, + "bestF1": { + "f1": 0.170478, + "threshold": 0.26 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.103774, + "recall": 0.000806 + }, + { + "t": 0.98, + "precision": 0.111504, + "recall": 0.002307 + }, + { + "t": 0.96, + "precision": 0.107317, + "recall": 0.004028 + }, + { + "t": 0.94, + "precision": 0.107018, + "recall": 0.006702 + }, + { + "t": 0.92, + "precision": 0.101859, + "recall": 0.010034 + }, + { + "t": 0.9, + "precision": 0.099822, + "recall": 0.014392 + }, + { + "t": 0.88, + "precision": 0.103905, + "recall": 0.021533 + }, + { + "t": 0.86, + "precision": 0.102137, + "recall": 0.029406 + }, + { + "t": 0.84, + "precision": 0.100859, + "recall": 0.039147 + }, + { + "t": 0.82, + "precision": 0.099813, + "recall": 0.050939 + }, + { + "t": 0.8, + "precision": 0.100875, + "recall": 0.069689 + }, + { + "t": 0.78, + "precision": 0.100573, + "recall": 0.087487 + }, + { + "t": 0.76, + "precision": 0.100027, + "recall": 0.107262 + }, + { + "t": 0.74, + "precision": 0.100277, + "recall": 0.131102 + }, + { + "t": 0.72, + "precision": 0.099699, + "recall": 0.155491 + }, + { + "t": 0.7, + "precision": 0.098625, + "recall": 0.181016 + }, + { + "t": 0.68, + "precision": 0.099095, + "recall": 0.210972 + }, + { + "t": 0.66, + "precision": 0.099261, + "recall": 0.242832 + }, + { + "t": 0.64, + "precision": 0.098699, + "recall": 0.273556 + }, + { + "t": 0.62, + "precision": 0.098736, + "recall": 0.306844 + }, + { + "t": 0.6, + "precision": 0.098714, + "recall": 0.348958 + }, + { + "t": 0.58, + "precision": 0.097907, + "recall": 0.381917 + }, + { + "t": 0.56, + "precision": 0.097912, + "recall": 0.417329 + }, + { + "t": 0.54, + "precision": 0.097512, + "recall": 0.451423 + }, + { + "t": 0.52, + "precision": 0.097426, + "recall": 0.486322 + }, + { + "t": 0.5, + "precision": 0.097383, + "recall": 0.521844 + }, + { + "t": 0.48, + "precision": 0.096964, + "recall": 0.555059 + }, + { + "t": 0.46, + "precision": 0.097119, + "recall": 0.590361 + }, + { + "t": 0.44, + "precision": 0.097247, + "recall": 0.625041 + }, + { + "t": 0.42, + "precision": 0.096867, + "recall": 0.65518 + }, + { + "t": 0.4, + "precision": 0.096591, + "recall": 0.691032 + }, + { + "t": 0.38, + "precision": 0.09636, + "recall": 0.71901 + }, + { + "t": 0.36, + "precision": 0.096159, + "recall": 0.745926 + }, + { + "t": 0.34, + "precision": 0.095736, + "recall": 0.768777 + }, + { + "t": 0.32, + "precision": 0.095462, + "recall": 0.791775 + }, + { + "t": 0.3, + "precision": 0.09515, + "recall": 0.812209 + }, + { + "t": 0.28, + "precision": 0.094936, + "recall": 0.832534 + }, + { + "t": 0.26, + "precision": 0.094727, + "recall": 0.850991 + }, + { + "t": 0.24, + "precision": 0.094504, + "recall": 0.867909 + }, + { + "t": 0.22, + "precision": 0.094272, + "recall": 0.882704 + }, + { + "t": 0.2, + "precision": 0.093949, + "recall": 0.898524 + }, + { + "t": 0.18, + "precision": 0.093663, + "recall": 0.90962 + }, + { + "t": 0.16, + "precision": 0.09341, + "recall": 0.920057 + }, + { + "t": 0.14, + "precision": 0.093188, + "recall": 0.930018 + }, + { + "t": 0.12, + "precision": 0.092912, + "recall": 0.937855 + }, + { + "t": 0.1, + "precision": 0.092715, + "recall": 0.946168 + }, + { + "t": 0.08, + "precision": 0.092575, + "recall": 0.954114 + }, + { + "t": 0.06, + "precision": 0.092479, + "recall": 0.962354 + }, + { + "t": 0.04, + "precision": 0.09242, + "recall": 0.970996 + }, + { + "t": 0.02, + "precision": 0.092181, + "recall": 0.979419 + }, + { + "t": 0.0, + "precision": 0.078288, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w037-ink9um.json b/docs/examples/bench/w037-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..a0e45c94d329dbb2aabaa31f7093ee201663ad8b --- /dev/null +++ b/docs/examples/bench/w037-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:51Z", + "scroll": "PHerc0139", + "segment": "w037", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w037_ink.png", + "sha256": "ccc197e70cacdda6a8f55b7aa084b38967378da0a063f6c591bb2dd9f801f4d1", + "width": 640, + "height": 545, + "pxPerMM": 9.4941, + "alignTargetFile": "data/bench/w037_ink.png", + "alignTargetSha256": "ccc197e70cacdda6a8f55b7aa084b38967378da0a063f6c591bb2dd9f801f4d1" + }, + "prediction": { + "name": "w037_gpu.png", + "sha256": "4b1f3efc88ebc02f44ade6d426371c7ceaa0f45a12d68828cae8a2a0948bcb33", + "format": "png", + "width": 640, + "height": 545, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.594, + "ty": 0.656, + "sx": 1.00064, + "sy": 1.00034, + "ncc": 0.348064, + "coarseNcc": 0.454299, + "evals": 36243, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 347616, + "validFraction": 0.996606, + "pearson": 0.348064, + "blurSigmaPx": 2.5, + "blurSigmaUM": 263.3, + "pearsonMatchedBlur": 0.385379, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.17065, + "dice": 0.291548, + "precision": 0.251118, + "recall": 0.347493, + "refPositiveFraction": 0.078555, + "predPositiveFraction": 0.108703, + "ap": 0.227863, + "bestF1": { + "f1": 0.292012, + "threshold": 0.46 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.757282, + "recall": 0.002856 + }, + { + "t": 0.98, + "precision": 0.514743, + "recall": 0.060094 + }, + { + "t": 0.96, + "precision": 0.477247, + "recall": 0.077196 + }, + { + "t": 0.94, + "precision": 0.452415, + "recall": 0.092614 + }, + { + "t": 0.92, + "precision": 0.434095, + "recall": 0.106493 + }, + { + "t": 0.9, + "precision": 0.415639, + "recall": 0.118541 + }, + { + "t": 0.88, + "precision": 0.401489, + "recall": 0.13037 + }, + { + "t": 0.86, + "precision": 0.389006, + "recall": 0.141502 + }, + { + "t": 0.84, + "precision": 0.378354, + "recall": 0.153367 + }, + { + "t": 0.82, + "precision": 0.369233, + "recall": 0.165599 + }, + { + "t": 0.8, + "precision": 0.360144, + "recall": 0.176438 + }, + { + "t": 0.78, + "precision": 0.351757, + "recall": 0.188084 + }, + { + "t": 0.76, + "precision": 0.343384, + "recall": 0.199766 + }, + { + "t": 0.74, + "precision": 0.334418, + "recall": 0.210679 + }, + { + "t": 0.72, + "precision": 0.325638, + "recall": 0.221079 + }, + { + "t": 0.7, + "precision": 0.318246, + "recall": 0.231735 + }, + { + "t": 0.68, + "precision": 0.310564, + "recall": 0.243088 + }, + { + "t": 0.66, + "precision": 0.30368, + "recall": 0.255649 + }, + { + "t": 0.64, + "precision": 0.29657, + "recall": 0.267257 + }, + { + "t": 0.62, + "precision": 0.288773, + "recall": 0.278061 + }, + { + "t": 0.6, + "precision": 0.282578, + "recall": 0.289376 + }, + { + "t": 0.58, + "precision": 0.275927, + "recall": 0.300582 + }, + { + "t": 0.56, + "precision": 0.269326, + "recall": 0.311825 + }, + { + "t": 0.54, + "precision": 0.262899, + "recall": 0.322628 + }, + { + "t": 0.52, + "precision": 0.25729, + "recall": 0.335079 + }, + { + "t": 0.5, + "precision": 0.251118, + "recall": 0.347493 + }, + { + "t": 0.48, + "precision": 0.245522, + "recall": 0.359871 + }, + { + "t": 0.46, + "precision": 0.239775, + "recall": 0.373347 + }, + { + "t": 0.44, + "precision": 0.233732, + "recall": 0.386714 + }, + { + "t": 0.42, + "precision": 0.227111, + "recall": 0.399238 + }, + { + "t": 0.4, + "precision": 0.221173, + "recall": 0.413227 + }, + { + "t": 0.38, + "precision": 0.215434, + "recall": 0.427546 + }, + { + "t": 0.36, + "precision": 0.21043, + "recall": 0.443732 + }, + { + "t": 0.34, + "precision": 0.205083, + "recall": 0.459772 + }, + { + "t": 0.32, + "precision": 0.199656, + "recall": 0.475556 + }, + { + "t": 0.3, + "precision": 0.194705, + "recall": 0.493134 + }, + { + "t": 0.28, + "precision": 0.189321, + "recall": 0.510419 + }, + { + "t": 0.26, + "precision": 0.183932, + "recall": 0.528106 + }, + { + "t": 0.24, + "precision": 0.178675, + "recall": 0.547442 + }, + { + "t": 0.22, + "precision": 0.1732, + "recall": 0.566338 + }, + { + "t": 0.2, + "precision": 0.167405, + "recall": 0.58648 + }, + { + "t": 0.18, + "precision": 0.16217, + "recall": 0.61032 + }, + { + "t": 0.16, + "precision": 0.156466, + "recall": 0.632951 + }, + { + "t": 0.14, + "precision": 0.150847, + "recall": 0.658036 + }, + { + "t": 0.12, + "precision": 0.144968, + "recall": 0.685868 + }, + { + "t": 0.1, + "precision": 0.138962, + "recall": 0.717252 + }, + { + "t": 0.08, + "precision": 0.132234, + "recall": 0.751016 + }, + { + "t": 0.06, + "precision": 0.124933, + "recall": 0.790164 + }, + { + "t": 0.04, + "precision": 0.116809, + "recall": 0.836745 + }, + { + "t": 0.02, + "precision": 0.106642, + "recall": 0.895192 + }, + { + "t": 0.0, + "precision": 0.078555, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w037-noise-baseline.json b/docs/examples/bench/w037-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..30b35b36ff004e9e48b7754dfe3c21da1a3c47b7 --- /dev/null +++ b/docs/examples/bench/w037-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:54Z", + "scroll": "PHerc0139", + "segment": "w037", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w037_ink.png", + "sha256": "ccc197e70cacdda6a8f55b7aa084b38967378da0a063f6c591bb2dd9f801f4d1", + "width": 640, + "height": 545, + "pxPerMM": 9.4941, + "alignTargetFile": "data/bench/w037_ink.png", + "alignTargetSha256": "ccc197e70cacdda6a8f55b7aa084b38967378da0a063f6c591bb2dd9f801f4d1" + }, + "prediction": { + "name": "noise-seed37.png", + "sha256": "069c42ab50caeaf483bef617e47052e084ef617274ab7891b7857cf79a279621", + "format": "png", + "width": 640, + "height": 545, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(37).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.003925, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 348800, + "validFraction": 1.0, + "pearson": 0.003925, + "blurSigmaPx": 2.0, + "blurSigmaUM": 210.7, + "pearsonMatchedBlur": 0.004883, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.073225, + "dice": 0.136458, + "precision": 0.078931, + "recall": 0.503204, + "refPositiveFraction": 0.078288, + "predPositiveFraction": 0.499106, + "ap": 0.078983, + "bestF1": { + "f1": 0.145209, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.077575, + "recall": 0.019775 + }, + { + "t": 0.96, + "precision": 0.080167, + "recall": 0.0401 + }, + { + "t": 0.94, + "precision": 0.078624, + "recall": 0.058849 + }, + { + "t": 0.92, + "precision": 0.07901, + "recall": 0.078917 + }, + { + "t": 0.9, + "precision": 0.079142, + "recall": 0.098473 + }, + { + "t": 0.88, + "precision": 0.079292, + "recall": 0.118504 + }, + { + "t": 0.86, + "precision": 0.079775, + "recall": 0.139232 + }, + { + "t": 0.84, + "precision": 0.080103, + "recall": 0.159996 + }, + { + "t": 0.82, + "precision": 0.080341, + "recall": 0.18032 + }, + { + "t": 0.8, + "precision": 0.080566, + "recall": 0.205332 + }, + { + "t": 0.78, + "precision": 0.080133, + "recall": 0.224485 + }, + { + "t": 0.76, + "precision": 0.079726, + "recall": 0.243088 + }, + { + "t": 0.74, + "precision": 0.079229, + "recall": 0.261398 + }, + { + "t": 0.72, + "precision": 0.0792, + "recall": 0.28121 + }, + { + "t": 0.7, + "precision": 0.079022, + "recall": 0.300289 + }, + { + "t": 0.68, + "precision": 0.079135, + "recall": 0.320504 + }, + { + "t": 0.66, + "precision": 0.079029, + "recall": 0.339766 + }, + { + "t": 0.64, + "precision": 0.079253, + "recall": 0.361043 + }, + { + "t": 0.62, + "precision": 0.079153, + "recall": 0.380415 + }, + { + "t": 0.6, + "precision": 0.079147, + "recall": 0.404292 + }, + { + "t": 0.58, + "precision": 0.079181, + "recall": 0.42458 + }, + { + "t": 0.56, + "precision": 0.079165, + "recall": 0.444465 + }, + { + "t": 0.54, + "precision": 0.079114, + "recall": 0.46424 + }, + { + "t": 0.52, + "precision": 0.078887, + "recall": 0.482697 + }, + { + "t": 0.5, + "precision": 0.078931, + "recall": 0.503204 + }, + { + "t": 0.48, + "precision": 0.078929, + "recall": 0.523236 + }, + { + "t": 0.46, + "precision": 0.078956, + "recall": 0.543231 + }, + { + "t": 0.44, + "precision": 0.078925, + "recall": 0.562457 + }, + { + "t": 0.42, + "precision": 0.078878, + "recall": 0.581939 + }, + { + "t": 0.4, + "precision": 0.078845, + "recall": 0.605339 + }, + { + "t": 0.38, + "precision": 0.078824, + "recall": 0.624712 + }, + { + "t": 0.36, + "precision": 0.078859, + "recall": 0.644999 + }, + { + "t": 0.34, + "precision": 0.078799, + "recall": 0.664189 + }, + { + "t": 0.32, + "precision": 0.078715, + "recall": 0.683158 + }, + { + "t": 0.3, + "precision": 0.078627, + "recall": 0.701945 + }, + { + "t": 0.28, + "precision": 0.078576, + "recall": 0.720914 + }, + { + "t": 0.26, + "precision": 0.078509, + "recall": 0.740067 + }, + { + "t": 0.24, + "precision": 0.078476, + "recall": 0.759073 + }, + { + "t": 0.22, + "precision": 0.078521, + "recall": 0.779068 + }, + { + "t": 0.2, + "precision": 0.078568, + "recall": 0.803018 + }, + { + "t": 0.18, + "precision": 0.078552, + "recall": 0.822134 + }, + { + "t": 0.16, + "precision": 0.078604, + "recall": 0.842495 + }, + { + "t": 0.14, + "precision": 0.078536, + "recall": 0.861354 + }, + { + "t": 0.12, + "precision": 0.078508, + "recall": 0.88069 + }, + { + "t": 0.1, + "precision": 0.078508, + "recall": 0.900282 + }, + { + "t": 0.08, + "precision": 0.07851, + "recall": 0.919471 + }, + { + "t": 0.06, + "precision": 0.078445, + "recall": 0.938514 + }, + { + "t": 0.04, + "precision": 0.078405, + "recall": 0.95763 + }, + { + "t": 0.02, + "precision": 0.078377, + "recall": 0.977112 + }, + { + "t": 0.0, + "precision": 0.078288, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w038-ct-baseline.json b/docs/examples/bench/w038-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..61f5fee23ab86f605a4cf02a97333569b6919934 --- /dev/null +++ b/docs/examples/bench/w038-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:00Z", + "scroll": "PHerc0139", + "segment": "w038", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w038_ink.png", + "sha256": "f4f7e94b9b7449a3a5861cbe309e4bdc22714cc18a1d5c15f12db410a7aac2eb", + "width": 640, + "height": 532, + "pxPerMM": 9.1888, + "alignTargetFile": "data/bench/w038_ink.png", + "alignTargetSha256": "f4f7e94b9b7449a3a5861cbe309e4bdc22714cc18a1d5c15f12db410a7aac2eb" + }, + "prediction": { + "name": "w038_ct.png", + "sha256": "3a7dc45d62f9da8ab65a46e3f388813749b058d1e93dfc6bed78d70cde7a76ee", + "format": "png", + "width": 640, + "height": 532, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.252078, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 340480, + "validFraction": 1.0, + "pearson": 0.252078, + "blurSigmaPx": 6.0, + "blurSigmaUM": 653.0, + "pearsonMatchedBlur": 0.403186, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.074925, + "dice": 0.139405, + "precision": 0.080061, + "recall": 0.538726, + "refPositiveFraction": 0.065831, + "predPositiveFraction": 0.442969, + "ap": 0.0799, + "bestF1": { + "f1": 0.142381, + "threshold": 0.02 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.101449, + "recall": 0.000937 + }, + { + "t": 0.98, + "precision": 0.09872, + "recall": 0.002409 + }, + { + "t": 0.96, + "precision": 0.088322, + "recall": 0.004015 + }, + { + "t": 0.94, + "precision": 0.089841, + "recall": 0.006826 + }, + { + "t": 0.92, + "precision": 0.092415, + "recall": 0.011198 + }, + { + "t": 0.9, + "precision": 0.085285, + "recall": 0.015437 + }, + { + "t": 0.88, + "precision": 0.082559, + "recall": 0.021415 + }, + { + "t": 0.86, + "precision": 0.085234, + "recall": 0.031007 + }, + { + "t": 0.84, + "precision": 0.085587, + "recall": 0.042786 + }, + { + "t": 0.82, + "precision": 0.083411, + "recall": 0.055546 + }, + { + "t": 0.8, + "precision": 0.08253, + "recall": 0.075221 + }, + { + "t": 0.78, + "precision": 0.082461, + "recall": 0.094896 + }, + { + "t": 0.76, + "precision": 0.081764, + "recall": 0.116802 + }, + { + "t": 0.74, + "precision": 0.081356, + "recall": 0.141162 + }, + { + "t": 0.72, + "precision": 0.0827, + "recall": 0.171768 + }, + { + "t": 0.7, + "precision": 0.082254, + "recall": 0.201258 + }, + { + "t": 0.68, + "precision": 0.082251, + "recall": 0.23289 + }, + { + "t": 0.66, + "precision": 0.082534, + "recall": 0.267601 + }, + { + "t": 0.64, + "precision": 0.082488, + "recall": 0.301151 + }, + { + "t": 0.62, + "precision": 0.081806, + "recall": 0.33314 + }, + { + "t": 0.6, + "precision": 0.081209, + "recall": 0.372446 + }, + { + "t": 0.58, + "precision": 0.08122, + "recall": 0.408361 + }, + { + "t": 0.56, + "precision": 0.080911, + "recall": 0.441421 + }, + { + "t": 0.54, + "precision": 0.080583, + "recall": 0.474302 + }, + { + "t": 0.52, + "precision": 0.080184, + "recall": 0.505845 + }, + { + "t": 0.5, + "precision": 0.080061, + "recall": 0.538726 + }, + { + "t": 0.48, + "precision": 0.079676, + "recall": 0.568886 + }, + { + "t": 0.46, + "precision": 0.07928, + "recall": 0.59793 + }, + { + "t": 0.44, + "precision": 0.079021, + "recall": 0.626707 + }, + { + "t": 0.42, + "precision": 0.078812, + "recall": 0.654546 + }, + { + "t": 0.4, + "precision": 0.078565, + "recall": 0.686892 + }, + { + "t": 0.38, + "precision": 0.078379, + "recall": 0.712233 + }, + { + "t": 0.36, + "precision": 0.078212, + "recall": 0.736013 + }, + { + "t": 0.34, + "precision": 0.077948, + "recall": 0.757562 + }, + { + "t": 0.32, + "precision": 0.07775, + "recall": 0.778621 + }, + { + "t": 0.3, + "precision": 0.077583, + "recall": 0.798117 + }, + { + "t": 0.28, + "precision": 0.077585, + "recall": 0.818016 + }, + { + "t": 0.26, + "precision": 0.077404, + "recall": 0.834434 + }, + { + "t": 0.24, + "precision": 0.077364, + "recall": 0.851031 + }, + { + "t": 0.22, + "precision": 0.077129, + "recall": 0.864415 + }, + { + "t": 0.2, + "precision": 0.07707, + "recall": 0.880967 + }, + { + "t": 0.18, + "precision": 0.077038, + "recall": 0.893504 + }, + { + "t": 0.16, + "precision": 0.076984, + "recall": 0.905015 + }, + { + "t": 0.14, + "precision": 0.076941, + "recall": 0.915499 + }, + { + "t": 0.12, + "precision": 0.076862, + "recall": 0.924824 + }, + { + "t": 0.1, + "precision": 0.076905, + "recall": 0.935487 + }, + { + "t": 0.08, + "precision": 0.076861, + "recall": 0.945034 + }, + { + "t": 0.06, + "precision": 0.076835, + "recall": 0.955028 + }, + { + "t": 0.04, + "precision": 0.076839, + "recall": 0.965557 + }, + { + "t": 0.02, + "precision": 0.076787, + "recall": 0.976756 + }, + { + "t": 0.0, + "precision": 0.065831, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w038-ink9um.json b/docs/examples/bench/w038-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..552c11b74ab7785ea880a8c74d6daaf226d1e7bf --- /dev/null +++ b/docs/examples/bench/w038-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:52:59Z", + "scroll": "PHerc0139", + "segment": "w038", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w038_ink.png", + "sha256": "f4f7e94b9b7449a3a5861cbe309e4bdc22714cc18a1d5c15f12db410a7aac2eb", + "width": 640, + "height": 532, + "pxPerMM": 9.1888, + "alignTargetFile": "data/bench/w038_ink.png", + "alignTargetSha256": "f4f7e94b9b7449a3a5861cbe309e4bdc22714cc18a1d5c15f12db410a7aac2eb" + }, + "prediction": { + "name": "w038_gpu.png", + "sha256": "a9e4b402fa9bc449ef35f0eb3edd3758ad1afafe4ac0178d05cd4245eaa74201", + "format": "png", + "width": 640, + "height": 532, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.469, + "ty": 0.5, + "sx": 1.00071, + "sy": 1.00189, + "ncc": 0.301921, + "coarseNcc": 0.425144, + "evals": 36193, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 339948, + "validFraction": 0.998438, + "pearson": 0.301921, + "blurSigmaPx": 2.5, + "blurSigmaUM": 272.1, + "pearsonMatchedBlur": 0.33623, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.133176, + "dice": 0.235049, + "precision": 0.204198, + "recall": 0.276881, + "refPositiveFraction": 0.065934, + "predPositiveFraction": 0.089402, + "ap": 0.171588, + "bestF1": { + "f1": 0.235811, + "threshold": 0.44 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.608696, + "recall": 0.001249 + }, + { + "t": 0.98, + "precision": 0.45101, + "recall": 0.039841 + }, + { + "t": 0.96, + "precision": 0.417947, + "recall": 0.053404 + }, + { + "t": 0.94, + "precision": 0.400223, + "recall": 0.063978 + }, + { + "t": 0.92, + "precision": 0.380217, + "recall": 0.07357 + }, + { + "t": 0.9, + "precision": 0.362336, + "recall": 0.082493 + }, + { + "t": 0.88, + "precision": 0.346655, + "recall": 0.091327 + }, + { + "t": 0.86, + "precision": 0.332488, + "recall": 0.099402 + }, + { + "t": 0.84, + "precision": 0.320887, + "recall": 0.108504 + }, + { + "t": 0.82, + "precision": 0.307213, + "recall": 0.11649 + }, + { + "t": 0.8, + "precision": 0.296674, + "recall": 0.124565 + }, + { + "t": 0.78, + "precision": 0.288025, + "recall": 0.133488 + }, + { + "t": 0.76, + "precision": 0.279333, + "recall": 0.142009 + }, + { + "t": 0.74, + "precision": 0.271207, + "recall": 0.1512 + }, + { + "t": 0.72, + "precision": 0.262662, + "recall": 0.160346 + }, + { + "t": 0.7, + "precision": 0.255856, + "recall": 0.169582 + }, + { + "t": 0.68, + "precision": 0.249535, + "recall": 0.179397 + }, + { + "t": 0.66, + "precision": 0.243203, + "recall": 0.189167 + }, + { + "t": 0.64, + "precision": 0.236511, + "recall": 0.198894 + }, + { + "t": 0.62, + "precision": 0.231429, + "recall": 0.209601 + }, + { + "t": 0.6, + "precision": 0.227095, + "recall": 0.220398 + }, + { + "t": 0.58, + "precision": 0.222474, + "recall": 0.232042 + }, + { + "t": 0.56, + "precision": 0.217289, + "recall": 0.242572 + }, + { + "t": 0.54, + "precision": 0.212917, + "recall": 0.253859 + }, + { + "t": 0.52, + "precision": 0.208614, + "recall": 0.265147 + }, + { + "t": 0.5, + "precision": 0.204198, + "recall": 0.276881 + }, + { + "t": 0.48, + "precision": 0.198861, + "recall": 0.288213 + }, + { + "t": 0.46, + "precision": 0.193787, + "recall": 0.299723 + }, + { + "t": 0.44, + "precision": 0.1894, + "recall": 0.312349 + }, + { + "t": 0.42, + "precision": 0.184511, + "recall": 0.324083 + }, + { + "t": 0.4, + "precision": 0.180424, + "recall": 0.338003 + }, + { + "t": 0.38, + "precision": 0.176611, + "recall": 0.352592 + }, + { + "t": 0.36, + "precision": 0.172591, + "recall": 0.368029 + }, + { + "t": 0.34, + "precision": 0.168427, + "recall": 0.383376 + }, + { + "t": 0.32, + "precision": 0.164318, + "recall": 0.399081 + }, + { + "t": 0.3, + "precision": 0.160497, + "recall": 0.416302 + }, + { + "t": 0.28, + "precision": 0.156888, + "recall": 0.434594 + }, + { + "t": 0.26, + "precision": 0.152756, + "recall": 0.454002 + }, + { + "t": 0.24, + "precision": 0.14907, + "recall": 0.475105 + }, + { + "t": 0.22, + "precision": 0.145305, + "recall": 0.4971 + }, + { + "t": 0.2, + "precision": 0.140815, + "recall": 0.51914 + }, + { + "t": 0.18, + "precision": 0.136281, + "recall": 0.542875 + }, + { + "t": 0.16, + "precision": 0.13182, + "recall": 0.567279 + }, + { + "t": 0.14, + "precision": 0.126894, + "recall": 0.59329 + }, + { + "t": 0.12, + "precision": 0.121778, + "recall": 0.622022 + }, + { + "t": 0.1, + "precision": 0.116552, + "recall": 0.654769 + }, + { + "t": 0.08, + "precision": 0.111035, + "recall": 0.691666 + }, + { + "t": 0.06, + "precision": 0.105237, + "recall": 0.736281 + }, + { + "t": 0.04, + "precision": 0.098657, + "recall": 0.791113 + }, + { + "t": 0.02, + "precision": 0.090286, + "recall": 0.86437 + }, + { + "t": 0.0, + "precision": 0.065934, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w038-noise-baseline.json b/docs/examples/bench/w038-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..25f675df44a7108c56411c8ef58cc63950a992a1 --- /dev/null +++ b/docs/examples/bench/w038-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:01Z", + "scroll": "PHerc0139", + "segment": "w038", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w038_ink.png", + "sha256": "f4f7e94b9b7449a3a5861cbe309e4bdc22714cc18a1d5c15f12db410a7aac2eb", + "width": 640, + "height": 532, + "pxPerMM": 9.1888, + "alignTargetFile": "data/bench/w038_ink.png", + "alignTargetSha256": "f4f7e94b9b7449a3a5861cbe309e4bdc22714cc18a1d5c15f12db410a7aac2eb" + }, + "prediction": { + "name": "noise-seed38.png", + "sha256": "3a22aebca64a490384af889bbd18eaf007ffacf4868add7e4757b1c931e7c4fb", + "format": "png", + "width": 640, + "height": 532, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(38).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.001169, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 340480, + "validFraction": 1.0, + "pearson": 0.001169, + "blurSigmaPx": 0.0, + "blurSigmaUM": 0.0, + "pearsonMatchedBlur": 0.001169, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.061853, + "dice": 0.1165, + "precision": 0.065951, + "recall": 0.498885, + "refPositiveFraction": 0.065831, + "predPositiveFraction": 0.497976, + "ap": 0.065906, + "bestF1": { + "f1": 0.123554, + "threshold": 0.02 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.068249, + "recall": 0.020523 + }, + { + "t": 0.96, + "precision": 0.066096, + "recall": 0.039261 + }, + { + "t": 0.94, + "precision": 0.06606, + "recall": 0.058669 + }, + { + "t": 0.92, + "precision": 0.067089, + "recall": 0.079816 + }, + { + "t": 0.9, + "precision": 0.065494, + "recall": 0.097662 + }, + { + "t": 0.88, + "precision": 0.065208, + "recall": 0.116356 + }, + { + "t": 0.86, + "precision": 0.065641, + "recall": 0.136923 + }, + { + "t": 0.84, + "precision": 0.06475, + "recall": 0.154323 + }, + { + "t": 0.82, + "precision": 0.064124, + "recall": 0.171589 + }, + { + "t": 0.8, + "precision": 0.064593, + "recall": 0.195503 + }, + { + "t": 0.78, + "precision": 0.064833, + "recall": 0.215356 + }, + { + "t": 0.76, + "precision": 0.065486, + "recall": 0.237307 + }, + { + "t": 0.74, + "precision": 0.065623, + "recall": 0.257607 + }, + { + "t": 0.72, + "precision": 0.065345, + "recall": 0.275944 + }, + { + "t": 0.7, + "precision": 0.065648, + "recall": 0.2966 + }, + { + "t": 0.68, + "precision": 0.065436, + "recall": 0.315294 + }, + { + "t": 0.66, + "precision": 0.065534, + "recall": 0.335371 + }, + { + "t": 0.64, + "precision": 0.065686, + "recall": 0.35576 + }, + { + "t": 0.62, + "precision": 0.065786, + "recall": 0.376104 + }, + { + "t": 0.6, + "precision": 0.065766, + "recall": 0.399839 + }, + { + "t": 0.58, + "precision": 0.065748, + "recall": 0.419158 + }, + { + "t": 0.56, + "precision": 0.06571, + "recall": 0.438387 + }, + { + "t": 0.54, + "precision": 0.065915, + "recall": 0.459356 + }, + { + "t": 0.52, + "precision": 0.065911, + "recall": 0.478942 + }, + { + "t": 0.5, + "precision": 0.065951, + "recall": 0.498885 + }, + { + "t": 0.48, + "precision": 0.06597, + "recall": 0.518738 + }, + { + "t": 0.46, + "precision": 0.065903, + "recall": 0.537566 + }, + { + "t": 0.44, + "precision": 0.06623, + "recall": 0.560052 + }, + { + "t": 0.42, + "precision": 0.06635, + "recall": 0.580753 + }, + { + "t": 0.4, + "precision": 0.066318, + "recall": 0.603864 + }, + { + "t": 0.38, + "precision": 0.066367, + "recall": 0.624119 + }, + { + "t": 0.36, + "precision": 0.066475, + "recall": 0.644865 + }, + { + "t": 0.34, + "precision": 0.066363, + "recall": 0.663335 + }, + { + "t": 0.32, + "precision": 0.066253, + "recall": 0.682163 + }, + { + "t": 0.3, + "precision": 0.066223, + "recall": 0.701526 + }, + { + "t": 0.28, + "precision": 0.066275, + "recall": 0.722093 + }, + { + "t": 0.26, + "precision": 0.06614, + "recall": 0.740073 + }, + { + "t": 0.24, + "precision": 0.066171, + "recall": 0.760239 + }, + { + "t": 0.22, + "precision": 0.066258, + "recall": 0.780762 + }, + { + "t": 0.2, + "precision": 0.066157, + "recall": 0.803426 + }, + { + "t": 0.18, + "precision": 0.066155, + "recall": 0.823102 + }, + { + "t": 0.16, + "precision": 0.065971, + "recall": 0.840725 + }, + { + "t": 0.14, + "precision": 0.066017, + "recall": 0.860891 + }, + { + "t": 0.12, + "precision": 0.066108, + "recall": 0.881904 + }, + { + "t": 0.1, + "precision": 0.066012, + "recall": 0.900464 + }, + { + "t": 0.08, + "precision": 0.065921, + "recall": 0.919113 + }, + { + "t": 0.06, + "precision": 0.065842, + "recall": 0.937405 + }, + { + "t": 0.04, + "precision": 0.06592, + "recall": 0.958107 + }, + { + "t": 0.02, + "precision": 0.065943, + "recall": 0.977782 + }, + { + "t": 0.0, + "precision": 0.065831, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w039-ct-baseline.json b/docs/examples/bench/w039-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..b68a1cb8b73fdd4cba4159237b28086c088b89cc --- /dev/null +++ b/docs/examples/bench/w039-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:10Z", + "scroll": "PHerc0139", + "segment": "w039", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w039_ink.png", + "sha256": "d9afe3383eb0864335d648f02df8ce59eda640d6a1d3e79f365d06f7015dfadb", + "width": 577, + "height": 640, + "pxPerMM": 7.9839, + "alignTargetFile": "data/bench/w039_ink.png", + "alignTargetSha256": "d9afe3383eb0864335d648f02df8ce59eda640d6a1d3e79f365d06f7015dfadb" + }, + "prediction": { + "name": "w039_ct.png", + "sha256": "927048bd37ec6e3838b6e957fc0dfc4c569f096688b088d48fca0a0dda8c2dbc", + "format": "png", + "width": 577, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.468189, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 369280, + "validFraction": 1.0, + "pearson": 0.468189, + "blurSigmaPx": 6.0, + "blurSigmaUM": 751.5, + "pearsonMatchedBlur": 0.698414, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.070479, + "dice": 0.131677, + "precision": 0.075419, + "recall": 0.518286, + "refPositiveFraction": 0.042724, + "predPositiveFraction": 0.293601, + "ap": 0.078568, + "bestF1": { + "f1": 0.131677, + "threshold": 0.5 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.136095, + "recall": 0.001458 + }, + { + "t": 0.98, + "precision": 0.131387, + "recall": 0.003423 + }, + { + "t": 0.96, + "precision": 0.123783, + "recall": 0.005641 + }, + { + "t": 0.94, + "precision": 0.118644, + "recall": 0.00843 + }, + { + "t": 0.92, + "precision": 0.1188, + "recall": 0.012296 + }, + { + "t": 0.9, + "precision": 0.115924, + "recall": 0.017304 + }, + { + "t": 0.88, + "precision": 0.121258, + "recall": 0.025417 + }, + { + "t": 0.86, + "precision": 0.113873, + "recall": 0.032516 + }, + { + "t": 0.84, + "precision": 0.107753, + "recall": 0.04158 + }, + { + "t": 0.82, + "precision": 0.103235, + "recall": 0.052989 + }, + { + "t": 0.8, + "precision": 0.097983, + "recall": 0.068961 + }, + { + "t": 0.78, + "precision": 0.096363, + "recall": 0.086328 + }, + { + "t": 0.76, + "precision": 0.095607, + "recall": 0.108005 + }, + { + "t": 0.74, + "precision": 0.09343, + "recall": 0.13133 + }, + { + "t": 0.72, + "precision": 0.091776, + "recall": 0.157381 + }, + { + "t": 0.7, + "precision": 0.088509, + "recall": 0.183178 + }, + { + "t": 0.68, + "precision": 0.086064, + "recall": 0.211447 + }, + { + "t": 0.66, + "precision": 0.08316, + "recall": 0.238385 + }, + { + "t": 0.64, + "precision": 0.08174, + "recall": 0.271027 + }, + { + "t": 0.62, + "precision": 0.080667, + "recall": 0.304114 + }, + { + "t": 0.6, + "precision": 0.0794, + "recall": 0.345566 + }, + { + "t": 0.58, + "precision": 0.078356, + "recall": 0.380491 + }, + { + "t": 0.56, + "precision": 0.077666, + "recall": 0.416112 + }, + { + "t": 0.54, + "precision": 0.076858, + "recall": 0.450973 + }, + { + "t": 0.52, + "precision": 0.076033, + "recall": 0.484439 + }, + { + "t": 0.5, + "precision": 0.075419, + "recall": 0.518286 + }, + { + "t": 0.48, + "precision": 0.074575, + "recall": 0.549914 + }, + { + "t": 0.46, + "precision": 0.074017, + "recall": 0.583064 + }, + { + "t": 0.44, + "precision": 0.073442, + "recall": 0.613742 + }, + { + "t": 0.42, + "precision": 0.07305, + "recall": 0.644926 + }, + { + "t": 0.4, + "precision": 0.072333, + "recall": 0.6789 + }, + { + "t": 0.38, + "precision": 0.071944, + "recall": 0.707676 + }, + { + "t": 0.36, + "precision": 0.071548, + "recall": 0.734297 + }, + { + "t": 0.34, + "precision": 0.071069, + "recall": 0.758699 + }, + { + "t": 0.32, + "precision": 0.070758, + "recall": 0.782024 + }, + { + "t": 0.3, + "precision": 0.070591, + "recall": 0.804906 + }, + { + "t": 0.28, + "precision": 0.070198, + "recall": 0.824555 + }, + { + "t": 0.26, + "precision": 0.069897, + "recall": 0.842492 + }, + { + "t": 0.24, + "precision": 0.069746, + "recall": 0.860493 + }, + { + "t": 0.22, + "precision": 0.069595, + "recall": 0.877036 + }, + { + "t": 0.2, + "precision": 0.069443, + "recall": 0.895798 + }, + { + "t": 0.18, + "precision": 0.069374, + "recall": 0.910439 + }, + { + "t": 0.16, + "precision": 0.069135, + "recall": 0.921088 + }, + { + "t": 0.14, + "precision": 0.068943, + "recall": 0.930722 + }, + { + "t": 0.12, + "precision": 0.068781, + "recall": 0.939976 + }, + { + "t": 0.1, + "precision": 0.0687, + "recall": 0.949293 + }, + { + "t": 0.08, + "precision": 0.068566, + "recall": 0.957723 + }, + { + "t": 0.06, + "precision": 0.068359, + "recall": 0.964759 + }, + { + "t": 0.04, + "precision": 0.068219, + "recall": 0.972872 + }, + { + "t": 0.02, + "precision": 0.067932, + "recall": 0.980478 + }, + { + "t": 0.0, + "precision": 0.042724, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w039-ink9um.json b/docs/examples/bench/w039-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..da747db02aafe3a0e8949bbc81e7f51a736be56f --- /dev/null +++ b/docs/examples/bench/w039-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:09Z", + "scroll": "PHerc0139", + "segment": "w039", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w039_ink.png", + "sha256": "d9afe3383eb0864335d648f02df8ce59eda640d6a1d3e79f365d06f7015dfadb", + "width": 577, + "height": 640, + "pxPerMM": 7.9839, + "alignTargetFile": "data/bench/w039_ink.png", + "alignTargetSha256": "d9afe3383eb0864335d648f02df8ce59eda640d6a1d3e79f365d06f7015dfadb" + }, + "prediction": { + "name": "w039_gpu.png", + "sha256": "373ddb854a2af7236703c3e4ebb58b148c335230fed2bb3e1498e8136c2c2870", + "format": "png", + "width": 577, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.469, + "ty": 0.5, + "sx": 1.00026, + "sy": 1.00005, + "ncc": 0.431841, + "coarseNcc": 0.652317, + "evals": 38358, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 368064, + "validFraction": 0.996707, + "pearson": 0.431841, + "blurSigmaPx": 2.0, + "blurSigmaUM": 250.5, + "pearsonMatchedBlur": 0.472161, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.159298, + "dice": 0.274818, + "precision": 0.247898, + "recall": 0.308297, + "refPositiveFraction": 0.042865, + "predPositiveFraction": 0.053309, + "ap": 0.214128, + "bestF1": { + "f1": 0.27508, + "threshold": 0.52 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.714286, + "recall": 0.000951 + }, + { + "t": 0.98, + "precision": 0.595696, + "recall": 0.066679 + }, + { + "t": 0.96, + "precision": 0.54557, + "recall": 0.081574 + }, + { + "t": 0.94, + "precision": 0.52011, + "recall": 0.095899 + }, + { + "t": 0.92, + "precision": 0.492424, + "recall": 0.107118 + }, + { + "t": 0.9, + "precision": 0.469285, + "recall": 0.118147 + }, + { + "t": 0.88, + "precision": 0.453488, + "recall": 0.128542 + }, + { + "t": 0.86, + "precision": 0.435397, + "recall": 0.13862 + }, + { + "t": 0.84, + "precision": 0.419715, + "recall": 0.14762 + }, + { + "t": 0.82, + "precision": 0.405419, + "recall": 0.157444 + }, + { + "t": 0.8, + "precision": 0.394166, + "recall": 0.167015 + }, + { + "t": 0.78, + "precision": 0.379485, + "recall": 0.175635 + }, + { + "t": 0.76, + "precision": 0.367745, + "recall": 0.185143 + }, + { + "t": 0.74, + "precision": 0.355726, + "recall": 0.194334 + }, + { + "t": 0.72, + "precision": 0.34556, + "recall": 0.203017 + }, + { + "t": 0.7, + "precision": 0.337322, + "recall": 0.212651 + }, + { + "t": 0.68, + "precision": 0.327504, + "recall": 0.221969 + }, + { + "t": 0.66, + "precision": 0.315361, + "recall": 0.230589 + }, + { + "t": 0.64, + "precision": 0.305381, + "recall": 0.239589 + }, + { + "t": 0.62, + "precision": 0.296673, + "recall": 0.249287 + }, + { + "t": 0.6, + "precision": 0.286712, + "recall": 0.257654 + }, + { + "t": 0.58, + "precision": 0.279061, + "recall": 0.268112 + }, + { + "t": 0.56, + "precision": 0.271117, + "recall": 0.277493 + }, + { + "t": 0.54, + "precision": 0.262908, + "recall": 0.287887 + }, + { + "t": 0.52, + "precision": 0.255135, + "recall": 0.298409 + }, + { + "t": 0.5, + "precision": 0.247898, + "recall": 0.308297 + }, + { + "t": 0.48, + "precision": 0.240571, + "recall": 0.319389 + }, + { + "t": 0.46, + "precision": 0.233823, + "recall": 0.330735 + }, + { + "t": 0.44, + "precision": 0.226599, + "recall": 0.340939 + }, + { + "t": 0.42, + "precision": 0.219727, + "recall": 0.353426 + }, + { + "t": 0.4, + "precision": 0.213102, + "recall": 0.365976 + }, + { + "t": 0.38, + "precision": 0.206186, + "recall": 0.378145 + }, + { + "t": 0.36, + "precision": 0.200207, + "recall": 0.391836 + }, + { + "t": 0.34, + "precision": 0.193894, + "recall": 0.405781 + }, + { + "t": 0.32, + "precision": 0.187601, + "recall": 0.420042 + }, + { + "t": 0.3, + "precision": 0.180938, + "recall": 0.433669 + }, + { + "t": 0.28, + "precision": 0.174514, + "recall": 0.448945 + }, + { + "t": 0.26, + "precision": 0.168177, + "recall": 0.465805 + }, + { + "t": 0.24, + "precision": 0.162047, + "recall": 0.483552 + }, + { + "t": 0.22, + "precision": 0.155426, + "recall": 0.500856 + }, + { + "t": 0.2, + "precision": 0.14894, + "recall": 0.520758 + }, + { + "t": 0.18, + "precision": 0.142083, + "recall": 0.540217 + }, + { + "t": 0.16, + "precision": 0.136158, + "recall": 0.565317 + }, + { + "t": 0.14, + "precision": 0.130079, + "recall": 0.591684 + }, + { + "t": 0.12, + "precision": 0.123926, + "recall": 0.621411 + }, + { + "t": 0.1, + "precision": 0.117701, + "recall": 0.655638 + }, + { + "t": 0.08, + "precision": 0.110836, + "recall": 0.694302 + }, + { + "t": 0.06, + "precision": 0.103727, + "recall": 0.7426 + }, + { + "t": 0.04, + "precision": 0.095294, + "recall": 0.799455 + }, + { + "t": 0.02, + "precision": 0.084743, + "recall": 0.874881 + }, + { + "t": 0.0, + "precision": 0.042865, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w039-noise-baseline.json b/docs/examples/bench/w039-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..ffdecd39fab7dbed1444626e0e4d4aab5478d445 --- /dev/null +++ b/docs/examples/bench/w039-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:11Z", + "scroll": "PHerc0139", + "segment": "w039", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w039_ink.png", + "sha256": "d9afe3383eb0864335d648f02df8ce59eda640d6a1d3e79f365d06f7015dfadb", + "width": 577, + "height": 640, + "pxPerMM": 7.9839, + "alignTargetFile": "data/bench/w039_ink.png", + "alignTargetSha256": "d9afe3383eb0864335d648f02df8ce59eda640d6a1d3e79f365d06f7015dfadb" + }, + "prediction": { + "name": "noise-seed39.png", + "sha256": "cc088b44e16ee91379ceca61e4737a531ffe5911c0b89e88a246b5b75ef6dc59", + "format": "png", + "width": 577, + "height": 640, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(39).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": -0.001205, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 369280, + "validFraction": 1.0, + "pearson": -0.001205, + "blurSigmaPx": 0.0, + "blurSigmaUM": 0.0, + "pearsonMatchedBlur": -0.001205, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.041183, + "dice": 0.079109, + "precision": 0.042956, + "recall": 0.499525, + "refPositiveFraction": 0.042724, + "predPositiveFraction": 0.496826, + "ap": 0.042999, + "bestF1": { + "f1": 0.081946, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.041948, + "recall": 0.019269 + }, + { + "t": 0.96, + "precision": 0.043664, + "recall": 0.040185 + }, + { + "t": 0.94, + "precision": 0.043674, + "recall": 0.060151 + }, + { + "t": 0.92, + "precision": 0.043975, + "recall": 0.081194 + }, + { + "t": 0.9, + "precision": 0.043369, + "recall": 0.099956 + }, + { + "t": 0.88, + "precision": 0.043496, + "recall": 0.120112 + }, + { + "t": 0.86, + "precision": 0.044109, + "recall": 0.141979 + }, + { + "t": 0.84, + "precision": 0.043771, + "recall": 0.160423 + }, + { + "t": 0.82, + "precision": 0.043836, + "recall": 0.180833 + }, + { + "t": 0.8, + "precision": 0.043357, + "recall": 0.202637 + }, + { + "t": 0.78, + "precision": 0.043149, + "recall": 0.221588 + }, + { + "t": 0.76, + "precision": 0.043271, + "recall": 0.241871 + }, + { + "t": 0.74, + "precision": 0.043152, + "recall": 0.261013 + }, + { + "t": 0.72, + "precision": 0.043278, + "recall": 0.281676 + }, + { + "t": 0.7, + "precision": 0.043177, + "recall": 0.300437 + }, + { + "t": 0.68, + "precision": 0.043141, + "recall": 0.320086 + }, + { + "t": 0.66, + "precision": 0.042974, + "recall": 0.338784 + }, + { + "t": 0.64, + "precision": 0.042961, + "recall": 0.35818 + }, + { + "t": 0.62, + "precision": 0.043015, + "recall": 0.378399 + }, + { + "t": 0.6, + "precision": 0.043215, + "recall": 0.404133 + }, + { + "t": 0.58, + "precision": 0.042977, + "recall": 0.421246 + }, + { + "t": 0.56, + "precision": 0.042984, + "recall": 0.440832 + }, + { + "t": 0.54, + "precision": 0.043012, + "recall": 0.460861 + }, + { + "t": 0.52, + "precision": 0.042967, + "recall": 0.479812 + }, + { + "t": 0.5, + "precision": 0.042956, + "recall": 0.499525 + }, + { + "t": 0.48, + "precision": 0.042811, + "recall": 0.517779 + }, + { + "t": 0.46, + "precision": 0.042625, + "recall": 0.535019 + }, + { + "t": 0.44, + "precision": 0.042671, + "recall": 0.554985 + }, + { + "t": 0.42, + "precision": 0.042701, + "recall": 0.575268 + }, + { + "t": 0.4, + "precision": 0.042699, + "recall": 0.5991 + }, + { + "t": 0.38, + "precision": 0.042633, + "recall": 0.617735 + }, + { + "t": 0.36, + "precision": 0.042624, + "recall": 0.637193 + }, + { + "t": 0.34, + "precision": 0.042714, + "recall": 0.658554 + }, + { + "t": 0.32, + "precision": 0.04279, + "recall": 0.679597 + }, + { + "t": 0.3, + "precision": 0.042813, + "recall": 0.699373 + }, + { + "t": 0.28, + "precision": 0.042811, + "recall": 0.718704 + }, + { + "t": 0.26, + "precision": 0.04288, + "recall": 0.739304 + }, + { + "t": 0.24, + "precision": 0.04281, + "recall": 0.757495 + }, + { + "t": 0.22, + "precision": 0.04279, + "recall": 0.7767 + }, + { + "t": 0.2, + "precision": 0.042844, + "recall": 0.801039 + }, + { + "t": 0.18, + "precision": 0.042797, + "recall": 0.819611 + }, + { + "t": 0.16, + "precision": 0.042815, + "recall": 0.84002 + }, + { + "t": 0.14, + "precision": 0.04276, + "recall": 0.858782 + }, + { + "t": 0.12, + "precision": 0.042732, + "recall": 0.877987 + }, + { + "t": 0.1, + "precision": 0.042637, + "recall": 0.895608 + }, + { + "t": 0.08, + "precision": 0.042605, + "recall": 0.914369 + }, + { + "t": 0.06, + "precision": 0.04259, + "recall": 0.933891 + }, + { + "t": 0.04, + "precision": 0.042648, + "recall": 0.954681 + }, + { + "t": 0.02, + "precision": 0.042705, + "recall": 0.975661 + }, + { + "t": 0.0, + "precision": 0.042724, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w040-ct-baseline.json b/docs/examples/bench/w040-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..95223bae8ba0cf1eb6408f1ce586c08dd43d30fc --- /dev/null +++ b/docs/examples/bench/w040-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:22Z", + "scroll": "PHerc0139", + "segment": "w040", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w040_ink.png", + "sha256": "751f3da4d502a160af033e3addaf20e7d528b9165997d35d53904201dbb983eb", + "width": 640, + "height": 513, + "pxPerMM": 8.5665, + "alignTargetFile": "data/bench/w040_ink.png", + "alignTargetSha256": "751f3da4d502a160af033e3addaf20e7d528b9165997d35d53904201dbb983eb" + }, + "prediction": { + "name": "w040_ct.png", + "sha256": "df430900b97f8459ea94ad284960bf61051731aca00b48db73ca07264d216e76", + "format": "png", + "width": 640, + "height": 513, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.238015, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 328320, + "validFraction": 1.0, + "pearson": 0.238015, + "blurSigmaPx": 6.0, + "blurSigmaUM": 700.4, + "pearsonMatchedBlur": 0.377102, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.09466, + "dice": 0.17295, + "precision": 0.104065, + "recall": 0.511587, + "refPositiveFraction": 0.085167, + "predPositiveFraction": 0.418683, + "ap": 0.103626, + "bestF1": { + "f1": 0.182173, + "threshold": 0.26 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.154589, + "recall": 0.001144 + }, + { + "t": 0.98, + "precision": 0.121387, + "recall": 0.002253 + }, + { + "t": 0.96, + "precision": 0.120123, + "recall": 0.004184 + }, + { + "t": 0.94, + "precision": 0.112903, + "recall": 0.006509 + }, + { + "t": 0.92, + "precision": 0.107301, + "recall": 0.009513 + }, + { + "t": 0.9, + "precision": 0.112554, + "recall": 0.014877 + }, + { + "t": 0.88, + "precision": 0.110945, + "recall": 0.021172 + }, + { + "t": 0.86, + "precision": 0.107364, + "recall": 0.028467 + }, + { + "t": 0.84, + "precision": 0.10722, + "recall": 0.039303 + }, + { + "t": 0.82, + "precision": 0.10813, + "recall": 0.052464 + }, + { + "t": 0.8, + "precision": 0.106584, + "recall": 0.070345 + }, + { + "t": 0.78, + "precision": 0.105801, + "recall": 0.088513 + }, + { + "t": 0.76, + "precision": 0.106035, + "recall": 0.109649 + }, + { + "t": 0.74, + "precision": 0.10502, + "recall": 0.131536 + }, + { + "t": 0.72, + "precision": 0.104754, + "recall": 0.156498 + }, + { + "t": 0.7, + "precision": 0.104806, + "recall": 0.184357 + }, + { + "t": 0.68, + "precision": 0.104392, + "recall": 0.212074 + }, + { + "t": 0.66, + "precision": 0.104823, + "recall": 0.243509 + }, + { + "t": 0.64, + "precision": 0.104204, + "recall": 0.273013 + }, + { + "t": 0.62, + "precision": 0.103724, + "recall": 0.303877 + }, + { + "t": 0.6, + "precision": 0.104355, + "recall": 0.34554 + }, + { + "t": 0.58, + "precision": 0.104259, + "recall": 0.378228 + }, + { + "t": 0.56, + "precision": 0.104255, + "recall": 0.411594 + }, + { + "t": 0.54, + "precision": 0.103997, + "recall": 0.444103 + }, + { + "t": 0.52, + "precision": 0.10412, + "recall": 0.478757 + }, + { + "t": 0.5, + "precision": 0.104065, + "recall": 0.511587 + }, + { + "t": 0.48, + "precision": 0.10414, + "recall": 0.544596 + }, + { + "t": 0.46, + "precision": 0.104089, + "recall": 0.576926 + }, + { + "t": 0.44, + "precision": 0.104121, + "recall": 0.608969 + }, + { + "t": 0.42, + "precision": 0.104033, + "recall": 0.639082 + }, + { + "t": 0.4, + "precision": 0.103668, + "recall": 0.672699 + }, + { + "t": 0.38, + "precision": 0.103605, + "recall": 0.70113 + }, + { + "t": 0.36, + "precision": 0.103342, + "recall": 0.726414 + }, + { + "t": 0.34, + "precision": 0.103236, + "recall": 0.752021 + }, + { + "t": 0.32, + "precision": 0.103003, + "recall": 0.775088 + }, + { + "t": 0.3, + "precision": 0.102795, + "recall": 0.796974 + }, + { + "t": 0.28, + "precision": 0.102467, + "recall": 0.816716 + }, + { + "t": 0.26, + "precision": 0.102237, + "recall": 0.835205 + }, + { + "t": 0.24, + "precision": 0.10184, + "recall": 0.851584 + }, + { + "t": 0.22, + "precision": 0.101578, + "recall": 0.867427 + }, + { + "t": 0.2, + "precision": 0.101366, + "recall": 0.885559 + }, + { + "t": 0.18, + "precision": 0.101188, + "recall": 0.899471 + }, + { + "t": 0.16, + "precision": 0.100945, + "recall": 0.91163 + }, + { + "t": 0.14, + "precision": 0.100739, + "recall": 0.923181 + }, + { + "t": 0.12, + "precision": 0.100538, + "recall": 0.934089 + }, + { + "t": 0.1, + "precision": 0.10034, + "recall": 0.944532 + }, + { + "t": 0.08, + "precision": 0.100161, + "recall": 0.954903 + }, + { + "t": 0.06, + "precision": 0.099942, + "recall": 0.964201 + }, + { + "t": 0.04, + "precision": 0.099628, + "recall": 0.972784 + }, + { + "t": 0.02, + "precision": 0.099209, + "recall": 0.981546 + }, + { + "t": 0.0, + "precision": 0.085167, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w040-ink9um.json b/docs/examples/bench/w040-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..72778f6c750eb63bf6c8cc244daed65fdf4d46e6 --- /dev/null +++ b/docs/examples/bench/w040-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:22Z", + "scroll": "PHerc0139", + "segment": "w040", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w040_ink.png", + "sha256": "751f3da4d502a160af033e3addaf20e7d528b9165997d35d53904201dbb983eb", + "width": 640, + "height": 513, + "pxPerMM": 8.5665, + "alignTargetFile": "data/bench/w040_ink.png", + "alignTargetSha256": "751f3da4d502a160af033e3addaf20e7d528b9165997d35d53904201dbb983eb" + }, + "prediction": { + "name": "w040_gpu.png", + "sha256": "5114fe17084c037b12c0dd883d93e65b287bdabce02e19d22fc1bfa0da0fd518", + "format": "png", + "width": 640, + "height": 513, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.563, + "ty": 0.625, + "sx": 0.99998, + "sy": 1.00004, + "ncc": 0.465038, + "coarseNcc": 0.55896, + "evals": 34283, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 327168, + "validFraction": 0.996491, + "pearson": 0.465038, + "blurSigmaPx": 1.5, + "blurSigmaUM": 175.1, + "pearsonMatchedBlur": 0.494103, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.251501, + "dice": 0.401919, + "precision": 0.360665, + "recall": 0.45383, + "refPositiveFraction": 0.085467, + "predPositiveFraction": 0.107544, + "ap": 0.370893, + "bestF1": { + "f1": 0.406319, + "threshold": 0.58 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.832653, + "recall": 0.007296 + }, + { + "t": 0.98, + "precision": 0.708208, + "recall": 0.14348 + }, + { + "t": 0.96, + "precision": 0.678885, + "recall": 0.168908 + }, + { + "t": 0.94, + "precision": 0.65297, + "recall": 0.18754 + }, + { + "t": 0.92, + "precision": 0.63386, + "recall": 0.204992 + }, + { + "t": 0.9, + "precision": 0.617462, + "recall": 0.2208 + }, + { + "t": 0.88, + "precision": 0.599817, + "recall": 0.23489 + }, + { + "t": 0.86, + "precision": 0.584861, + "recall": 0.247586 + }, + { + "t": 0.84, + "precision": 0.566781, + "recall": 0.259817 + }, + { + "t": 0.82, + "precision": 0.551545, + "recall": 0.272656 + }, + { + "t": 0.8, + "precision": 0.539064, + "recall": 0.285495 + }, + { + "t": 0.78, + "precision": 0.527121, + "recall": 0.29819 + }, + { + "t": 0.76, + "precision": 0.510286, + "recall": 0.308705 + }, + { + "t": 0.74, + "precision": 0.49643, + "recall": 0.320757 + }, + { + "t": 0.72, + "precision": 0.484705, + "recall": 0.332058 + }, + { + "t": 0.7, + "precision": 0.472338, + "recall": 0.34411 + }, + { + "t": 0.68, + "precision": 0.459802, + "recall": 0.355482 + }, + { + "t": 0.66, + "precision": 0.447041, + "recall": 0.366032 + }, + { + "t": 0.64, + "precision": 0.436282, + "recall": 0.377226 + }, + { + "t": 0.62, + "precision": 0.424696, + "recall": 0.388563 + }, + { + "t": 0.6, + "precision": 0.412774, + "recall": 0.399149 + }, + { + "t": 0.58, + "precision": 0.402442, + "recall": 0.410271 + }, + { + "t": 0.56, + "precision": 0.391147, + "recall": 0.420607 + }, + { + "t": 0.54, + "precision": 0.380506, + "recall": 0.431264 + }, + { + "t": 0.52, + "precision": 0.370212, + "recall": 0.442779 + }, + { + "t": 0.5, + "precision": 0.360665, + "recall": 0.45383 + }, + { + "t": 0.48, + "precision": 0.350435, + "recall": 0.464845 + }, + { + "t": 0.46, + "precision": 0.340617, + "recall": 0.475538 + }, + { + "t": 0.44, + "precision": 0.331665, + "recall": 0.488127 + }, + { + "t": 0.42, + "precision": 0.322639, + "recall": 0.499893 + }, + { + "t": 0.4, + "precision": 0.313129, + "recall": 0.511587 + }, + { + "t": 0.38, + "precision": 0.30415, + "recall": 0.524176 + }, + { + "t": 0.36, + "precision": 0.294674, + "recall": 0.536657 + }, + { + "t": 0.34, + "precision": 0.285097, + "recall": 0.549925 + }, + { + "t": 0.32, + "precision": 0.276116, + "recall": 0.563944 + }, + { + "t": 0.3, + "precision": 0.267405, + "recall": 0.578714 + }, + { + "t": 0.28, + "precision": 0.258227, + "recall": 0.592983 + }, + { + "t": 0.26, + "precision": 0.250202, + "recall": 0.609613 + }, + { + "t": 0.24, + "precision": 0.2411, + "recall": 0.625098 + }, + { + "t": 0.22, + "precision": 0.232585, + "recall": 0.642515 + }, + { + "t": 0.2, + "precision": 0.223838, + "recall": 0.661219 + }, + { + "t": 0.18, + "precision": 0.214824, + "recall": 0.679959 + }, + { + "t": 0.16, + "precision": 0.205623, + "recall": 0.699628 + }, + { + "t": 0.14, + "precision": 0.196356, + "recall": 0.720764 + }, + { + "t": 0.12, + "precision": 0.186351, + "recall": 0.742543 + }, + { + "t": 0.1, + "precision": 0.176622, + "recall": 0.768114 + }, + { + "t": 0.08, + "precision": 0.165805, + "recall": 0.795544 + }, + { + "t": 0.06, + "precision": 0.154312, + "recall": 0.828017 + }, + { + "t": 0.04, + "precision": 0.141355, + "recall": 0.867678 + }, + { + "t": 0.02, + "precision": 0.125155, + "recall": 0.916315 + }, + { + "t": 0.0, + "precision": 0.085467, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w040-noise-baseline.json b/docs/examples/bench/w040-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..8ef32cf248ed5896c126ca8daa3c1331acd2cc65 --- /dev/null +++ b/docs/examples/bench/w040-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:23Z", + "scroll": "PHerc0139", + "segment": "w040", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w040_ink.png", + "sha256": "751f3da4d502a160af033e3addaf20e7d528b9165997d35d53904201dbb983eb", + "width": 640, + "height": 513, + "pxPerMM": 8.5665, + "alignTargetFile": "data/bench/w040_ink.png", + "alignTargetSha256": "751f3da4d502a160af033e3addaf20e7d528b9165997d35d53904201dbb983eb" + }, + "prediction": { + "name": "noise-seed40.png", + "sha256": "fd99358bd951da2fc723a2a4cb0a2f89a2fdb30616028c65a7d6cb59cd50ec45", + "format": "png", + "width": 640, + "height": 513, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(40).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 5.1e-05, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 328320, + "validFraction": 1.0, + "pearson": 5.1e-05, + "blurSigmaPx": 2.0, + "blurSigmaUM": 233.5, + "pearsonMatchedBlur": 0.001069, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.078836, + "dice": 0.146151, + "precision": 0.085568, + "recall": 0.500536, + "refPositiveFraction": 0.085167, + "predPositiveFraction": 0.498191, + "ap": 0.085301, + "bestF1": { + "f1": 0.156966, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.084618, + "recall": 0.019634 + }, + { + "t": 0.96, + "precision": 0.08279, + "recall": 0.038588 + }, + { + "t": 0.94, + "precision": 0.084248, + "recall": 0.059044 + }, + { + "t": 0.92, + "precision": 0.084202, + "recall": 0.078607 + }, + { + "t": 0.9, + "precision": 0.086133, + "recall": 0.09985 + }, + { + "t": 0.88, + "precision": 0.085455, + "recall": 0.119054 + }, + { + "t": 0.86, + "precision": 0.08585, + "recall": 0.139153 + }, + { + "t": 0.84, + "precision": 0.086546, + "recall": 0.160325 + }, + { + "t": 0.82, + "precision": 0.08554, + "recall": 0.17792 + }, + { + "t": 0.8, + "precision": 0.085841, + "recall": 0.202131 + }, + { + "t": 0.78, + "precision": 0.085911, + "recall": 0.222373 + }, + { + "t": 0.76, + "precision": 0.085772, + "recall": 0.241864 + }, + { + "t": 0.74, + "precision": 0.085621, + "recall": 0.26114 + }, + { + "t": 0.72, + "precision": 0.085336, + "recall": 0.279987 + }, + { + "t": 0.7, + "precision": 0.085684, + "recall": 0.301051 + }, + { + "t": 0.68, + "precision": 0.085328, + "recall": 0.319541 + }, + { + "t": 0.66, + "precision": 0.085457, + "recall": 0.339675 + }, + { + "t": 0.64, + "precision": 0.085494, + "recall": 0.359702 + }, + { + "t": 0.62, + "precision": 0.085099, + "recall": 0.377798 + }, + { + "t": 0.6, + "precision": 0.085368, + "recall": 0.402367 + }, + { + "t": 0.58, + "precision": 0.085588, + "recall": 0.422967 + }, + { + "t": 0.56, + "precision": 0.085623, + "recall": 0.442493 + }, + { + "t": 0.54, + "precision": 0.08576, + "recall": 0.462521 + }, + { + "t": 0.52, + "precision": 0.085642, + "recall": 0.481189 + }, + { + "t": 0.5, + "precision": 0.085568, + "recall": 0.500536 + }, + { + "t": 0.48, + "precision": 0.085548, + "recall": 0.520313 + }, + { + "t": 0.46, + "precision": 0.085521, + "recall": 0.539554 + }, + { + "t": 0.44, + "precision": 0.085499, + "recall": 0.559223 + }, + { + "t": 0.42, + "precision": 0.085589, + "recall": 0.579537 + }, + { + "t": 0.4, + "precision": 0.085557, + "recall": 0.602675 + }, + { + "t": 0.38, + "precision": 0.085485, + "recall": 0.621915 + }, + { + "t": 0.36, + "precision": 0.085351, + "recall": 0.640762 + }, + { + "t": 0.34, + "precision": 0.085341, + "recall": 0.660074 + }, + { + "t": 0.32, + "precision": 0.085128, + "recall": 0.678206 + }, + { + "t": 0.3, + "precision": 0.084976, + "recall": 0.696731 + }, + { + "t": 0.28, + "precision": 0.084881, + "recall": 0.715256 + }, + { + "t": 0.26, + "precision": 0.084919, + "recall": 0.735212 + }, + { + "t": 0.24, + "precision": 0.08494, + "recall": 0.75506 + }, + { + "t": 0.22, + "precision": 0.084876, + "recall": 0.773764 + }, + { + "t": 0.2, + "precision": 0.084979, + "recall": 0.798226 + }, + { + "t": 0.18, + "precision": 0.085059, + "recall": 0.81879 + }, + { + "t": 0.16, + "precision": 0.085102, + "recall": 0.838495 + }, + { + "t": 0.14, + "precision": 0.085118, + "recall": 0.858844 + }, + { + "t": 0.12, + "precision": 0.085089, + "recall": 0.878013 + }, + { + "t": 0.1, + "precision": 0.085174, + "recall": 0.898577 + }, + { + "t": 0.08, + "precision": 0.085267, + "recall": 0.919069 + }, + { + "t": 0.06, + "precision": 0.085213, + "recall": 0.938095 + }, + { + "t": 0.04, + "precision": 0.085223, + "recall": 0.957585 + }, + { + "t": 0.02, + "precision": 0.085175, + "recall": 0.976683 + }, + { + "t": 0.0, + "precision": 0.085167, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w041-ct-baseline.json b/docs/examples/bench/w041-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..a6397f65e7a40f0980c20db25784e23553f90428 --- /dev/null +++ b/docs/examples/bench/w041-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:30Z", + "scroll": "PHerc0139", + "segment": "w041", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w041_ink.png", + "sha256": "1c4d6c1754bfbf75c0244cdc7e980b96edaa850895a769a6f2129187275f2d2b", + "width": 640, + "height": 494, + "pxPerMM": 8.5242, + "alignTargetFile": "data/bench/w041_ink.png", + "alignTargetSha256": "1c4d6c1754bfbf75c0244cdc7e980b96edaa850895a769a6f2129187275f2d2b" + }, + "prediction": { + "name": "w041_ct.png", + "sha256": "b4e58718a814135c14e2f7d6a2c556c259e9eaac0f72c4eb0abe52b8a288940f", + "format": "png", + "width": 640, + "height": 494, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.21203, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 316160, + "validFraction": 1.0, + "pearson": 0.21203, + "blurSigmaPx": 6.0, + "blurSigmaUM": 703.9, + "pearsonMatchedBlur": 0.353717, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.108698, + "dice": 0.196082, + "precision": 0.121199, + "recall": 0.513115, + "refPositiveFraction": 0.104188, + "predPositiveFraction": 0.441096, + "ap": 0.12076, + "bestF1": { + "f1": 0.212527, + "threshold": 0.04 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.066667, + "recall": 0.000334 + }, + { + "t": 0.98, + "precision": 0.087576, + "recall": 0.001305 + }, + { + "t": 0.96, + "precision": 0.097849, + "recall": 0.002763 + }, + { + "t": 0.94, + "precision": 0.099678, + "recall": 0.004706 + }, + { + "t": 0.92, + "precision": 0.107233, + "recall": 0.007741 + }, + { + "t": 0.9, + "precision": 0.116648, + "recall": 0.012508 + }, + { + "t": 0.88, + "precision": 0.119757, + "recall": 0.018549 + }, + { + "t": 0.86, + "precision": 0.124061, + "recall": 0.02708 + }, + { + "t": 0.84, + "precision": 0.125493, + "recall": 0.037705 + }, + { + "t": 0.82, + "precision": 0.127186, + "recall": 0.051002 + }, + { + "t": 0.8, + "precision": 0.124071, + "recall": 0.067942 + }, + { + "t": 0.78, + "precision": 0.122296, + "recall": 0.085458 + }, + { + "t": 0.76, + "precision": 0.120877, + "recall": 0.105131 + }, + { + "t": 0.74, + "precision": 0.121178, + "recall": 0.129114 + }, + { + "t": 0.72, + "precision": 0.121808, + "recall": 0.15507 + }, + { + "t": 0.7, + "precision": 0.121275, + "recall": 0.181633 + }, + { + "t": 0.68, + "precision": 0.122663, + "recall": 0.213692 + }, + { + "t": 0.66, + "precision": 0.122542, + "recall": 0.244384 + }, + { + "t": 0.64, + "precision": 0.12259, + "recall": 0.276594 + }, + { + "t": 0.62, + "precision": 0.122486, + "recall": 0.309745 + }, + { + "t": 0.6, + "precision": 0.122553, + "recall": 0.349879 + }, + { + "t": 0.58, + "precision": 0.121572, + "recall": 0.381087 + }, + { + "t": 0.56, + "precision": 0.121398, + "recall": 0.414238 + }, + { + "t": 0.54, + "precision": 0.121118, + "recall": 0.446812 + }, + { + "t": 0.52, + "precision": 0.121311, + "recall": 0.480753 + }, + { + "t": 0.5, + "precision": 0.121199, + "recall": 0.513115 + }, + { + "t": 0.48, + "precision": 0.121243, + "recall": 0.5449 + }, + { + "t": 0.46, + "precision": 0.121155, + "recall": 0.574833 + }, + { + "t": 0.44, + "precision": 0.120996, + "recall": 0.603916 + }, + { + "t": 0.42, + "precision": 0.120837, + "recall": 0.631967 + }, + { + "t": 0.4, + "precision": 0.120782, + "recall": 0.665574 + }, + { + "t": 0.38, + "precision": 0.12064, + "recall": 0.691135 + }, + { + "t": 0.36, + "precision": 0.12073, + "recall": 0.716849 + }, + { + "t": 0.34, + "precision": 0.120596, + "recall": 0.739891 + }, + { + "t": 0.32, + "precision": 0.120604, + "recall": 0.762538 + }, + { + "t": 0.3, + "precision": 0.120565, + "recall": 0.783424 + }, + { + "t": 0.28, + "precision": 0.120422, + "recall": 0.802398 + }, + { + "t": 0.26, + "precision": 0.120641, + "recall": 0.822951 + }, + { + "t": 0.24, + "precision": 0.12057, + "recall": 0.839769 + }, + { + "t": 0.22, + "precision": 0.120501, + "recall": 0.855616 + }, + { + "t": 0.2, + "precision": 0.120334, + "recall": 0.872647 + }, + { + "t": 0.18, + "precision": 0.120209, + "recall": 0.885853 + }, + { + "t": 0.16, + "precision": 0.120186, + "recall": 0.898391 + }, + { + "t": 0.14, + "precision": 0.120135, + "recall": 0.910291 + }, + { + "t": 0.12, + "precision": 0.12, + "recall": 0.920583 + }, + { + "t": 0.1, + "precision": 0.119831, + "recall": 0.931724 + }, + { + "t": 0.08, + "precision": 0.119597, + "recall": 0.942228 + }, + { + "t": 0.06, + "precision": 0.119519, + "recall": 0.953613 + }, + { + "t": 0.04, + "precision": 0.11942, + "recall": 0.964542 + }, + { + "t": 0.02, + "precision": 0.11916, + "recall": 0.975683 + }, + { + "t": 0.0, + "precision": 0.104188, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w041-ink9um.json b/docs/examples/bench/w041-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..cf6fa770284cf7abae33b3e84d891bd488eeef1b --- /dev/null +++ b/docs/examples/bench/w041-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:30Z", + "scroll": "PHerc0139", + "segment": "w041", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w041_ink.png", + "sha256": "1c4d6c1754bfbf75c0244cdc7e980b96edaa850895a769a6f2129187275f2d2b", + "width": 640, + "height": 494, + "pxPerMM": 8.5242, + "alignTargetFile": "data/bench/w041_ink.png", + "alignTargetSha256": "1c4d6c1754bfbf75c0244cdc7e980b96edaa850895a769a6f2129187275f2d2b" + }, + "prediction": { + "name": "w041_gpu.png", + "sha256": "d57c948b18da6c3c15569b858ad0ee34af03c97bb25bf51d2959e123115127c5", + "format": "png", + "width": 640, + "height": 494, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.531, + "ty": 0.5, + "sx": 1.00026, + "sy": 1.00204, + "ncc": 0.439035, + "coarseNcc": 0.539996, + "evals": 32194, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 315666, + "validFraction": 0.998438, + "pearson": 0.439035, + "blurSigmaPx": 2.0, + "blurSigmaUM": 234.6, + "pearsonMatchedBlur": 0.476365, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.235314, + "dice": 0.380979, + "precision": 0.411251, + "recall": 0.354857, + "refPositiveFraction": 0.104351, + "predPositiveFraction": 0.090041, + "ap": 0.352437, + "bestF1": { + "f1": 0.387712, + "threshold": 0.42 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.843931, + "recall": 0.004432 + }, + { + "t": 0.98, + "precision": 0.712142, + "recall": 0.080662 + }, + { + "t": 0.96, + "precision": 0.690936, + "recall": 0.09997 + }, + { + "t": 0.94, + "precision": 0.669744, + "recall": 0.114511 + }, + { + "t": 0.92, + "precision": 0.650993, + "recall": 0.127353 + }, + { + "t": 0.9, + "precision": 0.629711, + "recall": 0.13898 + }, + { + "t": 0.88, + "precision": 0.617167, + "recall": 0.150395 + }, + { + "t": 0.86, + "precision": 0.601959, + "recall": 0.162295 + }, + { + "t": 0.84, + "precision": 0.589627, + "recall": 0.173953 + }, + { + "t": 0.82, + "precision": 0.577351, + "recall": 0.184335 + }, + { + "t": 0.8, + "precision": 0.565634, + "recall": 0.195173 + }, + { + "t": 0.78, + "precision": 0.551809, + "recall": 0.204675 + }, + { + "t": 0.76, + "precision": 0.540249, + "recall": 0.214542 + }, + { + "t": 0.74, + "precision": 0.530014, + "recall": 0.224621 + }, + { + "t": 0.72, + "precision": 0.520624, + "recall": 0.236035 + }, + { + "t": 0.7, + "precision": 0.509799, + "recall": 0.247177 + }, + { + "t": 0.68, + "precision": 0.499115, + "recall": 0.256922 + }, + { + "t": 0.66, + "precision": 0.488184, + "recall": 0.267152 + }, + { + "t": 0.64, + "precision": 0.477599, + "recall": 0.277353 + }, + { + "t": 0.62, + "precision": 0.467916, + "recall": 0.288009 + }, + { + "t": 0.6, + "precision": 0.457622, + "recall": 0.299302 + }, + { + "t": 0.58, + "precision": 0.44804, + "recall": 0.309806 + }, + { + "t": 0.56, + "precision": 0.439417, + "recall": 0.321038 + }, + { + "t": 0.54, + "precision": 0.429576, + "recall": 0.332028 + }, + { + "t": 0.52, + "precision": 0.420628, + "recall": 0.343564 + }, + { + "t": 0.5, + "precision": 0.411251, + "recall": 0.354857 + }, + { + "t": 0.48, + "precision": 0.401059, + "recall": 0.365726 + }, + { + "t": 0.46, + "precision": 0.391779, + "recall": 0.377899 + }, + { + "t": 0.44, + "precision": 0.382493, + "recall": 0.390528 + }, + { + "t": 0.42, + "precision": 0.373484, + "recall": 0.403066 + }, + { + "t": 0.4, + "precision": 0.363605, + "recall": 0.414845 + }, + { + "t": 0.38, + "precision": 0.354521, + "recall": 0.427201 + }, + { + "t": 0.36, + "precision": 0.345618, + "recall": 0.441287 + }, + { + "t": 0.34, + "precision": 0.336374, + "recall": 0.455677 + }, + { + "t": 0.32, + "precision": 0.327121, + "recall": 0.470492 + }, + { + "t": 0.3, + "precision": 0.318158, + "recall": 0.486703 + }, + { + "t": 0.28, + "precision": 0.308376, + "recall": 0.502398 + }, + { + "t": 0.26, + "precision": 0.298815, + "recall": 0.518245 + }, + { + "t": 0.24, + "precision": 0.288809, + "recall": 0.535823 + }, + { + "t": 0.22, + "precision": 0.279358, + "recall": 0.554432 + }, + { + "t": 0.2, + "precision": 0.269009, + "recall": 0.57377 + }, + { + "t": 0.18, + "precision": 0.259517, + "recall": 0.596266 + }, + { + "t": 0.16, + "precision": 0.249196, + "recall": 0.61864 + }, + { + "t": 0.14, + "precision": 0.238533, + "recall": 0.644262 + }, + { + "t": 0.12, + "precision": 0.227368, + "recall": 0.672435 + }, + { + "t": 0.1, + "precision": 0.215328, + "recall": 0.70343 + }, + { + "t": 0.08, + "precision": 0.201749, + "recall": 0.735337 + }, + { + "t": 0.06, + "precision": 0.187414, + "recall": 0.773892 + }, + { + "t": 0.04, + "precision": 0.17131, + "recall": 0.823406 + }, + { + "t": 0.02, + "precision": 0.150962, + "recall": 0.885883 + }, + { + "t": 0.0, + "precision": 0.104351, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w041-noise-baseline.json b/docs/examples/bench/w041-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..991243fa2e4beaf3b3f904671bc0571ebc5b5abb --- /dev/null +++ b/docs/examples/bench/w041-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:31Z", + "scroll": "PHerc0139", + "segment": "w041", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w041_ink.png", + "sha256": "1c4d6c1754bfbf75c0244cdc7e980b96edaa850895a769a6f2129187275f2d2b", + "width": 640, + "height": 494, + "pxPerMM": 8.5242, + "alignTargetFile": "data/bench/w041_ink.png", + "alignTargetSha256": "1c4d6c1754bfbf75c0244cdc7e980b96edaa850895a769a6f2129187275f2d2b" + }, + "prediction": { + "name": "noise-seed41.png", + "sha256": "7fe2a94453e8ec2f26d5e895476481ed54e1d6e202eadb63dabb89b9374b3772", + "format": "png", + "width": 640, + "height": 494, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(41).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": -0.000591, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 316160, + "validFraction": 1.0, + "pearson": -0.000591, + "blurSigmaPx": 0.5, + "blurSigmaUM": 58.7, + "pearsonMatchedBlur": -0.000572, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.09375, + "dice": 0.171429, + "precision": 0.103705, + "recall": 0.49408, + "refPositiveFraction": 0.104188, + "predPositiveFraction": 0.496378, + "ap": 0.104346, + "bestF1": { + "f1": 0.188714, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.100847, + "recall": 0.018427 + }, + { + "t": 0.96, + "precision": 0.106104, + "recall": 0.039314 + }, + { + "t": 0.94, + "precision": 0.105962, + "recall": 0.058865 + }, + { + "t": 0.92, + "precision": 0.105419, + "recall": 0.078658 + }, + { + "t": 0.9, + "precision": 0.106951, + "recall": 0.099818 + }, + { + "t": 0.88, + "precision": 0.106725, + "recall": 0.119763 + }, + { + "t": 0.86, + "precision": 0.106227, + "recall": 0.138737 + }, + { + "t": 0.84, + "precision": 0.106663, + "recall": 0.159441 + }, + { + "t": 0.82, + "precision": 0.105813, + "recall": 0.178203 + }, + { + "t": 0.8, + "precision": 0.105473, + "recall": 0.201488 + }, + { + "t": 0.78, + "precision": 0.105519, + "recall": 0.221676 + }, + { + "t": 0.76, + "precision": 0.10497, + "recall": 0.239739 + }, + { + "t": 0.74, + "precision": 0.104981, + "recall": 0.25935 + }, + { + "t": 0.72, + "precision": 0.10477, + "recall": 0.278567 + }, + { + "t": 0.7, + "precision": 0.104306, + "recall": 0.29663 + }, + { + "t": 0.68, + "precision": 0.104251, + "recall": 0.315999 + }, + { + "t": 0.66, + "precision": 0.104125, + "recall": 0.335003 + }, + { + "t": 0.64, + "precision": 0.104102, + "recall": 0.354675 + }, + { + "t": 0.62, + "precision": 0.104073, + "recall": 0.374256 + }, + { + "t": 0.6, + "precision": 0.103996, + "recall": 0.397116 + }, + { + "t": 0.58, + "precision": 0.103761, + "recall": 0.415877 + }, + { + "t": 0.56, + "precision": 0.103502, + "recall": 0.434851 + }, + { + "t": 0.54, + "precision": 0.103651, + "recall": 0.4551 + }, + { + "t": 0.52, + "precision": 0.103862, + "recall": 0.47544 + }, + { + "t": 0.5, + "precision": 0.103705, + "recall": 0.49408 + }, + { + "t": 0.48, + "precision": 0.103793, + "recall": 0.513934 + }, + { + "t": 0.46, + "precision": 0.103939, + "recall": 0.534092 + }, + { + "t": 0.44, + "precision": 0.103945, + "recall": 0.553977 + }, + { + "t": 0.42, + "precision": 0.103965, + "recall": 0.573953 + }, + { + "t": 0.4, + "precision": 0.103709, + "recall": 0.596478 + }, + { + "t": 0.38, + "precision": 0.103826, + "recall": 0.616849 + }, + { + "t": 0.36, + "precision": 0.103821, + "recall": 0.63646 + }, + { + "t": 0.34, + "precision": 0.103541, + "recall": 0.654675 + }, + { + "t": 0.32, + "precision": 0.10365, + "recall": 0.67456 + }, + { + "t": 0.3, + "precision": 0.10372, + "recall": 0.69408 + }, + { + "t": 0.28, + "precision": 0.103734, + "recall": 0.714086 + }, + { + "t": 0.26, + "precision": 0.103894, + "recall": 0.73476 + }, + { + "t": 0.24, + "precision": 0.103841, + "recall": 0.75425 + }, + { + "t": 0.22, + "precision": 0.103855, + "recall": 0.77371 + }, + { + "t": 0.2, + "precision": 0.103772, + "recall": 0.796873 + }, + { + "t": 0.18, + "precision": 0.103898, + "recall": 0.817456 + }, + { + "t": 0.16, + "precision": 0.104006, + "recall": 0.837401 + }, + { + "t": 0.14, + "precision": 0.103903, + "recall": 0.856193 + }, + { + "t": 0.12, + "precision": 0.103916, + "recall": 0.875622 + }, + { + "t": 0.1, + "precision": 0.104046, + "recall": 0.896205 + }, + { + "t": 0.08, + "precision": 0.10411, + "recall": 0.916363 + }, + { + "t": 0.06, + "precision": 0.104005, + "recall": 0.935216 + }, + { + "t": 0.04, + "precision": 0.103971, + "recall": 0.954797 + }, + { + "t": 0.02, + "precision": 0.104008, + "recall": 0.97459 + }, + { + "t": 0.0, + "precision": 0.104188, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w042-ct-baseline.json b/docs/examples/bench/w042-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..7cb4e005c2a52ba30741f8ece870729cb6f3536d --- /dev/null +++ b/docs/examples/bench/w042-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:39Z", + "scroll": "PHerc0139", + "segment": "w042", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w042_ink.png", + "sha256": "4b5e61422e078d3bd92ff699d72f0579536c4e44c77e9a6c168c6a01e56a6942", + "width": 640, + "height": 486, + "pxPerMM": 8.6756, + "alignTargetFile": "data/bench/w042_ink.png", + "alignTargetSha256": "4b5e61422e078d3bd92ff699d72f0579536c4e44c77e9a6c168c6a01e56a6942" + }, + "prediction": { + "name": "w042_ct.png", + "sha256": "0231a5f2b89abba26b3983869727145b140a3d3f0f04b86911c736249d0fb097", + "format": "png", + "width": 640, + "height": 486, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.208662, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 311040, + "validFraction": 1.0, + "pearson": 0.208662, + "blurSigmaPx": 6.0, + "blurSigmaUM": 691.6, + "pearsonMatchedBlur": 0.342498, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.11346, + "dice": 0.203798, + "precision": 0.127192, + "recall": 0.512421, + "refPositiveFraction": 0.106768, + "predPositiveFraction": 0.430138, + "ap": 0.128229, + "bestF1": { + "f1": 0.216027, + "threshold": 0.02 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.165714, + "recall": 0.000873 + }, + { + "t": 0.98, + "precision": 0.12959, + "recall": 0.001807 + }, + { + "t": 0.96, + "precision": 0.133407, + "recall": 0.003644 + }, + { + "t": 0.94, + "precision": 0.13657, + "recall": 0.006354 + }, + { + "t": 0.92, + "precision": 0.129032, + "recall": 0.009636 + }, + { + "t": 0.9, + "precision": 0.130814, + "recall": 0.014906 + }, + { + "t": 0.88, + "precision": 0.135538, + "recall": 0.022464 + }, + { + "t": 0.86, + "precision": 0.139472, + "recall": 0.032792 + }, + { + "t": 0.84, + "precision": 0.141801, + "recall": 0.045048 + }, + { + "t": 0.82, + "precision": 0.141088, + "recall": 0.05899 + }, + { + "t": 0.8, + "precision": 0.141692, + "recall": 0.080581 + }, + { + "t": 0.78, + "precision": 0.140063, + "recall": 0.100063 + }, + { + "t": 0.76, + "precision": 0.137786, + "recall": 0.121684 + }, + { + "t": 0.74, + "precision": 0.136004, + "recall": 0.145804 + }, + { + "t": 0.72, + "precision": 0.13528, + "recall": 0.172724 + }, + { + "t": 0.7, + "precision": 0.134202, + "recall": 0.200759 + }, + { + "t": 0.68, + "precision": 0.132626, + "recall": 0.228733 + }, + { + "t": 0.66, + "precision": 0.13241, + "recall": 0.260773 + }, + { + "t": 0.64, + "precision": 0.132655, + "recall": 0.294227 + }, + { + "t": 0.62, + "precision": 0.131631, + "recall": 0.325123 + }, + { + "t": 0.6, + "precision": 0.130921, + "recall": 0.363124 + }, + { + "t": 0.58, + "precision": 0.130077, + "recall": 0.393869 + }, + { + "t": 0.56, + "precision": 0.129221, + "recall": 0.424132 + }, + { + "t": 0.54, + "precision": 0.128403, + "recall": 0.453612 + }, + { + "t": 0.52, + "precision": 0.127818, + "recall": 0.483363 + }, + { + "t": 0.5, + "precision": 0.127192, + "recall": 0.512421 + }, + { + "t": 0.48, + "precision": 0.126373, + "recall": 0.539763 + }, + { + "t": 0.46, + "precision": 0.126139, + "recall": 0.568822 + }, + { + "t": 0.44, + "precision": 0.125993, + "recall": 0.597368 + }, + { + "t": 0.42, + "precision": 0.125313, + "recall": 0.622181 + }, + { + "t": 0.4, + "precision": 0.124823, + "recall": 0.652082 + }, + { + "t": 0.38, + "precision": 0.124587, + "recall": 0.676865 + }, + { + "t": 0.36, + "precision": 0.124368, + "recall": 0.700322 + }, + { + "t": 0.34, + "precision": 0.124045, + "recall": 0.722846 + }, + { + "t": 0.32, + "precision": 0.123737, + "recall": 0.743985 + }, + { + "t": 0.3, + "precision": 0.123573, + "recall": 0.764732 + }, + { + "t": 0.28, + "precision": 0.123351, + "recall": 0.783824 + }, + { + "t": 0.26, + "precision": 0.123321, + "recall": 0.802554 + }, + { + "t": 0.24, + "precision": 0.123304, + "recall": 0.820862 + }, + { + "t": 0.22, + "precision": 0.123226, + "recall": 0.837604 + }, + { + "t": 0.2, + "precision": 0.123201, + "recall": 0.856695 + }, + { + "t": 0.18, + "precision": 0.122973, + "recall": 0.869945 + }, + { + "t": 0.16, + "precision": 0.122732, + "recall": 0.882291 + }, + { + "t": 0.14, + "precision": 0.122701, + "recall": 0.895932 + }, + { + "t": 0.12, + "precision": 0.122551, + "recall": 0.908669 + }, + { + "t": 0.1, + "precision": 0.122244, + "recall": 0.920202 + }, + { + "t": 0.08, + "precision": 0.121818, + "recall": 0.931916 + }, + { + "t": 0.06, + "precision": 0.121693, + "recall": 0.945858 + }, + { + "t": 0.04, + "precision": 0.121676, + "recall": 0.960071 + }, + { + "t": 0.02, + "precision": 0.121483, + "recall": 0.974194 + }, + { + "t": 0.0, + "precision": 0.106768, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w042-ink9um.json b/docs/examples/bench/w042-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..b210c732eb37fd016b6472975788fa8793d07f1b --- /dev/null +++ b/docs/examples/bench/w042-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:38Z", + "scroll": "PHerc0139", + "segment": "w042", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w042_ink.png", + "sha256": "4b5e61422e078d3bd92ff699d72f0579536c4e44c77e9a6c168c6a01e56a6942", + "width": 640, + "height": 486, + "pxPerMM": 8.6756, + "alignTargetFile": "data/bench/w042_ink.png", + "alignTargetSha256": "4b5e61422e078d3bd92ff699d72f0579536c4e44c77e9a6c168c6a01e56a6942" + }, + "prediction": { + "name": "w042_gpu.png", + "sha256": "726d4bc78e3256dc7076c9eae37d342f3253a436ee49a0a8993372f6c9746d3b", + "format": "png", + "width": 640, + "height": 486, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.531, + "ty": 0.594, + "sx": 1.00029, + "sy": 0.99995, + "ncc": 0.432681, + "coarseNcc": 0.518679, + "evals": 32230, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 309915, + "validFraction": 0.996383, + "pearson": 0.432681, + "blurSigmaPx": 2.0, + "blurSigmaUM": 230.5, + "pearsonMatchedBlur": 0.467094, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.238625, + "dice": 0.385306, + "precision": 0.392835, + "recall": 0.37806, + "refPositiveFraction": 0.107155, + "predPositiveFraction": 0.103125, + "ap": 0.347659, + "bestF1": { + "f1": 0.390839, + "threshold": 0.4 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.875, + "recall": 0.002951 + }, + { + "t": 0.98, + "precision": 0.686517, + "recall": 0.080189 + }, + { + "t": 0.96, + "precision": 0.653711, + "recall": 0.100786 + }, + { + "t": 0.94, + "precision": 0.632469, + "recall": 0.116023 + }, + { + "t": 0.92, + "precision": 0.611877, + "recall": 0.129694 + }, + { + "t": 0.9, + "precision": 0.597004, + "recall": 0.143997 + }, + { + "t": 0.88, + "precision": 0.58461, + "recall": 0.157397 + }, + { + "t": 0.86, + "precision": 0.569084, + "recall": 0.169171 + }, + { + "t": 0.84, + "precision": 0.554232, + "recall": 0.180794 + }, + { + "t": 0.82, + "precision": 0.544252, + "recall": 0.192026 + }, + { + "t": 0.8, + "precision": 0.533086, + "recall": 0.203047 + }, + { + "t": 0.78, + "precision": 0.522671, + "recall": 0.215213 + }, + { + "t": 0.76, + "precision": 0.512796, + "recall": 0.226866 + }, + { + "t": 0.74, + "precision": 0.502699, + "recall": 0.238339 + }, + { + "t": 0.72, + "precision": 0.49204, + "recall": 0.248487 + }, + { + "t": 0.7, + "precision": 0.481695, + "recall": 0.259508 + }, + { + "t": 0.68, + "precision": 0.471627, + "recall": 0.271041 + }, + { + "t": 0.66, + "precision": 0.463765, + "recall": 0.283658 + }, + { + "t": 0.64, + "precision": 0.453726, + "recall": 0.294077 + }, + { + "t": 0.62, + "precision": 0.44392, + "recall": 0.30558 + }, + { + "t": 0.6, + "precision": 0.434894, + "recall": 0.317504 + }, + { + "t": 0.58, + "precision": 0.42559, + "recall": 0.329128 + }, + { + "t": 0.56, + "precision": 0.416703, + "recall": 0.341052 + }, + { + "t": 0.54, + "precision": 0.408601, + "recall": 0.35391 + }, + { + "t": 0.52, + "precision": 0.400693, + "recall": 0.365533 + }, + { + "t": 0.5, + "precision": 0.392835, + "recall": 0.37806 + }, + { + "t": 0.48, + "precision": 0.385651, + "recall": 0.390737 + }, + { + "t": 0.46, + "precision": 0.376294, + "recall": 0.402662 + }, + { + "t": 0.44, + "precision": 0.367606, + "recall": 0.415249 + }, + { + "t": 0.42, + "precision": 0.358369, + "recall": 0.427444 + }, + { + "t": 0.4, + "precision": 0.35032, + "recall": 0.441959 + }, + { + "t": 0.38, + "precision": 0.341226, + "recall": 0.455208 + }, + { + "t": 0.36, + "precision": 0.33257, + "recall": 0.468006 + }, + { + "t": 0.34, + "precision": 0.32444, + "recall": 0.48264 + }, + { + "t": 0.32, + "precision": 0.315846, + "recall": 0.497275 + }, + { + "t": 0.3, + "precision": 0.306916, + "recall": 0.51206 + }, + { + "t": 0.28, + "precision": 0.297616, + "recall": 0.526574 + }, + { + "t": 0.26, + "precision": 0.288901, + "recall": 0.543588 + }, + { + "t": 0.24, + "precision": 0.279628, + "recall": 0.559728 + }, + { + "t": 0.22, + "precision": 0.270328, + "recall": 0.578036 + }, + { + "t": 0.2, + "precision": 0.26133, + "recall": 0.598001 + }, + { + "t": 0.18, + "precision": 0.251926, + "recall": 0.61926 + }, + { + "t": 0.16, + "precision": 0.242002, + "recall": 0.640338 + }, + { + "t": 0.14, + "precision": 0.231595, + "recall": 0.663947 + }, + { + "t": 0.12, + "precision": 0.221368, + "recall": 0.691168 + }, + { + "t": 0.1, + "precision": 0.210139, + "recall": 0.720226 + }, + { + "t": 0.08, + "precision": 0.198147, + "recall": 0.752146 + }, + { + "t": 0.06, + "precision": 0.185372, + "recall": 0.790749 + }, + { + "t": 0.04, + "precision": 0.170453, + "recall": 0.834774 + }, + { + "t": 0.02, + "precision": 0.152542, + "recall": 0.896805 + }, + { + "t": 0.0, + "precision": 0.107155, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w042-noise-baseline.json b/docs/examples/bench/w042-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..7ee9fae8f1d44af9857a5ec2f797b4c0e3acf936 --- /dev/null +++ b/docs/examples/bench/w042-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:39Z", + "scroll": "PHerc0139", + "segment": "w042", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w042_ink.png", + "sha256": "4b5e61422e078d3bd92ff699d72f0579536c4e44c77e9a6c168c6a01e56a6942", + "width": 640, + "height": 486, + "pxPerMM": 8.6756, + "alignTargetFile": "data/bench/w042_ink.png", + "alignTargetSha256": "4b5e61422e078d3bd92ff699d72f0579536c4e44c77e9a6c168c6a01e56a6942" + }, + "prediction": { + "name": "noise-seed42.png", + "sha256": "26b3a2e3089a8b0a9336925c2e1ac0f3a05ca502722e051f887e246b6dfba010", + "format": "png", + "width": 640, + "height": 486, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(42).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.000754, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 311040, + "validFraction": 1.0, + "pearson": 0.000754, + "blurSigmaPx": 0.0, + "blurSigmaUM": 0.0, + "pearsonMatchedBlur": 0.000754, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.097221, + "dice": 0.177213, + "precision": 0.107594, + "recall": 0.502093, + "refPositiveFraction": 0.106768, + "predPositiveFraction": 0.498235, + "ap": 0.106525, + "bestF1": { + "f1": 0.192936, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.104762, + "recall": 0.019212 + }, + { + "t": 0.96, + "precision": 0.101375, + "recall": 0.037068 + }, + { + "t": 0.94, + "precision": 0.101199, + "recall": 0.055678 + }, + { + "t": 0.92, + "precision": 0.102613, + "recall": 0.075431 + }, + { + "t": 0.9, + "precision": 0.104561, + "recall": 0.096239 + }, + { + "t": 0.88, + "precision": 0.105577, + "recall": 0.116173 + }, + { + "t": 0.86, + "precision": 0.105419, + "recall": 0.135325 + }, + { + "t": 0.84, + "precision": 0.106063, + "recall": 0.155861 + }, + { + "t": 0.82, + "precision": 0.107027, + "recall": 0.176398 + }, + { + "t": 0.8, + "precision": 0.10643, + "recall": 0.199133 + }, + { + "t": 0.78, + "precision": 0.106292, + "recall": 0.218796 + }, + { + "t": 0.76, + "precision": 0.106214, + "recall": 0.238249 + }, + { + "t": 0.74, + "precision": 0.106874, + "recall": 0.259418 + }, + { + "t": 0.72, + "precision": 0.106822, + "recall": 0.27881 + }, + { + "t": 0.7, + "precision": 0.106684, + "recall": 0.298353 + }, + { + "t": 0.68, + "precision": 0.10705, + "recall": 0.318709 + }, + { + "t": 0.66, + "precision": 0.107324, + "recall": 0.338944 + }, + { + "t": 0.64, + "precision": 0.10692, + "recall": 0.357253 + }, + { + "t": 0.62, + "precision": 0.107217, + "recall": 0.378361 + }, + { + "t": 0.6, + "precision": 0.107208, + "recall": 0.40206 + }, + { + "t": 0.58, + "precision": 0.107339, + "recall": 0.422024 + }, + { + "t": 0.56, + "precision": 0.10748, + "recall": 0.44229 + }, + { + "t": 0.54, + "precision": 0.107468, + "recall": 0.462164 + }, + { + "t": 0.52, + "precision": 0.107556, + "recall": 0.482189 + }, + { + "t": 0.5, + "precision": 0.107594, + "recall": 0.502093 + }, + { + "t": 0.48, + "precision": 0.107504, + "recall": 0.521846 + }, + { + "t": 0.46, + "precision": 0.107249, + "recall": 0.539974 + }, + { + "t": 0.44, + "precision": 0.107219, + "recall": 0.559216 + }, + { + "t": 0.42, + "precision": 0.107264, + "recall": 0.57915 + }, + { + "t": 0.4, + "precision": 0.107089, + "recall": 0.602096 + }, + { + "t": 0.38, + "precision": 0.107097, + "recall": 0.621639 + }, + { + "t": 0.36, + "precision": 0.106849, + "recall": 0.639887 + }, + { + "t": 0.34, + "precision": 0.106853, + "recall": 0.65961 + }, + { + "t": 0.32, + "precision": 0.106829, + "recall": 0.678641 + }, + { + "t": 0.3, + "precision": 0.106855, + "recall": 0.698244 + }, + { + "t": 0.28, + "precision": 0.106856, + "recall": 0.717998 + }, + { + "t": 0.26, + "precision": 0.106975, + "recall": 0.738264 + }, + { + "t": 0.24, + "precision": 0.106958, + "recall": 0.757746 + }, + { + "t": 0.22, + "precision": 0.106984, + "recall": 0.777771 + }, + { + "t": 0.2, + "precision": 0.106984, + "recall": 0.80171 + }, + { + "t": 0.18, + "precision": 0.106932, + "recall": 0.820771 + }, + { + "t": 0.16, + "precision": 0.106834, + "recall": 0.840073 + }, + { + "t": 0.14, + "precision": 0.106753, + "recall": 0.859375 + }, + { + "t": 0.12, + "precision": 0.10674, + "recall": 0.878647 + }, + { + "t": 0.1, + "precision": 0.106774, + "recall": 0.898612 + }, + { + "t": 0.08, + "precision": 0.106935, + "recall": 0.919871 + }, + { + "t": 0.06, + "precision": 0.106946, + "recall": 0.939565 + }, + { + "t": 0.04, + "precision": 0.10693, + "recall": 0.958656 + }, + { + "t": 0.02, + "precision": 0.106895, + "recall": 0.977898 + }, + { + "t": 0.0, + "precision": 0.106768, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w043-ct-baseline.json b/docs/examples/bench/w043-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..b9efc2fda8022a8092f24581c97088c2a1e8e2e9 --- /dev/null +++ b/docs/examples/bench/w043-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:49Z", + "scroll": "PHerc0139", + "segment": "w043", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w043_ink.png", + "sha256": "9167929c29941b793ac150cfc16d18f84a239f3eb5c8d28685c699402467f775", + "width": 640, + "height": 481, + "pxPerMM": 8.4188, + "alignTargetFile": "data/bench/w043_ink.png", + "alignTargetSha256": "9167929c29941b793ac150cfc16d18f84a239f3eb5c8d28685c699402467f775" + }, + "prediction": { + "name": "w043_ct.png", + "sha256": "8fe83037cdab7f905bcd55d583950e71956d8729576151cd6e93b9b5610a8a36", + "format": "png", + "width": 640, + "height": 481, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.238596, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 307840, + "validFraction": 1.0, + "pearson": 0.238596, + "blurSigmaPx": 6.0, + "blurSigmaUM": 712.7, + "pearsonMatchedBlur": 0.366188, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.153044, + "dice": 0.265461, + "precision": 0.173422, + "recall": 0.565681, + "refPositiveFraction": 0.14234, + "predPositiveFraction": 0.464296, + "ap": 0.178446, + "bestF1": { + "f1": 0.278645, + "threshold": 0.08 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.172043, + "recall": 0.00073 + }, + { + "t": 0.98, + "precision": 0.188596, + "recall": 0.001963 + }, + { + "t": 0.96, + "precision": 0.205306, + "recall": 0.004062 + }, + { + "t": 0.94, + "precision": 0.199324, + "recall": 0.006732 + }, + { + "t": 0.92, + "precision": 0.205549, + "recall": 0.01116 + }, + { + "t": 0.9, + "precision": 0.206504, + "recall": 0.016957 + }, + { + "t": 0.88, + "precision": 0.203134, + "recall": 0.023963 + }, + { + "t": 0.86, + "precision": 0.2044, + "recall": 0.034347 + }, + { + "t": 0.84, + "precision": 0.205779, + "recall": 0.046967 + }, + { + "t": 0.82, + "precision": 0.207139, + "recall": 0.063832 + }, + { + "t": 0.8, + "precision": 0.20246, + "recall": 0.086403 + }, + { + "t": 0.78, + "precision": 0.20134, + "recall": 0.109749 + }, + { + "t": 0.76, + "precision": 0.199738, + "recall": 0.135584 + }, + { + "t": 0.74, + "precision": 0.197706, + "recall": 0.16443 + }, + { + "t": 0.72, + "precision": 0.194127, + "recall": 0.19371 + }, + { + "t": 0.7, + "precision": 0.190955, + "recall": 0.224908 + }, + { + "t": 0.68, + "precision": 0.188991, + "recall": 0.258273 + }, + { + "t": 0.66, + "precision": 0.186351, + "recall": 0.291205 + }, + { + "t": 0.64, + "precision": 0.184353, + "recall": 0.325893 + }, + { + "t": 0.62, + "precision": 0.182123, + "recall": 0.360194 + }, + { + "t": 0.6, + "precision": 0.180451, + "recall": 0.402483 + }, + { + "t": 0.58, + "precision": 0.17882, + "recall": 0.436076 + }, + { + "t": 0.56, + "precision": 0.176936, + "recall": 0.468095 + }, + { + "t": 0.54, + "precision": 0.1756, + "recall": 0.500959 + }, + { + "t": 0.52, + "precision": 0.174481, + "recall": 0.53405 + }, + { + "t": 0.5, + "precision": 0.173422, + "recall": 0.565681 + }, + { + "t": 0.48, + "precision": 0.172473, + "recall": 0.596787 + }, + { + "t": 0.46, + "precision": 0.171505, + "recall": 0.626295 + }, + { + "t": 0.44, + "precision": 0.170698, + "recall": 0.654708 + }, + { + "t": 0.42, + "precision": 0.169925, + "recall": 0.682071 + }, + { + "t": 0.4, + "precision": 0.169385, + "recall": 0.714615 + }, + { + "t": 0.38, + "precision": 0.168684, + "recall": 0.738715 + }, + { + "t": 0.36, + "precision": 0.168055, + "recall": 0.76124 + }, + { + "t": 0.34, + "precision": 0.167463, + "recall": 0.782761 + }, + { + "t": 0.32, + "precision": 0.166719, + "recall": 0.801725 + }, + { + "t": 0.3, + "precision": 0.165958, + "recall": 0.81891 + }, + { + "t": 0.28, + "precision": 0.16537, + "recall": 0.83573 + }, + { + "t": 0.26, + "precision": 0.165065, + "recall": 0.851887 + }, + { + "t": 0.24, + "precision": 0.164817, + "recall": 0.867246 + }, + { + "t": 0.22, + "precision": 0.164519, + "recall": 0.880939 + }, + { + "t": 0.2, + "precision": 0.164379, + "recall": 0.896823 + }, + { + "t": 0.18, + "precision": 0.163956, + "recall": 0.907435 + }, + { + "t": 0.16, + "precision": 0.163821, + "recall": 0.91928 + }, + { + "t": 0.14, + "precision": 0.163666, + "recall": 0.929572 + }, + { + "t": 0.12, + "precision": 0.16332, + "recall": 0.938108 + }, + { + "t": 0.1, + "precision": 0.163167, + "recall": 0.947738 + }, + { + "t": 0.08, + "precision": 0.163054, + "recall": 0.957255 + }, + { + "t": 0.06, + "precision": 0.162582, + "recall": 0.964877 + }, + { + "t": 0.04, + "precision": 0.162279, + "recall": 0.972888 + }, + { + "t": 0.02, + "precision": 0.161708, + "recall": 0.981035 + }, + { + "t": 0.0, + "precision": 0.14234, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w043-ink9um.json b/docs/examples/bench/w043-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..9299d3509bab910c82d4290c18a859e20b4e95d7 --- /dev/null +++ b/docs/examples/bench/w043-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:49Z", + "scroll": "PHerc0139", + "segment": "w043", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w043_ink.png", + "sha256": "9167929c29941b793ac150cfc16d18f84a239f3eb5c8d28685c699402467f775", + "width": 640, + "height": 481, + "pxPerMM": 8.4188, + "alignTargetFile": "data/bench/w043_ink.png", + "alignTargetSha256": "9167929c29941b793ac150cfc16d18f84a239f3eb5c8d28685c699402467f775" + }, + "prediction": { + "name": "w043_gpu.png", + "sha256": "b92f6007d57a05b7c08958c60b5070e70f895a445a12f740f2ec8165d4be42ea", + "format": "png", + "width": 640, + "height": 481, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.75, + "ty": 0.469, + "sx": 1.0019, + "sy": 1.00196, + "ncc": 0.448243, + "coarseNcc": 0.548527, + "evals": 32200, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 307359, + "validFraction": 0.998438, + "pearson": 0.448243, + "blurSigmaPx": 2.0, + "blurSigmaUM": 237.6, + "pearsonMatchedBlur": 0.479192, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.263579, + "dice": 0.417194, + "precision": 0.414571, + "recall": 0.41985, + "refPositiveFraction": 0.142563, + "predPositiveFraction": 0.144378, + "ap": 0.397018, + "bestF1": { + "f1": 0.424172, + "threshold": 0.38 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.943089, + "recall": 0.002647 + }, + { + "t": 0.98, + "precision": 0.708305, + "recall": 0.09382 + }, + { + "t": 0.96, + "precision": 0.677461, + "recall": 0.118399 + }, + { + "t": 0.94, + "precision": 0.655273, + "recall": 0.138254 + }, + { + "t": 0.92, + "precision": 0.638566, + "recall": 0.155233 + }, + { + "t": 0.9, + "precision": 0.623154, + "recall": 0.171368 + }, + { + "t": 0.88, + "precision": 0.606343, + "recall": 0.185426 + }, + { + "t": 0.86, + "precision": 0.59157, + "recall": 0.198594 + }, + { + "t": 0.84, + "precision": 0.577191, + "recall": 0.211603 + }, + { + "t": 0.82, + "precision": 0.564249, + "recall": 0.223675 + }, + { + "t": 0.8, + "precision": 0.552127, + "recall": 0.236045 + }, + { + "t": 0.78, + "precision": 0.541194, + "recall": 0.248254 + }, + { + "t": 0.76, + "precision": 0.529849, + "recall": 0.260281 + }, + { + "t": 0.74, + "precision": 0.519057, + "recall": 0.272879 + }, + { + "t": 0.72, + "precision": 0.509434, + "recall": 0.28591 + }, + { + "t": 0.7, + "precision": 0.499981, + "recall": 0.298553 + }, + { + "t": 0.68, + "precision": 0.490096, + "recall": 0.310557 + }, + { + "t": 0.66, + "precision": 0.481864, + "recall": 0.322881 + }, + { + "t": 0.64, + "precision": 0.473122, + "recall": 0.334634 + }, + { + "t": 0.62, + "precision": 0.464079, + "recall": 0.346593 + }, + { + "t": 0.6, + "precision": 0.455598, + "recall": 0.359327 + }, + { + "t": 0.58, + "precision": 0.447075, + "recall": 0.371012 + }, + { + "t": 0.56, + "precision": 0.439348, + "recall": 0.383199 + }, + { + "t": 0.54, + "precision": 0.430341, + "recall": 0.394906 + }, + { + "t": 0.52, + "precision": 0.422707, + "recall": 0.407686 + }, + { + "t": 0.5, + "precision": 0.414571, + "recall": 0.41985 + }, + { + "t": 0.48, + "precision": 0.406662, + "recall": 0.432425 + }, + { + "t": 0.46, + "precision": 0.398701, + "recall": 0.445662 + }, + { + "t": 0.44, + "precision": 0.390721, + "recall": 0.458944 + }, + { + "t": 0.42, + "precision": 0.382263, + "recall": 0.47177 + }, + { + "t": 0.4, + "precision": 0.374829, + "recall": 0.486855 + }, + { + "t": 0.38, + "precision": 0.367342, + "recall": 0.501803 + }, + { + "t": 0.36, + "precision": 0.359401, + "recall": 0.515176 + }, + { + "t": 0.34, + "precision": 0.351684, + "recall": 0.530969 + }, + { + "t": 0.32, + "precision": 0.343609, + "recall": 0.54489 + }, + { + "t": 0.3, + "precision": 0.334959, + "recall": 0.559519 + }, + { + "t": 0.28, + "precision": 0.32665, + "recall": 0.575197 + }, + { + "t": 0.26, + "precision": 0.319374, + "recall": 0.593135 + }, + { + "t": 0.24, + "precision": 0.311558, + "recall": 0.611142 + }, + { + "t": 0.22, + "precision": 0.303437, + "recall": 0.630038 + }, + { + "t": 0.2, + "precision": 0.294917, + "recall": 0.649162 + }, + { + "t": 0.18, + "precision": 0.286353, + "recall": 0.669405 + }, + { + "t": 0.16, + "precision": 0.27779, + "recall": 0.690424 + }, + { + "t": 0.14, + "precision": 0.268931, + "recall": 0.714227 + }, + { + "t": 0.12, + "precision": 0.259519, + "recall": 0.739445 + }, + { + "t": 0.1, + "precision": 0.249489, + "recall": 0.766283 + }, + { + "t": 0.08, + "precision": 0.238095, + "recall": 0.794651 + }, + { + "t": 0.06, + "precision": 0.225835, + "recall": 0.827514 + }, + { + "t": 0.04, + "precision": 0.211803, + "recall": 0.867977 + }, + { + "t": 0.02, + "precision": 0.192824, + "recall": 0.913643 + }, + { + "t": 0.0, + "precision": 0.142563, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w043-noise-baseline.json b/docs/examples/bench/w043-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..663d27417519b89688c56180e64a89d293a25f68 --- /dev/null +++ b/docs/examples/bench/w043-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:53:50Z", + "scroll": "PHerc0139", + "segment": "w043", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w043_ink.png", + "sha256": "9167929c29941b793ac150cfc16d18f84a239f3eb5c8d28685c699402467f775", + "width": 640, + "height": 481, + "pxPerMM": 8.4188, + "alignTargetFile": "data/bench/w043_ink.png", + "alignTargetSha256": "9167929c29941b793ac150cfc16d18f84a239f3eb5c8d28685c699402467f775" + }, + "prediction": { + "name": "noise-seed43.png", + "sha256": "9aaf821a4849b935332137de1d96000c1b6a062f7187f04a6612c10acd458e1d", + "format": "png", + "width": 640, + "height": 481, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(43).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.001147, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 307840, + "validFraction": 1.0, + "pearson": 0.001147, + "blurSigmaPx": 0.5, + "blurSigmaUM": 59.4, + "pearsonMatchedBlur": 0.001162, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.124575, + "dice": 0.22155, + "precision": 0.142398, + "recall": 0.498813, + "refPositiveFraction": 0.14234, + "predPositiveFraction": 0.49861, + "ap": 0.143238, + "bestF1": { + "f1": 0.249208, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.148715, + "recall": 0.020608 + }, + { + "t": 0.96, + "precision": 0.146544, + "recall": 0.040303 + }, + { + "t": 0.94, + "precision": 0.145636, + "recall": 0.060477 + }, + { + "t": 0.92, + "precision": 0.145377, + "recall": 0.080127 + }, + { + "t": 0.9, + "precision": 0.145556, + "recall": 0.100575 + }, + { + "t": 0.88, + "precision": 0.145692, + "recall": 0.120635 + }, + { + "t": 0.86, + "precision": 0.1455, + "recall": 0.140376 + }, + { + "t": 0.84, + "precision": 0.144772, + "recall": 0.159752 + }, + { + "t": 0.82, + "precision": 0.145141, + "recall": 0.180246 + }, + { + "t": 0.8, + "precision": 0.144737, + "recall": 0.204094 + }, + { + "t": 0.78, + "precision": 0.144757, + "recall": 0.223858 + }, + { + "t": 0.76, + "precision": 0.1438, + "recall": 0.242092 + }, + { + "t": 0.74, + "precision": 0.143804, + "recall": 0.261924 + }, + { + "t": 0.72, + "precision": 0.143741, + "recall": 0.28146 + }, + { + "t": 0.7, + "precision": 0.14341, + "recall": 0.300949 + }, + { + "t": 0.68, + "precision": 0.14346, + "recall": 0.321192 + }, + { + "t": 0.66, + "precision": 0.14348, + "recall": 0.340362 + }, + { + "t": 0.64, + "precision": 0.143129, + "recall": 0.35919 + }, + { + "t": 0.62, + "precision": 0.143174, + "recall": 0.379022 + }, + { + "t": 0.6, + "precision": 0.142914, + "recall": 0.402004 + }, + { + "t": 0.58, + "precision": 0.142709, + "recall": 0.421037 + }, + { + "t": 0.56, + "precision": 0.142607, + "recall": 0.440504 + }, + { + "t": 0.54, + "precision": 0.142453, + "recall": 0.45972 + }, + { + "t": 0.52, + "precision": 0.142428, + "recall": 0.479392 + }, + { + "t": 0.5, + "precision": 0.142398, + "recall": 0.498813 + }, + { + "t": 0.48, + "precision": 0.142328, + "recall": 0.518463 + }, + { + "t": 0.46, + "precision": 0.142335, + "recall": 0.537519 + }, + { + "t": 0.44, + "precision": 0.142325, + "recall": 0.5571 + }, + { + "t": 0.42, + "precision": 0.142569, + "recall": 0.577229 + }, + { + "t": 0.4, + "precision": 0.14247, + "recall": 0.600415 + }, + { + "t": 0.38, + "precision": 0.142487, + "recall": 0.620088 + }, + { + "t": 0.36, + "precision": 0.142297, + "recall": 0.638824 + }, + { + "t": 0.34, + "precision": 0.141948, + "recall": 0.656488 + }, + { + "t": 0.32, + "precision": 0.142017, + "recall": 0.676526 + }, + { + "t": 0.3, + "precision": 0.141862, + "recall": 0.694988 + }, + { + "t": 0.28, + "precision": 0.141966, + "recall": 0.714935 + }, + { + "t": 0.26, + "precision": 0.141903, + "recall": 0.73431 + }, + { + "t": 0.24, + "precision": 0.142022, + "recall": 0.75437 + }, + { + "t": 0.22, + "precision": 0.142078, + "recall": 0.774271 + }, + { + "t": 0.2, + "precision": 0.142185, + "recall": 0.798211 + }, + { + "t": 0.18, + "precision": 0.142262, + "recall": 0.818111 + }, + { + "t": 0.16, + "precision": 0.14219, + "recall": 0.837761 + }, + { + "t": 0.14, + "precision": 0.142227, + "recall": 0.857684 + }, + { + "t": 0.12, + "precision": 0.142149, + "recall": 0.876831 + }, + { + "t": 0.1, + "precision": 0.14224, + "recall": 0.897166 + }, + { + "t": 0.08, + "precision": 0.142224, + "recall": 0.91702 + }, + { + "t": 0.06, + "precision": 0.142351, + "recall": 0.937423 + }, + { + "t": 0.04, + "precision": 0.142305, + "recall": 0.957004 + }, + { + "t": 0.02, + "precision": 0.142427, + "recall": 0.977178 + }, + { + "t": 0.0, + "precision": 0.14234, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w044-ct-baseline.json b/docs/examples/bench/w044-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..99bc2fb2199d3882f7ba54523d858aac8f45abef --- /dev/null +++ b/docs/examples/bench/w044-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:54:01Z", + "scroll": "PHerc0139", + "segment": "w044", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w044_ink.png", + "sha256": "f61355935663f133ef7d0c962fe28b2b470565e9aa4837b9b00956b699371ecc", + "width": 640, + "height": 474, + "pxPerMM": 8.3781, + "alignTargetFile": "data/bench/w044_ink.png", + "alignTargetSha256": "f61355935663f133ef7d0c962fe28b2b470565e9aa4837b9b00956b699371ecc" + }, + "prediction": { + "name": "w044_ct.png", + "sha256": "b537b3e4505f07bd871dfc6a680471ba88de93c354e0ef8c6d825c009039290c", + "format": "png", + "width": 640, + "height": 474, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.219483, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 303360, + "validFraction": 1.0, + "pearson": 0.219483, + "blurSigmaPx": 6.0, + "blurSigmaUM": 716.2, + "pearsonMatchedBlur": 0.33661, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.140652, + "dice": 0.246616, + "precision": 0.159041, + "recall": 0.54883, + "refPositiveFraction": 0.135657, + "predPositiveFraction": 0.468137, + "ap": 0.163658, + "bestF1": { + "f1": 0.264583, + "threshold": 0.04 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.266304, + "recall": 0.001191 + }, + { + "t": 0.98, + "precision": 0.219462, + "recall": 0.002576 + }, + { + "t": 0.96, + "precision": 0.212903, + "recall": 0.004811 + }, + { + "t": 0.94, + "precision": 0.212476, + "recall": 0.007946 + }, + { + "t": 0.92, + "precision": 0.202968, + "recall": 0.012296 + }, + { + "t": 0.9, + "precision": 0.202378, + "recall": 0.0182 + }, + { + "t": 0.88, + "precision": 0.19947, + "recall": 0.025612 + }, + { + "t": 0.86, + "precision": 0.197737, + "recall": 0.035672 + }, + { + "t": 0.84, + "precision": 0.193695, + "recall": 0.047481 + }, + { + "t": 0.82, + "precision": 0.191116, + "recall": 0.062207 + }, + { + "t": 0.8, + "precision": 0.187235, + "recall": 0.082546 + }, + { + "t": 0.78, + "precision": 0.181802, + "recall": 0.100746 + }, + { + "t": 0.76, + "precision": 0.17763, + "recall": 0.121789 + }, + { + "t": 0.74, + "precision": 0.174813, + "recall": 0.147134 + }, + { + "t": 0.72, + "precision": 0.17263, + "recall": 0.174811 + }, + { + "t": 0.7, + "precision": 0.169775, + "recall": 0.203096 + }, + { + "t": 0.68, + "precision": 0.169154, + "recall": 0.235803 + }, + { + "t": 0.66, + "precision": 0.167924, + "recall": 0.268875 + }, + { + "t": 0.64, + "precision": 0.166812, + "recall": 0.303064 + }, + { + "t": 0.62, + "precision": 0.164996, + "recall": 0.336816 + }, + { + "t": 0.6, + "precision": 0.163353, + "recall": 0.377955 + }, + { + "t": 0.58, + "precision": 0.162239, + "recall": 0.412509 + }, + { + "t": 0.56, + "precision": 0.161291, + "recall": 0.447161 + }, + { + "t": 0.54, + "precision": 0.160524, + "recall": 0.482225 + }, + { + "t": 0.52, + "precision": 0.159619, + "recall": 0.515297 + }, + { + "t": 0.5, + "precision": 0.159041, + "recall": 0.54883 + }, + { + "t": 0.48, + "precision": 0.158276, + "recall": 0.580419 + }, + { + "t": 0.46, + "precision": 0.158067, + "recall": 0.6132 + }, + { + "t": 0.44, + "precision": 0.157838, + "recall": 0.643695 + }, + { + "t": 0.42, + "precision": 0.157581, + "recall": 0.673244 + }, + { + "t": 0.4, + "precision": 0.156929, + "recall": 0.705878 + }, + { + "t": 0.38, + "precision": 0.156411, + "recall": 0.731125 + }, + { + "t": 0.36, + "precision": 0.156136, + "recall": 0.755911 + }, + { + "t": 0.34, + "precision": 0.1557, + "recall": 0.777926 + }, + { + "t": 0.32, + "precision": 0.155193, + "recall": 0.798265 + }, + { + "t": 0.3, + "precision": 0.154828, + "recall": 0.817607 + }, + { + "t": 0.28, + "precision": 0.154461, + "recall": 0.834957 + }, + { + "t": 0.26, + "precision": 0.154206, + "recall": 0.851846 + }, + { + "t": 0.24, + "precision": 0.154097, + "recall": 0.86798 + }, + { + "t": 0.22, + "precision": 0.153968, + "recall": 0.882609 + }, + { + "t": 0.2, + "precision": 0.153879, + "recall": 0.898452 + }, + { + "t": 0.18, + "precision": 0.153676, + "recall": 0.90963 + }, + { + "t": 0.16, + "precision": 0.153545, + "recall": 0.920443 + }, + { + "t": 0.14, + "precision": 0.153502, + "recall": 0.930382 + }, + { + "t": 0.12, + "precision": 0.153481, + "recall": 0.940053 + }, + { + "t": 0.1, + "precision": 0.153416, + "recall": 0.948631 + }, + { + "t": 0.08, + "precision": 0.153261, + "recall": 0.956431 + }, + { + "t": 0.06, + "precision": 0.153171, + "recall": 0.964425 + }, + { + "t": 0.04, + "precision": 0.153108, + "recall": 0.973027 + }, + { + "t": 0.02, + "precision": 0.152795, + "recall": 0.981022 + }, + { + "t": 0.0, + "precision": 0.135657, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w044-ink9um.json b/docs/examples/bench/w044-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..935d08c4cee7a0b921b28bf3131200f094f8ede7 --- /dev/null +++ b/docs/examples/bench/w044-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:54:00Z", + "scroll": "PHerc0139", + "segment": "w044", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w044_ink.png", + "sha256": "f61355935663f133ef7d0c962fe28b2b470565e9aa4837b9b00956b699371ecc", + "width": 640, + "height": 474, + "pxPerMM": 8.3781, + "alignTargetFile": "data/bench/w044_ink.png", + "alignTargetSha256": "f61355935663f133ef7d0c962fe28b2b470565e9aa4837b9b00956b699371ecc" + }, + "prediction": { + "name": "w044_gpu.png", + "sha256": "b5148db74ef0a5a4c705b7a6928656e36cbfaee0af8f42ed84bedb8df85199ae", + "format": "png", + "width": 640, + "height": 474, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.531, + "ty": 0.531, + "sx": 1.00036, + "sy": 1.00046, + "ncc": 0.454413, + "coarseNcc": 0.555729, + "evals": 32233, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 302247, + "validFraction": 0.996331, + "pearson": 0.454413, + "blurSigmaPx": 1.5, + "blurSigmaUM": 179.0, + "pearsonMatchedBlur": 0.483003, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.267864, + "dice": 0.422544, + "precision": 0.415906, + "recall": 0.429398, + "refPositiveFraction": 0.136157, + "predPositiveFraction": 0.140574, + "ap": 0.409529, + "bestF1": { + "f1": 0.426541, + "threshold": 0.42 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.924171, + "recall": 0.004738 + }, + { + "t": 0.98, + "precision": 0.764848, + "recall": 0.107647 + }, + { + "t": 0.96, + "precision": 0.721384, + "recall": 0.130173 + }, + { + "t": 0.94, + "precision": 0.694173, + "recall": 0.148203 + }, + { + "t": 0.92, + "precision": 0.671645, + "recall": 0.163925 + }, + { + "t": 0.9, + "precision": 0.65071, + "recall": 0.179355 + }, + { + "t": 0.88, + "precision": 0.632769, + "recall": 0.194445 + }, + { + "t": 0.86, + "precision": 0.619544, + "recall": 0.209365 + }, + { + "t": 0.84, + "precision": 0.603889, + "recall": 0.223386 + }, + { + "t": 0.82, + "precision": 0.58954, + "recall": 0.236386 + }, + { + "t": 0.8, + "precision": 0.574488, + "recall": 0.248317 + }, + { + "t": 0.78, + "precision": 0.559617, + "recall": 0.261172 + }, + { + "t": 0.76, + "precision": 0.546989, + "recall": 0.273953 + }, + { + "t": 0.74, + "precision": 0.535073, + "recall": 0.286565 + }, + { + "t": 0.72, + "precision": 0.522048, + "recall": 0.298034 + }, + { + "t": 0.7, + "precision": 0.510966, + "recall": 0.309674 + }, + { + "t": 0.68, + "precision": 0.501327, + "recall": 0.321386 + }, + { + "t": 0.66, + "precision": 0.491385, + "recall": 0.333341 + }, + { + "t": 0.64, + "precision": 0.481074, + "recall": 0.344665 + }, + { + "t": 0.62, + "precision": 0.472125, + "recall": 0.35786 + }, + { + "t": 0.6, + "precision": 0.461982, + "recall": 0.369839 + }, + { + "t": 0.58, + "precision": 0.452074, + "recall": 0.381868 + }, + { + "t": 0.56, + "precision": 0.442903, + "recall": 0.393288 + }, + { + "t": 0.54, + "precision": 0.434187, + "recall": 0.405268 + }, + { + "t": 0.52, + "precision": 0.425318, + "recall": 0.417442 + }, + { + "t": 0.5, + "precision": 0.415906, + "recall": 0.429398 + }, + { + "t": 0.48, + "precision": 0.408003, + "recall": 0.44303 + }, + { + "t": 0.46, + "precision": 0.400222, + "recall": 0.45586 + }, + { + "t": 0.44, + "precision": 0.391537, + "recall": 0.468326 + }, + { + "t": 0.42, + "precision": 0.382938, + "recall": 0.48135 + }, + { + "t": 0.4, + "precision": 0.374122, + "recall": 0.493233 + }, + { + "t": 0.38, + "precision": 0.365075, + "recall": 0.506038 + }, + { + "t": 0.36, + "precision": 0.356615, + "recall": 0.519598 + }, + { + "t": 0.34, + "precision": 0.348397, + "recall": 0.535028 + }, + { + "t": 0.32, + "precision": 0.339967, + "recall": 0.549996 + }, + { + "t": 0.3, + "precision": 0.331119, + "recall": 0.564382 + }, + { + "t": 0.28, + "precision": 0.322817, + "recall": 0.580833 + }, + { + "t": 0.26, + "precision": 0.314171, + "recall": 0.596384 + }, + { + "t": 0.24, + "precision": 0.305391, + "recall": 0.613661 + }, + { + "t": 0.22, + "precision": 0.296849, + "recall": 0.632469 + }, + { + "t": 0.2, + "precision": 0.287906, + "recall": 0.651131 + }, + { + "t": 0.18, + "precision": 0.278831, + "recall": 0.671178 + }, + { + "t": 0.16, + "precision": 0.269306, + "recall": 0.691639 + }, + { + "t": 0.14, + "precision": 0.259846, + "recall": 0.715039 + }, + { + "t": 0.12, + "precision": 0.250045, + "recall": 0.739752 + }, + { + "t": 0.1, + "precision": 0.239469, + "recall": 0.765582 + }, + { + "t": 0.08, + "precision": 0.228443, + "recall": 0.794912 + }, + { + "t": 0.06, + "precision": 0.215562, + "recall": 0.826938 + }, + { + "t": 0.04, + "precision": 0.201072, + "recall": 0.864481 + }, + { + "t": 0.02, + "precision": 0.18322, + "recall": 0.913275 + }, + { + "t": 0.0, + "precision": 0.136157, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w044-noise-baseline.json b/docs/examples/bench/w044-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..63064273fb407d07c3e3cd55ac33b53ff355b392 --- /dev/null +++ b/docs/examples/bench/w044-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:54:01Z", + "scroll": "PHerc0139", + "segment": "w044", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w044_ink.png", + "sha256": "f61355935663f133ef7d0c962fe28b2b470565e9aa4837b9b00956b699371ecc", + "width": 640, + "height": 474, + "pxPerMM": 8.3781, + "alignTargetFile": "data/bench/w044_ink.png", + "alignTargetSha256": "f61355935663f133ef7d0c962fe28b2b470565e9aa4837b9b00956b699371ecc" + }, + "prediction": { + "name": "noise-seed44.png", + "sha256": "8fe0db45189d2a6eec87cdb131fa1e278692663ec1c400e8160d7e768c6364ef", + "format": "png", + "width": 640, + "height": 474, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(44).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.000406, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 303360, + "validFraction": 1.0, + "pearson": 0.000406, + "blurSigmaPx": 6.0, + "blurSigmaUM": 716.2, + "pearsonMatchedBlur": 0.002128, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.11962, + "dice": 0.213679, + "precision": 0.135989, + "recall": 0.498433, + "refPositiveFraction": 0.135657, + "predPositiveFraction": 0.497218, + "ap": 0.135806, + "bestF1": { + "f1": 0.238905, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.135724, + "recall": 0.019683 + }, + { + "t": 0.96, + "precision": 0.133929, + "recall": 0.038952 + }, + { + "t": 0.94, + "precision": 0.136683, + "recall": 0.059631 + }, + { + "t": 0.92, + "precision": 0.136714, + "recall": 0.079265 + }, + { + "t": 0.9, + "precision": 0.135895, + "recall": 0.098729 + }, + { + "t": 0.88, + "precision": 0.136019, + "recall": 0.11863 + }, + { + "t": 0.86, + "precision": 0.135952, + "recall": 0.137924 + }, + { + "t": 0.84, + "precision": 0.136396, + "recall": 0.157923 + }, + { + "t": 0.82, + "precision": 0.136345, + "recall": 0.17763 + }, + { + "t": 0.8, + "precision": 0.135275, + "recall": 0.199985 + }, + { + "t": 0.78, + "precision": 0.135081, + "recall": 0.219425 + }, + { + "t": 0.76, + "precision": 0.135585, + "recall": 0.239983 + }, + { + "t": 0.74, + "precision": 0.13608, + "recall": 0.260613 + }, + { + "t": 0.72, + "precision": 0.135483, + "recall": 0.279178 + }, + { + "t": 0.7, + "precision": 0.135934, + "recall": 0.299614 + }, + { + "t": 0.68, + "precision": 0.135525, + "recall": 0.318081 + }, + { + "t": 0.66, + "precision": 0.136005, + "recall": 0.338566 + }, + { + "t": 0.64, + "precision": 0.135953, + "recall": 0.357957 + }, + { + "t": 0.62, + "precision": 0.13583, + "recall": 0.377178 + }, + { + "t": 0.6, + "precision": 0.136235, + "recall": 0.402231 + }, + { + "t": 0.58, + "precision": 0.136265, + "recall": 0.422108 + }, + { + "t": 0.56, + "precision": 0.135838, + "recall": 0.440114 + }, + { + "t": 0.54, + "precision": 0.135765, + "recall": 0.45931 + }, + { + "t": 0.52, + "precision": 0.135734, + "recall": 0.478507 + }, + { + "t": 0.5, + "precision": 0.135989, + "recall": 0.498433 + }, + { + "t": 0.48, + "precision": 0.135833, + "recall": 0.517411 + }, + { + "t": 0.46, + "precision": 0.135804, + "recall": 0.537263 + }, + { + "t": 0.44, + "precision": 0.135911, + "recall": 0.557481 + }, + { + "t": 0.42, + "precision": 0.136038, + "recall": 0.577868 + }, + { + "t": 0.4, + "precision": 0.136177, + "recall": 0.602168 + }, + { + "t": 0.38, + "precision": 0.135913, + "recall": 0.620902 + }, + { + "t": 0.36, + "precision": 0.135954, + "recall": 0.640755 + }, + { + "t": 0.34, + "precision": 0.135986, + "recall": 0.660584 + }, + { + "t": 0.32, + "precision": 0.135838, + "recall": 0.67927 + }, + { + "t": 0.3, + "precision": 0.135749, + "recall": 0.698467 + }, + { + "t": 0.28, + "precision": 0.135635, + "recall": 0.717639 + }, + { + "t": 0.26, + "precision": 0.135427, + "recall": 0.73618 + }, + { + "t": 0.24, + "precision": 0.135464, + "recall": 0.756057 + }, + { + "t": 0.22, + "precision": 0.135455, + "recall": 0.775375 + }, + { + "t": 0.2, + "precision": 0.135461, + "recall": 0.798775 + }, + { + "t": 0.18, + "precision": 0.135656, + "recall": 0.819551 + }, + { + "t": 0.16, + "precision": 0.135723, + "recall": 0.839307 + }, + { + "t": 0.14, + "precision": 0.135828, + "recall": 0.859257 + }, + { + "t": 0.12, + "precision": 0.135912, + "recall": 0.879377 + }, + { + "t": 0.1, + "precision": 0.135782, + "recall": 0.898112 + }, + { + "t": 0.08, + "precision": 0.135704, + "recall": 0.917843 + }, + { + "t": 0.06, + "precision": 0.135636, + "recall": 0.937088 + }, + { + "t": 0.04, + "precision": 0.135613, + "recall": 0.957014 + }, + { + "t": 0.02, + "precision": 0.135667, + "recall": 0.9766 + }, + { + "t": 0.0, + "precision": 0.135657, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w045-ct-baseline.json b/docs/examples/bench/w045-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..9ad1157e4b38e9d45672209f6875e2c3b6d49ed5 --- /dev/null +++ b/docs/examples/bench/w045-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:54:07Z", + "scroll": "PHerc0139", + "segment": "w045", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w045_ink.png", + "sha256": "efc6337fd58811d835a5e29d11f77cebad8b2be834cd60a9033125956faddd53", + "width": 640, + "height": 465, + "pxPerMM": 8.2966, + "alignTargetFile": "data/bench/w045_ink.png", + "alignTargetSha256": "efc6337fd58811d835a5e29d11f77cebad8b2be834cd60a9033125956faddd53" + }, + "prediction": { + "name": "w045_ct.png", + "sha256": "0d991e625d4abb1ffce31f904a3e47ed012cdfe291f02ceecd5fdf733fc4662b", + "format": "png", + "width": 640, + "height": 465, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.225947, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 297600, + "validFraction": 1.0, + "pearson": 0.225947, + "blurSigmaPx": 6.0, + "blurSigmaUM": 723.2, + "pearsonMatchedBlur": 0.373433, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.113873, + "dice": 0.204463, + "precision": 0.125031, + "recall": 0.560629, + "refPositiveFraction": 0.10619, + "predPositiveFraction": 0.476146, + "ap": 0.133863, + "bestF1": { + "f1": 0.214346, + "threshold": 0.1 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.246835, + "recall": 0.001234 + }, + { + "t": 0.98, + "precision": 0.208421, + "recall": 0.003133 + }, + { + "t": 0.96, + "precision": 0.203549, + "recall": 0.00617 + }, + { + "t": 0.94, + "precision": 0.21059, + "recall": 0.010949 + }, + { + "t": 0.92, + "precision": 0.213596, + "recall": 0.017499 + }, + { + "t": 0.9, + "precision": 0.201886, + "recall": 0.025062 + }, + { + "t": 0.88, + "precision": 0.195237, + "recall": 0.035283 + }, + { + "t": 0.86, + "precision": 0.184125, + "recall": 0.046389 + }, + { + "t": 0.84, + "precision": 0.176421, + "recall": 0.059522 + }, + { + "t": 0.82, + "precision": 0.170021, + "recall": 0.075122 + }, + { + "t": 0.8, + "precision": 0.163684, + "recall": 0.09781 + }, + { + "t": 0.78, + "precision": 0.158529, + "recall": 0.119803 + }, + { + "t": 0.76, + "precision": 0.153797, + "recall": 0.143504 + }, + { + "t": 0.74, + "precision": 0.148818, + "recall": 0.168945 + }, + { + "t": 0.72, + "precision": 0.144553, + "recall": 0.196158 + }, + { + "t": 0.7, + "precision": 0.141312, + "recall": 0.226125 + }, + { + "t": 0.68, + "precision": 0.137794, + "recall": 0.256155 + }, + { + "t": 0.66, + "precision": 0.135329, + "recall": 0.288399 + }, + { + "t": 0.64, + "precision": 0.133221, + "recall": 0.321182 + }, + { + "t": 0.62, + "precision": 0.131229, + "recall": 0.354218 + }, + { + "t": 0.6, + "precision": 0.128882, + "recall": 0.393709 + }, + { + "t": 0.58, + "precision": 0.128008, + "recall": 0.427884 + }, + { + "t": 0.56, + "precision": 0.127244, + "recall": 0.462249 + }, + { + "t": 0.54, + "precision": 0.126396, + "recall": 0.495348 + }, + { + "t": 0.52, + "precision": 0.125542, + "recall": 0.527846 + }, + { + "t": 0.5, + "precision": 0.125031, + "recall": 0.560629 + }, + { + "t": 0.48, + "precision": 0.124513, + "recall": 0.591703 + }, + { + "t": 0.46, + "precision": 0.124114, + "recall": 0.622556 + }, + { + "t": 0.44, + "precision": 0.123695, + "recall": 0.651668 + }, + { + "t": 0.42, + "precision": 0.123468, + "recall": 0.679767 + }, + { + "t": 0.4, + "precision": 0.122989, + "recall": 0.711126 + }, + { + "t": 0.38, + "precision": 0.122745, + "recall": 0.736536 + }, + { + "t": 0.36, + "precision": 0.122465, + "recall": 0.760205 + }, + { + "t": 0.34, + "precision": 0.122187, + "recall": 0.781596 + }, + { + "t": 0.32, + "precision": 0.121924, + "recall": 0.801816 + }, + { + "t": 0.3, + "precision": 0.121659, + "recall": 0.820328 + }, + { + "t": 0.28, + "precision": 0.121524, + "recall": 0.838776 + }, + { + "t": 0.26, + "precision": 0.121488, + "recall": 0.855927 + }, + { + "t": 0.24, + "precision": 0.121272, + "recall": 0.870483 + }, + { + "t": 0.22, + "precision": 0.121049, + "recall": 0.883552 + }, + { + "t": 0.2, + "precision": 0.121122, + "recall": 0.899975 + }, + { + "t": 0.18, + "precision": 0.12111, + "recall": 0.911936 + }, + { + "t": 0.16, + "precision": 0.121063, + "recall": 0.9226 + }, + { + "t": 0.14, + "precision": 0.120846, + "recall": 0.931397 + }, + { + "t": 0.12, + "precision": 0.120814, + "recall": 0.940827 + }, + { + "t": 0.1, + "precision": 0.120805, + "recall": 0.949718 + }, + { + "t": 0.08, + "precision": 0.120665, + "recall": 0.956965 + }, + { + "t": 0.06, + "precision": 0.120521, + "recall": 0.963895 + }, + { + "t": 0.04, + "precision": 0.120415, + "recall": 0.970825 + }, + { + "t": 0.02, + "precision": 0.120089, + "recall": 0.978356 + }, + { + "t": 0.0, + "precision": 0.10619, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w045-ink9um.json b/docs/examples/bench/w045-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..d877bc13144869c28e221a59aa28c56524c9914e --- /dev/null +++ b/docs/examples/bench/w045-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:54:06Z", + "scroll": "PHerc0139", + "segment": "w045", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w045_ink.png", + "sha256": "efc6337fd58811d835a5e29d11f77cebad8b2be834cd60a9033125956faddd53", + "width": 640, + "height": 465, + "pxPerMM": 8.2966, + "alignTargetFile": "data/bench/w045_ink.png", + "alignTargetSha256": "efc6337fd58811d835a5e29d11f77cebad8b2be834cd60a9033125956faddd53" + }, + "prediction": { + "name": "w045_gpu.png", + "sha256": "fe8cd07604368288da0eb2d4da667ca69d0349b7646814a6db77438d9d0c7bf9", + "format": "png", + "width": 640, + "height": 465, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.813, + "ty": 0.5, + "sx": 1.00193, + "sy": 1.00217, + "ncc": 0.420308, + "coarseNcc": 0.5311, + "evals": 32164, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 297135, + "validFraction": 0.998438, + "pearson": 0.420308, + "blurSigmaPx": 1.5, + "blurSigmaUM": 180.8, + "pearsonMatchedBlur": 0.449008, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.230325, + "dice": 0.374413, + "precision": 0.374041, + "recall": 0.374786, + "refPositiveFraction": 0.106356, + "predPositiveFraction": 0.106568, + "ap": 0.334415, + "bestF1": { + "f1": 0.378594, + "threshold": 0.4 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.692308, + "recall": 0.001709 + }, + { + "t": 0.98, + "precision": 0.666935, + "recall": 0.078571 + }, + { + "t": 0.96, + "precision": 0.627132, + "recall": 0.098886 + }, + { + "t": 0.94, + "precision": 0.603742, + "recall": 0.11436 + }, + { + "t": 0.92, + "precision": 0.587541, + "recall": 0.129232 + }, + { + "t": 0.9, + "precision": 0.572768, + "recall": 0.143092 + }, + { + "t": 0.88, + "precision": 0.559597, + "recall": 0.156288 + }, + { + "t": 0.86, + "precision": 0.548039, + "recall": 0.168945 + }, + { + "t": 0.84, + "precision": 0.53693, + "recall": 0.181729 + }, + { + "t": 0.82, + "precision": 0.523904, + "recall": 0.192804 + }, + { + "t": 0.8, + "precision": 0.513091, + "recall": 0.204639 + }, + { + "t": 0.78, + "precision": 0.501758, + "recall": 0.216727 + }, + { + "t": 0.76, + "precision": 0.488865, + "recall": 0.226441 + }, + { + "t": 0.74, + "precision": 0.478139, + "recall": 0.237042 + }, + { + "t": 0.72, + "precision": 0.468926, + "recall": 0.248307 + }, + { + "t": 0.7, + "precision": 0.460543, + "recall": 0.260015 + }, + { + "t": 0.68, + "precision": 0.451054, + "recall": 0.270901 + }, + { + "t": 0.66, + "precision": 0.442599, + "recall": 0.282545 + }, + { + "t": 0.64, + "precision": 0.4334, + "recall": 0.294159 + }, + { + "t": 0.62, + "precision": 0.425392, + "recall": 0.305455 + }, + { + "t": 0.6, + "precision": 0.416708, + "recall": 0.316467 + }, + { + "t": 0.58, + "precision": 0.408071, + "recall": 0.327005 + }, + { + "t": 0.56, + "precision": 0.400471, + "recall": 0.339124 + }, + { + "t": 0.54, + "precision": 0.392034, + "recall": 0.351022 + }, + { + "t": 0.52, + "precision": 0.382703, + "recall": 0.362952 + }, + { + "t": 0.5, + "precision": 0.374041, + "recall": 0.374786 + }, + { + "t": 0.48, + "precision": 0.365411, + "recall": 0.386178 + }, + { + "t": 0.46, + "precision": 0.356857, + "recall": 0.398266 + }, + { + "t": 0.44, + "precision": 0.348749, + "recall": 0.410923 + }, + { + "t": 0.42, + "precision": 0.340812, + "recall": 0.423486 + }, + { + "t": 0.4, + "precision": 0.333647, + "recall": 0.437536 + }, + { + "t": 0.38, + "precision": 0.325737, + "recall": 0.451395 + }, + { + "t": 0.36, + "precision": 0.317528, + "recall": 0.464717 + }, + { + "t": 0.34, + "precision": 0.309866, + "recall": 0.479115 + }, + { + "t": 0.32, + "precision": 0.302206, + "recall": 0.494114 + }, + { + "t": 0.3, + "precision": 0.294979, + "recall": 0.511233 + }, + { + "t": 0.28, + "precision": 0.287736, + "recall": 0.528764 + }, + { + "t": 0.26, + "precision": 0.279686, + "recall": 0.544902 + }, + { + "t": 0.24, + "precision": 0.271331, + "recall": 0.563097 + }, + { + "t": 0.22, + "precision": 0.262703, + "recall": 0.581609 + }, + { + "t": 0.2, + "precision": 0.254543, + "recall": 0.601956 + }, + { + "t": 0.18, + "precision": 0.246331, + "recall": 0.623473 + }, + { + "t": 0.16, + "precision": 0.237796, + "recall": 0.644611 + }, + { + "t": 0.14, + "precision": 0.228578, + "recall": 0.667426 + }, + { + "t": 0.12, + "precision": 0.218527, + "recall": 0.69138 + }, + { + "t": 0.1, + "precision": 0.208383, + "recall": 0.719733 + }, + { + "t": 0.08, + "precision": 0.197058, + "recall": 0.74894 + }, + { + "t": 0.06, + "precision": 0.184893, + "recall": 0.784412 + }, + { + "t": 0.04, + "precision": 0.169969, + "recall": 0.827005 + }, + { + "t": 0.02, + "precision": 0.15197, + "recall": 0.881147 + }, + { + "t": 0.0, + "precision": 0.106356, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w045-noise-baseline.json b/docs/examples/bench/w045-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..a537e3b8f4531f7219e28aa7833e19e039dbed87 --- /dev/null +++ b/docs/examples/bench/w045-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:54:08Z", + "scroll": "PHerc0139", + "segment": "w045", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w045_ink.png", + "sha256": "efc6337fd58811d835a5e29d11f77cebad8b2be834cd60a9033125956faddd53", + "width": 640, + "height": 465, + "pxPerMM": 8.2966, + "alignTargetFile": "data/bench/w045_ink.png", + "alignTargetSha256": "efc6337fd58811d835a5e29d11f77cebad8b2be834cd60a9033125956faddd53" + }, + "prediction": { + "name": "noise-seed45.png", + "sha256": "d881af26bd45dbbdf8dc4903c549899778414df5e974a25b6d6ef528763a6ea9", + "format": "png", + "width": 640, + "height": 465, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(45).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.001108, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 297600, + "validFraction": 1.0, + "pearson": 0.001108, + "blurSigmaPx": 2.0, + "blurSigmaUM": 241.1, + "pearsonMatchedBlur": 0.002276, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.096, + "dice": 0.175182, + "precision": 0.106257, + "recall": 0.498608, + "refPositiveFraction": 0.10619, + "predPositiveFraction": 0.49829, + "ap": 0.106219, + "bestF1": { + "f1": 0.191992, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.108333, + "recall": 0.019746 + }, + { + "t": 0.96, + "precision": 0.106786, + "recall": 0.039238 + }, + { + "t": 0.94, + "precision": 0.107153, + "recall": 0.059585 + }, + { + "t": 0.92, + "precision": 0.105619, + "recall": 0.078571 + }, + { + "t": 0.9, + "precision": 0.105861, + "recall": 0.09819 + }, + { + "t": 0.88, + "precision": 0.105345, + "recall": 0.117303 + }, + { + "t": 0.86, + "precision": 0.106188, + "recall": 0.137808 + }, + { + "t": 0.84, + "precision": 0.106802, + "recall": 0.158155 + }, + { + "t": 0.82, + "precision": 0.10663, + "recall": 0.177615 + }, + { + "t": 0.8, + "precision": 0.105788, + "recall": 0.199703 + }, + { + "t": 0.78, + "precision": 0.105714, + "recall": 0.219132 + }, + { + "t": 0.76, + "precision": 0.106051, + "recall": 0.23932 + }, + { + "t": 0.74, + "precision": 0.106086, + "recall": 0.259129 + }, + { + "t": 0.72, + "precision": 0.106308, + "recall": 0.279666 + }, + { + "t": 0.7, + "precision": 0.105857, + "recall": 0.297734 + }, + { + "t": 0.68, + "precision": 0.106182, + "recall": 0.318113 + }, + { + "t": 0.66, + "precision": 0.106057, + "recall": 0.337162 + }, + { + "t": 0.64, + "precision": 0.106326, + "recall": 0.357889 + }, + { + "t": 0.62, + "precision": 0.10637, + "recall": 0.377793 + }, + { + "t": 0.6, + "precision": 0.106281, + "recall": 0.401019 + }, + { + "t": 0.58, + "precision": 0.106085, + "recall": 0.419784 + }, + { + "t": 0.56, + "precision": 0.10598, + "recall": 0.438706 + }, + { + "t": 0.54, + "precision": 0.106224, + "recall": 0.459433 + }, + { + "t": 0.52, + "precision": 0.10636, + "recall": 0.479875 + }, + { + "t": 0.5, + "precision": 0.106257, + "recall": 0.498608 + }, + { + "t": 0.48, + "precision": 0.106133, + "recall": 0.517594 + }, + { + "t": 0.46, + "precision": 0.106225, + "recall": 0.537498 + }, + { + "t": 0.44, + "precision": 0.106359, + "recall": 0.558034 + }, + { + "t": 0.42, + "precision": 0.106252, + "recall": 0.577274 + }, + { + "t": 0.4, + "precision": 0.106162, + "recall": 0.600247 + }, + { + "t": 0.38, + "precision": 0.106278, + "recall": 0.620878 + }, + { + "t": 0.36, + "precision": 0.106244, + "recall": 0.640497 + }, + { + "t": 0.34, + "precision": 0.10638, + "recall": 0.660718 + }, + { + "t": 0.32, + "precision": 0.106556, + "recall": 0.681666 + }, + { + "t": 0.3, + "precision": 0.106435, + "recall": 0.700842 + }, + { + "t": 0.28, + "precision": 0.106373, + "recall": 0.720397 + }, + { + "t": 0.26, + "precision": 0.106181, + "recall": 0.738371 + }, + { + "t": 0.24, + "precision": 0.106128, + "recall": 0.757579 + }, + { + "t": 0.22, + "precision": 0.106046, + "recall": 0.77628 + }, + { + "t": 0.2, + "precision": 0.106072, + "recall": 0.799918 + }, + { + "t": 0.18, + "precision": 0.105934, + "recall": 0.818208 + }, + { + "t": 0.16, + "precision": 0.105784, + "recall": 0.836624 + }, + { + "t": 0.14, + "precision": 0.105939, + "recall": 0.857794 + }, + { + "t": 0.12, + "precision": 0.106009, + "recall": 0.877539 + }, + { + "t": 0.1, + "precision": 0.106101, + "recall": 0.897696 + }, + { + "t": 0.08, + "precision": 0.106039, + "recall": 0.916777 + }, + { + "t": 0.06, + "precision": 0.106158, + "recall": 0.937694 + }, + { + "t": 0.04, + "precision": 0.106105, + "recall": 0.956775 + }, + { + "t": 0.02, + "precision": 0.106096, + "recall": 0.975824 + }, + { + "t": 0.0, + "precision": 0.10619, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w046-ct-baseline.json b/docs/examples/bench/w046-ct-baseline.json index b63fdb68418d35413c5b03d4316b8a1a04cdadcb..d716c4657d50463954539bff653a3f0a32136e23 100644 --- a/docs/examples/bench/w046-ct-baseline.json +++ b/docs/examples/bench/w046-ct-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:30Z", + "created": "2026-10-16T03:54:14Z", "scroll": "PHerc0139", "segment": "w046", "reference": { diff --git a/docs/examples/bench/w046-ink9um.json b/docs/examples/bench/w046-ink9um.json index 1321879aaa8f3cf1d8ae9505e2f90af3bca6f7f3..d3955d8a87b9e2d1b950671e98989df53d203f5e 100644 --- a/docs/examples/bench/w046-ink9um.json +++ b/docs/examples/bench/w046-ink9um.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:29Z", + "created": "2026-10-16T03:54:14Z", "scroll": "PHerc0139", "segment": "w046", "reference": { diff --git a/docs/examples/bench/w046-noise-baseline.json b/docs/examples/bench/w046-noise-baseline.json index 918e3ed7b1a4c87fbf6a36cc7c3fec454eafef46..50fbc99af4ed70e10cbd9e447c9c43edb6a96732 100644 --- a/docs/examples/bench/w046-noise-baseline.json +++ b/docs/examples/bench/w046-noise-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:31Z", + "created": "2026-10-16T03:54:15Z", "scroll": "PHerc0139", "segment": "w046", "reference": { diff --git a/docs/examples/bench/w047-ct-baseline.json b/docs/examples/bench/w047-ct-baseline.json index a7b151ebdc8241109644c76b475c2282b060e54f..e094cc3ddd31cce4f4277f2cbeb178692312bb1a 100644 --- a/docs/examples/bench/w047-ct-baseline.json +++ b/docs/examples/bench/w047-ct-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:39Z", + "created": "2026-10-16T03:54:20Z", "scroll": "PHerc0139", "segment": "w047", "reference": { diff --git a/docs/examples/bench/w047-ink9um.json b/docs/examples/bench/w047-ink9um.json index af95905a7a60cf38d85cdc25368a6f8096e6bdf4..fa6908a7cfdd5fd4392c1f7b0f113952d6a7ace1 100644 --- a/docs/examples/bench/w047-ink9um.json +++ b/docs/examples/bench/w047-ink9um.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:39Z", + "created": "2026-10-16T03:54:19Z", "scroll": "PHerc0139", "segment": "w047", "reference": { diff --git a/docs/examples/bench/w047-noise-baseline.json b/docs/examples/bench/w047-noise-baseline.json index fefbabe9990ecc01fc30edf6ba8a44c567151446..c3a92ffdf36ff6b93ccf04f89abf134665d3c376 100644 --- a/docs/examples/bench/w047-noise-baseline.json +++ b/docs/examples/bench/w047-noise-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:40Z", + "created": "2026-10-16T03:54:20Z", "scroll": "PHerc0139", "segment": "w047", "reference": { diff --git a/docs/examples/bench/w048-ct-baseline.json b/docs/examples/bench/w048-ct-baseline.json index 23784f895995ce1d2ff5ced1845baf463f818a47..b9adb50e92e7ad7a218cdf704e57f820f84165e5 100644 --- a/docs/examples/bench/w048-ct-baseline.json +++ b/docs/examples/bench/w048-ct-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:46Z", + "created": "2026-10-16T03:54:29Z", "scroll": "PHerc0139", "segment": "w048", "reference": { diff --git a/docs/examples/bench/w048-ink9um.json b/docs/examples/bench/w048-ink9um.json index 6f443504457c8946e2a669a4316143756cb917d6..e1ef7c61d6282b03fd9047364d7649d861c719c1 100644 --- a/docs/examples/bench/w048-ink9um.json +++ b/docs/examples/bench/w048-ink9um.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:45Z", + "created": "2026-10-16T03:54:28Z", "scroll": "PHerc0139", "segment": "w048", "reference": { diff --git a/docs/examples/bench/w048-noise-baseline.json b/docs/examples/bench/w048-noise-baseline.json index b4e0cc83ed21faca0e7a505ca113d34dad18218a..936dc1950502636d78082b8bb20873f05c836823 100644 --- a/docs/examples/bench/w048-noise-baseline.json +++ b/docs/examples/bench/w048-noise-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:46Z", + "created": "2026-10-16T03:54:29Z", "scroll": "PHerc0139", "segment": "w048", "reference": { diff --git a/docs/examples/bench/w049-ct-baseline.json b/docs/examples/bench/w049-ct-baseline.json index 22d4e040611b2c6ae804333fb8fb11a100832fff..241166acdfd961cf0efb01bb6613d2bced5733a3 100644 --- a/docs/examples/bench/w049-ct-baseline.json +++ b/docs/examples/bench/w049-ct-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:53Z", + "created": "2026-10-16T03:54:34Z", "scroll": "PHerc0139", "segment": "w049", "reference": { diff --git a/docs/examples/bench/w049-ink9um.json b/docs/examples/bench/w049-ink9um.json index cb5072ee6a2fab7bf61a1d4a855d0bc100e7adb0..afb6cbc297575a05d63848b79541e4eabc1343c3 100644 --- a/docs/examples/bench/w049-ink9um.json +++ b/docs/examples/bench/w049-ink9um.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:53Z", + "created": "2026-10-16T03:54:33Z", "scroll": "PHerc0139", "segment": "w049", "reference": { diff --git a/docs/examples/bench/w049-noise-baseline.json b/docs/examples/bench/w049-noise-baseline.json index 6beac68dfc5dbecbd2a0c6e75c25f878622703c3..5a51c22bfe80f371e6245f26360bcbe87dd3274e 100644 --- a/docs/examples/bench/w049-noise-baseline.json +++ b/docs/examples/bench/w049-noise-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:16:54Z", + "created": "2026-10-16T03:54:35Z", "scroll": "PHerc0139", "segment": "w049", "reference": { diff --git a/docs/examples/bench/w050-ct-baseline.json b/docs/examples/bench/w050-ct-baseline.json index ec5135430d3a23887abec9f1de16a1cd5e7ab96c..9b45f4cd66eef0349ec59b9461d051a869132cb1 100644 --- a/docs/examples/bench/w050-ct-baseline.json +++ b/docs/examples/bench/w050-ct-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:17:02Z", + "created": "2026-10-16T03:54:42Z", "scroll": "PHerc0139", "segment": "w050", "reference": { diff --git a/docs/examples/bench/w050-ink9um.json b/docs/examples/bench/w050-ink9um.json index 3ec358768f5cbcd0014bed152445e98222732f03..adc92b327761799b23ebf8e4ca4405193fb9c9b3 100644 --- a/docs/examples/bench/w050-ink9um.json +++ b/docs/examples/bench/w050-ink9um.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:17:01Z", + "created": "2026-10-16T03:54:41Z", "scroll": "PHerc0139", "segment": "w050", "reference": { diff --git a/docs/examples/bench/w050-noise-baseline.json b/docs/examples/bench/w050-noise-baseline.json index 4f7c37564da188cdc1a90a62170a629397ea5592..1d468543d9dd90e2eef6dc71d67ed3e75c98c83e 100644 --- a/docs/examples/bench/w050-noise-baseline.json +++ b/docs/examples/bench/w050-noise-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:17:03Z", + "created": "2026-10-16T03:54:43Z", "scroll": "PHerc0139", "segment": "w050", "reference": { diff --git a/docs/examples/bench/w051-ct-baseline.json b/docs/examples/bench/w051-ct-baseline.json index 60602c23488686bac1c236889bbd1086a50bcdbb..4b845f71b8fb5d032b391d0223247f6d0948bcbd 100644 --- a/docs/examples/bench/w051-ct-baseline.json +++ b/docs/examples/bench/w051-ct-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:17:12Z", + "created": "2026-10-16T03:54:52Z", "scroll": "PHerc0139", "segment": "w051", "reference": { diff --git a/docs/examples/bench/w051-ink9um.json b/docs/examples/bench/w051-ink9um.json index 84f447778ce60b28d484e2582ac4209709af368c..e5920b7911181049f85606e7211e913ce8105953 100644 --- a/docs/examples/bench/w051-ink9um.json +++ b/docs/examples/bench/w051-ink9um.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:17:12Z", + "created": "2026-10-16T03:54:51Z", "scroll": "PHerc0139", "segment": "w051", "reference": { diff --git a/docs/examples/bench/w051-noise-baseline.json b/docs/examples/bench/w051-noise-baseline.json index 1ecda3a49d2cd34e6c4f3042678c98e8641ee012..1aa9b80568db400cb2648c4954e7d4e7cf6138f3 100644 --- a/docs/examples/bench/w051-noise-baseline.json +++ b/docs/examples/bench/w051-noise-baseline.json @@ -5,7 +5,7 @@ "version": "1.0.0", "impl": "python" }, - "created": "2026-10-16T03:17:13Z", + "created": "2026-10-16T03:54:53Z", "scroll": "PHerc0139", "segment": "w051", "reference": { diff --git a/docs/examples/bench/w052-ct-baseline.json b/docs/examples/bench/w052-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..8901d7030b275a94181ea1073fb02597b3354192 --- /dev/null +++ b/docs/examples/bench/w052-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:01Z", + "scroll": "PHerc0139", + "segment": "w052", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w052_ink.png", + "sha256": "26c924bb32909121e7727575c7a78aa95a0e67d6f8200a0f5ff935005d581383", + "width": 640, + "height": 504, + "pxPerMM": 7.0045, + "alignTargetFile": "data/bench/w052_ink.png", + "alignTargetSha256": "26c924bb32909121e7727575c7a78aa95a0e67d6f8200a0f5ff935005d581383" + }, + "prediction": { + "name": "w052_ct.png", + "sha256": "29b475c37e05f7a8078e2eadedbbf1863899ca48e39caec2e7fb76d91a508154", + "format": "png", + "width": 640, + "height": 504, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.231505, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 322560, + "validFraction": 1.0, + "pearson": 0.231505, + "blurSigmaPx": 6.0, + "blurSigmaUM": 856.6, + "pearsonMatchedBlur": 0.373234, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.074774, + "dice": 0.139144, + "precision": 0.079374, + "recall": 0.56338, + "refPositiveFraction": 0.066084, + "predPositiveFraction": 0.469051, + "ap": 0.09393, + "bestF1": { + "f1": 0.142862, + "threshold": 0.66 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.290123, + "recall": 0.002205 + }, + { + "t": 0.98, + "precision": 0.237986, + "recall": 0.004879 + }, + { + "t": 0.96, + "precision": 0.235782, + "recall": 0.009336 + }, + { + "t": 0.94, + "precision": 0.232525, + "recall": 0.015294 + }, + { + "t": 0.92, + "precision": 0.216569, + "recall": 0.022565 + }, + { + "t": 0.9, + "precision": 0.198437, + "recall": 0.030963 + }, + { + "t": 0.88, + "precision": 0.180833, + "recall": 0.039923 + }, + { + "t": 0.86, + "precision": 0.16622, + "recall": 0.052355 + }, + { + "t": 0.84, + "precision": 0.153524, + "recall": 0.067133 + }, + { + "t": 0.82, + "precision": 0.140552, + "recall": 0.082942 + }, + { + "t": 0.8, + "precision": 0.129652, + "recall": 0.107056 + }, + { + "t": 0.78, + "precision": 0.121958, + "recall": 0.129996 + }, + { + "t": 0.76, + "precision": 0.116254, + "recall": 0.156361 + }, + { + "t": 0.74, + "precision": 0.11061, + "recall": 0.184134 + }, + { + "t": 0.72, + "precision": 0.105436, + "recall": 0.213924 + }, + { + "t": 0.7, + "precision": 0.100395, + "recall": 0.243338 + }, + { + "t": 0.68, + "precision": 0.096309, + "recall": 0.274113 + }, + { + "t": 0.66, + "precision": 0.093138, + "recall": 0.306483 + }, + { + "t": 0.64, + "precision": 0.08999, + "recall": 0.338103 + }, + { + "t": 0.62, + "precision": 0.087735, + "recall": 0.370661 + }, + { + "t": 0.6, + "precision": 0.084995, + "recall": 0.40744 + }, + { + "t": 0.58, + "precision": 0.083397, + "recall": 0.439435 + }, + { + "t": 0.56, + "precision": 0.082113, + "recall": 0.471008 + }, + { + "t": 0.54, + "precision": 0.081117, + "recall": 0.502721 + }, + { + "t": 0.52, + "precision": 0.080474, + "recall": 0.535982 + }, + { + "t": 0.5, + "precision": 0.079374, + "recall": 0.56338 + }, + { + "t": 0.48, + "precision": 0.079009, + "recall": 0.594671 + }, + { + "t": 0.46, + "precision": 0.078455, + "recall": 0.622678 + }, + { + "t": 0.44, + "precision": 0.078042, + "recall": 0.650403 + }, + { + "t": 0.42, + "precision": 0.077705, + "recall": 0.67766 + }, + { + "t": 0.4, + "precision": 0.077354, + "recall": 0.708107 + }, + { + "t": 0.38, + "precision": 0.07702, + "recall": 0.731516 + }, + { + "t": 0.36, + "precision": 0.076749, + "recall": 0.754175 + }, + { + "t": 0.34, + "precision": 0.076511, + "recall": 0.775286 + }, + { + "t": 0.32, + "precision": 0.076203, + "recall": 0.794192 + }, + { + "t": 0.3, + "precision": 0.076006, + "recall": 0.812864 + }, + { + "t": 0.28, + "precision": 0.075668, + "recall": 0.829096 + }, + { + "t": 0.26, + "precision": 0.075575, + "recall": 0.846266 + }, + { + "t": 0.24, + "precision": 0.075305, + "recall": 0.86048 + }, + { + "t": 0.22, + "precision": 0.075272, + "recall": 0.875727 + }, + { + "t": 0.2, + "precision": 0.075055, + "recall": 0.891068 + }, + { + "t": 0.18, + "precision": 0.074963, + "recall": 0.9035 + }, + { + "t": 0.16, + "precision": 0.074891, + "recall": 0.915463 + }, + { + "t": 0.14, + "precision": 0.074731, + "recall": 0.92522 + }, + { + "t": 0.12, + "precision": 0.074673, + "recall": 0.935354 + }, + { + "t": 0.1, + "precision": 0.074614, + "recall": 0.945112 + }, + { + "t": 0.08, + "precision": 0.074455, + "recall": 0.952946 + }, + { + "t": 0.06, + "precision": 0.074348, + "recall": 0.961156 + }, + { + "t": 0.04, + "precision": 0.074262, + "recall": 0.968944 + }, + { + "t": 0.02, + "precision": 0.074117, + "recall": 0.977294 + }, + { + "t": 0.0, + "precision": 0.066084, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w052-ink9um.json b/docs/examples/bench/w052-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..ffadb685579699cf5963f797e6ee1990d2e7dfa4 --- /dev/null +++ b/docs/examples/bench/w052-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:00Z", + "scroll": "PHerc0139", + "segment": "w052", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w052_ink.png", + "sha256": "26c924bb32909121e7727575c7a78aa95a0e67d6f8200a0f5ff935005d581383", + "width": 640, + "height": 504, + "pxPerMM": 7.0045, + "alignTargetFile": "data/bench/w052_ink.png", + "alignTargetSha256": "26c924bb32909121e7727575c7a78aa95a0e67d6f8200a0f5ff935005d581383" + }, + "prediction": { + "name": "w052_gpu.png", + "sha256": "c79565130088f5a8d0dd423ab64bebcf604e871a7a0960632578227e21f72278", + "format": "png", + "width": 640, + "height": 504, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.5, + "ty": 0.375, + "sx": 1.00006, + "sy": 1.0015, + "ncc": 0.376893, + "coarseNcc": 0.570494, + "evals": 34294, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 322056, + "validFraction": 0.998438, + "pearson": 0.376893, + "blurSigmaPx": 2.0, + "blurSigmaUM": 285.5, + "pearsonMatchedBlur": 0.421537, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.188044, + "dice": 0.316561, + "precision": 0.299815, + "recall": 0.335288, + "refPositiveFraction": 0.066187, + "predPositiveFraction": 0.074018, + "ap": 0.244443, + "bestF1": { + "f1": 0.317318, + "threshold": 0.48 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.578947, + "recall": 0.000516 + }, + { + "t": 0.98, + "precision": 0.564663, + "recall": 0.043629 + }, + { + "t": 0.96, + "precision": 0.541824, + "recall": 0.060471 + }, + { + "t": 0.94, + "precision": 0.515319, + "recall": 0.07417 + }, + { + "t": 0.92, + "precision": 0.508635, + "recall": 0.088431 + }, + { + "t": 0.9, + "precision": 0.492565, + "recall": 0.101004 + }, + { + "t": 0.88, + "precision": 0.473475, + "recall": 0.111794 + }, + { + "t": 0.86, + "precision": 0.463073, + "recall": 0.124132 + }, + { + "t": 0.84, + "precision": 0.451836, + "recall": 0.13511 + }, + { + "t": 0.82, + "precision": 0.442178, + "recall": 0.147448 + }, + { + "t": 0.8, + "precision": 0.428244, + "recall": 0.15791 + }, + { + "t": 0.78, + "precision": 0.418801, + "recall": 0.169497 + }, + { + "t": 0.76, + "precision": 0.407238, + "recall": 0.180006 + }, + { + "t": 0.74, + "precision": 0.398382, + "recall": 0.191734 + }, + { + "t": 0.72, + "precision": 0.390305, + "recall": 0.203228 + }, + { + "t": 0.7, + "precision": 0.382211, + "recall": 0.215097 + }, + { + "t": 0.68, + "precision": 0.371625, + "recall": 0.22598 + }, + { + "t": 0.66, + "precision": 0.36439, + "recall": 0.239069 + }, + { + "t": 0.64, + "precision": 0.355644, + "recall": 0.250516 + }, + { + "t": 0.62, + "precision": 0.347648, + "recall": 0.262807 + }, + { + "t": 0.6, + "precision": 0.339164, + "recall": 0.274723 + }, + { + "t": 0.58, + "precision": 0.331253, + "recall": 0.286311 + }, + { + "t": 0.56, + "precision": 0.32384, + "recall": 0.299259 + }, + { + "t": 0.54, + "precision": 0.316645, + "recall": 0.312441 + }, + { + "t": 0.52, + "precision": 0.308595, + "recall": 0.324592 + }, + { + "t": 0.5, + "precision": 0.299815, + "recall": 0.335288 + }, + { + "t": 0.48, + "precision": 0.29187, + "recall": 0.347626 + }, + { + "t": 0.46, + "precision": 0.283333, + "recall": 0.360387 + }, + { + "t": 0.44, + "precision": 0.275556, + "recall": 0.373053 + }, + { + "t": 0.42, + "precision": 0.267011, + "recall": 0.38586 + }, + { + "t": 0.4, + "precision": 0.258361, + "recall": 0.399371 + }, + { + "t": 0.38, + "precision": 0.249879, + "recall": 0.412319 + }, + { + "t": 0.36, + "precision": 0.241961, + "recall": 0.426769 + }, + { + "t": 0.34, + "precision": 0.234085, + "recall": 0.441265 + }, + { + "t": 0.32, + "precision": 0.226883, + "recall": 0.456746 + }, + { + "t": 0.3, + "precision": 0.219116, + "recall": 0.472134 + }, + { + "t": 0.28, + "precision": 0.211749, + "recall": 0.488882 + }, + { + "t": 0.26, + "precision": 0.203882, + "recall": 0.505114 + }, + { + "t": 0.24, + "precision": 0.196067, + "recall": 0.522894 + }, + { + "t": 0.22, + "precision": 0.188456, + "recall": 0.542363 + }, + { + "t": 0.2, + "precision": 0.181057, + "recall": 0.563802 + }, + { + "t": 0.18, + "precision": 0.172758, + "recall": 0.585898 + }, + { + "t": 0.16, + "precision": 0.164666, + "recall": 0.609308 + }, + { + "t": 0.14, + "precision": 0.155664, + "recall": 0.634312 + }, + { + "t": 0.12, + "precision": 0.147103, + "recall": 0.662882 + }, + { + "t": 0.1, + "precision": 0.137853, + "recall": 0.693564 + }, + { + "t": 0.08, + "precision": 0.128385, + "recall": 0.729452 + }, + { + "t": 0.06, + "precision": 0.118045, + "recall": 0.771815 + }, + { + "t": 0.04, + "precision": 0.106707, + "recall": 0.82614 + }, + { + "t": 0.02, + "precision": 0.093053, + "recall": 0.894399 + }, + { + "t": 0.0, + "precision": 0.066187, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w052-noise-baseline.json b/docs/examples/bench/w052-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..f6e663ef2695a53f5fb0ea0d75b67104866d1b07 --- /dev/null +++ b/docs/examples/bench/w052-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:02Z", + "scroll": "PHerc0139", + "segment": "w052", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w052_ink.png", + "sha256": "26c924bb32909121e7727575c7a78aa95a0e67d6f8200a0f5ff935005d581383", + "width": 640, + "height": 504, + "pxPerMM": 7.0045, + "alignTargetFile": "data/bench/w052_ink.png", + "alignTargetSha256": "26c924bb32909121e7727575c7a78aa95a0e67d6f8200a0f5ff935005d581383" + }, + "prediction": { + "name": "noise-seed52.png", + "sha256": "bb3fe5fd265faaa91980c7c30583bb4e792de51baf87df1934c2739742c4a757", + "format": "png", + "width": 640, + "height": 504, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(52).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.003032, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 322560, + "validFraction": 1.0, + "pearson": 0.003032, + "blurSigmaPx": 3.5, + "blurSigmaUM": 499.7, + "pearsonMatchedBlur": 0.003071, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.062698, + "dice": 0.117997, + "precision": 0.066827, + "recall": 0.503659, + "refPositiveFraction": 0.066084, + "predPositiveFraction": 0.498059, + "ap": 0.066684, + "bestF1": { + "f1": 0.123975, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.066056, + "recall": 0.01961 + }, + { + "t": 0.96, + "precision": 0.066471, + "recall": 0.039313 + }, + { + "t": 0.94, + "precision": 0.06633, + "recall": 0.059204 + }, + { + "t": 0.92, + "precision": 0.06731, + "recall": 0.079893 + }, + { + "t": 0.9, + "precision": 0.066842, + "recall": 0.098893 + }, + { + "t": 0.88, + "precision": 0.066594, + "recall": 0.118221 + }, + { + "t": 0.86, + "precision": 0.067469, + "recall": 0.139567 + }, + { + "t": 0.84, + "precision": 0.067302, + "recall": 0.158989 + }, + { + "t": 0.82, + "precision": 0.066871, + "recall": 0.178129 + }, + { + "t": 0.8, + "precision": 0.066821, + "recall": 0.202008 + }, + { + "t": 0.78, + "precision": 0.06729, + "recall": 0.223213 + }, + { + "t": 0.76, + "precision": 0.066992, + "recall": 0.242447 + }, + { + "t": 0.74, + "precision": 0.067503, + "recall": 0.264778 + }, + { + "t": 0.72, + "precision": 0.067544, + "recall": 0.285138 + }, + { + "t": 0.7, + "precision": 0.067363, + "recall": 0.304232 + }, + { + "t": 0.68, + "precision": 0.067459, + "recall": 0.325155 + }, + { + "t": 0.66, + "precision": 0.067484, + "recall": 0.345843 + }, + { + "t": 0.64, + "precision": 0.067336, + "recall": 0.36489 + }, + { + "t": 0.62, + "precision": 0.067276, + "recall": 0.384265 + }, + { + "t": 0.6, + "precision": 0.067202, + "recall": 0.407534 + }, + { + "t": 0.58, + "precision": 0.067166, + "recall": 0.42705 + }, + { + "t": 0.56, + "precision": 0.067026, + "recall": 0.445487 + }, + { + "t": 0.54, + "precision": 0.066834, + "recall": 0.464158 + }, + { + "t": 0.52, + "precision": 0.066934, + "recall": 0.484566 + }, + { + "t": 0.5, + "precision": 0.066827, + "recall": 0.503659 + }, + { + "t": 0.48, + "precision": 0.066758, + "recall": 0.523363 + }, + { + "t": 0.46, + "precision": 0.066545, + "recall": 0.54133 + }, + { + "t": 0.44, + "precision": 0.066477, + "recall": 0.560424 + }, + { + "t": 0.42, + "precision": 0.066545, + "recall": 0.580456 + }, + { + "t": 0.4, + "precision": 0.066588, + "recall": 0.604522 + }, + { + "t": 0.38, + "precision": 0.066402, + "recall": 0.622537 + }, + { + "t": 0.36, + "precision": 0.066422, + "recall": 0.642475 + }, + { + "t": 0.34, + "precision": 0.066415, + "recall": 0.662038 + }, + { + "t": 0.32, + "precision": 0.066332, + "recall": 0.680897 + }, + { + "t": 0.3, + "precision": 0.066236, + "recall": 0.699568 + }, + { + "t": 0.28, + "precision": 0.066257, + "recall": 0.719553 + }, + { + "t": 0.26, + "precision": 0.066237, + "recall": 0.738929 + }, + { + "t": 0.24, + "precision": 0.066363, + "recall": 0.760133 + }, + { + "t": 0.22, + "precision": 0.066255, + "recall": 0.77857 + }, + { + "t": 0.2, + "precision": 0.066288, + "recall": 0.80273 + }, + { + "t": 0.18, + "precision": 0.066166, + "recall": 0.820651 + }, + { + "t": 0.16, + "precision": 0.066171, + "recall": 0.840542 + }, + { + "t": 0.14, + "precision": 0.066159, + "recall": 0.859824 + }, + { + "t": 0.12, + "precision": 0.06623, + "recall": 0.880512 + }, + { + "t": 0.1, + "precision": 0.06621, + "recall": 0.899794 + }, + { + "t": 0.08, + "precision": 0.066125, + "recall": 0.918043 + }, + { + "t": 0.06, + "precision": 0.066055, + "recall": 0.93662 + }, + { + "t": 0.04, + "precision": 0.066143, + "recall": 0.957262 + }, + { + "t": 0.02, + "precision": 0.066062, + "recall": 0.976074 + }, + { + "t": 0.0, + "precision": 0.066084, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w053-ct-baseline.json b/docs/examples/bench/w053-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..2ff1352b69d7de474318105928f78979acec6cf5 --- /dev/null +++ b/docs/examples/bench/w053-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:06Z", + "scroll": "PHerc0139", + "segment": "w053", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w053_ink.png", + "sha256": "dfddf39ba017f6bf40288741bf309c5edbf483419c20dfbd729f9f613bd77297", + "width": 640, + "height": 460, + "pxPerMM": 6.5986, + "alignTargetFile": "data/bench/w053_ink.png", + "alignTargetSha256": "dfddf39ba017f6bf40288741bf309c5edbf483419c20dfbd729f9f613bd77297" + }, + "prediction": { + "name": "w053_ct.png", + "sha256": "bb50da6580261cef973e4276ec7936d114d55845c8efac9088ea2c5a91f76e5d", + "format": "png", + "width": 640, + "height": 460, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.279033, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 294400, + "validFraction": 1.0, + "pearson": 0.279033, + "blurSigmaPx": 6.0, + "blurSigmaUM": 909.3, + "pearsonMatchedBlur": 0.503971, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.036345, + "dice": 0.07014, + "precision": 0.037671, + "recall": 0.507889, + "refPositiveFraction": 0.030785, + "predPositiveFraction": 0.415044, + "ap": 0.040615, + "bestF1": { + "f1": 0.071602, + "threshold": 0.72 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.028571, + "recall": 0.000331 + }, + { + "t": 0.98, + "precision": 0.077778, + "recall": 0.002317 + }, + { + "t": 0.96, + "precision": 0.069519, + "recall": 0.004303 + }, + { + "t": 0.94, + "precision": 0.069201, + "recall": 0.007834 + }, + { + "t": 0.92, + "precision": 0.072357, + "recall": 0.014123 + }, + { + "t": 0.9, + "precision": 0.072373, + "recall": 0.021737 + }, + { + "t": 0.88, + "precision": 0.070098, + "recall": 0.031557 + }, + { + "t": 0.86, + "precision": 0.063724, + "recall": 0.040936 + }, + { + "t": 0.84, + "precision": 0.059154, + "recall": 0.052301 + }, + { + "t": 0.82, + "precision": 0.055096, + "recall": 0.066093 + }, + { + "t": 0.8, + "precision": 0.052866, + "recall": 0.087609 + }, + { + "t": 0.78, + "precision": 0.049904, + "recall": 0.106146 + }, + { + "t": 0.76, + "precision": 0.048857, + "recall": 0.129979 + }, + { + "t": 0.74, + "precision": 0.046588, + "recall": 0.152598 + }, + { + "t": 0.72, + "precision": 0.044858, + "recall": 0.177314 + }, + { + "t": 0.7, + "precision": 0.043065, + "recall": 0.20192 + }, + { + "t": 0.68, + "precision": 0.042135, + "recall": 0.230167 + }, + { + "t": 0.66, + "precision": 0.041135, + "recall": 0.258303 + }, + { + "t": 0.64, + "precision": 0.040349, + "recall": 0.287432 + }, + { + "t": 0.62, + "precision": 0.039682, + "recall": 0.316893 + }, + { + "t": 0.6, + "precision": 0.03955, + "recall": 0.357829 + }, + { + "t": 0.58, + "precision": 0.0392, + "recall": 0.389496 + }, + { + "t": 0.56, + "precision": 0.038639, + "recall": 0.418405 + }, + { + "t": 0.54, + "precision": 0.03825, + "recall": 0.448748 + }, + { + "t": 0.52, + "precision": 0.037835, + "recall": 0.476774 + }, + { + "t": 0.5, + "precision": 0.037671, + "recall": 0.507889 + }, + { + "t": 0.48, + "precision": 0.037656, + "recall": 0.539777 + }, + { + "t": 0.46, + "precision": 0.037342, + "recall": 0.56692 + }, + { + "t": 0.44, + "precision": 0.037134, + "recall": 0.594174 + }, + { + "t": 0.42, + "precision": 0.036995, + "recall": 0.621979 + }, + { + "t": 0.4, + "precision": 0.036862, + "recall": 0.65464 + }, + { + "t": 0.38, + "precision": 0.036735, + "recall": 0.679687 + }, + { + "t": 0.36, + "precision": 0.036575, + "recall": 0.70352 + }, + { + "t": 0.34, + "precision": 0.036342, + "recall": 0.723932 + }, + { + "t": 0.32, + "precision": 0.036299, + "recall": 0.746993 + }, + { + "t": 0.3, + "precision": 0.036226, + "recall": 0.768068 + }, + { + "t": 0.28, + "precision": 0.036177, + "recall": 0.788591 + }, + { + "t": 0.26, + "precision": 0.036152, + "recall": 0.808562 + }, + { + "t": 0.24, + "precision": 0.036117, + "recall": 0.827099 + }, + { + "t": 0.22, + "precision": 0.036036, + "recall": 0.84354 + }, + { + "t": 0.2, + "precision": 0.036007, + "recall": 0.863732 + }, + { + "t": 0.18, + "precision": 0.036022, + "recall": 0.879731 + }, + { + "t": 0.16, + "precision": 0.035969, + "recall": 0.893413 + }, + { + "t": 0.14, + "precision": 0.035918, + "recall": 0.905881 + }, + { + "t": 0.12, + "precision": 0.035921, + "recall": 0.919673 + }, + { + "t": 0.1, + "precision": 0.035902, + "recall": 0.931921 + }, + { + "t": 0.08, + "precision": 0.03589, + "recall": 0.9445 + }, + { + "t": 0.06, + "precision": 0.035861, + "recall": 0.956306 + }, + { + "t": 0.04, + "precision": 0.035821, + "recall": 0.967229 + }, + { + "t": 0.02, + "precision": 0.035816, + "recall": 0.979036 + }, + { + "t": 0.0, + "precision": 0.030785, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w053-ink9um.json b/docs/examples/bench/w053-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..4b5ebc2262f4d1809f7f61a78e50cbe90b79d94e --- /dev/null +++ b/docs/examples/bench/w053-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:06Z", + "scroll": "PHerc0139", + "segment": "w053", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w053_ink.png", + "sha256": "dfddf39ba017f6bf40288741bf309c5edbf483419c20dfbd729f9f613bd77297", + "width": 640, + "height": 460, + "pxPerMM": 6.5986, + "alignTargetFile": "data/bench/w053_ink.png", + "alignTargetSha256": "dfddf39ba017f6bf40288741bf309c5edbf483419c20dfbd729f9f613bd77297" + }, + "prediction": { + "name": "w053_gpu.png", + "sha256": "09c8e76abc3c5f1d4d979709bee06692946fd8b9b1054d262234ab41717ad319", + "format": "png", + "width": 640, + "height": 460, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.031, + "ty": 0.563, + "sx": 1.0027, + "sy": 1.00001, + "ncc": 0.263665, + "coarseNcc": 0.423663, + "evals": 30069, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 293760, + "validFraction": 0.997826, + "pearson": 0.263665, + "blurSigmaPx": 2.0, + "blurSigmaUM": 303.1, + "pearsonMatchedBlur": 0.299131, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.092087, + "dice": 0.168643, + "precision": 0.136256, + "recall": 0.221229, + "refPositiveFraction": 0.030852, + "predPositiveFraction": 0.050092, + "ap": 0.097478, + "bestF1": { + "f1": 0.169197, + "threshold": 0.54 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.272727, + "recall": 0.000331 + }, + { + "t": 0.98, + "precision": 0.338095, + "recall": 0.023502 + }, + { + "t": 0.96, + "precision": 0.298067, + "recall": 0.032329 + }, + { + "t": 0.94, + "precision": 0.273425, + "recall": 0.039281 + }, + { + "t": 0.92, + "precision": 0.266708, + "recall": 0.047556 + }, + { + "t": 0.9, + "precision": 0.255471, + "recall": 0.05539 + }, + { + "t": 0.88, + "precision": 0.235717, + "recall": 0.061459 + }, + { + "t": 0.86, + "precision": 0.224107, + "recall": 0.06852 + }, + { + "t": 0.84, + "precision": 0.217734, + "recall": 0.076134 + }, + { + "t": 0.82, + "precision": 0.209492, + "recall": 0.082313 + }, + { + "t": 0.8, + "precision": 0.202437, + "recall": 0.089816 + }, + { + "t": 0.78, + "precision": 0.196543, + "recall": 0.09787 + }, + { + "t": 0.76, + "precision": 0.192996, + "recall": 0.107029 + }, + { + "t": 0.74, + "precision": 0.188074, + "recall": 0.115194 + }, + { + "t": 0.72, + "precision": 0.183754, + "recall": 0.122807 + }, + { + "t": 0.7, + "precision": 0.180978, + "recall": 0.131855 + }, + { + "t": 0.68, + "precision": 0.175803, + "recall": 0.14013 + }, + { + "t": 0.66, + "precision": 0.170142, + "recall": 0.147633 + }, + { + "t": 0.64, + "precision": 0.165439, + "recall": 0.156129 + }, + { + "t": 0.62, + "precision": 0.161865, + "recall": 0.165508 + }, + { + "t": 0.6, + "precision": 0.156593, + "recall": 0.173232 + }, + { + "t": 0.58, + "precision": 0.153145, + "recall": 0.183493 + }, + { + "t": 0.56, + "precision": 0.149277, + "recall": 0.192431 + }, + { + "t": 0.54, + "precision": 0.145089, + "recall": 0.202913 + }, + { + "t": 0.52, + "precision": 0.140207, + "recall": 0.210968 + }, + { + "t": 0.5, + "precision": 0.136256, + "recall": 0.221229 + }, + { + "t": 0.48, + "precision": 0.132019, + "recall": 0.230608 + }, + { + "t": 0.46, + "precision": 0.129419, + "recall": 0.242745 + }, + { + "t": 0.44, + "precision": 0.1261, + "recall": 0.254551 + }, + { + "t": 0.42, + "precision": 0.122774, + "recall": 0.26702 + }, + { + "t": 0.4, + "precision": 0.119146, + "recall": 0.278164 + }, + { + "t": 0.38, + "precision": 0.116004, + "recall": 0.291846 + }, + { + "t": 0.36, + "precision": 0.112061, + "recall": 0.303652 + }, + { + "t": 0.34, + "precision": 0.108381, + "recall": 0.317334 + }, + { + "t": 0.32, + "precision": 0.105212, + "recall": 0.333223 + }, + { + "t": 0.3, + "precision": 0.101511, + "recall": 0.347567 + }, + { + "t": 0.28, + "precision": 0.097818, + "recall": 0.363566 + }, + { + "t": 0.26, + "precision": 0.094103, + "recall": 0.379455 + }, + { + "t": 0.24, + "precision": 0.090811, + "recall": 0.398544 + }, + { + "t": 0.22, + "precision": 0.086874, + "recall": 0.41697 + }, + { + "t": 0.2, + "precision": 0.083979, + "recall": 0.439921 + }, + { + "t": 0.18, + "precision": 0.080323, + "recall": 0.46254 + }, + { + "t": 0.16, + "precision": 0.076947, + "recall": 0.48858 + }, + { + "t": 0.14, + "precision": 0.07325, + "recall": 0.515944 + }, + { + "t": 0.12, + "precision": 0.069562, + "recall": 0.548384 + }, + { + "t": 0.1, + "precision": 0.065701, + "recall": 0.585237 + }, + { + "t": 0.08, + "precision": 0.061622, + "recall": 0.630034 + }, + { + "t": 0.06, + "precision": 0.056827, + "recall": 0.679025 + }, + { + "t": 0.04, + "precision": 0.051472, + "recall": 0.744235 + }, + { + "t": 0.02, + "precision": 0.045217, + "recall": 0.834382 + }, + { + "t": 0.0, + "precision": 0.030852, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w053-noise-baseline.json b/docs/examples/bench/w053-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..36191415f9702ec4d766ba3401bf2aa76474b47b --- /dev/null +++ b/docs/examples/bench/w053-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:07Z", + "scroll": "PHerc0139", + "segment": "w053", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w053_ink.png", + "sha256": "dfddf39ba017f6bf40288741bf309c5edbf483419c20dfbd729f9f613bd77297", + "width": 640, + "height": 460, + "pxPerMM": 6.5986, + "alignTargetFile": "data/bench/w053_ink.png", + "alignTargetSha256": "dfddf39ba017f6bf40288741bf309c5edbf483419c20dfbd729f9f613bd77297" + }, + "prediction": { + "name": "noise-seed53.png", + "sha256": "5f1aebd5f378c0ffb9437c564599afa7d3f007876722c9fd0fabc49270f514ef", + "format": "png", + "width": 640, + "height": 460, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(53).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.000864, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 294400, + "validFraction": 1.0, + "pearson": 0.000864, + "blurSigmaPx": 0.0, + "blurSigmaUM": 0.0, + "pearsonMatchedBlur": 0.000864, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.029781, + "dice": 0.057839, + "precision": 0.030708, + "recall": 0.496414, + "refPositiveFraction": 0.030785, + "predPositiveFraction": 0.497646, + "ap": 0.031042, + "bestF1": { + "f1": 0.059811, + "threshold": 0.1 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.032852, + "recall": 0.020854 + }, + { + "t": 0.96, + "precision": 0.032685, + "recall": 0.041929 + }, + { + "t": 0.94, + "precision": 0.0319, + "recall": 0.061238 + }, + { + "t": 0.92, + "precision": 0.03167, + "recall": 0.080327 + }, + { + "t": 0.9, + "precision": 0.031501, + "recall": 0.100077 + }, + { + "t": 0.88, + "precision": 0.031089, + "recall": 0.118724 + }, + { + "t": 0.86, + "precision": 0.031424, + "recall": 0.139799 + }, + { + "t": 0.84, + "precision": 0.031093, + "recall": 0.158115 + }, + { + "t": 0.82, + "precision": 0.031, + "recall": 0.177756 + }, + { + "t": 0.8, + "precision": 0.031074, + "recall": 0.20203 + }, + { + "t": 0.78, + "precision": 0.030918, + "recall": 0.220677 + }, + { + "t": 0.76, + "precision": 0.030724, + "recall": 0.238663 + }, + { + "t": 0.74, + "precision": 0.030662, + "recall": 0.257641 + }, + { + "t": 0.72, + "precision": 0.030914, + "recall": 0.279598 + }, + { + "t": 0.7, + "precision": 0.031001, + "recall": 0.299901 + }, + { + "t": 0.68, + "precision": 0.030917, + "recall": 0.318879 + }, + { + "t": 0.66, + "precision": 0.030755, + "recall": 0.336864 + }, + { + "t": 0.64, + "precision": 0.031129, + "recall": 0.360918 + }, + { + "t": 0.62, + "precision": 0.030965, + "recall": 0.378462 + }, + { + "t": 0.6, + "precision": 0.03087, + "recall": 0.401081 + }, + { + "t": 0.58, + "precision": 0.030798, + "recall": 0.419508 + }, + { + "t": 0.56, + "precision": 0.030687, + "recall": 0.437052 + }, + { + "t": 0.54, + "precision": 0.030809, + "recall": 0.458788 + }, + { + "t": 0.52, + "precision": 0.030792, + "recall": 0.478318 + }, + { + "t": 0.5, + "precision": 0.030708, + "recall": 0.496414 + }, + { + "t": 0.48, + "precision": 0.030755, + "recall": 0.516716 + }, + { + "t": 0.46, + "precision": 0.030963, + "recall": 0.54077 + }, + { + "t": 0.44, + "precision": 0.030964, + "recall": 0.560079 + }, + { + "t": 0.42, + "precision": 0.031044, + "recall": 0.581044 + }, + { + "t": 0.4, + "precision": 0.031094, + "recall": 0.605539 + }, + { + "t": 0.38, + "precision": 0.031051, + "recall": 0.624848 + }, + { + "t": 0.36, + "precision": 0.031058, + "recall": 0.64493 + }, + { + "t": 0.34, + "precision": 0.031018, + "recall": 0.664019 + }, + { + "t": 0.32, + "precision": 0.030974, + "recall": 0.682666 + }, + { + "t": 0.3, + "precision": 0.030997, + "recall": 0.703078 + }, + { + "t": 0.28, + "precision": 0.030896, + "recall": 0.720733 + }, + { + "t": 0.26, + "precision": 0.030948, + "recall": 0.741476 + }, + { + "t": 0.24, + "precision": 0.030925, + "recall": 0.760565 + }, + { + "t": 0.22, + "precision": 0.030838, + "recall": 0.778109 + }, + { + "t": 0.2, + "precision": 0.030778, + "recall": 0.799846 + }, + { + "t": 0.18, + "precision": 0.03083, + "recall": 0.82092 + }, + { + "t": 0.16, + "precision": 0.030889, + "recall": 0.841774 + }, + { + "t": 0.14, + "precision": 0.030909, + "recall": 0.861966 + }, + { + "t": 0.12, + "precision": 0.030882, + "recall": 0.881055 + }, + { + "t": 0.1, + "precision": 0.03093, + "recall": 0.902571 + }, + { + "t": 0.08, + "precision": 0.030866, + "recall": 0.920446 + }, + { + "t": 0.06, + "precision": 0.030845, + "recall": 0.939424 + }, + { + "t": 0.04, + "precision": 0.030777, + "recall": 0.956747 + }, + { + "t": 0.02, + "precision": 0.030818, + "recall": 0.977712 + }, + { + "t": 0.0, + "precision": 0.030785, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w054-ct-baseline.json b/docs/examples/bench/w054-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..bd21f574e9a85aac6a945a26e033ff5e41d0c378 --- /dev/null +++ b/docs/examples/bench/w054-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:11Z", + "scroll": "PHerc0139", + "segment": "w054", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w054_ink.png", + "sha256": "7d2cd5825bb843ed0fe995db48a290eac3c219308e14e951f729ee034f00feda", + "width": 640, + "height": 439, + "pxPerMM": 6.1367, + "alignTargetFile": "data/bench/w054_ink.png", + "alignTargetSha256": "7d2cd5825bb843ed0fe995db48a290eac3c219308e14e951f729ee034f00feda" + }, + "prediction": { + "name": "w054_ct.png", + "sha256": "a6c5afcbeb31180ebfc9f286802fd07f01ae517a3db502e3972d73b68d6dd0b6", + "format": "png", + "width": 640, + "height": 439, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.292123, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 280960, + "validFraction": 1.0, + "pearson": 0.292123, + "blurSigmaPx": 6.0, + "blurSigmaUM": 977.7, + "pearsonMatchedBlur": 0.504129, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.068086, + "dice": 0.127492, + "precision": 0.072717, + "recall": 0.516695, + "refPositiveFraction": 0.056389, + "predPositiveFraction": 0.400673, + "ap": 0.077749, + "bestF1": { + "f1": 0.128193, + "threshold": 0.46 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.17757, + "recall": 0.001199 + }, + { + "t": 0.98, + "precision": 0.168582, + "recall": 0.002777 + }, + { + "t": 0.96, + "precision": 0.187135, + "recall": 0.006059 + }, + { + "t": 0.94, + "precision": 0.167224, + "recall": 0.009468 + }, + { + "t": 0.92, + "precision": 0.155065, + "recall": 0.014202 + }, + { + "t": 0.9, + "precision": 0.139857, + "recall": 0.019756 + }, + { + "t": 0.88, + "precision": 0.124268, + "recall": 0.025437 + }, + { + "t": 0.86, + "precision": 0.117799, + "recall": 0.034589 + }, + { + "t": 0.84, + "precision": 0.110452, + "recall": 0.044689 + }, + { + "t": 0.82, + "precision": 0.10516, + "recall": 0.057375 + }, + { + "t": 0.8, + "precision": 0.098007, + "recall": 0.074481 + }, + { + "t": 0.78, + "precision": 0.093486, + "recall": 0.092217 + }, + { + "t": 0.76, + "precision": 0.091618, + "recall": 0.114877 + }, + { + "t": 0.74, + "precision": 0.088508, + "recall": 0.138358 + }, + { + "t": 0.72, + "precision": 0.085718, + "recall": 0.163353 + }, + { + "t": 0.7, + "precision": 0.084127, + "recall": 0.191757 + }, + { + "t": 0.68, + "precision": 0.081922, + "recall": 0.219845 + }, + { + "t": 0.66, + "precision": 0.080847, + "recall": 0.252414 + }, + { + "t": 0.64, + "precision": 0.079621, + "recall": 0.285615 + }, + { + "t": 0.62, + "precision": 0.077613, + "recall": 0.315534 + }, + { + "t": 0.6, + "precision": 0.076032, + "recall": 0.353342 + }, + { + "t": 0.58, + "precision": 0.075223, + "recall": 0.38648 + }, + { + "t": 0.56, + "precision": 0.074289, + "recall": 0.418608 + }, + { + "t": 0.54, + "precision": 0.073683, + "recall": 0.451367 + }, + { + "t": 0.52, + "precision": 0.073399, + "recall": 0.48564 + }, + { + "t": 0.5, + "precision": 0.072717, + "recall": 0.516695 + }, + { + "t": 0.48, + "precision": 0.07245, + "recall": 0.549138 + }, + { + "t": 0.46, + "precision": 0.072065, + "recall": 0.579688 + }, + { + "t": 0.44, + "precision": 0.071461, + "recall": 0.607776 + }, + { + "t": 0.42, + "precision": 0.070887, + "recall": 0.634602 + }, + { + "t": 0.4, + "precision": 0.070622, + "recall": 0.66875 + }, + { + "t": 0.38, + "precision": 0.070364, + "recall": 0.695512 + }, + { + "t": 0.36, + "precision": 0.069961, + "recall": 0.719119 + }, + { + "t": 0.34, + "precision": 0.069662, + "recall": 0.74241 + }, + { + "t": 0.32, + "precision": 0.069385, + "recall": 0.764565 + }, + { + "t": 0.3, + "precision": 0.06922, + "recall": 0.785962 + }, + { + "t": 0.28, + "precision": 0.068925, + "recall": 0.804898 + }, + { + "t": 0.26, + "precision": 0.068867, + "recall": 0.825033 + }, + { + "t": 0.24, + "precision": 0.068823, + "recall": 0.844095 + }, + { + "t": 0.22, + "precision": 0.068791, + "recall": 0.862084 + }, + { + "t": 0.2, + "precision": 0.06861, + "recall": 0.880073 + }, + { + "t": 0.18, + "precision": 0.06848, + "recall": 0.894023 + }, + { + "t": 0.16, + "precision": 0.068308, + "recall": 0.906268 + }, + { + "t": 0.14, + "precision": 0.068271, + "recall": 0.918829 + }, + { + "t": 0.12, + "precision": 0.068221, + "recall": 0.930758 + }, + { + "t": 0.1, + "precision": 0.068214, + "recall": 0.943508 + }, + { + "t": 0.08, + "precision": 0.068045, + "recall": 0.953923 + }, + { + "t": 0.06, + "precision": 0.067818, + "recall": 0.963265 + }, + { + "t": 0.04, + "precision": 0.067638, + "recall": 0.973427 + }, + { + "t": 0.02, + "precision": 0.067357, + "recall": 0.982074 + }, + { + "t": 0.0, + "precision": 0.056389, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w054-ink9um.json b/docs/examples/bench/w054-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..886db7fe8c9f0c02ddae282f7ab29dd83b2bda2e --- /dev/null +++ b/docs/examples/bench/w054-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:10Z", + "scroll": "PHerc0139", + "segment": "w054", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w054_ink.png", + "sha256": "7d2cd5825bb843ed0fe995db48a290eac3c219308e14e951f729ee034f00feda", + "width": 640, + "height": 439, + "pxPerMM": 6.1367, + "alignTargetFile": "data/bench/w054_ink.png", + "alignTargetSha256": "7d2cd5825bb843ed0fe995db48a290eac3c219308e14e951f729ee034f00feda" + }, + "prediction": { + "name": "w054_gpu.png", + "sha256": "c3b3cf2331601645db43d56b402d49285a6ded4984bfcbcbc06582780dc05a80", + "format": "png", + "width": 640, + "height": 439, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.438, + "ty": 0.438, + "sx": 0.99994, + "sy": 1.00005, + "ncc": 0.28046, + "coarseNcc": 0.412182, + "evals": 30002, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 279882, + "validFraction": 0.996163, + "pearson": 0.28046, + "blurSigmaPx": 1.5, + "blurSigmaUM": 244.4, + "pearsonMatchedBlur": 0.314936, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.111432, + "dice": 0.200519, + "precision": 0.221865, + "recall": 0.18292, + "refPositiveFraction": 0.056606, + "predPositiveFraction": 0.04667, + "ap": 0.144248, + "bestF1": { + "f1": 0.213615, + "threshold": 0.36 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.333333, + "recall": 0.000126 + }, + { + "t": 0.98, + "precision": 0.454545, + "recall": 0.011361 + }, + { + "t": 0.96, + "precision": 0.420741, + "recall": 0.017926 + }, + { + "t": 0.94, + "precision": 0.401087, + "recall": 0.023291 + }, + { + "t": 0.92, + "precision": 0.367769, + "recall": 0.028088 + }, + { + "t": 0.9, + "precision": 0.350692, + "recall": 0.033579 + }, + { + "t": 0.88, + "precision": 0.347345, + "recall": 0.039639 + }, + { + "t": 0.86, + "precision": 0.341486, + "recall": 0.045825 + }, + { + "t": 0.84, + "precision": 0.342466, + "recall": 0.053651 + }, + { + "t": 0.82, + "precision": 0.330149, + "recall": 0.05889 + }, + { + "t": 0.8, + "precision": 0.322842, + "recall": 0.065392 + }, + { + "t": 0.78, + "precision": 0.315498, + "recall": 0.07183 + }, + { + "t": 0.76, + "precision": 0.30729, + "recall": 0.077952 + }, + { + "t": 0.74, + "precision": 0.300629, + "recall": 0.084517 + }, + { + "t": 0.72, + "precision": 0.293309, + "recall": 0.091586 + }, + { + "t": 0.7, + "precision": 0.284934, + "recall": 0.098719 + }, + { + "t": 0.68, + "precision": 0.27846, + "recall": 0.105914 + }, + { + "t": 0.66, + "precision": 0.273428, + "recall": 0.114183 + }, + { + "t": 0.64, + "precision": 0.264977, + "recall": 0.121442 + }, + { + "t": 0.62, + "precision": 0.259417, + "recall": 0.130846 + }, + { + "t": 0.6, + "precision": 0.253668, + "recall": 0.139683 + }, + { + "t": 0.58, + "precision": 0.245902, + "recall": 0.146753 + }, + { + "t": 0.56, + "precision": 0.239135, + "recall": 0.154895 + }, + { + "t": 0.54, + "precision": 0.232199, + "recall": 0.163227 + }, + { + "t": 0.52, + "precision": 0.225812, + "recall": 0.172063 + }, + { + "t": 0.5, + "precision": 0.221865, + "recall": 0.18292 + }, + { + "t": 0.48, + "precision": 0.2159, + "recall": 0.193524 + }, + { + "t": 0.46, + "precision": 0.211717, + "recall": 0.205517 + }, + { + "t": 0.44, + "precision": 0.204769, + "recall": 0.215742 + }, + { + "t": 0.42, + "precision": 0.197687, + "recall": 0.225525 + }, + { + "t": 0.4, + "precision": 0.191093, + "recall": 0.236698 + }, + { + "t": 0.38, + "precision": 0.185878, + "recall": 0.248248 + }, + { + "t": 0.36, + "precision": 0.180582, + "recall": 0.26144 + }, + { + "t": 0.34, + "precision": 0.173766, + "recall": 0.271666 + }, + { + "t": 0.32, + "precision": 0.169041, + "recall": 0.28612 + }, + { + "t": 0.3, + "precision": 0.163024, + "recall": 0.299943 + }, + { + "t": 0.28, + "precision": 0.157515, + "recall": 0.315218 + }, + { + "t": 0.26, + "precision": 0.152234, + "recall": 0.331566 + }, + { + "t": 0.24, + "precision": 0.147877, + "recall": 0.351449 + }, + { + "t": 0.22, + "precision": 0.143353, + "recall": 0.372846 + }, + { + "t": 0.2, + "precision": 0.138029, + "recall": 0.394685 + }, + { + "t": 0.18, + "precision": 0.132504, + "recall": 0.417976 + }, + { + "t": 0.16, + "precision": 0.127323, + "recall": 0.445812 + }, + { + "t": 0.14, + "precision": 0.121904, + "recall": 0.476867 + }, + { + "t": 0.12, + "precision": 0.116012, + "recall": 0.510699 + }, + { + "t": 0.1, + "precision": 0.109683, + "recall": 0.547813 + }, + { + "t": 0.08, + "precision": 0.103418, + "recall": 0.594711 + }, + { + "t": 0.06, + "precision": 0.096614, + "recall": 0.652591 + }, + { + "t": 0.04, + "precision": 0.088379, + "recall": 0.723727 + }, + { + "t": 0.02, + "precision": 0.07875, + "recall": 0.822761 + }, + { + "t": 0.0, + "precision": 0.056606, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w054-noise-baseline.json b/docs/examples/bench/w054-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..204553b0e4adb1ae1a3c6ea8d50f40723c95bf60 --- /dev/null +++ b/docs/examples/bench/w054-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:11Z", + "scroll": "PHerc0139", + "segment": "w054", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w054_ink.png", + "sha256": "7d2cd5825bb843ed0fe995db48a290eac3c219308e14e951f729ee034f00feda", + "width": 640, + "height": 439, + "pxPerMM": 6.1367, + "alignTargetFile": "data/bench/w054_ink.png", + "alignTargetSha256": "7d2cd5825bb843ed0fe995db48a290eac3c219308e14e951f729ee034f00feda" + }, + "prediction": { + "name": "noise-seed54.png", + "sha256": "bafe2a345b5c17b743935b3d0ea2decc9a4bf334df9cdab80325453d72ee509c", + "format": "png", + "width": 640, + "height": 439, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(54).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": -0.000293, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 280960, + "validFraction": 1.0, + "pearson": -0.000293, + "blurSigmaPx": 3.0, + "blurSigmaUM": 488.9, + "pearsonMatchedBlur": 0.000988, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.053274, + "dice": 0.101158, + "precision": 0.056297, + "recall": 0.498012, + "refPositiveFraction": 0.056389, + "predPositiveFraction": 0.498825, + "ap": 0.055937, + "bestF1": { + "f1": 0.106758, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.056328, + "recall": 0.019188 + }, + { + "t": 0.96, + "precision": 0.054898, + "recall": 0.037493 + }, + { + "t": 0.94, + "precision": 0.05518, + "recall": 0.057186 + }, + { + "t": 0.92, + "precision": 0.053892, + "recall": 0.074418 + }, + { + "t": 0.9, + "precision": 0.054557, + "recall": 0.0943 + }, + { + "t": 0.88, + "precision": 0.054419, + "recall": 0.112857 + }, + { + "t": 0.86, + "precision": 0.054976, + "recall": 0.132677 + }, + { + "t": 0.84, + "precision": 0.055133, + "recall": 0.152244 + }, + { + "t": 0.82, + "precision": 0.05558, + "recall": 0.17301 + }, + { + "t": 0.8, + "precision": 0.055434, + "recall": 0.195607 + }, + { + "t": 0.78, + "precision": 0.055255, + "recall": 0.214038 + }, + { + "t": 0.76, + "precision": 0.055173, + "recall": 0.232974 + }, + { + "t": 0.74, + "precision": 0.05538, + "recall": 0.253487 + }, + { + "t": 0.72, + "precision": 0.055126, + "recall": 0.271413 + }, + { + "t": 0.7, + "precision": 0.055191, + "recall": 0.291233 + }, + { + "t": 0.68, + "precision": 0.055585, + "recall": 0.31263 + }, + { + "t": 0.66, + "precision": 0.055921, + "recall": 0.333901 + }, + { + "t": 0.64, + "precision": 0.055793, + "recall": 0.352395 + }, + { + "t": 0.62, + "precision": 0.055815, + "recall": 0.372026 + }, + { + "t": 0.6, + "precision": 0.05589, + "recall": 0.395821 + }, + { + "t": 0.58, + "precision": 0.055892, + "recall": 0.415767 + }, + { + "t": 0.56, + "precision": 0.056058, + "recall": 0.436597 + }, + { + "t": 0.54, + "precision": 0.055944, + "recall": 0.455027 + }, + { + "t": 0.52, + "precision": 0.055939, + "recall": 0.474784 + }, + { + "t": 0.5, + "precision": 0.056297, + "recall": 0.498012 + }, + { + "t": 0.48, + "precision": 0.056525, + "recall": 0.520167 + }, + { + "t": 0.46, + "precision": 0.056469, + "recall": 0.539229 + }, + { + "t": 0.44, + "precision": 0.056575, + "recall": 0.559427 + }, + { + "t": 0.42, + "precision": 0.056574, + "recall": 0.579183 + }, + { + "t": 0.4, + "precision": 0.056579, + "recall": 0.602916 + }, + { + "t": 0.38, + "precision": 0.056665, + "recall": 0.623241 + }, + { + "t": 0.36, + "precision": 0.056545, + "recall": 0.64123 + }, + { + "t": 0.34, + "precision": 0.056344, + "recall": 0.658398 + }, + { + "t": 0.32, + "precision": 0.056492, + "recall": 0.679164 + }, + { + "t": 0.3, + "precision": 0.056394, + "recall": 0.697974 + }, + { + "t": 0.28, + "precision": 0.056328, + "recall": 0.71672 + }, + { + "t": 0.26, + "precision": 0.056254, + "recall": 0.735467 + }, + { + "t": 0.24, + "precision": 0.056341, + "recall": 0.756044 + }, + { + "t": 0.22, + "precision": 0.056335, + "recall": 0.775295 + }, + { + "t": 0.2, + "precision": 0.05641, + "recall": 0.80029 + }, + { + "t": 0.18, + "precision": 0.056512, + "recall": 0.821309 + }, + { + "t": 0.16, + "precision": 0.056411, + "recall": 0.83974 + }, + { + "t": 0.14, + "precision": 0.056293, + "recall": 0.857918 + }, + { + "t": 0.12, + "precision": 0.056215, + "recall": 0.876034 + }, + { + "t": 0.1, + "precision": 0.056239, + "recall": 0.896106 + }, + { + "t": 0.08, + "precision": 0.056364, + "recall": 0.917566 + }, + { + "t": 0.06, + "precision": 0.056428, + "recall": 0.938017 + }, + { + "t": 0.04, + "precision": 0.056366, + "recall": 0.956511 + }, + { + "t": 0.02, + "precision": 0.056362, + "recall": 0.976267 + }, + { + "t": 0.0, + "precision": 0.056389, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w055-ct-baseline.json b/docs/examples/bench/w055-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..de951f076d937c9786c7057047eb29aab391e49f --- /dev/null +++ b/docs/examples/bench/w055-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:18Z", + "scroll": "PHerc0139", + "segment": "w055", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w055_ink.png", + "sha256": "407d4526e39057eed7d402829ddc5853168197c005382352d45c92a8ac72a315", + "width": 640, + "height": 473, + "pxPerMM": 6.5607, + "alignTargetFile": "data/bench/w055_ink.png", + "alignTargetSha256": "407d4526e39057eed7d402829ddc5853168197c005382352d45c92a8ac72a315" + }, + "prediction": { + "name": "w055_ct.png", + "sha256": "dab0bba53acdd7d658ef4a2bf147d8bcf81016faae4d96402e1b1593a2aea178", + "format": "png", + "width": 640, + "height": 473, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.256452, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 302720, + "validFraction": 1.0, + "pearson": 0.256452, + "blurSigmaPx": 6.0, + "blurSigmaUM": 914.5, + "pearsonMatchedBlur": 0.448996, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.072267, + "dice": 0.134793, + "precision": 0.078053, + "recall": 0.493633, + "refPositiveFraction": 0.064859, + "predPositiveFraction": 0.410188, + "ap": 0.08418, + "bestF1": { + "f1": 0.142476, + "threshold": 0.1 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.145985, + "recall": 0.001019 + }, + { + "t": 0.98, + "precision": 0.172619, + "recall": 0.002954 + }, + { + "t": 0.96, + "precision": 0.161085, + "recall": 0.005144 + }, + { + "t": 0.94, + "precision": 0.146209, + "recall": 0.008251 + }, + { + "t": 0.92, + "precision": 0.143268, + "recall": 0.012682 + }, + { + "t": 0.9, + "precision": 0.142751, + "recall": 0.019558 + }, + { + "t": 0.88, + "precision": 0.133753, + "recall": 0.027045 + }, + { + "t": 0.86, + "precision": 0.128037, + "recall": 0.036773 + }, + { + "t": 0.84, + "precision": 0.123493, + "recall": 0.049557 + }, + { + "t": 0.82, + "precision": 0.116942, + "recall": 0.063563 + }, + { + "t": 0.8, + "precision": 0.108789, + "recall": 0.081644 + }, + { + "t": 0.78, + "precision": 0.104327, + "recall": 0.100693 + }, + { + "t": 0.76, + "precision": 0.099294, + "recall": 0.120302 + }, + { + "t": 0.74, + "precision": 0.095548, + "recall": 0.142661 + }, + { + "t": 0.72, + "precision": 0.09118, + "recall": 0.164918 + }, + { + "t": 0.7, + "precision": 0.088983, + "recall": 0.191861 + }, + { + "t": 0.68, + "precision": 0.086402, + "recall": 0.218091 + }, + { + "t": 0.66, + "precision": 0.084773, + "recall": 0.246868 + }, + { + "t": 0.64, + "precision": 0.08304, + "recall": 0.275491 + }, + { + "t": 0.62, + "precision": 0.08188, + "recall": 0.305338 + }, + { + "t": 0.6, + "precision": 0.081099, + "recall": 0.343842 + }, + { + "t": 0.58, + "precision": 0.080092, + "recall": 0.373994 + }, + { + "t": 0.56, + "precision": 0.079373, + "recall": 0.404095 + }, + { + "t": 0.54, + "precision": 0.078831, + "recall": 0.434094 + }, + { + "t": 0.52, + "precision": 0.078375, + "recall": 0.463685 + }, + { + "t": 0.5, + "precision": 0.078053, + "recall": 0.493633 + }, + { + "t": 0.48, + "precision": 0.077632, + "recall": 0.521901 + }, + { + "t": 0.46, + "precision": 0.077555, + "recall": 0.551747 + }, + { + "t": 0.44, + "precision": 0.077474, + "recall": 0.581135 + }, + { + "t": 0.42, + "precision": 0.077436, + "recall": 0.609402 + }, + { + "t": 0.4, + "precision": 0.077239, + "recall": 0.640471 + }, + { + "t": 0.38, + "precision": 0.077181, + "recall": 0.66665 + }, + { + "t": 0.36, + "precision": 0.077222, + "recall": 0.69237 + }, + { + "t": 0.34, + "precision": 0.077167, + "recall": 0.716563 + }, + { + "t": 0.32, + "precision": 0.077112, + "recall": 0.738769 + }, + { + "t": 0.3, + "precision": 0.077288, + "recall": 0.763064 + }, + { + "t": 0.28, + "precision": 0.077391, + "recall": 0.784761 + }, + { + "t": 0.26, + "precision": 0.077315, + "recall": 0.804523 + }, + { + "t": 0.24, + "precision": 0.077375, + "recall": 0.824539 + }, + { + "t": 0.22, + "precision": 0.077332, + "recall": 0.842212 + }, + { + "t": 0.2, + "precision": 0.077358, + "recall": 0.862636 + }, + { + "t": 0.18, + "precision": 0.077316, + "recall": 0.878069 + }, + { + "t": 0.16, + "precision": 0.07736, + "recall": 0.893552 + }, + { + "t": 0.14, + "precision": 0.077297, + "recall": 0.906896 + }, + { + "t": 0.12, + "precision": 0.07714, + "recall": 0.918865 + }, + { + "t": 0.1, + "precision": 0.077129, + "recall": 0.932668 + }, + { + "t": 0.08, + "precision": 0.076989, + "recall": 0.944637 + }, + { + "t": 0.06, + "precision": 0.076916, + "recall": 0.957268 + }, + { + "t": 0.04, + "precision": 0.076809, + "recall": 0.96939 + }, + { + "t": 0.02, + "precision": 0.076493, + "recall": 0.978863 + }, + { + "t": 0.0, + "precision": 0.064859, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w055-ink9um.json b/docs/examples/bench/w055-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..cec2bd9c339cf5b381b146cea2eb902cedbec475 --- /dev/null +++ b/docs/examples/bench/w055-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:17Z", + "scroll": "PHerc0139", + "segment": "w055", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w055_ink.png", + "sha256": "407d4526e39057eed7d402829ddc5853168197c005382352d45c92a8ac72a315", + "width": 640, + "height": 473, + "pxPerMM": 6.5607, + "alignTargetFile": "data/bench/w055_ink.png", + "alignTargetSha256": "407d4526e39057eed7d402829ddc5853168197c005382352d45c92a8ac72a315" + }, + "prediction": { + "name": "w055_gpu.png", + "sha256": "dc8f4c5ba397db806b852cb0075a23c008df3bdd2c8b554eee477d2c9dbaf86a", + "format": "png", + "width": 640, + "height": 473, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.5, + "ty": 0.5, + "sx": 0.99986, + "sy": 1.00212, + "ncc": 0.303177, + "coarseNcc": 0.451507, + "evals": 32210, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 302247, + "validFraction": 0.998438, + "pearson": 0.303177, + "blurSigmaPx": 2.0, + "blurSigmaUM": 304.8, + "pearsonMatchedBlur": 0.340059, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.130339, + "dice": 0.23062, + "precision": 0.241123, + "recall": 0.220994, + "refPositiveFraction": 0.06496, + "predPositiveFraction": 0.059537, + "ap": 0.171014, + "bestF1": { + "f1": 0.239023, + "threshold": 0.28 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.666667, + "recall": 0.000102 + }, + { + "t": 0.98, + "precision": 0.450249, + "recall": 0.018437 + }, + { + "t": 0.96, + "precision": 0.434343, + "recall": 0.028471 + }, + { + "t": 0.94, + "precision": 0.417607, + "recall": 0.03769 + }, + { + "t": 0.92, + "precision": 0.403855, + "recall": 0.04589 + }, + { + "t": 0.9, + "precision": 0.388638, + "recall": 0.052613 + }, + { + "t": 0.88, + "precision": 0.369759, + "recall": 0.059285 + }, + { + "t": 0.86, + "precision": 0.360011, + "recall": 0.06667 + }, + { + "t": 0.84, + "precision": 0.353141, + "recall": 0.074157 + }, + { + "t": 0.82, + "precision": 0.341192, + "recall": 0.081084 + }, + { + "t": 0.8, + "precision": 0.33513, + "recall": 0.088673 + }, + { + "t": 0.78, + "precision": 0.32516, + "recall": 0.095905 + }, + { + "t": 0.76, + "precision": 0.318986, + "recall": 0.1038 + }, + { + "t": 0.74, + "precision": 0.311361, + "recall": 0.111388 + }, + { + "t": 0.72, + "precision": 0.306157, + "recall": 0.119538 + }, + { + "t": 0.7, + "precision": 0.299869, + "recall": 0.127941 + }, + { + "t": 0.68, + "precision": 0.293028, + "recall": 0.136141 + }, + { + "t": 0.66, + "precision": 0.28668, + "recall": 0.144698 + }, + { + "t": 0.64, + "precision": 0.28102, + "recall": 0.153764 + }, + { + "t": 0.62, + "precision": 0.274213, + "recall": 0.162371 + }, + { + "t": 0.6, + "precision": 0.268312, + "recall": 0.171081 + }, + { + "t": 0.58, + "precision": 0.263596, + "recall": 0.180707 + }, + { + "t": 0.56, + "precision": 0.257764, + "recall": 0.190231 + }, + { + "t": 0.54, + "precision": 0.25088, + "recall": 0.199552 + }, + { + "t": 0.52, + "precision": 0.246436, + "recall": 0.2104 + }, + { + "t": 0.5, + "precision": 0.241123, + "recall": 0.220994 + }, + { + "t": 0.48, + "precision": 0.234308, + "recall": 0.23062 + }, + { + "t": 0.46, + "precision": 0.227441, + "recall": 0.240705 + }, + { + "t": 0.44, + "precision": 0.221101, + "recall": 0.252114 + }, + { + "t": 0.42, + "precision": 0.21455, + "recall": 0.263166 + }, + { + "t": 0.4, + "precision": 0.209603, + "recall": 0.275695 + }, + { + "t": 0.38, + "precision": 0.202881, + "recall": 0.2869 + }, + { + "t": 0.36, + "precision": 0.1976, + "recall": 0.30111 + }, + { + "t": 0.34, + "precision": 0.19213, + "recall": 0.31532 + }, + { + "t": 0.32, + "precision": 0.186929, + "recall": 0.329378 + }, + { + "t": 0.3, + "precision": 0.182285, + "recall": 0.34537 + }, + { + "t": 0.28, + "precision": 0.177856, + "recall": 0.364317 + }, + { + "t": 0.26, + "precision": 0.172159, + "recall": 0.381838 + }, + { + "t": 0.24, + "precision": 0.166388, + "recall": 0.400071 + }, + { + "t": 0.22, + "precision": 0.161126, + "recall": 0.420852 + }, + { + "t": 0.2, + "precision": 0.15578, + "recall": 0.442752 + }, + { + "t": 0.18, + "precision": 0.149849, + "recall": 0.466283 + }, + { + "t": 0.16, + "precision": 0.14496, + "recall": 0.494448 + }, + { + "t": 0.14, + "precision": 0.139366, + "recall": 0.524295 + }, + { + "t": 0.12, + "precision": 0.133356, + "recall": 0.558012 + }, + { + "t": 0.1, + "precision": 0.127105, + "recall": 0.596262 + }, + { + "t": 0.08, + "precision": 0.119986, + "recall": 0.639859 + }, + { + "t": 0.06, + "precision": 0.112129, + "recall": 0.694204 + }, + { + "t": 0.04, + "precision": 0.10302, + "recall": 0.760874 + }, + { + "t": 0.02, + "precision": 0.091784, + "recall": 0.854742 + }, + { + "t": 0.0, + "precision": 0.06496, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w055-noise-baseline.json b/docs/examples/bench/w055-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..57f7437b0f18788c119b7518155dc72d786cda8a --- /dev/null +++ b/docs/examples/bench/w055-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:18Z", + "scroll": "PHerc0139", + "segment": "w055", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w055_ink.png", + "sha256": "407d4526e39057eed7d402829ddc5853168197c005382352d45c92a8ac72a315", + "width": 640, + "height": 473, + "pxPerMM": 6.5607, + "alignTargetFile": "data/bench/w055_ink.png", + "alignTargetSha256": "407d4526e39057eed7d402829ddc5853168197c005382352d45c92a8ac72a315" + }, + "prediction": { + "name": "noise-seed55.png", + "sha256": "2364204dccd14b4e4850be4cc087a90514ad359bc09117f9cf8ba6b2fd70edc5", + "format": "png", + "width": 640, + "height": 473, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(55).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.003063, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 302720, + "validFraction": 1.0, + "pearson": 0.003063, + "blurSigmaPx": 0.0, + "blurSigmaUM": 0.0, + "pearsonMatchedBlur": 0.003063, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.061678, + "dice": 0.11619, + "precision": 0.065652, + "recall": 0.504686, + "refPositiveFraction": 0.064859, + "predPositiveFraction": 0.498586, + "ap": 0.06571, + "bestF1": { + "f1": 0.121816, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.068931, + "recall": 0.021188 + }, + { + "t": 0.96, + "precision": 0.067922, + "recall": 0.04151 + }, + { + "t": 0.94, + "precision": 0.069016, + "recall": 0.062545 + }, + { + "t": 0.92, + "precision": 0.068903, + "recall": 0.083376 + }, + { + "t": 0.9, + "precision": 0.067556, + "recall": 0.102119 + }, + { + "t": 0.88, + "precision": 0.066736, + "recall": 0.121015 + }, + { + "t": 0.86, + "precision": 0.066425, + "recall": 0.141133 + }, + { + "t": 0.84, + "precision": 0.065997, + "recall": 0.159978 + }, + { + "t": 0.82, + "precision": 0.065785, + "recall": 0.179281 + }, + { + "t": 0.8, + "precision": 0.065478, + "recall": 0.202047 + }, + { + "t": 0.78, + "precision": 0.065399, + "recall": 0.221402 + }, + { + "t": 0.76, + "precision": 0.065283, + "recall": 0.240552 + }, + { + "t": 0.74, + "precision": 0.065346, + "recall": 0.260874 + }, + { + "t": 0.72, + "precision": 0.065762, + "recall": 0.282164 + }, + { + "t": 0.7, + "precision": 0.066002, + "recall": 0.303249 + }, + { + "t": 0.68, + "precision": 0.065786, + "recall": 0.321891 + }, + { + "t": 0.66, + "precision": 0.065516, + "recall": 0.340634 + }, + { + "t": 0.64, + "precision": 0.065437, + "recall": 0.360039 + }, + { + "t": 0.62, + "precision": 0.065444, + "recall": 0.379953 + }, + { + "t": 0.6, + "precision": 0.06549, + "recall": 0.404197 + }, + { + "t": 0.58, + "precision": 0.06538, + "recall": 0.422991 + }, + { + "t": 0.56, + "precision": 0.065501, + "recall": 0.443618 + }, + { + "t": 0.54, + "precision": 0.065531, + "recall": 0.46394 + }, + { + "t": 0.52, + "precision": 0.065637, + "recall": 0.484619 + }, + { + "t": 0.5, + "precision": 0.065652, + "recall": 0.504686 + }, + { + "t": 0.48, + "precision": 0.065619, + "recall": 0.524498 + }, + { + "t": 0.46, + "precision": 0.065525, + "recall": 0.543853 + }, + { + "t": 0.44, + "precision": 0.06552, + "recall": 0.563716 + }, + { + "t": 0.42, + "precision": 0.065373, + "recall": 0.582255 + }, + { + "t": 0.4, + "precision": 0.065394, + "recall": 0.606295 + }, + { + "t": 0.38, + "precision": 0.065407, + "recall": 0.626159 + }, + { + "t": 0.36, + "precision": 0.065418, + "recall": 0.645869 + }, + { + "t": 0.34, + "precision": 0.065251, + "recall": 0.663797 + }, + { + "t": 0.32, + "precision": 0.065149, + "recall": 0.682184 + }, + { + "t": 0.3, + "precision": 0.06526, + "recall": 0.702913 + }, + { + "t": 0.28, + "precision": 0.065129, + "recall": 0.721402 + }, + { + "t": 0.26, + "precision": 0.06516, + "recall": 0.741265 + }, + { + "t": 0.24, + "precision": 0.065248, + "recall": 0.761842 + }, + { + "t": 0.22, + "precision": 0.065146, + "recall": 0.780483 + }, + { + "t": 0.2, + "precision": 0.06518, + "recall": 0.804574 + }, + { + "t": 0.18, + "precision": 0.064955, + "recall": 0.821789 + }, + { + "t": 0.16, + "precision": 0.064895, + "recall": 0.840481 + }, + { + "t": 0.14, + "precision": 0.064928, + "recall": 0.860497 + }, + { + "t": 0.12, + "precision": 0.064941, + "recall": 0.880004 + }, + { + "t": 0.1, + "precision": 0.064988, + "recall": 0.899969 + }, + { + "t": 0.08, + "precision": 0.064935, + "recall": 0.918967 + }, + { + "t": 0.06, + "precision": 0.064931, + "recall": 0.938576 + }, + { + "t": 0.04, + "precision": 0.064925, + "recall": 0.957777 + }, + { + "t": 0.02, + "precision": 0.06495, + "recall": 0.977845 + }, + { + "t": 0.0, + "precision": 0.064859, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w056-ct-baseline.json b/docs/examples/bench/w056-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..5ba31d707cda8cc4f62e2c483d1ab9e36c6c8000 --- /dev/null +++ b/docs/examples/bench/w056-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:22Z", + "scroll": "PHerc0139", + "segment": "w056", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w056_ink.png", + "sha256": "d2924fa38c99d7163bef5be976018ba08d85e038def67fc85cd2f3d7a35974ae", + "width": 640, + "height": 471, + "pxPerMM": 7.0183, + "alignTargetFile": "data/bench/w056_ink.png", + "alignTargetSha256": "d2924fa38c99d7163bef5be976018ba08d85e038def67fc85cd2f3d7a35974ae" + }, + "prediction": { + "name": "w056_ct.png", + "sha256": "156992b325cb9b192ed76c07a80d904b34b4a49139b20e5e81b662b2387c3798", + "format": "png", + "width": 640, + "height": 471, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.23305, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 301440, + "validFraction": 1.0, + "pearson": 0.23305, + "blurSigmaPx": 6.0, + "blurSigmaUM": 854.9, + "pearsonMatchedBlur": 0.397688, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.077846, + "dice": 0.144448, + "precision": 0.083248, + "recall": 0.545399, + "refPositiveFraction": 0.071099, + "predPositiveFraction": 0.465804, + "ap": 0.087624, + "bestF1": { + "f1": 0.151384, + "threshold": 0.08 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.134921, + "recall": 0.000793 + }, + { + "t": 0.98, + "precision": 0.139344, + "recall": 0.00238 + }, + { + "t": 0.96, + "precision": 0.131793, + "recall": 0.004526 + }, + { + "t": 0.94, + "precision": 0.135525, + "recall": 0.007885 + }, + { + "t": 0.92, + "precision": 0.129837, + "recall": 0.011898 + }, + { + "t": 0.9, + "precision": 0.130693, + "recall": 0.018477 + }, + { + "t": 0.88, + "precision": 0.125861, + "recall": 0.025569 + }, + { + "t": 0.86, + "precision": 0.119321, + "recall": 0.034434 + }, + { + "t": 0.84, + "precision": 0.115156, + "recall": 0.046193 + }, + { + "t": 0.82, + "precision": 0.110586, + "recall": 0.060004 + }, + { + "t": 0.8, + "precision": 0.106133, + "recall": 0.080347 + }, + { + "t": 0.78, + "precision": 0.102157, + "recall": 0.100317 + }, + { + "t": 0.76, + "precision": 0.098345, + "recall": 0.121454 + }, + { + "t": 0.74, + "precision": 0.094968, + "recall": 0.145483 + }, + { + "t": 0.72, + "precision": 0.092941, + "recall": 0.173292 + }, + { + "t": 0.7, + "precision": 0.091563, + "recall": 0.203667 + }, + { + "t": 0.68, + "precision": 0.090581, + "recall": 0.236702 + }, + { + "t": 0.66, + "precision": 0.088899, + "recall": 0.268617 + }, + { + "t": 0.64, + "precision": 0.087596, + "recall": 0.301978 + }, + { + "t": 0.62, + "precision": 0.086837, + "recall": 0.337579 + }, + { + "t": 0.6, + "precision": 0.085592, + "recall": 0.377706 + }, + { + "t": 0.58, + "precision": 0.084833, + "recall": 0.411814 + }, + { + "t": 0.56, + "precision": 0.084402, + "recall": 0.446389 + }, + { + "t": 0.54, + "precision": 0.083785, + "recall": 0.479003 + }, + { + "t": 0.52, + "precision": 0.083411, + "recall": 0.511898 + }, + { + "t": 0.5, + "precision": 0.083248, + "recall": 0.545399 + }, + { + "t": 0.48, + "precision": 0.082956, + "recall": 0.575448 + }, + { + "t": 0.46, + "precision": 0.082621, + "recall": 0.604657 + }, + { + "t": 0.44, + "precision": 0.082282, + "recall": 0.631579 + }, + { + "t": 0.42, + "precision": 0.082023, + "recall": 0.657615 + }, + { + "t": 0.4, + "precision": 0.082046, + "recall": 0.689296 + }, + { + "t": 0.38, + "precision": 0.082024, + "recall": 0.713699 + }, + { + "t": 0.36, + "precision": 0.081964, + "recall": 0.736702 + }, + { + "t": 0.34, + "precision": 0.081824, + "recall": 0.757885 + }, + { + "t": 0.32, + "precision": 0.081784, + "recall": 0.778229 + }, + { + "t": 0.3, + "precision": 0.08166, + "recall": 0.797079 + }, + { + "t": 0.28, + "precision": 0.081637, + "recall": 0.81481 + }, + { + "t": 0.26, + "precision": 0.081742, + "recall": 0.833427 + }, + { + "t": 0.24, + "precision": 0.08178, + "recall": 0.849711 + }, + { + "t": 0.22, + "precision": 0.081916, + "recall": 0.865668 + }, + { + "t": 0.2, + "precision": 0.081938, + "recall": 0.881626 + }, + { + "t": 0.18, + "precision": 0.082006, + "recall": 0.89427 + }, + { + "t": 0.16, + "precision": 0.081992, + "recall": 0.905282 + }, + { + "t": 0.14, + "precision": 0.082072, + "recall": 0.916807 + }, + { + "t": 0.12, + "precision": 0.082183, + "recall": 0.928005 + }, + { + "t": 0.1, + "precision": 0.082245, + "recall": 0.93841 + }, + { + "t": 0.08, + "precision": 0.082259, + "recall": 0.948208 + }, + { + "t": 0.06, + "precision": 0.082159, + "recall": 0.95754 + }, + { + "t": 0.04, + "precision": 0.082052, + "recall": 0.967199 + }, + { + "t": 0.02, + "precision": 0.081853, + "recall": 0.97751 + }, + { + "t": 0.0, + "precision": 0.071099, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w056-ink9um.json b/docs/examples/bench/w056-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..f9e22be4bc9c8c55f9f75f427f7b26bb382672fc --- /dev/null +++ b/docs/examples/bench/w056-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:22Z", + "scroll": "PHerc0139", + "segment": "w056", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w056_ink.png", + "sha256": "d2924fa38c99d7163bef5be976018ba08d85e038def67fc85cd2f3d7a35974ae", + "width": 640, + "height": 471, + "pxPerMM": 7.0183, + "alignTargetFile": "data/bench/w056_ink.png", + "alignTargetSha256": "d2924fa38c99d7163bef5be976018ba08d85e038def67fc85cd2f3d7a35974ae" + }, + "prediction": { + "name": "w056_gpu.png", + "sha256": "c2a59ccce65176ac716e507dd584b59128174330f1df1f285013921c87142539", + "format": "png", + "width": 640, + "height": 471, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.438, + "ty": 0.531, + "sx": 1.00012, + "sy": 1.00037, + "ncc": 0.31456, + "coarseNcc": 0.43439, + "evals": 32086, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 300330, + "validFraction": 0.996318, + "pearson": 0.31456, + "blurSigmaPx": 2.0, + "blurSigmaUM": 285.0, + "pearsonMatchedBlur": 0.350443, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.145508, + "dice": 0.25405, + "precision": 0.230854, + "recall": 0.282428, + "refPositiveFraction": 0.071362, + "predPositiveFraction": 0.087304, + "ap": 0.187315, + "bestF1": { + "f1": 0.260473, + "threshold": 0.4 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.541667, + "recall": 0.000607 + }, + { + "t": 0.98, + "precision": 0.446025, + "recall": 0.032195 + }, + { + "t": 0.96, + "precision": 0.417826, + "recall": 0.044839 + }, + { + "t": 0.94, + "precision": 0.400653, + "recall": 0.057297 + }, + { + "t": 0.92, + "precision": 0.386472, + "recall": 0.067982 + }, + { + "t": 0.9, + "precision": 0.375419, + "recall": 0.078387 + }, + { + "t": 0.88, + "precision": 0.357457, + "recall": 0.087346 + }, + { + "t": 0.86, + "precision": 0.345351, + "recall": 0.096538 + }, + { + "t": 0.84, + "precision": 0.334272, + "recall": 0.105263 + }, + { + "t": 0.82, + "precision": 0.324324, + "recall": 0.114782 + }, + { + "t": 0.8, + "precision": 0.313574, + "recall": 0.123414 + }, + { + "t": 0.78, + "precision": 0.306398, + "recall": 0.133399 + }, + { + "t": 0.76, + "precision": 0.299345, + "recall": 0.14287 + }, + { + "t": 0.74, + "precision": 0.293912, + "recall": 0.152716 + }, + { + "t": 0.72, + "precision": 0.287446, + "recall": 0.162607 + }, + { + "t": 0.7, + "precision": 0.281899, + "recall": 0.172359 + }, + { + "t": 0.68, + "precision": 0.275687, + "recall": 0.182158 + }, + { + "t": 0.66, + "precision": 0.270769, + "recall": 0.192982 + }, + { + "t": 0.64, + "precision": 0.266622, + "recall": 0.203947 + }, + { + "t": 0.62, + "precision": 0.259866, + "recall": 0.213233 + }, + { + "t": 0.6, + "precision": 0.254051, + "recall": 0.223824 + }, + { + "t": 0.58, + "precision": 0.248823, + "recall": 0.234276 + }, + { + "t": 0.56, + "precision": 0.244761, + "recall": 0.246314 + }, + { + "t": 0.54, + "precision": 0.239861, + "recall": 0.258025 + }, + { + "t": 0.52, + "precision": 0.235711, + "recall": 0.270157 + }, + { + "t": 0.5, + "precision": 0.230854, + "recall": 0.282428 + }, + { + "t": 0.48, + "precision": 0.22562, + "recall": 0.295026 + }, + { + "t": 0.46, + "precision": 0.221564, + "recall": 0.308744 + }, + { + "t": 0.44, + "precision": 0.216653, + "recall": 0.323068 + }, + { + "t": 0.42, + "precision": 0.211536, + "recall": 0.336926 + }, + { + "t": 0.4, + "precision": 0.206899, + "recall": 0.351484 + }, + { + "t": 0.38, + "precision": 0.201541, + "recall": 0.364828 + }, + { + "t": 0.36, + "precision": 0.196247, + "recall": 0.379199 + }, + { + "t": 0.34, + "precision": 0.190848, + "recall": 0.393664 + }, + { + "t": 0.32, + "precision": 0.185786, + "recall": 0.409108 + }, + { + "t": 0.3, + "precision": 0.18069, + "recall": 0.425952 + }, + { + "t": 0.28, + "precision": 0.176802, + "recall": 0.446715 + }, + { + "t": 0.26, + "precision": 0.172396, + "recall": 0.467059 + }, + { + "t": 0.24, + "precision": 0.167306, + "recall": 0.486702 + }, + { + "t": 0.22, + "precision": 0.161733, + "recall": 0.507092 + }, + { + "t": 0.2, + "precision": 0.156203, + "recall": 0.529349 + }, + { + "t": 0.18, + "precision": 0.150915, + "recall": 0.554031 + }, + { + "t": 0.16, + "precision": 0.145367, + "recall": 0.579181 + }, + { + "t": 0.14, + "precision": 0.139939, + "recall": 0.607689 + }, + { + "t": 0.12, + "precision": 0.134476, + "recall": 0.640164 + }, + { + "t": 0.1, + "precision": 0.127981, + "recall": 0.673526 + }, + { + "t": 0.08, + "precision": 0.12155, + "recall": 0.713139 + }, + { + "t": 0.06, + "precision": 0.114402, + "recall": 0.759985 + }, + { + "t": 0.04, + "precision": 0.106058, + "recall": 0.817469 + }, + { + "t": 0.02, + "precision": 0.095425, + "recall": 0.890491 + }, + { + "t": 0.0, + "precision": 0.071362, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w056-noise-baseline.json b/docs/examples/bench/w056-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..e4556fb758cacab432326bd27c882be8f2ed6a85 --- /dev/null +++ b/docs/examples/bench/w056-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:23Z", + "scroll": "PHerc0139", + "segment": "w056", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w056_ink.png", + "sha256": "d2924fa38c99d7163bef5be976018ba08d85e038def67fc85cd2f3d7a35974ae", + "width": 640, + "height": 471, + "pxPerMM": 7.0183, + "alignTargetFile": "data/bench/w056_ink.png", + "alignTargetSha256": "d2924fa38c99d7163bef5be976018ba08d85e038def67fc85cd2f3d7a35974ae" + }, + "prediction": { + "name": "noise-seed56.png", + "sha256": "ab1e4856e746c516ebe8dc6ec7cd429fdce722004b796056003cedf6b471d5a1", + "format": "png", + "width": 640, + "height": 471, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(56).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": -0.000703, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 301440, + "validFraction": 1.0, + "pearson": -0.000703, + "blurSigmaPx": 4.0, + "blurSigmaUM": 569.9, + "pearsonMatchedBlur": 0.000139, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.066129, + "dice": 0.124055, + "precision": 0.070882, + "recall": 0.496547, + "refPositiveFraction": 0.071099, + "predPositiveFraction": 0.498066, + "ap": 0.070887, + "bestF1": { + "f1": 0.132781, + "threshold": 0.02 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.073769, + "recall": 0.020063 + }, + { + "t": 0.96, + "precision": 0.072495, + "recall": 0.03966 + }, + { + "t": 0.94, + "precision": 0.071583, + "recall": 0.058791 + }, + { + "t": 0.92, + "precision": 0.069761, + "recall": 0.076521 + }, + { + "t": 0.9, + "precision": 0.069313, + "recall": 0.094998 + }, + { + "t": 0.88, + "precision": 0.06935, + "recall": 0.114222 + }, + { + "t": 0.86, + "precision": 0.069994, + "recall": 0.134612 + }, + { + "t": 0.84, + "precision": 0.070491, + "recall": 0.155095 + }, + { + "t": 0.82, + "precision": 0.070627, + "recall": 0.174785 + }, + { + "t": 0.8, + "precision": 0.070955, + "recall": 0.199095 + }, + { + "t": 0.78, + "precision": 0.070627, + "recall": 0.217805 + }, + { + "t": 0.76, + "precision": 0.07035, + "recall": 0.236515 + }, + { + "t": 0.74, + "precision": 0.070113, + "recall": 0.255133 + }, + { + "t": 0.72, + "precision": 0.07048, + "recall": 0.275896 + }, + { + "t": 0.7, + "precision": 0.070455, + "recall": 0.295166 + }, + { + "t": 0.68, + "precision": 0.070646, + "recall": 0.315743 + }, + { + "t": 0.66, + "precision": 0.070438, + "recall": 0.334546 + }, + { + "t": 0.64, + "precision": 0.070812, + "recall": 0.35615 + }, + { + "t": 0.62, + "precision": 0.070882, + "recall": 0.375793 + }, + { + "t": 0.6, + "precision": 0.070956, + "recall": 0.399729 + }, + { + "t": 0.58, + "precision": 0.070911, + "recall": 0.419186 + }, + { + "t": 0.56, + "precision": 0.070843, + "recall": 0.437943 + }, + { + "t": 0.54, + "precision": 0.070907, + "recall": 0.457867 + }, + { + "t": 0.52, + "precision": 0.071052, + "recall": 0.478117 + }, + { + "t": 0.5, + "precision": 0.070882, + "recall": 0.496547 + }, + { + "t": 0.48, + "precision": 0.071096, + "recall": 0.517404 + }, + { + "t": 0.46, + "precision": 0.070865, + "recall": 0.535274 + }, + { + "t": 0.44, + "precision": 0.070824, + "recall": 0.554871 + }, + { + "t": 0.42, + "precision": 0.070779, + "recall": 0.573908 + }, + { + "t": 0.4, + "precision": 0.070632, + "recall": 0.596258 + }, + { + "t": 0.38, + "precision": 0.0707, + "recall": 0.616508 + }, + { + "t": 0.36, + "precision": 0.070665, + "recall": 0.635405 + }, + { + "t": 0.34, + "precision": 0.07081, + "recall": 0.656542 + }, + { + "t": 0.32, + "precision": 0.070877, + "recall": 0.676512 + }, + { + "t": 0.3, + "precision": 0.070984, + "recall": 0.696948 + }, + { + "t": 0.28, + "precision": 0.070969, + "recall": 0.716312 + }, + { + "t": 0.26, + "precision": 0.071063, + "recall": 0.737309 + }, + { + "t": 0.24, + "precision": 0.071054, + "recall": 0.756579 + }, + { + "t": 0.22, + "precision": 0.0711, + "recall": 0.776829 + }, + { + "t": 0.2, + "precision": 0.071175, + "recall": 0.800765 + }, + { + "t": 0.18, + "precision": 0.07115, + "recall": 0.820082 + }, + { + "t": 0.16, + "precision": 0.071188, + "recall": 0.840566 + }, + { + "t": 0.14, + "precision": 0.071208, + "recall": 0.860302 + }, + { + "t": 0.12, + "precision": 0.071193, + "recall": 0.879806 + }, + { + "t": 0.1, + "precision": 0.071146, + "recall": 0.898936 + }, + { + "t": 0.08, + "precision": 0.071191, + "recall": 0.919186 + }, + { + "t": 0.06, + "precision": 0.071152, + "recall": 0.93785 + }, + { + "t": 0.04, + "precision": 0.07106, + "recall": 0.95614 + }, + { + "t": 0.02, + "precision": 0.071225, + "recall": 0.97807 + }, + { + "t": 0.0, + "precision": 0.071099, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w057-ct-baseline.json b/docs/examples/bench/w057-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..c6828084fe2441e0411d049626f8c9c435903e71 --- /dev/null +++ b/docs/examples/bench/w057-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:30Z", + "scroll": "PHerc0139", + "segment": "w057", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w057_ink.png", + "sha256": "754a17d5832e7b22c2ec1183c71a49dcabb65c543287ab7f2818dc62be64d8b0", + "width": 640, + "height": 496, + "pxPerMM": 6.99, + "alignTargetFile": "data/bench/w057_ink.png", + "alignTargetSha256": "754a17d5832e7b22c2ec1183c71a49dcabb65c543287ab7f2818dc62be64d8b0" + }, + "prediction": { + "name": "w057_ct.png", + "sha256": "14ef32a30434a86f24c3395c29d496d3602b336756304f5e32bc9c18148e1ac2", + "format": "png", + "width": 640, + "height": 496, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.273854, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 317440, + "validFraction": 1.0, + "pearson": 0.273854, + "blurSigmaPx": 6.0, + "blurSigmaUM": 858.4, + "pearsonMatchedBlur": 0.514485, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.045835, + "dice": 0.087652, + "precision": 0.04739, + "recall": 0.582778, + "refPositiveFraction": 0.040206, + "predPositiveFraction": 0.494437, + "ap": 0.050328, + "bestF1": { + "f1": 0.088772, + "threshold": 0.06 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.030534, + "recall": 0.000313 + }, + { + "t": 0.98, + "precision": 0.053985, + "recall": 0.001645 + }, + { + "t": 0.96, + "precision": 0.064885, + "recall": 0.003996 + }, + { + "t": 0.94, + "precision": 0.067202, + "recall": 0.007208 + }, + { + "t": 0.92, + "precision": 0.070635, + "recall": 0.012458 + }, + { + "t": 0.9, + "precision": 0.073551, + "recall": 0.01998 + }, + { + "t": 0.88, + "precision": 0.070719, + "recall": 0.027893 + }, + { + "t": 0.86, + "precision": 0.066584, + "recall": 0.037844 + }, + { + "t": 0.84, + "precision": 0.064513, + "recall": 0.05085 + }, + { + "t": 0.82, + "precision": 0.061756, + "recall": 0.065423 + }, + { + "t": 0.8, + "precision": 0.059868, + "recall": 0.087675 + }, + { + "t": 0.78, + "precision": 0.058426, + "recall": 0.109614 + }, + { + "t": 0.76, + "precision": 0.057744, + "recall": 0.135313 + }, + { + "t": 0.74, + "precision": 0.056302, + "recall": 0.163206 + }, + { + "t": 0.72, + "precision": 0.055007, + "recall": 0.193058 + }, + { + "t": 0.7, + "precision": 0.053354, + "recall": 0.222362 + }, + { + "t": 0.68, + "precision": 0.052361, + "recall": 0.256131 + }, + { + "t": 0.66, + "precision": 0.051823, + "recall": 0.292878 + }, + { + "t": 0.64, + "precision": 0.05079, + "recall": 0.327509 + }, + { + "t": 0.62, + "precision": 0.049875, + "recall": 0.361514 + }, + { + "t": 0.6, + "precision": 0.049064, + "recall": 0.404685 + }, + { + "t": 0.58, + "precision": 0.048959, + "recall": 0.44488 + }, + { + "t": 0.56, + "precision": 0.048377, + "recall": 0.480138 + }, + { + "t": 0.54, + "precision": 0.04794, + "recall": 0.514691 + }, + { + "t": 0.52, + "precision": 0.047648, + "recall": 0.549166 + }, + { + "t": 0.5, + "precision": 0.04739, + "recall": 0.582778 + }, + { + "t": 0.48, + "precision": 0.047081, + "recall": 0.614041 + }, + { + "t": 0.46, + "precision": 0.04695, + "recall": 0.645773 + }, + { + "t": 0.44, + "precision": 0.046876, + "recall": 0.67727 + }, + { + "t": 0.42, + "precision": 0.046712, + "recall": 0.704772 + }, + { + "t": 0.4, + "precision": 0.046528, + "recall": 0.735485 + }, + { + "t": 0.38, + "precision": 0.046416, + "recall": 0.759931 + }, + { + "t": 0.36, + "precision": 0.046305, + "recall": 0.782653 + }, + { + "t": 0.34, + "precision": 0.046295, + "recall": 0.804356 + }, + { + "t": 0.32, + "precision": 0.04628, + "recall": 0.824571 + }, + { + "t": 0.3, + "precision": 0.046314, + "recall": 0.844002 + }, + { + "t": 0.28, + "precision": 0.046318, + "recall": 0.861161 + }, + { + "t": 0.26, + "precision": 0.04632, + "recall": 0.876126 + }, + { + "t": 0.24, + "precision": 0.046277, + "recall": 0.889211 + }, + { + "t": 0.22, + "precision": 0.046328, + "recall": 0.902217 + }, + { + "t": 0.2, + "precision": 0.046417, + "recall": 0.916634 + }, + { + "t": 0.18, + "precision": 0.046458, + "recall": 0.926976 + }, + { + "t": 0.16, + "precision": 0.04646, + "recall": 0.935673 + }, + { + "t": 0.14, + "precision": 0.046478, + "recall": 0.943587 + }, + { + "t": 0.12, + "precision": 0.046474, + "recall": 0.950325 + }, + { + "t": 0.1, + "precision": 0.046458, + "recall": 0.955966 + }, + { + "t": 0.08, + "precision": 0.046522, + "recall": 0.963332 + }, + { + "t": 0.06, + "precision": 0.046516, + "recall": 0.969286 + }, + { + "t": 0.04, + "precision": 0.046436, + "recall": 0.974457 + }, + { + "t": 0.02, + "precision": 0.046385, + "recall": 0.982449 + }, + { + "t": 0.0, + "precision": 0.040206, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w057-ink9um.json b/docs/examples/bench/w057-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..17ea315bdfe0a9aa2d396ca528c41b7992b957bb --- /dev/null +++ b/docs/examples/bench/w057-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:30Z", + "scroll": "PHerc0139", + "segment": "w057", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w057_ink.png", + "sha256": "754a17d5832e7b22c2ec1183c71a49dcabb65c543287ab7f2818dc62be64d8b0", + "width": 640, + "height": 496, + "pxPerMM": 6.99, + "alignTargetFile": "data/bench/w057_ink.png", + "alignTargetSha256": "754a17d5832e7b22c2ec1183c71a49dcabb65c543287ab7f2818dc62be64d8b0" + }, + "prediction": { + "name": "w057_gpu.png", + "sha256": "0baae2f99ec2b4d643ddcb471b2605e9543a2e95e9dd37772c18f4abf513c263", + "format": "png", + "width": 640, + "height": 496, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.5, + "ty": 0.5, + "sx": 1.00007, + "sy": 0.99996, + "ncc": 0.215333, + "coarseNcc": 0.321414, + "evals": 34270, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 316305, + "validFraction": 0.996425, + "pearson": 0.215333, + "blurSigmaPx": 3.0, + "blurSigmaUM": 429.2, + "pearsonMatchedBlur": 0.260266, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.07363, + "dice": 0.13716, + "precision": 0.098568, + "recall": 0.225417, + "refPositiveFraction": 0.04035, + "predPositiveFraction": 0.092278, + "ap": 0.080963, + "bestF1": { + "f1": 0.137322, + "threshold": 0.48 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.2, + "recall": 0.000313 + }, + { + "t": 0.98, + "precision": 0.204682, + "recall": 0.023976 + }, + { + "t": 0.96, + "precision": 0.190147, + "recall": 0.034475 + }, + { + "t": 0.94, + "precision": 0.174123, + "recall": 0.042388 + }, + { + "t": 0.92, + "precision": 0.162439, + "recall": 0.049675 + }, + { + "t": 0.9, + "precision": 0.154663, + "recall": 0.05704 + }, + { + "t": 0.88, + "precision": 0.147466, + "recall": 0.063386 + }, + { + "t": 0.86, + "precision": 0.142288, + "recall": 0.069968 + }, + { + "t": 0.84, + "precision": 0.139152, + "recall": 0.078195 + }, + { + "t": 0.82, + "precision": 0.136539, + "recall": 0.085873 + }, + { + "t": 0.8, + "precision": 0.133787, + "recall": 0.093943 + }, + { + "t": 0.78, + "precision": 0.132015, + "recall": 0.103032 + }, + { + "t": 0.76, + "precision": 0.130037, + "recall": 0.111494 + }, + { + "t": 0.74, + "precision": 0.126967, + "recall": 0.119486 + }, + { + "t": 0.72, + "precision": 0.124599, + "recall": 0.127791 + }, + { + "t": 0.7, + "precision": 0.121322, + "recall": 0.135705 + }, + { + "t": 0.68, + "precision": 0.118813, + "recall": 0.143697 + }, + { + "t": 0.66, + "precision": 0.116511, + "recall": 0.152707 + }, + { + "t": 0.64, + "precision": 0.113628, + "recall": 0.161169 + }, + { + "t": 0.62, + "precision": 0.111773, + "recall": 0.17065 + }, + { + "t": 0.6, + "precision": 0.109051, + "recall": 0.178328 + }, + { + "t": 0.58, + "precision": 0.106685, + "recall": 0.187182 + }, + { + "t": 0.56, + "precision": 0.104386, + "recall": 0.1958 + }, + { + "t": 0.54, + "precision": 0.101706, + "recall": 0.204106 + }, + { + "t": 0.52, + "precision": 0.100227, + "recall": 0.214918 + }, + { + "t": 0.5, + "precision": 0.098568, + "recall": 0.225417 + }, + { + "t": 0.48, + "precision": 0.096756, + "recall": 0.236465 + }, + { + "t": 0.46, + "precision": 0.094654, + "recall": 0.246494 + }, + { + "t": 0.44, + "precision": 0.093045, + "recall": 0.258168 + }, + { + "t": 0.42, + "precision": 0.091833, + "recall": 0.271018 + }, + { + "t": 0.4, + "precision": 0.090209, + "recall": 0.283554 + }, + { + "t": 0.38, + "precision": 0.088811, + "recall": 0.297657 + }, + { + "t": 0.36, + "precision": 0.086663, + "recall": 0.309958 + }, + { + "t": 0.34, + "precision": 0.085309, + "recall": 0.325394 + }, + { + "t": 0.32, + "precision": 0.083756, + "recall": 0.341377 + }, + { + "t": 0.3, + "precision": 0.082586, + "recall": 0.358615 + }, + { + "t": 0.28, + "precision": 0.081134, + "recall": 0.376322 + }, + { + "t": 0.26, + "precision": 0.079408, + "recall": 0.394578 + }, + { + "t": 0.24, + "precision": 0.077411, + "recall": 0.412756 + }, + { + "t": 0.22, + "precision": 0.075895, + "recall": 0.435478 + }, + { + "t": 0.2, + "precision": 0.073832, + "recall": 0.457181 + }, + { + "t": 0.18, + "precision": 0.072241, + "recall": 0.48335 + }, + { + "t": 0.16, + "precision": 0.070377, + "recall": 0.510303 + }, + { + "t": 0.14, + "precision": 0.068757, + "recall": 0.544073 + }, + { + "t": 0.12, + "precision": 0.066593, + "recall": 0.57745 + }, + { + "t": 0.1, + "precision": 0.064574, + "recall": 0.616469 + }, + { + "t": 0.08, + "precision": 0.062929, + "recall": 0.668103 + }, + { + "t": 0.06, + "precision": 0.060578, + "recall": 0.724203 + }, + { + "t": 0.04, + "precision": 0.057448, + "recall": 0.790958 + }, + { + "t": 0.02, + "precision": 0.053063, + "recall": 0.872522 + }, + { + "t": 0.0, + "precision": 0.04035, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w057-noise-baseline.json b/docs/examples/bench/w057-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..68568d9ce5e849be1279b96415d6e7b7f2ec792c --- /dev/null +++ b/docs/examples/bench/w057-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:31Z", + "scroll": "PHerc0139", + "segment": "w057", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w057_ink.png", + "sha256": "754a17d5832e7b22c2ec1183c71a49dcabb65c543287ab7f2818dc62be64d8b0", + "width": 640, + "height": 496, + "pxPerMM": 6.99, + "alignTargetFile": "data/bench/w057_ink.png", + "alignTargetSha256": "754a17d5832e7b22c2ec1183c71a49dcabb65c543287ab7f2818dc62be64d8b0" + }, + "prediction": { + "name": "noise-seed57.png", + "sha256": "77517836578c7ce741c64b5a929421516431fff9dc8bcccb47078b91c67106fe", + "format": "png", + "width": 640, + "height": 496, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(57).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": -0.000821, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 317440, + "validFraction": 1.0, + "pearson": -0.000821, + "blurSigmaPx": 2.0, + "blurSigmaUM": 286.1, + "pearsonMatchedBlur": 0.000213, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.038772, + "dice": 0.074649, + "precision": 0.040332, + "recall": 0.500588, + "refPositiveFraction": 0.040206, + "predPositiveFraction": 0.499027, + "ap": 0.040142, + "bestF1": { + "f1": 0.077304, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.043464, + "recall": 0.021233 + }, + { + "t": 0.96, + "precision": 0.03844, + "recall": 0.037687 + }, + { + "t": 0.94, + "precision": 0.038459, + "recall": 0.056491 + }, + { + "t": 0.92, + "precision": 0.038228, + "recall": 0.075139 + }, + { + "t": 0.9, + "precision": 0.038623, + "recall": 0.094414 + }, + { + "t": 0.88, + "precision": 0.03912, + "recall": 0.114785 + }, + { + "t": 0.86, + "precision": 0.039114, + "recall": 0.133746 + }, + { + "t": 0.84, + "precision": 0.039558, + "recall": 0.154431 + }, + { + "t": 0.82, + "precision": 0.039659, + "recall": 0.174881 + }, + { + "t": 0.8, + "precision": 0.039412, + "recall": 0.196741 + }, + { + "t": 0.78, + "precision": 0.039817, + "recall": 0.218444 + }, + { + "t": 0.76, + "precision": 0.039961, + "recall": 0.238659 + }, + { + "t": 0.74, + "precision": 0.040266, + "recall": 0.260205 + }, + { + "t": 0.72, + "precision": 0.040636, + "recall": 0.282144 + }, + { + "t": 0.7, + "precision": 0.04056, + "recall": 0.301653 + }, + { + "t": 0.68, + "precision": 0.04042, + "recall": 0.320536 + }, + { + "t": 0.66, + "precision": 0.040125, + "recall": 0.337695 + }, + { + "t": 0.64, + "precision": 0.040011, + "recall": 0.356421 + }, + { + "t": 0.62, + "precision": 0.040006, + "recall": 0.375382 + }, + { + "t": 0.6, + "precision": 0.040265, + "recall": 0.401473 + }, + { + "t": 0.58, + "precision": 0.040177, + "recall": 0.420121 + }, + { + "t": 0.56, + "precision": 0.040348, + "recall": 0.442059 + }, + { + "t": 0.54, + "precision": 0.040405, + "recall": 0.462352 + }, + { + "t": 0.52, + "precision": 0.040315, + "recall": 0.480451 + }, + { + "t": 0.5, + "precision": 0.040332, + "recall": 0.500588 + }, + { + "t": 0.48, + "precision": 0.040476, + "recall": 0.522056 + }, + { + "t": 0.46, + "precision": 0.040647, + "recall": 0.544621 + }, + { + "t": 0.44, + "precision": 0.040635, + "recall": 0.564131 + }, + { + "t": 0.42, + "precision": 0.040653, + "recall": 0.58411 + }, + { + "t": 0.4, + "precision": 0.04064, + "recall": 0.607929 + }, + { + "t": 0.38, + "precision": 0.04067, + "recall": 0.628144 + }, + { + "t": 0.36, + "precision": 0.040657, + "recall": 0.647653 + }, + { + "t": 0.34, + "precision": 0.040546, + "recall": 0.665518 + }, + { + "t": 0.32, + "precision": 0.040501, + "recall": 0.684479 + }, + { + "t": 0.3, + "precision": 0.04034, + "recall": 0.701246 + }, + { + "t": 0.28, + "precision": 0.040354, + "recall": 0.72099 + }, + { + "t": 0.26, + "precision": 0.040228, + "recall": 0.738541 + }, + { + "t": 0.24, + "precision": 0.040148, + "recall": 0.756954 + }, + { + "t": 0.22, + "precision": 0.040179, + "recall": 0.776855 + }, + { + "t": 0.2, + "precision": 0.040205, + "recall": 0.800674 + }, + { + "t": 0.18, + "precision": 0.040183, + "recall": 0.819713 + }, + { + "t": 0.16, + "precision": 0.040206, + "recall": 0.839458 + }, + { + "t": 0.14, + "precision": 0.040162, + "recall": 0.858105 + }, + { + "t": 0.12, + "precision": 0.040161, + "recall": 0.877772 + }, + { + "t": 0.1, + "precision": 0.040181, + "recall": 0.897908 + }, + { + "t": 0.08, + "precision": 0.0401, + "recall": 0.915851 + }, + { + "t": 0.06, + "precision": 0.040083, + "recall": 0.934655 + }, + { + "t": 0.04, + "precision": 0.04015, + "recall": 0.956123 + }, + { + "t": 0.02, + "precision": 0.0402, + "recall": 0.97673 + }, + { + "t": 0.0, + "precision": 0.040206, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w058-ct-baseline.json b/docs/examples/bench/w058-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..39d0c6ccb117960d8787d892d7e89abf4a0ff4bb --- /dev/null +++ b/docs/examples/bench/w058-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:36Z", + "scroll": "PHerc0139", + "segment": "w058", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w058_ink.png", + "sha256": "f86e9e13b0f52351134791703fafbb5a4cd701d4ce8aab39cac07cf10d0d2ada", + "width": 640, + "height": 487, + "pxPerMM": 6.9189, + "alignTargetFile": "data/bench/w058_ink.png", + "alignTargetSha256": "f86e9e13b0f52351134791703fafbb5a4cd701d4ce8aab39cac07cf10d0d2ada" + }, + "prediction": { + "name": "w058_ct.png", + "sha256": "9a2d6eb4f994a44a02ccb7fe4b0134768b5b636da2faf21f6a1234ac24e79848", + "format": "png", + "width": 640, + "height": 487, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.269756, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 311680, + "validFraction": 1.0, + "pearson": 0.269756, + "blurSigmaPx": 6.0, + "blurSigmaUM": 867.2, + "pearsonMatchedBlur": 0.452888, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.11081, + "dice": 0.199512, + "precision": 0.120284, + "recall": 0.584517, + "refPositiveFraction": 0.098264, + "predPositiveFraction": 0.477512, + "ap": 0.121904, + "bestF1": { + "f1": 0.210231, + "threshold": 0.12 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.088235, + "recall": 0.000392 + }, + { + "t": 0.98, + "precision": 0.129902, + "recall": 0.00173 + }, + { + "t": 0.96, + "precision": 0.120755, + "recall": 0.003134 + }, + { + "t": 0.94, + "precision": 0.124093, + "recall": 0.005583 + }, + { + "t": 0.92, + "precision": 0.123779, + "recall": 0.008685 + }, + { + "t": 0.9, + "precision": 0.128707, + "recall": 0.013746 + }, + { + "t": 0.88, + "precision": 0.129079, + "recall": 0.020276 + }, + { + "t": 0.86, + "precision": 0.133488, + "recall": 0.030039 + }, + { + "t": 0.84, + "precision": 0.129512, + "recall": 0.04065 + }, + { + "t": 0.82, + "precision": 0.129202, + "recall": 0.055343 + }, + { + "t": 0.8, + "precision": 0.127283, + "recall": 0.07624 + }, + { + "t": 0.78, + "precision": 0.127969, + "recall": 0.098508 + }, + { + "t": 0.76, + "precision": 0.129032, + "recall": 0.124988 + }, + { + "t": 0.74, + "precision": 0.12869, + "recall": 0.153884 + }, + { + "t": 0.72, + "precision": 0.126731, + "recall": 0.182878 + }, + { + "t": 0.7, + "precision": 0.12696, + "recall": 0.218369 + }, + { + "t": 0.68, + "precision": 0.125931, + "recall": 0.252326 + }, + { + "t": 0.66, + "precision": 0.124082, + "recall": 0.286349 + }, + { + "t": 0.64, + "precision": 0.123375, + "recall": 0.323897 + }, + { + "t": 0.62, + "precision": 0.12301, + "recall": 0.362589 + }, + { + "t": 0.6, + "precision": 0.122117, + "recall": 0.406602 + }, + { + "t": 0.58, + "precision": 0.121498, + "recall": 0.443367 + }, + { + "t": 0.56, + "precision": 0.121011, + "recall": 0.47974 + }, + { + "t": 0.54, + "precision": 0.120501, + "recall": 0.515166 + }, + { + "t": 0.52, + "precision": 0.120719, + "recall": 0.551833 + }, + { + "t": 0.5, + "precision": 0.120284, + "recall": 0.584517 + }, + { + "t": 0.48, + "precision": 0.119953, + "recall": 0.616254 + }, + { + "t": 0.46, + "precision": 0.119964, + "recall": 0.648676 + }, + { + "t": 0.44, + "precision": 0.119805, + "recall": 0.678878 + }, + { + "t": 0.42, + "precision": 0.119751, + "recall": 0.707252 + }, + { + "t": 0.4, + "precision": 0.11957, + "recall": 0.739086 + }, + { + "t": 0.38, + "precision": 0.119565, + "recall": 0.764522 + }, + { + "t": 0.36, + "precision": 0.119182, + "recall": 0.78643 + }, + { + "t": 0.34, + "precision": 0.11911, + "recall": 0.807882 + }, + { + "t": 0.32, + "precision": 0.118883, + "recall": 0.82633 + }, + { + "t": 0.3, + "precision": 0.11878, + "recall": 0.844222 + }, + { + "t": 0.28, + "precision": 0.118661, + "recall": 0.860646 + }, + { + "t": 0.26, + "precision": 0.11842, + "recall": 0.874849 + }, + { + "t": 0.24, + "precision": 0.118456, + "recall": 0.889836 + }, + { + "t": 0.22, + "precision": 0.118417, + "recall": 0.903059 + }, + { + "t": 0.2, + "precision": 0.118343, + "recall": 0.917034 + }, + { + "t": 0.18, + "precision": 0.118268, + "recall": 0.927352 + }, + { + "t": 0.16, + "precision": 0.118274, + "recall": 0.937408 + }, + { + "t": 0.14, + "precision": 0.11815, + "recall": 0.945408 + }, + { + "t": 0.12, + "precision": 0.118133, + "recall": 0.953897 + }, + { + "t": 0.1, + "precision": 0.117962, + "recall": 0.960852 + }, + { + "t": 0.08, + "precision": 0.117838, + "recall": 0.967871 + }, + { + "t": 0.06, + "precision": 0.11762, + "recall": 0.974108 + }, + { + "t": 0.04, + "precision": 0.117398, + "recall": 0.980834 + }, + { + "t": 0.02, + "precision": 0.116929, + "recall": 0.986221 + }, + { + "t": 0.0, + "precision": 0.098264, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w058-ink9um.json b/docs/examples/bench/w058-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..4082b7303538c5550b9eb3739b63ed7b0493293e --- /dev/null +++ b/docs/examples/bench/w058-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:35Z", + "scroll": "PHerc0139", + "segment": "w058", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w058_ink.png", + "sha256": "f86e9e13b0f52351134791703fafbb5a4cd701d4ce8aab39cac07cf10d0d2ada", + "width": 640, + "height": 487, + "pxPerMM": 6.9189, + "alignTargetFile": "data/bench/w058_ink.png", + "alignTargetSha256": "f86e9e13b0f52351134791703fafbb5a4cd701d4ce8aab39cac07cf10d0d2ada" + }, + "prediction": { + "name": "w058_gpu.png", + "sha256": "c1547ad9c0c3909c8d8fc1a25150f3133235b84ee065cb55efe860719bc14164", + "format": "png", + "width": 640, + "height": 487, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.406, + "ty": 0.5, + "sx": 1.00036, + "sy": 1.00206, + "ncc": 0.340874, + "coarseNcc": 0.503774, + "evals": 32143, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 311193, + "validFraction": 0.998438, + "pearson": 0.340874, + "blurSigmaPx": 2.0, + "blurSigmaUM": 289.1, + "pearsonMatchedBlur": 0.376633, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.166883, + "dice": 0.286031, + "precision": 0.294956, + "recall": 0.277631, + "refPositiveFraction": 0.098418, + "predPositiveFraction": 0.092637, + "ap": 0.243806, + "bestF1": { + "f1": 0.299943, + "threshold": 0.3 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.538462, + "recall": 0.000457 + }, + { + "t": 0.98, + "precision": 0.575632, + "recall": 0.036406 + }, + { + "t": 0.96, + "precision": 0.542534, + "recall": 0.050184 + }, + { + "t": 0.94, + "precision": 0.511558, + "recall": 0.060698 + }, + { + "t": 0.92, + "precision": 0.487347, + "recall": 0.070428 + }, + { + "t": 0.9, + "precision": 0.464116, + "recall": 0.079603 + }, + { + "t": 0.88, + "precision": 0.450848, + "recall": 0.089398 + }, + { + "t": 0.86, + "precision": 0.441385, + "recall": 0.099455 + }, + { + "t": 0.84, + "precision": 0.428903, + "recall": 0.108728 + }, + { + "t": 0.82, + "precision": 0.419028, + "recall": 0.118784 + }, + { + "t": 0.8, + "precision": 0.408868, + "recall": 0.127665 + }, + { + "t": 0.78, + "precision": 0.399867, + "recall": 0.137558 + }, + { + "t": 0.76, + "precision": 0.390808, + "recall": 0.146309 + }, + { + "t": 0.74, + "precision": 0.381738, + "recall": 0.155614 + }, + { + "t": 0.72, + "precision": 0.374427, + "recall": 0.16541 + }, + { + "t": 0.7, + "precision": 0.366199, + "recall": 0.174748 + }, + { + "t": 0.68, + "precision": 0.357876, + "recall": 0.184412 + }, + { + "t": 0.66, + "precision": 0.350583, + "recall": 0.194436 + }, + { + "t": 0.64, + "precision": 0.343711, + "recall": 0.204754 + }, + { + "t": 0.62, + "precision": 0.335727, + "recall": 0.2137 + }, + { + "t": 0.6, + "precision": 0.327983, + "recall": 0.223496 + }, + { + "t": 0.58, + "precision": 0.321549, + "recall": 0.234303 + }, + { + "t": 0.56, + "precision": 0.314231, + "recall": 0.244555 + }, + { + "t": 0.54, + "precision": 0.308786, + "recall": 0.256016 + }, + { + "t": 0.52, + "precision": 0.30121, + "recall": 0.26666 + }, + { + "t": 0.5, + "precision": 0.294956, + "recall": 0.277631 + }, + { + "t": 0.48, + "precision": 0.289031, + "recall": 0.289418 + }, + { + "t": 0.46, + "precision": 0.282821, + "recall": 0.30189 + }, + { + "t": 0.44, + "precision": 0.276406, + "recall": 0.314265 + }, + { + "t": 0.42, + "precision": 0.270736, + "recall": 0.327717 + }, + { + "t": 0.4, + "precision": 0.263955, + "recall": 0.340125 + }, + { + "t": 0.38, + "precision": 0.257468, + "recall": 0.353446 + }, + { + "t": 0.36, + "precision": 0.251388, + "recall": 0.368139 + }, + { + "t": 0.34, + "precision": 0.245496, + "recall": 0.383485 + }, + { + "t": 0.32, + "precision": 0.239727, + "recall": 0.399811 + }, + { + "t": 0.3, + "precision": 0.233978, + "recall": 0.417703 + }, + { + "t": 0.28, + "precision": 0.227938, + "recall": 0.435923 + }, + { + "t": 0.26, + "precision": 0.222128, + "recall": 0.454925 + }, + { + "t": 0.24, + "precision": 0.216383, + "recall": 0.475659 + }, + { + "t": 0.22, + "precision": 0.210539, + "recall": 0.49809 + }, + { + "t": 0.2, + "precision": 0.204202, + "recall": 0.521337 + }, + { + "t": 0.18, + "precision": 0.198045, + "recall": 0.547197 + }, + { + "t": 0.16, + "precision": 0.191719, + "recall": 0.575244 + }, + { + "t": 0.14, + "precision": 0.185004, + "recall": 0.605185 + }, + { + "t": 0.12, + "precision": 0.177604, + "recall": 0.635877 + }, + { + "t": 0.1, + "precision": 0.170171, + "recall": 0.672283 + }, + { + "t": 0.08, + "precision": 0.162393, + "recall": 0.714304 + }, + { + "t": 0.06, + "precision": 0.153742, + "recall": 0.763542 + }, + { + "t": 0.04, + "precision": 0.143493, + "recall": 0.821465 + }, + { + "t": 0.02, + "precision": 0.130896, + "recall": 0.89395 + }, + { + "t": 0.0, + "precision": 0.098418, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w058-noise-baseline.json b/docs/examples/bench/w058-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..25cde2b272d54693b7c7340f92c1bf85c9e8f7ef --- /dev/null +++ b/docs/examples/bench/w058-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:37Z", + "scroll": "PHerc0139", + "segment": "w058", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w058_ink.png", + "sha256": "f86e9e13b0f52351134791703fafbb5a4cd701d4ce8aab39cac07cf10d0d2ada", + "width": 640, + "height": 487, + "pxPerMM": 6.9189, + "alignTargetFile": "data/bench/w058_ink.png", + "alignTargetSha256": "f86e9e13b0f52351134791703fafbb5a4cd701d4ce8aab39cac07cf10d0d2ada" + }, + "prediction": { + "name": "noise-seed58.png", + "sha256": "b30e07217188aa46bb47877407a1ea6411435918451eb55b8bd9e1e4c3fe204f", + "format": "png", + "width": 640, + "height": 487, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(58).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.001012, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 311680, + "validFraction": 1.0, + "pearson": 0.001012, + "blurSigmaPx": 6.0, + "blurSigmaUM": 867.2, + "pearsonMatchedBlur": 0.00161, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.089189, + "dice": 0.163771, + "precision": 0.098008, + "recall": 0.497763, + "refPositiveFraction": 0.098264, + "predPositiveFraction": 0.499063, + "ap": 0.097985, + "bestF1": { + "f1": 0.178945, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.093963, + "recall": 0.018905 + }, + { + "t": 0.96, + "precision": 0.096795, + "recall": 0.039442 + }, + { + "t": 0.94, + "precision": 0.097131, + "recall": 0.058804 + }, + { + "t": 0.92, + "precision": 0.096694, + "recall": 0.077938 + }, + { + "t": 0.9, + "precision": 0.096761, + "recall": 0.096941 + }, + { + "t": 0.88, + "precision": 0.096872, + "recall": 0.116172 + }, + { + "t": 0.86, + "precision": 0.097362, + "recall": 0.136416 + }, + { + "t": 0.84, + "precision": 0.098177, + "recall": 0.157214 + }, + { + "t": 0.82, + "precision": 0.098006, + "recall": 0.17638 + }, + { + "t": 0.8, + "precision": 0.09795, + "recall": 0.199889 + }, + { + "t": 0.78, + "precision": 0.09787, + "recall": 0.219186 + }, + { + "t": 0.76, + "precision": 0.098142, + "recall": 0.239331 + }, + { + "t": 0.74, + "precision": 0.09862, + "recall": 0.259869 + }, + { + "t": 0.72, + "precision": 0.098652, + "recall": 0.279361 + }, + { + "t": 0.7, + "precision": 0.098563, + "recall": 0.298462 + }, + { + "t": 0.68, + "precision": 0.098832, + "recall": 0.319065 + }, + { + "t": 0.66, + "precision": 0.098617, + "recall": 0.337741 + }, + { + "t": 0.64, + "precision": 0.098328, + "recall": 0.356907 + }, + { + "t": 0.62, + "precision": 0.098262, + "recall": 0.376726 + }, + { + "t": 0.6, + "precision": 0.098219, + "recall": 0.400268 + }, + { + "t": 0.58, + "precision": 0.097781, + "recall": 0.417932 + }, + { + "t": 0.56, + "precision": 0.097879, + "recall": 0.437947 + }, + { + "t": 0.54, + "precision": 0.098258, + "recall": 0.459497 + }, + { + "t": 0.52, + "precision": 0.098221, + "recall": 0.47912 + }, + { + "t": 0.5, + "precision": 0.098008, + "recall": 0.497763 + }, + { + "t": 0.48, + "precision": 0.097947, + "recall": 0.517158 + }, + { + "t": 0.46, + "precision": 0.098112, + "recall": 0.537598 + }, + { + "t": 0.44, + "precision": 0.098092, + "recall": 0.557221 + }, + { + "t": 0.42, + "precision": 0.098179, + "recall": 0.576681 + }, + { + "t": 0.4, + "precision": 0.098326, + "recall": 0.601397 + }, + { + "t": 0.38, + "precision": 0.098339, + "recall": 0.620923 + }, + { + "t": 0.36, + "precision": 0.09823, + "recall": 0.639664 + }, + { + "t": 0.34, + "precision": 0.098217, + "recall": 0.659026 + }, + { + "t": 0.32, + "precision": 0.098195, + "recall": 0.678421 + }, + { + "t": 0.3, + "precision": 0.098174, + "recall": 0.697816 + }, + { + "t": 0.28, + "precision": 0.098049, + "recall": 0.716557 + }, + { + "t": 0.26, + "precision": 0.098187, + "recall": 0.737193 + }, + { + "t": 0.24, + "precision": 0.098191, + "recall": 0.756783 + }, + { + "t": 0.22, + "precision": 0.098293, + "recall": 0.777027 + }, + { + "t": 0.2, + "precision": 0.098211, + "recall": 0.799393 + }, + { + "t": 0.18, + "precision": 0.098062, + "recall": 0.818036 + }, + { + "t": 0.16, + "precision": 0.098257, + "recall": 0.839521 + }, + { + "t": 0.14, + "precision": 0.098275, + "recall": 0.858981 + }, + { + "t": 0.12, + "precision": 0.098195, + "recall": 0.878016 + }, + { + "t": 0.1, + "precision": 0.098114, + "recall": 0.896497 + }, + { + "t": 0.08, + "precision": 0.098139, + "recall": 0.916022 + }, + { + "t": 0.06, + "precision": 0.098246, + "recall": 0.936331 + }, + { + "t": 0.04, + "precision": 0.098265, + "recall": 0.956117 + }, + { + "t": 0.02, + "precision": 0.098219, + "recall": 0.975577 + }, + { + "t": 0.0, + "precision": 0.098264, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w059-ct-baseline.json b/docs/examples/bench/w059-ct-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..3c69f6803f0e5a78086d6e34b94f8f6c1abf8b04 --- /dev/null +++ b/docs/examples/bench/w059-ct-baseline.json @@ -0,0 +1,334 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:47Z", + "scroll": "PHerc0139", + "segment": "w059", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w059_ink.png", + "sha256": "629864a76f1a92dedef9d228705bab26d8ff2ce0bb2a48949b5043c3309cf477", + "width": 640, + "height": 461, + "pxPerMM": 6.8223, + "alignTargetFile": "data/bench/w059_ink.png", + "alignTargetSha256": "629864a76f1a92dedef9d228705bab26d8ff2ce0bb2a48949b5043c3309cf477" + }, + "prediction": { + "name": "w059_ct.png", + "sha256": "fead2cd274c88bd50672df8d0537bc3eca86c27387b0ceb21312864c4d1622b7", + "format": "png", + "width": 640, + "height": 461, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: X-ray intensity (same grid, no alignment needed)" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.260932, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 295040, + "validFraction": 1.0, + "pearson": 0.260932, + "blurSigmaPx": 6.0, + "blurSigmaUM": 879.5, + "pearsonMatchedBlur": 0.427672, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.12384, + "dice": 0.220388, + "precision": 0.139417, + "recall": 0.525709, + "refPositiveFraction": 0.112917, + "predPositiveFraction": 0.425783, + "ap": 0.144145, + "bestF1": { + "f1": 0.241918, + "threshold": 0.08 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.170213, + "recall": 0.00048 + }, + { + "t": 0.98, + "precision": 0.166667, + "recall": 0.001381 + }, + { + "t": 0.96, + "precision": 0.187063, + "recall": 0.003212 + }, + { + "t": 0.94, + "precision": 0.181646, + "recall": 0.005763 + }, + { + "t": 0.92, + "precision": 0.183122, + "recall": 0.009965 + }, + { + "t": 0.9, + "precision": 0.185585, + "recall": 0.015999 + }, + { + "t": 0.88, + "precision": 0.182695, + "recall": 0.023893 + }, + { + "t": 0.86, + "precision": 0.18209, + "recall": 0.034729 + }, + { + "t": 0.84, + "precision": 0.170981, + "recall": 0.046195 + }, + { + "t": 0.82, + "precision": 0.165596, + "recall": 0.061114 + }, + { + "t": 0.8, + "precision": 0.16265, + "recall": 0.083866 + }, + { + "t": 0.78, + "precision": 0.157775, + "recall": 0.103677 + }, + { + "t": 0.76, + "precision": 0.155735, + "recall": 0.12772 + }, + { + "t": 0.74, + "precision": 0.154089, + "recall": 0.154015 + }, + { + "t": 0.72, + "precision": 0.151354, + "recall": 0.18133 + }, + { + "t": 0.7, + "precision": 0.149481, + "recall": 0.210866 + }, + { + "t": 0.68, + "precision": 0.147351, + "recall": 0.241123 + }, + { + "t": 0.66, + "precision": 0.145365, + "recall": 0.270629 + }, + { + "t": 0.64, + "precision": 0.14417, + "recall": 0.302206 + }, + { + "t": 0.62, + "precision": 0.143227, + "recall": 0.333934 + }, + { + "t": 0.6, + "precision": 0.142078, + "recall": 0.372385 + }, + { + "t": 0.58, + "precision": 0.141252, + "recall": 0.403752 + }, + { + "t": 0.56, + "precision": 0.140457, + "recall": 0.434399 + }, + { + "t": 0.54, + "precision": 0.140297, + "recall": 0.466006 + }, + { + "t": 0.52, + "precision": 0.139776, + "recall": 0.496233 + }, + { + "t": 0.5, + "precision": 0.139417, + "recall": 0.525709 + }, + { + "t": 0.48, + "precision": 0.139178, + "recall": 0.554555 + }, + { + "t": 0.46, + "precision": 0.138698, + "recall": 0.58211 + }, + { + "t": 0.44, + "precision": 0.138713, + "recall": 0.610446 + }, + { + "t": 0.42, + "precision": 0.138663, + "recall": 0.63704 + }, + { + "t": 0.4, + "precision": 0.138521, + "recall": 0.668498 + }, + { + "t": 0.38, + "precision": 0.138515, + "recall": 0.693471 + }, + { + "t": 0.36, + "precision": 0.138678, + "recall": 0.718565 + }, + { + "t": 0.34, + "precision": 0.138645, + "recall": 0.741318 + }, + { + "t": 0.32, + "precision": 0.138664, + "recall": 0.76314 + }, + { + "t": 0.3, + "precision": 0.138525, + "recall": 0.783341 + }, + { + "t": 0.28, + "precision": 0.138689, + "recall": 0.804653 + }, + { + "t": 0.26, + "precision": 0.138812, + "recall": 0.824554 + }, + { + "t": 0.24, + "precision": 0.138922, + "recall": 0.843254 + }, + { + "t": 0.22, + "precision": 0.13893, + "recall": 0.860543 + }, + { + "t": 0.2, + "precision": 0.13891, + "recall": 0.880024 + }, + { + "t": 0.18, + "precision": 0.138905, + "recall": 0.895272 + }, + { + "t": 0.16, + "precision": 0.138862, + "recall": 0.90908 + }, + { + "t": 0.14, + "precision": 0.138807, + "recall": 0.922317 + }, + { + "t": 0.12, + "precision": 0.138782, + "recall": 0.935284 + }, + { + "t": 0.1, + "precision": 0.138582, + "recall": 0.946931 + }, + { + "t": 0.08, + "precision": 0.13843, + "recall": 0.958397 + }, + { + "t": 0.06, + "precision": 0.138138, + "recall": 0.968393 + }, + { + "t": 0.04, + "precision": 0.137746, + "recall": 0.976647 + }, + { + "t": 0.02, + "precision": 0.137369, + "recall": 0.985052 + }, + { + "t": 0.0, + "precision": 0.112917, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w059-ink9um.json b/docs/examples/bench/w059-ink9um.json new file mode 100644 index 0000000000000000000000000000000000000000..45e526132010e24456e73a077649f2b25260b314 --- /dev/null +++ b/docs/examples/bench/w059-ink9um.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:46Z", + "scroll": "PHerc0139", + "segment": "w059", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w059_ink.png", + "sha256": "629864a76f1a92dedef9d228705bab26d8ff2ce0bb2a48949b5043c3309cf477", + "width": 640, + "height": 461, + "pxPerMM": 6.8223, + "alignTargetFile": "data/bench/w059_ink.png", + "alignTargetSha256": "629864a76f1a92dedef9d228705bab26d8ff2ce0bb2a48949b5043c3309cf477" + }, + "prediction": { + "name": "w059_gpu.png", + "sha256": "f70d59d0ffde0448a8b1905418f972e0850b61416f105d7003d987bd64c6165d", + "format": "png", + "width": 640, + "height": 461, + "normalization": "max255", + "invert": false, + "model": { + "name": "scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan", + "url": "https://huggingface.co/scrollprize/ink_9um" + } + }, + "alignment": { + "method": "auto", + "target": "ink", + "tx": 0.469, + "ty": 0.469, + "sx": 1.00014, + "sy": 1.00204, + "ncc": 0.402068, + "coarseNcc": 0.565716, + "evals": 30135, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 294579, + "validFraction": 0.998438, + "pearson": 0.402068, + "blurSigmaPx": 1.5, + "blurSigmaUM": 219.9, + "pearsonMatchedBlur": 0.43231, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.205065, + "dice": 0.340338, + "precision": 0.38031, + "recall": 0.307969, + "refPositiveFraction": 0.113094, + "predPositiveFraction": 0.091582, + "ap": 0.319417, + "bestF1": { + "f1": 0.355203, + "threshold": 0.32 + }, + "pr": [ + { + "t": 1.0, + "precision": 0.64, + "recall": 0.00048 + }, + { + "t": 0.98, + "precision": 0.725502, + "recall": 0.03689 + }, + { + "t": 0.96, + "precision": 0.681836, + "recall": 0.052169 + }, + { + "t": 0.94, + "precision": 0.656061, + "recall": 0.064986 + }, + { + "t": 0.92, + "precision": 0.625741, + "recall": 0.076032 + }, + { + "t": 0.9, + "precision": 0.606772, + "recall": 0.087678 + }, + { + "t": 0.88, + "precision": 0.590712, + "recall": 0.098124 + }, + { + "t": 0.86, + "precision": 0.574854, + "recall": 0.10938 + }, + { + "t": 0.84, + "precision": 0.559199, + "recall": 0.119796 + }, + { + "t": 0.82, + "precision": 0.546478, + "recall": 0.131112 + }, + { + "t": 0.8, + "precision": 0.53279, + "recall": 0.141198 + }, + { + "t": 0.78, + "precision": 0.520322, + "recall": 0.151403 + }, + { + "t": 0.76, + "precision": 0.508966, + "recall": 0.161879 + }, + { + "t": 0.74, + "precision": 0.498221, + "recall": 0.172325 + }, + { + "t": 0.72, + "precision": 0.486702, + "recall": 0.182921 + }, + { + "t": 0.7, + "precision": 0.476973, + "recall": 0.194297 + }, + { + "t": 0.68, + "precision": 0.465654, + "recall": 0.205103 + }, + { + "t": 0.66, + "precision": 0.455562, + "recall": 0.216479 + }, + { + "t": 0.64, + "precision": 0.445506, + "recall": 0.228216 + }, + { + "t": 0.62, + "precision": 0.433912, + "recall": 0.237971 + }, + { + "t": 0.6, + "precision": 0.424709, + "recall": 0.248477 + }, + { + "t": 0.58, + "precision": 0.414826, + "recall": 0.259853 + }, + { + "t": 0.56, + "precision": 0.4064, + "recall": 0.271409 + }, + { + "t": 0.54, + "precision": 0.396628, + "recall": 0.283146 + }, + { + "t": 0.52, + "precision": 0.389484, + "recall": 0.295723 + }, + { + "t": 0.5, + "precision": 0.38031, + "recall": 0.307969 + }, + { + "t": 0.48, + "precision": 0.371828, + "recall": 0.321057 + }, + { + "t": 0.46, + "precision": 0.362936, + "recall": 0.333664 + }, + { + "t": 0.44, + "precision": 0.353198, + "recall": 0.347621 + }, + { + "t": 0.42, + "precision": 0.34183, + "recall": 0.359478 + }, + { + "t": 0.4, + "precision": 0.332383, + "recall": 0.374486 + }, + { + "t": 0.38, + "precision": 0.323314, + "recall": 0.388744 + }, + { + "t": 0.36, + "precision": 0.315177, + "recall": 0.405043 + }, + { + "t": 0.34, + "precision": 0.306581, + "recall": 0.419901 + }, + { + "t": 0.32, + "precision": 0.299321, + "recall": 0.43674 + }, + { + "t": 0.3, + "precision": 0.291198, + "recall": 0.453129 + }, + { + "t": 0.28, + "precision": 0.283369, + "recall": 0.471439 + }, + { + "t": 0.26, + "precision": 0.275342, + "recall": 0.489689 + }, + { + "t": 0.24, + "precision": 0.26839, + "recall": 0.510671 + }, + { + "t": 0.22, + "precision": 0.261358, + "recall": 0.534444 + }, + { + "t": 0.2, + "precision": 0.253962, + "recall": 0.557947 + }, + { + "t": 0.18, + "precision": 0.24633, + "recall": 0.583191 + }, + { + "t": 0.16, + "precision": 0.238463, + "recall": 0.610326 + }, + { + "t": 0.14, + "precision": 0.229729, + "recall": 0.639172 + }, + { + "t": 0.12, + "precision": 0.22114, + "recall": 0.67207 + }, + { + "t": 0.1, + "precision": 0.211858, + "recall": 0.70899 + }, + { + "t": 0.08, + "precision": 0.200998, + "recall": 0.747231 + }, + { + "t": 0.06, + "precision": 0.189209, + "recall": 0.791115 + }, + { + "t": 0.04, + "precision": 0.17524, + "recall": 0.843764 + }, + { + "t": 0.02, + "precision": 0.157589, + "recall": 0.907129 + }, + { + "t": 0.0, + "precision": 0.113094, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/docs/examples/bench/w059-noise-baseline.json b/docs/examples/bench/w059-noise-baseline.json new file mode 100644 index 0000000000000000000000000000000000000000..1cb46ec6c2fcf31eb68499dde57c9b69effc2456 --- /dev/null +++ b/docs/examples/bench/w059-noise-baseline.json @@ -0,0 +1,335 @@ +{ + "schema": "unroll-inkbench/1", + "tool": { + "name": "inkbench", + "version": "1.0.0", + "impl": "python" + }, + "created": "2026-10-16T03:55:48Z", + "scroll": "PHerc0139", + "segment": "w059", + "reference": { + "layer": "ink", + "label": "Published ink (2.4 um), Vesuvius Challenge", + "file": "data/bench/w059_ink.png", + "sha256": "629864a76f1a92dedef9d228705bab26d8ff2ce0bb2a48949b5043c3309cf477", + "width": 640, + "height": 461, + "pxPerMM": 6.8223, + "alignTargetFile": "data/bench/w059_ink.png", + "alignTargetSha256": "629864a76f1a92dedef9d228705bab26d8ff2ce0bb2a48949b5043c3309cf477" + }, + "prediction": { + "name": "noise-seed59.png", + "sha256": "58e6a3decbfdea1688f541b415c3186a51e628b838ffa81a3a35da1f31f95563", + "format": "png", + "width": 640, + "height": 461, + "normalization": "max255", + "invert": false, + "model": { + "name": "baseline: uniform noise", + "notes": "numpy default_rng(59).random, 8 bit" + } + }, + "alignment": { + "method": "manual", + "target": "ink", + "tx": 0.0, + "ty": 0.0, + "sx": 1.0, + "sy": 1.0, + "ncc": 0.001838, + "coarseNcc": null, + "evals": 0, + "model": "bilinear, scale about grid centre, then shift; units are reference px" + }, + "metrics": { + "nValid": 295040, + "validFraction": 1.0, + "pearson": 0.001838, + "blurSigmaPx": 0.0, + "blurSigmaUM": 0.0, + "pearsonMatchedBlur": 0.001838, + "thresholds": { + "ref": 0.5, + "pred": 0.5 + }, + "iou": 0.102313, + "dice": 0.185633, + "precision": 0.113866, + "recall": 0.502086, + "refPositiveFraction": 0.112917, + "predPositiveFraction": 0.497902, + "ap": 0.113942, + "bestF1": { + "f1": 0.202921, + "threshold": 0.0 + }, + "pr": [ + { + "t": 1.0, + "precision": 1.0, + "recall": 0.0 + }, + { + "t": 0.98, + "precision": 0.118234, + "recall": 0.020261 + }, + { + "t": 0.96, + "precision": 0.11622, + "recall": 0.040132 + }, + { + "t": 0.94, + "precision": 0.117664, + "recall": 0.061144 + }, + { + "t": 0.92, + "precision": 0.117219, + "recall": 0.081285 + }, + { + "t": 0.9, + "precision": 0.115849, + "recall": 0.100465 + }, + { + "t": 0.88, + "precision": 0.114907, + "recall": 0.120156 + }, + { + "t": 0.86, + "precision": 0.114407, + "recall": 0.139727 + }, + { + "t": 0.84, + "precision": 0.114546, + "recall": 0.159718 + }, + { + "t": 0.82, + "precision": 0.114319, + "recall": 0.179139 + }, + { + "t": 0.8, + "precision": 0.114086, + "recall": 0.202191 + }, + { + "t": 0.78, + "precision": 0.114565, + "recall": 0.222933 + }, + { + "t": 0.76, + "precision": 0.114712, + "recall": 0.243404 + }, + { + "t": 0.74, + "precision": 0.114174, + "recall": 0.262194 + }, + { + "t": 0.72, + "precision": 0.114081, + "recall": 0.282005 + }, + { + "t": 0.7, + "precision": 0.113925, + "recall": 0.301486 + }, + { + "t": 0.68, + "precision": 0.113947, + "recall": 0.321297 + }, + { + "t": 0.66, + "precision": 0.113572, + "recall": 0.340057 + }, + { + "t": 0.64, + "precision": 0.113566, + "recall": 0.359898 + }, + { + "t": 0.62, + "precision": 0.113689, + "recall": 0.380369 + }, + { + "t": 0.6, + "precision": 0.113543, + "recall": 0.403092 + }, + { + "t": 0.58, + "precision": 0.113452, + "recall": 0.422122 + }, + { + "t": 0.56, + "precision": 0.113315, + "recall": 0.440973 + }, + { + "t": 0.54, + "precision": 0.113616, + "recall": 0.461564 + }, + { + "t": 0.52, + "precision": 0.113597, + "recall": 0.480925 + }, + { + "t": 0.5, + "precision": 0.113866, + "recall": 0.502086 + }, + { + "t": 0.48, + "precision": 0.113891, + "recall": 0.521837 + }, + { + "t": 0.46, + "precision": 0.113631, + "recall": 0.540267 + }, + { + "t": 0.44, + "precision": 0.113512, + "recall": 0.559328 + }, + { + "t": 0.42, + "precision": 0.11337, + "recall": 0.578208 + }, + { + "t": 0.4, + "precision": 0.113159, + "recall": 0.601141 + }, + { + "t": 0.38, + "precision": 0.113145, + "recall": 0.620591 + }, + { + "t": 0.36, + "precision": 0.113294, + "recall": 0.641183 + }, + { + "t": 0.34, + "precision": 0.113284, + "recall": 0.660663 + }, + { + "t": 0.32, + "precision": 0.113175, + "recall": 0.679814 + }, + { + "t": 0.3, + "precision": 0.113168, + "recall": 0.699565 + }, + { + "t": 0.28, + "precision": 0.113183, + "recall": 0.719676 + }, + { + "t": 0.26, + "precision": 0.113326, + "recall": 0.739967 + }, + { + "t": 0.24, + "precision": 0.113393, + "recall": 0.760048 + }, + { + "t": 0.22, + "precision": 0.113295, + "recall": 0.778658 + }, + { + "t": 0.2, + "precision": 0.113321, + "recall": 0.802581 + }, + { + "t": 0.18, + "precision": 0.113281, + "recall": 0.822032 + }, + { + "t": 0.16, + "precision": 0.113213, + "recall": 0.840973 + }, + { + "t": 0.14, + "precision": 0.113346, + "recall": 0.861624 + }, + { + "t": 0.12, + "precision": 0.113213, + "recall": 0.880174 + }, + { + "t": 0.1, + "precision": 0.113242, + "recall": 0.900105 + }, + { + "t": 0.08, + "precision": 0.113178, + "recall": 0.919136 + }, + { + "t": 0.06, + "precision": 0.113158, + "recall": 0.938706 + }, + { + "t": 0.04, + "precision": 0.113091, + "recall": 0.957887 + }, + { + "t": 0.02, + "precision": 0.113016, + "recall": 0.977338 + }, + { + "t": 0.0, + "precision": 0.112917, + "recall": 1.0 + } + ] + }, + "annotations": [], + "credits": { + "data": "PHerc. 139 scan, segments and published ink: Vesuvius Challenge open data (scrollprize.org), ESRF; held by the Biblioteca Nazionale di Napoli.", + "dataLicense": "CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/)", + "code": "Unroll Ink Bench by Chase Hendrick", + "codeLicense": "Apache-2.0" + }, + "disclaimer": "Scores measure agreement with another model's output (the Vesuvius Challenge published ink detection), not agreement with ground truth. Letter boxes are candidates for expert review, not readings." +} \ No newline at end of file diff --git a/index.html b/index.html index 3193c66e140845bfc5bfebdcdd8cb79b0913cfe1..12031dabe76eafd0ff16c0c52ea21b5f47672db9 100644 --- a/index.html +++ b/index.html @@ -90,7 +90,7 @@ if (!/(^|[#&])s=/.test(location.hash)) ['scrolls/index.json', 'scrolls/PHerc0139
- +

Published ink (2.4 µm): Vesuvius Challenge detection, 2.4 µm scan, all windings.

Why two ink layers?

Published ink is the Vesuvius Challenge result on the fine 2.4 µm scan. Our ink model (9 µm) is the open ink_9um model run by us on a scan about 4x coarser, so it is blurrier. It shows what an open model can recover from cheaper scans, as a baseline you can check.

diff --git a/lib/bench.js b/lib/bench.js index 8b50194e204678d1494e2c66eebe88776f57bbb1..342139124ffabf082502f362a96fd3c3dede10c1 100644 --- a/lib/bench.js +++ b/lib/bench.js @@ -341,7 +341,7 @@ async function loadBoard(sources) { S.board.push(...out); renderBoard(); return out.map((o) => ({ src: o.src, ok: !o.errors.length, errors: o.errors })); } -const EXAMPLES = ['w046', 'w047', 'w048', 'w049', 'w050', 'w051'].flatMap((w) => ['ink9um', 'ct-baseline', 'noise-baseline'].map((k) => `${BASE}docs/examples/bench/${w}-${k}.json`)); +const examples = () => Object.keys(S.index.segments).filter((w) => S.index.segments[w].layers.gpu).flatMap((w) => ['ink9um', 'ct-baseline', 'noise-baseline'].map((k) => `${BASE}docs/examples/bench/${w}-${k}.json`)); // ------------------------------------------------------------------ UI @@ -509,7 +509,7 @@ function wire() { $('bBoardLoad').onclick = () => guard(() => loadBoard($('bBoardURLs').value.split(/\s+/).filter(Boolean))); $('bBoardPick').onchange = (e) => guard(async () => { const bs = []; for (const f of e.target.files) { try { bs.push(JSON.parse(await f.text())); } catch { status(`${f.name} is not JSON.`, true); } } await loadBoard(bs); e.target.value = ''; }); $('bBoardCur').onclick = () => guard(() => loadBoard([bundle()])); - $('bBoardEx').onclick = () => guard(() => loadBoard(EXAMPLES)); + $('bBoardEx').onclick = () => guard(() => loadBoard(examples())); $('bBoardClear').onclick = () => { S.board = []; renderBoard(); }; $('bBoardMetric').onchange = renderBoard; stagePointer(); @@ -666,8 +666,8 @@ function renderBoard() { h += ''; } if (bad.length) h += `

Not loaded

    ${bad.map((o) => `
  • ${esc(o.src)}: ${esc(o.errors.join('; '))}
    • `).join('')}
`; - out.innerHTML = h || '

No bundles loaded. Try Load examples: the public ink_9um runs and two baselines (X-ray intensity, noise) on w046 to w051.

'; - if (good.length) out.insertAdjacentHTML('afterbegin', `

${good.length} bundle${good.length === 1 ? '' : 's'}, ranked by agreement with the published ink (another model's output, not ground truth). Thresholds differ between bundles if their authors chose so; compare r and AP for threshold-free numbers.

`); + out.innerHTML = h || '

No bundles loaded. Try Load examples: the public ink_9um run on every winding and two baselines (X-ray intensity, noise).

'; + if (good.length) out.insertAdjacentHTML('afterbegin', `

${good.length} bundle${good.length === 1 ? '' : 's'}, ranked by agreement with the published ink (another model's output, not ground truth). Thresholds differ between bundles if their authors chose so; compare r and AP for threshold-free numbers. Takeaway from the examples: rank by AP, where ink_9um (trained by the Vesuvius Challenge, run by Chase Hendrick) beats the X-ray floor on every winding; on Pearson r the X-ray floor wins on w039, w053, w054 and w057.

`); } // ------------------------------------------------------------------ open, close, API diff --git a/tests/__pycache__/synth.cpython-313.pyc b/tests/__pycache__/synth.cpython-313.pyc deleted file mode 100644 index ee46cda1e30928f1b46184ac7c14054024a80cab..0000000000000000000000000000000000000000 Binary files a/tests/__pycache__/synth.cpython-313.pyc and /dev/null differ diff --git a/tests/e2e_bench.py b/tests/e2e_bench.py index 895c289cb08df3b2c8e72307381c49bb4def7221..56319d1fd44789ba162ef1c069a64b026387eb4e 100644 --- a/tests/e2e_bench.py +++ b/tests/e2e_bench.py @@ -143,9 +143,9 @@ with sync_playwright() as pw: check(lb['alignment']['method'] == 'shared' and lb['metrics'] == zb['metrics'] and len(lb['annotations']) == 1, 'local share link restores after dropping the same file') p3.close() # 5. leaderboard and cite - p.click('[data-tab="board"]'); p.click('#bBoardEx'); p.wait_for_selector('.b-table', timeout=60000); p.wait_for_timeout(800) + p.click('[data-tab="board"]'); p.click('#bBoardEx'); p.wait_for_function("document.querySelectorAll('.b-table').length >= 37", timeout=180000); p.wait_for_timeout(800) rows = p.evaluate("[...document.querySelectorAll('.b-table')].map(t => [...t.querySelectorAll('tbody tr')].map(r => r.children[1].textContent))") - check(len(rows) == 6 and all(r[0].startswith('scrollprize/ink_9um') and r[-1] == 'baseline: uniform noise' for r in rows), 'leaderboard ranks the 18 example bundles per segment') + check(len(rows) == 37 and all(r[-1] == 'baseline: uniform noise' and any(x.startswith('scrollprize/ink_9um') for x in r[:2]) for r in rows), 'leaderboard ranks the 111 example bundles over 37 windings') rk = p.evaluate('async () => window.unrollBench.rank([window.unrollBench.bundle()], "ap")') check('w049' in rk, 'API rank works') p.screenshot(path=os.path.join(shots, 'bench_leaderboard.png')) @@ -155,7 +155,7 @@ with sync_playwright() as pw: p.screenshot(path=os.path.join(shots, 'bench_cite.png')) p.click('#bClose'); p.wait_for_timeout(200); p.click('#btnInfo', timeout=120000); p.wait_for_timeout(400) check('Why this matters for researchers' in p.inner_text('#about'), 'About panel has the researchers section') - p.locator('#about .bench-why').scroll_into_view_if_needed(); p.screenshot(path=os.path.join(shots, 'bench_about.png')) + p.evaluate("document.querySelector('#about .bench-why').scrollIntoView()"); p.wait_for_timeout(500); p.screenshot(path=os.path.join(shots, 'bench_about.png')) R['api'] = p.evaluate("({ shim: typeof window.unrollBench, viaUnroll: !!(window.unroll && window.unroll.bench), hasUnroll: typeof window.unroll })") check(R['api']['shim'] == 'object' and (R['api']['hasUnroll'] != 'object' or R['api']['viaUnroll']), f'agent API {R["api"]}') check(not errs, f'no page errors {errs[:3]}') diff --git a/tools/__pycache__/inkbench.cpython-313.pyc b/tools/__pycache__/inkbench.cpython-313.pyc deleted file mode 100644 index 4623038ff016038e738b5c856303a537fe1ef876..0000000000000000000000000000000000000000 Binary files a/tools/__pycache__/inkbench.cpython-313.pyc and /dev/null differ diff --git a/tools/make_bench_examples.py b/tools/make_bench_examples.py index 5b71d0c7097a5417db8bf63683940d2c9b68fb7c..a07af568a4a59aab165d9c2ce4f2c2d08c7206c8 100644 --- a/tools/make_bench_examples.py +++ b/tools/make_bench_examples.py @@ -1,5 +1,5 @@ # SPDX-License-Identifier: Apache-2.0 -"""Example Ink Bench bundles for the leaderboard: the public ink_9um runs (w046 to w051) and two +"""Example Ink Bench bundles for the leaderboard: the public ink_9um runs on every winding and two baselines any ink model should beat, X-ray intensity and uniform noise (seeded), on the same grid.""" import io, json, os, sys import numpy as np @@ -9,7 +9,7 @@ import inkbench as ib root = os.path.join(os.path.dirname(__file__), '..'); out = os.path.join(root, 'docs/examples/bench') refs = ib.Refs(root=root) ident = {'tx': 0.0, 'ty': 0.0, 'sx': 1.0, 'sy': 1.0, 'method': 'manual'} -for w in ['w046', 'w047', 'w048', 'w049', 'w050', 'w051']: +for w in [k for k, v in refs.index['segments'].items() if 'gpu' in v['layers']]: s = refs.segment(w) b = ib.compare(os.path.join(root, s['layers']['gpu']['file']), w, refs=refs, model={'name': 'scrollprize/ink_9um (seed 42, step 75000) on the 9.362 um scan', 'url': 'https://huggingface.co/scrollprize/ink_9um'})