README.md

dinovol embeddings of PHerc0841

Guide and live demo: huggingface.co/spaces/SharpMeow/dinovol-embeddings, where you can click a point of one slice and see every cube that looks like it.

Precomputed self-supervised embeddings of one whole public CT scan, so you can search, cluster and segment by feature similarity without running a GPU model yourself. Read any region by URL with zarr; nothing to download first.

In plain words. dinovol is a 3D model from the Vesuvius Challenge team, trained the DINO way (self-supervised, no labels). It turns each small cube of a CT scan into a list of numbers, so that cubes that look alike get lists that point the same way. Here it has been run over every voxel of PHerc0841: each 8 x 8 x 8 voxel cube (about 75 um across) has 480 numbers, and each 4 x 4 x 4 cube has 64 more. The dot product of two lists says how alike two cubes look, which is enough for click-a-point search, clustering, or a small classifier trained from a few clicks. The picture above shows one slice with the three directions in which the numbers vary most as red, green and blue.

• Scan: PHerc0841, 20250821151531-9.366um-1.2m-113keV-masked.zarr, level 0 (19400 x 7703 x 7703 voxels), from the Vesuvius Challenge open data (https://scrollprize.org). PHerc0841 is not eligible for the 2027 Grand Prize or First Letters prizes.
• Model: scrollprize/dinovol_8um_ps192_102126, checkpoint teacher_step_640000.pt (sha256 2354c493f050070e17ea0cda5cd3c1a6ebdde19a70a9c28bc9d368b80faabb86), run with villa's dinovol/dinovol_2/eval/embed_volume.py at commit 4ea26d9cbfc1778c68e0875c21c1f670922deba1 (branch dinovol-refactor), robust normalization with fixed volume-wide statistics.
• License: CC BY-NC 4.0, the license of the scan data it is computed from. Credit the Vesuvius Challenge for the scan and the dinovol authors for the model. The model's Hugging Face page states no license; villa's dinovol code is MIT.
• Made by Chase Hendrick, Independent Researcher · ORCID 0009-0002-9754-6087, on Hugging Face Jobs (L4 GPUs).

What is in it

One zarr v2 group, PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr, with three outputs. Each is stored as int8 with one float16 scale per vector: value = q * scale, then each vector has unit length.

Array	Shape (z, y, x, channels)	One vector per	Use
tokens_unit (+ tokens_unit_scale)	2448 x 963 x 963 x 480	8 x 8 x 8 voxels	concept search, clustering
dense (+ dense_scale)	4896 x 1926 x 1926 x 64	4 x 4 x 4 voxels	finer segmentation, same-sheet tests
ink (+ ink_scale)	4896 x 1926 x 1926 x 32	4 x 4 x 4 voxels	the model's ink head features

Index i along an axis covers voxels [i * step, (i + 1) * step) of the level-0 scan. Cells outside the scroll are all zero and their chunks are not stored. The scan was embedded in slabs of 192 slices, each on its own (its tiles read 32 voxels past its z edges), so values can step slightly at multiples of z 192.

Read it

import numpy as np, zarr                                     # zarr 3 (tested with 3.4.1) and fsspec with aiohttp
root = "https://huggingface.co/buckets/SharpMeow/dinovol-embeddings/resolve/PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr"
g = zarr.open_group(zarr.storage.FsspecStore.from_url(root, read_only=True), mode="r", zarr_format=2)
z, y, x = 9700 // 8, 3800 // 8, 3800 // 8                    # a level-0 voxel, as token indices
q = g["tokens_unit"][z - 2:z + 2, y - 16:y + 16, x - 16:x + 16].astype(np.float32)
s = g["tokens_unit_scale"][z - 2:z + 2, y - 16:y + 16, x - 16:x + 16].astype(np.float32)
t = q * s[..., None]                                         # unit vectors (all zero outside the scroll)
query = t[2, 16, 16]
similarity = t @ query                                       # cosine similarity to the clicked voxel

Download it

Everything streams, so most people never need a copy. To get files on disk, the free hf command (pip install -U huggingface_hub) copies any folder of the store, logged out. A copy opens with the same zarr code, with its folder's path in place of the URL: zarr.open_group("pherc0841-dinovol.zarr", mode="r", zarr_format=2).

What	Size
Everything	539 GB, about 940,000 files
tokens_unit with tokens_unit_scale	179 GB
dense with dense_scale	238 GB
ink with ink_scale	122 GB

Chunk folders follow the scan for every array: ARRAY/Z/Y/X with Z = z // 192 and Y, X = y // 96, x // 96 in scan voxels. A region (this strip, z 9,600 to 9,791 and y 3,840 to 3,935, is 40 MB and took 7 s logged out):

B=hf://buckets/SharpMeow/dinovol-embeddings/PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr
U=https://huggingface.co/buckets/SharpMeow/dinovol-embeddings/resolve/PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr
D=pherc0841-dinovol.zarr
mkdir -p $D/tokens_unit $D/tokens_unit_scale
for f in .zgroup .zattrs tokens_unit/.zarray tokens_unit_scale/.zarray; do curl -sL $U/$f -o $D/$f; done
hf buckets sync $B/tokens_unit/50/40 $D/tokens_unit/50/40
hf buckets sync $B/tokens_unit_scale/50/40 $D/tokens_unit_scale/50/40

One feature set is hf buckets sync $B/tokens_unit $D/tokens_unit plus the same for tokens_unit_scale (swap in dense or ink). For everything, log in first with a free account (hf auth login), because listing the whole store logged out gets rate limited, then hf buckets sync $B $D.

The whole scan in color

PHerc0841/20250821151531-dinovol_8um_ps192_102126-rgb.ome.zarr shows every cube's tokens_unit as one color: the three directions in which the numbers vary most across the whole scan, each scaled from its 1st to 99th percentile. It is a multiscale OME-Zarr (zarr v2, uint8, axes c, z, y, x; level 0 is one color per 8-voxel cube, 3 x 2448 x 963 x 963; levels 1 to 3 halve it), so neuroglancer streams it: the guide page has an Open in neuroglancer button. 0 means outside the scroll. The colors show how the numbers vary; they are not labels.

Checks

Before release, read back by URL: one 24 x 48 x 48 token block from every slab (5.6 million cells, 4.5 million inside the scroll). Every nonzero vector has unit length within 1 percent after scaling, and the nonzero cells match the nonzero CT voxels at pyramid level 3 exactly (the last slab, past the scan's end, was checked for norms only). The first slab checked in full (z 9600, 138,240 cells) matched too. These checks show the files are complete and aligned with the scan; they say nothing about how well the features separate sheets, ink or fibers.