Buckets:

SharpMeow
/

dinovol-embeddings

Files

xet

SharpMeow/dinovol-embeddings / README.md

SharpMeow

10 days ago

preview code

download

raw

6.96 kB

# dinovol embeddings of PHerc0841

**Guide and live demo: [huggingface.co/spaces/SharpMeow/dinovol-embeddings](https://huggingface.co/spaces/SharpMeow/dinovol-embeddings)**,
where you can click a point of one slice and see every cube that looks like it.

![One slice of PHerc0841 with its dinovol features shown as color](https://huggingface.co/spaces/SharpMeow/dinovol-embeddings/resolve/main/assets/dense.webp)

Precomputed self-supervised embeddings of one whole public CT scan, so you can search, cluster and segment by
feature similarity without running a GPU model yourself. Read any region by URL with zarr; nothing to download first.

**In plain words.** dinovol is a 3D model from the Vesuvius Challenge team, trained the DINO way (self-supervised, no
labels). It turns each small cube of a CT scan into a list of numbers, so that cubes that look alike get lists that
point the same way. Here it has been run over every voxel of PHerc0841: each 8 x 8 x 8 voxel cube (about 75 um
across) has 480 numbers, and each 4 x 4 x 4 cube has 64 more. The dot product of two lists says how alike two cubes
look, which is enough for click-a-point search, clustering, or a small classifier trained from a few clicks. The
picture above shows one slice with the three directions in which the numbers vary most as red, green and blue.

- **Scan:** PHerc0841, `20250821151531-9.366um-1.2m-113keV-masked.zarr`, level 0 (19400 x 7703 x 7703 voxels), from the
  Vesuvius Challenge open data (https://scrollprize.org). PHerc0841 is not eligible for the 2027 Grand Prize or
  First Letters prizes.
- **Model:** [`scrollprize/dinovol_8um_ps192_102126`](https://huggingface.co/scrollprize/dinovol_8um_ps192_102126),
  checkpoint `teacher_step_640000.pt` (sha256 `2354c493f050070e17ea0cda5cd3c1a6ebdde19a70a9c28bc9d368b80faabb86`), run
  with villa's `dinovol/dinovol_2/eval/embed_volume.py` at commit `4ea26d9cbfc1778c68e0875c21c1f670922deba1`
  (branch dinovol-refactor), robust normalization with fixed volume-wide statistics.
- **License:** CC BY-NC 4.0, the license of the scan data it is computed from. Credit the Vesuvius Challenge for the
  scan and the dinovol authors for the model. The model's Hugging Face page states no license; villa's dinovol code is
  MIT.
- Made by Chase Hendrick, Independent Researcher · ORCID 0009-0002-9754-6087, on Hugging Face Jobs (L4 GPUs).

## What is in it

One zarr v2 group, `PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr`, with three outputs. Each is stored as
int8 with one float16 scale per vector: `value = q * scale`, then each vector has unit length.

| Array | Shape (z, y, x, channels) | One vector per | Use |
| --- | --- | --- | --- |
| `tokens_unit` (+ `tokens_unit_scale`) | 2448 x 963 x 963 x 480 | 8 x 8 x 8 voxels | concept search, clustering |
| `dense` (+ `dense_scale`) | 4896 x 1926 x 1926 x 64 | 4 x 4 x 4 voxels | finer segmentation, same-sheet tests |
| `ink` (+ `ink_scale`) | 4896 x 1926 x 1926 x 32 | 4 x 4 x 4 voxels | the model's ink head features |

Index `i` along an axis covers voxels `[i * step, (i + 1) * step)` of the level-0 scan. Cells outside the scroll are
all zero and their chunks are not stored. The scan was embedded in slabs of 192 slices, each on its own (its tiles read
32 voxels past its z edges), so values can step slightly at multiples of z 192.

## Read it

```python
import numpy as np, zarr                                     # zarr 3 (tested with 3.4.1) and fsspec with aiohttp
root = "https://huggingface.co/buckets/SharpMeow/dinovol-embeddings/resolve/PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr"
g = zarr.open_group(zarr.storage.FsspecStore.from_url(root, read_only=True), mode="r", zarr_format=2)
z, y, x = 9700 // 8, 3800 // 8, 3800 // 8                    # a level-0 voxel, as token indices
q = g["tokens_unit"][z - 2:z + 2, y - 16:y + 16, x - 16:x + 16].astype(np.float32)
s = g["tokens_unit_scale"][z - 2:z + 2, y - 16:y + 16, x - 16:x + 16].astype(np.float32)
t = q * s[..., None]                                         # unit vectors (all zero outside the scroll)
query = t[2, 16, 16]
similarity = t @ query                                       # cosine similarity to the clicked voxel
```

## Download it

Everything streams, so most people never need a copy. To get files on disk, the free `hf` command
(`pip install -U huggingface_hub`) copies any folder of the store, logged out. A copy opens with the same zarr code,
with its folder's path in place of the URL: `zarr.open_group("pherc0841-dinovol.zarr", mode="r", zarr_format=2)`.

| What | Size |
| --- | --- |
| Everything | 539 GB, about 940,000 files |
| `tokens_unit` with `tokens_unit_scale` | 179 GB |
| `dense` with `dense_scale` | 238 GB |
| `ink` with `ink_scale` | 122 GB |

Chunk folders follow the scan for every array: `ARRAY/Z/Y/X` with Z = z // 192 and Y, X = y // 96, x // 96 in
scan voxels. A region (this strip, z 9,600 to 9,791 and y 3,840 to 3,935, is 40 MB and took 7 s logged out):

```bash
B=hf://buckets/SharpMeow/dinovol-embeddings/PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr
U=https://huggingface.co/buckets/SharpMeow/dinovol-embeddings/resolve/PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr
D=pherc0841-dinovol.zarr
mkdir -p $D/tokens_unit $D/tokens_unit_scale
for f in .zgroup .zattrs tokens_unit/.zarray tokens_unit_scale/.zarray; do curl -sL $U/$f -o $D/$f; done
hf buckets sync $B/tokens_unit/50/40 $D/tokens_unit/50/40
hf buckets sync $B/tokens_unit_scale/50/40 $D/tokens_unit_scale/50/40
```

One feature set is `hf buckets sync $B/tokens_unit $D/tokens_unit` plus the same for `tokens_unit_scale` (swap in
`dense` or `ink`). For everything, log in first with a free account (`hf auth login`), because listing the whole
store logged out gets rate limited, then `hf buckets sync $B $D`.

## The whole scan in color

`PHerc0841/20250821151531-dinovol_8um_ps192_102126-rgb.ome.zarr` shows every cube's `tokens_unit` as one color: the
three directions in which the numbers vary most across the whole scan, each scaled from its 1st to 99th percentile.
It is a multiscale OME-Zarr (zarr v2, uint8, axes c, z, y, x; level 0 is one color per 8-voxel cube, 3 x 2448 x 963
x 963; levels 1 to 3 halve it), so neuroglancer streams it: the guide page has an Open in neuroglancer button. 0
means outside the scroll. The colors show how the numbers vary; they are not labels.

## Checks

Before release, read back by URL: one 24 x 48 x 48 token block from every slab (5.6 million cells, 4.5 million inside
the scroll). Every nonzero vector has unit length within 1 percent after scaling, and the nonzero cells match the
nonzero CT voxels at pyramid level 3 exactly (the last slab, past the scan's end, was checked for norms only). The
first slab checked in full (z 9600, 138,240 cells) matched too. These checks show the files are complete and aligned
with the scan; they say nothing about how well the features separate sheets, ink or fibers.

Xet Storage Details

Size:: 6.96 kB
Xet hash:: 3af10672ddc5e1764e416720a0d79c09bd88f1449a3e84e5673aeb19088ed64e

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	# dinovol embeddings of PHerc0841

	Guide and live demo: [huggingface.co/spaces/SharpMeow/dinovol-embeddings](https://huggingface.co/spaces/SharpMeow/dinovol-embeddings),
	where you can click a point of one slice and see every cube that looks like it.

	![One slice of PHerc0841 with its dinovol features shown as color](https://huggingface.co/spaces/SharpMeow/dinovol-embeddings/resolve/main/assets/dense.webp)

	Precomputed self-supervised embeddings of one whole public CT scan, so you can search, cluster and segment by
	feature similarity without running a GPU model yourself. Read any region by URL with zarr; nothing to download first.

	In plain words. dinovol is a 3D model from the Vesuvius Challenge team, trained the DINO way (self-supervised, no
	labels). It turns each small cube of a CT scan into a list of numbers, so that cubes that look alike get lists that
	point the same way. Here it has been run over every voxel of PHerc0841: each 8 x 8 x 8 voxel cube (about 75 um
	across) has 480 numbers, and each 4 x 4 x 4 cube has 64 more. The dot product of two lists says how alike two cubes
	look, which is enough for click-a-point search, clustering, or a small classifier trained from a few clicks. The
	picture above shows one slice with the three directions in which the numbers vary most as red, green and blue.

	- Scan: PHerc0841, `20250821151531-9.366um-1.2m-113keV-masked.zarr`, level 0 (19400 x 7703 x 7703 voxels), from the
	Vesuvius Challenge open data (https://scrollprize.org). PHerc0841 is not eligible for the 2027 Grand Prize or
	First Letters prizes.
	- Model: [`scrollprize/dinovol_8um_ps192_102126`](https://huggingface.co/scrollprize/dinovol_8um_ps192_102126),
	checkpoint `teacher_step_640000.pt` (sha256 `2354c493f050070e17ea0cda5cd3c1a6ebdde19a70a9c28bc9d368b80faabb86`), run
	with villa's `dinovol/dinovol_2/eval/embed_volume.py` at commit `4ea26d9cbfc1778c68e0875c21c1f670922deba1`
	(branch dinovol-refactor), robust normalization with fixed volume-wide statistics.
	- License: CC BY-NC 4.0, the license of the scan data it is computed from. Credit the Vesuvius Challenge for the
	scan and the dinovol authors for the model. The model's Hugging Face page states no license; villa's dinovol code is
	MIT.
	- Made by Chase Hendrick, Independent Researcher · ORCID 0009-0002-9754-6087, on Hugging Face Jobs (L4 GPUs).

	## What is in it

	One zarr v2 group, `PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr`, with three outputs. Each is stored as
	int8 with one float16 scale per vector: `value = q * scale`, then each vector has unit length.

	\| Array \| Shape (z, y, x, channels) \| One vector per \| Use \|
	\| --- \| --- \| --- \| --- \|
	\| `tokens_unit` (+ `tokens_unit_scale`) \| 2448 x 963 x 963 x 480 \| 8 x 8 x 8 voxels \| concept search, clustering \|
	\| `dense` (+ `dense_scale`) \| 4896 x 1926 x 1926 x 64 \| 4 x 4 x 4 voxels \| finer segmentation, same-sheet tests \|
	\| `ink` (+ `ink_scale`) \| 4896 x 1926 x 1926 x 32 \| 4 x 4 x 4 voxels \| the model's ink head features \|

	Index `i` along an axis covers voxels `[i * step, (i + 1) * step)` of the level-0 scan. Cells outside the scroll are
	all zero and their chunks are not stored. The scan was embedded in slabs of 192 slices, each on its own (its tiles read
	32 voxels past its z edges), so values can step slightly at multiples of z 192.

	## Read it

	```python
	import numpy as np, zarr # zarr 3 (tested with 3.4.1) and fsspec with aiohttp
	root = "https://huggingface.co/buckets/SharpMeow/dinovol-embeddings/resolve/PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr"
	g = zarr.open_group(zarr.storage.FsspecStore.from_url(root, read_only=True), mode="r", zarr_format=2)
	z, y, x = 9700 // 8, 3800 // 8, 3800 // 8 # a level-0 voxel, as token indices
	q = g["tokens_unit"][z - 2:z + 2, y - 16:y + 16, x - 16:x + 16].astype(np.float32)
	s = g["tokens_unit_scale"][z - 2:z + 2, y - 16:y + 16, x - 16:x + 16].astype(np.float32)
	t = q * s[..., None] # unit vectors (all zero outside the scroll)
	query = t[2, 16, 16]
	similarity = t @ query # cosine similarity to the clicked voxel
	```

	## Download it

	Everything streams, so most people never need a copy. To get files on disk, the free `hf` command
	(`pip install -U huggingface_hub`) copies any folder of the store, logged out. A copy opens with the same zarr code,
	with its folder's path in place of the URL: `zarr.open_group("pherc0841-dinovol.zarr", mode="r", zarr_format=2)`.

	\| What \| Size \|
	\| --- \| --- \|
	\| Everything \| 539 GB, about 940,000 files \|
	\| `tokens_unit` with `tokens_unit_scale` \| 179 GB \|
	\| `dense` with `dense_scale` \| 238 GB \|
	\| `ink` with `ink_scale` \| 122 GB \|

	Chunk folders follow the scan for every array: `ARRAY/Z/Y/X` with Z = z // 192 and Y, X = y // 96, x // 96 in
	scan voxels. A region (this strip, z 9,600 to 9,791 and y 3,840 to 3,935, is 40 MB and took 7 s logged out):

	```bash
	B=hf://buckets/SharpMeow/dinovol-embeddings/PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr
	U=https://huggingface.co/buckets/SharpMeow/dinovol-embeddings/resolve/PHerc0841/20250821151531-dinovol_8um_ps192_102126.zarr
	D=pherc0841-dinovol.zarr
	mkdir -p $D/tokens_unit $D/tokens_unit_scale
	for f in .zgroup .zattrs tokens_unit/.zarray tokens_unit_scale/.zarray; do curl -sL $U/$f -o $D/$f; done
	hf buckets sync $B/tokens_unit/50/40 $D/tokens_unit/50/40
	hf buckets sync $B/tokens_unit_scale/50/40 $D/tokens_unit_scale/50/40
	```

	One feature set is `hf buckets sync $B/tokens_unit $D/tokens_unit` plus the same for `tokens_unit_scale` (swap in
	`dense` or `ink`). For everything, log in first with a free account (`hf auth login`), because listing the whole
	store logged out gets rate limited, then `hf buckets sync $B $D`.

	## The whole scan in color

	`PHerc0841/20250821151531-dinovol_8um_ps192_102126-rgb.ome.zarr` shows every cube's `tokens_unit` as one color: the
	three directions in which the numbers vary most across the whole scan, each scaled from its 1st to 99th percentile.
	It is a multiscale OME-Zarr (zarr v2, uint8, axes c, z, y, x; level 0 is one color per 8-voxel cube, 3 x 2448 x 963
	x 963; levels 1 to 3 halve it), so neuroglancer streams it: the guide page has an Open in neuroglancer button. 0
	means outside the scroll. The colors show how the numbers vary; they are not labels.

	## Checks

	Before release, read back by URL: one 24 x 48 x 48 token block from every slab (5.6 million cells, 4.5 million inside
	the scroll). Every nonzero vector has unit length within 1 percent after scaling, and the nonzero cells match the
	nonzero CT voxels at pyramid level 3 exactly (the last slab, past the scan's end, was checked for norms only). The
	first slab checked in full (z 9600, 138,240 cells) matched too. These checks show the files are complete and aligned
	with the scan; they say nothing about how well the features separate sheets, ink or fibers.