unroll / docs /ADDING_A_SCROLL.md
SharpMeow's picture
One page scroll bar and American English
757768f verified
|
Raw History Blame Contribute Delete
10.3 kB

Adding a scroll

The page draws whatever is listed in scrolls/index.json. Each entry points to a manifest that describes one scroll: its windings, the meshes and images behind them, its ink layers, and who made the data. Nothing about a particular scroll is hardcoded in app.js.

When the registry holds more than one scroll, a scroll picker appears at the top right. With only one (and nothing opened locally) the picker stays hidden.

Files

scrolls/index.json                 the registry
scrolls/<id>/manifest.json         one manifest per scroll
lib/manifest.js                    schema helpers and the validator (shared by the page and Node)
lib/scrolls.js                     loading: registry, local storage, mesh scrolls, atlases
lib/formats.js                     OBJ, PLY, TIFF and zip readers, all in the browser
lib/openown.js, lib/example.js     "Open your own" and the example package
tools/validate_scroll.mjs          the validator, from the command line
tools/precompute_template.py       a starting point for turning traced windings into a package

The registry

{
 "schema": "unroll-registry/1",
 "default": "PHerc0139",
 "scrolls": [
  { "id": "PHerc0139", "name": "PHerc. 139", "manifest": "scrolls/PHerc0139/manifest.json" }
 ]
}

default is the scroll shown when the URL does not name one. Other scrolls open with #s=<id> in the URL.

The manifest

{
 "schema": "unroll-scroll/1",
 "id": "MyScroll",
 "name": "My scroll",
 "description": "One or two sentences.",
 "voxelMM": 0.00791,
 "geometry": { "format": "mesh", "units": "voxels", "axis": "z" },
 "windings": [
  { "id": "w012", "mesh": "meshes/w012.obj" },
  { "id": "w011", "mesh": "meshes/w011.ply" }
 ],
 "outermost": "first",
 "layers": [
  { "id": "xray", "role": "ct", "label": "X-ray", "texture": "xray/{id}.jpg", "source": "Surface volume, 7.91 um scan", "resolutionUM": 7.91 },
  { "id": "ink", "role": "ink", "label": "Published ink (2.4 µm)", "texture": "ink/{id}.png", "files": { "w011": "ink/w011.tif" },
    "source": "Which model, on which scan", "resolutionUM": 2.4, "hi": "closeups/{id}.jpg" },
  { "id": "model", "role": "overlay", "label": "ink_9um (9 µm)", "windings": { "range": ["w011", "w012"] },
    "texture": "model/{id}.png", "source": "ink_9um on the 9.4 um scan", "resolutionUM": 9.362 }
 ],
 "credits": { "text": "Who made the data", "url": "https://scrollprize.org", "citation": "How to cite it" },
 "dataTerms": { "license": "CC BY-NC 4.0", "url": "https://creativecommons.org/licenses/by-nc/4.0/", "notes": "Anything else" }
}
Field Meaning
schema Always unroll-scroll/1.
id Letters, digits, ., -, _; up to 64 characters. Used in the URL.
voxelMM Scan voxel size in millimeters. Meshes in voxel units are scaled by it.
base Optional folder that asset paths are relative to, itself relative to the manifest. Registry manifests use "../../" to reach data/.
geometry.format mesh (OBJ or PLY per winding) or grid-int16 (the packed format tools/build.py writes; registry only).
geometry.lite Optional, grid-int16 only: { "index", "bin", "binPacked" } of a coarser copy of the mesh (python tools/build.py --coarsen 2 data/<scroll>) that phones load instead. ?mesh=full or ?mesh=lite overrides.
geometry.units voxels (default) or mm.
geometry.axis z (default: the scan convention, roll axis along z) or y. The page centers the roll on its bounding box.
geometry.flipV Default true: OBJ and PLY have v pointing up, so the image's top row is v = 1. Set false if your v points down.
windings A list of { "id", "mesh" }, or { "range": ["w023", "w059"], "mesh": "meshes/{id}.obj", "skip": [], "outermost": "last" }. A range keeps the zero padding.
outermost first (default) or last: which end of the list is the outside of the roll. The page unrolls from the outside in.
layers Up to one layer per role: ct (the X-ray chip), ink (amber) and overlay (cyan, with the compare swipe). At least ct or ink.
layers[].windings "all" (default), { "range": [first, last] }, or a list of ids.
layers[].texture Path pattern with {id}, for mesh scrolls and overlays. files overrides it for single windings.
layers[].atlas One atlas image for the whole scroll (grid-int16 only).
layers[].hi Optional close-up image per winding (pattern with {id}). Without it the close-up uses the layer's texture.
layers[].label, hint, title, closeupLabel, readoutLabel, bandTitle, notes, source, resolutionUM What the page says about the layer: chip text, the line under the chips, the tooltip, the close-up tab, the readout, the scrubber band, and the close-up caption.
layers[].stats Registry only: a JSON file with per-winding run details (see data/gpu.json).
credits, dataTerms Shown in the footer and the About sheet.
display Registry only: page title, headline, lede, About paragraphs and share-image text, as HTML. Manifests opened by visitors are always shown as plain text.

Image layers on a mesh

Each layer image covers its winding's UV square: the left edge is u = 0, the right edge u = 1, the top row v = 1. Images can be PNG, JPG or TIFF (8, 16 or 32 bit; none, LZW, Deflate or PackBits compression); color images use their red channel. 16- and 32-bit images are stretched to the range they use.

The page measures each winding's flattened width and height from the mesh and its UVs, packs the X-ray and ink images into two atlases (4096 px, 2048 on phones), and lays the windings end to end. Without an X-ray layer it paints the mesh's UV footprint instead, so Tear and Crush still know where the papyrus is.

What works with what

Feature Needs
Unroll, X-ray, ink a mesh and one image layer per winding
Tear, Crush, Shake the same (Tear cuts along the X-ray or the UV footprint)
Compare swipe an overlay layer on that winding
Scale bar always (the width comes from the mesh)
Readout row for the overlay stats (registry only)

Adding a scroll to the registry

  1. Build the package. tools/precompute_template.py turns tifxyz folders or OBJ files plus layer images into meshes, images and a manifest. Fill in its TODOs.
  2. Put it at scrolls/<id>/ and add an entry to scrolls/index.json.
  3. Run node tools/validate_scroll.mjs. It checks the registry, every manifest, every referenced file and the size limits, with the same code the page uses.
  4. Serve the folder (python3 -m http.server) and open http://localhost:8000/#s=<id>.
  5. Store images and meshes with Git LFS or Xet when you push to the Space; Hugging Face refuses plain binary files.

Assets load per scroll: only the chosen scroll's files are fetched. Within a scroll the overlay images load the first time the overlay is shown, and close-up images load per winding when a winding is opened.

Open your own

Visitors can open a scroll without adding it to the registry. Press + at the top right (or drag files onto the page).

  • Nothing is uploaded. The files are read by the page in the browser with the File API, checked with the same validator, and kept in this browser's IndexedDB until removed. No server receives them.
  • Ways in: drop or pick a zip, a folder, or a manifest together with its files; or paste the URL of a manifest or a zip. A URL is fetched by the browser directly from that server, so the server must allow cross-origin requests (CORS). Files beside a manifest URL are fetched relative to it.
  • Formats: OBJ or PLY meshes with UVs (geometry.format must be mesh), PNG, JPG or TIFF layers.
  • Limits: 64 MB per file, 128 MB per zip, 256 MB in all, 64 windings, 2 million vertices, 8192 px per image side.
  • Errors are listed by file and field: a missing file, a mesh without UVs, an image the browser cannot decode, a size over the limit.
  • A scroll opened this way appears in the picker as "(local)", with its own URL (#s=local-...) that only works in the browser that opened it. Remove it under +.
  • Manifests opened this way are shown as plain text: display and other HTML fields are ignored, so a shared link cannot inject markup into the page.

The example package

Download example package in the Open your own sheet builds unroll-synthetic-scroll.zip in the browser: three windings of a made-up spiral (two OBJ, one binary PLY, in millimeters with the roll axis along y), fake X-ray fibers, Greek text as ink in PNG, JPG and TIFF, and a blurred overlay on the middle winding so the compare swipe has something to show. Drop the zip straight back onto the page to see it unroll, then use its manifest.json as a starting point.

For agents

An agent can add or open a scroll without a mouse. The JSON schema of the manifest is in /api.json under manifestSchema, and /AGENTS.md covers the whole page API.

  • Check a manifest: await unroll.open.validate(manifest) returns { valid, errors, warnings, files } with the same rules as the page and tools/validate_scroll.mjs.
  • Open from a URL: await unroll.open.fromURL('https://example.org/my-scroll.zip') (the server must send CORS headers).
  • Open from bytes: await unroll.open.fromFiles([{ name: 'my-scroll.zip', data }]), where data is an ArrayBuffer, a Uint8Array, a Blob or a base64 string. For loose files, give each one with its relative path as name (manifest.json, meshes/w001.obj, ...). On success the page reloads into the scroll; then await unroll.waitFor('ready').
  • Start from the example: const ex = await unroll.open.example() returns the example zip as base64; feed it to open.fromFiles([{ name: ex.name, data: ex.base64 }]).
  • Clean up: open.listLocal() and open.removeLocal(key).
  • Registry scrolls: after step 3 above, node tests/coverage.mjs and node tests/api.mjs against a local server check that every control still has an API equivalent and that the API works on the new data.

Errors come back as { ok: false, error, code } with the file and field named, so an agent can fix the package and retry.