README.md

metadata

title: Fossil Agent
emoji: 🦴
colorFrom: yellow
colorTo: gray
sdk: docker
app_port: 7860
pinned: false
license: mit
short_description: Fossil photographs to a 3D mesh, via SAM3 and Meshy.

Fossil Agent — a Hugging Face Space

Chainlit UI over the fossil digitisation pipeline. Attach 1–4 photographs of one specimen from different angles; the agent removes the background, chooses its own SAM3 prompt points, segments each view, and reconstructs a 3D mesh from the cutouts.

Deploying

huggingface-cli repo create fossil-agent --type space --space_sdk docker
git clone https://huggingface.co/spaces/<you>/fossil-agent && cd fossil-agent

# the Space files, plus the project packages they import
cp -r <repo>/space/* .
cp -r <repo>/{config,helpers,tools,schemas,fossil_agents,skills} .

git add -A && git commit -m "fossil agent space" && git push

The project packages must sit next to app.py at the Space root — app.py puts the root on sys.path and imports them directly.

Secrets

OPENAI_API_KEY	required — the default model is gpt-5.6-luna
HF_TOKEN	facebook/sam3 is a gated repo; accept the licence on its model page or SAM3 will fail to load
MESHY_API_KEY	only needed if you enable 3D reconstruction

Variables

FOSSIL_MODEL_ID	default openai/gpt-5.6-luna
FOSSIL_BG_BACKEND	local (GrabCut, no token) or hf (RMBG-2.0)
FOSSIL_MESHY_TEXTURE	initial state of the texture switch

Hardware — read this before you pick the free tier

SAM3 needs a GPU to be usable. On free CPU Spaces it will load (slowly, ~3 GB) and run at minutes per view, on 12 MP photographs, for every view of every specimen. The Space will look hung. Budget a T4 or better, or expect a demo that technically works and practically doesn't.

Background removal is fine on CPU with FOSSIL_BG_BACKEND=local (GrabCut, no network, no token) — that path is deliberately dependency-light so the pipeline degrades rather than dies.

What the UI does that the CLI doesn't

A specimen is a set. Attachments become a folder of views. Gallery cards arm an image set, not a file. Attach more than 4 and the Space tells you what it dropped — Meshy's limit is a hard 400, and which four you keep matters more than how many you had.

The mesh is viewable. GLB opens in a real orbit-controlled 3D viewer in the right-hand panel, not an image. You cannot judge a reconstruction from a thumbnail: the question is always whether the faces your photographs didn't show were observed or invented, and you have to turn it around to see.

A live timer. The pipeline is slow by nature — a reasoning model choosing keypoints, SAM3 per view, then Meshy polling for minutes. Without a moving number a long turn is indistinguishable from a hung one.

3D is off by default. Meshy bills per task. The switch is off, the run reports the request it would have sent, and nothing is spent. Failed Meshy tasks are refunded; wrong-but-successful ones are not — so look at the SAM3 overlay before you flip it. That ordering is the whole reason the switch exists.

The toggles are per-session. They live in ADK session state, not process env. A Space serves many users from one process, and a global FOSSIL_MESHY_ENABLED would let one person's switch spend another person's credits.

Artefacts

Every turn surfaces buttons that open in the right-hand panel:

📐 coordinate canvas	the grid the analyst reads its coordinates off
📍 keypoints (per round)	where the analyst actually put its points, and how it revised
🎯 SAM3 overlay	look at this before enabling Meshy
🪨 SAM3 cutout	exactly what Meshy will see
🧊 mesh (GLB)	orbit it

Limits

A generative mesh is a visualisation, not a measurement. It is unitless, and Meshy infers any surface your photographs didn't cover. Multi-view triangulates rather than hallucinating the back — a real difference in kind from single-image — but it is still a generative model, not photogrammetry. For morphometrics you want a calibrated rig and a scale bar, and this pipeline would feed it masks rather than calling Meshy at all.