01 / THE PROBLEM
SNAP2SIM // INSIDE THE MACHINE
"You find a small metal cylinder at a flea market. What is it? How does it work inside?"
  • Curious makers, repair hobbyists, thrift scavengers, and robotics students constantly hold mystery hardware and cannot see what is happening inside.
  • Existing image AI stops at a label - "this is a valve" - instead of answering the real question: what moves, and why?
  • Teardowns, manuals, and exploded diagrams are scattered, intimidating, or do not exist for that exact part.
02 / THE SOLUTION
SNAP2SIM // INSIDE THE MACHINE
Snap a photo. Watch the mechanism come alive.

Snap2Sim turns a single photo of a hardware component into a narrated, animated 3D mechanical cutaway - naming internal parts and showing hidden motion.

[ identify ]

Likely mechanism

Confidence and mechanism type, not just a class name.

[ explain ]

Plain-language motion

Internal parts, trigger, and sequence: plunger, spring, rotor, pawl, port.

[ animate ]

Orbit the cutaway

Three.js field-manual view with orbit, zoom, reset, and play/pause.

Honesty is part of the product: when confident 3D reconstruction is not justified, Snap2Sim falls back to annotated callouts on the original photo instead of pretending.

03 / HOW IT WORKS
SNAP2SIM // INSIDE THE MACHINE
Snap2Sim architecture flow Photo upload flows through the Hugging Face Space, Modal GPU endpoint, NVIDIA Nemotron with llama.cpp, JSON validation, and deterministic browser rendering, with low confidence going to an annotated-photo fallback. Photo upload component image POST /analyze_image Hugging Face Space gradio.Server / cpu-basic bearer-token secret Modal GPU endpoint llama.cpp + mtmd NVIDIA Nemotron 3 Nano Omni 30B-A3B GGUF / UD-Q4_K_M mmproj-F16 returns JSON /generate_scene schema validation render-mode select Browser renders deterministic Three.js cutaway orbit / zoom / reset / play low confidence Annotated-photo fallback The model describes the mechanism. The browser owns the rendering.

No model-authored HTML/JS touches the page. The renderer consumes validated mechanism JSON, which keeps the live demo reliable and safe.

04 / LIVE DEMO
SNAP2SIM // INSIDE THE MACHINE

The moment: photo in, cutaway out.

CUTAWAY READY
Piston Mechanism
confidence 85%

Judges should feel the handoff.

  • Upload a mystery part photo.
  • Watch analysis become named parts and motion.
  • Orbit, zoom, reset, and play/pause the deterministic scene.
  • Move the confidence slider and see fallback behavior stay honest.
05 / DIFFERENTIATION
SNAP2SIM // INSIDE THE MACHINE

Most image demos stop at a label.

Snap2Sim answers the maker's next question: what would I see if I cut this open?

Typical image demo"This is a valve."
Snap2SimLikely mechanism, internal parts, trigger, and motion sequence.
Static outputCaption, chat answer, or generic class label.
Moving cutawayMaps parts into a technical field-manual scene with animation.
OverconfidentLooks plausible even when geometry is unsupported.
Honest uncertaintyConfidence threshold plus annotated-photo fallback.
Default app shellGeneric chat or form UI.
Branded maker toolCutaway aesthetic, same-origin client calls, server-side secrets.
06 / TECH STACK + HACKATHON FIT
SNAP2SIM // INSIDE THE MACHINE

Stack

  • Frontend: plain index.html + CSS + JS served by gradio.Server.
  • Rendering: deterministic Three.js primitives: box, cylinder, cone, capsule, sphere, rod, gear, torus, spring.
  • Controls: OrbitControls, camera fitting, reset view, confidence threshold, annotated-photo fallback.
  • Model: NVIDIA Nemotron 3 Nano Omni 30B-A3B GGUF, UD-Q4_K_M + mmproj-F16.
  • Budget: ~30B total / ~3B active per MoE token, under the 32B single-model budget.
  • GPU: Modal with weights cached in a Modal Volume and bearer-token protection.
  • Contract: JSON schema validation before anything renders.

Badges we are claiming

Backyard AI / core fit Llama Champion NVIDIA Nemotron Quest Modal Award Off-Brand OpenAI Codex Best Demo / pending video Sharing is Caring / pending post Off the Grid / not claimed

Inference currently runs on Modal, so Off the Grid is intentionally not claimed.

07 / CLOSE
SNAP2SIM // INSIDE THE MACHINE

Snap a photo. See inside the machine.

"You find a small metal cylinder at a flea market. What is it? How does it work inside?"

GitHub https://github.com/Bigstonks1/Snap2Sim
HF Space

https://huggingface.co/spaces/build-small-hackathon/Snap2Sim
Public submission Space under the Build Small Hackathon organization.

Credit

Built by Jason Do with implementation assistance from OpenAI Codex.

Snap2Sim // Inside the Machine // Build Small Hackathon