Spaces:

build-small-hackathon
/

Snap2Sim

Running

App Files Files Community

Snap2Sim / prompts /PROMPT.md

jasondo

Finalize docs and archive prompts

7006f4b 26 days ago

preview code

Raw

History Blame Contribute Delete

15.8 kB

A newer version of the Gradio SDK is available: 6.20.0

Upgrade

Snap2Sim — "Inside the Machine"

Build Small Hackathon · Backyard AI Track · huggingface.co/build-small-hackathon

Final Status (June 15, 2026)

This file is the original build prompt and contains historical implementation directions, including early A-Frame and private-Space assumptions that were superseded during development. The finished public submission uses deterministic browser-side Three.js from validated JSON, is hosted under build-small-hackathon/Snap2Sim, and is linked from the README.

Public Space: https://huggingface.co/spaces/build-small-hackathon/Snap2Sim
App host: https://build-small-hackathon-snap2sim.hf.space
Demo video: https://youtu.be/nuisDKMyyF8
X post: https://x.com/Ryno67114241/status/2066660199558152411

Goal

Build a Gradio app deployed as a Hugging Face Space that takes a photo of a hardware component (gear, valve, pump, lock, engine part, etc.) and produces an animated 3D visualization showing how that component works internally — "open it up and show me the moving parts and the mechanism."

Hard Constraints

Total model parameters across the entire pipeline ≤ 32B
Must be a Gradio app hosted as a Hugging Face Space
No cloud AI APIs at inference time where possible (targets "Off the Grid" bonus)
Plain HTML/CSS/JS frontend — no React, no build step, no bundler

Target User — The Curious Tinkerer / Maker

Someone who pulls apart old electronics, finds a mystery component at a thrift store or salvage yard, or cracks open a broken appliance wondering: "what does this actually do and how does it work inside?" Not an engineer — a curious, hands-on person who learns by taking things apart.

README must open with: "You find a small metal cylinder at a flea market. What is it? How does it work inside?" — before any technical description.

Archived Current State (June 13, 2026)

Scaffold existed at github.com/Bigstonks1/Snap2Sim, initially synced to HF Space jasondo111/Snap2Sim. The finished project was later transferred to the public Build Small Hackathon Space listed above.

Confirmed working:

Modal deployment: snap2sim-inside-the-machine (bigstonks1 workspace)
smoke_test_llamacpp_image → "ok": true for UD-Q4_K_M + mmproj-F16.gguf
analyze_image_llamacpp and scene-generation Modal endpoints deployed
GitHub → HF Space sync workflow live and passing
Local app code now uses gradio.Server plus a trusted index.html

Archived primary tasks (completed):

Deploy the current branch through GitHub → HF sync
Confirm the deployed Space loads the trusted index.html
Confirm /analyze_image and /generate_scene respond through the secured Modal bearer-token flow

Final result: deployed through GitHub-to-Hugging Face sync, verified on the public org-owned Space, and completed with public demo video and X post links.

Model Stack

Primary model: NVIDIA Nemotron 3 Nano Omni (30B-A3B, MoE, ~3B active params) Used for both vision analysis and A-Frame scene generation (two prompt turns, same model). Targets the NVIDIA Nemotron Quest sponsor award. ~31B total — under the 32B cap.

GGUF Path (confirmed working)

Setting	Value
Repo	`unsloth/NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF`
Primary quant	`UD-Q4_K_M` (~24 GB) + `mmproj-F16.gguf`
Fallback quant	`UD-IQ2_M` (~18.5 GB)
Runtime	`llama-mtmd-cli` via llama.cpp on Modal GPU

Fallback Split Pipeline

Only use if primary model code-gen quality is too weak:

Vision: NVIDIA Nemotron Nano V2 VL (12B)
Scene gen: Qwen2.5-Coder-14B
Total ~26B · still qualifies for Nemotron Quest

Model Runtime

Inference via llama.cpp / GGUF → targets "Llama Champion" bonus
Modal GPU endpoints called over HTTP with Bearer token auth (SNAP2SIM_API_TOKEN)
Backend swappable via INFERENCE_BACKEND=modal | zerogpu | local
If Modal deployed: "Off the Grid" not claimed, but Llama Champion + Nemotron Quest + Modal Award all apply
If ZeroGPU sufficient: "Off the Grid" additionally claimable

Deployment Architecture

[HF Space — CPU tier]              [Modal — GPU tier]
  gradio.Server                      analyze_image_llamacpp
  @app.get("/") → index.html    ←→   generate_scene
  @app.api() → /analyze_image        (weights cached in Modal Volume)
  @app.api() → /generate_scene

Gradio app calls Modal endpoints over HTTP via requests, image passed as base64
Modal cold starts on 30B model can take tens of seconds → show "WAKING THE WORKSHOP..." loading state

Architectural Shift — `gr.Blocks` → `gradio.Server`

This is the core change. Do not skip or partially implement it.

Why

gr.HTML strips <script> tags for security and HF Spaces CSP blocks external CDN imports in js_on_load. Any WebGL/Three.js/A-Frame output piped through gr.HTML will fail on the live Space — scripts get stripped, nothing renders. This is a confirmed, known issue.

How `gradio.Server` Fixes It

gradio.Server extends FastAPI. @app.get("/") serves index.html as a first-class trusted FastAPI response — the browser receives a full page with no stripping, no sandboxing, no CSP conflicts from Gradio's component system. A-Frame and Three.js CDN scripts load normally.

from gradio import Server

app = Server()

@app.get("/")
async def homepage():
    with open("index.html") as f:
        return HTMLResponse(f.read())

@app.api(name="analyze_image")
def analyze_image(image_b64: str) -> dict:
    return backend.run_analysis(image_b64)  # calls Modal or local placeholder

@app.api(name="generate_scene")
def generate_scene(mechanism_json: dict) -> str:
    return backend.run_scene_gen(mechanism_json)  # returns A-Frame HTML string

app.launch()

What to Change in the Scaffold

File	Action
`app.py`	`gradio.Server` app serving `index.html` and API routes
`index.html`	Trusted HTML/CSS/JS shell with pipeline orchestration
`modal_app.py`	`generate_scene` endpoints and A-Frame prompt
`snap2sim/backend.py`	`generate_scene` backend method
`snap2sim/prompts.py`	A-Frame scene-generation prompt
`snap2sim/aframe_scene.py`	Deterministic A-Frame placeholder scene

Rendering Stack — Two Layers

Layer 1 — Model-Generated Scene: A-Frame (declarative HTML)

A-Frame is a web framework built on Three.js that uses declarative HTML tags for 3D scenes. The model outputs HTML, not JavaScript — far more reliable for LLM generation.

Why A-Frame for model output:

LLMs generate HTML tags far more reliably than imperative JS
Injected via innerHTML, not eval() — no script execution risk
A-Frame runtime (already loaded in <head>) renders injected tags automatically
Built-in animation attribute handles motion without JS animation loops
Camera, lighting, and sky added automatically — less boilerplate to get wrong

Loading A-Frame in index.html:

<head>
  <script src="https://aframe.io/releases/1.6.0/aframe.min.js"></script>
</head>

CDN fallback: If HF Spaces blocks aframe.io, vendor the minified A-Frame JS (~1.1MB) as a static file served via gradio.Server's FastAPI static file mounting.

Injecting model output:

document.getElementById('viewport').innerHTML = modelGeneratedAframeHTML;
// A-Frame runtime picks up the new <a-scene> tags automatically

Example of what the model should output:

<a-scene>
  <a-sky color="#0F1318"></a-sky>
  <a-cylinder color="#E8A33D" radius="0.3" height="1" position="0 1 -3"
    animation="property: rotation; to: 0 360 0; loop: true; dur: 2000; easing: linear">
  </a-cylinder>
  <a-box color="#5FD4D0" position="0.8 0.5 -3"
    animation="property: position; to: 0.8 1 -3; dir: alternate; loop: true; dur: 1000">
  </a-box>
  <a-text value="Drive Shaft" position="0 2 -3" color="#5FD4D0" scale="0.5 0.5 0.5">
  </a-text>
</a-scene>

Prompt engineering for generate_scene endpoint:

Instruct the model to output only the <a-scene>...</a-scene> block — no preamble, no markdown fences, no explanation
A-Frame primitives to use: <a-box>, <a-cylinder>, <a-sphere>, <a-torus>, <a-cone>, <a-entity>
Animation format: animation="property: rotation; to: 0 360 0; loop: true; dur: 2000; easing: linear"
Keep scenes to 3–6 parts maximum for clarity
Set <a-sky color="#0F1318"> to match the page background

Layer 2 — Deterministic Fallback: Three.js (human-written)

If A-Frame output is empty, malformed, or renders blank after 3 seconds, immediately swap to buildDeterministicScene(json) — a JS function that reads the mechanism JSON and builds a reliable Three.js scene from geometric primitives.

function buildDeterministicScene(mechanismJson) {
  // Human-written. Always works given valid JSON.
  // Uses: BoxGeometry, CylinderGeometry, TorusGeometry per part shape
  // Applies rotation/translation per motion_type
  // Adds OrbitControls, annotation labels
  // The viewport must never be blank or show an error
}

Load Three.js in index.html alongside A-Frame:
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>

Pipeline / Application Flow

1. User uploads photo
        ↓
2. Frontend encodes as base64 → calls /analyze_image (Gradio JS client)
        ↓
3. Nemotron vision step → structured JSON:
   {
     "component_name": "Solenoid Valve",
     "parts": [
       { "name": "Coil", "shape": "cylinder", "color": "#E8A33D",
         "position": [0, 0, 0], "motion_type": "none", "motion_params": {} },
       { "name": "Plunger", "shape": "cylinder", "color": "#5FD4D0",
         "position": [0, 0.5, 0], "motion_type": "translate",
         "motion_params": { "axis": "y", "range": 0.3, "dur": 800 } }
     ],
     "summary": "When current flows through the coil, it generates a magnetic
                 field that pulls the plunger upward, opening the valve port."
   }
        ↓
4. Frontend populates analysis panel (name, part list, summary)
   → immediately calls /generate_scene with JSON
        ↓
5. Nemotron scene gen step → A-Frame HTML string (<a-scene>...</a-scene>)
        ↓
6. Frontend injects A-Frame HTML → innerHTML of #viewport
   A-Frame runtime renders automatically
        ↓
7. If A-Frame blank/failed after 3s → buildDeterministicScene(json)
   Viewport is NEVER blank

Visual Design — "Industrial Instrument Panel / Field Cutaway"

Implement the shell and CSS in Step 2. Do not work on [POLISH] items until Steps 1–4 are done.

Color System

:root {
  --bg:         #0F1318;
  --bg-panel:   #161B22;
  --bg-lift:    #1E2530;
  --amber:      #E8A33D;
  --amber-dim:  #7A5420;
  --cyan:       #5FD4D0;
  --cyan-dim:   #2A5E5C;
  --text:       #C8C0AC;
  --text-muted: #6B7280;
  --grid:       rgba(255,255,255,0.04);
}

Typography

Load from Bunny Fonts (not Google Fonts):

Display / headings / UI labels: Chakra Petch — technical, instrument-panel character
Monospace / data / callouts: Fira Code
Never use: Inter, Roboto, Arial, Space Grotesk, or any system font

Layout

Two-pane asymmetric split: 63% viewport (left) · 37% analysis panel (right)
Blueprint grid: 1px lines at --grid opacity, 32px spacing, on --bg base
Panel separator: 1px vertical line in --amber-dim
Upload drop zone: fills viewport · dashed 1px --amber-dim border (no border-radius) · centered "DROP COMPONENT PHOTO" in Chakra Petch uppercase --text-muted · on drag-over: border → --amber, text → --amber
Play/pause: minimal amber rectangle (no border-radius), Chakra Petch uppercase "PAUSE" / "RESUME", controls A-Frame animation playback via JS

Loading States

All in Chakra Petch uppercase, --amber color, with thin --amber indeterminate progress bar across viewport top:

State	Message
Modal cold start	`WAKING THE WORKSHOP...`
Vision inference	`ANALYZING ASSEMBLY...`
Scene generation	`RENDERING CUTAWAY...`

[POLISH] — Only After Steps 1–4 Work

Implement in this sub-order:

Component name watermark — large (clamp(4rem, 8vw, 9rem)) Chakra Petch uppercase in --bg-lift, absolutely positioned bleeding across both panes from bottom-left, z-index below content. Populated from component_name in JSON.
Noise texture — SVG feTurbulence grain at 3% opacity on viewport pane, inline data: URI, no external file. Makes the surface feel physical.
Vignette — radial gradient overlay on viewport edges, pointer-events: none so it floats above the A-Frame canvas without blocking interaction.
Scan-line reveal — when A-Frame scene first loads:
- 2px --cyan scan-line sweeps top→bottom over 0.7s
- Each A-Frame entity fades in as line passes it (opacity 0→1, translateY 12px→0, 0.35s ease-out, staggered by part index via animation-delay)
- Part labels fade in together (0.25s)
- Progress bar dissolves (0.2s)
- Total: ~1.2s · this is the signature moment

Deliverables

app.py — gradio.Server app (~50 lines)
index.html — self-contained HTML/CSS/JS; A-Frame + Three.js from CDN; Gradio JS client from CDN
snap2sim/backend.py — generate_scene
modal_app.py — generate_scene; A-Frame prompt
snap2sim/prompts.py — updated A-Frame scene generation prompt
requirements.txt — updated if needed for gradio.Server
README.md — tinkerer/maker story hook → project description → model stack with exact parameter breakdown (≤32B) → rendering stack rationale → bonus quest claims:

Quest	Status
Llama Champion	✅ Confirmed
NVIDIA Nemotron Quest	✅ Confirmed
Off-Brand	✅ Confirmed
Modal Award	✅ Confirmed
Off the Grid	⚡ If ZeroGPU used in final deploy
Field Notes	🎯 Stretch

Original Start Order (completed)

Follow this exactly. Do not skip ahead.

Step 1 — Critical Path

Deploy the current gradio.Server + index.html implementation through GitHub -> HF sync. Confirm the deployed Space loads and both /analyze_image and /generate_scene respond.

Step 2 — Make It Functional

Build buildDeterministicScene(json) in JS — Three.js scene from geometric primitives, always works given valid JSON. Wire the full pipeline: upload → /analyze_image → populate analysis panel → /generate_scene → inject A-Frame HTML via innerHTML → fallback to buildDeterministicScene(json) if A-Frame fails or blanks after 3 seconds. Confirm end-to-end with a real image and live Modal endpoints.

Step 3 — Apply the Design Shell

Add CSS variable system, Chakra Petch + Fira Code from Bunny Fonts, two-pane asymmetric layout, blueprint grid, loading states with progress bar, upload drop zone, panel separator. App should match the design direction above — minus [POLISH] items.

Step 4 — Harden

Error handling, 3-second blank-scene timeout before fallback triggers, Modal cold-start messaging, A-Frame <a-sky color="#0F1318"> matching page background, play/pause toggle wired to A-Frame animation playback. Verify GitHub -> HF Space sync pushes cleanly and the deployed Space runs correctly.

Step 5 — [POLISH] Only if Time Remains

Component name watermark → noise texture → vignette → scan-line reveal. In that order.