README.md

---
license: apache-2.0
title: CloneForge
sdk: gradio
emoji: 😻
colorFrom: blue
colorTo: green
short_description: Real-time multimodal object-cloning agent swarm — Gemma 4 31
---

# ⚒️ CloneForge

**Real-time multimodal object-cloning agent swarm — Gemma 4 31B on Cerebras.**

Upload or webcam a photo of a physical object; a swarm of specialized agents analyzes it,
plans a fabrication strategy, generates parametric 3D code, **visually critiques its own
result against your photo**, and emits a watertight, 3D-printable STL — in seconds, thanks
to Cerebras inference (~1,500+ tok/s).

```
photo(s) ─► 👁 Vision ─► 🧠 Planner ─► 🛠 Generator ─► 🔎 Visual Critic ─┐
              (specs)     (primitives)   (trimesh code)   (render vs photo) │
                                  ▲                                         │
                                  └──────── fix & regenerate (≤N) ◄─────────┘
                                                   │
                                      watertight STL + 3D preview + quality report
```

## Why it's different
- **The critic has eyes.** After building a mesh we render it (4 views) and send that render
  *back to Gemma alongside the original photo* — the model sees the mismatch and issues concrete
  fixes. This render→VLM→fix loop is the main fidelity lever (cf. Query2CAD, CADCodeVerify, LL3M).
- **Watertight by construction.** Output is a composition of parametric primitives
  (box/cylinder/sphere/torus + booleans), so meshes are print-ready with no repair pass —
  unlike neural image→3D models (TripoSR/TRELLIS/Hunyuan3D) that need GPUs and produce
  non-manifold draft meshes. It's also **editable**: ask for "20% taller" and it re-runs.
- **Speed is the demo.** A full clone (8–12 agent calls) runs in ~5 s of compute. The
  Speed Race tab shows Cerebras vs OpenAI side-by-side with live TTFT + tok/s.

## Quickstart
```bash
# Python 3.14, uv (python -m venv is unavailable here; uv handles the 3.14 wheels)
uv venv --python 3.14 .venv && . .venv/bin/activate
uv pip install -r requirements.txt

# .env needs:  CEREBRAS_API_KEY=...   OPENAI_API_KEY=...   (OpenAI = fallback + race lane)
python app.py        # open the printed local URL
```

## The app (3 tabs)
- **Clone** — photo (upload/webcam) + optional extra views → live agent transcript, mesh render,
  3D preview, downloadable STL. A **Refine** box applies text corrections ("thinner handle")
  reusing the cached analysis.
- **Examples** — curated reference objects with **published ground truth** (LEGO 3001, ISO 7089
  washer, DIN 934 nut, 16 mm die, mug, 20-tooth gear). One click clones them and reports
  **dimension match %** and Chamfer/voxel-IoU. Simple parts score high; the gear shows the
  fidelity gap on complex geometry — honest by design.
- **Speed Race** — same prompt, Cerebras ⚡ vs OpenAI 🐢, live first-token latency + tok/s.

## Architecture
| Module | Role |
|---|---|
| `cloneforge/llm.py` | Unified `AsyncOpenAI` client for **both** providers (Cerebras is OpenAI-compatible); streaming, multi-image input, strict JSON schema, 30-rpm backoff → OpenAI fallback |
| `cloneforge/schemas.py` | Pydantic agent I/O → strict `json_schema` (`strictify`) |
| `cloneforge/agents.py` | Vision · Planner · Generator · **Visual Critic** (zero tool-calling — see below) |
| `cloneforge/fabricate.py` | Sandboxed exec of generated code (whitelisted imports) + STL/GLB + watertight validation + stderr self-repair |
| `cloneforge/render.py` | Headless matplotlib 4-view shaded render (no GPU/X11/sudo) |
| `cloneforge/quality.py` | OBB dimension match + Chamfer + voxel-IoU vs ground truth |
| `cloneforge/silhouette.py` | Silhouette-IoU vs the input photo (best-of-N ranking + shape-match score) |
| `cloneforge/orchestrator.py` | Async-generator pipeline (streams to UI) + `refine_pipeline` |
| `cloneforge/examples.py` | Reference library from published part specs |
| `app.py` | Gradio UI |

## Key technical decisions (verified against the live API)
- **Images cannot be combined with tool calling** on Gemma 4 → we use **structured outputs
  everywhere, zero tool calling.**
- **`reasoning_effort` levels are equivalent** on Gemma 4 and *destabilize* structured output
  (empty JSON) → kept **off** on schema'd agents.
- **30 rpm** rate limit → vision is computed once and cached across critic/refine iterations;
  webcam is snapshot-only; 429 → bounded backoff → OpenAI `gpt-5.4-mini` fallback.
- **Python 3.14, no sudo** → CadQuery (≤3.12) and OpenSCAD (apt) are out; **trimesh + manifold3d**
  (pure pip) is the generator, **matplotlib** the renderer.

See [PLAN.md](PLAN.md) for the full build plan and [FIDELITY.md](FIDELITY.md) for the fidelity
analysis, competitive landscape, and roadmap. Demo script: [DEMO.md](DEMO.md).