README.md

metadata

title: Screenshot to React
emoji: 🖼️
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 5.50.0
app_file: app.py
pinned: false
license: gemma
short_description: Turn a UI screenshot into React code, rendered live
models:
  - Reubencf/gemma-3-27b-it-vlm-react-screenshot-to-code
  - Reubencf/gemma-3-4b-it-vlm-react-screenshot-to-code
datasets:
  - Reubencf/frontend-react-dataset
tags:
  - screenshot-to-code
  - react
  - gemma3
  - vlm

🖼️ → ⚛️ Screenshot to React

Upload a screenshot of a UI. Gemma 3 27B IT VLM with the gemma-3-27b-it-vlm-react-screenshot-to-code LoRA adapter writes a React component — and the Space renders it live.

How it works

1. The screenshot and instruction go through the Gemma 3 chat template (image → 256 soft tokens at 896×896).
2. The LoRA adapter is applied to the base model, and generation streams back token by token.
3. The React code is pulled out of the response (markdown fences stripped, truncated blocks recovered).
4. import statements are rewritten, not stripped — they're rebound onto UMD globals — and the default export is mounted in a sandboxed iframe behind an error boundary.

Running a 27B model on ZeroGPU

Gemma 3 27B is ~55 GB in bfloat16, which does not fit ZeroGPU's default 48 GB slice, so generation requests size="xlarge" — a full RTX Pro 6000 Blackwell at 96 GB. That draws 2× daily quota per call, so signed-out visitors get roughly one generation per day.

Two constraints shape the loading code:

• The base is loaded on CPU and moved with .to("cuda") once. spaces patches module-level CUDA placement and replays it when a GPU attaches; a direct safetensors CUDA load is not patched and fails with No CUDA GPUs are available. The same applies to PEFT, hence torch_device="cpu".
• The adapter is not merged, and the base is not quantised. merge_and_unload() would rebuild every layer's q/v projections on CPU while a ~55 GB copy is already resident. 4-bit would fit the smaller slice, but bitsandbytes quantises inside from_pretrained and needs a real GPU to do it — which does not exist at module scope on ZeroGPU. At rank 4 on two projections, leaving the adapter live costs a pair of tiny matmuls per layer.

BASE_MODEL_ID, ADAPTER_ID, MODEL_LABEL and GPU_SIZE are Space variables, so dropping back to the 4B pair is a settings change rather than a code change.

The preview sandbox

The training data leans heavily on a specific stack — across one shard, 187/187 responses import framer-motion, 183 import lucide-react — so the preview provides them rather than letting those identifiers come back undefined:

Import	Resolved to
react, react-dom/client	React 18 UMD
framer-motion, motion/react	framer-motion 11 UMD (Motion)
lucide-react	icons built on demand from the vanilla lucide UMD (1861 icons)
Tailwind classes	Tailwind Play CDN 3.4 (arbitrary values like bg-[#fffef8] included)
anything else	a passthrough component that renders its children, so layout survives

Code is compiled in-browser by Babel standalone with the typescript (TSX) and react presets, so TypeScript annotations are handled. Compile errors and render crashes surface in a red banner instead of a blank frame.

Only allow-scripts is granted to the iframe, so the generated code can't reach the parent page.

Notes and limits

• The adapter was trained for 26 steps at batch size 1 — treat output as a scaffold to edit, not finished code.
• The first request after the Space wakes has to load ~55 GB of weights before anything generates.
• Long components can exceed the token budget; raise Max new tokens under Generation settings if output cuts off mid-JSX. GPU time is requested in proportion to that budget.
• The Space defaults to unsloth/gemma-3-27b-it, an ungated mirror of the base model, so it runs without a token. Set BASE_MODEL_ID=google/gemma-3-27b-it and add an HF_TOKEN secret to use the canonical gated repo instead.
• Generated code is unverified model output. Review it before running it anywhere that matters.