| --- |
| title: VectorHD |
| emoji: 🖼️ |
| colorFrom: indigo |
| colorTo: blue |
| sdk: docker |
| app_port: 7860 |
| pinned: false |
| license: mit |
| short_description: HD raster-to-SVG vectorization with watertight geometry |
| --- |
| |
| <!-- The YAML block above configures this repo as a Hugging Face Docker Space (sdk/app_port are |
| the load-bearing keys; everything else is presentation). It is inert everywhere else — GitHub |
| renders it as a small table, and pip ignores it when this file is used as the package readme. |
| See docs/deploy/huggingface.md for the deployment runbook. --> |
| |
| # VectorHD |
| |
| **Tier-2 HD-mode server-side image vectorization.** A segmentation-driven, topology-aware, |
| sub-pixel-accurate raster→SVG pipeline in the spirit of Vectorizer.AI / Vector Magic, now with |
| learned perception, analytic geometry (lines / arcs / ellipses / whole primitives), symmetry |
| detection, and gradient fills — all optional and all preserving the watertight guarantee. |
| |
| This is the premium backend for a separate client-side web app (Nuxt) that does instant, |
| free, fully-private in-browser vectorization with VTracer/imagetracerjs. **VectorHD is the |
| "HD mode":** slower, server-side, and visibly cleaner on flat-color logos and illustrations — |
| fewer nodes, smoother curves, and **exact region boundaries with zero gaps or overlaps** |
| between adjacent shapes. |
| |
| The whole pipeline runs **CPU-only** with no optional extras; SAM 2 and DiffVG are isolated |
| add-ons that improve `hd` mode when present and degrade gracefully when not. See |
| [CLAUDE.md](CLAUDE.md) for the module map and the invariants that are enforced as tests. |
| |
| ## Pipeline |
| |
| ``` |
| preprocess → perceive → segment → topology → subpixel → fit → refine (opt) |
| → model-select → primitives → symmetry → gradients → export |
| ``` |
| |
| - **preprocess** — decode, EXIF-orient, downscale to ≤ `VECTORHD_MAX_MEGAPIXELS`, alpha. |
| - **perceive** *(optional, Tier-2)* — a 3-head U-Net restores the input and produces |
| boundary/corner maps that guide segmentation, snapping, and corner detection. No-op (Tier-1 |
| quality) when no weights are installed. **`restore` defaults off** pending the full perception |
| training run: the shipped *validation-run* checkpoint degrades clean/JPEG synthetic geometry |
| (measured C-4/C-5), so the net is gated on `restore` and stays available via `restore=true`. |
| Flip the default back after re-benchmarking with v2 weights. |
| - **segment** — partition into a complete label map (`quantize` k-means backend, or `sam2`); |
| adaptive simplification merges faint low-confidence borders. |
| - **topology** — build a **watertight planar boundary graph**: every boundary stored once and |
| shared by the two regions it separates. This is why the output has no gaps. |
| - **subpixel** — snap boundary points to the anti-aliasing intensity midpoint. |
| - **fit** — corner detection + Schneider cubic-Bézier fitting per shared edge. |
| - **refine** — optimize shared control points + colors against the (perception-restored) input. |
| Default engine is the **analytic MAP optimizer** (`refine_engine="analytic"`, numpy/scipy only, |
| no optional extras); DiffVG (`"diffvg"`, **deprecated** — see below) and `"none"` are also |
| selectable. Watertightness survives optimization by construction (shared graph). |
| - **model-select** *(Tier-2)* — replace each edge's cubics with the simplest analytic segment |
| (line / circular arc / elliptical arc / cubic) that fits within tolerance; arc-aware |
| watertightness holds (a shared edge reverses segment-by-segment). |
| - **primitives** *(Tier-2)* — detect whole-region circle/ellipse/rounded-rect and **project the |
| graph** onto them (junction nodes move onto the primitive; neighbours inherit — watertight). |
| - **symmetry** *(Tier-2)* — detect per-region mirror axes + rotational order (reported in stats; |
| enforcement is off by default). |
| - **gradients** *(Tier-2)* — reconstruct 2-stop linear/radial fills where a region's colour |
| varies smoothly; solid regions stay solid. |
| - **export** — clean SVG + stats over a small authenticated FastAPI API. Coordinates carry |
| `precision` decimals (default **3**; arc radii and rotation get 2 more, being per-arc scalars |
| shared with nothing), and every emitted arc is guaranteed to be *satisfiable* — its radii can |
| actually span its chord — so no renderer has to silently scale them. Both properties are |
| destroyed by re-rounding the output; see `integration/INTEGRATION.md`. |
|
|
| The Tier-2 structure/appearance passes run **after** refinement on the refined geometry, are each |
| individually toggleable, and never regress the watertight guarantee. `curve_types:["cubic"]` with |
| the detectors off reproduces the exact Tier-1 output. |
|
|
| ## Method & references |
|
|
| The **refine** stage is a classical **analytic MAP optimizer** (`src/vectorhd/analytic/`, default |
| `refine_engine="analytic"`): it inverts an exact, differentiable generative model of anti-aliased |
| rasterization to find the vector parameters whose rasterization best explains the pixels, under |
| bezigon shape priors, optimized over the shared planar graph so watertightness survives by |
| construction. Method and calibration are documented in [CLAUDE.md](CLAUDE.md#analytic-optimizer) and |
| the retrofit report at [docs/reports/retrofit-completion.md](docs/reports/retrofit-completion.md). |
|
|
| **The refine path is now fully classical** — no machine learning and no GPU: numpy/scipy only, zero |
| optional dependencies, deterministic. The optional **perception** network is an *upstream* |
| enhancement only (input restoration + boundary/corner guidance); it is gated behind `restore` |
| (default off) pending a full training run, and the pipeline runs end-to-end without it. |
|
|
| The method is published, and no commercial binary was reverse-engineered: |
|
|
| - **Diebel, J.** *Bayesian Image Vectorization: The Probabilistic Inversion of Vector Image |
| Rasterization.* PhD thesis, Stanford University, 2008 — the generative rasterization-inversion |
| formulation (the Vector Magic algorithm). |
| - **Yang, M., Chao, H., Zhang, C., Guo, J., Yuan, L., & Sun, J.** *Effective Clipart Image |
| Vectorization Through Direct Optimization of Bezigons.* IEEE TVCG, 2016. arXiv:1602.01913 — the |
| direct-bezigon extension whose energy design (the four priors + data term) this engine implements. |
|
|
| ## Quick start (CPU, no optional extras) |
|
|
| ```bash |
| uv sync # install core + dev deps (CPU-only, always works) |
| uv run pytest # tests must be green |
| uv run uvicorn vectorhd.api.app:app --reload # serve on :8000 |
| curl localhost:8000/healthz |
| ``` |
|
|
| Plain pip also works: |
|
|
| ```bash |
| python3.11 -m venv .venv && . .venv/bin/activate |
| pip install -e ".[dev]" |
| pytest |
| ``` |
|
|
| ## Optional extras |
|
|
| Both extras are **fully isolated** — the service works completely without them (fast mode is |
| CPU-only and needs neither). When absent, `hd` mode degrades to the quantize backend and skips |
| refinement, reporting the degradation in the response `warnings`. |
|
|
| ### `[sam]` — SAM 2 segmentation |
|
|
| SAM 2 is distributed from GitHub (the PyPI `sam2` package is an **unofficial third-party |
| upload** — do not use it): |
|
|
| ```bash |
| git clone https://github.com/facebookresearch/sam2.git |
| cd sam2 && SAM2_BUILD_CUDA=0 pip install -e . # add --no-build-isolation if needed |
| ``` |
|
|
| Then fetch the smallest checkpoint: |
|
|
| ```bash |
| uv run python scripts/download_weights.py # sam2.1_hiera_tiny by default |
| ``` |
|
|
| ### `vectorhd-core` — the Rust refine kernel (optional, experimental) |
|
|
| The hot kernel of the analytic optimizer (exact coverage rasterizer + boundary-integral gradient) |
| also exists as a Rust crate under `rust/vectorhd-core/`. **`refine_backend` defaults to `"auto"`: |
| the compiled extension when it is importable, the pure-Python kernel otherwise, silently.** The |
| default moved from `"python"` in port phase P1a, on measured evidence (`docs/reports/rust-port-p1a.md`) |
| — 2.10× on the aggregate benchmark, an 87/87 regression matrix, and kernel equivalence five orders |
| inside its gates. Use `"python"` to pin the reference kernel or `"rust"` to require the extension |
| (which warns if it is missing). |
| |
| It is optional in the strongest sense: `vectorhd` probes for the extension once |
| (`vectorhd/analytic/backend.py`) and uses the pure-Python kernel whenever it is absent, so neither |
| the package nor its test suite ever acquires a Rust dependency. Asking for `"rust"` without it |
| installed is not an error — the request is served by Python with a `rust_backend_unavailable` |
| warning. |
| |
| The kernel covers **flat and quadratic (C-5) per-region colour**, so gradient-heavy images benefit |
| too. Quadratic support is *capability-probed*, not assumed: an extension built before that landed |
| keeps the per-window Python fallback for those windows, because it would otherwise answer a |
| quadratic colour model with a flat-colour render — a wrong number rather than an error. |
| |
| ```bash |
| cd rust/vectorhd-core # run from HERE, not the repo root |
| VIRTUAL_ENV=../../.venv ../../.venv/bin/maturin develop --release |
| python -c "import vectorhd_core; print(vectorhd_core.kernel_info())" |
| ``` |
| |
| The working directory matters: cargo resolves `.cargo/config.toml` relative to the cwd, and |
| `rust/.cargo/config.toml` supplies the macOS linker flags a pyo3 `extension-module` build requires. |
| See `rust/vectorhd-core/README.md` for the full story, the three build targets (native / wasm / |
| Python) and the two-tier equivalence discipline; `docs/reports/rust-spike.md` and |
| `docs/reports/rust-port-p0.md` for the evidence. |
| |
| ### `[refine]` — DiffVG refinement (deprecated) |
| |
| > **Deprecated (C-4).** The default `refine_engine="analytic"` is a pure numpy/scipy MAP optimizer |
| > that needs **no optional build**, is roughly **15–130× faster** than DiffVG (logo fixtures to |
| > primitives; DiffVG runs its shipped ~90 s budget), and matches or beats it on PSNR and geometric |
| > error at equal-or-lower node counts (see the retrofit report). |
| > DiffVG remains **selectable** (`refine_engine="diffvg"`) for comparison but is **not** part of the |
| > recommended install; you only need the build below if you specifically want to reproduce it. |
| |
| DiffVG has **no pip package** and `pip install git+…` fails (it doesn't fetch submodules). |
| Build from source (needs CMake + Xcode CLT / a C++ toolchain): |
| |
| DiffVG is an old, largely-unmaintained C++/CMake project, so the naive `setup.py install` |
| fails against a modern toolchain (Python 3.11 + recent clang + CMake 4). The following recipe |
| **builds cleanly** (verified on macOS Apple Silicon, Python 3.11, CMake 4.4, CPU-only): |
| |
| ```bash |
| git clone --recursive https://github.com/BachiLi/diffvg.git |
| cd diffvg && git submodule update --init --recursive |
| |
| # 1) The vendored pybind11 predates Python 3.11's opaque PyFrameObject — upgrade it: |
| cd pybind11 && git fetch --tags --depth 1 origin v2.13.6 && git checkout v2.13.6 && cd .. |
| |
| # 2) The vendored thrust uses `_VSTD`, a libc++ macro removed in modern clang — define it back. |
| # CMake 4 also needs a minimum-policy shim for this old project. Build via setup.py (its |
| # pyproject.toml declares a poetry backend that skips the CMake build, so bypass PEP 517): |
| DIFFVG_CUDA=0 CMAKE_POLICY_VERSION_MINIMUM=3.5 CXXFLAGS="-D_VSTD=std" python setup.py install |
| |
| # 3) pydiffvg's pure-Python runtime deps: |
| pip install svgpathtools svgwrite cssutils matplotlib |
| ``` |
| |
| If it still won't build, the service ships fully without refinement — `refine_available()` |
| returns False and `hd` requests degrade with a `refine_unavailable` warning. Not a failure. |
|
|
| ## Privacy |
|
|
| Images are processed **entirely in memory / ephemeral temp files**; nothing is persisted and |
| request bodies are never logged. The client app's HD-mode disclosure copy depends on this. |
|
|
| ## API |
|
|
| Bearer-token auth: send `Authorization: Bearer <VECTORHD_API_TOKEN>`. If the token env var is |
| unset the service runs **open** (dev convenience) and logs a loud warning. |
|
|
| - `POST /v1/vectorize` — `multipart/form-data`: `image` file + optional `options` JSON field. |
| Returns `{ svg, stats, warnings }`. `stats` includes `width, height, scale, regions, paths, |
| nodes, svg_bytes`, per-stage `timings_ms`, and the Tier-2 summary `primitives`, `symmetries`, |
| `gradient_regions`, `edge_confidence_histogram`. Errors: `400` bad/oversized file, `401` auth, |
| `422` bad options, `503` pool saturated, `504` job timeout, `500` internal/worker crash. |
| **The job runs in a pre-warmed worker process under a hard, SIGKILL-enforced per-job deadline** |
| (`VECTORHD_JOB_TIMEOUT_S`) — the only way to actually stop an unbounded native stage; a mere |
| request-thread timeout cannot. Structured error bodies: |
| - `504` — `{"error":"job_timeout","stage":<exceeded>,"last_completed":<prior>,"elapsed_ms":<int>}` |
| - `500` (worker died mid-job) — `{"error":"worker_crash","stage":<last-stage>}` |
| - `503` (all workers busy) — `{"error":"server_busy"}` |
|
|
| Tier-2 options are documented in |
| [integration/INTEGRATION.md](integration/INTEGRATION.md) and reflected in `GET /v1/info`. |
| - `POST /v1/jobs` → **`202 { job_id, state, poll_after_ms }`** and `GET /v1/jobs/{job_id}` → |
| `200 { job_id, state, svg?, stats?, warnings, error? }` — the **same pipeline without holding the |
| connection**. `state` is `pending | running | done | error`; `svg`/`stats` appear exactly when |
| `state == "done"` and match `/v1/vectorize` field-for-field, and a failure arrives as `error` |
| carrying the same `code`/`status_code` the synchronous route would have returned. |
| |
| Use this whenever a proxy or gateway caps request duration. A dense 500 px badge takes ~38 s |
| end-to-end, but its **longest single HTTP call is 3 ms** — so a 25 s ceiling never applies. |
| Submission still validates synchronously (bad options → `422`, empty upload → `400`), so only |
| pipeline failures are deferred into the job. Records are held **in memory only**, bounded by TTL |
| and count, and never written to disk — which is why this lives in the service rather than in a |
| platform blob store. It is per-process, so run **one** uvicorn worker (as the container does). |
| - `GET /healthz` — `{ ok, device, version, build, engine, pool, sam_available, refine_available }`. |
| - **`build`** — `{ sha, ref, built_at }`: which **source commit** this container was built from, |
| e.g. `{"88d429f…", "main", "2026-07-28T15:20:00Z"}`. `version` cannot answer that — a service |
| can serve code well past its own version string, and this one did (the live Space reported |
| `0.2.0` while provably running post-0.2.0 code, so identifying the running commit meant |
| fetching source files off the Space mirror and diffing trees). Read **verbatim** from a stamp |
| the deploy flow writes into the pushed tree and **never re-derived at runtime**: an unstamped |
| build reports `null` rather than guessing from a git checkout or an env var, because a derived |
| value would describe the machine answering the request, not the image. This is the service's |
| counterpart to the web app's `/version.txt`. |
| - **`engine`** — `{ refine_engine_default, backend, level, core_version }`: the refine path a |
| request would actually take, e.g. `{"analytic", "rust", "handle", "0.1.0"}`. Derived by calling |
| the dispatcher's own `resolve_backend`/`resolve_level`, so it cannot disagree with what a job |
| gets. **This is the field to read when asking what is serving requests** — `refine_backend` in |
| a job's stats confirms it after the fact, but this answers it without running one. |
| - `pool` — `{ size, alive, busy, restarts }` (a growing `restarts` means jobs are timing out or |
| crashing). |
| - `sam_available` / `refine_available` — **legacy**, kept for backward compatibility. They report |
| *importability of the optional extras*, and `refine_available` tracks the retired DiffVG extra, |
| so it reads `false` on a perfectly healthy deployment. It has misled a live reader into thinking |
| refinement was unavailable; consult `engine` instead. Removal is a future major version. |
| - `GET /v1/info` — limits + option defaults/ranges (the web app builds its UI from this), plus the |
| same `build` block as `/healthz` from the same reader, so either response identifies the build. |
| |
| See [integration/INTEGRATION.md](integration/INTEGRATION.md) for the request/response contract |
| and a working Netlify proxy. |
|
|
| ## Configuration |
|
|
| All settings are environment variables with the `VECTORHD_` prefix. |
|
|
| | variable | default | what it does | |
| |---|---|---| |
| | `API_TOKEN` | unset | Bearer token for every protected route. **Unset = the API serves anyone.** | |
| | `REQUIRE_TOKEN` | `false` | Refuse to *start* when `API_TOKEN` is unset. **Set this in any public deployment** — a warning in a hosted log is easy to miss, a container that will not boot is not. | |
| | `DEVICE` | `auto` | `auto\|cpu\|cuda\|mps`. The default path is CPU-only. | |
| | `MAX_MEGAPIXELS` | `3.0` | Working image is downscaled to this. | |
| | `MAX_FILE_MB` | `10` | Upload size cap. | |
| | `HARD_DECODE_MEGAPIXELS` | `24` | Reject images decoding above this outright. | |
| | `REQUEST_TIMEOUT_S` | `120` | Wall-clock budget for one request. | |
| | `WORKER_POOL_SIZE` | `2` | Pre-warmed worker processes; also the concurrency bound. ≈ cores. | |
| | `JOB_TIMEOUT_S` | `REQUEST_TIMEOUT_S − MARGIN` | Hard per-job **SIGKILL** deadline. Refine's own 30 s best-so-far budget runs *inside* the job. | |
| | `JOB_TIMEOUT_MARGIN_S` | `5` | Subtracted from `REQUEST_TIMEOUT_S` when `JOB_TIMEOUT_S` is unset. | |
| | `ALLOW_TEST_HOOKS` | `false` | Master gate for test-only fault injection. **Never true in production.** | |
| | `WEIGHTS_DIR` | `weights` | Where SAM 2 / perception checkpoints are looked up. | |
| | `SAM_MODEL_SIZE` / `SAM_POINTS_PER_SIDE` | `tiny` / `16` | SAM 2 backend tuning (optional extra). | |
|
|
| Worker-pool rationale: |
| [docs/decisions/2026-07-22-stage-bounding-isolation.md](docs/decisions/2026-07-22-stage-bounding-isolation.md). |
| Thread caps (`OMP_NUM_THREADS` and friends) are set in the Dockerfile and inherited by the spawned |
| workers; cap them to roughly `cores / WORKER_POOL_SIZE` so one job does not fight itself. |
|
|
| ## Deployment |
|
|
| **One CPU-only image** ([Dockerfile](Dockerfile)). There is no GPU target and that is a |
| conclusion, not an omission: the retrofit replaced DiffVG — the only CUDA-relevant path — with the |
| numpy/scipy analytic optimizer, the Rust kernel accelerates it on CPU, and the perception net (the |
| last torch consumer) is gated off by default because every head measurably harmed geometry. |
|
|
| ```bash |
| docker build -t vectorhd . |
| docker run -p 7860:7860 -e VECTORHD_API_TOKEN=secret -e VECTORHD_REQUIRE_TOKEN=1 vectorhd |
| ``` |
|
|
| - **Multi-stage:** stage 1 compiles the optional `vectorhd_core` Rust extension with maturin; |
| stage 2 is a slim runtime with no compilers. Verify it loaded by checking that a `hd` response's |
| `stats.refine_backend` reads `rust` (it falls back to the pure-Python kernel silently otherwise). |
| - Listens on **7860** (Hugging Face Spaces convention), overridable with `$PORT`. Runs as |
| **non-root UID 1000**, with a `HEALTHCHECK` on `/healthz`. Nothing is written to disk. |
| - **Measured** (Apple M-series, native arm64 build, 2 BLAS threads, `WORKER_POOL_SIZE=1`): |
| image **2.09 GB**, cold start to healthy **~9 s**, `fast` on a 192 px icon **~2.9 s**, |
| `hd` on the same **~4.3 s**. A dense 93-region badge rides the 30 s refine budget and returns |
| best-so-far with a `refine_timeout` warning — by design, not a failure. |
| - **Live on the Hugging Face free tier** (2026-07-27, same image, same 192 px icon): `hd` |
| **14.95 s** total / 13.27 s refine, versus 4.25 s locally — **3.5× slower on a shared vCPU**, at |
| **identical** output (8 regions, 9 nodes, `refine_backend: rust`). The free tier costs latency, |
| not quality. |
| - **Job isolation is per-process, not per-thread.** Each `/v1/vectorize` job runs in one of |
| `WORKER_POOL_SIZE` pre-warmed worker processes; a job exceeding `JOB_TIMEOUT_S` is SIGKILLed and |
| its worker replenished. This makes the request timeout genuinely enforceable — before it, an |
| unbounded native stage (a pathological input pinned a worker for 66 min in corpus-v1). A |
| container OOM/segfault-class worker death is contained (clean 500, healthy server) rather than |
| taking the service down. Pool overhead vs in-process is IPC-only (single-digit ms at p50); see |
| [docs/reports/stage-bounding.md](docs/reports/stage-bounding.md). |
| - **SAM 2 is not installed** in the image, so `hd` emits a `sam_unavailable` warning and segments |
| with the quantize backend — the documented graceful degradation, and the configuration every |
| published quality number was measured under. |
|
|
| **Hugging Face Spaces** is the reference deployment: this repo is a ready Docker Space (the |
| README's YAML front-matter configures it). Step-by-step runbook, including secrets and live |
| verification: **[docs/deploy/huggingface.md](docs/deploy/huggingface.md)**. Free-tier expectations |
| are honest ones — 2 shared vCPU, the Space **sleeps when idle** (first request after a nap pays a |
| ~30–60 s cold start), and `hd` will be slower than the numbers above. `fast` mode stays snappy. |
|
|
| ## Tier-2 perception network (optional) |
|
|
| The perception net (`restore` option) is a 3-head U-Net (~1.95M params: restore / boundary / |
| corner). It ships **unweighted** — the whole pipeline runs at Tier-1 quality with no weights and |
| a `perception_unavailable` warning. To train and install it: |
|
|
| ```bash |
| # 1) Generate a synthetic dataset (procedural shapes + degradations; sharded .npz). |
| uv run python scripts/gen_dataset.py --n 40000 --out data/train --shard-size 512 |
| uv run python scripts/gen_dataset.py --n 4000 --out data/val --shard-size 512 |
| |
| # 2) Train (yaml-driven). Full run configs/perception.yaml; --smoke for a quick loop check. |
| uv run python scripts/train.py --config configs/perception.yaml |
| # Rough duration: a full run is O(hours) on a single modern GPU; the smoke config is minutes on CPU. |
| |
| # 3) No copy needed: the trainer's `export:` key writes the TorchScript weight straight to |
| # models/perception_v1.pt — exactly where the service loads it (the `perception_model` setting, |
| # VECTORHD_PERCEPTION_MODEL). (scripts/download_weights.py fetches the SAM2 checkpoint only; |
| # perception weights come from this training export, not a download.) |
| # Acceptance: uv run python scripts/eval_perception.py (boundary F1 vs Canny, corner recall vs |
| # Harris, restore PSNR). A validation checkpoint already beats the classical baselines; full |
| # training is left to the operator (not run in-repo). |
| ``` |
|
|
| **Data licensing:** all training and benchmark data is **procedurally generated in this repo** |
| (CC0, self-owned). No third-party datasets are used, and the network is **never** trained on the |
| output of Vectorizer.AI, Vector Magic, or any other vectorizer. See [DATA_LICENSES.md](DATA_LICENSES.md). |
|
|
| ## Tier-3 roadmap (not built here) |
|
|
| Multi-stop gradients (Tier-2 ships 2-stop) · a learned segmentation head (merging colour bands |
| under one gradient/primitive before fitting) · global / inter-region symmetry · regular-polygon & |
| star primitives (skipped as flaky) · DXF/EPS/PDF export · async job queues · batch endpoints · |
| default-on symmetry enforcement once correspondence is robust on complex art. |
|
|
