--- title: VectorHD emoji: πŸ–ΌοΈ colorFrom: indigo colorTo: blue sdk: docker app_port: 7860 pinned: false license: mit short_description: HD raster-to-SVG vectorization with watertight geometry --- # VectorHD **Tier-2 HD-mode server-side image vectorization.** A segmentation-driven, topology-aware, sub-pixel-accurate rasterβ†’SVG pipeline in the spirit of Vectorizer.AI / Vector Magic, now with learned perception, analytic geometry (lines / arcs / ellipses / whole primitives), symmetry detection, and gradient fills β€” all optional and all preserving the watertight guarantee. This is the premium backend for a separate client-side web app (Nuxt) that does instant, free, fully-private in-browser vectorization with VTracer/imagetracerjs. **VectorHD is the "HD mode":** slower, server-side, and visibly cleaner on flat-color logos and illustrations β€” fewer nodes, smoother curves, and **exact region boundaries with zero gaps or overlaps** between adjacent shapes. The whole pipeline runs **CPU-only** with no optional extras; SAM 2 and DiffVG are isolated add-ons that improve `hd` mode when present and degrade gracefully when not. See [CLAUDE.md](CLAUDE.md) for the module map and the invariants that are enforced as tests. ## Pipeline ``` preprocess β†’ perceive β†’ segment β†’ topology β†’ subpixel β†’ fit β†’ refine (opt) β†’ model-select β†’ primitives β†’ symmetry β†’ gradients β†’ export ``` - **preprocess** β€” decode, EXIF-orient, downscale to ≀ `VECTORHD_MAX_MEGAPIXELS`, alpha. - **perceive** *(optional, Tier-2)* β€” a 3-head U-Net restores the input and produces boundary/corner maps that guide segmentation, snapping, and corner detection. No-op (Tier-1 quality) when no weights are installed. **`restore` defaults off** pending the full perception training run: the shipped *validation-run* checkpoint degrades clean/JPEG synthetic geometry (measured C-4/C-5), so the net is gated on `restore` and stays available via `restore=true`. Flip the default back after re-benchmarking with v2 weights. - **segment** β€” partition into a complete label map (`quantize` k-means backend, or `sam2`); adaptive simplification merges faint low-confidence borders. - **topology** β€” build a **watertight planar boundary graph**: every boundary stored once and shared by the two regions it separates. This is why the output has no gaps. - **subpixel** β€” snap boundary points to the anti-aliasing intensity midpoint. - **fit** β€” corner detection + Schneider cubic-BΓ©zier fitting per shared edge. - **refine** β€” optimize shared control points + colors against the (perception-restored) input. Default engine is the **analytic MAP optimizer** (`refine_engine="analytic"`, numpy/scipy only, no optional extras); DiffVG (`"diffvg"`, **deprecated** β€” see below) and `"none"` are also selectable. Watertightness survives optimization by construction (shared graph). - **model-select** *(Tier-2)* β€” replace each edge's cubics with the simplest analytic segment (line / circular arc / elliptical arc / cubic) that fits within tolerance; arc-aware watertightness holds (a shared edge reverses segment-by-segment). - **primitives** *(Tier-2)* β€” detect whole-region circle/ellipse/rounded-rect and **project the graph** onto them (junction nodes move onto the primitive; neighbours inherit β€” watertight). - **symmetry** *(Tier-2)* β€” detect per-region mirror axes + rotational order (reported in stats; enforcement is off by default). - **gradients** *(Tier-2)* β€” reconstruct 2-stop linear/radial fills where a region's colour varies smoothly; solid regions stay solid. - **export** β€” clean SVG + stats over a small authenticated FastAPI API. Coordinates carry `precision` decimals (default **3**; arc radii and rotation get 2 more, being per-arc scalars shared with nothing), and every emitted arc is guaranteed to be *satisfiable* β€” its radii can actually span its chord β€” so no renderer has to silently scale them. Both properties are destroyed by re-rounding the output; see `integration/INTEGRATION.md`. The Tier-2 structure/appearance passes run **after** refinement on the refined geometry, are each individually toggleable, and never regress the watertight guarantee. `curve_types:["cubic"]` with the detectors off reproduces the exact Tier-1 output. ## Method & references The **refine** stage is a classical **analytic MAP optimizer** (`src/vectorhd/analytic/`, default `refine_engine="analytic"`): it inverts an exact, differentiable generative model of anti-aliased rasterization to find the vector parameters whose rasterization best explains the pixels, under bezigon shape priors, optimized over the shared planar graph so watertightness survives by construction. Method and calibration are documented in [CLAUDE.md](CLAUDE.md#analytic-optimizer) and the retrofit report at [docs/reports/retrofit-completion.md](docs/reports/retrofit-completion.md). **The refine path is now fully classical** β€” no machine learning and no GPU: numpy/scipy only, zero optional dependencies, deterministic. The optional **perception** network is an *upstream* enhancement only (input restoration + boundary/corner guidance); it is gated behind `restore` (default off) pending a full training run, and the pipeline runs end-to-end without it. The method is published, and no commercial binary was reverse-engineered: - **Diebel, J.** *Bayesian Image Vectorization: The Probabilistic Inversion of Vector Image Rasterization.* PhD thesis, Stanford University, 2008 β€” the generative rasterization-inversion formulation (the Vector Magic algorithm). - **Yang, M., Chao, H., Zhang, C., Guo, J., Yuan, L., & Sun, J.** *Effective Clipart Image Vectorization Through Direct Optimization of Bezigons.* IEEE TVCG, 2016. arXiv:1602.01913 β€” the direct-bezigon extension whose energy design (the four priors + data term) this engine implements. ## Quick start (CPU, no optional extras) ```bash uv sync # install core + dev deps (CPU-only, always works) uv run pytest # tests must be green uv run uvicorn vectorhd.api.app:app --reload # serve on :8000 curl localhost:8000/healthz ``` Plain pip also works: ```bash python3.11 -m venv .venv && . .venv/bin/activate pip install -e ".[dev]" pytest ``` ## Optional extras Both extras are **fully isolated** β€” the service works completely without them (fast mode is CPU-only and needs neither). When absent, `hd` mode degrades to the quantize backend and skips refinement, reporting the degradation in the response `warnings`. ### `[sam]` β€” SAM 2 segmentation SAM 2 is distributed from GitHub (the PyPI `sam2` package is an **unofficial third-party upload** β€” do not use it): ```bash git clone https://github.com/facebookresearch/sam2.git cd sam2 && SAM2_BUILD_CUDA=0 pip install -e . # add --no-build-isolation if needed ``` Then fetch the smallest checkpoint: ```bash uv run python scripts/download_weights.py # sam2.1_hiera_tiny by default ``` ### `vectorhd-core` β€” the Rust refine kernel (optional, experimental) The hot kernel of the analytic optimizer (exact coverage rasterizer + boundary-integral gradient) also exists as a Rust crate under `rust/vectorhd-core/`. **`refine_backend` defaults to `"auto"`: the compiled extension when it is importable, the pure-Python kernel otherwise, silently.** The default moved from `"python"` in port phase P1a, on measured evidence (`docs/reports/rust-port-p1a.md`) β€” 2.10Γ— on the aggregate benchmark, an 87/87 regression matrix, and kernel equivalence five orders inside its gates. Use `"python"` to pin the reference kernel or `"rust"` to require the extension (which warns if it is missing). It is optional in the strongest sense: `vectorhd` probes for the extension once (`vectorhd/analytic/backend.py`) and uses the pure-Python kernel whenever it is absent, so neither the package nor its test suite ever acquires a Rust dependency. Asking for `"rust"` without it installed is not an error β€” the request is served by Python with a `rust_backend_unavailable` warning. The kernel covers **flat and quadratic (C-5) per-region colour**, so gradient-heavy images benefit too. Quadratic support is *capability-probed*, not assumed: an extension built before that landed keeps the per-window Python fallback for those windows, because it would otherwise answer a quadratic colour model with a flat-colour render β€” a wrong number rather than an error. ```bash cd rust/vectorhd-core # run from HERE, not the repo root VIRTUAL_ENV=../../.venv ../../.venv/bin/maturin develop --release python -c "import vectorhd_core; print(vectorhd_core.kernel_info())" ``` The working directory matters: cargo resolves `.cargo/config.toml` relative to the cwd, and `rust/.cargo/config.toml` supplies the macOS linker flags a pyo3 `extension-module` build requires. See `rust/vectorhd-core/README.md` for the full story, the three build targets (native / wasm / Python) and the two-tier equivalence discipline; `docs/reports/rust-spike.md` and `docs/reports/rust-port-p0.md` for the evidence. ### `[refine]` β€” DiffVG refinement (deprecated) > **Deprecated (C-4).** The default `refine_engine="analytic"` is a pure numpy/scipy MAP optimizer > that needs **no optional build**, is roughly **15–130Γ— faster** than DiffVG (logo fixtures to > primitives; DiffVG runs its shipped ~90 s budget), and matches or beats it on PSNR and geometric > error at equal-or-lower node counts (see the retrofit report). > DiffVG remains **selectable** (`refine_engine="diffvg"`) for comparison but is **not** part of the > recommended install; you only need the build below if you specifically want to reproduce it. DiffVG has **no pip package** and `pip install git+…` fails (it doesn't fetch submodules). Build from source (needs CMake + Xcode CLT / a C++ toolchain): DiffVG is an old, largely-unmaintained C++/CMake project, so the naive `setup.py install` fails against a modern toolchain (Python 3.11 + recent clang + CMake 4). The following recipe **builds cleanly** (verified on macOS Apple Silicon, Python 3.11, CMake 4.4, CPU-only): ```bash git clone --recursive https://github.com/BachiLi/diffvg.git cd diffvg && git submodule update --init --recursive # 1) The vendored pybind11 predates Python 3.11's opaque PyFrameObject β€” upgrade it: cd pybind11 && git fetch --tags --depth 1 origin v2.13.6 && git checkout v2.13.6 && cd .. # 2) The vendored thrust uses `_VSTD`, a libc++ macro removed in modern clang β€” define it back. # CMake 4 also needs a minimum-policy shim for this old project. Build via setup.py (its # pyproject.toml declares a poetry backend that skips the CMake build, so bypass PEP 517): DIFFVG_CUDA=0 CMAKE_POLICY_VERSION_MINIMUM=3.5 CXXFLAGS="-D_VSTD=std" python setup.py install # 3) pydiffvg's pure-Python runtime deps: pip install svgpathtools svgwrite cssutils matplotlib ``` If it still won't build, the service ships fully without refinement β€” `refine_available()` returns False and `hd` requests degrade with a `refine_unavailable` warning. Not a failure. ## Privacy Images are processed **entirely in memory / ephemeral temp files**; nothing is persisted and request bodies are never logged. The client app's HD-mode disclosure copy depends on this. ## API Bearer-token auth: send `Authorization: Bearer `. If the token env var is unset the service runs **open** (dev convenience) and logs a loud warning. - `POST /v1/vectorize` β€” `multipart/form-data`: `image` file + optional `options` JSON field. Returns `{ svg, stats, warnings }`. `stats` includes `width, height, scale, regions, paths, nodes, svg_bytes`, per-stage `timings_ms`, and the Tier-2 summary `primitives`, `symmetries`, `gradient_regions`, `edge_confidence_histogram`. Errors: `400` bad/oversized file, `401` auth, `422` bad options, `503` pool saturated, `504` job timeout, `500` internal/worker crash. **The job runs in a pre-warmed worker process under a hard, SIGKILL-enforced per-job deadline** (`VECTORHD_JOB_TIMEOUT_S`) β€” the only way to actually stop an unbounded native stage; a mere request-thread timeout cannot. Structured error bodies: - `504` β€” `{"error":"job_timeout","stage":,"last_completed":,"elapsed_ms":}` - `500` (worker died mid-job) β€” `{"error":"worker_crash","stage":}` - `503` (all workers busy) β€” `{"error":"server_busy"}` Tier-2 options are documented in [integration/INTEGRATION.md](integration/INTEGRATION.md) and reflected in `GET /v1/info`. - `POST /v1/jobs` β†’ **`202 { job_id, state, poll_after_ms }`** and `GET /v1/jobs/{job_id}` β†’ `200 { job_id, state, svg?, stats?, warnings, error? }` β€” the **same pipeline without holding the connection**. `state` is `pending | running | done | error`; `svg`/`stats` appear exactly when `state == "done"` and match `/v1/vectorize` field-for-field, and a failure arrives as `error` carrying the same `code`/`status_code` the synchronous route would have returned. Use this whenever a proxy or gateway caps request duration. A dense 500 px badge takes ~38 s end-to-end, but its **longest single HTTP call is 3 ms** β€” so a 25 s ceiling never applies. Submission still validates synchronously (bad options β†’ `422`, empty upload β†’ `400`), so only pipeline failures are deferred into the job. Records are held **in memory only**, bounded by TTL and count, and never written to disk β€” which is why this lives in the service rather than in a platform blob store. It is per-process, so run **one** uvicorn worker (as the container does). - `GET /healthz` β€” `{ ok, device, version, build, engine, pool, sam_available, refine_available }`. - **`build`** β€” `{ sha, ref, built_at }`: which **source commit** this container was built from, e.g. `{"88d429f…", "main", "2026-07-28T15:20:00Z"}`. `version` cannot answer that β€” a service can serve code well past its own version string, and this one did (the live Space reported `0.2.0` while provably running post-0.2.0 code, so identifying the running commit meant fetching source files off the Space mirror and diffing trees). Read **verbatim** from a stamp the deploy flow writes into the pushed tree and **never re-derived at runtime**: an unstamped build reports `null` rather than guessing from a git checkout or an env var, because a derived value would describe the machine answering the request, not the image. This is the service's counterpart to the web app's `/version.txt`. - **`engine`** β€” `{ refine_engine_default, backend, level, core_version }`: the refine path a request would actually take, e.g. `{"analytic", "rust", "handle", "0.1.0"}`. Derived by calling the dispatcher's own `resolve_backend`/`resolve_level`, so it cannot disagree with what a job gets. **This is the field to read when asking what is serving requests** β€” `refine_backend` in a job's stats confirms it after the fact, but this answers it without running one. - `pool` β€” `{ size, alive, busy, restarts }` (a growing `restarts` means jobs are timing out or crashing). - `sam_available` / `refine_available` β€” **legacy**, kept for backward compatibility. They report *importability of the optional extras*, and `refine_available` tracks the retired DiffVG extra, so it reads `false` on a perfectly healthy deployment. It has misled a live reader into thinking refinement was unavailable; consult `engine` instead. Removal is a future major version. - `GET /v1/info` β€” limits + option defaults/ranges (the web app builds its UI from this), plus the same `build` block as `/healthz` from the same reader, so either response identifies the build. See [integration/INTEGRATION.md](integration/INTEGRATION.md) for the request/response contract and a working Netlify proxy. ## Configuration All settings are environment variables with the `VECTORHD_` prefix. | variable | default | what it does | |---|---|---| | `API_TOKEN` | unset | Bearer token for every protected route. **Unset = the API serves anyone.** | | `REQUIRE_TOKEN` | `false` | Refuse to *start* when `API_TOKEN` is unset. **Set this in any public deployment** β€” a warning in a hosted log is easy to miss, a container that will not boot is not. | | `DEVICE` | `auto` | `auto\|cpu\|cuda\|mps`. The default path is CPU-only. | | `MAX_MEGAPIXELS` | `3.0` | Working image is downscaled to this. | | `MAX_FILE_MB` | `10` | Upload size cap. | | `HARD_DECODE_MEGAPIXELS` | `24` | Reject images decoding above this outright. | | `REQUEST_TIMEOUT_S` | `120` | Wall-clock budget for one request. | | `WORKER_POOL_SIZE` | `2` | Pre-warmed worker processes; also the concurrency bound. β‰ˆ cores. | | `JOB_TIMEOUT_S` | `REQUEST_TIMEOUT_S βˆ’ MARGIN` | Hard per-job **SIGKILL** deadline. Refine's own 30 s best-so-far budget runs *inside* the job. | | `JOB_TIMEOUT_MARGIN_S` | `5` | Subtracted from `REQUEST_TIMEOUT_S` when `JOB_TIMEOUT_S` is unset. | | `ALLOW_TEST_HOOKS` | `false` | Master gate for test-only fault injection. **Never true in production.** | | `WEIGHTS_DIR` | `weights` | Where SAM 2 / perception checkpoints are looked up. | | `SAM_MODEL_SIZE` / `SAM_POINTS_PER_SIDE` | `tiny` / `16` | SAM 2 backend tuning (optional extra). | Worker-pool rationale: [docs/decisions/2026-07-22-stage-bounding-isolation.md](docs/decisions/2026-07-22-stage-bounding-isolation.md). Thread caps (`OMP_NUM_THREADS` and friends) are set in the Dockerfile and inherited by the spawned workers; cap them to roughly `cores / WORKER_POOL_SIZE` so one job does not fight itself. ## Deployment **One CPU-only image** ([Dockerfile](Dockerfile)). There is no GPU target and that is a conclusion, not an omission: the retrofit replaced DiffVG β€” the only CUDA-relevant path β€” with the numpy/scipy analytic optimizer, the Rust kernel accelerates it on CPU, and the perception net (the last torch consumer) is gated off by default because every head measurably harmed geometry. ```bash docker build -t vectorhd . docker run -p 7860:7860 -e VECTORHD_API_TOKEN=secret -e VECTORHD_REQUIRE_TOKEN=1 vectorhd ``` - **Multi-stage:** stage 1 compiles the optional `vectorhd_core` Rust extension with maturin; stage 2 is a slim runtime with no compilers. Verify it loaded by checking that a `hd` response's `stats.refine_backend` reads `rust` (it falls back to the pure-Python kernel silently otherwise). - Listens on **7860** (Hugging Face Spaces convention), overridable with `$PORT`. Runs as **non-root UID 1000**, with a `HEALTHCHECK` on `/healthz`. Nothing is written to disk. - **Measured** (Apple M-series, native arm64 build, 2 BLAS threads, `WORKER_POOL_SIZE=1`): image **2.09 GB**, cold start to healthy **~9 s**, `fast` on a 192 px icon **~2.9 s**, `hd` on the same **~4.3 s**. A dense 93-region badge rides the 30 s refine budget and returns best-so-far with a `refine_timeout` warning β€” by design, not a failure. - **Live on the Hugging Face free tier** (2026-07-27, same image, same 192 px icon): `hd` **14.95 s** total / 13.27 s refine, versus 4.25 s locally β€” **3.5Γ— slower on a shared vCPU**, at **identical** output (8 regions, 9 nodes, `refine_backend: rust`). The free tier costs latency, not quality. - **Job isolation is per-process, not per-thread.** Each `/v1/vectorize` job runs in one of `WORKER_POOL_SIZE` pre-warmed worker processes; a job exceeding `JOB_TIMEOUT_S` is SIGKILLed and its worker replenished. This makes the request timeout genuinely enforceable β€” before it, an unbounded native stage (a pathological input pinned a worker for 66 min in corpus-v1). A container OOM/segfault-class worker death is contained (clean 500, healthy server) rather than taking the service down. Pool overhead vs in-process is IPC-only (single-digit ms at p50); see [docs/reports/stage-bounding.md](docs/reports/stage-bounding.md). - **SAM 2 is not installed** in the image, so `hd` emits a `sam_unavailable` warning and segments with the quantize backend β€” the documented graceful degradation, and the configuration every published quality number was measured under. **Hugging Face Spaces** is the reference deployment: this repo is a ready Docker Space (the README's YAML front-matter configures it). Step-by-step runbook, including secrets and live verification: **[docs/deploy/huggingface.md](docs/deploy/huggingface.md)**. Free-tier expectations are honest ones β€” 2 shared vCPU, the Space **sleeps when idle** (first request after a nap pays a ~30–60 s cold start), and `hd` will be slower than the numbers above. `fast` mode stays snappy. ## Tier-2 perception network (optional) The perception net (`restore` option) is a 3-head U-Net (~1.95M params: restore / boundary / corner). It ships **unweighted** β€” the whole pipeline runs at Tier-1 quality with no weights and a `perception_unavailable` warning. To train and install it: ```bash # 1) Generate a synthetic dataset (procedural shapes + degradations; sharded .npz). uv run python scripts/gen_dataset.py --n 40000 --out data/train --shard-size 512 uv run python scripts/gen_dataset.py --n 4000 --out data/val --shard-size 512 # 2) Train (yaml-driven). Full run configs/perception.yaml; --smoke for a quick loop check. uv run python scripts/train.py --config configs/perception.yaml # Rough duration: a full run is O(hours) on a single modern GPU; the smoke config is minutes on CPU. # 3) No copy needed: the trainer's `export:` key writes the TorchScript weight straight to # models/perception_v1.pt β€” exactly where the service loads it (the `perception_model` setting, # VECTORHD_PERCEPTION_MODEL). (scripts/download_weights.py fetches the SAM2 checkpoint only; # perception weights come from this training export, not a download.) # Acceptance: uv run python scripts/eval_perception.py (boundary F1 vs Canny, corner recall vs # Harris, restore PSNR). A validation checkpoint already beats the classical baselines; full # training is left to the operator (not run in-repo). ``` **Data licensing:** all training and benchmark data is **procedurally generated in this repo** (CC0, self-owned). No third-party datasets are used, and the network is **never** trained on the output of Vectorizer.AI, Vector Magic, or any other vectorizer. See [DATA_LICENSES.md](DATA_LICENSES.md). ## Tier-3 roadmap (not built here) Multi-stop gradients (Tier-2 ships 2-stop) Β· a learned segmentation head (merging colour bands under one gradient/primitive before fitting) Β· global / inter-region symmetry Β· regular-polygon & star primitives (skipped as flaky) Β· DXF/EPS/PDF export Β· async job queues Β· batch endpoints Β· default-on symmetry enforcement once correspondence is robust on complex art.