README.md

metadata

title: VectorHD
emoji: 🖼️
colorFrom: indigo
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
license: mit
short_description: HD raster-to-SVG vectorization with watertight geometry

VectorHD

Tier-2 HD-mode server-side image vectorization. A segmentation-driven, topology-aware, sub-pixel-accurate raster→SVG pipeline in the spirit of Vectorizer.AI / Vector Magic, now with learned perception, analytic geometry (lines / arcs / ellipses / whole primitives), symmetry detection, and gradient fills — all optional and all preserving the watertight guarantee.

This is the premium backend for a separate client-side web app (Nuxt) that does instant, free, fully-private in-browser vectorization with VTracer/imagetracerjs. VectorHD is the "HD mode": slower, server-side, and visibly cleaner on flat-color logos and illustrations — fewer nodes, smoother curves, and exact region boundaries with zero gaps or overlaps between adjacent shapes.

The whole pipeline runs CPU-only with no optional extras; SAM 2 and DiffVG are isolated add-ons that improve hd mode when present and degrade gracefully when not. See CLAUDE.md for the module map and the invariants that are enforced as tests.

Pipeline

preprocess → perceive → segment → topology → subpixel → fit → refine (opt)
           → model-select → primitives → symmetry → gradients → export

• preprocess — decode, EXIF-orient, downscale to ≤ VECTORHD_MAX_MEGAPIXELS, alpha.
• perceive (optional, Tier-2) — a 3-head U-Net restores the input and produces boundary/corner maps that guide segmentation, snapping, and corner detection. No-op (Tier-1 quality) when no weights are installed. restore defaults off pending the full perception training run: the shipped validation-run checkpoint degrades clean/JPEG synthetic geometry (measured C-4/C-5), so the net is gated on restore and stays available via restore=true. Flip the default back after re-benchmarking with v2 weights.
• segment — partition into a complete label map (quantize k-means backend, or sam2); adaptive simplification merges faint low-confidence borders.
• topology — build a watertight planar boundary graph: every boundary stored once and shared by the two regions it separates. This is why the output has no gaps.
• subpixel — snap boundary points to the anti-aliasing intensity midpoint.
• fit — corner detection + Schneider cubic-Bézier fitting per shared edge.
• refine — optimize shared control points + colors against the (perception-restored) input. Default engine is the analytic MAP optimizer (refine_engine="analytic", numpy/scipy only, no optional extras); DiffVG ("diffvg", deprecated — see below) and "none" are also selectable. Watertightness survives optimization by construction (shared graph).
• model-select (Tier-2) — replace each edge's cubics with the simplest analytic segment (line / circular arc / elliptical arc / cubic) that fits within tolerance; arc-aware watertightness holds (a shared edge reverses segment-by-segment).
• primitives (Tier-2) — detect whole-region circle/ellipse/rounded-rect and project the graph onto them (junction nodes move onto the primitive; neighbours inherit — watertight).
• symmetry (Tier-2) — detect per-region mirror axes + rotational order (reported in stats; enforcement is off by default).
• gradients (Tier-2) — reconstruct 2-stop linear/radial fills where a region's colour varies smoothly; solid regions stay solid.
• export — clean SVG + stats over a small authenticated FastAPI API. Coordinates carry precision decimals (default 3; arc radii and rotation get 2 more, being per-arc scalars shared with nothing), and every emitted arc is guaranteed to be satisfiable — its radii can actually span its chord — so no renderer has to silently scale them. Both properties are destroyed by re-rounding the output; see integration/INTEGRATION.md.

The Tier-2 structure/appearance passes run after refinement on the refined geometry, are each individually toggleable, and never regress the watertight guarantee. curve_types:["cubic"] with the detectors off reproduces the exact Tier-1 output.

Method & references

The refine stage is a classical analytic MAP optimizer (src/vectorhd/analytic/, default refine_engine="analytic"): it inverts an exact, differentiable generative model of anti-aliased rasterization to find the vector parameters whose rasterization best explains the pixels, under bezigon shape priors, optimized over the shared planar graph so watertightness survives by construction. Method and calibration are documented in CLAUDE.md and the retrofit report at docs/reports/retrofit-completion.md.

The refine path is now fully classical — no machine learning and no GPU: numpy/scipy only, zero optional dependencies, deterministic. The optional perception network is an upstream enhancement only (input restoration + boundary/corner guidance); it is gated behind restore (default off) pending a full training run, and the pipeline runs end-to-end without it.

The method is published, and no commercial binary was reverse-engineered:

• Diebel, J. Bayesian Image Vectorization: The Probabilistic Inversion of Vector Image Rasterization. PhD thesis, Stanford University, 2008 — the generative rasterization-inversion formulation (the Vector Magic algorithm).
• Yang, M., Chao, H., Zhang, C., Guo, J., Yuan, L., & Sun, J. Effective Clipart Image Vectorization Through Direct Optimization of Bezigons. IEEE TVCG, 2016. arXiv:1602.01913 — the direct-bezigon extension whose energy design (the four priors + data term) this engine implements.

Quick start (CPU, no optional extras)

uv sync                    # install core + dev deps (CPU-only, always works)
uv run pytest              # tests must be green
uv run uvicorn vectorhd.api.app:app --reload   # serve on :8000
curl localhost:8000/healthz

Plain pip also works:

python3.11 -m venv .venv && . .venv/bin/activate
pip install -e ".[dev]"
pytest

Optional extras

Both extras are fully isolated — the service works completely without them (fast mode is CPU-only and needs neither). When absent, hd mode degrades to the quantize backend and skips refinement, reporting the degradation in the response warnings.

[sam] — SAM 2 segmentation

SAM 2 is distributed from GitHub (the PyPI sam2 package is an unofficial third-party upload — do not use it):

git clone https://github.com/facebookresearch/sam2.git
cd sam2 && SAM2_BUILD_CUDA=0 pip install -e .        # add --no-build-isolation if needed

Then fetch the smallest checkpoint:

uv run python scripts/download_weights.py            # sam2.1_hiera_tiny by default

vectorhd-core — the Rust refine kernel (optional, experimental)

The hot kernel of the analytic optimizer (exact coverage rasterizer + boundary-integral gradient) also exists as a Rust crate under rust/vectorhd-core/. refine_backend defaults to "auto": the compiled extension when it is importable, the pure-Python kernel otherwise, silently. The default moved from "python" in port phase P1a, on measured evidence (docs/reports/rust-port-p1a.md) — 2.10× on the aggregate benchmark, an 87/87 regression matrix, and kernel equivalence five orders inside its gates. Use "python" to pin the reference kernel or "rust" to require the extension (which warns if it is missing).

It is optional in the strongest sense: vectorhd probes for the extension once (vectorhd/analytic/backend.py) and uses the pure-Python kernel whenever it is absent, so neither the package nor its test suite ever acquires a Rust dependency. Asking for "rust" without it installed is not an error — the request is served by Python with a rust_backend_unavailable warning.

The kernel covers flat and quadratic (C-5) per-region colour, so gradient-heavy images benefit too. Quadratic support is capability-probed, not assumed: an extension built before that landed keeps the per-window Python fallback for those windows, because it would otherwise answer a quadratic colour model with a flat-colour render — a wrong number rather than an error.

cd rust/vectorhd-core                                   # run from HERE, not the repo root
VIRTUAL_ENV=../../.venv ../../.venv/bin/maturin develop --release
python -c "import vectorhd_core; print(vectorhd_core.kernel_info())"

The working directory matters: cargo resolves .cargo/config.toml relative to the cwd, and rust/.cargo/config.toml supplies the macOS linker flags a pyo3 extension-module build requires. See rust/vectorhd-core/README.md for the full story, the three build targets (native / wasm / Python) and the two-tier equivalence discipline; docs/reports/rust-spike.md and docs/reports/rust-port-p0.md for the evidence.

[refine] — DiffVG refinement (deprecated)

Deprecated (C-4). The default refine_engine="analytic" is a pure numpy/scipy MAP optimizer that needs no optional build, is roughly 15–130× faster than DiffVG (logo fixtures to primitives; DiffVG runs its shipped ~90 s budget), and matches or beats it on PSNR and geometric error at equal-or-lower node counts (see the retrofit report). DiffVG remains selectable (refine_engine="diffvg") for comparison but is not part of the recommended install; you only need the build below if you specifically want to reproduce it.

DiffVG has no pip package and pip install git+… fails (it doesn't fetch submodules). Build from source (needs CMake + Xcode CLT / a C++ toolchain):

DiffVG is an old, largely-unmaintained C++/CMake project, so the naive setup.py install fails against a modern toolchain (Python 3.11 + recent clang + CMake 4). The following recipe builds cleanly (verified on macOS Apple Silicon, Python 3.11, CMake 4.4, CPU-only):

git clone --recursive https://github.com/BachiLi/diffvg.git
cd diffvg && git submodule update --init --recursive

# 1) The vendored pybind11 predates Python 3.11's opaque PyFrameObject — upgrade it:
cd pybind11 && git fetch --tags --depth 1 origin v2.13.6 && git checkout v2.13.6 && cd ..

# 2) The vendored thrust uses `_VSTD`, a libc++ macro removed in modern clang — define it back.
#    CMake 4 also needs a minimum-policy shim for this old project. Build via setup.py (its
#    pyproject.toml declares a poetry backend that skips the CMake build, so bypass PEP 517):
DIFFVG_CUDA=0 CMAKE_POLICY_VERSION_MINIMUM=3.5 CXXFLAGS="-D_VSTD=std" python setup.py install

# 3) pydiffvg's pure-Python runtime deps:
pip install svgpathtools svgwrite cssutils matplotlib

If it still won't build, the service ships fully without refinement — refine_available() returns False and hd requests degrade with a refine_unavailable warning. Not a failure.

Privacy

Images are processed entirely in memory / ephemeral temp files; nothing is persisted and request bodies are never logged. The client app's HD-mode disclosure copy depends on this.

API

Bearer-token auth: send Authorization: Bearer <VECTORHD_API_TOKEN>. If the token env var is unset the service runs open (dev convenience) and logs a loud warning.

• POST /v1/vectorize — multipart/form-data: image file + optional options JSON field. Returns { svg, stats, warnings }. stats includes width, height, scale, regions, paths, nodes, svg_bytes, per-stage timings_ms, and the Tier-2 summary primitives, symmetries, gradient_regions, edge_confidence_histogram. Errors: 400 bad/oversized file, 401 auth, 422 bad options, 503 pool saturated, 504 job timeout, 500 internal/worker crash. The job runs in a pre-warmed worker process under a hard, SIGKILL-enforced per-job deadline (VECTORHD_JOB_TIMEOUT_S) — the only way to actually stop an unbounded native stage; a mere request-thread timeout cannot. Structured error bodies:

  • 504 — {"error":"job_timeout","stage":<exceeded>,"last_completed":<prior>,"elapsed_ms":<int>}
  • 500 (worker died mid-job) — {"error":"worker_crash","stage":<last-stage>}
  • 503 (all workers busy) — {"error":"server_busy"}

  Tier-2 options are documented in integration/INTEGRATION.md and reflected in GET /v1/info.

• POST /v1/jobs → 202 { job_id, state, poll_after_ms } and GET /v1/jobs/{job_id} → 200 { job_id, state, svg?, stats?, warnings, error? } — the same pipeline without holding the connection. state is pending | running | done | error; svg/stats appear exactly when state == "done" and match /v1/vectorize field-for-field, and a failure arrives as error carrying the same code/status_code the synchronous route would have returned.

  Use this whenever a proxy or gateway caps request duration. A dense 500 px badge takes ~38 s end-to-end, but its longest single HTTP call is 3 ms — so a 25 s ceiling never applies. Submission still validates synchronously (bad options → 422, empty upload → 400), so only pipeline failures are deferred into the job. Records are held in memory only, bounded by TTL and count, and never written to disk — which is why this lives in the service rather than in a platform blob store. It is per-process, so run one uvicorn worker (as the container does).

• GET /healthz — { ok, device, version, build, engine, pool, sam_available, refine_available }.

  • build — { sha, ref, built_at }: which source commit this container was built from, e.g. {"88d429f…", "main", "2026-07-28T15:20:00Z"}. version cannot answer that — a service can serve code well past its own version string, and this one did (the live Space reported 0.2.0 while provably running post-0.2.0 code, so identifying the running commit meant fetching source files off the Space mirror and diffing trees). Read verbatim from a stamp the deploy flow writes into the pushed tree and never re-derived at runtime: an unstamped build reports null rather than guessing from a git checkout or an env var, because a derived value would describe the machine answering the request, not the image. This is the service's counterpart to the web app's /version.txt.
  • engine — { refine_engine_default, backend, level, core_version }: the refine path a request would actually take, e.g. {"analytic", "rust", "handle", "0.1.0"}. Derived by calling the dispatcher's own resolve_backend/resolve_level, so it cannot disagree with what a job gets. This is the field to read when asking what is serving requests — refine_backend in a job's stats confirms it after the fact, but this answers it without running one.
  • pool — { size, alive, busy, restarts } (a growing restarts means jobs are timing out or crashing).
  • sam_available / refine_available — legacy, kept for backward compatibility. They report importability of the optional extras, and refine_available tracks the retired DiffVG extra, so it reads false on a perfectly healthy deployment. It has misled a live reader into thinking refinement was unavailable; consult engine instead. Removal is a future major version.

• GET /v1/info — limits + option defaults/ranges (the web app builds its UI from this), plus the same build block as /healthz from the same reader, so either response identifies the build.

See integration/INTEGRATION.md for the request/response contract and a working Netlify proxy.

Configuration

All settings are environment variables with the VECTORHD_ prefix.

variable	default	what it does
API_TOKEN	unset	Bearer token for every protected route. Unset = the API serves anyone.
REQUIRE_TOKEN	false	Refuse to start when API_TOKEN is unset. Set this in any public deployment — a warning in a hosted log is easy to miss, a container that will not boot is not.
DEVICE	auto	auto|cpu|cuda|mps. The default path is CPU-only.
MAX_MEGAPIXELS	3.0	Working image is downscaled to this.
MAX_FILE_MB	10	Upload size cap.
HARD_DECODE_MEGAPIXELS	24	Reject images decoding above this outright.
REQUEST_TIMEOUT_S	120	Wall-clock budget for one request.
WORKER_POOL_SIZE	2	Pre-warmed worker processes; also the concurrency bound. ≈ cores.
JOB_TIMEOUT_S	REQUEST_TIMEOUT_S − MARGIN	Hard per-job SIGKILL deadline. Refine's own 30 s best-so-far budget runs inside the job.
JOB_TIMEOUT_MARGIN_S	5	Subtracted from REQUEST_TIMEOUT_S when JOB_TIMEOUT_S is unset.
ALLOW_TEST_HOOKS	false	Master gate for test-only fault injection. Never true in production.
WEIGHTS_DIR	weights	Where SAM 2 / perception checkpoints are looked up.
SAM_MODEL_SIZE / SAM_POINTS_PER_SIDE	tiny / 16	SAM 2 backend tuning (optional extra).

Worker-pool rationale: docs/decisions/2026-07-22-stage-bounding-isolation.md. Thread caps (OMP_NUM_THREADS and friends) are set in the Dockerfile and inherited by the spawned workers; cap them to roughly cores / WORKER_POOL_SIZE so one job does not fight itself.

Deployment

One CPU-only image (Dockerfile). There is no GPU target and that is a conclusion, not an omission: the retrofit replaced DiffVG — the only CUDA-relevant path — with the numpy/scipy analytic optimizer, the Rust kernel accelerates it on CPU, and the perception net (the last torch consumer) is gated off by default because every head measurably harmed geometry.

docker build -t vectorhd .
docker run -p 7860:7860 -e VECTORHD_API_TOKEN=secret -e VECTORHD_REQUIRE_TOKEN=1 vectorhd

• Multi-stage: stage 1 compiles the optional vectorhd_core Rust extension with maturin; stage 2 is a slim runtime with no compilers. Verify it loaded by checking that a hd response's stats.refine_backend reads rust (it falls back to the pure-Python kernel silently otherwise).
• Listens on 7860 (Hugging Face Spaces convention), overridable with $PORT. Runs as non-root UID 1000, with a HEALTHCHECK on /healthz. Nothing is written to disk.
• Measured (Apple M-series, native arm64 build, 2 BLAS threads, WORKER_POOL_SIZE=1): image 2.09 GB, cold start to healthy ~9 s, fast on a 192 px icon ~2.9 s, hd on the same ~4.3 s. A dense 93-region badge rides the 30 s refine budget and returns best-so-far with a refine_timeout warning — by design, not a failure.
• Live on the Hugging Face free tier (2026-07-27, same image, same 192 px icon): hd 14.95 s total / 13.27 s refine, versus 4.25 s locally — 3.5× slower on a shared vCPU, at identical output (8 regions, 9 nodes, refine_backend: rust). The free tier costs latency, not quality.
• Job isolation is per-process, not per-thread. Each /v1/vectorize job runs in one of WORKER_POOL_SIZE pre-warmed worker processes; a job exceeding JOB_TIMEOUT_S is SIGKILLed and its worker replenished. This makes the request timeout genuinely enforceable — before it, an unbounded native stage (a pathological input pinned a worker for 66 min in corpus-v1). A container OOM/segfault-class worker death is contained (clean 500, healthy server) rather than taking the service down. Pool overhead vs in-process is IPC-only (single-digit ms at p50); see docs/reports/stage-bounding.md.
• SAM 2 is not installed in the image, so hd emits a sam_unavailable warning and segments with the quantize backend — the documented graceful degradation, and the configuration every published quality number was measured under.

Hugging Face Spaces is the reference deployment: this repo is a ready Docker Space (the README's YAML front-matter configures it). Step-by-step runbook, including secrets and live verification: docs/deploy/huggingface.md. Free-tier expectations are honest ones — 2 shared vCPU, the Space sleeps when idle (first request after a nap pays a ~30–60 s cold start), and hd will be slower than the numbers above. fast mode stays snappy.

Tier-2 perception network (optional)

The perception net (restore option) is a 3-head U-Net (~1.95M params: restore / boundary / corner). It ships unweighted — the whole pipeline runs at Tier-1 quality with no weights and a perception_unavailable warning. To train and install it:

# 1) Generate a synthetic dataset (procedural shapes + degradations; sharded .npz).
uv run python scripts/gen_dataset.py --n 40000 --out data/train --shard-size 512
uv run python scripts/gen_dataset.py --n 4000  --out data/val   --shard-size 512

# 2) Train (yaml-driven). Full run configs/perception.yaml; --smoke for a quick loop check.
uv run python scripts/train.py --config configs/perception.yaml
# Rough duration: a full run is O(hours) on a single modern GPU; the smoke config is minutes on CPU.

# 3) No copy needed: the trainer's `export:` key writes the TorchScript weight straight to
#    models/perception_v1.pt — exactly where the service loads it (the `perception_model` setting,
#    VECTORHD_PERCEPTION_MODEL). (scripts/download_weights.py fetches the SAM2 checkpoint only;
#    perception weights come from this training export, not a download.)
# Acceptance: uv run python scripts/eval_perception.py  (boundary F1 vs Canny, corner recall vs
# Harris, restore PSNR). A validation checkpoint already beats the classical baselines; full
# training is left to the operator (not run in-repo).

Data licensing: all training and benchmark data is procedurally generated in this repo (CC0, self-owned). No third-party datasets are used, and the network is never trained on the output of Vectorizer.AI, Vector Magic, or any other vectorizer. See DATA_LICENSES.md.

Tier-3 roadmap (not built here)

Multi-stop gradients (Tier-2 ships 2-stop) · a learned segmentation head (merging colour bands under one gradient/primitive before fitting) · global / inter-region symmetry · regular-polygon & star primitives (skipped as flaky) · DXF/EPS/PDF export · async job queues · batch endpoints · default-on symmetry enforcement once correspondence is robust on complex art.