Deploy CPU Depth Pro API for speed test

.dockerignore

@@ -0,0 +1,5 @@
+.venv/
+__pycache__/
+*.pyc
+*.pyo
+.DS_Store

Dockerfile

@@ -0,0 +1,31 @@
+FROM python:3.11-slim
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1 \
+    PIP_NO_CACHE_DIR=1 \
+    PORT=8000 \
+    DEPTH_PRO_MODEL_ID=apple/DepthPro-hf \
+    DEPTH_PRO_DEVICE=auto \
+    DEPTH_PRO_DTYPE=float32 \
+    DEPTH_PRO_EAGER_LOAD=false \
+    MAX_IMAGE_BYTES=16777216 \
+    HF_HUB_ENABLE_HF_TRANSFER=1
+
+WORKDIR /app
+
+RUN useradd -m -u 1000 appuser
+
+COPY requirements.txt .
+RUN python -m pip install --upgrade pip \
+    && python -m pip install --index-url https://download.pytorch.org/whl/cpu torch==2.5.1 \
+    && python -m pip install -r requirements.txt
+
+COPY main.py .
+COPY smoke_test.py .
+
+RUN chown -R appuser:appuser /app
+USER appuser
+
+EXPOSE 8000
+
+CMD ["sh", "-c", "uvicorn main:app --host 0.0.0.0 --port ${PORT:-8000}"]

Dockerfile.gpu

@@ -0,0 +1,30 @@
+FROM pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1 \
+    PIP_NO_CACHE_DIR=1 \
+    PORT=8000 \
+    DEPTH_PRO_MODEL_ID=/repository \
+    DEPTH_PRO_DEVICE=auto \
+    DEPTH_PRO_DTYPE=auto \
+    DEPTH_PRO_EAGER_LOAD=true \
+    MAX_IMAGE_BYTES=16777216 \
+    HF_HUB_ENABLE_HF_TRANSFER=1
+
+WORKDIR /app
+
+RUN useradd -m -u 1000 appuser
+
+COPY requirements.txt .
+RUN python -m pip install --upgrade pip \
+    && python -m pip install -r requirements.txt
+
+COPY main.py .
+COPY smoke_test.py .
+
+RUN chown -R appuser:appuser /app
+USER appuser
+
+EXPOSE 8000
+
+CMD ["sh", "-c", "uvicorn main:app --host 0.0.0.0 --port ${PORT:-8000}"]

README.md

@@ -1,10 +1,138 @@
 ---
-title: Spatialthings Depth Pro Free Cpu
-emoji: 🚀
-colorFrom: pink
-colorTo: pink
+title: SpatialThings Depth Pro API
 sdk: docker
-pinned: false
+app_port: 8000
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# SpatialThings Hosted Depth Pro API
+
+This directory packages a Hugging Face hosted Depth Pro server that preserves the Android API contract used by SpatialThings.
+
+## Deployment choice
+
+Use a Hugging Face Inference Endpoint custom container as the primary production path. The endpoint should select `apple/DepthPro-hf` as the model repository so Hugging Face mounts the model at `/repository`, while the Docker image contains only this FastAPI server and Python dependencies.
+
+Use a Docker Space only as a fallback. Free Spaces sleep when idle, so they do not satisfy the always-available requirement. Paid Spaces can run indefinitely, but Inference Endpoints have the cleaner production deployment and autoscaling controls.
+
+## API contract
+
+- `GET /health`
+- `POST /estimate-depth`
+- Request `Content-Type`: `image/jpeg`
+- Response `Content-Type`: `application/octet-stream`
+- Response body: contiguous `float32` little-endian depth map
+- Response headers:
+  - `X-Depth-Width`
+  - `X-Depth-Height`
+  - `X-Depth-Scale: metric_meters`
+  - `X-Process-Time-Sec`
+
+The server returns the `predicted_depth` tensor from `apple/DepthPro-hf` after `post_process_depth_estimation(..., target_sizes=[(image.height, image.width)])`. It does not normalize the output.
+
+## Cost and availability
+
+For always-on production, configure the endpoint with:
+
+- min replicas: `1`
+- max replicas: `1` to start, increase only after measuring traffic
+- scale-to-zero: disabled
+- hardware: start with 1x Nvidia L4; T4 can be cheaper but has less GPU memory
+
+As of 2026-07-02 from Hugging Face pricing docs:
+
+- Inference Endpoint AWS T4 x1: `$0.50/hr`, about `$365/month` at 730 hours
+- Inference Endpoint AWS L4 x1: `$0.80/hr`, about `$584/month`
+- Inference Endpoint GCP L4 x1: `$0.70/hr`, about `$511/month`
+- Space T4 small: `$0.40/hr`, Space T4 medium: `$0.60/hr`, Space L4 x1: `$0.80/hr`
+
+Do not enable scale-to-zero for the Android production URL. Hugging Face documents cold starts, temporary `503` responses while a replica initializes, and multi-minute scale-up time depending on the model. That behavior conflicts with an always-available mobile backend.
+
+## Build and push the container
+
+From the repository root:
+
+```bash
+docker build --platform linux/amd64 \
+  -f deploy/hf_depth_pro/Dockerfile.gpu \
+  -t <registry-user>/spatialthings-depth-pro:0.1.0 \
+  deploy/hf_depth_pro
+
+docker push <registry-user>/spatialthings-depth-pro:0.1.0
+```
+
+`--platform linux/amd64` matters on Apple Silicon Macs because Hugging Face Endpoint infrastructure expects x86_64 container images.
+
+## Create the Inference Endpoint
+
+Use the Inference Endpoints UI when deploying a custom container:
+
+1. Create a new endpoint.
+2. Model repository: `apple/DepthPro-hf`.
+3. Custom container image: `<registry-user>/spatialthings-depth-pro:0.1.0`.
+4. Container port: `8000`.
+5. Hardware: 1x Nvidia L4 recommended for the first production deployment.
+6. Autoscaling: `min replicas=1`, `max replicas=1`, scale-to-zero disabled.
+7. Visibility:
+   - Public keeps the current Android contract with no auth header, but exposes the endpoint to abuse.
+   - Protected requires adding `Authorization: Bearer ...` in the Android client.
+
+After the endpoint reaches Running, set the Android Depth Pro base URL to:
+
+```text
+https://<endpoint-id>.<region>.endpoints.huggingface.cloud
+```
+
+## Space fallback
+
+For the free CPU test path, create a Docker Space without `--flavor` and without `--sleep-time -1`:
+
+```bash
+hf repos create <user-or-org>/spatialthings-depth-pro \
+  --type space \
+  --space-sdk docker \
+  --public \
+  --exist-ok
+
+hf upload <user-or-org>/spatialthings-depth-pro deploy/hf_depth_pro . \
+  --type space
+```
+
+For Space fallback, set this runtime variable:
+
+```text
+DEPTH_PRO_MODEL_ID=apple/DepthPro-hf
+DEPTH_PRO_EAGER_LOAD=false
+```
+
+The Space URL is:
+
+```text
+https://<user-or-org>-spatialthings-depth-pro.hf.space
+```
+
+This free Space uses CPU Basic. It is suitable for cold-start and rough latency checks only. It can sleep when idle, and Depth Pro CPU inference is expected to be much slower than a paid GPU endpoint.
+
+For a paid always-on Space fallback, recreate or upgrade it with GPU hardware and `--sleep-time -1`.
+
+## Local development fallback
+
+Local execution is only for development validation:
+
+```bash
+cd deploy/hf_depth_pro
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
+
+DEPTH_PRO_MODEL_ID=apple/DepthPro-hf \
+DEPTH_PRO_DEVICE=auto \
+uvicorn main:app --host 0.0.0.0 --port 8000
+```
+
+Smoke-test the Android contract:
+
+```bash
+python smoke_test.py \
+  --base-url http://127.0.0.1:8000 \
+  --image ../../data/tmp_inputs/cat_fallback.jpg
+```

main.py

@@ -0,0 +1,230 @@
+from __future__ import annotations
+
+import logging
+import os
+import threading
+import time
+from contextlib import asynccontextmanager
+from io import BytesIO
+from pathlib import Path
+from typing import Any
+
+import numpy as np
+import torch
+from fastapi import Body, FastAPI, HTTPException, Request, Response
+from PIL import Image, UnidentifiedImageError
+from transformers import AutoImageProcessor, AutoModelForDepthEstimation
+
+LOGGER = logging.getLogger("spatialthings.depth_pro")
+
+DEFAULT_HF_MODEL_ID = "apple/DepthPro-hf"
+ENDPOINT_MODEL_PATH = Path("/repository")
+
+
+def _env_bool(name: str, default: bool | None = None) -> bool | None:
+    raw = os.getenv(name)
+    if raw is None or raw == "":
+        return default
+    return raw.strip().lower() in {"1", "true", "yes", "y", "on"}
+
+
+def _select_model_id() -> str:
+    configured = os.getenv("DEPTH_PRO_MODEL_ID")
+    if configured:
+        return configured
+    if ENDPOINT_MODEL_PATH.exists():
+        return str(ENDPOINT_MODEL_PATH)
+    return DEFAULT_HF_MODEL_ID
+
+
+def _select_device() -> torch.device:
+    configured = os.getenv("DEPTH_PRO_DEVICE", "auto").strip().lower()
+    if configured == "auto":
+        return torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    return torch.device(configured)
+
+
+def _select_dtype(device: torch.device) -> torch.dtype:
+    configured = os.getenv("DEPTH_PRO_DTYPE", "auto").strip().lower()
+    if configured == "auto":
+        return torch.float16 if device.type == "cuda" else torch.float32
+    if configured in {"float16", "fp16", "half"}:
+        return torch.float16
+    if configured in {"bfloat16", "bf16"}:
+        return torch.bfloat16
+    if configured in {"float32", "fp32"}:
+        return torch.float32
+    raise ValueError(f"Unsupported DEPTH_PRO_DTYPE={configured!r}")
+
+
+class ModelNotReadyError(RuntimeError):
+    pass
+
+
+class DepthProModel:
+    def __init__(self) -> None:
+        self.model_id = _select_model_id()
+        self.device = _select_device()
+        self.dtype = _select_dtype(self.device)
+        self.max_image_bytes = int(os.getenv("MAX_IMAGE_BYTES", str(16 * 1024 * 1024)))
+        self.use_fov_model = _env_bool("DEPTH_PRO_USE_FOV_MODEL", None)
+        self.processor: Any | None = None
+        self.model: Any | None = None
+        self.load_lock = threading.Lock()
+        self.lock = threading.Lock()
+
+    def load(self) -> None:
+        if self.processor is not None and self.model is not None:
+            return
+
+        with self.load_lock:
+            if self.processor is not None and self.model is not None:
+                return
+
+            model_kwargs: dict[str, Any] = {"torch_dtype": self.dtype}
+            if self.use_fov_model is not None:
+                model_kwargs["use_fov_model"] = self.use_fov_model
+
+            started = time.perf_counter()
+            LOGGER.info(
+                "Loading Depth Pro model model_id=%s device=%s dtype=%s use_fov_model=%s",
+                self.model_id,
+                self.device,
+                self.dtype,
+                self.use_fov_model,
+            )
+            self.processor = AutoImageProcessor.from_pretrained(self.model_id, use_fast=True)
+            self.model = AutoModelForDepthEstimation.from_pretrained(self.model_id, **model_kwargs)
+            self.model.to(self.device)
+            self.model.eval()
+            LOGGER.info("Loaded Depth Pro in %.3fs", time.perf_counter() - started)
+
+    def unload(self) -> None:
+        if self.model is not None:
+            self.model.to("cpu")
+        self.model = None
+        self.processor = None
+        if torch.cuda.is_available():
+            torch.cuda.empty_cache()
+
+    def require_loaded(self) -> tuple[Any, Any]:
+        if self.processor is None or self.model is None:
+            raise ModelNotReadyError("Depth Pro model is not loaded")
+        return self.processor, self.model
+
+    def health(self) -> dict[str, Any]:
+        return {
+            "ok": self.processor is not None and self.model is not None,
+            "model_id": self.model_id,
+            "device": str(self.device),
+            "dtype": str(self.dtype).replace("torch.", ""),
+            "scale": "metric_meters",
+            "max_image_bytes": self.max_image_bytes,
+        }
+
+    def estimate(self, image: Image.Image) -> tuple[np.ndarray, float]:
+        self.load()
+        processor, model = self.require_loaded()
+        width, height = image.size
+
+        with self.lock:
+            started = time.perf_counter()
+            inputs = processor(images=image, return_tensors="pt")
+            inputs = {
+                key: value.to(device=self.device)
+                if not torch.is_floating_point(value)
+                else value.to(device=self.device, dtype=self.dtype)
+                for key, value in inputs.items()
+            }
+
+            with torch.inference_mode():
+                outputs = model(**inputs)
+
+            post_processed = processor.post_process_depth_estimation(
+                outputs,
+                target_sizes=[(height, width)],
+            )
+            depth = post_processed[0]["predicted_depth"]
+            elapsed = time.perf_counter() - started
+
+        if isinstance(depth, torch.Tensor):
+            depth_np = depth.detach().to(dtype=torch.float32).cpu().numpy()
+        else:
+            depth_np = np.asarray(depth, dtype=np.float32)
+
+        if depth_np.ndim != 2:
+            raise RuntimeError(f"Depth output must be 2D, got shape={depth_np.shape}")
+
+        return np.ascontiguousarray(depth_np.astype(np.float32, copy=False)), elapsed
+
+
+depth_pro = DepthProModel()
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    if _env_bool("DEPTH_PRO_EAGER_LOAD", True):
+        depth_pro.load()
+    try:
+        yield
+    finally:
+        depth_pro.unload()
+
+
+app = FastAPI(title="SpatialThings Depth Pro API", lifespan=lifespan)
+
+
+@app.get("/")
+def root() -> dict[str, Any]:
+    return depth_pro.health()
+
+
+@app.get("/health")
+def health() -> dict[str, Any]:
+    return depth_pro.health()
+
+
+@app.post("/estimate-depth")
+def estimate_depth(
+    request: Request,
+    body: bytes = Body(..., media_type="image/jpeg"),
+) -> Response:
+    content_type = request.headers.get("content-type", "").split(";", maxsplit=1)[0].strip().lower()
+    if content_type not in {"image/jpeg", "image/jpg"}:
+        raise HTTPException(status_code=415, detail="Content-Type must be image/jpeg")
+    if not body:
+        raise HTTPException(status_code=400, detail="Missing request body")
+    if len(body) > depth_pro.max_image_bytes:
+        raise HTTPException(status_code=413, detail=f"Request body too large: {len(body)} bytes")
+
+    try:
+        with Image.open(BytesIO(body)) as image:
+            rgb = image.convert("RGB")
+    except (UnidentifiedImageError, OSError) as exc:
+        raise HTTPException(status_code=400, detail=f"Invalid JPEG image: {exc}") from exc
+
+    try:
+        depth, elapsed = depth_pro.estimate(rgb)
+    except ModelNotReadyError as exc:
+        raise HTTPException(status_code=503, detail=str(exc)) from exc
+    except RuntimeError as exc:
+        LOGGER.exception("Depth estimation failed")
+        raise HTTPException(status_code=500, detail=f"Depth estimation failed: {exc}") from exc
+
+    little_endian = np.ascontiguousarray(depth.astype("<f4", copy=False))
+    payload = little_endian.tobytes()
+    headers = {
+        "Content-Length": str(len(payload)),
+        "X-Depth-Width": str(depth.shape[1]),
+        "X-Depth-Height": str(depth.shape[0]),
+        "X-Depth-Scale": "metric_meters",
+        "X-Process-Time-Sec": f"{elapsed:.6f}",
+    }
+    return Response(content=payload, media_type="application/octet-stream", headers=headers)
+
+
+if __name__ == "__main__":
+    import uvicorn
+
+    logging.basicConfig(level=os.getenv("LOG_LEVEL", "INFO"))
+    uvicorn.run(app, host="0.0.0.0", port=int(os.getenv("PORT", "8000")))

requirements.txt

@@ -0,0 +1,9 @@
+accelerate>=0.34.0
+fastapi>=0.115.0
+hf-transfer>=0.1.8
+numpy>=1.26.0
+pillow>=10.4.0
+safetensors>=0.4.5
+torch>=2.4.0
+transformers>=4.50.0
+uvicorn[standard]>=0.30.0

smoke_test.py

@@ -0,0 +1,61 @@
+from __future__ import annotations
+
+import argparse
+import struct
+import sys
+import urllib.error
+import urllib.request
+from pathlib import Path
+
+
+def request(method: str, url: str, body: bytes | None = None, content_type: str | None = None):
+    headers = {}
+    if content_type:
+        headers["Content-Type"] = content_type
+    req = urllib.request.Request(url, data=body, headers=headers, method=method)
+    return urllib.request.urlopen(req, timeout=120)
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Smoke-test the SpatialThings Depth Pro API contract.")
+    parser.add_argument("--base-url", default="http://127.0.0.1:8000")
+    parser.add_argument("--image", required=True, type=Path)
+    args = parser.parse_args()
+
+    try:
+        with request("GET", f"{args.base_url.rstrip('/')}/health") as response:
+            print(f"health status={response.status} body={response.read().decode('utf-8')}")
+
+        image_bytes = args.image.read_bytes()
+        with request(
+            "POST",
+            f"{args.base_url.rstrip('/')}/estimate-depth",
+            body=image_bytes,
+            content_type="image/jpeg",
+        ) as response:
+            payload = response.read()
+            width = int(response.headers["X-Depth-Width"])
+            height = int(response.headers["X-Depth-Height"])
+            scale = response.headers["X-Depth-Scale"]
+            process_time = response.headers["X-Process-Time-Sec"]
+
+        expected_bytes = width * height * 4
+        if len(payload) != expected_bytes:
+            raise RuntimeError(f"Unexpected body length: got={len(payload)} expected={expected_bytes}")
+        if scale != "metric_meters":
+            raise RuntimeError(f"Unexpected X-Depth-Scale: {scale}")
+
+        first_depth = struct.unpack_from("<f", payload, 0)[0] if payload else float("nan")
+        print(
+            "estimate-depth ok "
+            f"size={width}x{height} bytes={len(payload)} "
+            f"scale={scale} process_time_sec={process_time} first_depth={first_depth:.6f}"
+        )
+        return 0
+    except (OSError, urllib.error.HTTPError, RuntimeError) as exc:
+        print(f"[ERROR] {exc}", file=sys.stderr)
+        return 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())