Spaces:
Running on RTX PRO 6000
Running on RTX PRO 6000
Validation through SimReady Central's validator: path + caption, verdict cache
Browse files/vlm_validate/search_result now goes to SimReady Central's validator (docker/hf-space/deeptag-validate/app.py, vendored from simready-central !582; supervisord's deeptag-validate on 127.0.0.1:8020) on the same engine: each hit's path and caption in words-only checks and an in-memory verdict cache (500k entries).
- Dockerfile +5 -0
- docker/hf-space/deeptag-validate/app.py +836 -0
- docker/hf-space/nginx.conf.template +3 -2
- docker/hf-space/supervisord-gpu.conf +18 -0
- docker/hf-space/supervisord.conf +18 -0
Dockerfile
CHANGED
|
@@ -284,6 +284,11 @@ COPY docker/hf-space/supervisord-gpu.conf /etc/supervisor/conf.d/hf-space.conf
|
|
| 284 |
|
| 285 |
COPY docker/hf-space/nginx.conf.template /etc/nginx/hf-space.conf.template
|
| 286 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 287 |
COPY services/explorer/dist /usr/share/nginx/explorer
|
| 288 |
|
| 289 |
|
|
|
|
| 284 |
|
| 285 |
COPY docker/hf-space/nginx.conf.template /etc/nginx/hf-space.conf.template
|
| 286 |
|
| 287 |
+
# SimReady Central's validator (supervisord's [program:deeptag-validate], nginx's
|
| 288 |
+
# /vlm_validate/search_result): one vendored file, run by the app venv, which already
|
| 289 |
+
# has its fastapi, uvicorn, httpx, pydantic and pillow (deepsearch-api's own).
|
| 290 |
+
COPY docker/hf-space/deeptag-validate/app.py /app/deeptag-validate/app.py
|
| 291 |
+
|
| 292 |
COPY services/explorer/dist /usr/share/nginx/explorer
|
| 293 |
|
| 294 |
|
docker/hf-space/deeptag-validate/app.py
ADDED
|
@@ -0,0 +1,836 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
| 2 |
+
# SPDX-License-Identifier: Apache-2.0
|
| 3 |
+
"""
|
| 4 |
+
deeptag-validate β VLM relevance-validation shim.
|
| 5 |
+
|
| 6 |
+
Two routes, one scorer:
|
| 7 |
+
|
| 8 |
+
- `POST /vlm_validate/search_result` β the original per-image contract: the
|
| 9 |
+
browser uploads one base64 thumbnail per call (~100 requests per search).
|
| 10 |
+
- `POST /vlm_validate/batch_stream` β the streamed batch contract (task #27):
|
| 11 |
+
ONE request per result page carrying only ids + image references; the shim
|
| 12 |
+
fetches each thumbnail ITSELF through the gateway's `/images` (LAN hop,
|
| 13 |
+
shared 7-day nginx cache the browsers already warmed, index-scoped by the
|
| 14 |
+
request's `index` slug) and streams verdicts back as NDJSON lines the
|
| 15 |
+
moment each lands β shim-cache hits first, then misses dispatched in
|
| 16 |
+
REQUEST (retrieval) order. Client disconnect cancels every queued engine
|
| 17 |
+
call (the whole point of the per-request task set β an abandoned query
|
| 18 |
+
must not strand work on the shared engine).
|
| 19 |
+
|
| 20 |
+
Bridges the Explorer's `/vlm_validate/*` contract to the self-hosted
|
| 21 |
+
unified vlm-parser-reranker engine (Qwen3.6-35B-A3B-FP8 on one RTX PRO 6000, served alias
|
| 22 |
+
"reranker" β the same
|
| 23 |
+
engine also serves query parsing under the alias "parser"). The engine is used
|
| 24 |
+
as a relevance scorer via the first-token logprob trick: it reads the verdict
|
| 25 |
+
distribution from one forward pass (enable_thinking=false; Qwen thinks by
|
| 26 |
+
default, so a missed flag silently corrupts verdicts), ~20x faster than
|
| 27 |
+
generating a verdict. We return the SAME shape and scale as usdsearch's own
|
| 28 |
+
validator (llm_client's log-prob method), minus the `reasoning` the UI no
|
| 29 |
+
longer shows:
|
| 30 |
+
|
| 31 |
+
{"is_match": bool, "confidence": float, "similarity_score": float, "model": str}
|
| 32 |
+
|
| 33 |
+
ONE verdict method on every route, the Qwen Yes/No log-prob verdict: a Yes/No
|
| 34 |
+
prompt (JUDGE_PROMPT for text queries, JUDGE_IMAGE_PROMPT for an image alone,
|
| 35 |
+
JUDGE_IMAGE_TEXT_PROMPT for an image with words), P(Yes) read from the
|
| 36 |
+
first-token log-probs (position scan, Yes/No mass renormalized), is_match =
|
| 37 |
+
P(Yes) > 0.5, confidence = P(Yes) in [0,1] and similarity_score = P(Yes) x 100
|
| 38 |
+
in [0,100] β usdsearch's scale, so the Explorer reads every validator's score
|
| 39 |
+
the same way (2026-10-06: the shim sent P(Yes) in [0,1] until then, and the
|
| 40 |
+
Explorer's per-value guess showed a usdsearch reject at P(Yes) 0.0076 as 76 %).
|
| 41 |
+
With JUDGE_ASSET_CONTEXT on (2026-10-07), a words-only check also reads the
|
| 42 |
+
candidate's file path and the index's caption, between the query and the
|
| 43 |
+
question (_select_prompt); image checks never do.
|
| 44 |
+
is_match coincides with the 50 % badge boundary the Explorer hardcodes (a
|
| 45 |
+
false is_match renders the orange "VLM rejected, likely false positive" chip,
|
| 46 |
+
and searchOutcome.js's WEAK state means "no hit at or above 50%"). This is the
|
| 47 |
+
verdict the 2026-09-14/15 evaluations measured
|
| 48 |
+
(docs/decisions/2026-09-search-quality.md). The graded 0/1/2 method and the
|
| 49 |
+
Gemma-4 engine it targeted were evaluated on 2026-09-15 and rejected (user
|
| 50 |
+
decision).
|
| 51 |
+
|
| 52 |
+
Validator-contract semantics (the authoritative engine + client contracts
|
| 53 |
+
live in INTEGRATION.md of https://gitlab-master.nvidia.com/omniverse/deeptag/parser-reranker):
|
| 54 |
+
- ABSTAIN (Yes/No probability mass below threshold at every position) β retry
|
| 55 |
+
the call once; a persistent abstain is returned as is_match=true with
|
| 56 |
+
abstain=true and confidence 0.0 β the candidate KEEPS its retrieval-order
|
| 57 |
+
place, unscored. An abstain must never be treated as "No": rejecting on an
|
| 58 |
+
unreliable read silently deletes good results.
|
| 59 |
+
- Transient failure (connect error / engine 5xx) β back off and retry once,
|
| 60 |
+
then fail-soft: 503/502, which the Explorer reads as "VLM unavailable": it
|
| 61 |
+
shows the VLM-unavailable banner and stops validating (verdicts already
|
| 62 |
+
collected keep their badges), leaving search intact. Scores are never
|
| 63 |
+
fabricated.
|
| 64 |
+
|
| 65 |
+
Throughput: the serving owner's operational contract caps validation fan-out
|
| 66 |
+
at 24 in-flight requests (engine-side 42.9 verdicts/s at that fan-out,
|
| 67 |
+
measured on the earlier L40S engine; end-to-end the shim sustains less because it also fetches the
|
| 68 |
+
thumbnails; more in-flight degrades p95 and adds nothing β the engine also
|
| 69 |
+
serves query parsing), so this shim is tuned
|
| 70 |
+
client-side:
|
| 71 |
+
- ONE pooled keep-alive HTTP client (lifespan-managed) β no per-call TCP
|
| 72 |
+
handshake tax (~an RTT each on ~100 calls per search);
|
| 73 |
+
- images reach the engine at most VALIDATE_MAX_EDGE (448) px on the longest
|
| 74 |
+
edge (2026-10-07): Isaac 6.2 stores every SimReady thumbnail at 1024x1024,
|
| 75 |
+
1,078 prompt tokens per verdict at that size against 250 at 448 (robots stay
|
| 76 |
+
256/512 px; anything at or under the cap passes through untouched). A JPEG
|
| 77 |
+
decodes at a reduced DCT scale first (draft), so the resize costs ~3 ms of
|
| 78 |
+
CPU per 1024-px image. Measured in docs/decisions/2026-09-search-quality.md:
|
| 79 |
+
65 vs 38 verdicts/s at saturation, 97 % of Yes/No verdicts unchanged;
|
| 80 |
+
- an asyncio.Semaphore caps in-flight reranker calls at the contract's
|
| 81 |
+
fan-out ceiling, converting bursts into queueing at the sweet spot;
|
| 82 |
+
- max_tokens 2: Qwen3.6 emits the verdict at position 0 (no wrapper tokens β
|
| 83 |
+
verified by probe); the position SCAN below is kept anyway, per the
|
| 84 |
+
integration contract, so a future wrapper-emitting model still extracts.
|
| 85 |
+
|
| 86 |
+
Environment:
|
| 87 |
+
RERANKER_BASE_URL OpenAI-compatible engine base URL
|
| 88 |
+
(default http://simready-central-prod-cluster.nvidia.com/vlm-parser-reranker/v1)
|
| 89 |
+
RERANKER_MODEL served model name (default "reranker")
|
| 90 |
+
RERANKER_API_KEY bearer key for an engine that wants one (default
|
| 91 |
+
none: the cluster engine is open). Sent on the
|
| 92 |
+
engine's chat calls only, never on an image fetch.
|
| 93 |
+
RERANKER_NAME display name in responses (default "Qwen3.6-35B-A3B-FP8";
|
| 94 |
+
the `model` field in every verdict)
|
| 95 |
+
RERANKER_COVER_MIN Yes/No verdict-token mass below this β abstain
|
| 96 |
+
(default 0.2).
|
| 97 |
+
RERANKER_TIMEOUT_S per-call timeout (default 30, per the serving
|
| 98 |
+
contract; a 50-hit page drained in β1.2 s, 1.46 s p95, on the earlier L40S)
|
| 99 |
+
RERANKER_MAX_TOKENS decode steps per call (default 2; raise if a
|
| 100 |
+
future model buries the verdict deeper)
|
| 101 |
+
RERANKER_MAX_CONCURRENCY in-flight cap toward the engine (default 48 for
|
| 102 |
+
the RTX PRO 6000 engine with async scheduling and
|
| 103 |
+
two API servers: 3 clients x c48 = 120.9 verdicts/s
|
| 104 |
+
at p95 1.41 s; 24 was the L40-class ceiling)
|
| 105 |
+
VALIDATE_MAX_EDGE downscale images whose longest edge exceeds this
|
| 106 |
+
many pixels before sending (default 448 since
|
| 107 |
+
2026-10-07; 4096 = no resize from 2026-09-21,
|
| 108 |
+
512 before 2026-09-15). The verdict cache key
|
| 109 |
+
includes it: changing it starts the shim's cache
|
| 110 |
+
cold.
|
| 111 |
+
RERANKER_BATCH_CONCURRENCY per-batch fan-out cap INSIDE the global
|
| 112 |
+
semaphore (default 24) β two users' concurrent
|
| 113 |
+
batches interleave at the engine instead of one
|
| 114 |
+
queueing whole behind the other.
|
| 115 |
+
SHIM_GATEWAY_BASE_URL the gateway's public listener, for shim-side
|
| 116 |
+
thumbnail fetches (default http://gateway:8080).
|
| 117 |
+
IMAGES_TIMEOUT_S per-thumbnail fetch timeout on the dedicated
|
| 118 |
+
image client (default 5; slow thumbnails must
|
| 119 |
+
not starve the engine connection pool).
|
| 120 |
+
BATCH_MAX_ITEMS batch size ceiling (default 200; larger β 413).
|
| 121 |
+
BATCH_HEARTBEAT_S blank-line heartbeat cadence on an otherwise
|
| 122 |
+
quiet stream. COUPLED CONSTANT β the gateway's
|
| 123 |
+
batch_stream proxy_read_timeout (60 s, hard-
|
| 124 |
+
coded) and the Explorer's stream stall guard
|
| 125 |
+
(45 s) are both sized off the default 10:
|
| 126 |
+
raising it past ~40 trips the client guard,
|
| 127 |
+
past 60 the gateway severs healthy streams.
|
| 128 |
+
VALIDATE_CACHE_SIZE verdict-cache entries (default 4000000, ~0.9 GB
|
| 129 |
+
at 240 bytes an entry; two entries per verdict;
|
| 130 |
+
0 disables; 20000 until 2026-10-07).
|
| 131 |
+
Verdicts are deterministic (temperature 0), so one
|
| 132 |
+
(model, max-edge, query, image-bytes) tuple always
|
| 133 |
+
scores the same β caching makes repeat waves
|
| 134 |
+
(re-searches, pagination bounces, other USERS
|
| 135 |
+
asking the same query) free. Complements the
|
| 136 |
+
Explorer's per-browser verdict cache: this one is
|
| 137 |
+
shared across every client of this shim. Keyed on
|
| 138 |
+
the prompt too (a different query never
|
| 139 |
+
coalesces).
|
| 140 |
+
|
| 141 |
+
Tests (tests/test_scoring.py: the scorer, the payload builder, the verdict
|
| 142 |
+
contract and the cache keys as pure functions; tests/test_routes.py: both
|
| 143 |
+
routes through fastapi.testclient with the engine call and the image fetch
|
| 144 |
+
monkeypatched, i.e. the prompt selection, the two cache paths, the NDJSON line
|
| 145 |
+
protocol and the per-item vs fatal error split; no engine, no network), from
|
| 146 |
+
the repo root:
|
| 147 |
+
|
| 148 |
+
cd deeptag-validate && pip install -r requirements-dev.txt && python -m pytest -q
|
| 149 |
+
# or, with no install step:
|
| 150 |
+
uv run --with pytest --with fastapi --with httpx --with pillow pytest deeptag-validate/tests
|
| 151 |
+
|
| 152 |
+
CI runs the first form as the `test-deeptag-validate` job, which gates
|
| 153 |
+
`build-deeptag-validate` exactly like the simready-api suite gates its image.
|
| 154 |
+
"""
|
| 155 |
+
import asyncio
|
| 156 |
+
import base64
|
| 157 |
+
import hashlib
|
| 158 |
+
import io
|
| 159 |
+
import json
|
| 160 |
+
import logging
|
| 161 |
+
import math
|
| 162 |
+
import os
|
| 163 |
+
import re
|
| 164 |
+
from collections import OrderedDict
|
| 165 |
+
from contextlib import asynccontextmanager
|
| 166 |
+
from typing import List, Optional
|
| 167 |
+
|
| 168 |
+
import httpx
|
| 169 |
+
from fastapi import FastAPI, HTTPException
|
| 170 |
+
from fastapi.responses import StreamingResponse
|
| 171 |
+
from PIL import Image
|
| 172 |
+
from pydantic import BaseModel
|
| 173 |
+
|
| 174 |
+
logger = logging.getLogger("deeptag-validate")
|
| 175 |
+
|
| 176 |
+
# The unified vlm-parser-reranker engine's "reranker" alias, OpenAI-compatible. Defaults to
|
| 177 |
+
# the cluster engine's ingress; override per deployment. It does not gate on an API key.
|
| 178 |
+
RERANKER_BASE_URL = os.environ.get("RERANKER_BASE_URL", "http://simready-central-prod-cluster.nvidia.com/vlm-parser-reranker/v1").rstrip("/")
|
| 179 |
+
RERANKER_MODEL = os.environ.get("RERANKER_MODEL", "reranker")
|
| 180 |
+
# A hosted engine (the HF Spaces' NVCF function / Inference Endpoint) wants a bearer key. It rides the
|
| 181 |
+
# engine's POSTs alone: the same pooled client also fetches http image URLs a request names, and a
|
| 182 |
+
# default header there would hand the key to any origin.
|
| 183 |
+
RERANKER_API_KEY = os.environ.get("RERANKER_API_KEY", "").strip()
|
| 184 |
+
_ENGINE_HEADERS = {"Authorization": f"Bearer {RERANKER_API_KEY}"} if RERANKER_API_KEY else None
|
| 185 |
+
RERANKER_NAME = os.environ.get("RERANKER_NAME", "Qwen3.6-35B-A3B-FP8")
|
| 186 |
+
|
| 187 |
+
# Yes/No verdict-token probability mass below this is unreliable β abstain
|
| 188 |
+
# (treat as unscored, never as "No").
|
| 189 |
+
COVER_MIN = float(os.environ.get("RERANKER_COVER_MIN", "0.2"))
|
| 190 |
+
TIMEOUT_S = float(os.environ.get("RERANKER_TIMEOUT_S", "30"))
|
| 191 |
+
# Qwen3.6 emits the verdict at position 0; one spare decode step is plenty and
|
| 192 |
+
# more only burns shared engine compute (it also serves query parsing).
|
| 193 |
+
RERANKER_MAX_TOKENS = int(os.environ.get("RERANKER_MAX_TOKENS", "2"))
|
| 194 |
+
# The serving contract's fan-out ceiling: past 24 in-flight, p95 degrades and
|
| 195 |
+
# throughput gains nothing.
|
| 196 |
+
RERANKER_MAX_CONCURRENCY = int(os.environ.get("RERANKER_MAX_CONCURRENCY", "48"))
|
| 197 |
+
# Vision-token cost scales with pixels; 448 px (longest edge) is the one view
|
| 198 |
+
# the production Yes/No verdict was measured at β the same edge the server-side
|
| 199 |
+
# verdict stage judges (docs/operations.md "Measured quality and the ordering
|
| 200 |
+
# rule"). The cache key carries the edge, so changing it never re-serves a
|
| 201 |
+
# verdict judged at another size.
|
| 202 |
+
VALIDATE_MAX_EDGE = int(os.environ.get("VALIDATE_MAX_EDGE", "448"))
|
| 203 |
+
|
| 204 |
+
# The asset's own facts in the words-only judge prompt (2026-10-07): "off" (the
|
| 205 |
+
# 2026-09 prompt), "path" (the candidate's file path) or "path+caption" (+ the
|
| 206 |
+
# index's one-line description). The client sends each hit's `path` and
|
| 207 |
+
# `caption`; a hit without a path is judged without facts, and image queries
|
| 208 |
+
# never use them (only words-only judging was evaluated). Measured in
|
| 209 |
+
# docs/decisions/2026-09-search-quality.md.
|
| 210 |
+
ASSET_CONTEXT_MODES = ("off", "path", "path+caption")
|
| 211 |
+
JUDGE_ASSET_CONTEXT = os.environ.get("JUDGE_ASSET_CONTEXT", "off").strip().lower()
|
| 212 |
+
if JUDGE_ASSET_CONTEXT not in ASSET_CONTEXT_MODES:
|
| 213 |
+
logger.warning(
|
| 214 |
+
"JUDGE_ASSET_CONTEXT=%r is not one of %s: judging without asset facts",
|
| 215 |
+
JUDGE_ASSET_CONTEXT, ", ".join(ASSET_CONTEXT_MODES),
|
| 216 |
+
)
|
| 217 |
+
JUDGE_ASSET_CONTEXT = "off"
|
| 218 |
+
# Prompt-length bounds for the facts: a path keeps its tail (the file and its
|
| 219 |
+
# folders name the asset), a caption its head.
|
| 220 |
+
ASSET_PATH_MAX = 300
|
| 221 |
+
ASSET_CAPTION_MAX = 200
|
| 222 |
+
|
| 223 |
+
# Streamed batch validation (task #27). The per-batch cap nests INSIDE the
|
| 224 |
+
# global engine semaphore: a 50-item batch occupies at most this many of the
|
| 225 |
+
# 24 engine slots at once, so concurrent users' batches interleave fairly.
|
| 226 |
+
RERANKER_BATCH_CONCURRENCY = int(os.environ.get("RERANKER_BATCH_CONCURRENCY", "24"))
|
| 227 |
+
# Shim-side thumbnail fetches go through the gateway's public listener β the
|
| 228 |
+
# same /images route (and shared nginx cache) the browsers use, index-scoped
|
| 229 |
+
# by the x-usdsearch-storage-backend header. NEVER a client-supplied URL: the
|
| 230 |
+
# base is pinned here and references travel as urlencoded query params only.
|
| 231 |
+
SHIM_GATEWAY_BASE_URL = os.environ.get("SHIM_GATEWAY_BASE_URL", "http://gateway:8080").rstrip("/")
|
| 232 |
+
IMAGES_TIMEOUT_S = float(os.environ.get("IMAGES_TIMEOUT_S", "5"))
|
| 233 |
+
BATCH_MAX_ITEMS = int(os.environ.get("BATCH_MAX_ITEMS", "200"))
|
| 234 |
+
BATCH_HEARTBEAT_S = float(os.environ.get("BATCH_HEARTBEAT_S", "10"))
|
| 235 |
+
|
| 236 |
+
# Cross-user verdict cache: temperature-0 verdicts are pure functions of
|
| 237 |
+
# (model, resize edge, query, image bytes), so identical pairs β from ANY
|
| 238 |
+
# client β re-serve without an engine call. Insertion-order eviction. An entry
|
| 239 |
+
# costs ~240 bytes (measured on python:3.11), and each fresh verdict writes two
|
| 240 |
+
# (its reference key and its content key), so 4,000,000 entries hold 2,000,000
|
| 241 |
+
# verdicts in ~0.9 GB: about half a day at the capacity plan's Phase 1 load
|
| 242 |
+
# (2026-10-07; 20000 before was ~30 s of traffic at 600 checks/s). In-process
|
| 243 |
+
# only: every shim restart starts cold.
|
| 244 |
+
VALIDATE_CACHE_SIZE = int(os.environ.get("VALIDATE_CACHE_SIZE", "4000000"))
|
| 245 |
+
_verdict_cache: "OrderedDict[str, object]" = OrderedDict()
|
| 246 |
+
|
| 247 |
+
|
| 248 |
+
def _cache_key(query: str, image_url: str, query_image_digest: str = "") -> str:
|
| 249 |
+
"""Content-keyed verdict-cache entry: the prompt, the PREPARED candidate
|
| 250 |
+
image and, for an image query, the query image's digest (every image query
|
| 251 |
+
shares one prompt, so without it two different query images would
|
| 252 |
+
coalesce)."""
|
| 253 |
+
h = hashlib.sha256()
|
| 254 |
+
for part in (RERANKER_MODEL, str(VALIDATE_MAX_EDGE), query, image_url, query_image_digest or ""):
|
| 255 |
+
h.update(part.encode())
|
| 256 |
+
h.update(b"\x00")
|
| 257 |
+
return h.hexdigest()
|
| 258 |
+
|
| 259 |
+
|
| 260 |
+
def _ref_key(prompt: str, query_image_digest: str, index: str, ref: str) -> str:
|
| 261 |
+
"""Reference-keyed verdict-cache entry for the batch path: keyed on the
|
| 262 |
+
IMAGE REFERENCE (image_key/asset_url + index slug) instead of prepared
|
| 263 |
+
bytes, so a cache hit costs no thumbnail fetch at all. The content key
|
| 264 |
+
(`_cache_key`) is still written alongside on every fresh verdict, so the
|
| 265 |
+
two paths coalesce. The query image participates as a PRE-COMPUTED digest
|
| 266 |
+
(image queries share one prompt and must not collide across different
|
| 267 |
+
reference images; hashing the multi-MB data URL per item would stall the
|
| 268 |
+
event loop; the route hashes it once)."""
|
| 269 |
+
h = hashlib.sha256()
|
| 270 |
+
for part in (RERANKER_MODEL, str(VALIDATE_MAX_EDGE), "ref", prompt, query_image_digest or "", index or "", ref):
|
| 271 |
+
h.update(part.encode())
|
| 272 |
+
h.update(b"\x00")
|
| 273 |
+
return h.hexdigest()
|
| 274 |
+
|
| 275 |
+
|
| 276 |
+
def _cache_get(key: str):
|
| 277 |
+
p = _verdict_cache.get(key)
|
| 278 |
+
if p is not None:
|
| 279 |
+
_verdict_cache.move_to_end(key)
|
| 280 |
+
return p
|
| 281 |
+
|
| 282 |
+
|
| 283 |
+
def _cache_put(key: str, score) -> None:
|
| 284 |
+
if VALIDATE_CACHE_SIZE <= 0:
|
| 285 |
+
return
|
| 286 |
+
_verdict_cache[key] = score
|
| 287 |
+
_verdict_cache.move_to_end(key)
|
| 288 |
+
while len(_verdict_cache) > VALIDATE_CACHE_SIZE:
|
| 289 |
+
_verdict_cache.popitem(last=False)
|
| 290 |
+
|
| 291 |
+
# Firm, one-word judge prompt (a weak prompt makes GLM describe the image β the
|
| 292 |
+
# logprob trick breaks). Text goes BEFORE the image in the message content.
|
| 293 |
+
JUDGE_PROMPT_HEAD = 'You are a visual relevance judge for a 3D-asset search. Query: "{query}". '
|
| 294 |
+
JUDGE_QUESTION = "Does the rendered asset match the query? Answer with exactly one word: Yes or No."
|
| 295 |
+
JUDGE_PROMPT = JUDGE_PROMPT_HEAD + JUDGE_QUESTION
|
| 296 |
+
# The asset facts (JUDGE_ASSET_CONTEXT), between the query and the question, as
|
| 297 |
+
# evaluated on 2026-10-07.
|
| 298 |
+
JUDGE_PATH_FACT = 'The candidate asset\'s file is "{path}". '
|
| 299 |
+
JUDGE_CAPTION_FACT = 'Catalog description: "{caption}". '
|
| 300 |
+
# An image alone (find-similar): usdsearch's own image-query question
|
| 301 |
+
# (llm_client LOGPROB_JUDGE_PROMPT_IMAGE_QUERY without its caption clause),
|
| 302 |
+
# word for word; the query image goes first. Without a dedicated prompt an
|
| 303 |
+
# image search would be scored against 'Query: ""' β pure noise. On 40 known
|
| 304 |
+
# pairs (2026-10-06) it matched the shim's earlier wording: every same-object
|
| 305 |
+
# pair Yes, every unrelated pair No.
|
| 306 |
+
JUDGE_IMAGE_PROMPT = (
|
| 307 |
+
"You are a visual relevance judge for a 3D-asset search. The first image is "
|
| 308 |
+
"the query image; the remaining images are rendered views of one asset. Does "
|
| 309 |
+
"the rendered asset match the query? Answer with exactly one word: Yes or No."
|
| 310 |
+
)
|
| 311 |
+
# An image WITH words: both count, the query image first. Not usdsearch's
|
| 312 |
+
# caption clause ('β¦ the query image and the text query is "β¦"; the remaining
|
| 313 |
+
# images are rendered views of one asset'), which let the model judge the query
|
| 314 |
+
# image itself against the words: on the same 40 pairs it passed 3-4 unrelated
|
| 315 |
+
# candidates, this wording none (the words alone, the shim's earlier rule: 1).
|
| 316 |
+
JUDGE_IMAGE_TEXT_PROMPT = (
|
| 317 |
+
'You are a visual relevance judge for a 3D-asset search. Query: "{query}", '
|
| 318 |
+
"together with the first image, which shows what the user is looking for. "
|
| 319 |
+
"The other image shows a candidate asset. Does the candidate asset match the "
|
| 320 |
+
"query? Answer with exactly one word: Yes or No."
|
| 321 |
+
)
|
| 322 |
+
YES_TOKENS = {"yes", "y", "true"}
|
| 323 |
+
NO_TOKENS = {"no", "n", "false"}
|
| 324 |
+
|
| 325 |
+
# One pooled keep-alive client for the process (created in lifespan): validation
|
| 326 |
+
# is ~100 calls per search against a remote engine, and a per-request client
|
| 327 |
+
# pays a fresh TCP handshake (an RTT) on every call.
|
| 328 |
+
_reranker_client: Optional[httpx.AsyncClient] = None
|
| 329 |
+
# Separate, short-timeout client for shim-side thumbnail fetches (batch path):
|
| 330 |
+
# 50 image GETs must never compete with engine calls for the reranker pool,
|
| 331 |
+
# and one slow origin must cost 5 s, not the engine's 30.
|
| 332 |
+
_images_client: Optional[httpx.AsyncClient] = None
|
| 333 |
+
# In-flight cap at the engine's measured saturation point β a burst beyond it
|
| 334 |
+
# queues here instead of doubling everyone's latency engine-side.
|
| 335 |
+
_reranker_sem = asyncio.Semaphore(RERANKER_MAX_CONCURRENCY)
|
| 336 |
+
|
| 337 |
+
|
| 338 |
+
@asynccontextmanager
|
| 339 |
+
async def _lifespan(app: FastAPI):
|
| 340 |
+
global _reranker_client, _images_client
|
| 341 |
+
_reranker_client = httpx.AsyncClient(
|
| 342 |
+
timeout=TIMEOUT_S,
|
| 343 |
+
limits=httpx.Limits(max_connections=64, max_keepalive_connections=64),
|
| 344 |
+
)
|
| 345 |
+
_images_client = httpx.AsyncClient(
|
| 346 |
+
# pool=None: waiting for a connection SLOT must queue, not error β a
|
| 347 |
+
# scalar timeout also bounds pool acquisition, and N concurrent
|
| 348 |
+
# batches Γ 12 workers over a finite pool would otherwise fabricate
|
| 349 |
+
# PoolTimeout "image fetch failed" verdicts for perfectly good
|
| 350 |
+
# thumbnails under load.
|
| 351 |
+
timeout=httpx.Timeout(connect=IMAGES_TIMEOUT_S, read=IMAGES_TIMEOUT_S, write=IMAGES_TIMEOUT_S, pool=None),
|
| 352 |
+
limits=httpx.Limits(max_connections=64, max_keepalive_connections=64),
|
| 353 |
+
)
|
| 354 |
+
try:
|
| 355 |
+
yield
|
| 356 |
+
finally:
|
| 357 |
+
await _reranker_client.aclose()
|
| 358 |
+
_reranker_client = None
|
| 359 |
+
await _images_client.aclose()
|
| 360 |
+
_images_client = None
|
| 361 |
+
|
| 362 |
+
|
| 363 |
+
app = FastAPI(title="deeptag-validate", version="1.1.0", lifespan=_lifespan)
|
| 364 |
+
|
| 365 |
+
|
| 366 |
+
class ValidateRequest(BaseModel):
|
| 367 |
+
query_text: Optional[str] = None
|
| 368 |
+
query_image: Optional[str] = None
|
| 369 |
+
images: List[str] = []
|
| 370 |
+
# The asset's facts for words-only judging (JUDGE_ASSET_CONTEXT).
|
| 371 |
+
path: Optional[str] = None
|
| 372 |
+
caption: Optional[str] = None
|
| 373 |
+
|
| 374 |
+
|
| 375 |
+
def p_yes_from_logprobs(content):
|
| 376 |
+
"""Scan token positions for the one carrying Yes/No mass (scan-mode is the
|
| 377 |
+
contract: robust to wrapper tokens some models emit before the verdict);
|
| 378 |
+
return (p_yes, cover) or (None, 0) if uncertain."""
|
| 379 |
+
for pos in content or []:
|
| 380 |
+
yes = no = 0.0
|
| 381 |
+
for entry in pos.get("top_logprobs", []) or []:
|
| 382 |
+
token = (entry.get("token") or "").strip().lower()
|
| 383 |
+
prob = math.exp(entry["logprob"])
|
| 384 |
+
if token in YES_TOKENS:
|
| 385 |
+
yes += prob
|
| 386 |
+
elif token in NO_TOKENS:
|
| 387 |
+
no += prob
|
| 388 |
+
if yes + no > COVER_MIN:
|
| 389 |
+
return yes / (yes + no), yes + no
|
| 390 |
+
return None, 0.0
|
| 391 |
+
|
| 392 |
+
|
| 393 |
+
def _shrink_bytes(raw: bytes, src: str, was_data_url: bool) -> str:
|
| 394 |
+
"""Sync (thread-pooled) half of image prep: decode, downscale if the longest
|
| 395 |
+
edge exceeds VALIDATE_MAX_EDGE (LANCZOS, JPEG q90), return a data URL. An
|
| 396 |
+
already-small data URL passes through UNTOUCHED; an already-small fetched
|
| 397 |
+
image is re-encoded as a data URL so the engine doesn't re-fetch it."""
|
| 398 |
+
with Image.open(io.BytesIO(raw)) as img:
|
| 399 |
+
w, h = img.size
|
| 400 |
+
if max(w, h) <= VALIDATE_MAX_EDGE:
|
| 401 |
+
logger.debug("image %dx%d <= %dpx: pass-through", w, h, VALIDATE_MAX_EDGE)
|
| 402 |
+
if was_data_url:
|
| 403 |
+
return src
|
| 404 |
+
fmt = (img.format or "png").lower()
|
| 405 |
+
mime = "image/jpeg" if fmt in ("jpg", "jpeg") else f"image/{fmt}"
|
| 406 |
+
return f"data:{mime};base64," + base64.b64encode(raw).decode()
|
| 407 |
+
# A JPEG decodes straight at a reduced DCT scale (1/2 for a 1024-px
|
| 408 |
+
# thumbnail; never below the cap): about half the shim CPU of a full
|
| 409 |
+
# decode before the resize. Other formats ignore it.
|
| 410 |
+
img.draft(None, (VALIDATE_MAX_EDGE, VALIDATE_MAX_EDGE))
|
| 411 |
+
rgb = img.convert("RGB")
|
| 412 |
+
rgb.thumbnail((VALIDATE_MAX_EDGE, VALIDATE_MAX_EDGE), Image.LANCZOS)
|
| 413 |
+
buf = io.BytesIO()
|
| 414 |
+
rgb.save(buf, format="JPEG", quality=90)
|
| 415 |
+
logger.debug(
|
| 416 |
+
"image resized %dx%d -> %dx%d (%d -> %d bytes)",
|
| 417 |
+
w, h, rgb.width, rgb.height, len(raw), buf.tell(),
|
| 418 |
+
)
|
| 419 |
+
return "data:image/jpeg;base64," + base64.b64encode(buf.getvalue()).decode()
|
| 420 |
+
|
| 421 |
+
|
| 422 |
+
async def _prepare_image(src: str) -> str:
|
| 423 |
+
"""Downscale-and-inline an image reference before it reaches the engine.
|
| 424 |
+
Defensive: anything unparseable is sent through untouched β a weird image
|
| 425 |
+
must degrade to the engine's own handling, never break validation."""
|
| 426 |
+
try:
|
| 427 |
+
if src.startswith("data:"):
|
| 428 |
+
_, _, b64 = src.partition(",")
|
| 429 |
+
raw = base64.b64decode(b64)
|
| 430 |
+
was_data_url = True
|
| 431 |
+
elif src.startswith(("http://", "https://")):
|
| 432 |
+
resp = await _reranker_client.get(src)
|
| 433 |
+
resp.raise_for_status()
|
| 434 |
+
raw = resp.content
|
| 435 |
+
was_data_url = False
|
| 436 |
+
else:
|
| 437 |
+
# Bare base64: the Explorer strips the data: prefix off its query
|
| 438 |
+
# image. Sent as is, the engine reads it as an image URL and 400s
|
| 439 |
+
# every image-only verdict; inline it as a data URL instead.
|
| 440 |
+
# Non-base64 strings still pass through (strict decode raises).
|
| 441 |
+
raw = base64.b64decode(src, validate=True)
|
| 442 |
+
was_data_url = False
|
| 443 |
+
return await asyncio.to_thread(_shrink_bytes, raw, src, was_data_url)
|
| 444 |
+
except Exception as exc: # noqa: BLE001 β image prep must never break validation
|
| 445 |
+
logger.warning("image prep failed (%s: %s); sending original", type(exc).__name__, exc)
|
| 446 |
+
return src
|
| 447 |
+
|
| 448 |
+
|
| 449 |
+
def _judge_payload(prompt: str, image_url: str, query_image_url: str = None) -> dict:
|
| 450 |
+
# Text before image(s) per the judge contract. For image queries the query
|
| 451 |
+
# image precedes the candidate (the image prompts' "first image" wording
|
| 452 |
+
# depends on this order). Plain sampling parameters only:
|
| 453 |
+
# no structured-outputs grammar (that belonged to the retired graded
|
| 454 |
+
# method), so an engine without a grammar backend serves this unchanged.
|
| 455 |
+
content = [{"type": "text", "text": prompt}]
|
| 456 |
+
if query_image_url:
|
| 457 |
+
content.append({"type": "image_url", "image_url": {"url": query_image_url}})
|
| 458 |
+
content.append({"type": "image_url", "image_url": {"url": image_url}})
|
| 459 |
+
return {
|
| 460 |
+
"model": RERANKER_MODEL,
|
| 461 |
+
"temperature": 0,
|
| 462 |
+
"max_tokens": RERANKER_MAX_TOKENS,
|
| 463 |
+
"logprobs": True,
|
| 464 |
+
"top_logprobs": 20,
|
| 465 |
+
"chat_template_kwargs": {"enable_thinking": False},
|
| 466 |
+
"messages": [{"role": "user", "content": content}],
|
| 467 |
+
}
|
| 468 |
+
|
| 469 |
+
|
| 470 |
+
async def _call_reranker(payload: dict) -> dict:
|
| 471 |
+
# Transient-failure semantics from the validator contract: back off and
|
| 472 |
+
# retry ONCE on connect errors / engine 5xx before failing soft. The
|
| 473 |
+
# semaphore holds in-flight calls at the engine's saturation point.
|
| 474 |
+
for attempt in (1, 2):
|
| 475 |
+
try:
|
| 476 |
+
async with _reranker_sem:
|
| 477 |
+
resp = await _reranker_client.post(
|
| 478 |
+
f"{RERANKER_BASE_URL}/chat/completions", json=payload, headers=_ENGINE_HEADERS
|
| 479 |
+
)
|
| 480 |
+
resp.raise_for_status()
|
| 481 |
+
return resp.json()
|
| 482 |
+
except (httpx.ConnectError, httpx.ConnectTimeout, httpx.ReadTimeout):
|
| 483 |
+
if attempt == 1:
|
| 484 |
+
await asyncio.sleep(0.25)
|
| 485 |
+
continue
|
| 486 |
+
# Engine down / slow β fail-soft. The Explorer reads 503 as
|
| 487 |
+
# "VLM unavailable": banner, validation stops; search is unaffected.
|
| 488 |
+
raise HTTPException(status_code=503, detail="reranker unavailable")
|
| 489 |
+
except httpx.HTTPStatusError as exc:
|
| 490 |
+
if attempt == 1 and exc.response.status_code >= 500:
|
| 491 |
+
await asyncio.sleep(0.25)
|
| 492 |
+
continue
|
| 493 |
+
raise EngineRejected(exc.response.status_code, f"reranker error: {exc}")
|
| 494 |
+
except httpx.HTTPError as exc:
|
| 495 |
+
raise HTTPException(status_code=502, detail=f"reranker error: {exc}")
|
| 496 |
+
|
| 497 |
+
|
| 498 |
+
# The configuration refusal's message: fixed, never the engine's own error (it names the engine's URL).
|
| 499 |
+
IMAGE_QUERY_REFUSED = "the validation engine refused an image query (it may take one image per prompt)"
|
| 500 |
+
|
| 501 |
+
|
| 502 |
+
class EngineRejected(HTTPException):
|
| 503 |
+
"""The engine ANSWERED with an error status (a 502 to the client, as before; the batch route keeps
|
| 504 |
+
treating it per item). `engine_status` lets the per-image route tell a refused request shape apart."""
|
| 505 |
+
|
| 506 |
+
def __init__(self, engine_status: int, detail: str):
|
| 507 |
+
super().__init__(status_code=502, detail=detail)
|
| 508 |
+
self.engine_status = engine_status
|
| 509 |
+
|
| 510 |
+
|
| 511 |
+
def _abstain_verdict() -> dict:
|
| 512 |
+
# Persistent abstain β keep the candidate in retrieval order, unscored
|
| 513 |
+
# (is_match=true so the Explorer does not reject it; confidence 0.0 and
|
| 514 |
+
# abstain=true make the non-verdict explicit rather than a fabricated
|
| 515 |
+
# score).
|
| 516 |
+
return {"is_match": True, "confidence": 0.0, "similarity_score": 0.0, "abstain": True, "model": RERANKER_NAME}
|
| 517 |
+
|
| 518 |
+
|
| 519 |
+
def _verdict_from_score(score: float) -> dict:
|
| 520 |
+
"""The ONE place a verdict dict is built from a cached or fresh SCORE
|
| 521 |
+
RECORD (the P(Yes) float), for both routes and both cache paths β a
|
| 522 |
+
boundary that lived in three copies would silently regress on cache hits.
|
| 523 |
+
similarity_score is P(Yes) x 100 (usdsearch's scale); confidence stays
|
| 524 |
+
P(Yes)."""
|
| 525 |
+
return {
|
| 526 |
+
"is_match": score > 0.5,
|
| 527 |
+
"confidence": round(score, 4),
|
| 528 |
+
"similarity_score": round(score * 100, 2),
|
| 529 |
+
"model": RERANKER_NAME,
|
| 530 |
+
}
|
| 531 |
+
|
| 532 |
+
|
| 533 |
+
def _extract_score(data: dict):
|
| 534 |
+
"""Pull the P(Yes) score record out of one engine response, or None
|
| 535 |
+
(abstain)."""
|
| 536 |
+
content = ((data.get("choices") or [{}])[0].get("logprobs") or {}).get("content")
|
| 537 |
+
p_yes, _cover = p_yes_from_logprobs(content)
|
| 538 |
+
return p_yes
|
| 539 |
+
|
| 540 |
+
|
| 541 |
+
def _fact_text(value, limit: int, keep_tail: bool = False) -> str:
|
| 542 |
+
"""One asset fact as prompt text: whitespace collapsed, double quotes made
|
| 543 |
+
single (the fact sits inside a quoted span), at most `limit` characters."""
|
| 544 |
+
text = " ".join(str(value or "").split()).replace('"', "'")
|
| 545 |
+
if len(text) > limit:
|
| 546 |
+
text = text[-limit:] if keep_tail else text[:limit]
|
| 547 |
+
return text
|
| 548 |
+
|
| 549 |
+
|
| 550 |
+
_ISAAC_ROOT = re.compile(r"/Isaac/[^/]+/Isaac/(.+)$")
|
| 551 |
+
# A Hugging Face dataset file (the hf-* indexes, the HF Spaces): below the dataset, and below
|
| 552 |
+
# `resolve|blob/<revision>/` where the key pins one, e.g.
|
| 553 |
+
# hf://huggingface.co/datasets/nvidia/simready-assets/resolve/2464ca32β¦/fixture_island_a01/simready_usd/x.usd
|
| 554 |
+
# β fixture_island_a01/simready_usd/x.usd.
|
| 555 |
+
_HF_DATASET_ROOT = re.compile(
|
| 556 |
+
r"^(?:[a-z][a-z0-9+.-]*://)?huggingface\.co/datasets/[^/]+/[^/]+/(?:(?:resolve|blob)/[^/]+/)?(.+)$", re.I
|
| 557 |
+
)
|
| 558 |
+
|
| 559 |
+
|
| 560 |
+
def _asset_path(path) -> str:
|
| 561 |
+
"""The path as the judge reads it: below the release-agnostic Isaac content
|
| 562 |
+
root (`Robots/Unitree/H2/H2.usda`, as evaluated), or below a Hugging Face
|
| 563 |
+
dataset's root, else without its leading slash."""
|
| 564 |
+
p = str(path or "").strip()
|
| 565 |
+
m = _ISAAC_ROOT.search(p) or _HF_DATASET_ROOT.match(p)
|
| 566 |
+
return _fact_text(m.group(1) if m else p.lstrip("/"), ASSET_PATH_MAX, keep_tail=True)
|
| 567 |
+
|
| 568 |
+
|
| 569 |
+
def _asset_facts(path, caption) -> str:
|
| 570 |
+
"""The words-only prompt's asset facts under JUDGE_ASSET_CONTEXT, or "":
|
| 571 |
+
the file path, then (path+caption) the caption. No path β no facts, so
|
| 572 |
+
only the evaluated combinations are ever sent."""
|
| 573 |
+
if JUDGE_ASSET_CONTEXT == "off":
|
| 574 |
+
return ""
|
| 575 |
+
p = _asset_path(path)
|
| 576 |
+
if not p:
|
| 577 |
+
return ""
|
| 578 |
+
facts = JUDGE_PATH_FACT.format(path=p)
|
| 579 |
+
caption_text = _fact_text(caption, ASSET_CAPTION_MAX) if JUDGE_ASSET_CONTEXT == "path+caption" else ""
|
| 580 |
+
if caption_text:
|
| 581 |
+
facts += JUDGE_CAPTION_FACT.format(caption=caption_text)
|
| 582 |
+
return facts
|
| 583 |
+
|
| 584 |
+
|
| 585 |
+
def _select_prompt(query_text: str, has_query_image: bool, path=None, caption=None) -> str:
|
| 586 |
+
"""The judge prompt for one candidate, over the stripped query text: an
|
| 587 |
+
image with words β JUDGE_IMAGE_TEXT_PROMPT, an image alone β
|
| 588 |
+
JUDGE_IMAGE_PROMPT, words alone β JUDGE_PROMPT, with the candidate's facts
|
| 589 |
+
before the question when JUDGE_ASSET_CONTEXT is on (a request with neither
|
| 590 |
+
words nor image keeps the historical 'Query: ""' prompt)."""
|
| 591 |
+
text = (query_text or "").strip()
|
| 592 |
+
if has_query_image:
|
| 593 |
+
return JUDGE_IMAGE_TEXT_PROMPT.format(query=text) if text else JUDGE_IMAGE_PROMPT
|
| 594 |
+
facts = _asset_facts(path, caption)
|
| 595 |
+
if not facts:
|
| 596 |
+
return JUDGE_PROMPT.format(query=text)
|
| 597 |
+
return JUDGE_PROMPT_HEAD.format(query=text) + facts + JUDGE_QUESTION
|
| 598 |
+
|
| 599 |
+
|
| 600 |
+
def _query_image_digest(query_image: Optional[str]) -> str:
|
| 601 |
+
"""Hash the (possibly multi-MB) query image ONCE per request β the cache
|
| 602 |
+
keys run per item, and per-item sha256 over a full data URL would stall
|
| 603 |
+
the loop."""
|
| 604 |
+
return hashlib.sha256(query_image.encode()).hexdigest() if query_image else ""
|
| 605 |
+
|
| 606 |
+
|
| 607 |
+
async def _score_prepared(prompt: str, image_url: str, query_image_url: str = None):
|
| 608 |
+
"""One verdict for a PREPARED (downscaled, inlined) image. Returns
|
| 609 |
+
(score | None, verdict_dict); score None means persistent abstain (the
|
| 610 |
+
dict already carries abstain=true; the CALLER decides caching, and
|
| 611 |
+
abstains are never cached). The score is the cache record (see
|
| 612 |
+
_verdict_from_score). Raises HTTPException 502/503 when the engine is
|
| 613 |
+
unreachable (fail-soft to the Explorer's "VLM unavailable")."""
|
| 614 |
+
data = await _call_reranker(_judge_payload(prompt, image_url, query_image_url))
|
| 615 |
+
score = _extract_score(data)
|
| 616 |
+
if score is None:
|
| 617 |
+
# Abstain semantics: retry once β never treat an unreliable read as "No".
|
| 618 |
+
data = await _call_reranker(_judge_payload(prompt, image_url, query_image_url))
|
| 619 |
+
score = _extract_score(data)
|
| 620 |
+
if score is None:
|
| 621 |
+
return None, _abstain_verdict()
|
| 622 |
+
return score, _verdict_from_score(score)
|
| 623 |
+
|
| 624 |
+
|
| 625 |
+
@app.get("/health")
|
| 626 |
+
def health():
|
| 627 |
+
return {"status": "ok", "reranker": RERANKER_BASE_URL, "model": RERANKER_MODEL, "asset_context": JUDGE_ASSET_CONTEXT}
|
| 628 |
+
|
| 629 |
+
|
| 630 |
+
@app.post("/vlm_validate/search_result")
|
| 631 |
+
async def validate(req: ValidateRequest):
|
| 632 |
+
if not req.images:
|
| 633 |
+
raise HTTPException(status_code=400, detail="no image provided")
|
| 634 |
+
# The same judge as the batch route: a query image (the Explorer sends it
|
| 635 |
+
# bare base64 when it finishes a cut batch stream here) goes to the engine
|
| 636 |
+
# first, under the image prompts.
|
| 637 |
+
prompt = _select_prompt(req.query_text, bool(req.query_image), req.path, req.caption)
|
| 638 |
+
query_image_url = await _prepare_image(req.query_image) if req.query_image else None
|
| 639 |
+
image_url = await _prepare_image(req.images[0])
|
| 640 |
+
# Serve deterministic repeats from the verdict cache β no engine call. The
|
| 641 |
+
# key hashes the PREPARED image (post-resize), so the same asset arriving
|
| 642 |
+
# as a data URL or a fetched http URL coalesces once prepared.
|
| 643 |
+
ckey = _cache_key(prompt, image_url, _query_image_digest(req.query_image))
|
| 644 |
+
cached = _cache_get(ckey) if VALIDATE_CACHE_SIZE > 0 else None
|
| 645 |
+
if cached is not None:
|
| 646 |
+
return {**_verdict_from_score(cached), "cached": True}
|
| 647 |
+
try:
|
| 648 |
+
score, verdict = await _score_prepared(prompt, image_url, query_image_url)
|
| 649 |
+
except EngineRejected as exc:
|
| 650 |
+
# An engine that takes one image per prompt (the dev HF Space's NVCF function) answers an image
|
| 651 |
+
# query (query image + candidate) with a 4xx. Nothing is down: answer usdsearch's configuration
|
| 652 |
+
# refusal, which the Space's Explorer reads as "this search goes unvalidated" instead of the
|
| 653 |
+
# VLM-unavailable banner. Central's Explorer reads any 503 as before.
|
| 654 |
+
if query_image_url and 400 <= exc.engine_status < 500 and exc.engine_status not in (408, 429):
|
| 655 |
+
raise HTTPException(
|
| 656 |
+
status_code=503,
|
| 657 |
+
detail={"error_type": "configuration", "retryable": False, "message": IMAGE_QUERY_REFUSED},
|
| 658 |
+
) from exc
|
| 659 |
+
raise
|
| 660 |
+
if score is not None:
|
| 661 |
+
_cache_put(ckey, score)
|
| 662 |
+
return verdict
|
| 663 |
+
|
| 664 |
+
|
| 665 |
+
class BatchItem(BaseModel):
|
| 666 |
+
id: str
|
| 667 |
+
image_key: Optional[str] = None
|
| 668 |
+
asset_url: Optional[str] = None
|
| 669 |
+
# The hit's source.path and the index's caption for it: facts for
|
| 670 |
+
# words-only judging (JUDGE_ASSET_CONTEXT).
|
| 671 |
+
path: Optional[str] = None
|
| 672 |
+
caption: Optional[str] = None
|
| 673 |
+
|
| 674 |
+
|
| 675 |
+
class BatchValidateRequest(BaseModel):
|
| 676 |
+
query_text: Optional[str] = None
|
| 677 |
+
query_image: Optional[str] = None
|
| 678 |
+
# The index SLUG (x-usdsearch-storage-backend value) the shim forwards on
|
| 679 |
+
# its /images fetches β routing, api-key injection, and the shared image
|
| 680 |
+
# cache all key off it upstream. Empty β the gateway's default index.
|
| 681 |
+
index: Optional[str] = None
|
| 682 |
+
# In the client's DISPLAY (retrieval) order β dispatch preserves it, so
|
| 683 |
+
# the most-likely matches score first and the ranked flow fills top-down.
|
| 684 |
+
items: List[BatchItem] = []
|
| 685 |
+
|
| 686 |
+
|
| 687 |
+
def _ndjson(obj) -> str:
|
| 688 |
+
return json.dumps(obj, separators=(",", ":")) + "\n"
|
| 689 |
+
|
| 690 |
+
|
| 691 |
+
@app.post("/vlm_validate/batch_stream")
|
| 692 |
+
async def validate_batch(req: BatchValidateRequest):
|
| 693 |
+
"""Streamed batch validation (NDJSON). Line protocol:
|
| 694 |
+
|
| 695 |
+
{"id", "cached", "verdict": {is_match, confidence, similarity_score,
|
| 696 |
+
model[, abstain]}} β one per scored hit
|
| 697 |
+
{"id", "error": "..."} β per-hit failure (bad thumbnail etc.);
|
| 698 |
+
NEVER poisons the rest of the batch
|
| 699 |
+
"" β blank heartbeat every BATCH_HEARTBEAT_S
|
| 700 |
+
{"fatal": "..."} β engine unreachable; treat as 502/503
|
| 701 |
+
{"done": {completed, errored, cached}} β ALWAYS last on a clean finish;
|
| 702 |
+
its absence means the stream was cut
|
| 703 |
+
(the client re-POSTs the missing ids)
|
| 704 |
+
|
| 705 |
+
Shim-cache hits flush first (no engine, no image fetch β the reference-
|
| 706 |
+
keyed entries); misses dispatch in request order under the per-batch cap.
|
| 707 |
+
Client disconnect cancels every pending worker; `async with _reranker_sem`
|
| 708 |
+
releases on cancellation, so an abandoned query frees its engine slots."""
|
| 709 |
+
if not req.items:
|
| 710 |
+
raise HTTPException(status_code=400, detail="no items")
|
| 711 |
+
if len(req.items) > BATCH_MAX_ITEMS:
|
| 712 |
+
raise HTTPException(status_code=413, detail=f"batch exceeds {BATCH_MAX_ITEMS} items")
|
| 713 |
+
if not (req.query_text or req.query_image):
|
| 714 |
+
raise HTTPException(status_code=400, detail="no query")
|
| 715 |
+
index = (req.index or "").strip()
|
| 716 |
+
qimg_digest = _query_image_digest(req.query_image)
|
| 717 |
+
# An image query (alone or with words): the judge sees the query image
|
| 718 |
+
# first, then each candidate. Prepared (downscaled) once. With asset facts
|
| 719 |
+
# on, the prompt (and so every cache key) is per item.
|
| 720 |
+
def prompt_for(it: BatchItem) -> str:
|
| 721 |
+
return _select_prompt(req.query_text, bool(req.query_image), it.path, it.caption)
|
| 722 |
+
|
| 723 |
+
query_image_url = await _prepare_image(req.query_image) if req.query_image else None
|
| 724 |
+
|
| 725 |
+
# Phase 0 β reference-keyed cache pass: free verdicts, emitted first.
|
| 726 |
+
cached_lines = [] # (kind, line) β kind: "verdict" | "error"
|
| 727 |
+
misses = []
|
| 728 |
+
for it in req.items:
|
| 729 |
+
ref = it.image_key or it.asset_url
|
| 730 |
+
if not ref:
|
| 731 |
+
cached_lines.append(("error", _ndjson({"id": it.id, "error": "no image reference"})))
|
| 732 |
+
continue
|
| 733 |
+
cached = _cache_get(_ref_key(prompt_for(it), qimg_digest, index, ref)) if VALIDATE_CACHE_SIZE > 0 else None
|
| 734 |
+
if cached is not None:
|
| 735 |
+
cached_lines.append(
|
| 736 |
+
("verdict", _ndjson({
|
| 737 |
+
"id": it.id,
|
| 738 |
+
"cached": True,
|
| 739 |
+
"verdict": _verdict_from_score(cached),
|
| 740 |
+
}))
|
| 741 |
+
)
|
| 742 |
+
else:
|
| 743 |
+
misses.append(it)
|
| 744 |
+
|
| 745 |
+
queue: asyncio.Queue = asyncio.Queue()
|
| 746 |
+
batch_sem = asyncio.Semaphore(RERANKER_BATCH_CONCURRENCY)
|
| 747 |
+
|
| 748 |
+
async def worker(it: BatchItem):
|
| 749 |
+
async with batch_sem:
|
| 750 |
+
ref = it.image_key or it.asset_url
|
| 751 |
+
prompt = prompt_for(it)
|
| 752 |
+
# Pinned base + urlencoded params ONLY β a reference can never
|
| 753 |
+
# change the request target (no SSRF surface); image_key (a
|
| 754 |
+
# specific matched view) wins over asset_url, mirroring the
|
| 755 |
+
# Explorer's own /images precedence.
|
| 756 |
+
params = {"image_key": it.image_key} if it.image_key else {"asset_url": it.asset_url}
|
| 757 |
+
headers = {"x-usdsearch-storage-backend": index} if index else {}
|
| 758 |
+
try:
|
| 759 |
+
resp = await _images_client.get(f"{SHIM_GATEWAY_BASE_URL}/images", params=params, headers=headers)
|
| 760 |
+
resp.raise_for_status()
|
| 761 |
+
raw = resp.content
|
| 762 |
+
except Exception as exc: # noqa: BLE001 β one bad thumbnail must not poison the batch
|
| 763 |
+
await queue.put(("error", _ndjson({"id": it.id, "error": f"image fetch failed: {type(exc).__name__}"})))
|
| 764 |
+
return
|
| 765 |
+
try:
|
| 766 |
+
# Raw bytes straight into the thread-pooled shrink: no base64
|
| 767 |
+
# round-trip on the loop, and the data URL's mime comes from
|
| 768 |
+
# PIL's own sniff (img.format), never the upstream's
|
| 769 |
+
# content-type header (S3-ish origins say octet-stream, which
|
| 770 |
+
# OpenAI-compatible engines 400 on).
|
| 771 |
+
image_url = await asyncio.to_thread(_shrink_bytes, raw, "", False)
|
| 772 |
+
except Exception as exc: # noqa: BLE001 β undecodable bytes must not reach the engine
|
| 773 |
+
await queue.put(("error", _ndjson({"id": it.id, "error": f"image decode failed: {type(exc).__name__}"})))
|
| 774 |
+
return
|
| 775 |
+
try:
|
| 776 |
+
score, verdict = await _score_prepared(prompt, image_url, query_image_url)
|
| 777 |
+
except HTTPException as exc:
|
| 778 |
+
if exc.status_code == 503:
|
| 779 |
+
# Engine UNREACHABLE β the whole batch is over.
|
| 780 |
+
await queue.put(("fatal", _ndjson({"fatal": exc.detail})))
|
| 781 |
+
else:
|
| 782 |
+
# Engine rejected THIS item (4xxβ502: e.g. an image it
|
| 783 |
+
# can't decode) β per-hit, never batch-fatal.
|
| 784 |
+
await queue.put(("error", _ndjson({"id": it.id, "error": f"engine rejected item: {exc.detail}"})))
|
| 785 |
+
return
|
| 786 |
+
except Exception as exc: # noqa: BLE001 β a dead worker with no line would
|
| 787 |
+
# strand `remaining` and heartbeat the stream FOREVER (the
|
| 788 |
+
# blank lines reset every downstream timeout). Anything
|
| 789 |
+
# unexpected (non-JSON 200 from a proxy, logprob shape drift)
|
| 790 |
+
# settles the hit as a per-hit error instead.
|
| 791 |
+
logger.warning("batch worker failed (%s: %s)", type(exc).__name__, exc)
|
| 792 |
+
await queue.put(("error", _ndjson({"id": it.id, "error": f"scoring failed: {type(exc).__name__}"})))
|
| 793 |
+
return
|
| 794 |
+
if score is not None:
|
| 795 |
+
_cache_put(_ref_key(prompt, qimg_digest, index, ref), score)
|
| 796 |
+
_cache_put(_cache_key(prompt, image_url, qimg_digest), score)
|
| 797 |
+
await queue.put(("verdict", _ndjson({"id": it.id, "cached": False, "verdict": verdict})))
|
| 798 |
+
|
| 799 |
+
async def stream():
|
| 800 |
+
completed = sum(1 for kind, _ in cached_lines if kind == "verdict")
|
| 801 |
+
errored = len(cached_lines) - completed
|
| 802 |
+
cached_count = completed
|
| 803 |
+
# Workers are created INSIDE the generator: a client that aborts
|
| 804 |
+
# before the body is ever iterated (POST β immediate abort) must not
|
| 805 |
+
# leak a batch onto the shared engine β a never-started generator
|
| 806 |
+
# never runs its finally, so task lifetime has to live in this frame.
|
| 807 |
+
# Dispatch in REQUEST order β the per-batch semaphore wakes FIFO, so
|
| 808 |
+
# completion approximates retrieval order and the flow's top fills
|
| 809 |
+
# first.
|
| 810 |
+
tasks = [asyncio.create_task(worker(it)) for it in misses]
|
| 811 |
+
try:
|
| 812 |
+
for _, ln in cached_lines:
|
| 813 |
+
yield ln
|
| 814 |
+
remaining = len(tasks)
|
| 815 |
+
while remaining > 0:
|
| 816 |
+
try:
|
| 817 |
+
kind, ln = await asyncio.wait_for(queue.get(), timeout=BATCH_HEARTBEAT_S)
|
| 818 |
+
except asyncio.TimeoutError:
|
| 819 |
+
yield "\n" # heartbeat β keeps proxies from severing a quiet stream
|
| 820 |
+
continue
|
| 821 |
+
yield ln
|
| 822 |
+
if kind == "fatal":
|
| 823 |
+
return # finally cancels the rest; no `done` line on a fatal
|
| 824 |
+
remaining -= 1
|
| 825 |
+
if kind == "error":
|
| 826 |
+
errored += 1
|
| 827 |
+
else:
|
| 828 |
+
completed += 1
|
| 829 |
+
yield _ndjson({"done": {"completed": completed, "errored": errored, "cached": cached_count}})
|
| 830 |
+
finally:
|
| 831 |
+
# Client gone, fatal, or clean finish β nothing may keep burning
|
| 832 |
+
# shared engine slots. Cancellation releases the semaphores.
|
| 833 |
+
for t in tasks:
|
| 834 |
+
t.cancel()
|
| 835 |
+
|
| 836 |
+
return StreamingResponse(stream(), media_type="application/x-ndjson")
|
docker/hf-space/nginx.conf.template
CHANGED
|
@@ -73,10 +73,11 @@ server {
|
|
| 73 |
proxy_pass http://127.0.0.1:8000;
|
| 74 |
}
|
| 75 |
|
|
|
|
|
|
|
| 76 |
location /vlm_validate/search_result {
|
| 77 |
if ($request_method = OPTIONS) { add_header Content-Length 0 always; add_header Content-Type "text/plain" always; return 204; }
|
| 78 |
-
|
| 79 |
-
proxy_pass http://127.0.0.1:8000;
|
| 80 |
}
|
| 81 |
|
| 82 |
location /llm_parse/ {
|
|
|
|
| 73 |
proxy_pass http://127.0.0.1:8000;
|
| 74 |
}
|
| 75 |
|
| 76 |
+
# SimReady Central's validator (supervisord's deeptag-validate, 127.0.0.1:8020) answers here in
|
| 77 |
+
# place of deepsearch-api's: the same judge plus each hit's path and caption and a verdict cache.
|
| 78 |
location /vlm_validate/search_result {
|
| 79 |
if ($request_method = OPTIONS) { add_header Content-Length 0 always; add_header Content-Type "text/plain" always; return 204; }
|
| 80 |
+
proxy_pass http://127.0.0.1:8020;
|
|
|
|
| 81 |
}
|
| 82 |
|
| 83 |
location /llm_parse/ {
|
docker/hf-space/supervisord-gpu.conf
CHANGED
|
@@ -155,6 +155,24 @@ stdout_logfile_maxbytes=0
|
|
| 155 |
stderr_logfile=/dev/stderr
|
| 156 |
stderr_logfile_maxbytes=0
|
| 157 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 158 |
[program:nginx]
|
| 159 |
command=/bin/sh -c "until curl -sf http://localhost:8000/health >/dev/null 2>&1 && curl -sf http://localhost:8010/docs >/dev/null 2>&1; do sleep 2; done; dl=false; case '%(ENV_DOWNLOAD_ENABLED)s' in [Tt]rue|1) if [ -f /app/hf-space-pipeline/.build/restore/hf-storage-api-catalog.yaml ]; then dl=true; else echo 'nginx: no hf-storage-api catalog, rendering download:false' >&2; fi;; esac; sed -e 's/__PORT__/%(ENV_PORT)s/' -e 's|__CORPUS_NAME__|%(ENV_CORPUS_NAME)s|g' -e 's/__DOWNLOAD_ENABLED__/'$dl'/' -e 's|__SUBMIT_CONTENT_URL__|%(ENV_SUBMIT_CONTENT_URL)s|g' /etc/nginx/hf-space.conf.template > /etc/nginx/conf.d/hf-space.conf && exec nginx -g 'daemon off;'"
|
| 160 |
user=root
|
|
|
|
| 155 |
stderr_logfile=/dev/stderr
|
| 156 |
stderr_logfile_maxbytes=0
|
| 157 |
|
| 158 |
+
[program:deeptag-validate]
|
| 159 |
+
; SimReady Central's validator (docker/hf-space/deeptag-validate/app.py, vendored from
|
| 160 |
+
; simready-central deeptag-validate/, !582) answers /vlm_validate/search_result in place of
|
| 161 |
+
; deepsearch-api's: the same Yes/No log-prob judge on the same engine, plus each hit's path
|
| 162 |
+
; and caption in words-only checks (JUDGE_ASSET_CONTEXT) and an in-memory verdict cache
|
| 163 |
+
; (VALIDATE_CACHE_SIZE, ~120 MB at 500k entries). The engine settings are deepsearch-api's
|
| 164 |
+
; USDSEARCH_* variables, read when the shell execs (an unset one never stops supervisord).
|
| 165 |
+
command=/usr/bin/env -u PYTHONPATH /bin/sh -c 'RERANKER_BASE_URL="$USDSEARCH_LLM_BASE_URL" RERANKER_MODEL="$USDSEARCH_VISION_VALIDATION_MODEL" RERANKER_NAME="$USDSEARCH_VISION_VALIDATION_MODEL" RERANKER_API_KEY="$USDSEARCH_LLM_API_KEY" exec /app/.venv/bin/python -m uvicorn app:app --host 127.0.0.1 --port 8020 --log-level warning'
|
| 166 |
+
environment=JUDGE_ASSET_CONTEXT="path+caption",VALIDATE_CACHE_SIZE="500000",VALIDATE_MAX_EDGE="448",RERANKER_MAX_CONCURRENCY="32",RERANKER_BATCH_CONCURRENCY="16",RERANKER_COVER_MIN="0.2"
|
| 167 |
+
directory=/app/deeptag-validate
|
| 168 |
+
user=1000
|
| 169 |
+
autorestart=true
|
| 170 |
+
startsecs=3
|
| 171 |
+
stdout_logfile=/dev/stdout
|
| 172 |
+
stdout_logfile_maxbytes=0
|
| 173 |
+
stderr_logfile=/dev/stderr
|
| 174 |
+
stderr_logfile_maxbytes=0
|
| 175 |
+
|
| 176 |
[program:nginx]
|
| 177 |
command=/bin/sh -c "until curl -sf http://localhost:8000/health >/dev/null 2>&1 && curl -sf http://localhost:8010/docs >/dev/null 2>&1; do sleep 2; done; dl=false; case '%(ENV_DOWNLOAD_ENABLED)s' in [Tt]rue|1) if [ -f /app/hf-space-pipeline/.build/restore/hf-storage-api-catalog.yaml ]; then dl=true; else echo 'nginx: no hf-storage-api catalog, rendering download:false' >&2; fi;; esac; sed -e 's/__PORT__/%(ENV_PORT)s/' -e 's|__CORPUS_NAME__|%(ENV_CORPUS_NAME)s|g' -e 's/__DOWNLOAD_ENABLED__/'$dl'/' -e 's|__SUBMIT_CONTENT_URL__|%(ENV_SUBMIT_CONTENT_URL)s|g' /etc/nginx/hf-space.conf.template > /etc/nginx/conf.d/hf-space.conf && exec nginx -g 'daemon off;'"
|
| 178 |
user=root
|
docker/hf-space/supervisord.conf
CHANGED
|
@@ -159,6 +159,24 @@ stdout_logfile_maxbytes=0
|
|
| 159 |
stderr_logfile=/dev/stderr
|
| 160 |
stderr_logfile_maxbytes=0
|
| 161 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
[program:nginx]
|
| 163 |
; nginx is the *only* process bound to the externally-proxied port, and
|
| 164 |
; Hugging Face's own Space "ready" detection is just "does something answer
|
|
|
|
| 159 |
stderr_logfile=/dev/stderr
|
| 160 |
stderr_logfile_maxbytes=0
|
| 161 |
|
| 162 |
+
[program:deeptag-validate]
|
| 163 |
+
; SimReady Central's validator (docker/hf-space/deeptag-validate/app.py, vendored from
|
| 164 |
+
; simready-central deeptag-validate/, !582) answers /vlm_validate/search_result in place of
|
| 165 |
+
; deepsearch-api's: the same Yes/No log-prob judge on the same engine, plus each hit's path
|
| 166 |
+
; and caption in words-only checks (JUDGE_ASSET_CONTEXT) and an in-memory verdict cache
|
| 167 |
+
; (VALIDATE_CACHE_SIZE, ~120 MB at 500k entries). The engine settings are deepsearch-api's
|
| 168 |
+
; USDSEARCH_* variables, read when the shell execs (an unset one never stops supervisord).
|
| 169 |
+
command=/usr/bin/env -u PYTHONPATH /bin/sh -c 'RERANKER_BASE_URL="$USDSEARCH_LLM_BASE_URL" RERANKER_MODEL="$USDSEARCH_VISION_VALIDATION_MODEL" RERANKER_NAME="$USDSEARCH_VISION_VALIDATION_MODEL" RERANKER_API_KEY="$USDSEARCH_LLM_API_KEY" exec /app/.venv/bin/python -m uvicorn app:app --host 127.0.0.1 --port 8020 --log-level warning'
|
| 170 |
+
environment=JUDGE_ASSET_CONTEXT="path+caption",VALIDATE_CACHE_SIZE="500000",VALIDATE_MAX_EDGE="448",RERANKER_MAX_CONCURRENCY="32",RERANKER_BATCH_CONCURRENCY="16",RERANKER_COVER_MIN="0.2"
|
| 171 |
+
directory=/app/deeptag-validate
|
| 172 |
+
user=1000
|
| 173 |
+
autorestart=true
|
| 174 |
+
startsecs=3
|
| 175 |
+
stdout_logfile=/dev/stdout
|
| 176 |
+
stdout_logfile_maxbytes=0
|
| 177 |
+
stderr_logfile=/dev/stderr
|
| 178 |
+
stderr_logfile_maxbytes=0
|
| 179 |
+
|
| 180 |
[program:nginx]
|
| 181 |
; nginx is the *only* process bound to the externally-proxied port, and
|
| 182 |
; Hugging Face's own Space "ready" detection is just "does something answer
|