nspasojevic commited on
Commit
d2a67d7
Β·
verified Β·
1 Parent(s): 1382249

Validation through SimReady Central's validator: path + caption, verdict cache

Browse files

/vlm_validate/search_result now goes to SimReady Central's validator (docker/hf-space/deeptag-validate/app.py, vendored from simready-central !582; supervisord's deeptag-validate on 127.0.0.1:8020) on the same engine: each hit's path and caption in words-only checks and an in-memory verdict cache (500k entries).

Dockerfile CHANGED
@@ -284,6 +284,11 @@ COPY docker/hf-space/supervisord-gpu.conf /etc/supervisor/conf.d/hf-space.conf
284
 
285
  COPY docker/hf-space/nginx.conf.template /etc/nginx/hf-space.conf.template
286
 
 
 
 
 
 
287
  COPY services/explorer/dist /usr/share/nginx/explorer
288
 
289
 
 
284
 
285
  COPY docker/hf-space/nginx.conf.template /etc/nginx/hf-space.conf.template
286
 
287
+ # SimReady Central's validator (supervisord's [program:deeptag-validate], nginx's
288
+ # /vlm_validate/search_result): one vendored file, run by the app venv, which already
289
+ # has its fastapi, uvicorn, httpx, pydantic and pillow (deepsearch-api's own).
290
+ COPY docker/hf-space/deeptag-validate/app.py /app/deeptag-validate/app.py
291
+
292
  COPY services/explorer/dist /usr/share/nginx/explorer
293
 
294
 
docker/hf-space/deeptag-validate/app.py ADDED
@@ -0,0 +1,836 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2
+ # SPDX-License-Identifier: Apache-2.0
3
+ """
4
+ deeptag-validate β€” VLM relevance-validation shim.
5
+
6
+ Two routes, one scorer:
7
+
8
+ - `POST /vlm_validate/search_result` β€” the original per-image contract: the
9
+ browser uploads one base64 thumbnail per call (~100 requests per search).
10
+ - `POST /vlm_validate/batch_stream` β€” the streamed batch contract (task #27):
11
+ ONE request per result page carrying only ids + image references; the shim
12
+ fetches each thumbnail ITSELF through the gateway's `/images` (LAN hop,
13
+ shared 7-day nginx cache the browsers already warmed, index-scoped by the
14
+ request's `index` slug) and streams verdicts back as NDJSON lines the
15
+ moment each lands β€” shim-cache hits first, then misses dispatched in
16
+ REQUEST (retrieval) order. Client disconnect cancels every queued engine
17
+ call (the whole point of the per-request task set β€” an abandoned query
18
+ must not strand work on the shared engine).
19
+
20
+ Bridges the Explorer's `/vlm_validate/*` contract to the self-hosted
21
+ unified vlm-parser-reranker engine (Qwen3.6-35B-A3B-FP8 on one RTX PRO 6000, served alias
22
+ "reranker" β€” the same
23
+ engine also serves query parsing under the alias "parser"). The engine is used
24
+ as a relevance scorer via the first-token logprob trick: it reads the verdict
25
+ distribution from one forward pass (enable_thinking=false; Qwen thinks by
26
+ default, so a missed flag silently corrupts verdicts), ~20x faster than
27
+ generating a verdict. We return the SAME shape and scale as usdsearch's own
28
+ validator (llm_client's log-prob method), minus the `reasoning` the UI no
29
+ longer shows:
30
+
31
+ {"is_match": bool, "confidence": float, "similarity_score": float, "model": str}
32
+
33
+ ONE verdict method on every route, the Qwen Yes/No log-prob verdict: a Yes/No
34
+ prompt (JUDGE_PROMPT for text queries, JUDGE_IMAGE_PROMPT for an image alone,
35
+ JUDGE_IMAGE_TEXT_PROMPT for an image with words), P(Yes) read from the
36
+ first-token log-probs (position scan, Yes/No mass renormalized), is_match =
37
+ P(Yes) > 0.5, confidence = P(Yes) in [0,1] and similarity_score = P(Yes) x 100
38
+ in [0,100] β€” usdsearch's scale, so the Explorer reads every validator's score
39
+ the same way (2026-10-06: the shim sent P(Yes) in [0,1] until then, and the
40
+ Explorer's per-value guess showed a usdsearch reject at P(Yes) 0.0076 as 76 %).
41
+ With JUDGE_ASSET_CONTEXT on (2026-10-07), a words-only check also reads the
42
+ candidate's file path and the index's caption, between the query and the
43
+ question (_select_prompt); image checks never do.
44
+ is_match coincides with the 50 % badge boundary the Explorer hardcodes (a
45
+ false is_match renders the orange "VLM rejected, likely false positive" chip,
46
+ and searchOutcome.js's WEAK state means "no hit at or above 50%"). This is the
47
+ verdict the 2026-09-14/15 evaluations measured
48
+ (docs/decisions/2026-09-search-quality.md). The graded 0/1/2 method and the
49
+ Gemma-4 engine it targeted were evaluated on 2026-09-15 and rejected (user
50
+ decision).
51
+
52
+ Validator-contract semantics (the authoritative engine + client contracts
53
+ live in INTEGRATION.md of https://gitlab-master.nvidia.com/omniverse/deeptag/parser-reranker):
54
+ - ABSTAIN (Yes/No probability mass below threshold at every position) β†’ retry
55
+ the call once; a persistent abstain is returned as is_match=true with
56
+ abstain=true and confidence 0.0 β€” the candidate KEEPS its retrieval-order
57
+ place, unscored. An abstain must never be treated as "No": rejecting on an
58
+ unreliable read silently deletes good results.
59
+ - Transient failure (connect error / engine 5xx) β†’ back off and retry once,
60
+ then fail-soft: 503/502, which the Explorer reads as "VLM unavailable": it
61
+ shows the VLM-unavailable banner and stops validating (verdicts already
62
+ collected keep their badges), leaving search intact. Scores are never
63
+ fabricated.
64
+
65
+ Throughput: the serving owner's operational contract caps validation fan-out
66
+ at 24 in-flight requests (engine-side 42.9 verdicts/s at that fan-out,
67
+ measured on the earlier L40S engine; end-to-end the shim sustains less because it also fetches the
68
+ thumbnails; more in-flight degrades p95 and adds nothing β€” the engine also
69
+ serves query parsing), so this shim is tuned
70
+ client-side:
71
+ - ONE pooled keep-alive HTTP client (lifespan-managed) β€” no per-call TCP
72
+ handshake tax (~an RTT each on ~100 calls per search);
73
+ - images reach the engine at most VALIDATE_MAX_EDGE (448) px on the longest
74
+ edge (2026-10-07): Isaac 6.2 stores every SimReady thumbnail at 1024x1024,
75
+ 1,078 prompt tokens per verdict at that size against 250 at 448 (robots stay
76
+ 256/512 px; anything at or under the cap passes through untouched). A JPEG
77
+ decodes at a reduced DCT scale first (draft), so the resize costs ~3 ms of
78
+ CPU per 1024-px image. Measured in docs/decisions/2026-09-search-quality.md:
79
+ 65 vs 38 verdicts/s at saturation, 97 % of Yes/No verdicts unchanged;
80
+ - an asyncio.Semaphore caps in-flight reranker calls at the contract's
81
+ fan-out ceiling, converting bursts into queueing at the sweet spot;
82
+ - max_tokens 2: Qwen3.6 emits the verdict at position 0 (no wrapper tokens β€”
83
+ verified by probe); the position SCAN below is kept anyway, per the
84
+ integration contract, so a future wrapper-emitting model still extracts.
85
+
86
+ Environment:
87
+ RERANKER_BASE_URL OpenAI-compatible engine base URL
88
+ (default http://simready-central-prod-cluster.nvidia.com/vlm-parser-reranker/v1)
89
+ RERANKER_MODEL served model name (default "reranker")
90
+ RERANKER_API_KEY bearer key for an engine that wants one (default
91
+ none: the cluster engine is open). Sent on the
92
+ engine's chat calls only, never on an image fetch.
93
+ RERANKER_NAME display name in responses (default "Qwen3.6-35B-A3B-FP8";
94
+ the `model` field in every verdict)
95
+ RERANKER_COVER_MIN Yes/No verdict-token mass below this β†’ abstain
96
+ (default 0.2).
97
+ RERANKER_TIMEOUT_S per-call timeout (default 30, per the serving
98
+ contract; a 50-hit page drained in β‰ˆ1.2 s, 1.46 s p95, on the earlier L40S)
99
+ RERANKER_MAX_TOKENS decode steps per call (default 2; raise if a
100
+ future model buries the verdict deeper)
101
+ RERANKER_MAX_CONCURRENCY in-flight cap toward the engine (default 48 for
102
+ the RTX PRO 6000 engine with async scheduling and
103
+ two API servers: 3 clients x c48 = 120.9 verdicts/s
104
+ at p95 1.41 s; 24 was the L40-class ceiling)
105
+ VALIDATE_MAX_EDGE downscale images whose longest edge exceeds this
106
+ many pixels before sending (default 448 since
107
+ 2026-10-07; 4096 = no resize from 2026-09-21,
108
+ 512 before 2026-09-15). The verdict cache key
109
+ includes it: changing it starts the shim's cache
110
+ cold.
111
+ RERANKER_BATCH_CONCURRENCY per-batch fan-out cap INSIDE the global
112
+ semaphore (default 24) β€” two users' concurrent
113
+ batches interleave at the engine instead of one
114
+ queueing whole behind the other.
115
+ SHIM_GATEWAY_BASE_URL the gateway's public listener, for shim-side
116
+ thumbnail fetches (default http://gateway:8080).
117
+ IMAGES_TIMEOUT_S per-thumbnail fetch timeout on the dedicated
118
+ image client (default 5; slow thumbnails must
119
+ not starve the engine connection pool).
120
+ BATCH_MAX_ITEMS batch size ceiling (default 200; larger β†’ 413).
121
+ BATCH_HEARTBEAT_S blank-line heartbeat cadence on an otherwise
122
+ quiet stream. COUPLED CONSTANT β€” the gateway's
123
+ batch_stream proxy_read_timeout (60 s, hard-
124
+ coded) and the Explorer's stream stall guard
125
+ (45 s) are both sized off the default 10:
126
+ raising it past ~40 trips the client guard,
127
+ past 60 the gateway severs healthy streams.
128
+ VALIDATE_CACHE_SIZE verdict-cache entries (default 4000000, ~0.9 GB
129
+ at 240 bytes an entry; two entries per verdict;
130
+ 0 disables; 20000 until 2026-10-07).
131
+ Verdicts are deterministic (temperature 0), so one
132
+ (model, max-edge, query, image-bytes) tuple always
133
+ scores the same β€” caching makes repeat waves
134
+ (re-searches, pagination bounces, other USERS
135
+ asking the same query) free. Complements the
136
+ Explorer's per-browser verdict cache: this one is
137
+ shared across every client of this shim. Keyed on
138
+ the prompt too (a different query never
139
+ coalesces).
140
+
141
+ Tests (tests/test_scoring.py: the scorer, the payload builder, the verdict
142
+ contract and the cache keys as pure functions; tests/test_routes.py: both
143
+ routes through fastapi.testclient with the engine call and the image fetch
144
+ monkeypatched, i.e. the prompt selection, the two cache paths, the NDJSON line
145
+ protocol and the per-item vs fatal error split; no engine, no network), from
146
+ the repo root:
147
+
148
+ cd deeptag-validate && pip install -r requirements-dev.txt && python -m pytest -q
149
+ # or, with no install step:
150
+ uv run --with pytest --with fastapi --with httpx --with pillow pytest deeptag-validate/tests
151
+
152
+ CI runs the first form as the `test-deeptag-validate` job, which gates
153
+ `build-deeptag-validate` exactly like the simready-api suite gates its image.
154
+ """
155
+ import asyncio
156
+ import base64
157
+ import hashlib
158
+ import io
159
+ import json
160
+ import logging
161
+ import math
162
+ import os
163
+ import re
164
+ from collections import OrderedDict
165
+ from contextlib import asynccontextmanager
166
+ from typing import List, Optional
167
+
168
+ import httpx
169
+ from fastapi import FastAPI, HTTPException
170
+ from fastapi.responses import StreamingResponse
171
+ from PIL import Image
172
+ from pydantic import BaseModel
173
+
174
+ logger = logging.getLogger("deeptag-validate")
175
+
176
+ # The unified vlm-parser-reranker engine's "reranker" alias, OpenAI-compatible. Defaults to
177
+ # the cluster engine's ingress; override per deployment. It does not gate on an API key.
178
+ RERANKER_BASE_URL = os.environ.get("RERANKER_BASE_URL", "http://simready-central-prod-cluster.nvidia.com/vlm-parser-reranker/v1").rstrip("/")
179
+ RERANKER_MODEL = os.environ.get("RERANKER_MODEL", "reranker")
180
+ # A hosted engine (the HF Spaces' NVCF function / Inference Endpoint) wants a bearer key. It rides the
181
+ # engine's POSTs alone: the same pooled client also fetches http image URLs a request names, and a
182
+ # default header there would hand the key to any origin.
183
+ RERANKER_API_KEY = os.environ.get("RERANKER_API_KEY", "").strip()
184
+ _ENGINE_HEADERS = {"Authorization": f"Bearer {RERANKER_API_KEY}"} if RERANKER_API_KEY else None
185
+ RERANKER_NAME = os.environ.get("RERANKER_NAME", "Qwen3.6-35B-A3B-FP8")
186
+
187
+ # Yes/No verdict-token probability mass below this is unreliable β†’ abstain
188
+ # (treat as unscored, never as "No").
189
+ COVER_MIN = float(os.environ.get("RERANKER_COVER_MIN", "0.2"))
190
+ TIMEOUT_S = float(os.environ.get("RERANKER_TIMEOUT_S", "30"))
191
+ # Qwen3.6 emits the verdict at position 0; one spare decode step is plenty and
192
+ # more only burns shared engine compute (it also serves query parsing).
193
+ RERANKER_MAX_TOKENS = int(os.environ.get("RERANKER_MAX_TOKENS", "2"))
194
+ # The serving contract's fan-out ceiling: past 24 in-flight, p95 degrades and
195
+ # throughput gains nothing.
196
+ RERANKER_MAX_CONCURRENCY = int(os.environ.get("RERANKER_MAX_CONCURRENCY", "48"))
197
+ # Vision-token cost scales with pixels; 448 px (longest edge) is the one view
198
+ # the production Yes/No verdict was measured at β€” the same edge the server-side
199
+ # verdict stage judges (docs/operations.md "Measured quality and the ordering
200
+ # rule"). The cache key carries the edge, so changing it never re-serves a
201
+ # verdict judged at another size.
202
+ VALIDATE_MAX_EDGE = int(os.environ.get("VALIDATE_MAX_EDGE", "448"))
203
+
204
+ # The asset's own facts in the words-only judge prompt (2026-10-07): "off" (the
205
+ # 2026-09 prompt), "path" (the candidate's file path) or "path+caption" (+ the
206
+ # index's one-line description). The client sends each hit's `path` and
207
+ # `caption`; a hit without a path is judged without facts, and image queries
208
+ # never use them (only words-only judging was evaluated). Measured in
209
+ # docs/decisions/2026-09-search-quality.md.
210
+ ASSET_CONTEXT_MODES = ("off", "path", "path+caption")
211
+ JUDGE_ASSET_CONTEXT = os.environ.get("JUDGE_ASSET_CONTEXT", "off").strip().lower()
212
+ if JUDGE_ASSET_CONTEXT not in ASSET_CONTEXT_MODES:
213
+ logger.warning(
214
+ "JUDGE_ASSET_CONTEXT=%r is not one of %s: judging without asset facts",
215
+ JUDGE_ASSET_CONTEXT, ", ".join(ASSET_CONTEXT_MODES),
216
+ )
217
+ JUDGE_ASSET_CONTEXT = "off"
218
+ # Prompt-length bounds for the facts: a path keeps its tail (the file and its
219
+ # folders name the asset), a caption its head.
220
+ ASSET_PATH_MAX = 300
221
+ ASSET_CAPTION_MAX = 200
222
+
223
+ # Streamed batch validation (task #27). The per-batch cap nests INSIDE the
224
+ # global engine semaphore: a 50-item batch occupies at most this many of the
225
+ # 24 engine slots at once, so concurrent users' batches interleave fairly.
226
+ RERANKER_BATCH_CONCURRENCY = int(os.environ.get("RERANKER_BATCH_CONCURRENCY", "24"))
227
+ # Shim-side thumbnail fetches go through the gateway's public listener β€” the
228
+ # same /images route (and shared nginx cache) the browsers use, index-scoped
229
+ # by the x-usdsearch-storage-backend header. NEVER a client-supplied URL: the
230
+ # base is pinned here and references travel as urlencoded query params only.
231
+ SHIM_GATEWAY_BASE_URL = os.environ.get("SHIM_GATEWAY_BASE_URL", "http://gateway:8080").rstrip("/")
232
+ IMAGES_TIMEOUT_S = float(os.environ.get("IMAGES_TIMEOUT_S", "5"))
233
+ BATCH_MAX_ITEMS = int(os.environ.get("BATCH_MAX_ITEMS", "200"))
234
+ BATCH_HEARTBEAT_S = float(os.environ.get("BATCH_HEARTBEAT_S", "10"))
235
+
236
+ # Cross-user verdict cache: temperature-0 verdicts are pure functions of
237
+ # (model, resize edge, query, image bytes), so identical pairs β€” from ANY
238
+ # client β€” re-serve without an engine call. Insertion-order eviction. An entry
239
+ # costs ~240 bytes (measured on python:3.11), and each fresh verdict writes two
240
+ # (its reference key and its content key), so 4,000,000 entries hold 2,000,000
241
+ # verdicts in ~0.9 GB: about half a day at the capacity plan's Phase 1 load
242
+ # (2026-10-07; 20000 before was ~30 s of traffic at 600 checks/s). In-process
243
+ # only: every shim restart starts cold.
244
+ VALIDATE_CACHE_SIZE = int(os.environ.get("VALIDATE_CACHE_SIZE", "4000000"))
245
+ _verdict_cache: "OrderedDict[str, object]" = OrderedDict()
246
+
247
+
248
+ def _cache_key(query: str, image_url: str, query_image_digest: str = "") -> str:
249
+ """Content-keyed verdict-cache entry: the prompt, the PREPARED candidate
250
+ image and, for an image query, the query image's digest (every image query
251
+ shares one prompt, so without it two different query images would
252
+ coalesce)."""
253
+ h = hashlib.sha256()
254
+ for part in (RERANKER_MODEL, str(VALIDATE_MAX_EDGE), query, image_url, query_image_digest or ""):
255
+ h.update(part.encode())
256
+ h.update(b"\x00")
257
+ return h.hexdigest()
258
+
259
+
260
+ def _ref_key(prompt: str, query_image_digest: str, index: str, ref: str) -> str:
261
+ """Reference-keyed verdict-cache entry for the batch path: keyed on the
262
+ IMAGE REFERENCE (image_key/asset_url + index slug) instead of prepared
263
+ bytes, so a cache hit costs no thumbnail fetch at all. The content key
264
+ (`_cache_key`) is still written alongside on every fresh verdict, so the
265
+ two paths coalesce. The query image participates as a PRE-COMPUTED digest
266
+ (image queries share one prompt and must not collide across different
267
+ reference images; hashing the multi-MB data URL per item would stall the
268
+ event loop; the route hashes it once)."""
269
+ h = hashlib.sha256()
270
+ for part in (RERANKER_MODEL, str(VALIDATE_MAX_EDGE), "ref", prompt, query_image_digest or "", index or "", ref):
271
+ h.update(part.encode())
272
+ h.update(b"\x00")
273
+ return h.hexdigest()
274
+
275
+
276
+ def _cache_get(key: str):
277
+ p = _verdict_cache.get(key)
278
+ if p is not None:
279
+ _verdict_cache.move_to_end(key)
280
+ return p
281
+
282
+
283
+ def _cache_put(key: str, score) -> None:
284
+ if VALIDATE_CACHE_SIZE <= 0:
285
+ return
286
+ _verdict_cache[key] = score
287
+ _verdict_cache.move_to_end(key)
288
+ while len(_verdict_cache) > VALIDATE_CACHE_SIZE:
289
+ _verdict_cache.popitem(last=False)
290
+
291
+ # Firm, one-word judge prompt (a weak prompt makes GLM describe the image β†’ the
292
+ # logprob trick breaks). Text goes BEFORE the image in the message content.
293
+ JUDGE_PROMPT_HEAD = 'You are a visual relevance judge for a 3D-asset search. Query: "{query}". '
294
+ JUDGE_QUESTION = "Does the rendered asset match the query? Answer with exactly one word: Yes or No."
295
+ JUDGE_PROMPT = JUDGE_PROMPT_HEAD + JUDGE_QUESTION
296
+ # The asset facts (JUDGE_ASSET_CONTEXT), between the query and the question, as
297
+ # evaluated on 2026-10-07.
298
+ JUDGE_PATH_FACT = 'The candidate asset\'s file is "{path}". '
299
+ JUDGE_CAPTION_FACT = 'Catalog description: "{caption}". '
300
+ # An image alone (find-similar): usdsearch's own image-query question
301
+ # (llm_client LOGPROB_JUDGE_PROMPT_IMAGE_QUERY without its caption clause),
302
+ # word for word; the query image goes first. Without a dedicated prompt an
303
+ # image search would be scored against 'Query: ""' β€” pure noise. On 40 known
304
+ # pairs (2026-10-06) it matched the shim's earlier wording: every same-object
305
+ # pair Yes, every unrelated pair No.
306
+ JUDGE_IMAGE_PROMPT = (
307
+ "You are a visual relevance judge for a 3D-asset search. The first image is "
308
+ "the query image; the remaining images are rendered views of one asset. Does "
309
+ "the rendered asset match the query? Answer with exactly one word: Yes or No."
310
+ )
311
+ # An image WITH words: both count, the query image first. Not usdsearch's
312
+ # caption clause ('… the query image and the text query is "…"; the remaining
313
+ # images are rendered views of one asset'), which let the model judge the query
314
+ # image itself against the words: on the same 40 pairs it passed 3-4 unrelated
315
+ # candidates, this wording none (the words alone, the shim's earlier rule: 1).
316
+ JUDGE_IMAGE_TEXT_PROMPT = (
317
+ 'You are a visual relevance judge for a 3D-asset search. Query: "{query}", '
318
+ "together with the first image, which shows what the user is looking for. "
319
+ "The other image shows a candidate asset. Does the candidate asset match the "
320
+ "query? Answer with exactly one word: Yes or No."
321
+ )
322
+ YES_TOKENS = {"yes", "y", "true"}
323
+ NO_TOKENS = {"no", "n", "false"}
324
+
325
+ # One pooled keep-alive client for the process (created in lifespan): validation
326
+ # is ~100 calls per search against a remote engine, and a per-request client
327
+ # pays a fresh TCP handshake (an RTT) on every call.
328
+ _reranker_client: Optional[httpx.AsyncClient] = None
329
+ # Separate, short-timeout client for shim-side thumbnail fetches (batch path):
330
+ # 50 image GETs must never compete with engine calls for the reranker pool,
331
+ # and one slow origin must cost 5 s, not the engine's 30.
332
+ _images_client: Optional[httpx.AsyncClient] = None
333
+ # In-flight cap at the engine's measured saturation point β€” a burst beyond it
334
+ # queues here instead of doubling everyone's latency engine-side.
335
+ _reranker_sem = asyncio.Semaphore(RERANKER_MAX_CONCURRENCY)
336
+
337
+
338
+ @asynccontextmanager
339
+ async def _lifespan(app: FastAPI):
340
+ global _reranker_client, _images_client
341
+ _reranker_client = httpx.AsyncClient(
342
+ timeout=TIMEOUT_S,
343
+ limits=httpx.Limits(max_connections=64, max_keepalive_connections=64),
344
+ )
345
+ _images_client = httpx.AsyncClient(
346
+ # pool=None: waiting for a connection SLOT must queue, not error β€” a
347
+ # scalar timeout also bounds pool acquisition, and N concurrent
348
+ # batches Γ— 12 workers over a finite pool would otherwise fabricate
349
+ # PoolTimeout "image fetch failed" verdicts for perfectly good
350
+ # thumbnails under load.
351
+ timeout=httpx.Timeout(connect=IMAGES_TIMEOUT_S, read=IMAGES_TIMEOUT_S, write=IMAGES_TIMEOUT_S, pool=None),
352
+ limits=httpx.Limits(max_connections=64, max_keepalive_connections=64),
353
+ )
354
+ try:
355
+ yield
356
+ finally:
357
+ await _reranker_client.aclose()
358
+ _reranker_client = None
359
+ await _images_client.aclose()
360
+ _images_client = None
361
+
362
+
363
+ app = FastAPI(title="deeptag-validate", version="1.1.0", lifespan=_lifespan)
364
+
365
+
366
+ class ValidateRequest(BaseModel):
367
+ query_text: Optional[str] = None
368
+ query_image: Optional[str] = None
369
+ images: List[str] = []
370
+ # The asset's facts for words-only judging (JUDGE_ASSET_CONTEXT).
371
+ path: Optional[str] = None
372
+ caption: Optional[str] = None
373
+
374
+
375
+ def p_yes_from_logprobs(content):
376
+ """Scan token positions for the one carrying Yes/No mass (scan-mode is the
377
+ contract: robust to wrapper tokens some models emit before the verdict);
378
+ return (p_yes, cover) or (None, 0) if uncertain."""
379
+ for pos in content or []:
380
+ yes = no = 0.0
381
+ for entry in pos.get("top_logprobs", []) or []:
382
+ token = (entry.get("token") or "").strip().lower()
383
+ prob = math.exp(entry["logprob"])
384
+ if token in YES_TOKENS:
385
+ yes += prob
386
+ elif token in NO_TOKENS:
387
+ no += prob
388
+ if yes + no > COVER_MIN:
389
+ return yes / (yes + no), yes + no
390
+ return None, 0.0
391
+
392
+
393
+ def _shrink_bytes(raw: bytes, src: str, was_data_url: bool) -> str:
394
+ """Sync (thread-pooled) half of image prep: decode, downscale if the longest
395
+ edge exceeds VALIDATE_MAX_EDGE (LANCZOS, JPEG q90), return a data URL. An
396
+ already-small data URL passes through UNTOUCHED; an already-small fetched
397
+ image is re-encoded as a data URL so the engine doesn't re-fetch it."""
398
+ with Image.open(io.BytesIO(raw)) as img:
399
+ w, h = img.size
400
+ if max(w, h) <= VALIDATE_MAX_EDGE:
401
+ logger.debug("image %dx%d <= %dpx: pass-through", w, h, VALIDATE_MAX_EDGE)
402
+ if was_data_url:
403
+ return src
404
+ fmt = (img.format or "png").lower()
405
+ mime = "image/jpeg" if fmt in ("jpg", "jpeg") else f"image/{fmt}"
406
+ return f"data:{mime};base64," + base64.b64encode(raw).decode()
407
+ # A JPEG decodes straight at a reduced DCT scale (1/2 for a 1024-px
408
+ # thumbnail; never below the cap): about half the shim CPU of a full
409
+ # decode before the resize. Other formats ignore it.
410
+ img.draft(None, (VALIDATE_MAX_EDGE, VALIDATE_MAX_EDGE))
411
+ rgb = img.convert("RGB")
412
+ rgb.thumbnail((VALIDATE_MAX_EDGE, VALIDATE_MAX_EDGE), Image.LANCZOS)
413
+ buf = io.BytesIO()
414
+ rgb.save(buf, format="JPEG", quality=90)
415
+ logger.debug(
416
+ "image resized %dx%d -> %dx%d (%d -> %d bytes)",
417
+ w, h, rgb.width, rgb.height, len(raw), buf.tell(),
418
+ )
419
+ return "data:image/jpeg;base64," + base64.b64encode(buf.getvalue()).decode()
420
+
421
+
422
+ async def _prepare_image(src: str) -> str:
423
+ """Downscale-and-inline an image reference before it reaches the engine.
424
+ Defensive: anything unparseable is sent through untouched β€” a weird image
425
+ must degrade to the engine's own handling, never break validation."""
426
+ try:
427
+ if src.startswith("data:"):
428
+ _, _, b64 = src.partition(",")
429
+ raw = base64.b64decode(b64)
430
+ was_data_url = True
431
+ elif src.startswith(("http://", "https://")):
432
+ resp = await _reranker_client.get(src)
433
+ resp.raise_for_status()
434
+ raw = resp.content
435
+ was_data_url = False
436
+ else:
437
+ # Bare base64: the Explorer strips the data: prefix off its query
438
+ # image. Sent as is, the engine reads it as an image URL and 400s
439
+ # every image-only verdict; inline it as a data URL instead.
440
+ # Non-base64 strings still pass through (strict decode raises).
441
+ raw = base64.b64decode(src, validate=True)
442
+ was_data_url = False
443
+ return await asyncio.to_thread(_shrink_bytes, raw, src, was_data_url)
444
+ except Exception as exc: # noqa: BLE001 β€” image prep must never break validation
445
+ logger.warning("image prep failed (%s: %s); sending original", type(exc).__name__, exc)
446
+ return src
447
+
448
+
449
+ def _judge_payload(prompt: str, image_url: str, query_image_url: str = None) -> dict:
450
+ # Text before image(s) per the judge contract. For image queries the query
451
+ # image precedes the candidate (the image prompts' "first image" wording
452
+ # depends on this order). Plain sampling parameters only:
453
+ # no structured-outputs grammar (that belonged to the retired graded
454
+ # method), so an engine without a grammar backend serves this unchanged.
455
+ content = [{"type": "text", "text": prompt}]
456
+ if query_image_url:
457
+ content.append({"type": "image_url", "image_url": {"url": query_image_url}})
458
+ content.append({"type": "image_url", "image_url": {"url": image_url}})
459
+ return {
460
+ "model": RERANKER_MODEL,
461
+ "temperature": 0,
462
+ "max_tokens": RERANKER_MAX_TOKENS,
463
+ "logprobs": True,
464
+ "top_logprobs": 20,
465
+ "chat_template_kwargs": {"enable_thinking": False},
466
+ "messages": [{"role": "user", "content": content}],
467
+ }
468
+
469
+
470
+ async def _call_reranker(payload: dict) -> dict:
471
+ # Transient-failure semantics from the validator contract: back off and
472
+ # retry ONCE on connect errors / engine 5xx before failing soft. The
473
+ # semaphore holds in-flight calls at the engine's saturation point.
474
+ for attempt in (1, 2):
475
+ try:
476
+ async with _reranker_sem:
477
+ resp = await _reranker_client.post(
478
+ f"{RERANKER_BASE_URL}/chat/completions", json=payload, headers=_ENGINE_HEADERS
479
+ )
480
+ resp.raise_for_status()
481
+ return resp.json()
482
+ except (httpx.ConnectError, httpx.ConnectTimeout, httpx.ReadTimeout):
483
+ if attempt == 1:
484
+ await asyncio.sleep(0.25)
485
+ continue
486
+ # Engine down / slow β†’ fail-soft. The Explorer reads 503 as
487
+ # "VLM unavailable": banner, validation stops; search is unaffected.
488
+ raise HTTPException(status_code=503, detail="reranker unavailable")
489
+ except httpx.HTTPStatusError as exc:
490
+ if attempt == 1 and exc.response.status_code >= 500:
491
+ await asyncio.sleep(0.25)
492
+ continue
493
+ raise EngineRejected(exc.response.status_code, f"reranker error: {exc}")
494
+ except httpx.HTTPError as exc:
495
+ raise HTTPException(status_code=502, detail=f"reranker error: {exc}")
496
+
497
+
498
+ # The configuration refusal's message: fixed, never the engine's own error (it names the engine's URL).
499
+ IMAGE_QUERY_REFUSED = "the validation engine refused an image query (it may take one image per prompt)"
500
+
501
+
502
+ class EngineRejected(HTTPException):
503
+ """The engine ANSWERED with an error status (a 502 to the client, as before; the batch route keeps
504
+ treating it per item). `engine_status` lets the per-image route tell a refused request shape apart."""
505
+
506
+ def __init__(self, engine_status: int, detail: str):
507
+ super().__init__(status_code=502, detail=detail)
508
+ self.engine_status = engine_status
509
+
510
+
511
+ def _abstain_verdict() -> dict:
512
+ # Persistent abstain β†’ keep the candidate in retrieval order, unscored
513
+ # (is_match=true so the Explorer does not reject it; confidence 0.0 and
514
+ # abstain=true make the non-verdict explicit rather than a fabricated
515
+ # score).
516
+ return {"is_match": True, "confidence": 0.0, "similarity_score": 0.0, "abstain": True, "model": RERANKER_NAME}
517
+
518
+
519
+ def _verdict_from_score(score: float) -> dict:
520
+ """The ONE place a verdict dict is built from a cached or fresh SCORE
521
+ RECORD (the P(Yes) float), for both routes and both cache paths β€” a
522
+ boundary that lived in three copies would silently regress on cache hits.
523
+ similarity_score is P(Yes) x 100 (usdsearch's scale); confidence stays
524
+ P(Yes)."""
525
+ return {
526
+ "is_match": score > 0.5,
527
+ "confidence": round(score, 4),
528
+ "similarity_score": round(score * 100, 2),
529
+ "model": RERANKER_NAME,
530
+ }
531
+
532
+
533
+ def _extract_score(data: dict):
534
+ """Pull the P(Yes) score record out of one engine response, or None
535
+ (abstain)."""
536
+ content = ((data.get("choices") or [{}])[0].get("logprobs") or {}).get("content")
537
+ p_yes, _cover = p_yes_from_logprobs(content)
538
+ return p_yes
539
+
540
+
541
+ def _fact_text(value, limit: int, keep_tail: bool = False) -> str:
542
+ """One asset fact as prompt text: whitespace collapsed, double quotes made
543
+ single (the fact sits inside a quoted span), at most `limit` characters."""
544
+ text = " ".join(str(value or "").split()).replace('"', "'")
545
+ if len(text) > limit:
546
+ text = text[-limit:] if keep_tail else text[:limit]
547
+ return text
548
+
549
+
550
+ _ISAAC_ROOT = re.compile(r"/Isaac/[^/]+/Isaac/(.+)$")
551
+ # A Hugging Face dataset file (the hf-* indexes, the HF Spaces): below the dataset, and below
552
+ # `resolve|blob/<revision>/` where the key pins one, e.g.
553
+ # hf://huggingface.co/datasets/nvidia/simready-assets/resolve/2464ca32…/fixture_island_a01/simready_usd/x.usd
554
+ # β†’ fixture_island_a01/simready_usd/x.usd.
555
+ _HF_DATASET_ROOT = re.compile(
556
+ r"^(?:[a-z][a-z0-9+.-]*://)?huggingface\.co/datasets/[^/]+/[^/]+/(?:(?:resolve|blob)/[^/]+/)?(.+)$", re.I
557
+ )
558
+
559
+
560
+ def _asset_path(path) -> str:
561
+ """The path as the judge reads it: below the release-agnostic Isaac content
562
+ root (`Robots/Unitree/H2/H2.usda`, as evaluated), or below a Hugging Face
563
+ dataset's root, else without its leading slash."""
564
+ p = str(path or "").strip()
565
+ m = _ISAAC_ROOT.search(p) or _HF_DATASET_ROOT.match(p)
566
+ return _fact_text(m.group(1) if m else p.lstrip("/"), ASSET_PATH_MAX, keep_tail=True)
567
+
568
+
569
+ def _asset_facts(path, caption) -> str:
570
+ """The words-only prompt's asset facts under JUDGE_ASSET_CONTEXT, or "":
571
+ the file path, then (path+caption) the caption. No path β†’ no facts, so
572
+ only the evaluated combinations are ever sent."""
573
+ if JUDGE_ASSET_CONTEXT == "off":
574
+ return ""
575
+ p = _asset_path(path)
576
+ if not p:
577
+ return ""
578
+ facts = JUDGE_PATH_FACT.format(path=p)
579
+ caption_text = _fact_text(caption, ASSET_CAPTION_MAX) if JUDGE_ASSET_CONTEXT == "path+caption" else ""
580
+ if caption_text:
581
+ facts += JUDGE_CAPTION_FACT.format(caption=caption_text)
582
+ return facts
583
+
584
+
585
+ def _select_prompt(query_text: str, has_query_image: bool, path=None, caption=None) -> str:
586
+ """The judge prompt for one candidate, over the stripped query text: an
587
+ image with words β†’ JUDGE_IMAGE_TEXT_PROMPT, an image alone β†’
588
+ JUDGE_IMAGE_PROMPT, words alone β†’ JUDGE_PROMPT, with the candidate's facts
589
+ before the question when JUDGE_ASSET_CONTEXT is on (a request with neither
590
+ words nor image keeps the historical 'Query: ""' prompt)."""
591
+ text = (query_text or "").strip()
592
+ if has_query_image:
593
+ return JUDGE_IMAGE_TEXT_PROMPT.format(query=text) if text else JUDGE_IMAGE_PROMPT
594
+ facts = _asset_facts(path, caption)
595
+ if not facts:
596
+ return JUDGE_PROMPT.format(query=text)
597
+ return JUDGE_PROMPT_HEAD.format(query=text) + facts + JUDGE_QUESTION
598
+
599
+
600
+ def _query_image_digest(query_image: Optional[str]) -> str:
601
+ """Hash the (possibly multi-MB) query image ONCE per request β€” the cache
602
+ keys run per item, and per-item sha256 over a full data URL would stall
603
+ the loop."""
604
+ return hashlib.sha256(query_image.encode()).hexdigest() if query_image else ""
605
+
606
+
607
+ async def _score_prepared(prompt: str, image_url: str, query_image_url: str = None):
608
+ """One verdict for a PREPARED (downscaled, inlined) image. Returns
609
+ (score | None, verdict_dict); score None means persistent abstain (the
610
+ dict already carries abstain=true; the CALLER decides caching, and
611
+ abstains are never cached). The score is the cache record (see
612
+ _verdict_from_score). Raises HTTPException 502/503 when the engine is
613
+ unreachable (fail-soft to the Explorer's "VLM unavailable")."""
614
+ data = await _call_reranker(_judge_payload(prompt, image_url, query_image_url))
615
+ score = _extract_score(data)
616
+ if score is None:
617
+ # Abstain semantics: retry once β€” never treat an unreliable read as "No".
618
+ data = await _call_reranker(_judge_payload(prompt, image_url, query_image_url))
619
+ score = _extract_score(data)
620
+ if score is None:
621
+ return None, _abstain_verdict()
622
+ return score, _verdict_from_score(score)
623
+
624
+
625
+ @app.get("/health")
626
+ def health():
627
+ return {"status": "ok", "reranker": RERANKER_BASE_URL, "model": RERANKER_MODEL, "asset_context": JUDGE_ASSET_CONTEXT}
628
+
629
+
630
+ @app.post("/vlm_validate/search_result")
631
+ async def validate(req: ValidateRequest):
632
+ if not req.images:
633
+ raise HTTPException(status_code=400, detail="no image provided")
634
+ # The same judge as the batch route: a query image (the Explorer sends it
635
+ # bare base64 when it finishes a cut batch stream here) goes to the engine
636
+ # first, under the image prompts.
637
+ prompt = _select_prompt(req.query_text, bool(req.query_image), req.path, req.caption)
638
+ query_image_url = await _prepare_image(req.query_image) if req.query_image else None
639
+ image_url = await _prepare_image(req.images[0])
640
+ # Serve deterministic repeats from the verdict cache β€” no engine call. The
641
+ # key hashes the PREPARED image (post-resize), so the same asset arriving
642
+ # as a data URL or a fetched http URL coalesces once prepared.
643
+ ckey = _cache_key(prompt, image_url, _query_image_digest(req.query_image))
644
+ cached = _cache_get(ckey) if VALIDATE_CACHE_SIZE > 0 else None
645
+ if cached is not None:
646
+ return {**_verdict_from_score(cached), "cached": True}
647
+ try:
648
+ score, verdict = await _score_prepared(prompt, image_url, query_image_url)
649
+ except EngineRejected as exc:
650
+ # An engine that takes one image per prompt (the dev HF Space's NVCF function) answers an image
651
+ # query (query image + candidate) with a 4xx. Nothing is down: answer usdsearch's configuration
652
+ # refusal, which the Space's Explorer reads as "this search goes unvalidated" instead of the
653
+ # VLM-unavailable banner. Central's Explorer reads any 503 as before.
654
+ if query_image_url and 400 <= exc.engine_status < 500 and exc.engine_status not in (408, 429):
655
+ raise HTTPException(
656
+ status_code=503,
657
+ detail={"error_type": "configuration", "retryable": False, "message": IMAGE_QUERY_REFUSED},
658
+ ) from exc
659
+ raise
660
+ if score is not None:
661
+ _cache_put(ckey, score)
662
+ return verdict
663
+
664
+
665
+ class BatchItem(BaseModel):
666
+ id: str
667
+ image_key: Optional[str] = None
668
+ asset_url: Optional[str] = None
669
+ # The hit's source.path and the index's caption for it: facts for
670
+ # words-only judging (JUDGE_ASSET_CONTEXT).
671
+ path: Optional[str] = None
672
+ caption: Optional[str] = None
673
+
674
+
675
+ class BatchValidateRequest(BaseModel):
676
+ query_text: Optional[str] = None
677
+ query_image: Optional[str] = None
678
+ # The index SLUG (x-usdsearch-storage-backend value) the shim forwards on
679
+ # its /images fetches β€” routing, api-key injection, and the shared image
680
+ # cache all key off it upstream. Empty β†’ the gateway's default index.
681
+ index: Optional[str] = None
682
+ # In the client's DISPLAY (retrieval) order β€” dispatch preserves it, so
683
+ # the most-likely matches score first and the ranked flow fills top-down.
684
+ items: List[BatchItem] = []
685
+
686
+
687
+ def _ndjson(obj) -> str:
688
+ return json.dumps(obj, separators=(",", ":")) + "\n"
689
+
690
+
691
+ @app.post("/vlm_validate/batch_stream")
692
+ async def validate_batch(req: BatchValidateRequest):
693
+ """Streamed batch validation (NDJSON). Line protocol:
694
+
695
+ {"id", "cached", "verdict": {is_match, confidence, similarity_score,
696
+ model[, abstain]}} β€” one per scored hit
697
+ {"id", "error": "..."} β€” per-hit failure (bad thumbnail etc.);
698
+ NEVER poisons the rest of the batch
699
+ "" β€” blank heartbeat every BATCH_HEARTBEAT_S
700
+ {"fatal": "..."} β€” engine unreachable; treat as 502/503
701
+ {"done": {completed, errored, cached}} β€” ALWAYS last on a clean finish;
702
+ its absence means the stream was cut
703
+ (the client re-POSTs the missing ids)
704
+
705
+ Shim-cache hits flush first (no engine, no image fetch β€” the reference-
706
+ keyed entries); misses dispatch in request order under the per-batch cap.
707
+ Client disconnect cancels every pending worker; `async with _reranker_sem`
708
+ releases on cancellation, so an abandoned query frees its engine slots."""
709
+ if not req.items:
710
+ raise HTTPException(status_code=400, detail="no items")
711
+ if len(req.items) > BATCH_MAX_ITEMS:
712
+ raise HTTPException(status_code=413, detail=f"batch exceeds {BATCH_MAX_ITEMS} items")
713
+ if not (req.query_text or req.query_image):
714
+ raise HTTPException(status_code=400, detail="no query")
715
+ index = (req.index or "").strip()
716
+ qimg_digest = _query_image_digest(req.query_image)
717
+ # An image query (alone or with words): the judge sees the query image
718
+ # first, then each candidate. Prepared (downscaled) once. With asset facts
719
+ # on, the prompt (and so every cache key) is per item.
720
+ def prompt_for(it: BatchItem) -> str:
721
+ return _select_prompt(req.query_text, bool(req.query_image), it.path, it.caption)
722
+
723
+ query_image_url = await _prepare_image(req.query_image) if req.query_image else None
724
+
725
+ # Phase 0 β€” reference-keyed cache pass: free verdicts, emitted first.
726
+ cached_lines = [] # (kind, line) β€” kind: "verdict" | "error"
727
+ misses = []
728
+ for it in req.items:
729
+ ref = it.image_key or it.asset_url
730
+ if not ref:
731
+ cached_lines.append(("error", _ndjson({"id": it.id, "error": "no image reference"})))
732
+ continue
733
+ cached = _cache_get(_ref_key(prompt_for(it), qimg_digest, index, ref)) if VALIDATE_CACHE_SIZE > 0 else None
734
+ if cached is not None:
735
+ cached_lines.append(
736
+ ("verdict", _ndjson({
737
+ "id": it.id,
738
+ "cached": True,
739
+ "verdict": _verdict_from_score(cached),
740
+ }))
741
+ )
742
+ else:
743
+ misses.append(it)
744
+
745
+ queue: asyncio.Queue = asyncio.Queue()
746
+ batch_sem = asyncio.Semaphore(RERANKER_BATCH_CONCURRENCY)
747
+
748
+ async def worker(it: BatchItem):
749
+ async with batch_sem:
750
+ ref = it.image_key or it.asset_url
751
+ prompt = prompt_for(it)
752
+ # Pinned base + urlencoded params ONLY β€” a reference can never
753
+ # change the request target (no SSRF surface); image_key (a
754
+ # specific matched view) wins over asset_url, mirroring the
755
+ # Explorer's own /images precedence.
756
+ params = {"image_key": it.image_key} if it.image_key else {"asset_url": it.asset_url}
757
+ headers = {"x-usdsearch-storage-backend": index} if index else {}
758
+ try:
759
+ resp = await _images_client.get(f"{SHIM_GATEWAY_BASE_URL}/images", params=params, headers=headers)
760
+ resp.raise_for_status()
761
+ raw = resp.content
762
+ except Exception as exc: # noqa: BLE001 β€” one bad thumbnail must not poison the batch
763
+ await queue.put(("error", _ndjson({"id": it.id, "error": f"image fetch failed: {type(exc).__name__}"})))
764
+ return
765
+ try:
766
+ # Raw bytes straight into the thread-pooled shrink: no base64
767
+ # round-trip on the loop, and the data URL's mime comes from
768
+ # PIL's own sniff (img.format), never the upstream's
769
+ # content-type header (S3-ish origins say octet-stream, which
770
+ # OpenAI-compatible engines 400 on).
771
+ image_url = await asyncio.to_thread(_shrink_bytes, raw, "", False)
772
+ except Exception as exc: # noqa: BLE001 β€” undecodable bytes must not reach the engine
773
+ await queue.put(("error", _ndjson({"id": it.id, "error": f"image decode failed: {type(exc).__name__}"})))
774
+ return
775
+ try:
776
+ score, verdict = await _score_prepared(prompt, image_url, query_image_url)
777
+ except HTTPException as exc:
778
+ if exc.status_code == 503:
779
+ # Engine UNREACHABLE β€” the whole batch is over.
780
+ await queue.put(("fatal", _ndjson({"fatal": exc.detail})))
781
+ else:
782
+ # Engine rejected THIS item (4xx→502: e.g. an image it
783
+ # can't decode) β€” per-hit, never batch-fatal.
784
+ await queue.put(("error", _ndjson({"id": it.id, "error": f"engine rejected item: {exc.detail}"})))
785
+ return
786
+ except Exception as exc: # noqa: BLE001 β€” a dead worker with no line would
787
+ # strand `remaining` and heartbeat the stream FOREVER (the
788
+ # blank lines reset every downstream timeout). Anything
789
+ # unexpected (non-JSON 200 from a proxy, logprob shape drift)
790
+ # settles the hit as a per-hit error instead.
791
+ logger.warning("batch worker failed (%s: %s)", type(exc).__name__, exc)
792
+ await queue.put(("error", _ndjson({"id": it.id, "error": f"scoring failed: {type(exc).__name__}"})))
793
+ return
794
+ if score is not None:
795
+ _cache_put(_ref_key(prompt, qimg_digest, index, ref), score)
796
+ _cache_put(_cache_key(prompt, image_url, qimg_digest), score)
797
+ await queue.put(("verdict", _ndjson({"id": it.id, "cached": False, "verdict": verdict})))
798
+
799
+ async def stream():
800
+ completed = sum(1 for kind, _ in cached_lines if kind == "verdict")
801
+ errored = len(cached_lines) - completed
802
+ cached_count = completed
803
+ # Workers are created INSIDE the generator: a client that aborts
804
+ # before the body is ever iterated (POST β†’ immediate abort) must not
805
+ # leak a batch onto the shared engine β€” a never-started generator
806
+ # never runs its finally, so task lifetime has to live in this frame.
807
+ # Dispatch in REQUEST order β€” the per-batch semaphore wakes FIFO, so
808
+ # completion approximates retrieval order and the flow's top fills
809
+ # first.
810
+ tasks = [asyncio.create_task(worker(it)) for it in misses]
811
+ try:
812
+ for _, ln in cached_lines:
813
+ yield ln
814
+ remaining = len(tasks)
815
+ while remaining > 0:
816
+ try:
817
+ kind, ln = await asyncio.wait_for(queue.get(), timeout=BATCH_HEARTBEAT_S)
818
+ except asyncio.TimeoutError:
819
+ yield "\n" # heartbeat β€” keeps proxies from severing a quiet stream
820
+ continue
821
+ yield ln
822
+ if kind == "fatal":
823
+ return # finally cancels the rest; no `done` line on a fatal
824
+ remaining -= 1
825
+ if kind == "error":
826
+ errored += 1
827
+ else:
828
+ completed += 1
829
+ yield _ndjson({"done": {"completed": completed, "errored": errored, "cached": cached_count}})
830
+ finally:
831
+ # Client gone, fatal, or clean finish β€” nothing may keep burning
832
+ # shared engine slots. Cancellation releases the semaphores.
833
+ for t in tasks:
834
+ t.cancel()
835
+
836
+ return StreamingResponse(stream(), media_type="application/x-ndjson")
docker/hf-space/nginx.conf.template CHANGED
@@ -73,10 +73,11 @@ server {
73
  proxy_pass http://127.0.0.1:8000;
74
  }
75
 
 
 
76
  location /vlm_validate/search_result {
77
  if ($request_method = OPTIONS) { add_header Content-Length 0 always; add_header Content-Type "text/plain" always; return 204; }
78
- rewrite ^/vlm_validate/search_result(.*)$ /v3/deepsearch/vlm_validate/search_result$1 break;
79
- proxy_pass http://127.0.0.1:8000;
80
  }
81
 
82
  location /llm_parse/ {
 
73
  proxy_pass http://127.0.0.1:8000;
74
  }
75
 
76
+ # SimReady Central's validator (supervisord's deeptag-validate, 127.0.0.1:8020) answers here in
77
+ # place of deepsearch-api's: the same judge plus each hit's path and caption and a verdict cache.
78
  location /vlm_validate/search_result {
79
  if ($request_method = OPTIONS) { add_header Content-Length 0 always; add_header Content-Type "text/plain" always; return 204; }
80
+ proxy_pass http://127.0.0.1:8020;
 
81
  }
82
 
83
  location /llm_parse/ {
docker/hf-space/supervisord-gpu.conf CHANGED
@@ -155,6 +155,24 @@ stdout_logfile_maxbytes=0
155
  stderr_logfile=/dev/stderr
156
  stderr_logfile_maxbytes=0
157
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
158
  [program:nginx]
159
  command=/bin/sh -c "until curl -sf http://localhost:8000/health >/dev/null 2>&1 && curl -sf http://localhost:8010/docs >/dev/null 2>&1; do sleep 2; done; dl=false; case '%(ENV_DOWNLOAD_ENABLED)s' in [Tt]rue|1) if [ -f /app/hf-space-pipeline/.build/restore/hf-storage-api-catalog.yaml ]; then dl=true; else echo 'nginx: no hf-storage-api catalog, rendering download:false' >&2; fi;; esac; sed -e 's/__PORT__/%(ENV_PORT)s/' -e 's|__CORPUS_NAME__|%(ENV_CORPUS_NAME)s|g' -e 's/__DOWNLOAD_ENABLED__/'$dl'/' -e 's|__SUBMIT_CONTENT_URL__|%(ENV_SUBMIT_CONTENT_URL)s|g' /etc/nginx/hf-space.conf.template > /etc/nginx/conf.d/hf-space.conf && exec nginx -g 'daemon off;'"
160
  user=root
 
155
  stderr_logfile=/dev/stderr
156
  stderr_logfile_maxbytes=0
157
 
158
+ [program:deeptag-validate]
159
+ ; SimReady Central's validator (docker/hf-space/deeptag-validate/app.py, vendored from
160
+ ; simready-central deeptag-validate/, !582) answers /vlm_validate/search_result in place of
161
+ ; deepsearch-api's: the same Yes/No log-prob judge on the same engine, plus each hit's path
162
+ ; and caption in words-only checks (JUDGE_ASSET_CONTEXT) and an in-memory verdict cache
163
+ ; (VALIDATE_CACHE_SIZE, ~120 MB at 500k entries). The engine settings are deepsearch-api's
164
+ ; USDSEARCH_* variables, read when the shell execs (an unset one never stops supervisord).
165
+ command=/usr/bin/env -u PYTHONPATH /bin/sh -c 'RERANKER_BASE_URL="$USDSEARCH_LLM_BASE_URL" RERANKER_MODEL="$USDSEARCH_VISION_VALIDATION_MODEL" RERANKER_NAME="$USDSEARCH_VISION_VALIDATION_MODEL" RERANKER_API_KEY="$USDSEARCH_LLM_API_KEY" exec /app/.venv/bin/python -m uvicorn app:app --host 127.0.0.1 --port 8020 --log-level warning'
166
+ environment=JUDGE_ASSET_CONTEXT="path+caption",VALIDATE_CACHE_SIZE="500000",VALIDATE_MAX_EDGE="448",RERANKER_MAX_CONCURRENCY="32",RERANKER_BATCH_CONCURRENCY="16",RERANKER_COVER_MIN="0.2"
167
+ directory=/app/deeptag-validate
168
+ user=1000
169
+ autorestart=true
170
+ startsecs=3
171
+ stdout_logfile=/dev/stdout
172
+ stdout_logfile_maxbytes=0
173
+ stderr_logfile=/dev/stderr
174
+ stderr_logfile_maxbytes=0
175
+
176
  [program:nginx]
177
  command=/bin/sh -c "until curl -sf http://localhost:8000/health >/dev/null 2>&1 && curl -sf http://localhost:8010/docs >/dev/null 2>&1; do sleep 2; done; dl=false; case '%(ENV_DOWNLOAD_ENABLED)s' in [Tt]rue|1) if [ -f /app/hf-space-pipeline/.build/restore/hf-storage-api-catalog.yaml ]; then dl=true; else echo 'nginx: no hf-storage-api catalog, rendering download:false' >&2; fi;; esac; sed -e 's/__PORT__/%(ENV_PORT)s/' -e 's|__CORPUS_NAME__|%(ENV_CORPUS_NAME)s|g' -e 's/__DOWNLOAD_ENABLED__/'$dl'/' -e 's|__SUBMIT_CONTENT_URL__|%(ENV_SUBMIT_CONTENT_URL)s|g' /etc/nginx/hf-space.conf.template > /etc/nginx/conf.d/hf-space.conf && exec nginx -g 'daemon off;'"
178
  user=root
docker/hf-space/supervisord.conf CHANGED
@@ -159,6 +159,24 @@ stdout_logfile_maxbytes=0
159
  stderr_logfile=/dev/stderr
160
  stderr_logfile_maxbytes=0
161
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
162
  [program:nginx]
163
  ; nginx is the *only* process bound to the externally-proxied port, and
164
  ; Hugging Face's own Space "ready" detection is just "does something answer
 
159
  stderr_logfile=/dev/stderr
160
  stderr_logfile_maxbytes=0
161
 
162
+ [program:deeptag-validate]
163
+ ; SimReady Central's validator (docker/hf-space/deeptag-validate/app.py, vendored from
164
+ ; simready-central deeptag-validate/, !582) answers /vlm_validate/search_result in place of
165
+ ; deepsearch-api's: the same Yes/No log-prob judge on the same engine, plus each hit's path
166
+ ; and caption in words-only checks (JUDGE_ASSET_CONTEXT) and an in-memory verdict cache
167
+ ; (VALIDATE_CACHE_SIZE, ~120 MB at 500k entries). The engine settings are deepsearch-api's
168
+ ; USDSEARCH_* variables, read when the shell execs (an unset one never stops supervisord).
169
+ command=/usr/bin/env -u PYTHONPATH /bin/sh -c 'RERANKER_BASE_URL="$USDSEARCH_LLM_BASE_URL" RERANKER_MODEL="$USDSEARCH_VISION_VALIDATION_MODEL" RERANKER_NAME="$USDSEARCH_VISION_VALIDATION_MODEL" RERANKER_API_KEY="$USDSEARCH_LLM_API_KEY" exec /app/.venv/bin/python -m uvicorn app:app --host 127.0.0.1 --port 8020 --log-level warning'
170
+ environment=JUDGE_ASSET_CONTEXT="path+caption",VALIDATE_CACHE_SIZE="500000",VALIDATE_MAX_EDGE="448",RERANKER_MAX_CONCURRENCY="32",RERANKER_BATCH_CONCURRENCY="16",RERANKER_COVER_MIN="0.2"
171
+ directory=/app/deeptag-validate
172
+ user=1000
173
+ autorestart=true
174
+ startsecs=3
175
+ stdout_logfile=/dev/stdout
176
+ stdout_logfile_maxbytes=0
177
+ stderr_logfile=/dev/stderr
178
+ stderr_logfile_maxbytes=0
179
+
180
  [program:nginx]
181
  ; nginx is the *only* process bound to the externally-proxied port, and
182
  ; Hugging Face's own Space "ready" detection is just "does something answer