FanLR commited on
Commit
019a613
·
verified ·
1 Parent(s): ec79b08

Restore original English demo UI exactly

Browse files

Restore the original app.py, English presentation helpers and console.js. Preserve the original purple/lime palette, fonts, sidebar and canvas layout. Backend remains the pinned GroundingPI/GroundingPI model on FanLR ZeroGPU. Local 51 tests pass. Live backend target detection completed with 29 targets, 281 output tokens and 13.47 seconds.

README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- title: GroundingPI 视觉感知工作台
3
  emoji: 🎯
4
  colorFrom: purple
5
  colorTo: green
@@ -12,15 +12,13 @@ models:
12
  - GroundingPI/GroundingPI
13
  ---
14
 
15
- # GroundingPI · 视觉感知工作台
16
 
17
- 沿用原版紫色与黄绿色配色、浅紫背景和双栏布局的中文视觉感知 demo。支持目标检测、密集检测、描述定位、三类点定位、视觉提示、OCR 与 GUI 定位,共 9 类任务、22 张原始案例。
18
 
19
- 模型固定为公开的 `GroundingPI/GroundingPI`,修订版本为 `127b0e1c54c68f24bf769a80067521b815b695dd`。保留模型原生处理器、聊天模板和 GAM 输出协议,中文 OCR 标签使用随项目打包的 Noto Sans CJK。
20
 
21
- 页面支持图像上传、案例切换、画布缩放、结果聚焦、逐实例查看、原图局部放大、图片下载、结构化坐标与原始输出。视频入口保留为不可选状态。案例提示词和坐标协议与原 demo 一致。
22
-
23
- ## 本地预览
24
 
25
  ```bash
26
  python3 -m venv .venv
@@ -28,17 +26,13 @@ python3 -m venv .venv
28
  GROUNDING_BACKEND=http DEMO_PORT=7862 .venv/bin/python space_app.py
29
  ```
30
 
31
- 预览默认 HTTP 地址为 `http://127.0.0.1:8000/v1`。未启动模型服务时,页面明确显示“推理未连接”,可以浏览图像与案例,不会展示伪造预测。已有推理服务可通过 `GROUNDING_API_BASE` 和 `GROUNDING_MODEL` 配置。
32
-
33
- ## ZeroGPU 部署
34
-
35
- 上传本目录内容到已有的 Gradio / ZeroGPU Space。本次目标为 `FanLR/GroundingPI-4B`,使用该账号已有的算力。模型仓库公开,不需要新建访问令牌。
36
 
37
- `space_app.py` 固定加载上述模型版本,并替换原 Space 的旧模型变量。不要同时设置本地模型目录与远程模型仓库。
38
 
39
- Static Space 无法在服务器加载模型。没有 ZeroGPU 权限的新账号可以发布前端,但完整 Gradio 应用需要计算后端。本次直接复用原 FanLR Space,避免跨账号访问私有服务的问题。
40
 
41
- ## 验证
42
 
43
  ```bash
44
  .venv/bin/python -m unittest discover -s tests -v
@@ -46,4 +40,4 @@ Static Space 无法在服务器加载模型。没有 ZeroGPU 权限的新账号
46
  .venv/bin/python scripts/build_bundle.py /absolute/path/studio.zip
47
  ```
48
 
49
- 本地验证与线上实际 GPU 推理分开记录。尚未完成的线上检查不会标注为通过。图片、模型及字体分别遵循其来源许可;见 `assets/SOURCES.md` 与模型仓库许可。
 
1
  ---
2
+ title: GroundingPI Perception Studio
3
  emoji: 🎯
4
  colorFrom: purple
5
  colorTo: green
 
12
  - GroundingPI/GroundingPI
13
  ---
14
 
15
+ # GroundingPI · Perception Studio
16
 
17
+ The original English demo, with its purple and lime palette, fonts, layout, nine task routes and 22 image examples. Supports image upload, canvas zoom, target focus, original-pixel detail, downloads and structured outputs. Video remains disabled.
18
 
19
+ The model is `GroundingPI/GroundingPI`, pinned at revision `127b0e1c54c68f24bf769a80067521b815b695dd`. Native processors, chat templates and GAM coordinates are preserved. Chinese OCR labels use the bundled Noto Sans CJK font.
20
 
21
+ ## Local preview
 
 
22
 
23
  ```bash
24
  python3 -m venv .venv
 
26
  GROUNDING_BACKEND=http DEMO_PORT=7862 .venv/bin/python space_app.py
27
  ```
28
 
29
+ The preview uses `http://127.0.0.1:8000/v1` by default. Without a model service, it reports inference offline and allows browsing examples. It does not substitute stored predictions.
 
 
 
 
30
 
31
+ ## ZeroGPU
32
 
33
+ Deploy this directory to the existing `FanLR/GroundingPI-4B` Gradio Space. The Space and GPU backend belong to FanLR; model weights come from the public GroundingPI account repository. `space_app.py` replaces the old Space model variables with this pinned release. Do not configure both a local model path and a remote model repository.
34
 
35
+ ## Verification
36
 
37
  ```bash
38
  .venv/bin/python -m unittest discover -s tests -v
 
40
  .venv/bin/python scripts/build_bundle.py /absolute/path/studio.zip
41
  ```
42
 
43
+ Local tests and CPU package validation are recorded separately from live GPU inference. Images, fonts and model weights retain their source licenses; see `assets/SOURCES.md` and the model repository.
app.py CHANGED
@@ -17,15 +17,15 @@ from grounding_demo.protocol import parse_gam, parse_reference_boxes
17
  from grounding_demo.render import draw_primitives
18
 
19
  ROUTE_META = {
20
- "grounding": ("目标检测", "开放词汇目标定位"),
21
- "dense": ("密集检测", "密集场景实例定位"),
22
- "referring": ("描述定位", "根据语言描述定位目标"),
23
- "dense_point": ("密集点定位", "定位多个目标的中心点"),
24
- "referring_point_in_mask": ("描述内点", "在描述目标内部预测位置"),
25
- "dense_point_in_mask": ("密集内点", "在多个目标内部预测位置"),
26
- "visual_prompt": ("视觉提示", "以参考区域寻找相似目标"),
27
- "ocr": ("文字识别 OCR", "识别文字并定位文本区域"),
28
- "gui": ("界面定位 GUI", "将操作意图映射到界面元素"),
29
  }
30
  CASES = load_cases()
31
  BY_ID = {row["id"]: row for row in CASES}
@@ -62,10 +62,10 @@ def query_update(canonical, route):
62
 
63
  def case_meta(row):
64
  return (
65
- "<div class='source-meta'><span>样例来源</span>"
66
- f"<b>{html.escape(row['source'])}</b><span>图像校验值</span>"
67
  f"<code>{row.get('image_sha256', '')[:16]}</code>"
68
- "<small>精选案例用于体验不同任务。模型仅接收当前图像与任务提示词。</small></div>"
69
  )
70
 
71
 
@@ -90,7 +90,7 @@ def reset_image(path, case_id, route, canonical, meta):
90
  targets_html([]),
91
  "[]",
92
  "",
93
- status_html("图像已就绪 · 点击开始解析"),
94
  task_help_html(route),
95
  )
96
 
@@ -124,7 +124,7 @@ def choose_gallery(route, evt: gr.SelectData):
124
 
125
  def upload_image(path, route, canonical):
126
  if not path:
127
- raise gr.Error("请先选择或上传一张图像。")
128
  with Image.open(path) as image:
129
  image.verify()
130
  return reset_image(
@@ -132,8 +132,8 @@ def upload_image(path, route, canonical):
132
  None,
133
  route,
134
  canonical,
135
- "<div class='source-meta'><span>你的图像</span><b>已上传图像</b>"
136
- "<small>选择任务,描述你希望识别或定位的内容。</small></div>",
137
  )
138
 
139
 
@@ -153,7 +153,7 @@ def edit_query(text, route, path):
153
  targets_html([]),
154
  "[]",
155
  "",
156
- status_html("提示词已更新 · 再次解析以刷新结果"),
157
  )
158
 
159
 
@@ -178,9 +178,9 @@ def result_bundle(path, canonical, route, raw, elapsed=None, usage=None):
178
  if not parsed.valid:
179
  state = "Protocol error"
180
  message = (
181
- f"解析完成 · {len(valid_items)} 个视觉目标"
182
  if parsed.valid
183
- else "输出校验未通过 · " + "; ".join(parsed.errors)
184
  )
185
  return dict(
186
  image=rendered,
@@ -203,15 +203,15 @@ def display_bundle(bundle):
203
  def inference_error(exc):
204
  timed_out = "timeout" in type(exc).__name__.lower()
205
  return (
206
- "请求超时,请检查连接后重试。图像与提示词已保留。"
207
  if timed_out else
208
- "推理暂不可用,请检查连接后重试。图像与提示词已保留。"
209
  )
210
 
211
 
212
  def run_inference(path, query_text, route):
213
  if not path:
214
- raise gr.Error("请先选择案例或上传图像。")
215
  canonical = compile_query(query_text, route)
216
  try:
217
  raw, elapsed, usage = GroundingClient().infer(path, canonical)
@@ -229,21 +229,21 @@ def run_inference(path, query_text, route):
229
  return display_bundle(bundle)
230
 
231
 
232
- RUN_LABEL = "开始解析"
233
 
234
 
235
  def run_inference_stream(path, query_text, route):
236
  """Expose real request state without an opaque progress mask or synthetic steps."""
237
  if not path:
238
- raise gr.Error("请先选择案例或上传图像。")
239
  yield (
240
  path,
241
- metrics_html(ROUTE_META[route][0], "等待推理"),
242
  targets_html([], pending=True),
243
  "[]",
244
  "",
245
- status_html("等待推理", "busy"),
246
- gr.update(value="正在解析…", interactive=False),
247
  )
248
  try:
249
  result = run_inference(path, query_text, route)
@@ -272,9 +272,9 @@ def service_status():
272
  ok, detail = GroundingClient().health()
273
  return (
274
  f"<div class='api-status {'online' if ok else 'offline'}' role='status' aria-live='polite'>"
275
- f"<i aria-hidden='true'></i><b>{'模型已就绪' if ok else '推理未连接'}</b>"
276
- f"<small>{'随时开始,探索图像中的目标。' if ok else '可以浏览案例;连接推理服务后即可解析。'}</small></div>"
277
- f"<details class='connection-detail'><summary>运行详情</summary><code>{html.escape(detail)}</code></details>"
278
  )
279
 
280
 
@@ -303,7 +303,7 @@ with gr.Blocks(
303
  css=CSS,
304
  js=JS,
305
  head=HOST_HEAD,
306
- title="GroundingPI · 视觉感知工作台",
307
  theme=gr.themes.Base(font=["Manrope", "sans-serif"], font_mono=["ui-monospace", "monospace"]),
308
  analytics_enabled=False,
309
  ) as demo:
@@ -315,52 +315,52 @@ with gr.Blocks(
315
  gr.HTML(
316
  """<div class="control-brand">
317
  <h2>Grounding<span>PI</span></h2>
318
- <p>面向物理智能的视觉基元。</p>
319
  </div>"""
320
  )
321
  with gr.Column(elem_id="control-settings"):
322
  with gr.Row(elem_id="control-fields"):
323
  gr.HTML(
324
- """<div class="media-field"><span>输入类型</span><div>
325
- <span class="media-option is-active" >图像</span>
326
  <button class="media-option" type="button" disabled
327
- title="暂未开放视频解析">视频</button></div></div>""",
328
  elem_id="media-type",
329
  )
330
  route = gr.Dropdown(
331
  [(name, key) for key, (name, _) in ROUTE_META.items()],
332
  value=first_route,
333
- label="选择任务",
334
  filterable=False,
335
  elem_id="route-select",
336
  )
337
  case = gr.Dropdown(
338
  route_choices(first_route),
339
  value=first_case["id"],
340
- label="已选案例",
341
  filterable=False,
342
  visible=False,
343
  elem_id="case-select",
344
  )
345
  upload = gr.UploadButton(
346
- "上传图像", file_types=["image"], type="filepath", elem_id="upload-button"
347
  )
348
- with gr.Accordion("运行配置", open=False, elem_id="advanced-options"):
349
  gr.HTML(
350
- "<div class='runtime-options'><span>模型</span><b>"
351
- + html.escape(os.getenv("GROUNDING_MODEL", "GroundingPI"))
352
- + "</b><span>最大输出长度</span><b>"
353
  + html.escape(os.getenv("GROUNDING_MAX_TOKENS", "1024"))
354
- + "</b><small>当前模型的运行配置。</small></div>"
355
  )
356
  run_btn = gr.Button(RUN_LABEL, variant="primary", elem_id="run-button")
357
  with gr.Column(elem_id="service-panel"):
358
- health = gr.HTML("<div class='api-status'>正在检查推理连接…</div>")
359
- health_btn = gr.Button("刷新连接", size="sm", elem_id="health-button")
360
  with gr.Column(scale=7, min_width=400, elem_id="visual-stage"):
361
  task_help = gr.HTML(task_help_html(first_route), elem_id="task-guide")
362
  with gr.Row(elem_id="stage-toolbar"):
363
- status = gr.HTML(status_html("图像已就绪"), elem_id="stage-status")
364
  with gr.Column(elem_id="canvas-viewport"):
365
  stage = gr.Image(
366
  value=input_view(str(ROOT / first_case["image"]), first_case["prompt"], first_route),
@@ -374,18 +374,18 @@ with gr.Blocks(
374
  show_fullscreen_button=False,
375
  )
376
  gr.HTML(
377
- """<div class="canvas-tools" aria-label="画布工具">
378
- <span class="image-dimensions">图像画布</span>
379
  <div class="zoom-tools">
380
- <button type="button" data-canvas="out" aria-label="缩小" title="缩小">
381
  <svg viewBox="0 0 20 20" aria-hidden="true"><path d="M5 10h10"/></svg></button>
382
- <output class="zoom-value" aria-label="画布缩放">100%</output>
383
- <button type="button" data-canvas="in" aria-label="放大" title="放大">
384
  <svg viewBox="0 0 20 20" aria-hidden="true"><path d="M5 10h10M10 5v10"/></svg></button>
385
  <span class="tool-divider"></span>
386
- <button type="button" data-canvas="fit" aria-label="适应画布" title="适应画布">
387
  <svg viewBox="0 0 20 20" aria-hidden="true"><path d="M7 3H3v4m10-4h4v4M3 13v4h4m10-4v4h-4"/></svg></button>
388
- <a class="canvas-download" download="grounding.png" aria-label="下载图像" title="下载图像">
389
  <svg viewBox="0 0 20 20" aria-hidden="true"><path d="M10 3v10m-4-4 4 4 4-4M4 14v3h12v-3"/></svg></a>
390
  </div>
391
  </div>""",
@@ -394,21 +394,21 @@ with gr.Blocks(
394
  with gr.Row(elem_id="query-row"):
395
  query = gr.Textbox(
396
  value=query_from_prompt(first_case["prompt"], first_route),
397
- label="描述你的目标",
398
  show_label=False,
399
  lines=2,
400
  max_lines=4,
401
  placeholder=TASK_HELP[first_route][3],
402
  elem_id="query-dock",
403
  )
404
- gr.HTML("<p class='query-hint'>Enter 开始解析 · Shift + Enter 换行</p>")
405
  mobile_run = gr.Button(RUN_LABEL, variant="primary", elem_id="mobile-run-button")
406
  with gr.Column(elem_id="targets-panel"):
407
  targets = gr.HTML(targets_html([]), elem_id="targets-content")
408
  with gr.Row(elem_id="review-row", equal_height=True):
409
  with gr.Column(scale=4, min_width=320, elem_id="sandbox-panel"):
410
  gr.HTML(
411
- "<div class='film-heading'><h2>探索案例</h2><small>精选视觉场景</small></div>"
412
  )
413
  gallery = gr.Gallery(
414
  value=gallery_items(first_route),
@@ -422,36 +422,36 @@ with gr.Blocks(
422
  interactive=False,
423
  elem_id="case-gallery",
424
  )
425
- with gr.Accordion("案例信息", open=False, elem_id="source-details"):
426
  meta = gr.HTML(case_meta(first_case))
427
  with gr.Column(scale=6, min_width=420, elem_id="review-summary"):
428
  with gr.Row(elem_id="result-columns", equal_height=True):
429
  with gr.Column(scale=4, min_width=200, elem_id="metrics-panel"):
430
- gr.HTML("<h2 class='inspector-subheading'>推理指标</h2>")
431
  metrics = gr.HTML(metrics_html(ROUTE_META[first_route][0]))
432
  with gr.Column(scale=6, min_width=240, elem_id="sidebar"):
433
  gr.HTML(
434
- "<div class='inspector-heading'><h2>目标细节</h2><span class='live-mark'>选择结果即可放大原始图像。</span></div>"
435
  )
436
  gr.HTML(
437
- "<div class='evidence-detail' hidden><div class='detail-heading'><span>局部细节</span><small>原始像素 · 局部放大</small></div><div class='detail-image'></div><div class='detail-coordinates'></div></div>",
438
  elem_id="detail-panel",
439
  )
440
- with gr.Accordion("模型提示词", open=False, elem_id="prompt-console"):
441
  prompt = gr.Textbox(
442
- value=first_case["prompt"], label="实际发送的提示词", lines=3, interactive=False
443
  )
444
  gr.HTML(
445
- "<div class='console-note'>只发送当前图像与提示词,模型独立生成预测结果。</div>"
446
  )
447
- with gr.Accordion("结构化结果", open=False, elem_id="output-console"):
448
  normalized = gr.Code(
449
- value="[]", language="json", label="目标与坐标 · 0–999", interactive=False
450
  )
451
- with gr.Accordion("模型原始输出", open=False):
452
- raw = gr.Textbox(label="原始 GAM 序列", lines=4, interactive=False)
453
  gr.HTML(
454
- "<footer class='studio-footer'><span>GroundingPI · 感知走向行动</span><span>探索目标、文字与交互位置 <i>图像 → 视觉基元</i></span></footer>"
455
  )
456
 
457
  reset_outputs = [
 
17
  from grounding_demo.render import draw_primitives
18
 
19
  ROUTE_META = {
20
+ "grounding": ("Grounding", "Open-vocabulary object localization"),
21
+ "dense": ("Dense", "Dense instance localization"),
22
+ "referring": ("Referring", "Language-guided referring"),
23
+ "dense_point": ("Dense Point", "Dense point prediction"),
24
+ "referring_point_in_mask": ("Referring Point in Mask", "Point inside a referred target"),
25
+ "dense_point_in_mask": ("Dense Point in Mask", "Point inside multiple targets"),
26
+ "visual_prompt": ("Visual Prompt", "Exemplar-guided localization"),
27
+ "ocr": ("OCR", "Text recognition and localization"),
28
+ "gui": ("GUI", "Interface element grounding"),
29
  }
30
  CASES = load_cases()
31
  BY_ID = {row["id"]: row for row in CASES}
 
62
 
63
  def case_meta(row):
64
  return (
65
+ "<div class='source-meta'><span>TRAINING SOURCE</span>"
66
+ f"<b>{html.escape(row['source'])}</b><span>IMAGE SHA256</span>"
67
  f"<code>{row.get('image_sha256', '')[:16]}</code>"
68
+ "<small>Curated research examples for qualitative exploration, not a held-out benchmark. Inference receives only the image and task prompt.</small></div>"
69
  )
70
 
71
 
 
90
  targets_html([]),
91
  "[]",
92
  "",
93
+ status_html("Image ready · Run grounding to extract visual evidence"),
94
  task_help_html(route),
95
  )
96
 
 
124
 
125
  def upload_image(path, route, canonical):
126
  if not path:
127
+ raise gr.Error("Select an image to continue.")
128
  with Image.open(path) as image:
129
  image.verify()
130
  return reset_image(
 
132
  None,
133
  route,
134
  canonical,
135
+ "<div class='source-meta'><span>YOUR IMAGE</span><b>Uploaded image</b>"
136
+ "<small>Select a task and describe the visual evidence to extract.</small></div>",
137
  )
138
 
139
 
 
153
  targets_html([]),
154
  "[]",
155
  "",
156
+ status_html("Prompt updated · Run grounding to refresh the overlay"),
157
  )
158
 
159
 
 
178
  if not parsed.valid:
179
  state = "Protocol error"
180
  message = (
181
+ f"Complete · {len(valid_items)} visual targets"
182
  if parsed.valid
183
+ else "Output validation failed · " + "; ".join(parsed.errors)
184
  )
185
  return dict(
186
  image=rendered,
 
203
  def inference_error(exc):
204
  timed_out = "timeout" in type(exc).__name__.lower()
205
  return (
206
+ "Request timed out. Check the connection and retry; your image and prompt are preserved."
207
  if timed_out else
208
+ "Inference unavailable. Check the connection and retry; your image and prompt are preserved."
209
  )
210
 
211
 
212
  def run_inference(path, query_text, route):
213
  if not path:
214
+ raise gr.Error("Choose an example or upload an image first.")
215
  canonical = compile_query(query_text, route)
216
  try:
217
  raw, elapsed, usage = GroundingClient().infer(path, canonical)
 
229
  return display_bundle(bundle)
230
 
231
 
232
+ RUN_LABEL = "Run Grounding"
233
 
234
 
235
  def run_inference_stream(path, query_text, route):
236
  """Expose real request state without an opaque progress mask or synthetic steps."""
237
  if not path:
238
+ raise gr.Error("Choose an example or upload an image first.")
239
  yield (
240
  path,
241
+ metrics_html(ROUTE_META[route][0], "Awaiting inference"),
242
  targets_html([], pending=True),
243
  "[]",
244
  "",
245
+ status_html("Awaiting inference", "busy"),
246
+ gr.update(value="Grounding in progress…", interactive=False),
247
  )
248
  try:
249
  result = run_inference(path, query_text, route)
 
272
  ok, detail = GroundingClient().health()
273
  return (
274
  f"<div class='api-status {'online' if ok else 'offline'}' role='status' aria-live='polite'>"
275
+ f"<i aria-hidden='true'></i><b>{'Inference ready' if ok else 'Inference offline'}</b>"
276
+ f"<small>{'Model loaded. Ready to process your image.' if ok else 'Explore examples while the inference service is unavailable.'}</small></div>"
277
+ f"<details class='connection-detail'><summary>Runtime details</summary><code>{html.escape(detail)}</code></details>"
278
  )
279
 
280
 
 
303
  css=CSS,
304
  js=JS,
305
  head=HOST_HEAD,
306
+ title="GroundingPI · Perception Studio",
307
  theme=gr.themes.Base(font=["Manrope", "sans-serif"], font_mono=["ui-monospace", "monospace"]),
308
  analytics_enabled=False,
309
  ) as demo:
 
315
  gr.HTML(
316
  """<div class="control-brand">
317
  <h2>Grounding<span>PI</span></h2>
318
+ <p>Visual primitives for physical intelligence.</p>
319
  </div>"""
320
  )
321
  with gr.Column(elem_id="control-settings"):
322
  with gr.Row(elem_id="control-fields"):
323
  gr.HTML(
324
+ """<div class="media-field"><span>Media Type</span><div>
325
+ <span class="media-option is-active">Image</span>
326
  <button class="media-option" type="button" disabled
327
+ title="Video is not available">Video</button></div></div>""",
328
  elem_id="media-type",
329
  )
330
  route = gr.Dropdown(
331
  [(name, key) for key, (name, _) in ROUTE_META.items()],
332
  value=first_route,
333
+ label="Task Type",
334
  filterable=False,
335
  elem_id="route-select",
336
  )
337
  case = gr.Dropdown(
338
  route_choices(first_route),
339
  value=first_case["id"],
340
+ label="Selected example",
341
  filterable=False,
342
  visible=False,
343
  elem_id="case-select",
344
  )
345
  upload = gr.UploadButton(
346
+ "Upload Image", file_types=["image"], type="filepath", elem_id="upload-button"
347
  )
348
+ with gr.Accordion("Advanced Parameters", open=False, elem_id="advanced-options"):
349
  gr.HTML(
350
+ "<div class='runtime-options'><span>Model</span><b>"
351
+ + html.escape(os.getenv("GROUNDING_MODEL", "grounding-pi-vlm"))
352
+ + "</b><span>Max output tokens</span><b>"
353
  + html.escape(os.getenv("GROUNDING_MAX_TOKENS", "1024"))
354
+ + "</b><small>Read-only runtime configuration. Only the current image and task prompt enter inference.</small></div>"
355
  )
356
  run_btn = gr.Button(RUN_LABEL, variant="primary", elem_id="run-button")
357
  with gr.Column(elem_id="service-panel"):
358
+ health = gr.HTML("<div class='api-status'>Checking inference service…</div>")
359
+ health_btn = gr.Button("Refresh connection", size="sm", elem_id="health-button")
360
  with gr.Column(scale=7, min_width=400, elem_id="visual-stage"):
361
  task_help = gr.HTML(task_help_html(first_route), elem_id="task-guide")
362
  with gr.Row(elem_id="stage-toolbar"):
363
+ status = gr.HTML(status_html("Image ready"), elem_id="stage-status")
364
  with gr.Column(elem_id="canvas-viewport"):
365
  stage = gr.Image(
366
  value=input_view(str(ROOT / first_case["image"]), first_case["prompt"], first_route),
 
374
  show_fullscreen_button=False,
375
  )
376
  gr.HTML(
377
+ """<div class="canvas-tools" aria-label="Canvas tools">
378
+ <span class="image-dimensions">Image canvas</span>
379
  <div class="zoom-tools">
380
+ <button type="button" data-canvas="out" aria-label="Zoom out" title="Zoom out">
381
  <svg viewBox="0 0 20 20" aria-hidden="true"><path d="M5 10h10"/></svg></button>
382
+ <output class="zoom-value" aria-label="Canvas zoom">100%</output>
383
+ <button type="button" data-canvas="in" aria-label="Zoom in" title="Zoom in">
384
  <svg viewBox="0 0 20 20" aria-hidden="true"><path d="M5 10h10M10 5v10"/></svg></button>
385
  <span class="tool-divider"></span>
386
+ <button type="button" data-canvas="fit" aria-label="Fit to canvas" title="Fit to canvas">
387
  <svg viewBox="0 0 20 20" aria-hidden="true"><path d="M7 3H3v4m10-4h4v4M3 13v4h4m10-4v4h-4"/></svg></button>
388
+ <a class="canvas-download" download="grounding.png" aria-label="Download image" title="Download image">
389
  <svg viewBox="0 0 20 20" aria-hidden="true"><path d="M10 3v10m-4-4 4 4 4-4M4 14v3h12v-3"/></svg></a>
390
  </div>
391
  </div>""",
 
394
  with gr.Row(elem_id="query-row"):
395
  query = gr.Textbox(
396
  value=query_from_prompt(first_case["prompt"], first_route),
397
+ label="What would you like to locate?",
398
  show_label=False,
399
  lines=2,
400
  max_lines=4,
401
  placeholder=TASK_HELP[first_route][3],
402
  elem_id="query-dock",
403
  )
404
+ gr.HTML("<p class='query-hint'>Enter to run · Shift + Enter for a new line</p>")
405
  mobile_run = gr.Button(RUN_LABEL, variant="primary", elem_id="mobile-run-button")
406
  with gr.Column(elem_id="targets-panel"):
407
  targets = gr.HTML(targets_html([]), elem_id="targets-content")
408
  with gr.Row(elem_id="review-row", equal_height=True):
409
  with gr.Column(scale=4, min_width=320, elem_id="sandbox-panel"):
410
  gr.HTML(
411
+ "<div class='film-heading'><h2>Explore Examples</h2><small>Curated visual scenarios</small></div>"
412
  )
413
  gallery = gr.Gallery(
414
  value=gallery_items(first_route),
 
422
  interactive=False,
423
  elem_id="case-gallery",
424
  )
425
+ with gr.Accordion("Data provenance", open=False, elem_id="source-details"):
426
  meta = gr.HTML(case_meta(first_case))
427
  with gr.Column(scale=6, min_width=420, elem_id="review-summary"):
428
  with gr.Row(elem_id="result-columns", equal_height=True):
429
  with gr.Column(scale=4, min_width=200, elem_id="metrics-panel"):
430
+ gr.HTML("<h2 class='inspector-subheading'>Inference Metrics</h2>")
431
  metrics = gr.HTML(metrics_html(ROUTE_META[first_route][0]))
432
  with gr.Column(scale=6, min_width=240, elem_id="sidebar"):
433
  gr.HTML(
434
+ "<div class='inspector-heading'><h2>Target Detail</h2><span class='live-mark'>Select a category below the canvas to magnify its original pixels.</span></div>"
435
  )
436
  gr.HTML(
437
+ "<div class='evidence-detail' hidden><div class='detail-heading'><span>Detail View</span><small>Original pixels · magnified</small></div><div class='detail-image'></div><div class='detail-coordinates'></div></div>",
438
  elem_id="detail-panel",
439
  )
440
+ with gr.Accordion("Prompt Console", open=False, elem_id="prompt-console"):
441
  prompt = gr.Textbox(
442
+ value=first_case["prompt"], label="COMPILED PROMPT", lines=3, interactive=False
443
  )
444
  gr.HTML(
445
+ "<div class='console-note'>Transparent inputs: only this image and compiled prompt are sent to the model. No annotations, training answers, or expected counts are included.</div>"
446
  )
447
+ with gr.Accordion("Structured Output", open=False, elem_id="output-console"):
448
  normalized = gr.Code(
449
+ value="[]", language="json", label="PARSED COORDINATES · 0–999", interactive=False
450
  )
451
+ with gr.Accordion("Native Token Sequence", open=False):
452
+ raw = gr.Textbox(label="RAW GAM SPECIAL TOKENS", lines=4, interactive=False)
453
  gr.HTML(
454
+ "<footer class='studio-footer'><span>GroundingPI · Towards Physical Intelligence</span><span>Research demonstration <i>Qualitative examples, not benchmark results.</i></span></footer>"
455
  )
456
 
457
  reset_outputs = [
assets/console.js CHANGED
@@ -43,7 +43,7 @@
43
  preview.style.backgroundImage = `url(${JSON.stringify(source?.src || image.currentSrc || image.src)})`;
44
  preview.style.backgroundSize = `${iw*scale}px ${ih*scale}px`;
45
  preview.style.backgroundPosition = `${preview.clientWidth/2-cx*scale}px ${preview.clientHeight/2-cy*scale}px`;
46
- $('.detail-coordinates').textContent = `实例 ${instanceIndex + 1} / ${coords.length} · [${p.join(', ')}]`;
47
  }
48
  function canvasTransform() {
49
  const stage = $('#stage-image'); if (!stage) return;
@@ -104,21 +104,21 @@
104
  const caption = document.createElement('div'); caption.className = 'focus-caption';
105
  const label = document.createElement('span');
106
  label.setAttribute('role', 'status'); label.setAttribute('aria-live', 'polite');
107
- const evidence = '预测目标';
108
  label.textContent = `${evidence} · ${selected.dataset.label} · ${instanceIndex + 1} / ${coords.length}`;
109
  label.title = label.textContent;
110
  caption.append(label);
111
- for (const [action, name, path] of [['prev', '上一个实例', 'm12 5-5 5 5 5'], ['next', '下一个实例', 'm8 5 5 5-5 5']]) {
112
  const button = document.createElement('button'); button.type = 'button'; button.dataset.instance = action;
113
  button.setAttribute('aria-label', name); button.title = name; button.disabled = coords.length < 2;
114
  const icon = svg('svg', {viewBox: '0 0 20 20', width: 16, height: 16, fill: 'none', 'aria-hidden': 'true'});
115
  icon.append(svg('path', {d: path, stroke: 'currentColor', 'stroke-width': 1.7}));
116
  button.append(icon); caption.append(button);
117
  }
118
- const close = document.createElement('button'); close.type = 'button'; close.className = 'focus-close'; close.setAttribute('aria-label', '关闭目标聚焦');
119
  const icon = svg('svg', { viewBox: '0 0 20 20', width: 16, height: 16, fill: 'none', 'aria-hidden': 'true' });
120
  icon.append(svg('path', { d: 'm6 6 8 8m0-8-8 8', stroke: 'currentColor', 'stroke-width': 1.7, 'stroke-linecap': 'round' }));
121
- close.title = '关闭聚焦,显示全部目标';
122
  close.append(icon); caption.append(close); stage.append(layer); $('#canvas-viewport').append(caption);
123
  if (activeAction) caption.querySelector(`[data-instance=${activeAction}]`)?.focus({preventScroll: true});
124
  showDetail(coords);
@@ -135,7 +135,7 @@
135
  for (const button of document.querySelectorAll('#run-button,#mobile-run-button')) {
136
  button.disabled = value;
137
  button.classList.toggle('is-waiting', value);
138
- if (value) button.setAttribute('aria-label', '正在解析');
139
  else button.removeAttribute('aria-label');
140
  }
141
  for (const id of ['controls', 'sandbox-panel', 'query-dock', 'upload-button', 'canvas-chrome']) {
@@ -169,10 +169,10 @@
169
  if (!ready || waiting) return;
170
  previousRunId = $('#stage-status .stage-status')?.dataset.runId || '';
171
  clientPending = true;
172
- clientStatus('请求已提交 · 等待推理', 'busy'); setWaiting(true);
173
  watchdog = setTimeout(() => {
174
  if (!waiting) return;
175
- clientStatus('尚未收到结果,请检查连接并刷新。服务器请求仍可能在处理。', 'error');
176
  setWaiting(false);
177
  }, 330000);
178
  };
@@ -201,7 +201,7 @@
201
  clearSelection(); resetCanvas(); lastSrc = src;
202
  }
203
  const dimensions = $('.image-dimensions');
204
- if (dimensions && image?.naturalWidth) dimensions.textContent = `${image.naturalWidth} × ${image.naturalHeight} px${zoom>1 ? ' / 拖动平移' : ''}`;
205
  drawSelection();
206
  }
207
  function onClick(event) {
 
43
  preview.style.backgroundImage = `url(${JSON.stringify(source?.src || image.currentSrc || image.src)})`;
44
  preview.style.backgroundSize = `${iw*scale}px ${ih*scale}px`;
45
  preview.style.backgroundPosition = `${preview.clientWidth/2-cx*scale}px ${preview.clientHeight/2-cy*scale}px`;
46
+ $('.detail-coordinates').textContent = `Instance ${instanceIndex + 1} / ${coords.length} · [${p.join(', ')}]`;
47
  }
48
  function canvasTransform() {
49
  const stage = $('#stage-image'); if (!stage) return;
 
104
  const caption = document.createElement('div'); caption.className = 'focus-caption';
105
  const label = document.createElement('span');
106
  label.setAttribute('role', 'status'); label.setAttribute('aria-live', 'polite');
107
+ const evidence = 'Prediction';
108
  label.textContent = `${evidence} · ${selected.dataset.label} · ${instanceIndex + 1} / ${coords.length}`;
109
  label.title = label.textContent;
110
  caption.append(label);
111
+ for (const [action, name, path] of [['prev', 'Previous instance', 'm12 5-5 5 5 5'], ['next', 'Next instance', 'm8 5 5 5-5 5']]) {
112
  const button = document.createElement('button'); button.type = 'button'; button.dataset.instance = action;
113
  button.setAttribute('aria-label', name); button.title = name; button.disabled = coords.length < 2;
114
  const icon = svg('svg', {viewBox: '0 0 20 20', width: 16, height: 16, fill: 'none', 'aria-hidden': 'true'});
115
  icon.append(svg('path', {d: path, stroke: 'currentColor', 'stroke-width': 1.7}));
116
  button.append(icon); caption.append(button);
117
  }
118
+ const close = document.createElement('button'); close.type = 'button'; close.className = 'focus-close'; close.setAttribute('aria-label', 'Close target focus');
119
  const icon = svg('svg', { viewBox: '0 0 20 20', width: 16, height: 16, fill: 'none', 'aria-hidden': 'true' });
120
  icon.append(svg('path', { d: 'm6 6 8 8m0-8-8 8', stroke: 'currentColor', 'stroke-width': 1.7, 'stroke-linecap': 'round' }));
121
+ close.title = 'Close focus and return to targets';
122
  close.append(icon); caption.append(close); stage.append(layer); $('#canvas-viewport').append(caption);
123
  if (activeAction) caption.querySelector(`[data-instance=${activeAction}]`)?.focus({preventScroll: true});
124
  showDetail(coords);
 
135
  for (const button of document.querySelectorAll('#run-button,#mobile-run-button')) {
136
  button.disabled = value;
137
  button.classList.toggle('is-waiting', value);
138
+ if (value) button.setAttribute('aria-label', 'Grounding in progress');
139
  else button.removeAttribute('aria-label');
140
  }
141
  for (const id of ['controls', 'sandbox-panel', 'query-dock', 'upload-button', 'canvas-chrome']) {
 
169
  if (!ready || waiting) return;
170
  previousRunId = $('#stage-status .stage-status')?.dataset.runId || '';
171
  clientPending = true;
172
+ clientStatus('Request submitted · Awaiting inference', 'busy'); setWaiting(true);
173
  watchdog = setTimeout(() => {
174
  if (!waiting) return;
175
+ clientStatus('No result received. Check the connection and refresh. The server request has not been cancelled.', 'error');
176
  setWaiting(false);
177
  }, 330000);
178
  };
 
201
  clearSelection(); resetCanvas(); lastSrc = src;
202
  }
203
  const dimensions = $('.image-dimensions');
204
+ if (dimensions && image?.naturalWidth) dimensions.textContent = `${image.naturalWidth} × ${image.naturalHeight} px${zoom>1 ? ' / drag to pan' : ''}`;
205
  drawSelection();
206
  }
207
  function onClick(event) {
docs/ui_case_pool_22_lock.json CHANGED
@@ -1,12 +1,12 @@
1
  {
2
  "files": {
3
- "app.py": "a5966d2b8dfbe7adf31e6b9542cafcdf6b3198b25661cc97c5580dda680ecf6c",
4
  "assets/Manrope-OFL.txt": "e01b637272e0cbdfb240184dd98ea5cc671556d9894dae2668d92ab2c906787c",
5
  "assets/Manrope-variable.ttf": "d0639be45d0af36e798172419d7bd173c4bd4f29e2b76cbb69db1d11bf8b0a40",
6
  "assets/NotoSansCJK-OFL.txt": "6a73f9541c2de74158c0e7cf6b0a58ef774f5a780bf191f2d7ec9cc53efe2bf2",
7
  "assets/NotoSansCJKsc-Regular.otf": "2c76254f6fc379fddfce0a7e84fb5385bb135d3e399294f6eeb6680d0365b74b",
8
  "assets/SOURCES.md": "0299a8a04951a07c896b28168496209ff900e60f6ae0c98bfdc86a7cc47c1712",
9
- "assets/console.js": "822e670f29a584ba201dc69e76cdbc6a2b836bee99b8dd1bc806aabdf3c7f5ec",
10
  "assets/frosted.css": "393bf5f1c04a49a6e9b17e855e573646489b4add85bc7973eb6bb277686b5c80",
11
  "assets/native.css": "24b80e61a37ca92f62a871f30114bf2839290abf9c899242033b1aeef3998c38",
12
  "assets/original.css": "7a244c7f65e40797850dcde7d6b6505b1b5de60aa24b129f44b33ae52291fd2e",
@@ -36,7 +36,7 @@
36
  "cases/visual_prompt/21_visual_prompt_round5_candidates_gam_visual_dense200_142.jpg": "34101e5a696ca089b3d6026fbafda21ba8a144bb2b4d4f36fd309273ed48e286",
37
  "cases/visual_prompt/22_visual_prompt_round5_candidates_gam_visual_dense200_187.jpg": "3daa8e2776a2dd0ab17abebd669d95728f64bfbd063986f14becfb3f1966463a",
38
  "grounding_demo/catalog.py": "75e543fed7f0ca018fb41c4b528efc2901aa6a0f05232959297d6d40798f676f",
39
- "grounding_demo/console.py": "3097928c7c6a77d5d4d379cbc298222e23d2751e5269e5efad39a71e8050c311",
40
  "grounding_demo/protocol.py": "cf043c9f026a1dcd63d55c5393f0d0921cd15eae4a0cc9a42e2834fab23486ec",
41
  "grounding_demo/render.py": "3b3fe5901edc41586bf5f1a1c263265d47c0fbe4d2ee4c30697c1bd0f78d484b"
42
  },
 
1
  {
2
  "files": {
3
+ "app.py": "79282810a5f15138950ffb9d2ff108067f7f73cf52c797a1931920e1fbff40b8",
4
  "assets/Manrope-OFL.txt": "e01b637272e0cbdfb240184dd98ea5cc671556d9894dae2668d92ab2c906787c",
5
  "assets/Manrope-variable.ttf": "d0639be45d0af36e798172419d7bd173c4bd4f29e2b76cbb69db1d11bf8b0a40",
6
  "assets/NotoSansCJK-OFL.txt": "6a73f9541c2de74158c0e7cf6b0a58ef774f5a780bf191f2d7ec9cc53efe2bf2",
7
  "assets/NotoSansCJKsc-Regular.otf": "2c76254f6fc379fddfce0a7e84fb5385bb135d3e399294f6eeb6680d0365b74b",
8
  "assets/SOURCES.md": "0299a8a04951a07c896b28168496209ff900e60f6ae0c98bfdc86a7cc47c1712",
9
+ "assets/console.js": "3af36a3aeafb72e85018b1c663f624b92877bad0152ee9e264a1c440d689f60e",
10
  "assets/frosted.css": "393bf5f1c04a49a6e9b17e855e573646489b4add85bc7973eb6bb277686b5c80",
11
  "assets/native.css": "24b80e61a37ca92f62a871f30114bf2839290abf9c899242033b1aeef3998c38",
12
  "assets/original.css": "7a244c7f65e40797850dcde7d6b6505b1b5de60aa24b129f44b33ae52291fd2e",
 
36
  "cases/visual_prompt/21_visual_prompt_round5_candidates_gam_visual_dense200_142.jpg": "34101e5a696ca089b3d6026fbafda21ba8a144bb2b4d4f36fd309273ed48e286",
37
  "cases/visual_prompt/22_visual_prompt_round5_candidates_gam_visual_dense200_187.jpg": "3daa8e2776a2dd0ab17abebd669d95728f64bfbd063986f14becfb3f1966463a",
38
  "grounding_demo/catalog.py": "75e543fed7f0ca018fb41c4b528efc2901aa6a0f05232959297d6d40798f676f",
39
+ "grounding_demo/console.py": "be37f29305a0282b4f7810a83d43f5f6196e66320f5f2faadaf9cfce57d4ccd0",
40
  "grounding_demo/protocol.py": "cf043c9f026a1dcd63d55c5393f0d0921cd15eae4a0cc9a42e2834fab23486ec",
41
  "grounding_demo/render.py": "3b3fe5901edc41586bf5f1a1c263265d47c0fbe4d2ee4c30697c1bd0f78d484b"
42
  },
grounding_demo/console.py CHANGED
@@ -23,15 +23,60 @@ CATEGORY_ROUTES = {"grounding", "dense", "layout", "dense_point_in_mask"}
23
 
24
  # Presentation only: never alter the released model prompt or expected answer.
25
  TASK_HELP = {
26
- "grounding": ["语言所指,清晰可见", "从一句描述中发现目标,以精确边框呈现它的位置。", "用逗号分隔类别;选择结果可查看原始像素。", "汽车,行人"],
27
- "dense": ["密集场景,逐一看清", "探索拥挤场景里的小目标、遮挡物和多个实例。", "输入目标类别,再逐个查看检测结果。", "行人,椅子"],
28
- "referring": ["把描述,落在图像上", "用外观、位置或关系,锁定你想找的那个目标。", "描述目标的特征与位置,例如颜色、衣着或相邻物体。", "站在桌子旁边的人"],
29
- "dense_point": ["多个目标,一点即明", "以轻量的坐标点表达多个对象的位置。", "描述需要定位的目标,点表示位置而非物体边界。", "所有可见的行人"],
30
- "referring_point_in_mask": ["在目标内部,找到落点", "根据文字描述,在目标对象内部预测一个位置。", "描述目标,结果将以图像内的坐标点呈现。", "穿着救生衣的人"],
31
- "dense_point_in_mask": ["复杂场景里的每个落点", "为同类目标逐一预测位于对象内部的位置。", "用逗号分隔类别,每个点对应一个目标实例。", "货车,公交车,汽车,行人"],
32
- "visual_prompt": ["给出参照,发现相似", "以参考框为线索,在图像中寻找相似对象。", "绿色 INPUT 框来自你的输入;坐标范围为 0–999。", "给出参考框,再描述需要寻找的相似目标"],
33
- "ocr": ["读懂文字,也找到它", "将识别的文字与所在区域连接,完整保留中文与标点。", "输入文字识别指令;选择一条结果即可检查原始图像。", "OCR task detect all the text in box format."],
34
- "gui": ["让交互意图,找到位置", "把你的操作描述映射为界面上的目标位置。", "描述要选择的元素;此功能仅定位,不执行点击。", "点击搜索按钮"]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
35
  }
36
 
37
 
@@ -66,12 +111,12 @@ def compile_query(query: str, route: str) -> str:
66
 
67
  def metrics_html(route, state="Ready", count=0, elapsed=None, usage=None):
68
  rows = [
69
- ("状态", {"Ready": "就绪", "Completed": "已完成", "Protocol error": "协议错误", "API error": "连接异常", "等待推理": "等待推理"}.get(state, state)),
70
- ("任务", route),
71
- ("模型", "GroundingPI"),
72
- ("目标数", str(count)),
73
- ("耗时", f"{elapsed:.2f} s" if elapsed is not None else "—"),
74
- ("输出 tokens", str((usage or {}).get("completion_tokens", "—"))),
75
  ]
76
  return (
77
  "<dl class='metrics-list'>"
@@ -82,11 +127,11 @@ def metrics_html(route, state="Ready", count=0, elapsed=None, usage=None):
82
 
83
  def targets_html(items, pending=False, image_source=""):
84
  if not items:
85
- title = "正在寻找视觉线索" if pending else "让图像给出答案"
86
  hint = (
87
- "解析进行中,当前图像会保持显示。"
88
  if pending
89
- else "选择案例或上传图像,开始解析后在这里探索结果。"
90
  )
91
  return (
92
  "<div class='empty-targets'><svg viewBox='0 0 40 40' width='40' height='40' "
@@ -98,18 +143,18 @@ def targets_html(items, pending=False, image_source=""):
98
  counts = Counter(item["label"] for item in items)
99
  chips = "".join(
100
  "<button type='button' class='target-chip' aria-pressed='false' "
101
- f"aria-label='{html.escape(f'查看 {label},共 {count} 个目标', quote=True)}' "
102
  f"data-label='{html.escape(label, quote=True)}' "
103
  f"data-coordinates='{html.escape(json.dumps([item['coordinates'] for item in items if item['label'] == label]), quote=True)}'>"
104
  f"<i aria-hidden='true' style='--category-color:rgb{PALETTE[idx % len(PALETTE)]}'></i><span>{html.escape(label)}</span><b>{count}</b></button>"
105
  for idx, (label, count) in enumerate(counts.items())
106
  )
107
- caption = "模型实时预测 · 选择目标查看细节"
108
  return (
109
  (f"<img class='focus-source' hidden alt='' src='{html.escape(image_source, quote=True)}'>" if image_source else "")
110
- + f"<div class='targets-summary'><b>{len(items)}</b> 个目标 <span>{len(counts)} 个类别</span></div>"
111
  f"<div class='target-chips'>{chips}</div>"
112
- "<div class='focus-help'><span>选择目标,查看局部细节</span>"
113
- "<button type='button' class='clear-focus' hidden>显示全部</button></div>"
114
  f"<div class='targets-caption'>{caption}</div>"
115
  )
 
23
 
24
  # Presentation only: never alter the released model prompt or expected answer.
25
  TASK_HELP = {
26
+ "grounding": [
27
+ "Open-Vocabulary Grounding",
28
+ "Turn category names into visual primitives: what is present, and where it is.",
29
+ "Separate categories with commas. Select a result to inspect its original pixels.",
30
+ "car, person"
31
+ ],
32
+ "dense": [
33
+ "Grounding in Dense Scenes",
34
+ "Explore object-level perception in crowded scenes, including small and overlapping instances.",
35
+ "Enter object categories. Inspect overlapping predictions one instance at a time.",
36
+ "person, chair"
37
+ ],
38
+ "referring": [
39
+ "Language, Grounded in Context",
40
+ "Link descriptions of appearance, position, and relationships to a specific visual target.",
41
+ "Describe appearance, position, or relationships in a complete phrase.",
42
+ "The person standing beside the table"
43
+ ],
44
+ "dense_point": [
45
+ "Dense Object Pointing",
46
+ "Represent multiple objects as localized points: a compact form of visual grounding.",
47
+ "Describe the targets to locate. Points indicate positions, not object boundaries.",
48
+ "All visible people"
49
+ ],
50
+ "referring_point_in_mask": [
51
+ "Point Inside a Referred Target",
52
+ "Use a description to predict a point inside the target object.",
53
+ "Describe the target. The output is an image-relative point inside the object.",
54
+ "The person wearing a flotation device"
55
+ ],
56
+ "visual_prompt": [
57
+ "Grounding by Visual Example",
58
+ "Use spatial exemplars to identify related instances through a shared grounding interface.",
59
+ "Green INPUT boxes come from your prompt, not model predictions. Coordinates use the 0–999 range.",
60
+ "Given reference boxes <|box_start|><100><100><200><200><|box_end|> ..."
61
+ ],
62
+ "ocr": [
63
+ "Reading with Spatial Context",
64
+ "Bind recognized text to its image region, connecting what is read with where it appears.",
65
+ "Use an OCR instruction. Select a transcription to inspect the original pixels.",
66
+ "Read and locate the text in the image"
67
+ ],
68
+ "dense_point_in_mask": [
69
+ "Point Inside Multiple Targets",
70
+ "Predict an interior point for each matching instance in a crowded scene.",
71
+ "Separate categories with commas. Each prediction marks a point inside an instance.",
72
+ "van, bus, truck, car, pedestrian"
73
+ ],
74
+ "gui": [
75
+ "Grounding Intent in Interfaces",
76
+ "Map an interaction instruction to a visual target within an interface.",
77
+ "Describe the element to select. The demo predicts a point; it does not execute clicks.",
78
+ "Click the search button"
79
+ ]
80
  }
81
 
82
 
 
111
 
112
  def metrics_html(route, state="Ready", count=0, elapsed=None, usage=None):
113
  rows = [
114
+ ("Status", state),
115
+ ("Task", route),
116
+ ("Model", "GroundingPI"),
117
+ ("Detections", str(count)),
118
+ ("Latency", f"{elapsed:.2f} s" if elapsed is not None else "—"),
119
+ ("Output tokens", str((usage or {}).get("completion_tokens", "—"))),
120
  ]
121
  return (
122
  "<dl class='metrics-list'>"
 
127
 
128
  def targets_html(items, pending=False, image_source=""):
129
  if not items:
130
+ title = "Awaiting visual evidence" if pending else "Explore a visual target"
131
  hint = (
132
+ "Your image stays visible while the model processes the request."
133
  if pending
134
+ else "Choose an example, run grounding, then select a category to inspect."
135
  )
136
  return (
137
  "<div class='empty-targets'><svg viewBox='0 0 40 40' width='40' height='40' "
 
143
  counts = Counter(item["label"] for item in items)
144
  chips = "".join(
145
  "<button type='button' class='target-chip' aria-pressed='false' "
146
+ f"aria-label='{html.escape(f'Inspect {label}, {count} target' + ('s' if count != 1 else ''), quote=True)}' "
147
  f"data-label='{html.escape(label, quote=True)}' "
148
  f"data-coordinates='{html.escape(json.dumps([item['coordinates'] for item in items if item['label'] == label]), quote=True)}'>"
149
  f"<i aria-hidden='true' style='--category-color:rgb{PALETTE[idx % len(PALETTE)]}'></i><span>{html.escape(label)}</span><b>{count}</b></button>"
150
  for idx, (label, count) in enumerate(counts.items())
151
  )
152
+ caption = "Live model predictions · select a target or category to inspect"
153
  return (
154
  (f"<img class='focus-source' hidden alt='' src='{html.escape(image_source, quote=True)}'>" if image_source else "")
155
+ + f"<div class='targets-summary'><b>{len(items)}</b> {'target' if len(items) == 1 else 'targets'} <span>{len(counts)} {'category' if len(counts) == 1 else 'categories'}</span></div>"
156
  f"<div class='target-chips'>{chips}</div>"
157
+ "<div class='focus-help'><span>Inspect individual instances</span>"
158
+ "<button type='button' class='clear-focus' hidden>Show all</button></div>"
159
  f"<div class='targets-caption'>{caption}</div>"
160
  )
tests/test_console.py CHANGED
@@ -44,12 +44,12 @@ class ConsoleTest(unittest.TestCase):
44
  self.assertEqual(len(choices["choices"]), len(app.BY_ROUTE[route]))
45
  self.assertEqual(len(gallery["value"]), len(app.BY_ROUTE[route]))
46
  self.assertEqual(loaded[8], "[]")
47
- self.assertIn("图像已就绪", loaded[10])
48
 
49
  def test_case_switch_resets_prediction(self):
50
  output = app.load_case(app.CASES[1]["id"])
51
  self.assertEqual(output[8], "[]")
52
- self.assertIn("图像已就绪", output[10])
53
 
54
  def test_upload_cannot_reveal_unrelated_gt(self):
55
  row = app.CASES[0]
@@ -70,7 +70,7 @@ class ConsoleTest(unittest.TestCase):
70
  str(app.ROOT / row["image"]), row["prompt"], row["route"], valid + invalid
71
  )
72
  self.assertEqual(result["normalized"], "[]")
73
- self.assertIn("协议错误", result["metrics"])
74
 
75
  def test_actual_request_uses_latest_query_without_reference(self):
76
  row = app.CASES[0]
@@ -81,7 +81,7 @@ class ConsoleTest(unittest.TestCase):
81
  infer.assert_called_once_with(path, compile_query("car", row["route"]))
82
  self.assertIsInstance(output[0], app.Image.Image)
83
  self.assertEqual(len(output), 6)
84
- self.assertIn("1 个视觉目标", output[-1])
85
 
86
  def test_labels_are_html_escaped(self):
87
  self.assertIn(
@@ -97,7 +97,7 @@ class ConsoleTest(unittest.TestCase):
97
  ])
98
  self.assertIn(label, result["normalized"])
99
  self.assertIn(label, result["targets"])
100
- self.assertIn("解析完成", result["status"])
101
 
102
  def test_focus_buttons_carry_exact_safe_geometry(self):
103
  class Buttons(HTMLParser):
@@ -138,7 +138,7 @@ class ConsoleTest(unittest.TestCase):
138
  self.assertIn("data-phase='busy'", pending[-2])
139
  done = next(stream)
140
  self.assertTrue(done[-1]["interactive"])
141
- self.assertIn("1 个视觉目标", done[-2])
142
  infer.assert_called_once_with(path, compile_query("car", row["route"]))
143
 
144
  def test_stream_error_releases_action_without_stale_prediction(self):
@@ -178,14 +178,14 @@ class ConsoleTest(unittest.TestCase):
178
  row = app.CASES[0]
179
  with patch.object(app.GroundingClient, "infer", side_effect=ConnectionError("test detail")):
180
  result = app.run_inference(str(app.ROOT / row["image"]), "car", "grounding")
181
- self.assertIn("请检查连接后重试", result[-1])
182
  self.assertNotIn("test detail", result[-1])
183
  self.assertIn("test detail", result[-2])
184
 
185
  def test_connection_status_explicit_and_escaped(self):
186
  with patch.object(app.GroundingClient, "health", return_value=(False, "<unsafe>")):
187
  output = app.service_status()
188
- self.assertIn("推理未连接", output)
189
  self.assertIn("&lt;unsafe&gt;", output)
190
 
191
  def test_both_run_buttons_share_real_stream(self):
 
44
  self.assertEqual(len(choices["choices"]), len(app.BY_ROUTE[route]))
45
  self.assertEqual(len(gallery["value"]), len(app.BY_ROUTE[route]))
46
  self.assertEqual(loaded[8], "[]")
47
+ self.assertIn("Image ready", loaded[10])
48
 
49
  def test_case_switch_resets_prediction(self):
50
  output = app.load_case(app.CASES[1]["id"])
51
  self.assertEqual(output[8], "[]")
52
+ self.assertIn("Image ready", output[10])
53
 
54
  def test_upload_cannot_reveal_unrelated_gt(self):
55
  row = app.CASES[0]
 
70
  str(app.ROOT / row["image"]), row["prompt"], row["route"], valid + invalid
71
  )
72
  self.assertEqual(result["normalized"], "[]")
73
+ self.assertIn("Protocol error", result["metrics"])
74
 
75
  def test_actual_request_uses_latest_query_without_reference(self):
76
  row = app.CASES[0]
 
81
  infer.assert_called_once_with(path, compile_query("car", row["route"]))
82
  self.assertIsInstance(output[0], app.Image.Image)
83
  self.assertEqual(len(output), 6)
84
+ self.assertIn("1 visual targets", output[-1])
85
 
86
  def test_labels_are_html_escaped(self):
87
  self.assertIn(
 
97
  ])
98
  self.assertIn(label, result["normalized"])
99
  self.assertIn(label, result["targets"])
100
+ self.assertIn("Complete", result["status"])
101
 
102
  def test_focus_buttons_carry_exact_safe_geometry(self):
103
  class Buttons(HTMLParser):
 
138
  self.assertIn("data-phase='busy'", pending[-2])
139
  done = next(stream)
140
  self.assertTrue(done[-1]["interactive"])
141
+ self.assertIn("1 visual targets", done[-2])
142
  infer.assert_called_once_with(path, compile_query("car", row["route"]))
143
 
144
  def test_stream_error_releases_action_without_stale_prediction(self):
 
178
  row = app.CASES[0]
179
  with patch.object(app.GroundingClient, "infer", side_effect=ConnectionError("test detail")):
180
  result = app.run_inference(str(app.ROOT / row["image"]), "car", "grounding")
181
+ self.assertIn("Check the connection and retry", result[-1])
182
  self.assertNotIn("test detail", result[-1])
183
  self.assertIn("test detail", result[-2])
184
 
185
  def test_connection_status_explicit_and_escaped(self):
186
  with patch.object(app.GroundingClient, "health", return_value=(False, "<unsafe>")):
187
  output = app.service_status()
188
+ self.assertIn("Inference offline", output)
189
  self.assertIn("&lt;unsafe&gt;", output)
190
 
191
  def test_both_run_buttons_share_real_stream(self):