Upload-driven benchmark + CHANGELOG catch-up

- run_benchmark_on_upload: takes a .zip / list / single file, runs the
parser benchmark on the uploaded corpus, returns headline metrics +
per-doc breakdown. GT-comparison metrics intentionally absent
(require labelled datasets). Surfaced in the Benchmark tab as a
second button alongside the regression-fixture button.
- Both benchmark buttons are also reachable via /gradio_api/call/
for remote validation: run_benchmark_in_space (fixtures) and
run_benchmark_on_upload (user-supplied corpus).
- CHANGELOG: catches up the past N pushes worth of work — ZeroGPU
integration, frontend overhaul, smoke/benchmark surfaces,
production-numbers section in the README.

CHANGELOG.md

@@ -7,6 +7,68 @@ pre-1.0 so minor bumps may include breaking changes.
 
 ## [Unreleased]
 
+### Added — Live ZeroGPU integration + Space deploy
+
+- Deployed to https://huggingface.co/spaces/arjun10g/zeroshotGPU on
+  `zero-a10g` hardware (ZeroGPU free tier).
+- ZeroGPU integration helper (`zsgdp/gpu/zero_gpu.py`): no-op decorator
+  off-Space, `@spaces.GPU(duration=N)` on-Space.
+- Two stateless GPU helpers — `_gpu_encode_batch` (embedding) and
+  `_gpu_run_pipeline` (transformers VLM/LLM) — that take only picklable
+  inputs and return only picklable outputs. State stays in the calling
+  process; the GPU worker is invoked just for compute.
+- `EmbeddingRetriever.index/query` and `TransformersClient.execute_task`
+  refactored to use the stateless helpers. Fixes a bug where the
+  in-method `@spaces.GPU` decoration silently dropped `self._chunk_ids`
+  / `self._vectors` / cached pipelines because they were set inside the
+  ephemeral worker process — yielding `recall@1=0` even with a
+  successful model run.
+- `_gpu_encode_batch` and `_gpu_run_pipeline` use `duration=60` to
+  stay under the ZeroGPU free-tier cap (180s rejected).
+- `runtime.py` now reports `zero_gpu_available` and adds an explicit
+  log note when the ZeroGPU SDK is detected vs. when hardware reports
+  ZeroGPU but the SDK is missing.
+
+### Added — Frontend (Gradio)
+
+- ZIP and multi-file upload (`gr.File(file_count="multiple")`,
+  accepts `.zip`); server-side extraction with path-traversal guard,
+  capped at `MAX_BATCH_DOCS` (default 20).
+- `gr.Progress` callback wired through `parse_uploaded_document` with
+  stage labels: "Validating uploads...", "Parsing N/M: <name>",
+  "Bundling artifacts...", "Done".
+- Three pipeline modes in the dropdown: `Docling + PyMuPDF`,
+  `Default lightweight`, `Live GPU repair` (loads
+  `configs/live_gpu_repair.yaml`, sets
+  `repair.execute_gpu_escalations=true`).
+- New tabs: `Chunks` (per-strategy detail with token-count distribution
+  + 3 sample chunks per strategy), `Artifacts` (individual file
+  downloads), `Smokes` (button bound to `run_smokes_in_space`),
+  `Benchmark` (regression-fixture + uploaded-corpus modes).
+- `run_smokes_in_space`, `run_benchmark_in_space`, and
+  `run_benchmark_on_upload` exposed at `/gradio_api/call/<name>` for
+  remote validation.
+- `parse_uploaded_document` returns a `summary.batch` block with
+  per-doc results and aggregate metrics when more than one doc parsed.
+- Agent-friendly docstring added to `parse_uploaded_document` so HF's
+  `agents.md` surface gives AI agents proper "when to use" guidance.
+- `chunking_payload` now includes a `detail` block with strategy
+  counts, token-count statistics, and 3 sample chunks per strategy
+  with 240-char previews.
+
+### Documentation — Production benchmark numbers landed
+
+- README's "Production benchmark numbers" section filled in with real
+  values from the live Space (commit `de03f34`):
+  - Smokes: 4/5 pass on `arjun10g/zeroshotGPU` (lexical, ablation,
+    embedding `recall@1=1.0` `recall@5=1.0`, gpu_repair-wiring).
+  - Benchmark: `mean_quality_score=0.964`, `mean_retrieval_recall_at_5=1.0`,
+    `mean_repair_resolution_rate=1.0`, `mean_repair_regression_rate=0.0`.
+- Live GPU repair invocation documented as wiring-verified but
+  end-to-end deferred (deterministic markdown table normalizer fixes
+  malformations before GPU escalation kicks in; needs a real PDF with
+  bbox-aware tables to genuinely fire Qwen).
+
 ### Documentation — README restructured
 
 - Reorganised into Install → Quick start → Opt-ins → Outputs →

app.py

@@ -524,6 +524,107 @@ def run_smokes_in_space() -> dict:
     return payload
 
 
+def run_benchmark_on_upload(file_obj: Any) -> dict:
+    """Run the parser benchmark against a user-supplied corpus.
+
+    Accepts the same upload shapes as `parse_uploaded_document`: a single
+    document, a list, or a `.zip` of documents. Per-file caps and batch
+    cap apply identically. Returns the benchmark headline metrics plus a
+    `documents` list with per-doc records.
+
+    For real §29 numbers against labelled datasets, use the
+    `omnidocbench` or `doclaynet` loader from a Pro-tier Dev Mode
+    terminal — those add layout F1 / table structure / formula CER which
+    require ground-truth annotations not available from a raw upload.
+    """
+
+    if file_obj is None:
+        return {"error": "Upload at least one document to benchmark."}
+
+    import tempfile
+    from zsgdp.benchmarks.parser_quality import run_parser_benchmark
+
+    if isinstance(file_obj, list):
+        upload_paths = [Path(item.name if hasattr(item, "name") else item) for item in file_obj if item is not None]
+    elif hasattr(file_obj, "name"):
+        upload_paths = [Path(file_obj.name)]
+    else:
+        upload_paths = [Path(str(file_obj))]
+    if not upload_paths:
+        return {"error": "Upload at least one document to benchmark."}
+
+    work_dir = Path(tempfile.mkdtemp(prefix="zsgdp_bench_upload_"))
+    docs = _extract_uploads_to_parse(upload_paths, work_dir)
+    if not docs:
+        return {
+            "error": "No supported documents found in the upload (accepted: pdf/md/txt/html, optionally inside a zip).",
+            "input_count": len(upload_paths),
+        }
+
+    # Per-file abuse guards.
+    for doc in docs:
+        try:
+            _validate_upload(doc)
+        except UploadRejected as exc:
+            return {"error": str(exc), "rejected": True, "source_path": str(doc)}
+
+    bench_input = work_dir / "input"
+    bench_input.mkdir()
+    for doc in docs:
+        target = bench_input / doc.name
+        # Avoid name collisions (different paths, same filename inside zips).
+        suffix = 2
+        while target.exists():
+            target = bench_input / f"{doc.stem}_{suffix}{doc.suffix}"
+            suffix += 1
+        shutil.copy2(doc, target)
+
+    out = work_dir / "out"
+    _logger.info(
+        "space_benchmark_upload_requested",
+        extra={"input_count": len(upload_paths), "docs_found": len(docs)},
+    )
+    summary = run_parser_benchmark(bench_input, out, dataset_name="custom_folder")
+
+    headline = {
+        "dataset_name": summary.get("dataset_name"),
+        "document_count": summary.get("document_count"),
+        "mean_quality_score": summary.get("mean_quality_score"),
+        "mean_retrieval_recall_at_1": summary.get("mean_retrieval_recall_at_1"),
+        "mean_retrieval_recall_at_5": summary.get("mean_retrieval_recall_at_5"),
+        "mean_retrieval_mrr": summary.get("mean_retrieval_mrr"),
+        "mean_parser_disagreement_rate": summary.get("mean_parser_disagreement_rate"),
+        "mean_repair_resolution_rate": summary.get("mean_repair_resolution_rate"),
+        "mean_repair_regression_rate": summary.get("mean_repair_regression_rate"),
+        "retrieval_evaluated_count": summary.get("retrieval_evaluated_count"),
+        "documents": [
+            {
+                "doc_id": doc.get("doc_id"),
+                "file_type": doc.get("file_type"),
+                "quality_score": doc.get("quality_score"),
+                "elements": doc.get("element_count"),
+                "tables": doc.get("table_count"),
+                "figures": doc.get("figure_count"),
+                "chunks": doc.get("chunk_count"),
+                "parser_disagreement_rate": doc.get("parser_disagreement_rate"),
+                "repair_resolution_rate": doc.get("repair_resolution_rate"),
+                "elapsed_seconds": doc.get("elapsed_seconds"),
+            }
+            for doc in summary.get("documents") or []
+        ],
+        "note": (
+            "GT-comparison metrics (layout F1, table structure, formula CER) "
+            "are unavailable for arbitrary uploads — they need labelled datasets "
+            "(omnidocbench / doclaynet)."
+        ),
+    }
+    _logger.info(
+        "space_benchmark_upload_complete",
+        extra={k: v for k, v in headline.items() if k != "documents" and not isinstance(v, list)},
+    )
+    return headline
+
+
 def run_benchmark_in_space() -> dict:
     """Run a benchmark against tests/regression/fixtures and return the headline numbers.
 
@@ -651,12 +752,26 @@ with gr.Blocks(title="zeroshotGPU") as demo:
             smoke_output = gr.JSON(label="Smoke report")
         with gr.Tab("Benchmark"):
             gr.Markdown(
-                "Runs the parser benchmark against the committed regression "
-                "fixtures (`tests/regression/fixtures/`). For real-corpus runs, "
-                "use `zsgdp benchmark` from a Dev Mode terminal. API endpoint: "
-                "`/gradio_api/call/run_benchmark_in_space`."
+                "**Two benchmark modes:**\n"
+                "- **Run on regression fixtures** — uses the committed seed "
+                "documents (`tests/regression/fixtures/`); reproducible without "
+                "any upload. API: `/gradio_api/call/run_benchmark_in_space`.\n"
+                "- **Run on uploaded corpus** — accepts a `.zip` of documents "
+                "(or a list of files). Returns headline metrics plus a per-doc "
+                "breakdown. GT-comparison metrics (layout F1, table structure, "
+                "formula CER) are NOT computed — those require labelled "
+                "datasets (`omnidocbench` / `doclaynet`) which can be loaded "
+                "via the CLI from a Pro-tier Dev Mode terminal. API: "
+                "`/gradio_api/call/run_benchmark_on_upload`."
+            )
+            with gr.Row():
+                benchmark_button = gr.Button("Run on regression fixtures", variant="primary")
+                benchmark_upload_button = gr.Button("Run on uploaded corpus")
+            benchmark_corpus = gr.File(
+                label="Optional upload — used only when 'Run on uploaded corpus' is clicked",
+                file_types=[".pdf", ".md", ".txt", ".html", ".htm", ".zip"],
+                file_count="multiple",
             )
-            benchmark_button = gr.Button("Run benchmark", variant="primary")
             benchmark_output = gr.JSON(label="Benchmark headline metrics")
     parse_button.click(
         parse_uploaded_document,
@@ -677,6 +792,12 @@ with gr.Blocks(title="zeroshotGPU") as demo:
     )
     smoke_button.click(run_smokes_in_space, inputs=[], outputs=smoke_output, api_name="run_smokes_in_space")
     benchmark_button.click(run_benchmark_in_space, inputs=[], outputs=benchmark_output, api_name="run_benchmark_in_space")
+    benchmark_upload_button.click(
+        run_benchmark_on_upload,
+        inputs=[benchmark_corpus],
+        outputs=benchmark_output,
+        api_name="run_benchmark_on_upload",
+    )
 
 
 if __name__ == "__main__":