Add Smoke Signal diagnostic audit trail and bug report template

README.md

@@ -11,3 +11,8 @@ pinned: false
 ---
 
 Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+
+## Smoke Signal Ops
+
+- Diagnostic audit trail: `smoke_signal/manifest/DIAGNOSTIC_AUDIT_TRAIL.md`
+- Bug report template: `smoke_signal/manifest/BUG_REPORT_TEMPLATE.md`

smoke_signal/manifest/BUG_REPORT_TEMPLATE.md

@@ -0,0 +1,52 @@
+# Smoke Signal — Bug Report Template
+
+Use this template when filing Smoke Signal issues.
+
+## Summary
+
+- Date/time:
+- Reporter:
+- Space URL:
+- Runtime SHA:
+- Stage affected: `ingest` | `profile` | `ocr` | `review` | `export`
+- Severity: `low` | `medium` | `high` | `blocking`
+
+## What Happened
+
+- Expected behavior:
+- Actual behavior:
+- First visible symptom:
+
+## Reproduction
+
+1. 
+2. 
+3. 
+
+## Evidence
+
+- Screenshot(s):
+- Step log excerpt (first 20 lines):
+- Browser + version:
+- Rights class used:
+- Test book ID:
+
+## Runtime State
+
+- `/api/spaces/.../runtime` stage:
+- `/api/spaces/.../runtime` sha:
+- Was fallback expected? `yes` | `no`
+- If OCR: did log show `using Tesseract fallback for this run`? `yes` | `no`
+
+## Impact
+
+- Affected users:
+- Data integrity risk: `none` | `possible` | `confirmed`
+- Rights/compliance risk: `none` | `possible` | `confirmed`
+
+## Cross-Reference
+
+- Related audit entry in `DIAGNOSTIC_AUDIT_TRAIL.md`:
+- Related commit(s):
+- Related issue ticket:
+

smoke_signal/manifest/DIAGNOSTIC_AUDIT_TRAIL.md

@@ -0,0 +1,72 @@
+# Smoke Signal — Diagnostic Audit Trail
+
+**Last updated:** 2026-05-16  
+**Scope:** Production fixes applied to Space `Pointf5ive/Codex_Extractor` during Gradio/OCR stabilization.
+
+## Purpose
+
+This file is the canonical incident and remediation trail for Smoke Signal runtime behavior.  
+Use it for bug reports, regression checks, and release notes.
+
+## Runtime Context
+
+- Space: `https://huggingface.co/spaces/Pointf5ive/Codex_Extractor`
+- Python runtime target: `3.12`
+- UI runtime: Gradio 6.x compatible path
+- OCR engines: Surya (primary), Tesseract (fallback)
+- Data root: `/tmp/smoke_signal`
+
+## Change Timeline
+
+| Local time (Europe/London) | Commit | Area | Symptom Observed | Root Cause | Fix Applied | Verification Signal |
+|---|---|---|---|---|---|---|
+| 2026-05-16 12:30 | `c4147bf` | UI startup | Smoke Signal tab hang on load | Blocking `wizard.load` + SSR interaction | Removed blocking load path | Tab can open without startup lock |
+| 2026-05-16 12:38 | `9556658` | Gradio compat | Runtime crash on app launch | Gradio API changes (`css/head`, image arg) | Moved `css/head` usage + removed deprecated image arg | Space builds and serves tab |
+| 2026-05-16 13:05 | `776da0b` | Dependency policy | Breakage risk across Gradio majors | Unpinned major updates | Pinned Gradio to `<7`, removed deprecated flags | Predictable runtime behavior |
+| 2026-05-16 13:27 | `600390b` | Dependency | Silent failures in workbook paths | Missing `openpyxl` | Added `openpyxl` to requirements | Excel-dependent flows stop failing |
+| 2026-05-16 14:38 | `894f8d5` | Queue/startup | Tabs unresponsive at startup | Blocking `demo.load` path | Removed startup blocking load call | Tabs interactive immediately |
+| 2026-05-16 14:44 | `2f1780f` | Event handlers | Gradio deprecation drift | Deprecated `queue=False` handler args | Removed legacy queue flags | Handlers register cleanly |
+| 2026-05-16 15:36 | `1ae819d` | Storage | Write failures on Space filesystem | Non-writable target paths | Moved operational state under `/tmp/smoke_signal` | Pipeline writes succeed in Space |
+| 2026-05-16 15:50 | `2d9935c` | OCR diagnostics | OCR step gave low observability | Sparse logging | Added debug output in OCR stage | Operator sees gating context in logs |
+| 2026-05-16 16:01 | `6233ead` | Ingest/rights | Rights edits blocked by duplicate uploads | Missing update path for existing rows | Added rights update behavior/UI | Rights corrections available post-upload |
+| 2026-05-16 16:44 | `e31e277` | Review UX | Missing page images after restart | `/tmp` cleanup removed rendered crops | Added on-demand re-render for review page images | Review image recovers after restart |
+| 2026-05-16 16:55 | `db859e4` | Governance diagnostics | Confusing OCR skip cases | Rights/status gating not explicit enough | Added rights/status gate diagnostics | Clear skip reasons in log |
+| 2026-05-16 17:03 | `6c9e2c1` | Runtime + shell | Sidebar/dashboard shell regressions | Compatibility drift + render shell timing | Restored compatibility + dashboard shell behavior | Left navigation shell visible |
+| 2026-05-16 17:18 | `be1ecc2` | Surya integration | `No module named 'surya.ocr'` fallback | Surya API changed in newer package versions | Added dual API support (predictor-v2 + legacy-v1) | Surya import path no longer hard-fails |
+| 2026-05-16 17:46 | `ad64790` | OCR performance | OCR perceived as frozen / very slow | Per-page model calls + repeated PDF opens + no progress | Added model reuse, batched OCR, one-open-per-book, live progress | OCR logs and progress move during run |
+| 2026-05-16 17:53 | `8c23eee` | OCR reliability | Long stalls before fallback activation | Surya load could block run path | Added Surya load timeout + background load + Tesseract run fallback | Fallback path no longer waits indefinitely |
+| 2026-05-16 18:16 | `16b10a3` | Fallback correctness | Fallback still stalled in practice | Fallback batches had missing image load path | Fixed Tesseract batch image loading + render/OCR progress separation | OCR advances under fallback mode |
+
+## Current OCR Safeguards
+
+- Surya runtime is loaded once and reused when available.
+- Surya load timeout is enforced (`SS_SURYA_LOAD_TIMEOUT_SEC`, default `20`).
+- If Surya is not ready in time, run continues with Tesseract.
+- If a Surya batch fails, that batch retries via Tesseract.
+- OCR run progress is split into render phase and OCR phase for visibility.
+
+## Active Runtime Knobs
+
+- `SS_SURYA_BATCH_SIZE` (default `4`)
+- `SS_SURYA_LOAD_TIMEOUT_SEC` (default `20`)
+- `SS_RENDER_DPI_SURYA` (default `300`)
+- `SS_RENDER_DPI_FALLBACK` (default `220`)
+
+## Release Validation Checklist
+
+Run this before closing any OCR incident:
+
+1. Confirm Space runtime shows latest commit SHA in API/runtime panel.
+2. Open Smoke Signal tab and verify tabs remain responsive.
+3. Run Step 3 OCR once and confirm progress text moves beyond `Preparing OCR run...`.
+4. Confirm one of these appears in log:
+   - `Surya models loaded (...)`
+   - `using Tesseract fallback for this run`
+5. Confirm output file exists for test book:
+   - `ocr_raw/<BOOK_ID>/<BOOK_ID>_ocr_raw.json`
+
+## Known Operational Risk
+
+- Hugging Face runtime can temporarily stick on older SHA during `RUNNING_APP_STARTING`.
+- If this happens, perform a controlled pause/restart cycle and re-check runtime SHA before testing behavior.
+