feat(upload): wait-for-extraction flow + status endpoint + extended voice guard

User directives integrated:
1. "generate the card inline ONLY after full data extraction" → card
no longer renders on the partial heuristic record; we wait for
the LLM pass to complete first.
2. "ensure re-grading actually happens" → new
/api/upload/extraction-status/{policy_id} exposes real-time
in-memory state of the background LLM extraction.
3. "kill the unprompted 'Could you please upload' message" → voice
auto-submit now blocked for the ENTIRE extraction window
(extractionInFlight state), not just the 8s upload-status window.
4. "same level of usability as the 148 catalogued, no less" → no
user-upload__ branches in PolicyPremiumWidget /
PerPolicyPremiumEstimator — parity by construction once LLM
extraction lands and HealthPolicy fields populate.

BACKEND
─────────────────────────────────────────────────────────────────────
- _UPLOAD_EXTRACTION_STATUS dict (in-process) tracks per-upload
extraction lifecycle: pending → running → complete | failed.
- extract_one_for_upload writes status at every phase + the final
completeness_pct + overall_grade it computed.
- New GET /api/upload/extraction-status/{policy_id} returns the
ExtractionStatusResponse for the frontend's poll loop. Unknown
policy_id returns status="unknown" so the client can stop polling.
- Upload endpoint pre-stamps "pending" BEFORE firing
asyncio.create_task so a fast frontend poll never races.

FRONTEND
─────────────────────────────────────────────────────────────────────
- New extractionInFlight state — true from upload start until
status="complete"|"failed" (or 120s hard timeout).
- Voice guard now: `if (uploadStatus || extractionInFlight) return`
(was: uploadStatus only — cleared in 8s while extraction kept
running 30-60s more, letting ambient sound fire an unprompted
"please upload" chat turn).
- handleFile restructured into 6 phases:
1. POST /api/upload-policy
2. push ack "Got it — I've received X. Give me a moment to read
through it fully (~30-60s) and I'll bring back a complete
picture" (NO citations → NO card render yet)
3. push choice prompt (finish profile / dive into PDF)
4. poll /api/upload/extraction-status/{id} every 3s for up to 120s
5. on status="complete" → push assistant msg WITH citations →
card renders inline at THIS point with full LLM-extracted data
6. on failed / timeout → push fallback message (no broken-state
card; user can still chat about the PDF since text is indexed)
- 3 new i18n keys (en + hi): upload.chat_ack_reading,
upload.chat_card_ready, upload.chat_extraction_failed.

VERIFY
─────────────────────────────────────────────────────────────────────
- py_compile clean
- npx tsc --noEmit clean
- Live audit deferred to deploy commit

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/main.py

@@ -1978,8 +1978,9 @@ async def upload_policy(
 
         # ── Fire LLM-assisted extraction in background (ADR-044) ─────────
         # Same extractor as the catalogued 148. Runs ~30-60s; the upload
-        # HTTP response returns now and the frontend polls the scorecard
-        # endpoint to refresh the card in place when extraction lands.
+        # HTTP response returns now and the frontend polls
+        # /api/upload/extraction-status/{policy_id} (see below) to know
+        # when the card-bearing chat message should be pushed.
         # Fail-silent: a failed LLM pass leaves the heuristic record
         # intact, so the card still has SOMETHING to show — never blocks
         # the user. NEVER blocks this request.
@@ -1992,6 +1993,20 @@ async def upload_policy(
                 if _resolved_insurer_slug != _udocs.UPLOAD_INSURER_SLUG
                 else _udocs.UPLOAD_INSURER_NAME,
             )
+            # Pre-stamp "pending" so a frontend poll that arrives BEFORE
+            # extract_one_for_upload's first await still sees a known
+            # state instead of HTTP 404.
+            await _udocs._set_extraction_status(
+                policy_id,
+                status="pending",
+                policy_name=policy_name,
+                insurer_slug=_resolved_insurer_slug,
+                started_at=None,
+                completed_at=None,
+                completeness_pct=None,
+                overall_grade=None,
+                error=None,
+            )
             asyncio.create_task(
                 _udocs.extract_one_for_upload(
                     policy_id=policy_id,
@@ -2028,6 +2043,58 @@ async def upload_policy(
     )
 
 
+# ---------------------------------------------------------------------------
+# GET /api/upload/extraction-status/{policy_id} — frontend poll target
+# (ADR-044, 2026-05-27).
+#
+# After the upload endpoint returns, the chat flow needs to know when
+# the background LLM extraction completes so it can push the card-bearing
+# assistant message into chat with the FULL data (not the heuristic
+# stub). This endpoint exposes _UPLOAD_EXTRACTION_STATUS so the
+# frontend can poll every ~3s for up to ~120s.
+# ---------------------------------------------------------------------------
+
+
+class ExtractionStatusResponse(BaseModel):
+    policy_id: str
+    status: str  # "pending" | "running" | "complete" | "failed" | "unknown"
+    policy_name: Optional[str] = None
+    insurer_slug: Optional[str] = None
+    started_at: Optional[str] = None
+    completed_at: Optional[str] = None
+    completeness_pct: Optional[float] = None
+    overall_grade: Optional[str] = None
+    error: Optional[str] = None
+
+
+@app.get(
+    "/api/upload/extraction-status/{policy_id}",
+    response_model=ExtractionStatusResponse,
+)
+async def upload_extraction_status(policy_id: str):
+    """Return the live status of a per-upload LLM-assisted extraction.
+
+    Returns `status="unknown"` for an unrecognised policy_id (e.g. the
+    frontend polled a stale id or a policy that was uploaded on a prior
+    container) so the client can stop polling without ambiguity.
+    """
+    from backend import uploaded_docs as _udocs
+    state = _udocs.get_extraction_status(policy_id)
+    if not state:
+        return ExtractionStatusResponse(policy_id=policy_id, status="unknown")
+    return ExtractionStatusResponse(
+        policy_id=policy_id,
+        status=state.get("status", "unknown"),
+        policy_name=state.get("policy_name"),
+        insurer_slug=state.get("insurer_slug"),
+        started_at=state.get("started_at"),
+        completed_at=state.get("completed_at"),
+        completeness_pct=state.get("completeness_pct"),
+        overall_grade=state.get("overall_grade"),
+        error=state.get("error"),
+    )
+
+
 class ScorecardSubScore(BaseModel):
     name: str
     score: int

backend/uploaded_docs.py

@@ -728,14 +728,47 @@ async def reingest_persisted_into_policies() -> dict:
 # 74% median for catalogued. After this change uploaded cards land in
 # the same completeness band by construction.
 #
-# Runs as a background asyncio task fired from the upload endpoint —
-# the upload's HTTP response returns immediately with the heuristic
-# record (sub-second), and the LLM pass (~30-60s) lands in the
-# background. The frontend polls /api/policies/{id}/scorecard after
-# the upload and refreshes the card in place when completeness jumps.
+# Runs as a background asyncio task fired from the upload endpoint.
+# The upload's HTTP response returns immediately (sub-second) with the
+# heuristic record; the LLM pass (~30-60s) lands in the background.
+# A new GET /api/upload/extraction-status/{policy_id} endpoint exposes
+# in-flight state to the frontend so the chat flow can wait for
+# extraction → THEN render the card with full data (no partial render).
 # ---------------------------------------------------------------------------
 
 
+# In-memory status dict — one entry per uploaded policy_id.
+# Shape:
+#   {
+#     "status": "pending" | "running" | "complete" | "failed",
+#     "policy_id": str,
+#     "policy_name": str,
+#     "insurer_slug": str,
+#     "started_at": ISO-8601 UTC,
+#     "completed_at": ISO-8601 UTC | None,
+#     "completeness_pct": float | None,  # populated on complete
+#     "overall_grade": str | None,
+#     "error": str | None,
+#   }
+# Survives only the live process — fine for the UX use case (the
+# frontend polls within ~120s of upload).
+_UPLOAD_EXTRACTION_STATUS: dict[str, dict] = {}
+_UPLOAD_EXTRACTION_LOCK = asyncio.Lock()
+
+
+async def _set_extraction_status(policy_id: str, **fields) -> None:
+    async with _UPLOAD_EXTRACTION_LOCK:
+        cur = _UPLOAD_EXTRACTION_STATUS.get(policy_id, {})
+        cur.update(fields)
+        cur["policy_id"] = policy_id
+        _UPLOAD_EXTRACTION_STATUS[policy_id] = cur
+
+
+def get_extraction_status(policy_id: str) -> Optional[dict]:
+    """Public read accessor used by the /api/upload/extraction-status endpoint."""
+    return _UPLOAD_EXTRACTION_STATUS.get(policy_id)
+
+
 async def extract_one_for_upload(
     policy_id: str,
     pdf_path: Path,
@@ -748,10 +781,25 @@ async def extract_one_for_upload(
     invalidates the marketplace grade cache so the next /api/policies/all
     + /api/policies/{id}/scorecard call returns the LLM-graded card.
 
+    Status is mirrored to `_UPLOAD_EXTRACTION_STATUS[policy_id]` at every
+    phase change so the frontend's poll loop sees progress in real time.
+
     Returns True iff a HealthPolicy was successfully extracted and written.
     Swallows all errors (returns False) — a failed LLM pass must NEVER
     affect the upload's HTTP response, which has already returned.
     """
+    _now = lambda: time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+    await _set_extraction_status(
+        policy_id,
+        status="running",
+        policy_name=policy_name,
+        insurer_slug=insurer_slug,
+        started_at=_now(),
+        completed_at=None,
+        completeness_pct=None,
+        overall_grade=None,
+        error=None,
+    )
     try:
         # Lazy imports — these touch the LLM client + DuckDB; we don't
         # want to pay that cost at module import time.
@@ -780,6 +828,11 @@ async def extract_one_for_upload(
                 "[upload-extract] read_full_text failed %s: %s: %s",
                 policy_id, type(e).__name__, e,
             )
+            await _set_extraction_status(
+                policy_id, status="failed",
+                completed_at=_now(),
+                error=f"read_full_text: {type(e).__name__}: {str(e)[:160]}",
+            )
             return False
 
         prompt = build_extract_prompt(text, schema_excerpt(), policy_id)
@@ -828,6 +881,11 @@ async def extract_one_for_upload(
                 "[upload-extract] no policy extracted for %s after retries; "
                 "card stays on heuristic record", policy_id,
             )
+            await _set_extraction_status(
+                policy_id, status="failed",
+                completed_at=_now(),
+                error="LLM returned no valid HealthPolicy after primary + fallback retries",
+            )
             return False
 
         # Write rag/extracted/<policy_id>.json — same shape as catalogued.
@@ -866,8 +924,38 @@ async def extract_one_for_upload(
             "[upload-extract] OK %s (extraction_confidence_pct=%s)",
             policy_id, getattr(policy, "extraction_confidence_pct", "n/a"),
         )
+
+        # Resolve the freshly-graded card so the status can report
+        # the actual completeness + grade the chat card will show.
+        _final_completeness = None
+        _final_grade = None
+        try:
+            import backend.main as _bm2
+            from backend.scorecard import build_scorecard as _bs
+            _doc = policy.model_dump()
+            _sc = _bs(_doc, profile=None)
+            if _sc is not None:
+                _final_completeness = float(_sc.data_completeness_pct)
+                _final_grade = _sc.overall_grade
+        except Exception:  # noqa: BLE001
+            pass
+
+        await _set_extraction_status(
+            policy_id, status="complete",
+            completed_at=_now(),
+            completeness_pct=_final_completeness,
+            overall_grade=_final_grade,
+        )
         return True
     except Exception as e:  # noqa: BLE001 — top-level catch-all
+        try:
+            await _set_extraction_status(
+                policy_id, status="failed",
+                completed_at=_now(),
+                error=f"{type(e).__name__}: {str(e)[:200]}",
+            )
+        except Exception:
+            pass
         _log.warning(
             "[upload-extract] unexpected failure for %s: %s: %s",
             policy_id, type(e).__name__, str(e)[:400],

frontend/src/app/page.tsx

@@ -196,6 +196,12 @@ export default function Page() {
     }
   }, [messages]);
   const [uploadStatus, setUploadStatus] = useState<string | null>(null);
+  // ADR-044 (2026-05-27) — extractionInFlight stays true from the moment
+  // a PDF starts uploading until the background LLM extraction either
+  // completes or hits its hard timeout. Voice auto-submit is gated on
+  // this so ambient noise / TTS playback during the 30-60s extraction
+  // window can no longer fire an unprompted chat turn.
+  const [extractionInFlight, setExtractionInFlight] = useState<boolean>(false);
   // KI-027 (2026-05-14) — voice UX simplification. The legacy `handsFree`
   // mode (its own VAD auto-cutoff + post-turn mic re-open loop) has been
   // removed. We now have exactly two voice paths, mutually exclusive:
@@ -971,14 +977,14 @@ export default function Page() {
     voiceSubmitRef.current = (text: string) => {
       const t = text.trim();
       if (t.length < 2) return;
-      // Suppress voice auto-submit while a PDF upload is in flight or
-      // just-completed (uploadStatus is non-null for ~8s after success
-      // / failure). A long upload + active mic + bot's TTS playing
-      // through speakers can otherwise auto-transcribe ambient sound
-      // and fire an "unprompted analysis" chat turn that drowns the
-      // upload-flow's choice prompt. Real user input still goes
-      // through the typed-input path / explicit Push-to-talk press.
-      if (uploadStatus) return;
+      // Suppress voice auto-submit while a PDF upload is in flight OR
+      // while the background LLM extraction is still running (ADR-044).
+      // A long upload + active mic + bot's TTS playing through speakers
+      // can otherwise auto-transcribe ambient sound and fire an
+      // "unprompted analysis" chat turn that drowns the upload-flow's
+      // choice prompt. Real user input still goes through the
+      // typed-input path / explicit Push-to-talk press.
+      if (uploadStatus || extractionInFlight) return;
       // V4 FIX 2 — dedup repeated finals within 500ms.
       const { text: prevText, at: prevAt } = lastFinalTextRef.current;
       const now = Date.now();
@@ -994,7 +1000,7 @@ export default function Page() {
     // send() reads `messages` / `sessionId` / `ttsLang` / view flags via
     // closure; rebind whenever they change so the latest values are used.
     // eslint-disable-next-line react-hooks/exhaustive-deps
-  }, [messages, sessionId, ttsLang, openPolicy, showMarketplace, showProfile, showPremium, uploadStatus]);
+  }, [messages, sessionId, ttsLang, openPolicy, showMarketplace, showProfile, showPremium, uploadStatus, extractionInFlight]);
 
   async function startRecording() {
     // KI-222 FIX 1 — silence any prior bot TTS BEFORE PTT recording starts.
@@ -1448,57 +1454,106 @@ export default function Page() {
   async function handleFile(ev: React.ChangeEvent<HTMLInputElement>) {
     const f = ev.target.files?.[0];
     if (!f) return;
-    // Earlier iteration also pushUser'd a "📎 Uploaded: <name>" breadcrumb
-    // into the transcript. That leaked into chat_history so a subsequent
-    // voice auto-fire (mic catching ambient sound during the long index
-    // wait) could trigger the brain to "analyse" the upload unprompted.
-    // Removed: the card rendered below the ack message is itself the
-    // user-visible breadcrumb that the upload happened. uploadStatus
-    // gates the voice-submit path during the indexing window.
+    // ADR-044 (2026-05-27) — new staged upload flow:
+    //   1. POST /api/upload-policy → indexes + persists + kicks the
+    //      background LLM extraction.
+    //   2. push assistant ack (NO card yet — we don't render the card
+    //      on the partial heuristic record).
+    //   3. push choice prompt (finish profile / dive into PDF).
+    //   4. poll /api/upload/extraction-status/{id} every 3s for up to
+    //      120s. While polling, `extractionInFlight=true` blocks voice
+    //      auto-submit so ambient sound during the wait can't trigger
+    //      an "unprompted please-upload" chat turn.
+    //   5. when status === "complete", push a NEW assistant message
+    //      with the citations → card renders inline at that point
+    //      with FULL data (catalogued-grade depth).
+    //   6. on "failed" / timeout, push a fallback ack with whatever
+    //      heuristic data we have, so the user is never stranded.
     setUploadStatus(t("upload.indexing", { name: f.name }));
+    setExtractionInFlight(true);
     try {
       // Pass the live chat session so the backend scopes the uploaded doc
       // to this user — the assistant can then answer questions about it
       // for the rest of THIS conversation.
       const r = await uploadPolicy(f, sessionId);
       setUploadStatus(t("upload.success", { name: r.policy_name }));
-      // ── In-chat acknowledgment + inline scorecard card ──────────────
-      // Push two assistant messages:
-      //   1. The "got it, here's the card" ack with a `citations` array
-      //      carrying the uploaded policy_id. The existing chat renderer
-      //      reads citations from an assistant message and fires
-      //      `getScorecard(policy_id, session_id)` per cited policy, so
-      //      the scorecard card appears inline under the bubble — same
-      //      treatment as a recommendation card.
-      //   2. The proceed-choice prompt — telling the user they can
-      //      finish their profile OR dive into the PDF, and noting that
-      //      a fuller profile makes the policy discussion more useful.
-      const ackText = t("upload.chat_ack", { name: r.policy_name });
-      pushAssistant(ackText, {
-        citations: [
-          {
-            policy_id: r.policy_id,
-            policy_name: r.policy_name,
-            insurer_slug: "user-upload",
-            page_start: 1,
-            page_end: r.pages_indexed,
-            source_url: "",
-            score: 1.0,
-          },
-        ],
-      });
+      // Step 2 — ack (NO citations yet → no card rendered)
+      pushAssistant(t("upload.chat_ack_reading", { name: r.policy_name }));
+      // Step 3 — choice prompt
       pushAssistant(t("upload.chat_choice"));
       // Refresh coverage so the uploaded doc shows up
       getCoverage().then(setCoverage).catch(() => {});
+
+      // Step 4 — poll extraction status
+      const POLL_INTERVAL_MS = 3000;
+      const MAX_TRIES = 40; // 40 × 3s = 120s
+      let landed = false;
+      let finalCompleteness: number | null = null;
+      let finalGrade: string | null = null;
+      let finalInsurerSlug: string = r.policy_id.startsWith("user-upload__") ? "user-upload" : "";
+      for (let i = 0; i < MAX_TRIES; i++) {
+        try {
+          const resp = await fetch(
+            `${BACKEND_URL}/api/upload/extraction-status/${encodeURIComponent(r.policy_id)}`,
+          );
+          if (resp.ok) {
+            const s = await resp.json();
+            if (s.status === "complete") {
+              landed = true;
+              finalCompleteness = s.completeness_pct ?? null;
+              finalGrade = s.overall_grade ?? null;
+              finalInsurerSlug = s.insurer_slug || finalInsurerSlug;
+              break;
+            }
+            if (s.status === "failed") {
+              break;
+            }
+            // pending / running / unknown — keep polling
+            if (s.insurer_slug) finalInsurerSlug = s.insurer_slug;
+          }
+        } catch (_) {
+          // tolerant of transient fetch errors; keep polling
+        }
+        await new Promise((res) => setTimeout(res, POLL_INTERVAL_MS));
+      }
+
+      // Step 5 — push the card-bearing assistant message
+      if (landed) {
+        pushAssistant(
+          t("upload.chat_card_ready", { name: r.policy_name }),
+          {
+            citations: [
+              {
+                policy_id: r.policy_id,
+                policy_name: r.policy_name,
+                insurer_slug: finalInsurerSlug || "user-upload",
+                page_start: 1,
+                page_end: r.pages_indexed,
+                source_url: "",
+                score: 1.0,
+              },
+            ],
+          },
+        );
+      } else {
+        // Step 6 — fallback. We DON'T render a card on the heuristic
+        // stub (per user directive — "lets generate the card inline
+        // ONLY after full data extraction"), so on timeout / failure
+        // just tell the user the deep-analysis didn't complete and
+        // they can ask questions about the PDF directly.
+        pushAssistant(
+          t("upload.chat_extraction_failed", { name: r.policy_name }),
+        );
+      }
     } catch (e: unknown) {
       const errMsg = e instanceof Error ? e.message : String(e);
       setUploadStatus(t("upload.error", { err: errMsg }));
-      // Surface the failure in chat too — a transient banner alone is
-      // easy to miss (originally reported as "no acknowledgment").
+      // Surface the failure in chat too.
       pushAssistant(t("upload.error", { err: errMsg }));
     } finally {
       if (fileInputRef.current) fileInputRef.current.value = "";
       setTimeout(() => setUploadStatus(null), 8000);
+      setExtractionInFlight(false);
     }
   }
 

frontend/src/lib/i18n.ts

@@ -43,6 +43,9 @@ export const UI_STRINGS = {
     "upload.error": "✗ Upload failed: ${err}",
     "upload.user_msg": "📎 Uploaded: ${name}",
     "upload.chat_ack": "Got it — I've read **${name}**. Here's how it grades against what we know about you so far:",
+    "upload.chat_ack_reading": "Got it — I've received **${name}**. Give me a moment to read it through fully (about 30–60 seconds) and I'll bring back a complete picture for you.",
+    "upload.chat_card_ready": "Here's the full picture of **${name}** — graded against what we know about you so far:",
+    "upload.chat_extraction_failed": "I couldn't pull a full analysis from this PDF this time. You can still ask me about anything inside the document — I have the full text indexed and can quote the exact wording.",
     "upload.chat_choice": "How would you like to proceed?\n\n• **Tell me more about yourself** — finish the short profile (age, family, location, budget, health) so I can speak to this policy more personally.\n• **Dive into the PDF first** — ask questions about coverage, waiting periods, exclusions, anything in the document.\n\nEither works. The more I know about you, the more useful the discussion of this policy will be.",
 
     // Marketplace panel
@@ -164,6 +167,9 @@ export const UI_STRINGS = {
     "upload.error": "✗ Upload विफल: ${err}",
     "upload.user_msg": "📎 Upload किया: ${name}",
     "upload.chat_ack": "मिल गया — **${name}** पढ़ ली। यह आपके profile के हिसाब से कैसी है:",
+    "upload.chat_ack_reading": "मिल गया — **${name}** मिल गई। थोड़ी देर दें (~30-60 सेकंड), पूरी तरह पढ़कर पूरा analysis लाता हूँ।",
+    "upload.chat_card_ready": "**${name}** का पूरा analysis — आपके profile के हिसाब से:",
+    "upload.chat_extraction_failed": "इस PDF का पूरा analysis इस बार नहीं निकाल पाया। फिर भी आप document के बारे में कुछ भी पूछ सकते हैं — पूरा text indexed है, exact wording quote कर सकता हूँ।",
     "upload.chat_choice": "आगे कैसे बढ़ें?\n\n• **अपने बारे में बताएं** — short profile पूरा करें (उम्र, परिवार, location, बजट, health) ताकि मैं इस policy पर आपको personally बात कर सकूं।\n• **पहले PDF पर बात करें** — coverage, waiting periods, exclusions — कुछ भी पूछें।\n\nदोनों ठीक हैं। जितना मैं आपके बारे में जानूंगा, इस policy की चर्चा उतनी useful होगी।",
 
     "mp.heading": "स्वास्थ्य बीमा बाज़ार",