rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
dd6172a
Β·
1 Parent(s): 2bb3898

docs: sync KI-071..KI-077 to ENTERPRISE_AUDIT + CLAUDE.md (post-KI-070 safeguards + Name field)

Browse files

After KI-070 shipped the single-LLM-call fact-find, six follow-up KIs
landed to make the experience robust under real-world LLM degradation
and to surface the captured profile in the UI. Recording them in
canonical docs so they're traceable across the repo.

ENTERPRISE_AUDIT.md "Fixes shipped today" β€” added 7 rows:
- KI-071 β€” 12-file docs reconciliation against actual code reality
- KI-072 β€” P0 fallback captures user's answer to last-asked slot
- KI-073 β€” Clear chat resets profile-completeness chip synchronously
- KI-074 β€” P0 greedy multi-slot capture + slot-trigger guards
- KI-075 β€” Brain timeout 12s β†’ 25s (root cause of "still robotic")
- KI-076 β€” HF Dataset viewer disabled (CastError gone)
- KI-077 β€” Name field + profile-panel pre-fill from chat state

CLAUDE.md "Fact-find loop" section extended with two new bullets
documenting the safeguards added since KI-070:
- KI-072/074 β€” fallback path no longer dead-ends; greedy multi-slot
capture with trigger guards
- KI-075 β€” 25s timeout chosen for NIM cold-start + 1 fallback hop

Triple-pushed (HF Space + GitHub).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (2) hide show
  1. 80-audit/ENTERPRISE_AUDIT.md +7 -0
  2. CLAUDE.md +2 -0
80-audit/ENTERPRISE_AUDIT.md CHANGED
@@ -258,6 +258,13 @@ The full eval was sending **every** QA question to `needs_finder`. Sample bot an
258
  | KI-068 | `26023b8` | Humanized fact-find readback (`primary_goal: first_buy` β†’ "goal: first health policy"; `30k_60k` β†’ "β‚Ή30,000–60,000/year"; `metro` β†’ "metro city"). Stripped `**bold**` markdown wrapper that was leaking as literal asterisks in chat + being read by TTS as "asterisk asterisk". Made KI-063 logging fire-and-forget. |
259
  | KI-069 | `26023b8` | Fixed KI-059 false-positive: regex was matching "this is correct" / "I'm good" / "first, show me ..." and routing users back to the name re-ask loop. Added `_NON_NAME_TOKENS` blocklist (50+ confirmatory words) + uppercase-name-only validation. **Now superseded by KI-070 (brain doesn't need this regex).** |
260
  | **KI-070** | **`364591b`** | **Single-LLM-call fact-find replaces 3-layer template stitching ([ADR-030](../70-docs/60-decisions/ADR-030-llm-driven-fact-find.md)).** Orchestrator fact-find branch ~387 β†’ ~95 lines. New `backend/fact_find_brain.py` (441 LOC). Deleted: `backend/question_paraphraser.py`, `_pick_opener`, `_NEUTRAL_OPENERS`, `_FAMILY_OPENERS`, `_contains_self_introduction`. Native multi-fact capture verified live 2026-05-15: opener "Hi, I'm Rohit Sar. I'm 32, just myself, living in Mumbai." captured `{name, age, dependents, location_tier}` in ONE turn. |
 
 
 
 
 
 
 
261
  | D-001 | (multi) | ChromaDB HNSW bloat 3-layer prevention (ADR-029) |
262
  | D-002 | (LaunchAgent edit) | Three silently-failing LaunchAgent scripts fixed |
263
  | D-009 | `bcb7079` | Removed `tmp_*.py` debug files from repo root |
 
258
  | KI-068 | `26023b8` | Humanized fact-find readback (`primary_goal: first_buy` β†’ "goal: first health policy"; `30k_60k` β†’ "β‚Ή30,000–60,000/year"; `metro` β†’ "metro city"). Stripped `**bold**` markdown wrapper that was leaking as literal asterisks in chat + being read by TTS as "asterisk asterisk". Made KI-063 logging fire-and-forget. |
259
  | KI-069 | `26023b8` | Fixed KI-059 false-positive: regex was matching "this is correct" / "I'm good" / "first, show me ..." and routing users back to the name re-ask loop. Added `_NON_NAME_TOKENS` blocklist (50+ confirmatory words) + uppercase-name-only validation. **Now superseded by KI-070 (brain doesn't need this regex).** |
260
  | **KI-070** | **`364591b`** | **Single-LLM-call fact-find replaces 3-layer template stitching ([ADR-030](../70-docs/60-decisions/ADR-030-llm-driven-fact-find.md)).** Orchestrator fact-find branch ~387 β†’ ~95 lines. New `backend/fact_find_brain.py` (441 LOC). Deleted: `backend/question_paraphraser.py`, `_pick_opener`, `_NEUTRAL_OPENERS`, `_FAMILY_OPENERS`, `_contains_self_introduction`. Native multi-fact capture verified live 2026-05-15: opener "Hi, I'm Rohit Sar. I'm 32, just myself, living in Mumbai." captured `{name, age, dependents, location_tier}` in ONE turn. |
261
+ | KI-071 | `2e60476` | Docs-vs-code reconciliation: 12 files updated to reflect actual chain primaries (Qwen 3-Next 80B brain / Nemotron Nano 30B fast brain / Mistral Large 3 675B judge per D-022 brain swap). DeepSeek V4-Pro / V4-Flash + Llama-4 Maverick correctly documented as fallback chain entries. README Β§1.2 + Β§4.3 rewritten; ADR-019 D-022 supersession note; kb/security + kb/eval INDEX; frontend/eval/rag subdirectory READMEs; ADR-029 filename + ADR-028 filename + env-var name fixed; dead `input.hands_free` i18n keys deleted; `welcome.subtitle` softened to "a few short questions" for KI-070's variable turn count. |
262
+ | KI-072 | `e098e1b` | **P0 fix.** `_canonical_fallback` now applies the user's current message to whichever slot was last asked (read from `session.awaiting_question_id`) via legacy normalizer BEFORE picking next slot. Previous behaviour wedged fact-find when LLM brain failed: "I am Don" / "Don" / "Don Jon" all bounced off the name slot's canonical re-ask. |
263
+ | KI-073 | `c53381c` | Frontend Clear chat now explicitly resets `profileCompleteness` state immediately so the "55% DONE" header chip clears synchronously for the new visitor, regardless of network latency on the backend session-reset call. |
264
+ | KI-074 | `4516e87` | **P0 fix.** KI-072 only checked the awaiting slot; if the LLM brain was driving slot X but the user supplied Y, Y was dropped. Now `_canonical_fallback` GREEDILY runs `_normalize_for_slot` against every unfilled slot in priority order (age β†’ dependents β†’ income_band β†’ existing_cover β†’ primary_goal β†’ location β†’ parents_age β†’ budget β†’ name), with slot-specific trigger guards to prevent cross-contamination ("29 years old" was getting written into existing_cover_inr AND parents_age_max AND age before the trigger guards). Name parser tightened: explicit-intro-only, 50+ word blocklist (my/your/first/looking/...), conjunction stop ("Rohit Sar and I am 32" β†’ "Rohit Sar"). Also stripped `**...**` markdown leak from health_conditions slot (different GRAPH entry than KI-068). |
265
+ | **KI-075** | **`5fc01a7`** | **Root cause of "still robotic" UX.** Live probe showed 4 of 5 fact-find turns hit `_TIMEOUT_S = 12s` asyncio.wait_for cap at exactly 13.2s latency β€” NIM cold-start eats 10-15s after a Space rebuild. Outer wait_for was killing brain calls BEFORE the chain's internal 22s `total_budget_s` could try cross-provider fallbacks (Groq, OpenRouter). Bumped `_TIMEOUT_S` to 25s. Brain success rate climbs from ~20% (1/5) to expected 80%+ for cold-start sessions; near-100% once warm. |
266
+ | KI-076 | (HF dataset) | Disabled the `rohitsar567/insurance-bot-data` dataset viewer by uploading a README with `viewer: false` YAML frontmatter. The viewer was failing with `StreamingRowsError: CastError` on heterogeneous JSON shapes (PDFs + Chroma binary + multi-schema JSONs). Dataset itself stays fully public + the HF Space `snapshot_download` is unaffected (schema-agnostic). Page now shows clean "Viewer disabled" notice + the new README we wrote. |
267
+ | KI-077 | `2bb3898` | "Build your profile" panel: added Name input field at top with "captured from chat" badge when populated. Backend `/api/profile/completeness` + `/api/profile` POST + `UserProfile` TypeScript type all extended with `name`. Panel pre-fills every field from the session's captured chat state via existing `initialProfile`. New `useEffect` keeps panel in sync when chat captures fields while panel is open. On Save, name persists to the named-profile JSON store (KI-040/062) so the user is auto-recognised on return visits. |
268
  | D-001 | (multi) | ChromaDB HNSW bloat 3-layer prevention (ADR-029) |
269
  | D-002 | (LaunchAgent edit) | Three silently-failing LaunchAgent scripts fixed |
270
  | D-009 | `bcb7079` | Removed `tmp_*.py` debug files from repo root |
CLAUDE.md CHANGED
@@ -37,6 +37,8 @@ Every LLM role is a `NimChainLLM` fallback chain, NOT a hardcoded single model.
37
  - **Safeguards.** JSON-block-must-parse β†’ fall to canonical `next_question(slot_id)`. Slot-not-progressing (3 turns same slot, no captures) β†’ bail to canonical. Hard 12s budget. Any chain exhaustion β†’ canonical fallback. Fact-find can never wedge.
38
  - **`backend/needs_finder.py::GRAPH`** retained as the safeguard fallback path only β€” never the primary path in steady state, but always available.
39
  - **DELETED in KI-070** (~500 LOC): `backend/question_paraphraser.py` module, `_pick_opener`, `_NEUTRAL_OPENERS` / `_FAMILY_OPENERS` constants, `_contains_self_introduction`, the KI-067 first-policy regex (brain captures natively from prose), the acknowledger template selection.
 
 
40
  - **Natural-conversation escape (KI-045):** intent_change phrases or off-topic questions still exit fact-find by routing through `should_route_to_fact_find` β€” handled upstream of `drive_fact_find` so the safeguard mechanism here applies to in-fact-find pivots.
41
  - **Indic queries** route through Sarvam-M for translation on input + output; the fact-find brain itself runs in English on the translated text.
42
 
 
37
  - **Safeguards.** JSON-block-must-parse β†’ fall to canonical `next_question(slot_id)`. Slot-not-progressing (3 turns same slot, no captures) β†’ bail to canonical. Hard 12s budget. Any chain exhaustion β†’ canonical fallback. Fact-find can never wedge.
38
  - **`backend/needs_finder.py::GRAPH`** retained as the safeguard fallback path only β€” never the primary path in steady state, but always available.
39
  - **DELETED in KI-070** (~500 LOC): `backend/question_paraphraser.py` module, `_pick_opener`, `_NEUTRAL_OPENERS` / `_FAMILY_OPENERS` constants, `_contains_self_introduction`, the KI-067 first-policy regex (brain captures natively from prose), the acknowledger template selection.
40
+ - **Fallback path is no longer a dead-end (KI-072 / KI-074).** When the brain times out / chain exhausts / `<FF>` block fails to parse, `_canonical_fallback` doesn't just return the canonical question β€” it GREEDILY applies the user's current message to every unfilled slot via `_normalize_for_slot`, applies captures to the profile, then picks the next still-empty slot. Slot-specific trigger guards (age needs "years old" or bare number; existing_cover needs β‚Ή/lakh/Cr cue or denial; name needs explicit "I'm X" intro pattern) prevent cross-contamination. So a NIM-degraded session walks fact-find via canonical questions but still progresses β€” never wedges.
41
+ - **Brain timeout: 25s (KI-075).** The `asyncio.wait_for` wrap around `drive_fact_find` was 12s originally β€” too tight. NIM cold-start eats 10-15s after a Space rebuild, killing the brain call before the chain's internal 22s `total_budget_s` could try Groq/OpenRouter fallbacks. 25s gives cold-start headroom + 1 chain fallback hop.
42
  - **Natural-conversation escape (KI-045):** intent_change phrases or off-topic questions still exit fact-find by routing through `should_route_to_fact_find` β€” handled upstream of `drive_fact_find` so the safeguard mechanism here applies to in-fact-find pivots.
43
  - **Indic queries** route through Sarvam-M for translation on input + output; the fact-find brain itself runs in English on the translated text.
44