Spaces:
Sleeping
Sleeping
| ## Why | |
| Qualitative trace analysis across 10 complaint flows identified 8 reproducible bugs spanning all 4 system layers. Left unaddressed, these cause silent data loss (prior_contact defaulting to false), blank UI panels (Verify Entities and Complaint Draft tabs never populating), misrouted complaints (BSNL classified as banking), and a broken document-upload path (Tesseract not found). Fixing them now, before additional traces are collected, ensures future evaluations test the system rather than its infrastructure gaps. | |
| ## What Changes | |
| - **UI — structured entity extraction**: Agent emits a fenced JSON block alongside its Rule 4 HITL summary; UI parses the JSON dict instead of regex-scraping reply prose to populate Verify Entities form fields. | |
| - **UI — prior_contact default**: Guard `prior == "Yes"` so a blank/None field does not silently become `False` in the confirmed entity payload. | |
| - **UI — draft extraction**: Relax `_extract_draft` regex to handle agent replies that omit the closing `---` marker or vary the `Subject:` line position. | |
| - **Prompt — Rule 4/5 tightening**: Clarify that emitting the HITL summary block is the gate action; agent must not ask a follow-up yes/no confirmation turn before the user reaches the Verify Entities tab. | |
| - **Classifier — telecom rebalancing**: Add SIM-activation and prepaid-recharge sentence templates; raise `supplement_per_class` from 2,000 to 5,000. Requires checkpoint retrain. | |
| - **NER — ORG coverage**: Add generic and lesser-known org tokens ("Indian bank", "Niva Bupa", "Agoda") to the NER synthetic ORG list. Requires checkpoint retrain. | |
| - **Privacy — REF_ID protection**: Register a deny-list PatternRecognizer in `PIIRedactor` that prevents `US_BANK_NUMBER` from consuming `REF-`, `TRN-`, `TXN-`, `OD-` prefixed complaint identifiers. | |
| - **Infrastructure — Tesseract pre-flight**: Add a startup check in `start.py` that detects a missing `tesseract` binary and exits with a clear install message before servers are launched. | |
| ## Capabilities | |
| ### New Capabilities | |
| - `hitl-structured-entities`: Agent emits structured entity JSON at the HITL gate; UI consumes it directly to populate Verify Entities form without regex parsing. | |
| - `complaint-id-pii-protection`: Deny-list recognizer shields complaint reference ID formats from false-positive Presidio redaction. | |
| - `tesseract-preflight`: Startup guard that validates the Tesseract system dependency before the server starts accepting requests. | |
| ### Modified Capabilities | |
| - `domain-classification`: Telecom supplement templates and sample count increased to correct class imbalance causing BSNL SIM complaints to be misclassified as banking. | |
| - `evidence-ner`: ORG entity list extended with generic and lesser-known Indian org names to reduce NER miss rate on real complaint inputs. | |
| ## Non-goals | |
| - Streaming UI (Gradio text streaming from the agent loop) — separate concern. | |
| - Replacing Tesseract with a cloud OCR fallback — out of scope; Tesseract remains a hard requirement. | |
| - Changing the 6-domain taxonomy or regulatory routing rules. | |
| - Fixing any bugs not identified in the 10-trace analysis. | |
| ## Impact | |
| - `ui/app.py`: `_parse_hitl_entities`, `_extract_draft`, `_handle_confirm` (prior_contact guard) | |
| - `src/agent/prompts.py`: Rule 4 and Rule 5 text | |
| - `src/classifier/train.py`: `_SUPPLEMENT_TEMPLATES["telecom"]`, `--supplement_per_class` default | |
| - `src/ner/train.py`: `_ENTITY_VALUES["ORG"]` | |
| - `src/privacy/redactor.py`: new `_complaint_id_deny_recognizer()` + registration | |
| - `start.py`: Tesseract binary check before `uvicorn` / Gradio launch | |
| - Model checkpoints `models/domain_classifier/` and `models/evidence_ner/` must be regenerated after ML changes | |