traces_for_coding.md

# G.U.I.DE — Traces for Coding

You are given interactions between a user and an GUIDE agent.
Each trace shows what the user asked and what the agent did.

**Your task**

1. **Open coding** — read each trace and write short labels (codes) that
   describe what you observe. Use your own words. Things to notice: prices,
   discounts, missing information, wrong items, refusals, defaults, modifiers.
   Aim for 2–4 codes per trace.

2. **Axial coding** — after coding all 18 traces, group your open codes into
   broader categories. Give each category a name and list the traces that belong
   to it. Look for relationships between categories.

There are no right or wrong codes at this stage.

---

## Trace 01

**Customer said:**  user: Indian bank charged 1000 rupee for international transaction with wrong accuse reference id  REF-20260501-001 on 15th june 2026 . I demand a full refund  


**Outcome:** Able to Extract information except bank

**Agent response:** 

assistant: Thank you. I've captured the following so far:

- **Amount:** ₹1000 (incorrect international transaction charge)
- **Reference ID:** REF-<US_BANK_NUMBER>-001
- **Date of incident:** 15 June 2026
- **Desired resolution:** Full refund

I just need a couple more details before drafting your letter.

**Which bank charged you this amount?** Please share the name of the Indian bank involved (your message mentioned "Indian bank" generically).

**_Open codes:_**
Indian bank is not being considered as bank instead it taking generic indian word because it's not used in NER training
In Verify Entities tab company name, amount was not filled out
When date was given followed by person name during the initial complaint  ( person on 01/05/2026). The redactor replaced it with a `<PERSON>` placeholder.  
Reference id do not have be redacted by presidio
Document processing failed: tesseract is not installed or it's not in your PATH. See README file for more information.
_Notes:_


## Trace 02

**Customer said:**  user: Axis bank charged 1000 rupee for late fee of not paying loan for my car reference id  REF-20260501-001 on 17th june 2026 . I demand a full refund. I've already reached to bank but they refused to refund. 

**Agent response:** 

assistant: Read the message and extracted entities rightly and provided escalation path.

**_Open codes:_**
In verify entties tab, it was given as false for "Have you already contacted the company" even though customer says yes during the complaint
Draft not showing on complaint draft tab after clicking "Confirm & generate Draft"
Document processing failed: tesseract is not installed or it's not in your PATH. See README file for more information.

## Trace 03

**Customer said:**  I took a loan of 100000 from Bank of baroda on may 1st 2026, it was not processed more than 10 days. loan refrence id REF-199384. I demand an explanation of delay. I reached to bank but they did not respond back 

**Agent response:** Read the complain and extracted entities rightly and provided escalation path.


**_Open codes:_**
Draft not showing on complaint draft tab after clicking "Confirm & generate Draft"
Document processing failed: tesseract is not installed or it's not in your PATH. See README file for more information.

## Trace 04
**Customer said:**  I recharged for 5450 rupee in airtel for 2 years on 3rd june, 2026.  my phone is number is 8092997182. recharge was successful. but it's not reflected. transaction reference id is TRN-9038493. I demand an immediate recharge.

**Agent response:** Read the complain and extracted entities rightly. and provided escalation path 

**_Open codes:_**
prior contact is not specified. when given yes upon agent prompted. but it's again being interpreted as "No" instead of "Yes"
mobile number is redacted on logs. but not shown on the draft. 
Draft not showing on complaint draft tab after clicking "Confirm & generate Draft".
Document processing failed: tesseract is not installed or it's not in your PATH. See README file for more information.

## Trace 05
**Customer said:** I bought a new sim card from BSNL for rs 450 on 11th january,2026. but sim card is not yet activiated. transaction reference is TRN-1345566. BSNL customer care did not respond back after the initial complaint. i demand an immediate activation of simcard. 

**Agent response:** 
The automatic classifier suggested "banking," but this complaint is clearly about a **telecom** service (BSNL SIM card activation) — BSNL is a telecommunications provider, not a bank. The classifier appears to have misjudged this

**open Codes:**
classifier is wrong

## Trace 06
**Customer said:**  I bought an health insurance from Niva Bupa for single year rs. 8000 on 3rd september 2025. but it was not activated transactoin id is TRN-12345664. I did not proper response from niva bupa customer care. I demand an immediate policy activation

**Agent response:** 
Asked to verify the entities

**_Open codes:_**
verified entities tab was filled out empty. In tool call extract_entities details are extracted


## Trace 07
**Customer said:** 
home loan for 35 lakhs from ICICI bank was not yet sanctioned. I requested for loan on June2nd, 2026. transaction id: TXN9234343. when prior contacted with bank, I did not recieve proper response from bank. i need resolution of credit to immediate fund 

**Agent reponse:** 
extracted entities of loan amount, bank name, date, transaction id, prior contact. asked clarification for desired resolution, sanction and dispurse loan immediately

**open codes:** 
after giving all details. agent forcing an yes or no confirmation at the end. 
draft complaint tab not showing draft.

## Trace 08
**Customer Said:** 
Joint account is not opened yet on axis bank. application is requested on month ago. reference id: REF-19334.when prior contacted with bank, I did not recieve proper response from bank.I demand an resolution of only proper reason on problem in opening the account

**Agent response:** 
Asked to verify the entities

**_Open codes:_**
verified entities tab was filled out empty. In tool call extract_entities details are extracted

## Trace 09

**Customer Said:** 
I placed an order on myntra by 1st june,2026 for 7000 rs. item not yet delivered. order id is OD-124564.  I contacted myntra two times but no proper response. i demand only full remand for the order. 

**Agent response:** 
extracted entities of  amount, compnay name, date, order id, prior contact. 

**_Open codes:**
Good response except Draft not showing on complaint draft tab after clicking "Confirm & generate Draft".

## Trace 10

**Customer Said:** 
I booked an hotel via agoda app on 17th june 2026 for 2000 rs. i got an order confirmation. od-12345. but when i enquired hotel. they are objecting. no prior contact to agoda. i demand only full remand for my order

**Agent response:** 
extracted entities of  amount, compnay name, date, order id, prior contact. 

**_Open codes:**
Good response except Draft not showing on complaint draft tab after clicking "Confirm & generate Draft".

## Axial Coding Worksheet

After coding all  traces, complete the table below.
Add or remove rows as needed.

| Category name | Description | Trace numbers |
| --- | --- | --- |
| empty_draft | Draft tab does not render the complaint after clicking "Confirm & Generate Draft", even though the agent's `draft_complaint` tool call succeeds. The letter is generated in the chat but not surfaced in the UI tab. | 2, 3, 4, 7, 9, 10 |
| verify_entities_tab_empty | The Verify Entities tab is pre-filled with empty/default values even when `extract_entities` has successfully returned structured data — indicating a disconnect between tool call output and the UI state population. | 6, 8 |
| pii_over_redaction | Presidio misidentifies non-PII tokens as PII and redacts them: a date following a person's name is consumed as part of the `<PERSON>` span (Trace 1), and the numeric part of a REF_ID is captured as `<US_BANK_NUMBER>` (Trace 1). | 1 |
| prior_contact_not_retained | The agent correctly captures "yes, I contacted" from the user's message but the HITL form defaults `prior_contact` to `false`/`No`, discarding the user's stated value before draft generation. | 2, 4 |
| domain_misclassification | The DomainClassifier assigns the wrong domain: BSNL SIM activation is labelled `banking` instead of `telecom` (Trace 5). Indicates the DistilBERT classifier does not generalise reliably to all telecom providers. | 5 |
| ner_entity_gap | EvidenceNER fails to extract an entity that is clearly present in the complaint: "Indian bank" is not recognised as an `ORG` because the phrase was not covered in synthetic training data. | 1 |
| tesseract_not_installed | Every trace that involves document upload fails with "tesseract is not installed or it's not in your PATH", blocking the OCR pipeline entirely. System-level dependency missing from setup. | 1, 2, 3, 4 |
| agent_confirmation_loop | After all required fields are provided, the agent asks an additional yes/no confirmation question rather than proceeding directly to the HITL gate, causing unnecessary friction for the user. | 7 |


**Relationships between categories**:

- **empty_draft** and **verify_entities_tab_empty** are both UI-layer failures: the agent and tool calls succeed but the results do not propagate into the correct Gradio tab. They likely share the same root cause — the frontend state-update logic (`gr.State` / tab refresh) is not triggered correctly after HITL confirmation.
- **pii_over_redaction** directly causes **ner_entity_gap** in Trace 1: the REF_ID is partially consumed by Presidio's `US_BANK_NUMBER` recogniser, leaving a malformed token that EvidenceNER cannot match to a `REF_ID` pattern.
- **prior_contact_not_retained** and **verify_entities_tab_empty** are both HITL form population failures — the Verify Entities tab is not seeded from the agent's extracted context, so fields that the agent already knows are lost at the confirmation step.
- **domain_misclassification** (Trace 5) feeds into **agent_confirmation_loop** behaviour: when the classifier is wrong, the agent must ask a clarifying domain question, which can cascade into extra confirmation turns before the HITL gate.
- **tesseract_not_installed** is an independent infrastructure gap that blocks all document-upload paths but does not affect text-only complaint flows.