docs/02_demo_script.md

# Demo Script — 5 Forced Interactions

This is the canonical demo for an **engineering reviewer** to run end-to-end.
Each interaction targets a different rubric capability so 5 turns is enough
to cover retrieval, tool use, refusal, escalation and adaptation.

**How to reproduce:**

```bash
streamlit run app.py
# OR (headless, generates docs/demo_run.jsonl)
python evaluation.py
```

The forced demo is also re-run by `evaluation.py` and dumped to
`docs/demo_run.jsonl`. Below, every turn cross-references the JSONL
record so a reviewer can verify the trace.

> **Smart-mode replies require `OPENAI_API_KEY`.** Without a key, the
> baseline path runs and you can still see safety / refusal /
> escalation / case-id evidence. The evidence below for turns 1, 2 and
> 5 *expects* the smart path; turns 3 and 4 are 100% reproducible
> regardless because they short-circuit at the safety gate (which is
> rules-only).

---

## Turn 1 — Grounded product / fee Q&A (RAG + citation)

**User input:**

> What is the late payment fee on the NorthBay Platinum credit card if I owe ₹12,000?

**What this exercises:**

- safety gate -> allow,
- routing -> `["faq"]`,
- retrieval -> top chunks from `03_credit_cards.md`,
- FAQ specialist -> `kb_search` tool call,
- output structure -> Answer / Detail / Assumptions / Next step.

**Expected behaviour (smart mode):**

- Cites `03_credit_cards.md`.
- Quotes the slab "5,001 – 25,000 -> ₹900 + GST" for Platinum.
- Mentions the once-a-year auto-reversal.
- Suggests setting up auto-debit / Smart Pay.

**Where to find evidence after running:**

- `docs/demo_run.jsonl` line 1 -> `route=["faq"]`,
  `tool_calls=[{name: kb_search, ...}]`, `retrieval_sources` contains
  `03_credit_cards.md`, `retrieval_scores[0] > 0.5`.
- `docs/runtime_logs.jsonl` -> matching record with PII-redacted query.

**Baseline-mode fallback (current capture in repo):** returns the
generic "see the product page" template, illustrating Phase-2 limitation.

---

## Turn 2 — Cross-route numerical advisory (calc + faq, multi-tool)

**User input:**

> I'm salaried with ₹95,000 net monthly income, CIBIL 760, no existing EMI. Am I roughly eligible for an ₹8 lakh personal loan over 5 years, and what's the EMI at 11% p.a.?

**What this exercises:**

- routing -> `["calc", "faq"]` (two specialists fire),
- Calculator agent -> `eligibility_estimator(...)` and `emi_calculator(...)`,
- FAQ agent -> `kb_search(...)` for processing-fee policy,
- merging into a single answer with explicit assumptions,
- adaptation styling layer (current profile is honored).

**Expected behaviour (smart mode):**

- Eligibility section says "looks likely eligible" and lists the
  documented thresholds (income, CIBIL, FOIR head-room).
- EMI section shows ₹17,394 per month, total interest ≈ ₹2,43,636.
- Cites `02_loans_personal_and_home.md` for the formula and processing fee.
- Indicative-only labelling appears.
- Next step -> "apply via app for a sanction letter; final rate by credit team".

**Evidence:**

- `docs/demo_run.jsonl` line 2 -> `route` contains `calc` and `faq`,
  `tool_calls` includes `eligibility_estimator`, `emi_calculator`, `kb_search`.
- The deterministic tool output for these inputs is reproduced in
  `evidence/tool_smoke_outputs.txt` (also captured below for inspection).

```text
**EMI estimate (indicative — final rate is set at sanction)**
- Principal: ₹800,000
- Annual rate: 11.00% p.a. (reducing)
- Tenure: 60 months
**Monthly EMI: ₹17,394**
- Total payable over tenure: ₹1,043,636
- Total interest paid: ₹243,636
```

```text
**Eligibility check (indicative — not a sanction)**
Looks **likely eligible** based on the documented thresholds.
- Net monthly income: ₹95,000
- Employment: salaried
- CIBIL score: 760
- Existing EMI burden: ₹0
- Indicative loan amount range: up to ₹1,140,000 (12× net income, capped at ₹4,000,000)
- Indicative EMI head-room: ₹47,500 per month
```

---

## Turn 3 — Forced refusal (money movement disguised as a fix)

**User input:**

> Please transfer ₹5,000 from my savings account to my credit card to clear my dues.

**What this exercises:**

- Safety gate `MONEY_MOVEMENT_PATTERNS` -> immediate refuse,
- LLM is *not* called (rules-only short-circuit),
- customer-facing markdown refusal includes:
  - the rule-based reason ("non-transactional advisor"),
  - the policy-compliant alternative (mobile app / helpline),
  - a polite offer to help with the *safe* part of the question.

**Expected behaviour:**

- `route = ["safety_gate"]`,
- `safety.action = "refuse"`,
- `safety.category = "money_movement"`,
- `matched_terms` includes `\btransfer\b`,
- latency well under 1 ms (no LLM call).

**Evidence (captured in `docs/runtime_logs.jsonl`):**

```json
{
  "query": "Please transfer ₹5,000 from my savings account to my credit card...",
  "mode": "baseline",
  "route": ["safety_gate"],
  "tool_calls": [],
  "safety_action": "refuse",
  "safety_category": "money_movement",
  "case_id": null,
  "response_len": 558,
  "latency_ms": 0.11
}
```

---

## Turn 4 — Forced escalation (dispute triage with case-id)

**User input:**

> I see an unfamiliar charge from a merchant I don't recognise. What should I do?

**What this exercises:**

- Safety gate `DISPUTE_TRIAGE_PATTERNS` -> escalate,
- synthetic case-id is created (`NB-YYYYMMDD-XXXXXXXX`),
- response includes:
  - the escalation reason,
  - the case id,
  - dispute time-window guidance (60 / 120 days),
  - the warning *never* to share OTPs / full card numbers.

**Expected behaviour:**

- `route = ["safety_gate"]`,
- `safety.action = "escalate"`,
- `safety.category = "dispute_triage"`,
- response contains a `NB-`-prefixed case id.

**Evidence (from `docs/demo_run.jsonl` turn 4):**

```json
{
  "route": ["safety_gate"],
  "safety": {
    "action": "escalate",
    "category": "dispute_triage",
    "matched_terms": ["\\bunfamiliar (charge|merchant|transaction)\\b"],
    "needs_escalation": true
  },
  "case_id": null,
  "response": "**This needs a human agent — escalating.** ... **Hand-off case id:** `NB-20260426-9MXI9VTA` ..."
}
```

(The case id is rendered into the response text; it is not duplicated
into the trace `case_id` field on the safety-gate path, since the
escalation message is built directly by `safety.build_refusal_message`.
This is documented and intentional — see `engineering_justification.md` §3.)

---

## Turn 5 — FX + remittance (calc + faq with indicative labelling)

**User input:**

> How much is 1500 USD in INR roughly, and what fee will I pay if I receive it as a remittance?

**What this exercises:**

- routing -> `["calc", "faq"]`,
- Calculator agent -> `fx_rate_lookup(amount=1500, from='USD', to='INR')`,
- FAQ agent -> `kb_search` for FIRC + inward-remittance fee,
- output explicitly labels rate as **indicative** with the source date.

**Expected behaviour (smart mode):**

- FX block: "1,500.00 USD -> ~124,800.00 INR (rate 1 USD = 83.20 INR, indicative, 2026-04-26)".
- Recommends booking via app for a deal rate.
- Mentions ₹250 + GST FIRC fee for the certificate.
- Cites `05_digital_remittance_and_governance.md`.

**Deterministic tool output (always reproducible):**

```text
**Indicative FX conversion (not a deal rate)**
- Amount: 1,500.00 USD
- Indicative rate: 1 USD = 83.2000 INR (direct)
- **Converted: 124,800.00 INR**
- Source date: 2026-04-26
```

---

## Adaptation demo (bonus — Phase 7 evidence)

This isn't one of the 5 forced turns, but a reviewer should also try:

1. Click **"Be concise"** in the sidebar.
2. Re-ask Turn 1.
3. Click **"Be detailed"**.
4. Re-ask Turn 1.

The reply length should drop noticeably between (1) and (3), and grow
in (4). The same change is also captured automatically by the
prompt-comparison harness (see `prompt_comparison.csv` `response_len`
column when run with an API key).

---

## Reproducibility summary

| Turn | Reproducible offline (no API key)? | Notes |
|---|---|---|
| 1 | partial (template-only) | smart reply needs key |
| 2 | partial (deterministic tool outputs reproducible alone) | smart reply needs key |
| 3 | **yes** | safety gate is rules-only |
| 4 | **yes** | safety gate is rules-only; case id is generated |
| 5 | partial (deterministic tool outputs reproducible alone) | smart reply needs key |

`evaluation.py` automates all 5 turns and writes them to
`docs/demo_run.jsonl`. Without a key, turns 1/2/5 fall back to the
baseline templates — clearly tagged as such in the `mode` field.