rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
d89a871
·
1 Parent(s): e610ee9

feat(profile): KI-063 — shown / selected / rejected policy tracking on profile

Browse files

Adds per-user, persistent policy interaction tracking so the bot remembers
across sessions which policies were shown to a user, which they shortlisted,
and which they rejected.

Schema (backend/needs_finder.py)
• Profile gains three list fields: shown_policies, selected_policies,
rejected_policies. Each entry: {policy_slug, insurer, event_at (ISO Z),
session_id, reason}.

Storage helpers (backend/profile_store.py)
• record_policy_event(persona_id_or_name, profile, event_type, policy_slug,
insurer, session_id, reason) — appends or dedup-updates on
(policy_slug, event_type); persists via existing save_profile so
persona-id keying stays consistent.
• get_shortlist(profile) — convenience wrapper around selected_policies.

Orchestrator wiring (backend/orchestrator.py)
• After every recommendation / comparison turn that passes faithfulness
and was issued by a named profile, iterate citations and log
event_type='shown' / reason='shown_in_recommendation'. Wrapped in a
try/except so logging never breaks the chat reply. Anonymous sessions
are skipped.
• _format_known_profile_summary now appends "Your shortlist: <insurer ·
slug>, ..." when selected_policies is non-empty, so KI-061's
welcome-back greeting surfaces the user's shortlist.

User-facing endpoints (backend/admin.py)
• POST /api/profile/select — {session_id, policy_slug, insurer, reason?}
• POST /api/profile/reject — {session_id, policy_slug, insurer, reason?}
Both look up the session, require profile.name (400 otherwise), and
delegate to record_policy_event. Returns {ok, event_type, policy_slug,
count}. NOT admin-gated — these are user actions.

Verification
• py_compile clean across all four files.
• Inline round-trip test: record one shown / selected / rejected event,
reload from disk, all three lists populate with correct shape + reason.
• Dedup test: two shown events for the same policy_slug → list length
stays 1, event_at updates to the later timestamp, session_id refreshes.
• Welcome-back summary correctly appends "Your shortlist: ..." line when
selected_policies is non-empty.
• tests/test_routing_regression.py: 15 passed, 13 subtests passed.

Frontend buttons are deferred — endpoints are exposed for when KI-063b lands.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend/admin.py CHANGED
@@ -302,6 +302,81 @@ async def admin_profiles(
302
  }
303
 
304
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
305
  # ---------------------------------------------------------------------------
306
  # /api/admin/performance — aggregated performance/quality metrics
307
  # ---------------------------------------------------------------------------
 
302
  }
303
 
304
 
305
+ # ---------------------------------------------------------------------------
306
+ # KI-063 (2026-05-15) — user-facing profile-event endpoints.
307
+ #
308
+ # These are NOT admin-gated — they're invoked by the frontend when a logged-
309
+ # in user (one with a stored profile.name) clicks the select/reject buttons
310
+ # on a policy card. Both look up the session, validate that the session has
311
+ # a named profile, then append the event through `profile_store.record_policy_event`.
312
+ #
313
+ # Anonymous sessions (no profile.name) get 400 — there's no key to persist
314
+ # against. The frontend should hide the buttons in that case.
315
+ # ---------------------------------------------------------------------------
316
+
317
+
318
+ class _PolicyEventBody(BaseModel):
319
+ session_id: str
320
+ policy_slug: str
321
+ insurer: str
322
+ reason: Optional[str] = None
323
+
324
+
325
+ def _do_record_policy_event(body: _PolicyEventBody, event_type: str) -> dict:
326
+ """Shared handler for /api/profile/select + /api/profile/reject."""
327
+ if not body.session_id or not body.policy_slug or not body.insurer:
328
+ raise HTTPException(
329
+ status_code=400,
330
+ detail="session_id, policy_slug, and insurer are required",
331
+ )
332
+ from backend.session_state import get_session
333
+ from backend.profile_store import record_policy_event
334
+
335
+ session = get_session(body.session_id)
336
+ if not session.profile.name:
337
+ raise HTTPException(
338
+ status_code=400,
339
+ detail="No named profile on this session — cannot persist event.",
340
+ )
341
+ ok = record_policy_event(
342
+ persona_id_or_name=session.profile.name,
343
+ profile=session.profile,
344
+ event_type=event_type, # type: ignore[arg-type]
345
+ policy_slug=body.policy_slug,
346
+ insurer=body.insurer,
347
+ session_id=body.session_id,
348
+ reason=body.reason,
349
+ )
350
+ if not ok:
351
+ raise HTTPException(status_code=500, detail="profile save failed")
352
+ # Also persist via the session flush so an in-memory consumer (e.g. the
353
+ # welcome-back greeter) reads the same state without a full disk reload.
354
+ session._flush()
355
+ field_name = {
356
+ "shown": "shown_policies",
357
+ "selected": "selected_policies",
358
+ "rejected": "rejected_policies",
359
+ }[event_type]
360
+ return {
361
+ "ok": True,
362
+ "event_type": event_type,
363
+ "policy_slug": body.policy_slug,
364
+ "count": len(getattr(session.profile, field_name, []) or []),
365
+ }
366
+
367
+
368
+ @router.post("/api/profile/select")
369
+ async def profile_select(body: _PolicyEventBody):
370
+ """Record a user clicking "shortlist / save" on a policy card."""
371
+ return _do_record_policy_event(body, "selected")
372
+
373
+
374
+ @router.post("/api/profile/reject")
375
+ async def profile_reject(body: _PolicyEventBody):
376
+ """Record a user clicking "not for me / reject" on a policy card."""
377
+ return _do_record_policy_event(body, "rejected")
378
+
379
+
380
  # ---------------------------------------------------------------------------
381
  # /api/admin/performance — aggregated performance/quality metrics
382
  # ---------------------------------------------------------------------------
backend/needs_finder.py CHANGED
@@ -45,6 +45,15 @@ class Profile:
45
  health_conditions: Optional[list[str]] = field(default_factory=list) # ["diabetes", "hypertension", ...]
46
  asked: list[str] = field(default_factory=list) # question IDs already asked
47
  free_form_session: bool = False # True = user asks free questions, not driven by us
 
 
 
 
 
 
 
 
 
48
 
49
 
50
  # ----------------------------------------------------------------------------
 
45
  health_conditions: Optional[list[str]] = field(default_factory=list) # ["diabetes", "hypertension", ...]
46
  asked: list[str] = field(default_factory=list) # question IDs already asked
47
  free_form_session: bool = False # True = user asks free questions, not driven by us
48
+ # KI-063 (2026-05-15) — per-user policy interaction log so the bot
49
+ # remembers which policies were shown / selected / rejected across
50
+ # sessions. Each entry is a dict with shape:
51
+ # {policy_slug, insurer, event_at (ISO Z), session_id, reason}
52
+ # Dedup at write-time on (policy_slug, event_type) — re-events just
53
+ # bump event_at + session_id rather than appending duplicates.
54
+ shown_policies: list[dict] = field(default_factory=list) # KI-063
55
+ selected_policies: list[dict] = field(default_factory=list) # KI-063
56
+ rejected_policies: list[dict] = field(default_factory=list) # KI-063
57
 
58
 
59
  # ----------------------------------------------------------------------------
backend/orchestrator.py CHANGED
@@ -284,7 +284,12 @@ _KNOWN_FIELD_FORMATTERS: tuple[tuple[str, str, "callable"], ...] = (
284
  def _format_known_profile_summary(profile) -> str:
285
  """Return a comma-separated rundown of what we already know, e.g.
286
  'age 34, covering you + spouse, income ₹10-25L, looking for first
287
- health policy'. Returns '' if nothing meaningful is stored."""
 
 
 
 
 
288
  parts: list[str] = []
289
  for field_name, label, fmt in _KNOWN_FIELD_FORMATTERS:
290
  val = getattr(profile, field_name, None)
@@ -296,7 +301,22 @@ def _format_known_profile_summary(profile) -> str:
296
  rendered = str(val)
297
  if rendered:
298
  parts.append(f"{label} {rendered}")
299
- return ", ".join(parts)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
300
 
301
 
302
  _HELPFUL_GAP_LABELS = {
@@ -1100,6 +1120,45 @@ async def handle_turn(
1100
  except Exception:
1101
  pass # if any step fails, return English — better than mis-translated
1102
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1103
  return TurnResult(
1104
  reply_text=reply,
1105
  citations=citations,
 
284
  def _format_known_profile_summary(profile) -> str:
285
  """Return a comma-separated rundown of what we already know, e.g.
286
  'age 34, covering you + spouse, income ₹10-25L, looking for first
287
+ health policy'. Returns '' if nothing meaningful is stored.
288
+
289
+ KI-063 (2026-05-15) — if the user has selected (shortlisted) policies
290
+ on file, append a "Your shortlist: <insurer · policy>, ..." line so the
291
+ returning-visitor greeting surfaces what they previously saved.
292
+ """
293
  parts: list[str] = []
294
  for field_name, label, fmt in _KNOWN_FIELD_FORMATTERS:
295
  val = getattr(profile, field_name, None)
 
301
  rendered = str(val)
302
  if rendered:
303
  parts.append(f"{label} {rendered}")
304
+ summary = ", ".join(parts)
305
+ # KI-063 — append shortlist line if any selected policies on file.
306
+ shortlist = list(getattr(profile, "selected_policies", None) or [])
307
+ if shortlist:
308
+ bits = []
309
+ for entry in shortlist:
310
+ insurer = (entry.get("insurer") or "").strip()
311
+ slug = (entry.get("policy_slug") or "").strip()
312
+ if insurer and slug:
313
+ bits.append(f"{insurer} · {slug}")
314
+ elif slug:
315
+ bits.append(slug)
316
+ if bits:
317
+ line = "Your shortlist: " + ", ".join(bits)
318
+ summary = f"{summary}. {line}" if summary else line
319
+ return summary
320
 
321
 
322
  _HELPFUL_GAP_LABELS = {
 
1120
  except Exception:
1121
  pass # if any step fails, return English — better than mis-translated
1122
 
1123
+ # KI-063 (2026-05-15) — auto-log "shown" policy events on the persisted
1124
+ # profile so a returning visitor's bot remembers which policies they've
1125
+ # seen. Only fires for context-dependent intents (recommendation /
1126
+ # comparison) AND only when faithfulness passed (we don't log cites that
1127
+ # the safety gates rejected). Anonymous users (no profile.name) get no
1128
+ # log — there's no key to persist against.
1129
+ try:
1130
+ if (
1131
+ intent in ("recommendation", "comparison")
1132
+ and verdict.passed
1133
+ and not blocked
1134
+ and session.profile.name
1135
+ and citations
1136
+ ):
1137
+ from backend.profile_store import record_policy_event
1138
+ seen_slugs: set[str] = set()
1139
+ for cite in citations:
1140
+ slug = cite.get("policy_id") or cite.get("policy_slug")
1141
+ insurer = cite.get("insurer_slug") or cite.get("insurer")
1142
+ if not slug or not insurer or slug in seen_slugs:
1143
+ continue
1144
+ seen_slugs.add(slug)
1145
+ record_policy_event(
1146
+ persona_id_or_name=session.profile.name,
1147
+ profile=session.profile,
1148
+ event_type="shown",
1149
+ policy_slug=slug,
1150
+ insurer=insurer,
1151
+ session_id=session_id,
1152
+ reason="shown_in_recommendation",
1153
+ )
1154
+ except Exception as e:
1155
+ # Never let logging failures break the chat reply.
1156
+ import logging
1157
+ logging.warning(
1158
+ "KI-063 shown_policies log failed (session=%s): %s: %s",
1159
+ session_id, type(e).__name__, str(e)[:200],
1160
+ )
1161
+
1162
  return TurnResult(
1163
  reply_text=reply,
1164
  citations=citations,
backend/profile_store.py CHANGED
@@ -32,7 +32,7 @@ import re
32
  import time
33
  from dataclasses import asdict
34
  from pathlib import Path
35
- from typing import Optional
36
 
37
  from backend.config import settings
38
  from backend.needs_finder import Profile
@@ -207,6 +207,101 @@ def save_profile(name: str, profile: Profile, *, session_id: Optional[str] = Non
207
  return False
208
 
209
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
210
  def list_profiles() -> list[dict]:
211
  """Return summary of all stored profiles — used by the admin Profile +
212
  Visitor Log view. One entry per file."""
 
32
  import time
33
  from dataclasses import asdict
34
  from pathlib import Path
35
+ from typing import Literal, Optional
36
 
37
  from backend.config import settings
38
  from backend.needs_finder import Profile
 
207
  return False
208
 
209
 
210
+ # ---------------------------------------------------------------------------
211
+ # KI-063 (2026-05-15) — per-user policy interaction tracking.
212
+ #
213
+ # Three event types are tracked on the Profile:
214
+ # shown — auto-logged by orchestrator when a policy is cited in a
215
+ # recommendation / comparison turn that passed faithfulness.
216
+ # selected — user clicked "save / shortlist" on a policy card (frontend
217
+ # POSTs to /api/profile/select).
218
+ # rejected — user clicked "not for me" (frontend POSTs to /api/profile/reject).
219
+ #
220
+ # Each entry persists across sessions on the JSON profile, so a returning
221
+ # visitor sees their shortlist and the bot can avoid re-pitching rejected
222
+ # policies.
223
+ # ---------------------------------------------------------------------------
224
+
225
+ _EVENT_TYPE_TO_FIELD = {
226
+ "shown": "shown_policies",
227
+ "selected": "selected_policies",
228
+ "rejected": "rejected_policies",
229
+ }
230
+
231
+
232
+ def record_policy_event(
233
+ persona_id_or_name: str,
234
+ profile: Profile,
235
+ event_type: Literal["shown", "selected", "rejected"],
236
+ policy_slug: str,
237
+ insurer: str,
238
+ session_id: Optional[str] = None,
239
+ reason: Optional[str] = None,
240
+ ) -> bool:
241
+ """Append a single policy-interaction event to the profile and persist.
242
+
243
+ Dedup: if the SAME `policy_slug` already exists in the matching list for
244
+ this event_type, the existing entry is updated in place (event_at +
245
+ session_id refreshed) rather than appending a duplicate. This keeps the
246
+ list bounded and chronologically meaningful — repeated shows of the same
247
+ policy collapse to the most recent timestamp.
248
+
249
+ Returns True on successful save, False on any failure (missing fields,
250
+ invalid event type, save error).
251
+ """
252
+ if event_type not in _EVENT_TYPE_TO_FIELD:
253
+ return False
254
+ if not policy_slug or not insurer:
255
+ return False
256
+ field_name = _EVENT_TYPE_TO_FIELD[event_type]
257
+ entries: list[dict] = list(getattr(profile, field_name, None) or [])
258
+ now_iso = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
259
+ default_reason = {
260
+ "shown": "shown_in_recommendation",
261
+ "selected": "user_clicked_select",
262
+ "rejected": "user_clicked_reject",
263
+ }[event_type]
264
+ payload = {
265
+ "policy_slug": policy_slug,
266
+ "insurer": insurer,
267
+ "event_at": now_iso,
268
+ "session_id": session_id,
269
+ "reason": reason or default_reason,
270
+ }
271
+ # Dedup on policy_slug within this event-type list. Bump timestamp +
272
+ # session_id; preserve original reason unless caller passed a new one.
273
+ dedup_idx = next(
274
+ (i for i, e in enumerate(entries) if e.get("policy_slug") == policy_slug),
275
+ None,
276
+ )
277
+ if dedup_idx is not None:
278
+ existing = dict(entries[dedup_idx])
279
+ existing["event_at"] = now_iso
280
+ if session_id:
281
+ existing["session_id"] = session_id
282
+ if reason:
283
+ existing["reason"] = reason
284
+ entries[dedup_idx] = existing
285
+ else:
286
+ entries.append(payload)
287
+ setattr(profile, field_name, entries)
288
+ # Persist through the existing save path so persona-id resolution + Chroma
289
+ # sync (if any) stay consistent.
290
+ save_name = profile.name or persona_id_or_name
291
+ if not save_name:
292
+ return False
293
+ return save_profile(save_name, profile, session_id=session_id)
294
+
295
+
296
+ def get_shortlist(profile: Profile) -> list[dict]:
297
+ """Return the user's selected (shortlisted) policies.
298
+
299
+ Thin convenience wrapper used by the admin panel + welcome-back greeting
300
+ so callers don't have to remember the field name.
301
+ """
302
+ return list(getattr(profile, "selected_policies", None) or [])
303
+
304
+
305
  def list_profiles() -> list[dict]:
306
  """Return summary of all stored profiles — used by the admin Profile +
307
  Visitor Log view. One entry per file."""