Spaces:
Sleeping
feat(profile): KI-063 — shown / selected / rejected policy tracking on profile
Browse filesAdds per-user, persistent policy interaction tracking so the bot remembers
across sessions which policies were shown to a user, which they shortlisted,
and which they rejected.
Schema (backend/needs_finder.py)
• Profile gains three list fields: shown_policies, selected_policies,
rejected_policies. Each entry: {policy_slug, insurer, event_at (ISO Z),
session_id, reason}.
Storage helpers (backend/profile_store.py)
• record_policy_event(persona_id_or_name, profile, event_type, policy_slug,
insurer, session_id, reason) — appends or dedup-updates on
(policy_slug, event_type); persists via existing save_profile so
persona-id keying stays consistent.
• get_shortlist(profile) — convenience wrapper around selected_policies.
Orchestrator wiring (backend/orchestrator.py)
• After every recommendation / comparison turn that passes faithfulness
and was issued by a named profile, iterate citations and log
event_type='shown' / reason='shown_in_recommendation'. Wrapped in a
try/except so logging never breaks the chat reply. Anonymous sessions
are skipped.
• _format_known_profile_summary now appends "Your shortlist: <insurer ·
slug>, ..." when selected_policies is non-empty, so KI-061's
welcome-back greeting surfaces the user's shortlist.
User-facing endpoints (backend/admin.py)
• POST /api/profile/select — {session_id, policy_slug, insurer, reason?}
• POST /api/profile/reject — {session_id, policy_slug, insurer, reason?}
Both look up the session, require profile.name (400 otherwise), and
delegate to record_policy_event. Returns {ok, event_type, policy_slug,
count}. NOT admin-gated — these are user actions.
Verification
• py_compile clean across all four files.
• Inline round-trip test: record one shown / selected / rejected event,
reload from disk, all three lists populate with correct shape + reason.
• Dedup test: two shown events for the same policy_slug → list length
stays 1, event_at updates to the later timestamp, session_id refreshes.
• Welcome-back summary correctly appends "Your shortlist: ..." line when
selected_policies is non-empty.
• tests/test_routing_regression.py: 15 passed, 13 subtests passed.
Frontend buttons are deferred — endpoints are exposed for when KI-063b lands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- backend/admin.py +75 -0
- backend/needs_finder.py +9 -0
- backend/orchestrator.py +61 -2
- backend/profile_store.py +96 -1
|
@@ -302,6 +302,81 @@ async def admin_profiles(
|
|
| 302 |
}
|
| 303 |
|
| 304 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 305 |
# ---------------------------------------------------------------------------
|
| 306 |
# /api/admin/performance — aggregated performance/quality metrics
|
| 307 |
# ---------------------------------------------------------------------------
|
|
|
|
| 302 |
}
|
| 303 |
|
| 304 |
|
| 305 |
+
# ---------------------------------------------------------------------------
|
| 306 |
+
# KI-063 (2026-05-15) — user-facing profile-event endpoints.
|
| 307 |
+
#
|
| 308 |
+
# These are NOT admin-gated — they're invoked by the frontend when a logged-
|
| 309 |
+
# in user (one with a stored profile.name) clicks the select/reject buttons
|
| 310 |
+
# on a policy card. Both look up the session, validate that the session has
|
| 311 |
+
# a named profile, then append the event through `profile_store.record_policy_event`.
|
| 312 |
+
#
|
| 313 |
+
# Anonymous sessions (no profile.name) get 400 — there's no key to persist
|
| 314 |
+
# against. The frontend should hide the buttons in that case.
|
| 315 |
+
# ---------------------------------------------------------------------------
|
| 316 |
+
|
| 317 |
+
|
| 318 |
+
class _PolicyEventBody(BaseModel):
|
| 319 |
+
session_id: str
|
| 320 |
+
policy_slug: str
|
| 321 |
+
insurer: str
|
| 322 |
+
reason: Optional[str] = None
|
| 323 |
+
|
| 324 |
+
|
| 325 |
+
def _do_record_policy_event(body: _PolicyEventBody, event_type: str) -> dict:
|
| 326 |
+
"""Shared handler for /api/profile/select + /api/profile/reject."""
|
| 327 |
+
if not body.session_id or not body.policy_slug or not body.insurer:
|
| 328 |
+
raise HTTPException(
|
| 329 |
+
status_code=400,
|
| 330 |
+
detail="session_id, policy_slug, and insurer are required",
|
| 331 |
+
)
|
| 332 |
+
from backend.session_state import get_session
|
| 333 |
+
from backend.profile_store import record_policy_event
|
| 334 |
+
|
| 335 |
+
session = get_session(body.session_id)
|
| 336 |
+
if not session.profile.name:
|
| 337 |
+
raise HTTPException(
|
| 338 |
+
status_code=400,
|
| 339 |
+
detail="No named profile on this session — cannot persist event.",
|
| 340 |
+
)
|
| 341 |
+
ok = record_policy_event(
|
| 342 |
+
persona_id_or_name=session.profile.name,
|
| 343 |
+
profile=session.profile,
|
| 344 |
+
event_type=event_type, # type: ignore[arg-type]
|
| 345 |
+
policy_slug=body.policy_slug,
|
| 346 |
+
insurer=body.insurer,
|
| 347 |
+
session_id=body.session_id,
|
| 348 |
+
reason=body.reason,
|
| 349 |
+
)
|
| 350 |
+
if not ok:
|
| 351 |
+
raise HTTPException(status_code=500, detail="profile save failed")
|
| 352 |
+
# Also persist via the session flush so an in-memory consumer (e.g. the
|
| 353 |
+
# welcome-back greeter) reads the same state without a full disk reload.
|
| 354 |
+
session._flush()
|
| 355 |
+
field_name = {
|
| 356 |
+
"shown": "shown_policies",
|
| 357 |
+
"selected": "selected_policies",
|
| 358 |
+
"rejected": "rejected_policies",
|
| 359 |
+
}[event_type]
|
| 360 |
+
return {
|
| 361 |
+
"ok": True,
|
| 362 |
+
"event_type": event_type,
|
| 363 |
+
"policy_slug": body.policy_slug,
|
| 364 |
+
"count": len(getattr(session.profile, field_name, []) or []),
|
| 365 |
+
}
|
| 366 |
+
|
| 367 |
+
|
| 368 |
+
@router.post("/api/profile/select")
|
| 369 |
+
async def profile_select(body: _PolicyEventBody):
|
| 370 |
+
"""Record a user clicking "shortlist / save" on a policy card."""
|
| 371 |
+
return _do_record_policy_event(body, "selected")
|
| 372 |
+
|
| 373 |
+
|
| 374 |
+
@router.post("/api/profile/reject")
|
| 375 |
+
async def profile_reject(body: _PolicyEventBody):
|
| 376 |
+
"""Record a user clicking "not for me / reject" on a policy card."""
|
| 377 |
+
return _do_record_policy_event(body, "rejected")
|
| 378 |
+
|
| 379 |
+
|
| 380 |
# ---------------------------------------------------------------------------
|
| 381 |
# /api/admin/performance — aggregated performance/quality metrics
|
| 382 |
# ---------------------------------------------------------------------------
|
|
@@ -45,6 +45,15 @@ class Profile:
|
|
| 45 |
health_conditions: Optional[list[str]] = field(default_factory=list) # ["diabetes", "hypertension", ...]
|
| 46 |
asked: list[str] = field(default_factory=list) # question IDs already asked
|
| 47 |
free_form_session: bool = False # True = user asks free questions, not driven by us
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
|
| 49 |
|
| 50 |
# ----------------------------------------------------------------------------
|
|
|
|
| 45 |
health_conditions: Optional[list[str]] = field(default_factory=list) # ["diabetes", "hypertension", ...]
|
| 46 |
asked: list[str] = field(default_factory=list) # question IDs already asked
|
| 47 |
free_form_session: bool = False # True = user asks free questions, not driven by us
|
| 48 |
+
# KI-063 (2026-05-15) — per-user policy interaction log so the bot
|
| 49 |
+
# remembers which policies were shown / selected / rejected across
|
| 50 |
+
# sessions. Each entry is a dict with shape:
|
| 51 |
+
# {policy_slug, insurer, event_at (ISO Z), session_id, reason}
|
| 52 |
+
# Dedup at write-time on (policy_slug, event_type) — re-events just
|
| 53 |
+
# bump event_at + session_id rather than appending duplicates.
|
| 54 |
+
shown_policies: list[dict] = field(default_factory=list) # KI-063
|
| 55 |
+
selected_policies: list[dict] = field(default_factory=list) # KI-063
|
| 56 |
+
rejected_policies: list[dict] = field(default_factory=list) # KI-063
|
| 57 |
|
| 58 |
|
| 59 |
# ----------------------------------------------------------------------------
|
|
@@ -284,7 +284,12 @@ _KNOWN_FIELD_FORMATTERS: tuple[tuple[str, str, "callable"], ...] = (
|
|
| 284 |
def _format_known_profile_summary(profile) -> str:
|
| 285 |
"""Return a comma-separated rundown of what we already know, e.g.
|
| 286 |
'age 34, covering you + spouse, income ₹10-25L, looking for first
|
| 287 |
-
health policy'. Returns '' if nothing meaningful is stored.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 288 |
parts: list[str] = []
|
| 289 |
for field_name, label, fmt in _KNOWN_FIELD_FORMATTERS:
|
| 290 |
val = getattr(profile, field_name, None)
|
|
@@ -296,7 +301,22 @@ def _format_known_profile_summary(profile) -> str:
|
|
| 296 |
rendered = str(val)
|
| 297 |
if rendered:
|
| 298 |
parts.append(f"{label} {rendered}")
|
| 299 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 300 |
|
| 301 |
|
| 302 |
_HELPFUL_GAP_LABELS = {
|
|
@@ -1100,6 +1120,45 @@ async def handle_turn(
|
|
| 1100 |
except Exception:
|
| 1101 |
pass # if any step fails, return English — better than mis-translated
|
| 1102 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1103 |
return TurnResult(
|
| 1104 |
reply_text=reply,
|
| 1105 |
citations=citations,
|
|
|
|
| 284 |
def _format_known_profile_summary(profile) -> str:
|
| 285 |
"""Return a comma-separated rundown of what we already know, e.g.
|
| 286 |
'age 34, covering you + spouse, income ₹10-25L, looking for first
|
| 287 |
+
health policy'. Returns '' if nothing meaningful is stored.
|
| 288 |
+
|
| 289 |
+
KI-063 (2026-05-15) — if the user has selected (shortlisted) policies
|
| 290 |
+
on file, append a "Your shortlist: <insurer · policy>, ..." line so the
|
| 291 |
+
returning-visitor greeting surfaces what they previously saved.
|
| 292 |
+
"""
|
| 293 |
parts: list[str] = []
|
| 294 |
for field_name, label, fmt in _KNOWN_FIELD_FORMATTERS:
|
| 295 |
val = getattr(profile, field_name, None)
|
|
|
|
| 301 |
rendered = str(val)
|
| 302 |
if rendered:
|
| 303 |
parts.append(f"{label} {rendered}")
|
| 304 |
+
summary = ", ".join(parts)
|
| 305 |
+
# KI-063 — append shortlist line if any selected policies on file.
|
| 306 |
+
shortlist = list(getattr(profile, "selected_policies", None) or [])
|
| 307 |
+
if shortlist:
|
| 308 |
+
bits = []
|
| 309 |
+
for entry in shortlist:
|
| 310 |
+
insurer = (entry.get("insurer") or "").strip()
|
| 311 |
+
slug = (entry.get("policy_slug") or "").strip()
|
| 312 |
+
if insurer and slug:
|
| 313 |
+
bits.append(f"{insurer} · {slug}")
|
| 314 |
+
elif slug:
|
| 315 |
+
bits.append(slug)
|
| 316 |
+
if bits:
|
| 317 |
+
line = "Your shortlist: " + ", ".join(bits)
|
| 318 |
+
summary = f"{summary}. {line}" if summary else line
|
| 319 |
+
return summary
|
| 320 |
|
| 321 |
|
| 322 |
_HELPFUL_GAP_LABELS = {
|
|
|
|
| 1120 |
except Exception:
|
| 1121 |
pass # if any step fails, return English — better than mis-translated
|
| 1122 |
|
| 1123 |
+
# KI-063 (2026-05-15) — auto-log "shown" policy events on the persisted
|
| 1124 |
+
# profile so a returning visitor's bot remembers which policies they've
|
| 1125 |
+
# seen. Only fires for context-dependent intents (recommendation /
|
| 1126 |
+
# comparison) AND only when faithfulness passed (we don't log cites that
|
| 1127 |
+
# the safety gates rejected). Anonymous users (no profile.name) get no
|
| 1128 |
+
# log — there's no key to persist against.
|
| 1129 |
+
try:
|
| 1130 |
+
if (
|
| 1131 |
+
intent in ("recommendation", "comparison")
|
| 1132 |
+
and verdict.passed
|
| 1133 |
+
and not blocked
|
| 1134 |
+
and session.profile.name
|
| 1135 |
+
and citations
|
| 1136 |
+
):
|
| 1137 |
+
from backend.profile_store import record_policy_event
|
| 1138 |
+
seen_slugs: set[str] = set()
|
| 1139 |
+
for cite in citations:
|
| 1140 |
+
slug = cite.get("policy_id") or cite.get("policy_slug")
|
| 1141 |
+
insurer = cite.get("insurer_slug") or cite.get("insurer")
|
| 1142 |
+
if not slug or not insurer or slug in seen_slugs:
|
| 1143 |
+
continue
|
| 1144 |
+
seen_slugs.add(slug)
|
| 1145 |
+
record_policy_event(
|
| 1146 |
+
persona_id_or_name=session.profile.name,
|
| 1147 |
+
profile=session.profile,
|
| 1148 |
+
event_type="shown",
|
| 1149 |
+
policy_slug=slug,
|
| 1150 |
+
insurer=insurer,
|
| 1151 |
+
session_id=session_id,
|
| 1152 |
+
reason="shown_in_recommendation",
|
| 1153 |
+
)
|
| 1154 |
+
except Exception as e:
|
| 1155 |
+
# Never let logging failures break the chat reply.
|
| 1156 |
+
import logging
|
| 1157 |
+
logging.warning(
|
| 1158 |
+
"KI-063 shown_policies log failed (session=%s): %s: %s",
|
| 1159 |
+
session_id, type(e).__name__, str(e)[:200],
|
| 1160 |
+
)
|
| 1161 |
+
|
| 1162 |
return TurnResult(
|
| 1163 |
reply_text=reply,
|
| 1164 |
citations=citations,
|
|
@@ -32,7 +32,7 @@ import re
|
|
| 32 |
import time
|
| 33 |
from dataclasses import asdict
|
| 34 |
from pathlib import Path
|
| 35 |
-
from typing import Optional
|
| 36 |
|
| 37 |
from backend.config import settings
|
| 38 |
from backend.needs_finder import Profile
|
|
@@ -207,6 +207,101 @@ def save_profile(name: str, profile: Profile, *, session_id: Optional[str] = Non
|
|
| 207 |
return False
|
| 208 |
|
| 209 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 210 |
def list_profiles() -> list[dict]:
|
| 211 |
"""Return summary of all stored profiles — used by the admin Profile +
|
| 212 |
Visitor Log view. One entry per file."""
|
|
|
|
| 32 |
import time
|
| 33 |
from dataclasses import asdict
|
| 34 |
from pathlib import Path
|
| 35 |
+
from typing import Literal, Optional
|
| 36 |
|
| 37 |
from backend.config import settings
|
| 38 |
from backend.needs_finder import Profile
|
|
|
|
| 207 |
return False
|
| 208 |
|
| 209 |
|
| 210 |
+
# ---------------------------------------------------------------------------
|
| 211 |
+
# KI-063 (2026-05-15) — per-user policy interaction tracking.
|
| 212 |
+
#
|
| 213 |
+
# Three event types are tracked on the Profile:
|
| 214 |
+
# shown — auto-logged by orchestrator when a policy is cited in a
|
| 215 |
+
# recommendation / comparison turn that passed faithfulness.
|
| 216 |
+
# selected — user clicked "save / shortlist" on a policy card (frontend
|
| 217 |
+
# POSTs to /api/profile/select).
|
| 218 |
+
# rejected — user clicked "not for me" (frontend POSTs to /api/profile/reject).
|
| 219 |
+
#
|
| 220 |
+
# Each entry persists across sessions on the JSON profile, so a returning
|
| 221 |
+
# visitor sees their shortlist and the bot can avoid re-pitching rejected
|
| 222 |
+
# policies.
|
| 223 |
+
# ---------------------------------------------------------------------------
|
| 224 |
+
|
| 225 |
+
_EVENT_TYPE_TO_FIELD = {
|
| 226 |
+
"shown": "shown_policies",
|
| 227 |
+
"selected": "selected_policies",
|
| 228 |
+
"rejected": "rejected_policies",
|
| 229 |
+
}
|
| 230 |
+
|
| 231 |
+
|
| 232 |
+
def record_policy_event(
|
| 233 |
+
persona_id_or_name: str,
|
| 234 |
+
profile: Profile,
|
| 235 |
+
event_type: Literal["shown", "selected", "rejected"],
|
| 236 |
+
policy_slug: str,
|
| 237 |
+
insurer: str,
|
| 238 |
+
session_id: Optional[str] = None,
|
| 239 |
+
reason: Optional[str] = None,
|
| 240 |
+
) -> bool:
|
| 241 |
+
"""Append a single policy-interaction event to the profile and persist.
|
| 242 |
+
|
| 243 |
+
Dedup: if the SAME `policy_slug` already exists in the matching list for
|
| 244 |
+
this event_type, the existing entry is updated in place (event_at +
|
| 245 |
+
session_id refreshed) rather than appending a duplicate. This keeps the
|
| 246 |
+
list bounded and chronologically meaningful — repeated shows of the same
|
| 247 |
+
policy collapse to the most recent timestamp.
|
| 248 |
+
|
| 249 |
+
Returns True on successful save, False on any failure (missing fields,
|
| 250 |
+
invalid event type, save error).
|
| 251 |
+
"""
|
| 252 |
+
if event_type not in _EVENT_TYPE_TO_FIELD:
|
| 253 |
+
return False
|
| 254 |
+
if not policy_slug or not insurer:
|
| 255 |
+
return False
|
| 256 |
+
field_name = _EVENT_TYPE_TO_FIELD[event_type]
|
| 257 |
+
entries: list[dict] = list(getattr(profile, field_name, None) or [])
|
| 258 |
+
now_iso = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
| 259 |
+
default_reason = {
|
| 260 |
+
"shown": "shown_in_recommendation",
|
| 261 |
+
"selected": "user_clicked_select",
|
| 262 |
+
"rejected": "user_clicked_reject",
|
| 263 |
+
}[event_type]
|
| 264 |
+
payload = {
|
| 265 |
+
"policy_slug": policy_slug,
|
| 266 |
+
"insurer": insurer,
|
| 267 |
+
"event_at": now_iso,
|
| 268 |
+
"session_id": session_id,
|
| 269 |
+
"reason": reason or default_reason,
|
| 270 |
+
}
|
| 271 |
+
# Dedup on policy_slug within this event-type list. Bump timestamp +
|
| 272 |
+
# session_id; preserve original reason unless caller passed a new one.
|
| 273 |
+
dedup_idx = next(
|
| 274 |
+
(i for i, e in enumerate(entries) if e.get("policy_slug") == policy_slug),
|
| 275 |
+
None,
|
| 276 |
+
)
|
| 277 |
+
if dedup_idx is not None:
|
| 278 |
+
existing = dict(entries[dedup_idx])
|
| 279 |
+
existing["event_at"] = now_iso
|
| 280 |
+
if session_id:
|
| 281 |
+
existing["session_id"] = session_id
|
| 282 |
+
if reason:
|
| 283 |
+
existing["reason"] = reason
|
| 284 |
+
entries[dedup_idx] = existing
|
| 285 |
+
else:
|
| 286 |
+
entries.append(payload)
|
| 287 |
+
setattr(profile, field_name, entries)
|
| 288 |
+
# Persist through the existing save path so persona-id resolution + Chroma
|
| 289 |
+
# sync (if any) stay consistent.
|
| 290 |
+
save_name = profile.name or persona_id_or_name
|
| 291 |
+
if not save_name:
|
| 292 |
+
return False
|
| 293 |
+
return save_profile(save_name, profile, session_id=session_id)
|
| 294 |
+
|
| 295 |
+
|
| 296 |
+
def get_shortlist(profile: Profile) -> list[dict]:
|
| 297 |
+
"""Return the user's selected (shortlisted) policies.
|
| 298 |
+
|
| 299 |
+
Thin convenience wrapper used by the admin panel + welcome-back greeting
|
| 300 |
+
so callers don't have to remember the field name.
|
| 301 |
+
"""
|
| 302 |
+
return list(getattr(profile, "selected_policies", None) or [])
|
| 303 |
+
|
| 304 |
+
|
| 305 |
def list_profiles() -> list[dict]:
|
| 306 |
"""Return summary of all stored profiles — used by the admin Profile +
|
| 307 |
Visitor Log view. One entry per file."""
|