saivivek6 commited on
Commit
47ad5bc
·
1 Parent(s): 16cd24d

Deploy: bandit/RAG flags(off)+knowledge mode, RAG/ChromaDB wiring, Vanguard decline tone, analyst commentary (Analysis heading + numbered blue points), no-dup-table, bigger HTML export, larger charts + hbar axis-label fix, JSON-leak fixes

Browse files
vivek/backend/combined_prompt.py CHANGED
@@ -386,7 +386,7 @@ Data grounding (STRICT — the widget must mirror your <RESPONSE>):
386
 
387
  Honoring an explicitly requested chart type:
388
  - If the user asks for a SPECIFIC chart type (e.g. "as a candlestick", "show a sankey", "pie chart") and it IS one of the supported kinds above, you MUST use exactly that kind — do not substitute another.
389
- - If the requested chart type is NOT in the supported kinds (e.g. 3D surface, map/choropleth, gantt, renko, marimekko, etc.), DO NOT silently render a different chart. Instead: in <RESPONSE>, say plainly that you cannot render that specific chart type yet, then name 1-3 supported kinds that would fit their data well and ask if they'd like one of those. In that case return <WIDGET></WIDGET> (empty) wait for the user to confirm before rendering an alternative.
390
  - Never pretend an unsupported type is supported, and never relabel a different chart as the requested type.
391
 
392
  Interactivity:
@@ -396,9 +396,17 @@ Interactivity:
396
  - If something cannot be expressed with the supported block types (e.g. a playable game, live sliders), DO NOT invent HTML — return <WIDGET></WIDGET> (empty) and explain in <RESPONSE> instead. Never dump raw index arrays like `[0,1,2]`.
397
  - For photographs, diagrams, or icons, use the `image` block with a valid https:// URL or a data: URI. Combine `image` with `text`, `chart`, and `table` blocks as needed.
398
 
399
- Dynamic layout (JSON) — avoid static, single-block dashboards:
400
- - Shape `layout` like a short story: context first, then metrics, then detail, then actions. Mix block types (text, kpi_row, chart, table, image, action_row) whenever it improves scanning; do not default to one lonely chart if KPIs or a sentence of framing would help.
401
- - Use `action_row` for obvious follow-up intents; keep blocks ordered top-to-bottom by importance so the widget feels purposeful, not generic.
 
 
 
 
 
 
 
 
402
  """
403
 
404
 
@@ -411,23 +419,41 @@ _REGISTRY_JSON_PATH = Path(__file__).resolve().parent.parent / "frontend-vue" /
411
 
412
  _JSON_RULE_PREAMBLE = """WIDGET JSON SCHEMA MODE (WIDGET_MODE=json):
413
  - WIDGET WARRANT (decide for yourself — there is no keyword trigger):
414
- Include a NON-EMPTY widget ONLY IF both are true: (1) a visual materially helps (a comparison of
415
- several numbers, a trend, a breakdown/share, a flow, or a matrix), AND (2) you can populate it with
416
- real values from your <RESPONSE> using the supported block types below. If you cannot build it from
417
- the allowed blocks (or there is no concrete data), return <WIDGET></WIDGET> (empty) and say so in
418
- <RESPONSE>. Do NOT chart a single number, a definition, a yes/no, or an opinion.
419
  - The content inside <WIDGET> MUST be valid JSON (no markdown fences, no comments).
420
  - Root object: { "version": "1.0", "layout": [ ... ] }
421
  - layout is an ordered array of blocks (top-to-bottom).
422
  - Supported block types ONLY (do not invent new ones):"""
423
 
424
- _JSON_RULE_TRAILER = """
425
- NO FABRICATED DATA (most important — applies to <RESPONSE> and <WIDGET>):
426
- - If you do NOT actually have the real figures to answer (e.g. live or historical stock prices, a fund's daily returns, exact financials you are not sure of), DO NOT invent, estimate, or use "illustrative"/"mock"/"example"/"est." data, and DO NOT draw a widget.
427
- - In that case: say plainly in <RESPONSE> that you do not have that specific data, suggest what the user could provide or ask instead, and return <WIDGET></WIDGET> (empty).
428
- - NEVER label a chart or value "mock", "illustrative", "estimated", or "example" if it is not real, do not render it. A widget must only ever show real, known values.
429
- - Educational math demos (e.g. compound-interest with user-given P/r/t) are fine because the user supplied the inputs; market data you don't have is NOT.
 
 
 
 
 
 
 
430
 
 
 
 
 
 
 
 
 
 
 
 
 
431
  Data grounding (STRICT — the widget must mirror your <RESPONSE>):
432
  - Use ONLY the exact numbers and labels stated in your <RESPONSE>. Do NOT invent values, round differently, or add labels/values not written in <RESPONSE>.
433
  - For category charts, "x_categories" MUST be the exact entity/segment names you named in <RESPONSE> (e.g. "Data Center", "Gaming") — NEVER 0, 1, 2. Each series value MUST equal the number in <RESPONSE>, aligned to those categories.
@@ -436,7 +462,7 @@ Data grounding (STRICT — the widget must mirror your <RESPONSE>):
436
 
437
  Honoring an explicitly requested chart type:
438
  - If the user asks for a SPECIFIC chart type (e.g. "as a candlestick", "show a sankey", "pie chart") and it IS one of the supported kinds above, you MUST use exactly that kind — do not substitute another.
439
- - If the requested chart type is NOT in the supported kinds (e.g. 3D surface, map/choropleth, gantt, renko, marimekko, etc.), DO NOT silently render a different chart. Instead: in <RESPONSE>, say plainly that you cannot render that specific chart type yet, then name 1-3 supported kinds that would fit their data well and ask if they'd like one of those. In that case return <WIDGET></WIDGET> (empty) wait for the user to confirm before rendering an alternative.
440
  - Never pretend an unsupported type is supported, and never relabel a different chart as the requested type.
441
 
442
  Interactivity:
@@ -446,9 +472,17 @@ Interactivity:
446
  - If something cannot be expressed with the supported block types (e.g. a playable game, live sliders), DO NOT invent HTML — return <WIDGET></WIDGET> (empty) and explain in <RESPONSE> instead. Never dump raw index arrays like `[0,1,2]`.
447
  - For photographs, diagrams, or icons, use the `image` block with a valid https:// URL or a data: URI. Combine `image` with `text`, `chart`, and `table` blocks as needed.
448
 
449
- Dynamic layout (JSON) — avoid static, single-block dashboards:
450
- - Shape `layout` like a short story: context first, then metrics, then detail, then actions. Mix block types whenever it improves scanning; do not default to one lonely chart if KPIs or a sentence of framing would help.
451
- - Use `action_row` for obvious follow-up intents; keep blocks ordered top-to-bottom by importance so the widget feels purposeful, not generic."""
 
 
 
 
 
 
 
 
452
 
453
 
454
  def build_json_widget_rule() -> str:
@@ -473,7 +507,11 @@ def build_json_widget_rule() -> str:
473
  entries.append(f"{header}\n{body}")
474
  if not entries:
475
  return _JSON_WIDGET_RULE.strip()
476
- return _JSON_RULE_PREAMBLE + "\n" + "\n".join(entries) + "\n" + _JSON_RULE_TRAILER
 
 
 
 
477
  except Exception:
478
  return _JSON_WIDGET_RULE.strip()
479
 
@@ -577,6 +615,7 @@ def build_combined_system_prompt(
577
  format_rule: str,
578
  primitive_extra_context: str,
579
  user_message: str,
 
580
  forbidden_components: list[str] | None = None,
581
  required_components: list[str] | None = None,
582
  ) -> str:
@@ -652,8 +691,30 @@ TOKEN LIMIT — you have ~{combined_max_tokens} tokens total for <RESPONSE> + <W
652
  - For comparison tables in <RESPONSE>: keep focused so the widget has room. Both must fit.
653
  """
654
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
655
  return f"""You are an expert AI assistant. Each turn you produce a written answer AND an OPTIONAL interactive widget built ONLY from a fixed set of UI components defined below — there is NO HTML and NO external charting libraries.
656
  {social_turn_banner}
 
657
  Output style: No emojis. Neat, clean, professional — in both <RESPONSE> text and <WIDGET>.
658
  {token_limit_block}
659
  {_OUTPUT_CONTRACT_STRICT}
 
386
 
387
  Honoring an explicitly requested chart type:
388
  - If the user asks for a SPECIFIC chart type (e.g. "as a candlestick", "show a sankey", "pie chart") and it IS one of the supported kinds above, you MUST use exactly that kind — do not substitute another.
389
+ - If the requested chart type is NOT in the supported kinds (e.g. 3D surface, map/choropleth, gantt, renko, marimekko, etc.), DO NOT silently render a different chart. Respond in a warm, professional, enterprise tone on behalf of the product "Vanguard" — briefly and politely explain that Vanguard's visualization suite does not currently support that chart type, then helpfully suggest 1-3 supported chart kinds that would present their data well and invite them to choose one. Return <WIDGET></WIDGET> (empty) and wait for them to pick. Do NOT use blunt or technical wording like "not supported in this system", "broken", "incorrect output", or "I cannot render" — keep it courteous and reassuring (e.g. "Vanguard doesn't support 3D surface charts just yet — but I'd be glad to show this as a 3D scatter, treemap, or heatmap instead. Which would you prefer?").
390
  - Never pretend an unsupported type is supported, and never relabel a different chart as the requested type.
391
 
392
  Interactivity:
 
396
  - If something cannot be expressed with the supported block types (e.g. a playable game, live sliders), DO NOT invent HTML — return <WIDGET></WIDGET> (empty) and explain in <RESPONSE> instead. Never dump raw index arrays like `[0,1,2]`.
397
  - For photographs, diagrams, or icons, use the `image` block with a valid https:// URL or a data: URI. Combine `image` with `text`, `chart`, and `table` blocks as needed.
398
 
399
+ Dynamic layout (JSON) — informative, not padded:
400
+ - ALWAYS provide proper DATA-ANALYST commentary with a chart. Put it as a `text` block in the widget BELOW the chart. The block's content MUST start with the heading line "### Analysis" then 3-4 NUMBERED points (each line: "1. ", "2. ", ...). Wrap the KEY figures/labels in **bold** (the renderer highlights bold in blue). Example content:
401
+ "### Analysis
402
+ 1. **Data Center** at **$47.5B** is ~**78%** of total FY2024 revenue — the clear dominant segment.
403
+ 2. It grew ~**3.2x YoY** (from **$15.0B**), driven by AI/HPC demand.
404
+ 3. **Gaming** is a distant second at **$10.4B**; every other segment is under **$2B**.
405
+ 4. Takeaway: revenue is heavily **concentrated in AI infrastructure**."
406
+ Write like a financial/data analyst — specific and quantitative. Across the points cover: leader & laggard with values; the key gap/ratio/share/growth rate; the trend/notable change; any outlier or concentration; and a one-line "so what" takeaway. Use real numbers. NEVER write filler like "this chart shows the data" or "as you can see" — every point must carry a concrete insight. Always 3-4 points (not one). Never show a bare, unexplained chart.
407
+ - Add a `table` when it genuinely HELPS the reader — e.g. exact reference figures, extra columns/notes, or a breakdown the chart can't show. Do NOT add a table that merely restates a simple chart's few values with nothing extra (that is clutter); use one when it adds real detail.
408
+ - You MAY add a few KPI tiles for headline numbers when useful. Mix block types only when each adds value, and keep blocks ordered top-to-bottom by importance.
409
+ - Use `action_row` for obvious follow-up intents so the widget feels purposeful, not generic.
410
  """
411
 
412
 
 
419
 
420
  _JSON_RULE_PREAMBLE = """WIDGET JSON SCHEMA MODE (WIDGET_MODE=json):
421
  - WIDGET WARRANT (decide for yourself — there is no keyword trigger):
422
+ Include a NON-EMPTY widget when a visual materially helps (a comparison of several numbers, a trend,
423
+ a breakdown/share, a flow, or a matrix) and you can populate it with concrete values using the
424
+ supported block types below (see the data rule). Do NOT chart a single number, a definition, a
425
+ yes/no, or an opinion.
 
426
  - The content inside <WIDGET> MUST be valid JSON (no markdown fences, no comments).
427
  - Root object: { "version": "1.0", "layout": [ ... ] }
428
  - layout is an ordered array of blocks (top-to-bottom).
429
  - Supported block types ONLY (do not invent new ones):"""
430
 
431
+ # Data-source rule — strict (RAG on) vs. knowledge (RAG off). Selected in build_json_widget_rule().
432
+ _DATA_RULE_STRICT = """
433
+ NO FABRICATED DATA ground every figure in the GROUNDING CONTEXT (applies to <RESPONSE> and <WIDGET>):
434
+ - Use ONLY figures that appear in the GROUNDING CONTEXT provided below (retrieved financial data),
435
+ or numbers the user supplied in their message. Do NOT invent, estimate, recall from memory, or use
436
+ "illustrative"/"mock"/"example"/"est." data.
437
+ - If the GROUNDING CONTEXT does not contain the figures needed to answer, say plainly in <RESPONSE>
438
+ that you do not have that data, suggest what to ask instead, and return <WIDGET></WIDGET> (empty).
439
+ - NEVER label a chart or value "mock"/"illustrative"/"estimated" — if it is not in the context, do
440
+ not render it. A widget must only ever show real values from the GROUNDING CONTEXT.
441
+ - Educational math demos (e.g. compound interest with user-given P/r/t) are fine — the user supplied
442
+ the inputs.
443
+ """
444
 
445
+ _DATA_RULE_KNOWLEDGE = """
446
+ DATA FROM YOUR KNOWLEDGE (applies to <RESPONSE> and <WIDGET>):
447
+ - Use your own world knowledge to provide concrete, realistic figures and DRAW the widget. You do NOT
448
+ need a live source — give your best-known real-world values (e.g. company revenue, segments, market
449
+ caps) and render the chart rather than refusing.
450
+ - If a number is approximate or from memory, you MAY briefly note that in <RESPONSE> (e.g. "approx",
451
+ "FY2023"), but still render the widget.
452
+ - Do NOT fill charts with meaningless placeholder values (0,1,2,3 or random filler) — use genuine
453
+ best-estimate numbers that reflect reality.
454
+ """
455
+
456
+ _JSON_RULE_TRAILER = """
457
  Data grounding (STRICT — the widget must mirror your <RESPONSE>):
458
  - Use ONLY the exact numbers and labels stated in your <RESPONSE>. Do NOT invent values, round differently, or add labels/values not written in <RESPONSE>.
459
  - For category charts, "x_categories" MUST be the exact entity/segment names you named in <RESPONSE> (e.g. "Data Center", "Gaming") — NEVER 0, 1, 2. Each series value MUST equal the number in <RESPONSE>, aligned to those categories.
 
462
 
463
  Honoring an explicitly requested chart type:
464
  - If the user asks for a SPECIFIC chart type (e.g. "as a candlestick", "show a sankey", "pie chart") and it IS one of the supported kinds above, you MUST use exactly that kind — do not substitute another.
465
+ - If the requested chart type is NOT in the supported kinds (e.g. 3D surface, map/choropleth, gantt, renko, marimekko, etc.), DO NOT silently render a different chart. Respond in a warm, professional, enterprise tone on behalf of the product "Vanguard" — briefly and politely explain that Vanguard's visualization suite does not currently support that chart type, then helpfully suggest 1-3 supported chart kinds that would present their data well and invite them to choose one. Return <WIDGET></WIDGET> (empty) and wait for them to pick. Do NOT use blunt or technical wording like "not supported in this system", "broken", "incorrect output", or "I cannot render" — keep it courteous and reassuring (e.g. "Vanguard doesn't support 3D surface charts just yet — but I'd be glad to show this as a 3D scatter, treemap, or heatmap instead. Which would you prefer?").
466
  - Never pretend an unsupported type is supported, and never relabel a different chart as the requested type.
467
 
468
  Interactivity:
 
472
  - If something cannot be expressed with the supported block types (e.g. a playable game, live sliders), DO NOT invent HTML — return <WIDGET></WIDGET> (empty) and explain in <RESPONSE> instead. Never dump raw index arrays like `[0,1,2]`.
473
  - For photographs, diagrams, or icons, use the `image` block with a valid https:// URL or a data: URI. Combine `image` with `text`, `chart`, and `table` blocks as needed.
474
 
475
+ Dynamic layout (JSON) — informative, not padded:
476
+ - ALWAYS provide proper DATA-ANALYST commentary with a chart. Put it as a `text` block in the widget BELOW the chart. The block's content MUST start with the heading line "### Analysis" then 3-4 NUMBERED points (each line: "1. ", "2. ", ...). Wrap the KEY figures/labels in **bold** (the renderer highlights bold in blue). Example content:
477
+ "### Analysis
478
+ 1. **Data Center** at **$47.5B** is ~**78%** of total FY2024 revenue — the clear dominant segment.
479
+ 2. It grew ~**3.2x YoY** (from **$15.0B**), driven by AI/HPC demand.
480
+ 3. **Gaming** is a distant second at **$10.4B**; every other segment is under **$2B**.
481
+ 4. Takeaway: revenue is heavily **concentrated in AI infrastructure**."
482
+ Write like a financial/data analyst — specific and quantitative. Across the points cover: leader & laggard with values; the key gap/ratio/share/growth rate; the trend/notable change; any outlier or concentration; and a one-line "so what" takeaway. Use real numbers. NEVER write filler like "this chart shows the data" or "as you can see" — every point must carry a concrete insight. Always 3-4 points (not one). Never show a bare, unexplained chart.
483
+ - Add a `table` when it genuinely HELPS the reader — e.g. exact reference figures, extra columns/notes, or a breakdown the chart can't show. Do NOT add a table that merely restates a simple chart's few values with nothing extra (that is clutter); use one when it adds real detail.
484
+ - You MAY add a few KPI tiles for headline numbers when useful. Mix block types only when each adds value, and keep blocks ordered top-to-bottom by importance.
485
+ - Use `action_row` for obvious follow-up intents so the widget feels purposeful, not generic."""
486
 
487
 
488
  def build_json_widget_rule() -> str:
 
507
  entries.append(f"{header}\n{body}")
508
  if not entries:
509
  return _JSON_WIDGET_RULE.strip()
510
+ # Strict grounding when RAG is on; otherwise let the model use its own knowledge.
511
+ data_rule = _DATA_RULE_STRICT if getattr(config, "USE_RAG", False) else _DATA_RULE_KNOWLEDGE
512
+ return (
513
+ _JSON_RULE_PREAMBLE + "\n" + "\n".join(entries) + "\n" + data_rule + _JSON_RULE_TRAILER
514
+ )
515
  except Exception:
516
  return _JSON_WIDGET_RULE.strip()
517
 
 
615
  format_rule: str,
616
  primitive_extra_context: str,
617
  user_message: str,
618
+ grounding_context: str = "",
619
  forbidden_components: list[str] | None = None,
620
  required_components: list[str] | None = None,
621
  ) -> str:
 
691
  - For comparison tables in <RESPONSE>: keep focused so the widget has room. Both must fit.
692
  """
693
 
694
+ if grounding_context.strip():
695
+ grounding_block = (
696
+ "\n═══════════════════════════════════════════════════════\n"
697
+ "GROUNDING CONTEXT — retrieved financial data (the ONLY source for your figures)\n"
698
+ "═══════════════════════════════════════════════════════\n"
699
+ "Use ONLY numbers/labels that appear below. Do not add figures from memory.\n\n"
700
+ f"{grounding_context.strip()}\n"
701
+ )
702
+ elif getattr(config, "USE_RAG", False):
703
+ # RAG on but nothing relevant retrieved → decline rather than fabricate.
704
+ grounding_block = (
705
+ "\n════════════════════��══════════════════════════════════\n"
706
+ "GROUNDING CONTEXT — none retrieved\n"
707
+ "═══════════════════════════════════════════════════════\n"
708
+ "No financial data was retrieved for this query. You do NOT have figures for it: "
709
+ "say so plainly in <RESPONSE> and return <WIDGET></WIDGET> (empty). Do NOT use memory or estimates.\n"
710
+ )
711
+ else:
712
+ # RAG off → knowledge mode (the data rule lets the model use its own figures).
713
+ grounding_block = ""
714
+
715
  return f"""You are an expert AI assistant. Each turn you produce a written answer AND an OPTIONAL interactive widget built ONLY from a fixed set of UI components defined below — there is NO HTML and NO external charting libraries.
716
  {social_turn_banner}
717
+ {grounding_block}
718
  Output style: No emojis. Neat, clean, professional — in both <RESPONSE> text and <WIDGET>.
719
  {token_limit_block}
720
  {_OUTPUT_CONTRACT_STRICT}
vivek/backend/config.py CHANGED
@@ -40,6 +40,17 @@ ADAPTIVE_LLM_MODE = os.getenv("ADAPTIVE_LLM_MODE", LLM_MODE).lower()
40
  # Widget output: "html" (full HTML in iframe) or "json" (UI schema for frontend renderer).
41
  WIDGET_MODE = os.getenv("WIDGET_MODE", "json").strip().lower()
42
 
 
 
 
 
 
 
 
 
 
 
 
43
  # Max time (sec) and tokens for combined response+widget generation.
44
  # Tight defaults so latency stays predictable; raise only when the UI shows truncation.
45
  COMBINED_TIMEOUT_SECONDS = int(os.getenv("COMBINED_TIMEOUT_SECONDS", "45"))
 
40
  # Widget output: "html" (full HTML in iframe) or "json" (UI schema for frontend renderer).
41
  WIDGET_MODE = os.getenv("WIDGET_MODE", "json").strip().lower()
42
 
43
+
44
+ def _flag(name: str, default: str = "0") -> bool:
45
+ return os.getenv(name, default).strip().lower() in ("1", "true", "yes", "on")
46
+
47
+ # Feature flags (default OFF):
48
+ # - USE_BANDIT: when off, no Thompson-Sampling strategy selection or reward learning;
49
+ # the response uses a neutral style.
50
+ # - USE_RAG: when off, no ChromaDB retrieval; the model fills figures from its own knowledge.
51
+ USE_BANDIT = _flag("USE_BANDIT", "0")
52
+ USE_RAG = _flag("USE_RAG", "0")
53
+
54
  # Max time (sec) and tokens for combined response+widget generation.
55
  # Tight defaults so latency stays predictable; raise only when the UI shows truncation.
56
  COMBINED_TIMEOUT_SECONDS = int(os.getenv("COMBINED_TIMEOUT_SECONDS", "45"))
vivek/backend/server.py CHANGED
@@ -293,6 +293,19 @@ app.add_middleware(GZipMiddleware, minimum_size=1000)
293
  def _on_startup():
294
  persistence.init_db()
295
 
 
 
 
 
 
 
 
 
 
 
 
 
 
296
  if getattr(config, "ADMIN_CONFIGURED", False) and config.ADMIN_USERNAME and config.ADMIN_PASSWORD and config.ADMIN_EMAIL:
297
  try:
298
  register_user(username=config.ADMIN_USERNAME, email=config.ADMIN_EMAIL, password=config.ADMIN_PASSWORD)
@@ -822,22 +835,29 @@ def _build_adaptive_prompt(uid: str, msg: str):
822
  ev = fast_valence(msg, user["last_response"])
823
  auto_detected = False
824
  auto_r = None
825
-
826
- if user["last_response"] and user["last_x"] is not None and user["last_strategy"]:
827
- reward = float(np.clip(0.5 + 0.45 * ev["pos"] - 0.45 * ev["neg"], 0.05, 0.95))
828
- engine.update(uid, user["last_strategy"], np.array(user["last_x"]), reward)
829
- auto_detected = True
830
- auto_r = reward
831
-
832
  explicit = False
833
- force_explore = bool(detect_explore_trigger(msg) or (ev.get("neg", 0.0) >= config.NEG_EXPLORE_THRESHOLD))
834
- neg_s = negative_strength(ev)
835
-
836
- strat, scores, x, prev = engine.select(
837
- uid, msg, force_explore=force_explore, neg_strength=neg_s, explicit_strategy=None
838
- )
839
-
840
- format_rule = config.STRATEGIES.get(strat, "Be helpful and clear.")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
841
 
842
  prim_block = ""
843
  if _is_admin_user(uid):
@@ -852,11 +872,24 @@ def _build_adaptive_prompt(uid: str, msg: str):
852
  if prim_lines:
853
  prim_block = "\n\n## User primitives (follow these as constraints)\n" + "\n".join(prim_lines) + "\n"
854
 
 
 
 
 
 
 
 
 
 
 
 
 
855
  combined_system = build_combined_system_prompt(
856
  strategy_id=strat,
857
  format_rule=format_rule,
858
  primitive_extra_context=(getattr(config, "SKILLS_CONTENT", "") or "") + prim_block,
859
  user_message=msg,
 
860
  forbidden_components=None,
861
  required_components=None,
862
  )
 
293
  def _on_startup():
294
  persistence.init_db()
295
 
296
+ # RAG: only when enabled — ingest the financial corpus into ChromaDB on first boot
297
+ # (the binary store is not shipped). When USE_RAG is off, skip entirely (no model download).
298
+ if getattr(config, "USE_RAG", False):
299
+ try:
300
+ from rag_finance.retriever import _load as _rag_load
301
+ from rag_finance.ingest import ingest as _rag_ingest
302
+ col, _ = _rag_load()
303
+ if col is None or col.count() == 0:
304
+ stats = _rag_ingest()
305
+ print(f"[rag] ingested {stats.get('documents')} financial docs into ChromaDB")
306
+ except Exception as e:
307
+ print(f"[rag] ingest skipped: {e}")
308
+
309
  if getattr(config, "ADMIN_CONFIGURED", False) and config.ADMIN_USERNAME and config.ADMIN_PASSWORD and config.ADMIN_EMAIL:
310
  try:
311
  register_user(username=config.ADMIN_USERNAME, email=config.ADMIN_EMAIL, password=config.ADMIN_PASSWORD)
 
835
  ev = fast_valence(msg, user["last_response"])
836
  auto_detected = False
837
  auto_r = None
 
 
 
 
 
 
 
838
  explicit = False
839
+ force_explore = False
840
+
841
+ if config.USE_BANDIT:
842
+ # Implicit reward for the previous turn from this message's sentiment.
843
+ if user["last_response"] and user["last_x"] is not None and user["last_strategy"]:
844
+ reward = float(np.clip(0.5 + 0.45 * ev["pos"] - 0.45 * ev["neg"], 0.05, 0.95))
845
+ engine.update(uid, user["last_strategy"], np.array(user["last_x"]), reward)
846
+ auto_detected = True
847
+ auto_r = reward
848
+ force_explore = bool(detect_explore_trigger(msg) or (ev.get("neg", 0.0) >= config.NEG_EXPLORE_THRESHOLD))
849
+ neg_s = negative_strength(ev)
850
+ strat, scores, x, prev = engine.select(
851
+ uid, msg, force_explore=force_explore, neg_strength=neg_s, explicit_strategy=None
852
+ )
853
+ format_rule = config.STRATEGIES.get(strat, "Be helpful and clear.")
854
+ else:
855
+ # Bandit OFF: neutral style, no strategy selection, no learning.
856
+ strat = "neutral"
857
+ scores = {}
858
+ prev = None
859
+ x = engine.featurize(msg, user) # still compute the feature vector for the Insights panel
860
+ format_rule = "Answer naturally, clearly, and concisely. There is no required text format."
861
 
862
  prim_block = ""
863
  if _is_admin_user(uid):
 
872
  if prim_lines:
873
  prim_block = "\n\n## User primitives (follow these as constraints)\n" + "\n".join(prim_lines) + "\n"
874
 
875
+ # RAG: when ON, retrieve clean financial figures (entity-gated) → GROUNDING CONTEXT.
876
+ # When OFF, no context → the model fills figures from its own knowledge.
877
+ grounding_context = ""
878
+ if config.USE_RAG:
879
+ try:
880
+ from rag_finance.retriever import retrieve as _rag_retrieve
881
+ rag = _rag_retrieve(msg, k=6)
882
+ if rag.get("relevant") and rag.get("chunks"):
883
+ grounding_context = "\n\n".join(rag["chunks"])
884
+ except Exception as e:
885
+ print(f"[rag] retrieve failed: {e}")
886
+
887
  combined_system = build_combined_system_prompt(
888
  strategy_id=strat,
889
  format_rule=format_rule,
890
  primitive_extra_context=(getattr(config, "SKILLS_CONTENT", "") or "") + prim_block,
891
  user_message=msg,
892
+ grounding_context=grounding_context,
893
  forbidden_components=None,
894
  required_components=None,
895
  )
vivek/frontend-vue/dist/assets/{ActionRow-ByJHOpPb.js → ActionRow-yGTZNE6v.js} RENAMED
@@ -1 +1 @@
1
- import{G as e,M as t,P as n,f as r,h as i,i as a,jt as o,u as s,v as c}from"./runtime-core.esm-bundler-olIhRSij.js";import{tr as l}from"./index-0cd4ouHe.js";var u={class:`flex flex-wrap gap-2`},d=c({__name:`ActionRow`,props:{block:{}},emits:[`action`],setup(c,{emit:d}){let f=d;function p(e){let t=String(e.label||e.intent||``).trim();t&&f(`action`,t)}return(d,f)=>(t(),r(`div`,u,[(t(!0),r(a,null,n(c.block.buttons||[],(n,r)=>(t(),s(l,{key:r,type:`button`,variant:`outline`,size:`sm`,title:n.intent?`Ask: ${n.label||n.intent}`:void 0,onClick:e=>p(n)},{default:e(()=>[i(o(n.label||`Action`),1)]),_:2},1032,[`title`,`onClick`]))),128))]))}});export{d as default};
 
1
+ import{G as e,M as t,P as n,f as r,h as i,i as a,jt as o,u as s,v as c}from"./runtime-core.esm-bundler-olIhRSij.js";import{nr as l}from"./index-Dv2_3L6X.js";var u={class:`flex flex-wrap gap-2`},d=c({__name:`ActionRow`,props:{block:{}},emits:[`action`],setup(c,{emit:d}){let f=d;function p(e){let t=String(e.label||e.intent||``).trim();t&&f(`action`,t)}return(d,f)=>(t(),r(`div`,u,[(t(!0),r(a,null,n(c.block.buttons||[],(n,r)=>(t(),s(l,{key:r,type:`button`,variant:`outline`,size:`sm`,title:n.intent?`Ask: ${n.label||n.intent}`:void 0,onClick:e=>p(n)},{default:e(()=>[i(o(n.label||`Action`),1)]),_:2},1032,[`title`,`onClick`]))),128))]))}});export{d as default};
vivek/frontend-vue/dist/assets/{ChartBlock-D02q0rey.js → ChartBlock-fdWObMZH.js} RENAMED
The diff for this file is too large to render. See raw diff
 
vivek/frontend-vue/dist/assets/TextBlock-BpmAPyAu.js ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ import{M as e,c as t,f as n,v as r}from"./runtime-core.esm-bundler-olIhRSij.js";import{r as i}from"./index-Dv2_3L6X.js";var a=[`innerHTML`],o=i(r({__name:`TextBlock`,props:{block:{}},setup(r){let i=r;function o(e){return e.replace(/&/g,`&amp;`).replace(/</g,`&lt;`).replace(/>/g,`&gt;`)}function s(e){let t=o(e);return t=t.replace(/\*\*(.+?)\*\*/g,`<strong class="ws-accent">$1</strong>`),t=t.replace(/__(.+?)__/g,`<strong class="ws-accent">$1</strong>`),t=t.replace(/`([^`]+?)`/g,`<code class="ws-code">$1</code>`),t}let c=t(()=>{let e=String(i.block.content||``).replace(/\r\n/g,`
2
+ `).trim();if(!e)return``;let t=e.split(`
3
+ `),n=[],r=null,a=()=>{r&&=(n.push(`</${r}>`),null)};for(let e of t){let t=e.match(/^\s*#{1,3}\s+(.*)$/),i=e.match(/^\s*(\d+)[.)]\s+(.*)$/),o=e.match(/^\s*[-*•]\s+(.*)$/);t?(a(),n.push(`<div class="ws-h">${s(t[1])}</div>`)):i?(r!==`ol`&&(a(),n.push(`<ol class="ws-ol">`),r=`ol`),n.push(`<li>${s(i[2])}</li>`)):o?(r!==`ul`&&(a(),n.push(`<ul class="ws-bullets">`),r=`ul`),n.push(`<li>${s(o[1])}</li>`)):(a(),e.trim()&&n.push(`<p>${s(e)}</p>`))}return a(),n.join(``)});return(t,r)=>(e(),n(`div`,{class:`ws-text leading-relaxed motion-safe:transition-opacity motion-safe:duration-300`,innerHTML:c.value},null,8,a))}}),[[`__scopeId`,`data-v-51d1042a`]]);export{o as default};
vivek/frontend-vue/dist/assets/TextBlock-ChIy4qJQ.css ADDED
@@ -0,0 +1 @@
 
 
1
+ .ws-text[data-v-51d1042a] p{margin:0 0 .45em}.ws-text[data-v-51d1042a] p:last-child{margin-bottom:0}.ws-text[data-v-51d1042a] .ws-h{letter-spacing:.04em;text-transform:uppercase;color:hsl(var(--muted-foreground));margin:.2em 0 .5em;font-size:.8rem;font-weight:700}.ws-text[data-v-51d1042a] .ws-bullets{flex-direction:column;gap:.4em;margin:.25em 0;padding-left:0;list-style:none;display:flex}.ws-text[data-v-51d1042a] .ws-bullets li{padding-left:1.1em;position:relative}.ws-text[data-v-51d1042a] .ws-bullets li:before{content:"";background:hsl(var(--primary));border-radius:9999px;width:6px;height:6px;position:absolute;top:.55em;left:0}.ws-text[data-v-51d1042a] .ws-ol{counter-reset:ws;flex-direction:column;gap:.5em;margin:.25em 0;padding-left:0;list-style:none;display:flex}.ws-text[data-v-51d1042a] .ws-ol li{counter-increment:ws;padding-left:1.9em;position:relative}.ws-text[data-v-51d1042a] .ws-ol li:before{content:counter(ws);width:1.35em;height:1.35em;color:hsl(var(--primary));background:hsl(var(--primary) / .12);border-radius:9999px;justify-content:center;align-items:center;font-size:.72em;font-weight:700;display:inline-flex;position:absolute;top:.05em;left:0}.ws-text[data-v-51d1042a] .ws-accent{color:hsl(var(--primary));font-weight:600}.ws-text[data-v-51d1042a] .ws-code{background:hsl(var(--primary) / .1);color:hsl(var(--primary));border-radius:.3rem;padding:.05em .35em;font-size:.88em}
vivek/frontend-vue/dist/assets/TextBlock-CoLO1TqP.js DELETED
@@ -1 +0,0 @@
1
- import{M as e,f as t,jt as n,v as r}from"./runtime-core.esm-bundler-olIhRSij.js";var i={class:`whitespace-pre-wrap leading-relaxed motion-safe:transition-opacity motion-safe:duration-300`},a=r({__name:`TextBlock`,props:{block:{}},setup(r){return(a,o)=>(e(),t(`div`,i,n(r.block.content||``),1))}});export{a as default};
 
 
vivek/frontend-vue/dist/assets/index-CKwtM7do.css ADDED
The diff for this file is too large to render. See raw diff
 
vivek/frontend-vue/dist/assets/{index-0cd4ouHe.js → index-Dv2_3L6X.js} RENAMED
The diff for this file is too large to render. See raw diff
 
vivek/frontend-vue/dist/assets/index-yy7aehFb.css DELETED
The diff for this file is too large to render. See raw diff
 
vivek/frontend-vue/dist/index.html CHANGED
@@ -5,9 +5,9 @@
5
  <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
6
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7
  <title>frontend-vue</title>
8
- <script type="module" crossorigin src="/assets/index-0cd4ouHe.js"></script>
9
  <link rel="modulepreload" crossorigin href="/assets/runtime-core.esm-bundler-olIhRSij.js">
10
- <link rel="stylesheet" crossorigin href="/assets/index-yy7aehFb.css">
11
  </head>
12
  <body>
13
  <div id="app"></div>
 
5
  <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
6
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7
  <title>frontend-vue</title>
8
+ <script type="module" crossorigin src="/assets/index-Dv2_3L6X.js"></script>
9
  <link rel="modulepreload" crossorigin href="/assets/runtime-core.esm-bundler-olIhRSij.js">
10
+ <link rel="stylesheet" crossorigin href="/assets/index-CKwtM7do.css">
11
  </head>
12
  <body>
13
  <div id="app"></div>
vivek/frontend-vue/src/components/WidgetSchemaChart.vue CHANGED
@@ -96,7 +96,7 @@ watch(() => props.chart, refresh, { deep: true })
96
  <ArrowDownTrayIcon class="h-3 w-3" /> PNG
97
  </button>
98
  </div>
99
- <div v-if="renderable" ref="rootEl" class="w-full h-[min(360px,52vh)] min-h-[220px]" />
100
  <div v-else class="px-3 py-6 text-xs text-muted-foreground text-center">
101
  No renderable data for chart type "{{ kind }}".
102
  </div>
 
96
  <ArrowDownTrayIcon class="h-3 w-3" /> PNG
97
  </button>
98
  </div>
99
+ <div v-if="renderable" ref="rootEl" class="w-full h-[clamp(340px,56vh,560px)] min-h-[300px]" />
100
  <div v-else class="px-3 py-6 text-xs text-muted-foreground text-center">
101
  No renderable data for chart type "{{ kind }}".
102
  </div>
vivek/frontend-vue/src/components/widgets/TextBlock.vue CHANGED
@@ -1,9 +1,150 @@
1
  <script setup lang="ts">
2
- defineProps<{ block: { content?: string } }>()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  </script>
4
 
5
  <template>
6
- <div class="whitespace-pre-wrap leading-relaxed motion-safe:transition-opacity motion-safe:duration-300">
7
- {{ block.content || '' }}
8
- </div>
 
9
  </template>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  <script setup lang="ts">
2
+ import { computed } from 'vue'
3
+
4
+ const props = defineProps<{ block: { content?: string } }>()
5
+
6
+ // Escape first (safe), then apply a tiny markdown subset so the model's analysis can use
7
+ // "- " bullets and **bold**/`$ figures` highlighted in the accent color. No raw HTML allowed.
8
+ function esc(s: string): string {
9
+ return s.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;')
10
+ }
11
+
12
+ function inline(s: string): string {
13
+ let t = esc(s)
14
+ // **bold** and __bold__ → accent (blue) emphasis
15
+ t = t.replace(/\*\*(.+?)\*\*/g, '<strong class="ws-accent">$1</strong>')
16
+ t = t.replace(/__(.+?)__/g, '<strong class="ws-accent">$1</strong>')
17
+ // `code` / inline figures
18
+ t = t.replace(/`([^`]+?)`/g, '<code class="ws-code">$1</code>')
19
+ return t
20
+ }
21
+
22
+ const html = computed(() => {
23
+ const raw = String(props.block.content || '').replace(/\r\n/g, '\n').trim()
24
+ if (!raw) return ''
25
+ const lines = raw.split('\n')
26
+ const out: string[] = []
27
+ let list: 'ul' | 'ol' | null = null
28
+ const closeList = () => {
29
+ if (list) {
30
+ out.push(`</${list}>`)
31
+ list = null
32
+ }
33
+ }
34
+ for (const line of lines) {
35
+ const heading = line.match(/^\s*#{1,3}\s+(.*)$/)
36
+ const ordered = line.match(/^\s*(\d+)[.)]\s+(.*)$/)
37
+ const bullet = line.match(/^\s*[-*•]\s+(.*)$/)
38
+ if (heading) {
39
+ closeList()
40
+ out.push(`<div class="ws-h">${inline(heading[1])}</div>`)
41
+ } else if (ordered) {
42
+ if (list !== 'ol') {
43
+ closeList()
44
+ out.push('<ol class="ws-ol">')
45
+ list = 'ol'
46
+ }
47
+ out.push(`<li>${inline(ordered[2])}</li>`)
48
+ } else if (bullet) {
49
+ if (list !== 'ul') {
50
+ closeList()
51
+ out.push('<ul class="ws-bullets">')
52
+ list = 'ul'
53
+ }
54
+ out.push(`<li>${inline(bullet[1])}</li>`)
55
+ } else {
56
+ closeList()
57
+ if (line.trim()) out.push(`<p>${inline(line)}</p>`)
58
+ }
59
+ }
60
+ closeList()
61
+ return out.join('')
62
+ })
63
  </script>
64
 
65
  <template>
66
+ <div
67
+ class="ws-text leading-relaxed motion-safe:transition-opacity motion-safe:duration-300"
68
+ v-html="html"
69
+ />
70
  </template>
71
+
72
+ <style scoped>
73
+ .ws-text :deep(p) {
74
+ margin: 0 0 0.45em;
75
+ }
76
+ .ws-text :deep(p:last-child) {
77
+ margin-bottom: 0;
78
+ }
79
+ .ws-text :deep(.ws-h) {
80
+ font-size: 0.8rem;
81
+ font-weight: 700;
82
+ letter-spacing: 0.04em;
83
+ text-transform: uppercase;
84
+ color: hsl(var(--muted-foreground));
85
+ margin: 0.2em 0 0.5em;
86
+ }
87
+ .ws-text :deep(.ws-bullets) {
88
+ margin: 0.25em 0;
89
+ padding-left: 0;
90
+ list-style: none;
91
+ display: flex;
92
+ flex-direction: column;
93
+ gap: 0.4em;
94
+ }
95
+ .ws-text :deep(.ws-bullets li) {
96
+ position: relative;
97
+ padding-left: 1.1em;
98
+ }
99
+ .ws-text :deep(.ws-bullets li)::before {
100
+ content: '';
101
+ position: absolute;
102
+ left: 0;
103
+ top: 0.55em;
104
+ width: 6px;
105
+ height: 6px;
106
+ border-radius: 9999px;
107
+ background: hsl(var(--primary));
108
+ }
109
+ .ws-text :deep(.ws-ol) {
110
+ margin: 0.25em 0;
111
+ padding-left: 0;
112
+ list-style: none;
113
+ counter-reset: ws;
114
+ display: flex;
115
+ flex-direction: column;
116
+ gap: 0.5em;
117
+ }
118
+ .ws-text :deep(.ws-ol li) {
119
+ position: relative;
120
+ padding-left: 1.9em;
121
+ counter-increment: ws;
122
+ }
123
+ .ws-text :deep(.ws-ol li)::before {
124
+ content: counter(ws);
125
+ position: absolute;
126
+ left: 0;
127
+ top: 0.05em;
128
+ width: 1.35em;
129
+ height: 1.35em;
130
+ display: inline-flex;
131
+ align-items: center;
132
+ justify-content: center;
133
+ font-size: 0.72em;
134
+ font-weight: 700;
135
+ border-radius: 9999px;
136
+ color: hsl(var(--primary));
137
+ background: hsl(var(--primary) / 0.12);
138
+ }
139
+ .ws-text :deep(.ws-accent) {
140
+ color: hsl(var(--primary));
141
+ font-weight: 600;
142
+ }
143
+ .ws-text :deep(.ws-code) {
144
+ font-size: 0.88em;
145
+ padding: 0.05em 0.35em;
146
+ border-radius: 0.3rem;
147
+ background: hsl(var(--primary) / 0.1);
148
+ color: hsl(var(--primary));
149
+ }
150
+ </style>
vivek/frontend-vue/src/lib/echartsOption.ts CHANGED
@@ -706,8 +706,11 @@ export function buildEChartsOption(chart: ChartSpec, title = '', opts: { dark?:
706
  type: 'category' as const,
707
  data: cats,
708
  name: (isHBar ? chart?.y_label : chart?.x_label) || '',
709
- nameLocation: 'middle' as const,
710
- nameGap: 30,
 
 
 
711
  axisLabel: { fontSize: 10, rotate: !isHBar && (cats as string[]).length > 6 ? 30 : 0 },
712
  }
713
  : {
 
706
  type: 'category' as const,
707
  data: cats,
708
  name: (isHBar ? chart?.y_label : chart?.x_label) || '',
709
+ // For hbar the category axis is vertical, so a centered axis name overlaps the
710
+ // (often long) category labels — place it at the axis end instead.
711
+ nameLocation: (isHBar ? 'end' : 'middle') as 'end' | 'middle',
712
+ nameGap: isHBar ? 12 : (cats as string[]).length > 6 ? 42 : 28,
713
+ nameTextStyle: isHBar ? { align: 'right' as const, color: tc } : { color: tc },
714
  axisLabel: { fontSize: 10, rotate: !isHBar && (cats as string[]).length > 6 ? 30 : 0 },
715
  }
716
  : {
vivek/frontend-vue/src/lib/exportWidgetHtml.ts CHANGED
@@ -176,8 +176,8 @@ function renderBlock(block: Block, idx: number, charts: ChartEntry[]): string {
176
 
177
  const STYLES = `
178
  *{box-sizing:border-box}
179
- body{margin:0;background:#f7f8fa;color:#111318;font:14px/1.5 system-ui,-apple-system,Segoe UI,Roboto,sans-serif;padding:24px}
180
- .ws-doc{max-width:880px;margin:0 auto;display:flex;flex-direction:column;gap:14px}
181
  .ws-card{background:#fff;border:1px solid #e6e8ec;border-radius:14px;padding:16px;box-shadow:0 1px 2px rgba(0,0,0,.04)}
182
  .ws-text{margin:0;color:#2b2f38}
183
  .ws-h{font-weight:600;font-size:13px;margin-bottom:8px}
@@ -202,10 +202,10 @@ body{margin:0;background:#f7f8fa;color:#111318;font:14px/1.5 system-ui,-apple-sy
202
  .ws-action-row{display:flex;flex-wrap:wrap;gap:8px}
203
  .ws-action{font-size:13px;padding:6px 12px;border:1px solid #d7dae0;border-radius:8px;background:#fff;color:#475569}
204
  .ws-fig{margin:0}
205
- .ws-fig img{width:100%;max-height:420px;border-radius:10px}
206
  .ws-fig figcaption{font-size:12px;color:#9aa1ad;margin-top:6px;text-align:center}
207
- .ws-chart{width:100%;height:380px}
208
- .ws-chart-3d{height:460px}
209
  .ws-note{font-size:12px;color:#9aa1ad}
210
  .ws-footer{text-align:center;font-size:11px;color:#b3b9c4;margin-top:6px}
211
  `
 
176
 
177
  const STYLES = `
178
  *{box-sizing:border-box}
179
+ body{margin:0;background:#f7f8fa;color:#111318;font:15px/1.6 system-ui,-apple-system,Segoe UI,Roboto,sans-serif;padding:clamp(16px,3vw,40px)}
180
+ .ws-doc{max-width:1200px;width:96vw;margin:0 auto;display:flex;flex-direction:column;gap:18px}
181
  .ws-card{background:#fff;border:1px solid #e6e8ec;border-radius:14px;padding:16px;box-shadow:0 1px 2px rgba(0,0,0,.04)}
182
  .ws-text{margin:0;color:#2b2f38}
183
  .ws-h{font-weight:600;font-size:13px;margin-bottom:8px}
 
202
  .ws-action-row{display:flex;flex-wrap:wrap;gap:8px}
203
  .ws-action{font-size:13px;padding:6px 12px;border:1px solid #d7dae0;border-radius:8px;background:#fff;color:#475569}
204
  .ws-fig{margin:0}
205
+ .ws-fig img{width:100%;max-height:min(70vh,640px);object-fit:contain;border-radius:10px}
206
  .ws-fig figcaption{font-size:12px;color:#9aa1ad;margin-top:6px;text-align:center}
207
+ .ws-chart{width:100%;height:clamp(420px,68vh,720px)}
208
+ .ws-chart-3d{height:clamp(480px,78vh,820px)}
209
  .ws-note{font-size:12px;color:#9aa1ad}
210
  .ws-footer{text-align:center;font-size:11px;color:#b3b9c4;margin-top:6px}
211
  `
vivek/frontend-vue/src/widget-registry.json CHANGED
@@ -3,9 +3,9 @@
3
  "blocks": [
4
  {
5
  "type": "text",
6
- "whenToUse": "Narrative explanation, framing sentence, or any prose paragraph.",
7
  "spec": [
8
- "{ \"type\": \"text\", \"id\": \"...\", \"content\": \"...\" }"
9
  ],
10
  "example": {
11
  "type": "text",
 
3
  "blocks": [
4
  {
5
  "type": "text",
6
+ "whenToUse": "Narrative explanation, analysis, or framing. Supports a light markdown subset in content: \"- \" bullet lines and **bold** (bold renders highlighted in the accent/blue color). Great for analyst commentary as bullet points.",
7
  "spec": [
8
+ "{ \"type\": \"text\", \"id\": \"...\", \"content\": \"- **Data Center** at **$47.5B** is ~**78%** of total\\n- **Gaming** recovered to **$10.4B** (+14% YoY)\" }"
9
  ],
10
  "example": {
11
  "type": "text",
vivek/rag_finance/retriever.py CHANGED
@@ -87,27 +87,32 @@ def retrieve(query: str, k: int = 6) -> dict:
87
  return _empty(query)
88
  except Exception:
89
  pass
90
- # Retrieve a wider pool so entity filtering can find the right company's docs.
91
- res = col.query(query_texts=[_clean_for_retrieval(query)], n_results=max(k, 12))
92
- ids = res["ids"][0]
93
- docs = res["documents"][0]
94
- metas = res["metadatas"][0]
95
- dists = res["distances"][0]
96
- rows = list(zip(ids, docs, metas, dists))
97
-
98
- # ENTITY-AWARE RELEVANCE GATE:
99
- # - Query names a company we HAVE → keep that company's docs (let the synthesizer
100
- # handle any missing metric). Distance alone is unreliable for minor metrics.
101
- # - Query names NO known company → distance gate (catches off-topic / unknown companies).
102
  mentioned = _mentioned_ids(query)
103
  if mentioned:
104
- kept = [r for r in rows if r[2].get("company_id") in mentioned]
105
- if not kept: # company named but its docs weren't in the pool → loose distance fallback
106
- kept = [r for r in rows if r[3] <= ENTITY_THRESHOLD]
 
 
 
 
 
 
 
 
 
 
 
 
107
  else:
108
- kept = [r for r in rows if r[3] <= RELEVANCE_THRESHOLD]
109
-
110
- kept = kept[:9] # cap context size
111
  relevant = len(kept) > 0
112
 
113
  doc_ids = [x[0] for x in kept]
@@ -124,7 +129,7 @@ def retrieve(query: str, k: int = 6) -> dict:
124
  "metas": kept_metas,
125
  "evidence": packet,
126
  "relevant": relevant,
127
- "best_distance": min(dists) if dists else None,
128
  }
129
 
130
 
 
87
  return _empty(query)
88
  except Exception:
89
  pass
90
+ # ENTITY-AWARE RETRIEVAL:
91
+ # - Query NAMES companies we have → fetch ALL their docs (every fiscal year) by metadata,
92
+ # so multi-company comparisons get full coverage (embedding top-k can miss some).
93
+ # The synthesizer then picks the right year(s).
94
+ # - Query names NO known company → embedding search + distance gate (catches off-topic).
 
 
 
 
 
 
 
95
  mentioned = _mentioned_ids(query)
96
  if mentioned:
97
+ try:
98
+ got = col.get(where={"company_id": {"$in": list(mentioned)}})
99
+ gids = got.get("ids") or []
100
+ gdocs = got.get("documents") or []
101
+ gmetas = got.get("metadatas") or []
102
+ kept = list(zip(gids, gdocs, gmetas, [0.0] * len(gids)))
103
+ except Exception:
104
+ kept = []
105
+ if not kept: # fallback: embedding pool filtered to mentioned companies
106
+ res = col.query(query_texts=[_clean_for_retrieval(query)], n_results=24)
107
+ rows = list(zip(res["ids"][0], res["documents"][0], res["metadatas"][0], res["distances"][0]))
108
+ kept = [r for r in rows if r[2].get("company_id") in mentioned] or [
109
+ r for r in rows if r[3] <= ENTITY_THRESHOLD
110
+ ]
111
+ kept = kept[:21] # all docs for up to ~7 companies × 3 years
112
  else:
113
+ res = col.query(query_texts=[_clean_for_retrieval(query)], n_results=max(k, 12))
114
+ rows = list(zip(res["ids"][0], res["documents"][0], res["metadatas"][0], res["distances"][0]))
115
+ kept = [r for r in rows if r[3] <= RELEVANCE_THRESHOLD][:9]
116
  relevant = len(kept) > 0
117
 
118
  doc_ids = [x[0] for x in kept]
 
129
  "metas": kept_metas,
130
  "evidence": packet,
131
  "relevant": relevant,
132
+ "best_distance": (min((x[3] for x in kept), default=None) if kept else None),
133
  }
134
 
135
 
vivek/requirements.txt CHANGED
@@ -6,3 +6,4 @@ uvicorn[standard]
6
  SQLAlchemy
7
  PyJWT
8
  bcrypt
 
 
6
  SQLAlchemy
7
  PyJWT
8
  bcrypt
9
+ chromadb>=1.0