ishaq101's picture
fix/ check and new chart tool (#16)
5a60e93
Raw
History Blame Contribute Delete
7.22 kB
"""`is_report_ready` — deterministic report-readiness signal (seam #5, KM-652).
The Help skill asks "can the user generate a report yet?" before it offers that as
a next step. This is the producer of that answer; Help only *consumes* it (see
`handlers/help.ReportReadiness`). No LLM — readiness is a fact about persisted state,
not a judgement.
The rule mirrors what makes a real report non-empty and worth generating, so Help can
never suggest an action that would 409 or produce a duplicate:
1. (removed 2026-06-24) a validated problem statement — the report no longer gates on
the goal (now user-entered `objective` + `business_questions`, no agent validation).
2. at least one **substantive** persisted `AnalysisRecord` — a record whose
*analysis* task succeeded. A failed run still persists a record WITH findings
(they narrate the failure), and data-access tasks (check_/retrieve_) succeed even
when the analysis fails — so neither "has findings" nor "any task succeeded" is
enough. We require a genuine analysis tool (analyze_*) to have completed. We count
*results*, not chat turns.
3. delta-since-report — if a report already exists (`state.report_id`), only ready
when there's a substantive analysis newer than the latest report; otherwise the
new report would be identical.
`missing` names whichever criterion is absent, so Help can tell the user the next gap
to fill (the team values `missing` over the bare boolean). Bias is anti-false-positive
(report is also button-triggered): a record-store read failure fails **closed**
(not ready); a report-store read failure during the delta check fails **open** (we
can't prove staleness, and the button is always there).
NOT in scope (deferred, pending the readiness eval set): semantic *alignment* of the
analyses to the problem statement and *depth*/variety scoring — both need an LLM judge
and shouldn't sit in the per-turn Help hot path until eval justifies the cost.
"""
from __future__ import annotations
from datetime import UTC, datetime
from typing import TYPE_CHECKING
from src.middlewares.logging import get_logger
from ..handlers.help import ReportReadiness
if TYPE_CHECKING:
from ..gate import AnalysisState
logger = get_logger("report_readiness")
# Human-readable gaps surfaced to the user via Help (kept stable for the prompt).
# _MISSING_PROBLEM retired 2026-06-24 — the report no longer gates on a validated goal.
_MISSING_ANALYSIS = "at least one completed analysis"
_MISSING_DELTA = "a new analysis since the last report"
def _default_record_store():
from ..slow_path.store import PostgresReportInputStore
return PostgresReportInputStore()
def _default_report_store():
from .store import ReportStore
return ReportStore()
def _is_newer(a: datetime, b: datetime) -> bool:
"""True if `a` is later than `b`, tolerating naive/aware mismatch (assume UTC)."""
if a.tzinfo is None:
a = a.replace(tzinfo=UTC)
if b.tzinfo is None:
b = b.replace(tzinfo=UTC)
return a > b
def has_successful_analysis(record) -> bool:
"""True if the record has at least one *result-producing* task that succeeded.
A failed run still writes findings (narrating the failure) and its data-access
tasks (check_/retrieve_) succeed, so we can't key on findings or on "any task
succeeded". A completed analysis tool (analyze_*) — or, since W2 charts
(2026-07-14), a completed `render_chart`, whose viz-tail upstream necessarily
computed the numbers being charted — is the real "we produced a result"
signal. A chart-only session therefore satisfies the report floor.
"""
return any(
t.status == "success"
and any(tool.startswith("analyze") or tool == "render_chart" for tool in t.tools_used)
for t in record.tasks_run
)
async def report_floor(
analysis_id: str | None,
state: AnalysisState,
*,
record_store=None,
) -> tuple[list[str], list]:
"""The report **floor**: ≥1 substantive analysis.
Returns `(missing, substantive_records)`. This is the shared gate both the Help
readiness signal AND the report API enforce, so the button and Help can't drift
(T-D / T11).
CHANGED 2026-06-24: the `problem_validated` precondition was dropped — analysis is no
longer gated on a validated goal (now user-entered `objective` + `business_questions`,
no agent validation), so the only floor is "is there anything worth reporting". The
delta-since-report check stays advisory and lives only in `is_report_ready`; the
report button is always allowed to cut a new version (decision 4A). Fails closed
(counts as missing analysis) on a record-store read error. `record_store` is
injectable for tests. `state` stays in the signature (callers + the `is_report_ready`
delta check use it).
"""
missing: list[str] = []
substantive: list = []
if analysis_id:
try:
store = record_store or _default_record_store()
records = await store.list_for_analysis(analysis_id)
substantive = [r for r in records if has_successful_analysis(r)]
except Exception as exc: # noqa: BLE001 — never-throw; fail closed to not-ready
logger.warning(
"report_floor: record store read failed — not ready",
analysis_id=analysis_id,
error=str(exc),
)
return [_MISSING_ANALYSIS], []
if not substantive:
missing.append(_MISSING_ANALYSIS)
return missing, substantive
async def is_report_ready(
analysis_id: str | None,
state: AnalysisState,
*,
record_store=None,
report_store=None,
) -> ReportReadiness:
"""Return whether a report can be generated for this analysis, and the gaps if not.
`record_store` / `report_store` are injectable for tests; they default to the
real Postgres stores.
"""
missing, substantive = await report_floor(
analysis_id, state, record_store=record_store
)
if not substantive:
# No analyses to report on → the delta check is moot.
return ReportReadiness(ready=not missing, missing=missing)
# Delta-since-report: a report already exists, so only ready if a substantive
# analysis is newer than the latest report. Fail-open on a report-store error.
if state.report_id:
last_report_at: datetime | None = None
try:
rstore = report_store or _default_report_store()
reports = await rstore.list_for_analysis(analysis_id)
last_report_at = max((r.generated_at for r in reports), default=None)
except Exception as exc: # noqa: BLE001 — skip delta; can't prove staleness
logger.warning(
"is_report_ready: report store read failed — skipping delta check",
analysis_id=analysis_id,
error=str(exc),
)
if last_report_at is not None and not any(
_is_newer(r.created_at, last_report_at) for r in substantive
):
missing.append(_MISSING_DELTA)
return ReportReadiness(ready=not missing, missing=missing)