Upgrade RAGForge to v1.7 evaluation correctness and adaptive efficiency

README.md

@@ -10,12 +10,27 @@ pinned: false
 
 # RAGForge
 
-**RAGForge v1.6 - a production-style, portfolio-ready agentic RAG and analytical synthesis system for Hugging Face Spaces.**
+**RAGForge v1.7 - a production-style, portfolio-ready agentic RAG and analytical synthesis system for Hugging Face Spaces.**
 
 RAGForge combines hybrid document retrieval, source-level/hierarchical retrieval, semantic query planning, corrective RAG, Self-RAG-style verification, Text2SQL and an “Ask-the-Web” research path in one CPU-friendly application. The default LLM is **Google Gemini 3.5 Flash-Lite**; the UI also exposes Gemini 3.1 Flash-Lite and stronger Flash models.
 
 
 
+## What is new in v1.7
+
+v1.7 is an evidence-driven correctness, provenance and efficiency release based on the first full v1.6 Hard Mode/profile benchmark. It fixes evaluator/UI artifacts before changing retrieval behavior and turns measured profile/reranker results into explicit runtime policy.
+
+- **Markdown-safe source cards** - retrieved Markdown is rendered as escaped plain text inside uniform source cards, so a snippet beginning with `#` can no longer become a giant UI heading.
+- **Graphical latency waterfall** - Pipeline Inspector replaces ASCII `#####` bars with proportional HTML latency bars and exposes the reranker decision reason plus grounded-absence state.
+- **Grounded absence handling** - answers such as “the retrieved policy does not mention a dispute fee” are treated as calibrated no-answer responses instead of low-confidence hallucinations. This avoids unnecessary revise calls and gives Hard Mode a robust missing-answer matcher.
+- **Markdown-aware citation coverage** - numbered/bulleted claims count even when short, while generic list introductions/headings do not. This fixes false 0% coverage for answers such as the four NIST AI RMF functions.
+- **Table-source validity for overviews** - global corpus overviews surface deterministic DuckDB table evidence as `[T#]` sources, so structured claims can be cited validly instead of referring to a table ID absent from the source list.
+- **Fresh-vs-saved evaluation provenance** - each saved report gets a run ID and server-boot ID. Fresh execution messages are no longer overwritten by the saved-run selector, and saved-run loading is triggered only by explicit user input.
+- **Profile-policy summary** - optional Fast/Balanced/Agentic benchmarking now produces aggregated profile metrics and an evidence-scoped recommendation rather than only six raw rows.
+- **Context-efficiency diagnostics** - evaluation now calls out the case where Recall@5 is excellent but Precision@5 is low, separating “found the right source” from “sent too many distractors to generation”.
+- **Small-corpus reranker policy tightened** - because v1.6 showed identical source and chunk metrics with/without reranking while adding multi-second latency, the cross-encoder is skipped even in Agentic on small corpora. It remains available for larger corpora.
+- **Diagnostic readability** - each `Next:` recommendation starts on its own line.
+
 ## What is new in v1.6
 
 v1.6 moves RAGForge from a mostly saturated demo benchmark into harder evaluation and analytical synthesis. The goal is to make the next improvements measurable rather than simply adding more RAG components.

SECURITY.md

@@ -50,3 +50,7 @@ sanitized from a fixed UI table label and evaluation depth; user-supplied paths
 - The existing Text2SQL path still validates model-authored SQL as a single read-only `SELECT`/CTE before execution.
 - Semantic citation attribution uses the already-loaded local embedding model only as a high-threshold fallback. It can attach a citation label, but it cannot change source text, execute instructions, or grant retrieved content instruction priority.
 - Hard Mode includes an explicit prompt-injection detector case so regressions in stored-instruction detection remain visible in the evaluation report.
+
+## Evaluation provenance (v1.7)
+
+Saved evaluation metadata contains only operational provenance (short run ID, server-boot ID, model, benchmark version, corpus version and timestamp). It does not add user document content to browser storage. The opaque browser session ID remains the only client-persisted workspace identifier.

docs/ARCHITECTURE_API.md

@@ -1,4 +1,4 @@
-# Architecture and API - v1.6
+# Architecture and API - v1.7
 
 ## Runtime architecture
 
@@ -130,3 +130,11 @@ new network API: it materializes CSV/TSV/Markdown files inside the current ephem
 ## v1.6 evaluation observability
 
 The UI exposes Hard Mode, optional profile comparison, node-latency summaries and timestamped evaluation history. `GET /api/v1/evaluation/history/{session_id}` exposes the same archived-run metadata to API clients. The normal latest-run endpoints remain unchanged.
+
+## v1.7 runtime provenance and calibrated absence
+
+Evaluation cache metadata includes a short run ID and server-boot ID. These fields make it possible to distinguish a fresh benchmark from a saved report in the UI/API without relying on ambiguous status text. `GET /api/v1/evaluation/saved/{session_id}` includes this provenance in its inventory.
+
+The verify path now recognizes a grounded absence answer: an evidence-cited statement that the requested fact is not present in the selected sources. This state skips the normal low-confidence revise branch, preventing a second generation call whose only purpose would be to restate the same absence.
+
+For corpus overviews, structured tables are surfaced as deterministic `[T#]` evidence alongside the source-balanced document set. Analytical synthesis continues to use the same table evidence path.

docs/EVALUATION.md

@@ -1,4 +1,13 @@
-# RAGForge evaluation - v1.6
+# RAGForge evaluation - v1.7
+
+
+## v1.7 evaluator correctness and provenance
+
+v1.7 changes benchmark semantics where the v1.6 report exposed evaluator artifacts rather than RAG failures. Citation coverage is Markdown-aware: short numbered/bulleted factual items count, headings and generic list introductions do not. Missing-answer cases use a grounded-absence matcher that accepts natural uncertainty language instead of a single canned phrase. Global overviews include table evidence so `[T#]` citations are part of the actual returned source set.
+
+Saved reports now carry a `run_id` and `server_boot_id`. A fresh run is reported as fresh, while loading/reusing a saved run explicitly reports zero new Gemini requests. Programmatic updates to the saved-run selector no longer trigger a second UI callback that can overwrite fresh-run status.
+
+Profile benchmarking also emits `profile_summary` rows (accuracy, citation quality, median latency, LLM-call count and reranker rate) plus an evidence-scoped profile recommendation. Evaluation diagnostics additionally surface context-efficiency when recall is high but source precision is low, and identify the dominant node-latency bottleneck.
 
 ## v1.6 hard-mode and analytical evaluation
 

docs/FEATURE_MATRIX.md

@@ -39,7 +39,7 @@
 | Caching | TTL result cache keyed by session/config/corpus version | latency/quota reduction without stale cross-corpus answers |
 | Rate limiting | sliding-window per IP | protects a public shared model key |
 | Observability | Prometheus + semantic plan/evidence/correction/node trace + node time/estimated LLM calls/web/correction/reranker/citation-repair flags | makes agent decisions and efficiency inspectable |
-| Evaluation | transparent v1.6 benchmark + source/chunk ranking metrics + hard-mode robustness + citations + planner/web policy + typed Text2SQL + abstention + pacing-aware latency + calibrated Deep judge | makes the benchmark harder as the original demo saturates and separates retrieval, orchestration, analysis and generation failures |
+| Evaluation | transparent v1.7 benchmark + Markdown-aware citation claims + grounded-absence robustness + source/chunk ranking metrics + hard-mode + planner/web policy + typed Text2SQL + pacing-aware latency + calibrated Deep judge | makes the benchmark harder as the original demo saturates and separates retrieval, orchestration, analysis and generation failures |
 | Saved evaluation history | latest Quick/Standard/Deep cache plus timestamped archived runs and deltas | allows instant run switching and lightweight within-workspace regression tracking without consuming Gemini quota again |
 | Evaluation report serialization | JSON-safe normalization at save/load/UI/API boundaries + formatted raw JSON view | prevents `root={...}` wrapper leakage and keeps fresh/restored reports identical |
 | Evaluation table export | CSV/TSV/Markdown export for every benchmark table | makes Text2SQL/planner/QA/overview/ablation/abstention/comparison results easy to copy or download |
@@ -67,3 +67,7 @@ The demo uses embedded Qdrant, in-memory DuckDB, deterministic source profiles,
 
 | Profile benchmark | optional Fast/Balanced/Agentic labeled comparison | quantifies whether additional planning, reranking and verification cost is justified for representative tasks |
 | Hard-mode robustness | paraphrase/distractor/missing/multi-hop/insight/SQL/local-freshness/injection cases | prevents a near-perfect easy benchmark from becoming meaningless as a regression signal |
+
+| Evaluation provenance | run ID + server-boot ID + fresh/reused status | makes cache/rebuild behavior auditable and prevents fresh runs from being mistaken for saved reuse |
+| Context-efficiency diagnostics | source Precision@5 beside Recall@5 + targeted recommendations | exposes distractor-heavy context without sacrificing overview/synthesis breadth prematurely |
+| Grounded absence | evidence-cited missing-information answers skip unnecessary revise | rewards calibrated uncertainty and reduces extra model calls on unanswerable local questions |

docs/MIGRATION_1.7.md

@@ -0,0 +1,19 @@
+# Migration to RAGForge v1.7
+
+v1.7 is a correctness, provenance and measured-efficiency release over v1.6. Runtime dependency pins are unchanged.
+
+## Highlights
+
+- escaped uniform source cards;
+- graphical node-latency waterfall;
+- Markdown-aware citation coverage;
+- grounded missing-information answers and no unnecessary revise call;
+- valid `[T#]` evidence on corpus overviews;
+- evaluation run IDs/server-boot provenance and user-input-only saved-run switching;
+- profile-policy and context-efficiency diagnostics;
+- small-corpus reranker skip extended to Agentic after source+chunk ablation showed no gain;
+- benchmark version `1.7`.
+
+Because citation/missing-answer scoring semantics changed, v1.6 saved reports remain historical and are not eligible for v1.7 automatic benchmark reuse.
+
+Apply the patch, rebuild the Space, index the demo corpus, run Quick first, then Standard. A fresh run should say `Fresh ... evaluation complete`, while an explicit saved-run load should say `Loaded saved ... run`.

docs/QUERY_PLANNING.md

@@ -142,3 +142,7 @@ The `retrieve`/`web` trace records `reranker_used` and `reranker_reason`. The ex
 - “Compare our NIST document with the latest online guidance” remains `comparison -> hierarchical` with mixed/web relevance.
 
 The planner is explicitly told that a collection-wide insight question should stay local/analytical even when structured tables are present; the existence of a table alone does not force SQL routing.
+
+## v1.7 grounded absence behavior
+
+A corpus-scoped fact lookup can legitimately conclude that the indexed evidence does not state the requested fact. v1.7 treats an evidence-cited absence statement as calibrated uncertainty, not as a hallucination signal. This does not change routing to the web: web fallback still requires semantic relevance/permission. The grounded-absence state simply prevents an unnecessary answer-revision call when the model has already answered conservatively from the local evidence.

docs/RESUME_BULLETS.md

@@ -25,3 +25,5 @@
 
 - Added JSON-safe evaluation persistence and copy/export tooling for all benchmark result tables, keeping saved Quick/Standard/Deep reports directly comparable without rerunning model calls.
 - Hardened zero-call citation post-processing with grouped-citation normalization, duplicate-tail cleanup and conservative preamble skipping to improve citation usability without increasing inference cost.
+
+- Built an evaluation-driven RAG quality loop (v1.7) that corrected Markdown-aware citation scoring and missing-answer evaluation, added run-level cache provenance, and converted source/chunk ablations plus Fast/Balanced/Agentic benchmarks into adaptive reranker/profile policy diagnostics.

docs/SOURCES.md

@@ -63,3 +63,7 @@ The adaptive reranker policy and incremental evaluation design are internal engi
 ## v1.6 evaluation-driven policy
 
 Insight synthesis, hard-mode benchmark cases, table citation semantics, chunk-level reranker labels, profile comparison and evaluation-history deltas are RAGForge-specific engineering additions derived from the project's own observed evaluation gaps. They are not claims that one routing taxonomy or benchmark design is universally optimal.
+
+## v1.7 evaluation-policy note
+
+v1.7 does not add a new external dependency or benchmark dataset. The new policies are derived from RAGForge's own auditable v1.6 run data: perfect source/chunk ablation quality with large reranker latency, a false-negative missing-answer case, and Markdown citation-coverage artifacts.

docs/UX_LIFECYCLE.md

@@ -84,3 +84,11 @@ and download file are derived from the selected saved run; no Gemini request is
 Saving an evaluation still updates the latest Quick/Standard/Deep cache, but v1.6 also writes a timestamped historical copy inside the workspace. History is read-only and consumes no model quota. It is ephemeral with the Space container, just like the corpus.
 
 The query inspector adds a text-based node-latency waterfall. This uses existing LangGraph trace timings and introduces no plotting dependency or extra model call. Structured evidence produced by analytical synthesis appears in the source panel as `[T#]` table cards with row count, schema and a bounded preview.
+
+## v1.7 source, trace and saved-run UX
+
+Source snippets are rendered as escaped plain text inside uniform source cards. Source-controlled Markdown therefore cannot change font size or create headings in the Sources accordion.
+
+The Pipeline Inspector latency waterfall is now a proportional graphical bar display rather than a row of `#` characters. The same section exposes the adaptive reranker decision and whether the response is a grounded absence answer.
+
+Saved evaluation switching is bound to user input rather than generic change events. A fresh benchmark can update the selected depth without immediately reloading itself and overwriting the completion message. Fresh and reused statuses include the saved run ID; evaluation metadata also records the server boot that produced the report.

docs/architecture.mmd

@@ -66,3 +66,5 @@ flowchart TD
     EH -. compatible Standard baseline .-> EJ
 
     API[FastAPI /docs + OpenAPI + Prometheus] -. live introspection .-> UI
+
+%% v1.7: evidence-cited grounded-absence answers bypass the low-confidence revise loop.

evals/README.md

@@ -38,3 +38,7 @@ The `hard_mode_cases` section intentionally targets failure modes that the origi
 Selected QA cases also define `chunk_must_contain`. These labels are used only for chunk-level reranker Hit@1/MRR; unlabeled cases are excluded from those averages.
 
 The optional profile benchmark is not stored as a separate question set: it reuses a focused QA case and a cross-document case across Fast, Balanced and Agentic so the comparison is controlled.
+
+## v1.7 scoring corrections
+
+Citation coverage is Markdown-aware, missing-answer cases accept natural grounded-absence language, and global overview table citations are evaluated against actual `[T#]` source records. Reports also include run/server provenance and optional aggregated profile-policy summaries. Benchmark version `1.7` intentionally prevents automatic reuse of v1.6 reports under the changed scoring semantics.

evals/demo_benchmark.json

@@ -1,6 +1,6 @@
 {
-  "version": "1.6",
-  "description": "Hard-mode, transparent multi-layer benchmark for the bundled RAGForge demo corpus, including focused QA, analytical insight synthesis, adversarial routing, missing-answer robustness, chunk-level retrieval labels, Text2SQL and lifecycle abstention.",
+  "version": "1.7",
+  "description": "RAGForge v1.7 demo benchmark with Markdown-aware citation claims, grounded-absence robustness, analytical synthesis, table evidence, hard-mode cases, and source/chunk retrieval ablations.",
   "qa_cases": [
     {
       "id": "qa_acme_sev1_ack",

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "ragforge"
-version = "1.6.0"
+version = "1.7.0"
 description = "Production-style agentic RAG demo for Hugging Face Spaces"
 requires-python = ">=3.11"
 dependencies = []

src/ragforge/__init__.py

@@ -1,3 +1,3 @@
 """RAGForge: production-style agentic retrieval augmented generation demo."""
 
-__version__ = "1.6.0"
+__version__ = "1.7.0"

src/ragforge/api.py

@@ -29,7 +29,7 @@ def _auth(authorization: Annotated[str | None, Header()] = None) -> None:
 
 
 def create_api() -> FastAPI:
-    app = FastAPI(title="RAGForge API", version="1.6.0")
+    app = FastAPI(title="RAGForge API", version="1.7.0")
 
     @app.get("/api/health")
     def health():
@@ -54,7 +54,9 @@ def create_api() -> FastAPI:
                 "saved-evaluation-history", "incremental-deep-evaluation", "typed-text2sql-evaluation",
                 "adaptive-reranking", "semantic-citation-attribution", "insight-synthesis",
                 "table-citations", "hard-mode-evaluation", "chunk-level-reranker-ablation",
-                "optional-profile-benchmark", "node-latency-observability", "evaluation-run-history"
+                "optional-profile-benchmark", "node-latency-observability", "evaluation-run-history",
+                "grounded-absence-handling", "markdown-aware-citation-coverage", "evaluation-run-provenance",
+                "profile-policy-diagnostics", "context-efficiency-diagnostics"
             ],
         }
 

src/ragforge/citations.py

@@ -47,7 +47,12 @@ def repair_missing_citations(
     *,
     semantic_support: bool = True,
 ) -> tuple[str, int]:
-    """Attach citations only when an uncited factual unit clearly matches evidence."""
+    """Attach citations only when an uncited factual unit clearly matches evidence.
+
+    v1.7 repairs prose at sentence granularity. A paragraph that already has a
+    citation in sentence one must not cause sentence two to be treated as cited.
+    Bullets remain whole units so list formatting is preserved.
+    """
     if not answer or not sources:
         return answer, 0
     answer = normalize_citation_syntax(answer)
@@ -65,12 +70,14 @@ def repair_missing_citations(
         }
 
     evidence: list[tuple[str, set[str]]] = []
+    by_id: dict[str, dict[str, Any]] = {}
     for source in sources:
         sid = str(source.get("id", ""))
         if not re.fullmatch(r"(?:D|W|T)\d+", sid):
             continue
         text = f"{source.get('title', '')} {source.get('snippet', '')}"
         evidence.append((sid, toks(text)))
+        by_id[sid] = source
     if not evidence:
         return answer, 0
 
@@ -82,36 +89,20 @@ def repair_missing_citations(
             from .retrieval import ModelRegistry
 
             semantic_ids = [sid for sid, _ in evidence]
-            texts = []
-            by_id = {str(source.get("id", "")): source for source in sources}
-            for sid in semantic_ids:
-                source = by_id.get(sid, {})
-                texts.append(f"{source.get('title', '')} {source.get('snippet', '')}"[:2400])
+            texts = [
+                f"{by_id[sid].get('title', '')} {by_id[sid].get('snippet', '')}"[:2400]
+                for sid in semantic_ids
+            ]
             semantic_vectors = np.asarray(list(ModelRegistry.embedding().passage_embed(texts)), dtype=float)
             norms = np.linalg.norm(semantic_vectors, axis=1, keepdims=True) + 1e-9
             semantic_vectors = semantic_vectors / norms
         except Exception:
             semantic_vectors = None
 
-    repaired = 0
-    out: list[str] = []
-    for line in answer.splitlines():
-        stripped = line.strip()
-        plain = re.sub(r"[`*_#>-]", "", stripped).strip()
-        if (
-            not stripped
-            or re.search(r"\[(?:D|W|T)\d+\]", line)
-            or stripped.startswith("```")
-            or stripped.endswith(":")
-            or len(plain) < 24
-        ):
-            out.append(line)
-            continue
-
+    def choose_ids(plain: str) -> list[str]:
         unit_tokens = toks(plain)
         if not unit_tokens:
-            out.append(line)
-            continue
+            return []
         ranked: list[tuple[int, float, str]] = []
         for sid, source_tokens in evidence:
             overlap = len(unit_tokens & source_tokens)
@@ -125,7 +116,11 @@ def repair_missing_citations(
             selected_ids = [best_sid]
             if len(ranked) > 1:
                 second_overlap, second_support, second_sid = ranked[1]
-                if second_overlap >= 2 and second_support >= 0.20 and second_support >= best_score * 0.65:
+                if (
+                    second_overlap >= 2
+                    and second_support >= 0.20
+                    and second_support >= best_score * 0.65
+                ):
                     selected_ids.append(second_sid)
         elif semantic_vectors is not None and semantic_ids:
             try:
@@ -139,16 +134,73 @@ def repair_missing_citations(
                 best_idx = int(order[0])
                 best_sem = float(sims[best_idx])
                 second_sem = float(sims[int(order[1])]) if len(order) > 1 else -1.0
-                # Conservative attribution: require high semantic similarity and
-                # a margin over the next source. This avoids citation decoration.
                 if best_sem >= 0.68 and (best_sem - second_sem >= 0.055 or best_sem >= 0.78):
                     selected_ids = [semantic_ids[best_idx]]
             except Exception:
                 selected_ids = []
+        return selected_ids
 
-        if selected_ids:
-            out.append(line.rstrip() + " " + " ".join(f"[{sid}]" for sid in selected_ids))
-            repaired += len(selected_ids)
-        else:
+    def repair_unit(unit: str) -> tuple[str, int]:
+        stripped = unit.strip()
+        plain = re.sub(r"[`*_#>-]", "", stripped).strip()
+        if (
+            not stripped
+            or re.search(r"\[(?:D|W|T)\d+\]", unit)
+            or stripped.startswith("```")
+            or stripped.endswith(":")
+            or len(plain) < 24
+        ):
+            return unit, 0
+        selected_ids = choose_ids(plain)
+        if not selected_ids:
+            return unit, 0
+        citation_text = " ".join(f"[{sid}]" for sid in selected_ids)
+        trimmed = unit.rstrip()
+        terminal = trimmed[-1] if trimmed and trimmed[-1] in ".!?" else ""
+        if terminal:
+            trimmed = trimmed[:-1].rstrip()
+            return f"{trimmed} {citation_text}{terminal}", len(selected_ids)
+        return f"{trimmed} {citation_text}", len(selected_ids)
+
+    repaired = 0
+    out: list[str] = []
+    for line in answer.splitlines():
+        stripped = line.strip()
+        if not stripped or stripped.startswith("```") or stripped.endswith(":"):
             out.append(line)
+            continue
+
+        # Keep list items intact. The evaluator also treats one list item as one
+        # factual unit, so this preserves readable Markdown and avoids citation
+        # decoration on every short clause inside a bullet.
+        if re.match(r"^\s*(?:[-*+]\s+|\d+[.)]\s+)", line):
+            repaired_line, count = repair_unit(line)
+            out.append(repaired_line)
+            repaired += count
+            continue
+
+        # Move a citation written after sentence punctuation back onto that
+        # sentence, then protect common abbreviations before splitting.
+        split_text = re.sub(
+            r"([.!?])\s+((?:\[(?:D|W|T)\d+(?:\s*,\s*(?:D|W|T)\d+)*\]\s*)+)",
+            r" \2\1 ",
+            line,
+        )
+        protected = (
+            split_text.replace("vs.", "vs<prd>")
+            .replace("e.g.", "e<prd>g<prd>")
+            .replace("i.e.", "i<prd>e<prd>")
+            .replace("etc.", "etc<prd>")
+        )
+        units = [part.strip().replace("<prd>", ".") for part in re.split(r"(?<=[.!?])\s+", protected)]
+        repaired_units: list[str] = []
+        for unit in units:
+            if not unit:
+                continue
+            repaired_unit, count = repair_unit(unit)
+            repaired_units.append(repaired_unit)
+            repaired += count
+        out.append(" ".join(repaired_units) if repaired_units else line)
+
     return normalize_citation_syntax("\n".join(out)), repaired
+

src/ragforge/eval_metrics.py

@@ -52,6 +52,93 @@ def answer_key_match(answer: str, case: dict[str, Any]) -> bool:
     return bool(expected_all or expected_any)
 
 
+
+def missing_answer_match(answer: str, case: dict[str, Any] | None = None) -> bool:
+    """Recognize a grounded "not present in the evidence" answer.
+
+    Missing-answer evaluation should reward calibrated uncertainty, not require a
+    single canned phrase. The matcher therefore accepts benchmark-specific cues
+    plus a conservative generic vocabulary for absence/insufficiency.
+    """
+    text = re.sub(r"\s+", " ", (answer or "").strip().casefold())
+    if not text:
+        return False
+    case = case or {}
+    expected = [str(x).casefold() for x in case.get("expected_missing_any", [])]
+    generic = [
+        "not specified", "does not specify", "doesn't specify",
+        "not provided", "does not provide", "doesn't provide",
+        "does not mention", "doesn't mention", "do not mention", "not mentioned",
+        "does not contain", "doesn't contain", "no information", "no fee information",
+        "insufficient to answer", "insufficient evidence", "cannot determine", "can't determine",
+        "not stated", "not available in", "not present in", "no evidence of",
+    ]
+    return any(cue in text for cue in [*expected, *generic] if cue)
+
+
+def substantive_claim_units(answer: str) -> list[str]:
+    """Extract Markdown-aware factual units for citation coverage.
+
+    Headings and generic list introductions are presentation structure, not
+    factual claims. Short numbered/bulleted values are factual units even when
+    they are much shorter than prose sentences.
+    """
+    units: list[str] = []
+    in_code = False
+    for raw_line in (answer or "").splitlines():
+        line = raw_line.strip()
+        if line.startswith("```"):
+            in_code = not in_code
+            continue
+        if in_code or not line:
+            continue
+        if re.match(r"^#{1,6}\s+", line):
+            continue
+
+        is_list = bool(re.match(r"^(?:[-*+]\s+|\d+[.)]\s+)", line))
+        content = re.sub(r"^(?:[-*+]\s+|\d+[.)]\s+)", "", line).strip()
+        plain = re.sub(r"\[(?:D|W|T)\d+(?:\s*,\s*(?:D|W|T)\d+)*\]", "", content)
+        plain = re.sub(r"[`*_#>]", "", plain).strip()
+        words = re.findall(r"[A-Za-z0-9][A-Za-z0-9_.%$+-]*", plain)
+
+        # Preambles such as "The following documents contain:" introduce the
+        # claims in following bullets and should not depress citation coverage.
+        if not is_list and plain.endswith(":"):
+            continue
+
+        if is_list:
+            if len(words) >= 1 and len(plain) >= 3:
+                units.append(content)
+            continue
+
+        # Split long prose lines into sentence-level claims. Citations are often
+        # written after punctuation (``claim. [D1]``); move that citation tail
+        # onto the claim before splitting. Protect common abbreviations such as
+        # ``vs.`` so they do not become fake uncited sentence fragments.
+        split_text = re.sub(
+            r"([.!?])\s+((?:\[(?:D|W|T)\d+(?:\s*,\s*(?:D|W|T)\d+)*\]\s*)+)",
+            r" \2\1 ",
+            content,
+        )
+        protected = (
+            split_text.replace("vs.", "vs<prd>")
+            .replace("e.g.", "e<prd>g<prd>")
+            .replace("i.e.", "i<prd>e<prd>")
+            .replace("etc.", "etc<prd>")
+        )
+        segments = [
+            seg.strip().replace("<prd>", ".")
+            for seg in re.split(r"(?<=[.!?])\s+", protected)
+            if seg.strip()
+        ]
+        for segment in segments:
+            segment_plain = re.sub(r"\[(?:D|W|T)\d+(?:\s*,\s*(?:D|W|T)\d+)*\]", "", segment)
+            segment_plain = re.sub(r"[`*_#>]", "", segment_plain).strip()
+            seg_words = re.findall(r"[A-Za-z0-9][A-Za-z0-9_.%$+-]*", segment_plain)
+            if len(segment_plain) >= 24 or len(seg_words) >= 5:
+                units.append(segment)
+    return units
+
 def scalar_value_match(observed: Any, expected: Any) -> bool:
     """Compare tabular scalar values without relying on Markdown rendering.
 
@@ -158,15 +245,13 @@ def citation_metrics(answer: str, result_sources: list[dict[str, Any]]) -> dict[
     valid = sum(1 for citation in cited if citation in valid_ids)
     validity = safe_div(valid, len(cited)) if cited else 0.0
 
-    units = [
-        unit.strip()
-        for unit in re.split(r"(?<=[.!?])\s+|\n+", answer or "")
-        if len(re.sub(r"[`*_#>-]", "", unit).strip()) >= 24
-    ]
+    units = substantive_claim_units(answer or "")
     cited_units = sum(1 for unit in units if extract_citation_ids(unit))
     coverage = safe_div(cited_units, len(units)) if units else 0.0
     return {
         "citation_count": len(cited),
         "citation_validity": validity,
         "citation_coverage": coverage,
+        "substantive_units": len(units),
+        "cited_units": cited_units,
     }

src/ragforge/evaluation.py

@@ -11,6 +11,7 @@ from .eval_metrics import (
     answer_key_match,
     citation_metrics,
     mean,
+    missing_answer_match,
     percentile,
     safe_div,
     scalar_value_match,
@@ -607,15 +608,14 @@ def _hard_mode_eval(
         returned = _document_sources(result.sources, 5)
         retrieval = source_metrics(returned, case.get("relevant_sources", [])) if case.get("relevant_sources") else {}
         if kind == "missing":
-            answer_l = result.answer.lower()
-            passed = any(term.lower() in answer_l for term in case.get("expected_missing_any", []))
+            passed = missing_answer_match(result.answer, case)
         elif kind == "insight":
             table_cited = any(str(src.get("id", "")).startswith("T") for src in result.sources) and "[T" in result.answer
             evidence = result.trace.get("evidence", {})
             passed = plan.get("task_type") == case.get("expected_task") and plan.get("retrieval_strategy") == case.get("expected_strategy") and float(evidence.get("source_coverage", 0.0) or 0.0) >= float(case.get("min_source_coverage", 0.0)) and (table_cited or not case.get("requires_table_citation"))
         else:
             passed = answer_key_match(result.answer, case) and float(retrieval.get("source_recall@5", 1.0)) >= 1.0
-        row.update({"route":plan.get("route"),"task":plan.get("task_type"),"strategy":plan.get("retrieval_strategy"),"answer_key_match":answer_key_match(result.answer,case) if kind=="qa" else None,"citation_validity":round(float(citations["citation_validity"]),3),"citation_coverage":round(float(citations["citation_coverage"]),3),"source_recall@5":round(float(retrieval.get("source_recall@5",1.0)),3),"latency_ms":round(max(0.0,wall-pace),1),"pass":passed,"gemini_calls":int(result.trace.get("metrics",{}).get("llm_calls_estimate",0) or 0),"_answer":result.answer,"_sources":result.sources,"_node_times":_trace_node_times(result.trace)})
+        row.update({"route":plan.get("route"),"task":plan.get("task_type"),"strategy":plan.get("retrieval_strategy"),"answer_key_match":answer_key_match(result.answer,case) if kind=="qa" else None,"missing_answer_match":missing_answer_match(result.answer, case) if kind=="missing" else None,"citation_validity":round(float(citations["citation_validity"]),3),"citation_coverage":round(float(citations["citation_coverage"]),3),"source_recall@5":round(float(retrieval.get("source_recall@5",1.0)),3),"latency_ms":round(max(0.0,wall-pace),1),"pass":passed,"gemini_calls":int(result.trace.get("metrics",{}).get("llm_calls_estimate",0) or 0),"_answer":result.answer,"_sources":result.sources,"_node_times":_trace_node_times(result.trace)})
         rows.append(row)
     return rows
 
@@ -651,12 +651,53 @@ def _profile_benchmark(
     return rows
 
 
+def _profile_summary(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    summaries: list[dict[str, Any]] = []
+    for profile in ("Fast", "Balanced", "Agentic"):
+        group = [row for row in rows if row.get("profile") == profile]
+        if not group:
+            continue
+        summaries.append({
+            "profile": profile,
+            "answer_accuracy": round(mean([float(bool(row.get("answer_key_match"))) for row in group]), 3),
+            "citation_validity": round(mean([float(row.get("citation_validity", 0.0) or 0.0) for row in group]), 3),
+            "citation_coverage": round(mean([float(row.get("citation_coverage", 0.0) or 0.0) for row in group]), 3),
+            "median_latency_ms": round(percentile([float(row.get("latency_ms", 0.0) or 0.0) for row in group], 0.5), 1),
+            "mean_llm_calls": round(mean([float(row.get("llm_calls_estimate", 0.0) or 0.0) for row in group]), 2),
+            "reranker_rate": round(mean([float(bool(row.get("reranker_used"))) for row in group]), 3),
+            "cases": len(group),
+        })
+    return summaries
+
+
+def _profile_recommendation(profile_summary: list[dict[str, Any]]) -> str:
+    if not profile_summary:
+        return ""
+    by_name = {row["profile"]: row for row in profile_summary}
+    fast = by_name.get("Fast")
+    balanced = by_name.get("Balanced")
+    agentic = by_name.get("Agentic")
+    if fast and balanced:
+        quality_close = (
+            float(fast.get("answer_accuracy", 0.0)) >= float(balanced.get("answer_accuracy", 0.0)) - 0.01
+            and float(fast.get("citation_coverage", 0.0)) >= float(balanced.get("citation_coverage", 0.0)) - 0.05
+        )
+        faster = float(fast.get("median_latency_ms", 0.0) or 0.0) < float(balanced.get("median_latency_ms", 0.0) or 0.0)
+        if quality_close and faster:
+            return "Fast matched Balanced quality on the sampled explicit-Documents cases with lower median latency. Keep Balanced as the general Auto default, but prefer Fast for simple local lookups."
+    if agentic and balanced and float(agentic.get("median_latency_ms", 0.0) or 0.0) > 2 * max(1.0, float(balanced.get("median_latency_ms", 0.0) or 0.0)):
+        return "Agentic was materially slower than Balanced on the sampled cases. Reserve Agentic for difficult or low-confidence work rather than routine lookups."
+    return "Profile differences were not large enough on this sample to justify changing the default execution policy."
+
+
 def _diagnostics(
     summary: dict[str, Any],
     ablation_rows: list[dict[str, Any]],
     planner_rows: list[dict[str, Any]],
     sql_rows: list[dict[str, Any]],
     hard_rows: list[dict[str, Any]] | None = None,
+    profile_summary: list[dict[str, Any]] | None = None,
+    node_latency_rows: list[dict[str, Any]] | None = None,
 ) -> list[dict[str, str]]:
     findings: list[dict[str, str]] = []
 
@@ -673,6 +714,22 @@ def _diagnostics(
             }
         )
 
+    if (
+        summary.get("source_recall@5", 0.0) >= 0.95
+        and summary.get("source_precision@5", 1.0) < 0.60
+    ):
+        findings.append(
+            {
+                "severity": "info",
+                "area": "context efficiency",
+                "finding": (
+                    f"Source Recall@5 is {float(summary.get('source_recall@5', 0.0)):.0%} while source Precision@5 is "
+                    f"{float(summary.get('source_precision@5', 0.0)):.0%}; the correct source is consistently present, but extra distractor sources are also entering the context."
+                ),
+                "recommendation": "Treat this as a context-budget signal. Test conservative focused-query pruning before reducing top-k globally, because overview and synthesis tasks still need breadth.",
+            }
+        )
+
     if summary.get("citation_coverage", 1.0) < 0.90:
         findings.append(
             {
@@ -731,6 +788,29 @@ def _diagnostics(
                     "recommendation": "Consider disabling reranking in Fast mode/small corpora, while retaining it for harder or larger corpora where chunk-level quality may improve.",
                 }
             )
+    profile_summary = profile_summary or []
+    recommendation = _profile_recommendation(profile_summary)
+    if recommendation:
+        findings.append({
+            "severity": "info",
+            "area": "profile policy",
+            "finding": recommendation,
+            "recommendation": "Use the profile benchmark as local evidence only; repeat it on larger user corpora before making a global policy claim.",
+        })
+
+    node_latency_rows = node_latency_rows or []
+    if node_latency_rows:
+        dominant = max(node_latency_rows, key=lambda row: float(row.get("mean_ms", 0.0) or 0.0))
+        total_mean = sum(float(row.get("mean_ms", 0.0) or 0.0) for row in node_latency_rows)
+        share = safe_div(float(dominant.get("mean_ms", 0.0) or 0.0), total_mean)
+        if share >= 0.60:
+            findings.append({
+                "severity": "info",
+                "area": "latency",
+                "finding": f"{dominant.get('node', 'generation')} dominates mean node time at approximately {share:.0%} of measured pipeline-node latency.",
+                "recommendation": "Prioritize model/generation efficiency before micro-optimizing millisecond-scale retrieval stages.",
+            })
+
     if not findings:
         findings.append(
             {
@@ -819,6 +899,8 @@ def _deep_from_standard_cache(
         report.get("semantic_planner", []),
         report.get("text2sql", []),
         report.get("hard_mode", []),
+        report.get("profile_summary", []),
+        report.get("node_latency", []),
     )
     report.setdefault("methodology", {})["evaluation_cache"] = (
         "Deep reused the current cached Standard deterministic baseline and issued only sampled judge calls."
@@ -994,10 +1076,15 @@ def run_demo_eval(
         **{key: round(value, 3) for key, value in judge_summary.items()},
     }
 
-    diagnostics = _diagnostics(summary, ablation_rows, planner_rows, sql_rows, hard_rows)
-    summary["evaluation_wall_ms"] = round((time.perf_counter() - wall_started) * 1000, 1)
-
     node_latency_rows = _node_latency_summary(qa_rows + overview_rows + hard_rows)
+    profile_summary_rows = _profile_summary(profile_rows)
+    profile_recommendation = _profile_recommendation(profile_summary_rows)
+    if profile_recommendation:
+        summary["profile_recommendation"] = profile_recommendation
+    diagnostics = _diagnostics(
+        summary, ablation_rows, planner_rows, sql_rows, hard_rows, profile_summary_rows, node_latency_rows
+    )
+    summary["evaluation_wall_ms"] = round((time.perf_counter() - wall_started) * 1000, 1)
     progress(1.0, "Evaluation complete")
     return {
         "summary": summary,
@@ -1010,6 +1097,7 @@ def run_demo_eval(
         "retrieval_ablation": ablation_rows,
         "hard_mode": hard_rows,
         "profile_benchmark": profile_rows,
+        "profile_summary": profile_summary_rows,
         "node_latency": node_latency_rows,
         "methodology": {
             "deterministic": (

src/ragforge/pipeline.py

@@ -45,6 +45,7 @@ class GraphState(TypedDict, total=False):
     abstain_reason: str
     evidence: EvidenceAssessment
     trace: dict[str, Any]
+    grounded_absence: bool
 
 
 class RAGEngine:
@@ -104,7 +105,11 @@ class RAGEngine:
         graph.add_edge("generate", "verify")
         graph.add_conditional_edges(
             "verify",
-            lambda s: "revise" if s.get("attempts", 0) < 1 and s.get("confidence", 1.0) < 0.58 else "end",
+            lambda s: "revise" if (
+                s.get("attempts", 0) < 1
+                and s.get("confidence", 1.0) < 0.58
+                and not s.get("grounded_absence", False)
+            ) else "end",
             {"revise": "revise", "end": END},
         )
         graph.add_edge("revise", "verify")
@@ -184,7 +189,7 @@ class RAGEngine:
                 "q": query,
                 "c": config.model_dump(),
                 "v": self.workspace.version,
-                "pipeline": 7,
+                "pipeline": 8,
             },
             sort_keys=True,
         )
@@ -209,13 +214,18 @@ class RAGEngine:
         chunk_count = len(self.workspace.chunks)
         if plan.retrieval_strategy == "global":
             return False, "global_source_profiles_already_balance_sources"
+        # The v1.6 source- and chunk-level ablation showed no ranking gain on
+        # the bundled five-source/90-chunk corpus while reranking added seconds.
+        # Treat the reranker as a large/hard-corpus capability, not a profile tax.
+        if chunk_count < 250 and source_count < 10:
+            return False, "small_corpus_source_and_chunk_benchmark_no_gain"
         if cfg.profile == "Agentic":
-            return True, "agentic_profile"
+            return True, "agentic_larger_corpus"
         if plan.task_type in {"comparison", "cross_document_synthesis", "insight_synthesis"}:
-            return True, "multi_source_reasoning"
+            return True, "multi_source_reasoning_larger_corpus"
         if chunk_count >= 250 or source_count >= 10:
             return True, "larger_corpus"
-        return False, "small_corpus_no_benchmark_gain"
+        return False, "adaptive_skip"
 
     @staticmethod
     def _normalize_citation_syntax(answer: str) -> str:
@@ -245,6 +255,7 @@ class RAGEngine:
             "reranker_used": any(bool(n.get("reranker_used", False)) for n in nodes),
             "citation_repairs": sum(int(n.get("citation_repairs", 0) or 0) for n in nodes),
             "table_evidence_used": any(int(n.get("table_sources", 0) or 0) > 0 for n in nodes),
+            "grounded_absence": any(bool(n.get("grounded_absence", False)) for n in nodes),
         }
 
     def _record(self, state: GraphState, node: str, started: float, **extra: Any) -> None:
@@ -553,6 +564,13 @@ class RAGEngine:
         elif strategy == "global":
             state["doc_hits"] = self.workspace.global_evidence(query, cfg.top_k, use_reranker)
             state["selected_sources"] = list(dict.fromkeys(h.chunk.source for h in state["doc_hits"]))
+            # Corpus overviews may summarize structured sources too. Surface the
+            # deterministic table evidence explicitly so [T#] citations are
+            # valid and numeric/table claims do not rely on manifest metadata.
+            if plan.task_type == "overview" and self.workspace.sql.tables:
+                structured_context, table_sources = self.workspace.sql.analytics_context(max_rows=12)
+                state["structured_context"] = structured_context
+                state["table_sources"] = table_sources
         elif strategy == "hierarchical":
             queries = list(state.get("document_queries") or [query])
             if state.get("hyde"):
@@ -988,6 +1006,8 @@ Requirements:
             evidence.score if evidence else None,
         )
         state["confidence"] = min(float(state.get("confidence", local)), local)
+        grounded_absence = self._looks_like_grounded_absence(answer)
+        state["grounded_absence"] = grounded_absence
         self_rag_used = False
         if cfg.profile == "Agentic" and cfg.use_self_rag and state.get("context"):
             try:
@@ -1003,6 +1023,7 @@ Requirements:
             t,
             confidence=round(state["confidence"], 3),
             self_rag=self_rag_used,
+            grounded_absence=grounded_absence,
             llm_calls=int(self_rag_used),
         )
         return state
@@ -1103,6 +1124,18 @@ Return only the revised answer."""
             )
         return sources
 
+    @staticmethod
+    def _looks_like_grounded_absence(answer: str) -> bool:
+        text = re.sub(r"\s+", " ", (answer or "").casefold())
+        cues = (
+            "not specified", "does not specify", "doesn't specify", "not provided",
+            "does not mention", "doesn't mention", "do not mention", "not mentioned",
+            "does not contain", "no information", "insufficient to answer",
+            "insufficient evidence", "cannot determine", "can't determine",
+            "not stated", "not present in",
+        )
+        return bool(re.search(r"\[(?:D|W|T)\d+\]", answer or "")) and any(cue in text for cue in cues)
+
     @staticmethod
     def _local_confidence(
         answer: str,
@@ -1116,7 +1149,10 @@ Return only the revised answer."""
         citation_count = len(re.findall(r"\[(?:D|W|T)\d+\]", answer))
         citation_factor = min(1.0, citation_count / max(1, min(3, sources)))
         evidence = evidence_score if evidence_score is not None else RAGEngine._evidence_strength(doc_hits, web_hits)
-        unsupported_language = 0.25 if "don't have enough" in answer.lower() or "insufficient" in answer.lower() else 0.0
+        cautious = "don't have enough" in answer.lower() or "insufficient" in answer.lower()
+        # Calibrated uncertainty is not a hallucination signal when the answer
+        # explicitly grounds the absence claim in retrieved evidence.
+        unsupported_language = 0.25 if cautious and not RAGEngine._looks_like_grounded_absence(answer) else 0.0
         return max(0.05, min(0.98, 0.30 + 0.35 * citation_factor + 0.35 * evidence - unsupported_language))
 
     @staticmethod

src/ragforge/ui.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import html
 import re
 from pathlib import Path
 from typing import Any
@@ -24,6 +25,16 @@ CSS = """
 .muted {opacity: .75;}
 .status-ready {padding: 8px 10px; border-radius: 8px;}
 .status-line {padding: 8px 10px; border: 1px solid rgba(128,128,128,.25); border-radius: 8px; margin: 6px 0;}
+#source-panel .source-card {padding: 4px 0;}
+#source-panel .source-title {font-size: 1rem; font-weight: 650; line-height: 1.35;}
+#source-panel .source-meta {font-size: .88rem; opacity: .78; line-height: 1.4; margin-top: 2px;}
+#source-panel .source-snippet {font-size: .95rem; line-height: 1.5; margin-top: 8px; white-space: normal;}
+.latency-waterfall {display: grid; gap: 7px; margin-top: 8px;}
+.latency-row {display: grid; grid-template-columns: minmax(72px, 110px) 1fr minmax(70px, 90px); gap: 8px; align-items: center;}
+.latency-label {font-size: .88rem; font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;}
+.latency-track {height: 9px; border-radius: 999px; background: rgba(128,128,128,.18); overflow: hidden;}
+.latency-fill {height: 100%; min-width: 2px; border-radius: 999px; background: var(--primary-500, currentColor);}
+.latency-time {font-size: .84rem; text-align: right; opacity: .8;}
 """
 
 
@@ -39,7 +50,7 @@ def _ensure_session(session_id: str | None) -> tuple[str, Any]:
 def _ui_text(text: str) -> str:
     # Keep UI punctuation visually compact even when model/source text contains
     # typographic dash glyphs. The underlying retrieved evidence is unchanged.
-    return (text or "").replace("\u2014", "-").replace("\u2013", "-")
+    return (text or "").replace("\u2014", " - ").replace("\u2013", " - ")
 
 
 def _truncate_preview(text: str, limit: int = 420) -> str:
@@ -64,48 +75,67 @@ def _corpus_markdown(summary, prefix: str | None = None) -> str:
 
 
 def _sources_markdown(sources: list[dict[str, Any]]) -> str:
+    """Render source cards with plain-text snippets and uniform typography.
+
+    Retrieved Markdown headings such as ``# Acme Cloud`` must never become UI
+    headings inside the source panel. All source-controlled text is HTML-escaped
+    before rendering, while the surrounding card markup owns the typography.
+    """
     if not sources:
         return "*No sources returned.*"
     blocks: list[str] = []
-    for s in sources:
-        sid = s.get("id", "?")
-        title = s.get("title", "Source")
-        if s.get("type") == "web" and s.get("url"):
-            preview = _truncate_preview(s.get("snippet", ""))
-            body = f"**[{sid}] {title}**  \n{s['url']}  \nRetrieval rank: `#{s.get('rank', '-')}`"
-            if preview:
-                body += f"  \n\n{preview}"
-            blocks.append(body)
-        elif s.get("type") in {"sql", "table"}:
-            preview = _truncate_preview(s.get("snippet", ""))
-            body = f"**[{sid}] {title}**  \nRows: `{s.get('rows', '-')}`"
-            if s.get("schema"):
-                body += f"  \nSchema: `{_ui_text(str(s.get('schema')))}`"
-            if preview:
-                body += f"  \n\n{preview}"
-            blocks.append(body)
+    for source in sources:
+        sid = html.escape(str(source.get("id", "?")))
+        title = html.escape(_ui_text(str(source.get("title", "Source"))))
+        preview = html.escape(_truncate_preview(str(source.get("snippet", ""))))
+        source_type = str(source.get("type", "document"))
+        page = source.get("page")
+        page_text = f" - page {html.escape(str(page))}" if page else ""
+
+        meta: list[str] = []
+        if source_type == "web" and source.get("url"):
+            url = html.escape(str(source.get("url")), quote=True)
+            meta.append(f'<a href="{url}" target="_blank" rel="noopener noreferrer">Open web source</a>')
+            meta.append(f"Retrieval rank: #{html.escape(str(source.get('rank', '-')))}")
+        elif source_type in {"sql", "table"}:
+            meta.append(f"Rows: {html.escape(str(source.get('rows', '-')))}")
+            if source.get("schema"):
+                meta.append("Schema: " + html.escape(_ui_text(str(source.get("schema")))) )
         else:
-            page = f" - page {s['page']}" if s.get("page") else ""
-            preview = _truncate_preview(s.get("snippet", ""))
-            blocks.append(
-                f"**[{sid}] {title}{page}**  \n"
-                f"Retrieval rank: `#{s.get('rank', '-')}` - hybrid signal: `{s.get('retrieval_signal', '-')}`  \n\n"
-                f"{preview}"
-            )
-    return "\n\n---\n\n".join(blocks)
+            meta.append(f"Retrieval rank: #{html.escape(str(source.get('rank', '-')))}")
+            meta.append(f"hybrid signal: {html.escape(str(source.get('retrieval_signal', '-')))}")
+
+        blocks.append(
+            '<div class="source-card">'
+            f'<div class="source-title">[{sid}] {title}{page_text}</div>'
+            f'<div class="source-meta">{" - ".join(meta)}</div>'
+            + (f'<div class="source-snippet">{preview}</div>' if preview else '')
+            + '</div>'
+        )
+    return '<hr>'.join(blocks)
 
 
 def _latency_waterfall(trace: dict[str, Any]) -> str:
-    nodes = [n for n in trace.get("nodes", []) if float(n.get("ms", 0.0) or 0.0) > 0]
+    """Render a compact proportional latency bar instead of ASCII hashes."""
+    nodes = [n for n in trace.get("nodes", []) if float(n.get("ms", 0.0) or 0.0) >= 0.0]
     if not nodes:
         return "*No node timings available.*"
     max_ms = max(float(n.get("ms", 0.0) or 0.0) for n in nodes) or 1.0
-    lines = []
+    rows: list[str] = []
     for node in nodes:
         ms = float(node.get("ms", 0.0) or 0.0)
-        width = max(1, min(18, int(round(18 * ms / max_ms))))
-        lines.append(f"`{str(node.get('node','-')):<9}` {'#' * width} {ms:.0f} ms")
-    return "  \n".join(lines)
+        width = 0.0 if ms <= 0 else max(1.5, min(100.0, 100.0 * ms / max_ms))
+        name = html.escape(str(node.get("node", "-")))
+        rows.append(
+            '<div class="latency-row">'
+            f'<div class="latency-label">{name}</div>'
+            '<div class="latency-track">'
+            f'<div class="latency-fill" style="width:{width:.1f}%"></div>'
+            '</div>'
+            f'<div class="latency-time">{ms:.0f} ms</div>'
+            '</div>'
+        )
+    return '<div class="latency-waterfall">' + ''.join(rows) + '</div>'
 
 
 def _inspector_markdown(trace: dict[str, Any]) -> str:
@@ -116,11 +146,13 @@ def _inspector_markdown(trace: dict[str, Any]) -> str:
     evidence = trace.get("evidence", {})
     nodes = [n.get("node") for n in trace.get("nodes", []) if n.get("node")]
     metrics = trace.get("metrics", {})
+    retrieve_node = next((node for node in trace.get("nodes", []) if node.get("node") == "retrieve"), {})
     coverage = float(evidence.get("source_coverage", 0.0) or 0.0)
+    table_count = int(workspace.get("tables", 0) or 0)
     return (
         "**Workspace**  \n"
         f"{workspace.get('sources', 0)} sources - {workspace.get('chunks', 0)} chunks - "
-        f"{workspace.get('tables', 0)} tables - version {workspace.get('version', 0)}\n\n"
+        f"{table_count} {'table' if table_count == 1 else 'tables'} - version {workspace.get('version', 0)}\n\n"
         "**Semantic plan**  \n"
         f"Route: `{plan.get('route', '-')}` - scope: `{plan.get('knowledge_scope', '-')}` - "
         f"task: `{plan.get('task_type', '-')}` - strategy: `{plan.get('retrieval_strategy', '-')}` - "
@@ -134,7 +166,9 @@ def _inspector_markdown(trace: dict[str, Any]) -> str:
         f"web used: `{bool(metrics.get('web_used', False))}` - "
         f"correction used: `{bool(metrics.get('correction_used', False))}` - "
         f"reranker used: `{bool(metrics.get('reranker_used', False))}` - "
-        f"citation repairs: `{int(metrics.get('citation_repairs', 0) or 0)}`\n\n"
+        f"citation repairs: `{int(metrics.get('citation_repairs', 0) or 0)}` - "
+        f"grounded absence: `{bool(metrics.get('grounded_absence', False))}`  \n"
+        f"Reranker decision: `{retrieve_node.get('reranker_reason', '-')}`\n\n"
         f"**Execution path**  \n`{' -> '.join(nodes) if nodes else '-'}`\n\n"
         "**Node latency waterfall**  \n" + _latency_waterfall(trace)
     )
@@ -149,6 +183,7 @@ def _eval_summary_markdown(report: dict[str, Any]) -> str:
         "",
         f"**Deterministic quality:** `{float(summary.get('deterministic_quality_score', 0.0)):.3f}` - "
         f"**answer accuracy:** `{float(summary.get('answer_accuracy', 0.0)):.0%}` - "
+        f"**source Precision@5:** `{float(summary.get('source_precision@5', 0.0)):.0%}` - "
         f"**source Recall@5:** `{float(summary.get('source_recall@5', 0.0)):.0%}` - "
         f"**Hit@1:** `{float(summary.get('source_hit@1', 0.0)):.0%}` - "
         f"**MRR:** `{float(summary.get('source_mrr', 0.0)):.3f}` - "
@@ -201,6 +236,8 @@ def _eval_summary_markdown(report: dict[str, Any]) -> str:
             "",
             f"**Profile benchmark:** `{int(summary.get('profile_benchmark_cases', 0) or 0)}` profile/case runs included.",
         ])
+        if summary.get("profile_recommendation"):
+            lines.extend(["", f"**Profile policy:** {summary.get('profile_recommendation')}"])
     if summary.get("reused_standard_baseline"):
         lines += [
             "",
@@ -218,8 +255,8 @@ def _eval_diagnostics_markdown(report: dict[str, Any]) -> str:
     for row in diagnostics:
         severity = str(row.get("severity", "info")).upper()
         lines.append(
-            f"- **{severity} - {row.get('area', 'benchmark')}:** {row.get('finding', '')} "
-            f"**Next:** {row.get('recommendation', '')}"
+            f"- **{severity} - {row.get('area', 'benchmark')}:** {row.get('finding', '')}  \n"
+            f"  **Next:** {row.get('recommendation', '')}"
         )
     return "\n".join(lines)
 
@@ -270,7 +307,7 @@ def _architecture_snapshot(session_id: str | None) -> tuple[str, str, str, dict[
     settings = get_settings()
     stats = ws.stats()
     runtime_json = {
-        "ragforge_version": "1.6.0",
+        "ragforge_version": "1.7.0",
         "workspace": stats,
         "models": {
             "generation": settings.default_model,
@@ -291,7 +328,7 @@ def _architecture_snapshot(session_id: str | None) -> tuple[str, str, str, dict[
     }
     runtime = (
         "### Live runtime\n"
-        f"**RAGForge:** `v1.6.0` - **workspace:** `{sid[:12]}...` - **status:** `{stats['status']}`\n\n"
+        f"**RAGForge:** `v1.7.0` - **workspace:** `{sid[:12]}...` - **status:** `{stats['status']}`\n\n"
         f"**Corpus:** `{stats['sources']}` sources - `{stats['chunks']}` chunks - "
         f"`{stats['source_profiles']}` source profiles - `{stats['tables']}` tables - "
         f"corpus version `{stats['version']}`\n\n"
@@ -343,6 +380,7 @@ EVAL_TABLE_KEYS = {
     "Retrieval ablation": "retrieval_ablation",
     "Hard mode": "hard_mode",
     "Profile benchmark": "profile_benchmark",
+    "Profile summary": "profile_summary",
     "Node latency": "node_latency",
     "Abstention": "abstention",
     "Compare saved runs": "__compare__",
@@ -392,6 +430,7 @@ def _eval_comparison_frame(ws) -> pd.DataFrame:
                 "pacing_wait_s": round(float(summary.get("pacing_sleep_ms", 0.0) or 0.0) / 1000, 1),
                 "deep_judge_overall": summary.get("judge_overall", ""),
                 "current_corpus": item.get("current_corpus", False),
+                "run_id": item.get("run_id", ""),
                 "saved_at": item.get("saved_at", ""),
             }
         )
@@ -405,10 +444,19 @@ def _saved_eval_status(ws) -> str:
     parts = []
     for item in inventory:
         stale = "" if item.get("current_corpus") else " (stale corpus)"
-        parts.append(f"`{item['level']}` grade {item.get('grade', '-')}{stale}")
+        run_id = item.get("run_id") or "legacy"
+        parts.append(f"`{item['level']}` grade {item.get('grade', '-')} - run `{run_id}`{stale}")
     return "**Saved runs:** " + " - ".join(parts)
 
 
+def _saved_eval_message(level: str, report: dict[str, Any], *, stale: bool = False) -> str:
+    meta = report.get("evaluation_cache", {}) if report else {}
+    run_id = meta.get("run_id") or "legacy"
+    saved_at = meta.get("saved_at") or "unknown time"
+    suffix = "This result belongs to an older corpus version." if stale else "0 new Gemini requests were used."
+    return f"**Loaded saved {level} run `{run_id}` from {saved_at}.** {suffix}"
+
+
 def build_ui() -> gr.Blocks:
     settings = get_settings()
     with gr.Blocks(css=CSS, title="RAGForge") as demo:
@@ -480,7 +528,7 @@ def build_ui() -> gr.Blocks:
                         ask_btn = gr.Button("Ask", variant="primary")
                         query_status = gr.Markdown("Ready.", elem_classes=["status-line"])
                         with gr.Accordion("Sources", open=True):
-                            source_view = gr.Markdown("*Sources appear here.*")
+                            source_view = gr.Markdown("*Sources appear here.*", elem_id="source-panel")
                         with gr.Accordion("Pipeline inspector", open=False):
                             inspector_summary = gr.Markdown(
                                 "*Run a query to inspect routing, retrieval and evidence decisions.*"
@@ -895,10 +943,7 @@ def build_ui() -> gr.Blocks:
                         )
                     meta = report.get("evaluation_cache", {})
                     stale = int(meta.get("workspace_version", -1)) != int(ws.version)
-                    status = (
-                        f"**Loaded saved {level} evaluation.** "
-                        + ("This result belongs to an older corpus version." if stale else "No Gemini requests were used.")
-                    )
+                    status = _saved_eval_message(level, report, stale=stale)
                     outputs = _evaluation_outputs(ws, report, status)
                     return (sid, outputs[-1], *outputs[:-1])
 
@@ -921,7 +966,7 @@ def build_ui() -> gr.Blocks:
                             outputs = _evaluation_outputs(
                                 ws,
                                 cached,
-                                f"**Loaded saved {level} evaluation.** No Gemini requests were used.",
+                                _saved_eval_message(level, cached, stale=False),
                             )
                             return (
                                 sid,
@@ -964,10 +1009,15 @@ def build_ui() -> gr.Blocks:
                             if report.get("summary", {}).get("reused_standard_baseline")
                             else ""
                         )
+                        meta = report.get("evaluation_cache", {})
+                        run_id = meta.get("run_id") or "unknown"
+                        requests = int(report.get("summary", {}).get("gemini_requests", 0) or 0)
                         outputs = _evaluation_outputs(
                             ws,
                             report,
-                            f"**Evaluation complete.** {level} benchmark finished.{skipped_note}{incremental_note}",
+                            f"**Fresh {level} evaluation complete - run `{run_id}`.** "
+                            f"This execution issued {requests} Gemini request(s) and was saved for reuse."
+                            f"{skipped_note}{incremental_note}",
                         )
                         return (
                             sid,
@@ -1057,7 +1107,7 @@ def build_ui() -> gr.Blocks:
                     show_progress="hidden",
                 )
 
-                eval_saved_level.change(
+                eval_saved_level.input(
                     load_saved_eval,
                     [session_state, eval_saved_level],
                     [

src/ragforge/workspace.py

@@ -21,6 +21,9 @@ from .schemas import CorpusSummary, Document, SearchHit, SourceProfile
 from .sql_agent import SQLWorkspace
 
 
+SERVER_BOOT_ID = uuid.uuid4().hex[:12]
+
+
 class Workspace:
     def __init__(self, session_id: str):
         self.session_id = session_id
@@ -111,6 +114,7 @@ class Workspace:
             "table_names": list(self.sql.tables),
             "saved_evaluations": sorted(self.evaluation_reports),
             "status": "empty" if self.is_empty else "ready",
+            "server_boot_id": SERVER_BOOT_ID,
         }
 
     @property
@@ -148,6 +152,8 @@ class Workspace:
                 "benchmark_version": benchmark_version,
                 "workspace_version": self.version,
                 "saved_at": datetime.now(timezone.utc).isoformat(),
+                "run_id": uuid.uuid4().hex[:12],
+                "server_boot_id": SERVER_BOOT_ID,
             }
             self.evaluation_reports[level] = saved
             try:
@@ -211,6 +217,9 @@ class Workspace:
                     "workspace_version": meta.get("workspace_version"),
                     "current_corpus": int(meta.get("workspace_version", -1)) == int(self.version),
                     "saved_at": meta.get("saved_at", ""),
+                    "run_id": meta.get("run_id", ""),
+                    "server_boot_id": meta.get("server_boot_id", ""),
+                    "current_server": meta.get("server_boot_id") in {None, "", SERVER_BOOT_ID},
                 }
             )
         return rows
@@ -240,6 +249,8 @@ class Workspace:
                     "hard_mode_pass": summary.get("hard_mode_pass_rate"),
                     "p50_ms": summary.get("latency_p50_ms"),
                     "gemini_requests": summary.get("gemini_requests"),
+                    "run_id": meta.get("run_id", ""),
+                    "server_boot_id": meta.get("server_boot_id", ""),
                 }
                 prev = previous_by_level.get(level)
                 if prev:

tests/test_citations.py

@@ -32,3 +32,22 @@ def test_repair_adds_supported_citation_to_factual_bullet():
     repaired, count = repair_missing_citations(answer, sources)
     assert "[D1]" in repaired
     assert count == 1
+
+
+def test_repair_missing_citations_repairs_uncited_sentence_inside_cited_paragraph():
+    from ragforge.citations import repair_missing_citations
+
+    answer = (
+        "Support tiers [T1]: Starter costs 0. "
+        "Intermediate tiers include Team at 49 and Business at 199. "
+        "Enterprise costs 799 [T1]."
+    )
+    sources = [{
+        "id": "T1",
+        "type": "table",
+        "title": "support_matrix",
+        "snippet": "Starter 0 Team 49 Business 199 Enterprise 799",
+    }]
+    repaired, count = repair_missing_citations(answer, sources, semantic_support=False)
+    assert "Intermediate tiers include Team at 49 and Business at 199 [T1]." in repaired
+    assert count >= 1

tests/test_evaluation_assets.py

@@ -5,7 +5,7 @@ from pathlib import Path
 def test_demo_benchmark_is_multilayer_and_auditable():
     path = Path("evals/demo_benchmark.json")
     data = json.loads(path.read_text(encoding="utf-8"))
-    assert data["version"] == "1.6"
+    assert data["version"] == "1.7"
     assert len(data["qa_cases"]) >= 9
     assert len(data["planner_cases"]) >= 10
     assert len(data["overview_cases"]) >= 2
@@ -49,7 +49,7 @@ def test_demo_evaluation_and_introspection_are_available_through_api():
     assert "/api/v1/session/{session_id}" in text
     assert "/api/v1/evaluation/saved/{session_id}" in text
     assert "/api/v1/evaluation/saved/{session_id}/{level}" in text
-    assert 'version="1.6.0"' in text
+    assert 'version="1.7.0"' in text
 
 
 def test_v15_evaluation_cache_and_incremental_deep_are_present():
@@ -99,7 +99,7 @@ def test_v15_text2sql_cases_have_typed_expected_values():
 def test_v15_pipeline_contains_adaptive_reranking_and_citation_repair():
     text = Path("src/ragforge/pipeline.py").read_text(encoding="utf-8")
     assert "_reranker_decision" in text
-    assert "small_corpus_no_benchmark_gain" in text
+    assert "small_corpus_source_and_chunk_benchmark_no_gain" in text
     assert "_repair_missing_citations" in text
     assert "citation_repairs" in text
 

tests/test_ui_copy.py

@@ -54,7 +54,7 @@ def test_ui_has_quota_safe_evaluation_controls_and_score_card_spacing():
 
 def test_architecture_snapshot_returns_complete_runtime_payload():
     text = Path("src/ragforge/ui.py").read_text(encoding="utf-8")
-    assert '"ragforge_version": "1.6.0"' in text
+    assert '"ragforge_version": "1.7.0"' in text
     assert "return sid, runtime, curl, runtime_json" in text
     assert "curl = f\ndef _eval_frame" not in text
 
@@ -63,7 +63,7 @@ def test_ui_can_switch_saved_evaluations_without_rerunning():
     text = Path("src/ragforge/ui.py").read_text(encoding="utf-8")
     assert "View saved evaluation" in text
     assert "Refresh saved runs" in text
-    assert "No Gemini requests were used" in text
+    assert "0 new Gemini requests were used" in text
 
 
 def test_ui_exposes_copyable_evaluation_exports():

tests/test_v16_features.py

@@ -4,7 +4,7 @@ from pathlib import Path
 
 def test_v16_benchmark_has_hard_mode_and_insight_plans():
     data = json.loads(Path("evals/demo_benchmark.json").read_text(encoding="utf-8"))
-    assert data["version"] == "1.6"
+    assert data["version"] == "1.7"
     assert len(data.get("hard_mode_cases", [])) >= 8
     insight = [c for c in data["planner_cases"] if c.get("task") == "insight_synthesis"]
     assert insight

tests/test_v17_features.py

@@ -0,0 +1,86 @@
+from pathlib import Path
+
+from ragforge.eval_metrics import citation_metrics, missing_answer_match
+
+
+def test_v17_markdown_aware_citation_coverage_counts_short_numbered_items():
+    answer = (
+        "Based on the provided documents, the four core functions are:\n\n"
+        "1. **GOVERN** [D1] [D2]\n"
+        "2. **MAP** [D1] [D2]\n"
+        "3. **MEASURE** [D1] [D2]\n"
+        "4. **MANAGE** [D1] [D2]"
+    )
+    metrics = citation_metrics(answer, [{"id": "D1"}, {"id": "D2"}])
+    assert metrics["citation_validity"] == 1.0
+    assert metrics["citation_coverage"] == 1.0
+    assert metrics["substantive_units"] == 4
+
+
+def test_v17_citation_coverage_ignores_generic_list_preamble():
+    answer = (
+        "Based on the demo documents, escalation guidance is contained in:\n\n"
+        "* **acme_cloud_runbook.md:** Incident escalation guidance [D1].\n"
+        "* **orbitpay_policy.txt:** Security escalation guidance [D2]."
+    )
+    metrics = citation_metrics(answer, [{"id": "D1"}, {"id": "D2"}])
+    assert metrics["citation_coverage"] == 1.0
+
+
+def test_v17_grounded_missing_answer_matches_natural_absence_language():
+    answer = (
+        "Based on the provided evidence, the retrieved documents do not mention any fee "
+        "charged by OrbitPay for opening a card dispute [D1]. Therefore, the supplied "
+        "context is insufficient to answer the question."
+    )
+    case = {"expected_missing_any": ["not specified", "does not contain"]}
+    assert missing_answer_match(answer, case)
+
+
+def test_v17_ui_uses_plain_source_cards_and_graphical_latency_bars():
+    text = Path("src/ragforge/ui.py").read_text(encoding="utf-8")
+    assert 'elem_id="source-panel"' in text
+    assert "html.escape(_truncate_preview" in text
+    assert "latency-track" in text
+    assert "latency-fill" in text
+    assert "'#' * width" not in text
+
+
+def test_v17_saved_run_switching_is_user_input_only_and_has_provenance():
+    ui = Path("src/ragforge/ui.py").read_text(encoding="utf-8")
+    workspace = Path("src/ragforge/workspace.py").read_text(encoding="utf-8")
+    assert "eval_saved_level.input(" in ui
+    assert "eval_saved_level.change(" not in ui
+    assert "Fresh {level} evaluation complete" in ui
+    assert "run_id" in workspace
+    assert "SERVER_BOOT_ID" in workspace
+
+
+def test_v17_global_overviews_surface_table_evidence():
+    text = Path("src/ragforge/pipeline.py").read_text(encoding="utf-8")
+    assert 'plan.task_type == "overview" and self.workspace.sql.tables' in text
+    assert "analytics_context(max_rows=12)" in text
+
+
+def test_v17_grounded_absence_skips_revise_path():
+    text = Path("src/ragforge/pipeline.py").read_text(encoding="utf-8")
+    assert "grounded_absence" in text
+    assert "and not s.get(\"grounded_absence\", False)" in text
+
+
+def test_v17_diagnostics_put_next_on_following_line():
+    text = Path("src/ragforge/ui.py").read_text(encoding="utf-8")
+    assert '  **Next:**' in text
+
+
+def test_v17_citation_coverage_attaches_citation_after_sentence_punctuation():
+    answer = "The corpus does not provide complete financial audits. [D1]"
+    metrics = citation_metrics(answer, [{"id": "D1"}])
+    assert metrics["citation_coverage"] == 1.0
+
+
+def test_v17_citation_coverage_does_not_split_vs_abbreviation():
+    answer = "Operational strictness vs. flexibility is a notable contrast [D1]."
+    metrics = citation_metrics(answer, [{"id": "D1"}])
+    assert metrics["citation_coverage"] == 1.0
+    assert metrics["substantive_units"] == 1