docs/FEATURE_MATRIX.md

# Feature matrix - v2.0 final

The table below describes the final planned portfolio release. Future changes should be driven by real deployment/corpus requirements rather than feature-count growth.

| Capability | RAGForge implementation | Why it matters |
|---|---|---|
| Semantic query planning | schema-constrained Gemini `QueryPlan` in Balanced/Agentic | resolves intent before retrieval instead of relying on freshness/SQL keyword regexes |
| Knowledge-scope classification | corpus / external / mixed / structured-data | prevents session-local phrases such as “current corpus” from being mistaken for web freshness |
| Task classification | fact / overview / cross-doc / comparison / aggregation / insight synthesis / follow-up | lets evidence requirements depend on the task and distinguishes broad description from analytical pattern-finding |
| Independent retrieval queries | separate document and web queries | search engines and private corpora often need different formulations |
| Corpus manifest | source metadata + deterministic representative excerpts | gives the planner grounded knowledge of what is actually indexed |
| Source-profile index | one retrieval-only profile per source | enables document-level routing before chunk-level retrieval |
| Hierarchical retrieval | source profile → selected source(s) → source-scoped chunk retrieval | improves broad/source-specific questions and avoids unrelated chunk competition |
| Global retrieval | source-balanced evidence selection | corpus overviews represent distinct files instead of the longest document |
| Analytical retrieval | source-balanced document evidence + deterministic DuckDB table context | supports grounded trend/insight synthesis across mixed local evidence without extra table-summary LLM calls |
| Dense retrieval | FastEmbed BGE-small + Qdrant local | semantic recall without hosted embedding cost |
| Source-scoped dense retrieval | cached normalized embedding matrix | efficient hierarchical search inside selected sources without rebuilding stores |
| Sparse retrieval | BM25 | exact terms, identifiers, error codes and names |
| Hybrid fusion | Reciprocal Rank Fusion | robust combination of lexical + semantic rankings |
| Reranking | local MiniLM cross-encoder behind an adaptive profile/task/corpus policy | retains second-stage precision capability while avoiding measured multi-second overhead on easy/small-corpus paths |
| Relevance display | rank + bounded hybrid retrieval signal | avoids presenting uncalibrated cross-encoder logits as probabilities |
| Chunking | sentence-aware overlap + optional semantic breakpoints | balances context continuity and retrieval granularity |
| Multi-query | planner-generated variants in Balanced/Agentic, bounded by profile | improves recall across alternate wording |
| HyDE | optional Agentic hypothetical passage used only for retrieval | bridges abstract questions to document language |
| Corrective RAG | retrieve → task-aware grade → semantic correction → retrieve again | fixes weak retrieval before reaching for external data |
| Conditional web fallback | web requires both permission and semantic relevance | prevents irrelevant internet pollution of private/session-local questions |
| Evidence grading | relevance + method agreement + task-aware source coverage + optional LLM judge | a global overview and a focused fact lookup should not share one confidence rule |
| Abstention | corpus-only tasks can answer “insufficient evidence” after correction | weak retrieval does not automatically imply the internet is relevant |
| Self-RAG | faithfulness audit + bounded revision | reflection without unbounded recursion |
| Ask-the-Web | independent query fan-out, parallel search/fetch, extraction, rerank, synthesis | fresh-information path separated from private retrieval |
| Native search | Gemini grounding submodel | provider-native search option while keeping main RAG model independent |
| Routing controls | Auto/Documents/Web/Hybrid/Data(SQL) | semantic defaults plus explicit user override |
| Text2SQL | isolated DuckDB + validated read-only SQL | structured questions are computed rather than guessed from text chunks |
| OCR | optional Gemini file transcription | scanned documents/images remain usable |
| Multiformat ingestion | PDF/TXT/MD/DOCX/PPTX/CSV/XLSX/JSON/HTML/code/images/ZIP | realistic enterprise ingestion surface |
| ZIP hardening | traversal/file-count/uncompressed-size/type limits | archive UX without naive extraction risk |
| Citations | `[D#]` documents, `[T#]` tables, `[W#]` web + conservative zero-call lexical/semantic repair | auditable answer grounding across text and structured evidence with no extra citation-fixer request |
| Prompt-injection defense | untrusted-context rules + heuristic scoring/downranking | retrieval is an attack surface |
| SSRF defense | public URL/DNS checks + redirects disabled | web agents must not become internal-network fetchers |
| Session isolation | per-session corpus/index/DuckDB/history/cache version | prevents accidental cross-user context |
| Caching | TTL result cache keyed by session/config/corpus version | latency/quota reduction without stale cross-corpus answers |
| Rate limiting | sliding-window per IP | protects a public shared model key |
| Observability | Prometheus + semantic plan/evidence/correction/node trace + node time/estimated LLM calls/web/correction/reranker/citation-repair flags | makes agent decisions and efficiency inspectable |
| Adaptive retrieval depth | corpus-scale-aware 6/8/10/12 candidate depth | avoids assuming the 90-chunk demo top-k is sufficient for larger uploads |
| Adaptive context budget | focused 2-5 chunk budget from scale, score gap, confidence and source ambiguity | shrinks generation context without collapsing broad or ambiguous evidence needs |
| Focused evidence compression | deterministic query-relevant sentence selection for generator context only | reduces prompt load without mutating source cards or spending another LLM call |
| Scale stress | 1x/5x/20x long-document distractor cloning with precomputed vector reuse | tests retrieval/budget robustness around ~1000 chunks without Gemini or re-embedding cost |
| Workspace diagnostics | corpus scale/capacity, index readiness, TTL/idle, estimated vector memory | makes demo operational limits explicit before they become silent failures |
| Acceptance checks | transparent threshold checklist over quality, safety, efficiency and scale robustness | summarizes ship/no-ship evidence without hiding component metrics |
| Evaluation | transparent v1.9 benchmark + Markdown-aware citation claims + grounded-absence robustness + source/chunk ranking metrics + hard-mode + adaptive-context/compression/scale ablations + planner/web policy + typed Text2SQL + pacing-corrected node latency + calibrated Deep judge | separates retrieval, orchestration, context economics, scale robustness, analysis and generation failures |
| Saved evaluation history | latest Quick/Standard/Deep cache plus timestamped archived runs and deltas | allows instant run switching and lightweight within-workspace regression tracking without consuming Gemini quota again |
| Evaluation report serialization | JSON-safe normalization at save/load/UI/API boundaries + formatted raw JSON view | prevents `root={...}` wrapper leakage and keeps fresh/restored reports identical |
| Evaluation table export | CSV/TSV/Markdown export for every benchmark table | makes Text2SQL/planner/QA/overview/ablation/abstention/comparison results easy to copy or download |
| Incremental Deep evaluation | compatible Standard deterministic baseline + judge-only Deep delta | cuts repeated Standard -> Deep Gemini usage from a full benchmark rerun to the representative judge sample |
| Adaptive reranking | runtime skips cross-encoder for Fast/small/easy cases while retaining explicit ablation and harder-query support | converts measured latency-vs-quality evidence into a runtime optimization without deleting the reranker capability |
| Adaptive focused context pruning | focused semantic/hierarchical fact lookups keep a three-chunk safety floor; broad tasks bypass pruning | improves prompt efficiency while protecting overview, insight and multi-source breadth |
| Context-budget ablation | full top-k vs focused pruning with Precision/Recall/Hit/MRR and context size/token metrics | proves that pruning preserves retrieval quality before treating token reduction as a win |
| Prompt-budget observability | traces expose manifest inclusion, evidence-context chars and estimated generation-prompt tokens | connects latency/cost changes to actual model-input changes |
| Citation repair | conservative lexical evidence matching for uncited factual units, preamble-skip policy and grouped-citation normalization, zero extra LLM calls | improves citation completeness while reducing incorrect or redundant automatically attached citations |
| Evaluation quota control | shared rolling Gemini request ledger, configurable target RPM, provider retry-delay handling and request/pacing telemetry | prevents Standard/Deep benchmark bursts from repeatedly exhausting low free-tier RPM quotas |
| Quality gates | subsystem thresholds cap the letter grade | prevents a strong weighted average from hiding weak Text2SQL/routing/citation behavior |
| Evaluation diagnostics | structured warnings/recommendations for citations, planner taxonomy, Text2SQL and reranker tradeoffs | turns benchmark output into actionable engineering feedback |
| API | FastAPI session/status/ingest/query/evaluate/benchmark-info/saved-evaluations/health/metrics + Swagger/OpenAPI | usable beyond the UI and introspectable from the Architecture + API tab |
| Query/evaluation run state | dedicated status lines + disabled buttons while work is active | prevents silent waits and accidental duplicate submissions without reintroducing overlapping progress overlays |
| Browser session continuity | `gr.BrowserState` stores only the opaque workspace ID | refreshes can reconnect without storing corpus data client-side |
| Lazy demo recovery | empty demo workspace rebuilds on first non-Web question | public demo remains usable after refresh/Space restart without manual lifecycle knowledge |
| Workspace preflight | empty document/table state short-circuits before retrieval/generation | lifecycle failures are not mistaken for retrieval failures |
| Explicit abstain node | LangGraph terminal path for missing local data / insufficient corpus-only evidence | avoids pointless verification/revision and evidence-free generations |
| Inspector summary | workspace stats + semantic plan + evidence + execution path + node-latency waterfall above raw trace | makes architecture and latency bottlenecks legible without hiding low-level diagnostics |
| Source preview hygiene | word-boundary truncation with explicit `...` | prevents visibly broken snippets such as half a final word |
| UI | Gradio demo corpus + RAG switches + progress + trace | recruiters can exercise the architecture without setup |
| Deployment | one Docker Space, non-root writable workspace/cache paths | cheap reproducible hosting without permission failures |
| CI | Ruff + pytest | software-engineering discipline |

## Deliberate production trade-offs

The demo uses embedded Qdrant, in-memory DuckDB, deterministic source profiles, an in-process cache and ephemeral sessions because a small public Hugging Face Space should stay cheap and understandable. The interfaces are intentionally separable so a real service can swap in managed Qdrant, Redis, object storage, a warehouse, OIDC and queued ingestion.

| Profile benchmark | optional Fast/Balanced/Agentic labeled comparison | quantifies whether additional planning, reranking and verification cost is justified for representative tasks |
| Hard-mode robustness | paraphrase/distractor/missing/multi-hop/insight/SQL/local-freshness/injection cases | prevents a near-perfect easy benchmark from becoming meaningless as a regression signal |

| Evaluation provenance | run ID + server-boot ID + fresh/reused status | makes cache/rebuild behavior auditable and prevents fresh runs from being mistaken for saved reuse |
| Context-efficiency diagnostics | source Precision@5 beside Recall@5 + targeted recommendations | exposes distractor-heavy context without sacrificing overview/synthesis breadth prematurely |
| Grounded absence | evidence-cited missing-information answers skip unnecessary revise | rewards calibrated uncertainty and reduces extra model calls on unanswerable local questions |