docs/limitations.md

# Limitations

This repository is designed as a reproducible RAG evaluation command center over bundled synthetic/offline artifacts.

## Scope boundaries

- No live LLM or embedding calls.
- No production vector database.
- No online document ingestion pipeline.
- No authentication, RBAC, multi-tenant controls, or API-key lifecycle.
- No scheduled monitoring jobs or alert delivery.
- No persistent user state beyond Streamlit session state.
- No production incident automation.

## Data boundaries

- The bundled tables are synthetic/offline evaluation artifacts.
- Metrics should be interpreted as evaluation diagnostics, not production SLOs.
- Policy simulation uses offline evidence-strength signals and should not be treated as a deployable gate without fresh validation.
- Evidence-strength scores are retrieval-side diagnostics, not calibrated model confidence.

## Implementation boundaries

- Filtering functions favor defensive copies over in-place mutation. This keeps Streamlit reruns predictable for the bundled dataset size, but very large evaluation tables may require a more memory-conscious filtering strategy.
- Risk and configuration scores use documented deterministic review weights, not learned coefficients. Recalibrate them before applying the dashboard to a real production corpus.

## Why these boundaries are intentional

The goal of v1.0.0 is to be deterministic, inspectable, testable, and easy to run from a clean clone. Live adapters and production integrations would add secrets, network variance, cost, and non-deterministic test behavior.