Update environment configuration and Streamlit app for enhanced timeout handling

- Added new environment variables `DOC_AUDI_API_BASE` and `DOC_AUDI_HTTP_READ_TIMEOUT` to `.env.example`, with updated default values and constraints.
- Improved timeout handling in `streamlit_app.py`, allowing for a configurable read timeout of up to 7200 seconds.
- Revised documentation across multiple files to reflect changes in timeout settings and environment variable usage, ensuring clarity for users.
- Removed outdated `LOGICAL_DEVELOPMENT_SEQUENCE.md` as part of documentation cleanup.

.env.example

@@ -44,5 +44,8 @@ JOBS_DB_PATH=./data/jobs.db
 # Limits
 MAX_DOCUMENTS_PER_BATCH=100
 
-# Streamlit → API
+# Streamlit → API (Streamlit process reads these when set in the shell / OS env)
 STREAMLIT_BACKEND_URL=http://localhost:8000
+DOC_AUDI_API_BASE=http://127.0.0.1:8000
+# Read timeout (seconds) for Ask/Summarise HTTP calls; default in code is 3600 if unset
+DOC_AUDI_HTTP_READ_TIMEOUT=3600

LOGICAL_DEVELOPMENT_SEQUENCE.md

@@ -1,506 +0,0 @@
-# DocuAudit AI - Milestone Development Sequence (Build -> Verify -> Move On)
-
-This guide is written so you can develop strictly milestone-by-milestone and validate your output at each step.
-
-Rule for progression:
-- If a milestone verification fails, do not continue to the next milestone.
-- Only move forward when all checks for the current milestone pass.
-
----
-
-## Shared Setup and Run Commands
-
-### One-time setup
-```bash
-uv venv --python 3.11
-uv init --python 3.11
-uv pip install -r requirements.txt
-copy .env.example .env
-```
-
-### Run backend
-```bash
-uv run uvicorn api.main:app --host 0.0.0.0 --port 8000 --reload
-```
-
-### Run UI
-```bash
-uv run streamlit run streamlit_app.py
-```
-
-### Smoke checks
-```bash
-curl http://localhost:8000/health
-curl http://localhost:8000/docs
-```
-
----
-
-## Milestone 1 - FastAPI Foundation
-
-### Dependencies 
-- `fastapi`
-- `uvicorn[standard]`
-
-### After adding dependencies 
-- `uv add requirements.txt`
-- `uv pip install -r requirements.txt`
-
-### Depends on previous milestones
-- None (starting point).
-
-### Expected input
-- Fresh repo with Python environment ready.
-- `requirements.txt` and `.env` prepared.
-
-### Build scope
-- Create `api/main.py`.
-- Add `GET /health`.
-- App starts with Uvicorn.
-
-### Expected output/result
-- API starts without errors.
-- `/health` returns success JSON.
-- Swagger loads at `/docs`.
-
-### Start backend server- fastapi
-  - `uv run uvicorn api.main:app --host 0.0.0.0 --port 8000 --reload`
-
-### Verification checks
-```bash
-curl http://localhost:8000/health
-curl http://localhost:8000/docs
-```
-
-### Pass criteria
-- No startup exception.
-- No 404/500 for `/health` and `/docs`.
-
----
-
-## Milestone 2 - Route Skeletons (Placeholder Only)
-
-### Dependencies 
-- Milestone 1 dependencies only.
-- (No new runtime dependency required if routes are placeholders.)
-
-### Depends on previous milestones
-- Milestone 1 must pass (`/health` and `/docs` working).
-
-### Expected input
-- Running FastAPI app from Milestone 1.
-- Router module structure available under `api/routes`.
-
-### Build scope
-- Add routers:
-  - `api/routes/ingest.py`
-  - `api/routes/query.py`
-  - `api/routes/jobs.py`
-  - `api/routes/audit.py`
-- Register routes in `api/main.py`.
-- Keep responses as placeholders only.
-
-### Expected output/result
-- All route paths exist and respond.
-- Placeholder payloads return consistently.
-
-### Start backend server- fastapi
-  - `uv run uvicorn api.main:app --host 0.0.0.0 --port 8000 --reload`
-
-### Verification checks
-- Open Swagger and call each endpoint once.
-- Confirm no 404/500.
-
-### Pass criteria
-- Route wiring complete.
-- No real business logic yet.
-
----
-
-## Milestone 3 - Config + Request/Response Contracts
-
-### Dependencies 
-- `pydantic`
-- `pydantic-settings`
-- `python-dotenv`
-
-### After adding dependencies 
-- `uv add requirements.txt`
-- `uv pip install -r requirements.txt`
-
-### Depends on previous milestones
-- Milestone 2 route skeletons must be wired and reachable.
-
-### Expected input
-- Existing route handlers ready for request body integration.
-- `.env` file available for settings values.
-
-### Build scope
-- Add `api/config.py` (env-backed settings).
-- Add `models/requests.py` and `models/responses.py`.
-- Apply request validation in routes.
-
-### Expected output/result
-- Config values read from `.env`.
-- Valid requests succeed.
-- Invalid payloads show schema errors.
-
-### Start backend server- fastapi
-  - `uv run uvicorn api.main:app --host 0.0.0.0 --port 8000 --reload`
-
-### Verification checks
-```bash
-curl -X POST http://localhost:8000/query/ask ^
-  -H "Content-Type: application/json" ^
-  -d "{\"question\":\"What are key risks?\",\"collection_name\":\"default\"}"
-```
-- Also test one invalid payload (e.g., missing `question`).
-
-### Pass criteria
-- Validation behavior is visible and correct.
-
----
-
-## Milestone 4 - RAG Ingestion Pipeline (No Answer Generation)
-
-### Dependencies
-- `langchain`
-- `langchain-core` (pulled via LangChain ecosystem)
-- `langchain-chroma`
-- `chromadb`
-- `langchain-community`
-- `langchain-ollama` (if using Ollama embeddings)
-- `langchain-openai` + `openai` (if using OpenAI embeddings)
-- `pymupdf` (PDF loading)
-- `python-multipart` (file upload handling in ingest route)
-
-### After adding dependencies 
-- `uv add requirements.txt`
-- `uv pip install -r requirements.txt`
-
-### Depends on previous milestones
-- Milestone 3 contracts/config must pass.
-
-### Provider selection
-- Choose provider in `.env` via `LLM_PROVIDER`.
-- Supported values:
-  - `ollama`
-  - `openai`
-  - `anthropic`
-  - `huggingface`
-
-### Expected input
-- Valid upload route available to accept files.
-- Configuration values for chunking, Chroma path, and provider present.
-- Test documents (PDF/TXT/MD) ready for ingestion.
-
-### Build scope
-- Implement:
-  - `rag/loader.py`
-  - `rag/chunker.py`
-  - `rag/embedder.py`
-  - `rag/vector_store.py`
-- Wire ingest flow: load -> chunk -> embed -> persist.
-- Preserve metadata:
-  - `source`
-  - `page`
-  - `chunk_index`
-
-### Expected output/result
-- Upload/ingest creates vectors in Chroma.
-- Collection has stored documents/chunks.
-
-### Start backend server- fastapi
-  - `uv run uvicorn api.main:app --host 0.0.0.0 --port 8000 --reload`
-
-### Verification checks
-- Ingest sample PDF(s).
-- Confirm collection appears and has document count > 0.
-
-### Pass criteria
-- Vector persistence works.
-- No LLM answer quality evaluation yet (that is Milestone 5).
-
----
-
-## Milestone 5 - Retrieval + Grounded LLM Answer
-
-### Dependencies
-- Keep Milestone 4 dependencies.
-- Add provider package(s) for your selected chat LLM:
-  - Ollama: `langchain-ollama`
-  - OpenAI: `langchain-openai`, `openai`
-  - Anthropic: `langchain-anthropic`, `anthropic`
-  - Hugging Face endpoint: `langchain-community` (+ API key)
-
-### After adding dependencies 
-- `uv add requirements.txt`
-- `uv pip install -r requirements.txt`
-
-### Depends on previous milestones
-- Milestone 4 vectors must exist in Chroma (ingestion verified).
-
-### Expected input
-- Ingested collection with non-zero document/chunk vectors.
-- Query endpoint contract from Milestone 3.
-- Valid LLM API key/local model access based on selected provider.
-
-### Build scope
-- Implement `rag/retriever.py` to:
-  - retrieve top-k chunks
-  - format context
-  - invoke configured LLM
-  - return answer + sources
-- Wire query routes to retriever.
-
-### Expected output/result
-- Query returns grounded answer based on retrieved chunks.
-- Source citations are included.
-- Empty/no-match returns safe fallback answer.
-
-### Start backend server- fastapi
-  - `uv run uvicorn api.main:app --host 0.0.0.0 --port 8000 --reload`
-
-### Verification checks
-- Ask question that is clearly answerable from uploaded docs.
-- Ask question not present in docs and verify fallback message.
-
-### Pass criteria
-- Retrieve-then-generate flow works reliably.
-
----
-
-## Milestone 6 - Audit Persistence
-
-### Dependencies
-- `aiosqlite`
-
-### After adding dependencies 
-- `uv add requirements.txt`
-- `uv pip install -r requirements.txt`
-
-### Depends on previous milestones
-- Milestone 5 query flow must return answers reliably.
-
-### Expected input
-- Query route already producing response payload (`answer`, `sources`, metadata).
-- Writable SQLite path configured in environment.
-
-### Build scope
-- Implement `storage/audit_store.py` fully.
-- Persist query request/response metadata.
-- Add audit list/detail retrieval endpoints.
-
-### Expected output/result
-- Every query creates an audit record.
-- You can fetch log entries by list and id.
-
-### Start backend server- fastapi
-  - `uv run uvicorn api.main:app --host 0.0.0.0 --port 8000 --reload`
-
-### Verification checks
-- Run one query.
-- Fetch corresponding audit entry.
-
-### Pass criteria
-- Audit trail is complete and query-linked.
-
----
-
-## Milestone 7 - Background Ingestion Jobs
-
-
-### Dependencies
-- No mandatory new package (uses FastAPI background tasks + existing modules).
-
-### After adding dependencies (if any)
-- `uv add requirements.txt`
-- `uv pip install -r requirements.txt`
-
-### Depends on previous milestones
-- Milestone 4 ingestion logic must work synchronously first.
-- Milestone 6 persistence layer should be available for job tracking.
-
-### Expected input
-- Working ingest function (`load -> chunk -> add_documents`).
-- Job status storage schema and endpoints available.
-
-### Build scope
-- Implement `workers/ingest_worker.py`.
-- Move ingestion processing to background.
-- Track status in jobs endpoints/store.
-
-### Expected output/result
-- Upload returns `job_id`.
-- Status transitions:
-  - `queued`
-  - `processing`
-  - `completed` or `failed`
-
-### Start backend server- fastapi
-  - `uv run uvicorn api.main:app --host 0.0.0.0 --port 8000 --reload`
-
-### Verification checks
-- Upload docs and poll job status endpoint.
-
-### Pass criteria
-- API remains responsive while ingestion runs.
-
----
-
-## Milestone 8 - Endpoint Completion (Production Shape)
-
-### Dependencies
-- `httpx` (URL ingestion/download flow)
-
-### After adding dependencies 
-- `uv add requirements.txt`
-- `uv pip install -r requirements.txt`
-
-### Depends on previous milestones
-- Milestones 1 through 7 should be passing individually.
-
-### Expected input
-- Stable ingestion, retrieval, audit, and jobs internals.
-- Final request/response models already defined.
-
-### Build scope
-- Ensure behavior and contracts are complete for:
-  - `POST /ingest/upload`
-  - `POST /ingest/url`
-  - `GET /ingest/collections`
-  - `DELETE /ingest/collection/{collection_name}`
-  - `POST /query/ask`
-  - `POST /query/summarise`
-  - `GET /jobs`
-  - `GET /jobs/{job_id}`
-  - `GET /audit/logs`
-  - `GET /audit/logs/{query_id}`
-
-### Expected output/result
-- Full backend flow works from upload to audited answer.
-
-### Start backend server
-  - `uv run uvicorn api.main:app --host 0.0.0.0 --port 8000 --reload`
-
-### Verification checks
-- Run one complete cycle using API only:
-  - ingest -> job complete -> ask -> inspect sources -> fetch audit
-
-### Pass criteria
-- No contract mismatches or broken endpoints.
-
----
-
-## Milestone 9 - Streamlit UI Integration
-
-### Dependencies
-- `streamlit`
-
-### After adding dependencies 
-- `uv add requirements.txt`
-- `uv pip install -r requirements.txt`
-
-### Depends on previous milestones
-- Milestone 8 full backend flow must be stable.
-
-### Expected input
-- Running backend server with all finalized endpoints.
-- Predictable API payload shapes for upload/query/jobs/audit.
-
-### Build scope
-- Connect `streamlit_app.py` to backend API.
-- Include Upload, Jobs, Ask, Summarise, Audit sections.
-
-### Expected output/result
-- Full flow works from UI alone.
-
-### Start backend server- fastapi
-```bash
-uv run uvicorn api.main:app --host 0.0.0.0 --port 8000
-```
-### Start frontend server- Streamlit
-```bash
-uv run streamlit run streamlit_app.py --server.address=0.0.0.0 --server.port=8501
-```
-
-### Verification checks
-- Perform end-to-end cycle from Streamlit without manual curl.
-
-### Pass criteria
-- UI reflects backend status and responses correctly.
-
----
-
-## Milestone 10 - Tests and Hardening
-
-### Dependencies
-- `pytest`
-- `pytest-asyncio`
-
-### After adding dependencies 
-- `uv add requirements.txt`
-- `uv pip install -r requirements.txt`
-
-### Depends on previous milestones
-- Milestones 1 through 9 completed and stable enough to test.
-
-### Expected input
-- Final endpoint behavior and contracts.
-- Representative sample docs/test data and deterministic test cases.
-
-### Build scope
-- Add/update:
-  - `tests/test_ingest.py`
-  - `tests/test_query.py`
-  - `tests/test_audit.py`
-- Cover success + validation + failure paths.
-
-### Expected output/result
-- Automated tests pass and catch regressions.
-
-### Verification checks
-```bash
-uv run pytest -q
-uv run pytest tests/test_ingest.py -q
-uv run pytest tests/test_query.py -q
-uv run pytest tests/test_audit.py -q
-```
-
-### Pass criteria
-- Core behavior is test-covered and stable.
-
----
-
-## Milestone-by-Milestone Output Checklist
-
-Use this quick gate before advancing:
-
-1. Milestone 1: API up + `/health` + `/docs`
-2. Milestone 2: all route stubs reachable
-3. Milestone 3: schema validation enforced
-4. Milestone 4: vectors written to Chroma
-5. Milestone 5: grounded answer + citations
-6. Milestone 6: audit log persisted and fetchable
-7. Milestone 7: background job lifecycle visible
-8. Milestone 8: full API flow complete
-9. Milestone 9: full UI flow complete
-10. Milestone 10: tests passing
-
-If any line fails, fix that milestone before moving forward.
-
----
-
-## Development Completion Dependency Chain
-
-Use this chain to understand what must be complete before a later milestone is considered valid:
-
-- Milestone 2 depends on 1
-- Milestone 3 depends on 2
-- Milestone 4 depends on 3
-- Milestone 5 depends on 4
-- Milestone 6 depends on 5
-- Milestone 7 depends on 4 and 6
-- Milestone 8 depends on 1-7
-- Milestone 9 depends on 8
-- Milestone 10 depends on 1-9

streamlit_app.py

@@ -11,14 +11,22 @@ import streamlit as st
 
 DEFAULT_API_BASE = os.environ.get("DOC_AUDI_API_BASE", "http://127.0.0.1:8000")
 
+# httpx read timeout for Ask/Summarise: embeddings + LLM on CPU or cold Ollama often exceeds 10 minutes.
+_HTTP_READ_TIMEOUT_DEFAULT_S = 3600.0
+_HTTP_READ_TIMEOUT_MIN_S = 60.0
+_HTTP_READ_TIMEOUT_MAX_S = 7200.0
+
 
 def _http_read_timeout_seconds() -> float:
-    raw = os.environ.get("DOC_AUDI_HTTP_READ_TIMEOUT", "600")
+    raw = os.environ.get(
+        "DOC_AUDI_HTTP_READ_TIMEOUT",
+        str(int(_HTTP_READ_TIMEOUT_DEFAULT_S)),
+    )
     try:
         read_s = float(raw)
     except ValueError:
-        read_s = 600.0
-    return max(60.0, min(read_s, 3600.0))
+        read_s = _HTTP_READ_TIMEOUT_DEFAULT_S
+    return max(_HTTP_READ_TIMEOUT_MIN_S, min(read_s, _HTTP_READ_TIMEOUT_MAX_S))
 
 
 def _http_timeout() -> httpx.Timeout:
@@ -29,8 +37,10 @@ def _http_timeout() -> httpx.Timeout:
 
 def _fmt_timeout_hint() -> str:
     cap = int(_http_read_timeout_seconds())
+    lo, hi = int(_HTTP_READ_TIMEOUT_MIN_S), int(_HTTP_READ_TIMEOUT_MAX_S)
     return (
-        f"The UI stops waiting after **{cap}s** per request (set **DOC_AUDI_HTTP_READ_TIMEOUT** to raise it, max 3600). "
+        f"The UI stops waiting after **{cap}s** per request (set **DOC_AUDI_HTTP_READ_TIMEOUT**, "
+        f"allowed **{lo}–{hi}** s). "
         "Ensure `ollama serve` is running; cold models or CPU inference can exceed a few minutes."
     )
 
@@ -178,7 +188,7 @@ def main() -> None:
         )
         st.caption(
             f"Ask/Summarise wait up to **{int(_http_read_timeout_seconds())}s** per request "
-            "(env `DOC_AUDI_HTTP_READ_TIMEOUT`, range 60–3600)."
+            f"(env `DOC_AUDI_HTTP_READ_TIMEOUT`, range {int(_HTTP_READ_TIMEOUT_MIN_S)}–{int(_HTTP_READ_TIMEOUT_MAX_S)})."
         )
         if st.button("Test connection"):
             ok, msg = _health_check()