docs: update all references Render → HF Spaces (Stage 12)

- README.md: deployment row, design decisions, architecture diagram
- docs/architecture.md: component table, key decisions, known limits, future work
- docs/decisions.md: Render decision superseded, HF Spaces decision added
- docs/evolution.md: Stage 12 added, Current State Snapshot updated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

README.md

@@ -88,7 +88,7 @@ If retrieval degrades, you see it before the user does.
 | **Eval** | RAGAS (pre-computed JSON) + LLM-as-Judge per turn |
 | **Observability** | LangSmith |
 | **Frontend** | React 19 + Vite + Tailwind CSS v4 |
-| **Deployment** | Render (Docker backend) + Vercel (frontend) |
+| **Deployment** | HF Spaces Docker (backend, 16GB RAM) + Vercel (frontend) |
 
 ---
 
@@ -232,10 +232,11 @@ prism/
 
 ## Design Decisions Worth Noting
 
-- **Singleton retriever cache per workspace** — without cache, every request rebuilt the Chroma instance (full embedding reload) → OOM after 2–3 queries. Cache invalidated after ingest.
-- **CPU-only torch in Dockerfile** — sentence-transformers pulls CUDA torch (~2GB) by default, OOMing Render 512MB before uvicorn binds port. Pre-installing CPU torch (~200MB) is mandatory.
-- **RAGAS pre-computed locally** — `nest_asyncio` cannot patch `uvloop` (uvicorn's event loop on Linux). Live RAGAS eval on Render always 500s. Run locally, commit JSON, Vercel reads file.
-- **TinyBERT-L-2-v2 reranker** — MiniLM-L-6-v2 (~85MB) + base memory exceeded 512MB on web queries. TinyBERT (~17MB) saves 68MB permanently.
+- **Singleton retriever cache per workspace** — without cache, every request rebuilt the Chroma instance (full embedding reload). Cache invalidated after ingest.
+- **CPU-only torch in Dockerfile** — sentence-transformers pulls CUDA torch (~2GB) by default. Pre-installing CPU torch keeps the image lean.
+- **HF Spaces UID 1000** — HF runs Docker containers as user 1000. Reranker weights baked under `HF_HOME=/app/.cache/huggingface` as user 1000 at build time; `HF_HUB_OFFLINE=1` set after download to block runtime Hub calls.
+- **RAGAS pre-computed locally** — `nest_asyncio` cannot patch `uvloop` (uvicorn's event loop on Linux). Live RAGAS eval always 500s. Run locally, commit JSON, Vercel reads file.
+- **TinyBERT-L-2-v2 reranker** — same ranking quality as MiniLM-L-6-v2 at demo corpus scale; ~17MB vs ~85MB.
 - **Web search bypasses chain** — `ConversationalRetrievalChain` condensation step strips prepended Tavily context before LLM sees it. Web path uses direct LLM call with chat history.
 - **Idempotent ingestion** — chunk IDs are `md5(source + page + text)`. Re-ingesting same doc does not duplicate chunks.
 

docs/architecture.md

@@ -13,8 +13,8 @@ Query → hybrid retrieval (ChromaDB dense + BM25 sparse) → weighted RRF fusio
 | Vector store | ChromaDB (persistent) | Dense embedding storage and retrieval |
 | Sparse retrieval | rank_bm25 (BM25Okapi) | Keyword-match retrieval for regulatory text |
 | Hybrid fusion | Weighted RRF (dense 0.7 + sparse 0.3) | Merge dense + sparse result lists |
-| Reranker | cross-encoder/ms-marco-TinyBERT-L-2-v2 | Re-score top-10 → return top-5 (~17MB vs 85MB; OOM prevention on 512MB Render) |
-| Embeddings | Euron API (text-embedding-3-small) | API-based; avoids OOM on Render free tier. Groq has no embeddings endpoint. |
+| Reranker | cross-encoder/ms-marco-TinyBERT-L-2-v2 | Re-score top-10 → return top-5 (~17MB; chosen over MiniLM-L-6-v2 ~85MB for lower memory footprint) |
+| Embeddings | Euron API (text-embedding-3-small) | API-based; Groq has no embeddings endpoint. |
 | LLM | Groq (llama-3.3-70b-versatile) via langchain-groq | Fast open-weight inference; OpenAI-compatible |
 | Chunking | LangChain ParentDocumentRetriever | Child 200-char indexed, parent 800-char sent to LLM |
 | Memory | ConversationBufferWindowMemory (k=10) | Last 10 conversation turns |
@@ -28,7 +28,7 @@ Query → hybrid retrieval (ChromaDB dense + BM25 sparse) → weighted RRF fusio
 | Document parsing | LlamaParse (primary), pypdf (fallback) | PDF extraction |
 | Backend | FastAPI + Uvicorn | REST API |
 | Frontend | React 19 + Vite + Tailwind CSS v4 | Chat / Upload tabs |
-| Deployment | Render (Docker backend) + Vercel (frontend) + Vercel (eval-dashboard) | Production |
+| Deployment | HF Spaces (Docker backend, 16GB RAM) + Vercel (frontend) + Vercel (eval-dashboard) | Production |
 
 ## Data flow
 
@@ -49,24 +49,26 @@ Query → hybrid retrieval (ChromaDB dense + BM25 sparse) → weighted RRF fusio
 7. Response includes: answer, sources (with scores), retrieval_method
 
 ## Key design decisions
-- **API embeddings over local**: sentence-transformers ~400MB OOMs on Render 512MB free tier; Euron API ~0MB. Groq used for LLM; Euron retained for embeddings (Groq exposes no embeddings endpoint).
+- **API embeddings over local**: Euron API embeddings ~0MB RAM; Groq has no embeddings endpoint so Euron is retained for embeddings.
 - **ParentDocumentRetriever**: small chunks improve retrieval precision; large parent chunks improve answer faithfulness
 - **Cross-encoder reranker**: bi-encoder (ChromaDB) is fast but approximate; cross-encoder is slower but more accurate on top-20 pool
 - **BM25 weight 0.3**: regulatory text has exact keyword matches (section numbers); sparse retrieval catches what dense misses
-- **RAGAS benchmark pre-computed locally**: `nest_asyncio` cannot patch `uvloop` (used by uvicorn on Render Linux), making live RAGAS eval impossible on prod. Run `scripts/run_ragas_local.py` locally, commit JSON results, Vercel builds dashboard from file.
-- **TinyBERT-L-2-v2 reranker**: MiniLM-L-6-v2 (~85MB) + base memory (~250MB) + Tavily content + LLM call exceeded Render 512MB on web queries. TinyBERT-L-2-v2 is ~17MB — same ranking quality at demo corpus scale.
-- **Singleton vectorstore/retriever cache**: each workspace caches its Chroma vectorstore + HybridRetriever in a module-level dict. Without cache, every chat request created a new Chroma instance (full embedding reload), causing OOM on repeated queries. Cache is invalidated after ingest.
+- **RAGAS benchmark pre-computed locally**: `nest_asyncio` cannot patch `uvloop` (used by uvicorn on Linux), making live RAGAS eval impossible on prod. Run `scripts/run_ragas_local.py` locally, commit JSON results, Vercel builds dashboard from file.
+- **TinyBERT-L-2-v2 reranker**: MiniLM-L-6-v2 (~85MB) vs TinyBERT-L-2-v2 (~17MB) — same ranking quality at demo corpus scale with lower memory footprint.
+- **Singleton vectorstore/retriever cache**: each workspace caches its Chroma vectorstore + HybridRetriever in a module-level dict. Without cache, every chat request created a new Chroma instance (full embedding reload). Cache is invalidated after ingest.
 - **Multi-workspace isolation**: each workspace maps to one ChromaDB collection. Frontend workspace switcher passes `workspace_id` on every request; backend resolves the correct collection before retrieval.
-- **URL size guard**: `url_loader.py` enforces a max content size before embedding URL content, preventing OOM from large external pages.
+- **URL size guard**: `url_loader.py` enforces a max content size before embedding URL content, preventing memory spikes from large external pages.
 - **Eval dashboard separate site**: eval runs offline, results versioned as JSON. Separates eval tooling from user-facing app; no live eval endpoint on prod backend. Per-message faithfulness badge removed from UI — moved to dedicated dashboard.
 - **answer_correctness over faithfulness**: faithfulness (LLM judge vs retrieved chunks) is circular — inflates when eval pairs are corpus-aligned. answer_correctness (LLM judge vs ground_truth reference) is an independent signal.
+- **HF Spaces Docker (UID 1000)**: model weights baked into image under `HF_HOME=/app/.cache/huggingface` as user 1000 at build time; `HF_HUB_OFFLINE=1` set after download to block runtime network calls.
 
 ## Known limitations
 - InMemoryStore for parent chunks: does not survive server restart (re-ingest required)
 - BM25 index rebuilt in memory on each startup (not persisted to disk)
-- Render free tier: 512MB RAM, cold starts after inactivity
+- HF Spaces free tier: ephemeral filesystem — chroma_db lost on cold start (re-upload required)
+- Euron embedding API sequential: ~5s/chunk — 30 chunks = ~150s blocking upload
 
 ## Future improvements
 - Persist BM25 index to disk (pickle)
-- Multi-collection ChromaDB (one per document set)
+- Move Euron embedding to background task (return 202, poll for ready)
 - Streaming responses from LLM

docs/decisions.md

@@ -4,15 +4,16 @@
 
 | Date | Decision | Rationale | Status |
 |------|----------|-----------|--------|
-| 2026-05 | Euron API for embeddings (not local sentence-transformers) | Local model ~400MB → OOM on Render 512MB free tier | Active |
+| 2026-05 | Euron API for embeddings (not local sentence-transformers) | Groq has no embeddings endpoint; Euron API keeps RAM footprint near zero vs local model | Active |
 | 2026-05-24 | Groq (llama-3.3-70b-versatile) for LLM; Euron retained for embeddings | Groq: faster inference, open-weight model. Euron kept for embeddings — Groq has no embeddings endpoint. | Active |
-| 2026-05 | Render (Docker) over Railway for backend | Free tier RAM fit confirmed: embeddings ~150MB + reranker ~17MB = ~230MB total | Active |
+| 2026-05 | Render (Docker) over Railway for backend | Free tier RAM fit confirmed: embeddings ~150MB + reranker ~17MB = ~230MB total | **Superseded 2026-06-22** |
+| 2026-06-22 | HF Spaces Docker over Render for backend | Render 512MB free tier caused repeated OOM under contextual refresh + concurrent chat. HF Spaces CPU Basic = 16GB RAM free. No memory guards needed. Port 7860, user UID 1000, HF_HOME=/app/.cache/huggingface baked at build time. | Active |
 | 2026-05 | ParentDocumentRetriever (child 200 / parent 800) | Better faithfulness: small chunks retrieved precisely, large chunks give LLM full context | Active |
 | 2026-05 | BM25 weight 0.3 in RRF fusion | Regulatory text has exact keyword matches; sparse recall is complementary, not dominant | Active |
 | 2026-05-30 | RAGAS pre-computed locally, dashboard JSON-driven | `nest_asyncio` cannot patch `uvloop` on Render — live eval endpoint always 500s. Run `scripts/run_ragas_local.py` (8B judge model) → commit `ragas_benchmark.json` → Vercel builds dashboard. | Active |
 | 2026-05-30 | RAGAS judge uses `llama-3.1-8b-instant` not `llama-3.3-70b` | 70B model exhausts Groq free-tier 100k TPD in one eval run. 8B has 500k TPD and is sufficient for statement-level faithfulness checks. Answer generation still uses 70B. | Active |
 | 2026-05 | LangSmith tracing via env var (no code changes) | LangChain reads LANGCHAIN_TRACING_V2 automatically; zero instrumentation cost | Active |
-| 2026-05-30 | Reranker switched to TinyBERT-L-2-v2 (~17MB) from MiniLM-L-6-v2 (~85MB) | MiniLM + base memory + Tavily content + LLM call exceeded 512MB on web queries; TinyBERT saves 68MB permanently with acceptable ranking quality at demo scale | Active |
+| 2026-05-30 | Reranker switched to TinyBERT-L-2-v2 (~17MB) from MiniLM-L-6-v2 (~85MB) | Lower memory footprint with acceptable ranking quality at demo corpus scale | Active |
 | 2026-05 | Cross-encoder reranker pre-downloaded at Docker build time | Avoids cold-start latency on first request in production | Active |
 | 2026-05 | Idempotent ingestion via md5(source+page+text) chunk IDs | Re-running ingest does not duplicate chunks in ChromaDB | Active |
 | 2026-06-14 | Multi-workspace: one ChromaDB collection per workspace | Isolated document sets per workspace; `workspace_id` passed on every request; `list_collections()` normalised for chromadb ≥0.5.4 (returns `list[str]`) and <0.5 (returns `list[Collection]`) | Active |

docs/evolution.md

@@ -1,7 +1,7 @@
 # Prism — Project Evolution
 
 > End-to-end record of what was broken at each stage, what was built to fix it, and what is planned next.
-> Updated as the project evolves. Last updated: 2026-06-20 (Stage 11).
+> Updated as the project evolves. Last updated: 2026-06-22 (Stage 12).
 
 ---
 
@@ -18,6 +18,7 @@
 9. [Stage 8 — Eval Dashboard + Rigorous Metrics](#stage-8--eval-dashboard--rigorous-metrics-2026-06-17)
 10. [Stage 9 — Multi-Query Retrieval](#stage-9--multi-query-retrieval-2026-06-19)
 11. [Stage 10 — Contextual Retrieval (Eval)](#stage-10--contextual-retrieval-eval-2026-06-20)
+12. [Stage 12 — HF Spaces Migration](#stage-12--hf-spaces-migration-2026-06-22)
 12. [Stage 11 — Contextual Retrieval in Production + Dashboard Polish](#stage-11--contextual-retrieval-in-production--dashboard-polish-2026-06-20)
 13. [Current State Snapshot](#current-state-snapshot)
 13. [Roadmap — Retrieval & Answer Quality](#roadmap--retrieval--answer-quality)
@@ -487,15 +488,43 @@ Eval:         Separate eval-dashboard/ static site → https://askprism-eval.ver
               Versioning: MAJOR.MINOR.PATCH — name changes on MAJOR only (v1.x.x=Violet, v2.x.x=Indigo)
 Frontend:     Violet v1.3 badge in sidebar footer. Maintenance banner config-driven (frontend/src/config.js).
 Workspaces:   Per-workspace ChromaDB collection, singleton retriever cache
-Infra:        Render (backend, ephemeral FS — re-upload required after cold start) +
+Infra:        HF Spaces CPU Basic (backend, 16GB RAM, ephemeral FS — re-upload required after cold start) +
               https://askprism.vercel.app/ (frontend) + https://askprism-eval.vercel.app/ (eval)
-Known limits: Euron embed ~52s for 30 chunks (sync, blocking). Next: move to background.
-              Render ephemeral FS: chroma_db lost on restart. Fix: Render persistent disk.
+              Backend URL: https://benroshan-prism.hf.space
+Known limits: Euron embed ~5s/chunk sequential — 30 chunks = ~150s blocking upload. Next: move embed to background.
+              HF Spaces ephemeral FS: chroma_db lost on cold start. Fix: mount HF persistent storage bucket.
 Observability: LangSmith traces all LLM + retrieval calls (optional, env var)
 ```
 
 ---
 
+## Stage 12 — HF Spaces Migration (2026-06-22)
+
+### What was wrong
+Render free tier (512MB RAM) caused repeated OOM crashes under contextual retrieval:
+- Base RSS after upload = 524MB (over the 512MB limit)
+- `gc.collect()` had no effect — ChromaDB HNSW index + torch runtime held by native allocators, not Python heap
+- Contextual refresh (3 async Groq coroutines) + simultaneous chat (Tavily + LLM + CrossEncoder) = peak exceeded 512MB
+- Workarounds (RSS guard skipping contextual retrieval, web search suppression during refresh) negated the +18% recall improvement
+
+### What we built
+
+| File | Change |
+|------|--------|
+| `Dockerfile` | Port 8000 → 7860 (HF convention). Add `useradd -m -u 1000 user` + `chown -R user /app` (HF runs containers as UID 1000). Set `HF_HOME=/app/.cache/huggingface` BEFORE pre-download so user 1000 owns cached weights. Set `HF_HUB_OFFLINE=1` AFTER download. |
+| `README.md` | Added HF Spaces frontmatter (`sdk: docker`, `app_port: 7860`). Updated deploy instructions. |
+| `server/routes/chat.py` | Removed `is_contextualizing` web search suppression guard (Render-specific). |
+| `server/routes/upload.py` | Removed `RSS > 460MB` contextual retrieval skip guard (Render-specific). |
+| `docs/`, `decisions.md` | Render → HF Spaces across all infra references. |
+
+### Key discoveries
+- HF_HUB_OFFLINE must be set AFTER the pre-download RUN step — setting it before blocks the download itself
+- Docker build runs pre-download as root by default; must `USER 1000` first then set `HF_HOME` under `/app` so runtime user 1000 can read the cached weights
+- HF Spaces free CPU Basic: 2 vCPUs, 16GB RAM — resolves all Render OOM issues permanently
+- Contextual retrieval now runs fully in production (was silently skipped by RSS guard on Render)
+
+---
+
 ## Roadmap — Retrieval & Answer Quality
 
 ### Phase 1 — Quick wins (no infra change, measurable RAGAS lift)
@@ -528,7 +557,7 @@ Observability: LangSmith traces all LLM + retrieval calls (optional, env var)
 ### Phase 3 — UX + trust
 
 #### Streaming Responses
-- **Problem:** User submits question → 8–15s wait → full answer appears. On Render free vCPU this feels broken.
+- **Problem:** User submits question → 8–15s wait → full answer appears. Feels broken even on fast hardware.
 - **How:** Backend: `chain.astream_events()` → `StreamingResponse` yielding SSE tokens. Frontend: `EventSource` or `fetch` + `ReadableStream` — append tokens as they arrive. Faithfulness scoring runs as background task after full answer assembled.
 - **Effort:** High — both backend and frontend change. `ConversationalRetrievalChain` supports `astream_events()` in LangChain ≥0.2.
 - **Impact:** Perceived latency drops from 10s to ~1s. Single biggest UX improvement.