Deploy: DeepTrust workspace (v2 clean history)

.cursor/plans/client-side_rag_knowledge_upload_c1d69919.plan.md

@@ -0,0 +1,141 @@
+---
+name: Client-side RAG knowledge upload
+overview: "Implement the Context panel as a real client-side RAG flow: persist PDFs, notes, and URLs in IndexedDB, index them with in-browser embeddings and a vector store, retrieve relevant chunks at query time, and pass that context to the research agent so the LLM uses it when planning and synthesizing."
+todos: []
+isProject: false
+---
+
+# Client-Side RAG: Knowledge Upload and Retrieval
+
+## Current state
+
+- **Frontend** ([app/page.tsx](app/page.tsx)): The Context panel stores only metadata in React state (`KnowledgeItem`: id, type, label, meta). File content is never read; notes are plain text; URLs are just strings. No persistence, no embeddings, no retrieval.
+- **API** ([app/api/research/route.ts](app/api/research/route.ts)): Reads only `query` from the body; `knowledge` is ignored.
+- **Agent** ([lib/agent/graph.ts](lib/agent/graph.ts), [lib/agent/state.ts](lib/agent/state.ts)): `runResearch(userQuery, sessionName)` and `createInitialState({ threadId, userQuery, sessionName })`; no field for retrieved context. The Thinker builds the prompt from `state.userQuery` only ([lib/agent/nodes/thinker.ts](lib/agent/nodes/thinker.ts) line 117).
+
+## Target flow
+
+```mermaid
+sequenceDiagram
+  participant User
+  participant UI
+  participant IndexedDB
+  participant VectorStore
+  participant API
+  participant Agent
+
+  User->>UI: Add PDF / note / URL
+  UI->>UI: Extract text (PDF: pdf.js; note: as-is; URL: see below)
+  UI->>UI: Chunk and embed (browser)
+  UI->>VectorStore: Store chunks + vectors in IndexedDB
+  User->>UI: Run research
+  UI->>VectorStore: Embed query; top-k similarity search
+  UI->>API: POST { query, retrievedContext }
+  API->>Agent: runResearch(query, { knowledgeContext })
+  Agent->>Agent: Thinker / Synthesizer use knowledgeContext in prompts
+```
+
+
+
+## 1. Client: Storage, extraction, embedding, and retrieval
+
+### 1.1 Persistence and data model
+
+- Introduce a **client-side knowledge store** backed by **IndexedDB** (separate from the existing React state so refreshes and future sessions retain data).
+- **Schema** (conceptual):  
+  - **Documents**: `id`, `type` (file | url | note), `label`, `createdAt`, optional `url` for type url.  
+  - **Chunks**: `id`, `documentId`, `text`, `embedding` (array of numbers), `startIndex`, `endIndex`.
+- Use a single IndexedDB database (e.g. `deeptrust-knowledge`) with object stores for documents and chunks (or a library that wraps this).
+
+### 1.2 Text extraction
+
+- **PDFs**: Use **pdf.js** (Mozilla) in the browser to extract text from dropped/selected files. No server round-trip; keep PDFs client-side only.
+- **Notes**: Use the note text as a single “document”; optional chunking by paragraph or fixed size.
+- **URLs**: Two options (choose one for v1):
+  - **A (recommended for scope)**  
+  Store URL as metadata only. Do **not** fetch or embed URL content in the browser (avoids CORS and complexity). When building the research payload, send the list of “context URLs” with the request; the backend can pass them into the Thinker so the plan may include `document_fetch` steps for those URLs, or the Thinker can be prompted to consider “the user provided these URLs as references” in the plan.
+  - **B (full client-side)**  
+  Fetch URL in the browser (e.g. via a Next.js API route that proxies the fetch to avoid CORS), extract text (HTML → text), then chunk and embed like PDFs. Adds proxy and error handling.
+
+Recommendation: **A** for the first iteration; document B as a follow-up.
+
+### 1.3 Chunking and embedding in the browser
+
+- **Chunking**: Split document text into overlapping or fixed-size segments (e.g. 256–512 tokens or ~500 chars with 50–100 char overlap). No server call.
+- **Embeddings**: Use a **browser-run embedding model** so everything stays client-side:
+  - **Option A**: **Transformers.js** (`@xenova/transformers`) with a small feature-extraction model (e.g. `Xenova/all-MiniLM-L6-v2` or similar). Runs in WebAssembly; single dependency consistent with “local-first”.
+  - **Option B**: A dedicated client-side vector DB that includes embeddings (e.g. **idbvec** with external embeddings, or **entity-db** which uses Transformers.js under the hood). Prefer one that stores vectors in IndexedDB and supports similarity search.
+- Store each chunk’s embedding in IndexedDB with a reference to its document and text.
+
+### 1.4 Vector store and retrieval
+
+- **Option A**: **idbvec** (`@brainwires/idbvec`): WASM + IndexedDB, HNSW index, configurable distance. You supply embeddings (e.g. from Transformers.js); idbvec handles storage and k-NN search.
+- **Option B**: **Custom**: Store chunks (with embeddings) in IndexedDB; on query, load relevant chunks and run brute-force cosine similarity (or a tiny WASM k-NN) in a worker to avoid blocking the main thread. Good for small corpora (< ~10k chunks).
+- At “Run research” time: embed the user query with the same model, run top-k similarity search (e.g. k = 5–10), build a single `retrievedContext` string (e.g. concatenate chunk texts with source labels) and send it with the request.
+
+### 1.5 UI and sync with existing Context panel
+
+- Keep the existing **Context panel** UX: drag-and-drop, “Attach files”, URL input, “Short note”, and the list of items.
+- **On add**:
+  - For **files** (PDF): read file, extract text with pdf.js, chunk, embed, and write documents + chunks (+ vectors) to IndexedDB. Add a corresponding entry to React state for the list (label, type, id) and optionally sync the list from IndexedDB on load.
+  - For **notes**: create one document, chunk if needed, embed, store; update UI list.
+  - For **URLs** (v1): store URL in IndexedDB as a document with no chunks (or a single placeholder chunk with URL as “text”); backend will receive “context URLs” and can use them in the prompt or in plan steps.
+- **On remove**: Delete document (and its chunks) from IndexedDB and from React state. Add a remove control next to each item in the list (currently there is none).
+- **On load**: Hydrate the Context list from IndexedDB so persisted knowledge survives refresh.
+
+## 2. Backend: Accept and use retrieved context
+
+### 2.1 Research API
+
+- In [app/api/research/route.ts](app/api/research/route.ts), read from the request body: `query` and either `retrievedContext` (string) or `knowledge` (array or object that includes `retrievedContext` and optionally `contextUrls: string[]`).
+- Call the agent with this context, e.g. `runResearch(query, sessionName, { retrievedContext, contextUrls })`.
+
+### 2.2 Agent state and graph
+
+- In [lib/agent/state.ts](lib/agent/state.ts), add an optional field to `ResearchState`, e.g.  
+`knowledgeContext: z.string().optional()`  
+and optionally `contextUrls: z.array(z.string()).optional()`.
+- In [lib/agent/graph.ts](lib/agent/graph.ts), add the new channel(s) to the state graph (e.g. `knowledgeContext: { value: (_, n) => n }`) and ensure they are part of the initial state when provided.
+- Extend [lib/agent/state.ts](lib/agent/state.ts) `createInitialState` to accept optional `knowledgeContext` (and `contextUrls` if used), and pass them through in [lib/agent/graph.ts](lib/agent/graph.ts) when calling `createInitialState` and streaming.
+
+### 2.3 Thinker and Synthesizer
+
+- In [lib/agent/nodes/thinker.ts](lib/agent/nodes/thinker.ts), when building the user message (around line 117), if `state.knowledgeContext` is present, append it to the prompt, e.g.  
+“The user provided the following retrieved context from their local knowledge base. Use it to inform the plan and prefer steps that leverage this context where relevant:\n\n” + state.knowledgeContext.  
+If `state.contextUrls` is used, add a line like “The user also referenced these URLs: …” so the Thinker can emit `document_fetch` steps for them.
+- In [lib/agent/nodes/synthesizer.ts](lib/agent/nodes/synthesizer.ts), if `state.knowledgeContext` is present, include it in the context passed to the LLM when synthesizing the final report (so the report can cite or summarize the user’s local knowledge).
+
+## 3. Dependencies and build
+
+- Add npm packages: e.g. **pdfjs-dist** (or **react-pdf** if you prefer a React wrapper) for PDF text extraction; **@xenova/transformers** (or the chosen embedding solution); and **idbvec** or a minimal IndexedDB + vector search helper.
+- Ensure the embedding model is loaded only when the user first adds knowledge (or on first “Run research” with non-empty context) to avoid blocking initial page load; consider a small “Preparing knowledge…” state in the UI when the embedding pipeline is loading.
+
+## 4. Scope summary
+
+
+| Area    | In scope                                                                                                                                                                                                                     |
+| ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Client  | IndexedDB persistence for documents/chunks; PDF text extraction (pdf.js); notes as documents; URLs stored as references (no client fetch in v1).                                                                             |
+| Client  | Browser embeddings (Transformers.js or equivalent); vector store in IndexedDB (idbvec or custom); top-k retrieval at query time; send `retrievedContext` (+ optional `contextUrls`) with POST /api/research.                 |
+| Client  | Context panel: remove item; hydrate list from IndexedDB on load; show “indexed” or “ready” per item after processing.                                                                                                        |
+| Backend | Research route parses `retrievedContext` (and optional `contextUrls`); runResearch(..., { knowledgeContext, contextUrls }); state + createInitialState + graph channels; Thinker and Synthesizer include context in prompts. |
+
+
+## 5. Files to add or touch
+
+- **New**: `lib/knowledge/` (or `app/lib/knowledge/`) — IndexedDB schema, chunking, embedding pipeline (or wrapper around Transformers.js + idbvec), and retrieval function. Keep it UI-agnostic so it can be called from the Context panel and before `fetch('/api/research')`.
+- **New** (optional): `app/components/KnowledgePanel.tsx` — Extract the Context panel into a component that uses the knowledge store and exposes “items” + “retrievedContext” for the parent.
+- **Edit**: [app/page.tsx](app/page.tsx) — Wire file drop/note/URL to the knowledge store; before runResearch, call retrieval and send `retrievedContext` (and `contextUrls`) in the request body; add remove button; hydrate from IndexedDB.
+- **Edit**: [app/api/research/route.ts](app/api/research/route.ts) — Read `retrievedContext` / `contextUrls`; pass to runResearch.
+- **Edit**: [lib/agent/state.ts](lib/agent/state.ts) — Add `knowledgeContext` (and optionally `contextUrls`); extend `createInitialState`.
+- **Edit**: [lib/agent/graph.ts](lib/agent/graph.ts) — Add channel(s); pass context into `createInitialState` in runResearch.
+- **Edit**: [lib/agent/nodes/thinker.ts](lib/agent/nodes/thinker.ts) — Include `state.knowledgeContext` (and URLs) in the Thinker prompt.
+- **Edit**: [lib/agent/nodes/synthesizer.ts](lib/agent/nodes/synthesizer.ts) — Include `state.knowledgeContext` in the synthesis prompt.
+- **Edit**: [lib/agent/index.ts](lib/agent/index.ts) — Export any new types if needed; ensure runResearch signature is updated and documented.
+
+## 6. Optional follow-ups
+
+- **URL content in browser**: Proxy fetch in an API route; extract text; chunk and embed like PDFs so URL content is fully in the vector store.
+- **Delete-all / export**: Clear all knowledge from IndexedDB; export documents/chunks as JSON.
+- **Progress UX**: Show “Extracting…”, “Embedding…”, “Indexed” per file so large PDFs don’t look stuck.
+

.dockerignore

@@ -0,0 +1,30 @@
+# Dependencies (reinstalled in image)
+node_modules
+npm-debug.log*
+.pnpm-debug.log*
+.yarn*
+
+# Build outputs (rebuilt in image)
+.next
+dist
+out
+
+# Git and IDE
+.git
+.gitignore
+.cursor
+.vscode
+*.md
+!README.md
+
+# Env and secrets
+.env
+.env.*
+!.env.example
+
+# Misc
+.DS_Store
+*.log
+coverage
+.nyc_output
+.hf-cache

.gitignore

@@ -0,0 +1,47 @@
+# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
+
+# dependencies
+/node_modules
+/.pnp
+.pnp.*
+.yarn/*
+!.yarn/patches
+!.yarn/plugins
+!.yarn/releases
+!.yarn/versions
+
+# testing
+/coverage
+
+# next.js
+/.next/
+/out/
+
+# production
+/build
+
+# misc
+.DS_Store
+*.pem
+
+# debug
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+.pnpm-debug.log*
+
+# env files (can opt-in for committing if needed)
+.env*
+
+# vercel
+.vercel
+
+# typescript
+*.tsbuildinfo
+next-env.d.ts
+
+# huggingface model cache
+.hf-cache/
+
+# HF Spaces reject binary files in git; keep .pptx out of history
+docs/*.pptx

AGENTS.md

@@ -0,0 +1,16 @@
+# DeepTrust — Agent context
+
+**What this is:** TypeScript research agent (LangGraph state graph + local LLM + Next.js). Plan → audit → tools → synthesize. Real-time workspace UI with SSE streaming and context/knowledge upload.
+
+**Where the truth lives:** Structure and features change over time. Prefer these for current details:
+- **README.md** — Overview, stack, project structure, API, frontend, run/config.
+- **docs/ARCHITECTURE.md** — Low-level design: state machine, nodes, routing, LLM layer, SSE protocol, frontend architecture.
+
+**Rough layout (may evolve):**
+- `lib/agent/` — Graph, state, nodes, LLM client (worker thread), utils.
+- `app/` — Next.js app; `page.tsx` = workspace UI; `app/api/` = research + model endpoints.
+- `dist/llm/` — Built worker (from `npm run build:worker`).
+
+**Commands:** `npm run build:worker` before first run; `npm run dev` to develop; `npm run build` then `npm run start` for production.
+
+**When editing:** Follow existing patterns in the area you change. For agent graph or state, check `lib/agent/state.ts` and `lib/agent/graph.ts`; for API/streaming, see the route handlers and README/ARCHITECTURE.

DEPLOY.md

@@ -0,0 +1,47 @@
+# Deploying DeepTrust on Render
+
+The app is containerized with **Docker**. You do **not** need Docker installed locally—Render builds the image in the cloud when you push.
+
+## Prerequisites
+
+- Git repo pushed to GitHub (or GitLab)
+- Account on [Render](https://render.com)
+
+## Deploy steps
+
+1. Go to [dashboard.render.com](https://dashboard.render.com), connect your GitHub repo.
+2. Add a **Web Service**.
+3. Set:
+   - **Build:** Docker (Render will use the repo `Dockerfile`).
+   - **Plan:** See [Memory and plan](#memory-and-plan) below.
+4. Create the service. Render builds and deploys on each push.
+
+**Blueprint:** Alternatively, use the repo’s `render.yaml` (Blueprint deploy) so the service is defined in code.
+
+## Memory and plan
+
+Render **Free** instances have **512 MB RAM**. Model loading often exceeds this:
+
+- **SmolLM2-360M (q4)** ≈ 300–450 MB for weights alone, plus Node, Next.js, worker, and inference → typically **OOM on Free** (502 when loading).
+- **SmolLM2-135M (q4)** ≈ 150–200 MB → may work on Free but is tight; 502 can still occur under load.
+
+**Recommendations:**
+
+- **Free tier (512 MB):** Set env `HF_MODEL=HuggingFaceTB/SmolLM2-135M-Instruct` and use the 135M model (q4) in the UI. If you still get 502 on “Load model”, the instance is likely OOM — upgrade to Starter.
+- **Starter (2 GB) or higher:** Safe for SmolLM2-360M (q4) and 135M. Use Starter if you need 360M or reliable inference.
+
+## Environment variables (optional)
+
+- `HF_MODEL` — Hugging Face model id (default in code). On Free tier, use `HuggingFaceTB/SmolLM2-135M-Instruct` to reduce OOM risk.
+- `HF_CACHE_DIR` — Where to cache model files (default `./.hf-cache`). On ephemeral disks this is lost between restarts; models re-download on cold start.
+- `DEEPTRUST_LOAD_MODEL_AT_STARTUP` — Set to `0` or `false` to disable loading the model when the server starts (default: load at startup). Use this only if you prefer to load on first user request.
+
+## Hugging Face Spaces (Docker)
+
+When running as a **Docker Space**, the app **skips startup model preload** so the Space becomes healthy within the 30‑minute launch timeout. You’ll see in the logs: `[instrumentation] Hugging Face Space detected; skipping startup model preload (model will load on first use).` The model loads on first use (e.g. when you click “Load model” or run research). If your Space still times out, set the variable `DEEPTRUST_LOAD_MODEL_AT_STARTUP=0` in the Space’s **Settings → Variables**. In the Space README YAML you can set `app_port: 3000` to match the Dockerfile.
+
+## Notes
+
+- **Model at startup:** The app loads the default model (from `HF_MODEL`, or 135M q4) when the Node server starts, so the first user request does not wait for download/load. (On Hugging Face Spaces this is skipped so the Space stays healthy.) If preload fails (e.g. OOM), the server still starts and the UI "Load model" button can be used to retry.
+- **First deploy/cold start** can be slow while the model downloads and loads during server boot; subsequent requests are fast.
+- **Standalone:** The image uses Next.js `output: "standalone"` for a smaller build and includes the compiled LLM worker under `dist/llm/`.

Dockerfile

@@ -0,0 +1,53 @@
+# DeepTrust — multi-stage build for Render
+# Build runs in the cloud; no local Docker required.
+# Uses node:20-slim (glibc) so onnxruntime-node and tokenizers native bindings work.
+# Alpine (musl) causes 502 on model load due to incompatible native modules.
+
+FROM node:20-slim AS deps
+WORKDIR /app
+COPY package.json package-lock.json* ./
+RUN npm ci
+
+FROM node:20-slim AS builder
+WORKDIR /app
+COPY --from=deps /app/node_modules ./node_modules
+COPY . .
+
+# Build LLM worker (dist/llm/) then Next.js (standalone)
+ENV NEXT_TELEMETRY_DISABLED=1
+RUN npm run build
+
+# Pre-download the default model so the container doesn't fetch at runtime
+ENV HF_CACHE_DIR=/app/.hf-cache-build
+RUN mkdir -p /app/.hf-cache-build && node scripts/preload-hf-model.cjs
+
+FROM node:20-slim AS runner
+WORKDIR /app
+
+ENV NODE_ENV=production
+ENV NEXT_TELEMETRY_DISABLED=1
+
+RUN groupadd --gid 1001 nodejs && useradd --uid 1001 --gid nodejs --shell /bin/false nextjs
+
+# Standalone output: server + minimal node_modules
+COPY --from=builder /app/.next/standalone ./
+COPY --from=builder /app/.next/static ./.next/static
+COPY --from=builder /app/public ./public
+# LLM worker (required at runtime; path is process.cwd()/dist/llm)
+COPY --from=builder /app/dist ./dist
+# Worker runs in a separate thread and requires these at runtime; standalone does not trace them.
+COPY --from=builder /app/package.json ./package.json
+COPY --from=builder /app/package-lock.json ./package-lock.json
+RUN npm install @huggingface/transformers onnxruntime-node --omit=dev --ignore-scripts --no-save
+
+# Pre-downloaded model cache (avoids runtime download; worker reads from here)
+COPY --from=builder --chown=nextjs:nodejs /app/.hf-cache-build ./.hf-cache
+ENV HF_CACHE_DIR=/app/.hf-cache
+
+USER nextjs
+
+EXPOSE 3000
+ENV PORT=3000
+ENV HOSTNAME="0.0.0.0"
+
+CMD ["node", "server.js"]

README.md

@@ -0,0 +1,643 @@
+---
+title: DeepTrust Research Agent
+emoji: 🔬
+colorFrom: blue
+colorTo: indigo
+sdk: docker
+app_port: 3000
+pinned: false
+short_description: LangGraph research agent with local LLM and Next.js workspace.
+tags:
+  - langgraph
+  - nextjs
+  - transformers
+  - research-agent
+  - llm
+  - typescript
+---
+
+# DeepTrust Research Agent
+
+A TypeScript implementation of an autonomous research agent: LangGraph state machines, local LLM inference (Hugging Face Transformers in a worker thread), and a real-time, AI-centric Next.js workspace. The UI is designed for a Cursor/Gemini-like flow—immediate feedback, Server-Sent Events (SSE) streaming, optimistic updates, a knowledge/context drop zone, and quick-action chips—so the full application from graph nodes to the browser is understandable in one read.
+
+## Documentation
+
+- **[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)** — Low-level design: state machine, channels, routing, LLM layer, API protocols, and frontend architecture (SSE, streaming UX, knowledge flow).
+
+## Table of Contents
+
+1. [Architecture Overview](#architecture-overview)
+2. [Technology Stack](#technology-stack)
+3. [Project Structure](#project-structure)
+4. [State Management](#state-management)
+5. [Graph Nodes](#graph-nodes)
+6. [Routing and Conditional Edges](#routing-and-conditional-edges)
+7. [LLM Integration](#llm-integration)
+8. [API Layer](#api-layer)
+9. [Frontend: Real-Time Workspace](#frontend-real-time-workspace)
+10. [Running the Project](#running-the-project)
+11. [Configuration](#configuration)
+
+---
+
+## Architecture Overview
+
+DeepTrust implements a cyclic state graph where a research query flows through multiple specialized nodes:
+
+```
+┌──────────────────────────────────────────────────────────┐
+│                                                          │
+│   [START] ──► thinker ──► auditor ──► tool_executor      │
+│                  ▲           │              │            │
+│                  │    reject │              │            │
+│                  └───────────┘              ▼            │
+│                                       synthesizer        │
+│                                            │             │
+│                                         [END]            │
+└──────────────────────────────────────────────────────────┘
+```
+
+### Node Responsibilities
+
+- **Thinker**: Decomposes a research question into a structured, multi-step plan
+- **Auditor**: Validates the plan against organizational policy; rejects non-compliant plans
+- **HITL Gate**: Pauses execution for human approval before tool execution
+- **Tool Executor**: Executes each plan step sequentially (web search, document fetch, etc.)
+- **Synthesizer**: Aggregates tool outputs into a final research report
+
+The graph supports revision loops: if the Auditor rejects a plan, control returns to the Thinker with structured feedback. A configurable ceiling (`maxPlanRevisions`) prevents infinite loops.
+
+---
+
+## Technology Stack
+
+| Layer | Technology | Purpose |
+|-------|------------|---------|
+| **Frontend** | Next.js 16, React 19, Tailwind CSS 4 | Server-side rendering, streaming UI updates |
+| **State Machine** | LangGraph.js | Graph construction, checkpointing, conditional routing |
+| **LLM Inference** | @huggingface/transformers | Local model loading and text generation |
+| **Schema Validation** | Zod 4 | Runtime type validation for state and API contracts |
+| **Type System** | TypeScript 5 | Static type safety across the codebase |
+
+### Key Dependencies
+
+```json
+{
+  "@langchain/langgraph": "^1.1.5",
+  "@huggingface/transformers": "^3.8.1",
+  "zod": "^4.3.6",
+  "next": "16.1.6",
+  "react": "19.2.3"
+}
+```
+
+---
+
+## Project Structure
+
+```
+lib/agent/
+├── index.ts           # Public API exports
+├── graph.ts           # StateGraph construction and compilation
+├── state.ts           # Zod schemas and TypeScript types
+├── routing.ts         # Conditional edge functions
+├── llm/
+│   ├── index.ts       # Worker proxy: loadModel, chatComplete, getModelStatus
+│   ├── pipeline.ts    # Pipeline config (used by worker)
+│   └── worker-entry.ts # Worker entry: runs Transformers in a separate thread
+├── nodes/
+│   ├── index.ts       # Node exports
+│   ├── thinker.ts     # Plan generation node
+│   ├── auditor.ts     # Policy validation node
+│   ├── hitl-gate.ts   # Human approval checkpoint
+│   ├── tool-executor.ts # Tool dispatch node
+│   └── synthesizer.ts # Report synthesis node
+└── utils/
+    ├── index.ts       # Utility exports
+    ├── extract-json.ts # Robust JSON parsing
+    └── policy.ts      # Policy file loader
+
+dist/llm/              # Built by npm run build:worker
+├── worker-entry.js    # Compiled worker
+└── pipeline.js        # Pipeline bundle
+
+app/
+├── page.tsx           # Workspace UI: chat, context panel, model card, SSE client
+├── layout.tsx
+├── globals.css
+└── api/
+    ├── research/
+    │   └── route.ts   # POST: SSE stream of research events
+    └── model/
+        └── load/
+            └── route.ts # GET/POST: model load + SSE progress
+```
+
+---
+
+## State Management
+
+### The ResearchState Schema
+
+All state flows through a single Zod-validated schema. This ensures runtime type safety and enables serialization for checkpointing.
+
+```typescript
+export const ResearchState = z.object({
+  // Identity
+  threadId: z.string().uuid(),
+  sessionName: z.string().default("Unnamed Session"),
+
+  // Input
+  userQuery: z.string().min(1),
+
+  // Planning
+  plan: ResearchPlan.nullable().default(null),
+  rejectionFeedback: z.string().nullable().default(null),
+  planRevisionCount: z.number().int().nonnegative().default(0),
+  maxPlanRevisions: z.number().int().positive().default(5),
+
+  // Auditing
+  auditResult: AuditResult.nullable().default(null),
+
+  // Execution
+  currentStepIndex: z.number().int().nonnegative().default(0),
+  humanApproved: z.boolean().default(false),
+
+  // Output
+  finalReport: z.string().nullable().default(null),
+
+  // Observability
+  reasoning: z.array(ReasoningEntry).default([]),
+  status: RunStatus.default("idle"),
+  updatedAt: z.string().datetime(),
+  errorMessage: z.string().nullable().default(null),
+});
+```
+
+### Channel Configuration
+
+LangGraph requires explicit channel definitions for state merging. Most fields use last-write-wins semantics, but the `reasoning` array uses append-only concatenation:
+
+```typescript
+const graph = new StateGraph<ResearchState>({
+  channels: {
+    threadId: { value: (_, n) => n },
+    // ... other scalar fields use (_, n) => n
+
+    // Append-only reasoning log
+    reasoning: {
+      value: (existing: ReasoningEntry[], incoming: ReasoningEntry[]) =>
+        [...(existing ?? []), ...(incoming ?? [])],
+      default: () => [],
+    },
+  },
+});
+```
+
+### Sub-Schemas
+
+**ResearchStep**: A single action in the research plan.
+
+```typescript
+export const ResearchStep = z.object({
+  id: z.string().uuid(),
+  tool: z.enum(["web_search", "document_fetch", "code_interpreter", "summarize"]),
+  input: z.string().min(1),
+  rationale: z.string(),
+  output: z.string().optional(),
+  executedAt: z.string().datetime().optional(),
+});
+```
+
+**AuditResult**: Structured feedback from the Auditor.
+
+```typescript
+export const AuditResult = z.object({
+  verdict: z.enum(["approved", "rejected", "needs_revision"]),
+  policyViolations: z.array(z.string()).default([]),
+  suggestions: z.array(z.string()).default([]),
+  auditedAt: z.string().datetime(),
+});
+```
+
+---
+
+## Graph Nodes
+
+Each node is an async function that receives the current state and returns a partial state update.
+
+### Thinker Node
+
+Generates or revises a research plan. Prompts the LLM with structured output requirements:
+
+```typescript
+async function thinkerNode(state: ResearchState): Promise<Partial<ResearchState>> {
+  const isRevision = state.planRevisionCount > 0 && state.rejectionFeedback;
+
+  const system = `
+You are the Thinker node of DeepTrust, an autonomous research agent.
+Return ONLY a valid JSON object matching:
+{
+  "objective": string,
+  "steps": Array<{ "id": UUID, "tool": string, "input": string, "rationale": string }>,
+  "estimatedTokenBudget": number,
+  "createdAt": ISO8601,
+  "revision": number
+}`;
+
+  const userMessage = isRevision
+    ? `Research question: "${state.userQuery}"\n\nPREVIOUS PLAN REJECTED:\n${state.rejectionFeedback}`
+    : `Research question: "${state.userQuery}"`;
+
+  const rawThought = await chatComplete(system, userMessage);
+  const parsed = extractJSON(rawThought);
+  const plan = ResearchPlan.parse({ ...parsed, revision: state.planRevisionCount });
+
+  return {
+    plan,
+    status: "thinking",
+    rejectionFeedback: null,
+    reasoning: appendReasoning(state, { node: "thinker", summary: "..." }),
+    updatedAt: new Date().toISOString(),
+  };
+}
+```
+
+### Auditor Node
+
+Validates plans against `POLICY.md`. Returns structured violations and suggestions:
+
+```typescript
+async function auditorNode(state: ResearchState): Promise<Partial<ResearchState>> {
+  const policy = loadPolicy();
+
+  const system = `
+You are the Auditor node. Evaluate research plans against policy.
+Return ONLY: { "verdict": "approved"|"rejected"|"needs_revision", ... }`;
+
+  const rawThought = await chatComplete(system, `POLICY:\n${policy}\n\nPLAN:\n${JSON.stringify(state.plan)}`);
+  const auditResult = AuditResult.parse(extractJSON(rawThought));
+
+  const isRejected = auditResult.verdict !== "approved";
+
+  return {
+    auditResult,
+    rejectionFeedback: isRejected ? formatFeedback(auditResult) : null,
+    planRevisionCount: isRejected ? state.planRevisionCount + 1 : state.planRevisionCount,
+    status: isRejected ? "thinking" : "awaiting_approval",
+  };
+}
+```
+
+### HITL Gate Node
+
+Uses LangGraph's `interrupt()` primitive to pause execution and write a checkpoint:
+
+```typescript
+async function hitlGateNode(state: ResearchState): Promise<Partial<ResearchState>> {
+  interrupt({
+    message: "Plan ready for review. Set humanApproved=true to continue.",
+    plan: state.plan,
+    auditResult: state.auditResult,
+  });
+
+  return { updatedAt: new Date().toISOString() };
+}
+```
+
+### Tool Executor Node
+
+Iterates through plan steps. Each invocation processes one step and increments `currentStepIndex`:
+
+```typescript
+async function toolExecutorNode(state: ResearchState): Promise<Partial<ResearchState>> {
+  const step = state.plan.steps[state.currentStepIndex];
+  const output = await dispatchTool(step.tool, step.input);
+
+  const updatedSteps = state.plan.steps.map((s, i) =>
+    i === state.currentStepIndex ? { ...s, output, executedAt: new Date().toISOString() } : s
+  );
+
+  return {
+    plan: { ...state.plan, steps: updatedSteps },
+    currentStepIndex: state.currentStepIndex + 1,
+    status: "executing",
+  };
+}
+```
+
+### Synthesizer Node
+
+Aggregates all step outputs into a final report:
+
+```typescript
+async function synthesizerNode(state: ResearchState): Promise<Partial<ResearchState>> {
+  const stepsContext = state.plan.steps
+    .map((s, i) => `Step ${i + 1} [${s.tool}]: ${s.output}`)
+    .join("\n\n");
+
+  const finalReport = await chatComplete(
+    "Write a comprehensive research report.",
+    `Objective: ${state.plan.objective}\n\nResults:\n${stepsContext}`
+  );
+
+  return { finalReport, status: "complete" };
+}
+```
+
+---
+
+## Routing and Conditional Edges
+
+LangGraph uses routing functions to determine the next node based on current state.
+
+### Post-Audit Routing
+
+```typescript
+function routeAfterAudit(state: ResearchState): "thinker" | "hitl_gate" | typeof END {
+  if (state.planRevisionCount >= state.maxPlanRevisions) {
+    return END; // Safety ceiling reached
+  }
+  if (state.auditResult?.verdict !== "approved") {
+    return "thinker"; // Loop back for revision
+  }
+  return "hitl_gate"; // Proceed to human approval
+}
+```
+
+### Post-HITL Routing
+
+```typescript
+function routeAfterHitl(state: ResearchState): "tool_executor" | typeof END {
+  if (!state.humanApproved) {
+    return END; // Fail-safe if approval missing
+  }
+  return "tool_executor";
+}
+```
+
+### Post-Tool Routing
+
+```typescript
+function routeAfterToolStep(state: ResearchState): "tool_executor" | "synthesizer" {
+  if (state.currentStepIndex < state.plan.steps.length) {
+    return "tool_executor"; // More steps remain
+  }
+  return "synthesizer"; // All steps complete
+}
+```
+
+---
+
+## LLM Integration
+
+### Hugging Face Transformers
+
+The project uses `@huggingface/transformers` for local inference. Models are cached to `.hf-cache/` for persistence across restarts.
+
+```typescript
+import { pipeline, TextGenerationPipeline, env } from "@huggingface/transformers";
+
+env.cacheDir = process.env.HF_CACHE_DIR || "./.hf-cache";
+
+const MODEL_ID = process.env.HF_MODEL || "HuggingFaceTB/SmolLM2-360M-Instruct";
+
+let generatorPromise: Promise<TextGenerationPipeline> | null = null;
+
+export function loadModel(onProgress?: ProgressCallback): Promise<TextGenerationPipeline> {
+  if (generatorPromise) return generatorPromise;
+
+  generatorPromise = pipeline("text-generation", MODEL_ID, {
+    progress_callback: (data) => {
+      onProgress?.({
+        status: data.status === "progress" ? "downloading" : "loading",
+        progress: Math.round((data.progress || 0) * 100),
+        file: data.file || "",
+        message: `Downloading ${data.file?.split("/").pop()}`,
+      });
+    },
+  });
+
+  return generatorPromise;
+}
+```
+
+### Chat Completion Interface
+
+```typescript
+export async function chatComplete(systemPrompt: string, userMessage: string): Promise<string> {
+  const generator = await loadModel();
+
+  const output = await generator(
+    [
+      { role: "system", content: systemPrompt },
+      { role: "user", content: userMessage },
+    ],
+    { max_new_tokens: 4096, do_sample: true, temperature: 0.7 }
+  );
+
+  const result = output[0] as { generated_text: Array<{ role: string; content: string }> };
+  return result.generated_text.find((m) => m.role === "assistant")?.content ?? "";
+}
+```
+
+### JSON Extraction
+
+Small models often produce malformed JSON. The `extractJSON` utility handles common issues:
+
+```typescript
+export function extractJSON(text: string): unknown {
+  // Try direct parse
+  try { return JSON.parse(text); } catch {}
+
+  // Remove markdown fences
+  const cleaned = text.replace(/```json\s*/gi, "").replace(/```\s*/g, "");
+
+  // Extract JSON object
+  const match = cleaned.match(/\{[\s\S]*\}/);
+  if (match) {
+    try { return JSON.parse(match[0]); } catch {}
+  }
+
+  throw new Error(`Could not extract JSON from: ${text.slice(0, 200)}`);
+}
+```
+
+---
+
+## API Layer
+
+### Research Endpoint (SSE)
+
+`POST /api/research` streams research state updates as **Server-Sent Events** so the client can show progress immediately and parse events by type. Using SSE (instead of raw NDJSON) gives a standard, well-supported streaming protocol and allows future event names (e.g. `ping`, `heartbeat`) without changing the wire format.
+
+**Request body:** `{ "query": string, "knowledge"?: Array<{ id, type, label, meta? }> }`. The `knowledge` array is sent by the workspace when the user adds files, URLs, or notes in the Context panel; the backend currently uses only `query` and may later use `knowledge` for RAG or plan conditioning.
+
+**Response:** `Content-Type: text/event-stream`. Each message is an SSE message:
+
+- `event: start` — First event; signals that the run has started (enables optimistic UI).
+- `event: research` — One per graph node update; `data` is `{ node, state }`.
+- `event: error` — On exception; `data` includes `node: "_error"` and `state.errorMessage`.
+
+```typescript
+// Server: send helper
+const send = (event: string, payload: { node: string; state: Record<string, unknown> }) => {
+  controller.enqueue(encoder.encode(`event: ${event}\ndata: ${JSON.stringify(payload)}\n\n`));
+};
+send("start", { node: "_start", state: { status: "started", ... } });
+for await (const event of runResearch(query)) {
+  send("research", event);
+}
+// On catch: send("error", { node: "_error", state: { status: "failed", errorMessage } });
+```
+
+### Model Loading Endpoint
+
+`GET /api/model/load?...modelId=...&dtype=...` (or POST with same query) streams download/load progress via SSE. The client parses `data: {...}` lines to drive the progress bar and status pill. See [Frontend: Real-Time Workspace](#frontend-real-time-workspace) for how the UI consumes these streams.
+
+---
+
+## Frontend: Real-Time Workspace
+
+The React client (`app/page.tsx`) is a single-page workspace that mirrors a Cursor/Gemini-style flow: immediate, non-blocking feedback, streaming AI responses, and a dedicated context/knowledge area.
+
+### What the UI Provides
+
+| Area | Purpose |
+|------|--------|
+| **Chat** | User messages and assistant replies. When a run finishes, the final report is streamed **word-by-word** into the last assistant message to mimic a live conversation. |
+| **Context / Knowledge** | Drag-and-drop zone for PDFs, text files, or URLs; optional fields to add URLs and short notes. Stored in React state and sent as `knowledge` with each research request for future backend use. |
+| **Quick-action chips** | A row of buttons below the input (e.g. “Help me learn this topic”, “Summarize these docs”) that set or extend the query and trigger a run—Gemini-style shortcuts. |
+| **Starter cards** | Empty state with example prompts (e.g. “How does Gemini Pro work…”) that fill the input and can be run in one click. |
+| **Model card** | Compact panel for model selection, load/progress, and status (Ready / Loading / Error). |
+| **Reasoning trace** | Scrollable list of the latest reasoning entries from the event stream (node + summary) for observability. |
+
+### Optimistic UI and Shimmer
+
+- On “Run Research”, the UI immediately appends the user message and a placeholder assistant message with a shimmer skeleton, then consumes SSE and updates that message when the final report arrives.
+- Shimmer and loading states use Tailwind (e.g. `animate-pulse`, neutral backgrounds) so the interface feels responsive even when the agent is still planning or executing.
+
+### SSE Consumption (Research)
+
+The client uses `EventSource`-style parsing on the `ReadableStream`: split by `\n\n`, then for each line look for `event:` and `data:` and dispatch by event type. Accumulated `research` events update both the reasoning trace and the chat when `finalReport` is present.
+
+```typescript
+// Conceptual: read stream, split by double newline, parse "event:" and "data:"
+const chunks = buffer.split("\n\n");
+for (const chunk of chunks) {
+  const eventMatch = chunk.match(/event:\s*(\w+)/);
+  const dataMatch = chunk.match(/data:\s*(\{[\s\S]*\})/);
+  if (eventMatch && dataMatch) {
+    const payload = JSON.parse(dataMatch[1]);
+    if (eventMatch[1] === "research") setEvents((prev) => [...prev, payload]);
+    // ... handle start, error; when payload.state.finalReport exists, run word-by-word animation
+  }
+}
+```
+
+### Word-by-Word Streaming
+
+When an event contains `state.finalReport`, the full text is not dumped at once. A small timer (e.g. every 40ms) reveals the report word-by-word in the last assistant message and clears the “streaming” state when done. This keeps the same SSE event payload while making the reply feel live.
+
+---
+
+## Running the Project
+
+### Prerequisites
+
+- Node.js 20+
+- npm or pnpm
+
+### Installation
+
+```bash
+npm install
+```
+
+### Build the LLM worker (required for local inference)
+
+Inference runs in a Node.js worker thread. Compile the worker once before using the app locally:
+
+```bash
+npm run build:worker
+```
+
+This writes `dist/llm/worker-entry.js` and `dist/llm/pipeline.js`. The production build runs this step automatically.
+
+### Development
+
+```bash
+npm run dev
+```
+
+Open http://localhost:3000. Click "Load Model" to download and initialize the LLM, then run research queries.
+
+If you see an error that the worker was not found, run `npm run build:worker` first.
+
+### Production build
+
+```bash
+npm run build
+```
+
+This runs `build:worker` then builds the Next.js app with webpack. The app is served with:
+
+```bash
+npm run start
+```
+
+### First run
+
+The first model load downloads weights to `.hf-cache/` (approximately 400MB for SmolLM2-360M Q4). Subsequent loads are fast.
+
+### Deploy (Render)
+
+The repo includes a **Dockerfile** and config for [Render](https://render.com). Render builds the image in the cloud (no local Docker required). See **[DEPLOY.md](DEPLOY.md)** for steps.
+
+---
+
+## Configuration
+
+Create `.env.local` from `.env.example`:
+
+```bash
+# Model selection
+HF_MODEL=HuggingFaceTB/SmolLM2-360M-Instruct
+HF_CACHE_DIR=./.hf-cache
+```
+
+### Observability (LangSmith)
+
+To make agent behavior observable and debug failures (e.g. thinker returning invalid JSON), use [LangSmith](https://smith.langchain.com/). Set in `.env.local`:
+
+```bash
+LANGCHAIN_TRACING_V2=true
+LANGCHAIN_API_KEY=your-langsmith-api-key
+LANGCHAIN_PROJECT=deeptrust
+```
+
+With tracing enabled, every graph run is recorded. You can inspect prompts, raw LLM outputs, and state transitions in the LangSmith UI, which helps diagnose schema validation errors and long-running or looping runs.
+
+### Available Models
+
+| Model | Size | Speed | Quality |
+|-------|------|-------|---------|
+| `HuggingFaceTB/SmolLM2-360M-Instruct` | 400MB | Fast | Basic |
+| `HuggingFaceTB/SmolLM2-1.7B-Instruct` | 1.7GB | Moderate | Better |
+
+---
+
+## Key Concepts Demonstrated
+
+1. **State Machines for Agents**: Using LangGraph to model complex, cyclic agent workflows
+2. **Type-Safe State**: Zod schemas with TypeScript inference for runtime validation
+3. **Local LLM Inference**: Running models in-process without external API dependencies
+4. **Streaming Responses**: Server-Sent Events and ReadableStream for real-time updates
+5. **Human-in-the-Loop**: Checkpoint interrupts for manual approval gates
+6. **Revision Loops**: Cyclic graph edges for iterative refinement with safety ceilings
+
+---
+
+## License
+
+MIT

app/api/model/load/route.ts

@@ -0,0 +1,109 @@
+// Avoid loading @huggingface/transformers on Vercel (no native runtime / disk); use dynamic import only when needed.
+const isVercel = process.env.VERCEL === "1";
+
+function unsupportedStream(encoder: TextEncoder): ReadableStream<Uint8Array> {
+  const msg = {
+    status: "error",
+    progress: 0,
+    message:
+      "Model loading is not supported on Vercel (serverless). Run the app locally or self-host for local inference.",
+    models: [] as { id: string; label: string; dtype?: string; sizeNote?: string }[],
+  };
+  return new ReadableStream({
+    start(controller) {
+      controller.enqueue(encoder.encode(`data: ${JSON.stringify(msg)}\n\n`));
+      controller.close();
+    },
+  });
+}
+
+export async function GET(request: Request) {
+  const encoder = new TextEncoder();
+
+  if (isVercel) {
+    return new Response(unsupportedStream(encoder), {
+      headers: {
+        "Content-Type": "text/event-stream",
+        "Cache-Control": "no-cache",
+        Connection: "keep-alive",
+      },
+    });
+  }
+
+  const { loadModel, getModelStatus, MODELS } = await import("@/lib/agent/llm");
+  const { searchParams } = new URL(request.url);
+  const modelId = searchParams.get("modelId") ?? undefined;
+  const dtype = searchParams.get("dtype") ?? undefined;
+
+  const stream = new ReadableStream({
+    async start(controller) {
+      const send = (data: object) => {
+        controller.enqueue(encoder.encode(`data: ${JSON.stringify(data)}\n\n`));
+      };
+
+      const status = await getModelStatus(modelId, dtype);
+      const requestedId = modelId ?? status.modelId;
+      const requestedDtype = dtype ?? status.dtype;
+      if (
+        status.status === "ready" &&
+        status.modelId === requestedId &&
+        status.dtype === requestedDtype
+      ) {
+        send({ ...status, models: status.models ?? MODELS });
+        controller.close();
+        return;
+      }
+
+      send({
+        status: "loading",
+        progress: 0,
+        message: "Starting model load...",
+        modelId: modelId ?? status.modelId,
+        dtype: dtype ?? status.dtype,
+        models: status.models ?? MODELS,
+      });
+
+      try {
+        await loadModel(modelId, dtype as "q4" | "fp16" | "fp32" | undefined, (progress) => {
+          send({ ...progress, models: (progress as { models?: typeof MODELS }).models ?? MODELS });
+        });
+        controller.close();
+      } catch (error) {
+        send({
+          status: "error",
+          progress: 0,
+          message: error instanceof Error ? error.message : "Unknown error",
+          modelId: modelId ?? undefined,
+          dtype: dtype ?? undefined,
+          models: MODELS,
+        });
+        controller.close();
+      }
+    },
+  });
+
+  return new Response(stream, {
+    headers: {
+      "Content-Type": "text/event-stream",
+      "Cache-Control": "no-cache",
+      Connection: "keep-alive",
+    },
+  });
+}
+
+export async function POST() {
+  if (isVercel) {
+    return Response.json({
+      status: "error",
+      progress: 0,
+      file: "",
+      message:
+        "Model loading is not supported on Vercel (serverless). Run the app locally or self-host for local inference.",
+      models: [],
+    });
+  }
+
+  const { getModelStatus, MODELS } = await import("@/lib/agent/llm");
+  const status = await getModelStatus();
+  return Response.json({ ...status, models: status.models ?? MODELS });
+}

app/api/research/approve/route.ts

@@ -0,0 +1,55 @@
+import { approveAndResume } from "@/lib/agent";
+import { NextRequest } from "next/server";
+
+export async function POST(req: NextRequest) {
+  const body = await req.json();
+  const { threadId } = body as { threadId?: string };
+
+  if (!threadId) {
+    return new Response(
+      JSON.stringify({ error: "threadId is required to approve a run" }),
+      { status: 400, headers: { "Content-Type": "application/json" } }
+    );
+  }
+
+  const encoder = new TextEncoder();
+
+  const stream = new ReadableStream({
+    async start(controller) {
+      const send = (
+        event: string,
+        payload: { node: string; state: Record<string, unknown> }
+      ) => {
+        const data = `event: ${event}\ndata: ${JSON.stringify(payload)}\n\n`;
+        controller.enqueue(encoder.encode(data));
+      };
+
+      try {
+        for await (const event of approveAndResume(threadId)) {
+          send("research", event);
+        }
+      } catch (err) {
+        const message = err instanceof Error ? err.message : String(err);
+        send("error", {
+          node: "_error",
+          state: {
+            status: "failed",
+            errorMessage: message,
+          },
+        });
+      } finally {
+        controller.close();
+      }
+    },
+  });
+
+  return new Response(stream, {
+    headers: {
+      "Content-Type": "text/event-stream; charset=utf-8",
+      "Cache-Control": "no-cache, no-transform",
+      Connection: "keep-alive",
+      "X-Accel-Buffering": "no",
+    },
+  });
+}
+

app/api/research/route.ts

@@ -0,0 +1,80 @@
+import { runResearch } from "@/lib/agent";
+import { NextRequest } from "next/server";
+
+export async function POST(req: NextRequest) {
+  const body = await req.json();
+  const {
+    query,
+    retrievedContext,
+    contextUrls,
+  }: {
+    query: string;
+    retrievedContext?: string;
+    contextUrls?: string[];
+  } = body;
+
+  const encoder = new TextEncoder();
+
+  const stream = new ReadableStream({
+    async start(controller) {
+      const send = (
+        event: string,
+        payload: { node: string; state: Record<string, unknown> }
+      ) => {
+        const data = `event: ${event}\ndata: ${JSON.stringify(payload)}\n\n`;
+        controller.enqueue(encoder.encode(data));
+      };
+
+      // Initial "cursor-like" optimistic event
+      send("start", {
+        node: "_start",
+        state: {
+          status: "started",
+          plan: { objective: "Starting…", steps: [] },
+          reasoning: [
+            {
+              node: "_start",
+              summary:
+                "Research started. First step (planning) may take 1–2 minutes on slower devices.",
+            },
+          ],
+        },
+      });
+
+      try {
+        const options =
+          retrievedContext != null || (contextUrls?.length ?? 0) > 0
+            ? { knowledgeContext: retrievedContext ?? "", contextUrls: contextUrls ?? [] }
+            : undefined;
+        for await (const event of runResearch(query, "Research Session", options)) {
+          if (event.node === "__interrupt__") {
+            // Surface HITL interrupt to the client and close the stream.
+            send("hitl_waiting", event);
+            return;
+          }
+          send("research", event);
+        }
+      } catch (err) {
+        const message = err instanceof Error ? err.message : String(err);
+        send("error", {
+          node: "_error",
+          state: {
+            status: "failed",
+            errorMessage: message,
+          },
+        });
+      } finally {
+        controller.close();
+      }
+    },
+  });
+
+  return new Response(stream, {
+    headers: {
+      "Content-Type": "text/event-stream; charset=utf-8",
+      "Cache-Control": "no-cache, no-transform",
+      Connection: "keep-alive",
+      "X-Accel-Buffering": "no",
+    },
+  });
+}

app/globals.css

@@ -0,0 +1,26 @@
+@import "tailwindcss";
+
+:root {
+  --background: #ffffff;
+  --foreground: #171717;
+}
+
+@theme inline {
+  --color-background: var(--background);
+  --color-foreground: var(--foreground);
+  --font-sans: var(--font-geist-sans);
+  --font-mono: var(--font-geist-mono);
+}
+
+@media (prefers-color-scheme: dark) {
+  :root {
+    --background: #0a0a0a;
+    --foreground: #ededed;
+  }
+}
+
+body {
+  background: var(--background);
+  color: var(--foreground);
+  font-family: Arial, Helvetica, sans-serif;
+}

app/layout.tsx

@@ -0,0 +1,34 @@
+import type { Metadata } from "next";
+import { Geist, Geist_Mono } from "next/font/google";
+import "./globals.css";
+
+const geistSans = Geist({
+  variable: "--font-geist-sans",
+  subsets: ["latin"],
+});
+
+const geistMono = Geist_Mono({
+  variable: "--font-geist-mono",
+  subsets: ["latin"],
+});
+
+export const metadata: Metadata = {
+  title: "Create Next App",
+  description: "Generated by create next app",
+};
+
+export default function RootLayout({
+  children,
+}: Readonly<{
+  children: React.ReactNode;
+}>) {
+  return (
+    <html lang="en">
+      <body
+        className={`${geistSans.variable} ${geistMono.variable} antialiased`}
+      >
+        {children}
+      </body>
+    </html>
+  );
+}

app/page.tsx

@@ -0,0 +1,1120 @@
+"use client";
+
+import { useState, useCallback, useEffect, useRef } from "react";
+
+interface ModelProgress {
+  status: "idle" | "loading" | "downloading" | "ready" | "error";
+  progress: number;
+  file: string;
+  message: string;
+  modelId?: string;
+  dtype?: string;
+  models?: ModelOption[];
+}
+
+interface ModelOption {
+  id: string;
+  label: string;
+  dtype?: "q4" | "fp16" | "fp32";
+  sizeNote?: string;
+}
+
+interface ResearchEvent {
+  node: string;
+  state: {
+    threadId?: string;
+    interrupt?: unknown;
+    status?: string;
+    plan?: {
+      objective: string;
+      steps: Array<{ tool: string; input: string; rationale?: string }>;
+    };
+    auditResult?: {
+      verdict: "approved" | "rejected" | "needs_revision";
+      policyViolations?: string[];
+      suggestions?: string[];
+    };
+    finalReport?: string;
+    reasoning?: Array<{ node: string; summary: string }>;
+    errorMessage?: string;
+  };
+}
+
+type ChatRole = "user" | "assistant";
+
+interface ChatMessage {
+  id: string;
+  role: ChatRole;
+  content: string;
+  isStreaming?: boolean;
+}
+
+type KnowledgeItemType = "file" | "url" | "note";
+
+interface KnowledgeItem {
+  id: string;
+  type: KnowledgeItemType;
+  label: string;
+  meta?: string;
+  status?: "pending" | "indexing" | "indexed" | "error";
+}
+
+const DEFAULT_MODELS: ModelOption[] = [
+  {
+    id: "HuggingFaceTB/SmolLM2-135M-Instruct",
+    label: "SmolLM2 135M (Q4, tiny)",
+    dtype: "q4",
+    sizeNote: "~150–200 MB (approx)",
+  },
+  {
+    id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+    label: "SmolLM2 360M (Q4)",
+    dtype: "q4",
+    sizeNote: "~388 MB",
+  },
+  {
+    id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+    label: "SmolLM2 360M (FP16)",
+    dtype: "fp16",
+    sizeNote: "~725 MB",
+  },
+  {
+    id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+    label: "SmolLM2 360M (full)",
+    dtype: "fp32",
+    sizeNote: "~1.45 GB",
+  },
+];
+
+const QUICK_ACTIONS = [
+  "Create image-ready research brief",
+  "Help me learn this topic",
+  "Summarize these docs for me",
+  "Audit my assumptions",
+  "Turn this into an implementation plan",
+];
+
+const PREVIEW_QUERIES = [
+  "How does Gemini Pro work for deep research workflows?",
+  "Design a learning plan for mastering LangGraph in 30 days.",
+  "Compare self-hosted vs managed LLM stacks for a fintech startup.",
+];
+
+export default function DeepTrustWorkspace() {
+  const [modelStatus, setModelStatus] = useState<ModelProgress>({
+    status: "idle",
+    progress: 0,
+    file: "",
+    message: "Model not loaded",
+  });
+  const [selectedModelIndex, setSelectedModelIndex] = useState(0);
+  const models = modelStatus.models?.length ? modelStatus.models : DEFAULT_MODELS;
+  const selectedModel = models[selectedModelIndex] ?? null;
+
+  const [query, setQuery] = useState(PREVIEW_QUERIES[0]);
+  const [chat, setChat] = useState<ChatMessage[]>([]);
+  const [events, setEvents] = useState<ResearchEvent[]>([]);
+  const [hitlThreadId, setHitlThreadId] = useState<string | null>(null);
+  const [hitlPayload, setHitlPayload] = useState<unknown | null>(null);
+  const [error, setError] = useState<string | null>(null);
+  const [isStreaming, setIsStreaming] = useState(false);
+  const [knowledgeItems, setKnowledgeItems] = useState<KnowledgeItem[]>([]);
+  const [isDragOver, setIsDragOver] = useState(false);
+  const [noteDraft, setNoteDraft] = useState("");
+  const [urlDraft, setUrlDraft] = useState("");
+
+  const streamAbortRef = useRef<AbortController | null>(null);
+  const streamingTargetRef = useRef<string | null>(null);
+  const streamingTimerRef = useRef<number | null>(null);
+  const pendingFullTextRef = useRef<string | null>(null);
+
+  const chatScrollRef = useRef<HTMLDivElement | null>(null);
+
+  useEffect(() => {
+    fetch("/api/model/load", { method: "POST" })
+      .then((res) => res.json())
+      .then((data: ModelProgress) => {
+        setModelStatus((prev) => ({ ...prev, ...data }));
+        if (data.models?.length && selectedModelIndex >= data.models.length) {
+          setSelectedModelIndex(0);
+        }
+      })
+      .catch(() => {});
+    // eslint-disable-next-line react-hooks/exhaustive-deps
+  }, []);
+
+  const hydrateKnowledge = useCallback(() => {
+    import("@/lib/knowledge")
+      .then(({ listKnowledgeItems }) => listKnowledgeItems())
+      .then((items) => setKnowledgeItems(items))
+      .catch(() => {});
+  }, []);
+  useEffect(() => {
+    hydrateKnowledge();
+  }, [hydrateKnowledge]);
+
+  const startStreamingAnimation = useCallback((fullText: string, messageId: string) => {
+    if (!fullText) return;
+    if (streamingTimerRef.current) {
+      window.clearInterval(streamingTimerRef.current);
+    }
+
+    const words = fullText.split(/\s+/);
+    let index = 0;
+
+    streamingTargetRef.current = messageId;
+
+    setChat((prev) =>
+      prev.map((m) =>
+        m.id === messageId
+          ? {
+              ...m,
+              content: "",
+              isStreaming: true,
+            }
+          : m
+      )
+    );
+
+    const timer = window.setInterval(() => {
+      index += 1;
+      const nextContent = words.slice(0, index).join(" ");
+
+      setChat((prev) =>
+        prev.map((m) =>
+          m.id === messageId
+            ? {
+                ...m,
+                content: nextContent,
+              }
+            : m
+        )
+      );
+
+      if (index >= words.length) {
+        if (streamingTimerRef.current) {
+          window.clearInterval(streamingTimerRef.current);
+        }
+        streamingTimerRef.current = null;
+        pendingFullTextRef.current = null;
+        setChat((prev) =>
+          prev.map((m) =>
+            m.id === messageId
+              ? {
+                  ...m,
+                  isStreaming: false,
+                }
+              : m
+          )
+        );
+      }
+    }, 40);
+
+    streamingTimerRef.current = timer;
+  }, []);
+
+  useEffect(() => {
+    if (chatScrollRef.current) {
+      chatScrollRef.current.scrollTop = chatScrollRef.current.scrollHeight;
+    }
+  }, [chat]);
+
+  const isModelReady =
+    modelStatus.status === "ready" &&
+    selectedModel &&
+    modelStatus.modelId === selectedModel.id &&
+    modelStatus.dtype === selectedModel.dtype;
+
+  const isModelLoading =
+    modelStatus.status === "loading" || modelStatus.status === "downloading";
+
+  const loadModel = useCallback(async () => {
+    if (!selectedModel) return;
+    setModelStatus((prev) => ({ ...prev, status: "loading", message: "Starting..." }));
+
+    try {
+      const params = new URLSearchParams({ modelId: selectedModel.id });
+      if (selectedModel.dtype) params.set("dtype", selectedModel.dtype);
+      const response = await fetch(`/api/model/load?${params.toString()}`);
+      const reader = response.body?.getReader();
+      if (!reader) throw new Error("No response body");
+
+      const decoder = new TextDecoder();
+      let buffer = "";
+
+      while (true) {
+        const { done, value } = await reader.read();
+        if (done) break;
+
+        buffer += decoder.decode(value, { stream: true });
+        const chunks = buffer.split("\n\n");
+        buffer = chunks.pop() || "";
+
+        for (const chunk of chunks) {
+          if (chunk.startsWith("data: ")) {
+            try {
+              const data = JSON.parse(chunk.slice(6)) as ModelProgress;
+              setModelStatus((prev) => ({ ...prev, ...data }));
+            } catch {
+              // ignore malformed progress chunks
+            }
+          }
+        }
+      }
+    } catch (err) {
+      setModelStatus((prev) => ({
+        ...prev,
+        status: "error",
+        message: err instanceof Error ? err.message : "Failed to load model",
+      }));
+    }
+  }, [selectedModel]);
+
+  const registerKnowledgeFiles = useCallback((files: FileList | null) => {
+    if (!files || files.length === 0) return;
+    const pdfs = Array.from(files).filter((f) => f.type === "application/pdf");
+    import("@/lib/knowledge").then(({ addPdfFile }) => {
+      for (const file of pdfs) {
+        const tempId = `temp-${file.name}-${Date.now()}-${Math.random().toString(36).slice(2)}`;
+        setKnowledgeItems((prev) => [
+          ...prev,
+          { id: tempId, type: "file" as KnowledgeItemType, label: file.name, meta: "Indexing…", status: "indexing" as const },
+        ]);
+        addPdfFile(file)
+          .then((meta: KnowledgeItem) => {
+            setKnowledgeItems((prev) => prev.map((x) => (x.id === tempId ? { ...meta, status: "indexed" as const } : x)));
+          })
+          .catch(() => {
+            setKnowledgeItems((prev) => prev.map((x) => (x.id === tempId ? { ...x, status: "error" as const, meta: "Failed" } : x)));
+          });
+      }
+    });
+  }, []);
+
+  const handleDrop: React.DragEventHandler<HTMLDivElement> = (event) => {
+    event.preventDefault();
+    event.stopPropagation();
+    setIsDragOver(false);
+
+    if (event.dataTransfer.files?.length) registerKnowledgeFiles(event.dataTransfer.files);
+
+    const urlPayload =
+      event.dataTransfer.getData("text/uri-list") ||
+      event.dataTransfer.getData("text/plain");
+
+    if (urlPayload && /^https?:\/\//i.test(urlPayload.trim())) {
+      const url = urlPayload.trim();
+      import("@/lib/knowledge")
+        .then(({ addUrl }) => addUrl(url))
+        .then((meta) => setKnowledgeItems((prev) => [...prev, meta]))
+        .catch(() => {});
+    }
+  };
+
+  const handleDragOver: React.DragEventHandler<HTMLDivElement> = (event) => {
+    event.preventDefault();
+    event.stopPropagation();
+    setIsDragOver(true);
+  };
+
+  const handleDragLeave: React.DragEventHandler<HTMLDivElement> = (event) => {
+    event.preventDefault();
+    event.stopPropagation();
+    setIsDragOver(false);
+  };
+
+  const handleAddNote = () => {
+    const value = noteDraft.trim();
+    if (!value) return;
+    import("@/lib/knowledge")
+      .then(({ addNote }) => addNote(value))
+      .then((meta) => {
+        setKnowledgeItems((prev) => [...prev, meta]);
+        setNoteDraft("");
+      })
+      .catch(() => {});
+  };
+
+  const handleAddUrl = () => {
+    const value = urlDraft.trim();
+    if (!value) return;
+    import("@/lib/knowledge")
+      .then(({ addUrl }) => addUrl(value))
+      .then((meta) => {
+        setKnowledgeItems((prev) => [...prev, meta]);
+        setUrlDraft("");
+      })
+      .catch(() => {});
+  };
+
+  const handleRemoveKnowledgeItem = (id: string) => {
+    import("@/lib/knowledge")
+      .then(({ removeKnowledgeDocument }) => removeKnowledgeDocument(id))
+      .then(() => setKnowledgeItems((prev) => prev.filter((x) => x.id !== id)))
+      .catch(() => {});
+  };
+
+  const resetStreaming = () => {
+    if (streamAbortRef.current) {
+      streamAbortRef.current.abort();
+      streamAbortRef.current = null;
+    }
+    if (streamingTimerRef.current) {
+      window.clearInterval(streamingTimerRef.current);
+      streamingTimerRef.current = null;
+    }
+    pendingFullTextRef.current = null;
+    streamingTargetRef.current = null;
+    setIsStreaming(false);
+  };
+
+  const runResearch = useCallback(
+    async (promptOverride?: string) => {
+      const nextQuery = (promptOverride ?? query).trim();
+      if (!nextQuery || !isModelReady || isStreaming) return;
+
+      if (streamAbortRef.current) {
+        streamAbortRef.current.abort();
+      }
+
+      const controller = new AbortController();
+      streamAbortRef.current = controller;
+
+      setError(null);
+      setEvents([]);
+      setIsStreaming(true);
+
+      const userMessage: ChatMessage = {
+        id: `user-${Date.now()}`,
+        role: "user",
+        content: nextQuery,
+      };
+      const assistantMessage: ChatMessage = {
+        id: `assistant-${Date.now()}`,
+        role: "assistant",
+        content: "",
+        isStreaming: true,
+      };
+
+      setChat((prev) => [...prev, userMessage, assistantMessage]);
+
+      try {
+        let retrievedContext = "";
+        let contextUrls: string[] = [];
+        if (knowledgeItems.length > 0) {
+          const { retrieve } = await import("@/lib/knowledge");
+          const result = await retrieve(nextQuery);
+          retrievedContext = result.retrievedContext;
+          contextUrls = result.contextUrls;
+        }
+        const response = await fetch("/api/research", {
+          method: "POST",
+          headers: { "Content-Type": "application/json" },
+          body: JSON.stringify({
+            query: nextQuery,
+            retrievedContext,
+            contextUrls,
+          }),
+          signal: controller.signal,
+        });
+
+        if (!response.ok) {
+          throw new Error(`HTTP ${response.status}: ${response.statusText}`);
+        }
+
+        const reader = response.body?.getReader();
+        if (!reader) {
+          throw new Error("No response body");
+        }
+
+        const decoder = new TextDecoder();
+        let buffer = "";
+        let hadError = false;
+        const assistantId = assistantMessage.id;
+
+        while (true) {
+          const { done, value } = await reader.read();
+          if (done) break;
+
+          buffer += decoder.decode(value, { stream: true });
+          const eventsRaw = buffer.split("\n\n");
+          buffer = eventsRaw.pop() || "";
+
+          for (const raw of eventsRaw) {
+            const lines = raw.split("\n");
+            let eventType = "message";
+            let dataLine = "";
+            for (const line of lines) {
+              if (line.startsWith("event: ")) {
+                eventType = line.slice(7);
+              }
+              if (line.startsWith("data: ")) {
+                dataLine = line.slice(6);
+              }
+            }
+
+            if (!dataLine) continue;
+
+            try {
+              const parsed = JSON.parse(dataLine) as ResearchEvent;
+              if (eventType === "hitl_waiting" && parsed.node === "__interrupt__") {
+                setHitlThreadId(parsed.state.threadId ?? null);
+                setHitlPayload(parsed.state.interrupt ?? null);
+                setIsStreaming(false);
+                continue;
+              }
+
+              setEvents((prev) => [...prev, parsed]);
+
+              if (parsed.node === "_error" || parsed.state.status === "failed") {
+                const message =
+                  parsed.state.errorMessage ?? "Research failed. See server logs for details.";
+                setError(message);
+                hadError = true;
+                resetStreaming();
+              }
+
+              if (parsed.state.finalReport && !hadError) {
+                pendingFullTextRef.current = parsed.state.finalReport;
+              }
+
+              if (pendingFullTextRef.current && !streamingTimerRef.current) {
+                startStreamingAnimation(pendingFullTextRef.current, assistantId);
+              }
+            } catch (e) {
+              console.warn("Failed to parse SSE event:", e);
+            }
+          }
+        }
+
+        if (!hadError) {
+          setIsStreaming(false);
+        }
+      } catch (err) {
+        if ((err as Error).name === "AbortError") {
+          return;
+        }
+
+        setError(err instanceof Error ? err.message : "Unknown error");
+        resetStreaming();
+      }
+    },
+    [
+      knowledgeItems,
+      isModelReady,
+      isStreaming,
+      query,
+      startStreamingAnimation,
+      resetStreaming,
+    ]
+  );
+
+  const handleQuickAction = (template: string) => {
+    const composed = query ? `${query}\n\n${template}` : template;
+    setQuery(composed);
+    void runResearch(composed);
+  };
+
+  const handleSubmit: React.FormEventHandler<HTMLFormElement> = (event) => {
+    event.preventDefault();
+    void runResearch();
+  };
+
+  const isInputDisabled = !isModelReady || isStreaming;
+
+  const handleApprovePlan = useCallback(async () => {
+    if (!hitlThreadId) return;
+
+    setIsStreaming(true);
+    setError(null);
+
+    const assistantMessage: ChatMessage = {
+      id: `assistant-${Date.now()}`,
+      role: "assistant",
+      content: "",
+      isStreaming: true,
+    };
+    setChat((prev) => [...prev, assistantMessage]);
+
+    try {
+      const response = await fetch("/api/research/approve", {
+        method: "POST",
+        headers: { "Content-Type": "application/json" },
+        body: JSON.stringify({ threadId: hitlThreadId }),
+      });
+
+      if (!response.ok) {
+        throw new Error(`HTTP ${response.status}: ${response.statusText}`);
+      }
+
+      const reader = response.body?.getReader();
+      if (!reader) {
+        throw new Error("No response body");
+      }
+
+      const decoder = new TextDecoder();
+      let buffer = "";
+      let hadError = false;
+      const assistantId = assistantMessage.id;
+
+      while (true) {
+        const { done, value } = await reader.read();
+        if (done) break;
+
+        buffer += decoder.decode(value, { stream: true });
+        const eventsRaw = buffer.split("\n\n");
+        buffer = eventsRaw.pop() || "";
+
+        for (const raw of eventsRaw) {
+          const lines = raw.split("\n");
+          let eventType = "message";
+          let dataLine = "";
+          for (const line of lines) {
+            if (line.startsWith("event: ")) {
+              eventType = line.slice(7);
+            }
+            if (line.startsWith("data: ")) {
+              dataLine = line.slice(6);
+            }
+          }
+
+          if (!dataLine) continue;
+
+          try {
+            const parsed = JSON.parse(dataLine) as ResearchEvent;
+            if (eventType === "error" || parsed.node === "_error" || parsed.state.status === "failed") {
+              const message =
+                parsed.state.errorMessage ?? "Research failed after approval. See server logs for details.";
+              setError(message);
+              hadError = true;
+              resetStreaming();
+            }
+
+            setEvents((prev) => [...prev, parsed]);
+
+            if (parsed.state.finalReport && !hadError) {
+              pendingFullTextRef.current = parsed.state.finalReport;
+            }
+
+            if (pendingFullTextRef.current && !streamingTimerRef.current) {
+              startStreamingAnimation(pendingFullTextRef.current, assistantId);
+            }
+          } catch (e) {
+            console.warn("Failed to parse SSE event (approve):", e);
+          }
+        }
+      }
+
+      if (!hadError) {
+        setIsStreaming(false);
+      }
+    } catch (err) {
+      setError(err instanceof Error ? err.message : "Unknown error during approval");
+      resetStreaming();
+    } finally {
+      setHitlThreadId(null);
+      setHitlPayload(null);
+    }
+  }, [
+    hitlThreadId,
+    resetStreaming,
+    startStreamingAnimation,
+  ]);
+
+  const handleRejectPlan = useCallback(() => {
+    setHitlThreadId(null);
+    setHitlPayload(null);
+    setIsStreaming(false);
+    setChat((prev) => [
+      ...prev,
+      {
+        id: `assistant-${Date.now()}`,
+        role: "assistant",
+        content:
+          "The proposed plan was rejected. Please refine your question or constraints, then try again.",
+      },
+    ]);
+  }, []);
+
+  return (
+    <div className="min-h-screen bg-zinc-950 text-zinc-100 flex flex-col">
+      <main className="flex-1 flex flex-col lg:flex-row gap-6 px-4 sm:px-8 py-6 max-w-6xl mx-auto w-full">
+        <section className="flex-1 flex flex-col border border-zinc-900/80 bg-zinc-950/60 rounded-3xl shadow-[0_0_0_1px_rgba(255,255,255,0.03)]">
+          <header className="px-6 pt-5 pb-4 border-b border-zinc-900/80 flex items-center justify-between">
+            <div>
+              <p className="text-sm font-medium text-zinc-400">Hi there</p>
+              <h2 className="text-2xl sm:text-3xl font-semibold tracking-tight">
+                Where should we start?
+              </h2>
+            </div>
+            <div
+              className={`px-3 py-1 rounded-full text-xs font-medium border ${
+                isModelReady
+                  ? "border-emerald-500/40 bg-emerald-500/10 text-emerald-300"
+                  : isModelLoading
+                  ? "border-sky-500/40 bg-sky-500/10 text-sky-300"
+                  : modelStatus.status === "error"
+                  ? "border-red-500/40 bg-red-500/10 text-red-300"
+                  : "border-zinc-700 bg-zinc-900 text-zinc-400"
+              }`}
+            >
+              {isModelReady && "Model ready"}
+              {isModelLoading && "Loading model…"}
+              {modelStatus.status === "error" && "Model error"}
+              {modelStatus.status === "idle" && "Model not loaded"}
+            </div>
+          </header>
+
+          <div className="flex-1 flex flex-col">
+            <div
+              ref={chatScrollRef}
+              className="flex-1 px-6 py-4 space-y-4 overflow-y-auto scrollbar-thin scrollbar-thumb-zinc-800 scrollbar-track-transparent"
+            >
+              {chat.length === 0 && (
+                <div className="mt-4 grid gap-3 sm:grid-cols-2">
+                  {PREVIEW_QUERIES.map((example) => (
+                    <button
+                      key={example}
+                      type="button"
+                      disabled={!isModelReady}
+                      onClick={() => void runResearch(example)}
+                      className="group text-left rounded-2xl border border-zinc-800 bg-zinc-900/60 px-4 py-3 text-sm text-zinc-300 hover:border-zinc-500/80 hover:bg-zinc-900 transition-colors disabled:opacity-40 disabled:cursor-not-allowed"
+                    >
+                      <span className="block font-medium text-zinc-100 mb-1">
+                        Try this
+                      </span>
+                      <span className="block text-zinc-400 text-xs line-clamp-3">
+                        {example}
+                      </span>
+                    </button>
+                  ))}
+                </div>
+              )}
+
+              {chat.map((message) => (
+                <div
+                  key={message.id}
+                  className={`flex ${
+                    message.role === "user" ? "justify-end" : "justify-start"
+                  }`}
+                >
+                  <div
+                    className={`max-w-[80%] rounded-2xl px-4 py-2.5 text-sm leading-relaxed shadow-sm ${
+                      message.role === "user"
+                        ? "bg-zinc-100 text-zinc-900"
+                        : "bg-zinc-900 text-zinc-100 border border-zinc-800"
+                    }`}
+                  >
+                    {message.content || (message.isStreaming && (
+                      <div className="space-y-2">
+                        <div className="h-3 rounded-full bg-zinc-700/60 animate-pulse" />
+                        <div className="h-3 w-2/3 rounded-full bg-zinc-800/60 animate-pulse" />
+                      </div>
+                    ))}
+                  </div>
+                </div>
+              ))}
+            </div>
+
+            <div className="px-4 pb-4 pt-2 border-t border-zinc-900/80">
+              <form
+                onSubmit={handleSubmit}
+                className="flex flex-col gap-3 rounded-2xl bg-zinc-950/80 border border-zinc-800 px-3 pt-2.5 pb-3 shadow-[0_0_0_1px_rgba(255,255,255,0.03)]"
+              >
+                <textarea
+                  rows={2}
+                  value={query}
+                  onChange={(event) => setQuery(event.target.value)}
+                  disabled={isInputDisabled}
+                  placeholder="Ask DeepTrust anything about your code, docs, or ideas…"
+                  className="w-full resize-none bg-transparent text-sm outline-none placeholder:text-zinc-500 text-zinc-100"
+                />
+                <div className="flex items-center justify-between gap-2">
+                  <div className="flex flex-wrap gap-1.5">
+                    {QUICK_ACTIONS.map((action) => (
+                      <button
+                        key={action}
+                        type="button"
+                        disabled={isInputDisabled}
+                        onClick={() => handleQuickAction(action)}
+                        className="rounded-full border border-zinc-800 bg-zinc-900/70 px-3 py-1 text-[11px] text-zinc-300 hover:bg-zinc-800 hover:border-zinc-600 transition-colors disabled:opacity-40 disabled:cursor-not-allowed"
+                      >
+                        {action}
+                      </button>
+                    ))}
+                  </div>
+                  <div className="flex items-center gap-2">
+                    {isStreaming && (
+                      <button
+                        type="button"
+                        onClick={resetStreaming}
+                        className="text-xs text-zinc-400 hover:text-zinc-200 px-2 py-1 rounded-full border border-zinc-800"
+                      >
+                        Stop
+                      </button>
+                    )}
+                    <button
+                      type="submit"
+                      disabled={isInputDisabled}
+                      className="inline-flex items-center justify-center rounded-full bg-zinc-100 text-zinc-950 text-xs font-medium px-4 py-2 hover:bg-white transition-colors disabled:bg-zinc-700 disabled:text-zinc-300 disabled:cursor-not-allowed"
+                    >
+                      {isStreaming ? "Thinking…" : "Run"}
+                    </button>
+                  </div>
+                </div>
+              </form>
+              {error && (
+                <p className="mt-2 text-xs text-red-400 border border-red-900/70 bg-red-950/60 rounded-xl px-3 py-2">
+                  {error}
+                </p>
+              )}
+            </div>
+          </div>
+        </section>
+
+        <aside className="w-full lg:w-[320px] flex flex-col gap-4">
+          <div className="rounded-3xl border border-zinc-900 bg-zinc-950/70 p-4 space-y-3 shadow-[0_0_0_1px_rgba(255,255,255,0.02)]">
+            <div className="flex items-center justify-between">
+              <div>
+                <p className="text-xs uppercase tracking-wide text-zinc-500">
+                  Model
+                </p>
+                <p className="text-sm font-medium">
+                  {modelStatus.modelId || "Select a model"}
+                </p>
+              </div>
+              <span
+                className={`inline-flex items-center rounded-full px-2 py-0.5 text-[10px] font-medium border ${
+                  isModelReady
+                    ? "border-emerald-500/40 bg-emerald-500/10 text-emerald-300"
+                    : isModelLoading
+                    ? "border-sky-500/40 bg-sky-500/10 text-sky-300"
+                    : modelStatus.status === "error"
+                    ? "border-red-500/40 bg-red-500/10 text-red-300"
+                    : "border-zinc-700 bg-zinc-900 text-zinc-400"
+                }`}
+              >
+                {isModelReady && "Ready"}
+                {isModelLoading && "Loading…"}
+                {modelStatus.status === "error" && "Error"}
+                {modelStatus.status === "idle" && "Not loaded"}
+              </span>
+            </div>
+
+            {models.length > 0 && (
+              <div>
+                <select
+                  value={selectedModelIndex}
+                  onChange={(event) => setSelectedModelIndex(Number(event.target.value))}
+                  disabled={isModelLoading}
+                  className="w-full mt-1 rounded-xl border border-zinc-800 bg-zinc-950 px-3 py-2 text-xs text-zinc-200 focus:outline-none focus:ring-1 focus:ring-zinc-500 disabled:opacity-50 disabled:cursor-not-allowed"
+                >
+                  {models.map((model, index) => (
+                    <option
+                      key={`${model.id}-${model.dtype ?? "default"}`}
+                      value={index}
+                    >
+                      {model.label} {model.sizeNote ? `— ${model.sizeNote}` : ""}
+                    </option>
+                  ))}
+                </select>
+              </div>
+            )}
+
+            {isModelLoading && (
+              <div className="space-y-1.5">
+                <div className="flex justify-between text-[11px] text-zinc-400">
+                  <span>{modelStatus.message}</span>
+                  <span>{modelStatus.progress}%</span>
+                </div>
+                <div className="h-1.5 rounded-full bg-zinc-900 overflow-hidden">
+                  <div
+                    className="h-full bg-sky-500 transition-all duration-300"
+                    style={{ width: `${modelStatus.progress}%` }}
+                  />
+                </div>
+              </div>
+            )}
+
+            {!isModelReady && !isModelLoading && (
+              <button
+                type="button"
+                onClick={loadModel}
+                className="w-full inline-flex items-center justify-center text-xs font-medium rounded-xl bg-zinc-100 text-zinc-950 py-2.5 hover:bg-white transition-colors"
+              >
+                Load model
+              </button>
+            )}
+
+            {modelStatus.status === "error" && (
+              <button
+                type="button"
+                onClick={loadModel}
+                className="w-full text-[11px] text-red-300 underline underline-offset-2 hover:no-underline text-left"
+              >
+                Retry loading
+              </button>
+            )}
+          </div>
+
+          <div
+            onDrop={handleDrop}
+            onDragOver={handleDragOver}
+            onDragLeave={handleDragLeave}
+            className={`rounded-3xl border-2 border-dashed px-4 py-4 space-y-3 transition-colors cursor-pointer ${
+              isDragOver
+                ? "border-zinc-200 bg-zinc-900/80"
+                : "border-zinc-700/80 bg-zinc-950/60"
+            }`}
+          >
+            <div className="flex items-center justify-between gap-2">
+              <div>
+                <p className="text-xs uppercase tracking-wide text-zinc-500">
+                  Context
+                </p>
+                <p className="text-sm font-medium">
+                  Drop PDFs, notes, or URLs
+                </p>
+              </div>
+            </div>
+            <p className="text-xs text-zinc-500">
+              Drag files or links here. DeepTrust will use them as additional
+              knowledge when answering.
+            </p>
+
+            <div className="flex gap-2">
+              <label className="inline-flex cursor-pointer rounded-xl border border-zinc-700 bg-zinc-900/70 px-3 py-1.5 text-[11px] text-zinc-200 hover:bg-zinc-800 transition-colors">
+                Attach files
+                <input
+                  type="file"
+                  accept="application/pdf"
+                  multiple
+                  className="hidden"
+                  onChange={(event) => registerKnowledgeFiles(event.target.files)}
+                />
+              </label>
+            </div>
+
+            <div className="space-y-2 pt-1">
+              <div className="flex gap-2">
+                <input
+                  type="text"
+                  value={urlDraft}
+                  onChange={(event) => setUrlDraft(event.target.value)}
+                  placeholder="Paste URL"
+                  className="flex-1 rounded-xl bg-zinc-900/70 border border-zinc-800 px-3 py-1.5 text-xs text-zinc-100 placeholder:text-zinc-500 focus:outline-none focus:ring-1 focus:ring-zinc-500"
+                />
+                <button
+                  type="button"
+                  onClick={handleAddUrl}
+                  className="rounded-xl border border-zinc-800 bg-zinc-900/80 px-2.5 py-1 text-[11px] text-zinc-200 hover:bg-zinc-800"
+                >
+                  Add
+                </button>
+              </div>
+              <div className="flex gap-2">
+                <input
+                  type="text"
+                  value={noteDraft}
+                  onChange={(event) => setNoteDraft(event.target.value)}
+                  placeholder="Short note or hint"
+                  className="flex-1 rounded-xl bg-zinc-900/70 border border-zinc-800 px-3 py-1.5 text-xs text-zinc-100 placeholder:text-zinc-500 focus:outline-none focus:ring-1 focus:ring-zinc-500"
+                />
+                <button
+                  type="button"
+                  onClick={handleAddNote}
+                  className="rounded-xl border border-zinc-800 bg-zinc-900/80 px-2.5 py-1 text-[11px] text-zinc-200 hover:bg-zinc-800"
+                >
+                  Save
+                </button>
+              </div>
+            </div>
+
+            {knowledgeItems.length > 0 && (
+              <div className="max-h-40 overflow-y-auto space-y-1 pt-1">
+                {knowledgeItems.map((item) => (
+                  <div
+                    key={item.id}
+                    className="flex items-center justify-between gap-2 rounded-xl bg-zinc-900/70 border border-zinc-800 px-3 py-1.5 text-[11px] text-zinc-200"
+                  >
+                    <span className="truncate min-w-0">
+                      {item.type === "file" && "📄 "}
+                      {item.type === "url" && "🔗 "}
+                      {item.type === "note" && "✏️ "}
+                      {item.label}
+                    </span>
+                    <span className="flex items-center gap-1.5 shrink-0">
+                      {item.meta && (
+                        <span className="text-[10px] text-zinc-500">
+                          {item.meta}
+                        </span>
+                      )}
+                      <button
+                        type="button"
+                        onClick={() => handleRemoveKnowledgeItem(item.id)}
+                        className="rounded p-0.5 text-zinc-500 hover:text-zinc-300 hover:bg-zinc-800 transition-colors"
+                        aria-label="Remove"
+                      >
+                        ×
+                      </button>
+                    </span>
+                  </div>
+                ))}
+              </div>
+            )}
+          </div>
+
+          {events.length > 0 && (
+            <div className="space-y-3">
+              {/* HITL approval banner */}
+              {hitlThreadId && (
+                <div className="rounded-3xl border border-amber-500/40 bg-amber-500/5 p-3 space-y-1.5">
+                  <p className="text-xs font-medium text-amber-200">
+                    Human review required
+                  </p>
+                  <p className="text-[11px] text-amber-100/90">
+                    The agent has prepared a plan and is waiting for your approval before executing tools.
+                  </p>
+                  <div className="flex items-center gap-2 pt-1">
+                    <button
+                      type="button"
+                      onClick={handleApprovePlan}
+                      className="px-3 py-1.5 rounded-full text-[11px] font-medium bg-emerald-500 text-emerald-950 hover:bg-emerald-400 transition-colors"
+                    >
+                      Approve & run tools
+                    </button>
+                    <button
+                      type="button"
+                      onClick={handleRejectPlan}
+                      className="px-3 py-1.5 rounded-full text-[11px] font-medium border border-amber-400/60 text-amber-100 hover:bg-amber-500/10 transition-colors"
+                    >
+                      Reject plan
+                    </button>
+                  </div>
+                </div>
+              )}
+
+              {/* Plan & audit snapshot */}
+              {(() => {
+                const latestWithPlan = [...events]
+                  .reverse()
+                  .find((e) => e.state.plan);
+                const latestWithAudit = [...events]
+                  .reverse()
+                  .find((e) => e.state.auditResult);
+                if (!latestWithPlan && !latestWithAudit) return null;
+
+                const plan = latestWithPlan?.state.plan;
+                const audit = latestWithAudit?.state.auditResult;
+
+                return (
+                  <div className="rounded-3xl border border-zinc-900 bg-zinc-950/70 p-3 space-y-2">
+                    <div className="flex items-center justify-between gap-2">
+                      <p className="text-xs font-medium text-zinc-400">
+                        Plan & audit
+                      </p>
+                      {audit && (
+                        <span
+                          className={`px-2 py-0.5 rounded-full text-[10px] font-medium ${
+                            audit.verdict === "approved"
+                              ? "bg-emerald-500/10 text-emerald-300 border border-emerald-500/30"
+                              : audit.verdict === "rejected"
+                              ? "bg-red-500/10 text-red-300 border border-red-500/30"
+                              : "bg-amber-500/10 text-amber-300 border border-amber-500/30"
+                          }`}
+                        >
+                          Audit: {audit.verdict}
+                        </span>
+                      )}
+                    </div>
+                    {plan && (
+                      <div className="space-y-1.5">
+                        <p className="text-[11px] text-zinc-300">
+                          {plan.objective}
+                        </p>
+                        <ol className="space-y-1 max-h-24 overflow-y-auto">
+                          {plan.steps.map((step, idx) => (
+                            <li
+                              key={`${step.tool}-${idx}`}
+                              className="text-[11px] text-zinc-400 flex gap-1.5"
+                            >
+                              <span className="mt-0.5 text-zinc-500">
+                                {idx + 1}.
+                              </span>
+                              <span className="flex-1">
+                                <span className="font-mono text-[10px] uppercase tracking-wide text-zinc-500 mr-1">
+                                  {step.tool}
+                                </span>
+                                {step.input}
+                              </span>
+                            </li>
+                          ))}
+                        </ol>
+                      </div>
+                    )}
+                    {audit?.policyViolations &&
+                      audit.policyViolations.length > 0 && (
+                        <div className="pt-1 border-t border-zinc-800/80 mt-1">
+                          <p className="text-[10px] font-medium text-red-300 mb-0.5">
+                            Policy flags
+                          </p>
+                          <ul className="space-y-0.5">
+                            {audit.policyViolations.map((v, i) => (
+                              <li
+                                key={`${v}-${i}`}
+                                className="text-[10px] text-red-200/90"
+                              >
+                                • {v}
+                              </li>
+                            ))}
+                          </ul>
+                        </div>
+                      )}
+                  </div>
+                );
+              })()}
+
+              {/* Reasoning trace */}
+              <div className="rounded-3xl border border-zinc-900 bg-zinc-950/70 p-3 space-y-1.5">
+                <p className="text-xs font-medium text-zinc-400 mb-1">
+                  Reasoning trace
+                </p>
+                <div className="max-h-40 overflow-y-auto space-y-1.5">
+                  {events
+                    .filter((event) => event.state.reasoning?.length)
+                    .map((event, index) => {
+                      const latest =
+                        event.state.reasoning?.[
+                          event.state.reasoning.length - 1
+                        ];
+                      if (!latest) return null;
+                      return (
+                        <div
+                          key={`${event.node}-${index}`}
+                          className="rounded-2xl bg-zinc-900/80 border border-zinc-800 px-3 py-1.5"
+                        >
+                          <div className="flex items-center justify-between mb-0.5">
+                            <span className="text-[10px] font-mono text-zinc-500">
+                              {event.node}
+                            </span>
+                            {event.state.status && (
+                              <span className="text-[10px] text-zinc-500">
+                                {event.state.status}
+                              </span>
+                            )}
+                          </div>
+                          <p className="text-[11px] text-zinc-300 line-clamp-3">
+                            {latest.summary}
+                          </p>
+                        </div>
+                      );
+                    })}
+                </div>
+              </div>
+            </div>
+          )}
+        </aside>
+      </main>
+    </div>
+  );
+}

dist/llm/pipeline.js

@@ -0,0 +1,195 @@
+"use strict";
+/**
+ * Pipeline — runs inside the LLM worker thread only.
+ * Hugging Face Transformers + onnxruntime-node; state lives in this thread.
+ */
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.MODELS = exports.MODEL_ID = void 0;
+exports.getModelStatus = getModelStatus;
+exports.loadModel = loadModel;
+exports.chatComplete = chatComplete;
+const transformers_1 = require("@huggingface/transformers");
+const node_path_1 = __importDefault(require("node:path"));
+transformers_1.env.cacheDir =
+    process.env.HF_CACHE_DIR ||
+        node_path_1.default.join(process.cwd(), ".hf-cache");
+// Default to a very small instruct model for low-spec machines.
+// If this model ID does not exist or you prefer another tiny model,
+// set HF_MODEL in your environment to override.
+exports.MODEL_ID = process.env.HF_MODEL || "HuggingFaceTB/SmolLM2-135M-Instruct";
+exports.MODELS = [
+    {
+        id: "HuggingFaceTB/SmolLM2-135M-Instruct",
+        label: "SmolLM2 135M (Q4, tiny)",
+        dtype: "q4",
+        sizeNote: "~150–200 MB (approx)",
+    },
+    {
+        id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+        label: "SmolLM2 360M (Q4)",
+        dtype: "q4",
+        sizeNote: "~388 MB",
+    },
+    {
+        id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+        label: "SmolLM2 360M (FP16)",
+        dtype: "fp16",
+        sizeNote: "~725 MB",
+    },
+    {
+        id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+        label: "SmolLM2 360M (full)",
+        dtype: "fp32",
+        sizeNote: "~1.45 GB",
+    },
+];
+let currentModelId = exports.MODELS[0].id;
+let currentDtype = exports.MODELS[0].dtype;
+let generatorPromise = null;
+let isModelLoaded = false;
+let currentProgress = 0;
+let currentStatus = "idle";
+let currentFile = "";
+function getModelStatus(forModelId, forDtype) {
+    const isOther = forModelId !== undefined && (forModelId !== currentModelId || forDtype !== currentDtype);
+    if (isOther) {
+        return {
+            status: "idle",
+            progress: 0,
+            file: "",
+            message: "Model not loaded",
+            modelId: forModelId,
+            dtype: forDtype,
+        };
+    }
+    if (isModelLoaded) {
+        return {
+            status: "ready",
+            progress: 100,
+            file: "",
+            message: "Model ready",
+            modelId: currentModelId,
+            dtype: currentDtype,
+        };
+    }
+    if (generatorPromise) {
+        return {
+            status: currentStatus,
+            progress: currentProgress,
+            file: currentFile,
+            message: currentFile ? `Downloading ${currentFile}` : "Loading model...",
+            modelId: currentModelId,
+            dtype: currentDtype,
+        };
+    }
+    return {
+        status: "idle",
+        progress: 0,
+        file: "",
+        message: "Model not loaded",
+        modelId: currentModelId,
+        dtype: currentDtype,
+    };
+}
+function loadModel(modelId, dtype, onProgress) {
+    console.log("[worker] loadModel called");
+    const nextId = modelId ?? currentModelId;
+    const nextDtype = dtype ?? currentDtype;
+    if (nextId !== currentModelId || nextDtype !== currentDtype) {
+        generatorPromise = null;
+        isModelLoaded = false;
+        currentModelId = nextId;
+        currentDtype = nextDtype;
+        currentStatus = "idle";
+        currentProgress = 0;
+        currentFile = "";
+    }
+    if (generatorPromise) {
+        return generatorPromise;
+    }
+    console.log(`\n🔄 [worker] Loading model: ${currentModelId}${currentDtype ? ` (${currentDtype})` : ""}`);
+    console.log(`   Cache directory: ${transformers_1.env.cacheDir}\n`);
+    currentStatus = "loading";
+    const startTime = Date.now();
+    const pipelineOptions = {
+        progress_callback: (progressData) => {
+            currentStatus = progressData.status === "progress" ? "downloading" : "loading";
+            currentFile = progressData.file || progressData.name || "";
+            currentProgress = Math.round(progressData.progress ?? 0);
+            const update = {
+                status: currentStatus,
+                progress: currentProgress,
+                file: currentFile,
+                message: currentFile
+                    ? `Downloading ${currentFile.split("/").pop()} (${currentProgress}%)`
+                    : `${progressData.status}...`,
+                modelId: currentModelId,
+                dtype: currentDtype,
+            };
+            console.log(`   ${update.message}`);
+            onProgress?.(update);
+        },
+    };
+    if (currentDtype) {
+        pipelineOptions.dtype = currentDtype;
+    }
+    const pipelinePromise = (0, transformers_1.pipeline)("text-generation", currentModelId, pipelineOptions);
+    generatorPromise = pipelinePromise
+        .then((gen) => {
+        const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
+        console.log(`\n✅ [worker] Model loaded in ${elapsed}s\n`);
+        isModelLoaded = true;
+        currentStatus = "ready";
+        currentProgress = 100;
+        onProgress?.({
+            status: "ready",
+            progress: 100,
+            file: "",
+            message: "Model ready",
+            modelId: currentModelId,
+            dtype: currentDtype,
+        });
+        return gen;
+    })
+        .catch((err) => {
+        currentStatus = "error";
+        generatorPromise = null;
+        onProgress?.({
+            status: "error",
+            progress: 0,
+            file: "",
+            message: err.message,
+            modelId: currentModelId,
+            dtype: currentDtype,
+        });
+        throw err;
+    });
+    return generatorPromise;
+}
+async function chatComplete(systemPrompt, userMessage) {
+    console.log("[worker] chatComplete called");
+    const generator = await loadModel();
+    const messages = [
+        { role: "system", content: systemPrompt },
+        { role: "user", content: userMessage },
+    ];
+    const preview = userMessage.slice(0, 60).replace(/\n/g, " ");
+    console.log(`🤖 [worker] Generating response for: "${preview}..."`);
+    const startTime = Date.now();
+    const output = await generator(messages, {
+        max_new_tokens: 512,
+        do_sample: true,
+        temperature: 0.7,
+    });
+    const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
+    const result = output[0];
+    const assistantMessage = result.generated_text.find((msg) => msg.role === "assistant");
+    if (!assistantMessage) {
+        throw new Error("No assistant response generated");
+    }
+    console.log(`✅ [worker] Generated ${assistantMessage.content.length} chars in ${elapsed}s`);
+    return assistantMessage.content;
+}

dist/llm/worker-entry.js

@@ -0,0 +1,47 @@
+"use strict";
+/**
+ * Worker entry — runs in a Node worker_thread. Handles getStatus, load, chat and forwards progress.
+ */
+Object.defineProperty(exports, "__esModule", { value: true });
+const node_worker_threads_1 = require("node:worker_threads");
+const pipeline_1 = require("./pipeline");
+function reply(msg) {
+    node_worker_threads_1.parentPort?.postMessage(msg);
+}
+node_worker_threads_1.parentPort?.on("message", (msg) => {
+    const { id, type, payload } = msg;
+    (async () => {
+        try {
+            switch (type) {
+                case "getStatus": {
+                    const status = (0, pipeline_1.getModelStatus)(payload.modelId, payload.dtype);
+                    reply({ id, type: "resolve", payload: { ...status, models: pipeline_1.MODELS } });
+                    break;
+                }
+                case "load": {
+                    await (0, pipeline_1.loadModel)(payload.modelId, payload.dtype, (progress) => {
+                        reply({
+                            id,
+                            type: "progress",
+                            payload: { ...progress, models: pipeline_1.MODELS },
+                        });
+                    });
+                    const status = (0, pipeline_1.getModelStatus)(payload.modelId, payload.dtype);
+                    reply({ id, type: "resolve", payload: { ...status, models: pipeline_1.MODELS } });
+                    break;
+                }
+                case "chat": {
+                    const text = await (0, pipeline_1.chatComplete)(payload.systemPrompt, payload.userMessage);
+                    reply({ id, type: "resolve", payload: text });
+                    break;
+                }
+                default:
+                    reply({ id, type: "reject", payload: `Unknown message type: ${msg.type}` });
+            }
+        }
+        catch (err) {
+            const message = err instanceof Error ? err.message : String(err);
+            reply({ id, type: "reject", payload: message });
+        }
+    })();
+});

docs/ARCHITECTURE.md

@@ -0,0 +1,503 @@
+# DeepTrust Architecture
+
+This document provides a detailed technical overview of the DeepTrust Research Agent architecture, covering the state machine design, data flow, and implementation patterns.
+
+## System Overview
+
+DeepTrust is a research automation system that orchestrates an LLM through a multi-stage workflow. The system decomposes research questions into executable plans, validates them against policy, executes tool calls, and synthesizes results into reports.
+
+```
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                         FRONTEND (app/page.tsx)                             │
+│  ┌─────────────┐  ┌─────────────────────────────────────────────────────┐  │
+│  │ Model Card  │  │  Chat + Context panel                                 │  │
+│  │ (SSE load)  │  │  • Messages (user / assistant with word-by-word)    │  │
+│  └──────┬──────┘  │  • Knowledge drop zone (files, URLs, notes)          │  │
+│         │         │  • Quick-action chips, preview prompts               │  │
+│         │         │  • Reasoning trace (node summaries)                   │  │
+│         │         └──────────────────────────┬──────────────────────────┘  │
+└─────────┼───────────────────────────────────┼─────────────────────────────┘
+          │                                     │
+          ▼                                     ▼
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                              API LAYER                                      │
+│  ┌─────────────────────┐  ┌─────────────────────────────────────┐          │
+│  │ GET/POST             │  │ POST /api/research                   │          │
+│  │ /api/model/load      │  │ (SSE: event + data per research step)│          │
+│  │ (SSE progress)       │  └─────────────────┬───────────────────┘          │
+│  └─────────┬───────────┘                      │                             │
+└────────────┼─────────────────────────────────┼─────────────────────────────┘
+             │                                 │
+             ▼                                 ▼
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                              AGENT CORE (lib/agent)                         │
+│  ┌─────────────────┐  ┌─────────────────────────────────────────────────┐  │
+│  │   LLM Client     │  │              StateGraph                         │  │
+│  │ (worker thread)  │  │  ┌─────────┐  ┌─────────┐  ┌──────────────┐    │  │
+│  │  loadModel()     │  │  │ Thinker │──│ Auditor │──│ Tool Executor │    │  │
+│  │  chatComplete()  │◄─┼──└────▲────┘  └────┬────┘  └──────┬───────┘    │  │
+│  └─────────────────┘  │       │            │               │            │  │
+│                        │       └────────────┘               ▼            │  │
+│                        │                        ┌───────────────┐       │  │
+│                        │                        │  Synthesizer  │       │  │
+│                        │                        └───────────────���       │  │
+│                        └─────────────────────────────────────────────────┘  │
+└─────────────────────────────────────────────────────────────────────────────┘
+```
+
+## State Machine Design
+
+### LangGraph StateGraph
+
+The agent is implemented as a LangGraph `StateGraph<ResearchState>`. LangGraph provides:
+
+1. **Typed State Channels**: Each state field has a merge strategy
+2. **Conditional Routing**: Functions determine the next node based on state
+3. **Checkpointing**: State snapshots enable pause/resume workflows
+4. **Streaming**: Events are yielded as nodes complete
+
+### State Flow
+
+```
+Initial State
+     │
+     ▼
+┌─────────┐
+│ thinker │ ◄──────────────────────────────┐
+└────┬────┘                                │
+     │ produces plan                       │
+     ▼                                     │
+┌─────────┐                                │
+│ auditor │                                │
+└────┬────┘                                │
+     │                                     │
+     ├── verdict: "rejected" ──────────────┘
+     │   (planRevisionCount++)              
+     │                                      
+     ├── planRevisionCount >= max ──────► END
+     │                                      
+     └── verdict: "approved"
+         │
+         ▼
+   ┌───────────┐
+   │ hitl_gate │ ◄─── interrupt() pauses here
+   └─────┬─────┘
+         │ humanApproved = true (external)
+         ▼
+  ┌───────────────┐
+  │ tool_executor │ ◄─────────────┐
+  └───────┬───────┘               │
+          │                       │
+          ├── more steps ─────────┘
+          │                        
+          └── all steps done
+              │
+              ▼
+       ┌─────────────┐
+       │ synthesizer │
+       └──────┬──────┘
+              │
+              ▼
+             END
+```
+
+### Channel Merge Strategies
+
+LangGraph channels define how incoming state updates merge with existing state:
+
+| Field | Strategy | Rationale |
+|-------|----------|-----------|
+| `reasoning` | Append | Accumulates full reasoning trace |
+| All others | Replace | Last-write-wins for scalar values |
+
+```typescript
+channels: {
+  reasoning: {
+    value: (existing, incoming) => [...(existing ?? []), ...(incoming ?? [])],
+    default: () => [],
+  },
+  plan: { value: (_, n) => n },
+  // ...
+}
+```
+
+## Data Schemas
+
+### Type System Philosophy
+
+All data structures use Zod for runtime validation. TypeScript types are inferred from Zod schemas, ensuring a single source of truth.
+
+```typescript
+// Schema definition
+export const ResearchStep = z.object({
+  id: z.string().uuid(),
+  tool: z.enum(["web_search", "document_fetch", "code_interpreter", "summarize"]),
+  input: z.string().min(1),
+  rationale: z.string(),
+  output: z.string().optional(),
+});
+
+// Type inference (no manual duplication)
+export type ResearchStep = z.infer<typeof ResearchStep>;
+```
+
+### Schema Hierarchy
+
+```
+ResearchState (root)
+├── threadId: UUID
+├── userQuery: string
+├── plan: ResearchPlan | null
+│   ├── objective: string
+│   ├── steps: ResearchStep[]
+│   │   ├── id: UUID
+│   │   ├── tool: enum
+│   │   ├── input: string
+│   │   ├── rationale: string
+│   │   └── output?: string
+│   ├── estimatedTokenBudget: number
+│   └── revision: number
+├── auditResult: AuditResult | null
+│   ├── verdict: enum
+│   ├── policyViolations: string[]
+│   └── suggestions: string[]
+├── reasoning: ReasoningEntry[]
+│   ├── node: enum
+│   ├── timestamp: datetime
+│   ├── summary: string
+│   └── rawThought?: string
+└── status: enum
+```
+
+## Node Implementation Patterns
+
+### Node Function Signature
+
+All nodes follow the same pattern:
+
+```typescript
+async function nodeName(state: ResearchState): Promise<Partial<ResearchState>> {
+  // 1. Read required state
+  // 2. Perform computation (LLM calls, tool execution, etc.)
+  // 3. Return partial state update
+}
+```
+
+LangGraph merges the returned partial state into the existing state using channel strategies.
+
+### Prompt Engineering Pattern
+
+Each LLM-calling node structures prompts for reliable JSON output:
+
+```typescript
+const system = `
+You are the [Role] node of DeepTrust.
+[Brief description of responsibility]
+
+Return ONLY a valid JSON object matching:
+{
+  "field1": type,
+  "field2": type
+}
+
+Rules:
+- [Constraint 1]
+- [Constraint 2]
+- Do not include markdown fences or prose outside JSON.
+`;
+
+const userMessage = `[Contextual input]`;
+const raw = await chatComplete(system, userMessage);
+const parsed = extractJSON(raw);
+const validated = Schema.parse(parsed);
+```
+
+### Error Handling Pattern
+
+Nodes append to the reasoning trace even on failure, enabling debugging:
+
+```typescript
+async function node(state: ResearchState) {
+  try {
+    // ... main logic
+    return { /* success state */ };
+  } catch (error) {
+    const reasoning = appendReasoning(state, {
+      node: "node_name",
+      summary: `Error: ${error.message}`,
+    });
+    return {
+      status: "failed",
+      errorMessage: error.message,
+      reasoning,
+    };
+  }
+}
+```
+
+## Routing Logic
+
+### Conditional Edges
+
+LangGraph `addConditionalEdges` accepts a router function that returns the next node name:
+
+```typescript
+graph.addConditionalEdges("auditor", routeAfterAudit, {
+  thinker: "thinker",
+  hitl_gate: "hitl_gate",
+  [END]: END,
+});
+```
+
+The mapping object defines legal transitions. If the router returns a key not in the map, LangGraph throws an error.
+
+### Router Functions
+
+Routers are pure functions that inspect state:
+
+```typescript
+function routeAfterAudit(state: ResearchState): "thinker" | "hitl_gate" | typeof END {
+  // Safety ceiling check
+  if (state.planRevisionCount >= state.maxPlanRevisions) {
+    return END;
+  }
+
+  // Rejection triggers revision
+  if (state.auditResult?.verdict !== "approved") {
+    return "thinker";
+  }
+
+  // Approval proceeds to HITL
+  return "hitl_gate";
+}
+```
+
+## LLM Integration Layer
+
+### Architecture
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                     llm/index.ts                            │
+├─────────────────────────────────────────────────────────────┤
+│  Module-level State                                         │
+│  ├── generatorPromise: Promise<TextGenerationPipeline>      │
+│  ├── isModelLoaded: boolean                                 │
+│  └── currentProgress: number                                │
+├─────────────────────────────────────────────────────────────┤
+│  Exports                                                    │
+│  ├── loadModel(onProgress?) → Promise<Pipeline>             │
+│  ├── chatComplete(system, user) → Promise<string>           │
+│  ├── getModelStatus() → ModelProgress                       │
+│  └── MODEL_ID: string                                       │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Lazy Loading Pattern
+
+The model is loaded once and reused across all requests:
+
+```typescript
+let generatorPromise: Promise<TextGenerationPipeline> | null = null;
+
+export function loadModel(): Promise<TextGenerationPipeline> {
+  if (generatorPromise) return generatorPromise; // Return cached promise
+
+  generatorPromise = pipeline("text-generation", MODEL_ID, {
+    progress_callback: handleProgress,
+  });
+
+  return generatorPromise;
+}
+```
+
+### Progress Streaming
+
+The Hugging Face Transformers library supports progress callbacks:
+
+```typescript
+pipeline("text-generation", MODEL_ID, {
+  progress_callback: (data: { status: string; file?: string; progress?: number }) => {
+    // data.status: "download", "progress", "ready"
+    // data.file: current file being downloaded
+    // data.progress: 0-1 fraction
+  },
+});
+```
+
+## API Design
+
+### Research Streaming: Server-Sent Events (SSE)
+
+Research results are streamed as **Server-Sent Events** so the client can distinguish event types and get low-latency, non-blocking updates. The response uses `Content-Type: text/event-stream` and each message has an `event` name plus a `data` payload (JSON).
+
+**Why SSE (not NDJSON):** Standard SSE gives a single, well-understood protocol for streaming; event names (`start`, `research`, `error`) allow the client to handle each kind of message without guessing. Proxies and browsers handle SSE well, and we can add `ping`/`heartbeat` later without changing the wire format.
+
+**Wire format:**
+
+```
+event: start
+data: {"node":"_start","state":{"status":"started","plan":{...},"reasoning":[...]}}
+
+event: research
+data: {"node":"thinker","state":{"status":"thinking","plan":{...}}}
+
+event: research
+data: {"node":"auditor","state":{"status":"awaiting_approval","auditResult":{...}}}
+
+event: research
+data: {"node":"synthesizer","state":{"status":"complete","finalReport":"..."}}
+
+event: error
+data: {"node":"_error","state":{"status":"failed","errorMessage":"..."}}
+```
+
+**Server (route):** The route encodes each event with `event: <name>\ndata: <JSON>\n\n` and enqueues into a `ReadableStream`, then closes the stream when the graph run finishes or throws.
+
+**Client:** The client reads the response body with `getReader()`, accumulates chunks, splits on `\n\n` to get full SSE messages, then for each message parses the `event:` line and the `data:` line (JSON). Events of type `research` (and `start`) are appended to the events list; the last event carrying `finalReport` is used to drive the word-by-word streaming animation in the chat.
+
+### Model Loading Protocol
+
+Model load also uses SSE for download progress:
+
+```
+data: {"status":"downloading","progress":25,"file":"model.onnx"}\n\n
+data: {"status":"downloading","progress":50,"file":"model.onnx"}\n\n
+data: {"status":"ready","progress":100,"modelId":"...","dtype":"q4"}\n\n
+```
+
+SSE format: `data: ` prefix, JSON body, double newline (`\n\n`) between events.
+
+## Frontend Architecture
+
+The workspace (`app/page.tsx`) is built for a Cursor/Gemini-like flow: immediate feedback, non-blocking streaming, and clear separation between chat, context, and observability.
+
+### Layout and Responsibilities
+
+| Area | Purpose |
+|------|--------|
+| **Chat** | User messages and assistant replies. Assistant messages show a shimmer placeholder while waiting, then the final report is revealed word-by-word for a live-conversation feel. |
+| **Context panel** | Knowledge drop zone: drag-and-drop files (PDF, text, etc.) or add URLs/notes. Items are listed and sent as `knowledge` in the research request for future agent use. |
+| **Model card** | Model selection, load/progress, status pill (Ready / Loading / Error). Uses the same SSE pattern as model load API. |
+| **Reasoning trace** | Scrollable list of the latest reasoning summaries per node so you can follow the graph’s flow while the chat shows the final answer. |
+
+### Optimistic UI and Streaming Flow
+
+1. **On submit:** The client immediately appends a user message and an assistant placeholder (with shimmer) to the chat and sets `isStreaming = true`. No wait for the first byte.
+2. **SSE consumption:** `POST /api/research` is read with `response.body.getReader()`. Chunks are decoded and split on `\n\n`. Each SSE message is parsed for `event:` and `data:`; `start` and `research` events are appended to the events list.
+3. **Final report:** When an event contains `state.finalReport`, that text is stored and a word-by-word animation is started for the latest assistant message: a timer (e.g. every 40ms) reveals the next word until the full report is shown, then `isStreaming` is cleared.
+4. **Abort/cleanup:** A ref holds an `AbortController` for the in-flight request; starting a new run aborts the previous one and clears the streaming timer so only one “live” reply runs at a time.
+
+### Knowledge / Context Flow
+
+- **Drop zone:** Accepts drag-and-drop files and paste/drop of URLs (`text/uri-list` or plain text). Files and URLs are turned into `KnowledgeItem` entries (id, type, label, optional meta).
+- **Request payload:** The research request sends `{ query, knowledge: knowledgeItems }`. The backend currently uses only `query`; `knowledge` is reserved for future use (e.g. RAG, plan conditioning).
+- **UI copy:** The panel explains that added context can be used to ground answers; when the agent supports it, no frontend change is required beyond the existing payload.
+
+### Quick Actions and Starter Cards
+
+- **Quick-action chips** below the input (e.g. “Help me learn this topic”, “Summarize these docs”) set or extend the query and optionally trigger a run, similar to Gemini suggestion chips.
+- **Starter cards** in the empty state show example prompts (e.g. “How does Gemini Pro work…”) and populate the input or start a run when clicked, so the app feels ready to use without typing.
+
+### Why This Structure
+
+- **Single page:** All controls (model, context, chat, trace) stay on one screen to reduce context switching and match a “flow state” tool.
+- **SSE end-to-end:** Both research and model load use SSE so the client has one mental model: stream events, parse by type, update UI.
+- **Word-by-word:** The synthesizer returns the full report in one event; animating it word-by-word on the client gives a streaming feel without changing the backend contract.
+
+## File Organization
+
+### Separation of Concerns
+
+| Directory | Responsibility |
+|-----------|----------------|
+| `lib/agent/nodes/` | Individual node implementations |
+| `lib/agent/llm/` | LLM client abstraction |
+| `lib/agent/utils/` | Shared utilities (JSON extraction, policy loading) |
+| `lib/agent/` | Graph construction, state schemas, routing |
+| `app/api/` | HTTP endpoints |
+| `app/` | React UI components |
+
+### Import Hierarchy
+
+```
+app/api/research/route.ts
+└── @/lib/agent (public API)
+    └── graph.ts
+        ├── state.ts
+        ├── nodes/index.ts
+        │   ├── thinker.ts → llm, utils, state
+        │   ├── auditor.ts → llm, utils, state
+        │   └── ...
+        └── routing.ts → state
+```
+
+## Checkpointing and Persistence
+
+### MemorySaver (Development)
+
+Default checkpointer stores state in memory. State is lost on server restart.
+
+```typescript
+import { MemorySaver } from "@langchain/langgraph";
+const checkpointer = new MemorySaver();
+```
+
+### Production Persistence
+
+For production, swap to a persistent checkpointer:
+
+```typescript
+import { PostgresSaver } from "@langchain/langgraph-checkpoint-postgres";
+const checkpointer = await PostgresSaver.fromConnString(process.env.DATABASE_URL);
+```
+
+### Thread-Based State
+
+Each research session has a unique `threadId`. The checkpointer keys state by thread:
+
+```typescript
+const config = { configurable: { thread_id: initialState.threadId } };
+
+// Stream with checkpointing
+for await (const event of graph.stream(initialState, config)) { ... }
+
+// Resume from checkpoint
+await graph.updateState(config, { humanApproved: true });
+for await (const event of graph.stream(null, config)) { ... }
+```
+
+## Security Considerations
+
+### Policy Enforcement
+
+The Auditor node validates plans against `POLICY.md` before execution. Policy rules should cover:
+
+- Data access restrictions
+- External request limits
+- Allowed tool types
+- Content guidelines
+
+### Tool Sandboxing
+
+Tool implementations (currently stubs) should sandbox external operations:
+
+- Network requests: Rate limiting, allowlists
+- Code execution: Containerized environments
+- File access: Scoped to specific directories
+
+### Input Validation
+
+All state mutations pass through Zod schemas, preventing malformed data from propagating.
+
+---
+
+## Future Considerations
+
+### Potential Enhancements
+
+1. **Persistent Checkpointing**: PostgreSQL or Redis for production state storage
+2. **Tool Implementations**: Real web search (Tavily), document fetch (Playwright), code execution
+3. **Multi-Model Support**: Router to select appropriate model per task complexity
+4. **Observability**: OpenTelemetry traces for node-level metrics
+5. **Parallel Tool Execution**: Execute independent steps concurrently

eslint.config.mjs

@@ -0,0 +1,18 @@
+import { defineConfig, globalIgnores } from "eslint/config";
+import nextVitals from "eslint-config-next/core-web-vitals";
+import nextTs from "eslint-config-next/typescript";
+
+const eslintConfig = defineConfig([
+  ...nextVitals,
+  ...nextTs,
+  // Override default ignores of eslint-config-next.
+  globalIgnores([
+    // Default ignores of eslint-config-next:
+    ".next/**",
+    "out/**",
+    "build/**",
+    "next-env.d.ts",
+  ]),
+]);
+
+export default eslintConfig;

instrumentation.ts

@@ -0,0 +1,56 @@
+/**
+ * Next.js instrumentation — runs once when the Node server starts.
+ * Preloads the LLM model so the first request doesn't wait for download + load.
+ * Skip on Vercel (no local inference). Disable with DEEPTRUST_LOAD_MODEL_AT_STARTUP=0.
+ * Skip on Hugging Face Spaces so the Space becomes healthy immediately (model loads on first use).
+ *
+ * Logs progress from the worker so long-running downloads/loads are visible in Space logs.
+ * After PRELOAD_TIMEOUT_MS (default 10 min) logs a timeout message if still loading.
+ */
+
+const PRELOAD_TIMEOUT_MS = Number(process.env.DEEPTRUST_PRELOAD_TIMEOUT_MS) || 10 * 60 * 1000;
+
+/** True when running inside a Hugging Face Space (avoids startup preload so health check passes). */
+function isHuggingFaceSpace(): boolean {
+  if (process.env.SPACE_ID || process.env.HF_SPACE_ID) return true;
+  if (process.env.PORT === "7860") return true; // Spaces default Docker app_port
+  return false;
+}
+
+export async function register() {
+  if (process.env.NEXT_RUNTIME !== "nodejs") return;
+  if (process.env.VERCEL === "1") return;
+  if (process.env.DEEPTRUST_LOAD_MODEL_AT_STARTUP === "0" || process.env.DEEPTRUST_LOAD_MODEL_AT_STARTUP === "false") return;
+  if (isHuggingFaceSpace()) {
+    console.log("[instrumentation] Hugging Face Space detected; skipping startup model preload (model will load on first use).");
+    return;
+  }
+
+  const { loadModel } = await import("@/lib/agent/llm");
+  const modelId = process.env.HF_MODEL || "HuggingFaceTB/SmolLM2-135M-Instruct";
+  console.log(`[instrumentation] Preloading model in background: ${modelId}`);
+
+  let completed = false;
+  const timeoutId = setTimeout(() => {
+    if (!completed) {
+      console.log(
+        `[instrumentation] Model preload still in progress (timeout after ${PRELOAD_TIMEOUT_MS / 60000} min). Check worker logs for download/load progress.`
+      );
+    }
+  }, PRELOAD_TIMEOUT_MS);
+
+  const onProgress = (p: { status: string; message: string; file?: string; progress?: number }) => {
+    console.log(`[instrumentation] ${p.message}${p.progress != null ? ` ${p.progress}%` : ""}`);
+  };
+
+  void loadModel(undefined, "q4", onProgress)
+    .then(() => {
+      completed = true;
+      console.log("[instrumentation] Model preload complete.");
+    })
+    .catch((err) => {
+      completed = true;
+      console.error("[instrumentation] Model preload failed:", err instanceof Error ? err.message : err);
+    })
+    .finally(() => clearTimeout(timeoutId));
+}

lib/agent/POLICY.md

@@ -0,0 +1,4 @@
+# Research Policy
+- Do not access private or personal data.
+- Do not make more than 10 external requests per session.
+- All sources must be publicly accessible.

lib/agent/graph.ts

@@ -0,0 +1,217 @@
+/**
+ * graph.ts — DeepTrust Research Agent
+ *
+ * Defines the full StateGraph lifecycle:
+ *
+ *   ┌──────────────────────────────────────────────────────────┐
+ *   │                                                          │
+ *   │   [START] ──► thinker ──► auditor ──► tool_executor      │
+ *   │                  ▲           │              │            │
+ *   │                  │    reject │              │            │
+ *   │                  └───────────┘              ▼            │
+ *   │                                       synthesizer        │
+ *   │                                            │             │
+ *   │                                         [END]            │
+ *   └──────────────────────────────────────────────────────────┘
+ *
+ * HITL interrupt fires between auditor approval and tool_executor,
+ * giving operators a chance to review the plan before any tool runs.
+ *
+ * Every node writes to `state.reasoning` for full observability.
+ */
+
+import { StateGraph, END, START, MemorySaver, Command } from "@langchain/langgraph";
+import { v4 as uuidv4 } from "uuid";
+
+import {
+  ResearchState,
+  ReasoningEntry,
+  createInitialState,
+} from "./state";
+
+import {
+  thinkerNode,
+  auditorNode,
+  hitlGateNode,
+  toolExecutorNode,
+  synthesizerNode,
+} from "./nodes";
+
+import {
+  routeAfterAudit,
+  routeAfterHitl,
+  routeAfterToolStep,
+} from "./routing";
+
+// ─────────────────────────────────────────────────────────────
+// Graph construction
+// ─────────────────────────────────────────────────────────────
+
+/**
+ * Builds and compiles the DeepTrust StateGraph.
+ *
+ * The `checkpointer` defaults to MemorySaver (in-process, dev-only).
+ * Pass a PostgresSaver instance for production persistence.
+ */
+export function buildDeepTrustGraph(
+  checkpointer: MemorySaver = new MemorySaver()
+) {
+  const graph = new StateGraph<ResearchState>({
+    channels: {
+      threadId:         { value: (_, n) => n },
+      sessionName:      { value: (_, n) => n },
+      userQuery:        { value: (_, n) => n },
+      knowledgeContext: { value: (_, n) => n },
+      contextUrls:      { value: (_, n) => n },
+      plan:             { value: (_, n) => n },
+      rejectionFeedback:{ value: (_, n) => n },
+      planRevisionCount:{ value: (_, n) => n },
+      maxPlanRevisions: { value: (_, n) => n },
+      auditResult:      { value: (_, n) => n },
+      currentStepIndex: { value: (_, n) => n },
+      humanApproved:    { value: (_, n) => n },
+      finalReport:      { value: (_, n) => n },
+      reasoning: {
+        value: (existing: ReasoningEntry[], incoming: ReasoningEntry[]) =>
+          [...(existing ?? []), ...(incoming ?? [])],
+        default: () => [],
+      },
+      status:       { value: (_, n) => n },
+      updatedAt:    { value: (_, n) => n },
+      errorMessage: { value: (_, n) => n },
+    },
+  })
+    // Nodes
+    .addNode("thinker",       thinkerNode)
+    .addNode("auditor",       auditorNode)
+    .addNode("hitl_gate",     hitlGateNode)
+    .addNode("tool_executor", toolExecutorNode)
+    .addNode("synthesizer",   synthesizerNode)
+
+    // Edges
+    .addEdge(START,         "thinker")
+    .addEdge("thinker",     "auditor")
+
+    .addConditionalEdges("auditor", routeAfterAudit, {
+      thinker:   "thinker",
+      hitl_gate: "hitl_gate",
+      [END]:      END,
+    })
+
+    .addConditionalEdges("hitl_gate", routeAfterHitl, {
+      tool_executor: "tool_executor",
+      [END]:          END,
+    })
+
+    .addConditionalEdges("tool_executor", routeAfterToolStep, {
+      tool_executor: "tool_executor",
+      synthesizer:   "synthesizer",
+    })
+
+    .addEdge("synthesizer", END);
+
+  return graph.compile({ checkpointer, interruptBefore: [] });
+}
+
+// ─────────────────────────────────────────────────────────────
+// Singleton instance
+// ─────────────────────────────────────────────────────────────
+
+const checkpointer = new MemorySaver();
+export const deepTrustGraph = buildDeepTrustGraph(checkpointer);
+
+// ─────────────────────────────────────────────────────────────
+// Public API
+// ─────���───────────────────────────────────────────────────────
+
+export interface RunResearchOptions {
+  knowledgeContext?: string;
+  contextUrls?: string[];
+  /**
+   * Optional metadata forwarded to LangGraph / LangSmith.
+   * Useful for grouping and searching traces.
+   */
+  metadata?: Record<string, unknown>;
+}
+
+/**
+ * Start a new research session and stream events to the caller.
+ * Optionally pass retrieved context from client-side RAG (knowledgeContext, contextUrls).
+ *
+ * @example
+ * ```ts
+ * for await (const event of runResearch("What caused the 2008 crisis?")) {
+ *   console.log(event);
+ * }
+ * for await (const event of runResearch("Summarize my docs", "Session", { knowledgeContext: "..." })) {
+ *   console.log(event);
+ * }
+ * ```
+ */
+export async function* runResearch(
+  userQuery: string,
+  sessionName = "Research Session",
+  options: RunResearchOptions = {}
+): AsyncGenerator<{ node: string; state: Partial<ResearchState> }> {
+  const initialState = createInitialState({
+    threadId: uuidv4(),
+    userQuery,
+    sessionName,
+    knowledgeContext: options.knowledgeContext,
+    contextUrls: options.contextUrls,
+  });
+
+  const config = {
+    configurable: { thread_id: initialState.threadId },
+    metadata: {
+      project: process.env.LANGCHAIN_PROJECT ?? "deeptrust",
+      run_name: "DeepTrust research session",
+      source: "deeptrust-ui",
+      ...(options.metadata ?? {}),
+    },
+  };
+
+  for await (const event of await deepTrustGraph.stream(initialState, config)) {
+    // LangGraph interrupts surface under the __interrupt__ key.
+    if ("__interrupt__" in event) {
+      const interruptInfo = (event as any).__interrupt__;
+      yield {
+        node: "__interrupt__",
+        // Expose threadId so callers can resume the run.
+        state: {
+          threadId: initialState.threadId,
+          interrupt: interruptInfo,
+        } as any,
+      };
+      return;
+    }
+
+    for (const [node, state] of Object.entries(event)) {
+      yield { node, state: state as Partial<ResearchState> };
+    }
+  }
+}
+
+/**
+ * Resume a paused session after human approval.
+ *
+ * The caller passes the threadId and streams the remaining events.
+ */
+export async function* approveAndResume(
+  threadId: string
+): AsyncGenerator<{ node: string; state: Partial<ResearchState> }> {
+  const config = { configurable: { thread_id: threadId } };
+
+  await deepTrustGraph.updateState(config, { humanApproved: true });
+
+  // Resume execution from the last checkpoint.
+  const command = new Command({
+    resume: { approved: true },
+  });
+
+  for await (const event of await deepTrustGraph.stream(command, config)) {
+    for (const [node, state] of Object.entries(event)) {
+      yield { node, state: state as Partial<ResearchState> };
+    }
+  }
+}

lib/agent/index.ts

@@ -0,0 +1,32 @@
+/**
+ * DeepTrust Research Agent — Public API
+ */
+
+// Graph and public functions
+export {
+  buildDeepTrustGraph,
+  deepTrustGraph,
+  runResearch,
+  approveAndResume,
+} from "./graph";
+export type { RunResearchOptions } from "./graph";
+
+// State types and helpers
+export {
+  ResearchState,
+  ResearchPlan,
+  ResearchStep,
+  AuditResult,
+  ReasoningEntry,
+  NodeName,
+  createInitialState,
+  appendReasoning,
+} from "./state";
+
+// LLM client (for advanced usage)
+export { chatComplete, loadModel, getModelStatus, MODEL_ID, MODELS } from "./llm";
+export type { ModelProgress, ProgressCallback, ModelOption } from "./llm";
+
+// Utilities
+export { extractJSON, loadPolicy } from "./utils";
+

lib/agent/llm/index.ts

@@ -0,0 +1,121 @@
+/**
+ * LLM Client — server-side inference via a worker thread
+ *
+ * Pipeline runs in lib/agent/llm/worker-entry (compiled to dist/llm). This module
+ * spawns the worker and proxies getModelStatus, loadModel, chatComplete so API routes
+ * and the agent keep the same API. Keeps the main Node process non-blocking.
+ */
+
+import { Worker } from "node:worker_threads";
+import path from "node:path";
+import fs from "node:fs";
+
+import type { ModelOption, ModelProgress } from "./pipeline";
+
+// Static list for API responses; worker also returns models in getStatus/load payloads
+export type { ModelOption, ModelProgress };
+export const MODEL_ID =
+  process.env.HF_MODEL || "HuggingFaceTB/SmolLM2-135M-Instruct";
+
+export const MODELS: ModelOption[] = [
+  {
+    id: "HuggingFaceTB/SmolLM2-135M-Instruct",
+    label: "SmolLM2 135M (Q4, tiny)",
+    dtype: "q4",
+    sizeNote: "~150–200 MB (approx)",
+  },
+  {
+    id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+    label: "SmolLM2 360M (Q4)",
+    dtype: "q4",
+    sizeNote: "~388 MB",
+  },
+  {
+    id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+    label: "SmolLM2 360M (FP16)",
+    dtype: "fp16",
+    sizeNote: "~725 MB",
+  },
+  {
+    id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+    label: "SmolLM2 360M (full)",
+    dtype: "fp32",
+    sizeNote: "~1.45 GB",
+  },
+];
+
+export type ProgressCallback = (progress: ModelProgress) => void;
+
+let worker: Worker | null = null;
+const pending = new Map<
+  string,
+  { resolve: (v: unknown) => void; reject: (e: Error) => void; onProgress?: ProgressCallback }
+>();
+
+function getWorkerPath(): string {
+  return path.join(process.cwd(), "dist", "llm", "worker-entry.js");
+}
+
+function getWorker(): Worker {
+  if (worker) return worker;
+  const workerPath = getWorkerPath();
+  if (!fs.existsSync(workerPath)) {
+    throw new Error(
+      `LLM worker not found at ${workerPath}. Run "npm run build:worker" (or "npm run build") first.`
+    );
+  }
+  worker = new Worker(workerPath, {
+    stdout: true,
+    stderr: true,
+  });
+  worker.on("message", (msg: { id: string; type: string; payload: unknown }) => {
+    const entry = pending.get(msg.id);
+    if (!entry) return;
+    if (msg.type === "progress" && entry.onProgress) {
+      entry.onProgress(msg.payload as ModelProgress);
+      return;
+    }
+    pending.delete(msg.id);
+    if (msg.type === "resolve") entry.resolve(msg.payload);
+    else if (msg.type === "reject") entry.reject(new Error(String(msg.payload)));
+  });
+  worker.on("error", (err) => {
+    for (const [, entry] of pending) entry.reject(err);
+    pending.clear();
+  });
+  worker.on("exit", (code) => {
+    if (code !== 0) {
+      for (const [, entry] of pending) entry.reject(new Error(`Worker exited with code ${code}`));
+      pending.clear();
+    }
+    worker = null;
+  });
+  return worker;
+}
+
+function send<T>(type: string, payload: unknown, onProgress?: ProgressCallback): Promise<T> {
+  const id = `${Date.now()}-${Math.random().toString(36).slice(2)}`;
+  return new Promise<T>((resolve, reject) => {
+    pending.set(id, { resolve: resolve as (v: unknown) => void, reject, onProgress });
+    getWorker().postMessage({ id, type, payload });
+  });
+}
+
+export type ModelStatusResponse = ModelProgress & { models: ModelOption[] };
+
+export function getModelStatus(forModelId?: string, forDtype?: string): Promise<ModelStatusResponse> {
+  return send<ModelStatusResponse>("getStatus", { modelId: forModelId, dtype: forDtype });
+}
+
+/** Resolves when the model is loaded; progress is reported via onProgress. */
+export function loadModel(
+  modelId?: string,
+  dtype?: ModelOption["dtype"],
+  onProgress?: ProgressCallback
+): Promise<ModelStatusResponse> {
+  return send<ModelStatusResponse>("load", { modelId, dtype }, onProgress);
+}
+
+export async function chatComplete(systemPrompt: string, userMessage: string): Promise<string> {
+  return send<string>("chat", { systemPrompt, userMessage });
+}

lib/agent/llm/pipeline.ts

@@ -0,0 +1,243 @@
+/**
+ * Pipeline — runs inside the LLM worker thread only.
+ * Hugging Face Transformers + onnxruntime-node; state lives in this thread.
+ */
+
+import { pipeline, TextGenerationPipeline, env } from "@huggingface/transformers";
+import path from "node:path";
+
+env.cacheDir =
+  process.env.HF_CACHE_DIR ||
+  path.join(process.cwd(), ".hf-cache");
+
+// Default to a very small instruct model for low-spec machines.
+// If this model ID does not exist or you prefer another tiny model,
+// set HF_MODEL in your environment to override.
+export const MODEL_ID =
+  process.env.HF_MODEL || "HuggingFaceTB/SmolLM2-135M-Instruct";
+
+export interface ModelOption {
+  id: string;
+  label: string;
+  dtype?: "q4" | "fp16" | "fp32";
+  sizeNote?: string;
+}
+
+export const MODELS: ModelOption[] = [
+  {
+    id: "HuggingFaceTB/SmolLM2-135M-Instruct",
+    label: "SmolLM2 135M (Q4, tiny)",
+    dtype: "q4",
+    sizeNote: "~150–200 MB (approx)",
+  },
+  {
+    id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+    label: "SmolLM2 360M (Q4)",
+    dtype: "q4",
+    sizeNote: "~388 MB",
+  },
+  {
+    id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+    label: "SmolLM2 360M (FP16)",
+    dtype: "fp16",
+    sizeNote: "~725 MB",
+  },
+  {
+    id: "HuggingFaceTB/SmolLM2-360M-Instruct",
+    label: "SmolLM2 360M (full)",
+    dtype: "fp32",
+    sizeNote: "~1.45 GB",
+  },
+];
+
+let currentModelId = MODELS[0].id;
+let currentDtype = MODELS[0].dtype;
+let generatorPromise: Promise<TextGenerationPipeline> | null = null;
+let isModelLoaded = false;
+let currentProgress = 0;
+let currentStatus = "idle";
+let currentFile = "";
+
+export interface ModelProgress {
+  status: "idle" | "loading" | "downloading" | "ready" | "error";
+  progress: number;
+  file: string;
+  message: string;
+  modelId?: string;
+  dtype?: string;
+}
+
+export function getModelStatus(forModelId?: string, forDtype?: string): ModelProgress {
+  const isOther = forModelId !== undefined && (forModelId !== currentModelId || forDtype !== currentDtype);
+  if (isOther) {
+    return {
+      status: "idle",
+      progress: 0,
+      file: "",
+      message: "Model not loaded",
+      modelId: forModelId,
+      dtype: forDtype,
+    };
+  }
+  if (isModelLoaded) {
+    return {
+      status: "ready",
+      progress: 100,
+      file: "",
+      message: "Model ready",
+      modelId: currentModelId,
+      dtype: currentDtype,
+    };
+  }
+  if (generatorPromise) {
+    return {
+      status: currentStatus as ModelProgress["status"],
+      progress: currentProgress,
+      file: currentFile,
+      message: currentFile ? `Downloading ${currentFile}` : "Loading model...",
+      modelId: currentModelId,
+      dtype: currentDtype,
+    };
+  }
+  return {
+    status: "idle",
+    progress: 0,
+    file: "",
+    message: "Model not loaded",
+    modelId: currentModelId,
+    dtype: currentDtype,
+  };
+}
+
+export type ProgressCallback = (progress: ModelProgress) => void;
+
+export function loadModel(
+  modelId?: string,
+  dtype?: ModelOption["dtype"],
+  onProgress?: ProgressCallback
+): Promise<TextGenerationPipeline> {
+  const nextId = modelId ?? currentModelId;
+  const nextDtype = dtype ?? currentDtype;
+
+  if (nextId !== currentModelId || nextDtype !== currentDtype) {
+    generatorPromise = null;
+    isModelLoaded = false;
+    currentModelId = nextId;
+    currentDtype = nextDtype;
+    currentStatus = "idle";
+    currentProgress = 0;
+    currentFile = "";
+  }
+
+  if (generatorPromise) {
+    return generatorPromise;
+  }
+
+  console.log(`\n🔄 [worker] Loading model: ${currentModelId}${currentDtype ? ` (${currentDtype})` : ""}`);
+  console.log(`   Cache directory: ${env.cacheDir}\n`);
+
+  currentStatus = "loading";
+  const startTime = Date.now();
+
+  const pipelineOptions: Parameters<typeof pipeline>[2] = {
+    progress_callback: (progressData: {
+      status: string;
+      name?: string;
+      file?: string;
+      loaded?: number;
+      total?: number;
+      progress?: number;
+    }) => {
+      currentStatus = progressData.status === "progress" ? "downloading" : "loading";
+      currentFile = progressData.file || progressData.name || "";
+      currentProgress = Math.round(progressData.progress ?? 0);
+
+      const update: ModelProgress = {
+        status: currentStatus as ModelProgress["status"],
+        progress: currentProgress,
+        file: currentFile,
+        message: currentFile
+          ? `Downloading ${currentFile.split("/").pop()} (${currentProgress}%)`
+          : `${progressData.status}...`,
+        modelId: currentModelId,
+        dtype: currentDtype,
+      };
+
+      console.log(`   ${update.message}`);
+      onProgress?.(update);
+    },
+  };
+  if (currentDtype) {
+    (pipelineOptions as Record<string, unknown>).dtype = currentDtype;
+  }
+
+  const pipelinePromise = pipeline("text-generation", currentModelId, pipelineOptions);
+  generatorPromise = (pipelinePromise as Promise<TextGenerationPipeline>)
+    .then((gen) => {
+      const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
+      console.log(`\n✅ [worker] Model loaded in ${elapsed}s\n`);
+      isModelLoaded = true;
+      currentStatus = "ready";
+      currentProgress = 100;
+      onProgress?.({
+        status: "ready",
+        progress: 100,
+        file: "",
+        message: "Model ready",
+        modelId: currentModelId,
+        dtype: currentDtype,
+      });
+      return gen;
+    })
+    .catch((err) => {
+      currentStatus = "error";
+      generatorPromise = null;
+      onProgress?.({
+        status: "error",
+        progress: 0,
+        file: "",
+        message: err.message,
+        modelId: currentModelId,
+        dtype: currentDtype,
+      });
+      throw err;
+    });
+
+  return generatorPromise;
+}
+
+export async function chatComplete(
+  systemPrompt: string,
+  userMessage: string
+): Promise<string> {
+  const generator = await loadModel();
+
+  const messages = [
+    { role: "system", content: systemPrompt },
+    { role: "user", content: userMessage },
+  ];
+
+  const preview = userMessage.slice(0, 60).replace(/\n/g, " ");
+  console.log(`🤖 [worker] Generating response for: "${preview}..."`);
+  const startTime = Date.now();
+
+  const output = await generator(messages, {
+    max_new_tokens: 4096,
+    do_sample: true,
+    temperature: 0.7,
+  });
+
+  const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
+
+  const result = output[0] as { generated_text: Array<{ role: string; content: string }> };
+  const assistantMessage = result.generated_text.find(
+    (msg) => msg.role === "assistant"
+  );
+
+  if (!assistantMessage) {
+    throw new Error("No assistant response generated");
+  }
+
+  console.log(`✅ [worker] Generated ${assistantMessage.content.length} chars in ${elapsed}s`);
+  return assistantMessage.content;
+}

lib/agent/llm/worker-entry.ts

@@ -0,0 +1,65 @@
+/**
+ * Worker entry — runs in a Node worker_thread. Handles getStatus, load, chat and forwards progress.
+ */
+
+import { parentPort } from "node:worker_threads";
+import {
+  getModelStatus,
+  loadModel,
+  chatComplete,
+  MODELS,
+  type ModelProgress,
+  type ModelOption,
+} from "./pipeline";
+
+type Incoming =
+  | { id: string; type: "getStatus"; payload: { modelId?: string; dtype?: string } }
+  | { id: string; type: "load"; payload: { modelId?: string; dtype?: ModelOption["dtype"] } }
+  | { id: string; type: "chat"; payload: { systemPrompt: string; userMessage: string } };
+
+type Outgoing =
+  | { id: string; type: "resolve"; payload: unknown }
+  | { id: string; type: "progress"; payload: ModelProgress & { models: ModelOption[] } }
+  | { id: string; type: "reject"; payload: string };
+
+function reply(msg: Outgoing) {
+  parentPort?.postMessage(msg);
+}
+
+parentPort?.on("message", (msg: Incoming) => {
+  const { id, type, payload } = msg;
+
+  (async () => {
+    try {
+      switch (type) {
+        case "getStatus": {
+          const status = getModelStatus(payload.modelId, payload.dtype);
+          reply({ id, type: "resolve", payload: { ...status, models: MODELS } });
+          break;
+        }
+        case "load": {
+          await loadModel(payload.modelId, payload.dtype, (progress) => {
+            reply({
+              id,
+              type: "progress",
+              payload: { ...progress, models: MODELS },
+            });
+          });
+          const status = getModelStatus(payload.modelId, payload.dtype);
+          reply({ id, type: "resolve", payload: { ...status, models: MODELS } });
+          break;
+        }
+        case "chat": {
+          const text = await chatComplete(payload.systemPrompt, payload.userMessage);
+          reply({ id, type: "resolve", payload: text });
+          break;
+        }
+        default:
+          reply({ id, type: "reject", payload: `Unknown message type: ${(msg as { type: string }).type}` });
+      }
+    } catch (err) {
+      const message = err instanceof Error ? err.message : String(err);
+      reply({ id, type: "reject", payload: message });
+    }
+  })();
+});

lib/agent/nodes/auditor.ts

@@ -0,0 +1,101 @@
+/**
+ * Auditor Node — validates plans against organizational policy
+ */
+
+import { chatComplete } from "../llm";
+import { extractJSON, loadPolicy } from "../utils";
+import { ResearchState, AuditResult, appendReasoning } from "../state";
+
+/**
+ * Compares the current plan against POLICY.md.
+ * Sets `auditResult` with verdict + structured feedback.
+ * If rejected, also sets `rejectionFeedback` and increments
+ * `planRevisionCount` so the router can loop back to thinker.
+ */
+export async function auditorNode(
+  state: ResearchState
+): Promise<Partial<ResearchState>> {
+  if (!state.plan) {
+    throw new Error("auditorNode called with no plan in state");
+  }
+
+  const policy = loadPolicy();
+
+  const system = `
+You are the Auditor node of DeepTrust.
+You evaluate research plans against an organisational policy and return a structured verdict.
+
+Return ONLY a valid JSON object matching:
+{
+  "verdict": "approved" | "rejected" | "needs_revision",
+  "policyViolations": string[],
+  "suggestions": string[],
+  "auditedAt": string (ISO 8601)
+}
+
+Rules:
+- "approved" means the plan fully complies with policy.
+- "rejected" means the plan has hard violations that cannot be patched.
+- "needs_revision" means soft issues exist but the plan is salvageable.
+- Treat "needs_revision" as rejection for routing purposes.
+- Do not include markdown fences or any prose outside the JSON object.
+  `.trim();
+
+  const userMessage = `
+POLICY:
+${policy}
+
+PLAN TO AUDIT:
+${JSON.stringify(state.plan, null, 2)}
+  `.trim();
+
+  const rawThought = await chatComplete(system, userMessage);
+
+  let parsed: unknown;
+  try {
+    parsed = extractJSON(rawThought);
+  } catch {
+    throw new Error(`Auditor produced non-JSON output: ${rawThought.slice(0, 300)}`);
+  }
+
+  const auditResult = AuditResult.parse({
+    ...(parsed as object),
+    auditedAt: new Date().toISOString(),
+  });
+
+  const isRejected = auditResult.verdict !== "approved";
+
+  const rejectionFeedback: string | null = isRejected
+    ? [
+        `Verdict: ${auditResult.verdict}`,
+        auditResult.policyViolations.length
+          ? `Violations:\n${auditResult.policyViolations.map((v) => `  - ${v}`).join("\n")}`
+          : null,
+        auditResult.suggestions.length
+          ? `Suggestions:\n${auditResult.suggestions.map((s) => `  - ${s}`).join("\n")}`
+          : null,
+      ]
+        .filter(Boolean)
+        .join("\n\n")
+    : null;
+
+  const reasoning = appendReasoning(state, {
+    node: "auditor",
+    summary: isRejected
+      ? `Plan REJECTED (${auditResult.verdict}). ${auditResult.policyViolations.length} violation(s) found.`
+      : "Plan APPROVED — no policy violations detected.",
+    rawThought,
+  });
+
+  return {
+    auditResult,
+    rejectionFeedback,
+    planRevisionCount: isRejected
+      ? state.planRevisionCount + 1
+      : state.planRevisionCount,
+    status: isRejected ? "thinking" : "awaiting_approval",
+    reasoning,
+    updatedAt: new Date().toISOString(),
+  };
+}
+

lib/agent/nodes/hitl-gate.ts

@@ -0,0 +1,40 @@
+/**
+ * HITL Gate Node — Human-in-the-Loop interrupt point
+ */
+
+import { interrupt } from "@langchain/langgraph";
+import { ResearchState, appendReasoning } from "../state";
+
+/**
+ * Pauses the graph and surfaces the plan to the operator.
+ * When the graph is resumed (via .updateState()), the caller
+ * must set `state.humanApproved = true` to proceed.
+ *
+ * Using LangGraph's `interrupt()` primitive ensures the
+ * checkpoint is written BEFORE the interrupt fires, so the
+ * session can be resumed from any client or process.
+ */
+export async function hitlGateNode(
+  state: ResearchState
+): Promise<Partial<ResearchState>> {
+  const reasoning = appendReasoning(state, {
+    node: "thinker", // logged under thinker namespace for UI grouping
+    summary: "⏸ HITL gate: awaiting human approval before tool execution.",
+  });
+
+  // This call writes a checkpoint and suspends until the graph
+  // is resumed externally. The value passed to interrupt() is
+  // surfaced to the caller of `graph.stream()`.
+  interrupt({
+    message: "Plan ready for review. Set humanApproved=true to continue.",
+    plan: state.plan,
+    auditResult: state.auditResult,
+  });
+
+  // Code below only runs after the interrupt is resolved.
+  return {
+    reasoning,
+    updatedAt: new Date().toISOString(),
+  };
+}
+

lib/agent/nodes/index.ts

@@ -0,0 +1,6 @@
+export { thinkerNode } from "./thinker";
+export { auditorNode } from "./auditor";
+export { hitlGateNode } from "./hitl-gate";
+export { toolExecutorNode } from "./tool-executor";
+export { synthesizerNode } from "./synthesizer";
+

lib/agent/nodes/synthesizer.ts

@@ -0,0 +1,66 @@
+/**
+ * Synthesizer Node — produces the final research report
+ */
+
+import { chatComplete } from "../llm";
+import { ResearchState, appendReasoning } from "../state";
+
+/**
+ * Reads all step outputs and produces the final research report.
+ */
+export async function synthesizerNode(
+  state: ResearchState
+): Promise<Partial<ResearchState>> {
+  if (!state.plan) throw new Error("synthesizerNode called with no plan");
+
+  const system = `
+You are the Synthesizer node of DeepTrust.
+You receive a research plan with all step outputs filled in,
+and you write a comprehensive, well-structured research report.
+
+Guidelines:
+- Lead with an executive summary.
+- Organise findings by theme, not by tool execution order.
+- Cite which step produced each finding.
+- End with actionable conclusions.
+  `.trim();
+
+  const stepsContext = state.plan.steps
+    .map(
+      (s, i) =>
+        `Step ${i + 1} [${s.tool}] — "${s.input}"\nOutput: ${s.output ?? "(no output)"}`
+    )
+    .join("\n\n---\n\n");
+
+  const knowledgeBlock =
+    state.knowledgeContext?.trim() ?
+      `
+
+User's local knowledge (use to ground or cite the report where relevant):
+${state.knowledgeContext}
+`
+    : "";
+
+  const userMessage = `
+Objective: ${state.plan.objective}
+
+Research results:
+${stepsContext}
+${knowledgeBlock}
+  `.trim();
+
+  const finalReport = await chatComplete(system, userMessage);
+
+  const reasoning = appendReasoning(state, {
+    node: "synthesizer",
+    summary: `Final report synthesized (${finalReport.length} chars).`,
+  });
+
+  return {
+    finalReport,
+    status: "complete",
+    reasoning,
+    updatedAt: new Date().toISOString(),
+  };
+}
+

lib/agent/nodes/thinker.ts

@@ -0,0 +1,228 @@
+/**
+ * Thinker Node — produces or revises a structured ResearchPlan
+ *
+ * Uses safeParse + retry with Zod error feedback so that invalid LLM output
+ * (e.g. missing fields, wrong types) is corrected by the model instead of
+ * causing a 500. After max retries, throws so the route can stream an error event.
+ */
+
+import { chatComplete } from "../llm";
+import { extractJSON } from "../utils";
+import { ResearchState, ResearchPlan, appendReasoning } from "../state";
+import { v4 as uuidv4 } from "uuid";
+
+const MAX_PLAN_ATTEMPTS = 3;
+
+/**
+ * Format Zod errors for inclusion in the next prompt so the LLM can self-correct.
+ */
+function formatZodErrors(issues: { path: unknown[]; message: string }[]): string {
+  return issues
+    .map((i) => `  - ${i.path.map((p) => String(p)).join(".")}: ${i.message}`)
+    .join("\n");
+}
+
+const UUID_REGEX =
+  /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
+
+/**
+ * Normalize LLM output so it passes ResearchPlan: fix non-UUID step ids and
+ * missing rationales (small/local models often omit these).
+ */
+function normalizePlan(raw: Record<string, unknown>): Record<string, unknown> {
+  const steps = Array.isArray(raw.steps) ? raw.steps : [];
+  const normalizedSteps = steps.map((s: unknown, i: number) => {
+    const step = typeof s === "object" && s !== null ? (s as Record<string, unknown>) : {};
+    const id =
+      typeof step.id === "string" && UUID_REGEX.test(step.id) ? step.id : uuidv4();
+    const rationale =
+      typeof step.rationale === "string" && step.rationale.length > 0
+        ? step.rationale
+        : typeof step.input === "string"
+          ? step.input
+          : `Step ${i + 1}`;
+    return {
+      ...step,
+      id,
+      rationale,
+      input: typeof step.input === "string" ? step.input : "",
+      tool: step.tool ?? "web_search",
+    };
+  });
+
+  const createdAt =
+    typeof raw.createdAt === "string" && raw.createdAt.length > 0
+      ? raw.createdAt
+      : new Date().toISOString();
+  const estimatedTokenBudget =
+    typeof raw.estimatedTokenBudget === "number" && raw.estimatedTokenBudget > 0
+      ? raw.estimatedTokenBudget
+      : 2048;
+
+  const objective =
+    typeof raw.objective === "string" && raw.objective.trim().length > 0
+      ? raw.objective.trim()
+      : "Research objective";
+
+  return {
+    ...raw,
+    objective,
+    steps: normalizedSteps,
+    createdAt,
+    estimatedTokenBudget,
+  };
+}
+
+/**
+ * Produces (or revises) a structured ResearchPlan.
+ * When `state.rejectionFeedback` is set, the model receives the
+ * Auditor's critique and is instructed to produce a corrected plan.
+ *
+ * If the LLM returns invalid JSON or fails schema validation, we retry up to
+ * MAX_PLAN_ATTEMPTS times, feeding the validation errors back into the prompt.
+ */
+export async function thinkerNode(
+  state: ResearchState
+): Promise<Partial<ResearchState>> {
+  const isRevision = state.planRevisionCount > 0 && state.rejectionFeedback;
+
+  const system = `
+You are the Thinker node of DeepTrust, an autonomous research agent.
+Your sole job is to decompose a research question into a concrete, step-by-step plan.
+
+Return ONLY a valid JSON object that matches this TypeScript type:
+{
+  "objective": string,
+  "steps": Array<{
+    "id": string (UUID v4),
+    "tool": "web_search" | "document_fetch" | "code_interpreter" | "summarize",
+    "input": string,
+    "rationale": string
+  }>,
+  "estimatedTokenBudget": number,
+  "createdAt": string (ISO 8601),
+  "revision": number
+}
+
+Rules:
+- Maximum 20 steps.
+- Every step must have a clear rationale.
+- Do not include markdown fences or any prose outside the JSON object.
+  `.trim();
+
+  let revisionContext = isRevision
+    ? `\n\nPREVIOUS PLAN WAS REJECTED. Auditor feedback:\n${state.rejectionFeedback}\n\nRevision #${state.planRevisionCount + 1}: Produce a corrected plan.`
+    : "";
+
+  let knowledgeBlock = "";
+  if (state.knowledgeContext?.trim()) {
+    knowledgeBlock = `\n\nThe user provided the following retrieved context from their local knowledge base. Use it to inform the plan and prefer steps that leverage this context where relevant:\n\n${state.knowledgeContext}`;
+  }
+  if (state.contextUrls?.length) {
+    knowledgeBlock += `\n\nThe user also referenced these URLs (consider document_fetch steps for them where useful): ${state.contextUrls.join(", ")}`;
+  }
+
+  let userMessage = `Research question: "${state.userQuery}"${revisionContext}${knowledgeBlock}`;
+
+  let lastRawThought: string | null = null;
+  let lastParseError: string | null = null;
+
+  for (let attempt = 1; attempt <= MAX_PLAN_ATTEMPTS; attempt++) {
+    const parseFeedback =
+      lastParseError &&
+      `\n\nYour previous response had validation errors. Fix them and return ONLY valid JSON:\n${lastParseError}`;
+
+    const rawThought = await chatComplete(
+      system,
+      userMessage + (parseFeedback ?? "")
+    );
+    lastRawThought = rawThought;
+
+    let parsed: unknown;
+    try {
+      parsed = extractJSON(rawThought);
+    } catch {
+      lastParseError = `Could not parse as JSON. Output started with: ${rawThought.slice(0, 200)}`;
+      continue;
+    }
+
+    const normalized = normalizePlan(
+      typeof parsed === "object" && parsed !== null ? (parsed as Record<string, unknown>) : {}
+    );
+    const withRevision = {
+      ...normalized,
+      revision: state.planRevisionCount,
+    };
+
+    const result = ResearchPlan.safeParse(withRevision);
+    if (result.success) {
+      const plan = result.data;
+      const reasoning = appendReasoning(state, {
+        node: "thinker",
+        summary: isRevision
+          ? `Revised plan (attempt ${state.planRevisionCount + 1}): ${plan.steps.length} steps for "${plan.objective}"`
+          : `Initial plan created: ${plan.steps.length} steps for "${plan.objective}"`,
+        rawThought,
+      });
+
+      return {
+        plan,
+        status: "thinking",
+        rejectionFeedback: null,
+        reasoning,
+        updatedAt: new Date().toISOString(),
+      };
+    }
+
+    lastParseError = formatZodErrors(result.error.issues);
+  }
+
+  // Fallback: if the model failed to produce valid JSON after all attempts,
+  // synthesize a minimal but valid single-step plan directly from the user query
+  // so the run can still proceed instead of hard-failing.
+  const fallbackPlanResult = ResearchPlan.safeParse({
+    objective:
+      typeof state.userQuery === "string" && state.userQuery.trim().length > 0
+        ? state.userQuery.trim()
+        : "Research objective",
+    steps: [
+      {
+        id: uuidv4(),
+        tool: "web_search",
+        input:
+          typeof state.userQuery === "string" && state.userQuery.trim().length > 0
+            ? state.userQuery.trim()
+            : "Initial research query",
+        rationale:
+          typeof lastRawThought === "string" && lastRawThought.trim().length > 0
+            ? lastRawThought.slice(0, 500)
+            : "Initial search to understand the question.",
+      },
+    ],
+    estimatedTokenBudget: 2048,
+    createdAt: new Date().toISOString(),
+    revision: state.planRevisionCount,
+  });
+
+  if (fallbackPlanResult.success) {
+    const plan = fallbackPlanResult.data;
+    const reasoning = appendReasoning(state, {
+      node: "thinker",
+      summary: `Fallback plan created after ${MAX_PLAN_ATTEMPTS} failed JSON attempts: ${plan.steps.length} step for "${plan.objective}"`,
+      rawThought: lastRawThought ?? undefined,
+    });
+
+    return {
+      plan,
+      status: "thinking",
+      rejectionFeedback: null,
+      reasoning,
+      updatedAt: new Date().toISOString(),
+    };
+  }
+
+  throw new Error(
+    `Thinker failed to produce a valid plan after ${MAX_PLAN_ATTEMPTS} attempts. Last validation errors:\n${lastParseError ?? "unknown"}\n\nLast output (excerpt): ${(lastRawThought ?? "").slice(0, 400)}`
+  );
+}
+

lib/agent/nodes/tool-executor.ts

@@ -0,0 +1,79 @@
+/**
+ * Tool Executor Node — executes research plan steps
+ */
+
+import { ResearchState, ResearchPlan, ResearchStep, appendReasoning } from "../state";
+
+/**
+ * Executes one step at a time (the step at `currentStepIndex`).
+ * After each execution the graph loops back through the router;
+ * when all steps are complete it advances to synthesizer.
+ *
+ * Real tool integrations (Tavily, Playwright, etc.) replace the
+ * stub `dispatchTool` function.
+ */
+export async function toolExecutorNode(
+  state: ResearchState
+): Promise<Partial<ResearchState>> {
+  if (!state.plan) throw new Error("toolExecutorNode called with no plan");
+
+  const step = state.plan.steps[state.currentStepIndex];
+  if (!step) throw new Error(`No step at index ${state.currentStepIndex}`);
+
+  // Dispatch the tool
+  const output = await dispatchTool(step.tool, step.input);
+
+  // Persist the output back into the plan's steps array immutably
+  const updatedSteps = state.plan.steps.map((s, i) =>
+    i === state.currentStepIndex
+      ? { ...s, output, executedAt: new Date().toISOString() }
+      : s
+  );
+
+  const updatedPlan = ResearchPlan.parse({
+    ...state.plan,
+    steps: updatedSteps,
+  });
+
+  const reasoning = appendReasoning(state, {
+    node: "tool_executor",
+    summary: `Executed step ${state.currentStepIndex + 1}/${state.plan.steps.length}: [${step.tool}] "${step.input.slice(0, 80)}…"`,
+    rawThought: output,
+  });
+
+  return {
+    plan: updatedPlan,
+    currentStepIndex: state.currentStepIndex + 1,
+    status: "executing",
+    reasoning,
+    updatedAt: new Date().toISOString(),
+  };
+}
+
+/**
+ * Dispatch a tool call. Replace stubs with real implementations.
+ */
+async function dispatchTool(
+  tool: ResearchStep["tool"],
+  input: string
+): Promise<string> {
+  switch (tool) {
+    case "web_search":
+      // e.g. return await tavilySearch(input);
+      return `[STUB] web_search result for: "${input}"`;
+    case "document_fetch":
+      // e.g. return await fetchDocument(input);
+      return `[STUB] document_fetch result for: "${input}"`;
+    case "code_interpreter":
+      // e.g. return await runSandboxedCode(input);
+      return `[STUB] code_interpreter result for: "${input}"`;
+    case "summarize":
+      // e.g. return await summarizeText(input);
+      return `[STUB] summarize result for: "${input}"`;
+    default: {
+      const exhaustiveCheck: never = tool;
+      throw new Error(`Unknown tool: ${exhaustiveCheck}`);
+    }
+  }
+}
+

lib/agent/routing.ts

@@ -0,0 +1,49 @@
+/**
+ * Routing functions — conditional edges for the state graph
+ */
+
+import { END } from "@langchain/langgraph";
+import { ResearchState } from "./state";
+
+/**
+ * After auditor: loop back to thinker or advance to HITL gate.
+ */
+export function routeAfterAudit(
+  state: ResearchState
+): "thinker" | "hitl_gate" | typeof END {
+  // Hard stop — too many failed revisions
+  if (state.planRevisionCount >= state.maxPlanRevisions) {
+    return END;
+  }
+  if (state.auditResult?.verdict !== "approved") {
+    return "thinker";
+  }
+  return "hitl_gate";
+}
+
+/**
+ * After HITL gate: block until human approves, then execute.
+ */
+export function routeAfterHitl(
+  state: ResearchState
+): "tool_executor" | typeof END {
+  if (!state.humanApproved) {
+    // Graph should not reach here without approval; fail safe.
+    return END;
+  }
+  return "tool_executor";
+}
+
+/**
+ * After each tool step: keep executing or move to synthesizer.
+ */
+export function routeAfterToolStep(
+  state: ResearchState
+): "tool_executor" | "synthesizer" {
+  const totalSteps = state.plan?.steps.length ?? 0;
+  if (state.currentStepIndex < totalSteps) {
+    return "tool_executor";
+  }
+  return "synthesizer";
+}
+

lib/agent/state.ts

@@ -0,0 +1,209 @@
+/**
+ * state.ts — DeepTrust Research Agent
+ *
+ * Single source of truth for every field that flows through the
+ * StateGraph.  Zod gives us runtime validation; the inferred TS
+ * types keep every node strictly typed with zero `any` leakage.
+ */
+
+import { z } from "zod";
+
+// ─────────────────────────────────────────────────────────────
+// Enumerations
+// ─────────────────────────────────────────────────────────────
+
+/** Every legal node name in the graph – used to type routing returns. */
+export const NodeName = z.enum([
+  "thinker",
+  "auditor",
+  "tool_executor",
+  "synthesizer",
+  "__end__",
+]);
+export type NodeName = z.infer<typeof NodeName>;
+
+/** Disposition returned by the Auditor node. */
+export const AuditVerdict = z.enum(["approved", "rejected", "needs_revision"]);
+export type AuditVerdict = z.infer<typeof AuditVerdict>;
+
+/** Lifecycle of a single research run. */
+export const RunStatus = z.enum([
+  "idle",
+  "thinking",
+  "awaiting_approval", // HITL interrupt point
+  "executing",
+  "synthesizing",
+  "complete",
+  "failed",
+]);
+export type RunStatus = z.infer<typeof RunStatus>;
+
+// ─────────────────────────────────────────────────────────────
+// Sub-schemas
+// ─────────────────────────────────────────────────────────────
+
+/**
+ * A single step inside a research plan.
+ * The Thinker produces an array of these; the Tool Executor
+ * works through them one-by-one.
+ */
+export const ResearchStep = z.object({
+  id: z.string().uuid(),
+  tool: z.enum(["web_search", "document_fetch", "code_interpreter", "summarize"]),
+  input: z.string().min(1),
+  rationale: z.string(),
+  /** Populated by the Tool Executor after the step runs. */
+  output: z.string().optional(),
+  executedAt: z.string().datetime().optional(),
+});
+export type ResearchStep = z.infer<typeof ResearchStep>;
+
+/**
+ * The structured plan produced by the Thinker and evaluated
+ * by the Auditor before any tool is touched.
+ */
+export const ResearchPlan = z.object({
+  objective: z.string().min(1),
+  steps: z.array(ResearchStep).min(1).max(20),
+  estimatedTokenBudget: z.number().int().positive(),
+  createdAt: z.string().datetime(),
+  revision: z.number().int().nonnegative().default(0),
+});
+export type ResearchPlan = z.infer<typeof ResearchPlan>;
+
+/**
+ * One entry in the reasoning trail — every node appends here
+ * so the UI can replay the full thought process.
+ */
+export const ReasoningEntry = z.object({
+  node: NodeName,
+  timestamp: z.string().datetime(),
+  summary: z.string(),
+  /** Optional verbatim model output for deep-dive inspection. */
+  rawThought: z.string().optional(),
+});
+export type ReasoningEntry = z.infer<typeof ReasoningEntry>;
+
+/** Structured feedback from the Auditor when it rejects a plan. */
+export const AuditResult = z.object({
+  verdict: AuditVerdict,
+  policyViolations: z.array(z.string()).default([]),
+  suggestions: z.array(z.string()).default([]),
+  auditedAt: z.string().datetime(),
+});
+export type AuditResult = z.infer<typeof AuditResult>;
+
+// ─────────────────────────────────────────────────────────────
+// Root State Schema
+// ─────────────────────────────────────────────────────────────
+
+/**
+ * ResearchState — the complete, checkpointable state object.
+ *
+ * LangGraph.js will serialise/deserialise this on every step,
+ * so every field must be JSON-safe.  The checkpointer stores
+ * snapshots keyed by `threadId`, enabling full rehydration.
+ */
+export const ResearchState = z.object({
+  // ── Identity & Persistence ───────────────────────────────
+  /** Stable identifier — passed to the checkpointer as the config key. */
+  threadId: z.string().uuid(),
+  /** Human-readable label shown in the UI. */
+  sessionName: z.string().default("Unnamed Session"),
+
+  // ── Input ────────────────────────────────────────────────
+  /** The raw research question submitted by the user. */
+  userQuery: z.string().min(1),
+
+  /**
+   * Retrieved context from the user's local knowledge base (client-side RAG).
+   * Passed into Thinker and Synthesizer so the LLM can use it when planning and reporting.
+   */
+  knowledgeContext: z.string().optional(),
+
+  /**
+   * URLs the user added as references. Thinker can emit document_fetch steps for these.
+   */
+  contextUrls: z.array(z.string()).optional(),
+
+  // ── Planning ─────────────────────────────────────────────
+  /** Current plan produced by the Thinker.  Null before first plan. */
+  plan: ResearchPlan.nullable().default(null),
+
+  /**
+   * Feedback injected into the Thinker's context when the Auditor
+   * rejects a plan.  Cleared after each successful audit.
+   */
+  rejectionFeedback: z.string().nullable().default(null),
+
+  /** How many times the plan has cycled through Thinker→Auditor. */
+  planRevisionCount: z.number().int().nonnegative().default(0),
+
+  /** Safety ceiling — prevents runaway revision loops. */
+  maxPlanRevisions: z.number().int().positive().default(5),
+
+  // ── Auditing ─────────────────────────────────────────────
+  /** The most recent audit result.  Null before first audit. */
+  auditResult: AuditResult.nullable().default(null),
+
+  // ── Execution ────────────────────────────────────────────
+  /** Index into `plan.steps` for the currently-executing step. */
+  currentStepIndex: z.number().int().nonnegative().default(0),
+
+  /**
+   * Whether the user has explicitly approved the plan.
+   * The graph checks this before transitioning to tool_executor.
+   */
+  humanApproved: z.boolean().default(false),
+
+  // ── Output ───────────────────────────────────────────────
+  /** Final synthesized report.  Null until synthesizer runs. */
+  finalReport: z.string().nullable().default(null),
+
+  // ── Observability ────────────────────────────────────────
+  /**
+   * Append-only log of every node's reasoning.
+   * The UI streams this list to render a live "thought process" view.
+   */
+  reasoning: z.array(ReasoningEntry).default([]),
+
+  // ── Lifecycle ────────────────────────────────────────────
+  status: RunStatus.default("idle"),
+
+  /** ISO timestamp of the last state mutation. */
+  updatedAt: z.string().datetime(),
+
+  /** Non-null when the run terminated with an unrecoverable error. */
+  errorMessage: z.string().nullable().default(null),
+});
+
+export type ResearchState = z.infer<typeof ResearchState>;
+
+// ─────────────────────────────────────────────────────────────
+// Helpers
+// ─────────────────────────────────────────────────────────────
+
+/** Append a reasoning entry without mutating the caller's reference. */
+export function appendReasoning(
+  state: ResearchState,
+  entry: Omit<ReasoningEntry, "timestamp">
+): ReasoningEntry[] {
+  const stamped: ReasoningEntry = {
+    ...entry,
+    timestamp: new Date().toISOString(),
+  };
+  return [...state.reasoning, stamped];
+}
+
+/** Returns a fresh, validated initial state for a new session. */
+export function createInitialState(
+  params: Pick<
+    ResearchState,
+    "threadId" | "userQuery" | "sessionName" | "knowledgeContext" | "contextUrls"
+  >
+): ResearchState {
+  return ResearchState.parse({
+    ...params,
+    updatedAt: new Date().toISOString(),
+  });
+}

lib/agent/utils/extract-json.ts

@@ -0,0 +1,42 @@
+/**
+ * JSON extraction helper — handles models that add extra text
+ */
+
+/**
+ * Extracts JSON from model output that may contain extra text,
+ * markdown fences, or other non-JSON content.
+ */
+export function extractJSON(text: string): unknown {
+  // Try direct parse first
+  try {
+    return JSON.parse(text);
+  } catch {
+    // Continue to extraction methods
+  }
+
+  // Remove markdown code fences
+  const cleaned = text.replace(/```json\s*/gi, "").replace(/```\s*/g, "");
+
+  // Try to find JSON object boundaries
+  const jsonMatch = cleaned.match(/\{[\s\S]*\}/);
+  if (jsonMatch) {
+    try {
+      return JSON.parse(jsonMatch[0]);
+    } catch {
+      // Continue
+    }
+  }
+
+  // Try to find JSON array boundaries
+  const arrayMatch = cleaned.match(/\[[\s\S]*\]/);
+  if (arrayMatch) {
+    try {
+      return JSON.parse(arrayMatch[0]);
+    } catch {
+      // Continue
+    }
+  }
+
+  throw new Error(`Could not extract valid JSON from: ${text.slice(0, 200)}...`);
+}
+

lib/agent/utils/index.ts

@@ -0,0 +1,3 @@
+export { extractJSON } from "./extract-json";
+export { loadPolicy } from "./policy";
+

lib/agent/utils/policy.ts

@@ -0,0 +1,27 @@
+/**
+ * Policy loader — reads POLICY.md for the Auditor node
+ */
+
+import { readFileSync } from "fs";
+import { join } from "path";
+
+const DEFAULT_POLICY = `
+# DeepTrust Default Policy
+- Never access personal, private, or confidential data sources.
+- Never execute code that modifies the host filesystem.
+- Never make more than 10 external HTTP requests per session.
+- Research must be directly related to the user's stated objective.
+- All sources must be attributable and verifiable.
+`.trim();
+
+/**
+ * Load the policy from POLICY.md or return a default fallback.
+ */
+export function loadPolicy(): string {
+  try {
+    return readFileSync(join(process.cwd(), "POLICY.md"), "utf-8");
+  } catch {
+    return DEFAULT_POLICY;
+  }
+}
+

lib/knowledge/chunk.ts

@@ -0,0 +1,34 @@
+/**
+ * Text chunking for RAG. Fixed-size segments with overlap.
+ */
+
+const DEFAULT_CHUNK_SIZE = 500;
+const DEFAULT_OVERLAP = 80;
+
+export function chunkText(
+  text: string,
+  chunkSize = DEFAULT_CHUNK_SIZE,
+  overlap = DEFAULT_OVERLAP
+): { text: string; startIndex: number; endIndex: number }[] {
+  const trimmed = text.trim();
+  if (!trimmed) return [];
+
+  const chunks: { text: string; startIndex: number; endIndex: number }[] = [];
+  let start = 0;
+
+  while (start < trimmed.length) {
+    let end = Math.min(start + chunkSize, trimmed.length);
+    if (end < trimmed.length) {
+      const lastSpace = trimmed.lastIndexOf(" ", end);
+      if (lastSpace > start) end = lastSpace;
+    }
+    const slice = trimmed.slice(start, end);
+    if (slice.length > 0) {
+      chunks.push({ text: slice, startIndex: start, endIndex: end });
+    }
+    start = end - (end < trimmed.length ? overlap : 0);
+    if (start >= end) start = end;
+  }
+
+  return chunks;
+}

lib/knowledge/db.ts

@@ -0,0 +1,111 @@
+/**
+ * IndexedDB schema and access for the knowledge store.
+ * Database: deeptrust-knowledge
+ * Stores: documents, chunks
+ */
+
+import type { KnowledgeDocument, KnowledgeChunk } from "./types";
+
+const DB_NAME = "deeptrust-knowledge";
+const DB_VERSION = 1;
+const STORE_DOCUMENTS = "documents";
+const STORE_CHUNKS = "chunks";
+
+let dbPromise: Promise<IDBDatabase> | null = null;
+
+export function openDB(): Promise<IDBDatabase> {
+  if (typeof window === "undefined") {
+    return Promise.reject(new Error("IndexedDB is only available in the browser"));
+  }
+  if (dbPromise) return dbPromise;
+  dbPromise = new Promise((resolve, reject) => {
+    const req = indexedDB.open(DB_NAME, DB_VERSION);
+    req.onerror = () => reject(req.error);
+    req.onsuccess = () => resolve(req.result);
+    req.onupgradeneeded = (event) => {
+      const db = (event.target as IDBOpenDBRequest).result;
+      if (!db.objectStoreNames.contains(STORE_DOCUMENTS)) {
+        db.createObjectStore(STORE_DOCUMENTS, { keyPath: "id" });
+      }
+      if (!db.objectStoreNames.contains(STORE_CHUNKS)) {
+        const chunkStore = db.createObjectStore(STORE_CHUNKS, { keyPath: "id" });
+        chunkStore.createIndex("byDocument", "documentId", { unique: false });
+      }
+    };
+  });
+  return dbPromise;
+}
+
+export async function putDocument(doc: KnowledgeDocument): Promise<void> {
+  const db = await openDB();
+  return new Promise((resolve, reject) => {
+    const tx = db.transaction(STORE_DOCUMENTS, "readwrite");
+    tx.objectStore(STORE_DOCUMENTS).put(doc);
+    tx.oncomplete = () => resolve();
+    tx.onerror = () => reject(tx.error);
+  });
+}
+
+export async function getDocument(id: string): Promise<KnowledgeDocument | undefined> {
+  const db = await openDB();
+  return new Promise((resolve, reject) => {
+    const tx = db.transaction(STORE_DOCUMENTS, "readonly");
+    const req = tx.objectStore(STORE_DOCUMENTS).get(id);
+    tx.oncomplete = () => resolve(req.result);
+    tx.onerror = () => reject(tx.error);
+  });
+}
+
+export async function listDocuments(): Promise<KnowledgeDocument[]> {
+  const db = await openDB();
+  return new Promise((resolve, reject) => {
+    const tx = db.transaction(STORE_DOCUMENTS, "readonly");
+    const req = tx.objectStore(STORE_DOCUMENTS).getAll();
+    tx.oncomplete = () => resolve(req.result ?? []);
+    tx.onerror = () => reject(tx.error);
+  });
+}
+
+export async function deleteDocument(id: string): Promise<void> {
+  const db = await openDB();
+  return new Promise((resolve, reject) => {
+    const tx = db.transaction([STORE_DOCUMENTS, STORE_CHUNKS], "readwrite");
+    const docStore = tx.objectStore(STORE_DOCUMENTS);
+    const chunkStore = tx.objectStore(STORE_CHUNKS);
+    docStore.delete(id);
+    const index = chunkStore.index("byDocument");
+    const range = IDBKeyRange.only(id);
+    const req = index.openCursor(range);
+    req.onsuccess = () => {
+      const cursor = req.result;
+      if (cursor) {
+        cursor.delete();
+        cursor.continue();
+      }
+    };
+    tx.oncomplete = () => resolve();
+    tx.onerror = () => reject(tx.error);
+  });
+}
+
+export async function putChunks(chunks: KnowledgeChunk[]): Promise<void> {
+  if (chunks.length === 0) return;
+  const db = await openDB();
+  return new Promise((resolve, reject) => {
+    const tx = db.transaction(STORE_CHUNKS, "readwrite");
+    const store = tx.objectStore(STORE_CHUNKS);
+    for (const chunk of chunks) store.put(chunk);
+    tx.oncomplete = () => resolve();
+    tx.onerror = () => reject(tx.error);
+  });
+}
+
+export async function getAllChunks(): Promise<KnowledgeChunk[]> {
+  const db = await openDB();
+  return new Promise((resolve, reject) => {
+    const tx = db.transaction(STORE_CHUNKS, "readonly");
+    const req = tx.objectStore(STORE_CHUNKS).getAll();
+    tx.oncomplete = () => resolve(req.result ?? []);
+    tx.onerror = () => reject(tx.error);
+  });
+}

lib/knowledge/embeddings.ts

@@ -0,0 +1,59 @@
+/**
+ * Browser-side embeddings via @xenova/transformers.
+ * Lazy-loads the pipeline on first use.
+ */
+
+const MODEL = "Xenova/all-MiniLM-L6-v2";
+
+let pipelinePromise: Promise<unknown> | null = null;
+
+async function getPipeline(): Promise<unknown> {
+  if (typeof window === "undefined") {
+    throw new Error("Embeddings are only available in the browser");
+  }
+  if (!pipelinePromise) {
+    const { pipeline } = await import("@xenova/transformers");
+    pipelinePromise = pipeline("feature-extraction", MODEL);
+  }
+  return pipelinePromise;
+}
+
+/** Embed a single text. Returns normalized vector for cosine similarity. */
+export async function embed(text: string): Promise<number[]> {
+  const pipe = (await getPipeline()) as (input: string, options?: { pooling?: string; normalize?: boolean }) => Promise<{ data: Float32Array }>;
+  const output = await pipe(text, { pooling: "mean", normalize: true });
+  const data = output.data;
+  if (!data) throw new Error("Embedding output has no data");
+  return Array.from(data);
+}
+
+/** Embed multiple texts in one batch (more efficient). */
+export async function embedBatch(texts: string[]): Promise<number[][]> {
+  if (texts.length === 0) return [];
+  const pipe = (await getPipeline()) as (input: string | string[], options?: { pooling?: string; normalize?: boolean }) => Promise<{ data: Float32Array; dims: number[] }>;
+  const output = await pipe(texts, { pooling: "mean", normalize: true });
+  const data = output.data;
+  const dims = output.dims;
+  if (!data || !dims?.length) throw new Error("Batch embedding output has no data");
+  const dim = dims[dims.length - 1] ?? data.length;
+  const results: number[][] = [];
+  for (let i = 0; i < dims[0]; i++) {
+    const start = i * dim;
+    results.push(Array.from(data.slice(start, start + dim)));
+  }
+  return results;
+}
+
+export function cosineSimilarity(a: number[], b: number[]): number {
+  if (a.length !== b.length) return 0;
+  let dot = 0;
+  let normA = 0;
+  let normB = 0;
+  for (let i = 0; i < a.length; i++) {
+    dot += a[i] * b[i];
+    normA += a[i] * a[i];
+    normB += b[i] * b[i];
+  }
+  const denom = Math.sqrt(normA) * Math.sqrt(normB);
+  return denom === 0 ? 0 : dot / denom;
+}

lib/knowledge/index.ts

@@ -0,0 +1,10 @@
+/**
+ * Client-side knowledge store for RAG.
+ * Use from "use client" components only (IndexedDB + Transformers.js run in browser).
+ */
+
+export type { KnowledgeDocument, KnowledgeChunk, KnowledgeItemMeta, RetrieveResult, DocumentType } from "./types";
+export { listKnowledgeItems, addPdfFile, addNote, addUrl, removeKnowledgeDocument, retrieve } from "./store";
+export { chunkText } from "./chunk";
+export { extractTextFromPdf } from "./pdf";
+export { embed, embedBatch, cosineSimilarity } from "./embeddings";

lib/knowledge/pdf.ts

@@ -0,0 +1,28 @@
+/**
+ * PDF text extraction in the browser using pdfjs-dist.
+ * Only runs client-side.
+ */
+
+export async function extractTextFromPdf(file: File): Promise<string> {
+  if (typeof window === "undefined") {
+    throw new Error("PDF extraction is only available in the browser");
+  }
+  const pdfjsLib = await import("pdfjs-dist");
+  const GlobalWorkerOptions = (pdfjsLib as unknown as { GlobalWorkerOptions?: { workerSrc: string } }).GlobalWorkerOptions;
+  if (GlobalWorkerOptions && !GlobalWorkerOptions.workerSrc) {
+    GlobalWorkerOptions.workerSrc = "https://unpkg.com/pdfjs-dist@4.7.76/build/pdf.worker.min.mjs";
+  }
+  const arrayBuffer = await file.arrayBuffer();
+  const doc = await pdfjsLib.getDocument({ data: arrayBuffer }).promise;
+  const numPages = doc.numPages;
+  const parts: string[] = [];
+  for (let i = 1; i <= numPages; i++) {
+    const page = await doc.getPage(i);
+    const content = await page.getTextContent();
+    const text = content.items
+      .map((item) => ("str" in item ? item.str : ""))
+      .join("");
+    parts.push(text);
+  }
+  return parts.join("\n\n").trim();
+}

lib/knowledge/store.ts

@@ -0,0 +1,156 @@
+/**
+ * High-level knowledge store: add documents, remove, list, retrieve.
+ * Coordinates IndexedDB, chunking, PDF extraction, and embeddings.
+ */
+
+import { v4 as uuidv4 } from "uuid";
+import type { KnowledgeDocument, KnowledgeChunk, KnowledgeItemMeta, RetrieveResult } from "./types";
+import { openDB, putDocument, putChunks, listDocuments, deleteDocument, getAllChunks } from "./db";
+import { chunkText } from "./chunk";
+import { extractTextFromPdf } from "./pdf";
+import { embed, cosineSimilarity } from "./embeddings";
+
+const TOP_K = 8;
+
+function docToMeta(doc: KnowledgeDocument): KnowledgeItemMeta {
+  return {
+    id: doc.id,
+    type: doc.type,
+    label: doc.label,
+    meta: doc.type === "file" ? undefined : doc.url,
+    status: "indexed",
+  };
+}
+
+/** List all documents as UI items. */
+export async function listKnowledgeItems(): Promise<KnowledgeItemMeta[]> {
+  const docs = await listDocuments();
+  return docs.map(docToMeta);
+}
+
+/** Add a PDF file: extract text, chunk, embed, store. */
+export async function addPdfFile(file: File): Promise<KnowledgeItemMeta> {
+  const text = await extractTextFromPdf(file);
+  const id = uuidv4();
+  const doc: KnowledgeDocument = {
+    id,
+    type: "file",
+    label: file.name,
+    createdAt: new Date().toISOString(),
+  };
+  await putDocument(doc);
+  const chunks = chunkText(text);
+  const chunksWithEmbeddings: KnowledgeChunk[] = [];
+  for (let i = 0; i < chunks.length; i++) {
+    const ch = chunks[i];
+    const embedding = await embed(ch.text);
+    chunksWithEmbeddings.push({
+      id: `${id}-chunk-${i}`,
+      documentId: id,
+      text: ch.text,
+      embedding,
+      startIndex: ch.startIndex,
+      endIndex: ch.endIndex,
+    });
+  }
+  await putChunks(chunksWithEmbeddings);
+  return docToMeta(doc);
+}
+
+/** Add a note: chunk, embed, store. */
+export async function addNote(label: string): Promise<KnowledgeItemMeta> {
+  const id = uuidv4();
+  const doc: KnowledgeDocument = {
+    id,
+    type: "note",
+    label: label.slice(0, 80) + (label.length > 80 ? "…" : ""),
+    createdAt: new Date().toISOString(),
+  };
+  await putDocument(doc);
+  const chunks = chunkText(label);
+  const chunksWithEmbeddings: KnowledgeChunk[] = [];
+  for (let i = 0; i < chunks.length; i++) {
+    const ch = chunks[i];
+    const embedding = await embed(ch.text);
+    chunksWithEmbeddings.push({
+      id: `${id}-chunk-${i}`,
+      documentId: id,
+      text: ch.text,
+      embedding,
+      startIndex: ch.startIndex,
+      endIndex: ch.endIndex,
+    });
+  }
+  if (chunksWithEmbeddings.length === 0) {
+    const embedding = await embed(label);
+    chunksWithEmbeddings.push({
+      id: `${id}-chunk-0`,
+      documentId: id,
+      text: label,
+      embedding,
+      startIndex: 0,
+      endIndex: label.length,
+    });
+  }
+  await putChunks(chunksWithEmbeddings);
+  return docToMeta(doc);
+}
+
+/** Add a URL as reference (no fetch in v1). Stored as document with one placeholder chunk so we can return contextUrls. */
+export async function addUrl(url: string): Promise<KnowledgeItemMeta> {
+  const id = uuidv4();
+  const doc: KnowledgeDocument = {
+    id,
+    type: "url",
+    label: url,
+    url,
+    createdAt: new Date().toISOString(),
+  };
+  await putDocument(doc);
+  const embedding = await embed(`URL: ${url}`);
+  await putChunks([
+    {
+      id: `${id}-chunk-0`,
+      documentId: id,
+      text: url,
+      embedding,
+      startIndex: 0,
+      endIndex: url.length,
+    },
+  ]);
+  return docToMeta(doc);
+}
+
+/** Remove a document and all its chunks. */
+export async function removeKnowledgeDocument(id: string): Promise<void> {
+  await deleteDocument(id);
+}
+
+/** Retrieve relevant context for a query: embed query, top-k similarity, build retrievedContext + contextUrls. */
+export async function retrieve(query: string): Promise<RetrieveResult> {
+  const chunks = await getAllChunks();
+  if (chunks.length === 0) {
+    return { retrievedContext: "", contextUrls: [] };
+  }
+  const docs = await listDocuments();
+  const docMap = new Map(docs.map((d) => [d.id, d]));
+  const queryEmbedding = await embed(query);
+  const withScore = chunks.map((ch) => ({
+    chunk: ch,
+    score: cosineSimilarity(ch.embedding, queryEmbedding),
+  }));
+  withScore.sort((a, b) => b.score - a.score);
+  const top = withScore.slice(0, TOP_K);
+  const parts: string[] = [];
+  const urlSet = new Set<string>();
+  for (const { chunk } of top) {
+    const doc = docMap.get(chunk.documentId);
+    const source = doc ? doc.label : "Unknown";
+    parts.push(`[${source}]\n${chunk.text}`);
+    if (doc?.type === "url" && doc.url) urlSet.add(doc.url);
+  }
+  return {
+    retrievedContext: parts.join("\n\n"),
+    contextUrls: Array.from(urlSet),
+  };
+}

lib/knowledge/types.ts

@@ -0,0 +1,37 @@
+/**
+ * Client-side knowledge store types.
+ * Documents and chunks are persisted in IndexedDB.
+ */
+
+export type DocumentType = "file" | "url" | "note";
+
+export interface KnowledgeDocument {
+  id: string;
+  type: DocumentType;
+  label: string;
+  url?: string;
+  createdAt: string;
+}
+
+export interface KnowledgeChunk {
+  id: string;
+  documentId: string;
+  text: string;
+  embedding: number[];
+  startIndex: number;
+  endIndex: number;
+}
+
+/** UI-facing item (matches existing KnowledgeItem shape). */
+export interface KnowledgeItemMeta {
+  id: string;
+  type: DocumentType;
+  label: string;
+  meta?: string;
+  status?: "pending" | "indexing" | "indexed" | "error";
+}
+
+export interface RetrieveResult {
+  retrievedContext: string;
+  contextUrls: string[];
+}

next.config.ts

@@ -0,0 +1,11 @@
+import type { NextConfig } from "next";
+
+const nextConfig: NextConfig = {
+  // Required for @huggingface/transformers in API routes: use Node runtime
+  // for onnxruntime-node instead of bundling (which can break or hang).
+  serverExternalPackages: ["onnxruntime-node"],
+  // Smaller Docker image: output standalone server + minimal node_modules
+  output: "standalone",
+};
+
+export default nextConfig;

package.json

@@ -0,0 +1,39 @@
+{
+  "name": "deeptrust",
+  "version": "0.1.0",
+  "private": true,
+  "scripts": {
+    "dev": "next dev",
+    "build": "npm run build:worker && next build --webpack",
+    "build:worker": "tsc -p tsconfig.worker.json",
+    "start": "next start",
+    "lint": "eslint"
+  },
+  "dependencies": {
+    "@anthropic-ai/sdk": "^0.78.0",
+    "@huggingface/transformers": "^3.8.1",
+    "@xenova/transformers": "^2.17.2",
+    "next": "16.1.6",
+    "pdfjs-dist": "^4.7.76",
+    "ollama": "^0.6.3",
+    "react": "19.2.3",
+    "react-dom": "19.2.3"
+  },
+  "devDependencies": {
+    "@langchain/core": "^1.1.28",
+    "@langchain/langgraph": "^1.1.5",
+    "@tailwindcss/postcss": "^4",
+    "@types/node": "^20",
+    "@types/react": "^19",
+    "@types/react-dom": "^19",
+    "@types/uuid": "^10.0.0",
+    "eslint": "^9",
+    "eslint-config-next": "16.1.6",
+    "install": "^0.13.0",
+    "npm": "^11.11.0",
+    "tailwindcss": "^4",
+    "typescript": "^5",
+    "uuid": "^13.0.0",
+    "zod": "^4.3.6"
+  }
+}

postcss.config.mjs

@@ -0,0 +1,7 @@
+const config = {
+  plugins: {
+    "@tailwindcss/postcss": {},
+  },
+};
+
+export default config;