diff --git a/.cursor/plans/client-side_rag_knowledge_upload_c1d69919.plan.md b/.cursor/plans/client-side_rag_knowledge_upload_c1d69919.plan.md new file mode 100644 index 0000000000000000000000000000000000000000..832373798841f15a14ce28743400ed49a2d78514 --- /dev/null +++ b/.cursor/plans/client-side_rag_knowledge_upload_c1d69919.plan.md @@ -0,0 +1,141 @@ +--- +name: Client-side RAG knowledge upload +overview: "Implement the Context panel as a real client-side RAG flow: persist PDFs, notes, and URLs in IndexedDB, index them with in-browser embeddings and a vector store, retrieve relevant chunks at query time, and pass that context to the research agent so the LLM uses it when planning and synthesizing." +todos: [] +isProject: false +--- + +# Client-Side RAG: Knowledge Upload and Retrieval + +## Current state + +- **Frontend** ([app/page.tsx](app/page.tsx)): The Context panel stores only metadata in React state (`KnowledgeItem`: id, type, label, meta). File content is never read; notes are plain text; URLs are just strings. No persistence, no embeddings, no retrieval. +- **API** ([app/api/research/route.ts](app/api/research/route.ts)): Reads only `query` from the body; `knowledge` is ignored. +- **Agent** ([lib/agent/graph.ts](lib/agent/graph.ts), [lib/agent/state.ts](lib/agent/state.ts)): `runResearch(userQuery, sessionName)` and `createInitialState({ threadId, userQuery, sessionName })`; no field for retrieved context. The Thinker builds the prompt from `state.userQuery` only ([lib/agent/nodes/thinker.ts](lib/agent/nodes/thinker.ts) line 117). + +## Target flow + +```mermaid +sequenceDiagram + participant User + participant UI + participant IndexedDB + participant VectorStore + participant API + participant Agent + + User->>UI: Add PDF / note / URL + UI->>UI: Extract text (PDF: pdf.js; note: as-is; URL: see below) + UI->>UI: Chunk and embed (browser) + UI->>VectorStore: Store chunks + vectors in IndexedDB + User->>UI: Run research + UI->>VectorStore: Embed query; top-k similarity search + UI->>API: POST { query, retrievedContext } + API->>Agent: runResearch(query, { knowledgeContext }) + Agent->>Agent: Thinker / Synthesizer use knowledgeContext in prompts +``` + + + +## 1. Client: Storage, extraction, embedding, and retrieval + +### 1.1 Persistence and data model + +- Introduce a **client-side knowledge store** backed by **IndexedDB** (separate from the existing React state so refreshes and future sessions retain data). +- **Schema** (conceptual): + - **Documents**: `id`, `type` (file | url | note), `label`, `createdAt`, optional `url` for type url. + - **Chunks**: `id`, `documentId`, `text`, `embedding` (array of numbers), `startIndex`, `endIndex`. +- Use a single IndexedDB database (e.g. `deeptrust-knowledge`) with object stores for documents and chunks (or a library that wraps this). + +### 1.2 Text extraction + +- **PDFs**: Use **pdf.js** (Mozilla) in the browser to extract text from dropped/selected files. No server round-trip; keep PDFs client-side only. +- **Notes**: Use the note text as a single “document”; optional chunking by paragraph or fixed size. +- **URLs**: Two options (choose one for v1): + - **A (recommended for scope)** + Store URL as metadata only. Do **not** fetch or embed URL content in the browser (avoids CORS and complexity). When building the research payload, send the list of “context URLs” with the request; the backend can pass them into the Thinker so the plan may include `document_fetch` steps for those URLs, or the Thinker can be prompted to consider “the user provided these URLs as references” in the plan. + - **B (full client-side)** + Fetch URL in the browser (e.g. via a Next.js API route that proxies the fetch to avoid CORS), extract text (HTML → text), then chunk and embed like PDFs. Adds proxy and error handling. + +Recommendation: **A** for the first iteration; document B as a follow-up. + +### 1.3 Chunking and embedding in the browser + +- **Chunking**: Split document text into overlapping or fixed-size segments (e.g. 256–512 tokens or ~500 chars with 50–100 char overlap). No server call. +- **Embeddings**: Use a **browser-run embedding model** so everything stays client-side: + - **Option A**: **Transformers.js** (`@xenova/transformers`) with a small feature-extraction model (e.g. `Xenova/all-MiniLM-L6-v2` or similar). Runs in WebAssembly; single dependency consistent with “local-first”. + - **Option B**: A dedicated client-side vector DB that includes embeddings (e.g. **idbvec** with external embeddings, or **entity-db** which uses Transformers.js under the hood). Prefer one that stores vectors in IndexedDB and supports similarity search. +- Store each chunk’s embedding in IndexedDB with a reference to its document and text. + +### 1.4 Vector store and retrieval + +- **Option A**: **idbvec** (`@brainwires/idbvec`): WASM + IndexedDB, HNSW index, configurable distance. You supply embeddings (e.g. from Transformers.js); idbvec handles storage and k-NN search. +- **Option B**: **Custom**: Store chunks (with embeddings) in IndexedDB; on query, load relevant chunks and run brute-force cosine similarity (or a tiny WASM k-NN) in a worker to avoid blocking the main thread. Good for small corpora (< ~10k chunks). +- At “Run research” time: embed the user query with the same model, run top-k similarity search (e.g. k = 5–10), build a single `retrievedContext` string (e.g. concatenate chunk texts with source labels) and send it with the request. + +### 1.5 UI and sync with existing Context panel + +- Keep the existing **Context panel** UX: drag-and-drop, “Attach files”, URL input, “Short note”, and the list of items. +- **On add**: + - For **files** (PDF): read file, extract text with pdf.js, chunk, embed, and write documents + chunks (+ vectors) to IndexedDB. Add a corresponding entry to React state for the list (label, type, id) and optionally sync the list from IndexedDB on load. + - For **notes**: create one document, chunk if needed, embed, store; update UI list. + - For **URLs** (v1): store URL in IndexedDB as a document with no chunks (or a single placeholder chunk with URL as “text”); backend will receive “context URLs” and can use them in the prompt or in plan steps. +- **On remove**: Delete document (and its chunks) from IndexedDB and from React state. Add a remove control next to each item in the list (currently there is none). +- **On load**: Hydrate the Context list from IndexedDB so persisted knowledge survives refresh. + +## 2. Backend: Accept and use retrieved context + +### 2.1 Research API + +- In [app/api/research/route.ts](app/api/research/route.ts), read from the request body: `query` and either `retrievedContext` (string) or `knowledge` (array or object that includes `retrievedContext` and optionally `contextUrls: string[]`). +- Call the agent with this context, e.g. `runResearch(query, sessionName, { retrievedContext, contextUrls })`. + +### 2.2 Agent state and graph + +- In [lib/agent/state.ts](lib/agent/state.ts), add an optional field to `ResearchState`, e.g. +`knowledgeContext: z.string().optional()` +and optionally `contextUrls: z.array(z.string()).optional()`. +- In [lib/agent/graph.ts](lib/agent/graph.ts), add the new channel(s) to the state graph (e.g. `knowledgeContext: { value: (_, n) => n }`) and ensure they are part of the initial state when provided. +- Extend [lib/agent/state.ts](lib/agent/state.ts) `createInitialState` to accept optional `knowledgeContext` (and `contextUrls` if used), and pass them through in [lib/agent/graph.ts](lib/agent/graph.ts) when calling `createInitialState` and streaming. + +### 2.3 Thinker and Synthesizer + +- In [lib/agent/nodes/thinker.ts](lib/agent/nodes/thinker.ts), when building the user message (around line 117), if `state.knowledgeContext` is present, append it to the prompt, e.g. +“The user provided the following retrieved context from their local knowledge base. Use it to inform the plan and prefer steps that leverage this context where relevant:\n\n” + state.knowledgeContext. +If `state.contextUrls` is used, add a line like “The user also referenced these URLs: …” so the Thinker can emit `document_fetch` steps for them. +- In [lib/agent/nodes/synthesizer.ts](lib/agent/nodes/synthesizer.ts), if `state.knowledgeContext` is present, include it in the context passed to the LLM when synthesizing the final report (so the report can cite or summarize the user’s local knowledge). + +## 3. Dependencies and build + +- Add npm packages: e.g. **pdfjs-dist** (or **react-pdf** if you prefer a React wrapper) for PDF text extraction; **@xenova/transformers** (or the chosen embedding solution); and **idbvec** or a minimal IndexedDB + vector search helper. +- Ensure the embedding model is loaded only when the user first adds knowledge (or on first “Run research” with non-empty context) to avoid blocking initial page load; consider a small “Preparing knowledge…” state in the UI when the embedding pipeline is loading. + +## 4. Scope summary + + +| Area | In scope | +| ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Client | IndexedDB persistence for documents/chunks; PDF text extraction (pdf.js); notes as documents; URLs stored as references (no client fetch in v1). | +| Client | Browser embeddings (Transformers.js or equivalent); vector store in IndexedDB (idbvec or custom); top-k retrieval at query time; send `retrievedContext` (+ optional `contextUrls`) with POST /api/research. | +| Client | Context panel: remove item; hydrate list from IndexedDB on load; show “indexed” or “ready” per item after processing. | +| Backend | Research route parses `retrievedContext` (and optional `contextUrls`); runResearch(..., { knowledgeContext, contextUrls }); state + createInitialState + graph channels; Thinker and Synthesizer include context in prompts. | + + +## 5. Files to add or touch + +- **New**: `lib/knowledge/` (or `app/lib/knowledge/`) — IndexedDB schema, chunking, embedding pipeline (or wrapper around Transformers.js + idbvec), and retrieval function. Keep it UI-agnostic so it can be called from the Context panel and before `fetch('/api/research')`. +- **New** (optional): `app/components/KnowledgePanel.tsx` — Extract the Context panel into a component that uses the knowledge store and exposes “items” + “retrievedContext” for the parent. +- **Edit**: [app/page.tsx](app/page.tsx) — Wire file drop/note/URL to the knowledge store; before runResearch, call retrieval and send `retrievedContext` (and `contextUrls`) in the request body; add remove button; hydrate from IndexedDB. +- **Edit**: [app/api/research/route.ts](app/api/research/route.ts) — Read `retrievedContext` / `contextUrls`; pass to runResearch. +- **Edit**: [lib/agent/state.ts](lib/agent/state.ts) — Add `knowledgeContext` (and optionally `contextUrls`); extend `createInitialState`. +- **Edit**: [lib/agent/graph.ts](lib/agent/graph.ts) — Add channel(s); pass context into `createInitialState` in runResearch. +- **Edit**: [lib/agent/nodes/thinker.ts](lib/agent/nodes/thinker.ts) — Include `state.knowledgeContext` (and URLs) in the Thinker prompt. +- **Edit**: [lib/agent/nodes/synthesizer.ts](lib/agent/nodes/synthesizer.ts) — Include `state.knowledgeContext` in the synthesis prompt. +- **Edit**: [lib/agent/index.ts](lib/agent/index.ts) — Export any new types if needed; ensure runResearch signature is updated and documented. + +## 6. Optional follow-ups + +- **URL content in browser**: Proxy fetch in an API route; extract text; chunk and embed like PDFs so URL content is fully in the vector store. +- **Delete-all / export**: Clear all knowledge from IndexedDB; export documents/chunks as JSON. +- **Progress UX**: Show “Extracting…”, “Embedding…”, “Indexed” per file so large PDFs don’t look stuck. + diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000000000000000000000000000000000000..2c504bb5a7cb6b87e0cc9fc4d054c8bf131d509d --- /dev/null +++ b/.dockerignore @@ -0,0 +1,30 @@ +# Dependencies (reinstalled in image) +node_modules +npm-debug.log* +.pnpm-debug.log* +.yarn* + +# Build outputs (rebuilt in image) +.next +dist +out + +# Git and IDE +.git +.gitignore +.cursor +.vscode +*.md +!README.md + +# Env and secrets +.env +.env.* +!.env.example + +# Misc +.DS_Store +*.log +coverage +.nyc_output +.hf-cache diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000000000000000000000000000000000000..9541e334069513931eebdd284dde91f6be4e049d --- /dev/null +++ b/.gitignore @@ -0,0 +1,47 @@ +# See https://help.github.com/articles/ignoring-files/ for more about ignoring files. + +# dependencies +/node_modules +/.pnp +.pnp.* +.yarn/* +!.yarn/patches +!.yarn/plugins +!.yarn/releases +!.yarn/versions + +# testing +/coverage + +# next.js +/.next/ +/out/ + +# production +/build + +# misc +.DS_Store +*.pem + +# debug +npm-debug.log* +yarn-debug.log* +yarn-error.log* +.pnpm-debug.log* + +# env files (can opt-in for committing if needed) +.env* + +# vercel +.vercel + +# typescript +*.tsbuildinfo +next-env.d.ts + +# huggingface model cache +.hf-cache/ + +# HF Spaces reject binary files in git; keep .pptx out of history +docs/*.pptx diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000000000000000000000000000000000..36827804a3e99a4223feba854ee89e8646434723 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,16 @@ +# DeepTrust — Agent context + +**What this is:** TypeScript research agent (LangGraph state graph + local LLM + Next.js). Plan → audit → tools → synthesize. Real-time workspace UI with SSE streaming and context/knowledge upload. + +**Where the truth lives:** Structure and features change over time. Prefer these for current details: +- **README.md** — Overview, stack, project structure, API, frontend, run/config. +- **docs/ARCHITECTURE.md** — Low-level design: state machine, nodes, routing, LLM layer, SSE protocol, frontend architecture. + +**Rough layout (may evolve):** +- `lib/agent/` — Graph, state, nodes, LLM client (worker thread), utils. +- `app/` — Next.js app; `page.tsx` = workspace UI; `app/api/` = research + model endpoints. +- `dist/llm/` — Built worker (from `npm run build:worker`). + +**Commands:** `npm run build:worker` before first run; `npm run dev` to develop; `npm run build` then `npm run start` for production. + +**When editing:** Follow existing patterns in the area you change. For agent graph or state, check `lib/agent/state.ts` and `lib/agent/graph.ts`; for API/streaming, see the route handlers and README/ARCHITECTURE. diff --git a/DEPLOY.md b/DEPLOY.md new file mode 100644 index 0000000000000000000000000000000000000000..a1c0b9294e27421667ec6cd3f2a73b8b366ca3d3 --- /dev/null +++ b/DEPLOY.md @@ -0,0 +1,47 @@ +# Deploying DeepTrust on Render + +The app is containerized with **Docker**. You do **not** need Docker installed locally—Render builds the image in the cloud when you push. + +## Prerequisites + +- Git repo pushed to GitHub (or GitLab) +- Account on [Render](https://render.com) + +## Deploy steps + +1. Go to [dashboard.render.com](https://dashboard.render.com), connect your GitHub repo. +2. Add a **Web Service**. +3. Set: + - **Build:** Docker (Render will use the repo `Dockerfile`). + - **Plan:** See [Memory and plan](#memory-and-plan) below. +4. Create the service. Render builds and deploys on each push. + +**Blueprint:** Alternatively, use the repo’s `render.yaml` (Blueprint deploy) so the service is defined in code. + +## Memory and plan + +Render **Free** instances have **512 MB RAM**. Model loading often exceeds this: + +- **SmolLM2-360M (q4)** ≈ 300–450 MB for weights alone, plus Node, Next.js, worker, and inference → typically **OOM on Free** (502 when loading). +- **SmolLM2-135M (q4)** ≈ 150–200 MB → may work on Free but is tight; 502 can still occur under load. + +**Recommendations:** + +- **Free tier (512 MB):** Set env `HF_MODEL=HuggingFaceTB/SmolLM2-135M-Instruct` and use the 135M model (q4) in the UI. If you still get 502 on “Load model”, the instance is likely OOM — upgrade to Starter. +- **Starter (2 GB) or higher:** Safe for SmolLM2-360M (q4) and 135M. Use Starter if you need 360M or reliable inference. + +## Environment variables (optional) + +- `HF_MODEL` — Hugging Face model id (default in code). On Free tier, use `HuggingFaceTB/SmolLM2-135M-Instruct` to reduce OOM risk. +- `HF_CACHE_DIR` — Where to cache model files (default `./.hf-cache`). On ephemeral disks this is lost between restarts; models re-download on cold start. +- `DEEPTRUST_LOAD_MODEL_AT_STARTUP` — Set to `0` or `false` to disable loading the model when the server starts (default: load at startup). Use this only if you prefer to load on first user request. + +## Hugging Face Spaces (Docker) + +When running as a **Docker Space**, the app **skips startup model preload** so the Space becomes healthy within the 30‑minute launch timeout. You’ll see in the logs: `[instrumentation] Hugging Face Space detected; skipping startup model preload (model will load on first use).` The model loads on first use (e.g. when you click “Load model” or run research). If your Space still times out, set the variable `DEEPTRUST_LOAD_MODEL_AT_STARTUP=0` in the Space’s **Settings → Variables**. In the Space README YAML you can set `app_port: 3000` to match the Dockerfile. + +## Notes + +- **Model at startup:** The app loads the default model (from `HF_MODEL`, or 135M q4) when the Node server starts, so the first user request does not wait for download/load. (On Hugging Face Spaces this is skipped so the Space stays healthy.) If preload fails (e.g. OOM), the server still starts and the UI "Load model" button can be used to retry. +- **First deploy/cold start** can be slow while the model downloads and loads during server boot; subsequent requests are fast. +- **Standalone:** The image uses Next.js `output: "standalone"` for a smaller build and includes the compiled LLM worker under `dist/llm/`. diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000000000000000000000000000000000000..f9ddcb80ddffd716e314ad711060425354891af6 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,53 @@ +# DeepTrust — multi-stage build for Render +# Build runs in the cloud; no local Docker required. +# Uses node:20-slim (glibc) so onnxruntime-node and tokenizers native bindings work. +# Alpine (musl) causes 502 on model load due to incompatible native modules. + +FROM node:20-slim AS deps +WORKDIR /app +COPY package.json package-lock.json* ./ +RUN npm ci + +FROM node:20-slim AS builder +WORKDIR /app +COPY --from=deps /app/node_modules ./node_modules +COPY . . + +# Build LLM worker (dist/llm/) then Next.js (standalone) +ENV NEXT_TELEMETRY_DISABLED=1 +RUN npm run build + +# Pre-download the default model so the container doesn't fetch at runtime +ENV HF_CACHE_DIR=/app/.hf-cache-build +RUN mkdir -p /app/.hf-cache-build && node scripts/preload-hf-model.cjs + +FROM node:20-slim AS runner +WORKDIR /app + +ENV NODE_ENV=production +ENV NEXT_TELEMETRY_DISABLED=1 + +RUN groupadd --gid 1001 nodejs && useradd --uid 1001 --gid nodejs --shell /bin/false nextjs + +# Standalone output: server + minimal node_modules +COPY --from=builder /app/.next/standalone ./ +COPY --from=builder /app/.next/static ./.next/static +COPY --from=builder /app/public ./public +# LLM worker (required at runtime; path is process.cwd()/dist/llm) +COPY --from=builder /app/dist ./dist +# Worker runs in a separate thread and requires these at runtime; standalone does not trace them. +COPY --from=builder /app/package.json ./package.json +COPY --from=builder /app/package-lock.json ./package-lock.json +RUN npm install @huggingface/transformers onnxruntime-node --omit=dev --ignore-scripts --no-save + +# Pre-downloaded model cache (avoids runtime download; worker reads from here) +COPY --from=builder --chown=nextjs:nodejs /app/.hf-cache-build ./.hf-cache +ENV HF_CACHE_DIR=/app/.hf-cache + +USER nextjs + +EXPOSE 3000 +ENV PORT=3000 +ENV HOSTNAME="0.0.0.0" + +CMD ["node", "server.js"] diff --git a/README.md b/README.md new file mode 100644 index 0000000000000000000000000000000000000000..67bca2a84717411d0a90f695e49c1812ebcbecae --- /dev/null +++ b/README.md @@ -0,0 +1,643 @@ +--- +title: DeepTrust Research Agent +emoji: 🔬 +colorFrom: blue +colorTo: indigo +sdk: docker +app_port: 3000 +pinned: false +short_description: LangGraph research agent with local LLM and Next.js workspace. +tags: + - langgraph + - nextjs + - transformers + - research-agent + - llm + - typescript +--- + +# DeepTrust Research Agent + +A TypeScript implementation of an autonomous research agent: LangGraph state machines, local LLM inference (Hugging Face Transformers in a worker thread), and a real-time, AI-centric Next.js workspace. The UI is designed for a Cursor/Gemini-like flow—immediate feedback, Server-Sent Events (SSE) streaming, optimistic updates, a knowledge/context drop zone, and quick-action chips—so the full application from graph nodes to the browser is understandable in one read. + +## Documentation + +- **[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)** — Low-level design: state machine, channels, routing, LLM layer, API protocols, and frontend architecture (SSE, streaming UX, knowledge flow). + +## Table of Contents + +1. [Architecture Overview](#architecture-overview) +2. [Technology Stack](#technology-stack) +3. [Project Structure](#project-structure) +4. [State Management](#state-management) +5. [Graph Nodes](#graph-nodes) +6. [Routing and Conditional Edges](#routing-and-conditional-edges) +7. [LLM Integration](#llm-integration) +8. [API Layer](#api-layer) +9. [Frontend: Real-Time Workspace](#frontend-real-time-workspace) +10. [Running the Project](#running-the-project) +11. [Configuration](#configuration) + +--- + +## Architecture Overview + +DeepTrust implements a cyclic state graph where a research query flows through multiple specialized nodes: + +``` +┌──────────────────────────────────────────────────────────┐ +│ │ +│ [START] ──► thinker ──► auditor ──► tool_executor │ +│ ▲ │ │ │ +│ │ reject │ │ │ +│ └───────────┘ ▼ │ +│ synthesizer │ +│ │ │ +│ [END] │ +└──────────────────────────────────────────────────────────┘ +``` + +### Node Responsibilities + +- **Thinker**: Decomposes a research question into a structured, multi-step plan +- **Auditor**: Validates the plan against organizational policy; rejects non-compliant plans +- **HITL Gate**: Pauses execution for human approval before tool execution +- **Tool Executor**: Executes each plan step sequentially (web search, document fetch, etc.) +- **Synthesizer**: Aggregates tool outputs into a final research report + +The graph supports revision loops: if the Auditor rejects a plan, control returns to the Thinker with structured feedback. A configurable ceiling (`maxPlanRevisions`) prevents infinite loops. + +--- + +## Technology Stack + +| Layer | Technology | Purpose | +|-------|------------|---------| +| **Frontend** | Next.js 16, React 19, Tailwind CSS 4 | Server-side rendering, streaming UI updates | +| **State Machine** | LangGraph.js | Graph construction, checkpointing, conditional routing | +| **LLM Inference** | @huggingface/transformers | Local model loading and text generation | +| **Schema Validation** | Zod 4 | Runtime type validation for state and API contracts | +| **Type System** | TypeScript 5 | Static type safety across the codebase | + +### Key Dependencies + +```json +{ + "@langchain/langgraph": "^1.1.5", + "@huggingface/transformers": "^3.8.1", + "zod": "^4.3.6", + "next": "16.1.6", + "react": "19.2.3" +} +``` + +--- + +## Project Structure + +``` +lib/agent/ +├── index.ts # Public API exports +├── graph.ts # StateGraph construction and compilation +├── state.ts # Zod schemas and TypeScript types +├── routing.ts # Conditional edge functions +├── llm/ +│ ├── index.ts # Worker proxy: loadModel, chatComplete, getModelStatus +│ ├── pipeline.ts # Pipeline config (used by worker) +│ └── worker-entry.ts # Worker entry: runs Transformers in a separate thread +├── nodes/ +│ ├── index.ts # Node exports +│ ├── thinker.ts # Plan generation node +│ ├── auditor.ts # Policy validation node +│ ├── hitl-gate.ts # Human approval checkpoint +│ ├── tool-executor.ts # Tool dispatch node +│ └── synthesizer.ts # Report synthesis node +└── utils/ + ├── index.ts # Utility exports + ├── extract-json.ts # Robust JSON parsing + └── policy.ts # Policy file loader + +dist/llm/ # Built by npm run build:worker +├── worker-entry.js # Compiled worker +└── pipeline.js # Pipeline bundle + +app/ +├── page.tsx # Workspace UI: chat, context panel, model card, SSE client +├── layout.tsx +├── globals.css +└── api/ + ├── research/ + │ └── route.ts # POST: SSE stream of research events + └── model/ + └── load/ + └── route.ts # GET/POST: model load + SSE progress +``` + +--- + +## State Management + +### The ResearchState Schema + +All state flows through a single Zod-validated schema. This ensures runtime type safety and enables serialization for checkpointing. + +```typescript +export const ResearchState = z.object({ + // Identity + threadId: z.string().uuid(), + sessionName: z.string().default("Unnamed Session"), + + // Input + userQuery: z.string().min(1), + + // Planning + plan: ResearchPlan.nullable().default(null), + rejectionFeedback: z.string().nullable().default(null), + planRevisionCount: z.number().int().nonnegative().default(0), + maxPlanRevisions: z.number().int().positive().default(5), + + // Auditing + auditResult: AuditResult.nullable().default(null), + + // Execution + currentStepIndex: z.number().int().nonnegative().default(0), + humanApproved: z.boolean().default(false), + + // Output + finalReport: z.string().nullable().default(null), + + // Observability + reasoning: z.array(ReasoningEntry).default([]), + status: RunStatus.default("idle"), + updatedAt: z.string().datetime(), + errorMessage: z.string().nullable().default(null), +}); +``` + +### Channel Configuration + +LangGraph requires explicit channel definitions for state merging. Most fields use last-write-wins semantics, but the `reasoning` array uses append-only concatenation: + +```typescript +const graph = new StateGraph({ + channels: { + threadId: { value: (_, n) => n }, + // ... other scalar fields use (_, n) => n + + // Append-only reasoning log + reasoning: { + value: (existing: ReasoningEntry[], incoming: ReasoningEntry[]) => + [...(existing ?? []), ...(incoming ?? [])], + default: () => [], + }, + }, +}); +``` + +### Sub-Schemas + +**ResearchStep**: A single action in the research plan. + +```typescript +export const ResearchStep = z.object({ + id: z.string().uuid(), + tool: z.enum(["web_search", "document_fetch", "code_interpreter", "summarize"]), + input: z.string().min(1), + rationale: z.string(), + output: z.string().optional(), + executedAt: z.string().datetime().optional(), +}); +``` + +**AuditResult**: Structured feedback from the Auditor. + +```typescript +export const AuditResult = z.object({ + verdict: z.enum(["approved", "rejected", "needs_revision"]), + policyViolations: z.array(z.string()).default([]), + suggestions: z.array(z.string()).default([]), + auditedAt: z.string().datetime(), +}); +``` + +--- + +## Graph Nodes + +Each node is an async function that receives the current state and returns a partial state update. + +### Thinker Node + +Generates or revises a research plan. Prompts the LLM with structured output requirements: + +```typescript +async function thinkerNode(state: ResearchState): Promise> { + const isRevision = state.planRevisionCount > 0 && state.rejectionFeedback; + + const system = ` +You are the Thinker node of DeepTrust, an autonomous research agent. +Return ONLY a valid JSON object matching: +{ + "objective": string, + "steps": Array<{ "id": UUID, "tool": string, "input": string, "rationale": string }>, + "estimatedTokenBudget": number, + "createdAt": ISO8601, + "revision": number +}`; + + const userMessage = isRevision + ? `Research question: "${state.userQuery}"\n\nPREVIOUS PLAN REJECTED:\n${state.rejectionFeedback}` + : `Research question: "${state.userQuery}"`; + + const rawThought = await chatComplete(system, userMessage); + const parsed = extractJSON(rawThought); + const plan = ResearchPlan.parse({ ...parsed, revision: state.planRevisionCount }); + + return { + plan, + status: "thinking", + rejectionFeedback: null, + reasoning: appendReasoning(state, { node: "thinker", summary: "..." }), + updatedAt: new Date().toISOString(), + }; +} +``` + +### Auditor Node + +Validates plans against `POLICY.md`. Returns structured violations and suggestions: + +```typescript +async function auditorNode(state: ResearchState): Promise> { + const policy = loadPolicy(); + + const system = ` +You are the Auditor node. Evaluate research plans against policy. +Return ONLY: { "verdict": "approved"|"rejected"|"needs_revision", ... }`; + + const rawThought = await chatComplete(system, `POLICY:\n${policy}\n\nPLAN:\n${JSON.stringify(state.plan)}`); + const auditResult = AuditResult.parse(extractJSON(rawThought)); + + const isRejected = auditResult.verdict !== "approved"; + + return { + auditResult, + rejectionFeedback: isRejected ? formatFeedback(auditResult) : null, + planRevisionCount: isRejected ? state.planRevisionCount + 1 : state.planRevisionCount, + status: isRejected ? "thinking" : "awaiting_approval", + }; +} +``` + +### HITL Gate Node + +Uses LangGraph's `interrupt()` primitive to pause execution and write a checkpoint: + +```typescript +async function hitlGateNode(state: ResearchState): Promise> { + interrupt({ + message: "Plan ready for review. Set humanApproved=true to continue.", + plan: state.plan, + auditResult: state.auditResult, + }); + + return { updatedAt: new Date().toISOString() }; +} +``` + +### Tool Executor Node + +Iterates through plan steps. Each invocation processes one step and increments `currentStepIndex`: + +```typescript +async function toolExecutorNode(state: ResearchState): Promise> { + const step = state.plan.steps[state.currentStepIndex]; + const output = await dispatchTool(step.tool, step.input); + + const updatedSteps = state.plan.steps.map((s, i) => + i === state.currentStepIndex ? { ...s, output, executedAt: new Date().toISOString() } : s + ); + + return { + plan: { ...state.plan, steps: updatedSteps }, + currentStepIndex: state.currentStepIndex + 1, + status: "executing", + }; +} +``` + +### Synthesizer Node + +Aggregates all step outputs into a final report: + +```typescript +async function synthesizerNode(state: ResearchState): Promise> { + const stepsContext = state.plan.steps + .map((s, i) => `Step ${i + 1} [${s.tool}]: ${s.output}`) + .join("\n\n"); + + const finalReport = await chatComplete( + "Write a comprehensive research report.", + `Objective: ${state.plan.objective}\n\nResults:\n${stepsContext}` + ); + + return { finalReport, status: "complete" }; +} +``` + +--- + +## Routing and Conditional Edges + +LangGraph uses routing functions to determine the next node based on current state. + +### Post-Audit Routing + +```typescript +function routeAfterAudit(state: ResearchState): "thinker" | "hitl_gate" | typeof END { + if (state.planRevisionCount >= state.maxPlanRevisions) { + return END; // Safety ceiling reached + } + if (state.auditResult?.verdict !== "approved") { + return "thinker"; // Loop back for revision + } + return "hitl_gate"; // Proceed to human approval +} +``` + +### Post-HITL Routing + +```typescript +function routeAfterHitl(state: ResearchState): "tool_executor" | typeof END { + if (!state.humanApproved) { + return END; // Fail-safe if approval missing + } + return "tool_executor"; +} +``` + +### Post-Tool Routing + +```typescript +function routeAfterToolStep(state: ResearchState): "tool_executor" | "synthesizer" { + if (state.currentStepIndex < state.plan.steps.length) { + return "tool_executor"; // More steps remain + } + return "synthesizer"; // All steps complete +} +``` + +--- + +## LLM Integration + +### Hugging Face Transformers + +The project uses `@huggingface/transformers` for local inference. Models are cached to `.hf-cache/` for persistence across restarts. + +```typescript +import { pipeline, TextGenerationPipeline, env } from "@huggingface/transformers"; + +env.cacheDir = process.env.HF_CACHE_DIR || "./.hf-cache"; + +const MODEL_ID = process.env.HF_MODEL || "HuggingFaceTB/SmolLM2-360M-Instruct"; + +let generatorPromise: Promise | null = null; + +export function loadModel(onProgress?: ProgressCallback): Promise { + if (generatorPromise) return generatorPromise; + + generatorPromise = pipeline("text-generation", MODEL_ID, { + progress_callback: (data) => { + onProgress?.({ + status: data.status === "progress" ? "downloading" : "loading", + progress: Math.round((data.progress || 0) * 100), + file: data.file || "", + message: `Downloading ${data.file?.split("/").pop()}`, + }); + }, + }); + + return generatorPromise; +} +``` + +### Chat Completion Interface + +```typescript +export async function chatComplete(systemPrompt: string, userMessage: string): Promise { + const generator = await loadModel(); + + const output = await generator( + [ + { role: "system", content: systemPrompt }, + { role: "user", content: userMessage }, + ], + { max_new_tokens: 4096, do_sample: true, temperature: 0.7 } + ); + + const result = output[0] as { generated_text: Array<{ role: string; content: string }> }; + return result.generated_text.find((m) => m.role === "assistant")?.content ?? ""; +} +``` + +### JSON Extraction + +Small models often produce malformed JSON. The `extractJSON` utility handles common issues: + +```typescript +export function extractJSON(text: string): unknown { + // Try direct parse + try { return JSON.parse(text); } catch {} + + // Remove markdown fences + const cleaned = text.replace(/```json\s*/gi, "").replace(/```\s*/g, ""); + + // Extract JSON object + const match = cleaned.match(/\{[\s\S]*\}/); + if (match) { + try { return JSON.parse(match[0]); } catch {} + } + + throw new Error(`Could not extract JSON from: ${text.slice(0, 200)}`); +} +``` + +--- + +## API Layer + +### Research Endpoint (SSE) + +`POST /api/research` streams research state updates as **Server-Sent Events** so the client can show progress immediately and parse events by type. Using SSE (instead of raw NDJSON) gives a standard, well-supported streaming protocol and allows future event names (e.g. `ping`, `heartbeat`) without changing the wire format. + +**Request body:** `{ "query": string, "knowledge"?: Array<{ id, type, label, meta? }> }`. The `knowledge` array is sent by the workspace when the user adds files, URLs, or notes in the Context panel; the backend currently uses only `query` and may later use `knowledge` for RAG or plan conditioning. + +**Response:** `Content-Type: text/event-stream`. Each message is an SSE message: + +- `event: start` — First event; signals that the run has started (enables optimistic UI). +- `event: research` — One per graph node update; `data` is `{ node, state }`. +- `event: error` — On exception; `data` includes `node: "_error"` and `state.errorMessage`. + +```typescript +// Server: send helper +const send = (event: string, payload: { node: string; state: Record }) => { + controller.enqueue(encoder.encode(`event: ${event}\ndata: ${JSON.stringify(payload)}\n\n`)); +}; +send("start", { node: "_start", state: { status: "started", ... } }); +for await (const event of runResearch(query)) { + send("research", event); +} +// On catch: send("error", { node: "_error", state: { status: "failed", errorMessage } }); +``` + +### Model Loading Endpoint + +`GET /api/model/load?...modelId=...&dtype=...` (or POST with same query) streams download/load progress via SSE. The client parses `data: {...}` lines to drive the progress bar and status pill. See [Frontend: Real-Time Workspace](#frontend-real-time-workspace) for how the UI consumes these streams. + +--- + +## Frontend: Real-Time Workspace + +The React client (`app/page.tsx`) is a single-page workspace that mirrors a Cursor/Gemini-style flow: immediate, non-blocking feedback, streaming AI responses, and a dedicated context/knowledge area. + +### What the UI Provides + +| Area | Purpose | +|------|--------| +| **Chat** | User messages and assistant replies. When a run finishes, the final report is streamed **word-by-word** into the last assistant message to mimic a live conversation. | +| **Context / Knowledge** | Drag-and-drop zone for PDFs, text files, or URLs; optional fields to add URLs and short notes. Stored in React state and sent as `knowledge` with each research request for future backend use. | +| **Quick-action chips** | A row of buttons below the input (e.g. “Help me learn this topic”, “Summarize these docs”) that set or extend the query and trigger a run—Gemini-style shortcuts. | +| **Starter cards** | Empty state with example prompts (e.g. “How does Gemini Pro work…”) that fill the input and can be run in one click. | +| **Model card** | Compact panel for model selection, load/progress, and status (Ready / Loading / Error). | +| **Reasoning trace** | Scrollable list of the latest reasoning entries from the event stream (node + summary) for observability. | + +### Optimistic UI and Shimmer + +- On “Run Research”, the UI immediately appends the user message and a placeholder assistant message with a shimmer skeleton, then consumes SSE and updates that message when the final report arrives. +- Shimmer and loading states use Tailwind (e.g. `animate-pulse`, neutral backgrounds) so the interface feels responsive even when the agent is still planning or executing. + +### SSE Consumption (Research) + +The client uses `EventSource`-style parsing on the `ReadableStream`: split by `\n\n`, then for each line look for `event:` and `data:` and dispatch by event type. Accumulated `research` events update both the reasoning trace and the chat when `finalReport` is present. + +```typescript +// Conceptual: read stream, split by double newline, parse "event:" and "data:" +const chunks = buffer.split("\n\n"); +for (const chunk of chunks) { + const eventMatch = chunk.match(/event:\s*(\w+)/); + const dataMatch = chunk.match(/data:\s*(\{[\s\S]*\})/); + if (eventMatch && dataMatch) { + const payload = JSON.parse(dataMatch[1]); + if (eventMatch[1] === "research") setEvents((prev) => [...prev, payload]); + // ... handle start, error; when payload.state.finalReport exists, run word-by-word animation + } +} +``` + +### Word-by-Word Streaming + +When an event contains `state.finalReport`, the full text is not dumped at once. A small timer (e.g. every 40ms) reveals the report word-by-word in the last assistant message and clears the “streaming” state when done. This keeps the same SSE event payload while making the reply feel live. + +--- + +## Running the Project + +### Prerequisites + +- Node.js 20+ +- npm or pnpm + +### Installation + +```bash +npm install +``` + +### Build the LLM worker (required for local inference) + +Inference runs in a Node.js worker thread. Compile the worker once before using the app locally: + +```bash +npm run build:worker +``` + +This writes `dist/llm/worker-entry.js` and `dist/llm/pipeline.js`. The production build runs this step automatically. + +### Development + +```bash +npm run dev +``` + +Open http://localhost:3000. Click "Load Model" to download and initialize the LLM, then run research queries. + +If you see an error that the worker was not found, run `npm run build:worker` first. + +### Production build + +```bash +npm run build +``` + +This runs `build:worker` then builds the Next.js app with webpack. The app is served with: + +```bash +npm run start +``` + +### First run + +The first model load downloads weights to `.hf-cache/` (approximately 400MB for SmolLM2-360M Q4). Subsequent loads are fast. + +### Deploy (Render) + +The repo includes a **Dockerfile** and config for [Render](https://render.com). Render builds the image in the cloud (no local Docker required). See **[DEPLOY.md](DEPLOY.md)** for steps. + +--- + +## Configuration + +Create `.env.local` from `.env.example`: + +```bash +# Model selection +HF_MODEL=HuggingFaceTB/SmolLM2-360M-Instruct +HF_CACHE_DIR=./.hf-cache +``` + +### Observability (LangSmith) + +To make agent behavior observable and debug failures (e.g. thinker returning invalid JSON), use [LangSmith](https://smith.langchain.com/). Set in `.env.local`: + +```bash +LANGCHAIN_TRACING_V2=true +LANGCHAIN_API_KEY=your-langsmith-api-key +LANGCHAIN_PROJECT=deeptrust +``` + +With tracing enabled, every graph run is recorded. You can inspect prompts, raw LLM outputs, and state transitions in the LangSmith UI, which helps diagnose schema validation errors and long-running or looping runs. + +### Available Models + +| Model | Size | Speed | Quality | +|-------|------|-------|---------| +| `HuggingFaceTB/SmolLM2-360M-Instruct` | 400MB | Fast | Basic | +| `HuggingFaceTB/SmolLM2-1.7B-Instruct` | 1.7GB | Moderate | Better | + +--- + +## Key Concepts Demonstrated + +1. **State Machines for Agents**: Using LangGraph to model complex, cyclic agent workflows +2. **Type-Safe State**: Zod schemas with TypeScript inference for runtime validation +3. **Local LLM Inference**: Running models in-process without external API dependencies +4. **Streaming Responses**: Server-Sent Events and ReadableStream for real-time updates +5. **Human-in-the-Loop**: Checkpoint interrupts for manual approval gates +6. **Revision Loops**: Cyclic graph edges for iterative refinement with safety ceilings + +--- + +## License + +MIT diff --git a/app/api/model/load/route.ts b/app/api/model/load/route.ts new file mode 100644 index 0000000000000000000000000000000000000000..97abcb063c5a149db6f8c7fc999b7e549057cd80 --- /dev/null +++ b/app/api/model/load/route.ts @@ -0,0 +1,109 @@ +// Avoid loading @huggingface/transformers on Vercel (no native runtime / disk); use dynamic import only when needed. +const isVercel = process.env.VERCEL === "1"; + +function unsupportedStream(encoder: TextEncoder): ReadableStream { + const msg = { + status: "error", + progress: 0, + message: + "Model loading is not supported on Vercel (serverless). Run the app locally or self-host for local inference.", + models: [] as { id: string; label: string; dtype?: string; sizeNote?: string }[], + }; + return new ReadableStream({ + start(controller) { + controller.enqueue(encoder.encode(`data: ${JSON.stringify(msg)}\n\n`)); + controller.close(); + }, + }); +} + +export async function GET(request: Request) { + const encoder = new TextEncoder(); + + if (isVercel) { + return new Response(unsupportedStream(encoder), { + headers: { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + Connection: "keep-alive", + }, + }); + } + + const { loadModel, getModelStatus, MODELS } = await import("@/lib/agent/llm"); + const { searchParams } = new URL(request.url); + const modelId = searchParams.get("modelId") ?? undefined; + const dtype = searchParams.get("dtype") ?? undefined; + + const stream = new ReadableStream({ + async start(controller) { + const send = (data: object) => { + controller.enqueue(encoder.encode(`data: ${JSON.stringify(data)}\n\n`)); + }; + + const status = await getModelStatus(modelId, dtype); + const requestedId = modelId ?? status.modelId; + const requestedDtype = dtype ?? status.dtype; + if ( + status.status === "ready" && + status.modelId === requestedId && + status.dtype === requestedDtype + ) { + send({ ...status, models: status.models ?? MODELS }); + controller.close(); + return; + } + + send({ + status: "loading", + progress: 0, + message: "Starting model load...", + modelId: modelId ?? status.modelId, + dtype: dtype ?? status.dtype, + models: status.models ?? MODELS, + }); + + try { + await loadModel(modelId, dtype as "q4" | "fp16" | "fp32" | undefined, (progress) => { + send({ ...progress, models: (progress as { models?: typeof MODELS }).models ?? MODELS }); + }); + controller.close(); + } catch (error) { + send({ + status: "error", + progress: 0, + message: error instanceof Error ? error.message : "Unknown error", + modelId: modelId ?? undefined, + dtype: dtype ?? undefined, + models: MODELS, + }); + controller.close(); + } + }, + }); + + return new Response(stream, { + headers: { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + Connection: "keep-alive", + }, + }); +} + +export async function POST() { + if (isVercel) { + return Response.json({ + status: "error", + progress: 0, + file: "", + message: + "Model loading is not supported on Vercel (serverless). Run the app locally or self-host for local inference.", + models: [], + }); + } + + const { getModelStatus, MODELS } = await import("@/lib/agent/llm"); + const status = await getModelStatus(); + return Response.json({ ...status, models: status.models ?? MODELS }); +} diff --git a/app/api/research/approve/route.ts b/app/api/research/approve/route.ts new file mode 100644 index 0000000000000000000000000000000000000000..eaa2ea7f7a498878e6e53db0eba827f1fe5aacf6 --- /dev/null +++ b/app/api/research/approve/route.ts @@ -0,0 +1,55 @@ +import { approveAndResume } from "@/lib/agent"; +import { NextRequest } from "next/server"; + +export async function POST(req: NextRequest) { + const body = await req.json(); + const { threadId } = body as { threadId?: string }; + + if (!threadId) { + return new Response( + JSON.stringify({ error: "threadId is required to approve a run" }), + { status: 400, headers: { "Content-Type": "application/json" } } + ); + } + + const encoder = new TextEncoder(); + + const stream = new ReadableStream({ + async start(controller) { + const send = ( + event: string, + payload: { node: string; state: Record } + ) => { + const data = `event: ${event}\ndata: ${JSON.stringify(payload)}\n\n`; + controller.enqueue(encoder.encode(data)); + }; + + try { + for await (const event of approveAndResume(threadId)) { + send("research", event); + } + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + send("error", { + node: "_error", + state: { + status: "failed", + errorMessage: message, + }, + }); + } finally { + controller.close(); + } + }, + }); + + return new Response(stream, { + headers: { + "Content-Type": "text/event-stream; charset=utf-8", + "Cache-Control": "no-cache, no-transform", + Connection: "keep-alive", + "X-Accel-Buffering": "no", + }, + }); +} + diff --git a/app/api/research/route.ts b/app/api/research/route.ts new file mode 100644 index 0000000000000000000000000000000000000000..5336ebb65c84bb7a9a2826dfde4d0223c79836f4 --- /dev/null +++ b/app/api/research/route.ts @@ -0,0 +1,80 @@ +import { runResearch } from "@/lib/agent"; +import { NextRequest } from "next/server"; + +export async function POST(req: NextRequest) { + const body = await req.json(); + const { + query, + retrievedContext, + contextUrls, + }: { + query: string; + retrievedContext?: string; + contextUrls?: string[]; + } = body; + + const encoder = new TextEncoder(); + + const stream = new ReadableStream({ + async start(controller) { + const send = ( + event: string, + payload: { node: string; state: Record } + ) => { + const data = `event: ${event}\ndata: ${JSON.stringify(payload)}\n\n`; + controller.enqueue(encoder.encode(data)); + }; + + // Initial "cursor-like" optimistic event + send("start", { + node: "_start", + state: { + status: "started", + plan: { objective: "Starting…", steps: [] }, + reasoning: [ + { + node: "_start", + summary: + "Research started. First step (planning) may take 1–2 minutes on slower devices.", + }, + ], + }, + }); + + try { + const options = + retrievedContext != null || (contextUrls?.length ?? 0) > 0 + ? { knowledgeContext: retrievedContext ?? "", contextUrls: contextUrls ?? [] } + : undefined; + for await (const event of runResearch(query, "Research Session", options)) { + if (event.node === "__interrupt__") { + // Surface HITL interrupt to the client and close the stream. + send("hitl_waiting", event); + return; + } + send("research", event); + } + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + send("error", { + node: "_error", + state: { + status: "failed", + errorMessage: message, + }, + }); + } finally { + controller.close(); + } + }, + }); + + return new Response(stream, { + headers: { + "Content-Type": "text/event-stream; charset=utf-8", + "Cache-Control": "no-cache, no-transform", + Connection: "keep-alive", + "X-Accel-Buffering": "no", + }, + }); +} \ No newline at end of file diff --git a/app/favicon.ico b/app/favicon.ico new file mode 100644 index 0000000000000000000000000000000000000000..718d6fea4835ec2d246af9800eddb7ffb276240c Binary files /dev/null and b/app/favicon.ico differ diff --git a/app/globals.css b/app/globals.css new file mode 100644 index 0000000000000000000000000000000000000000..a2dc41ecee5ec435200fe7cba2bde4107f823774 --- /dev/null +++ b/app/globals.css @@ -0,0 +1,26 @@ +@import "tailwindcss"; + +:root { + --background: #ffffff; + --foreground: #171717; +} + +@theme inline { + --color-background: var(--background); + --color-foreground: var(--foreground); + --font-sans: var(--font-geist-sans); + --font-mono: var(--font-geist-mono); +} + +@media (prefers-color-scheme: dark) { + :root { + --background: #0a0a0a; + --foreground: #ededed; + } +} + +body { + background: var(--background); + color: var(--foreground); + font-family: Arial, Helvetica, sans-serif; +} diff --git a/app/layout.tsx b/app/layout.tsx new file mode 100644 index 0000000000000000000000000000000000000000..f7fa87eb875260ed98651bc419c8139b5119e554 --- /dev/null +++ b/app/layout.tsx @@ -0,0 +1,34 @@ +import type { Metadata } from "next"; +import { Geist, Geist_Mono } from "next/font/google"; +import "./globals.css"; + +const geistSans = Geist({ + variable: "--font-geist-sans", + subsets: ["latin"], +}); + +const geistMono = Geist_Mono({ + variable: "--font-geist-mono", + subsets: ["latin"], +}); + +export const metadata: Metadata = { + title: "Create Next App", + description: "Generated by create next app", +}; + +export default function RootLayout({ + children, +}: Readonly<{ + children: React.ReactNode; +}>) { + return ( + + + {children} + + + ); +} diff --git a/app/page.tsx b/app/page.tsx new file mode 100644 index 0000000000000000000000000000000000000000..75bb9eca3b376a5894b77f36158ef674fe4aad77 --- /dev/null +++ b/app/page.tsx @@ -0,0 +1,1120 @@ +"use client"; + +import { useState, useCallback, useEffect, useRef } from "react"; + +interface ModelProgress { + status: "idle" | "loading" | "downloading" | "ready" | "error"; + progress: number; + file: string; + message: string; + modelId?: string; + dtype?: string; + models?: ModelOption[]; +} + +interface ModelOption { + id: string; + label: string; + dtype?: "q4" | "fp16" | "fp32"; + sizeNote?: string; +} + +interface ResearchEvent { + node: string; + state: { + threadId?: string; + interrupt?: unknown; + status?: string; + plan?: { + objective: string; + steps: Array<{ tool: string; input: string; rationale?: string }>; + }; + auditResult?: { + verdict: "approved" | "rejected" | "needs_revision"; + policyViolations?: string[]; + suggestions?: string[]; + }; + finalReport?: string; + reasoning?: Array<{ node: string; summary: string }>; + errorMessage?: string; + }; +} + +type ChatRole = "user" | "assistant"; + +interface ChatMessage { + id: string; + role: ChatRole; + content: string; + isStreaming?: boolean; +} + +type KnowledgeItemType = "file" | "url" | "note"; + +interface KnowledgeItem { + id: string; + type: KnowledgeItemType; + label: string; + meta?: string; + status?: "pending" | "indexing" | "indexed" | "error"; +} + +const DEFAULT_MODELS: ModelOption[] = [ + { + id: "HuggingFaceTB/SmolLM2-135M-Instruct", + label: "SmolLM2 135M (Q4, tiny)", + dtype: "q4", + sizeNote: "~150–200 MB (approx)", + }, + { + id: "HuggingFaceTB/SmolLM2-360M-Instruct", + label: "SmolLM2 360M (Q4)", + dtype: "q4", + sizeNote: "~388 MB", + }, + { + id: "HuggingFaceTB/SmolLM2-360M-Instruct", + label: "SmolLM2 360M (FP16)", + dtype: "fp16", + sizeNote: "~725 MB", + }, + { + id: "HuggingFaceTB/SmolLM2-360M-Instruct", + label: "SmolLM2 360M (full)", + dtype: "fp32", + sizeNote: "~1.45 GB", + }, +]; + +const QUICK_ACTIONS = [ + "Create image-ready research brief", + "Help me learn this topic", + "Summarize these docs for me", + "Audit my assumptions", + "Turn this into an implementation plan", +]; + +const PREVIEW_QUERIES = [ + "How does Gemini Pro work for deep research workflows?", + "Design a learning plan for mastering LangGraph in 30 days.", + "Compare self-hosted vs managed LLM stacks for a fintech startup.", +]; + +export default function DeepTrustWorkspace() { + const [modelStatus, setModelStatus] = useState({ + status: "idle", + progress: 0, + file: "", + message: "Model not loaded", + }); + const [selectedModelIndex, setSelectedModelIndex] = useState(0); + const models = modelStatus.models?.length ? modelStatus.models : DEFAULT_MODELS; + const selectedModel = models[selectedModelIndex] ?? null; + + const [query, setQuery] = useState(PREVIEW_QUERIES[0]); + const [chat, setChat] = useState([]); + const [events, setEvents] = useState([]); + const [hitlThreadId, setHitlThreadId] = useState(null); + const [hitlPayload, setHitlPayload] = useState(null); + const [error, setError] = useState(null); + const [isStreaming, setIsStreaming] = useState(false); + const [knowledgeItems, setKnowledgeItems] = useState([]); + const [isDragOver, setIsDragOver] = useState(false); + const [noteDraft, setNoteDraft] = useState(""); + const [urlDraft, setUrlDraft] = useState(""); + + const streamAbortRef = useRef(null); + const streamingTargetRef = useRef(null); + const streamingTimerRef = useRef(null); + const pendingFullTextRef = useRef(null); + + const chatScrollRef = useRef(null); + + useEffect(() => { + fetch("/api/model/load", { method: "POST" }) + .then((res) => res.json()) + .then((data: ModelProgress) => { + setModelStatus((prev) => ({ ...prev, ...data })); + if (data.models?.length && selectedModelIndex >= data.models.length) { + setSelectedModelIndex(0); + } + }) + .catch(() => {}); + // eslint-disable-next-line react-hooks/exhaustive-deps + }, []); + + const hydrateKnowledge = useCallback(() => { + import("@/lib/knowledge") + .then(({ listKnowledgeItems }) => listKnowledgeItems()) + .then((items) => setKnowledgeItems(items)) + .catch(() => {}); + }, []); + useEffect(() => { + hydrateKnowledge(); + }, [hydrateKnowledge]); + + const startStreamingAnimation = useCallback((fullText: string, messageId: string) => { + if (!fullText) return; + if (streamingTimerRef.current) { + window.clearInterval(streamingTimerRef.current); + } + + const words = fullText.split(/\s+/); + let index = 0; + + streamingTargetRef.current = messageId; + + setChat((prev) => + prev.map((m) => + m.id === messageId + ? { + ...m, + content: "", + isStreaming: true, + } + : m + ) + ); + + const timer = window.setInterval(() => { + index += 1; + const nextContent = words.slice(0, index).join(" "); + + setChat((prev) => + prev.map((m) => + m.id === messageId + ? { + ...m, + content: nextContent, + } + : m + ) + ); + + if (index >= words.length) { + if (streamingTimerRef.current) { + window.clearInterval(streamingTimerRef.current); + } + streamingTimerRef.current = null; + pendingFullTextRef.current = null; + setChat((prev) => + prev.map((m) => + m.id === messageId + ? { + ...m, + isStreaming: false, + } + : m + ) + ); + } + }, 40); + + streamingTimerRef.current = timer; + }, []); + + useEffect(() => { + if (chatScrollRef.current) { + chatScrollRef.current.scrollTop = chatScrollRef.current.scrollHeight; + } + }, [chat]); + + const isModelReady = + modelStatus.status === "ready" && + selectedModel && + modelStatus.modelId === selectedModel.id && + modelStatus.dtype === selectedModel.dtype; + + const isModelLoading = + modelStatus.status === "loading" || modelStatus.status === "downloading"; + + const loadModel = useCallback(async () => { + if (!selectedModel) return; + setModelStatus((prev) => ({ ...prev, status: "loading", message: "Starting..." })); + + try { + const params = new URLSearchParams({ modelId: selectedModel.id }); + if (selectedModel.dtype) params.set("dtype", selectedModel.dtype); + const response = await fetch(`/api/model/load?${params.toString()}`); + const reader = response.body?.getReader(); + if (!reader) throw new Error("No response body"); + + const decoder = new TextDecoder(); + let buffer = ""; + + while (true) { + const { done, value } = await reader.read(); + if (done) break; + + buffer += decoder.decode(value, { stream: true }); + const chunks = buffer.split("\n\n"); + buffer = chunks.pop() || ""; + + for (const chunk of chunks) { + if (chunk.startsWith("data: ")) { + try { + const data = JSON.parse(chunk.slice(6)) as ModelProgress; + setModelStatus((prev) => ({ ...prev, ...data })); + } catch { + // ignore malformed progress chunks + } + } + } + } + } catch (err) { + setModelStatus((prev) => ({ + ...prev, + status: "error", + message: err instanceof Error ? err.message : "Failed to load model", + })); + } + }, [selectedModel]); + + const registerKnowledgeFiles = useCallback((files: FileList | null) => { + if (!files || files.length === 0) return; + const pdfs = Array.from(files).filter((f) => f.type === "application/pdf"); + import("@/lib/knowledge").then(({ addPdfFile }) => { + for (const file of pdfs) { + const tempId = `temp-${file.name}-${Date.now()}-${Math.random().toString(36).slice(2)}`; + setKnowledgeItems((prev) => [ + ...prev, + { id: tempId, type: "file" as KnowledgeItemType, label: file.name, meta: "Indexing…", status: "indexing" as const }, + ]); + addPdfFile(file) + .then((meta: KnowledgeItem) => { + setKnowledgeItems((prev) => prev.map((x) => (x.id === tempId ? { ...meta, status: "indexed" as const } : x))); + }) + .catch(() => { + setKnowledgeItems((prev) => prev.map((x) => (x.id === tempId ? { ...x, status: "error" as const, meta: "Failed" } : x))); + }); + } + }); + }, []); + + const handleDrop: React.DragEventHandler = (event) => { + event.preventDefault(); + event.stopPropagation(); + setIsDragOver(false); + + if (event.dataTransfer.files?.length) registerKnowledgeFiles(event.dataTransfer.files); + + const urlPayload = + event.dataTransfer.getData("text/uri-list") || + event.dataTransfer.getData("text/plain"); + + if (urlPayload && /^https?:\/\//i.test(urlPayload.trim())) { + const url = urlPayload.trim(); + import("@/lib/knowledge") + .then(({ addUrl }) => addUrl(url)) + .then((meta) => setKnowledgeItems((prev) => [...prev, meta])) + .catch(() => {}); + } + }; + + const handleDragOver: React.DragEventHandler = (event) => { + event.preventDefault(); + event.stopPropagation(); + setIsDragOver(true); + }; + + const handleDragLeave: React.DragEventHandler = (event) => { + event.preventDefault(); + event.stopPropagation(); + setIsDragOver(false); + }; + + const handleAddNote = () => { + const value = noteDraft.trim(); + if (!value) return; + import("@/lib/knowledge") + .then(({ addNote }) => addNote(value)) + .then((meta) => { + setKnowledgeItems((prev) => [...prev, meta]); + setNoteDraft(""); + }) + .catch(() => {}); + }; + + const handleAddUrl = () => { + const value = urlDraft.trim(); + if (!value) return; + import("@/lib/knowledge") + .then(({ addUrl }) => addUrl(value)) + .then((meta) => { + setKnowledgeItems((prev) => [...prev, meta]); + setUrlDraft(""); + }) + .catch(() => {}); + }; + + const handleRemoveKnowledgeItem = (id: string) => { + import("@/lib/knowledge") + .then(({ removeKnowledgeDocument }) => removeKnowledgeDocument(id)) + .then(() => setKnowledgeItems((prev) => prev.filter((x) => x.id !== id))) + .catch(() => {}); + }; + + const resetStreaming = () => { + if (streamAbortRef.current) { + streamAbortRef.current.abort(); + streamAbortRef.current = null; + } + if (streamingTimerRef.current) { + window.clearInterval(streamingTimerRef.current); + streamingTimerRef.current = null; + } + pendingFullTextRef.current = null; + streamingTargetRef.current = null; + setIsStreaming(false); + }; + + const runResearch = useCallback( + async (promptOverride?: string) => { + const nextQuery = (promptOverride ?? query).trim(); + if (!nextQuery || !isModelReady || isStreaming) return; + + if (streamAbortRef.current) { + streamAbortRef.current.abort(); + } + + const controller = new AbortController(); + streamAbortRef.current = controller; + + setError(null); + setEvents([]); + setIsStreaming(true); + + const userMessage: ChatMessage = { + id: `user-${Date.now()}`, + role: "user", + content: nextQuery, + }; + const assistantMessage: ChatMessage = { + id: `assistant-${Date.now()}`, + role: "assistant", + content: "", + isStreaming: true, + }; + + setChat((prev) => [...prev, userMessage, assistantMessage]); + + try { + let retrievedContext = ""; + let contextUrls: string[] = []; + if (knowledgeItems.length > 0) { + const { retrieve } = await import("@/lib/knowledge"); + const result = await retrieve(nextQuery); + retrievedContext = result.retrievedContext; + contextUrls = result.contextUrls; + } + const response = await fetch("/api/research", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + query: nextQuery, + retrievedContext, + contextUrls, + }), + signal: controller.signal, + }); + + if (!response.ok) { + throw new Error(`HTTP ${response.status}: ${response.statusText}`); + } + + const reader = response.body?.getReader(); + if (!reader) { + throw new Error("No response body"); + } + + const decoder = new TextDecoder(); + let buffer = ""; + let hadError = false; + const assistantId = assistantMessage.id; + + while (true) { + const { done, value } = await reader.read(); + if (done) break; + + buffer += decoder.decode(value, { stream: true }); + const eventsRaw = buffer.split("\n\n"); + buffer = eventsRaw.pop() || ""; + + for (const raw of eventsRaw) { + const lines = raw.split("\n"); + let eventType = "message"; + let dataLine = ""; + for (const line of lines) { + if (line.startsWith("event: ")) { + eventType = line.slice(7); + } + if (line.startsWith("data: ")) { + dataLine = line.slice(6); + } + } + + if (!dataLine) continue; + + try { + const parsed = JSON.parse(dataLine) as ResearchEvent; + if (eventType === "hitl_waiting" && parsed.node === "__interrupt__") { + setHitlThreadId(parsed.state.threadId ?? null); + setHitlPayload(parsed.state.interrupt ?? null); + setIsStreaming(false); + continue; + } + + setEvents((prev) => [...prev, parsed]); + + if (parsed.node === "_error" || parsed.state.status === "failed") { + const message = + parsed.state.errorMessage ?? "Research failed. See server logs for details."; + setError(message); + hadError = true; + resetStreaming(); + } + + if (parsed.state.finalReport && !hadError) { + pendingFullTextRef.current = parsed.state.finalReport; + } + + if (pendingFullTextRef.current && !streamingTimerRef.current) { + startStreamingAnimation(pendingFullTextRef.current, assistantId); + } + } catch (e) { + console.warn("Failed to parse SSE event:", e); + } + } + } + + if (!hadError) { + setIsStreaming(false); + } + } catch (err) { + if ((err as Error).name === "AbortError") { + return; + } + + setError(err instanceof Error ? err.message : "Unknown error"); + resetStreaming(); + } + }, + [ + knowledgeItems, + isModelReady, + isStreaming, + query, + startStreamingAnimation, + resetStreaming, + ] + ); + + const handleQuickAction = (template: string) => { + const composed = query ? `${query}\n\n${template}` : template; + setQuery(composed); + void runResearch(composed); + }; + + const handleSubmit: React.FormEventHandler = (event) => { + event.preventDefault(); + void runResearch(); + }; + + const isInputDisabled = !isModelReady || isStreaming; + + const handleApprovePlan = useCallback(async () => { + if (!hitlThreadId) return; + + setIsStreaming(true); + setError(null); + + const assistantMessage: ChatMessage = { + id: `assistant-${Date.now()}`, + role: "assistant", + content: "", + isStreaming: true, + }; + setChat((prev) => [...prev, assistantMessage]); + + try { + const response = await fetch("/api/research/approve", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ threadId: hitlThreadId }), + }); + + if (!response.ok) { + throw new Error(`HTTP ${response.status}: ${response.statusText}`); + } + + const reader = response.body?.getReader(); + if (!reader) { + throw new Error("No response body"); + } + + const decoder = new TextDecoder(); + let buffer = ""; + let hadError = false; + const assistantId = assistantMessage.id; + + while (true) { + const { done, value } = await reader.read(); + if (done) break; + + buffer += decoder.decode(value, { stream: true }); + const eventsRaw = buffer.split("\n\n"); + buffer = eventsRaw.pop() || ""; + + for (const raw of eventsRaw) { + const lines = raw.split("\n"); + let eventType = "message"; + let dataLine = ""; + for (const line of lines) { + if (line.startsWith("event: ")) { + eventType = line.slice(7); + } + if (line.startsWith("data: ")) { + dataLine = line.slice(6); + } + } + + if (!dataLine) continue; + + try { + const parsed = JSON.parse(dataLine) as ResearchEvent; + if (eventType === "error" || parsed.node === "_error" || parsed.state.status === "failed") { + const message = + parsed.state.errorMessage ?? "Research failed after approval. See server logs for details."; + setError(message); + hadError = true; + resetStreaming(); + } + + setEvents((prev) => [...prev, parsed]); + + if (parsed.state.finalReport && !hadError) { + pendingFullTextRef.current = parsed.state.finalReport; + } + + if (pendingFullTextRef.current && !streamingTimerRef.current) { + startStreamingAnimation(pendingFullTextRef.current, assistantId); + } + } catch (e) { + console.warn("Failed to parse SSE event (approve):", e); + } + } + } + + if (!hadError) { + setIsStreaming(false); + } + } catch (err) { + setError(err instanceof Error ? err.message : "Unknown error during approval"); + resetStreaming(); + } finally { + setHitlThreadId(null); + setHitlPayload(null); + } + }, [ + hitlThreadId, + resetStreaming, + startStreamingAnimation, + ]); + + const handleRejectPlan = useCallback(() => { + setHitlThreadId(null); + setHitlPayload(null); + setIsStreaming(false); + setChat((prev) => [ + ...prev, + { + id: `assistant-${Date.now()}`, + role: "assistant", + content: + "The proposed plan was rejected. Please refine your question or constraints, then try again.", + }, + ]); + }, []); + + return ( +
+
+
+
+
+

Hi there

+

+ Where should we start? +

+
+
+ {isModelReady && "Model ready"} + {isModelLoading && "Loading model…"} + {modelStatus.status === "error" && "Model error"} + {modelStatus.status === "idle" && "Model not loaded"} +
+
+ +
+
+ {chat.length === 0 && ( +
+ {PREVIEW_QUERIES.map((example) => ( + + ))} +
+ )} + + {chat.map((message) => ( +
+
+ {message.content || (message.isStreaming && ( +
+
+
+
+ ))} +
+
+ ))} +
+ +
+
+