DocWeave - One Page
What I Built
DocWeave is an agentic document intelligence platform that transforms unstructured documents into a governed knowledge register.
It processes documents through an AI pipeline that extracts text, chunks content, creates embeddings, extracts structured knowledge, reconciles it with existing knowledge, validates proposals against configurable rules, and decides whether the knowledge can be committed automatically or requires human review.
The core idea is simple: AI proposes knowledge, rules provide quality gates, humans resolve uncertainty, and approved changes become committed knowledge with an audit trail.
Think of it as "GitHub for documents": every knowledge change is proposed, validated, reviewed when necessary, and committed with its source evidence and history.
Who It's For
DocWeave is designed for teams that manage large document collections and need structured, searchable, and auditable knowledge.
Typical users include:
- Compliance teams
- Research groups
- Clinical teams
- Legal departments
- Technical documentation teams
- Organizations maintaining large policy or research corpora
The problem it addresses is familiar: "I know we read this somewhere, but I cannot remember which document said it."
DocWeave makes the extracted knowledge searchable while preserving where that knowledge came from and how it was reviewed.
Results
- End-to-end processing: A PDF can be processed into structured knowledge items such as ENTITY, CLAIM, METHOD, METRIC, and OBSERVATION, with evidence quotes and confidence scores in approximately 20 to 45 seconds.
- Zero-loss batch extraction: Documents are processed in batches and sequentially so large documents are not silently truncated during knowledge extraction.
- Human-in-the-loop: Four configurable validation operators,
min_confidence,required_evidence,allowed_proposal_types, andtopic_relevance, determine whether proposals can be committed automatically or need human review. - Crash resilience: LangGraph checkpointing allows workflows to survive server restarts. Graceful shutdown waits for active work, while auto-resume can pick up orphaned workflows.
- MCP integration: The Model Context Protocol server exposes the DocWeave lifecycle to AI agents, allowing them to operate documents, workflows, proposals, knowledge, and validation rules programmatically.
- Deployed architecture: The frontend is deployed on Vercel, the backend runs in Docker on Hugging Face Spaces, and the database uses Neon PostgreSQL with pgvector.
How the System Works
Document
|
v
Extract
PyMuPDF + OCR
|
v
Chunk
|
v
Embed
sentence-transformers
|
v
Knowledge Extraction
LLM JSON
|
v
Reconcile
Deduplicate / detect conflicts
|
v
Validate
Configurable rules
|
v
Decide
Auto-commit or Human Review
|
+----------------------------+
| |
| Auto-approved | Rules fail
v v
Commit Human Review
Audit Trail |
+---- Approve --> Commit
|
+---- Reject
|
+---- Archive
|
Restore
|
Review
The workflow is stateful and checkpointed. A human review step is therefore a real workflow boundary rather than just a UI confirmation dialog.
Key Trade-Offs
| Decision | Why |
|---|---|
| Background threads instead of Celery | Keeps deployment simple and avoids requiring Redis or another message broker. The trade-off is limited horizontal scaling, which is acceptable for the current demo and small-team use case. |
| Groq free-tier LLM | Provides fast, low-cost inference. The trade-off is strict request-size and rate limitations, so knowledge extraction uses smaller batches. A paid tier or self-hosted model would remove much of this limitation. |
| Ephemeral file storage on Hugging Face | Keeps the current deployment simple. The trade-off is that uploaded files can be lost across container restarts. A production deployment should use persistent object storage such as S3 or GCS. |
| Local sentence-transformers embeddings | Removes an external embedding API dependency and keeps embedding generation under application control. The trade-off is memory usage, making very small servers unsuitable. |
| PostgreSQL for relational data and vectors | Keeps documents, knowledge, proposals, workflow data, and vector search in one database. The trade-off is that a dedicated vector database may scale better for very large workloads. |
| Native JSON mode instead of tool calling for extraction | Provides reliable structured output across the selected model/provider combination. The trade-off is less strict schema enforcement than dedicated function calling, so the application uses JSON parsing and fallback handling. |
Governance and Human Review
A central design choice in DocWeave is that extracted AI knowledge is not automatically treated as truth.
A proposal can pass validation and be committed automatically when it meets the configured rules.
When it does not, the workflow pauses at the human review boundary.
The reviewer can:
- Approve the proposal
- Reject the proposal
- Archive the proposal for later
- Restore an archived proposal
Archiving is deliberately reversible. It does not commit or reject the knowledge item.
HUMAN REVIEW
|
+-----------+-----------+
| | |
Approve Reject Archive
| | |
Commit Record Archived
| |
ACTIVE Restore
state |
v
PENDING
|
HUMAN REVIEW
This gives DocWeave a clear separation between what the model suggested and what the system has accepted as committed knowledge.
Agent Integration
DocWeave exposes its document lifecycle through MCP so an AI agent can operate the system directly.
An agent can perform operations such as:
- Discover workspaces
- Upload documents
- List and manage documents
- Monitor workflows
- Cancel or retry processing
- Inspect pending proposals
- Approve, reject, archive, and restore proposals
- Browse and search committed knowledge
- Create and manage validation rules
- Inspect workflow metrics
- Read dashboard statistics
- Read the activity feed
This means the same governed workflow can be used from the web application or through an AI agent.
What I'd Add With More Time
- Persistent cloud storage such as S3 for uploaded documents
- WebSocket-based real-time progress instead of polling
- Multi-user collaboration with role-based access control
- Knowledge graph visualization for entity relationships
- More advanced automatic conflict resolution using LLM reasoning
- PDF annotation overlays showing exactly where each knowledge item was extracted
- Larger evaluation datasets for measuring extraction, reconciliation, and retrieval quality
- More durable distributed workflow execution for larger deployments
The Main Design Principle
DocWeave is not intended to be an AI that claims to know everything.
It is designed to make document-heavy knowledge work faster while keeping the important parts visible:
where information came from, what the AI proposed, which rules were applied, whether a human reviewed it, and what ultimately became committed knowledge.
Concurrency and Safety
Two documents uploaded at the same time run in isolated threads, each with its own database session and its own LangGraph checkpoint thread ID. They cannot overwrite each other's state.
The one race condition that existed — two simultaneous approve calls on the same proposal — is closed with a PostgreSQL row-level lock (SELECT ... FOR UPDATE) acquired before the proposal status is read. The second caller blocks until the first commits, then sees the already-reviewed status and returns a 409.
The RunTracker registry uses its own mutex. The TestConcurrency suite verifies that 5 threads writing to the same tracker simultaneously produce exact totals without corruption.
The remaining honest limitation: the in-memory workflow thread registry and tracker store are process-local. A multi-worker deployment (uvicorn --workers N) would need these moved to a shared store. The current single-worker Docker deployment is unaffected.
Stage-by-Stage Timing and Cost
Every workflow stage is instrumented with start_stage / end_stage calls. The ten tracked stages are:
extraction → chunking → embedding → classification → knowledge_extraction → reconciliation → validation → decision → linking → complete
Token counts (input_tokens, output_tokens, llm_calls) are captured per LLM call with provider-specific fallbacks for Groq, Anthropic, and OpenAI response formats.
The full report is persisted to a WorkflowCheckpoint row so it survives server restarts, and is retrievable via the get_run_metrics MCP tool at any time after the run completes.
Task 1 Requirement Summary
| # | Requirement | Status | How |
|---|---|---|---|
| 1 | Visible stages, branching decisions | ✅ | LangGraph StateGraph, 11 nodes, conditional CONTINUE/REVIEW branch at decision node |
| 2 | Survives being stopped | ✅ | LangGraph PostgresSaver checkpoints after every node; startup orphan recovery; WAITING_FOR_REVIEW DB state |
| 3 | Human holds the gate | ✅ | Per-proposal approve/reject/archive/restore; each is a single-row DB operation; rejecting one does not affect others |
| 4 | Machine can drive it | ✅ | 24-tool MCP server; approve_proposal and reject_proposal are explicit MCP operations using the same service as the REST API |
| 5 | Never bluffs | ✅ | KnowledgeEvidence table with verbatim quote, page, section; LLM prompt requires evidence; items stay PENDING until explicitly approved |
| 6 | Stranger can run it | ✅ | LLM_API_KEY=x docker-compose up --build — one command, auto-migrations |
| 7 | Tests without live keys | ✅ | 35 offline tests covering validation, decision routing, kill-and-resume, concurrency, prompt injection — zero external dependencies |
| 8 | Does not take orders from documents | ✅ | Every section wrapped in data-boundary markers before LLM; 12 injection patterns detected; content preserved, not stripped |
| 9 | Two runs stay two runs | ✅ | Isolated threads + sessions; LangGraph thread IDs; with_for_update() on proposal review; tracker mutex |
| 10 | Knows what it cost | ✅ | All 11 nodes instrumented; per-stage elapsed time + token counts; persisted to DB; accessible via MCP |