Spaces:
Sleeping
Sleeping
File size: 4,896 Bytes
f65e025 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | # Architecture — Document Processing AI Agent
**Decision record from research (see `RESEARCH.md`). Stack chosen: custom composed stack.**
## High-level
```
┌──────────────────────────────────────────────────────────────┐
│ FRONTEND (Next.js + React + TypeScript + Tailwind) │
│ 40% Chat │ 60% Workspace │
│ ──────── │ ── Tabs: Viewer · Fields · Classify · Summary · │
│ streaming │ Export ── │
│ agent │ Document viewer w/ bounding-box highlights │
└─────┬────────────────────────────────────────────────────────┘
│ REST + SSE (streaming)
┌─────▼────────────────────────────────────────────────────────┐
│ BACKEND (FastAPI, async) │
│ │
│ api/ routes: documents, chat(SSE), extract, │
│ classify, summary, export, health │
│ agent/ orchestrator (tool-calling loop) + prompts │
│ llm/ provider abstraction: Gemini(default)⇄OpenAI │
│ services/ ingestion(Docling) · extraction · classification│
│ · summary · qa(RAG) · anomaly · export · │
│ vectorstore · storage │
│ schemas/ Pydantic models (incl. invoice/contract) │
│ core/ config · logging · deps │
└─────┬───────────────────────────┬────────────────────────────┘
│ │
┌─────▼─────┐ ┌─────────▼──────────┐
│ Docling │ │ Storage │
│ (local │ │ • files (disk/S3) │
│ extract) │ │ • SQLite/Postgres │
└───────────┘ │ • vectors (chroma/│
│ pgvector) │
└────────────────────┘
```
## The 7 core functions → implementation
| Function | How | Tool exposed to agent |
|---|---|---|
| 1. Upload & ingest | `POST /documents` → Docling parse → DoclingDocument JSON + page images | `ingest_document` |
| 2. Parse & extract structured data | Docling structure + LLM structured-output against Pydantic schema (fields, tables, KV, entities) + confidence | `extract_fields` |
| 3. Classify | LLM zero-shot classification → type + confidence | `classify_document` |
| 4. Summarize | Map-reduce summarization over chunks | `summarize_document` |
| 5. Conversational Q&A | Agentic RAG: retrieve chunks → answer **with citations** (page + bbox) | `query_document` |
| 6. Flag anomalies | Rule + LLM checks: missing required fields, inconsistencies, low-confidence | `flag_anomalies` |
| 7. Export | Serialize extracted data → JSON / CSV / Excel | `export_data` |
## Agent design
Single **orchestrator agent** with the tools above (tool-calling loop), agentic-RAG retrieval, structured outputs (Pydantic), human-in-the-loop on low-confidence fields. Pattern mirrors LlamaIndex ADW: *parse → maintain state → retrieve → reason → surface for validation*.
## LLM strategy
- **Default (free):** Gemini Flash (1M context, native vision for scans).
- **Fallback (switchable):** GPT-4o-mini.
- `llm/base.py` defines a `LLMProvider` interface; runtime switch via config/env or per-request header. Cheap model for classify, stronger for reasoning.
## Tech choices (pragmatic defaults for a runnable, impressive build)
- **Backend:** FastAPI, Pydantic v2, Docling, `google-generativeai`, `openai`, ChromaDB (embedded vector store — no external service), SQLite (dev) with a path to Postgres/pgvector.
- **Frontend:** Next.js (App Router) + React + TypeScript + Tailwind + a PDF/image viewer with overlay layer for bounding boxes; SSE for streaming chat.
- **Local-first:** runs with only a free Gemini API key. No paid services required.
## Separation of concerns
Routes are thin → call services. Agent orchestrates services as tools. LLM access only via `llm/`. Storage only via `services/storage` + `services/vectorstore`. Schemas shared. This is the Dify-grade discipline we borrowed.
|