docs/01_requirements.md

Requirements Specification — Document Extraction Agent

1. Overview

An autonomous agent that ingests invoices, receipts, and similar semi-structured financial documents, extracts their key fields into a validated structured record, and routes anything it is not confident about to a human review queue. The agent runs unattended over a stream of incoming documents and is also exposed through a small public web demo.

The reasoning step (turning document text/images into structured fields) is performed by a swappable LLM backend. Everything around it — triggering, parsing, validation, confidence scoring, routing, persistence, logging — is application code. The engineering value of this project is the system around the model, not the model itself.

2. Problem statement

Manually keying fields off invoices and receipts is slow and error-prone, and the documents arrive in inconsistent formats and qualities (clean PDFs, flatbed scans, phone photos). We want a pipeline that processes them automatically, is confident only when it should be, and surfaces the rest for a human — with measurable accuracy.

3. Goals

• Ingest documents from a watched location with no human trigger per document.
• Support three input modalities: native-text PDFs, scanned images, and phone photos.
• Extract a defined set of fields into a strict JSON schema.
• Validate extracted values, including arithmetic consistency checks.
• Assign a confidence to each document and route low-confidence documents to review rather than auto-accepting them.
• Persist accepted records and export them to CSV.
• Provide a public demo URL where a single uploaded document is processed and its result shown.
• Run entirely on free infrastructure and free model access.
• Be measurable against ground-truth datasets (precision, recall, F1).

4. Non-goals (explicitly out of scope for v1)

• Fine-tuning or training a model. Off-the-shelf models only.
• A full review application with auth, multi-user workflows, or audit trails. The review queue is a directory plus a CSV, not a product.
• Persistent multi-tenant storage in the cloud demo. The demo is presentation-only and stateless.
• Handling non-financial document types (contracts, IDs, medical records).
• Real-time / low-latency guarantees. This is a background batch system.
• Production hardening (SLAs, horizontal scale, queue infrastructure).

5. Users and usage modes

1. Autonomous batch mode (primary). Operator drops files into an inbox/ directory (local or mounted). The agent processes each, writes accepted records to storage, and moves uncertain ones to review/. No interaction per document.
2. Demo mode (secondary). A visitor uploads one document to the public web UI and sees the extracted fields, per-field confidence, validation results, and the accept/review decision. Nothing is persisted.

Both modes call the same core pipeline.

6. Functional requirements

• FR-1 Ingestion. Detect new files in inbox/ (file-watcher or poll) and enqueue them for processing. Supported types: .pdf, .png, .jpg, .jpeg, .webp, .tif/.tiff.
• FR-2 Parsing / text acquisition. For native-text PDFs, extract text and layout. For scans/photos, obtain content either via OCR or via a multimodal model that reads the image directly. The chosen path is backend-dependent (see architecture).
• FR-3 Field extraction. Produce a JSON object conforming to the schema in the data spec, using the active model backend with structured-output enforcement.
• FR-4 Validation. Apply type/format checks and arithmetic cross-checks. Each field carries a validation status.
• FR-5 Confidence + routing. Compute a document-level confidence from model signal, validation results, and required-field completeness. If it clears the threshold, auto-accept; otherwise route to review.
• FR-6 Persistence. Append accepted records to a local SQLite database and export to CSV. Move source files to processed/ or review/ accordingly.
• FR-7 Logging. Emit structured logs for every document: inputs, backend used, decision, validation failures, and timings. Never crash the loop on a single bad document — isolate, log, and continue.
• FR-8 Web demo. Accept one uploaded document, run the core pipeline, and render fields, confidence, validation, and decision. Stateless.
• FR-9 Backend selection. The model backend is chosen by configuration at startup with no code change (Gemini free tier or local Ollama).
• FR-10 Evaluation. A harness runs the pipeline over a labelled dataset and reports field-level precision, recall, and F1, plus document-level routing statistics.

7. Non-functional requirements

• NFR-1 Cost. Zero spend for development and demo. Local model = no quota; hosted model = free tier only.
• NFR-2 Privacy. Free hosted backends may use inputs for training; the public demo must process only synthetic/public documents. This must be stated in the demo UI. Sensitive data is handled only via the local backend.
• NFR-3 Swappability. Adding or replacing a backend requires implementing one interface and changing config — nothing else.
• NFR-4 Robustness. A malformed or unreadable document produces a logged failure and a review routing, never a crash.
• NFR-5 Reproducibility. Pinned dependencies; deterministic config; documented setup that runs from a clean checkout.
• NFR-6 Portability. The core pipeline is independent of both entry points and of any specific host.

8. Success criteria

The project is successful when:

• The agent processes a mixed batch (native PDFs + scans + phone photos) end-to-end with no per-document intervention, persisting accepted records and correctly diverting uncertain ones to review.
• On a held-out labelled set, auto-accept precision on the critical fields (total, tax, invoice_number) is ≥ 0.98, with recall reported at that operating point. (Rationale and method in the data spec.)
• A public demo URL processes an uploaded document of each modality and displays a correct, validated result.
• Swapping between the Gemini and Ollama backends requires only a config change.

9. Key assumptions

• Documents are predominantly English. Multilingual handling is best-effort.
• Volume during development is low (tens to low hundreds of documents), well within free-tier limits.
• The operator's local machine or chosen free host can run lightweight Python continuously; the model itself runs locally (Ollama) or via free API.
• Free-tier quotas and free hosting behaviour (idle sleep, CPU-only) are acceptable for a portfolio demo.

10. Glossary

• Auto-accept: a document whose confidence clears the threshold and whose record is persisted without human review.
• Review: a document routed to a human because confidence is below threshold or a hard validation rule failed.
• Critical fields: fields where a confidently-wrong value is most costly — total, tax, invoice_number.
• Backend: an implementation of the model interface that turns a document into structured fields.
• Core pipeline: the host- and entry-point-independent function that takes a document and returns an extraction result.