docs/05_build_plan.md

Build Plan — Ordered Tasks for Claude Code

Execute phases in order. Each task lists what to build and its acceptance criteria. Every phase ends in something runnable or testable. Keep the core decoupled from entry points and backends throughout (see CLAUDE.md).

Legend: [AC] = acceptance criteria.

---

Phase 0 — Scaffolding

0.1 Project skeleton. Create the repo layout from the setup doc and initialize the project with uv (uv init). Pin the interpreter to 3.11 (uv python pin 3.11) and set requires-python = ">=3.11" in pyproject.toml. Add dependencies via uv add (committing uv.lock); add .env.example and .gitignore (ignore data/, .env, .venv/, caches — but commit uv.lock and .python-version); add an empty README.md. [AC] .python-version reads 3.11; uv sync succeeds and uv run python -c "import sys, doc_agent; assert sys.version_info[:2]==(3,11)" passes.

0.2 Config loader. Implement config.py using pydantic-settings: load env, validate combinations (gemini requires key; vision_direct requires multimodal backend), fail fast with clear messages. [AC] Unit test: invalid combos raise; valid config parses.

---

Phase 1 — Contract & validation (pure, no model yet)

1.1 Schema. Implement schema/models.py (Document, LineItem) per the data spec, with normalization helpers (money → float, date → ISO). [AC] test_schema.py: valid dicts parse; malformed money/date handled.

1.2 Validation rules. Implement validation/rules.py: hard rules H1–H4, soft rules S1–S4, monetary epsilon, returning a structured report. [AC] test_validation.py: reconciling totals pass H2/H3; mismatches fail; soft failures recorded without forcing review.

1.3 Confidence & routing. Implement routing/score.py: pure score(data, report, model_signal) -> float and route(score, report) -> decision, with hard-failure short-circuit. [AC] test_routing.py: hard failure ⇒ review regardless of score; threshold boundary behaves; missing required fields lower score.

These three modules are fully testable before any model exists.

---

Phase 2 — Parsing & backends

2.1 Modality detection. parsing/detect.py: map file → native_pdf | image by extension/MIME. [AC] Correctly classifies the supported extensions.

2.2 Docling parser. parsing/docling_parser.py: native PDF → text/layout payload; optionally retain page image. [AC] A sample text PDF yields non-empty structured text.

2.3 OCR path (optional strategy). parsing/ocr.py: image → text via PaddleOCR/Tesseract, behind the same payload interface. [AC] A sample receipt image yields text; absence of the OCR engine degrades gracefully with a clear error.

2.4 Backend interface + factory. backends/base.py: the ExtractionBackend protocol, BackendResult, and a factory that builds the backend from config. [AC] Factory returns the configured backend; unknown backend ⇒ clear error.

2.5 Gemini backend. backends/gemini.py: multimodal call to the free tier; text for native PDFs, image for scans/photos; schema-constrained JSON output; bounded retries + timeout; model id from config. [AC] Given a sample document, returns schema-valid JSON; transient errors retry then route to review (don't crash).

2.6 Ollama backend. backends/ollama.py: local call with JSON-schema/grammar constrained decoding; text-in (pairs with OCR path). [AC] With a local model present, returns schema-valid JSON for a text payload. Skipped/marked if no local server (documented).

---

Phase 3 — Core pipeline

3.1 Assemble core. core.py: process_document(path) -> ExtractionResult chaining detect → acquire → extract → validate → score → route. Pure of side-effects (no file moves, no DB) — returns a result object. [AC] test_core_smoke.py: runs end-to-end on a couple of sample files with a stub backend; returns a populated result with a decision.

3.2 Idempotency. Content-hash helper so the same file isn't reprocessed. [AC] Same file hashed identically across runs.

---

Phase 4 — Entry points

4.1 Persistence. store/db.py (SQLite append of accepted records) and store/export.py (CSV export). [AC] Accepted record persists and appears in CSV; schema columns match.

4.2 Watcher / batch runner. ingest/watcher.py: watch (or poll) inbox/, call core, persist accepted, move source to processed/ or review/, per-document try/except with structured logging. [AC] Dropping a batch (mixed PDFs/scans/photos) processes all; one deliberately corrupt file routes to review with a logged reason and does not stop the loop.

4.3 Web demo. web/app.py: Gradio single-upload UI rendering fields, per-field confidence, validation report, and decision; explicit "synthetic/public only" notice; stateless. [AC] Local Gradio URL processes one upload of each modality and displays a correct, validated result.

---

Phase 5 — Evaluation

5.1 Dataset loaders. eval/datasets/: scripts to fetch a held-out slice of SROIE and the labelled invoice JSON set; map gold labels to the schema. No data committed to git. [AC] Loader yields (document, gold) pairs for a slice.

5.2 Metrics harness. eval/run_eval.py: run core over a slice; normalize; compute per-field and per-critical-field precision/recall/F1; compute auto-accept precision on critical fields; sweep threshold. [AC] Prints a metrics table; produces the precision/recall trade-off across thresholds; recommends a CONFIDENCE_THRESHOLD.

5.3 Tune & record. Set CONFIDENCE_THRESHOLD so auto-accept precision on critical fields ≥ 0.98; record the resulting recall. [AC] README contains a results table per dataset (precision/recall/F1 + routing stats) — the portfolio evidence.

---

Phase 6 — Deploy & document

6.1 Hugging Face Space. Add Space app.py and a requirements.txt for the Gradio builder, generated from uv (uv export --no-hashes --no-dev -o requirements.txt); set secrets (GEMINI_API_KEY, backend=gemini, vision_direct). Deploy. [AC] Public URL processes an uploaded document of each modality.

6.2 README. Quickstart (both modes), the swappable-backend explanation, the results table, the demo URL, and the free-tier/privacy caveats. [AC] A reader can run locally from scratch and understands the design and the evidence.

---

Build order rationale

Pure logic first (Phases 1) so the hard-to-test parts (validation, routing) are locked down before any model variance enters. Parsing and backends next (Phase 2), then the core that composes them (Phase 3). Entry points (Phase 4) are thin glue added only once the core is proven. Evaluation (Phase 5) sets the one tunable threshold with evidence. Deployment (Phase 6) is last and small because the core was kept host-independent from the start.