LEARN — a working knowledge of what you built
If a hiring manager pulls this project up in a screen and asks you to walk them through it, this doc is the map you narrate from. Every design decision, every trade-off, every "why not the other thing" is captured here. Read it end to end once. Reread the section for whichever thing they poke at.
Table of contents
- The one-liner (what it is, why it exists)
- The end-to-end request flow
- Every layer, one at a time
- 3.1 Pydantic schemas + the strict-mode constraint
- 3.2 The extractor + the envelope pattern
- 3.3 The document loader
- 3.4 The OpenAI client (why
temperature=None, whyreasoning_effort="minimal") - 3.5 Section chunker for 10-Ks
- 3.6 FastAPI backend + dependency injection for tests
- 3.7 Evaluation harness — the P/R/F1 machinery
- 3.8 UI (Paper & Ink, React + Motion + R3F)
- 3.9 Docker + HF Spaces single-container deploy
- 3.10 GitHub Actions CI
- The 14 design decisions you should be able to defend
- The roadmap explained — v1 v2 v3 v4
- Interview cheat sheet
- File map — what lives where
1. The one-liner
A production-grade LLM service that turns unstructured business documents (invoices, receipts, SEC 10-K filings) into schema-validated JSON with per-field confidence, cost accounting, and a reproducible evaluation harness.
The point of the project isn't the extraction. Every LLM demo does that. The point is everything around it — schemas, evaluation, cost tracking, multi- model benchmarking, section chunking for long documents, tests, CI, a real frontend, Docker, and a live public URL. That's the shape of shipped LLM software, and it's what hiring managers screen for.
Who would use this: a fintech that gets 10,000 vendor invoices a month and
needs to auto-populate their AP system. A hedge fund analyst who wants 10-K
financials in Snowflake. A small business owner who wants receipts categorized
into QuickBooks. All three use the same API — different doc_type, different
schema, same envelope.
2. The end-to-end request flow
Here's what happens when a user drops a PDF into the UI. Follow this sequence carefully — every layer earns its keep.
┌────────────┐ 1. multipart POST ┌──────────────┐ 2. dispatch on doc_type ┌───────────────┐
│ React UI │──────────────────────►│ FastAPI │─────────────────────────────►│ Extractor │
│ (Dropzone) │◄──────────────────────│ /extract │◄─── 6. ExtractionResult ─────│ orchestrator │
└────────────┘ └──────────────┘ └───────┬───────┘
│
│ 3. load
▼
┌───────────────┐
│ document │
│ loader │
│ (pdf/img/txt) │
└───────┬───────┘
│
│ text + images
│
┌─────────────────────────────────────────────────────┤
│ 4. filing? → chunk into cover + Item 1A + Item 8 │
│ invoice/receipt? → send whole doc │
▼ ▼
┌─────────────┐ ┌──────────────────┐
│ system │ │ OpenAI │
│ prompt │──── messages ─────────────────────►│ chat.completions │
│ (per type) │ │ .parse │
└─────────────┘ └────────┬─────────┘
│
5. envelope validated ◄──────── strict JSON│
by Pydantic v2 │
(extra=forbid) ▼
┌──────────────────┐
│ ExtractionResult │
│ + metrics │
└──────────────────┘
Steps 1-6 in words:
UI POSTs
file + doc_type + optional modelto/extract. The React Dropzone builds the multipart request.doc_typeis one ofreceipt | invoice | filing. The optionalmodelflag is what powers the multi-model benchmark.FastAPI receives, dispatches to
DocumentExtractor.extract(). The router (src/api/routers/extract.py) validates the file size (413 error over 10 MB), the content type (415), the doc_type (400), and injects the extractor viaDepends(get_extractor). That dependency injection is important — it lets tests swap in a fake extractor.Document loader parses the bytes.
document_loader.pysniffs the filename and dispatches. PDFs go through pdfplumber for text; if text-density is under 100 chars per page it falls back to PyMuPDF page rendering → PNG → vision. Images pass straight through as base64..txt/.mddecode as UTF-8. Returns aLoadedDocument(text, images_b64, source_type).Dispatch on
doc_type. For invoices/receipts, one system prompt + the full document text + images → one OpenAI call. For filings, the section chunker runs first, slicing the doc into cover + Item 8 + Item 1A blocks, and the message builder stitches those into three labeled sections. This is the trick that makes 10-Ks affordable (~$0.06/doc instead of ~$0.60/doc if you ship the whole 150K-token document).OpenAI
chat.completions.parsewith a Pydantic response format. This is the game-changer. Instead of asking for JSON in a prompt and hoping, we hand OpenAI the envelope schema — Pydantic v2 → JSON Schema — and OpenAI's strict mode guarantees the return matches. No JSON parsing errors, no missing fields, no extra fields. If the model can't comply, the API returns 400 instead of garbage.Response gets wrapped in
ExtractionResult. The extractor computesoverall_confidencefrom the per-field scores, attaches the raw text snippet for debugging, and returns(result, metrics). Metrics include tokens, cost, latency, and the model used — every request is fully accounted for.
3. Every layer, one at a time
3.1 Pydantic schemas + the strict-mode constraint
src/schemas/{base,common,invoice,receipt,filing,registry}.py.
The base: StrictModel has extra="forbid". This one config choice
cascades through everything. It means:
- The model cannot invent fields. If it hallucinates
internal_note, Pydantic rejects the whole response. - OpenAI's structured-outputs strict mode requires
additionalProperties: falseon every object in the schema — which is exactly whatextra="forbid"produces. - Adding a new field means adding it to the schema. There's no other place it can come from.
The registry pattern: src/schemas/registry.py maps "invoice" | "receipt" | "filing" to schema classes. Adding a new domain (e.g. purchase orders)
is a one-line change here plus a new schema file. Downstream code — the API,
the eval CLI, the extractor, the envelope factory — all look up via
get_schema(doc_type). No hardcoded branches anywhere.
Money as float, not Decimal. Read this carefully because it's a
question you will be asked. OpenAI structured outputs represent Decimal
as string in JSON Schema — which pushes the numeric parsing back onto the
model as a string task and reduces precision. float is number in JSON
Schema, which the model handles natively. We round to 2 decimals on assignment
via a validator. If this were an accounting system-of-record, we'd use
Decimal at the application layer after extraction — but at the LLM
boundary, float is correct.
3.2 The extractor + the envelope pattern
src/extractors/extractor.py orchestrates. src/extractors/envelope.py
handles the Pydantic gymnastics.
The envelope pattern answers a subtle question: what does the LLM
return? Not ExtractionResult directly, because some of that (like
overall_confidence, computed as mean-of-scores) is our code's job, not the
model's. Instead we generate — dynamically per doc type — an envelope class:
Envelope = create_model(
"InvoiceEnvelope",
data=(Invoice, ...),
field_confidences=(list[FieldConfidence], ...),
warnings=(list[ExtractionWarning], ...),
)
The model outputs the envelope, our code wraps it in ExtractionResult and
adds the computed pieces. Envelopes are LRU-cached so they're created once
per doc type.
3.3 The document loader
src/extractors/document_loader.py. Dispatches on filename + magic bytes:
.pdf→ try pdfplumber first. If text density is under 100 chars/page, fall back to PyMuPDF page rendering → PNG → send images to vision..png / .jpg / .jpeg / .webp / .tiff / .bmp→ base64 encode, ship as vision input..txt / .md / .log / .text→ UTF-8 decode, ship as text.- Anything else →
source_type="empty", extractor raises.
The text/image branch matters because a scanned receipt with no OCR layer looks empty to pdfplumber. The rendering fallback lets vision do the work.
3.4 The OpenAI client
src/extractors/openai_client.py. Two non-obvious details:
temperature=None (not 0.0). The gpt-5 family only accepts the default
temperature; passing 0.0 returns a 400. We use a nullable field and only
send it if the caller explicitly requests one. This bit us early — the
original code hardcoded temperature=0.0 and every gpt-5-nano call failed.
reasoning_effort="minimal". gpt-5 models have an internal chain-of-
thought that's billed to you as reasoning tokens. On our schema, the default
effort costs $0.042/doc and takes 30 seconds. minimal costs $0.012/doc and
takes 7 seconds with no measured quality loss. The flag is exposed all the
way through the CLI so the benchmark can compare.
Tenacity retry. The client retries on RateLimitError and APIError with
exponential backoff. Nothing exotic — just a decorator.
3.5 Section chunker for 10-Ks
src/extractors/section_chunker.py. This one's pretty:
A 10-K is 50-250 pages. Even at gpt-5's 400K token context, feeding the whole
thing costs ~$0.60/doc. But 10-Ks have structured Items — Item 1A. Risk Factors, Item 7. MD&A, Item 8. Financial Statements — that the chunker
finds via regex.
The regex tolerates every real-world variant: ITEM 1A., Item 1A —,
item 1a:. It runs across the whole document and returns every hit, then
keeps only the last occurrence of each item ID — because every 10-K
mentions each Item at least twice (Table of Contents + real section), and the
last one is always the real body.
Cover slice = first 4KB before the first Item. Section bodies = from just
after the heading to the start of the next heading. Result: a ChunkedFiling
with .get("1A"), .has("8"), .get_text("1A", default="…").
Downstream, the extractor stitches cover + Item 8 + Item 1A into three labeled blocks in one prompt. Prompt tokens drop from ~150K to ~30-40K, cost drops from ~$0.60/doc to ~$0.06/doc, and the model has less distractor text to sift through.
3.6 FastAPI backend
src/api/main.py, routers/, middleware.py, errors.py.
- Endpoints:
GET /(banner),GET /health(probe for Docker),GET /schemas(list doc types),GET /schemas/{doc_type}(JSON Schema),POST /extract(the money endpoint). - Dependency injection:
get_extractorreturns a singletonDocumentExtractorvialru_cache. In tests,app.dependency_overridesswaps it for a fake — so the whole test suite runs without an OpenAI key. - Middleware:
RequestIDMiddlewareattaches anX-Request-IDheader;AccessLogMiddlewareemits one structured JSON log line per request. - Error envelope: every 4xx/5xx returns
{"error": {code, message, request_id, details}}. Consistent shape for the frontend to render. - CORS: wide open. Fine for a portfolio; you'd lock this down in prod.
3.7 Evaluation harness
src/eval/{flatten,comparators,metrics,runner,report,cli}.py.
This is the piece I'm proudest of. Given a JSONL of ground-truth records
and an extractor function, it produces per-record CSV, JSON summary, and
resume-worthy markdown — all reproducible from python scripts/run_eval.py.
Flattener walks a Pydantic model + dicts recursively and produces
{"total": 12.99, "line_items[0].description": "Coffee", ...}. So nested
schemas score the same way flat ones do.
Type classifier looks at each field's Python type and JSON value:
floatfields withmoneyin the name → money comparator (0.01 absolute OR 0.5% relative tolerance — either passes)- other
float/int→ number comparator (exact) strwith currency/SKU/phone/ID hints → exact comparator- other
str→ fuzzy comparator (rapidfuzztoken_set_ratio ≥ 85) date/time→ ISO equality
Metrics compute per-field TP/FP/FN/TN, then micro F1 (weighted by field support), macro F1 (mean of per-field F1 across every field with non-zero support), and doc-exact-match (fraction of docs where every field is right).
Runner takes an injected extractor function (ExtractorFn protocol),
so --mode selfcheck uses a mock that returns ground truth verbatim
(validates the eval pipeline itself always hits F1=1.0), and --mode live
uses DocumentExtractor. Same code path.
The CLI accepts --model and --reasoning-effort — which is what makes the
multi-model benchmark a one-command sweep.
3.8 UI — Paper & Ink
ui/src/. React + Vite + TypeScript + Tailwind + Motion + React Three Fiber.
Deliberate choice: do not look like a generic AI-SaaS demo. Editorial aesthetic. Cream paper + deep ink navy in light mode; warm charcoal + parchment in dark. Instrument Serif italic for the display face. Grain overlay via inline SVG turbulence.
Hero: a 3D paper sheet with mouse parallax (React Three Fiber), procedural canvas texture (no glTF file), that folds a corner and turns green when the extraction succeeds. Reduced-motion respected.
Dropzone: drag-drop or browse, sample buttons for a real receipt PNG and
invoice PDF that ship in ui/public/samples/. Doc-type picker with Receipt | Invoice | 10-K. Confidence rendered as an ink well that fills from the
bottom; cost as a wax-stamped number.
API contract in ui/src/lib/api.ts. Types mirror the FastAPI response
shape 1:1.
3.9 Docker + HF Spaces single-container deploy
Root Dockerfile is three-stage:
node:20-alpinebuilds the UI →dist/.python:3.11-sliminstalls the Python venv.- Runtime: nginx + uvicorn + tini as PID 1.
docker/nginx.hf.conflistens on 7860 (HF's required port), servesdist/at/, proxies/api/*to127.0.0.1:8000. Entrypoint launches uvicorn in the background, polls/healthfor up to 30s so nginx never serves 502s during boot, then exec's nginx in the foreground with a SIGTERM trap.
docker/api.Dockerfile + docker/ui.Dockerfile + docker-compose.yml are
the two-container local-dev variant. Same code, different packaging.
HF Space YAML frontmatter (sdk: docker, app_port: 7860) tells HF to build
the root Dockerfile. OPENAI_API_KEY is a Space secret.
3.10 GitHub Actions CI
.github/workflows/ci.yml. Three parallel jobs on every push:
- python-lint-and-test — ruff + pytest across 126 unit tests (no OpenAI key needed, thanks to the injected extractor).
- ui-typecheck-and-build — tsc + vite build.
- docker-build — buildx multi-arch build with GHA cache, gated on both lint jobs.
4. The 14 design decisions you should be able to defend
If asked "why did you do X instead of Y," here's the honest answer for each.
| # | Decision | Alternative | Why we chose ours |
|---|---|---|---|
| 1 | Pydantic v2 with extra="forbid" |
dataclasses, Marshmallow, JSON Schema alone | OpenAI's structured-outputs strict mode maps directly onto Pydantic's config. Free validation + JSON Schema generation. |
| 2 | Money as float with rounder validator |
Decimal |
OpenAI structured outputs render Decimal as string in JSON Schema — model has to parse. float is native number. Round on assignment. Convert to Decimal at the app layer if needed. |
| 3 | Envelope pattern (dynamically generated) | Model outputs full ExtractionResult |
Some fields (overall_confidence, raw_text_snippet) are our code's job. Envelope is the LLM contract; ExtractionResult is the app contract. |
| 4 | Registry for doc types | if/elif on doc_type |
One place to add a new domain. Downstream code is generic. |
| 5 | client.beta.chat.completions.parse |
Prompt-then-JSON.loads | Native structured outputs eliminate a whole class of parsing errors and hallucinated schemas. |
| 6 | reasoning_effort="minimal" on gpt-5 |
Default effort | 3.5× cheaper, 4× faster, no measurable quality drop on structured extraction. Verified with the eval harness. |
| 7 | temperature=None (opt-in) |
temperature=0.0 hardcoded |
gpt-5 family rejects temperature=0.0. Had to be discovered by breakage. |
| 8 | Section chunking for 10-Ks | Whole-doc single call | ~10× cost reduction ($0.60→$0.06) with equal-or-better signal ratio. |
| 9 | Regex chunker, "keep last occurrence" | Full parser (unstructured, llama_index) | Real 10-Ks mention each Item at least twice (TOC + body). Keep-last is deterministic and works. |
| 10 | Injected extractor in eval + API | Real extractor everywhere | Test suite runs without OpenAI. CI stays deterministic and free. |
| 11 | Rapidfuzz token_set_ratio ≥ 85 for text |
Exact string match | Merchant names print as "SANYU STATIONERY" vs "Sanyu Stationery Shop"; those are the same merchant. |
| 12 | Money tolerance: 0.01 abs OR 0.5% rel | Exact match | Real receipts round differently. Both prevent silly-false-negatives. |
| 13 | React + Motion + R3F for UI | Streamlit / Gradio | Portfolio project. Recruiters can tell in 5 seconds whether the UI was thrown together. Editorial aesthetic on purpose. |
| 14 | Single Docker container on HF Spaces | HF SDK Gradio, or Vercel + Railway | HF Spaces is the AI-community-recognized host, single container = one URL, one deploy, one health check. |
5. The roadmap explained
The README roadmap has four checkboxes. Here's what each one actually means in concrete engineering terms.
v1 — Invoices & Receipts pipeline + multi-model benchmark ✅
Shipped 2026-07-04 → 2026-07-05. Full stack: schemas, extractor, FastAPI, React UI, Docker, HF Spaces, 96 tests, CI green. Live eval on gpt-5-nano hit 0.94 F1 on receipts at $0.012/doc, 6s latency. Multi-model benchmark swept nano vs mini vs full at reasoning-effort=minimal on the same 10 records; nano was Pareto-optimal (0.896 micro F1 at $0.0116/doc) — larger tiers led on macro F1 only.
v2 — SEC 10-K pipeline ✅
Shipped 2026-07-05 → 2026-07-06. Adds the filing doc type end-to-end:
FilingCover + FilingFinancials + RiskFactor Pydantic schemas, regex-based
section chunker for Items 1A/8, filing-specific system prompt with unit-of-
measure normalization rules, EDGAR downloader script that respects SEC's
10 req/s + User-Agent policy, XBRL-driven auto-ground-truth builder. First
live eval: 0.56 micro F1 at $0.06/doc. v2.2 attempted three targeted
fixes (prompt worked examples, FIN_MAP expansion for banks, cover backfill)
— per-field improvements on revenue/total_equity/operating_income, aggregate
stayed flat because filing_date regressed and total_debt didn't move. Full
diagnose→try→measure narrative in the README. 30 new tests, 126 total.
v3 — Streaming extraction + async batch API ✅
Shipped 2026-07-06. Two extra endpoints landed alongside POST /extract:
Streaming (POST /extract/stream). Returns Server-Sent Events. Each
frame is event: <name>\ndata: <json>\n\n. The stages we emit are
starting, loading, model_call, validated, then result with the
full validated ExtractionResult, then done. Errors are in-band
error events (the HTTP 200 has already gone out by the time the
extractor raises).
Why progress events instead of partial JSON deltas: OpenAI's
chat.completions.parse() with stream=True yields raw token chunks,
not partial validated Pydantic objects. To emit field-by-field you'd
have to run your own streaming JSON parser (ijson / json-stream)
that tolerates mid-value boundaries, then re-validate the accumulated
object against the envelope schema — that gets fragile on nested arrays
(line items, risk factors). Shipping progress events is the honest UX
without the JSON-stream footguns. If OpenAI adds object-level streaming
for structured outputs later, we extend this endpoint with partial
events on top of the existing progress ones — the SSE format welcomes
both.
Batch (POST /extract/batch + GET /extract/batch/{job_id}).
Multipart with N files. Returns 202 with a job_id immediately;
extraction runs in FastAPI BackgroundTasks. GET returns a snapshot:
overall status, per-item status, per-item result/metrics/error as they
land. Poll every 1-3s.
Concurrency architecture (src/api/batch_store.py):
- One in-memory
dictguarded by anasyncio.Lock= the job store. Restart wipes it — that's the accepted tradeoff at this scale. Prod swaps Redis + RQ in without touching the API. - One
asyncio.Semaphore(5)initialized on first use, shared across every in-flight job. OpenAI rate limits are per-org, not per-job, so job-scoped limits wouldn't protect us. - Extraction runs via
asyncio.to_thread(extractor.extract, …)because the openai-python SDK is synchronous under.parse(). Threading keeps the event loop free to serviceGET /extract/batch/{id}polls and to fan work out to the next job.
Tests: 12 new (in tests/unit/test_api_v3.py), 138 total. All use the
injected fake extractor — no OpenAI key required in CI.
Estimated work: shipped in 1 session, no API cost. Same extractor, different wrappers.
v4 — Fine-tuning experiment vs. base GPT-5 nano ✅
Shipped 2026-07-06 (code + tests + docs). Real-scale training run pending on your side.
Three scripts, one loop:
scripts/prep_ft_dataset.py— reads a receipt/invoice JSONL (must havetext+ground_truth), converts each row to OpenAI's chat-completions fine-tuning format. The system message is the exact production prompt (get_prompt(doc_type)); the user message is the exact production shape (---BEGIN DOCUMENT TEXT---block); the assistant message is the envelope ({data, field_confidences, warnings}) with the ground truth asdata. If any of those diverge from whatDocumentExtractorsends at inference, the fine-tune won't transfer — hence the shape-matching tests. 80/20 train/val split, deterministic per--seed.scripts/launch_finetune.py— uploads train + val files viaclient.files.create(purpose="fine-tune"), kicks off the job viaclient.fine_tuning.jobs.create(), then polls status every 30 s until the job finishes.Ctrl-Cdetaches cleanly (the job keeps running on OpenAI's side). Default target isgpt-4o-mini-2024-07-18— the cheapest reliable fine-tune base at ~$3 / 1M training tokens.scripts/compare_finetune.py— takes the resultingft-modelid and runs the same 10-record smoke eval (SROIE + CORD) against both the base gpt-5-nano @ minimal and the fine-tune, emits a side-by-side comparison table (evaluation/finetuning/<timestamp>/comparison.{md,csv,json}) — same shape as the multi-model benchmark.
9 new tests in tests/unit/test_prep_ft_dataset.py — pure JSON
transformation tests, no network. Confirms the fine-tuning JSONL is
valid OpenAI chat format, the assistant content is a valid envelope, the
system prompt matches production, the user shape matches
DocumentExtractor._build_messages(), the 80/20 split is deterministic,
malformed rows are skipped without crashing, and OpenAI's ≥10-example
minimum is warned about explicitly.
The interesting resume story isn't "F1 went up." It's the tradeoff: gpt-5-nano @ minimal already hits 0.94 F1 on receipts at $0.012/doc with no schema lock-in. A fine-tuned gpt-4o-mini will probably beat that on F1 (say 0.96+) at ~2× per-token cost, and only for this schema — a schema change means retraining. Whether you ship it is a business-tradeoff question, not a technical one. Senior answer: "I built the pipeline, ran it, and the numbers didn't justify shipping the fine-tune for a schema this small and this well-served by prompting." Junior answer: "I fine-tuned and F1 went up." Pick your audience.
To run the real experiment, you need > 10 training examples — the
smoke datasets have 5 each. Pull the full SROIE / CORD splits with
python scripts/prep_datasets.py all first, then point
prep_ft_dataset.py --input data/processed/sroie.jsonl at the fuller
data. Realistic cost: $0.50-$2 to train, $0.30-$0.80 for the
comparison eval on gpt-4o-mini.
6. Interview cheat sheet
"Walk me through your project."
Take 90 seconds. Don't start with the stack. Start with the story:
I built a service that turns three kinds of business documents — invoices, receipts, SEC 10-Ks — into schema-validated JSON. The interesting part isn't the extraction itself; it's everything around it. Pydantic v2 with
extra="forbid"gives OpenAI's structured-outputs strict mode a schema to enforce, so the model can't hallucinate fields. There's an evaluation harness that computes per-field precision, recall, and F1 against public ground truth. I used that harness to benchmark three GPT-5 tiers and proved that nano is Pareto-optimal on this workload. For 10-Ks I built a regex-based section chunker so we only ship Item 8 and Item 1A to the model — that drops cost from $0.60/doc to $0.06/doc. Deployed on Hugging Face Spaces via a single Docker container. Full stack: FastAPI backend, React + Motion + R3F frontend, GitHub Actions CI running 126 tests.
"Why Pydantic?"
Runtime validation + JSON Schema generation for free. And its extra="forbid"
config maps 1:1 onto OpenAI's strict mode requirement of
additionalProperties: false. If I hadn't used Pydantic I'd have written
that JSON Schema by hand and validated the response manually.
"How do you handle model errors or hallucinations?"
Three layers. First, extra="forbid" means the model cannot invent fields —
OpenAI's strict mode rejects those before we ever see them. Second, per-field
confidence scores let us surface uncertainty to the UI. Third, the eval
harness measures F1 against ground truth so we know empirically which
fields the model gets wrong — you can't catch what you don't measure.
"How do you evaluate quality?"
Per-field precision, recall, F1 with type-appropriate comparators — money uses a 0.01 absolute or 0.5% relative tolerance; text uses rapidfuzz's token-set ratio at 85; dates use ISO equality. Then micro F1 for overall, macro F1 for how rare fields do, and doc-level exact-match as the strictest metric.
"What was the hardest part?"
Two things. One: 10-K unit-of-measure — "in millions" means multiply
by a million and the model kept forgetting. I did a diagnose→try→measure
loop where I added worked examples to the prompt. Revenue precision doubled;
some other fields regressed. Real ML/eng work. Two: the streaming
consideration — I punted it to v3 because getting partial validated Pydantic
objects out of a token stream is a whole architectural rethink; the current
.parse() API only gives you the object at completion.
"What would you do differently?"
Two-pass extract-then-verify for money fields. First call extracts as we do now; second call is prompted with "here's what you just returned — verify each dollar figure against the scale header in the source." Empirically that kind of self-review pattern is where the next 15-20 F1 points on 10-K financials live.
"How did you deploy?"
Single-container Docker on Hugging Face Spaces. Multi-stage build: node
builds the UI to dist/, python installs the venv, runtime image has nginx
- uvicorn + tini. Entrypoint launches uvicorn in the background, polls
/healthfor up to 30 seconds so nginx never serves 502s while boot is in progress, then exec's nginx as PID 1. Space secret carries the OpenAI key.
"Why not just use LangChain?"
For this use case it would add complexity without helping. LangChain's value is when you need agents, tool use, RAG. Structured extraction with a Pydantic response format is one API call — LangChain would be a wrapper over what I already have. If v3 grows an agent-style tool-use pattern (e.g. "look up this vendor in our CRM") I'd reconsider.
"How much did all this cost you to build?"
Under $2 in OpenAI API cost total. Multi-model benchmark was $0.36. Two 10-K eval runs were $0.30 each. Everything else is free — HF Spaces free tier, GitHub free-tier CI, the sandbox for dev.
7. File map
04-structured-data-extraction/
├── src/ # Backend Python
│ ├── schemas/ # Pydantic schemas + registry
│ │ ├── base.py # StrictModel, ExtractionResult, FieldConfidence, ExtractionWarning
│ │ ├── common.py # Address, Party, MoneyMixin, currency helpers
│ │ ├── invoice.py # Invoice, LineItem
│ │ ├── receipt.py # Receipt, ReceiptLineItem
│ │ ├── filing.py # Filing, FilingCover, FilingFinancials, RiskFactor ← v2
│ │ └── registry.py # doc_type → schema class
│ ├── extractors/
│ │ ├── document_loader.py # PDF/image/txt → LoadedDocument
│ │ ├── prompts.py # SYSTEM_PROMPT_{INVOICE,RECEIPT,FILING} + common rules
│ │ ├── envelope.py # Dynamic envelope Pydantic model, per-schema LRU cached
│ │ ├── openai_client.py # OpenAI wrapper, retry, reasoning_effort, temperature=None
│ │ ├── section_chunker.py # Regex-based 10-K Item splitter, TOC dedup ← v2
│ │ └── extractor.py # Top-level DocumentExtractor, dispatches on doc_type
│ ├── api/
│ │ ├── main.py # FastAPI app + CORS + middleware wiring
│ │ ├── deps.py # get_extractor dependency (LRU-cached singleton)
│ │ ├── errors.py # APIError subclasses, exception handler
│ │ ├── middleware.py # RequestID + AccessLog middleware
│ │ └── routers/
│ │ ├── health.py # GET /, /health
│ │ ├── schemas.py # GET /schemas, /schemas/{doc_type}
│ │ └── extract.py # POST /extract (multipart)
│ ├── data_prep/ # SROIE + CORD normalizers, JSONL writer
│ ├── eval/
│ │ ├── flatten.py # Nested Pydantic → flat dotted-path dict
│ │ ├── comparators.py # money / number / text / exact / date comparators
│ │ ├── metrics.py # TP/FP/FN/TN, micro F1, macro F1, doc-exact
│ │ ├── runner.py # run_eval(records, extractor_fn, doc_type) → EvalReport
│ │ ├── report.py # write CSV, JSON summary, markdown
│ │ └── cli.py # argparse entry point (--mode, --model, --reasoning-effort)
│ └── utils/ # config (dotenv), cost_tracker, logging (loguru)
├── ui/ # React + Vite + TS + Tailwind + Motion + R3F
│ ├── src/
│ │ ├── App.tsx # Page composition
│ │ ├── main.tsx # Vite entry
│ │ ├── components/
│ │ │ ├── Hero.tsx # Kinetic-headline + 3D paper
│ │ │ ├── PaperScene.tsx # R3F Canvas, procedural paper texture
│ │ │ ├── ExtractSection.tsx# Workbench (Dropzone + ResultsPanel)
│ │ │ ├── Dropzone.tsx # File upload + doc-type picker + samples
│ │ │ ├── ResultsPanel.tsx # JSON view + confidence + cost
│ │ │ ├── ConfidenceInkwell.tsx, JsonView.tsx, MetricsStrip.tsx, WarningsList.tsx
│ │ │ ├── HowItWorks.tsx, Numbers.tsx, TopNav.tsx, Footer.tsx
│ │ │ ├── ThemeToggle.tsx # dark/light one-attribute flip
│ │ │ └── CustomCursor.tsx # ink-dot cursor with spring lag
│ │ ├── hooks/ # useExtract, useTheme
│ │ ├── lib/ # api.ts (fetch client), samples.ts (Coffee/Software)
│ │ ├── styles/ # theme.css (tokens), globals.css (grain, cursor)
│ │ └── types.ts # DocType, ExtractResponse mirror the API
│ ├── public/samples/ # coffee_receipt.png + software_invoice.pdf (real files!)
│ ├── package.json, tsconfig.json, vite.config.ts, tailwind.config.js
├── scripts/ # CLI tools
│ ├── prep_datasets.py # Download + normalize SROIE/CORD
│ ├── run_eval.py # Thin wrapper over src/eval/cli.py
│ ├── run_multimodel_benchmark.py # Matrix sweep → comparison.{md,csv,json}
│ ├── download_edgar.py # 5-issuer 10-K download from SEC EDGAR ← v2
│ └── build_filings_gt.py # XBRL companyfacts → ground truth JSONL ← v2
├── tests/
│ └── unit/ # 126 tests, no OpenAI key required
│ ├── test_schemas.py # Invoice/Receipt schema
│ ├── test_filing_schema.py # Filing schema — 20 tests ← v2
│ ├── test_section_chunker.py # Section chunker — 10 tests ← v2
│ ├── test_extractor.py # DocumentExtractor with FakeClient
│ ├── test_eval.py # Flatten, comparators, metrics
│ ├── test_data_prep.py # SROIE + CORD normalizers
│ └── test_api.py # FastAPI TestClient + dependency override
├── data/
│ ├── samples/ # Committed 10 hand-crafted samples
│ └── raw/10k/ # Downloaded 10-Ks (gitignored by default) ← v2
├── evaluation/
│ ├── smoke_sroie_sample.jsonl # 5 receipts (SROIE, live eval)
│ ├── smoke_cord_sample.jsonl # 5 receipts (CORD, live eval)
│ ├── smoke_filings_sample.jsonl # 5 10-Ks (live eval) ← v2
│ ├── reports/<timestamp>/ # Per-run reports
│ └── benchmarks/<timestamp>/ # Multi-model comparison rollups
├── docker/
│ ├── api.Dockerfile # Two-container local dev
│ ├── ui.Dockerfile # Two-container local dev
│ ├── nginx.conf # Two-container proxy config
│ ├── nginx.hf.conf # HF Space single-container config (port 7860)
│ └── hf-entrypoint.sh # tini + uvicorn + nginx orchestration
├── Dockerfile # Single-container HF build (3-stage)
├── docker-compose.yml # Two-container local dev
├── .github/workflows/ci.yml # 3 parallel jobs
├── requirements.txt # Python deps
├── pyproject.toml # ruff + black + pytest config
├── .env.example # OPENAI_API_KEY placeholder
├── .gitignore, .gitattributes
├── README.md # The narrative + numbers
├── SHIPPING.md # Resume bullet + LinkedIn draft
└── LEARN.md # This document
Read this once. Skim it before any interview. Every design decision has an answer here — no hand-waving needed.