DEVLOG.md

DEVLOG

Append-only log of design decisions, what changed, what didn't work, and why. Newest entries at the bottom. Don't rewrite history here — commit fixes go in git.

---

2026-05-14 — v0.1 — initial scaffold

Goal: Stand up an end-to-end pipeline that takes any scientific PDF and produces structured per-figure / per-table output, deployable as a Gradio app.

Decisions:

• Switched the extraction backend from raw PyMuPDF (notebook v0) to docling. Reason: PyMuPDF's doc.extract_image() only yields raster images, which silently skips vector figures (most plots in modern scientific PDFs). docling uses a layout model + region rendering, so it gets both.
• Caption-to-figure pairing now comes from docling's layout (not center-distance matching). More robust on multi-column papers.
• Added a linker.py step that scans body text for "Fig N" / "Table N" references and attaches surrounding paragraphs to each figure/table. This is the "figure context" the project brief calls out — wasn't in the notebook v0.
• Summarization uses Claude Sonnet 4.6 by default with vision input. Schema is structured JSON so it's downstream-indexable for the LENR dashboard.

Stack:

• miner.py (docling) → linker.py (regex on body text) → summarizer.py (Anthropic vision API) → app.py (Gradio).

Open questions:

• Quality on LENR-conference-proceedings PDFs (older scans) — might need OCR enabled. Test before deploy.
• Whether multi-panel figures (a/b/c) need panel-level splitting.

---

2026-05-14 — v0.1.1 — Gradio type-mismatch fix

Bug: First click on "Mine the paper" crashed with AttributeError: 'list' object has no attribute 'expandtabs' from inside gradio/components/markdown.py.

Cause: Progress yield statements were passing [] to the tables_view slot, but that's a gr.Markdown component which needs str. Same for json_download which needs None not "".

Fix: Corrected types in all intermediate yields and the no-PDF early return.

---

2026-05-14 — v0.1.2 — transformers version pin

Bug: ValueError: ... model type rt_detr_v2 but Transformers does not recognize this architecture.

Cause: docling's layout model uses RT-DETR v2, which needs transformers >= 4.49. Local env had an older version.

Fix: Pinned transformers>=4.49, docling>=2.40, docling-ibm-models>=3.4 in requirements.txt. Reinstalled.

---

2026-05-15 — v0.2 — first deployment + evaluation

Deployment:

• Live on Hugging Face Spaces: https://huggingface.co/spaces/deepesh-goel/figure-table-miner
• Pinned gradio_sdk_version: 5.29.0 in README YAML to fix dark-mode rendering bug (5.0.0 default had broken theme switching)
• Pinned huggingface_hub<1.0 to fix HfFolder import error
• Patched gradio_client.utils._json_schema_to_python_type to skip bool schemas (HF Spaces "No API found" bug)
• Wrapped pipeline in try/except so backend errors surface in the Status box instead of generic "Error"

Evaluation:

• Tested on 5 papers spanning ML, ACM/SIGKDD, Elsevier medical, and ICCF/LENR proceedings (55 pages, 27 figures, 16 tables ground truth)
• Tables: 16/16 detected (perfect across all five papers)
• Figures: 28 detected vs 27 ground truth, but heterogeneous — 3 papers near-perfect, 1 over-detected (P1: nested example figures inside a meta paper), 1 under-detected (P4: six panel figures merged into one composite)
• Caption number parsing succeeded 16/28 figures, 14/16 tables; failures concentrate in P1 and P4
• Discussion-paragraph linking attached context to 22/28 figures, 13/16 tables; ~100% when caption number parsed
• All summaries populated successfully; no LLM errors after Anthropic credits topped up
• Added two limitations to the report: "Nested figures (meta-content)" from P1, expanded "Multi-panel figures" with P4 as concrete case

(next entry goes here)