pdf-script/README.md

# PDF Accessibility Pipeline

Automatically analyses a PDF document and produces a fully tagged, screen-reader-accessible PDF conforming to the **PDF/UA-1** standard.

---

## How it works

```
data/input.pdf
      │
      ▼  Part 1 — generate_metadata  (Python)
      │
      ├─ YOLOv11 layout detection  ──► segmented regions
      ├─ Tesseract OCR             ──► extracted text
      ├─ LaTeX chunking            ──► document order + reading order
      └─ GPT-4o labeling + alt-text
      │
      ▼
output/elements.txt
      │
      ▼  Part 2 — embed_metadata  (Java / iText 9)
      │
      └─ PDF structure tree (tags) ──► output/tagged_output.pdf  (PDF/UA-1)
```

---

## Prerequisites

### 1 · Python (≥ 3.9)

```bash
pip install -r requirements.txt
```

> **System packages also needed:**
> | Tool | Purpose | macOS | Linux (apt) |
> |------|---------|-------|-------------|
> | **Tesseract** | OCR engine | `brew install tesseract` | `apt install tesseract-ocr` |
> | **Poppler** | PDF → image | `brew install poppler` | `apt install poppler-utils` |

### 2 · Java (≥ 11) + Maven (≥ 3.6)

```bash
brew install openjdk@11 maven   # macOS
# or: apt install openjdk-11-jdk maven  (Linux)
```

### 3 · YOLOv11 model weights

Download the pre-trained document-layout model:

```
https://github.com/moured/YOLOv11-Document-Layout-Analysis/releases
   → yolov11x_best.pt
```

Save it anywhere and set the path (see Configuration below).

### 4 · OpenAI API key

Required for LaTeX chunk mapping and figure alt-text generation.

---

## Configuration

Create a `.env` file in the project root (or export as environment variables):

```dotenv
# .env — copy this block and fill in your values

# REQUIRED: OpenAI API key
OPENAI_API_KEY=sk-...

# REQUIRED: Path to downloaded YOLOv11 weights
YOLO_MODEL_PATH=/absolute/path/to/yolov11x_best.pt
```

All other settings can be tweaked in [`generate_metadata/config.py`](generate_metadata/config.py).

---

## Input files

Place both inputs in the `data/` directory before running:

| What | Where to put it |
|------|----------------|
| Input PDF | `data/input.pdf` |
| LaTeX source | any subfolder inside `data/` e.g. `data/my-paper/` |

The pipeline auto-detects the LaTeX subfolder inside `data/`.
Everything else (`elements.txt`, tagged PDF, visualizations) is generated by the pipeline into `output/`.

---

## Quick start

```bash
# 1. Clone and enter the repo
git clone <repo-url>
cd pdf-script

# 2. Configure
cp .env.example .env        # then fill in OPENAI_API_KEY and YOLO_MODEL_PATH
# (or just export them in your shell)

# 3. Install Python dependencies
pip install -r requirements.txt

# 4. Build the Java tagger
mvn package -DskipTests     # creates pdftagger.jar

# 5. Place your inputs in data/
cp your-paper.pdf data/input.pdf
cp -r your-latex-project/ data/my-paper/   # any subfolder name works

# 6. Run the full pipeline
python pipeline.py
```

Outputs land in `output/`:

```
output/
├── elements.txt            ← structured metadata (roles + bboxes)
├── tagged_output.pdf       ← accessible PDF/UA-1 output
├── latex_chunks.txt        ← debug: LaTeX semantic chunks
├── segments2chunks_gpt.txt ← debug: OCR → chunk mapping log
├── pdf-imgs/               ← per-page PNG images
├── segmentations-1/        ← YOLO detection visualisations
├── segmentations-2/        ← refined segment visualisations
└── visualizations/
    ├── labels.pdf          ← annotated pages with element labels
    └── reading_order.pdf   ← annotated pages with reading-order numbers
```

---

## Usage reference

### Full pipeline
```bash
# Place data/input.pdf and data/<your-latex-folder>/ first, then:
python pipeline.py                              # auto-detects everything in data/
python pipeline.py --pdf path/to/paper.pdf     # custom PDF location
python pipeline.py --latex path/to/latex-dir/  # custom LaTeX parent directory
python pipeline.py --pages 3                   # first 3 pages only
```

### Part 1 only (analysis → elements.txt)
```bash
python -m generate_metadata.main \
    --pdf  data/input.pdf \
    --output output/
```

### Part 2 only (tag an existing elements.txt)
```bash
python pipeline.py --skip-metadata

# or call the Java tagger directly:
java -jar pdftagger.jar data/input.pdf output/elements.txt output/tagged_output.pdf
```

### Rebuild JAR (after changing Java code)
```bash
python pipeline.py --rebuild-jar
# or:
mvn package -DskipTests
```

### CLI reference
```
python pipeline.py --help

  --pdf PATH          Input PDF (default: data/input.pdf)
  --latex DIR         Directory containing the LaTeX source subfolder (default: data/)
  --output DIR        Output directory (default: output/)
  --pages N           Process first N pages only (-1 = all)
  --skip-metadata     Skip Part 1, reuse existing output/elements.txt
  --skip-tagging      Skip Part 2 (Java tagger)
  --rebuild-jar       Force rebuild of pdftagger.jar
```

---

## Project structure

```
pdf-script/
│
├── pipeline.py                     ← Main entry point (runs both parts)
├── requirements.txt                ← Python dependencies
├── pom.xml                         ← Maven build (embed_metadata JAR)
├── .env                            ← Your secrets (not committed)
│
├── data/
│   └── input.pdf                   ← Place your PDF here
│
├── output/                         ← All generated files (gitignored)
│
├── generate_metadata/              ← Part 1: Python analysis pipeline
│   ├── main.py                     ← Entry point for Part 1
│   ├── config.py                   ← API keys, model paths, settings
│   ├── pdf_loader.py               ── PDF → page images (pdf2image)
│   ├── segmentation.py             ── YOLOv11 layout detection + bbox utils
│   ├── ocr.py                      ── Tesseract OCR
│   ├── latex_processor.py          ── LaTeX flattening + semantic chunking
│   ├── mapping.py                  ── OCR ↔ YOLO mapping + refinement
│   ├── chunk_mapper.py             ── Segment → LaTeX chunk matching (GPT-4o)
│   ├── refinement.py               ── Semantic split / merge / clean
│   ├── labeling.py                 ── Accessibility role labeling
│   ├── reading_order.py            ── Reading order determination
│   ├── alt_text.py                 ── Figure alt-text (\\Description or GPT-4o)
│   ├── sanity_check.py             ── GPT-4o label + reading-order QA pass
│   └── output.py                   ── Write elements.txt + visualisation PDFs
│
└── embed_metadata/                 ← Part 2: Java PDF tagger (iText 9 / PDF/UA-1)
    └── src/main/java/com/pdftag/
        ├── itagpdf.java            ← CLI entry point
        └── TagPDF.java             ← Core tagging logic
```

---

## elements.txt format

Pipe-delimited, with leading `|` characters indicating nesting depth:

```
Sect|1|100|50|800|900|
|H1|1|120|60|780|120|Introduction
|P|1|120|130|780|200|This paper presents…
|FIGURE|1|120|210|500|450|Chart showing accuracy over time
|TABLE|1|120|460|780|650|
||TR|1|125|465|775|495|
|||TH|1|125|465|300|495|Method
|||TH|1|305|465|500|495|Accuracy
```

Columns: `[depth-pipes] role | page | x1 | y1 | x2 | y2 | text`

**Supported roles:** `H1` `H2` `H3` `P` `Sect` `FIGURE` `CAPTION` `TABLE` `TR` `TH` `TD` `L` `LI` `AUTHOR` `BIBENTRY` `FORMULA` `NOTE` `ARTIFACT`

---

## Troubleshooting

| Error | Fix |
|-------|-----|
| `OPENAI_API_KEY is not set` | Add key to `.env` or export in shell |
| `YOLO_MODEL_PATH` not found | Download weights and update `.env` |
| `tesseract: command not found` | `brew install tesseract` / `apt install tesseract-ocr` |
| `pdftoppm: command not found` | `brew install poppler` / `apt install poppler-utils` |
| `mvn: command not found` | `brew install maven` / `apt install maven` |
| JAR build fails | Check `java -version` — needs Java 11+. On macOS: `export JAVA_HOME=$(/usr/libexec/java_home -v 11)` |