README.md

---
title: Hybrid Vision RAG PDF Processor
emoji: 📄
colorFrom: indigo
colorTo: blue
sdk: gradio
sdk_version: "5.20.0"
app_file: main.py
pinned: false
license: mit
---

# Hybrid Vision RAG PDF Processor

A demo-ready Python project that extracts PDF content, analyzes charts/graphs via a Vision LLM, and answers questions with a Hybrid RAG system (Vector + Knowledge Graph + BM25).

## Tech stack

- **Docling** — PDF content extraction (text, pages, figures)
- **OpenRouter** — Vision LLM for chart/graph analysis
- **Groq** — LLM powering the RAG pipeline
- **LlamaIndex** — Vector + Knowledge Graph + BM25 retrieval with RRF fusion
- **Gradio** — demo UI

## What's new

- ✅ **Clean layered architecture** — `config`, `models`, `services`, `retrieval`, `session`, `web`
- ✅ **Own session architecture** — `Session` entity + pluggable `SessionStore` + `SessionManager` with janitor
- ✅ **Pydantic configuration** — all settings loaded from `.env` with validation
- ✅ **Structured logging** — no more raw `print()` statements
- ✅ **Polished Gradio demo UI** — live progress, status badge, commands, and session controls
- ✅ **Pre-flight API validation** — OpenRouter + Groq are live-checked before the pipeline starts; failures pop a warning and block processing (see the **API Status** panel)
- ✅ **Unit tests** for session lifecycle and BM25 retrieval
- ✅ **Unified CLI** — `extract`, `preprocess`, `app`

## Quick start

### 1. Install dependencies

```bash
uv sync
# or
pip install -e .
```

### 2. Configure API keys

Copy `.env.example` to `.env` and add your keys:

```bash
cp .env.example .env
```

```env
OPENROUTER_API_KEY=sk-or-v1-...
GROQ_API_KEY=gsk_...
```

### 3. Launch the demo

```bash
app
# or
python -m docling_pdf_processor
```

Open `http://localhost:7860` in your browser.

---

## Docker

Run the demo in a container (secrets are injected at runtime — never baked into the image):

```bash
# 1. Put real keys in .env (the image does NOT see your host's OS env vars)
#    OPENROUTER_API_KEY=sk-or-v1-...
#    GROQ_API_KEY=gsk_...

# 2. Build & run
docker compose up --build
```

Then open `http://localhost:8010`.

Without compose:

```bash
docker build -t docling-pdf-processor .
docker run -p 8010:8010 --env-file .env docling-pdf-processor
```

The compose file mounts a named volume at `/app/.cache/huggingface` so Docling's model weights download once and persist across restarts.

> ⚠️ **Key source caveat for containers:** on your host, `GROQ_API_KEY` may live in an OS environment variable that overrides the `.env` placeholder (the API Status panel shows `[source: OS env $GROQ_API_KEY]`). A container does **not** inherit host OS env vars — only what you pass via `--env-file` / `env_file`. So you must put the **real** Groq key in `.env` before `docker compose up`, or the pre-flight validation will block processing with `401 Invalid API key`.

---

## CLI usage

```bash
# Extract images + markdown from a PDF
extract --pdf path/to/doc.pdf --output ./extracted_images

# Preprocess extracted images with a Vision LLM
preprocess --quarters ./extracted_images --output graphs_description.pkl

# Launch the Gradio demo
app
```

---

## Project architecture

```
docling_pdf_processor/
├── config.py              # Pydantic Settings from .env
├── models.py              # Shared dataclasses
├── exceptions.py          # Domain errors
├── logging_config.py      # Structured logging setup
├── cli.py                 # Unified CLI
├── pipeline.py            # Session-aware PipelineOrchestrator
├── services/
│   ├── extractor.py       # Docling PDF extraction
│   ├── vision.py          # OpenRouter Vision-LLM
│   └── rag.py             # Hybrid RAG builder + query
├── retrieval/
│   ├── bm25.py            # BM25 keyword retriever
│   └── hybrid_rrf.py      # RRF fusion retriever
├── session/
│   ├── models.py          # Session + SessionStatus
│   ├── store.py           # SessionStore protocol + implementations
│   └── manager.py         # Lifecycle + janitor
└── web/
    ├── gradio_app.py      # Demo UI
    └── components.py      # UI helpers
```

### Data flow

```
User uploads PDF
        │
        ▼
┌─────────────────────┐
│  SessionManager     │  Creates UUID workspace in temp dir
│  (own session arch) │
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│ PipelineOrchestrator│  Streams progress via Queue
│  • ExtractorService   │  Docling → text, pages, figures
│  • VisionService      │  OpenRouter → graph descriptions
│  • RAGService         │  Vector + KG + BM25 indexes
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│   Gradio Chat UI    │  Session-scoped memory
│   !reasoning        │  Inspect retriever contributions
│   !compare          │  Side-by-side retriever comparison
└─────────────────────┘
```

---

## Session architecture

Every upload creates a `Session` with:

- A unique UUID
- An isolated workspace under `tempdir/docling_sessions/{uuid}/`
- A `SessionStatus` lifecycle: `created → extracting → analyzing → indexing → ready`
- A heartbeat timestamp for stale-session cleanup

The `SessionStore` protocol has two built-in implementations:

- `InMemorySessionStore` — default for single-process demos
- `FileSystemSessionStore` — persists session metadata to disk

A background janitor cleans sessions idle longer than `SESSION_MAX_AGE_SECONDS` (default 30 min).

---

## Chat commands

- `!reasoning <query>` — show how Vector / KG / BM25 contribute to the final answer
- `!compare <query>` — compare Vector, KG, BM25, and Hybrid answers side-by-side

---

## API validation

Before the pipeline runs, the app live-checks the two external APIs it depends on:

| API | Check | Endpoint |
|-----|-------|----------|
| OpenRouter (Vision LLM) | key + credits | `GET https://openrouter.ai/api/v1/key` |
| Groq (RAG LLM) | key + model **actually generate** (1-token chat completion) | `POST https://api.groq.com/openai/v1/chat/completions` |

- The **API Status** panel in the *Process PDF* tab shows each API as ✅ Valid / ❌ Invalid, with a **masked key prefix and the key's source** (`UI field` / `OS env $VAR` / `.env`) so it's clear which credential is being tested.
- Click **Validate APIs** to re-run the checks on demand; editing any key/model field live-refreshes the panel.
- When you click **Process PDF**, validation runs first. If any API fails (bad key → 401, wrong model → 404, no credits → 402), a popup warning shows which API and why, and the pipeline **does not start**.

> **Note on key sources:** pydantic-settings loads OS environment variables *with higher priority than* `.env`. So if a key is set in your shell/system environment, it overrides a placeholder in `.env`. The source tag in the panel makes this visible — e.g. `OS env $GROQ_API_KEY` instead of `.env`.

Validation calls are cheap and run in parallel, typically returning in ~1s.

---

## Production notes

- **Concurrency** — Gradio runs with `default_concurrency_limit=1` so only one pipeline processes at a time per process instance.
- **Multi-tenant scaling** — For true multi-user production, run each session in a separate worker or process.
- **Vector store** — Uses LlamaIndex's in-memory `SimpleVectorStore` (the previous DeepLake store is abandoned and breaks on NumPy 2 / Python 3.13). Indexes are rebuilt from the extracted files each session, which is the existing behavior, so nothing is lost — but for very large corpora consider swapping in a persisted store (FAISS, Chroma, pgvector).
- **API keys** — Never commit `.env`. It is already ignored by `.gitignore`.
- **Large PDFs** — PDFs > 100 MB trigger a warning but are not blocked.

---

## Development

### Run tests

```bash
pytest
```

### Run a specific stage from the CLI

```bash
# Extraction only
extract --pdf sample.pdf --output ./out

# Vision analysis only
preprocess --quarters ./out --output ./out/graphs.pkl --api-key $OPENROUTER_API_KEY
```

---

## License

MIT