ARCHITECTURE.md

Personal Document Intel & Archiver — Architecture Plan

Overview

Local-first PDF RAG tool for caregivers. Upload medical documents, ask questions, get answers. Runs entirely offline.

Track & Badges

• Track: Backyard AI
• Badge targets: Off the Grid (no cloud), Llama Champion (llama.cpp), Off-Brand (custom UI), Field Notes (blog post)

Tech Stack

Component	Choice	Why
Runtime	llama.cpp (llama-cpp-python)	Off the Grid + Llama Champion badges
Model	Qwen 2.5 14B GGUF (Q4_K_M)	≤32B, strong on document reasoning, ~8GB RAM
Embeddings	all-MiniLM-L6-v2 via sentence-transformers	Small, fast, no cloud, runs locally
Vector store	FAISS (in-memory, persisted to disk)	Lightweight, no server needed
PDF parser	PyMuPDF (fitz)	Pure Python, fast, no cloud deps
Text splitter	LangChain's RecursiveCharacterTextSplitter	Clean chunking with overlap
UI	Gradio (custom theme)	Required by hackathon rules
Frontend polish	Custom CSS + gr.ChatInterface	Off-Brand badge

File Structure

personal-doc-archiver/
├── app.py                  # Gradio entry point
├── rag_engine.py           # RAG pipeline (embed → retrieve → generate)
├── pdf_loader.py           # PDF parsing + text chunking
├── config.py               # Paths, model name, chunk settings
├── requirements.txt        # Python dependencies
├── models/
│   └── (download target for Qwen GGUF)
├── data/
│   └── documents/          # Uploaded PDFs go here
├── cache/
│   ├── index.faiss         # Persisted FAISS index
│   └── chunks.json         # Chunk metadata
├── assets/
│   └── (custom CSS / branding)
└── README.md               # Submission notes

Data Flow

User uploads PDF
    ↓
pdf_loader.py extracts text (PyMuPDF)
    ↓
Text splitter chunks with overlap (500 chars, 50 overlap)
    ↓
sentence-transformers embeds each chunk
    ↓
FAISS stores embeddings in index
    ↓
User asks question
    ↓
Question embedded with same model
    ↓
FAISS searches top-3 chunks
    ↓
Context + question → llama.cpp model → answer
    ↓
Answer displayed in Gradio chat

Chunk Strategy

• Size: 500 characters
• Overlap: 50 characters
• Why: Medical documents have dense info; small chunks keep retrieval precise

Model Settings (llama.cpp)

• Context: 4096 tokens (enough for Qwen 14B with chunked context)
• Temperature: 0.1 (low — we want factual, not creative)
• GPU layers: depends on VRAM. RTX 5070 = all layers GPU. Fallback: CPU-only.
• Prompt format: Qwen 2.5 chat template

System Prompt (for the model)

You are a medical document assistant. You help caregivers find information in uploaded medical documents.
Answer using ONLY the provided context. If the answer isn't in the context, say "I couldn't find that in your documents."
Be concise and specific — cite dates, names, and numbers when available.

UI Layout (Gradio)

┌──────────────────────────────────────────┐
│  🏥 Personal Document Intel & Archiver    │
│  ─────────────────────────────────────── │
│  [⚠️ System: Offline — No data leaves    │
│   this computer]                         │
│                                          │
│  ┌──────────┐  ┌──────────────────────┐  │
│  │ 📁       │  │                      │  │
│  │ Upload   │  │   Chat Interface     │  │
│  │ Documents│  │                      │  │
│  │          │  │  "What did Dr. Chen  │  │
│  │ [Doc 1]  │  │   say about my       │  │
│  │ [Doc 2]  │  │   blood work?"       │  │
│  │          │  │                      │  │
│  │ Status:  │  │  "Your March labs   │  │
│  │ ✅ 2 docs│  │   showed elevated    │  │
│  │ loaded   │  │   cholesterol..."    │  │
│  └──────────┘  └──────────────────────┘  │
│                                          │
│  Examples: [What did Dr. Chen say...]    │
│  [When is the next cardiology appt?]     │
└──────────────────────────────────────────┘

Development Phases

Phase 1 — Core (today/tomorrow)

• Set up Gradio skeleton with custom theme
• Build PDF parser + chunker
• Wire up sentence-transformers + FAISS
• Integrate llama.cpp with Qwen 14B
• End-to-end: upload PDF → ask question → get answer

Phase 2 — Polish (middle week)

• Off-Brand UI polish (custom CSS, icons, status indicators)
• Document management (list, delete, re-index)
• Better error handling (bad PDFs, no model, empty context)
• Performance: multi-document search, file type detection

Phase 3 — Ship (last 2 days)

• Full test cycle on target hardware (mom's laptop / user's PC)
• Demo video recording
• Social media post
• Field Notes blog post in Obsidian
• Submit to build-small-hackathon org on HF Spaces
  • Note: Gradio app goes on HF Spaces for submission
  • Model itself stays local (badge requirement)
  • Space will use a placeholder model or OpenRouter for the demo
  • The "real" offline version is what runs on local machine

Questions for Mom

(To be filled once she sends them — placeholder structure)

• Category: Appointments
• Category: Medications
• Category: Lab Results
• Category: Insurance / Billing
• Category: Doctor Instructions