| # Personal Document Intel & Archiver — Architecture Plan |
|
|
| ## Overview |
| Local-first PDF RAG tool for caregivers. Upload medical documents, ask questions, get answers. Runs entirely offline. |
|
|
| ## Track & Badges |
| - **Track:** Backyard AI |
| - **Badge targets:** Off the Grid (no cloud), Llama Champion (llama.cpp), Off-Brand (custom UI), Field Notes (blog post) |
|
|
| ## Tech Stack |
|
|
| | Component | Choice | Why | |
| |---|---|---| |
| | **Runtime** | llama.cpp (`llama-cpp-python`) | Off the Grid + Llama Champion badges | |
| | **Model** | Qwen 2.5 14B GGUF (Q4_K_M) | ≤32B, strong on document reasoning, ~8GB RAM | |
| | **Embeddings** | `all-MiniLM-L6-v2` via sentence-transformers | Small, fast, no cloud, runs locally | |
| | **Vector store** | FAISS (in-memory, persisted to disk) | Lightweight, no server needed | |
| | **PDF parser** | PyMuPDF (`fitz`) | Pure Python, fast, no cloud deps | |
| | **Text splitter** | LangChain's RecursiveCharacterTextSplitter | Clean chunking with overlap | |
| | **UI** | Gradio (custom theme) | Required by hackathon rules | |
| | **Frontend polish** | Custom CSS + gr.ChatInterface | Off-Brand badge | |
|
|
| ## File Structure |
|
|
| ``` |
| personal-doc-archiver/ |
| ├── app.py # Gradio entry point |
| ├── rag_engine.py # RAG pipeline (embed → retrieve → generate) |
| ├── pdf_loader.py # PDF parsing + text chunking |
| ├── config.py # Paths, model name, chunk settings |
| ├── requirements.txt # Python dependencies |
| ├── models/ |
| │ └── (download target for Qwen GGUF) |
| ├── data/ |
| │ └── documents/ # Uploaded PDFs go here |
| ├── cache/ |
| │ ├── index.faiss # Persisted FAISS index |
| │ └── chunks.json # Chunk metadata |
| ├── assets/ |
| │ └── (custom CSS / branding) |
| └── README.md # Submission notes |
| ``` |
|
|
| ## Data Flow |
|
|
| ``` |
| User uploads PDF |
| ↓ |
| pdf_loader.py extracts text (PyMuPDF) |
| ↓ |
| Text splitter chunks with overlap (500 chars, 50 overlap) |
| ↓ |
| sentence-transformers embeds each chunk |
| ↓ |
| FAISS stores embeddings in index |
| ↓ |
| User asks question |
| ↓ |
| Question embedded with same model |
| ↓ |
| FAISS searches top-3 chunks |
| ↓ |
| Context + question → llama.cpp model → answer |
| ↓ |
| Answer displayed in Gradio chat |
| ``` |
|
|
| ## Chunk Strategy |
| - Size: 500 characters |
| - Overlap: 50 characters |
| - Why: Medical documents have dense info; small chunks keep retrieval precise |
|
|
| ## Model Settings (llama.cpp) |
| - Context: 4096 tokens (enough for Qwen 14B with chunked context) |
| - Temperature: 0.1 (low — we want factual, not creative) |
| - GPU layers: depends on VRAM. RTX 5070 = all layers GPU. Fallback: CPU-only. |
| - Prompt format: Qwen 2.5 chat template |
|
|
| ## System Prompt (for the model) |
| ``` |
| You are a medical document assistant. You help caregivers find information in uploaded medical documents. |
| Answer using ONLY the provided context. If the answer isn't in the context, say "I couldn't find that in your documents." |
| Be concise and specific — cite dates, names, and numbers when available. |
| ``` |
|
|
| ## UI Layout (Gradio) |
| ``` |
| ┌──────────────────────────────────────────┐ |
| │ 🏥 Personal Document Intel & Archiver │ |
| │ ─────────────────────────────────────── │ |
| │ [⚠️ System: Offline — No data leaves │ |
| │ this computer] │ |
| │ │ |
| │ ┌──────────┐ ┌──────────────────────┐ │ |
| │ │ 📁 │ │ │ │ |
| │ │ Upload │ │ Chat Interface │ │ |
| │ │ Documents│ │ │ │ |
| │ │ │ │ "What did Dr. Chen │ │ |
| │ │ [Doc 1] │ │ say about my │ │ |
| │ │ [Doc 2] │ │ blood work?" │ │ |
| │ │ │ │ │ │ |
| │ │ Status: │ │ "Your March labs │ │ |
| │ │ ✅ 2 docs│ │ showed elevated │ │ |
| │ │ loaded │ │ cholesterol..." │ │ |
| │ └──────────┘ └──────────────────────┘ │ |
| │ │ |
| │ Examples: [What did Dr. Chen say...] │ |
| │ [When is the next cardiology appt?] │ |
| └──────────────────────────────────────────┘ |
| ``` |
|
|
| ## Development Phases |
|
|
| ### Phase 1 — Core (today/tomorrow) |
| - [ ] Set up Gradio skeleton with custom theme |
| - [ ] Build PDF parser + chunker |
| - [ ] Wire up sentence-transformers + FAISS |
| - [ ] Integrate llama.cpp with Qwen 14B |
| - [ ] End-to-end: upload PDF → ask question → get answer |
|
|
| ### Phase 2 — Polish (middle week) |
| - [ ] Off-Brand UI polish (custom CSS, icons, status indicators) |
| - [ ] Document management (list, delete, re-index) |
| - [ ] Better error handling (bad PDFs, no model, empty context) |
| - [ ] Performance: multi-document search, file type detection |
|
|
| ### Phase 3 — Ship (last 2 days) |
| - [ ] Full test cycle on target hardware (mom's laptop / user's PC) |
| - [ ] Demo video recording |
| - [ ] Social media post |
| - [ ] Field Notes blog post in Obsidian |
| - [ ] Submit to build-small-hackathon org on HF Spaces |
| - Note: Gradio app goes on HF Spaces for submission |
| - Model itself stays local (badge requirement) |
| - Space will use a placeholder model or OpenRouter for the demo |
| - The "real" offline version is what runs on local machine |
|
|
| ## Questions for Mom |
| (To be filled once she sends them — placeholder structure) |
| - Category: Appointments |
| - Category: Medications |
| - Category: Lab Results |
| - Category: Insurance / Billing |
| - Category: Doctor Instructions |
|
|
