Spaces:
Running
Running
| title: ClinIQ — Clinical Document Assistant | |
| emoji: 🩺 | |
| colorFrom: blue | |
| colorTo: green | |
| sdk: gradio | |
| sdk_version: "6.11.0" | |
| app_file: app.py | |
| python_version: "3.11" | |
| pinned: false | |
| license: apache-2.0 | |
| tags: | |
| - medical | |
| - rag | |
| - qwen | |
| - privacy | |
| - langgraph | |
| - llama-cpp | |
| - hackathon | |
| - build-small-2026 | |
| - tiny-titan | |
| # ClinIQ — Privacy-First Clinical Document Assistant | |
| ClinIQ is an AI-powered clinical document assistant built for small medical clinics. Upload patient records — discharge summaries, intake notes, consult reports, scanned documents — and instantly get answers, structured extractions, and proactive drug safety alerts. | |
| **Model:** Qwen2.5-3B-Instruct (Q4_K_M) · **Inference:** llama.cpp on Modal A10G GPU · **Framework:** Gradio gr.Server + LangGraph | |
| --- | |
| ## The Problem | |
| Small clinics handle dozens of patients daily. Nurses upload a discharge summary from the hospital, an intake note from last week, and a specialist consult from yesterday — all as separate files. Nobody has time to cross-reference all three. A medication prescribed in one document may conflict with an allergy documented in another. These conflicts get missed. | |
| ClinIQ solves this by reading all documents together and flagging dangerous combinations automatically — before any question is asked. | |
| --- | |
| ## What ClinIQ Does | |
| ### 1. Proactive Drug Safety Check | |
| The moment documents are uploaded, ClinIQ automatically scans every medication and allergy across all files and flags dangerous combinations. No question needed. | |
| Example: a discharge summary prescribes Aspirin 81mg. An intake note documents an Aspirin allergy causing bronchospasm. These are two separate files from two different visits. ClinIQ catches the conflict instantly: | |
| ``` | |
| DANGER Aspirin allergy contraindication | |
| Patient prescribed Aspirin 81mg but has documented bronchospasm reaction | |
| Recommendation: Switch to Clopidogrel (noted in cardiology consult) | |
| ``` | |
| ### 2. Clinical Q&A | |
| Ask any question in plain English. ClinIQ retrieves relevant chunks from all uploaded documents and generates a precise, grounded answer in ~2 seconds. | |
| - *"Who is the attending physician?"* → direct fact lookup | |
| - *"What are the follow-up instructions?"* → extracts from discharge summary | |
| - *"Is it safe to give this patient aspirin?"* → reasons across multiple documents | |
| - *"What alternative was recommended instead of aspirin?"* → retrieves from consult note | |
| ### 3. Structured Extraction | |
| Ask for a list and get a formatted table — not a paragraph of text. | |
| - *"List all medications"* → table with name, dose, frequency, route | |
| - *"What are all the allergies?"* → table with substance, reaction, severity | |
| - *"List all diagnoses"* → primary and secondary conditions | |
| - *"What are the vital signs?"* → structured vitals object | |
| ### 4. Cross-Document Reasoning | |
| ClinIQ retrieves context from all uploaded documents simultaneously. A question about drug interactions can pull the medication list from the discharge summary, the allergy list from the intake note, and the cardiology recommendation from the consult — combining all three into a single coherent answer. | |
| ### 5. OCR Support | |
| Supports scanned PDFs and image files (JPG, PNG, TIFF). Tesseract OCR extracts text from scanned documents automatically before indexing. | |
| --- | |
| ## Architecture | |
| ### Document Ingestion Pipeline | |
| ``` | |
| Upload (PDF / TXT / JPG / PNG / TIFF / scanned PDF) | |
| ↓ | |
| Text Extraction | |
| ├── Native PDF → pypdf (fast, text-selectable) | |
| ├── Scanned PDF → pdf2image + tesseract OCR | |
| ├── Image → tesseract OCR (upscaled to 1000px for accuracy) | |
| └── TXT / MD → direct read | |
| ↓ | |
| Chunking — 400 character chunks with 80 character overlap | |
| ↓ | |
| Dual Indexing | |
| ├── BM25 index (rank_bm25) — keyword matching | |
| └── Dense index (sentence-transformers/all-MiniLM-L6-v2) — semantic matching | |
| ``` | |
| ### Query Pipeline — LangGraph 5-Node Agent | |
| ``` | |
| User Question | |
| ↓ | |
| [1] CLASSIFY | |
| Rule-based classification — no LLM call | |
| Detects query type: simple / structured / complex / comparison | |
| ↓ | |
| [2] DECOMPOSE | |
| For complex and comparison queries only | |
| Qwen breaks the question into 2-3 focused sub-queries | |
| ↓ | |
| [3] RETRIEVE | |
| Runs each sub-query through hybrid retrieval: | |
| ├── BM25 score (keyword overlap) | |
| ├── Dense score (semantic similarity via cosine) | |
| └── Score fusion → top-6 unique chunks across all documents | |
| ↓ | |
| [4] GENERATE | |
| Qwen2.5-3B-Instruct synthesizes an answer from retrieved chunks | |
| Structured queries → returns JSON parsed into a table | |
| Free-text queries → returns a concise clinical answer | |
| ↓ | |
| [5] REFLECT | |
| Heuristic grounding check — verifies answer overlaps with context | |
| Flags potential hallucinations without an extra LLM call | |
| ↓ | |
| Streaming response → custom HTML/JS frontend (gr.Server) | |
| ``` | |
| ### Drug Safety Pipeline (runs in parallel with Q&A) | |
| ``` | |
| All uploaded documents | |
| ↓ | |
| Step 1 — Regex Extraction (no LLM, instantaneous) | |
| Scans for 20+ medication section header variants: | |
| MEDICATIONS, Current Medications, Active Meds, Home Meds, Rx, Drug List... | |
| Scans for 15+ allergy section header variants: | |
| ALLERGIES, Drug Allergies, Known Allergies, Sensitivities, Adverse Reactions... | |
| Also handles inline format: "Allergies: Penicillin, Sulfa" | |
| Deduplicates across all documents | |
| ↓ | |
| Step 2 — LLM Safety Reasoning (Qwen2.5-3B) | |
| Feeds clean extracted lists to the model | |
| Asks: "are there dangerous drug-allergy or drug-drug interactions?" | |
| Returns structured JSON: alerts with DANGER / WARNING / INFO severity | |
| ↓ | |
| Colour-coded alert panel in the UI | |
| ``` | |
| ### Inference Stack | |
| ``` | |
| HF Space (CPU) Modal Cloud (GPU) | |
| ───────────────── ────────────────────────────── | |
| gr.Server (Gradio 6.11) nvidia/cuda:12.4.0-devel image | |
| LangGraph agent llama.cpp built with CUDA | |
| Hybrid retriever Qwen2.5-3B-Instruct Q4_K_M GGUF | |
| Safety checker llama-server on A10G | |
| │ │ | |
| └──── HTTP POST ────────────────────┘ | |
| (httpx, timeout=300s) | |
| payload: {prompt, n_predict, temperature: 0.0} | |
| ``` | |
| --- | |
| ## Tech Stack | |
| | Component | Technology | | |
| |-----------|-----------| | |
| | Frontend | Custom HTML/CSS/JS via `gr.Server` (Gradio 6.11) | | |
| | Agent framework | LangGraph (StateGraph, 5 nodes) | | |
| | Retrieval | BM25 (rank_bm25) + Dense (sentence-transformers) | | |
| | Vector index | FAISS (faiss-cpu) | | |
| | LLM | Qwen2.5-3B-Instruct via llama.cpp Q4_K_M | | |
| | Inference server | llama-server (llama.cpp) on Modal A10G | | |
| | OCR | tesseract-ocr + pdf2image + pytesseract | | |
| | PDF parsing | pypdf | | |
| | Trace sharing | HuggingFace Hub API (huggingface_hub) | | |
| --- | |
| ## Why a 3B Model Works Here | |
| Clinical document extraction is a **retrieval + pattern matching** problem, not a general reasoning problem. The hard work is done by the hybrid retriever (finds the right chunks) and the two-step safety checker (regex extracts reliably, LLM only reasons on clean lists). The model's job is to synthesize well-retrieved context into a clean answer. | |
| Qwen2.5-3B-Instruct handles this well: | |
| - Instruction-tuned for structured output (JSON tables) | |
| - Fast — ~2 seconds on A10G at Q4_K_M quantization | |
| - Efficient — ~1.8GB on disk, ~$0.001 per query | |
| - Accurate enough for fact retrieval from clinical text | |
| --- | |
| ## Model | |
| [bartowski/Qwen2.5-3B-Instruct-GGUF](https://huggingface.co/bartowski/Qwen2.5-3B-Instruct-GGUF) — Q4_K_M quantization — Apache 2.0 license | |
| Base model: [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) | |
| --- | |
| ## Dataset | |
| Demo documents are synthetic clinical notes styled after MTSamples — a discharge summary, an intake note, and a cardiology consult for two fictional patients. They contain an intentional cross-document drug-allergy conflict (Aspirin prescribed in one, Aspirin allergy in another) to demonstrate the safety checker. | |
| Live agent traces from queries are published at [karthikmulugu08/cliniq-traces](https://huggingface.co/datasets/karthikmulugu08/cliniq-traces). | |
| --- | |
| ## Running Locally | |
| ```bash | |
| pip install -r requirements.txt | |
| python app.py | |
| # visit http://localhost:7860 | |
| ``` | |
| ## Deploying to Modal + HF Space | |
| ```bash | |
| # 1. Deploy the GPU inference server | |
| pip install modal | |
| modal setup | |
| modal deploy modal_inference.py | |
| # Copy the printed endpoint URL | |
| # 2. Set Space secrets | |
| # MODAL_ENDPOINT → URL from modal deploy | |
| # HF_TOKEN → your HF write token | |
| # HF_DATASET_REPO → karthikmulugu08/cliniq-traces | |
| ``` | |
| --- | |
| ## Submission Links | |
| - **Space:** https://huggingface.co/spaces/build-small-hackathon/cliniq | |
| - **Demo Video:** https://www.loom.com/share/d72ef56097194dd08fdaa734ff64d90f | |
| - **Social Post:** https://www.linkedin.com/posts/karthikmulugu_healthcareai-gradio-huggingface-share-7469807577778819072-dQx6/ | |
| - **Field Notes:** https://huggingface.co/blog/build-small-hackathon/cliniq-on-device-pharmacist |