--- title: ClinIQ โ€” Clinical Document Assistant emoji: ๐Ÿฉบ colorFrom: blue colorTo: green sdk: gradio sdk_version: "6.11.0" app_file: app.py python_version: "3.11" pinned: false license: apache-2.0 tags: - medical - rag - qwen - privacy - langgraph - llama-cpp - hackathon - build-small-2026 - tiny-titan --- # ClinIQ โ€” Privacy-First Clinical Document Assistant ClinIQ is an AI-powered clinical document assistant built for small medical clinics. Upload patient records โ€” discharge summaries, intake notes, consult reports, scanned documents โ€” and instantly get answers, structured extractions, and proactive drug safety alerts. **Model:** Qwen2.5-3B-Instruct (Q4_K_M) ยท **Inference:** llama.cpp on Modal A10G GPU ยท **Framework:** Gradio gr.Server + LangGraph --- ## The Problem Small clinics handle dozens of patients daily. Nurses upload a discharge summary from the hospital, an intake note from last week, and a specialist consult from yesterday โ€” all as separate files. Nobody has time to cross-reference all three. A medication prescribed in one document may conflict with an allergy documented in another. These conflicts get missed. ClinIQ solves this by reading all documents together and flagging dangerous combinations automatically โ€” before any question is asked. --- ## What ClinIQ Does ### 1. Proactive Drug Safety Check The moment documents are uploaded, ClinIQ automatically scans every medication and allergy across all files and flags dangerous combinations. No question needed. Example: a discharge summary prescribes Aspirin 81mg. An intake note documents an Aspirin allergy causing bronchospasm. These are two separate files from two different visits. ClinIQ catches the conflict instantly: ``` DANGER Aspirin allergy contraindication Patient prescribed Aspirin 81mg but has documented bronchospasm reaction Recommendation: Switch to Clopidogrel (noted in cardiology consult) ``` ### 2. Clinical Q&A Ask any question in plain English. ClinIQ retrieves relevant chunks from all uploaded documents and generates a precise, grounded answer in ~2 seconds. - *"Who is the attending physician?"* โ†’ direct fact lookup - *"What are the follow-up instructions?"* โ†’ extracts from discharge summary - *"Is it safe to give this patient aspirin?"* โ†’ reasons across multiple documents - *"What alternative was recommended instead of aspirin?"* โ†’ retrieves from consult note ### 3. Structured Extraction Ask for a list and get a formatted table โ€” not a paragraph of text. - *"List all medications"* โ†’ table with name, dose, frequency, route - *"What are all the allergies?"* โ†’ table with substance, reaction, severity - *"List all diagnoses"* โ†’ primary and secondary conditions - *"What are the vital signs?"* โ†’ structured vitals object ### 4. Cross-Document Reasoning ClinIQ retrieves context from all uploaded documents simultaneously. A question about drug interactions can pull the medication list from the discharge summary, the allergy list from the intake note, and the cardiology recommendation from the consult โ€” combining all three into a single coherent answer. ### 5. OCR Support Supports scanned PDFs and image files (JPG, PNG, TIFF). Tesseract OCR extracts text from scanned documents automatically before indexing. --- ## Architecture ### Document Ingestion Pipeline ``` Upload (PDF / TXT / JPG / PNG / TIFF / scanned PDF) โ†“ Text Extraction โ”œโ”€โ”€ Native PDF โ†’ pypdf (fast, text-selectable) โ”œโ”€โ”€ Scanned PDF โ†’ pdf2image + tesseract OCR โ”œโ”€โ”€ Image โ†’ tesseract OCR (upscaled to 1000px for accuracy) โ””โ”€โ”€ TXT / MD โ†’ direct read โ†“ Chunking โ€” 400 character chunks with 80 character overlap โ†“ Dual Indexing โ”œโ”€โ”€ BM25 index (rank_bm25) โ€” keyword matching โ””โ”€โ”€ Dense index (sentence-transformers/all-MiniLM-L6-v2) โ€” semantic matching ``` ### Query Pipeline โ€” LangGraph 5-Node Agent ``` User Question โ†“ [1] CLASSIFY Rule-based classification โ€” no LLM call Detects query type: simple / structured / complex / comparison โ†“ [2] DECOMPOSE For complex and comparison queries only Qwen breaks the question into 2-3 focused sub-queries โ†“ [3] RETRIEVE Runs each sub-query through hybrid retrieval: โ”œโ”€โ”€ BM25 score (keyword overlap) โ”œโ”€โ”€ Dense score (semantic similarity via cosine) โ””โ”€โ”€ Score fusion โ†’ top-6 unique chunks across all documents โ†“ [4] GENERATE Qwen2.5-3B-Instruct synthesizes an answer from retrieved chunks Structured queries โ†’ returns JSON parsed into a table Free-text queries โ†’ returns a concise clinical answer โ†“ [5] REFLECT Heuristic grounding check โ€” verifies answer overlaps with context Flags potential hallucinations without an extra LLM call โ†“ Streaming response โ†’ custom HTML/JS frontend (gr.Server) ``` ### Drug Safety Pipeline (runs in parallel with Q&A) ``` All uploaded documents โ†“ Step 1 โ€” Regex Extraction (no LLM, instantaneous) Scans for 20+ medication section header variants: MEDICATIONS, Current Medications, Active Meds, Home Meds, Rx, Drug List... Scans for 15+ allergy section header variants: ALLERGIES, Drug Allergies, Known Allergies, Sensitivities, Adverse Reactions... Also handles inline format: "Allergies: Penicillin, Sulfa" Deduplicates across all documents โ†“ Step 2 โ€” LLM Safety Reasoning (Qwen2.5-3B) Feeds clean extracted lists to the model Asks: "are there dangerous drug-allergy or drug-drug interactions?" Returns structured JSON: alerts with DANGER / WARNING / INFO severity โ†“ Colour-coded alert panel in the UI ``` ### Inference Stack ``` HF Space (CPU) Modal Cloud (GPU) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ gr.Server (Gradio 6.11) nvidia/cuda:12.4.0-devel image LangGraph agent llama.cpp built with CUDA Hybrid retriever Qwen2.5-3B-Instruct Q4_K_M GGUF Safety checker llama-server on A10G โ”‚ โ”‚ โ””โ”€โ”€โ”€โ”€ HTTP POST โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ (httpx, timeout=300s) payload: {prompt, n_predict, temperature: 0.0} ``` --- ## Tech Stack | Component | Technology | |-----------|-----------| | Frontend | Custom HTML/CSS/JS via `gr.Server` (Gradio 6.11) | | Agent framework | LangGraph (StateGraph, 5 nodes) | | Retrieval | BM25 (rank_bm25) + Dense (sentence-transformers) | | Vector index | FAISS (faiss-cpu) | | LLM | Qwen2.5-3B-Instruct via llama.cpp Q4_K_M | | Inference server | llama-server (llama.cpp) on Modal A10G | | OCR | tesseract-ocr + pdf2image + pytesseract | | PDF parsing | pypdf | | Trace sharing | HuggingFace Hub API (huggingface_hub) | --- ## Why a 3B Model Works Here Clinical document extraction is a **retrieval + pattern matching** problem, not a general reasoning problem. The hard work is done by the hybrid retriever (finds the right chunks) and the two-step safety checker (regex extracts reliably, LLM only reasons on clean lists). The model's job is to synthesize well-retrieved context into a clean answer. Qwen2.5-3B-Instruct handles this well: - Instruction-tuned for structured output (JSON tables) - Fast โ€” ~2 seconds on A10G at Q4_K_M quantization - Efficient โ€” ~1.8GB on disk, ~$0.001 per query - Accurate enough for fact retrieval from clinical text --- ## Model [bartowski/Qwen2.5-3B-Instruct-GGUF](https://huggingface.co/bartowski/Qwen2.5-3B-Instruct-GGUF) โ€” Q4_K_M quantization โ€” Apache 2.0 license Base model: [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) --- ## Dataset Demo documents are synthetic clinical notes styled after MTSamples โ€” a discharge summary, an intake note, and a cardiology consult for two fictional patients. They contain an intentional cross-document drug-allergy conflict (Aspirin prescribed in one, Aspirin allergy in another) to demonstrate the safety checker. Live agent traces from queries are published at [karthikmulugu08/cliniq-traces](https://huggingface.co/datasets/karthikmulugu08/cliniq-traces). --- ## Running Locally ```bash pip install -r requirements.txt python app.py # visit http://localhost:7860 ``` ## Deploying to Modal + HF Space ```bash # 1. Deploy the GPU inference server pip install modal modal setup modal deploy modal_inference.py # Copy the printed endpoint URL # 2. Set Space secrets # MODAL_ENDPOINT โ†’ URL from modal deploy # HF_TOKEN โ†’ your HF write token # HF_DATASET_REPO โ†’ karthikmulugu08/cliniq-traces ``` --- ## Submission Links - **Space:** https://huggingface.co/spaces/build-small-hackathon/cliniq - **Demo Video:** https://www.loom.com/share/d72ef56097194dd08fdaa734ff64d90f - **Social Post:** https://www.linkedin.com/posts/karthikmulugu_healthcareai-gradio-huggingface-share-7469807577778819072-dQx6/ - **Field Notes:** https://huggingface.co/blog/build-small-hackathon/cliniq-on-device-pharmacist