README.md

---
title: ClinIQ — Clinical Document Assistant
emoji: 🩺
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: "6.11.0"
app_file: app.py
python_version: "3.11"
pinned: false
license: apache-2.0
tags:
  - medical
  - rag
  - qwen
  - privacy
  - langgraph
  - llama-cpp
  - hackathon
  - build-small-2026
  - tiny-titan
---

# ClinIQ — Privacy-First Clinical Document Assistant

ClinIQ is an AI-powered clinical document assistant built for small medical clinics. Upload patient records — discharge summaries, intake notes, consult reports, scanned documents — and instantly get answers, structured extractions, and proactive drug safety alerts.

**Model:** Qwen2.5-3B-Instruct (Q4_K_M) · **Inference:** llama.cpp on Modal A10G GPU · **Framework:** Gradio gr.Server + LangGraph

---

## The Problem

Small clinics handle dozens of patients daily. Nurses upload a discharge summary from the hospital, an intake note from last week, and a specialist consult from yesterday — all as separate files. Nobody has time to cross-reference all three. A medication prescribed in one document may conflict with an allergy documented in another. These conflicts get missed.

ClinIQ solves this by reading all documents together and flagging dangerous combinations automatically — before any question is asked.

---

## What ClinIQ Does

### 1. Proactive Drug Safety Check
The moment documents are uploaded, ClinIQ automatically scans every medication and allergy across all files and flags dangerous combinations. No question needed.

Example: a discharge summary prescribes Aspirin 81mg. An intake note documents an Aspirin allergy causing bronchospasm. These are two separate files from two different visits. ClinIQ catches the conflict instantly:

```
DANGER  Aspirin allergy contraindication
        Patient prescribed Aspirin 81mg but has documented bronchospasm reaction
        Recommendation: Switch to Clopidogrel (noted in cardiology consult)
```

### 2. Clinical Q&A
Ask any question in plain English. ClinIQ retrieves relevant chunks from all uploaded documents and generates a precise, grounded answer in ~2 seconds.

- *"Who is the attending physician?"* → direct fact lookup
- *"What are the follow-up instructions?"* → extracts from discharge summary
- *"Is it safe to give this patient aspirin?"* → reasons across multiple documents
- *"What alternative was recommended instead of aspirin?"* → retrieves from consult note

### 3. Structured Extraction
Ask for a list and get a formatted table — not a paragraph of text.

- *"List all medications"* → table with name, dose, frequency, route
- *"What are all the allergies?"* → table with substance, reaction, severity
- *"List all diagnoses"* → primary and secondary conditions
- *"What are the vital signs?"* → structured vitals object

### 4. Cross-Document Reasoning
ClinIQ retrieves context from all uploaded documents simultaneously. A question about drug interactions can pull the medication list from the discharge summary, the allergy list from the intake note, and the cardiology recommendation from the consult — combining all three into a single coherent answer.

### 5. OCR Support
Supports scanned PDFs and image files (JPG, PNG, TIFF). Tesseract OCR extracts text from scanned documents automatically before indexing.

---

## Architecture

### Document Ingestion Pipeline

```
Upload (PDF / TXT / JPG / PNG / TIFF / scanned PDF)
        ↓
Text Extraction
  ├── Native PDF  → pypdf (fast, text-selectable)
  ├── Scanned PDF → pdf2image + tesseract OCR
  ├── Image       → tesseract OCR (upscaled to 1000px for accuracy)
  └── TXT / MD    → direct read
        ↓
Chunking — 400 character chunks with 80 character overlap
        ↓
Dual Indexing
  ├── BM25 index  (rank_bm25) — keyword matching
  └── Dense index (sentence-transformers/all-MiniLM-L6-v2) — semantic matching
```

### Query Pipeline — LangGraph 5-Node Agent

```
User Question
      ↓
  [1] CLASSIFY
      Rule-based classification — no LLM call
      Detects query type: simple / structured / complex / comparison
      ↓
  [2] DECOMPOSE
      For complex and comparison queries only
      Qwen breaks the question into 2-3 focused sub-queries
      ↓
  [3] RETRIEVE
      Runs each sub-query through hybrid retrieval:
      ├── BM25 score (keyword overlap)
      ├── Dense score (semantic similarity via cosine)
      └── Score fusion → top-6 unique chunks across all documents
      ↓
  [4] GENERATE
      Qwen2.5-3B-Instruct synthesizes an answer from retrieved chunks
      Structured queries → returns JSON parsed into a table
      Free-text queries → returns a concise clinical answer
      ↓
  [5] REFLECT
      Heuristic grounding check — verifies answer overlaps with context
      Flags potential hallucinations without an extra LLM call
      ↓
Streaming response → custom HTML/JS frontend (gr.Server)
```

### Drug Safety Pipeline (runs in parallel with Q&A)

```
All uploaded documents
        ↓
Step 1 — Regex Extraction (no LLM, instantaneous)
  Scans for 20+ medication section header variants:
  MEDICATIONS, Current Medications, Active Meds, Home Meds, Rx, Drug List...
  Scans for 15+ allergy section header variants:
  ALLERGIES, Drug Allergies, Known Allergies, Sensitivities, Adverse Reactions...
  Also handles inline format: "Allergies: Penicillin, Sulfa"
  Deduplicates across all documents
        ↓
Step 2 — LLM Safety Reasoning (Qwen2.5-3B)
  Feeds clean extracted lists to the model
  Asks: "are there dangerous drug-allergy or drug-drug interactions?"
  Returns structured JSON: alerts with DANGER / WARNING / INFO severity
        ↓
Colour-coded alert panel in the UI
```

### Inference Stack

```
HF Space (CPU)                    Modal Cloud (GPU)
─────────────────                 ──────────────────────────────
gr.Server (Gradio 6.11)          nvidia/cuda:12.4.0-devel image
LangGraph agent                   llama.cpp built with CUDA
Hybrid retriever                  Qwen2.5-3B-Instruct Q4_K_M GGUF
Safety checker                    llama-server on A10G
        │                                   │
        └──── HTTP POST ────────────────────┘
              (httpx, timeout=300s)
              payload: {prompt, n_predict, temperature: 0.0}
```

---

## Tech Stack

| Component | Technology |
|-----------|-----------|
| Frontend | Custom HTML/CSS/JS via `gr.Server` (Gradio 6.11) |
| Agent framework | LangGraph (StateGraph, 5 nodes) |
| Retrieval | BM25 (rank_bm25) + Dense (sentence-transformers) |
| Vector index | FAISS (faiss-cpu) |
| LLM | Qwen2.5-3B-Instruct via llama.cpp Q4_K_M |
| Inference server | llama-server (llama.cpp) on Modal A10G |
| OCR | tesseract-ocr + pdf2image + pytesseract |
| PDF parsing | pypdf |
| Trace sharing | HuggingFace Hub API (huggingface_hub) |

---

## Why a 3B Model Works Here

Clinical document extraction is a **retrieval + pattern matching** problem, not a general reasoning problem. The hard work is done by the hybrid retriever (finds the right chunks) and the two-step safety checker (regex extracts reliably, LLM only reasons on clean lists). The model's job is to synthesize well-retrieved context into a clean answer.

Qwen2.5-3B-Instruct handles this well:
- Instruction-tuned for structured output (JSON tables)
- Fast — ~2 seconds on A10G at Q4_K_M quantization
- Efficient — ~1.8GB on disk, ~$0.001 per query
- Accurate enough for fact retrieval from clinical text

---

## Model

[bartowski/Qwen2.5-3B-Instruct-GGUF](https://huggingface.co/bartowski/Qwen2.5-3B-Instruct-GGUF) — Q4_K_M quantization — Apache 2.0 license

Base model: [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)

---

## Dataset

Demo documents are synthetic clinical notes styled after MTSamples — a discharge summary, an intake note, and a cardiology consult for two fictional patients. They contain an intentional cross-document drug-allergy conflict (Aspirin prescribed in one, Aspirin allergy in another) to demonstrate the safety checker.

Live agent traces from queries are published at [karthikmulugu08/cliniq-traces](https://huggingface.co/datasets/karthikmulugu08/cliniq-traces).

---

## Running Locally

```bash
pip install -r requirements.txt
python app.py
# visit http://localhost:7860
```

## Deploying to Modal + HF Space

```bash
# 1. Deploy the GPU inference server
pip install modal
modal setup
modal deploy modal_inference.py
# Copy the printed endpoint URL

# 2. Set Space secrets
# MODAL_ENDPOINT → URL from modal deploy
# HF_TOKEN       → your HF write token
# HF_DATASET_REPO → karthikmulugu08/cliniq-traces
```
---

## Submission Links

- **Space:** https://huggingface.co/spaces/build-small-hackathon/cliniq
- **Demo Video:** https://www.loom.com/share/d72ef56097194dd08fdaa734ff64d90f
- **Social Post:** https://www.linkedin.com/posts/karthikmulugu_healthcareai-gradio-huggingface-share-7469807577778819072-dQx6/
- **Field Notes:** https://huggingface.co/blog/build-small-hackathon/cliniq-on-device-pharmacist