README.md

metadata

title: Chest X-ray Recommender
emoji: 🩻
colorFrom: blue
colorTo: indigo
sdk: gradio
app_file: app.py
pinned: false
license: mit
short_description: Visual chest X-ray recommender powered by CLIP

🩻 Chest X-ray Recommender

🚀 Live demo: https://huggingface.co/spaces/Matanech/CXR_Similarity_Engine

A visual recommendation engine for chest X-rays, built with CLIP embeddings and Gradio. Upload an X-ray or describe a finding in words and the app returns 3–5 of the most visually similar studies from a pre-computed catalog drawn from MLforHealthcare/mimic-cxr.

⚠️ Educational demo only. Not a medical device. Do not use for clinical decisions.

How it works

1. The companion notebook (Assignment_3_MIMIC_CXR_Recommender_v4.ipynb) sub-samples 2,000 chest X-rays from MIMIC-CXR.
2. Each image is embedded with openai/clip-vit-base-patch32 into a 512-dimensional vector and L2-normalised.
3. Embeddings, base64 thumbnails, KMeans cluster labels, and the original radiology reports are saved together to embeddings.parquet.
4. This Space loads that single parquet on startup. At query time, the user's text or image is encoded with the same CLIP model and the catalog is ranked by cosine similarity.
5. The app returns the top-3 matches plus up to 2 extra matches (5 total) when consecutive scores differ by ≤ 0.02, giving the user "second opinions" when the model is uncertain.

Files in this Space

File	Purpose
app.py	Gradio interface + recommendation logic
requirements.txt	Python dependencies
embeddings.parquet	Pre-computed catalog (built by the notebook)
Assignment_3_MIMIC_CXR_Recommender_v4.ipynb	Companion notebook (EDA, embeddings, clustering, app)
README.md	This file (also drives the Space card)

Running locally

pip install -r requirements.txt
python app.py

The app launches at http://127.0.0.1:7860.

Configuration

A few environment variables tweak the behaviour without code changes (set them in your Space under Settings → Variables and secrets):

Variable	Default	Description
CLIP_MODEL_ID	openai/clip-vit-base-patch32	Any HF CLIP model. Try flaviagiammarino/pubmed-clip-vit-base-patch32 for medical fine-tuning.
EMBEDDINGS_FILE	embeddings.parquet	Path to the catalog file.
K_MIN	3	Minimum number of recommendations.
K_MAX	5	Maximum number of recommendations (if score gaps are tight).
GAP_THRESHOLD	0.02	Cosine-similarity gap below which to include extra matches.
VIDEO_EMBED_ID	(empty)	YouTube video ID for the walk-through. When set, the Space adds an embed at the bottom.

Acknowledgements

• Dataset: MLforHealthcare/mimic-cxr
• Embedding model: openai/clip-vit-base-patch32
• Course: Data Science Assignment 3 — Embeddings, RecSys, Spaces