---
title: Chest X-ray Recommender
emoji: 🩻
colorFrom: blue
colorTo: indigo
sdk: gradio
app_file: app.py
pinned: false
license: mit
short_description: Visual chest X-ray recommender powered by CLIP
---

# 🩻 Chest X-ray Recommender

**🚀 Live demo:** <https://huggingface.co/spaces/Matanech/CXR_Similarity_Engine>

A visual recommendation engine for chest X-rays, built with CLIP embeddings
and Gradio. Upload an X-ray **or** describe a finding in words and the app
returns 3–5 of the most visually similar studies from a pre-computed catalog
drawn from [`MLforHealthcare/mimic-cxr`](https://huggingface.co/datasets/MLforHealthcare/mimic-cxr).

> ⚠️ **Educational demo only. Not a medical device. Do not use for clinical decisions.**

## How it works

1. The companion notebook (`Assignment_3_MIMIC_CXR_Recommender_v4.ipynb`) sub-samples
   2,000 chest X-rays from MIMIC-CXR.
2. Each image is embedded with
   [`openai/clip-vit-base-patch32`](https://huggingface.co/openai/clip-vit-base-patch32)
   into a 512-dimensional vector and L2-normalised.
3. Embeddings, base64 thumbnails, KMeans cluster labels, and the original
   radiology reports are saved together to `embeddings.parquet`.
4. This Space loads that single parquet on startup. At query time, the user's
   text or image is encoded with the same CLIP model and the catalog is ranked
   by cosine similarity.
5. The app returns the **top-3** matches plus up to **2 extra** matches (5 total)
   when consecutive scores differ by ≤ 0.02, giving the user "second opinions"
   when the model is uncertain.

## Files in this Space

| File                                          | Purpose |
|-----------------------------------------------|---------|
| `app.py`                                      | Gradio interface + recommendation logic |
| `requirements.txt`                            | Python dependencies |
| `embeddings.parquet`                          | Pre-computed catalog (built by the notebook) |
| `Assignment_3_MIMIC_CXR_Recommender_v4.ipynb` | Companion notebook (EDA, embeddings, clustering, app) |
| `README.md`                                   | This file (also drives the Space card) |

## Running locally

```bash
pip install -r requirements.txt
python app.py
```

The app launches at <http://127.0.0.1:7860>.

## Configuration

A few environment variables tweak the behaviour without code changes (set them
in your Space under **Settings → Variables and secrets**):

| Variable          | Default                            | Description |
|-------------------|------------------------------------|-------------|
| `CLIP_MODEL_ID`   | `openai/clip-vit-base-patch32`     | Any HF CLIP model. Try `flaviagiammarino/pubmed-clip-vit-base-patch32` for medical fine-tuning. |
| `EMBEDDINGS_FILE` | `embeddings.parquet`               | Path to the catalog file. |
| `K_MIN`           | `3`                                | Minimum number of recommendations. |
| `K_MAX`           | `5`                                | Maximum number of recommendations (if score gaps are tight). |
| `GAP_THRESHOLD`   | `0.02`                             | Cosine-similarity gap below which to include extra matches. |
| `VIDEO_EMBED_ID`  | (empty)                            | YouTube video ID for the walk-through. When set, the Space adds an embed at the bottom. |

## Acknowledgements

- Dataset: [MLforHealthcare/mimic-cxr](https://huggingface.co/datasets/MLforHealthcare/mimic-cxr)
- Embedding model: [openai/clip-vit-base-patch32](https://huggingface.co/openai/clip-vit-base-patch32)
- Course: Data Science Assignment 3 — Embeddings, RecSys, Spaces