README.md

# CardioSyntax v2: SYNTAX Score Prediction from Coronary Angiography

This repository contains code and pre-trained models for automated SYNTAX score prediction from coronary angiography videos. The SYNTAX score is a validated metric for quantifying the anatomical complexity of coronary artery disease.

## Overview

**CardioSyntax v2** is a large-scale dataset of 11,410 angiography studies with SYNTAX scores, designed for developing machine learning models for automated coronary lesion complexity assessment. This repository provides:

- **Dataset**: 11,209 training studies (Philips Allura Clarity + Azurion 7) + 120 test studies (100 Philips + 20 Siemens) annotated by 6 independent interventional cardiologists
- **Baseline Models**: 
  - Backbone network (R3D-18): single projection → SYNTAX contribution
  - RNN head (LSTM/GRU/Transformer): multi-view aggregation → final SYNTAX score
- **Pre-trained Weights**: 5-fold ensemble with post-calibration
- **Evaluation**: Pearson correlation r ≈ 0.81, balanced accuracy ≈ 82% for SYNTAX > 22 classification

The dataset addresses inter-observer variability through multi-expert annotation and enables systematic investigation of domain shift effects across imaging systems.

## Repository Structure

```
coronary-syntax-prediction/
├── backbone/                          # Single-frame backbone (3D ResNet)
│   ├── dataset.py                    # SyntaxDataset for single-view training
│   ├── pl_model.py                   # SyntaxLightningModule (Lightning wrapper)
│   ├── pl_train.py                   # Training script for backbone
│   └── __init__.py
├── full_model/                        # Multi-view RNN head + backbone
│   ├── rnn_dataset.py                # SyntaxDataset for multi-view (RNN) training
│   ├── rnn_model.py                  # SyntaxLightningModule with RNN variants
│   ├── rnn_train.py                  # Training script for full model
│   └── __init__.py
├── inference/                         # Inference & evaluation
│   ├── rnn_apply.py                  # Ensemble inference script
│   ├── metrics_visualization.py       # Plotly-based metrics & visualization
│   └── __init__.py
├── backbone_weights/                 # Backbone .pt checkpoints (5 folds × 2 arteries)
├── full_model_weights/               # Full model .pt checkpoints (5 folds × 2 arteries × variants)
├── scaling_coeffs/                   # Calibration coefficients (a, b) per fold
├── requirements.txt
└── README.md
```

## Installation

```bash
# Clone the repository
git clone https://huggingface.co/MesserMMP/coronary-syntax-prediction
cd coronary-syntax-prediction

# Install dependencies
pip install -r requirements.txt
```

### Requirements

Key packages:
- PyTorch 2.0+ with CUDA support
- Lightning 2.x
- torchvision, pytorchvideo
- numpy, scikit-learn, pandas
- plotly for visualization
- click for CLI

See `requirements.txt` for full list.

## Training

### Stage 1: Backbone Training (Single-View)

Train 3D ResNet-18 on single angiographic projections (left/right coronary artery separately):

```bash
python backbone/pl_train.py \
  -r /path/to/dataset_root \
  --fold 0 \
  -a right \
  --num-classes 2 \
  -b 50 \
  -f 32 \
  -v 256 256 \
  --max-epochs 10 \
  --num-workers 8 \
  --devices 0 \
  --precision bf16-mixed \
  --logdir ./logs/backbone \
  --use-weighted-sampler
```

**Parameters:**
- `-r, --dataset-root`: Path to dataset root (default: `.`)
- `--fold`: Fold number (0-4, default: 4)
- `-a, --artery`: `left` or `right` (default: `right`)
- `-nc, --num-classes`: Output units: 2 for (classification, regression) (default: 2)
- `-b, --batch-size`: Batch size (default: 50)
- `-f, --frames-per-clip`: Frames per clip (default: 32)
- `-v, --video-size`: Frame resolution H W (default: 256 256)
- `--max-epochs`: Epochs for full training (default: 10)
- `--num-workers`: DataLoader workers (default: 8)
- `--devices`: GPU device IDs (default: )
- `--precision`: Training precision mode (default: `bf16-mixed`)
- `--logdir`: Log directory (default: `./logs/backbone`)
- `--use-weighted-sampler`: Balance classes by score intervals (flag)
- `--seed`: Random seed (default: 42)

**Output:** TensorBoard logs + `.ckpt` checkpoints in `--logdir`

### Stage 2: RNN Head Training (Multi-View)

Train LSTM/GRU/Transformer aggregation head on multi-view studies:

```bash
python full_model/rnn_train.py \
  -r /path/to/dataset_root \
  --fold 0 \
  -a right \
  --variant lstm_mean \
  --num-classes 2 \
  -b 8 \
  -f 32 \
  -v 256 256 \
  --max-epochs 15 \
  --num-workers 16 \
  --devices 0 \
  --precision bf16-mixed \
  --logdir ./logs/rnn \
  --backbone-pt-dir backbone_weights \
  --backbone-from-pt \
  --rnn-folds-dir rnn_folds
```

**Key Parameters:**
- `--variant`: Head architecture:
  - `mean_out`: Mean of projection scores (no RNN)
  - `mean`: MLP on mean pooled backbone features
  - `lstm_mean`: LSTM with mean pooling of hidden states
  - `lstm_last`: LSTM, use last hidden state
  - `gru_mean`, `gru_last`: GRU variants
  - `bert_mean`, `bert_cls`, `bert_cls2`: Transformer encoder variants
  - (default: `lstm_mean`)
- `--backbone-pt-dir`: Path to .pt backbone weights (default: `backbone_weights`)
- `--backbone-from-pt`: Load backbone from `.pt` files (flag, default: True)
- `--backbone-logdir`: Alternative: load backbone from Lightning logs
- `--rnn-folds-dir`: Directory with `rnn_fold{fold}_train.json`, `rnn_fold{fold}_eval.json` (default: `rnn_folds`)
- `--use-weighted-sampler`: Balance by score (flag)
- `--pt-weights-format`: If loading pre-trained head, expect `.pt` instead of `.ckpt` (flag)

**Output:** Full model checkpoints in `--logdir/{artery}BinSyntax_R3D_fold{fold:02d}_{variant}_post/...`

## Inference

Run ensemble inference on test datasets:

```bash
python inference/rnn_apply.py \
  -d "test_philips_100.json" "test_siemens_20.json" \
  -n "Philips Test" "Siemens Test" \
  -p "philips_100" "siemens_20" \
  -r /path/to/dataset_root \
  --model-dir full_model_weights \
  -v 256 256 \
  --frames-per-clip 32 \
  --num-workers 8 \
  --variant lstm_mean \
  --pt-weights-format \
  --use-scaling \
  --scaling-file scaling_coeffs.json \
  -e "Ensemble_lstm_mean" \
  -m metrics.json
```

**Parameters:**
- `-d, --dataset-paths`: Multiple JSON test dataset paths (relative to `--dataset-root`)
- `-n, --dataset-names`: Display names for each dataset (must match `-d` count)
- `-p, --postfixes`: Suffixes for result files (must match `-d` count)
- `-r, --dataset-root`: Dataset root (default: `.`)
- `--model-dir`: Directory with full model `.pt` weights (default: `full_model_weights`)
- `-v, --video-size`: Frame resolution H W (default: 256 256)
- `--frames-per-clip`: Frames per clip (default: 32)
- `--num-workers`: DataLoader workers (default: 8)
- `--variant`: Head model variant (default: `lstm_mean`)
- `--pt-weights-format`: Model weights are `.pt` raw state_dict (flag, default: True)
- `--use-scaling`: Apply post-calibration scaling a*x+b from JSON (flag)
- `--scaling-file`: Calibration coefficients file relative to `--dataset-root`
- `-e, --ensemble-name`: Experiment name for metrics (e.g., "Ensemble_lstm_mean")
- `-m, --metrics-file`: JSON output file path for all metrics
- `--seed`: Random seed (default: 42)

**Output:**
- Per-dataset predictions: `results/{postfix}.json`
- Ensemble metrics: `{metrics_file}` with Pearson correlation, balanced accuracy, per-fold statistics
- Plotly HTML plots: `visualizations/{postfix}.html`

## Dataset

**Link:** [https://huggingface.co/datasets/MesserMMP/coronary-angiography-syntax](https://huggingface.co/datasets/MesserMMP/coronary-angiography-syntax)

### Data Format

The dataset provides two different JSON annotation formats depending on the training stage:

#### Format 1: Backbone Training (Single-View/Per-Video)

Used for training the backbone model (`backbone/` scripts). Each record represents **one angiographic video projection**.

**File structure:** `folds/fold{fold_id}_{split}.json`

```json
[
  {
    "study_uid": "1.3.46.670589.28.26690171363123020190823114413193175",
    "series_uid": "1.3.46.670589.28.26690171363123020190823115102913895.2.2",
    "sop_uid": "1.3.46.670589.28.266901713631230201908231158145991852211512",
    "path": "../anon_data/chunk1/.../IM-2254-0039.dcm",
    "shape": [38, 512, 512],
    "artery": 0,
    "artery_prob": 1.9078343029832467e-05,
    "syntax": 0.0,
    "syntax_left": 0.0,
    "syntax_right": 0.0,
    "bin_syntax": 0,
    "manufacturer": "Philips",
    "device_model": "Allura Clarity"
  },
  {
    "study_uid": "1.3.46.670589.28.26690171363123020190823114413193175",
    "series_uid": "1.3.46.670589.28.26690171363123020190823115648342907.2.2",
    "sop_uid": "1.3.46.670589.28.266901713631230201908231158148282212211512",
    "path": "../anon_data/chunk1/.../IM-2260-0022.dcm",
    "shape": [15, 512, 512],
    "artery": 1,
    "artery_prob": 0.9559882879257202,
    "syntax": 0.0,
    "syntax_left": 0.0,
    "syntax_right": 0.0,
    "bin_syntax": 0,
    "manufacturer": "Philips",
    "device_model": "Allura Clarity"
  }
]
```

**Field descriptions:**

| Field | Type | Description |
|-------|------|-------------|
| `study_uid` | str | Unique study identifier (DICOM) |
| `series_uid` | str | Unique series identifier (DICOM) |
| `sop_uid` | str | Unique SOP instance UID (DICOM) |
| `path` | str | Relative path to DICOM video file |
| `shape` | list[int] | Video dimensions: [T, H, W] (frames, height, width) |
| `artery` | int | Coronary artery: 0 = left (LCA), 1 = right (RCA) |
| `artery_prob` | float | Model confidence for artery classification (0–1) |
| `syntax` | float | Total SYNTAX score (LCA + RCA) |
| `syntax_left` | float | SYNTAX score for left coronary artery |
| `syntax_right` | float | SYNTAX score for right coronary artery |
| `bin_syntax` | int | Binary classification: 0 = SYNTAX < threshold, 1 = SYNTAX ≥ threshold |
| `manufacturer` | str | Imaging equipment manufacturer (e.g., "Philips", "Siemens") |
| `device_model` | str | Equipment model (e.g., "Allura Clarity", "AXIOM-Artis") |

***

#### Format 2: RNN Head Training (Multi-View/Per-Patient)

Used for training the full RNN model (`full_model/` scripts). Each record represents **one patient study** with all angiographic projections grouped by artery.

**File structure:** `rnn_folds/rnn_fold{fold_id}_{split}.json`

```json
[
  {
    "study_uid": "1.3.46.670589.28.26690171363123020190823114413193175",
    "manufacturer": "Philips",
    "device_model": "Allura Clarity",
    "syntax": 0.0,
    "syntax_left": 0.0,
    "syntax_right": 0.0,
    "bin_syntax": 0,
    "videos": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115102913895.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158145991852211512",
        "path": "../anon_data/chunk1/.../IM-2254-0039.dcm",
        "shape": [38, 512, 512],
        "artery": 0,
        "artery_prob": 1.9078343029832467e-05
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115648342907.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158148282212211512",
        "path": "../anon_data/chunk1/.../IM-2260-0022.dcm",
        "shape": [15, 512, 512],
        "artery": 1,
        "artery_prob": 0.9559882879257202
      }
    ],
    "videos_left": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115102913895.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158145991852211512",
        "path": "../anon_data/chunk1/.../IM-2254-0039.dcm",
        "shape": [38, 512, 512],
        "artery": 0,
        "artery_prob": 1.9078343029832467e-05
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115533239902.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158147562092211512",
        "path": "../anon_data/chunk1/.../IM-2258-0055.dcm",
        "shape": [50, 512, 512],
        "artery": 0,
        "artery_prob": 0.0003698925720527768
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115052517893.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158145581792211512",
        "path": "../anon_data/chunk1/.../IM-2253-0047.dcm",
        "shape": [47, 512, 512],
        "artery": 0,
        "artery_prob": 0.00018035581160802394
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115550658904.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158147942152211512",
        "path": "../anon_data/chunk1/.../IM-2259-0047.dcm",
        "shape": [41, 512, 512],
        "artery": 0,
        "artery_prob": 0.00022948876721784472
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115523054900.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158147162032211512",
        "path": "../anon_data/chunk1/.../IM-2257-0047.dcm",
        "shape": [43, 512, 512],
        "artery": 0,
        "artery_prob": 0.00034276593942195177
      }
    ],
    "videos_right": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115648342907.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158148282212211512",
        "path": "../anon_data/chunk1/.../IM-2260-0022.dcm",
        "shape": [15, 512, 512],
        "artery": 1,
        "artery_prob": 0.9559882879257202
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115657547909.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158149262272211512",
        "path": "../anon_data/chunk1/.../IM-2261-0070.dcm",
        "shape": [62, 512, 512],
        "artery": 1,
        "artery_prob": 0.9999938011169434
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115712699912.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158149652332211512",
        "path": "../anon_data/chunk1/.../IM-2262-0063.dcm",
        "shape": [54, 512, 512],
        "artery": 1,
        "artery_prob": 0.999855637550354
      }
    ],
    "videos_other": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115128658897.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158146391912211512",
        "path": "../anon_data/chunk1/.../IM-2255-0035.dcm",
        "shape": [33, 512, 512],
        "artery": 0,
        "artery_prob": 0.2866898477077484
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115132067898.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158146771972211512",
        "path": "../anon_data/chunk1/.../IM-2256-0035.dcm",
        "shape": [32, 512, 512],
        "artery": 0,
        "artery_prob": 0.3251875042915344
      }
    ]
  }
]
```

**Field descriptions:**

| Field | Type | Description |
|-------|------|-------------|
| `study_uid` | str | Unique study identifier (same across all videos for this patient) |
| `manufacturer` | str | Imaging equipment manufacturer |
| `device_model` | str | Equipment model |
| `syntax` | float | Total SYNTAX score (LCA + RCA) |
| `syntax_left` | float | SYNTAX score for left coronary artery |
| `syntax_right` | float | SYNTAX score for right coronary artery |
| `bin_syntax` | int | Binary classification label |
| `videos` | list[object] | All video projections (unfiltered) |
| `videos_left` | list[object] | Projections classified as left coronary (artery=0, high confidence) |
| `videos_right` | list[object] | Projections classified as right coronary (artery=1, high confidence) |
| `videos_other` | list[object] | Low-confidence projections (excluded from training); typically ~5–7% of total |

**Video object fields:**

| Field | Type | Description |
|-------|------|-------------|
| `series_uid` | str | Unique series identifier (DICOM) |
| `sop_uid` | str | Unique SOP instance UID (DICOM) |
| `path` | str | Relative path to DICOM video file |
| `shape` | list[int] | Video dimensions: [T, H, W] |
| `artery` | int | Artery classification: 0 = left, 1 = right |
| `artery_prob` | float | Model confidence (0–1); threshold typically 0.5 for `videos_left`/`videos_right` inclusion |

***

#### Test Set Format (Multi-Expert Annotations)

Test data (`test_philips_100.json`, `test_siemens_20.json`) follows Format 2 with additional expert annotations:

```json
[
  {
    "study_uid": "...",
    "manufacturer": "Philips",
    "device_model": "Allura Clarity",
    "syntax": 18.5,
    "syntax_left": 10.0,
    "syntax_right": 8.5,
    "mean_syntax": 18.5,
    "expert0": 18,
    "expert1": 19,
    "expert2": 17,
    "expert3": 19,
    "expert4": 20,
    "expert5": 18,
    "videos_left": [...],
    "videos_right": [...],
    "videos_other": [...]
  }
]
```

**Additional fields for test data:**

| Field | Type | Description |
|-------|------|-------------|
| `mean_syntax` | float | Consensus SYNTAX score (mean of 6 experts) |
| `expert0`–`expert5` | float | Individual expert SYNTAX score (6 independent cardiologists) |

**Note:** `expert0` typically represents aggregated/consensus annotations, while `expert1`–`expert5` are individual readers.

***

#### DICOM Video Format

Each DICOM file contains:

- **Dimensions:** [T, H, W] = [frames, 512, 512] at 15 fps
- **Data type:** `uint8` (8-bit grayscale)
- **Frame count:** Typically 15–60 frames per projection
- **Resolution:** 512×512 pixels (standard for X-ray angiography)

***

### Data Splits

| Set | Files | Size | Equipment | Artery Classification |
|-----|-------|------|-----------|----------------------|
| **Backbone training** | `folds/fold{0-4}_train.json` | ~8,000 videos | Philips | Single video |
| **Backbone validation** | `folds/fold{0-4}_eval.json` | ~1,000 videos | Philips | Single video |
| **RNN training** | `rnn_folds/rnn_fold{0-4}_train.json` | ~2,200 studies | Philips | Multi-view (patient) |
| **RNN validation** | `rnn_folds/rnn_fold{0-4}_eval.json` | ~290 studies | Philips | Multi-view (patient) |
| **Test 1 (Philips)** | `test_philips_100.json` | 100 studies | Philips Allura Clarity | 6 experts |
| **Test 2 (Siemens)** | `test_siemens_20.json` | 20 studies | Siemens AXIOM-Artis | 6 experts |

**SYNTAX > 0 prevalence:** 39.26% in training data

## Results

### Baseline Model Performance (5-Fold Ensemble)

| Metric | Test Set (Philips) | Test Set (Siemens) | Description |
|--------|--------------------|--------------------|------------|
| Pearson r | 0.814 | 0.482 | Raw predictions |
| Balanced Accuracy | 0.687 | 0.566 | Pre-calibration, SYNTAX > 22 |
| **Post-Calibration** | | | |
| Pearson r | 0.816 | 0.482 | After scaling |
| Balanced Accuracy | **0.825** | **0.643** | After scaling, SYNTAX > 22 |

**Key findings:**
- Strong performance on in-distribution (Philips) test data
- Domain shift effect on out-of-distribution (Siemens) data
- Calibration effective for Philips, limited benefit for Siemens
- Inter-observer variability constrains upper bound on accuracy

### Model Architecture

- **Backbone:** R3D-18 (video ResNet) pre-trained on Kinetics-400
- **Head (default):** LSTM with mean-pooling aggregation + linear regressor
- **Output:** 2 channels (classification logit + log-transformed regression)
- **Loss:** Weighted BCE (classification) + scaled MSE (regression)

## Weights & Checkpoints

Pre-trained weights available in repository:

- `backbone_weights/`: `.pt` files for 5 folds × 2 arteries
  - Example: `RightBinSyntax_R3D_full_fold00.pt`
- `full_model_weights/`: `.pt` files for ensemble variants
  - Example: `RightBinSyntax_R3D_fold00_lstm_mean_post_best.pt`
- `scaling_coeffs/`: Calibration parameters (a, b) per fold

## Citation

If you use this dataset or code, please cite:

```bibtex
@dataset{cardiosyntax_v2_2025,
  title={CardioSyntax v2: Angiographic dataset for SYNTAX score estimation},
  author={MesserMMP and collaborators},
  year={2025},
  url={https://huggingface.co/datasets/MesserMMP/coronary-angiography-syntax}
}
```

## Related Work

- **SYNTAX Score Definition:** Sianos et al., EuroIntervention (2005)
- **Original CardioSyntax:** Ponomarchuk et al., arXiv:2407.19894 (2024)
- **Video Understanding:** Hara et al., CVPR (2018) — R3D architecture

## License

CC0 1.0 (Public Domain)

## Contact & Support

For issues, questions, or contributions, please refer to the [HuggingFace repository discussions](https://huggingface.co/MesserMMP/coronary-syntax-prediction/discussions).

***

**Last updated:** January 2026