README.md

---
tags:
- medical-imaging
- diffusion
- 3d
- lidc-idri
- lesion-generation
license: apache-2.0
---

# LesionGenerate (LeFusion)

This repository contains the LeFusion code, model weights, preprocessed LIDC data, bounding-box masks, and generated LIDC examples used in our lesion-generation experiments. It is based on [LeFusion](https://github.com/HINTLab/LeFusion).

## Repository contents

- `LeFusion/`: training, inference, diffusion, and dataset code.
- `LeFusion_Model/LIDC/lidc.pt`: LIDC model checkpoint.
- `data/LIDC/Pathological/`: preprocessed pathological CT crops, lesion masks, bbox masks, and split files.
- `data/LIDC/Normal/`: preprocessed normal CT crops and masks.
- `data/LIDC/Demo/`: demo images, lesion masks, and bbox masks.
- `data/LIDC/Demo_bbox_norm_infer/`: outputs generated with bbox conditioning and normalized-input preprocessing.
- `data/LIDC/Demo_bbox_softedge_norm_infer/`: outputs generated with bbox conditioning, normalized-input preprocessing, and soft-edge repaint blending.
- `scripts/convert_lidc_masks_to_bbox.py`: reproducible conversion from lesion masks to 3D bbox masks.
- `datasets/LesionGenerate_LIDC.tar`: complete `data/LIDC` snapshot (original preprocessed data, bbox masks, and generated demo datasets).

Runtime directories `logs/` and `outputs/` are intentionally not included.

The Hub repository stores the many small NIfTI files in one uncompressed tar archive to avoid API-rate-limit failures. The dataset archive is located at:

```text
datasets/LesionGenerate_LIDC.tar
```

Its size is about 306 MB and it contains the complete `data/LIDC` directory snapshot. Extract it from the repository root with:

```bash
tar -xf datasets/LesionGenerate_LIDC.tar
```

## LIDC preprocessing

The downloaded LIDC data comes from the preprocessed data released for LeFusion. Run:

```bash
bash scripts/download_lidc_data.sh
```

This downloads and extracts the `Pathological`, `Normal`, and `Demo` archives from `YuheLiuu/LeFusion_Preprocessed_Data`.

The original LeFusion LIDC inference path uses the `hu` preprocessing mode:

1. Clamp CT intensity to `[-1000, 400]` HU.
2. Linearly rescale that range to `[-1, 1]`.
3. Crop or pad each sample to `(depth, height, width) = (32, 64, 64)`.
4. Crop or pad the mask to the same shape without intensity interpolation.

For the normalized demo experiments, `lidc_preprocessing=normalized` was added. These demo volumes are already stored in the model's normalized intensity domain, so this mode deliberately skips HU clamp/rescaling and only applies `CropOrPad(32, 64, 64)`. Applying the HU transform again would incorrectly compress already-normalized values.

## Generating `Pathological/Mask_bbox`

Each binary lesion mask is converted into one axis-aligned 3D bounding cuboid:

1. Treat every voxel with value greater than zero as foreground.
2. Find the minimum and maximum foreground coordinate along all three axes.
3. Fill the inclusive bounding cuboid with value `1`; keep all other voxels `0`.
4. Save as `uint8` NIfTI while preserving affine, header, qform, sform, and the original relative path.

Reproduce it with:

```bash
python scripts/convert_lidc_masks_to_bbox.py \
  --input-dir data/LIDC/Pathological/Mask \
  --output-dir data/LIDC/Pathological/Mask_bbox \
  --overwrite
```

The same conversion was applied to the demo masks:

```bash
python scripts/convert_lidc_masks_to_bbox.py \
  --input-dir data/LIDC/Demo/Mask \
  --output-dir data/LIDC/Demo/Mask_bbox \
  --overwrite
```

`data/LIDC/Demo/Image_bbox` is a symbolic link to `Demo/Image`. Its name is intentional: the inference loader replaces `/Image` with `/Mask`, so using `Image_bbox` makes it resolve the corresponding masks under `Mask_bbox` without duplicating the CT files.
The two bbox inference shell scripts recreate this link automatically if an archive or Hub upload does not preserve directory symlinks.

## Generated directories and producing scripts

The following generated directories in this snapshot were produced by these shell entry points. Commands should be run from the repository root.

| Generated directory | Producing script | Main configuration |
|---|---|---|
| `/data/project/MedEdit/LeFusion/data/LIDC/gen` | `scripts/lidc_inference.sh` | Standard LeFusion LIDC inference using `data/LIDC/Normal/Image`, the original HU clamp/rescale preprocessing, three histogram conditions, and the LIDC checkpoint. |
| `/data/project/MedEdit/LeFusion/data/LIDC/Demo_bbox_norm_infer` | `scripts/lidc_inference_demo_bbox_normalized.sh` | Demo inference using `Demo/Mask_bbox` as a hard bbox repaint mask and `lidc_preprocessing=normalized`. |
| `/data/project/MedEdit/LeFusion/data/LIDC/Demo_bbox_softedge_norm_infer` | `scripts/lidc_inference_demo_bbox_softedge_normalized.sh` | The same normalized bbox demo inference with soft-edge repaint alpha and final soft blending enabled. |

Reproduction commands:

```bash
bash scripts/lidc_inference.sh
bash scripts/lidc_inference_demo_bbox_normalized.sh
bash scripts/lidc_inference_demo_bbox_softedge_normalized.sh
```

Each output root contains generated images under `Image/` and the masks used for generation under `Mask/`. For the three-condition LIDC runs, results are separated into `Image_1` to `Image_3` and `Mask_1` to `Mask_3`.

## Generated bbox demo datasets

### `Demo_bbox_norm_infer`

Generated by:

```bash
bash scripts/lidc_inference_demo_bbox_normalized.sh
```

It uses the original demo CT volumes as background/anatomy, `Demo/Mask_bbox` as the editable region, three histogram condition types, the LIDC checkpoint, and `lidc_preprocessing=normalized`. RePaint uses the bbox as a hard binary edit mask: diffusion synthesis occurs inside the box and the input anatomy is retained outside it. Generated images and their conditioning bbox masks are written to the `Image/Image_{1,2,3}` and `Mask/Mask_{1,2,3}` folders.

### `Demo_bbox_softedge_norm_infer`

Generated by:

```bash
bash scripts/lidc_inference_demo_bbox_softedge_normalized.sh
```

This starts from the same images, bbox masks, checkpoint, conditions, and normalized preprocessing, but changes boundary compositing:

- `lidc_repaint_mask_mode=soft_edge`
- radius `3` (`7 x 7 x 7` average-pooling kernel)
- `2` smoothing passes
- power `1.0`, floor `0.0`
- final soft blend enabled

The smoothed alpha remains restricted to the bbox interior. It gradually reduces synthesized-image weight near the bbox boundary, mixes the noised input anatomy into every RePaint step, and performs one final alpha blend with the input. This reduces the hard rectangular seam produced by a binary bbox while keeping voxels outside the bbox unchanged.

## Code changes from upstream

- Added selectable `hu` and `normalized` LIDC inference preprocessing.
- Propagated `lidc_preprocessing` through the inference dataloader and Hydra config.
- Added configurable hard/soft-edge LIDC repaint masks and optional final soft blending.
- Added bbox-mask conversion, demo inference, evaluation, and visualization utilities under `scripts/`.
- Corrected the LIDC checkpoint path in `scripts/lidc_inference.sh`.

The training preprocessing remains unchanged; the new normalization switch applies to inference inputs.

## Environment

Install the pinned dependencies with:

```bash
pip install -r requirements.txt
```

The shell entry points in this snapshot contain paths for the original local environment. Update the repository root, Conda path, and environment name when running elsewhere.

## License and data

Code is provided under the repository's Apache-2.0 license. LIDC-IDRI data remains subject to its original dataset terms and attribution requirements. This repository is for research use and does not provide medical advice or a clinically validated system.