Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,73 @@
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-sa-4.0
|
| 3 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-sa-4.0
|
| 3 |
+
pipeline_tag: image-segmentation
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
<img src="imgs/nnInteractive_header_white.png">
|
| 7 |
+
|
| 8 |
+
# Model Checkpoint for `nnInteractive: Redefining 3D Promptable Segmentation`
|
| 9 |
+
|
| 10 |
+
This repository provides the official checkpoints for `nnInteractive`, a state-of-the-art framework for 3D promptable segmentation.
|
| 11 |
+
For installation instructions and usage guidance, please refer to the official [python backend](https://github.com/MIC-DKFZ/nnInteractive).
|
| 12 |
+
The backend is designed for seamless integration into Python-based workflows—ideal for researchers, developers, and power users working directly with code.
|
| 13 |
+
`nnInteractive` is also available through graphical viewers (GUI) for those who prefer a visual workflow. The napari and MITK integrations are developed and maintained by our team. Thanks to the community for contributing the 3D Slicer extension!
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
<div align="center">
|
| 17 |
+
|
| 18 |
+
| **<div align="center">[napari plugin](https://github.com/MIC-DKFZ/napari-nninteractive)</div>** | **<div align="center">[MITK integration](https://www.mitk.org/wiki/MITK-nnInteractive)</div>** | **<div align="center">[3D Slicer extension](https://github.com/coendevente/SlicerNNInteractive)</div>** |
|
| 19 |
+
|-------------------|----------------------|-------------------------|
|
| 20 |
+
| [<img src="imgs/Logos/napari.jpg" width="200">](https://github.com/MIC-DKFZ/napari-nninteractive) | [<img src="imgs/Logos/mitk.jpg" width="200">](https://www.mitk.org/wiki/MITK-nnInteractive) | [<img src="imgs/Logos/3DSlicer.png" width="200">](https://github.com/coendevente/SlicerNNInteractive) |
|
| 21 |
+
|
| 22 |
+
</div>
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
## What is nnInteractive?
|
| 26 |
+
|
| 27 |
+
> Isensee, F.\*, Rokuss, M.\*, Krämer, L.\*, Dinkelacker, S., Ravindran, A., Stritzke, F., Hamm, B., Wald, T., Langenberg, M., Ulrich, C., Deissler, J., Floca, R., & Maier-Hein, K. (2025). nnInteractive: Redefining 3D Promptable Segmentation. https://arxiv.org/abs/2503.08373 \
|
| 28 |
+
> *: equal contribution
|
| 29 |
+
|
| 30 |
+
Link: [](https://arxiv.org/abs/2503.08373)
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
##### Abstract:
|
| 34 |
+
|
| 35 |
+
Accurate and efficient 3D segmentation is essential for both clinical and research applications. While foundation
|
| 36 |
+
models like SAM have revolutionized interactive segmentation, their 2D design and domain shift limitations make them
|
| 37 |
+
ill-suited for 3D medical images. Current adaptations address some of these challenges but remain limited, either
|
| 38 |
+
lacking volumetric awareness, offering restricted interactivity, or supporting only a small set of structures and
|
| 39 |
+
modalities. Usability also remains a challenge, as current tools are rarely integrated into established imaging
|
| 40 |
+
platforms and often rely on cumbersome web-based interfaces with restricted functionality. We introduce nnInteractive,
|
| 41 |
+
the first comprehensive 3D interactive open-set segmentation method. It supports diverse prompts—including points,
|
| 42 |
+
scribbles, boxes, and a novel lasso prompt—while leveraging intuitive 2D interactions to generate full 3D
|
| 43 |
+
segmentations. Trained on 120+ diverse volumetric 3D datasets (CT, MRI, PET, 3D Microscopy, etc.), nnInteractive
|
| 44 |
+
sets a new state-of-the-art in accuracy, adaptability, and usability. Crucially, it is the first method integrated
|
| 45 |
+
into widely used image viewers (e.g., Napari, MITK), ensuring broad accessibility for real-world clinical and research
|
| 46 |
+
applications. Extensive benchmarking demonstrates that nnInteractive far surpasses existing methods, setting a new
|
| 47 |
+
standard for AI-driven interactive 3D segmentation.
|
| 48 |
+
|
| 49 |
+
<img src="imgs/figure1_method.png" width="1200">
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
## Citation
|
| 53 |
+
When using nnInteractive, please cite the following paper:
|
| 54 |
+
|
| 55 |
+
> Isensee, F.\*, Rokuss, M.\*, Krämer, L.\*, Dinkelacker, S., Ravindran, A., Stritzke, F., Hamm, B., Wald, T., Langenberg, M., Ulrich, C., Deissler, J., Floca, R., & Maier-Hein, K. (2025). nnInteractive: Redefining 3D Promptable Segmentation. https://arxiv.org/abs/2503.08373 \
|
| 56 |
+
> *: equal contribution
|
| 57 |
+
|
| 58 |
+
Link: [](https://arxiv.org/abs/2503.08373)
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
# License
|
| 62 |
+
Note that the [model checkpoint](https://huggingface.co/nnInteractive/nnInteractive) is `Creative Commons Attribution Non Commercial Share Alike 4.0`!
|
| 63 |
+
|
| 64 |
+
## Acknowledgments
|
| 65 |
+
|
| 66 |
+
<p align="left">
|
| 67 |
+
<img src="imgs/Logos/HI_Logo.png" width="150">
|
| 68 |
+
<img src="imgs/Logos/DKFZ_Logo.png" width="500">
|
| 69 |
+
</p>
|
| 70 |
+
|
| 71 |
+
This repository is developed and maintained by the Applied Computer Vision Lab (ACVL)
|
| 72 |
+
of [Helmholtz Imaging](https://www.helmholtz-imaging.de/) and the
|
| 73 |
+
[Division of Medical Image Computing](https://www.dkfz.de/en/medical-image-computing) at DKFZ.
|