Image Segmentation
Transformers
PyTorch
repcontur
feature-extraction
medical
radiology
ct
3d
tumor-segmentation
referring-segmentation
report-generation
visual-question-answering
custom_code
Instructions to use Oncosoft-Research/RepConTUR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Oncosoft-Research/RepConTUR with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-segmentation", model="Oncosoft-Research/RepConTUR", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Oncosoft-Research/RepConTUR", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Link to paper page
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,18 +1,18 @@
|
|
| 1 |
---
|
| 2 |
-
|
| 3 |
library_name: transformers
|
|
|
|
| 4 |
pipeline_tag: image-segmentation
|
| 5 |
-
base_model: OpenGVLab/InternVL2_5-1B
|
| 6 |
tags:
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
---
|
| 17 |
|
| 18 |
# RepConTUR-1B
|
|
@@ -24,7 +24,7 @@ scan.
|
|
| 24 |
|
| 25 |
- **Code:** <https://github.com/Oncosoft-Research/RepConTUR>
|
| 26 |
- **Data:** <https://huggingface.co/datasets/Oncosoft-Research/RepConTUR>
|
| 27 |
-
- **Paper:** RepConTUR: A Report-Conditioned Volumetric Multimodal Model for Abdominal Tumor Segmentation
|
| 28 |
|
| 29 |
## Model
|
| 30 |
|
|
@@ -102,10 +102,14 @@ runtime = build_runtime('Oncosoft-Research/RepConTUR') # model, tokenize
|
|
| 102 |
case = load_case(runtime, 'ct.nii.gz') # any 3D abdominal CT in NIfTI
|
| 103 |
report_text = open('report.txt').read() # the narrative report for GSEG
|
| 104 |
|
| 105 |
-
prompt = runtime.image_token_string + '
|
|
|
|
| 106 |
Task.GSEG, 'tumor in the right kidney', report_text)
|
| 107 |
input_ids = runtime.tokenizer.encode(
|
| 108 |
-
f'<|user|>
|
|
|
|
|
|
|
|
|
|
| 109 |
|
| 110 |
output = runtime.model.predict_volume(
|
| 111 |
pixel_values=case.pixel_values.cuda(), input_ids=input_ids, g_pixel_values=case.grounding_volume)
|
|
@@ -155,4 +159,4 @@ non-commercial use only, and adaptations under the same license.
|
|
| 155 |
|
| 156 |
Built on [Sa2VA](https://github.com/bytedance/Sa2VA), [VoxTell](https://github.com/MIC-DKFZ/VoxTell),
|
| 157 |
[CT-CLIP / CT-RATE](https://github.com/ibrahimethemhamamci/CT-CLIP), [InternVL](https://github.com/OpenGVLab/InternVL),
|
| 158 |
-
UniBioMed and AbdomenAtlas 3.0 / RadGPT.
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model: OpenGVLab/InternVL2_5-1B
|
| 3 |
library_name: transformers
|
| 4 |
+
license: cc-by-nc-sa-4.0
|
| 5 |
pipeline_tag: image-segmentation
|
|
|
|
| 6 |
tags:
|
| 7 |
+
- medical
|
| 8 |
+
- radiology
|
| 9 |
+
- ct
|
| 10 |
+
- 3d
|
| 11 |
+
- tumor-segmentation
|
| 12 |
+
- referring-segmentation
|
| 13 |
+
- report-generation
|
| 14 |
+
- visual-question-answering
|
| 15 |
+
- custom_code
|
| 16 |
---
|
| 17 |
|
| 18 |
# RepConTUR-1B
|
|
|
|
| 24 |
|
| 25 |
- **Code:** <https://github.com/Oncosoft-Research/RepConTUR>
|
| 26 |
- **Data:** <https://huggingface.co/datasets/Oncosoft-Research/RepConTUR>
|
| 27 |
+
- **Paper:** [RepConTUR: A Report-Conditioned Volumetric Multimodal Model for Abdominal Tumor Segmentation](https://huggingface.co/papers/2610.40094)
|
| 28 |
|
| 29 |
## Model
|
| 30 |
|
|
|
|
| 102 |
case = load_case(runtime, 'ct.nii.gz') # any 3D abdominal CT in NIfTI
|
| 103 |
report_text = open('report.txt').read() # the narrative report for GSEG
|
| 104 |
|
| 105 |
+
prompt = runtime.image_token_string + '
|
| 106 |
+
' + build_task_prompt(
|
| 107 |
Task.GSEG, 'tumor in the right kidney', report_text)
|
| 108 |
input_ids = runtime.tokenizer.encode(
|
| 109 |
+
f'<|user|>
|
| 110 |
+
{prompt}<|end|>
|
| 111 |
+
<|assistant|>
|
| 112 |
+
', add_special_tokens=False)
|
| 113 |
|
| 114 |
output = runtime.model.predict_volume(
|
| 115 |
pixel_values=case.pixel_values.cuda(), input_ids=input_ids, g_pixel_values=case.grounding_volume)
|
|
|
|
| 159 |
|
| 160 |
Built on [Sa2VA](https://github.com/bytedance/Sa2VA), [VoxTell](https://github.com/MIC-DKFZ/VoxTell),
|
| 161 |
[CT-CLIP / CT-RATE](https://github.com/ibrahimethemhamamci/CT-CLIP), [InternVL](https://github.com/OpenGVLab/InternVL),
|
| 162 |
+
UniBioMed and AbdomenAtlas 3.0 / RadGPT.
|