yet-another-simon commited on
Commit
6502057
·
verified ·
1 Parent(s): ab57dd2

Complete model card

Browse files
Files changed (1) hide show
  1. README.md +42 -2
README.md CHANGED
@@ -1,9 +1,49 @@
1
  ---
2
  license: cc-by-nc-sa-4.0
3
- arxiv: 2509.06831
4
  ---
5
 
6
  # Leveraging generic foundation models for multimodal surgical data analysis
7
 
8
  Model repository for the paper **Leveraging generic foundation models for multimodal surgical data analysis** by Simon
9
- Pezold, Jérôme A. Kurylec, Jan S. Liechti, Beat P. Müller, and Joël L. Lavanchy.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-sa-4.0
 
3
  ---
4
 
5
  # Leveraging generic foundation models for multimodal surgical data analysis
6
 
7
  Model repository for the paper **Leveraging generic foundation models for multimodal surgical data analysis** by Simon
8
+ Pezold, Jérôme A. Kurylec, Jan S. Liechti, Beat P. Müller, and Joël L. Lavanchy. For more details, see the [paper](https://arxiv.org/abs/2509.06831) and its
9
+ [code repository](https://github.com/DigitalSurgeryLab-Basel/ML-CDS-2025).
10
+
11
+ ## Model weights and usage
12
+
13
+ * **Finetuned V-JEPA** (ours): We finetuned *V-JEPA ViT-L* on unlabeled endoscopic surgical videos, as described in the paper.
14
+ The resulting model weights can be downloaded [here](./jepa-12h.pth.tar).
15
+
16
+ * **Pretrained V-JEPA** (third-party): The model weights that formed the basis for finetuning can be downloaded via the links provided in Meta Research's
17
+ [V-JEPA v1 code repository](https://github.com/facebookresearch/jepa): use the link under *model zoo / pretrained models /
18
+ ViT-L / checkpoint* or [this direct link](https://dl.fbaipublicfiles.com/jepa/vitl16/vitl16.pth.tar). Please note that we are not
19
+ affiliated with Meta Research in any way.
20
+
21
+ For details on how to use the provided model weights with the approach proposed in the paper, follow the instructions in its
22
+ [code repository](https://github.com/DigitalSurgeryLab-Basel/ML-CDS-2025).
23
+
24
+ ## Data, distribution, and licensing
25
+
26
+ As we utilized public datasets for finetuning, we distribute our model weights under the same license:
27
+ [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).
28
+ For more details, please refer to the license file linked above.
29
+
30
+ Videos from the following datasets were used for finetuning:
31
+
32
+ * [Heidelberg colorectal (HeiCo)](https://www.synapse.org/Synapse:syn21903917/files/)
33
+ * [Cholec80](http://camma.u-strasbg.fr/datasets/)
34
+ * [MultiBypass140](https://github.com/CAMMA-public/MultiBypass140)
35
+
36
+ We are grateful to all authors for creating and publicly releasing these datasets!
37
+
38
+ ## Citing
39
+
40
+ If you find our work useful, please consider citing:
41
+ ```bibtex
42
+ @article{pezold2025leveraging,
43
+ title = {Leveraging Generic Foundation Models for Multimodal Surgical Data Analysis},
44
+ author = {Pezold, Simon and Kurylec, Jérôme A. and Liechti, Jan S. and Müller, Beat P. and Lavanchy, Joël L.},
45
+ journal = {arXiv preprint},
46
+ year = {2025},
47
+ doi = {10.48550/arXiv.2509.06831},
48
+ }
49
+ ```