Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,3 +1,101 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language: en
|
| 3 |
+
license: mit
|
| 4 |
+
tags:
|
| 5 |
+
- surgical-video
|
| 6 |
+
- remaining-surgery-duration
|
| 7 |
+
- multibypass140
|
| 8 |
+
- cholec80
|
| 9 |
+
- workflow-conditioning
|
| 10 |
+
datasets:
|
| 11 |
+
- cholec80
|
| 12 |
+
- multibypass140
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# Surgical workflow models
|
| 16 |
+
|
| 17 |
+
A collection of checkpoints from the Chen lab covering surgical-workflow
|
| 18 |
+
analysis tasks. This repository is intended to host weights for several
|
| 19 |
+
papers and projects; new project subfolders will be added over time.
|
| 20 |
+
|
| 21 |
+
## Project: rsd-mb140-cholec80-2026 (current)
|
| 22 |
+
|
| 23 |
+
Trained model checkpoints for the NeurIPS 2026 submission *"When Workflow
|
| 24 |
+
Conditioning Helps... A Variability-Scaling Study on MultiBypass140 and
|
| 25 |
+
Cholec80."*
|
| 26 |
+
|
| 27 |
+
12 PyTorch state-dicts produced by the training scripts in the
|
| 28 |
+
[bariatric_rsd GitHub repo](https://github.com/billchenxi/bariatric_rsd):
|
| 29 |
+
|
| 30 |
+
### Cholec80 (used for the §6.5 ensemble + isotonic 3.56 result)
|
| 31 |
+
- `cholec80/run018_seed42.pth`
|
| 32 |
+
- `cholec80/run018_seed123.pth`
|
| 33 |
+
- `cholec80/run018_seed777.pth`
|
| 34 |
+
|
| 35 |
+
### MultiBypass140 fold 0 strict prefix-only (Run 033, §6.1 headline)
|
| 36 |
+
- `mb140/run033_no_token_seed{42,123,777}.pth`
|
| 37 |
+
- `mb140/run033_oracle_seed{42,123,777}.pth`
|
| 38 |
+
- `mb140/run033_decoupled_seed{42,123,777}.pth`
|
| 39 |
+
|
| 40 |
+
Each checkpoint is the best-val-MAE state-dict from a 15-epoch training
|
| 41 |
+
run (cosine LR schedule, AdamW, batch 64, sequence_len=8, frame_stride=5).
|
| 42 |
+
|
| 43 |
+
## How to use
|
| 44 |
+
|
| 45 |
+
See the reproducibility package in the GitHub repo:
|
| 46 |
+
<https://github.com/billchenxi/bariatric_rsd/tree/master/reproducibility>
|
| 47 |
+
|
| 48 |
+
Quick verification (requires Cholec80 frames extracted; see DATA_PREP.md):
|
| 49 |
+
|
| 50 |
+
```bash
|
| 51 |
+
git clone https://github.com/billchenxi/bariatric_rsd.git
|
| 52 |
+
cd bariatric_rsd/reproducibility
|
| 53 |
+
pip install -r requirements.txt
|
| 54 |
+
bash scripts/download_weights.sh --only cholec80
|
| 55 |
+
CHOLEC80_ROOT=/path/to/cholec80 bash scripts/verify_cholec80_3.56.sh
|
| 56 |
+
# Expected: 3.563 ± 0.01 min
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
## Model architecture
|
| 60 |
+
|
| 61 |
+
- **Visual encoder:** ViT-B/16 (timm `vit_base_patch16_224`), ImageNet
|
| 62 |
+
pretrained, layers 0-5 frozen.
|
| 63 |
+
- **Temporal head:** Hierarchical Temporal Attention (HTA) adapted from
|
| 64 |
+
Surgformer (Yang et al., 2024).
|
| 65 |
+
- **Heads:** RSD regression, phase classification (auxiliary), deviation
|
| 66 |
+
classification (auxiliary).
|
| 67 |
+
- **Workflow conditioning:** prepended workflow-cluster token (oracle,
|
| 68 |
+
decoupled-oracle, or none) at the temporal-head input.
|
| 69 |
+
|
| 70 |
+
Total parameters: ~93 M (ViT-B + temporal heads).
|
| 71 |
+
|
| 72 |
+
## Caveats and intended use
|
| 73 |
+
|
| 74 |
+
- These checkpoints are released for **academic reproducibility** of the
|
| 75 |
+
NeurIPS 2026 submission. Not intended for clinical use.
|
| 76 |
+
- The 3.56 min Cholec80 result depends on a 3-seed ensemble + horizontal-
|
| 77 |
+
flip TTA + per-video isotonic post-processing. Isotonic post-processing
|
| 78 |
+
alone accounts for ~83% of the gain — see the manuscript §6.5 for the
|
| 79 |
+
full caveats.
|
| 80 |
+
- Cholec80 results are on a 30-video public phase-labeled subset, *not*
|
| 81 |
+
the canonical 40-video test split.
|
| 82 |
+
- MultiBypass140 numbers are validation-set (training-time best-checkpoint
|
| 83 |
+
selection); 5-fold extension is in flight.
|
| 84 |
+
|
| 85 |
+
## License
|
| 86 |
+
|
| 87 |
+
MIT — same as the GitHub repo. Note that the underlying datasets
|
| 88 |
+
(Cholec80, MultiBypass140) have their own licenses and are not
|
| 89 |
+
redistributed here.
|
| 90 |
+
|
| 91 |
+
## Citation
|
| 92 |
+
|
| 93 |
+
```bibtex
|
| 94 |
+
@inproceedings{bariatric_rsd_2026,
|
| 95 |
+
title = {When Workflow Conditioning Helps: A Variability-Scaling Study on MultiBypass140 and Cholec80},
|
| 96 |
+
author = {Chen, Bill and others},
|
| 97 |
+
booktitle = {Submitted to NeurIPS 2026 (Evaluations \& Datasets track)},
|
| 98 |
+
year = {2026},
|
| 99 |
+
url = {https://github.com/billchenxi/bariatric_rsd}
|
| 100 |
+
}
|
| 101 |
+
```
|