File size: 3,019 Bytes
b58da11 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ---
library_name: peft
base_model: jwang01/surgvidlm
base_model_relation: adapter
pipeline_tag: image-text-to-text
tags:
- peft
- lora
- llama-factory
- qwen2-vl
- medical
- surgical-vqa
- heico
- lapchole
---
# ORENA SurgVidLM HEICO + LapChole Adapter
This repository contains the ORENA LoRA adaptation of SurgVidLM for
single-frame surgical visual question answering. The adapter was trained
jointly on HEICO and LapChole and is stored under `surgvidLM/`.
## Base model
The required base checkpoint is the `surgvidlm-stage2` subfolder of
[`jwang01/surgvidlm`](https://huggingface.co/jwang01/surgvidlm). This
repository contains only the downstream LoRA adapter; it does not redistribute
the SurgVidLM base weights or training datasets.
## Training data
The converted data contained:
- Training: 7,200 HEICO and 5,182 LapChole examples
- Validation: 800 HEICO and 566 LapChole examples
No images, videos, annotations, or other training data are included in this
repository.
## Adapter details
- Fine-tuning method: LoRA supervised fine-tuning
- LoRA rank: 8
- LoRA alpha: 16
- LoRA dropout: 0.05
- Vision tower: frozen
- Multimodal projector: frozen
- Learning rate: 5e-5
- Epochs: 3
- Effective training batch size: 32
- Seed: 42
- Framework: LLaMA-Factory / PEFT
The selected adapter corresponds to checkpoint 700. The identical adapter is
stored at the training output root because `load_best_model_at_end` was
enabled.
## Validation
- Combined HEICO + LapChole validation loss: 0.3538476527
- Training loss: 0.3713347467
These are loss values, not official-test accuracy scores. Official test-set
results should be reported separately when available.
## Download
```bash
hf download ahnaf1393/ORENA_weights \
--include "surgvidLM/*" \
--local-dir ORENA_weights
hf download jwang01/surgvidlm \
--include "surgvidlm-stage2/*" \
--local-dir surgvidlm_base
```
## Inference with the ORENA SurgVidLM script
```bash
python SurgVidLM/frame_vqa_inference_batch.py \
--model-path surgvidlm_base/surgvidlm-stage2 \
--adapter-path ORENA_weights/surgvidLM \
--data-path PATH_TO_INPUT.json \
--result-path predictions.json
```
## Intended use and limitations
This adapter is intended for research on surgical-scene understanding and
frame-level visual question answering. It has not been validated as a medical
device and must not be used for diagnosis, clinical decisions, or autonomous
surgical control.
Performance may vary across procedures, hospitals, imaging systems, patient
populations, question formats, and conditions not represented during
training.
## License
Public redistribution terms are pending confirmation from the SurgVidLM model
authors and the applicable HEICO and LapChole data-use agreements. Do not
treat the absence of a license field as permission for unrestricted use or
redistribution.
## Acknowledgements
This adapter is derived from
[`SurgVidLM`](https://github.com/gkw0010/SurgVidLM). See also the
[`SurgVidLM` paper](https://arxiv.org/abs/2506.17873).
|