| --- |
| library_name: peft |
| base_model: jwang01/surgvidlm |
| base_model_relation: adapter |
| pipeline_tag: image-text-to-text |
| tags: |
| - peft |
| - lora |
| - llama-factory |
| - qwen2-vl |
| - medical |
| - surgical-vqa |
| - heico |
| - lapchole |
| --- |
| |
| # ORENA SurgVidLM HEICO + LapChole Adapter |
|
|
| This repository contains the ORENA LoRA adaptation of SurgVidLM for |
| single-frame surgical visual question answering. The adapter was trained |
| jointly on HEICO and LapChole and is stored under `surgvidLM/`. |
|
|
| ## Base model |
|
|
| The required base checkpoint is the `surgvidlm-stage2` subfolder of |
| [`jwang01/surgvidlm`](https://huggingface.co/jwang01/surgvidlm). This |
| repository contains only the downstream LoRA adapter; it does not redistribute |
| the SurgVidLM base weights or training datasets. |
|
|
| ## Training data |
|
|
| The converted data contained: |
|
|
| - Training: 7,200 HEICO and 5,182 LapChole examples |
| - Validation: 800 HEICO and 566 LapChole examples |
|
|
| No images, videos, annotations, or other training data are included in this |
| repository. |
|
|
| ## Adapter details |
|
|
| - Fine-tuning method: LoRA supervised fine-tuning |
| - LoRA rank: 8 |
| - LoRA alpha: 16 |
| - LoRA dropout: 0.05 |
| - Vision tower: frozen |
| - Multimodal projector: frozen |
| - Learning rate: 5e-5 |
| - Epochs: 3 |
| - Effective training batch size: 32 |
| - Seed: 42 |
| - Framework: LLaMA-Factory / PEFT |
|
|
| The selected adapter corresponds to checkpoint 700. The identical adapter is |
| stored at the training output root because `load_best_model_at_end` was |
| enabled. |
|
|
| ## Validation |
|
|
| - Combined HEICO + LapChole validation loss: 0.3538476527 |
| - Training loss: 0.3713347467 |
|
|
| These are loss values, not official-test accuracy scores. Official test-set |
| results should be reported separately when available. |
|
|
| ## Download |
|
|
| ```bash |
| hf download ahnaf1393/ORENA_weights \ |
| --include "surgvidLM/*" \ |
| --local-dir ORENA_weights |
| |
| hf download jwang01/surgvidlm \ |
| --include "surgvidlm-stage2/*" \ |
| --local-dir surgvidlm_base |
| ``` |
|
|
| ## Inference with the ORENA SurgVidLM script |
|
|
| ```bash |
| python SurgVidLM/frame_vqa_inference_batch.py \ |
| --model-path surgvidlm_base/surgvidlm-stage2 \ |
| --adapter-path ORENA_weights/surgvidLM \ |
| --data-path PATH_TO_INPUT.json \ |
| --result-path predictions.json |
| ``` |
|
|
| ## Intended use and limitations |
|
|
| This adapter is intended for research on surgical-scene understanding and |
| frame-level visual question answering. It has not been validated as a medical |
| device and must not be used for diagnosis, clinical decisions, or autonomous |
| surgical control. |
|
|
| Performance may vary across procedures, hospitals, imaging systems, patient |
| populations, question formats, and conditions not represented during |
| training. |
|
|
| ## License |
|
|
| Public redistribution terms are pending confirmation from the SurgVidLM model |
| authors and the applicable HEICO and LapChole data-use agreements. Do not |
| treat the absence of a license field as permission for unrestricted use or |
| redistribution. |
|
|
| ## Acknowledgements |
|
|
| This adapter is derived from |
| [`SurgVidLM`](https://github.com/gkw0010/SurgVidLM). See also the |
| [`SurgVidLM` paper](https://arxiv.org/abs/2506.17873). |
|
|
