---
library_name: peft
base_model: jwang01/surgvidlm
base_model_relation: adapter
pipeline_tag: image-text-to-text
tags:
- peft
- lora
- llama-factory
- qwen2-vl
- medical
- surgical-vqa
- heico
- lapchole
---

# ORENA SurgVidLM HEICO + LapChole Adapter

This repository contains the ORENA LoRA adaptation of SurgVidLM for
single-frame surgical visual question answering. The adapter was trained
jointly on HEICO and LapChole and is stored under `surgvidLM/`.

## Base model

The required base checkpoint is the `surgvidlm-stage2` subfolder of
[`jwang01/surgvidlm`](https://huggingface.co/jwang01/surgvidlm). This
repository contains only the downstream LoRA adapter; it does not redistribute
the SurgVidLM base weights or training datasets.

## Training data

The converted data contained:

- Training: 7,200 HEICO and 5,182 LapChole examples
- Validation: 800 HEICO and 566 LapChole examples

No images, videos, annotations, or other training data are included in this
repository.

## Adapter details

- Fine-tuning method: LoRA supervised fine-tuning
- LoRA rank: 8
- LoRA alpha: 16
- LoRA dropout: 0.05
- Vision tower: frozen
- Multimodal projector: frozen
- Learning rate: 5e-5
- Epochs: 3
- Effective training batch size: 32
- Seed: 42
- Framework: LLaMA-Factory / PEFT

The selected adapter corresponds to checkpoint 700. The identical adapter is
stored at the training output root because `load_best_model_at_end` was
enabled.

## Validation

- Combined HEICO + LapChole validation loss: 0.3538476527
- Training loss: 0.3713347467

These are loss values, not official-test accuracy scores. Official test-set
results should be reported separately when available.

## Download

```bash
hf download ahnaf1393/ORENA_weights \
  --include "surgvidLM/*" \
  --local-dir ORENA_weights

hf download jwang01/surgvidlm \
  --include "surgvidlm-stage2/*" \
  --local-dir surgvidlm_base
```

## Inference with the ORENA SurgVidLM script

```bash
python SurgVidLM/frame_vqa_inference_batch.py \
  --model-path surgvidlm_base/surgvidlm-stage2 \
  --adapter-path ORENA_weights/surgvidLM \
  --data-path PATH_TO_INPUT.json \
  --result-path predictions.json
```

## Intended use and limitations

This adapter is intended for research on surgical-scene understanding and
frame-level visual question answering. It has not been validated as a medical
device and must not be used for diagnosis, clinical decisions, or autonomous
surgical control.

Performance may vary across procedures, hospitals, imaging systems, patient
populations, question formats, and conditions not represented during
training.

## License

Public redistribution terms are pending confirmation from the SurgVidLM model
authors and the applicable HEICO and LapChole data-use agreements. Do not
treat the absence of a license field as permission for unrestricted use or
redistribution.

## Acknowledgements

This adapter is derived from
[`SurgVidLM`](https://github.com/gkw0010/SurgVidLM). See also the
[`SurgVidLM` paper](https://arxiv.org/abs/2506.17873).