---
library_name: peft
license: llama2
base_model:
- Qwen/Qwen3-VL-8B-Instruct
- openbmb/MiniCPM-V-4_5
- OpenGVLab/InternVL3_5-8B-HF
- llava-hf/llava-v1.6-vicuna-13b-hf
- llava-hf/llava-onevision-qwen2-7b-ov-hf
datasets:
- ReefNet/ReefNet-1.0
pipeline_tag: image-text-to-text
tags:
- lora
- llama-factory
- coral-reef
- biodiversity
- fine-grained-classification
- reefnet
---

# ReefNet LoRA adapters: few-shot, variance and rebuttal runs

Supplementary LoRA adapters from the ReefNet paper (NeurIPS 2026, Evaluations & Datasets track) and its rebuttal.
The main fine-tuned models have their own repos:
[Qwen3-VL](https://huggingface.co/ReefNet/reefnet-qwen3-vl-8b-lora),
[MiniCPM-V 4.5](https://huggingface.co/ReefNet/reefnet-minicpm-v-4.5-lora),
[InternVL3.5](https://huggingface.co/ReefNet/reefnet-internvl3.5-8b-lora).

Each subfolder is a self-contained PEFT adapter with its `train_config.yaml`, trainer logs and `predictions/`.
Few-shot support sets are nested: k=1 is a subset of k=5, which is a subset of k=10 and of k=20. They are sampled from the ReefNet
train split, and the sample lists are in the reefnet-mllm repository. The LLaVA-NeXT adapter inherits the Llama 2
licence of its base model. The other adapters are Apache-2.0.

| subfolder | base model | what | ReefNet | Al-Wajh |
|---|---|---|---|---|
| `few_shot/k1/qwen3_vl` | Qwen3-VL-8B-Instruct | Table 4, 1-shot | 5.56 | 10.93 |
| `few_shot/k1/minicpm_v` | MiniCPM-V 4.5 | Table 4, 1-shot | 2.85 | 7.68 |
| `few_shot/k1/internvl3_5` | InternVL3.5-8B | Table 4, 1-shot | 2.00 | 10.31 |
| `few_shot/k5/qwen3_vl` | Qwen3-VL-8B-Instruct | Table 4, 5-shot | 9.15 | 13.77 |
| `few_shot/k5/minicpm_v` | MiniCPM-V 4.5 | Table 4, 5-shot | 7.32 | 8.52 |
| `few_shot/k5/internvl3_5` | InternVL3.5-8B | Table 4, 5-shot | 2.87 | 9.01 |
| `few_shot/k10/qwen3_vl` | Qwen3-VL-8B-Instruct | Table 4, 10-shot | 17.24 | 25.87 |
| `few_shot/k10/minicpm_v` | MiniCPM-V 4.5 | Table 4, 10-shot | 14.95 | 12.77 |
| `few_shot/k10/internvl3_5` | InternVL3.5-8B | Table 4, 10-shot | 10.44 | 12.80 |
| `few_shot/k20/qwen3_vl` | Qwen3-VL-8B-Instruct | Table 4, 20-shot | 27.22 | 31.27 |
| `few_shot/k20/minicpm_v` | MiniCPM-V 4.5 | Table 4, 20-shot | 21.95 | 20.88 |
| `few_shot/k20/internvl3_5` | InternVL3.5-8B | Table 4, 20-shot | 18.34 | 12.87 |
| `seed_variance/qwen3_vl_cap300_seed1` | Qwen3-VL-8B-Instruct | rebuttal: run-to-run variance, class-capped subsample (<=300/genus) | 50.96 | 34.72 |
| `seed_variance/qwen3_vl_cap300_seed2` | Qwen3-VL-8B-Instruct | rebuttal: run-to-run variance, class-capped subsample (<=300/genus) | 50.45 | 34.68 |
| `seed_variance/qwen3_vl_cap300_seed3` | Qwen3-VL-8B-Instruct | rebuttal: run-to-run variance, class-capped subsample (<=300/genus) | 50.16 | 32.66 |
| `convergence/qwen3_vl_cont_2p5ep` | Qwen3-VL-8B-Instruct | rebuttal: +0.5 epoch continuation | 72.75 | 56.94 |
| `llava/llava_next_13b_ckpt14000` | LLaVA-NeXT (Vicuna-13B) | rebuttal: LLaVA-NeXT, step 14000 | 61.70 | 38.62 |
| `llava/llava_onevision_7b_ckpt26000` | LLaVA-OneVision-7B | rebuttal: LLaVA-OneVision, step 26000, 3 GPUs (effective batch 24) | 60.50 | 35.39 |

Base model revisions: `Qwen/Qwen3-VL-8B-Instruct@0c351dd0`, `openbmb/MiniCPM-V-4_5@c1a6986b`, `OpenGVLab/InternVL3_5-8B-HF@741a7d03`, `llava-hf/llava-v1.6-vicuna-13b-hf@745cfbdb`, `llava-hf/llava-onevision-qwen2-7b-ov-hf@0d506805`

## Usage (example: 20-shot Qwen3-VL)

```python
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

base, rev = "Qwen/Qwen3-VL-8B-Instruct", "0c351dd01ed87e9c1b53cbc748cba10e6187ff3b"
model = AutoModelForImageTextToText.from_pretrained(base, revision=rev, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "ReefNet/reefnet-mllm-lora-extras", subfolder="few_shot/k20/qwen3_vl").eval()
processor = AutoProcessor.from_pretrained(base, revision=rev)

# input: a 1024x1024 crop centred on the annotated point
messages = [{"role": "user", "content": [
    {"type": "image", "image": "crop.png"},
    {"type": "text", "text": "What is the genus in the center of the image? Answer with one word"}]}]
inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,
                                       return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=32, do_sample=False)
print(processor.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```
During training the image token sat inside the prompt text (see Prompts below). This snippet puts the image first, which
is how the chat template inserts it.

## Training

LoRA fine-tuning with [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) @ `589da21d`.
The exact config is in `train_config.yaml`: rank 8 on all linear layers, AdamW, lr 5e-5, cosine schedule,
warmup ratio 0.03, bf16, per-device batch 8, no gradient accumulation, `image_max_pixels` 1024x1024, no augmentation.
Software: transformers 4.57.1, peft 0.18.1, torch 2.9.1.
Model selection used the ReefNet validation split. Al-Wajh is never used for training, tuning or selection.

## Prompts (exact strings, `<image>` marks the image position)

| split | user prompt | target |
|---|---|---|
| train / val / ReefNet test | `What is the genus in the center of the image <image>? Answer with one word` | genus name |
| Al-Wajh test | `<image>Identify the hard coral genus in this image. Respond with only the genus name.` | genus name |

Decoding: greedy (`top_k=1`), `max_new_tokens=32`. The output is normalised (strip, capitalise) and must match a genus exactly.

## Metric

Macro recall (balanced accuracy). ReefNet: 39 labels, 23,043 test crops. Al-Wajh: averaged over the 13 genera present
(4,624 crops), while the model keeps the full 39-way output. A prediction outside the 13 counts as an error.
`predictions/` holds the per-sample outputs behind the reported numbers. Recompute them with `tools/recompute_macro_recall.py`
from the reefnet-mllm repository.

## Data

[ReefNet-1.0](https://huggingface.co/datasets/ReefNet/ReefNet-1.0): 1024x1024 crops centred on each point annotation.
Training uses the high-confidence benchmark split (445,985 train / 9,999 val).

## Citation

```bibtex
@inproceedings{felemban2026reefnet,
  title     = {ReefNet: A Large-Scale Dataset and Benchmark for Fine-Grained Coral Reef Recognition},
  author    = {Felemban, Abdulwahab and Battach, Yahia and Khan, Faizan Farooq and Fu, Yuqian and Liu, Xuhui and
               Khattab, Yesmeen and Radwan, Yousef A. and Li, Xiang and Dunn, Natalie and B{\"a}hr, Susanne and
               Terraneo, Tullia Isotta and Marchese, Fabio and Beery, Sara and Jones, Burton H. and
               Benzoni, Francesca and Elhoseiny, Mohamed},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS) Evaluations and Datasets Track},
  year      = {2026}
}
```