Configuration Parsing Warning:In adapter_config.json: "peft.task_type" must be a string

Opsis-V

Opsis-V is the typed question interface of One state, many questions: Opsis gives a frontier model typed eyes for pathology inside Qwen3-VL-8B-Instruct: a LoRA adapter (rank 16, alpha 32, dropout 0.05) on the query, key, value, output, gate, up and down projections of the language tower, with the vision tower frozen. It reads a field at its native 0.2 mpp and its 448 um context canvas at 1.0 mpp as pixels, with the two images as a shared prefix, and answers each typed multiple choice question with the option letter. The answer distribution is the softmax over the candidate letters. Code: https://github.com/gezhixuan/opsis.

Training

NuCLS field questions (19 operators, fields/nucls of the question release), our patient split, 3 epochs, 16 questions per field, learning rate 1e-4, bf16. The paper reports three seeds; this is the seed with the lowest validation NLL (seeds.json lists all three). Prompt, images and scoring are in scripts/opsis_v/40_train_vlm.py of the code repository.

Use

from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor
base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3-VL-8B-Instruct", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "gezx1004/opsis-v")
processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-8B-Instruct")

Evaluation and shared prefix inference: python scripts/opsis_v/40_train_vlm.py --eval-only and scripts/opsis_v/41_vlm_shared_prefix.py.

Limits

Trained on breast cancer fields of TCGA slides at one resolution. It answers the operators it was trained on, it is not a diagnostic device and was not validated for clinical use.

Licence

Apache 2.0, as the base model. The training labels come from NuCLS and BCSS (CC0) on TCGA slides.

Downloads last month
10
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for gezx1004/opsis-v

Adapter
(260)
this model