qwen2vl-2b-kd-ce-only-full

LoRA adapter (r=128, alpha=256) for Qwen/Qwen2-VL-2B-Instruct, trained with the ce-only objective, distilling from teacher Qwen/Qwen2.5-VL-7B-Instruct on the full llava_v1_5_mix665k mixture (1,525,381 samples, 1 epoch, effective batch 128, lr 1e-5 cosine, bf16, image_resolution=low, max_len 2048).

Results (fast_signal suite, Qwen2.5-32B-Instruct judge)

This run: mean 52.58 (MMBench 73.71 / MMStar 47.8 / MMMU 40.22 / MathVista 48.6)

• eval-lib golden record: ukaz-uit/eval-lib run_id rg306cry (spec sha256:ed2050c5764bfca6)

Full campaign comparison:

Method	MMBench_DEV_EN	MMStar	MMMU_DEV_VAL	MathVista_MINI	Mean
ce_only (baseline)	73.71	47.80	40.22	48.60	52.58
cgkd	73.54	47.60	40.78	47.30	52.30
scva	72.85	47.40	40.11	47.90	52.07
emkd	73.37	47.20	40.44	47.10	52.03
sre	72.59	47.60	40.33	46.00	51.63
base Qwen2-VL-2B	-	-	-	-	51.40
unit_aligned	71.91	47.27	39.67	46.00	51.21

Usage

Merge the adapter into the base model with PEFT, or load directly:

from peft import PeftModel
from transformers import Qwen2VLForConditionalGeneration

base = Qwen2VLForConditionalGeneration.from_pretrained("Qwen/Qwen2-VL-2B-Instruct", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "DiningSystem/qwen2vl-2b-kd-ce-only-full")
model = model.merge_and_unload()

projectors/ (if present) holds the auxiliary alignment projector used only during training.