Fix load example: Qwen3-VL needs AutoModelForImageTextToText, not AutoModelForCausalLM

README.md

@@ -19,11 +19,23 @@ Reward scales are **not comparable across the two** — they come from different
 reward models. Compare policies by downstream task performance, not by these numbers.
 
 ```python
-from peft import AutoPeftModelForCausalLM
-model = AutoPeftModelForCausalLM.from_pretrained(
-    "YeMoKoo/AF_RL", subfolder="scalar-firsterror-20260817", torch_dtype="bfloat16")
+import torch
+from transformers import AutoModelForImageTextToText
+from peft import PeftModel
+
+base = AutoModelForImageTextToText.from_pretrained(
+    "Qwen/Qwen3-VL-2B-Instruct",
+    revision="89644892e4d85e24eaac8bacfd4f463576704203",
+    dtype=torch.bfloat16, device_map="cuda")
+model = PeftModel.from_pretrained(
+    base, "YeMoKoo/AF_RL", subfolder="scalar-firsterror-20260817")
 ```
 
+Qwen3-VL is a vision-language model, so `AutoModelForCausalLM` /
+`AutoPeftModelForCausalLM` do **not** work — use `AutoModelForImageTextToText`.
+After loading, assert
+`sum("lora" in n.lower() for n, _ in model.named_parameters()) == 392`.
+
 Both were trained on the `first_error` split (2,662 samples) of
 `Gyubeum/AndroidFlux_RL_Train_Test` @ `4545695c26951a4884e8f18e6468d69cbdf9d174`,
 for 1 epoch with identical hyperparameters — the reward model is the only

discrete-firsterror-20260817/README.md

@@ -9,30 +9,32 @@ Reward model: **discrete** — judge-token `<|+|>` / `<|-|>` log-probability mar
 ## Load
 
 ```python
-from peft import AutoPeftModelForCausalLM
-model = AutoPeftModelForCausalLM.from_pretrained(
-    "YeMoKoo/AF_RL", subfolder="discrete-firsterror-20260817", torch_dtype="bfloat16")
-```
+import torch
+from transformers import AutoModelForImageTextToText, AutoProcessor
+from peft import PeftModel
 
-Or with the base model loaded explicitly:
+BASE, REV = "Qwen/Qwen3-VL-2B-Instruct", "89644892e4d85e24eaac8bacfd4f463576704203"
 
-```python
-from transformers import AutoModelForCausalLM
-from peft import PeftModel
-base = AutoModelForCausalLM.from_pretrained(
-    "Qwen/Qwen3-VL-2B-Instruct",
-    revision="89644892e4d85e24eaac8bacfd4f463576704203",
-    torch_dtype="bfloat16")
+base = AutoModelForImageTextToText.from_pretrained(
+    BASE, revision=REV, dtype=torch.bfloat16, device_map="cuda")
 model = PeftModel.from_pretrained(base, "YeMoKoo/AF_RL", subfolder="discrete-firsterror-20260817")
+processor = AutoProcessor.from_pretrained(BASE, revision=REV)
+
+# sanity check: must be 392, not 0
+assert sum("lora" in n.lower() for n, _ in model.named_parameters()) == 392
 ```
 
-The base checkpoint is pinned in `adapter_config.json`
-(`Qwen/Qwen3-VL-2B-Instruct` @ `89644892e4d85e24eaac8bacfd4f463576704203`) and is
-downloaded automatically.
+Qwen3-VL is a vision-language model (`Qwen3VLForConditionalGeneration`) and is
+**not** registered in `MODEL_FOR_CAUSAL_LM_MAPPING`, so `AutoModelForCausalLM` and
+`AutoPeftModelForCausalLM` do not work here — use `AutoModelForImageTextToText`.
+
+The base checkpoint is also pinned in `adapter_config.json`
+(`Qwen/Qwen3-VL-2B-Instruct` @ `89644892e4d85e24eaac8bacfd4f463576704203`).
 
-> `target_modules` is a **regex** over `model.language_model.*`. Verify the adapter
-> actually attached (`sum("lora" in n for n, _ in model.named_parameters()) > 0`);
-> a different `transformers` version can silently match zero modules.
+> `target_modules` is a **regex** over `model.language_model.*`. A different
+> `transformers` version can rename modules so the regex matches **zero** of them;
+> the adapter then loads without any error and the model behaves exactly like the
+> base. Always assert the 392 LoRA parameter count above.
 > Trained with `transformers==4.57.6`, `peft==0.19.1`, `torch==2.8.0`.
 
 ## Training data

scalar-firsterror-20260817/README.md

@@ -9,30 +9,32 @@ Reward model: **scalar** — Bradley-Terry scalar head; raw scalar reward.
 ## Load
 
 ```python
-from peft import AutoPeftModelForCausalLM
-model = AutoPeftModelForCausalLM.from_pretrained(
-    "YeMoKoo/AF_RL", subfolder="scalar-firsterror-20260817", torch_dtype="bfloat16")
-```
+import torch
+from transformers import AutoModelForImageTextToText, AutoProcessor
+from peft import PeftModel
 
-Or with the base model loaded explicitly:
+BASE, REV = "Qwen/Qwen3-VL-2B-Instruct", "89644892e4d85e24eaac8bacfd4f463576704203"
 
-```python
-from transformers import AutoModelForCausalLM
-from peft import PeftModel
-base = AutoModelForCausalLM.from_pretrained(
-    "Qwen/Qwen3-VL-2B-Instruct",
-    revision="89644892e4d85e24eaac8bacfd4f463576704203",
-    torch_dtype="bfloat16")
+base = AutoModelForImageTextToText.from_pretrained(
+    BASE, revision=REV, dtype=torch.bfloat16, device_map="cuda")
 model = PeftModel.from_pretrained(base, "YeMoKoo/AF_RL", subfolder="scalar-firsterror-20260817")
+processor = AutoProcessor.from_pretrained(BASE, revision=REV)
+
+# sanity check: must be 392, not 0
+assert sum("lora" in n.lower() for n, _ in model.named_parameters()) == 392
 ```
 
-The base checkpoint is pinned in `adapter_config.json`
-(`Qwen/Qwen3-VL-2B-Instruct` @ `89644892e4d85e24eaac8bacfd4f463576704203`) and is
-downloaded automatically.
+Qwen3-VL is a vision-language model (`Qwen3VLForConditionalGeneration`) and is
+**not** registered in `MODEL_FOR_CAUSAL_LM_MAPPING`, so `AutoModelForCausalLM` and
+`AutoPeftModelForCausalLM` do not work here — use `AutoModelForImageTextToText`.
+
+The base checkpoint is also pinned in `adapter_config.json`
+(`Qwen/Qwen3-VL-2B-Instruct` @ `89644892e4d85e24eaac8bacfd4f463576704203`).
 
-> `target_modules` is a **regex** over `model.language_model.*`. Verify the adapter
-> actually attached (`sum("lora" in n for n, _ in model.named_parameters()) > 0`);
-> a different `transformers` version can silently match zero modules.
+> `target_modules` is a **regex** over `model.language_model.*`. A different
+> `transformers` version can rename modules so the regex matches **zero** of them;
+> the adapter then loads without any error and the model behaves exactly like the
+> base. Always assert the 392 LoRA parameter count above.
 > Trained with `transformers==4.57.6`, `peft==0.19.1`, `torch==2.8.0`.
 
 ## Training data