Sikar/9b-vl-4-source-mu-lora-phase-2

Qwen3.5-9B VLM Phase 2 LoRA — Reverse-Engineering Screenshots to TUI Plugins

Overview

Phase 2 VLM SFT adapter for Qwen3.5-9B. Trained on 1090 reverse-engineering traces where the model sees a screenshot and builds a TUI (terminal UI) plugin that replicates it.

Base Model

Qwen3.5-9B with Phase 1 text-only LoRA already merged in (trained on 1566 tool-use/plugin-building examples for 2000 steps).

Architecture: Qwen3_5ForConditionalGeneration (VLM)

Training Config

Parameter	Value
Framework	TRL 1.5.0 + DeepSpeed ZeRO-3
GPUs	8x A100 80GB
max_length	20480
LoRA rank	16
LoRA alpha	32.0
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Learning rate	5e-6, cosine schedule
Warmup	30 steps
Batch size	1 per GPU (effective 8)
Total steps	1500
Attention	SDPA
Loss	Full sequence

Training Data

1090 examples from run_v5_reveng_img:

• Screenshots from: design2code, websight, webui, pico8
• Task: Given a screenshot, reverse-engineer it into a mu TUI panel plugin
• Format: Multi-turn tool-calling conversations with image in tool result
• Token length: 14K-31K (median 19.5K)

Loss Curve

Step	Loss	Accuracy	Epoch
5	4.52	64.3%	0.04
50	1.25	82.2%	0.37
100	0.70	86.6%	0.73
200	0.39	90.4%	1.50
400	0.17	95.3%	2.92
550	0.16	95.5%	4.0

Checkpoints

• step_200/ - Loss 0.39, Epoch 1.5 (good generalization)
• step_400/ - Loss 0.17, Epoch 2.9 (best quality/overfit trade-off)
• More checkpoints added as training continues

Recommended Checkpoint

step_400 is likely the best trade-off between quality and generalization:

• The model has seen the data about 3 times (epoch 2.9)
• Loss is 0.17 with 95.3% token accuracy
• Later checkpoints risk overfitting on 1090 examples