alex-novineer/validating_cad_draws

Teaching Vision-Language Models to validate Technical Drawings and CAD Models

LoRA adapters and data for the paper Teaching Vision-Language Models to validate Technical Drawings and CAD Models (CV4DC Workshop, ACCV 2026).

• Paper: https://alexvalgui.github.io/validating_cad_draws/paper.pdf
• Code: https://github.com/alexvalgui/validating_cad_draws

The task is binary verification: given a 2D engineering drawing and a render of a 3D CAD model, does the pair show the same part? The adapters turn Qwen3-VL-2B and Gemma-3-4B, which answer Yes to almost every pair zero-shot, into calibrated verifiers that run on a single workstation GPU.

Contents

validating_cad_draws/
├── qwen3-vl-2b-lora/   LoRA adapter for Qwen/Qwen3-VL-2B-Instruct (text + vision)
├── gemma3-4b-lora/     LoRA adapter for google/gemma-3-4b-it (text + vision)
└── data/
    ├── omnicad_*.zip   Hidden-line drawings and renders for our Omni-CAD subset
    └── medi_bench.zip  Medical prosthetics benchmark

Adapters

Both adapters were trained for one epoch on 107,160 drawing–render pairs from Omni-CAD with LoRA rank 16, α 32, dropout 0.05, applied to the attention and MLP projections of the language model and the vision encoder. Training ran in bf16 on one NVIDIA L40S (48 GB). Full hyperparameters are in the paper and the GitHub repository.

Adapter	Base model	Omni-CAD Acc / AUC	Prosthetics Acc / AUC
qwen3-vl-2b-lora	Qwen3-VL-2B-Instruct	92.0 / 0.976	75.0 / 0.815
gemma3-4b-lora	Gemma-3-4B-it	90.3 / 0.973	72.2 / 0.806

Prosthetics numbers use the Omni-CAD adapters with no further training.

Data

Omni-CAD subset. 10,000 parts selected from Omni-CAD by farthest-point sampling over geometric features, split by part into 8,930 training parts and 470 held-out parts. For each part and each of six axis-aligned views we provide a 1024×1024 hidden-line drawing (OpenCascade HLR, visible edges solid, hidden edges dashed) and a 512×512 Blender render. A dimensioned variant of the drawings is included for ablation. The underlying Omni-CAD models are released by their authors under the MIT license.

Medical prosthetics benchmark (medi_bench.zip). 199 upper-limb prosthesis meshes collected from GrabCAD and the NIH 3D Print Exchange, grouped into 88 design families and eight functional categories (fingers and phalanges, pins, tensioners, palms and arms, locking mechanisms, forearm inserts, enclosures, straps and small hardware). Each mesh comes with aligned drawings and renders produced by the same pipeline, plus family and category labels so that same-family negatives can be excluded. The evaluation set has 2,284 balanced pairs.

Usage

The model is prompted once and the answer is read from the next-token log-probabilities of Yes and No, not sampled.

import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForImageTextToText
from peft import PeftModel

BASE = "Qwen/Qwen3-VL-2B-Instruct"
ADAPTER = "alex-novineer/validating_cad_draws"
SUBFOLDER = "qwen3-vl-2b-lora"

processor = AutoProcessor.from_pretrained(BASE)
model = AutoModelForImageTextToText.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="cuda")
model = PeftModel.from_pretrained(model, ADAPTER, subfolder=SUBFOLDER).eval()

drawing = Image.open("drawing.png").convert("RGB")
render = Image.open("render.png").convert("RGB")

messages = [
    {"role": "system", "content": "You are an expert mechanical engineer performing quality inspection."},
    {"role": "user", "content": [
        {"type": "image"}, {"type": "image"},
        {"type": "text", "text": "The first image is a 2D engineering drawing. The second image is a render "
                                 "of a 3D CAD model. Do the drawing and the model show the same part? "
                                 "Answer with exactly one word: Yes or No."},
    ]},
]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=[prompt], images=[drawing, render], return_tensors="pt").to(model.device)

with torch.no_grad():
    logits = model(**inputs).logits[0, -1]

tok = processor.tokenizer
yes_ids = {tok.encode(t, add_special_tokens=False)[0] for t in ["Yes", " Yes", "yes", " yes"]}
no_ids = {tok.encode(t, add_special_tokens=False)[0] for t in ["No", " No", "no", " no"]}
log_probs = torch.log_softmax(logits.float(), dim=-1)
p_yes = torch.logsumexp(log_probs[list(yes_ids)], 0).exp()
p_no = torch.logsumexp(log_probs[list(no_ids)], 0).exp()
score = (p_yes / (p_yes + p_no)).item()
print("same part" if score >= 0.5 else "different part", round(score, 3))

For Gemma, set BASE = "google/gemma-3-4b-it" and SUBFOLDER = "gemma3-4b-lora". The evaluation scripts in the GitHub repository reproduce the tables in the paper.

Limitations

• Thresholds reported as Acc* in the paper are chosen on the evaluation set.
• Checkpoints were selected on a 500-pair subset of the held-out set; there is no separate test split.
• Remaining errors are concentrated on hard negatives: close relatives of the correct part that differ in holes, fillets, or bosses. On the prosthetics benchmark each adapter fails on one parametric family (Qwen on locking mechanisms, Gemma on tensioners).
• One seed and one epoch per adapter.

License

The adapters and the drawings/renders we generated are released under the MIT license. Base models keep their own terms: Qwen3-VL is Apache 2.0 and Gemma 3 is subject to the Gemma Terms of Use. Omni-CAD is MIT-licensed by its authors; prosthesis meshes retain the terms of their original GrabCAD and NIH 3D Print Exchange uploads.

Citation

@inproceedings{valverde2026teaching,
  title     = {Teaching Vision-Language Models to validate Technical Drawings and CAD Models},
  author    = {Valverde, Alexander and Xie-Li, Danny and Wang, Zhichao and Conde, Marcos V.},
  booktitle = {Proceedings of the Asian Conference on Computer Vision (ACCV) Workshops},
  year      = {2026}
}