CADReasoner-CM (cross-modality variant)

CADReasoner-CM is a variant of CADReasoner, a vision–language model for iterative CAD reverse engineering. It generates a runnable CadQuery program and refines it over several iterations using geometric feedback from the discrepancy between the target shape and the current prediction.

This checkpoint conditions on both the point cloud and multi-view renders of the target shape. Combining the two modalities improves geometric alignment and recovers fine details that either modality alone tends to miss.

Accepted to CVPR 2026 Findings Track

Paper: https://arxiv.org/abs/2603.29847 Code: https://github.com/zhemdi/CADReasoner (see the pc_cm/ directory) HF paper page: https://huggingface.co/papers/2603.29847

Variants

Model	Input modality	Flags
kulibinai/cadreasoner	multi-view renders	— (root scripts)
kulibinai/cadreasoner-pc	point cloud	--use_pc true --use_img false
kulibinai/cadreasoner-cm	point cloud + multi-view renders	--use_pc true --use_img true

Architecture

This checkpoint uses the Cadrille architecture: a Qwen2-VL-2B backbone plus a Fourier point-cloud encoder. The point features are projected to the hidden size and scattered into a run of pad tokens prepended to the prompt, so the model cannot be loaded with a bare Qwen2VLForConditionalGeneration.from_pretrained — config.json declares architectures: ["Cadrille"], and the class lives in pc_cm/cadrille.py in the repository.

The point cloud is built from the discrepancy between the target mesh and the current prediction, not from the target alone: both surfaces are sampled, points whose distance to the opposite shape exceeds a percentile threshold are kept, and farthest point sampling reduces them. Each direction contributes n_points points — n_points from GT→pred and n_points from pred→GT — so the prompt reserves 2 * n_points pad tokens and each point carries 6 features (position + displacement vector). On the first iteration, where no prediction exists yet, the bounding-box centre is used in place of the predicted mesh.

--n_points must match the value used in training: 128.

Setup

git clone https://github.com/zhemdi/CADReasoner.git
cd CADReasoner

Build the environment from the provided Dockerfile, then add the two packages the pc_cm/ scripts need on top of it:

pip install opencv-python rtree

opencv-python is used by the image augmentations, and rtree backs trimesh's closest-point queries — without it the code silently falls back to a KD-tree over mesh vertices, which is less accurate on coarse meshes.

The scripts load the model with attn_implementation="flash_attention_2" and torch.bfloat16, so an Ampere-or-newer GPU with flash-attn installed is required. Inference shards samples across all visible GPUs, one process per GPU, and needs at least one.

Inference

python3 pc_cm/test.py \
    --dataset <test_dataset> \
    --checkpoint kulibinai/cadreasoner-cm \
    --use_pc true --use_img true \
    --n_points 128 \
    --n_iters 3 \
    --n_samples 4 \
    --outdir preds_cm

<test_dataset> is either a local directory of .stl files or a Hugging Face dataset repo id:

• maksimko123/deepcad_test_mesh
• maksimko123/fusion360_test_mesh
• kulibinai/mcb_test
• kulibinai/deepcad_test_scan
• kulibinai/fusion360_test_scan
• kulibinai/mcb_test_scan

Each iteration writes a CadQuery program and its compiled mesh per candidate:

preds_cm/<dataset_name>/<iter>/<shape_stem>/<chamfer_distance>.py
preds_cm/<dataset_name>/<iter>/<shape_stem>/<chamfer_distance>.stl

Files are renamed to their chamfer distance so the next iteration can pick the best candidates to refine. --n_samples controls how many candidates are sampled per shape (the first is greedy, the rest use temperature 1.2).

Evaluation

python3 pc_cm/evaluate.py \
    --dataset <test_dataset> \
    --pred_dir preds_cm/<dataset_name>

For each refinement iteration this reports median chamfer distance (scaled by 1000), mean IoU, and the invalidity ratio — the fraction of shapes with no valid prediction. Each shape is scored with the best candidate seen up to that iteration, so the numbers are cumulative across iterations.

Training

The curriculum runs over dataset groups 0, 1, 2 (see data/README.md for preparing the split). Group 0 starts from Qwen/Qwen2-VL-2B-Instruct; each later group starts from the previous group's final checkpoint.

torchrun --nproc_per_node <n_gpus> pc_cm/train.py \
    --dataset_dir <train_dataset_dir> \
    --use_pc true --use_img true \
    --n_points 128

Checkpoints and logs are written to runs/<experiment>/<experiment>_<timestamp>/, with per-group weights under model/<group>/final. Pass --run_dir and --groups to resume an interrupted run, and --skip_generate_code / --skip_generate_meshes to reuse refinement samples already on disk.

Loading the model directly

If you are integrating the model rather than using the scripts:

import torch
from transformers import AutoProcessor
from cadrille import Cadrille  # from pc_cm/

processor = AutoProcessor.from_pretrained(
    "kulibinai/cadreasoner-cm",
    resized_width=14 * 17 * 2,
    resized_height=14 * 17 * 4,
    padding_side="left",
    use_fast=True,
)
model = Cadrille.from_pretrained(
    "kulibinai/cadreasoner-cm",
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
).cuda().eval()

forward and generate additionally require point_clouds (a (batch, 2 * n_points, 6) tensor), is_pc and is_img — they are not optional, and the pad-token prefix must already be present in the prompt. See generate_predictions_process in pc_cm/test.py for a complete, working example.

Limitations

The image-conditioned kulibinai/cadreasoner gives the best results in our implementation; the geometry-conditioned variants are released for reproducibility and for settings where rendered views are unavailable. The point-cloud modality also proved unstable under RL fine-tuning, so it is not being developed further at present.

Citation

@InProceedings{Kabisov_2026_CVPR,
    author    = {Kabisov, Soslan and Kirichuk, Vsevolod and Volkov, Andrey and Barannikov, Marina and Savrasov, Gennadiy and Konushin, Anton and Kuznetsov, Andrey and Zhemchuzhnikov, Dmitrii},
    title     = {CADReasoner: Iterative Program Editing for CAD Reverse Engineering},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings},
    month     = {June},
    year      = {2026},
    pages     = {6143-6153}
}