README.md

---
license: other
library_name: pytorch
pipeline_tag: image-to-3d
tags:
  - cad
  - parametric-cad
  - multimodal
  - deepcad
  - image-to-cad
  - pytorch
---

# DM-CAD Checkpoints

This repository archives the key trained checkpoints for the undergraduate thesis project:

**Research on a Prototype for CAD Command Sequence Generation Based on Multimodal Inputs**

The model predicts a DeepCAD latent vector from multi-view CAD renderings, optionally corrected by a text description. The predicted latent is then decoded by the frozen DeepCAD decoder into a parametric CAD command sequence.

## Files

| Path | Role |
| --- | --- |
| `checkpoints/resnet18_transformer_text_v1_best.pt` | Main final multimodal checkpoint. This is the official thesis checkpoint for image+text inference. |
| `checkpoints/resnet18_transformer_image_v1_best.pt` | Final image-only Transformer checkpoint. Useful for pure visual inference and comparison. |
| `deepcad_pretrained/pretrained.tar` | DeepCAD pretrained autoencoder package used as the frozen CAD encoder/decoder dependency. |
| `config_summary.json` | Extracted checkpoint metadata, training args, and stored metrics. |
| `results/FINAL_THESIS_RESULTS_2026-05-15.md` | Human-readable final experiment summary. |
| `results/*.json` | Machine-readable evaluation outputs for the final checkpoints. |
| `checksums_sha256.txt` | SHA256 checksums for all uploaded files. |

## Official Checkpoints

The final thesis record selects:

- Image-only: `runs/deepcad_latent/resnet18_transformer_v1_ddp2/best.pt`
- Multimodal: `runs/deepcad_latent/resnet18_transformer_text_v1_ddp2/best.pt`
- Enhanced inference: multimodal checkpoint with `blend(alpha=0.5)`

The later `transformer_text_v2` run was not adopted because it did not surpass `transformer_text_v1` on test latent MSE.

## Main Results

Held-out test set:

| Method | Seq Exact | Solid Valid | ACC_cmd | ACC_param | Invalidity | CD mean | CD median |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Image-only Transformer Direct | 0.1313 | 0.8135 | 0.8634 | 0.7415 | 0.1865 | 0.12981 | 0.03433 |
| Multimodal Transformer Direct | 0.1423 | 0.8450 | 0.8721 | 0.7528 | 0.1550 | 0.12944 | 0.02992 |
| Multimodal Transformer Blend 0.5 | 0.1742 | 0.9464 | 0.8620 | 0.7720 | 0.0536 | 0.12508 | 0.03365 |

ARE-reference subset:

| Method | Seq Exact | Solid Valid | ACC_cmd | ACC_param | Invalidity | CD mean | CD median |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Multimodal Transformer Direct | 0.1462 | 0.8480 | 0.8775 | 0.7577 | 0.1520 | 0.12649 | 0.02712 |
| Multimodal Transformer Blend 0.5 | 0.1795 | 0.9478 | 0.8680 | 0.7766 | 0.0522 | 0.12248 | 0.03090 |

## Expected Runtime Context

These checkpoints are project checkpoints rather than standalone Hugging Face Transformers models. They are intended to be used with the original `dm-cad2` source code and the DeepCAD dependency code.

Typical evaluation command:

```bash
python scripts/evaluate_image_to_cad.py \
  --checkpoint checkpoints/resnet18_transformer_text_v1_best.pt \
  --ids <test_ids.txt> \
  --text-root <text_embedding_root> \
  --data-root <dataset_root> \
  --device cuda \
  --batch-size 64 \
  --num-workers 4 \
  --retrieval-mode blend \
  --blend-alpha 0.5 \
  --retrieval-latent-root <train_latent_root> \
  --check-solid-validity \
  --compute-chamfer \
  --num-cd-points 2000 \
  --output <output.json>
```

The dataset and latent/text embedding shards are not included in this model repository.

## Checkpoint Notes

- Main checkpoint SHA256: `26efa55128cca01fb8d797615fb9470a31e30d4f168ef68a681934f5c5c2ff11`
- Image-only checkpoint SHA256: `9192081b95b9823a8199edf605b43e67ae9363c13166ab4d7ef174044359ebbd`
- DeepCAD pretrained package SHA256: `511eb29e945f1c84f10b9b60127e27a38c4fa0b968ad160fc58c675bfc8473e4`