EulerCAD-Deep

Deep is a decoder-only transformer pretrained on 2D CAD drawings. Each drawing is a graph of points and straight segments, and the model reads it as a random Euler walk: an ordered sequence of rows that crosses every segment once. It learns to predict the next row. A drawing's embedding is the mean of one layer's hidden states over the walk.

Code, training and analysis: github.com/NicoloLocatelli/EulerCAD.

Model

  • 14 layers of width 384, 6 attention heads, 36.9M parameters.
  • Gated attention, QK normalization, value residual.
  • Vocabulary of 512 node identities and 256 bins for coordinates and distances. Drawings with more than 512 points are dropped.

Training

  • Data: the 246,108 training drawings of EulerCAD-ABC-Preprocessed, 2D views of 3D objects from the ABC dataset.
  • 208,200 updates of 160 drawings, with the Muon optimizer and a warmup-stable-decay schedule, on one RTX 4060 Ti (16 GB).
  • These weights are the checkpoint at update 208,000, which has the best validation loss: 0.7144.

Files

  • model.safetensors: the weights.
  • config.json: the tokenizer, the model configuration, and the update the weights come from.

Use

git clone https://github.com/NicoloLocatelli/EulerCAD.git
cd EulerCAD
uv sync
uv run eulercad model download          # into runs/deep/checkpoints/
uv run eulercad analyze runs/deep/checkpoints/model.safetensors --split val

analyze measures every layer and scores retrieval of the other views of each 3D object. On the validation split, retrieval is best at layer 7: mean average precision 0.250, recall@1 0.538.

In Python:

from pathlib import Path
from eulercad.training.artifacts import load_model

model = load_model(Path("runs/deep/checkpoints/model.safetensors"))

License

The weights and the code are under the MIT License. The training data derive from ABC, whose CAD models belong to their creators under the Onshape Terms of Use, section 1.g.ii.

Downloads last month
25
Safetensors
Model size
36.9M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for NicoloLocatelli/EulerCAD-Deep

Finetunes
1 model

Dataset used to train NicoloLocatelli/EulerCAD-Deep