--- license: mit tags: - cad - graphs - transformer - self-supervised datasets: - NicoloLocatelli/EulerCAD-ABC-Preprocessed --- # EulerCAD-Deep Deep is a decoder-only transformer pretrained on 2D CAD drawings. Each drawing is a graph of points and straight segments, and the model reads it as a random Euler walk: an ordered sequence of rows that crosses every segment once. It learns to predict the next row. A drawing's embedding is the mean of one layer's hidden states over the walk. Code, training and analysis: [github.com/NicoloLocatelli/EulerCAD](https://github.com/NicoloLocatelli/EulerCAD). ## Model - 14 layers of width 384, 6 attention heads, 36.9M parameters. - Gated attention, QK normalization, value residual. - Vocabulary of 512 node identities and 256 bins for coordinates and distances. Drawings with more than 512 points are dropped. ## Training - Data: the 246,108 training drawings of [EulerCAD-ABC-Preprocessed](https://huggingface.co/datasets/NicoloLocatelli/EulerCAD-ABC-Preprocessed), 2D views of 3D objects from the [ABC dataset](https://deep-geometry.github.io/abc-dataset/). - 208,200 updates of 160 drawings, with the Muon optimizer and a warmup-stable-decay schedule, on one RTX 4060 Ti (16 GB). - These weights are the checkpoint at update 208,000, which has the best validation loss: 0.7144. ## Files - `model.safetensors`: the weights. - `config.json`: the tokenizer, the model configuration, and the update the weights come from. ## Use ```bash git clone https://github.com/NicoloLocatelli/EulerCAD.git cd EulerCAD uv sync uv run eulercad model download # into runs/deep/checkpoints/ uv run eulercad analyze runs/deep/checkpoints/model.safetensors --split val ``` `analyze` measures every layer and scores retrieval of the other views of each 3D object. On the validation split, retrieval is best at layer 7: mean average precision 0.250, recall@1 0.538. In Python: ```python from pathlib import Path from eulercad.training.artifacts import load_model model = load_model(Path("runs/deep/checkpoints/model.safetensors")) ``` ## License The weights and the code are under the MIT License. The training data derive from ABC, whose CAD models belong to their creators under the [Onshape Terms of Use](https://www.onshape.com/legal/terms-of-use), section 1.g.ii.