EulerCAD-Deep / README.md
NicoloLocatelli's picture
Model card
5859526 verified
|
Raw History Blame Contribute Delete
2.34 kB
---
license: mit
tags:
- cad
- graphs
- transformer
- self-supervised
datasets:
- NicoloLocatelli/EulerCAD-ABC-Preprocessed
---
# EulerCAD-Deep
Deep is a decoder-only transformer pretrained on 2D CAD drawings. Each drawing is a graph of points and
straight segments, and the model reads it as a random Euler walk: an ordered sequence of rows that crosses
every segment once. It learns to predict the next row. A drawing's embedding is the mean of one layer's
hidden states over the walk.
Code, training and analysis: [github.com/NicoloLocatelli/EulerCAD](https://github.com/NicoloLocatelli/EulerCAD).
## Model
- 14 layers of width 384, 6 attention heads, 36.9M parameters.
- Gated attention, QK normalization, value residual.
- Vocabulary of 512 node identities and 256 bins for coordinates and distances. Drawings with more than
512 points are dropped.
## Training
- Data: the 246,108 training drawings of
[EulerCAD-ABC-Preprocessed](https://huggingface.co/datasets/NicoloLocatelli/EulerCAD-ABC-Preprocessed),
2D views of 3D objects from the [ABC dataset](https://deep-geometry.github.io/abc-dataset/).
- 208,200 updates of 160 drawings, with the Muon optimizer and a warmup-stable-decay schedule, on one
RTX 4060 Ti (16 GB).
- These weights are the checkpoint at update 208,000, which has the best validation loss: 0.7144.
## Files
- `model.safetensors`: the weights.
- `config.json`: the tokenizer, the model configuration, and the update the weights come from.
## Use
```bash
git clone https://github.com/NicoloLocatelli/EulerCAD.git
cd EulerCAD
uv sync
uv run eulercad model download # into runs/deep/checkpoints/
uv run eulercad analyze runs/deep/checkpoints/model.safetensors --split val
```
`analyze` measures every layer and scores retrieval of the other views of each 3D object. On the
validation split, retrieval is best at layer 7: mean average precision 0.250, recall@1 0.538.
In Python:
```python
from pathlib import Path
from eulercad.training.artifacts import load_model
model = load_model(Path("runs/deep/checkpoints/model.safetensors"))
```
## License
The weights and the code are under the MIT License. The training data derive from ABC, whose CAD models
belong to their creators under the [Onshape Terms of Use](https://www.onshape.com/legal/terms-of-use),
section 1.g.ii.