File size: 2,069 Bytes
350b5c6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
---
{}
---
## References to read
[1]: Prein, T., Pan, E., Doerr, T., Olivetti, E., & Rupp, J. L. M. 2024. **MTEncoder: A Transformer-Based Framework for Materials Representation Learning.** *Materials Today*. Available at chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://openreview.net/pdf?id=wug7i3O7y1
<br>
[2] Schmidt, Jonathan, Hai-Chen Wang, Tiago F.T. Cerqueira, Silvana Botti, Aldo H. Romero, and Miguel A.L. Marques. 2024. "Improving Machine-Learning Models in Materials Science through Large Datasets." Journal Name (To be determined). https://www.sciencedirect.com/science/article/pii/S2542529324002360.
# MTEncoder (SyntMTE)
## Overview
MTEncoder is a transformer-based model for encoding materials’ elemental compositions into dense vector representations. Each material is tokenized into:
- Individual element tokens (e.g., Na, Fe, O)
- A special `Compound` token (`[CPD]`) that aggregates elemental information
These tokens are fed into a transformer encoder, which produces context-rich embeddings. The embedding of the `[CPD]` token serves as the learned representation of the material and is passed through an MLP head to predict various properties[1].
## Pretraining Tasks
MTEncoder is pretrained on the Alexandria dataset [2] across 12 tasks:
| Pretraining Objective |
|----------------------------------------------|
| Stress |
| Band Gap (Direct) |
| Band Gap (Indirect) |
| Density of States at Fermi Level |
| Energy Above Hull |
| Formation Energy |
| Corrected Total Energy |
| Phase Separation Energy |
| Number of Atomic Sites |
| Total Magnetic Moment |
| Crystal Space Group |
| Masked Element Reconstruction (Self-Supervised) |
*Table: Pretraining objectives for MTEncoder (drawn from the Alexandria materials dataset).* |