|
|
--- |
|
|
language: |
|
|
- en |
|
|
license: mit |
|
|
library_name: transformers |
|
|
tags: |
|
|
- materials-science |
|
|
- crystallography |
|
|
- generative-ai |
|
|
- inverse-design |
|
|
- chemistry |
|
|
- photovoltaics |
|
|
datasets: |
|
|
- c-bone/mpdb-slme-full |
|
|
base_model: c-bone/CrystaLLM-pi_base |
|
|
pipeline_tag: text-generation |
|
|
--- |
|
|
|
|
|
# Model Card for CrystaLLM-pi_SLME |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
|
|
|
**CrystaLLM-pi_SLME** is a conditional generative model designed for the discovery of high-performance photovoltaic materials. It is a fine-tuned version of the `CrystaLLM-pi` framework, based on a GPT-2 decoder-only architecture. This variant employs the **Property-Key-Value (PKV)** attention mechanism to condition the generation of Crystallographic Information Files (CIFs) on the **Spectroscopic Limited Maximum Efficiency (SLME)** metric. |
|
|
|
|
|
The model generates crystal structures based on a single target scalar property: |
|
|
1. **SLME** (%) - A theoretical maximum efficiency metric for photovoltaic absorbers. |
|
|
|
|
|
- **Developed by:** Bone et al. (University College London) |
|
|
- **Model type:** Autoregressive Transformer with Prefix Attention Conditioning |
|
|
- **Language(s):** CIF (Crystallographic Information File) syntax |
|
|
- **License:** MIT |
|
|
- **Finetuned from model:** `c-bone/CrystaLLM-pi_base` |
|
|
|
|
|
### Model Sources |
|
|
|
|
|
- **Repository:** [GitHub: CrystaLLM-pi](https://github.com/C-Bone-UCL/CrystaLLM-pi) |
|
|
- **Paper:** [Discovery and recovery of crystalline materials with property-conditioned transformers (arXiv:2511.21299)](https://arxiv.org/abs/2511.21299) |
|
|
- **Dataset:** [HuggingFace: c-bone/mpdb-slme-full](https://huggingface.co/datasets/c-bone/mpdb-slme-full) |
|
|
|
|
|
## Uses |
|
|
|
|
|
### Direct Use |
|
|
|
|
|
The model is intended for the exploration of chemical space for new photovoltaic candidates. Users can condition generation on high SLME values (e.g., >25%) to discover novel materials with optimal optical and electronic properties for solar energy conversion. |
|
|
|
|
|
### Out-of-Scope Use |
|
|
|
|
|
- **Large Unit Cells:** Context window limit applies (~1024 tokens). |
|
|
- **Production Deployment:** Generated structures are theoretical predictions. Verification via Hybrid-DFT calculations and experimental synthesis is required. |
|
|
|
|
|
## Bias, Risks, and Limitations |
|
|
|
|
|
- **Implicit Learning:** The model was not explicitly trained on band gap data, but implicitly learned to target the optimal Shockley-Queisser range (1.2-1.4 eV) via the SLME metric. It may be less effective at targeting SLME values driven by mechanisms outside the primary training distribution. |
|
|
- **Data Scarcity:** The model was fine-tuned on a relatively small dataset (~5.3K materials). |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
|
|
For instructions on how to load and run generation with this model, please refer to the `_load_and_generate.py` script in the [CrystaLLM-pi GitHub Repository](https://github.com/C-Bone-UCL/CrystaLLM-pi). |
|
|
|
|
|
## Training Details |
|
|
|
|
|
### Training Data |
|
|
|
|
|
The model was fine-tuned on the **MP SLME** dataset, containing inorganic structures labeled with their calculated Spectroscopic Limited Maximum Efficiency. |
|
|
|
|
|
- **Source:** Materials Project / Derived from Walker and Butler (via `c-bone/mpdb-slme-full`) |
|
|
- **Preprocessing:** CIFs are augmented, tokenized, and SLME values are normalized. |
|
|
|
|
|
### Training Procedure |
|
|
|
|
|
- **Architecture:** GPT-2 with Property-Key-Value (PKV) encoder layers. (~38.7M parameters) |
|
|
- **Mechanism:** Prefix Tuning (PKV) is used to inject the SLME target directly into the attention mechanism. |
|
|
|
|
|
## Evaluation |
|
|
|
|
|
### Metrics |
|
|
|
|
|
The model is evaluated based on: |
|
|
1. **Hit-Rate:** Fraction of generated materials with predicted SLME values near the target. |
|
|
2. **VSUN:** Validity, Stability, Uniqueness, and Novelty of the generated candidates. |
|
|
|
|
|
### Results |
|
|
|
|
|
The model successfully generated stable, novel candidates (e.g., $Rb_2(NbBr_3)_3$) with high predicted efficiencies, demonstrating the ability to map complex structure-property relationships from limited data. |
|
|
|
|
|
## Citation |
|
|
|
|
|
```bibtex |
|
|
@misc{bone2025discoveryrecoverycrystallinematerials, |
|
|
title={Discovery and recovery of crystalline materials with property-conditioned transformers}, |
|
|
author={Cyprien Bone and Matthew Walker and Kuangdai Leng and Luis M. Antunes and Ricardo Grau-Crespo and Amil Aligayev and Javier Dominguez and Keith T. Butler}, |
|
|
year={2025}, |
|
|
eprint={2511.21299}, |
|
|
archivePrefix={arXiv}, |
|
|
primaryClass={cond-mat.mtrl-sci}, |
|
|
url={[https://arxiv.org/abs/2511.21299](https://arxiv.org/abs/2511.21299)}, |
|
|
} |
