Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,108 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
# CSLLM: Crystal Structure Large Language Model
|
| 6 |
+
|
| 7 |
+
## Model Description
|
| 8 |
+
|
| 9 |
+
CSLLM (Crystal Structure Large Language Model) is a specialized framework of fine-tuned large language models designed for crystal structure synthesizability prediction. The CSLLM framework consists of three specialized LLMs that can predict the synthesizability of arbitrary 3D crystal structures, identify possible synthetic methods, and recommend suitable precursors.
|
| 10 |
+
|
| 11 |
+
## Model Details
|
| 12 |
+
|
| 13 |
+
- **Repository:** https://github.com/szl666/CSLLM
|
| 14 |
+
- **Model type:** Fine-tuned Large Language Model for Crystal Structure Synthesizability Prediction
|
| 15 |
+
- **Language(s):** English
|
| 16 |
+
- **Base models:** LLaMA-7B, LLaMA3-8B
|
| 17 |
+
|
| 18 |
+
## Model Variants
|
| 19 |
+
|
| 20 |
+
The CSLLM family includes several specialized variants:
|
| 21 |
+
|
| 22 |
+
- **`method_llm_llama3`**: Specialized for crystal synthesis method prediction
|
| 23 |
+
- **`precursor_llm_llama3`**: Focused on precursor identification and selection for crystal synthesis
|
| 24 |
+
- **`synthesis_llm_llama`**: Synthesizability prediction for crystals using LLaMA-7B
|
| 25 |
+
- **`synthesis_llm_llama3`**: Synthesizability prediction for crystals using LLaMA3-8B
|
| 26 |
+
|
| 27 |
+
## Usage
|
| 28 |
+
|
| 29 |
+
### Installation and Setup
|
| 30 |
+
|
| 31 |
+
First, clone and install the LMFlow library:
|
| 32 |
+
```bash
|
| 33 |
+
git clone https://github.com/OptimalScale/LMFlow
|
| 34 |
+
cd LMFlow
|
| 35 |
+
pip install -e .
|
| 36 |
+
```
|
| 37 |
+
|
| 38 |
+
### Batch Evaluation Script
|
| 39 |
+
|
| 40 |
+
Create a `run_evaluation.sh` script in the LMFlow main directory:
|
| 41 |
+
|
| 42 |
+
```bash
|
| 43 |
+
#!/bin/bash
|
| 44 |
+
|
| 45 |
+
CUDA_VISIBLE_DEVICES=0 \
|
| 46 |
+
deepspeed examples/evaluate.py \
|
| 47 |
+
--answer_type math \
|
| 48 |
+
--model_name_or_path {model_name_or_path} \
|
| 49 |
+
--lora_model_path {lora_model_path} \
|
| 50 |
+
--dataset_path {dataset_path} \
|
| 51 |
+
--prompt_structure "input: {input}" \
|
| 52 |
+
--deepspeed examples/ds_config.json \
|
| 53 |
+
--metric accuracy
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
### Configuration Parameters
|
| 57 |
+
|
| 58 |
+
#### Base Model Options (`model_name_or_path`)
|
| 59 |
+
- `llama-7b-hf`: LLaMA 7B base model
|
| 60 |
+
- `llama3-8bf-hf`: LLaMA3 8B base model
|
| 61 |
+
|
| 62 |
+
#### Fine-tuned Model Variants (`lora_model_path`)
|
| 63 |
+
- `method_llm_llama3`: Crystal synthesis method prediction
|
| 64 |
+
- `precursor_llm_llama3`: Precursor recommendation
|
| 65 |
+
- `synthesis_llm_llama`: Synthesizability prediction (LLaMA-7B based)
|
| 66 |
+
- `synthesis_llm_llama3`: Synthesizability prediction (LLaMA3-8B based)
|
| 67 |
+
|
| 68 |
+
#### Dataset Path (dataset_path)
|
| 69 |
+
|
| 70 |
+
Example test data is provided in the repository at:
|
| 71 |
+
https://github.com/szl666/CSLLM/tree/main/data
|
| 72 |
+
|
| 73 |
+
## Training Data
|
| 74 |
+
|
| 75 |
+
CSLLM models were trained on curated datasets including:
|
| 76 |
+
- Crystal structure databases (ICSD, COD, Materials Project)
|
| 77 |
+
- Synthesis protocols and experimental procedures from materials science literature
|
| 78 |
+
|
| 79 |
+
## Performance
|
| 80 |
+
|
| 81 |
+
The models demonstrate strong performance across various crystallographic tasks:
|
| 82 |
+
- Synthesizability Prediction: 98.6% on unseen crystal structures
|
| 83 |
+
- Synthesis method prediction: Exceeds 90% accuracy in classifying synthetic methods
|
| 84 |
+
- Precursor selection: Exceeds 90% accuracy in identifying suitable solid-state synthetic precursors for common binary and ternary compounds
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
## Technical Requirements
|
| 88 |
+
|
| 89 |
+
- **Hardware**: NVIDIA GPU with at least 40GB VRAM recommended
|
| 90 |
+
- **Software**: PyTorch, DeepSpeed, LMFlow framework
|
| 91 |
+
- **Memory**: Sufficient RAM for loading large language models
|
| 92 |
+
|
| 93 |
+
|
| 94 |
+
## Citation
|
| 95 |
+
|
| 96 |
+
If you use CSLLM in your research, please cite the repository:
|
| 97 |
+
|
| 98 |
+
```bibtex
|
| 99 |
+
@article{song2025accurate,
|
| 100 |
+
title={Accurate prediction of synthesizability and precursors of 3D crystal structures via large language models},
|
| 101 |
+
author={Song, Z and Lu, S and Ju, M and others},
|
| 102 |
+
journal={Nature Communications},
|
| 103 |
+
volume={16},
|
| 104 |
+
number={1},
|
| 105 |
+
pages={6530},
|
| 106 |
+
year={2025}
|
| 107 |
+
}
|
| 108 |
+
```
|