Upload folder using huggingface_hub
Browse files- README.md +118 -0
- demo/nep_AuAg/dcu_1/AuAg_nep_train.json +80 -0
- demo/nep_AuAg/dcu_1/std_input.json +72 -0
- demo/nep_AuAg/dcu_1/submit.sh +50 -0
- demo/nep_AuAg/dcu_16/AuAg_nep_train_16card.json +51 -0
- demo/nep_AuAg/dcu_16/submit_16card.sh +61 -0
- demo/nep_AuAg/dcu_8/AuAg_nep_train_8card.json +51 -0
- demo/nep_AuAg/dcu_8/submit_8card.sh +51 -0
- demo/nep_Cu/Cu_nep_train.json +62 -0
- demo/nep_Cu/std_input.json +62 -0
- demo/nep_Cu/submit.sh +49 -0
- demo/nep_HfO2/dcu_1/HfO2_nep_test.json +87 -0
- demo/nep_HfO2/dcu_1/HfO2_nep_train.json +91 -0
- demo/nep_HfO2/dcu_1/submit.sh +48 -0
- demo/nep_HfO2/dcu_16/HfO2_nep_train_16card.json +91 -0
- demo/nep_HfO2/dcu_16/submit_16card.sh +61 -0
- demo/nep_HfO2/dcu_8/HfO2_nep_train_8card.json +91 -0
- demo/nep_HfO2/dcu_8/submit_8card.sh +50 -0
- demo/nep_LiSiC/LiSiC_nep_train.json +72 -0
- demo/nep_LiSiC/std_input.json +72 -0
- demo/nep_LiSiC/submit.sh +51 -0
- matchem_env.sh +48 -0
- matpl_install.sh +179 -0
README.md
ADDED
|
@@ -0,0 +1,118 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<p align="center">
|
| 2 |
+
<strong>
|
| 3 |
+
<span style="font-size: 30px;">NEP</span>
|
| 4 |
+
</strong>
|
| 5 |
+
</p>
|
| 6 |
+
|
| 7 |
+
# Model Introduction
|
| 8 |
+
|
| 9 |
+
NEP (Neural Evolution Potential) is a neural-network potential training example based on MatPL. It learns material interaction information such as energy, forces, and virial from atomic structure data.
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
# Model Description
|
| 13 |
+
|
| 14 |
+
NEP is based on the MatPL framework and is trained on atomic-system data such as Cu and LiSiC. It performs interatomic potential function training and molecular dynamics simulation for materials systems.
|
| 15 |
+
|
| 16 |
+
# Applicable Scenarios
|
| 17 |
+
|
| 18 |
+
| Scenario | Description |
|
| 19 |
+
| :---: | :--- |
|
| 20 |
+
| Cu system NEP training | Train the Cu potential using `demo/nep_Cu/Cu_nep_train.json` |
|
| 21 |
+
| LiSiC system NEP training | Train the LiSiC potential using `demo/nep_LiSiC/LiSiC_nep_train.json` |
|
| 22 |
+
| SLURM job submission | Refer to `demo/nep_Cu/submit.sh` to run training on a cluster |
|
| 23 |
+
| Custom data migration | Organize your data into MatPL-supported formats such as `pwmat/movement` and replace the training path |
|
| 24 |
+
|
| 25 |
+
# Usage Instructions
|
| 26 |
+
|
| 27 |
+
## 1. Using OneCode
|
| 28 |
+
|
| 29 |
+
You can try out intelligent one-click AI4S programming in the OneCode online environment:
|
| 30 |
+
|
| 31 |
+
[Try intelligent one-click AI4S programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)
|
| 32 |
+
|
| 33 |
+
## 2. Manual Installation and Usage
|
| 34 |
+
|
| 35 |
+
**Hardware Requirements**
|
| 36 |
+
|
| 37 |
+
- DCU or GPU is recommended for training.
|
| 38 |
+
- CPU can be used for import and small-configuration connectivity checks; full training will be slow.
|
| 39 |
+
- DCU users need to install DTK in advance. DTK 25.04.2 or above, or the OneScience-recommended version matching the current cluster, is suggested.
|
| 40 |
+
|
| 41 |
+
### Download the Model Package
|
| 42 |
+
|
| 43 |
+
```bash
|
| 44 |
+
modelscope download --model OneScience/NEP --local_dir ./nep
|
| 45 |
+
cd nep
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
### Install the Runtime Environment
|
| 49 |
+
|
| 50 |
+
**DCU Environment**
|
| 51 |
+
|
| 52 |
+
```bash
|
| 53 |
+
# Please activate DTK and CONDA first
|
| 54 |
+
conda create -n onescience311 python=3.11 -y
|
| 55 |
+
conda activate onescience311
|
| 56 |
+
# uv installation is also supported
|
| 57 |
+
pip install onescience[matchem-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
|
| 58 |
+
```
|
| 59 |
+
|
| 60 |
+
**GPU Environment**
|
| 61 |
+
|
| 62 |
+
```bash
|
| 63 |
+
# Please activate CONDA first
|
| 64 |
+
conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
|
| 65 |
+
conda activate onescience311
|
| 66 |
+
# uv installation is also supported
|
| 67 |
+
pip install onescience[matchem-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
### Install MatPL
|
| 71 |
+
|
| 72 |
+
```bash
|
| 73 |
+
# Uses the test_pip environment by default; if you use another conda environment name, please specify it first:
|
| 74 |
+
# export MATCHEM_CONDA_NAME=your_env
|
| 75 |
+
bash matpl_install.sh
|
| 76 |
+
```
|
| 77 |
+
|
| 78 |
+
### Training Data Description
|
| 79 |
+
|
| 80 |
+
This repository does not include built-in training data. Taking the MatPL dataset as an example, download it from ModelScope and place it under `data/` in the repository root:
|
| 81 |
+
|
| 82 |
+
```bash
|
| 83 |
+
modelscope download --dataset OneScience/MatPL --local_dir ./data
|
| 84 |
+
```
|
| 85 |
+
|
| 86 |
+
After downloading, the data path will be `data/MatPL/`.
|
| 87 |
+
|
| 88 |
+
### Training
|
| 89 |
+
|
| 90 |
+
Single-GPU:
|
| 91 |
+
|
| 92 |
+
```bash
|
| 93 |
+
cd demo/nep_Cu
|
| 94 |
+
MatPL train Cu_nep_train.json
|
| 95 |
+
```
|
| 96 |
+
|
| 97 |
+
SLURM submission:
|
| 98 |
+
|
| 99 |
+
```bash
|
| 100 |
+
cd demo/nep_Cu
|
| 101 |
+
bash submit.sh
|
| 102 |
+
```
|
| 103 |
+
|
| 104 |
+
### Training Weights
|
| 105 |
+
|
| 106 |
+
This repository currently does not include built-in trained weights. Weights can be obtained through the training steps above.
|
| 107 |
+
|
| 108 |
+
# OneScience Official Information
|
| 109 |
+
|
| 110 |
+
| Platform | OneScience Main Repository | Skills Repository |
|
| 111 |
+
| --- | --- | --- |
|
| 112 |
+
| Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
|
| 113 |
+
| GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |
|
| 114 |
+
|
| 115 |
+
# Citation and License
|
| 116 |
+
|
| 117 |
+
- The NEP example code comes from the OneScience repository. This repository retains the source attribution and is organized for OneScience ModelScope automatic execution scenarios.
|
| 118 |
+
- If you use NEP or MatPL training results in scientific research, we recommend citing the relevant OneScience project information, MatPL/NEP methods, and the sources of the datasets actually used.
|
demo/nep_AuAg/dcu_1/AuAg_nep_train.json
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": [
|
| 4 |
+
"Ag",
|
| 5 |
+
"Au"
|
| 6 |
+
],
|
| 7 |
+
"seed": 2023,
|
| 8 |
+
"recover_train": true,
|
| 9 |
+
"model": {
|
| 10 |
+
"descriptor": {
|
| 11 |
+
"cutoff": [
|
| 12 |
+
6.0,
|
| 13 |
+
4.0
|
| 14 |
+
],
|
| 15 |
+
"n_max": [
|
| 16 |
+
16,
|
| 17 |
+
12
|
| 18 |
+
],
|
| 19 |
+
"basis_size": [
|
| 20 |
+
12,
|
| 21 |
+
8
|
| 22 |
+
],
|
| 23 |
+
"l_max": [
|
| 24 |
+
4,
|
| 25 |
+
2,
|
| 26 |
+
1
|
| 27 |
+
]
|
| 28 |
+
},
|
| 29 |
+
"fitting_net": {
|
| 30 |
+
"network_size": [
|
| 31 |
+
60,
|
| 32 |
+
1
|
| 33 |
+
]
|
| 34 |
+
}
|
| 35 |
+
},
|
| 36 |
+
"optimizer": {
|
| 37 |
+
"optimizer": "ADAM",
|
| 38 |
+
"epochs": 30,
|
| 39 |
+
"batch_size": 8,
|
| 40 |
+
"print_freq": 10,
|
| 41 |
+
"learning_rate": 0.001,
|
| 42 |
+
"stop_lr": 3.51e-08,
|
| 43 |
+
"stop_step": 1000000,
|
| 44 |
+
"decay_step": 5000,
|
| 45 |
+
"train_energy": true,
|
| 46 |
+
"train_force": true,
|
| 47 |
+
"train_virial": true,
|
| 48 |
+
"start_pre_fac_force": 1000,
|
| 49 |
+
"start_pre_fac_etot": 0.02,
|
| 50 |
+
"start_pre_fac_virial": 50.0,
|
| 51 |
+
"end_pre_fac_force": 1.0,
|
| 52 |
+
"end_pre_fac_etot": 1.0,
|
| 53 |
+
"end_pre_fac_virial": 1.0,
|
| 54 |
+
"block_size": 5120,
|
| 55 |
+
"kalman_lambda": 0.98,
|
| 56 |
+
"kalman_nue": 0.9987,
|
| 57 |
+
"pre_fac_force": 2.0,
|
| 58 |
+
"pre_fac_etot": 1.0,
|
| 59 |
+
"pre_fac_virial": 1.0,
|
| 60 |
+
"max_norm": 0.5,
|
| 61 |
+
"norm_type": 2,
|
| 62 |
+
"t_0": 1,
|
| 63 |
+
"t_mult": 2,
|
| 64 |
+
"verbose": true,
|
| 65 |
+
"_clip_value": 1.0
|
| 66 |
+
},
|
| 67 |
+
"format": "pwmlff/npy",
|
| 68 |
+
"train_data": [
|
| 69 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag10Au22",
|
| 70 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag10Au38",
|
| 71 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag10Au6",
|
| 72 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag11Au21",
|
| 73 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag11Au37",
|
| 74 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag11Au5",
|
| 75 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag12"
|
| 76 |
+
],
|
| 77 |
+
"valid_data": [
|
| 78 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag12"
|
| 79 |
+
]
|
| 80 |
+
}
|
demo/nep_AuAg/dcu_1/std_input.json
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": [
|
| 4 |
+
47,
|
| 5 |
+
79
|
| 6 |
+
],
|
| 7 |
+
"seed": 2023,
|
| 8 |
+
"recover_train": true,
|
| 9 |
+
"model": {
|
| 10 |
+
"descriptor": {
|
| 11 |
+
"cutoff": [
|
| 12 |
+
6.0,
|
| 13 |
+
4.0
|
| 14 |
+
],
|
| 15 |
+
"n_max": [
|
| 16 |
+
16,
|
| 17 |
+
12
|
| 18 |
+
],
|
| 19 |
+
"basis_size": [
|
| 20 |
+
12,
|
| 21 |
+
8
|
| 22 |
+
],
|
| 23 |
+
"l_max": [
|
| 24 |
+
4,
|
| 25 |
+
2,
|
| 26 |
+
1
|
| 27 |
+
]
|
| 28 |
+
},
|
| 29 |
+
"fitting_net": {
|
| 30 |
+
"network_size": [
|
| 31 |
+
60,
|
| 32 |
+
1
|
| 33 |
+
]
|
| 34 |
+
}
|
| 35 |
+
},
|
| 36 |
+
"optimizer": {
|
| 37 |
+
"optimizer": "ADAM",
|
| 38 |
+
"epochs": 30,
|
| 39 |
+
"batch_size": 8,
|
| 40 |
+
"print_freq": 10,
|
| 41 |
+
"learning_rate": 0.001,
|
| 42 |
+
"stop_lr": 3.51e-08,
|
| 43 |
+
"stop_step": 1000000,
|
| 44 |
+
"decay_step": 5000,
|
| 45 |
+
"train_energy": true,
|
| 46 |
+
"train_force": true,
|
| 47 |
+
"train_virial": true,
|
| 48 |
+
"start_pre_fac_force": 1000,
|
| 49 |
+
"start_pre_fac_etot": 0.02,
|
| 50 |
+
"start_pre_fac_virial": 50.0,
|
| 51 |
+
"end_pre_fac_force": 1.0,
|
| 52 |
+
"end_pre_fac_etot": 1.0,
|
| 53 |
+
"end_pre_fac_virial": 1.0,
|
| 54 |
+
"max_norm": 0.5,
|
| 55 |
+
"norm_type": 2,
|
| 56 |
+
"t_0": 1,
|
| 57 |
+
"t_mult": 2
|
| 58 |
+
},
|
| 59 |
+
"format": "pwmlff/npy",
|
| 60 |
+
"train_data": [
|
| 61 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag10Au22",
|
| 62 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag10Au38",
|
| 63 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag10Au6",
|
| 64 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag11Au21",
|
| 65 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag11Au37",
|
| 66 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag11Au5",
|
| 67 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag12"
|
| 68 |
+
],
|
| 69 |
+
"valid_data": [
|
| 70 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/pwdata/Ag12"
|
| 71 |
+
]
|
| 72 |
+
}
|
demo/nep_AuAg/dcu_1/submit.sh
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH --job-name=nep_AuAg_train
|
| 3 |
+
#SBATCH --partition=hx1hdexclu12
|
| 4 |
+
#SBATCH --nodes=1
|
| 5 |
+
#SBATCH --ntasks-per-node=1
|
| 6 |
+
#SBATCH --gres=dcu:1
|
| 7 |
+
#SBATCH --cpus-per-task=16
|
| 8 |
+
#SBATCH --time=2:00:00
|
| 9 |
+
#SBATCH --output=slurm_%j.out
|
| 10 |
+
#SBATCH --error=slurm_%j.err
|
| 11 |
+
|
| 12 |
+
if [[ -n "${SCRIPT_DIR}" ]]; then
|
| 13 |
+
:
|
| 14 |
+
elif [[ -n "${SLURM_SUBMIT_DIR}" ]]; then
|
| 15 |
+
SCRIPT_DIR="${SLURM_SUBMIT_DIR}"
|
| 16 |
+
else
|
| 17 |
+
echo "ERROR: 未检测到外部 SCRIPT_DIR 变量,也不在 Slurm 任务环境(SLURM_SUBMIT_DIR 为空)"
|
| 18 |
+
exit 1
|
| 19 |
+
fi
|
| 20 |
+
echo $SCRIPT_DIR
|
| 21 |
+
|
| 22 |
+
export MATCHEM_CONDA_NAME="${MATCHEM_CONDA_NAME:-test_pip}"
|
| 23 |
+
source "$SCRIPT_DIR/matchem_env.sh"
|
| 24 |
+
|
| 25 |
+
# MatPL 运行时环境(不侵入 matchem_env.sh,各算例自行维护)
|
| 26 |
+
# 校验MatPL根路径
|
| 27 |
+
if [[ -z "${MATPL_SRC_DIR}" || "${MATPL_SRC_DIR}" == "/path/to/matpl_dcu" || ! -d "${MATPL_SRC_DIR}" ]]; then
|
| 28 |
+
echo "=============================================="
|
| 29 |
+
echo "ERROR: MATPL_SRC_DIR 路径配置错误!"
|
| 30 |
+
echo "当前值:${MATPL_SRC_DIR}"
|
| 31 |
+
echo "请设置为真实有效的MatPL源码目录,示例:"
|
| 32 |
+
echo "export MATPL_SRC_DIR=/public/home/xxx/real_matpl_path"
|
| 33 |
+
echo "=============================================="
|
| 34 |
+
exit 1
|
| 35 |
+
fi
|
| 36 |
+
|
| 37 |
+
if [ -f "$MATPL_SRC_DIR/env.sh" ]; then
|
| 38 |
+
source "$MATPL_SRC_DIR/env.sh"
|
| 39 |
+
fi
|
| 40 |
+
echo "MATPL_SRC_DIR = $MATPL_SRC_DIR"
|
| 41 |
+
export LD_LIBRARY_PATH="$MATPL_SRC_DIR/src/op/build/lib:${LD_LIBRARY_PATH:-}"
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
# 输入 JSON 使用环境变量,运行前展开为临时文件
|
| 45 |
+
INPUT_JSON="AuAg_nep_train.json"
|
| 46 |
+
EXPANDED_JSON=".${INPUT_JSON%.json}_expanded_$$.json"
|
| 47 |
+
trap 'rm -f "$EXPANDED_JSON"' EXIT
|
| 48 |
+
python3 -c "import os; open('$EXPANDED_JSON','w').write(os.path.expandvars(open('$INPUT_JSON').read()))"
|
| 49 |
+
|
| 50 |
+
MatPL train "$EXPANDED_JSON"
|
demo/nep_AuAg/dcu_16/AuAg_nep_train_16card.json
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": ["Ag", "Au"],
|
| 4 |
+
"seed": 2023,
|
| 5 |
+
"recover_train": true,
|
| 6 |
+
"model": {
|
| 7 |
+
"descriptor": {
|
| 8 |
+
"cutoff": [6.0, 4.0],
|
| 9 |
+
"n_max": [16, 12],
|
| 10 |
+
"basis_size": [12, 8],
|
| 11 |
+
"l_max": [4, 2, 1]
|
| 12 |
+
},
|
| 13 |
+
"fitting_net": {
|
| 14 |
+
"network_size": [100, 1]
|
| 15 |
+
}
|
| 16 |
+
},
|
| 17 |
+
"optimizer": {
|
| 18 |
+
"optimizer": "ADAM",
|
| 19 |
+
"epochs": 30,
|
| 20 |
+
"batch_size": 128,
|
| 21 |
+
"print_freq": 10,
|
| 22 |
+
"learning_rate": 0.001,
|
| 23 |
+
"stop_lr": 3.51e-08,
|
| 24 |
+
"stop_step": 1000000,
|
| 25 |
+
"decay_step": 5000,
|
| 26 |
+
"train_energy": true,
|
| 27 |
+
"train_force": true,
|
| 28 |
+
"train_virial": true,
|
| 29 |
+
"start_pre_fac_force": 1000,
|
| 30 |
+
"start_pre_fac_etot": 0.02,
|
| 31 |
+
"start_pre_fac_virial": 50.0,
|
| 32 |
+
"end_pre_fac_force": 1.0,
|
| 33 |
+
"end_pre_fac_etot": 1.0,
|
| 34 |
+
"end_pre_fac_virial": 1.0,
|
| 35 |
+
"block_size": 5120,
|
| 36 |
+
"kalman_lambda": 0.98,
|
| 37 |
+
"kalman_nue": 0.9987,
|
| 38 |
+
"pre_fac_force": 2.0,
|
| 39 |
+
"pre_fac_etot": 1.0,
|
| 40 |
+
"pre_fac_virial": 1.0,
|
| 41 |
+
"max_norm": 0.5,
|
| 42 |
+
"norm_type": 2,
|
| 43 |
+
"t_0": 1,
|
| 44 |
+
"t_mult": 2,
|
| 45 |
+
"verbose": true,
|
| 46 |
+
"_clip_value": 1.0
|
| 47 |
+
},
|
| 48 |
+
"format": "extxyz",
|
| 49 |
+
"train_data": ["/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/AuAg-5762.xyz"],
|
| 50 |
+
"valid_data": ["/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/AuAg-5762.xyz"]
|
| 51 |
+
}
|
demo/nep_AuAg/dcu_16/submit_16card.sh
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH --job-name=nep_AuAg_16card
|
| 3 |
+
#SBATCH --partition=hx1hdexclu12
|
| 4 |
+
#SBATCH --nodes=2
|
| 5 |
+
#SBATCH --ntasks-per-node=8
|
| 6 |
+
#SBATCH --gres=dcu:8
|
| 7 |
+
#SBATCH --cpus-per-task=4
|
| 8 |
+
#SBATCH --mem=256G
|
| 9 |
+
#SBATCH --time=2:00:00
|
| 10 |
+
#SBATCH --output=slurm_16card_%j.out
|
| 11 |
+
#SBATCH --error=slurm_16card_%j.err
|
| 12 |
+
|
| 13 |
+
if [[ -n "${SCRIPT_DIR}" ]]; then
|
| 14 |
+
:
|
| 15 |
+
elif [[ -n "${SLURM_SUBMIT_DIR}" ]]; then
|
| 16 |
+
SCRIPT_DIR="${SLURM_SUBMIT_DIR}"
|
| 17 |
+
else
|
| 18 |
+
echo "ERROR: 未检测到外部 SCRIPT_DIR 变量,也不在 Slurm 任务环境(SLURM_SUBMIT_DIR 为空)"
|
| 19 |
+
exit 1
|
| 20 |
+
fi
|
| 21 |
+
echo $SCRIPT_DIR
|
| 22 |
+
|
| 23 |
+
export MATCHEM_CONDA_NAME="${MATCHEM_CONDA_NAME:-test_pip}"
|
| 24 |
+
source "$SCRIPT_DIR/matchem_env.sh"
|
| 25 |
+
|
| 26 |
+
# MatPL 运行时环境(不侵入 matchem_env.sh,各算例自行维护)
|
| 27 |
+
# 校验MatPL根路径
|
| 28 |
+
if [[ -z "${MATPL_SRC_DIR}" || "${MATPL_SRC_DIR}" == "/path/to/matpl_dcu" || ! -d "${MATPL_SRC_DIR}" ]]; then
|
| 29 |
+
echo "=============================================="
|
| 30 |
+
echo "ERROR: MATPL_SRC_DIR 路径配置错误!"
|
| 31 |
+
echo "当前值:${MATPL_SRC_DIR}"
|
| 32 |
+
echo "请设置为真实有效的MatPL源码目录,示例:"
|
| 33 |
+
echo "export MATPL_SRC_DIR=/public/home/xxx/real_matpl_path"
|
| 34 |
+
echo "=============================================="
|
| 35 |
+
exit 1
|
| 36 |
+
fi
|
| 37 |
+
|
| 38 |
+
if [ -f "$MATPL_SRC_DIR/env.sh" ]; then
|
| 39 |
+
source "$MATPL_SRC_DIR/env.sh"
|
| 40 |
+
fi
|
| 41 |
+
echo "MATPL_SRC_DIR = $MATPL_SRC_DIR"
|
| 42 |
+
export LD_LIBRARY_PATH="$MATPL_SRC_DIR/src/op/build/lib:${LD_LIBRARY_PATH:-}"
|
| 43 |
+
|
| 44 |
+
# 多节点通信设置
|
| 45 |
+
MASTER_ADDR=$(scontrol show hostnames $SLURM_JOB_NODELIST | head -n 1)
|
| 46 |
+
function get_free_port() {
|
| 47 |
+
python -c 'import socket; s = socket.socket(socket.AF_INET, socket.SOCK_STREAM); s.bind(("", 0)); print(s.getsockname()[1]); s.close()'
|
| 48 |
+
}
|
| 49 |
+
MASTER_PORT=$(get_free_port)
|
| 50 |
+
export MASTER_ADDR=$MASTER_ADDR
|
| 51 |
+
export MASTER_PORT=$MASTER_PORT
|
| 52 |
+
|
| 53 |
+
# 16 卡训练(2 节点 × 8 DCU)
|
| 54 |
+
|
| 55 |
+
# 输入 JSON 使用环境变量,运行前展开为临时文件
|
| 56 |
+
INPUT_JSON="AuAg_nep_train_16card.json"
|
| 57 |
+
EXPANDED_JSON=".${INPUT_JSON%.json}_expanded_$$.json"
|
| 58 |
+
trap 'rm -f "$EXPANDED_JSON"' EXIT
|
| 59 |
+
python3 -c "import os; open('$EXPANDED_JSON','w').write(os.path.expandvars(open('$INPUT_JSON').read()))"
|
| 60 |
+
|
| 61 |
+
srun MatPL train "$EXPANDED_JSON"
|
demo/nep_AuAg/dcu_8/AuAg_nep_train_8card.json
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": ["Ag", "Au"],
|
| 4 |
+
"seed": 2023,
|
| 5 |
+
"recover_train": true,
|
| 6 |
+
"model": {
|
| 7 |
+
"descriptor": {
|
| 8 |
+
"cutoff": [6.0, 4.0],
|
| 9 |
+
"n_max": [16, 12],
|
| 10 |
+
"basis_size": [12, 8],
|
| 11 |
+
"l_max": [4, 2, 1]
|
| 12 |
+
},
|
| 13 |
+
"fitting_net": {
|
| 14 |
+
"network_size": [100, 1]
|
| 15 |
+
}
|
| 16 |
+
},
|
| 17 |
+
"optimizer": {
|
| 18 |
+
"optimizer": "ADAM",
|
| 19 |
+
"epochs": 30,
|
| 20 |
+
"batch_size": 64,
|
| 21 |
+
"print_freq": 10,
|
| 22 |
+
"learning_rate": 0.001,
|
| 23 |
+
"stop_lr": 3.51e-08,
|
| 24 |
+
"stop_step": 1000000,
|
| 25 |
+
"decay_step": 5000,
|
| 26 |
+
"train_energy": true,
|
| 27 |
+
"train_force": true,
|
| 28 |
+
"train_virial": true,
|
| 29 |
+
"start_pre_fac_force": 1000,
|
| 30 |
+
"start_pre_fac_etot": 0.02,
|
| 31 |
+
"start_pre_fac_virial": 50.0,
|
| 32 |
+
"end_pre_fac_force": 1.0,
|
| 33 |
+
"end_pre_fac_etot": 1.0,
|
| 34 |
+
"end_pre_fac_virial": 1.0,
|
| 35 |
+
"block_size": 5120,
|
| 36 |
+
"kalman_lambda": 0.98,
|
| 37 |
+
"kalman_nue": 0.9987,
|
| 38 |
+
"pre_fac_force": 2.0,
|
| 39 |
+
"pre_fac_etot": 1.0,
|
| 40 |
+
"pre_fac_virial": 1.0,
|
| 41 |
+
"max_norm": 0.5,
|
| 42 |
+
"norm_type": 2,
|
| 43 |
+
"t_0": 1,
|
| 44 |
+
"t_mult": 2,
|
| 45 |
+
"verbose": true,
|
| 46 |
+
"_clip_value": 1.0
|
| 47 |
+
},
|
| 48 |
+
"format": "extxyz",
|
| 49 |
+
"train_data": ["/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/AuAg-5762.xyz"],
|
| 50 |
+
"valid_data": ["/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/AuAg/AuAg-5762.xyz"]
|
| 51 |
+
}
|
demo/nep_AuAg/dcu_8/submit_8card.sh
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH --job-name=nep_AuAg_8card
|
| 3 |
+
#SBATCH --partition=hx1hdexclu12
|
| 4 |
+
#SBATCH --nodes=1
|
| 5 |
+
#SBATCH --ntasks-per-node=8
|
| 6 |
+
#SBATCH --gres=dcu:8
|
| 7 |
+
#SBATCH --cpus-per-task=4
|
| 8 |
+
#SBATCH --mem=256G
|
| 9 |
+
#SBATCH --time=2:00:00
|
| 10 |
+
#SBATCH --output=slurm_8card_%j.out
|
| 11 |
+
#SBATCH --error=slurm_8card_%j.err
|
| 12 |
+
|
| 13 |
+
if [[ -n "${SCRIPT_DIR}" ]]; then
|
| 14 |
+
:
|
| 15 |
+
elif [[ -n "${SLURM_SUBMIT_DIR}" ]]; then
|
| 16 |
+
SCRIPT_DIR="${SLURM_SUBMIT_DIR}"
|
| 17 |
+
else
|
| 18 |
+
echo "ERROR: 未检测到外部 SCRIPT_DIR 变量,也不在 Slurm 任务环境(SLURM_SUBMIT_DIR 为空)"
|
| 19 |
+
exit 1
|
| 20 |
+
fi
|
| 21 |
+
echo $SCRIPT_DIR
|
| 22 |
+
|
| 23 |
+
export MATCHEM_CONDA_NAME="${MATCHEM_CONDA_NAME:-test_pip}"
|
| 24 |
+
source "$SCRIPT_DIR/matchem_env.sh"
|
| 25 |
+
|
| 26 |
+
# MatPL 运行时环境(不侵入 matchem_env.sh,各算例自行维护)
|
| 27 |
+
# 校验MatPL根路径
|
| 28 |
+
if [[ -z "${MATPL_SRC_DIR}" || "${MATPL_SRC_DIR}" == "/path/to/matpl_dcu" || ! -d "${MATPL_SRC_DIR}" ]]; then
|
| 29 |
+
echo "=============================================="
|
| 30 |
+
echo "ERROR: MATPL_SRC_DIR 路径配置错误!"
|
| 31 |
+
echo "当前值:${MATPL_SRC_DIR}"
|
| 32 |
+
echo "请设置为真实有效的MatPL源码目录,示例:"
|
| 33 |
+
echo "export MATPL_SRC_DIR=/public/home/xxx/real_matpl_path"
|
| 34 |
+
echo "=============================================="
|
| 35 |
+
exit 1
|
| 36 |
+
fi
|
| 37 |
+
|
| 38 |
+
if [ -f "$MATPL_SRC_DIR/env.sh" ]; then
|
| 39 |
+
source "$MATPL_SRC_DIR/env.sh"
|
| 40 |
+
fi
|
| 41 |
+
echo "MATPL_SRC_DIR = $MATPL_SRC_DIR"
|
| 42 |
+
export LD_LIBRARY_PATH="$MATPL_SRC_DIR/src/op/build/lib:${LD_LIBRARY_PATH:-}"
|
| 43 |
+
|
| 44 |
+
# 8 卡训练
|
| 45 |
+
# 输入 JSON 使用环境变量,运行前展开为临时文件
|
| 46 |
+
INPUT_JSON="AuAg_nep_train_8card.json"
|
| 47 |
+
EXPANDED_JSON=".${INPUT_JSON%.json}_expanded_$$.json"
|
| 48 |
+
trap 'rm -f "$EXPANDED_JSON"' EXIT
|
| 49 |
+
python3 -c "import os; open('$EXPANDED_JSON','w').write(os.path.expandvars(open('$INPUT_JSON').read()))"
|
| 50 |
+
|
| 51 |
+
MatPL train "$EXPANDED_JSON"
|
demo/nep_Cu/Cu_nep_train.json
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": [
|
| 4 |
+
29
|
| 5 |
+
],
|
| 6 |
+
"seed": 2023,
|
| 7 |
+
"recover_train": true,
|
| 8 |
+
"model": {
|
| 9 |
+
"descriptor": {
|
| 10 |
+
"cutoff": [
|
| 11 |
+
6.0,
|
| 12 |
+
6.0
|
| 13 |
+
],
|
| 14 |
+
"n_max": [
|
| 15 |
+
4,
|
| 16 |
+
4
|
| 17 |
+
],
|
| 18 |
+
"basis_size": [
|
| 19 |
+
12,
|
| 20 |
+
12
|
| 21 |
+
],
|
| 22 |
+
"l_max": [
|
| 23 |
+
4,
|
| 24 |
+
2,
|
| 25 |
+
1
|
| 26 |
+
]
|
| 27 |
+
},
|
| 28 |
+
"fitting_net": {
|
| 29 |
+
"network_size": [
|
| 30 |
+
40,
|
| 31 |
+
1
|
| 32 |
+
]
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"optimizer": {
|
| 36 |
+
"optimizer": "ADAM",
|
| 37 |
+
"epochs": 30,
|
| 38 |
+
"batch_size": 1,
|
| 39 |
+
"print_freq": 10,
|
| 40 |
+
"learning_rate": 0.001,
|
| 41 |
+
"stop_lr": 3.51e-08,
|
| 42 |
+
"stop_step": 1000000,
|
| 43 |
+
"decay_step": 5000,
|
| 44 |
+
"train_energy": true,
|
| 45 |
+
"train_force": true,
|
| 46 |
+
"train_virial": false,
|
| 47 |
+
"start_pre_fac_force": 1000,
|
| 48 |
+
"start_pre_fac_etot": 0.02,
|
| 49 |
+
"start_pre_fac_virial": 50.0,
|
| 50 |
+
"end_pre_fac_force": 1.0,
|
| 51 |
+
"end_pre_fac_etot": 1.0,
|
| 52 |
+
"end_pre_fac_virial": 1.0
|
| 53 |
+
},
|
| 54 |
+
"format": "pwmat/movement",
|
| 55 |
+
"train_data": [
|
| 56 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/Cu/pwdata/0_300_MOVEMENT",
|
| 57 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/Cu/pwdata/1_500_MOVEMENT"
|
| 58 |
+
],
|
| 59 |
+
"valid_data": [
|
| 60 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/Cu/pwdata/valid_movement"
|
| 61 |
+
]
|
| 62 |
+
}
|
demo/nep_Cu/std_input.json
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": [
|
| 4 |
+
29
|
| 5 |
+
],
|
| 6 |
+
"seed": 2023,
|
| 7 |
+
"recover_train": true,
|
| 8 |
+
"model": {
|
| 9 |
+
"descriptor": {
|
| 10 |
+
"cutoff": [
|
| 11 |
+
6.0,
|
| 12 |
+
6.0
|
| 13 |
+
],
|
| 14 |
+
"n_max": [
|
| 15 |
+
4,
|
| 16 |
+
4
|
| 17 |
+
],
|
| 18 |
+
"basis_size": [
|
| 19 |
+
12,
|
| 20 |
+
12
|
| 21 |
+
],
|
| 22 |
+
"l_max": [
|
| 23 |
+
4,
|
| 24 |
+
2,
|
| 25 |
+
1
|
| 26 |
+
]
|
| 27 |
+
},
|
| 28 |
+
"fitting_net": {
|
| 29 |
+
"network_size": [
|
| 30 |
+
40,
|
| 31 |
+
1
|
| 32 |
+
]
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"optimizer": {
|
| 36 |
+
"optimizer": "ADAM",
|
| 37 |
+
"epochs": 30,
|
| 38 |
+
"batch_size": 1,
|
| 39 |
+
"print_freq": 10,
|
| 40 |
+
"learning_rate": 0.001,
|
| 41 |
+
"stop_lr": 3.51e-08,
|
| 42 |
+
"stop_step": 1000000,
|
| 43 |
+
"decay_step": 5000,
|
| 44 |
+
"train_energy": true,
|
| 45 |
+
"train_force": true,
|
| 46 |
+
"train_virial": false,
|
| 47 |
+
"start_pre_fac_force": 1000,
|
| 48 |
+
"start_pre_fac_etot": 0.02,
|
| 49 |
+
"start_pre_fac_virial": 50.0,
|
| 50 |
+
"end_pre_fac_force": 1.0,
|
| 51 |
+
"end_pre_fac_etot": 1.0,
|
| 52 |
+
"end_pre_fac_virial": 1.0
|
| 53 |
+
},
|
| 54 |
+
"format": "pwmat/movement",
|
| 55 |
+
"train_data": [
|
| 56 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/Cu/pwdata/0_300_MOVEMENT",
|
| 57 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/Cu/pwdata/1_500_MOVEMENT"
|
| 58 |
+
],
|
| 59 |
+
"valid_data": [
|
| 60 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/Cu/pwdata/valid_movement"
|
| 61 |
+
]
|
| 62 |
+
}
|
demo/nep_Cu/submit.sh
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH --job-name=nep_Cu_train
|
| 3 |
+
#SBATCH --partition=hx1hdexclu12
|
| 4 |
+
#SBATCH --nodes=1
|
| 5 |
+
#SBATCH --ntasks-per-node=1
|
| 6 |
+
#SBATCH --gres=dcu:1
|
| 7 |
+
#SBATCH --cpus-per-task=16
|
| 8 |
+
#SBATCH --time=2:00:00
|
| 9 |
+
#SBATCH --output=slurm_%j.out
|
| 10 |
+
#SBATCH --error=slurm_%j.err
|
| 11 |
+
|
| 12 |
+
if [[ -n "${SCRIPT_DIR}" ]]; then
|
| 13 |
+
:
|
| 14 |
+
elif [[ -n "${SLURM_SUBMIT_DIR}" ]]; then
|
| 15 |
+
SCRIPT_DIR="${SLURM_SUBMIT_DIR}"
|
| 16 |
+
else
|
| 17 |
+
echo "ERROR: 未检测到外部 SCRIPT_DIR 变量,也不在 Slurm 任务环境(SLURM_SUBMIT_DIR 为空)"
|
| 18 |
+
exit 1
|
| 19 |
+
fi
|
| 20 |
+
echo $SCRIPT_DIR
|
| 21 |
+
|
| 22 |
+
export MATCHEM_CONDA_NAME="${MATCHEM_CONDA_NAME:-test_pip}"
|
| 23 |
+
source "$SCRIPT_DIR/matchem_env.sh"
|
| 24 |
+
|
| 25 |
+
# MatPL 运行时环境(不侵入 matchem_env.sh,各算例自行维护)
|
| 26 |
+
# 校验MatPL根路径
|
| 27 |
+
if [[ -z "${MATPL_SRC_DIR}" || "${MATPL_SRC_DIR}" == "/path/to/matpl_dcu" || ! -d "${MATPL_SRC_DIR}" ]]; then
|
| 28 |
+
echo "=============================================="
|
| 29 |
+
echo "ERROR: MATPL_SRC_DIR 路径配置错误!"
|
| 30 |
+
echo "当前值:${MATPL_SRC_DIR}"
|
| 31 |
+
echo "请设置为真实有效的MatPL源码目录,示例:"
|
| 32 |
+
echo "export MATPL_SRC_DIR=/public/home/xxx/real_matpl_path"
|
| 33 |
+
echo "=============================================="
|
| 34 |
+
exit 1
|
| 35 |
+
fi
|
| 36 |
+
|
| 37 |
+
if [ -f "$MATPL_SRC_DIR/env.sh" ]; then
|
| 38 |
+
source "$MATPL_SRC_DIR/env.sh"
|
| 39 |
+
fi
|
| 40 |
+
echo "MATPL_SRC_DIR = $MATPL_SRC_DIR"
|
| 41 |
+
export LD_LIBRARY_PATH="$MATPL_SRC_DIR/src/op/build/lib:${LD_LIBRARY_PATH:-}"
|
| 42 |
+
|
| 43 |
+
# 输入 JSON 使用环境变量,运行前展开为临时文件
|
| 44 |
+
INPUT_JSON="Cu_nep_train.json"
|
| 45 |
+
EXPANDED_JSON=".${INPUT_JSON%.json}_expanded_$$.json"
|
| 46 |
+
trap 'rm -f "$EXPANDED_JSON"' EXIT
|
| 47 |
+
python3 -c "import os; open('$EXPANDED_JSON','w').write(os.path.expandvars(open('$INPUT_JSON').read()))"
|
| 48 |
+
|
| 49 |
+
MatPL train "$EXPANDED_JSON"
|
demo/nep_HfO2/dcu_1/HfO2_nep_test.json
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": [
|
| 4 |
+
8,
|
| 5 |
+
72
|
| 6 |
+
],
|
| 7 |
+
"seed": 2023,
|
| 8 |
+
"recover_train": true,
|
| 9 |
+
"model": {
|
| 10 |
+
"descriptor": {
|
| 11 |
+
"cutoff": [
|
| 12 |
+
6.0,
|
| 13 |
+
6.0
|
| 14 |
+
],
|
| 15 |
+
"n_max": [
|
| 16 |
+
4,
|
| 17 |
+
4
|
| 18 |
+
],
|
| 19 |
+
"basis_size": [
|
| 20 |
+
12,
|
| 21 |
+
12
|
| 22 |
+
],
|
| 23 |
+
"l_max": [
|
| 24 |
+
4,
|
| 25 |
+
2,
|
| 26 |
+
1
|
| 27 |
+
]
|
| 28 |
+
},
|
| 29 |
+
"fitting_net": {
|
| 30 |
+
"network_size": [
|
| 31 |
+
40,
|
| 32 |
+
1
|
| 33 |
+
]
|
| 34 |
+
}
|
| 35 |
+
},
|
| 36 |
+
"optimizer": {
|
| 37 |
+
"optimizer": "ADAM",
|
| 38 |
+
"epochs": 30,
|
| 39 |
+
"batch_size": 1,
|
| 40 |
+
"print_freq": 10,
|
| 41 |
+
"learning_rate": 0.001,
|
| 42 |
+
"stop_lr": 3.51e-08,
|
| 43 |
+
"stop_step": 1000000,
|
| 44 |
+
"decay_step": 5000,
|
| 45 |
+
"train_energy": true,
|
| 46 |
+
"train_force": true,
|
| 47 |
+
"train_virial": false,
|
| 48 |
+
"start_pre_fac_force": 1000,
|
| 49 |
+
"start_pre_fac_etot": 0.02,
|
| 50 |
+
"start_pre_fac_virial": 50.0,
|
| 51 |
+
"end_pre_fac_force": 1.0,
|
| 52 |
+
"end_pre_fac_etot": 1.0,
|
| 53 |
+
"end_pre_fac_virial": 1.0
|
| 54 |
+
},
|
| 55 |
+
"format": "pwmlff/npy",
|
| 56 |
+
"model_load_file": "./model_record/nep_model.ckpt",
|
| 57 |
+
"test_data": [
|
| 58 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_000_50",
|
| 59 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_004_50",
|
| 60 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_008_50",
|
| 61 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_012_50",
|
| 62 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_016_50",
|
| 63 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_020_20",
|
| 64 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_024_20",
|
| 65 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_001_50",
|
| 66 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_005_50",
|
| 67 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_009_50",
|
| 68 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_013_50",
|
| 69 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_017_50",
|
| 70 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_021_20",
|
| 71 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_025_20",
|
| 72 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_002_50",
|
| 73 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_006_50",
|
| 74 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_010_50",
|
| 75 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_014_50",
|
| 76 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_018_50",
|
| 77 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_022_20",
|
| 78 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_026_20",
|
| 79 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_003_50",
|
| 80 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_007_50",
|
| 81 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_011_50",
|
| 82 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_015_30",
|
| 83 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_019_50",
|
| 84 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_023_20",
|
| 85 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_027_20"
|
| 86 |
+
]
|
| 87 |
+
}
|
demo/nep_HfO2/dcu_1/HfO2_nep_train.json
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": [
|
| 4 |
+
8,
|
| 5 |
+
72
|
| 6 |
+
],
|
| 7 |
+
"seed": 2023,
|
| 8 |
+
"recover_train": true,
|
| 9 |
+
"model": {
|
| 10 |
+
"descriptor": {
|
| 11 |
+
"cutoff": [
|
| 12 |
+
6.0,
|
| 13 |
+
6.0
|
| 14 |
+
],
|
| 15 |
+
"n_max": [
|
| 16 |
+
4,
|
| 17 |
+
4
|
| 18 |
+
],
|
| 19 |
+
"basis_size": [
|
| 20 |
+
12,
|
| 21 |
+
12
|
| 22 |
+
],
|
| 23 |
+
"l_max": [
|
| 24 |
+
4,
|
| 25 |
+
2,
|
| 26 |
+
1
|
| 27 |
+
]
|
| 28 |
+
},
|
| 29 |
+
"fitting_net": {
|
| 30 |
+
"network_size": [
|
| 31 |
+
40,
|
| 32 |
+
1
|
| 33 |
+
]
|
| 34 |
+
}
|
| 35 |
+
},
|
| 36 |
+
"optimizer": {
|
| 37 |
+
"optimizer": "ADAM",
|
| 38 |
+
"epochs": 30,
|
| 39 |
+
"batch_size": 1,
|
| 40 |
+
"print_freq": 10,
|
| 41 |
+
"learning_rate": 0.001,
|
| 42 |
+
"stop_lr": 3.51e-08,
|
| 43 |
+
"stop_step": 1000000,
|
| 44 |
+
"decay_step": 5000,
|
| 45 |
+
"train_energy": true,
|
| 46 |
+
"train_force": true,
|
| 47 |
+
"train_virial": false,
|
| 48 |
+
"start_pre_fac_force": 1000,
|
| 49 |
+
"start_pre_fac_etot": 0.02,
|
| 50 |
+
"start_pre_fac_virial": 50.0,
|
| 51 |
+
"end_pre_fac_force": 1.0,
|
| 52 |
+
"end_pre_fac_etot": 1.0,
|
| 53 |
+
"end_pre_fac_virial": 1.0
|
| 54 |
+
},
|
| 55 |
+
"format": "pwmlff/npy",
|
| 56 |
+
"train_data": [
|
| 57 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_000_50/",
|
| 58 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_002_50/",
|
| 59 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_004_50/",
|
| 60 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_006_50/",
|
| 61 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_008_50/",
|
| 62 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_010_50/",
|
| 63 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_012_50/",
|
| 64 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_014_50/",
|
| 65 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_016_50/",
|
| 66 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_018_50/",
|
| 67 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_020_20/",
|
| 68 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_022_20/",
|
| 69 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_024_20/",
|
| 70 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_026_20/",
|
| 71 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_001_50/",
|
| 72 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_003_50/",
|
| 73 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_005_50/",
|
| 74 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_007_50/",
|
| 75 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_009_50/",
|
| 76 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_011_50/",
|
| 77 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_013_50/",
|
| 78 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_015_30/",
|
| 79 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_017_50/",
|
| 80 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_019_50/",
|
| 81 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_021_20/",
|
| 82 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_023_20/",
|
| 83 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_025_20/",
|
| 84 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_027_20/"
|
| 85 |
+
],
|
| 86 |
+
"valid_data": [
|
| 87 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_000_50/",
|
| 88 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_004_50/",
|
| 89 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_008_50/"
|
| 90 |
+
]
|
| 91 |
+
}
|
demo/nep_HfO2/dcu_1/submit.sh
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH --job-name=nep_HfO2_train
|
| 3 |
+
#SBATCH --partition=hx1hdexclu12
|
| 4 |
+
#SBATCH --nodes=1
|
| 5 |
+
#SBATCH --ntasks-per-node=1
|
| 6 |
+
#SBATCH --gres=dcu:1
|
| 7 |
+
#SBATCH --cpus-per-task=16
|
| 8 |
+
#SBATCH --time=2:00:00
|
| 9 |
+
#SBATCH --output=slurm_%j.out
|
| 10 |
+
#SBATCH --error=slurm_%j.err
|
| 11 |
+
|
| 12 |
+
if [[ -n "${SCRIPT_DIR}" ]]; then
|
| 13 |
+
:
|
| 14 |
+
elif [[ -n "${SLURM_SUBMIT_DIR}" ]]; then
|
| 15 |
+
SCRIPT_DIR="${SLURM_SUBMIT_DIR}"
|
| 16 |
+
else
|
| 17 |
+
echo "ERROR: 未检测到外部 SCRIPT_DIR 变量,也不在 Slurm 任务环境(SLURM_SUBMIT_DIR 为空)"
|
| 18 |
+
exit 1
|
| 19 |
+
fi
|
| 20 |
+
echo $SCRIPT_DIR
|
| 21 |
+
|
| 22 |
+
export MATCHEM_CONDA_NAME="${MATCHEM_CONDA_NAME:-test_pip}"
|
| 23 |
+
source "$SCRIPT_DIR/matchem_env.sh"
|
| 24 |
+
|
| 25 |
+
# MatPL 运行时环境(不侵入 matchem_env.sh,各算例自行维护)
|
| 26 |
+
# 校验MatPL根路径
|
| 27 |
+
if [[ -z "${MATPL_SRC_DIR}" || "${MATPL_SRC_DIR}" == "/path/to/matpl_dcu" || ! -d "${MATPL_SRC_DIR}" ]]; then
|
| 28 |
+
echo "=============================================="
|
| 29 |
+
echo "ERROR: MATPL_SRC_DIR 路径配置错误!"
|
| 30 |
+
echo "当前值:${MATPL_SRC_DIR}"
|
| 31 |
+
echo "请设置为真实有效的MatPL源码目录,示例:"
|
| 32 |
+
echo "export MATPL_SRC_DIR=/public/home/xxx/real_matpl_path"
|
| 33 |
+
echo "=============================================="
|
| 34 |
+
exit 1
|
| 35 |
+
fi
|
| 36 |
+
|
| 37 |
+
if [ -f "$MATPL_SRC_DIR/env.sh" ]; then
|
| 38 |
+
source "$MATPL_SRC_DIR/env.sh"
|
| 39 |
+
fi
|
| 40 |
+
echo "MATPL_SRC_DIR = $MATPL_SRC_DIR"
|
| 41 |
+
export LD_LIBRARY_PATH="$MATPL_SRC_DIR/src/op/build/lib:${LD_LIBRARY_PATH:-}"
|
| 42 |
+
# 输入 JSON 使用环境变量,运行前展开为临时文件
|
| 43 |
+
INPUT_JSON="HfO2_nep_train.json"
|
| 44 |
+
EXPANDED_JSON=".${INPUT_JSON%.json}_expanded_$$.json"
|
| 45 |
+
trap 'rm -f "$EXPANDED_JSON"' EXIT
|
| 46 |
+
python3 -c "import os; open('$EXPANDED_JSON','w').write(os.path.expandvars(open('$INPUT_JSON').read()))"
|
| 47 |
+
|
| 48 |
+
MatPL train "$EXPANDED_JSON"
|
demo/nep_HfO2/dcu_16/HfO2_nep_train_16card.json
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": [
|
| 4 |
+
8,
|
| 5 |
+
72
|
| 6 |
+
],
|
| 7 |
+
"seed": 2023,
|
| 8 |
+
"recover_train": true,
|
| 9 |
+
"model": {
|
| 10 |
+
"descriptor": {
|
| 11 |
+
"cutoff": [
|
| 12 |
+
6.0,
|
| 13 |
+
6.0
|
| 14 |
+
],
|
| 15 |
+
"n_max": [
|
| 16 |
+
4,
|
| 17 |
+
4
|
| 18 |
+
],
|
| 19 |
+
"basis_size": [
|
| 20 |
+
12,
|
| 21 |
+
12
|
| 22 |
+
],
|
| 23 |
+
"l_max": [
|
| 24 |
+
4,
|
| 25 |
+
2,
|
| 26 |
+
1
|
| 27 |
+
]
|
| 28 |
+
},
|
| 29 |
+
"fitting_net": {
|
| 30 |
+
"network_size": [
|
| 31 |
+
40,
|
| 32 |
+
1
|
| 33 |
+
]
|
| 34 |
+
}
|
| 35 |
+
},
|
| 36 |
+
"optimizer": {
|
| 37 |
+
"optimizer": "ADAM",
|
| 38 |
+
"epochs": 30,
|
| 39 |
+
"batch_size": 16,
|
| 40 |
+
"print_freq": 10,
|
| 41 |
+
"learning_rate": 0.001,
|
| 42 |
+
"stop_lr": 3.51e-08,
|
| 43 |
+
"stop_step": 1000000,
|
| 44 |
+
"decay_step": 5000,
|
| 45 |
+
"train_energy": true,
|
| 46 |
+
"train_force": true,
|
| 47 |
+
"train_virial": false,
|
| 48 |
+
"start_pre_fac_force": 1000,
|
| 49 |
+
"start_pre_fac_etot": 0.02,
|
| 50 |
+
"start_pre_fac_virial": 50.0,
|
| 51 |
+
"end_pre_fac_force": 1.0,
|
| 52 |
+
"end_pre_fac_etot": 1.0,
|
| 53 |
+
"end_pre_fac_virial": 1.0
|
| 54 |
+
},
|
| 55 |
+
"format": "pwmlff/npy",
|
| 56 |
+
"train_data": [
|
| 57 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_000_50/",
|
| 58 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_002_50/",
|
| 59 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_004_50/",
|
| 60 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_006_50/",
|
| 61 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_008_50/",
|
| 62 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_010_50/",
|
| 63 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_012_50/",
|
| 64 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_014_50/",
|
| 65 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_016_50/",
|
| 66 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_018_50/",
|
| 67 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_020_20/",
|
| 68 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_022_20/",
|
| 69 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_024_20/",
|
| 70 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_026_20/",
|
| 71 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_001_50/",
|
| 72 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_003_50/",
|
| 73 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_005_50/",
|
| 74 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_007_50/",
|
| 75 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_009_50/",
|
| 76 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_011_50/",
|
| 77 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_013_50/",
|
| 78 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_015_30/",
|
| 79 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_017_50/",
|
| 80 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_019_50/",
|
| 81 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_021_20/",
|
| 82 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_023_20/",
|
| 83 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_025_20/",
|
| 84 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_027_20/"
|
| 85 |
+
],
|
| 86 |
+
"valid_data": [
|
| 87 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_000_50/",
|
| 88 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_004_50/",
|
| 89 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_008_50/"
|
| 90 |
+
]
|
| 91 |
+
}
|
demo/nep_HfO2/dcu_16/submit_16card.sh
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH --job-name=nep_HfO2_16card
|
| 3 |
+
#SBATCH --partition=hx1hdexclu12
|
| 4 |
+
#SBATCH --nodes=2
|
| 5 |
+
#SBATCH --ntasks-per-node=8
|
| 6 |
+
#SBATCH --gres=dcu:8
|
| 7 |
+
#SBATCH --cpus-per-task=4
|
| 8 |
+
#SBATCH --mem=256G
|
| 9 |
+
#SBATCH --time=2:00:00
|
| 10 |
+
#SBATCH --output=slurm_16card_%j.out
|
| 11 |
+
#SBATCH --error=slurm_16card_%j.err
|
| 12 |
+
|
| 13 |
+
if [[ -n "${SCRIPT_DIR}" ]]; then
|
| 14 |
+
:
|
| 15 |
+
elif [[ -n "${SLURM_SUBMIT_DIR}" ]]; then
|
| 16 |
+
SCRIPT_DIR="${SLURM_SUBMIT_DIR}"
|
| 17 |
+
else
|
| 18 |
+
echo "ERROR: 未检测到外部 SCRIPT_DIR 变量,也不在 Slurm 任务环境(SLURM_SUBMIT_DIR 为空)"
|
| 19 |
+
exit 1
|
| 20 |
+
fi
|
| 21 |
+
echo $SCRIPT_DIR
|
| 22 |
+
|
| 23 |
+
export MATCHEM_CONDA_NAME="${MATCHEM_CONDA_NAME:-test_pip}"
|
| 24 |
+
source "$SCRIPT_DIR/matchem_env.sh"
|
| 25 |
+
|
| 26 |
+
# MatPL 运行时环境(不侵入 matchem_env.sh,各算例自行维护)
|
| 27 |
+
# 校验MatPL根路径
|
| 28 |
+
if [[ -z "${MATPL_SRC_DIR}" || "${MATPL_SRC_DIR}" == "/path/to/matpl_dcu" || ! -d "${MATPL_SRC_DIR}" ]]; then
|
| 29 |
+
echo "=============================================="
|
| 30 |
+
echo "ERROR: MATPL_SRC_DIR 路径配置错误!"
|
| 31 |
+
echo "当前值:${MATPL_SRC_DIR}"
|
| 32 |
+
echo "请设置为真实有效的MatPL源码目录,示例:"
|
| 33 |
+
echo "export MATPL_SRC_DIR=/public/home/xxx/real_matpl_path"
|
| 34 |
+
echo "=============================================="
|
| 35 |
+
exit 1
|
| 36 |
+
fi
|
| 37 |
+
|
| 38 |
+
if [ -f "$MATPL_SRC_DIR/env.sh" ]; then
|
| 39 |
+
source "$MATPL_SRC_DIR/env.sh"
|
| 40 |
+
fi
|
| 41 |
+
echo "MATPL_SRC_DIR = $MATPL_SRC_DIR"
|
| 42 |
+
export LD_LIBRARY_PATH="$MATPL_SRC_DIR/src/op/build/lib:${LD_LIBRARY_PATH:-}"
|
| 43 |
+
|
| 44 |
+
# 多节点通信设置
|
| 45 |
+
MASTER_ADDR=$(scontrol show hostnames $SLURM_JOB_NODELIST | head -n 1)
|
| 46 |
+
function get_free_port() {
|
| 47 |
+
python -c 'import socket; s = socket.socket(socket.AF_INET, socket.SOCK_STREAM); s.bind(("", 0)); print(s.getsockname()[1]); s.close()'
|
| 48 |
+
}
|
| 49 |
+
MASTER_PORT=$(get_free_port)
|
| 50 |
+
export MASTER_ADDR=$MASTER_ADDR
|
| 51 |
+
export MASTER_PORT=$MASTER_PORT
|
| 52 |
+
|
| 53 |
+
# 16 卡训练(2 节点 × 8 DCU)
|
| 54 |
+
|
| 55 |
+
# 输入 JSON 使用环境变量,运行前展开为临时文件
|
| 56 |
+
INPUT_JSON="HfO2_nep_train_16card.json"
|
| 57 |
+
EXPANDED_JSON=".${INPUT_JSON%.json}_expanded_$$.json"
|
| 58 |
+
trap 'rm -f "$EXPANDED_JSON"' EXIT
|
| 59 |
+
python3 -c "import os; open('$EXPANDED_JSON','w').write(os.path.expandvars(open('$INPUT_JSON').read()))"
|
| 60 |
+
|
| 61 |
+
srun MatPL train "$EXPANDED_JSON"
|
demo/nep_HfO2/dcu_8/HfO2_nep_train_8card.json
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": [
|
| 4 |
+
8,
|
| 5 |
+
72
|
| 6 |
+
],
|
| 7 |
+
"seed": 2023,
|
| 8 |
+
"recover_train": true,
|
| 9 |
+
"model": {
|
| 10 |
+
"descriptor": {
|
| 11 |
+
"cutoff": [
|
| 12 |
+
6.0,
|
| 13 |
+
6.0
|
| 14 |
+
],
|
| 15 |
+
"n_max": [
|
| 16 |
+
4,
|
| 17 |
+
4
|
| 18 |
+
],
|
| 19 |
+
"basis_size": [
|
| 20 |
+
12,
|
| 21 |
+
12
|
| 22 |
+
],
|
| 23 |
+
"l_max": [
|
| 24 |
+
4,
|
| 25 |
+
2,
|
| 26 |
+
1
|
| 27 |
+
]
|
| 28 |
+
},
|
| 29 |
+
"fitting_net": {
|
| 30 |
+
"network_size": [
|
| 31 |
+
40,
|
| 32 |
+
1
|
| 33 |
+
]
|
| 34 |
+
}
|
| 35 |
+
},
|
| 36 |
+
"optimizer": {
|
| 37 |
+
"optimizer": "ADAM",
|
| 38 |
+
"epochs": 30,
|
| 39 |
+
"batch_size": 8,
|
| 40 |
+
"print_freq": 10,
|
| 41 |
+
"learning_rate": 0.001,
|
| 42 |
+
"stop_lr": 3.51e-08,
|
| 43 |
+
"stop_step": 1000000,
|
| 44 |
+
"decay_step": 5000,
|
| 45 |
+
"train_energy": true,
|
| 46 |
+
"train_force": true,
|
| 47 |
+
"train_virial": false,
|
| 48 |
+
"start_pre_fac_force": 1000,
|
| 49 |
+
"start_pre_fac_etot": 0.02,
|
| 50 |
+
"start_pre_fac_virial": 50.0,
|
| 51 |
+
"end_pre_fac_force": 1.0,
|
| 52 |
+
"end_pre_fac_etot": 1.0,
|
| 53 |
+
"end_pre_fac_virial": 1.0
|
| 54 |
+
},
|
| 55 |
+
"format": "pwmlff/npy",
|
| 56 |
+
"train_data": [
|
| 57 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_000_50/",
|
| 58 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_002_50/",
|
| 59 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_004_50/",
|
| 60 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_006_50/",
|
| 61 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_008_50/",
|
| 62 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_010_50/",
|
| 63 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_012_50/",
|
| 64 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_014_50/",
|
| 65 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_016_50/",
|
| 66 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_018_50/",
|
| 67 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_020_20/",
|
| 68 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_022_20/",
|
| 69 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_024_20/",
|
| 70 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_026_20/",
|
| 71 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_001_50/",
|
| 72 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_003_50/",
|
| 73 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_005_50/",
|
| 74 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_007_50/",
|
| 75 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_009_50/",
|
| 76 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_011_50/",
|
| 77 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_013_50/",
|
| 78 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_015_30/",
|
| 79 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_017_50/",
|
| 80 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_019_50/",
|
| 81 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_021_20/",
|
| 82 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_023_20/",
|
| 83 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_025_20/",
|
| 84 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_027_20/"
|
| 85 |
+
],
|
| 86 |
+
"valid_data": [
|
| 87 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_000_50/",
|
| 88 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_004_50/",
|
| 89 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/HfO2/pwdata/init_008_50/"
|
| 90 |
+
]
|
| 91 |
+
}
|
demo/nep_HfO2/dcu_8/submit_8card.sh
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH --job-name=nep_HfO2_8card
|
| 3 |
+
#SBATCH --partition=hx1hdexclu12
|
| 4 |
+
#SBATCH --nodes=1
|
| 5 |
+
#SBATCH --ntasks-per-node=8
|
| 6 |
+
#SBATCH --gres=dcu:8
|
| 7 |
+
#SBATCH --cpus-per-task=4
|
| 8 |
+
#SBATCH --mem=400G
|
| 9 |
+
#SBATCH --time=2:00:00
|
| 10 |
+
#SBATCH --output=slurm_8card_%j.out
|
| 11 |
+
#SBATCH --error=slurm_8card_%j.err
|
| 12 |
+
|
| 13 |
+
if [[ -n "${SCRIPT_DIR}" ]]; then
|
| 14 |
+
:
|
| 15 |
+
elif [[ -n "${SLURM_SUBMIT_DIR}" ]]; then
|
| 16 |
+
SCRIPT_DIR="${SLURM_SUBMIT_DIR}"
|
| 17 |
+
else
|
| 18 |
+
echo "ERROR: 未检测到外部 SCRIPT_DIR 变量,也不在 Slurm 任务环境(SLURM_SUBMIT_DIR 为空)"
|
| 19 |
+
exit 1
|
| 20 |
+
fi
|
| 21 |
+
echo $SCRIPT_DIR
|
| 22 |
+
|
| 23 |
+
export MATCHEM_CONDA_NAME="${MATCHEM_CONDA_NAME:-test_pip}"
|
| 24 |
+
source "$SCRIPT_DIR/matchem_env.sh"
|
| 25 |
+
|
| 26 |
+
# MatPL 运行时环境(不侵入 matchem_env.sh,各算例自行维护)
|
| 27 |
+
# 校验MatPL根路径
|
| 28 |
+
if [[ -z "${MATPL_SRC_DIR}" || "${MATPL_SRC_DIR}" == "/path/to/matpl_dcu" || ! -d "${MATPL_SRC_DIR}" ]]; then
|
| 29 |
+
echo "=============================================="
|
| 30 |
+
echo "ERROR: MATPL_SRC_DIR 路径配置错误!"
|
| 31 |
+
echo "当前值:${MATPL_SRC_DIR}"
|
| 32 |
+
echo "请设置为真实有效的MatPL源码目录,示例:"
|
| 33 |
+
echo "export MATPL_SRC_DIR=/public/home/xxx/real_matpl_path"
|
| 34 |
+
echo "=============================================="
|
| 35 |
+
exit 1
|
| 36 |
+
fi
|
| 37 |
+
|
| 38 |
+
if [ -f "$MATPL_SRC_DIR/env.sh" ]; then
|
| 39 |
+
source "$MATPL_SRC_DIR/env.sh"
|
| 40 |
+
fi
|
| 41 |
+
echo "MATPL_SRC_DIR = $MATPL_SRC_DIR"
|
| 42 |
+
export LD_LIBRARY_PATH="$MATPL_SRC_DIR/src/op/build/lib:${LD_LIBRARY_PATH:-}"
|
| 43 |
+
|
| 44 |
+
# 输入 JSON 使用环境变量,运行前展开为临时文件
|
| 45 |
+
INPUT_JSON="HfO2_nep_train_8card.json"
|
| 46 |
+
EXPANDED_JSON=".${INPUT_JSON%.json}_expanded_$$.json"
|
| 47 |
+
trap 'rm -f "$EXPANDED_JSON"' EXIT
|
| 48 |
+
python3 -c "import os; open('$EXPANDED_JSON','w').write(os.path.expandvars(open('$INPUT_JSON').read()))"
|
| 49 |
+
|
| 50 |
+
MatPL train "$EXPANDED_JSON"
|
demo/nep_LiSiC/LiSiC_nep_train.json
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": [
|
| 4 |
+
3,
|
| 5 |
+
14,
|
| 6 |
+
6
|
| 7 |
+
],
|
| 8 |
+
"seed": 2023,
|
| 9 |
+
"recover_train": true,
|
| 10 |
+
"model": {
|
| 11 |
+
"descriptor": {
|
| 12 |
+
"cutoff": [
|
| 13 |
+
6.0,
|
| 14 |
+
6.0
|
| 15 |
+
],
|
| 16 |
+
"n_max": [
|
| 17 |
+
4,
|
| 18 |
+
4
|
| 19 |
+
],
|
| 20 |
+
"basis_size": [
|
| 21 |
+
12,
|
| 22 |
+
12
|
| 23 |
+
],
|
| 24 |
+
"l_max": [
|
| 25 |
+
4,
|
| 26 |
+
2,
|
| 27 |
+
1
|
| 28 |
+
]
|
| 29 |
+
},
|
| 30 |
+
"fitting_net": {
|
| 31 |
+
"network_size": [
|
| 32 |
+
40,
|
| 33 |
+
1
|
| 34 |
+
]
|
| 35 |
+
}
|
| 36 |
+
},
|
| 37 |
+
"optimizer": {
|
| 38 |
+
"optimizer": "ADAM",
|
| 39 |
+
"epochs": 30,
|
| 40 |
+
"batch_size": 1,
|
| 41 |
+
"print_freq": 10,
|
| 42 |
+
"learning_rate": 0.001,
|
| 43 |
+
"stop_lr": 3.51e-08,
|
| 44 |
+
"stop_step": 1000000,
|
| 45 |
+
"decay_step": 5000,
|
| 46 |
+
"train_energy": true,
|
| 47 |
+
"train_force": true,
|
| 48 |
+
"train_virial": false,
|
| 49 |
+
"start_pre_fac_force": 1000,
|
| 50 |
+
"start_pre_fac_etot": 0.02,
|
| 51 |
+
"start_pre_fac_virial": 50.0,
|
| 52 |
+
"end_pre_fac_force": 1.0,
|
| 53 |
+
"end_pre_fac_etot": 1.0,
|
| 54 |
+
"end_pre_fac_virial": 1.0
|
| 55 |
+
},
|
| 56 |
+
"format": "pwmlff/npy",
|
| 57 |
+
"train_data": [
|
| 58 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/C2",
|
| 59 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/C448",
|
| 60 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/C448Li75",
|
| 61 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/C64Si32",
|
| 62 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Li1Si24",
|
| 63 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Li3Si8",
|
| 64 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Li8",
|
| 65 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Li88Si20",
|
| 66 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Si1",
|
| 67 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Si217"
|
| 68 |
+
],
|
| 69 |
+
"valid_data": [
|
| 70 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/C2"
|
| 71 |
+
]
|
| 72 |
+
}
|
demo/nep_LiSiC/std_input.json
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "NEP",
|
| 3 |
+
"atom_type": [
|
| 4 |
+
3,
|
| 5 |
+
14,
|
| 6 |
+
6
|
| 7 |
+
],
|
| 8 |
+
"seed": 2023,
|
| 9 |
+
"recover_train": true,
|
| 10 |
+
"model": {
|
| 11 |
+
"descriptor": {
|
| 12 |
+
"cutoff": [
|
| 13 |
+
6.0,
|
| 14 |
+
6.0
|
| 15 |
+
],
|
| 16 |
+
"n_max": [
|
| 17 |
+
4,
|
| 18 |
+
4
|
| 19 |
+
],
|
| 20 |
+
"basis_size": [
|
| 21 |
+
12,
|
| 22 |
+
12
|
| 23 |
+
],
|
| 24 |
+
"l_max": [
|
| 25 |
+
4,
|
| 26 |
+
2,
|
| 27 |
+
1
|
| 28 |
+
]
|
| 29 |
+
},
|
| 30 |
+
"fitting_net": {
|
| 31 |
+
"network_size": [
|
| 32 |
+
40,
|
| 33 |
+
1
|
| 34 |
+
]
|
| 35 |
+
}
|
| 36 |
+
},
|
| 37 |
+
"optimizer": {
|
| 38 |
+
"optimizer": "ADAM",
|
| 39 |
+
"epochs": 30,
|
| 40 |
+
"batch_size": 1,
|
| 41 |
+
"print_freq": 10,
|
| 42 |
+
"learning_rate": 0.001,
|
| 43 |
+
"stop_lr": 3.51e-08,
|
| 44 |
+
"stop_step": 1000000,
|
| 45 |
+
"decay_step": 5000,
|
| 46 |
+
"train_energy": true,
|
| 47 |
+
"train_force": true,
|
| 48 |
+
"train_virial": false,
|
| 49 |
+
"start_pre_fac_force": 1000,
|
| 50 |
+
"start_pre_fac_etot": 0.02,
|
| 51 |
+
"start_pre_fac_virial": 50.0,
|
| 52 |
+
"end_pre_fac_force": 1.0,
|
| 53 |
+
"end_pre_fac_etot": 1.0,
|
| 54 |
+
"end_pre_fac_virial": 1.0
|
| 55 |
+
},
|
| 56 |
+
"format": "pwmlff/npy",
|
| 57 |
+
"train_data": [
|
| 58 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/C2",
|
| 59 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/C448",
|
| 60 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/C448Li75",
|
| 61 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/C64Si32",
|
| 62 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Li1Si24",
|
| 63 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Li3Si8",
|
| 64 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Li8",
|
| 65 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Li88Si20",
|
| 66 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Si1",
|
| 67 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/Si217"
|
| 68 |
+
],
|
| 69 |
+
"valid_data": [
|
| 70 |
+
"/public/share/sugonhpcapp01/onestore/onedatasets/matchem/matpl/LiSiC/pwdata/C2"
|
| 71 |
+
]
|
| 72 |
+
}
|
demo/nep_LiSiC/submit.sh
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH --job-name=nep_LiSiC_train
|
| 3 |
+
#SBATCH --partition=hx1hdexclu12
|
| 4 |
+
#SBATCH --nodes=1
|
| 5 |
+
#SBATCH --ntasks-per-node=1
|
| 6 |
+
#SBATCH --gres=dcu:1
|
| 7 |
+
#SBATCH --cpus-per-task=16
|
| 8 |
+
#SBATCH --time=2:00:00
|
| 9 |
+
#SBATCH --output=slurm_%j.out
|
| 10 |
+
#SBATCH --error=slurm_%j.err
|
| 11 |
+
|
| 12 |
+
if [[ -n "${SCRIPT_DIR}" ]]; then
|
| 13 |
+
:
|
| 14 |
+
elif [[ -n "${SLURM_SUBMIT_DIR}" ]]; then
|
| 15 |
+
SCRIPT_DIR="${SLURM_SUBMIT_DIR}"
|
| 16 |
+
else
|
| 17 |
+
echo "ERROR: 未检测到外部 SCRIPT_DIR 变量,也不在 Slurm 任务环境(SLURM_SUBMIT_DIR 为空)"
|
| 18 |
+
exit 1
|
| 19 |
+
fi
|
| 20 |
+
echo $SCRIPT_DIR
|
| 21 |
+
|
| 22 |
+
export MATCHEM_CONDA_NAME="${MATCHEM_CONDA_NAME:-test_pip}"
|
| 23 |
+
source "$SCRIPT_DIR/matchem_env.sh"
|
| 24 |
+
|
| 25 |
+
# MatPL 运行时环境(不侵入 matchem_env.sh,各算例自行维护)
|
| 26 |
+
# 校验MatPL根路径
|
| 27 |
+
if [[ -z "${MATPL_SRC_DIR}" || "${MATPL_SRC_DIR}" == "/path/to/matpl_dcu" || ! -d "${MATPL_SRC_DIR}" ]]; then
|
| 28 |
+
echo "=============================================="
|
| 29 |
+
echo "ERROR: MATPL_SRC_DIR 路径配置错误!"
|
| 30 |
+
echo "当前值:${MATPL_SRC_DIR}"
|
| 31 |
+
echo "请设置为真实有效的MatPL源码目录,示例:"
|
| 32 |
+
echo "export MATPL_SRC_DIR=/public/home/xxx/real_matpl_path"
|
| 33 |
+
echo "=============================================="
|
| 34 |
+
exit 1
|
| 35 |
+
fi
|
| 36 |
+
|
| 37 |
+
if [ -f "$MATPL_SRC_DIR/env.sh" ]; then
|
| 38 |
+
source "$MATPL_SRC_DIR/env.sh"
|
| 39 |
+
fi
|
| 40 |
+
echo "MATPL_SRC_DIR = $MATPL_SRC_DIR"
|
| 41 |
+
export LD_LIBRARY_PATH="$MATPL_SRC_DIR/src/op/build/lib:${LD_LIBRARY_PATH:-}"
|
| 42 |
+
|
| 43 |
+
# 单卡训练
|
| 44 |
+
|
| 45 |
+
# 输入 JSON 使用环境变量,运行前展开为临时文件
|
| 46 |
+
INPUT_JSON="LiSiC_nep_train.json"
|
| 47 |
+
EXPANDED_JSON=".${INPUT_JSON%.json}_expanded_$$.json"
|
| 48 |
+
trap 'rm -f "$EXPANDED_JSON"' EXIT
|
| 49 |
+
python3 -c "import os; open('$EXPANDED_JSON','w').write(os.path.expandvars(open('$INPUT_JSON').read()))"
|
| 50 |
+
|
| 51 |
+
MatPL train "$EXPANDED_JSON"
|
matchem_env.sh
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# ==========================================
|
| 3 |
+
# MatChem 统一环境配置脚本
|
| 4 |
+
# 用途:加载模块、激活 conda、导出各组件路径
|
| 5 |
+
# 用法:source matchem_env.sh
|
| 6 |
+
# 注意:默认激活 conda 环境 test_pip;若使用其他环境名,请执行:
|
| 7 |
+
# MATCHEM_CONDA_NAME=your_env source matchem_env.sh
|
| 8 |
+
# ==========================================
|
| 9 |
+
|
| 10 |
+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
| 11 |
+
|
| 12 |
+
# ---------- 1. 基础环境配置 ----------
|
| 13 |
+
export MATCHEM_CONDA_NAME="${MATCHEM_CONDA_NAME:-test_pip}"
|
| 14 |
+
|
| 15 |
+
# ---------- 2. OneScience 运行时环境变量 ----------
|
| 16 |
+
export ONESCIENCE_DATASETS_DIR="/public/share/sugonhpcapp01/onestore/onedatasets"
|
| 17 |
+
export ONESCIENCE_MODELS_DIR="/public/share/sugonhpcapp01/onestore/onemodels"
|
| 18 |
+
export device="gpu" # 根据实际平台改为 gpu 或 dcu
|
| 19 |
+
export LD_LIBRARY_PATH="${CONDA_PREFIX:-}/lib:${LD_LIBRARY_PATH:-}"
|
| 20 |
+
|
| 21 |
+
# ---------- 3. 外部软件路径(使用 dp_install.sh / matpl_install.sh / lmp_install.sh 时会自动更新) ----------
|
| 22 |
+
# DeepMD-kit 源码目录
|
| 23 |
+
export DEEPMD_SRC_DIR="${SCRIPT_DIR}/deepmd-kit"
|
| 24 |
+
|
| 25 |
+
# MatPL 源码目录
|
| 26 |
+
export MATPL_SRC_DIR="${SCRIPT_DIR}/matpl_dcu"
|
| 27 |
+
|
| 28 |
+
# LAMMPS 安装目录
|
| 29 |
+
export LAMMPS_INSTALL_DIR="${SCRIPT_DIR}/lammps_dcu"
|
| 30 |
+
|
| 31 |
+
# DeepMD C++ 接口目录
|
| 32 |
+
export DP_CPP_DIR="${SCRIPT_DIR}/dp_cpp_dcu"
|
| 33 |
+
|
| 34 |
+
# ---------- 4. 加载集群模块与 conda ----------
|
| 35 |
+
set +u
|
| 36 |
+
source ~/.bashrc
|
| 37 |
+
set -u
|
| 38 |
+
module load sghpcdas/25.6 # DTK / PyTorch 等 SDK
|
| 39 |
+
module load sghpc-mpi-gcc/26.3 # MPI 与 GCC 编译器
|
| 40 |
+
|
| 41 |
+
conda activate "$MATCHEM_CONDA_NAME"
|
| 42 |
+
|
| 43 |
+
# ---------- 6. LAMMPS 运行时环境 ----------
|
| 44 |
+
export LD_LIBRARY_PATH=${LAMMPS_INSTALL_DIR}/lib64:${LD_LIBRARY_PATH:-}
|
| 45 |
+
export LD_LIBRARY_PATH=${LAMMPS_INSTALL_DIR}/lib_override:${LD_LIBRARY_PATH:-}
|
| 46 |
+
export LAMMPS_PLUGIN_PATH=${DP_CPP_DIR}/lib
|
| 47 |
+
|
| 48 |
+
echo "✅ MatChem 环境已激活: ${MATCHEM_CONDA_NAME}"
|
matpl_install.sh
ADDED
|
@@ -0,0 +1,179 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# -----------------------------------------------------------------------------
|
| 3 |
+
# MatPL DCU 一键安装脚本
|
| 4 |
+
# 流程:源码拉取 → 编译安装 → 安装验证
|
| 5 |
+
# 用法:bash matpl_install.sh
|
| 6 |
+
# MATPL_SRC_DIR=/path/to/src bash matpl_install.sh # 指定源码路径
|
| 7 |
+
# -----------------------------------------------------------------------------
|
| 8 |
+
|
| 9 |
+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
| 10 |
+
cd "$SCRIPT_DIR"
|
| 11 |
+
|
| 12 |
+
# 1. 环境准备
|
| 13 |
+
echo ">>> Step 1: 加载环境"
|
| 14 |
+
source "$SCRIPT_DIR/../matchem_env.sh"
|
| 15 |
+
|
| 16 |
+
# 1.5 确保 gflags/glog 运行库存在(torch cmake 的运行时依赖,pip 环境通常缺失)
|
| 17 |
+
echo ">>> Step 1.5: 检查 gflags/glog 运行库"
|
| 18 |
+
MISSING_PKGS=""
|
| 19 |
+
ls "$CONDA_PREFIX"/lib/libgflags.so* >/dev/null 2>&1 || MISSING_PKGS="${MISSING_PKGS} gflags"
|
| 20 |
+
ls "$CONDA_PREFIX"/lib/libglog.so* >/dev/null 2>&1 || MISSING_PKGS="${MISSING_PKGS} glog"
|
| 21 |
+
if [ -n "${MISSING_PKGS}" ]; then
|
| 22 |
+
echo ">>> 安装缺失的运行库:${MISSING_PKGS}(conda-forge)"
|
| 23 |
+
conda install -y -c conda-forge ${MISSING_PKGS}
|
| 24 |
+
else
|
| 25 |
+
echo ">>> gflags/glog 已存在,跳过"
|
| 26 |
+
fi
|
| 27 |
+
|
| 28 |
+
# 下载辅助:优先 curl,失败时回退 wget(部分节点 curl 存在 TLS/代理问题)
|
| 29 |
+
download_file() {
|
| 30 |
+
local url="$1" out="$2"
|
| 31 |
+
if command -v curl >/dev/null 2>&1 && curl -fL -o "$out" "$url"; then
|
| 32 |
+
return 0
|
| 33 |
+
fi
|
| 34 |
+
echo "[提示] curl 下载失败,改用 wget: $url"
|
| 35 |
+
if command -v wget >/dev/null 2>&1 && wget -O "$out" "$url"; then
|
| 36 |
+
return 0
|
| 37 |
+
fi
|
| 38 |
+
echo "[错误] 下载失败: $url"
|
| 39 |
+
return 1
|
| 40 |
+
}
|
| 41 |
+
|
| 42 |
+
# 2. 交互式配置源码路径
|
| 43 |
+
MATPL_SRC="${MATPL_SRC_DIR:-${SCRIPT_DIR}/matpl_dcu}"
|
| 44 |
+
if [ -t 0 ]; then
|
| 45 |
+
read -rp "请输入 MatPL 源码路径 [默认: ${MATPL_SRC}]: " input_src
|
| 46 |
+
MATPL_SRC="${input_src:-${MATPL_SRC}}"
|
| 47 |
+
fi
|
| 48 |
+
echo "[提示] 使用 MatPL 源码路径: ${MATPL_SRC}"
|
| 49 |
+
|
| 50 |
+
# 3. 源码准备
|
| 51 |
+
# 说明:
|
| 52 |
+
# - 开发/测试阶段:自动通过 HTTPS + 代理拉取源码
|
| 53 |
+
# - 生产/客户场景:建议提前上传源码到集群,通过 MATPL_SRC_DIR 指定
|
| 54 |
+
if [ ! -d "$MATPL_SRC/.git" ] && [ ! -f "$MATPL_SRC/main.py" ]; then
|
| 55 |
+
echo ">>> Step 3: 拉取 MatPL DCU 源码"
|
| 56 |
+
# 计算节点需通过 HTTP 代理访问外网,配置 git 代理,如代理实效,可向集群管理员重新申请
|
| 57 |
+
git config --global http.proxy "http://scnethpc2601:sWMtqVS@10.16.1.52:3120"
|
| 58 |
+
git clone --depth 1 --branch nep-dcu/2026.3 "https://gitee.com/wang-rui-sugon/matpl_dcu.git" "$MATPL_SRC"
|
| 59 |
+
else
|
| 60 |
+
echo ">>> Step 3: 源码已存在,跳过拉取"
|
| 61 |
+
fi
|
| 62 |
+
|
| 63 |
+
cd "$MATPL_SRC"
|
| 64 |
+
|
| 65 |
+
echo "=========================================="
|
| 66 |
+
echo " MatPL DCU 一键安装脚本"
|
| 67 |
+
echo "=========================================="
|
| 68 |
+
echo "工作目录: $MATPL_SRC"
|
| 69 |
+
|
| 70 |
+
# 3.5 Patch MatPL CMakeLists.txt:避免在登录节点触发 torch 动态库加载
|
| 71 |
+
echo ">>> Step 3.5: Patch MatPL CMakeLists.txt,避免登录节点 import torch"
|
| 72 |
+
OP_CMAKE="$MATPL_SRC/src/op/CMakeLists.txt"
|
| 73 |
+
CHEB_CMAKE="$MATPL_SRC/src/feature/chebyshev/CMakeLists.txt"
|
| 74 |
+
|
| 75 |
+
if [ -f "$OP_CMAKE" ]; then
|
| 76 |
+
sed -i "s|import torch; print(torch.utils.cmake_prefix_path)|import importlib.util, os; spec = importlib.util.find_spec('torch'); p = os.path.dirname(spec.origin) if spec and spec.origin else ''; print((p + '/share/cmake') if p else '')|" "$OP_CMAKE"
|
| 77 |
+
sed -i "s|import torch; print(torch.__path__\[0\])|import importlib.util, os; spec = importlib.util.find_spec('torch'); print(os.path.dirname(spec.origin) if spec and spec.origin else '')|" "$OP_CMAKE"
|
| 78 |
+
sed -i "s|import torch; print(torch.version.hip is not None)|print(True)|" "$OP_CMAKE"
|
| 79 |
+
echo ">>> 已 patch $OP_CMAKE"
|
| 80 |
+
fi
|
| 81 |
+
|
| 82 |
+
if [ -f "$CHEB_CMAKE" ]; then
|
| 83 |
+
sed -i "s|import torch; print(torch.utils.cmake_prefix_path)|import importlib.util, os; spec = importlib.util.find_spec('torch'); p = os.path.dirname(spec.origin) if spec and spec.origin else ''; print((p + '/share/cmake') if p else '')|" "$CHEB_CMAKE"
|
| 84 |
+
echo ">>> 已 patch $CHEB_CMAKE"
|
| 85 |
+
fi
|
| 86 |
+
|
| 87 |
+
# 4. 修复 torch cmake 中硬编码的 /opt/dtk 路径
|
| 88 |
+
TORCH_CMAKE="$CONDA_PREFIX/lib/python3.11/site-packages/torch/share/cmake/Caffe2/Caffe2Targets.cmake"
|
| 89 |
+
if [ -f "$TORCH_CMAKE" ]; then
|
| 90 |
+
if grep -q "/opt/dtk" "$TORCH_CMAKE"; then
|
| 91 |
+
echo ">>> Step 4: 修复 torch cmake 硬编码路径"
|
| 92 |
+
sed -i "s|/opt/dtk|$ROCM_PATH|g" "$TORCH_CMAKE"
|
| 93 |
+
else
|
| 94 |
+
echo ">>> Step 4: torch cmake 路径已正确,跳过修复"
|
| 95 |
+
fi
|
| 96 |
+
else
|
| 97 |
+
echo "警告: 未找到 torch Caffe2Targets.cmake,跳过修复"
|
| 98 |
+
fi
|
| 99 |
+
|
| 100 |
+
# 5. 确保安装兼容版 glog(0.7+ API 不兼容)
|
| 101 |
+
echo ">>> Step 5: 确保 glog 0.6 已安装"
|
| 102 |
+
if [ -f "$CONDA_PREFIX/lib/libglog.so.1" ]; then
|
| 103 |
+
echo "glog 0.6+ 已存在,跳过安装"
|
| 104 |
+
else
|
| 105 |
+
echo "从源码编译 glog 0.6 ..."
|
| 106 |
+
GLOG_BUILD_DIR=$(mktemp -d)
|
| 107 |
+
cd "$GLOG_BUILD_DIR"
|
| 108 |
+
download_file "https://github.com/google/glog/archive/refs/tags/v0.6.0.tar.gz" v0.6.0.tar.gz || exit 1
|
| 109 |
+
tar -xzf v0.6.0.tar.gz
|
| 110 |
+
cd glog-0.6.0
|
| 111 |
+
cmake -S . -B build \
|
| 112 |
+
-DCMAKE_INSTALL_PREFIX="$CONDA_PREFIX" \
|
| 113 |
+
-DBUILD_SHARED_LIBS=ON \
|
| 114 |
+
-DWITH_GTEST=OFF
|
| 115 |
+
cmake --build build -j$(nproc)
|
| 116 |
+
cmake --install build
|
| 117 |
+
cd "$MATPL_SRC"
|
| 118 |
+
rm -rf "$GLOG_BUILD_DIR"
|
| 119 |
+
fi
|
| 120 |
+
|
| 121 |
+
# 6. 导出编译所需环境变量
|
| 122 |
+
echo ">>> Step 6: 导出编译环境变量"
|
| 123 |
+
export ROCM_PATH="$ROCM_PATH"
|
| 124 |
+
export CUDA_TOOLKIT_ROOT_DIR="${ROCM_PATH}/cuda/cuda-12"
|
| 125 |
+
export PATH="${ROCM_PATH}/cuda/cuda-12/bin:$PATH"
|
| 126 |
+
export CMAKE_PREFIX_PATH="${ROCM_PATH}/lib/cmake:${ROCM_PATH}/dcc/comgr/lib64/cmake/amd_comgr:${CMAKE_PREFIX_PATH:-}"
|
| 127 |
+
export LIBRARY_PATH="${ROCM_PATH}/cuda/cuda-12/targets/x86_64-linux/lib:${ROCM_PATH}/dcc/lib/clang/17.0.0/lib/linux:${LIBRARY_PATH:-}"
|
| 128 |
+
export CPLUS_INCLUDE_PATH="$CONDA_PREFIX/include:${CPLUS_INCLUDE_PATH:-}"
|
| 129 |
+
|
| 130 |
+
# 修复 GCC 版本:强制使用 sghpc-mpi-gcc/26.3 提供的 GCC 12.4.0
|
| 131 |
+
export CC=/public/software/sghpc_sdk.bak/Linux_x86_64/26.3/compilers/gcc-12.4.0/bin/gcc
|
| 132 |
+
export CXX=/public/software/sghpc_sdk.bak/Linux_x86_64/26.3/compilers/gcc-12.4.0/bin/g++
|
| 133 |
+
|
| 134 |
+
# 7. 清理历史编译产物并重新编译
|
| 135 |
+
echo ">>> Step 7: 开始编译 MatPL"
|
| 136 |
+
cd src
|
| 137 |
+
rm -rf feature/nep_find_neigh/build feature/NEP_GPU/build op/build
|
| 138 |
+
|
| 139 |
+
# 登录节点内存有限,限制并行编译数以避免 OOM
|
| 140 |
+
NPROC=$(nproc)
|
| 141 |
+
if [ "$NPROC" -gt 2 ]; then
|
| 142 |
+
NPROC=2
|
| 143 |
+
fi
|
| 144 |
+
echo ">>> 限制并行编译数为 ${NPROC}(避免节点 OOM)"
|
| 145 |
+
bash build.sh -j${NPROC}
|
| 146 |
+
|
| 147 |
+
# 8. 生成 env.sh
|
| 148 |
+
cd "$MATPL_SRC"
|
| 149 |
+
cat > env.sh <<EOF
|
| 150 |
+
# Load for MatPL
|
| 151 |
+
export PYTHONPATH=$MATPL_SRC:\${PYTHONPATH:-}
|
| 152 |
+
export PATH=$MATPL_SRC/src/bin:\${PATH:-}
|
| 153 |
+
EOF
|
| 154 |
+
|
| 155 |
+
# 9. 验证
|
| 156 |
+
echo ">>> Step 9: 验证安装"
|
| 157 |
+
source env.sh
|
| 158 |
+
python -c "import matpl; print('MatPL import OK')" 2>/dev/null || \
|
| 159 |
+
python -c "import sys; sys.path.insert(0, '$MATPL_SRC'); import matpl; print('MatPL import OK')" 2>/dev/null || \
|
| 160 |
+
echo "注意: MatPL Python 包导入验证跳过(不影响 C++ 算子使用)"
|
| 161 |
+
|
| 162 |
+
# 10. 将配置写回 matchem_env.sh
|
| 163 |
+
MATCHEM_ENV_FILE="${SCRIPT_DIR}/../matchem_env.sh"
|
| 164 |
+
if [ -f "${MATCHEM_ENV_FILE}" ]; then
|
| 165 |
+
echo "[提示] 更新 ${MATCHEM_ENV_FILE} ..."
|
| 166 |
+
sed -i "s|^export MATPL_SRC_DIR=.*|export MATPL_SRC_DIR=${MATPL_SRC}|" "${MATCHEM_ENV_FILE}"
|
| 167 |
+
else
|
| 168 |
+
echo "[警告] 未找到 ${MATCHEM_ENV_FILE},跳过写入配置。"
|
| 169 |
+
fi
|
| 170 |
+
|
| 171 |
+
echo ""
|
| 172 |
+
echo "=========================================="
|
| 173 |
+
echo " MatPL DCU 安装完成!"
|
| 174 |
+
echo "=========================================="
|
| 175 |
+
echo "源码路径: $MATPL_SRC"
|
| 176 |
+
echo "环境文件: $MATPL_SRC/env.sh"
|
| 177 |
+
echo ""
|
| 178 |
+
echo "每次使用前请执行:"
|
| 179 |
+
echo " source $SCRIPT_DIR/../matchem_env.sh"
|