Spaces:
Sleeping
Sleeping
Marek Bukowicki commited on
Commit ·
5f0e06a
1
Parent(s): 7bc8c9d
update readme
Browse files
README.md
CHANGED
|
@@ -11,222 +11,9 @@ short_description: ShimNet Spectra Correction
|
|
| 11 |
|
| 12 |
# ShimNet
|
| 13 |
ShimNet is a data-driven AI solution to improve high-resolution nuclear magnetic resonance (NMR) spectra
|
| 14 |
-
distorted by the inhomogeneous magnetic field (less than optimal shimming). To use it, the experimental training data has to be collected (see **Data collection** below).
|
| 15 |
-
Example data can also be downloaded (see below).
|
| 16 |
|
| 17 |
-
Paper: [ShimNet: A neural network for post-acquisition improvement of NMR spectra distorted by magnetic-field inhomogeneity](https://
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
GPU version (for training and inference)
|
| 24 |
-
```
|
| 25 |
-
pip install -r requirements-gpu.txt
|
| 26 |
-
```
|
| 27 |
-
|
| 28 |
-
CPU version (for inference, not recommended for training)
|
| 29 |
-
```
|
| 30 |
-
pip install -r requirements-cpu.txt --extra-index-url https://download.pytorch.org/whl/cpu
|
| 31 |
-
```
|
| 32 |
-
|
| 33 |
-
## Usage
|
| 34 |
-
To correct spectra presented in the paper:
|
| 35 |
-
1. download weights (model parameters):
|
| 36 |
-
```
|
| 37 |
-
python download_files.py
|
| 38 |
-
```
|
| 39 |
-
or directly from [Google Drive 700MHz](https://drive.google.com/uc?export=download&id=17fTNWl7YW6mPbbZWga0EfdoF_6S8fCke) and [Google Drive 600MHz](https://drive.google.com/uc?export=download&id=1_VxOpFGJcFsOa5DHOW2GJbP8RvHCmC1N) and place it in `weights` directory
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
2. : run correction (e.g. `Azarone_20ul_700MHz.csv`):
|
| 43 |
-
```
|
| 44 |
-
python predict.py sample_data/Azarone_20ul_700MHz.csv -o output --config configs/shimnet_700.yaml --weights weights/shimnet_700MHz.pt
|
| 45 |
-
```
|
| 46 |
-
The output will be `output/Azarone_20ul_700MHz_processed.csv` file
|
| 47 |
-
|
| 48 |
-
Multiple files may be processed using "*" syntax:
|
| 49 |
-
```
|
| 50 |
-
python predict.py sample_data/*700MHz.csv -o output --config configs/shimnet_700.yaml --weights weights/shimnet_700MH
|
| 51 |
-
z.pt
|
| 52 |
-
```
|
| 53 |
-
|
| 54 |
-
For 600 MHz data use `--config configs/shimnet_600.yaml` and `--weights weights/shimnet_600MHz.pt`, e.g.:
|
| 55 |
-
|
| 56 |
-
```
|
| 57 |
-
python predict.py sample_data/CresolRed_after_styrene_600MHz.csv -o output --config configs/shimnet_600.yaml --weights weights/shimnet_600MHz.pt
|
| 58 |
-
```
|
| 59 |
-
|
| 60 |
-
### input format
|
| 61 |
-
|
| 62 |
-
The spectrum file for reconstruction should be in the format of two columns separated by a space and without the sign at the end of the line at the end of the file(example below):
|
| 63 |
-
```csv
|
| 64 |
-
-1.97134 0.0167137
|
| 65 |
-
-1.97085 -0.00778748
|
| 66 |
-
-1.97036 -0.0109595
|
| 67 |
-
-1.96988 0.00825978
|
| 68 |
-
-1.96939 0.0133886
|
| 69 |
-
```
|
| 70 |
-
|
| 71 |
-
## Train on your data
|
| 72 |
-
|
| 73 |
-
For the model to function properly, it should be trained on calibration data from the spectrometer used for the measurements. To train a model on data from your spectrometer, please follow the instructions below.
|
| 74 |
-
|
| 75 |
-
### Training data collection
|
| 76 |
-
|
| 77 |
-
Below we describe the training data collection for Agilent/Varian spectrometers. For machines of other vendors similar procedure can be implemented.
|
| 78 |
-
To collect ShimNet training data use Python script (sweep_shims_lineshape_Z1Z2.py) from the calibration_loop folder to drive the spectrometer:
|
| 79 |
-
1. Install TReNDS package ( trends.spektrino.com )
|
| 80 |
-
2. Open VnmrJ and type: 'listenon'
|
| 81 |
-
3. Put the lineshape sample (1% CHCl3 in deuterated acetone), set standard PROTON parameters, and set nt=1 (do not modify sw and at!)
|
| 82 |
-
4. Shim the sample and collect the data. Save the optimally shimmed dataset
|
| 83 |
-
5. Edit the sweep_shims_lineshape_Z1Z2.py script
|
| 84 |
-
6. Put optimum z1 and z2 shim values as optiz1 and optiz2 below
|
| 85 |
-
7. Define the calibration range as range_z1 and range_z2 (default is ok)
|
| 86 |
-
8. Start the python script:
|
| 87 |
-
```
|
| 88 |
-
python3 ./sweep_shims_lineshape_Z1Z2.py
|
| 89 |
-
```
|
| 90 |
-
The spectrometer will start collecting spectra
|
| 91 |
-
|
| 92 |
-
### SCRF extraction
|
| 93 |
-
Shim Coil Response Functions (SCRF) should be extracted from the spectra with `extract_scrf_from_fids.py` script.
|
| 94 |
-
```
|
| 95 |
-
python extract_scrf_from_fids.py
|
| 96 |
-
```
|
| 97 |
-
|
| 98 |
-
The script uses hardcoded paths to the NMR signals (fid-s) in Agilent/Varian format: a directory with optimal measurement (`opti_fid_path` available) and a directory with calibration loop measurements (`data_dir`):
|
| 99 |
-
```python
|
| 100 |
-
# input
|
| 101 |
-
data_dir = "../../sample_run/loop"
|
| 102 |
-
opti_fid_path = "../../sample_run/opti.fid"
|
| 103 |
-
|
| 104 |
-
```
|
| 105 |
-
|
| 106 |
-
The output files are also hardcoded:
|
| 107 |
-
```python
|
| 108 |
-
# output
|
| 109 |
-
spectra_file = "../../sample_run/total.npy"
|
| 110 |
-
spectra_file_names = "../../sample_run/total.csv"
|
| 111 |
-
opi_spectrum_file = "../../sample_run/opti.npy"
|
| 112 |
-
responses_file = "../../sample_run/scrf_61.pt"
|
| 113 |
-
```
|
| 114 |
-
where only the `responses_file` is used in ShimNet training.
|
| 115 |
-
|
| 116 |
-
If the measurements are stored in a format other than Varian, you may need to change this line:
|
| 117 |
-
```python
|
| 118 |
-
dic, data = ng.varian.read(varian_fid_path)
|
| 119 |
-
```
|
| 120 |
-
(see nmrglue package documentation for details)
|
| 121 |
-
|
| 122 |
-
### Training
|
| 123 |
-
|
| 124 |
-
1. Download multiplets database:
|
| 125 |
-
```
|
| 126 |
-
python download_files.py --multiplets
|
| 127 |
-
```
|
| 128 |
-
2. Configure run:
|
| 129 |
-
- create a run directory, e.g. `runs/my_lab_spectrometer_2025`
|
| 130 |
-
- create a configuration file:
|
| 131 |
-
1. copy `configs/shimnet_template.py` to the run directory and rename it to `config.yaml`
|
| 132 |
-
```bash
|
| 133 |
-
cp configs/shimnet_template.py runs/my_lab_spectrometer_2025/config.yaml
|
| 134 |
-
```
|
| 135 |
-
2. edit the SCRF in path in the config file:
|
| 136 |
-
```yaml
|
| 137 |
-
response_functions_files:
|
| 138 |
-
- path/to/srcf_file
|
| 139 |
-
```
|
| 140 |
-
e.g.
|
| 141 |
-
```yaml
|
| 142 |
-
response_functions_files:
|
| 143 |
-
- ../../sample_run/scrf_61.pt
|
| 144 |
-
```
|
| 145 |
-
3. adjust spectrometer frequency step `frq_step` to match your data (spectrometer range in Hz divided by number of points in spectrum):
|
| 146 |
-
```yaml
|
| 147 |
-
frq_step: 0.34059797
|
| 148 |
-
```
|
| 149 |
-
4. adjust spectromer frequency in the metadata
|
| 150 |
-
```yaml
|
| 151 |
-
metadata: # additional metadata, not used in the training process
|
| 152 |
-
spectrometer_frequency: 700.0 # MHz
|
| 153 |
-
```
|
| 154 |
-
3. Run training:
|
| 155 |
-
```
|
| 156 |
-
python train.py runs/my_lab_spectrometer_2025
|
| 157 |
-
```
|
| 158 |
-
Training results will appear in `runs/my_lab_spectrometer_2025` directory.
|
| 159 |
-
Model parameters are stored in `runs/my_lab_spectrometer_2025/model.pt` file
|
| 160 |
-
4. Use trained model:
|
| 161 |
-
|
| 162 |
-
use `--config runs/my_lab_spectrometer_2025/config.yaml` and `--weights runs/my_lab_spectrometer_2025/model.pt` flags, e.g.
|
| 163 |
-
```
|
| 164 |
-
python predict.py my_sample1.csv -o my_output --config runs/my_lab_spectrometer_2025/config.yaml --weights runs/my_lab_spectrometer_2025/model.pt
|
| 165 |
-
```
|
| 166 |
-
|
| 167 |
-
## Repeat training on our data
|
| 168 |
-
|
| 169 |
-
If you want to train the network using the calibration data from our paper, follow the procedure below.
|
| 170 |
-
|
| 171 |
-
1. Download multiplets database and our SCRF files:
|
| 172 |
-
```
|
| 173 |
-
python download_files.py --multiplets --SCRF --no-weights
|
| 174 |
-
```
|
| 175 |
-
or directly download from Google Drive and store in `data/` directory: [Response Functions 600MHz](https://drive.google.com/file/d/1J-DsPtaITXU3TFrbxaZPH800U1uIiwje/view?usp=sharing), [Response Functions 700MHz](https://drive.google.com/file/d/113al7A__yYALx_2hkESuzFIDU3feVtNY/view?usp=sharing), [Multiplets data](https://drive.google.com/file/d/1QGvV-Au50ZxaP1vFsmR_auI299Dw-Wrt/view?usp=sharing)
|
| 176 |
-
|
| 177 |
-
2. Configure run
|
| 178 |
-
- For 600MHz spectrometer:
|
| 179 |
-
```bash
|
| 180 |
-
mkdir -p runs/repeat_paper_training_600MHz
|
| 181 |
-
cp configs/shimnet_600.yaml runs/repeat_paper_training_600MHz/config.yaml
|
| 182 |
-
```
|
| 183 |
-
- For 700 MHz spectrometer:
|
| 184 |
-
```bash
|
| 185 |
-
mkdir -p runs/repeat_paper_training_700MHz
|
| 186 |
-
cp configs/shimnet_700.yaml runs/repeat_paper_training_700MHz/config.yaml
|
| 187 |
-
```
|
| 188 |
-
3. Run training:
|
| 189 |
-
```
|
| 190 |
-
python train.py runs/repeat_paper_training_600MHz
|
| 191 |
-
```
|
| 192 |
-
or
|
| 193 |
-
```
|
| 194 |
-
python train.py runs/repeat_paper_training_700MHz
|
| 195 |
-
```
|
| 196 |
-
Training results will appear in `runs/repeat_paper_training_600MHz` or `runs/repeat_paper_training_700MHz` directory.
|
| 197 |
-
|
| 198 |
-
## GUI
|
| 199 |
-
|
| 200 |
-
### Installation
|
| 201 |
-
|
| 202 |
-
To use the ShimNet GUI, ensure you have Python 3.10 installed (not tested with Python 3.11+). After installing the ShimNet requirements (CPU/GPU), install the additional dependencies for the GUI:
|
| 203 |
-
|
| 204 |
-
```bash
|
| 205 |
-
pip install -r requirements-gui.txt
|
| 206 |
-
```
|
| 207 |
-
|
| 208 |
-
### Launching the GUI
|
| 209 |
-
|
| 210 |
-
The ShimNet GUI is built using Gradio. To start the application, run:
|
| 211 |
-
|
| 212 |
-
```bash
|
| 213 |
-
python predict-gui.py
|
| 214 |
-
```
|
| 215 |
-
|
| 216 |
-
Once the application starts, open your browser and navigate to:
|
| 217 |
-
|
| 218 |
-
```
|
| 219 |
-
http://127.0.0.1:7860
|
| 220 |
-
```
|
| 221 |
-
|
| 222 |
-
to access the GUI locally.
|
| 223 |
-
|
| 224 |
-
### Sharing the GUI
|
| 225 |
-
|
| 226 |
-
To make the GUI accessible over the internet, use the `--share` flag:
|
| 227 |
-
|
| 228 |
-
```bash
|
| 229 |
-
python predict-gui.py --share
|
| 230 |
-
```
|
| 231 |
-
|
| 232 |
-
A public web address will be displayed in the terminal, which you can use to access the GUI remotely or share with others.
|
|
|
|
| 11 |
|
| 12 |
# ShimNet
|
| 13 |
ShimNet is a data-driven AI solution to improve high-resolution nuclear magnetic resonance (NMR) spectra
|
|
|
|
|
|
|
| 14 |
|
| 15 |
+
Paper: [ShimNet: A neural network for post-acquisition improvement of NMR spectra distorted by magnetic-field inhomogeneity](https://doi.org/10.1021/acs.jpcb.5c02632)
|
| 16 |
|
| 17 |
+
Full documentation may be found in the original repository: https://github.com/center4ml/shimnet
|
| 18 |
|
| 19 |
+
Sample input data: https://huggingface.co/spaces/NMR-CeNT-UW/ShimNet/tree/main/sample_data
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|