| --- |
| license: cc-by-4.0 |
| datasets: |
| - HZBSolarOptics/MultiLayerThinFilms |
| metrics: |
| - mae |
| tags: |
| - science |
| - material |
| - inverse |
| - design |
| --- |
| |
| <div align="center"> |
| <img src="https://huggingface.co/datasets/HZBSolarOptics/shared-assets/raw/main/optollama.svg" style="height: 300px;"> |
| </div> |
|
|
| # OptoLlama |
|
|
| [](./LICENSE) |
| [](mailto:SE-AOPT-office@helmholtz-berlin.de) |
|
|
| Meet OptoLlama — a masked diffusion language transformer aimed to solve inverse design of multi-layer thin film structures. |
|
|
| **Key Features** |
|
|
| - Masked diffusion language model (**MDLM**) |
| - Support for reflectance, absorption and transmittance **RAT spectra** 📈 |
| - Wave length from **300-2,000nm** 💡 |
| - State-of-the-art **predictive performance** for inverse material design 😎 |
|
|
|
|
| **Supporting Material** |
|
|
| **ArXiV Paper on MDLM**: 📝 https://arxiv.org/pdf/2406.07524 |
|
|
|
|
| ## Usage |
|
|
| ### Install Dependencies |
|
|
| ```bash |
| python -m pip install torch |
| python -m pip install safetensors |
| ``` |
|
|
| ### Load Model Checkpoint |
|
|
| ```python |
| from safetensors.torch import load_file |
| |
| model = OptoLlama() |
| |
| safetensors_path = "optollama-model.safetensors" |
| state_dict = load_file(safetensors_path) |
| model.load_state_dict(state_dict) |
| ``` |
|
|
| ## Useful Information |
|
|
| | Stat | Value | |
| | :------------------ | ----------: | |
| | #Parameters | 111,555,513 | |
| | Best validation MAE | 0.0140 | |
| | top_p | 0.9 | |
| | top_k | 5 | |
| | Epochs trained | 1,000 | |
| | Best epoch | 866 | |
| | Batch size | 256 | |
| | n_blocks | 6 | |
| | n_heads | 8 | |
| | d_model | 1,024 | |
| | max_seq_length | 20 | |
| |
| ## Acknowledgements |
| |
| This work is supported by the Helmholtz Association Initiative and Networking Fund through the Helmholtz AI platform, and the HAICORE@KIT grant. |
| |
| ---- |
| |
| <div style="float: left; position: relative; left: 50%; transform: translateX(-50%);"> |
| <a href="https://www.helmholtz-berlin.de/index_en.html"></a><img src="https://huggingface.co/datasets/HZBSolarOptics/shared-assets/raw/main/logo_hzb.svg" style="height: 80px; float: left; margin: 0 1em"></a> |
| <a href="https://www.kit.edu/english/index.php"></a><img src="https://huggingface.co/datasets/HZBSolarOptics/shared-assets/raw/main/logo_kit.svg" style="height: 80px; float: left; margin: 0 1em"></a> |
| <a href="https://www.zib.de/"></a><img src="https://huggingface.co/datasets/HZBSolarOptics/shared-assets/resolve/main/logo_zib.png" style="height: 80px; float: left; margin: 0 1em"></a> |
| |
| <a href="https://www.helmholtz.ai/"></a><img src="https://huggingface.co/datasets/HZBSolarOptics/shared-assets/raw/main/logo_hai.svg" style="height: 20px; float: left; margin: 2em 0; position: relative; left: 50%; transform: translateX(-50%);"></a> |
| </div> |
