victor-shirasuna
commited on
Commit
·
2f0e472
1
Parent(s):
a71b2b1
Updated README.md
Browse files
README.md
CHANGED
|
@@ -25,7 +25,7 @@ This repository provides PyTorch source code associated with our publication, "S
|
|
| 25 |
|
| 26 |
**Paper:** [OpenReview Link](https://openreview.net/pdf?id=0uWNuJ1xtz)
|
| 27 |
|
| 28 |
-
**
|
| 29 |
|
| 30 |
For more information contact: vshirasuna@ibm.com or evital@br.ibm.com.
|
| 31 |
|
|
@@ -33,7 +33,7 @@ For more information contact: vshirasuna@ibm.com or evital@br.ibm.com.
|
|
| 33 |
|
| 34 |
## Introduction
|
| 35 |
|
| 36 |
-
We present a large encoder-decoder chemical foundation model based on the IBM Bamba architecture, a hybrid of Transformers and Mamba-2 layers, designed to support multi-representational molecular string inputs. The model is pre-trained in a BERT-style on 588 million samples, resulting in a corpus of approximately 29 billion molecular tokens. These models serve as a foundation for language chemical research in supporting different complex tasks, including molecular properties prediction, classification, and molecular translation. **Additionally, the STR-Bamba architecture allows for the aggregation of multiple representations in a single text input, as it does not contain any token length limitation, except for hardware limitations.** Our experiments across multiple benchmark datasets demonstrate state-of-the-art performance for various tasks. Model weights are available at: [
|
| 37 |
|
| 38 |
The STR-Bamba model supports the following **molecular representations**:
|
| 39 |
- SMILES
|
|
@@ -60,7 +60,7 @@ The STR-Bamba model supports the following **molecular representations**:
|
|
| 60 |
|
| 61 |
### Pretrained Models and Training Logs
|
| 62 |
|
| 63 |
-
We provide checkpoints of the STR-Bamba model pre-trained on a dataset of ~118M small molecules, ~2M polymer structures, and 258 formulations. The pre-trained model shows competitive performance on classification and regression benchmarks across small and polymer molecules, and electrolyte formulations. For model weights: [
|
| 64 |
|
| 65 |
Add the STR-Bamba `pre-trained weights.pt` to the `inference/` or `finetune/` directory according to your needs. The directory structure should look like the following:
|
| 66 |
|
|
|
|
| 25 |
|
| 26 |
**Paper:** [OpenReview Link](https://openreview.net/pdf?id=0uWNuJ1xtz)
|
| 27 |
|
| 28 |
+
**GitHub:** [GitHub Link](https://github.com/IBM/materials/tree/main/models/str_bamba)
|
| 29 |
|
| 30 |
For more information contact: vshirasuna@ibm.com or evital@br.ibm.com.
|
| 31 |
|
|
|
|
| 33 |
|
| 34 |
## Introduction
|
| 35 |
|
| 36 |
+
We present a large encoder-decoder chemical foundation model based on the IBM Bamba architecture, a hybrid of Transformers and Mamba-2 layers, designed to support multi-representational molecular string inputs. The model is pre-trained in a BERT-style on 588 million samples, resulting in a corpus of approximately 29 billion molecular tokens. These models serve as a foundation for language chemical research in supporting different complex tasks, including molecular properties prediction, classification, and molecular translation. **Additionally, the STR-Bamba architecture allows for the aggregation of multiple representations in a single text input, as it does not contain any token length limitation, except for hardware limitations.** Our experiments across multiple benchmark datasets demonstrate state-of-the-art performance for various tasks. Model weights are available at: [GitHub Link](https://github.com/IBM/materials/tree/main/models/str_bamba).
|
| 37 |
|
| 38 |
The STR-Bamba model supports the following **molecular representations**:
|
| 39 |
- SMILES
|
|
|
|
| 60 |
|
| 61 |
### Pretrained Models and Training Logs
|
| 62 |
|
| 63 |
+
We provide checkpoints of the STR-Bamba model pre-trained on a dataset of ~118M small molecules, ~2M polymer structures, and 258 formulations. The pre-trained model shows competitive performance on classification and regression benchmarks across small and polymer molecules, and electrolyte formulations. For model weights: [GitHub Link](https://github.com/IBM/materials/tree/main/models/str_bamba)
|
| 64 |
|
| 65 |
Add the STR-Bamba `pre-trained weights.pt` to the `inference/` or `finetune/` directory according to your needs. The directory structure should look like the following:
|
| 66 |
|