| | --- |
| | language: en |
| | license: apache-2.0 |
| | tags: |
| | - biology |
| | - chemistry |
| | - smiles |
| | - bert |
| | - molecular-modeling |
| | --- |
| | |
| | # chemical_structure_smiles_bert |
| | |
| | ## Overview |
| | This model is a BERT-style encoder pre-trained on millions of SMILES (Simplified Molecular Input Line Entry System) strings. It learns the "grammar" of chemistry to provide high-dimensional embeddings of molecules, which can then be used for downstream tasks like property prediction, toxicity screening, and drug-target interaction modeling. |
| | |
| | |
| | |
| | ## Model Architecture |
| | The model adapts the **BERT** (Bidirectional Encoder Representations from Transformers) architecture for molecular sequences: |
| | - **Tokenizer**: Custom regex-based tokenizer that treats atoms (e.g., `[Fe+2]`, `Cl`) and structural markers (`=`, `#`, `(`, `)`) as individual tokens. |
| | - **Encoder**: 12 layers of multi-head self-attention. |
| | - **Pre-training Task**: Masked Language Modeling (MLM), where 15% of atoms/bonds are hidden and predicted based on context. |
| | |
| | ## Intended Use |
| | - **Molecular Fingerprinting**: Generating dense vector representations for similarity searches in chemical databases. |
| | - **Lead Optimization**: Serving as a feature extractor for models predicting LogP, solubility, or binding affinity. |
| | - **Reaction Prediction**: Analyzing chemical transformations by comparing reactant and product embeddings. |
| | |
| | ## Limitations |
| | - **3D Conformation**: This model only understands 1D string representations and does not account for 3D spatial stereochemistry or bond angles. |
| | - **Sequence Length**: Extremely large polymers or proteins exceeding 512 SMILES tokens are truncated. |
| | - **Chemical Diversity**: Predictions may be biased toward drug-like small molecules if the chemical space is outside the pre-training distribution. |
