Create README.md

README.md

@@ -0,0 +1,33 @@
+---
+language: en
+license: apache-2.0
+tags:
+- biology
+- chemistry
+- smiles
+- bert
+- molecular-modeling
+---
+
+# chemical_structure_smiles_bert
+
+## Overview
+This model is a BERT-style encoder pre-trained on millions of SMILES (Simplified Molecular Input Line Entry System) strings. It learns the "grammar" of chemistry to provide high-dimensional embeddings of molecules, which can then be used for downstream tasks like property prediction, toxicity screening, and drug-target interaction modeling.
+
+
+
+## Model Architecture
+The model adapts the **BERT** (Bidirectional Encoder Representations from Transformers) architecture for molecular sequences:
+- **Tokenizer**: Custom regex-based tokenizer that treats atoms (e.g., `[Fe+2]`, `Cl`) and structural markers (`=`, `#`, `(`, `)`) as individual tokens.
+- **Encoder**: 12 layers of multi-head self-attention.
+- **Pre-training Task**: Masked Language Modeling (MLM), where 15% of atoms/bonds are hidden and predicted based on context.
+
+## Intended Use
+- **Molecular Fingerprinting**: Generating dense vector representations for similarity searches in chemical databases.
+- **Lead Optimization**: Serving as a feature extractor for models predicting LogP, solubility, or binding affinity.
+- **Reaction Prediction**: Analyzing chemical transformations by comparing reactant and product embeddings.
+
+## Limitations
+- **3D Conformation**: This model only understands 1D string representations and does not account for 3D spatial stereochemistry or bond angles.
+- **Sequence Length**: Extremely large polymers or proteins exceeding 512 SMILES tokens are truncated.
+- **Chemical Diversity**: Predictions may be biased toward drug-like small molecules if the chemical space is outside the pre-training distribution.