|
|
--- |
|
|
tags: |
|
|
- sentence-transformers |
|
|
- smiles-similarity |
|
|
- feature-extraction |
|
|
- molecular-similarity |
|
|
pipeline_tag: sentence-similarity |
|
|
library_name: sentence-transformers |
|
|
metrics: |
|
|
- spearmanr |
|
|
license: apache-2.0 |
|
|
new_version: Derify/ChemMRL |
|
|
--- |
|
|
|
|
|
# Chem-MRL (SentenceTransformer) |
|
|
|
|
|
This is a trained [Chem-MRL](https://github.com/emapco/chem-mrl) [sentence-transformers](https://www.SBERT.net) model. It maps SMILES to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, database indexing, molecular classification, clustering, and more. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
- **Model Type:** Sentence Transformer |
|
|
- **Maximum Sequence Length:** 128 tokens |
|
|
- **Output Dimensionality:** 1024 dimensions |
|
|
- **Similarity Function:** Cosine Similarity |
|
|
|
|
|
### Model Sources |
|
|
|
|
|
- **Repository:** [Chem-MRL on GitHub](https://github.com/emapco/chem-mrl) |
|
|
- **Demo App Repository:** [Chem-MRL-demo on GitHub](https://github.com/emapco/chem-mrl-demo) |
|
|
|
|
|
### Full Model Architecture |
|
|
|
|
|
``` |
|
|
SentenceTransformer( |
|
|
(0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: RobertaModel (ChemBERTa) |
|
|
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) |
|
|
(2): Normalize() |
|
|
) |
|
|
``` |
|
|
|
|
|
## Usage |
|
|
|
|
|
### Direct Usage (Sentence Transformers) |
|
|
|
|
|
First install the Sentence Transformers library: |
|
|
|
|
|
```bash |
|
|
pip install -U sentence-transformers |
|
|
``` |
|
|
|
|
|
Then you can load this model and run inference. |
|
|
```python |
|
|
from sentence_transformers import SentenceTransformer |
|
|
|
|
|
# Download from the 🤗 Hub |
|
|
model = SentenceTransformer("Derify/ChemMRL-alpha") |
|
|
# Run inference |
|
|
sentences = [ |
|
|
'CCO', |
|
|
"CC(C)O", |
|
|
'CC(=O)O', |
|
|
] |
|
|
embeddings = model.encode(sentences) |
|
|
print(embeddings.shape) |
|
|
# [3, 1024] |
|
|
|
|
|
# Get the similarity scores for the embeddings |
|
|
similarities = model.similarity(embeddings, embeddings) |
|
|
print(similarities.shape) |
|
|
# [3, 3] |
|
|
``` |
|
|
|
|
|
## Training Details |
|
|
|
|
|
### Framework Versions |
|
|
- Python: 3.12.9 |
|
|
- Sentence Transformers: 4.0.1 |
|
|
- Transformers: 4.48.2 |
|
|
- PyTorch: 2.6.0+cu124 |
|
|
- Accelerate: 1.4.0 |
|
|
- Datasets: 3.3.2 |
|
|
- Tokenizers: 0.21.0 |
|
|
|
|
|
## Citation |
|
|
- Chithrananda, Seyone, et al. "ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction." _arXiv [Cs.LG]_, 2020. [Link](http://arxiv.org/abs/2010.09885). |
|
|
- Ahmad, Walid, et al. "ChemBERTa-2: Towards Chemical Foundation Models." _arXiv [Cs.LG]_, 2022. [Link](http://arxiv.org/abs/2209.01712). |
|
|
- Kusupati, Aditya, et al. "Matryoshka Representation Learning." _arXiv [Cs.LG]_, 2022. [Link](https://arxiv.org/abs/2205.13147). |
|
|
- Li, Xianming, et al. "2D Matryoshka Sentence Embeddings." _arXiv [Cs.CL]_, 2024. [Link](http://arxiv.org/abs/2402.14776). |
|
|
- Bajusz, Dávid, et al. "Why is the Tanimoto Index an Appropriate Choice for Fingerprint-Based Similarity Calculations?" _J Cheminform_, 7, 20 (2015). [Link](https://doi.org/10.1186/s13321-015-0069-3). |
|
|
- Li, Xiaoya, et al. "Dice Loss for Data-imbalanced NLP Tasks." _arXiv [Cs.CL]_, 2020. [Link](https://arxiv.org/abs/1911.02855) |
|
|
- Reimers, Nils, and Gurevych, Iryna. "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks." _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing_, 2019. [Link](https://arxiv.org/abs/1908.10084). |
|
|
|
|
|
## Model Card Authors |
|
|
|
|
|
[@eacortes](https://huggingface.co/eacortes) |
|
|
|
|
|
## Model Card Contact |
|
|
|
|
|
Manny Cortes (manny@derifyai.com) |
