Upload README.md
Browse files
README.md
CHANGED
|
@@ -52,7 +52,7 @@ model-index:
|
|
| 52 |
|
| 53 |
This [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) is finetuned from [Derify/ModChemBERT-IR-BASE](https://huggingface.co/Derify/ModChemBERT-IR-BASE) using hard-negative triplets derived from [Derify/pubchem_10m_genmol_similarity](https://huggingface.co/datasets/Derify/pubchem_10m_genmol_similarity). Positive SMILES pairs are first filtered by quality and similarity constraints, then reduced to one strongest positive target per anchor molecule to create a high-signal training set for reranking. The model computes relevance scores for pairs of SMILES strings, enabling SMILES reranking and molecular semantic search.
|
| 54 |
|
| 55 |
-
For this variant, positives are selected with a composite ranking criterion that combines high QED and similarity without an additional similarity-contribution cutoff. The quality stage uses strict inequality filtering (`QED > 0.85`, `similarity > 0.5`, with similarity also bounded below 1.0), and then keeps the top-scoring pair per anchor molecule.
|
| 56 |
|
| 57 |
Hard negatives are mined with [Sentence Transformers](https://www.sbert.net/) using [Derify/ChemMRL-beta](https://huggingface.co/Derify/ChemMRL-beta) as the teacher model and a TopK-PercPos-style margin setting based on [NV-Retriever](https://arxiv.org/abs/2407.15831), with `relative_margin=0.05` and `max_negative_score_threshold = pos_score * percentage_margin`. Training uses triplet-format samples with 5 mined negatives per anchor-positive pair and optimizes a multiple-negatives ranking objective, while reranking evaluation uses n-tuple samples with 30 mined negatives per query.
|
| 58 |
|
|
@@ -65,7 +65,6 @@ Hard negatives are mined with [Sentence Transformers](https://www.sbert.net/) us
|
|
| 65 |
- **Number of Output Labels:** 1 label
|
| 66 |
- **Training Dataset:**
|
| 67 |
- [Derify/pubchem_10m_genmol_similarity](https://huggingface.co/datasets/Derify/pubchem_10m_genmol_similarity) Mined Hard Negatives
|
| 68 |
-
<!-- - **Language:** Unknown -->
|
| 69 |
- **License:** apache-2.0
|
| 70 |
|
| 71 |
### Model Sources
|
|
@@ -258,6 +257,7 @@ You can finetune this model on your own dataset.
|
|
| 258 |
- `torch_compile`: True
|
| 259 |
- `torch_compile_backend`: inductor
|
| 260 |
- `torch_compile_mode`: max-autotune
|
|
|
|
| 261 |
- `batch_sampler`: no_duplicates
|
| 262 |
|
| 263 |
#### All Hyperparameters
|
|
@@ -372,7 +372,7 @@ You can finetune this model on your own dataset.
|
|
| 372 |
- `neftune_noise_alpha`: None
|
| 373 |
- `optim_target_modules`: None
|
| 374 |
- `batch_eval_metrics`: False
|
| 375 |
-
- `eval_on_start`:
|
| 376 |
- `use_liger_kernel`: False
|
| 377 |
- `liger_kernel_config`: None
|
| 378 |
- `eval_use_gather_object`: False
|
|
|
|
| 52 |
|
| 53 |
This [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) is finetuned from [Derify/ModChemBERT-IR-BASE](https://huggingface.co/Derify/ModChemBERT-IR-BASE) using hard-negative triplets derived from [Derify/pubchem_10m_genmol_similarity](https://huggingface.co/datasets/Derify/pubchem_10m_genmol_similarity). Positive SMILES pairs are first filtered by quality and similarity constraints, then reduced to one strongest positive target per anchor molecule to create a high-signal training set for reranking. The model computes relevance scores for pairs of SMILES strings, enabling SMILES reranking and molecular semantic search.
|
| 54 |
|
| 55 |
+
For this variant, the positives are selected with a composite ranking criterion that combines high QED and similarity without an additional similarity-contribution cutoff. The quality stage uses strict inequality filtering (`QED > 0.85`, `similarity > 0.5`, with similarity also bounded below 1.0), and then keeps the top-scoring pair per anchor molecule.
|
| 56 |
|
| 57 |
Hard negatives are mined with [Sentence Transformers](https://www.sbert.net/) using [Derify/ChemMRL-beta](https://huggingface.co/Derify/ChemMRL-beta) as the teacher model and a TopK-PercPos-style margin setting based on [NV-Retriever](https://arxiv.org/abs/2407.15831), with `relative_margin=0.05` and `max_negative_score_threshold = pos_score * percentage_margin`. Training uses triplet-format samples with 5 mined negatives per anchor-positive pair and optimizes a multiple-negatives ranking objective, while reranking evaluation uses n-tuple samples with 30 mined negatives per query.
|
| 58 |
|
|
|
|
| 65 |
- **Number of Output Labels:** 1 label
|
| 66 |
- **Training Dataset:**
|
| 67 |
- [Derify/pubchem_10m_genmol_similarity](https://huggingface.co/datasets/Derify/pubchem_10m_genmol_similarity) Mined Hard Negatives
|
|
|
|
| 68 |
- **License:** apache-2.0
|
| 69 |
|
| 70 |
### Model Sources
|
|
|
|
| 257 |
- `torch_compile`: True
|
| 258 |
- `torch_compile_backend`: inductor
|
| 259 |
- `torch_compile_mode`: max-autotune
|
| 260 |
+
- `eval_on_start`: True
|
| 261 |
- `batch_sampler`: no_duplicates
|
| 262 |
|
| 263 |
#### All Hyperparameters
|
|
|
|
| 372 |
- `neftune_noise_alpha`: None
|
| 373 |
- `optim_target_modules`: None
|
| 374 |
- `batch_eval_metrics`: False
|
| 375 |
+
- `eval_on_start`: True
|
| 376 |
- `use_liger_kernel`: False
|
| 377 |
- `liger_kernel_config`: None
|
| 378 |
- `eval_use_gather_object`: False
|