Update README.md
Browse files
README.md
CHANGED
|
@@ -100,16 +100,14 @@ model gives *t* ≈ 0.61, *p* ≈ 0.56. The 0.005 gap in favour of BRAID sits we
|
|
| 100 |
noise, and the per-seed ranges overlap almost completely (BRAID 0.710–0.751, control
|
| 101 |
0.712–0.738).
|
| 102 |
|
| 103 |
-
The correct reading is
|
| 104 |
|
| 105 |
> Under matched pretraining and finetuning, BRAID's guaranteed-validity property comes
|
| 106 |
> **at no measurable cost to downstream predictive performance** on BBBP.
|
| 107 |
|
| 108 |
-
That is the useful claim. A notation that buys 100% generative validity while leaving
|
| 109 |
-
property prediction unchanged is worth having; a notation that claims a 0.005 AUC win on
|
| 110 |
-
n = 5 is not.
|
| 111 |
|
| 112 |
-
|
|
|
|
| 113 |
at this scale reporting a single seed is measuring noise.
|
| 114 |
|
| 115 |
---
|
|
@@ -164,56 +162,25 @@ tokenizer/`Vocab` builder, and the test suites) lives at:
|
|
| 164 |
|
| 165 |
## Limitations
|
| 166 |
|
| 167 |
-
1. **
|
| 168 |
-
shortest notation — `>k` and `^d` are two characters where SMILES uses one. Aromatic mode
|
| 169 |
-
reaches parity with SMILES, not dominance.
|
| 170 |
-
|
| 171 |
-
| molecule | SMILES | DeepSMILES | SELFIES(tok) | BRAID(kekulé) | BRAID(aromatic) |
|
| 172 |
-
|---|--:|--:|--:|--:|--:|
|
| 173 |
-
| benzene | 8 | 7 | 8 | 11 | 8 |
|
| 174 |
-
| aspirin | 21 | 18 | 19 | 24 | 21 |
|
| 175 |
-
| caffeine | 26 | 24 | 24 | 28 | 27 |
|
| 176 |
-
| morphine | 36 | 35 | 36 | 42 | — |
|
| 177 |
-
|
| 178 |
-
2. **Stereochemistry is partial.** Tetrahedral chirality and E/Z double bonds round-trip on
|
| 179 |
26/26 test cases. Allene/axial/planar chirality, atropisomers, and non-tetrahedral stereo
|
| 180 |
centres fall back to unspecified.
|
| 181 |
|
| 182 |
-
|
| 183 |
for common ions but liable to mis-clamp hypervalent, organometallic, or unusual-charge
|
| 184 |
atoms. It is not a substitute for RDKit's model.
|
| 185 |
|
| 186 |
-
|
| 187 |
-
and over-valent bonds are silent no-ops, so a mutated string can decode to something only
|
| 188 |
-
loosely related to its neighbour. Validity is guaranteed; locality under mutation is not.
|
| 189 |
-
|
| 190 |
-
5. **Insertion/deletion is non-local.** Inserting or deleting an atom shifts every `^d` that
|
| 191 |
spans the edit point. Inherent to relative back-references; applies equally to DeepSMILES.
|
| 192 |
|
| 193 |
-
|
| 194 |
RDKit canonical-rank rooting for determinism, but uniqueness is not proven.
|
| 195 |
|
| 196 |
-
7. **Depends on RDKit** for parsing and sanitization. This is a reference implementation,
|
| 197 |
-
not a self-contained spec.
|
| 198 |
-
|
| 199 |
-
### Limitations of the evaluation specifically
|
| 200 |
-
|
| 201 |
-
- **One dataset, one split.** BBBP only. No claim is made about BACE, HIV, Tox21, SIDER, or
|
| 202 |
-
ClinTox. Results on a single scaffold split of a ~2k-molecule dataset are fragile.
|
| 203 |
-
- **Small pretraining corpus.** ZINC 100k is tiny by chemical-LM standards. Any notation
|
| 204 |
-
effect that only emerges at scale would be invisible here.
|
| 205 |
-
- **Seed variance, not split variance.** The ± reflects finetuning seed noise under a fixed
|
| 206 |
-
split. It does not capture how much the answer would move under a different scaffold split,
|
| 207 |
-
which is typically larger.
|
| 208 |
-
- **n = 5.** Adequate to show two models are *indistinguishable*; nowhere near adequate to
|
| 209 |
-
establish a small real difference in either direction.
|
| 210 |
-
|
| 211 |
---
|
| 212 |
|
| 213 |
## Citation
|
| 214 |
|
| 215 |
-
BRAID is a recombination of published mechanisms
|
| 216 |
-
notation. Please cite the underlying work:
|
| 217 |
|
| 218 |
- O'Boyle, N. & Dalke, A. *DeepSMILES: An Adaptation of SMILES for Use in Machine-Learning
|
| 219 |
of Chemical Structures.* ChemRxiv, 2018.
|
|
|
|
| 100 |
noise, and the per-seed ranges overlap almost completely (BRAID 0.710–0.751, control
|
| 101 |
0.712–0.738).
|
| 102 |
|
| 103 |
+
The correct reading is
|
| 104 |
|
| 105 |
> Under matched pretraining and finetuning, BRAID's guaranteed-validity property comes
|
| 106 |
> **at no measurable cost to downstream predictive performance** on BBBP.
|
| 107 |
|
|
|
|
|
|
|
|
|
|
| 108 |
|
| 109 |
+
|
| 110 |
+
Note: The seed-to-seed spread (±0.017) exceeds the between-model difference. Any comparison
|
| 111 |
at this scale reporting a single seed is measuring noise.
|
| 112 |
|
| 113 |
---
|
|
|
|
| 162 |
|
| 163 |
## Limitations
|
| 164 |
|
| 165 |
+
1. **Stereochemistry is partial.** Tetrahedral chirality and E/Z double bonds round-trip on
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 166 |
26/26 test cases. Allene/axial/planar chirality, atropisomers, and non-tetrahedral stereo
|
| 167 |
centres fall back to unspecified.
|
| 168 |
|
| 169 |
+
2. **The valence state machine is approximate.** A hand-rolled charge→valence rule, correct
|
| 170 |
for common ions but liable to mis-clamp hypervalent, organometallic, or unusual-charge
|
| 171 |
atoms. It is not a substitute for RDKit's model.
|
| 172 |
|
| 173 |
+
3. **Insertion/deletion is non-local.** Inserting or deleting an atom shifts every `^d` that
|
|
|
|
|
|
|
|
|
|
|
|
|
| 174 |
spans the edit point. Inherent to relative back-references; applies equally to DeepSMILES.
|
| 175 |
|
| 176 |
+
4. **Not a canonical hash.** One molecule has many valid BRAID strings. The encoder uses
|
| 177 |
RDKit canonical-rank rooting for determinism, but uniqueness is not proven.
|
| 178 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 179 |
---
|
| 180 |
|
| 181 |
## Citation
|
| 182 |
|
| 183 |
+
BRAID is a recombination of published mechanisms. Please cite the underlying work:
|
|
|
|
| 184 |
|
| 185 |
- O'Boyle, N. & Dalke, A. *DeepSMILES: An Adaptation of SMILES for Use in Machine-Learning
|
| 186 |
of Chemical Structures.* ChemRxiv, 2018.
|