aakothari commited on
Commit
2a3b71d
·
verified ·
1 Parent(s): 17e2514

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -41
README.md CHANGED
@@ -100,16 +100,14 @@ model gives *t* ≈ 0.61, *p* ≈ 0.56. The 0.005 gap in favour of BRAID sits we
100
  noise, and the per-seed ranges overlap almost completely (BRAID 0.710–0.751, control
101
  0.712–0.738).
102
 
103
- The correct reading is **not** "BRAID beats SMILES." It is:
104
 
105
  > Under matched pretraining and finetuning, BRAID's guaranteed-validity property comes
106
  > **at no measurable cost to downstream predictive performance** on BBBP.
107
 
108
- That is the useful claim. A notation that buys 100% generative validity while leaving
109
- property prediction unchanged is worth having; a notation that claims a 0.005 AUC win on
110
- n = 5 is not.
111
 
112
- Note that seed-to-seed spread (±0.017) exceeds the between-model difference. Any comparison
 
113
  at this scale reporting a single seed is measuring noise.
114
 
115
  ---
@@ -164,56 +162,25 @@ tokenizer/`Vocab` builder, and the test suites) lives at:
164
 
165
  ## Limitations
166
 
167
- 1. **Not actually more compact.** The "D" in BRAID is aspirational. BRAID is never the
168
- shortest notation — `>k` and `^d` are two characters where SMILES uses one. Aromatic mode
169
- reaches parity with SMILES, not dominance.
170
-
171
- | molecule | SMILES | DeepSMILES | SELFIES(tok) | BRAID(kekulé) | BRAID(aromatic) |
172
- |---|--:|--:|--:|--:|--:|
173
- | benzene | 8 | 7 | 8 | 11 | 8 |
174
- | aspirin | 21 | 18 | 19 | 24 | 21 |
175
- | caffeine | 26 | 24 | 24 | 28 | 27 |
176
- | morphine | 36 | 35 | 36 | 42 | — |
177
-
178
- 2. **Stereochemistry is partial.** Tetrahedral chirality and E/Z double bonds round-trip on
179
  26/26 test cases. Allene/axial/planar chirality, atropisomers, and non-tetrahedral stereo
180
  centres fall back to unspecified.
181
 
182
- 3. **The valence state machine is approximate.** A hand-rolled charge→valence rule, correct
183
  for common ions but liable to mis-clamp hypervalent, organometallic, or unusual-charge
184
  atoms. It is not a substitute for RDKit's model.
185
 
186
- 4. **"Every string is valid" holds only because the decoder is total.** Unparseable tokens
187
- and over-valent bonds are silent no-ops, so a mutated string can decode to something only
188
- loosely related to its neighbour. Validity is guaranteed; locality under mutation is not.
189
-
190
- 5. **Insertion/deletion is non-local.** Inserting or deleting an atom shifts every `^d` that
191
  spans the edit point. Inherent to relative back-references; applies equally to DeepSMILES.
192
 
193
- 6. **Not a canonical hash.** One molecule has many valid BRAID strings. The encoder uses
194
  RDKit canonical-rank rooting for determinism, but uniqueness is not proven.
195
 
196
- 7. **Depends on RDKit** for parsing and sanitization. This is a reference implementation,
197
- not a self-contained spec.
198
-
199
- ### Limitations of the evaluation specifically
200
-
201
- - **One dataset, one split.** BBBP only. No claim is made about BACE, HIV, Tox21, SIDER, or
202
- ClinTox. Results on a single scaffold split of a ~2k-molecule dataset are fragile.
203
- - **Small pretraining corpus.** ZINC 100k is tiny by chemical-LM standards. Any notation
204
- effect that only emerges at scale would be invisible here.
205
- - **Seed variance, not split variance.** The ± reflects finetuning seed noise under a fixed
206
- split. It does not capture how much the answer would move under a different scaffold split,
207
- which is typically larger.
208
- - **n = 5.** Adequate to show two models are *indistinguishable*; nowhere near adequate to
209
- establish a small real difference in either direction.
210
-
211
  ---
212
 
213
  ## Citation
214
 
215
- BRAID is a recombination of published mechanisms and should not be cited as a novel
216
- notation. Please cite the underlying work:
217
 
218
  - O'Boyle, N. & Dalke, A. *DeepSMILES: An Adaptation of SMILES for Use in Machine-Learning
219
  of Chemical Structures.* ChemRxiv, 2018.
 
100
  noise, and the per-seed ranges overlap almost completely (BRAID 0.710–0.751, control
101
  0.712–0.738).
102
 
103
+ The correct reading is
104
 
105
  > Under matched pretraining and finetuning, BRAID's guaranteed-validity property comes
106
  > **at no measurable cost to downstream predictive performance** on BBBP.
107
 
 
 
 
108
 
109
+
110
+ Note: The seed-to-seed spread (±0.017) exceeds the between-model difference. Any comparison
111
  at this scale reporting a single seed is measuring noise.
112
 
113
  ---
 
162
 
163
  ## Limitations
164
 
165
+ 1. **Stereochemistry is partial.** Tetrahedral chirality and E/Z double bonds round-trip on
 
 
 
 
 
 
 
 
 
 
 
166
  26/26 test cases. Allene/axial/planar chirality, atropisomers, and non-tetrahedral stereo
167
  centres fall back to unspecified.
168
 
169
+ 2. **The valence state machine is approximate.** A hand-rolled charge→valence rule, correct
170
  for common ions but liable to mis-clamp hypervalent, organometallic, or unusual-charge
171
  atoms. It is not a substitute for RDKit's model.
172
 
173
+ 3. **Insertion/deletion is non-local.** Inserting or deleting an atom shifts every `^d` that
 
 
 
 
174
  spans the edit point. Inherent to relative back-references; applies equally to DeepSMILES.
175
 
176
+ 4. **Not a canonical hash.** One molecule has many valid BRAID strings. The encoder uses
177
  RDKit canonical-rank rooting for determinism, but uniqueness is not proven.
178
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
179
  ---
180
 
181
  ## Citation
182
 
183
+ BRAID is a recombination of published mechanisms. Please cite the underlying work:
 
184
 
185
  - O'Boyle, N. & Dalke, A. *DeepSMILES: An Adaptation of SMILES for Use in Machine-Learning
186
  of Chemical Structures.* ChemRxiv, 2018.