File size: 12,150 Bytes
ea77201
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
"""Feature extraction for the hERG Phase-5 models — end-to-end from SMILES, RDKit only.

Reproduces exactly the feature blocks used by the Phase-5 XGBoost models (Approach A
and the XGBoost legs of Approaches B/C), and is backwards-compatible with the Phase-3/4
blocks (Morgan / MACCS / 217 RDKit descriptors are computed with the same settings as
work/step0_lock_splits_features.py, so Phase-5 models can be compared like-for-like).

Blocks (in canonical assembly order):
  morgan          ECFP4 Morgan bits, radius 2, 2048 bits                         (Phase 3/4)
  maccs           MACCS keys, 166 bits (167-bit vector with placeholder bit 0 dropped) (Phase 3/4)
  desc            217 RDKit molecular descriptors (Descriptors.descList, the exact
                  set/order of the Phase-3/4 cache, work/desc_names.json)       (Phase 3/4)
  physchem_extra  [QED] — the one standard physicochemical property absent from the
                  217-descriptor set (verified programmatically; TPSA, MolLogP,
                  FractionCSP3, NumAromaticRings etc. are already inside `desc`,
                  so they are deliberately NOT duplicated here)                  (Phase 5)
  morgan_counts   Morgan radius-2 atom-environment COUNTS folded to 2048
                  (assembly applies log1p)                                       (Phase 5)
  ecfp6           Morgan bits, radius 3, 2048 bits                               (Phase 5)
  fcfp4           Morgan bits, radius 2, 2048 bits, feature-class atom invariants
                  (FCFP4)                                                        (Phase 5)
  herg            hERG-targeted features (13, continuous) — see featurize_herg()   (Phase 5)
  pharm           2D pharmacophore pair counts (60, counts) — see featurize_pharm() (Phase 5)
  morgan1024      Morgan ECFP4 bits folded to 1024 (added 2026-11-13 at Hossain's
                  instruction, alongside the 2048-bit default)                    (Phase 5)
  morgan1024_counts  Morgan ECFP4 counts folded to 1024 (assembly applies log1p)  (Phase 5)
  cdk             OPTIONAL, not computed here: CDK fingerprints come from
                  PaDEL-Descriptor (separate Java tool); see PHASE5_RESULTS.md.

Usage:
  from feature_extraction import featurize_smiles, BLOCK_ORDER
  X, blocks = featurize_smiles("CCOc1ccc(cc1)...")   # blocks: dict name -> 1-D np.array

Preprocessing applied at model time (NOT in this module; fit on training folds only):
  - desc / physchem_extra / herg: drop near-constant & >5%-NaN columns (fit on train),
    median-impute (train medians), standardize (train mean/SD) — Phase 3/4 DescPrep logic.
  - morgan_counts / pharm: log1p, then the same standardisation as above.
  - bit fingerprints (morgan / maccs / ecfp6 / fcfp4 / cdk): used raw.
"""
import os
import numpy as np
from rdkit import Chem, RDConfig, RDLogger
from rdkit.Chem import AllChem, MACCSkeys, rdMolDescriptors, rdFingerprintGenerator
from rdkit.Chem import ChemicalFeatures, QED
from rdkit.Chem.Descriptors import MolLogP

RDLogger.DisableLog("rdApp.*")

from rdkit.Chem import Descriptors as _Desc
DESC_NAMES = [n for n, _ in _Desc.descList]  # 217, fixed order (== Phase-3/4 desc_names.json)
_DESC_FNS = dict(_Desc.descList)
BLOCK_ORDER = ["morgan", "maccs", "desc", "physchem_extra", "morgan_counts",
               "ecfp6", "fcfp4", "herg", "pharm", "morgan1024", "morgan1024_counts"]
BLOCK_SIZES = {"morgan": 2048, "maccs": 166, "desc": 217, "physchem_extra": 1,
               "morgan_counts": 2048, "ecfp6": 2048, "fcfp4": 2048, "herg": 13, "pharm": 60,
               "morgan1024": 1024, "morgan1024_counts": 1024}

# ---------------------------------------------------------------- hERG block ----
# Method (documented approximation, no fitted parameters):
# * Protonatable / ionizable centres are identified with RDKit's BaseFeatures
#   feature factory (PosIonizable / NegIonizable families).
# * Each PosIonizable centre is assigned a class pKa by atom environment:
#     aromatic pyridine-like N ......... 5.2
#     imidazole-like aromatic N (5-ring, >=2 ring N) ... 7.0
#     aniline-like N (attached to aromatic C) ......... 4.6
#     other (aliphatic amine etc.) .................... 9.5
#   Fraction protonated at pH 7.4 for a centre: f = 1 / (1 + 10^(7.4 - pKa)).
# * logD(7.4) estimate: MolLogP - log10(1 + 10^(pKa_base - 7.4)) using the strongest
#   basic centre, and additionally - log10(1 + 10^(7.4 - 4.5)) if a carboxylic-acid
#   group is present (assumed acid pKa 4.5). This is a crude class-pKa approximation
#   of logD, NOT a measured or rigorously predicted logD — it is labelled as such
#   everywhere it is used.
_CLASS_PKA = {"pyridine": 5.2, "imidazole": 7.0, "aniline": 4.6, "aliphatic": 9.5}
_ACID_PKA = 4.5
_PH = 7.4
_factory = None


def _get_factory():
    global _factory
    if _factory is None:
        cands = [os.path.join(RDConfig.RDDataDir, "BaseFeatures.fdef"),
                 os.path.join(RDConfig.RDDataDir, "Code", "GraphMol", "ChemicalFeatures",
                              "BaseFeatures.fdef")]
        fdef = next(p for p in cands if os.path.exists(p))
        _factory = ChemicalFeatures.BuildFeatureFactory(fdef)
    return _factory


def _classify_base(atom, mol):
    if atom.GetIsAromatic():
        ring = mol.GetRingInfo()
        for r in ring.AtomRings():
            if atom.GetIdx() in r and len(r) == 5 and sum(1 for i in r if mol.GetAtomWithIdx(i).GetAtomicNum() == 7) >= 2:
                return "imidazole"
        return "pyridine"
    for nb in atom.GetNeighbors():
        if nb.GetIsAromatic():
            return "aniline"
    return "aliphatic"


def _frac_prot(pka, ph=_PH):
    return 1.0 / (1.0 + 10.0 ** (ph - pka))


HERG_NAMES = ["n_pos_ionizable", "n_neg_ionizable", "expected_protonated_pH74",
              "strongest_base_pKa", "logD74_est", "formal_charge", "abs_formal_charge",
              "n_aromatic_rings", "frac_aromatic_atoms", "n_amine_like_N",
              "n_carboxylic_acid", "has_base", "has_acid"]


def featurize_herg(mol):
    feats = _get_factory().GetFeaturesForMol(mol)
    pos, neg = [], []
    for f in feats:
        if f.GetFamily() == "PosIonizable":
            pos.append(f)
        elif f.GetFamily() == "NegIonizable":
            neg.append(f)
    pkas = []
    exp_prot = 0.0
    for f in pos:
        aidx = f.GetAtomIds()[0]
        cls = _classify_base(mol.GetAtomWithIdx(aidx), mol)
        pka = _CLASS_PKA[cls]
        pkas.append(pka)
        exp_prot += _frac_prot(pka)
    logp = MolLogP(mol)
    logd = logp
    if pkas:
        logd -= np.log10(1.0 + 10.0 ** (max(pkas) - _PH))
    n_acid = len(mol.GetSubstructMatches(Chem.MolFromSmarts("[CX3](=O)[OX2H1]")))
    if n_acid:
        logd -= np.log10(1.0 + 10.0 ** (_PH - _ACID_PKA))
    n_ar = sum(1 for a in mol.GetAtoms() if a.GetIsAromatic())
    amine_like = len(mol.GetSubstructMatches(
        Chem.MolFromSmarts("[NX3;!$(NC=O);!$(NS(=O)(=O));!$(N=*);!$(N#*)]")))
    fc = Chem.GetFormalCharge(mol)
    return np.array([
        len(pos), len(neg), exp_prot,
        max(pkas) if pkas else 0.0,
        logd, fc, abs(fc),
        rdMolDescriptors.CalcNumAromaticRings(mol),
        n_ar / max(mol.GetNumAtoms(), 1),
        amine_like, n_acid,
        1.0 if pkas else 0.0, 1.0 if n_acid else 0.0,
    ], dtype=float)


# ---------------------------------------------------------------- pharm block ---
# 2D pharmacophore pair counts. Feature points from RDKit's BaseFeatures factory,
# families mapped to 5 classes: Donor, Acceptor, Hydrophobe (Hydrophobe +
# LumpedHydrophobe + AromaticRing), PosIonizable, NegIonizable. For every pair of
# points (same point excluded; pairs taken with repetition over classes), the
# minimum topological distance between the two points' atoms is binned as
# <=2, 3-4, 5-6, >=7, giving 15 class-pairs x 4 bins = 60 counts.
_PHARM_FAMS = ["Donor", "Acceptor", "Hydrophobe", "PosIonizable", "NegIonizable"]
_FAM_MAP = {"Donor": "Donor", "Acceptor": "Acceptor", "Hydrophobe": "Hydrophobe",
            "LumpedHydrophobe": "Hydrophobe", "AromaticRing": "Hydrophobe",
            "PosIonizable": "PosIonizable", "NegIonizable": "NegIonizable",
            "RingDonor": "Donor", "RingAcceptor": "Acceptor"}
PHARM_NAMES = [f"{a}|{b}|d{db}" for i, a in enumerate(_PHARM_FAMS) for b in _PHARM_FAMS[i:]
               for db in ["<=2", "3-4", "5-6", ">=7"]]


def _dist_bin(d):
    return 0 if d <= 2 else 1 if d <= 4 else 2 if d <= 6 else 3


def featurize_pharm(mol):
    feats = _get_factory().GetFeaturesForMol(mol)
    pts = []
    for f in feats:
        fam = _FAM_MAP.get(f.GetFamily())
        if fam is not None:
            pts.append((fam, set(f.GetAtomIds())))
    out = np.zeros(60, dtype=float)
    if len(pts) < 2:
        return out
    dm = Chem.GetDistanceMatrix(mol)
    pair_index = {}
    k = 0
    for i, a in enumerate(_PHARM_FAMS):
        for b in _PHARM_FAMS[i:]:
            pair_index[(a, b)] = k
            k += 4
    for i in range(len(pts)):
        for j in range(i + 1, len(pts)):
            fa, aa = pts[i]
            fb, ab = pts[j]
            d = min(dm[x, y] for x in aa for y in ab)
            key = (fa, fb) if (fa, fb) in pair_index else (fb, fa)
            out[pair_index[key] + _dist_bin(int(d))] += 1.0
    return out


# ---------------------------------------------------------------- main entry ----
_morgan_gen = rdFingerprintGenerator.GetMorganGenerator(radius=2, fpSize=2048)
_morgan_gen_1024 = rdFingerprintGenerator.GetMorganGenerator(radius=2, fpSize=1024)
_morgan_gen_r3 = rdFingerprintGenerator.GetMorganGenerator(radius=3, fpSize=2048)
_fcfp_gen = rdFingerprintGenerator.GetMorganGenerator(
    radius=2, fpSize=2048,
    atomInvariantsGenerator=rdFingerprintGenerator.GetMorganFeatureAtomInvGen())
from rdkit.ML.Descriptors.MoleculeDescriptors import MolecularDescriptorCalculator
_CALC = MolecularDescriptorCalculator(DESC_NAMES)


def _desc_vector(mol):
    # C++-loop calculator over the descList functions; identical values/order to
    # calling each descList function in Python, several-fold faster.
    try:
        return np.array(_CALC.CalcDescriptors(mol), dtype=float)
    except Exception:
        out = np.empty(len(DESC_NAMES), dtype=float)
        for i, n in enumerate(DESC_NAMES):
            try:
                out[i] = _DESC_FNS[n](mol)
            except Exception:
                out[i] = np.nan
        return out


def featurize_mol(mol):
    """mol: RDKit Mol -> dict block name -> 1-D float32 array (NaN-free except desc)."""
    out = {}
    out["morgan"] = np.asarray(_morgan_gen.GetFingerprint(mol), dtype=np.float32)
    out["maccs"] = np.asarray(MACCSkeys.GenMACCSKeys(mol), dtype=np.float32)[1:]  # drop bit 0
    out["desc"] = _desc_vector(mol)
    out["physchem_extra"] = np.array([QED.qed(mol)], dtype=float)
    counts = _morgan_gen.GetCountFingerprint(mol)
    arr = np.zeros(2048, dtype=np.float32)
    for idx, c in counts.GetNonzeroElements().items():
        arr[idx % 2048] += c
    out["morgan_counts"] = arr
    out["ecfp6"] = np.asarray(_morgan_gen_r3.GetFingerprint(mol), dtype=np.float32)
    out["fcfp4"] = np.asarray(_fcfp_gen.GetFingerprint(mol), dtype=np.float32)
    out["herg"] = featurize_herg(mol)
    out["pharm"] = featurize_pharm(mol)
    # Morgan ECFP4 at 1024 bits (bits + counts) — added to the Phase-5 ablation by
    # Hossain's instruction of 2026-11-13 (tests whether the smaller, denser folding
    # beats the 2048-bit default).
    out["morgan1024"] = np.asarray(_morgan_gen_1024.GetFingerprint(mol), dtype=np.float32)
    counts = _morgan_gen_1024.GetCountFingerprint(mol)
    arr = np.zeros(1024, dtype=np.float32)
    for idx, c in counts.GetNonzeroElements().items():
        arr[idx % 1024] += c
    out["morgan1024_counts"] = arr
    return out


def featurize_smiles(smiles):
    """SMILES -> (assembled float32 vector in BLOCK_ORDER, blocks dict). None if unparseable."""
    mol = Chem.MolFromSmiles(smiles)
    if mol is None:
        return None, None
    blocks = featurize_mol(mol)
    return np.concatenate([blocks[b] for b in BLOCK_ORDER]).astype(np.float32), blocks