|
|
--- |
|
|
language: |
|
|
- en |
|
|
license: mit |
|
|
tags: |
|
|
- drug-discovery |
|
|
- binding-affinity |
|
|
- protein-ligand |
|
|
- graph-neural-network |
|
|
- esm2 |
|
|
- drug-repurposing |
|
|
- multimodal |
|
|
- transfer-learning |
|
|
datasets: |
|
|
- pdbbind-v2020 |
|
|
metrics: |
|
|
- rmse |
|
|
- pearsonr |
|
|
pipeline_tag: other |
|
|
--- |
|
|
|
|
|
# DeepPharm: Multi-Modal Transfer Learning for Drug-Target Affinity Prediction |
|
|
|
|
|
## Model Description |
|
|
|
|
|
**DeepPharm** is a multi-modal deep learning framework for predicting protein–ligand binding affinity ($pK$). It combines: |
|
|
|
|
|
- **GATv2** molecular graph encoder (3 layers, 4 heads) |
|
|
- **ECFP4** fingerprint MLP encoder (2048→128) |
|
|
- **Gated Fusion** mechanism for adaptive ligand representation |
|
|
- **ESM-2** protein language model (150M params, fine-tuned) |
|
|
- **Stacked Cross-Attention** (2 layers, 4 heads) for drug-protein interaction |
|
|
- **Residual Prediction Head** with SiLU activation |
|
|
|
|
|
### Two Modes of Operation |
|
|
|
|
|
| Mode | Task | Input | Output | |
|
|
|------|------|-------|--------| |
|
|
| **Mode A** | Supervised affinity prediction | Drug SMILES + Protein sequence | pK value | |
|
|
| **Mode B** | Weakly supervised drug repurposing | Drug + Disease signature | Ranked candidates | |
|
|
|
|
|
## Performance |
|
|
|
|
|
### Systematic Ablation (PDBbind v2020, $N_{test}=3{,}775$) |
|
|
|
|
|
| Config | RMSE ↓ | Pearson ↑ | Spearman ↑ | |
|
|
|--------|--------|-----------|------------| |
|
|
| V1 Baseline (ESM-35M) | 1.266 | 0.743 | 0.743 | |
|
|
| V2 Architecture | 1.258 | 0.748 | 0.746 | |
|
|
| V2 + CosineWR | 1.244 | 0.753 | 0.750 | |
|
|
| **V2 + ESM-150M (Best)** | **1.229** | **0.762** | **0.760** | |
|
|
| V2 + EMA | 1.247 | 0.753 | 0.753 | |
|
|
|
|
|
### Five-Seed Ensemble (Best Configuration) |
|
|
|
|
|
| Metric | Mean ± Std | |
|
|
|--------|-----------| |
|
|
| RMSE | 1.246 ± 0.005 | |
|
|
| Pearson r | 0.751 ± 0.002 | |
|
|
| Spearman ρ | 0.750 ± 0.002 | |
|
|
|
|
|
CV < 0.4% confirms high reproducibility. |
|
|
|
|
|
### Baselines (all re-implemented on same split) |
|
|
|
|
|
| Model | RMSE ↓ | Pearson ↑ | |
|
|
|-------|--------|-----------| |
|
|
| DeepDTA (CNN) | 1.48 | 0.61 | |
|
|
| GraphDTA (GCN) | 1.39 | 0.67 | |
|
|
| MolCLR* | 1.30 | 0.74 | |
|
|
| DrugBAN | 1.28 | 0.76 | |
|
|
| **DeepPharm V2** | **1.23** | **0.76** | |
|
|
|
|
|
## Intended Use |
|
|
|
|
|
- High-throughput virtual screening of drug candidates |
|
|
- Binding affinity prediction for drug-target pairs |
|
|
- Hypothesis generation for drug repurposing in orphan diseases |
|
|
- Research and academic purposes |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- 2D topological encoder; cannot distinguish stereoisomers |
|
|
- Trained on PDBbind v2020, which overrepresents kinases |
|
|
- Mode B uses drug priors (guilt-by-association), not zero-shot inference |
|
|
- Predictions require experimental validation |
|
|
|
|
|
## Training Details |
|
|
|
|
|
- **Dataset:** PDBbind v2020 General Set (15,100 train / 3,775 test, seed=42) |
|
|
- **Hardware:** 1× NVIDIA H100 80 GB |
|
|
- **Optimizer:** AdamW (backbone LR: 5e-6, head LR: 8e-4) |
|
|
- **Scheduler:** CosineAnnealing with Warm Restarts ($T_0$=10, $T_{mult}$=2) |
|
|
- **Loss:** MSE + 0.3·RankingLoss + 0.2·HuberLoss |
|
|
- **Training time:** ~11 min/epoch (ESM-2 150M), best checkpoint at epoch 18 |
|
|
|
|
|
## Available Checkpoints |
|
|
|
|
|
| File | Description | RMSE | |
|
|
|------|-------------|------| |
|
|
| `best_v2_esm150m.pt` | Best V2 model (ESM-2 150M) | 1.229 | |
|
|
| `best_v1_esm35m.pt` | V1 Baseline (ESM-2 35M) | 1.266 | |
|
|
|
|
|
## How to Use |
|
|
|
|
|
```python |
|
|
from huggingface_hub import hf_hub_download |
|
|
|
|
|
# Download the best model |
|
|
path = hf_hub_download("chamoso/DeepPharm", "best_v2_esm150m.pt") |
|
|
|
|
|
# Load in PyTorch |
|
|
import torch |
|
|
checkpoint = torch.load(path, map_location="cpu") |
|
|
``` |
|
|
|
|
|
For full inference with data preprocessing: |
|
|
|
|
|
```bash |
|
|
git clone https://github.com/chamoso/DeepPharm.git |
|
|
cd DeepPharm |
|
|
python scripts/predict.py \ |
|
|
--checkpoint weights/best_v2_esm150m.pt \ |
|
|
--smiles "CC(=O)Oc1ccccc1C(=O)O" \ |
|
|
--sequence "MKTAYIAKQRQISFVKSHFSRQLE..." |
|
|
``` |
|
|
|
|
|
## Links |
|
|
|
|
|
- **GitHub:** [chamoso/DeepPharm](https://github.com/chamoso/DeepPharm) |
|
|
- **Live Demo:** [HuggingFace Spaces](https://huggingface.co/spaces/chamoso/DeepPharm) |
|
|
|
|
|
## Citation |
|
|
|
|
|
*Preprint coming soon.* |
|
|
|
|
|
## License |
|
|
|
|
|
MIT License |
|
|
|
