File size: 2,875 Bytes
08e8a4c 4106c06 08e8a4c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 |
---
license: mit
tags:
- chemistry
- molecular-generation
- structure-based-design
- transformer
- reinforcement-learning
- gpt-2
datasets:
- HUBioDataLab/DrugGEN-chembl-smiles
- antoinebcx/smiles-molecules-moses
base_model:
- openai-community/gpt2
pipeline_tag: reinforcement-learning
library_name: transformers
---
# 🏂 ProteinSkier
**ProteinSkier** is a GPT-2–based language model that “carves fresh lines” through chemical space, producing drug-like SMILES strings with an explicit bias toward **ADMET quality, novelty, and synthesizability**.
## 1 · Why another generative model?
Traditional generative models often rediscover known scaffolds or output molecules that fail late-stage ADMET filters.
ProteinSkier addresses this by coupling large-scale pre-training on ~2 M curated molecules **with a second-stage Reinforcement Fine-Tuning (RFT)** that rewards:
| Component | Reward signal (λ) | Source |
|-----------|-------------------|--------|
| **Validity** | hard filter | RDKit sanitisation |
| **QED ↑** | 0.35 | RDKit |
| **Novelty ↑** | 0.25 | training-set hash table |
| **Lipinski pass ↑** | 0.20 | RDKit |
| **logP in [–1, 4]** | 0.10 | RDKit |
| **Predicted tox ↓** | 0.10 | internal classifier |
The policy is updated with *policy-gradient REINFORCE*; low-quality trajectories are rejected via an adaptive threshold (see `FullDatasetRFTTrainer` in the code).
## 2 · Intended uses & scope
| Stage | Example use-case | Not a good fit |
|-------|------------------|----------------|
| *Hit finding* | Rapidly scaffold-hop around a weak binder identified by docking. | Predicting absolute IC₅₀/Kᵢ values. |
| *Lead optimisation* | Generating analogues that respect Lipinski & BBB guidelines. | Ensuring synthetic accessibility without chemist review. |
| *Ideation / teaching* | Demonstrating language-model chemistry in the classroom. | Production-scale enumeration without downstream filtering. |
## 3 · Quick start
> Requires `transformers ≥ 4.42`, `torch ≥ 2.2`, `rdkit`, `accelerate`.
```python
from transformers import AutoTokenizer, GPT2LMHeadModel
model_id = "ProteinDance/ProteinSkier"
tok = AutoTokenizer.from_pretrained(model_id)
model = GPT2LMHeadModel.from_pretrained(model_id)
# Generate 5 novel molecules
prompt = tok("<bos>", return_tensors="pt").input_ids
gen = model.generate(
prompt.repeat(5, 1),
max_length=128,
do_sample=True,
top_p=0.95,
temperature=0.7,
)
smiles = tok.batch_decode(gen, skip_special_tokens=True)
print("\n".join(smiles))
```
## 4 · Limitations & caveats
- No guaranteed synthesizability – always perform retrosynthetic analysis.
- Property estimators used in RFT are fast; wet-lab assays will vary.
- Output may include patented molecules – run IP checks.
- ADMET focus biases chemistry toward oral drugs; unsuitable for agrochemicals or materials.
|