File size: 4,861 Bytes
8775a98 575718f 8775a98 575718f 66ee258 575718f e4534d7 575718f 5f87c43 575718f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 | ---
license: mit
base_model: facebook/esm2_t33_650M_UR50D
inference: false
tags:
- protein
- system-one
- protein-engineering
- protein-language-model
- mutation-effect-prediction
- biology
- AI4Science
- esm2
---
# A System One Approach to General Protein Evolution
Pev proposes the next single-residue edit to a protein and says how likely it is to
help. One forward pass over the parent scores **every** legal substitution and STOP
in parallel, so a decision costs one backbone pass rather than one per candidate --
over **800×** the throughput of a per-mutant encoder at 1,000 candidates. Trained
once on measured outcomes from many families, applied **zero-shot** to families it
has never seen: peptides, small folded domains and antibodies, under binding or
stability.
Two heads, two questions. `p_choice[a]` is the probability that edit `a` wins the
panel it was offered; `p_improve[a]` that it improves the measured outcome. Both are
calibrated post-hoc on held-out families.
## Results
The discovery screen: one measured reference protein, every measured single
substitution of it, ten wells, on families reserved before training.
| method | Hit rate @10 ↑ | Enrichment @10 ↑ | AUROC ↑ | Wells to first hit ↓ |
| --- | --- | --- | --- | --- |
| *Perfect ranking* | *0.949* | *18.94×* | *1.000* | *1.0* |
| **Pev** | **0.424** | **7.19×** | **0.729** | **6.1** |
| ESM-2 zero-shot | 0.231 | 2.80× | 0.545 | 48.3 |
| Random | 0.128 | 1.02× | 0.497 | 27.8 |
By class: **10.96×** on domains, **1.32×** on peptides, **2.16×** on antibodies.
Calibration halves the calibration error (ECE 0.182 → 0.096 on `p_improve`) and is
rank-preserving, so every number holds with and without it.
## Files
| file | |
| --- | --- |
| `checkpoints/pev/weights.pt` | 6 MB. Heads, conditioning and LoRA slots only. |
| `checkpoints/pev/metadata.json` | architecture, backbone fingerprint, and what the two probabilities are probabilities *of*. |
| `checkpoints/pev/calibration.json` | the post-hoc calibrator. |
The frozen ESM-2 650M backbone is **not** stored here: it is 400× the size of what
was trained and reconstructible by name, so it is fetched from
`facebook/esm2_t33_650M_UR50D` on first use and checked against the fingerprint in
`metadata.json` -- a different cached copy is reported rather than silently changing
every number.
## Usage
Pev is not a `transformers` model and does not load with `AutoModel`. Install the
`pev` package from the project repository, then point it at this checkpoint:
```python
import json
from huggingface_hub import snapshot_download
from pev.loading import config_from_metadata, load_checkpoint
from pev.model import DecisionModel
path = f"{snapshot_download('Yibooooo/Pev')}/checkpoints/pev"
model = DecisionModel(config_from_metadata(json.load(open(f"{path}/metadata.json"))))
if model.cfg.lora_last_n_layers:
model.encoder.enable_lora()
load_checkpoint(path, model) # warns if the backbone differs
model.eval()
```
The project repository also carries the discovery screen and the pinned 45-panel
benchmark, so the table above can be reproduced end to end against a verified copy
of the evaluation split.
## Intended use
For **ranking candidate single substitutions** in a protein-engineering campaign, as
a prior when no assay data on the target exists yet -- exactly when a family-specific
model cannot be fitted. Out of scope, and wrong if you use it this way:
- **Insertions, deletions, multi-residue designs.** Single substitutions only.
- **Objectives it was not trained on.** Binding and stability only; not expression,
solubility, immunogenicity or developability.
- **A substitute for assays.** The headline means "2 of every 5 wells return a
beneficial edit rather than 1 in 8", not that the top-ranked edit works.
- **Antibodies, with care.** The weakest class: five test families, and the untrained
backbone still ranks slightly better on enrichment.
- **Anything clinical or safety-critical.**
## Data
Trained on public mutational-scanning and binding datasets -- AlphaSeq, MegaScale,
FireProtDB, ProteinGym, SKEMPI 2.0, SLiM DMS, AbAgym, FLAb -- split by family, a
family held out for one property held out for every property. The evaluation split is
214,378 variants across 165 held-out families, carrying no training families; the
training families and the training code are not distributed. Please cite the upstream
datasets alongside this work.
## Citation
```bibtex
@misc{wen2026pev,
title={A System One Approach to General Protein Evolution},
author={Pev Team},
year={2026},
howpublished={\url{https://yibow.me/pev}},
url={https://yibow.me/pev},
}
```
|