| --- |
| license: mit |
| base_model: facebook/esm2_t33_650M_UR50D |
| inference: false |
| tags: |
| - protein |
| - system-one |
| - protein-engineering |
| - protein-language-model |
| - mutation-effect-prediction |
| - biology |
| - AI4Science |
| - esm2 |
| --- |
| |
| # A System One Approach to General Protein Evolution |
|
|
|  |
|
|
| Pev proposes the next single-residue edit to a protein and says how likely it is to |
| help. One forward pass over the parent scores **every** legal substitution and STOP |
| in parallel, so a decision costs one backbone pass rather than one per candidate -- |
| over **800×** the throughput of a per-mutant encoder at 1,000 candidates. Trained |
| once on measured outcomes from many families, applied **zero-shot** to families it |
| has never seen: peptides, small folded domains and antibodies, under binding or |
| stability. |
|
|
| Two heads, two questions. `p_choice[a]` is the probability that edit `a` wins the |
| panel it was offered; `p_improve[a]` that it improves the measured outcome. Both are |
| calibrated post-hoc on held-out families. |
|
|
| ## Results |
|
|
| The discovery screen: one measured reference protein, every measured single |
| substitution of it, ten wells, on families reserved before training. |
|
|
| | method | Hit rate @10 ↑ | Enrichment @10 ↑ | AUROC ↑ | Wells to first hit ↓ | |
| | --- | --- | --- | --- | --- | |
| | *Perfect ranking* | *0.949* | *18.94×* | *1.000* | *1.0* | |
| | **Pev** | **0.424** | **7.19×** | **0.729** | **6.1** | |
| | ESM-2 zero-shot | 0.231 | 2.80× | 0.545 | 48.3 | |
| | Random | 0.128 | 1.02× | 0.497 | 27.8 | |
|
|
| By class: **10.96×** on domains, **1.32×** on peptides, **2.16×** on antibodies. |
| Calibration halves the calibration error (ECE 0.182 → 0.096 on `p_improve`) and is |
| rank-preserving, so every number holds with and without it. |
|
|
| ## Files |
|
|
| | file | | |
| | --- | --- | |
| | `checkpoints/pev/weights.pt` | 6 MB. Heads, conditioning and LoRA slots only. | |
| | `checkpoints/pev/metadata.json` | architecture, backbone fingerprint, and what the two probabilities are probabilities *of*. | |
| | `checkpoints/pev/calibration.json` | the post-hoc calibrator. | |
|
|
| The frozen ESM-2 650M backbone is **not** stored here: it is 400× the size of what |
| was trained and reconstructible by name, so it is fetched from |
| `facebook/esm2_t33_650M_UR50D` on first use and checked against the fingerprint in |
| `metadata.json` -- a different cached copy is reported rather than silently changing |
| every number. |
|
|
| ## Usage |
|
|
| Pev is not a `transformers` model and does not load with `AutoModel`. Install the |
| `pev` package from the project repository, then point it at this checkpoint: |
|
|
| ```python |
| import json |
| from huggingface_hub import snapshot_download |
| from pev.loading import config_from_metadata, load_checkpoint |
| from pev.model import DecisionModel |
| |
| path = f"{snapshot_download('Yibooooo/Pev')}/checkpoints/pev" |
| model = DecisionModel(config_from_metadata(json.load(open(f"{path}/metadata.json")))) |
| if model.cfg.lora_last_n_layers: |
| model.encoder.enable_lora() |
| load_checkpoint(path, model) # warns if the backbone differs |
| model.eval() |
| ``` |
|
|
| The project repository also carries the discovery screen and the pinned 45-panel |
| benchmark, so the table above can be reproduced end to end against a verified copy |
| of the evaluation split. |
|
|
| ## Intended use |
|
|
| For **ranking candidate single substitutions** in a protein-engineering campaign, as |
| a prior when no assay data on the target exists yet -- exactly when a family-specific |
| model cannot be fitted. Out of scope, and wrong if you use it this way: |
|
|
| - **Insertions, deletions, multi-residue designs.** Single substitutions only. |
| - **Objectives it was not trained on.** Binding and stability only; not expression, |
| solubility, immunogenicity or developability. |
| - **A substitute for assays.** The headline means "2 of every 5 wells return a |
| beneficial edit rather than 1 in 8", not that the top-ranked edit works. |
| - **Antibodies, with care.** The weakest class: five test families, and the untrained |
| backbone still ranks slightly better on enrichment. |
| - **Anything clinical or safety-critical.** |
|
|
| ## Data |
|
|
| Trained on public mutational-scanning and binding datasets -- AlphaSeq, MegaScale, |
| FireProtDB, ProteinGym, SKEMPI 2.0, SLiM DMS, AbAgym, FLAb -- split by family, a |
| family held out for one property held out for every property. The evaluation split is |
| 214,378 variants across 165 held-out families, carrying no training families; the |
| training families and the training code are not distributed. Please cite the upstream |
| datasets alongside this work. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{wen2026pev, |
| title={A System One Approach to General Protein Evolution}, |
| author={Pev Team}, |
| year={2026}, |
| howpublished={\url{https://yibow.me/pev}}, |
| url={https://yibow.me/pev}, |
| } |
| ``` |
|
|
