keras/esm2_t6_8M

Model Overview

Evolutionary Scale Modeling (ESM) version 2 architecture.

Reference

• Evolutionary-scale prediction of atomic level protein structure with a language model

ESM-2 is a state-of-the-art protein model trained on a masked language modelling objective. It is suitable for fine-tuning on a wide range of tasks that take protein sequences as input.

Links

• [ESM-2 Quickstart Notebook] - coming soon
• [ESM-2 API Documentation] - coming soon
• [ESM-2 Beginner Guide] - coming soon
• KerasHub Model Publishing Guide

Installation

Keras and KerasHub can be installed with:

pip install -U -q keras-hub
pip install -U -q keras

Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the Keras Getting Started page.

Presets

The following model checkpoints are provided by the Keras team. Weights have been ported from: https://huggingface.co. Full code examples for each are available below.

Preset name	Parameters	Description
esm2_t6_8M	8M	6 transformer layers version of the ESM-2 protein language model, trained on the UniRef50 clustered protein sequence dataset.
esm2_t12_35M	35M	12 transformer layers version of the ESM-2 protein language model, trained on the UniRef50 clustered protein sequence dataset.
esm2_t30_150M	150M	30 transformer layers version of the ESM-2 protein language model, trained on the UniRef50 clustered protein sequence dataset.
esm2_t33_650M	650M	33 transformer layers version of the ESM-2 protein language model, trained on the UniRef50 clustered protein sequence dataset.