|
|
--- |
|
|
title: "ESM2 Long Models" |
|
|
--- |
|
|
|
|
|
## ESM2 Long |
|
|
|
|
|
ESM2 Long is an adapted version of the ESM2 architectures. It uses local attention instead of global attention, allowing for models with longer input sizes. ESM2 Long models have a context size of 2,050, double that of the standard ESM2 model. Several ESM2 Long models are available: |
|
|
|
|
|
| Model | Num layers | |
|
|
|------------------------------|----| |
|
|
| [gabrielbianchin/esm2_t33_long](https://huggingface.co/gabrielbianchin/esm2_t33_long) | 33 | |
|
|
| [gabrielbianchin/esm2_t30_long](https://huggingface.co/gabrielbianchin/esm2_t30_long) | 30 | |
|
|
| [gabrielbianchin/esm2_t12_long](https://huggingface.co/gabrielbianchin/esm2_t12_long) | 12 | |
|
|
| [gabrielbianchin/esm2_t6_long](https://huggingface.co/gabrielbianchin/esm2_t6_long) | 6 | |
|
|
|
|
|
For detailed information, please refer to the paper. |
