prasadsachin commited on
Commit
95b1f97
·
verified ·
1 Parent(s): c4db4fb

Update README.md with new model card content

Browse files
Files changed (1) hide show
  1. README.md +39 -22
README.md CHANGED
@@ -1,25 +1,42 @@
1
  ---
2
  library_name: keras-hub
3
  ---
4
- This is a [`ESM` model](https://keras.io/api/keras_hub/models/esm) uploaded using the KerasHub library and can be used with JAX, TensorFlow, and PyTorch backends.
5
- This model is related to a `MaskedPLM` task.
6
-
7
- Model config:
8
- * **name:** esm_backbone
9
- * **trainable:** True
10
- * **vocabulary_size:** 33
11
- * **num_layers:** 30
12
- * **num_heads:** 20
13
- * **hidden_dim:** 640
14
- * **intermediate_dim:** 2560
15
- * **dropout:** 0.0
16
- * **max_wavelength:** 10000
17
- * **use_bias:** True
18
- * **activation:** gelu
19
- * **layer_norm_eps:** 1e-05
20
- * **use_pre_layer_norm:** False
21
- * **position_embedding_type:** rotary
22
- * **max_sequence_length:** 1026
23
- * **pad_token_id:** 1
24
-
25
- This model card has been generated automatically and should be completed by the model author. See [Model Cards documentation](https://huggingface.co/docs/hub/model-cards) for more information.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  library_name: keras-hub
3
  ---
4
+ ### Model Overview
5
+ Evolutionary Scale Modeling (ESM) version 2 architecture.
6
+
7
+ **Reference**
8
+
9
+ - [Evolutionary-scale prediction of atomic level protein structure with a language model](https://www.biorxiv.org/content/10.1101/2022.07.20.500902v2)
10
+
11
+ ESM-2 is a state-of-the-art protein model trained on a masked language modelling objective. It is suitable for fine-tuning on a wide range of tasks that take protein sequences as input.
12
+
13
+ ## Links
14
+
15
+ * [ESM-2 Quickstart Notebook] - coming soon
16
+ * [ESM-2 API Documentation] - coming soon
17
+ * [ESM-2 Beginner Guide] - coming soon
18
+ * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
19
+
20
+
21
+ ## Installation
22
+
23
+ Keras and KerasHub can be installed with:
24
+
25
+ ```
26
+ pip install -U -q keras-hub
27
+ pip install -U -q keras
28
+ ```
29
+
30
+ Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.
31
+
32
+ ## Presets
33
+
34
+ The following model checkpoints are provided by the Keras team. Weights have been ported from: https://huggingface.co. Full code examples for each are available below.
35
+
36
+ | Preset name | Parameters | Description |
37
+ |------------------------------------|------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
38
+ | esm2_t6_8M | 8M | 6 transformer layers version of the ESM-2 protein language model, trained on the UniRef50 clustered protein sequence dataset. |
39
+ | esm2_t12_35M | 35M | 12 transformer layers version of the ESM-2 protein language model, trained on the UniRef50 clustered protein sequence dataset. |
40
+ | esm2_t30_150M | 150M | 30 transformer layers version of the ESM-2 protein language model, trained on the UniRef50 clustered protein sequence dataset. |
41
+ | esm2_t33_650M | 650M | 33 transformer layers version of the ESM-2 protein language model, trained on the UniRef50 clustered protein sequence dataset. |
42
+