Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -26,7 +26,7 @@ length <= 2048), and switches to an RNA-native vocabulary (U instead of T).
|
|
| 26 |
| Intermediate size | 3072 |
|
| 27 |
| Vocabulary size | 11 |
|
| 28 |
| Positional encoding | Absolute learned |
|
| 29 |
-
| Architecture |
|
| 30 |
| Max sequence length | 2048 |
|
| 31 |
|
| 32 |
**Vocabulary:** `[PAD]=0, [UNK]=1, [CLS]=2, [EOS]=3, [SEP]=4, [MASK]=5, A=6, U=7, C=8, G=9, N=10`
|
|
|
|
| 26 |
| Intermediate size | 3072 |
|
| 27 |
| Vocabulary size | 11 |
|
| 28 |
| Positional encoding | Absolute learned |
|
| 29 |
+
| Architecture | Post-LN BERT / BertForMaskedLM |
|
| 30 |
| Max sequence length | 2048 |
|
| 31 |
|
| 32 |
**Vocabulary:** `[PAD]=0, [UNK]=1, [CLS]=2, [EOS]=3, [SEP]=4, [MASK]=5, A=6, U=7, C=8, G=9, N=10`
|