Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -26,6 +26,7 @@ pre-trained on 36 million non-coding RNA sequences.
|
|
| 26 |
| Vocabulary size | 22 |
|
| 27 |
| Positional encoding | RoPE (base=10000, non-interleaved) |
|
| 28 |
| Architecture | Pre-LN Transformer with SwiGLU FFN |
|
|
|
|
| 29 |
|
| 30 |
**Vocabulary** (index order): `<cls>` (0), `<pad>` (1), `<eos>` (2), `<unk>` (3),
|
| 31 |
`<mask>` (4), A (5), C (6), G (7), T (8), I (9), R (10), Y (11), K (12), M (13),
|
|
|
|
| 26 |
| Vocabulary size | 22 |
|
| 27 |
| Positional encoding | RoPE (base=10000, non-interleaved) |
|
| 28 |
| Architecture | Pre-LN Transformer with SwiGLU FFN |
|
| 29 |
+
| Max sequence length | ~8192 (practical; RoPE has no hard limit) |
|
| 30 |
|
| 31 |
**Vocabulary** (index order): `<cls>` (0), `<pad>` (1), `<eos>` (2), `<unk>` (3),
|
| 32 |
`<mask>` (4), A (5), C (6), G (7), T (8), I (9), R (10), Y (11), K (12), M (13),
|