Add model specs, GitHub link, and update citation
Browse files
README.md
CHANGED
|
@@ -27,6 +27,19 @@ HELM-BERT is a BERT-style encoder designed specifically for peptide sequences in
|
|
| 27 |
- **Span Masking**: Contiguous token masking for improved contextual learning
|
| 28 |
- **nGiE**: n-gram Induced Encoding layer for local pattern recognition
|
| 29 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
## How to Use
|
| 31 |
|
| 32 |
```python
|
|
@@ -61,4 +74,4 @@ Pretrained on deduplicated peptide sequences from:
|
|
| 61 |
|
| 62 |
## License
|
| 63 |
|
| 64 |
-
MIT License
|
|
|
|
| 27 |
- **Span Masking**: Contiguous token masking for improved contextual learning
|
| 28 |
- **nGiE**: n-gram Induced Encoding layer for local pattern recognition
|
| 29 |
|
| 30 |
+
Please check the [official repository](https://github.com/clinfo/HELM-BERT) for more implementation details and updates.
|
| 31 |
+
|
| 32 |
+
## Model Specifications
|
| 33 |
+
|
| 34 |
+
| Parameter | Value |
|
| 35 |
+
|-----------|-------|
|
| 36 |
+
| Parameters | 54.8M |
|
| 37 |
+
| Hidden size | 768 |
|
| 38 |
+
| Layers | 6 |
|
| 39 |
+
| Attention heads | 12 |
|
| 40 |
+
| Vocab size | 78 |
|
| 41 |
+
| Max token length | 512 |
|
| 42 |
+
|
| 43 |
## How to Use
|
| 44 |
|
| 45 |
```python
|
|
|
|
| 74 |
|
| 75 |
## License
|
| 76 |
|
| 77 |
+
MIT License
|