Update README.md
Browse files
README.md
CHANGED
|
@@ -44,6 +44,7 @@ python bin/PhaStyle.py \
|
|
| 44 |
|
| 45 |
```
|
| 46 |
|
|
|
|
| 47 |
### Datasets Used:
|
| 48 |
|
| 49 |
- **BACPHLIP (without E. coli)**: 1,868 training sequences and 246 validation sequences.
|
|
@@ -53,18 +54,55 @@ python bin/PhaStyle.py \
|
|
| 53 |
Each dataset was processed using **512bp segment lengths** to simulate fragmented metagenomic assemblies.
|
| 54 |
|
| 55 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
|
| 57 |
-
|
| 58 |
|
| 59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
-
|
| 62 |
-
- **Balanced Accuracy**: 0.94 (on 1022bp fragments from the *Escherichia* dataset)
|
| 63 |
-
- **MCC (Matthews Correlation Coefficient)**: 0.91
|
| 64 |
-
- **Sensitivity**: 0.97
|
| 65 |
-
- **Specificity**: 0.91
|
| 66 |
|
|
|
|
| 67 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
|
| 69 |
## Limitations
|
| 70 |
|
|
|
|
| 44 |
|
| 45 |
```
|
| 46 |
|
| 47 |
+
|
| 48 |
### Datasets Used:
|
| 49 |
|
| 50 |
- **BACPHLIP (without E. coli)**: 1,868 training sequences and 246 validation sequences.
|
|
|
|
| 54 |
Each dataset was processed using **512bp segment lengths** to simulate fragmented metagenomic assemblies.
|
| 55 |
|
| 56 |
---
|
| 57 |
+
## Performance Results
|
| 58 |
+
|
| 59 |
+
The performance of ProkBERT PhaStyle was evaluated on various datasets, including *Escherichia* and EXTREMOPHILE phages, using segment lengths of 512bp and 1022bp. The results are summarized below:
|
| 60 |
+
|
| 61 |
+
### Performance on *Escherichia* Dataset (512bp and 1022bp segments)
|
| 62 |
+
|
| 63 |
+
| Method | Balanced Accuracy | MCC | Sensitivity | Specificity |
|
| 64 |
+
|--------------------------|-------------------|-------|-------------|-------------|
|
| 65 |
+
| **ProkBERT-mini (512bp)** | 0.91 | 0.83 | 0.94 | 0.89 |
|
| 66 |
+
| ProkBERT-mini-long (512bp)| 0.90 | 0.82 | 0.96 | 0.85 |
|
| 67 |
+
| ProkBERT-mini-c (512bp) | 0.89 | 0.80 | 0.95 | 0.84 |
|
| 68 |
+
| DNABERT-2-117M (512bp) | 0.84 | 0.72 | 0.95 | 0.74 |
|
| 69 |
+
| Nuc. Trans.-50m (512bp) | 0.85 | 0.72 | 0.92 | 0.78 |
|
| 70 |
+
| **ProkBERT-mini (1022bp)**| **0.94** | **0.88** | **0.97** | **0.91** |
|
| 71 |
+
| ProkBERT-mini-long (1022bp)| 0.94 | 0.89 | 0.97 | 0.91 |
|
| 72 |
|
| 73 |
+
### Performance on EXTREMOPHILE Dataset (512bp and 1022bp segments)
|
| 74 |
|
| 75 |
+
| Method | Balanced Accuracy | MCC | Sensitivity | Specificity |
|
| 76 |
+
|--------------------------|-------------------|-------|-------------|-------------|
|
| 77 |
+
| **ProkBERT-mini (512bp)** | 0.93 | 0.83 | 0.99 | 0.87 |
|
| 78 |
+
| ProkBERT-mini-long (512bp)| 0.93 | 0.82 | **1.00** | 0.86 |
|
| 79 |
+
| ProkBERT-mini-c (512bp) | 0.92 | 0.80 | 0.99 | 0.84 |
|
| 80 |
+
| DNABERT-2-117M (512bp) | 0.89 | 0.74 | 0.99 | 0.79 |
|
| 81 |
+
| **ProkBERT-mini (1022bp)**| **0.96** | **0.91** | **1.00** | **0.93** |
|
| 82 |
+
| ProkBERT-mini-long (1022bp)| 0.96 | 0.90 | 1.00 | 0.92 |
|
| 83 |
|
| 84 |
+
These tables highlight the high accuracy, MCC, and generalization capability of ProkBERT models, particularly on challenging datasets like *Escherichia* and extremophile phages. The ProkBERT-mini and ProkBERT-mini-long models consistently performed well on both datasets.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
|
| 86 |
+
For more detailed results, including additional metrics, please refer to the original research paper.
|
| 87 |
---
|
| 88 |
+
## Inference Speed and Running Times
|
| 89 |
+
|
| 90 |
+
The computational performance of ProkBERT PhaStyle was evaluated using 1,000 randomly selected sequences from the BACPHLIP dataset. The evaluation was performed on a consistent hardware setup with NVIDIA Tesla A100 GPUs. The execution times and inference speeds of various models are summarized below:
|
| 91 |
+
|
| 92 |
+
### Execution Times (in seconds)
|
| 93 |
+
|
| 94 |
+
| Model | Execution Time (seconds) | Inference Speed (MB/sec) |
|
| 95 |
+
|--------------------------|--------------------------|--------------------------|
|
| 96 |
+
| **ProkBERT-mini-long** | **132** | **0.52** |
|
| 97 |
+
| ProkBERT-mini | 141 | 0.49 |
|
| 98 |
+
| ProkBERT-mini-c | 146 | 0.47 |
|
| 99 |
+
| DNABERT-2-117M | 248 | 0.25 |
|
| 100 |
+
| Nucleotide Transformer-50m| 342 | 0.18 |
|
| 101 |
+
| Nucleotide Transformer-500m| 502 | 0.12 |
|
| 102 |
+
| DeePhage | 159 | 0.43 |
|
| 103 |
+
| PhaTYP | 2,718 | 0.03 |
|
| 104 |
+
| BACPHLIP | 7,125 | 0.01 |
|
| 105 |
+
|
| 106 |
|
| 107 |
## Limitations
|
| 108 |
|