Add Comparison-Table to Readme
Browse files
README.md
CHANGED
|
@@ -41,21 +41,22 @@ library_name: transformers
|
|
| 41 |
---
|
| 42 |
|
| 43 |
## 📊 Performance Metrics
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
|
| 48 |
-
|
| 49 |
-
|
|
| 50 |
-
|
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
|
| 54 |
-
|
|
| 55 |
-
|
|
| 56 |
-
|
|
| 57 |
-
|
|
| 58 |
-
|
|
|
|
|
| 59 |
|
| 60 |
---
|
| 61 |
|
|
|
|
| 41 |
---
|
| 42 |
|
| 43 |
## 📊 Performance Metrics
|
| 44 |
+
This project features four models exploring the trade-offs between recurrent (LSTM) and attention-based (Transformer) architectures, as well as the effects of fine-tuning on capchas generated by the [Python Captcha Library](https://captcha.lepture.com/).
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
| Metric | **CRNN (Base)** | **CRNN (Finetuned)** | **Conv-Transformer (Base)** | **Conv-Transformer (Finetuned)** |
|
| 48 |
+
|--------|-----------------|----------------------|-----------------------------|----------------------------------|
|
| 49 |
+
| Architecture | CRNN | CRNN | Convolutional Transformer | Convolutional Transformer |
|
| 50 |
+
| Training Data | [hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data) | [hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data) <br> [Python Captcha Library](https://captcha.lepture.com/) | [hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data) | [hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data) <br> [Python Captcha Library](https://captcha.lepture.com/) |
|
| 51 |
+
| # Parameters | **3,570,943** | **3,570,943** | 12,279,551 | 12,279,551 |
|
| 52 |
+
| Model Size | **14.3 MB** | **14.3 MB** | 51.7 MB | 51.7 MB |
|
| 53 |
+
| Sequence Accuracy <br> ([hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data)) | 96.81% | 92.98% | **97.38%** | 95.36% |
|
| 54 |
+
| Character Error Rate (CER) <br> ([hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data)) | 0.70% | 1.59% | **0.57%** | 1.03% |
|
| 55 |
+
| Sequence Accuracy <br> ([Python Captcha Library](https://captcha.lepture.com/)) | 9.65% | 86.20% | 11.59% | **88.42%** |
|
| 56 |
+
| Character Error Rate (CER) <br> ([Python Captcha Library](https://captcha.lepture.com/)) | 43.98% | 2.53% | 38.63% | **2.08%** |
|
| 57 |
+
| Throughput (img/sec) | 447.26 | 447.26 | **733.00** | **733.00** |
|
| 58 |
+
| Compute Hardware | NVIDIA RTX A6000 | NVIDIA RTX A6000 | NVIDIA RTX A6000 | NVIDIA RTX A6000 |
|
| 59 |
+
| Link | [Graf-J/captcha-crnn-base](https://huggingface.co/Graf-J/captcha-crnn-base) | [Graf-J/captcha-crnn-finetuned](https://huggingface.co/Graf-J/captcha-crnn-finetuned) | [Graf-J/captcha-conv-transformer-base](https://huggingface.co/Graf-J/captcha-conv-transformer-base) | [Graf-J/captcha-conv-transformer-finetuned](https://huggingface.co/Graf-J/captcha-conv-transformer-finetuned)
|
| 60 |
|
| 61 |
---
|
| 62 |
|