Graf-J commited on
Commit
8f3eed2
·
verified ·
1 Parent(s): 7cc8a91

Add Comparison-Table to Readme

Browse files
Files changed (1) hide show
  1. README.md +16 -15
README.md CHANGED
@@ -41,21 +41,22 @@ library_name: transformers
41
  ---
42
 
43
  ## 📊 Performance Metrics
44
-
45
- ### **Test Set Results**
46
-
47
- | Dataset | Sequence Accuracy | Character Error Rate (CER) |
48
- | --- | --- | --- |
49
- | **[hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data)** | `92.98%` | `1.59%` |
50
- | **[Python Captcha Library](https://captcha.lepture.com/)** | `86.20%` | `2.53%` |
51
-
52
- ### **Hardware & Efficiency**
53
- | Metric | Value |
54
- | --- | --- |
55
- | **Model Parameters** | `3,570,943` |
56
- | **Model Size (Disk)** | `14.3 MB` |
57
- | **Throughput (Images/sec)** | `447.26 467.29` |
58
- | **Compute Hardware** | **NVIDIA RTX A6000** |
 
59
 
60
  ---
61
 
 
41
  ---
42
 
43
  ## 📊 Performance Metrics
44
+ This project features four models exploring the trade-offs between recurrent (LSTM) and attention-based (Transformer) architectures, as well as the effects of fine-tuning on capchas generated by the [Python Captcha Library](https://captcha.lepture.com/).
45
+
46
+
47
+ | Metric | **CRNN (Base)** | **CRNN (Finetuned)** | **Conv-Transformer (Base)** | **Conv-Transformer (Finetuned)** |
48
+ |--------|-----------------|----------------------|-----------------------------|----------------------------------|
49
+ | Architecture | CRNN | CRNN | Convolutional Transformer | Convolutional Transformer |
50
+ | Training Data | [hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data) | [hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data) <br> [Python Captcha Library](https://captcha.lepture.com/) | [hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data) | [hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data) <br> [Python Captcha Library](https://captcha.lepture.com/) |
51
+ | # Parameters | **3,570,943** | **3,570,943** | 12,279,551 | 12,279,551 |
52
+ | Model Size | **14.3 MB** | **14.3 MB** | 51.7 MB | 51.7 MB |
53
+ | Sequence Accuracy <br> ([hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data)) | 96.81% | 92.98% | **97.38%** | 95.36% |
54
+ | Character Error Rate (CER) <br> ([hammer888/captcha-data](https://huggingface.co/datasets/hammer888/captcha-data)) | 0.70% | 1.59% | **0.57%** | 1.03% |
55
+ | Sequence Accuracy <br> ([Python Captcha Library](https://captcha.lepture.com/)) | 9.65% | 86.20% | 11.59% | **88.42%** |
56
+ | Character Error Rate (CER) <br> ([Python Captcha Library](https://captcha.lepture.com/)) | 43.98% | 2.53% | 38.63% | **2.08%** |
57
+ | Throughput (img/sec) | 447.26 | 447.26 | **733.00** | **733.00** |
58
+ | Compute Hardware | NVIDIA RTX A6000 | NVIDIA RTX A6000 | NVIDIA RTX A6000 | NVIDIA RTX A6000 |
59
+ | Link | [Graf-J/captcha-crnn-base](https://huggingface.co/Graf-J/captcha-crnn-base) | [Graf-J/captcha-crnn-finetuned](https://huggingface.co/Graf-J/captcha-crnn-finetuned) | [Graf-J/captcha-conv-transformer-base](https://huggingface.co/Graf-J/captcha-conv-transformer-base) | [Graf-J/captcha-conv-transformer-finetuned](https://huggingface.co/Graf-J/captcha-conv-transformer-finetuned)
60
 
61
  ---
62