Update README.md
Browse files
README.md
CHANGED
|
@@ -15,11 +15,92 @@ license: apache-2.0
|
|
| 15 |
* **Max Input Length:** 20 tokens
|
| 16 |
* **Performance:** ~97% Accuracy on 4-operation math (Validation Set)
|
| 17 |
|
| 18 |
-
##
|
| 19 |
| Epoch | Training Loss | Val Accuracy | Status |
|
| 20 |
| :--- | :--- | :--- | :--- |
|
| 21 |
| 1 | 1.1420 | 54.89% | π΄ Learnt Format |
|
| 22 |
| 2 | 0.3931 | 78.79% | π‘ Learnt Digits |
|
| 23 |
| 5 | 0.1638 | 91.91% | π’ Learning subtleties |
|
| 24 |
| 9 | 0.1051 | 97.15% | π΅ High Precision |
|
| 25 |
-
| **10** | **0.1004** | **97.73%** | π **Near Perfect** |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
* **Max Input Length:** 20 tokens
|
| 16 |
* **Performance:** ~97% Accuracy on 4-operation math (Validation Set)
|
| 17 |
|
| 18 |
+
## Performance Chart
|
| 19 |
| Epoch | Training Loss | Val Accuracy | Status |
|
| 20 |
| :--- | :--- | :--- | :--- |
|
| 21 |
| 1 | 1.1420 | 54.89% | π΄ Learnt Format |
|
| 22 |
| 2 | 0.3931 | 78.79% | π‘ Learnt Digits |
|
| 23 |
| 5 | 0.1638 | 91.91% | π’ Learning subtleties |
|
| 24 |
| 9 | 0.1051 | 97.15% | π΅ High Precision |
|
| 25 |
+
| **10** | **0.1004** | **97.73%** | π **Near Perfect** |
|
| 26 |
+
|
| 27 |
+
## How to use
|
| 28 |
+
To use this model, download `model.pt` and `use.py` and run it on any type of device with Python3.
|
| 29 |
+
|
| 30 |
+
## Examples
|
| 31 |
+
Model loaded (Accuracy: 97.73% from epoch 10)
|
| 32 |
+
|
| 33 |
+
--- Mini Math Model interactive ---
|
| 34 |
+
Enter an arithmetic task (e.g. 15*15) or type 'exit' to quit this.
|
| 35 |
+
|
| 36 |
+
Task > 0*567
|
| 37 |
+
Model: 0 | Correct: 0 β
|
| 38 |
+
|
| 39 |
+
Task > 999+999
|
| 40 |
+
Model: 1998 | Correct: 1998 β
|
| 41 |
+
|
| 42 |
+
Task > 1/1
|
| 43 |
+
Model: 1 | Correct: 1 β
|
| 44 |
+
|
| 45 |
+
Task > 1684*8787
|
| 46 |
+
Model: 6398 | Correct: 14797308 β
|
| 47 |
+
|
| 48 |
+
Task > 124*598
|
| 49 |
+
Model: 2452 | Correct: 74152 β
|
| 50 |
+
|
| 51 |
+
Task > 12/68
|
| 52 |
+
Model: 4 | Correct: 0 β
|
| 53 |
+
|
| 54 |
+
Task > 123*123
|
| 55 |
+
Model: 499 | Correct: 15129 β
|
| 56 |
+
|
| 57 |
+
Task > 47*5
|
| 58 |
+
Model: 235 | Correct: 235 β
|
| 59 |
+
|
| 60 |
+
Task > 456+125
|
| 61 |
+
Model: 581 | Correct: 581 β
|
| 62 |
+
|
| 63 |
+
Task > 957-234
|
| 64 |
+
Model: 723 | Correct: 723 β
|
| 65 |
+
|
| 66 |
+
Task > 120-7650
|
| 67 |
+
Model: -550 | Correct: -7530 β
|
| 68 |
+
|
| 69 |
+
Task > 450-750
|
| 70 |
+
Model: -300 | Correct: -300 β
|
| 71 |
+
|
| 72 |
+
Task > 453-97
|
| 73 |
+
Model: 356 | Correct: 356 β
|
| 74 |
+
|
| 75 |
+
Task > 129-462
|
| 76 |
+
Model: -333 | Correct: -333 β
|
| 77 |
+
|
| 78 |
+
Task > 8*8
|
| 79 |
+
Model: 64 | Correct: 64 β
|
| 80 |
+
|
| 81 |
+
Task > 54*54
|
| 82 |
+
Model: 2916 | Correct: 2916 β
|
| 83 |
+
|
| 84 |
+
Task > 102*78
|
| 85 |
+
Model: 748 | Correct: 7956 β
|
| 86 |
+
|
| 87 |
+
Task > 74*9
|
| 88 |
+
Model: 666 | Correct: 666 β
|
| 89 |
+
|
| 90 |
+
Task > 103-34
|
| 91 |
+
Model: 69 | Correct: 69 β
|
| 92 |
+
|
| 93 |
+
## Overall accuracy
|
| 94 |
+
The overall accuracy after 10 epochs of training is ~97% for tasks with max. 3 digits each like `74*9` or `103-34`.
|
| 95 |
+
|
| 96 |
+
## Limitations
|
| 97 |
+
The can't do:
|
| 98 |
+
- Tasks with more than 3 digits like `3984-125`
|
| 99 |
+
- Multiplication tasks with numbers above 99 like `293*21`
|
| 100 |
+
- Complex tasks
|
| 101 |
+
|
| 102 |
+
## Training
|
| 103 |
+
We trained for 10 epochs (~20 minutes of training on Kaggle 2x T4) with 2 million randomly generated training samples.
|
| 104 |
+
|
| 105 |
+
## Final thoughts
|
| 106 |
+
We may be releasing an improved version of this that can solve really complex tasks and much more...stay tuned!
|