Update README.md
Browse files
README.md
CHANGED
|
@@ -100,6 +100,7 @@ We measured the average inference speed (tokens/s) of generating 1024 new tokens
|
|
| 100 |
|Quantization | Speed (3022 tokens) | Speed (8192 tokens)|
|
| 101 |
|--- |--- |---|
|
| 102 |
|BF16 | 33.40 | 31.91 |
|
|
|
|
| 103 |
|
| 104 |
|
| 105 |
## 🚀 How to use the model
|
|
|
|
| 100 |
|Quantization | Speed (3022 tokens) | Speed (8192 tokens)|
|
| 101 |
|--- |--- |---|
|
| 102 |
|BF16 | 33.40 | 31.91 |
|
| 103 |
+
|INT4 | - | 31.95 |
|
| 104 |
|
| 105 |
|
| 106 |
## 🚀 How to use the model
|