KucLab commited on
Commit
52829b4
·
verified ·
1 Parent(s): e29c9cf

Add report benchmark.md

Browse files
Files changed (1) hide show
  1. reports/benchmark.md +6 -6
reports/benchmark.md CHANGED
@@ -24,18 +24,18 @@ Generated by `scripts/benchmark.py`.
24
 
25
  | Metric | Value |
26
  | --- | --- |
27
- | Single text latency | 1.29 ms |
28
- | Single text throughput | 775.0 texts/s |
29
- | Batch of 8 latency | 1.43 ms |
30
- | Batched throughput | 5609.9 texts/s |
31
  | Peak VRAM (inference) | 84.7 MB |
32
 
33
  ## Accuracy
34
 
35
  | Split | Rows | Accuracy | F1 | Precision | Recall |
36
  | --- | --- | --- | --- | --- | --- |
37
- | val | 6585 | 0.8559 | 0.8182 | 0.8626 | 0.7781 |
38
- | test | 6496 | 0.8556 | 0.8180 | 0.8615 | 0.7787 |
39
  | probe (hand-written) | 97 | 0.9897 | 0.9882 | 0.9767 | 1.0000 |
40
 
41
  The probe set is hand-written and absent from all training data.
 
24
 
25
  | Metric | Value |
26
  | --- | --- |
27
+ | Single text latency | 1.06 ms |
28
+ | Single text throughput | 946.8 texts/s |
29
+ | Batch of 8 latency | 1.2 ms |
30
+ | Batched throughput | 6646.8 texts/s |
31
  | Peak VRAM (inference) | 84.7 MB |
32
 
33
  ## Accuracy
34
 
35
  | Split | Rows | Accuracy | F1 | Precision | Recall |
36
  | --- | --- | --- | --- | --- | --- |
37
+ | val | 6585 | 0.8553 | 0.8187 | 0.8563 | 0.7843 |
38
+ | test | 6496 | 0.8590 | 0.8238 | 0.8595 | 0.7909 |
39
  | probe (hand-written) | 97 | 0.9897 | 0.9882 | 0.9767 | 1.0000 |
40
 
41
  The probe set is hand-written and absent from all training data.