Barist.2 / reports /benchmark_cpu.md
KucLab's picture
Add report benchmark_cpu.md
f8175a7 verified
|
Raw History Blame Contribute Delete
1.02 kB

Barist.1 — Benchmark

Generated by scripts/benchmark.py.

Hardware

  • Device: cpu (cpu)
  • torch: 2.13.0+rocm7.1
  • Python: 3.12.12
  • OS: Linux-7.1.7-100.fc43.x86_64-x86_64-with-glibc2.43

Model

Property Value
Parameters 903,552
Checkpoint size 3.45 MB
Vocabulary 2048
Context 4096 tokens
Input limit 4000 characters
Layers / d_model / heads / d_ff 3 / 128 / 4 / 384

Inference

Metric Value
Single text latency 0.55 ms
Single text throughput 1820.1 texts/s
Batch of 8 latency 1.52 ms
Batched throughput 5266.0 texts/s
Peak VRAM (inference) n/a (CPU)

Accuracy

Split Rows Accuracy F1 Precision Recall
val 6585 0.8547 0.8178 0.8561 0.7828
test 6496 0.8581 0.8221 0.8603 0.7872
probe (hand-written) 97 0.9897 0.9882 0.9767 1.0000

The probe set is hand-written and absent from all training data.