mudler commited on
Commit
03e921a
·
verified ·
1 Parent(s): 3b0427d

Add benchmark graphs to card

Browse files
Files changed (1) hide show
  1. README.md +12 -0
README.md CHANGED
@@ -38,6 +38,18 @@ Which to pick: **q5_k** or **q5_0** for the best size and accuracy (byte-identic
38
 
39
  Only the large `ggml_mul_mat`-fed weights (Qwen3 and Whisper attention/FFN projections, the adaptor linears, and the token embedding, 343 tensors) are quantized; norms, biases, the conv stem, positional embeddings, and the mel filterbank stay F32.
40
 
 
 
 
 
 
 
 
 
 
 
 
 
41
  ## Usage
42
 
43
  ```sh
 
38
 
39
  Only the large `ggml_mul_mat`-fed weights (Qwen3 and Whisper attention/FFN projections, the adaptor linears, and the token embedding, 343 tensors) are quantized; norms, biases, the conv stem, positional embeddings, and the mel filterbank stay F32.
40
 
41
+ ## Benchmarks
42
+
43
+ Same audio, same F32 weights, same threads, byte-identical transcript. moss-transcribe.cpp (ggml, CPU) stays under real time where PyTorch does not, and the gap holds as clips get longer:
44
+
45
+ ![RTF vs audio length: moss-transcribe.cpp vs PyTorch on CPU](benchmarks/rtf_vs_length.png)
46
+
47
+ Quantization makes the model both smaller and faster (the decode is memory-bandwidth bound), with the transcript byte-identical through q5:
48
+
49
+ ![Quantization ladder: size and speed by dtype](benchmarks/quant_ladder.png)
50
+
51
+ Full methodology and the reproducible harness are in the [benchmarks](https://github.com/mudler/moss-transcribe.cpp/blob/master/benchmarks/BENCHMARK.md).
52
+
53
  ## Usage
54
 
55
  ```sh