Add benchmark graphs to card
Browse files
README.md
CHANGED
|
@@ -38,6 +38,18 @@ Which to pick: **q5_k** or **q5_0** for the best size and accuracy (byte-identic
|
|
| 38 |
|
| 39 |
Only the large `ggml_mul_mat`-fed weights (Qwen3 and Whisper attention/FFN projections, the adaptor linears, and the token embedding, 343 tensors) are quantized; norms, biases, the conv stem, positional embeddings, and the mel filterbank stay F32.
|
| 40 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
## Usage
|
| 42 |
|
| 43 |
```sh
|
|
|
|
| 38 |
|
| 39 |
Only the large `ggml_mul_mat`-fed weights (Qwen3 and Whisper attention/FFN projections, the adaptor linears, and the token embedding, 343 tensors) are quantized; norms, biases, the conv stem, positional embeddings, and the mel filterbank stay F32.
|
| 40 |
|
| 41 |
+
## Benchmarks
|
| 42 |
+
|
| 43 |
+
Same audio, same F32 weights, same threads, byte-identical transcript. moss-transcribe.cpp (ggml, CPU) stays under real time where PyTorch does not, and the gap holds as clips get longer:
|
| 44 |
+
|
| 45 |
+

|
| 46 |
+
|
| 47 |
+
Quantization makes the model both smaller and faster (the decode is memory-bandwidth bound), with the transcript byte-identical through q5:
|
| 48 |
+
|
| 49 |
+

|
| 50 |
+
|
| 51 |
+
Full methodology and the reproducible harness are in the [benchmarks](https://github.com/mudler/moss-transcribe.cpp/blob/master/benchmarks/BENCHMARK.md).
|
| 52 |
+
|
| 53 |
## Usage
|
| 54 |
|
| 55 |
```sh
|