Add OWSM-CTC v4 models and documentation
Browse files- .gitattributes +2 -0
- OWSM-CTC-GGUF/README.md +84 -0
- OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf +3 -0
- OWSM-CTC-GGUF/owsm-ctc-v4-1b-q8_0.gguf +3 -0
.gitattributes
CHANGED
|
@@ -130,3 +130,5 @@ OWSM-GGUF/owsm-v4-medium-1b-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
|
| 130 |
OWSM-GGUF/owsm-v4-small-370m-f32.gguf filter=lfs diff=lfs merge=lfs -text
|
| 131 |
OWSM-GGUF/owsm-v4-small-370m-q4_k.gguf filter=lfs diff=lfs merge=lfs -text
|
| 132 |
OWSM-GGUF/owsm-v4-small-370m-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 130 |
OWSM-GGUF/owsm-v4-small-370m-f32.gguf filter=lfs diff=lfs merge=lfs -text
|
| 131 |
OWSM-GGUF/owsm-v4-small-370m-q4_k.gguf filter=lfs diff=lfs merge=lfs -text
|
| 132 |
OWSM-GGUF/owsm-v4-small-370m-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 133 |
+
OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf filter=lfs diff=lfs merge=lfs -text
|
| 134 |
+
OWSM-CTC-GGUF/owsm-ctc-v4-1b-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
OWSM-CTC-GGUF/README.md
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-4.0
|
| 3 |
+
tags:
|
| 4 |
+
- audio.cpp
|
| 5 |
+
- automatic-speech-recognition
|
| 6 |
+
- speech-translation
|
| 7 |
+
- gguf
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
# OWSM-CTC GGUF
|
| 11 |
+
|
| 12 |
+
For [audio.cpp](https://github.com/0xShug0/audio.cpp).
|
| 13 |
+
|
| 14 |
+
Upstream: [espnet/owsm_ctc_v4_1B](https://huggingface.co/espnet/owsm_ctc_v4_1B), revision `5f526a73d919afd1bf51ab425909c8ab18801670`.
|
| 15 |
+
|
| 16 |
+
`owsm-ctc-v4-1b-f32.gguf` and `owsm-ctc-v4-1b-q8_0.gguf` use a
|
| 17 |
+
self-conditioned E-Branchformer encoder with a CTC head instead of the
|
| 18 |
+
autoregressive decoder. Both transcribe and translate with the separate `owsm_ctc`
|
| 19 |
+
family. CTC uses best-path decoding, without beam search or predicted timestamps.
|
| 20 |
+
For long recordings it uses 30-second windows, discards two seconds of context
|
| 21 |
+
on each side, then collapses repeated frame tokens across window boundaries.
|
| 22 |
+
|
| 23 |
+
### CTC Parity
|
| 24 |
+
|
| 25 |
+
F32 matched official Python exactly on the five-request short sequence on CPU,
|
| 26 |
+
CUDA, and Vulkan: two English recordings, English-to-German translation,
|
| 27 |
+
unknown-language conditioning, and a repeated first request. The 327.6-second
|
| 28 |
+
recording's full 910-word transcript also matched exactly on CPU and CUDA.
|
| 29 |
+
The short request following the long recording remained unchanged.
|
| 30 |
+
These are transcript comparisons, not a guarantee of identical internal tensors
|
| 31 |
+
or a multilingual accuracy benchmark.
|
| 32 |
+
|
| 33 |
+
Q8_0 is not exact-parity. On the long recording it had one normalized-word
|
| 34 |
+
edit on CPU and CUDA (0.11%) relative to Python F32. These measure
|
| 35 |
+
disagreement with Python, not error against a human transcript. The short German
|
| 36 |
+
translation also changed, and Q8_0 outputs differed slightly between backends.
|
| 37 |
+
Use F32 when reference fidelity matters; quantization does not improve the
|
| 38 |
+
upstream model's translation quality.
|
| 39 |
+
|
| 40 |
+
### Warmed Performance
|
| 41 |
+
|
| 42 |
+
Hardware: AMD Ryzen 9 7900X, NVIDIA GeForce RTX 5090, eight CPU threads.
|
| 43 |
+
Software: audio.cpp debug build, ESPnet 202610.post2, PyTorch 2.11.0+cu128.
|
| 44 |
+
C++ CPU measurements use a full-model CPU-only build with CUDA and Vulkan
|
| 45 |
+
disabled, native CPU instructions, and OpenMP. A paired F32 check reduced peak
|
| 46 |
+
RSS by about 282 MiB versus the GPU-enabled binary. GPU measurements still use
|
| 47 |
+
the separate GPU-enabled build.
|
| 48 |
+
|
| 49 |
+
One 3.505-second warmup precedes
|
| 50 |
+
one 327.6-second request, with English best-path decoding, 30-second windows,
|
| 51 |
+
two-second left/right context, and one window per batch. No TF32 or precision
|
| 52 |
+
overrides are used. Python uses `best_path` for the padded warmup and
|
| 53 |
+
`decode_long` for the long request. Timing excludes loading, warmup, and file
|
| 54 |
+
reading; C++ uses `session.wall_ms`, and Python uses an external timer with CUDA
|
| 55 |
+
synchronization. Memory is sampled across loading and inference using process
|
| 56 |
+
RSS and NVML process GPU memory: every 20 ms for CPU-only C++, and every 100 ms
|
| 57 |
+
for Python and the GPU runs. The numbers below are performance
|
| 58 |
+
measurements, separate from the focused output checks above.
|
| 59 |
+
|
| 60 |
+
| Backend / implementation | Warmed inference | RTF | Peak RAM or VRAM |
|
| 61 |
+
| --- | ---: | ---: | ---: |
|
| 62 |
+
| CPU Python F32 | 29.586 s | 0.09031 | 8,729 MiB RAM |
|
| 63 |
+
| CPU C++ F32 | 33.527 s | 0.10234 | 5,630 MiB RAM |
|
| 64 |
+
| CPU C++ Q8_0 | 30.262 s | 0.09238 | 3,050 MiB RAM |
|
| 65 |
+
| CUDA Python F32 | 1.082 s | 0.00330 | 10,728 MiB VRAM |
|
| 66 |
+
| CUDA C++ F32 | 0.627 s | 0.00191 | 6,080 MiB VRAM |
|
| 67 |
+
| CUDA C++ Q8_0 | 0.650 s | 0.00198 | 3,508 MiB VRAM |
|
| 68 |
+
|
| 69 |
+
**C++ CTC is still slower than Python F32 on CPU in this test:** about 13% for
|
| 70 |
+
F32 and 2% for Q8_0.
|
| 71 |
+
|
| 72 |
+
```bash
|
| 73 |
+
audiocpp_cli --task asr --family owsm_ctc \
|
| 74 |
+
--model /path/to/OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf \
|
| 75 |
+
--backend cuda --threads 8 --audio input.wav \
|
| 76 |
+
--request-option language=eng --text-out transcript.txt --log
|
| 77 |
+
```
|
| 78 |
+
|
| 79 |
+
`language=auto` uses unknown-language
|
| 80 |
+
conditioning in each window. `target_language=deu`, for example, selects German
|
| 81 |
+
translation. Add `--mode streaming` to emit committed text after each window
|
| 82 |
+
from a complete audio file. This produces the same transcript as offline mode;
|
| 83 |
+
it is not live microphone streaming. In server mode, use a streaming model
|
| 84 |
+
entry and `stream=true` on `/v1/audio/transcriptions`.
|
OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:03b272596f3552a3c511c655e00205ad56c46696204e9f21c3d958999cb5e49e
|
| 3 |
+
size 4046590976
|
OWSM-CTC-GGUF/owsm-ctc-v4-1b-q8_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:836d27af7220b3815ebca378a193d3af6ef1cd6787724fa2ecdc0b3226b7c38b
|
| 3 |
+
size 1296685184
|