audio-cpp commited on
Commit
ab9531f
·
verified ·
1 Parent(s): ba87e41

Add OWSM-CTC v4 models and documentation

Browse files
.gitattributes CHANGED
@@ -130,3 +130,5 @@ OWSM-GGUF/owsm-v4-medium-1b-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
130
  OWSM-GGUF/owsm-v4-small-370m-f32.gguf filter=lfs diff=lfs merge=lfs -text
131
  OWSM-GGUF/owsm-v4-small-370m-q4_k.gguf filter=lfs diff=lfs merge=lfs -text
132
  OWSM-GGUF/owsm-v4-small-370m-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
 
 
 
130
  OWSM-GGUF/owsm-v4-small-370m-f32.gguf filter=lfs diff=lfs merge=lfs -text
131
  OWSM-GGUF/owsm-v4-small-370m-q4_k.gguf filter=lfs diff=lfs merge=lfs -text
132
  OWSM-GGUF/owsm-v4-small-370m-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
133
+ OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf filter=lfs diff=lfs merge=lfs -text
134
+ OWSM-CTC-GGUF/owsm-ctc-v4-1b-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
OWSM-CTC-GGUF/README.md ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ tags:
4
+ - audio.cpp
5
+ - automatic-speech-recognition
6
+ - speech-translation
7
+ - gguf
8
+ ---
9
+
10
+ # OWSM-CTC GGUF
11
+
12
+ For [audio.cpp](https://github.com/0xShug0/audio.cpp).
13
+
14
+ Upstream: [espnet/owsm_ctc_v4_1B](https://huggingface.co/espnet/owsm_ctc_v4_1B), revision `5f526a73d919afd1bf51ab425909c8ab18801670`.
15
+
16
+ `owsm-ctc-v4-1b-f32.gguf` and `owsm-ctc-v4-1b-q8_0.gguf` use a
17
+ self-conditioned E-Branchformer encoder with a CTC head instead of the
18
+ autoregressive decoder. Both transcribe and translate with the separate `owsm_ctc`
19
+ family. CTC uses best-path decoding, without beam search or predicted timestamps.
20
+ For long recordings it uses 30-second windows, discards two seconds of context
21
+ on each side, then collapses repeated frame tokens across window boundaries.
22
+
23
+ ### CTC Parity
24
+
25
+ F32 matched official Python exactly on the five-request short sequence on CPU,
26
+ CUDA, and Vulkan: two English recordings, English-to-German translation,
27
+ unknown-language conditioning, and a repeated first request. The 327.6-second
28
+ recording's full 910-word transcript also matched exactly on CPU and CUDA.
29
+ The short request following the long recording remained unchanged.
30
+ These are transcript comparisons, not a guarantee of identical internal tensors
31
+ or a multilingual accuracy benchmark.
32
+
33
+ Q8_0 is not exact-parity. On the long recording it had one normalized-word
34
+ edit on CPU and CUDA (0.11%) relative to Python F32. These measure
35
+ disagreement with Python, not error against a human transcript. The short German
36
+ translation also changed, and Q8_0 outputs differed slightly between backends.
37
+ Use F32 when reference fidelity matters; quantization does not improve the
38
+ upstream model's translation quality.
39
+
40
+ ### Warmed Performance
41
+
42
+ Hardware: AMD Ryzen 9 7900X, NVIDIA GeForce RTX 5090, eight CPU threads.
43
+ Software: audio.cpp debug build, ESPnet 202610.post2, PyTorch 2.11.0+cu128.
44
+ C++ CPU measurements use a full-model CPU-only build with CUDA and Vulkan
45
+ disabled, native CPU instructions, and OpenMP. A paired F32 check reduced peak
46
+ RSS by about 282 MiB versus the GPU-enabled binary. GPU measurements still use
47
+ the separate GPU-enabled build.
48
+
49
+ One 3.505-second warmup precedes
50
+ one 327.6-second request, with English best-path decoding, 30-second windows,
51
+ two-second left/right context, and one window per batch. No TF32 or precision
52
+ overrides are used. Python uses `best_path` for the padded warmup and
53
+ `decode_long` for the long request. Timing excludes loading, warmup, and file
54
+ reading; C++ uses `session.wall_ms`, and Python uses an external timer with CUDA
55
+ synchronization. Memory is sampled across loading and inference using process
56
+ RSS and NVML process GPU memory: every 20 ms for CPU-only C++, and every 100 ms
57
+ for Python and the GPU runs. The numbers below are performance
58
+ measurements, separate from the focused output checks above.
59
+
60
+ | Backend / implementation | Warmed inference | RTF | Peak RAM or VRAM |
61
+ | --- | ---: | ---: | ---: |
62
+ | CPU Python F32 | 29.586 s | 0.09031 | 8,729 MiB RAM |
63
+ | CPU C++ F32 | 33.527 s | 0.10234 | 5,630 MiB RAM |
64
+ | CPU C++ Q8_0 | 30.262 s | 0.09238 | 3,050 MiB RAM |
65
+ | CUDA Python F32 | 1.082 s | 0.00330 | 10,728 MiB VRAM |
66
+ | CUDA C++ F32 | 0.627 s | 0.00191 | 6,080 MiB VRAM |
67
+ | CUDA C++ Q8_0 | 0.650 s | 0.00198 | 3,508 MiB VRAM |
68
+
69
+ **C++ CTC is still slower than Python F32 on CPU in this test:** about 13% for
70
+ F32 and 2% for Q8_0.
71
+
72
+ ```bash
73
+ audiocpp_cli --task asr --family owsm_ctc \
74
+ --model /path/to/OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf \
75
+ --backend cuda --threads 8 --audio input.wav \
76
+ --request-option language=eng --text-out transcript.txt --log
77
+ ```
78
+
79
+ `language=auto` uses unknown-language
80
+ conditioning in each window. `target_language=deu`, for example, selects German
81
+ translation. Add `--mode streaming` to emit committed text after each window
82
+ from a complete audio file. This produces the same transcript as offline mode;
83
+ it is not live microphone streaming. In server mode, use a streaming model
84
+ entry and `stream=true` on `/v1/audio/transcriptions`.
OWSM-CTC-GGUF/owsm-ctc-v4-1b-f32.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:03b272596f3552a3c511c655e00205ad56c46696204e9f21c3d958999cb5e49e
3
+ size 4046590976
OWSM-CTC-GGUF/owsm-ctc-v4-1b-q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:836d27af7220b3815ebca378a193d3af6ef1cd6787724fa2ecdc0b3226b7c38b
3
+ size 1296685184