Sync model card with upstream GitHub inference README
Browse files
README.md
CHANGED
|
@@ -20,10 +20,10 @@ acoustic echo cancellation (AEC), noise suppression, and dereverberation of
|
|
| 20 |
16 kHz speech, designed to run on commodity CPUs in real time.
|
| 21 |
|
| 22 |
- Two sizes — choose by CPU budget:
|
| 23 |
-
- **v1.3 (current)** — 4.8 M parameters (~19 MB F32), ~3.
|
| 24 |
-
frame on Zen4 (4 threads), **≈
|
| 25 |
- **v1.2** — 1.3 M parameters (~5 MB F32), ~1.6 ms per 16 ms frame on
|
| 26 |
-
Zen4 (4 threads), **≈
|
| 27 |
- Causal, streaming: 256-sample hop, 16 ms algorithmic latency
|
| 28 |
- F32 reference inference in C++ via [GGML](https://github.com/ggml-org/ggml);
|
| 29 |
PyTorch reference included for verification and research
|
|
@@ -279,6 +279,30 @@ compositor, browser). On a dedicated embedded device with no
|
|
| 279 |
compositor contending for the queue, expect the quieter end of the
|
| 280 |
range.
|
| 281 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 282 |
## Running Inference
|
| 283 |
|
| 284 |
Download a GGUF from the file list above — `localvqe-v1.3-4.8M-f32.gguf`
|
|
|
|
| 20 |
16 kHz speech, designed to run on commodity CPUs in real time.
|
| 21 |
|
| 22 |
- Two sizes — choose by CPU budget:
|
| 23 |
+
- **v1.3 (current)** — 4.8 M parameters (~19 MB F32), ~3.2 ms per 16 ms
|
| 24 |
+
frame on Zen4 (4 threads), **≈5× realtime**, ~34 MiB peak RSS.
|
| 25 |
- **v1.2** — 1.3 M parameters (~5 MB F32), ~1.6 ms per 16 ms frame on
|
| 26 |
+
Zen4 (4 threads), **≈10× realtime**, ~20 MiB peak RSS.
|
| 27 |
- Causal, streaming: 256-sample hop, 16 ms algorithmic latency
|
| 28 |
- F32 reference inference in C++ via [GGML](https://github.com/ggml-org/ggml);
|
| 29 |
PyTorch reference included for verification and research
|
|
|
|
| 279 |
compositor contending for the queue, expect the quieter end of the
|
| 280 |
range.
|
| 281 |
|
| 282 |
+
### Memory footprint (CPU)
|
| 283 |
+
|
| 284 |
+
Process RSS from `bench` on Zen4, measured via `/proc/self/status`.
|
| 285 |
+
Same numbers under every thread count from 1 to 16 — the runtime has
|
| 286 |
+
no per-thread arenas of meaningful size, so peak RSS is set by
|
| 287 |
+
weights + activations + history scratch.
|
| 288 |
+
|
| 289 |
+
| Model | Post-load delta ¹ | Peak RSS (VmHWM) ² |
|
| 290 |
+
|---------------------|------------------:|-------------------:|
|
| 291 |
+
| **v1.3** (4.8 M) | +24.4 MiB | 34.1 MiB |
|
| 292 |
+
| **v1.2** (1.3 M) | +10.0 MiB | 19.6 MiB |
|
| 293 |
+
|
| 294 |
+
¹ RSS added by loading the model + initialising the CPU backend, on
|
| 295 |
+
top of a ~7 MiB binary-and-libs baseline. This is the portable
|
| 296 |
+
"working set the model brings" number; the absolute peak will depend
|
| 297 |
+
on your host process baseline.
|
| 298 |
+
|
| 299 |
+
² Steady-state ceiling after warmup + sustained streaming. v1.3 is
|
| 300 |
+
~1.75× v1.2 in RSS terms despite carrying ~3.7× more parameters —
|
| 301 |
+
activation/history buffers don't scale with channel width. GPU
|
| 302 |
+
backends are not reflected here (VRAM doesn't appear in
|
| 303 |
+
`/proc/self/status`); for those, `bench --profile` prints the
|
| 304 |
+
backend-internal weight/activation buffer sizes.
|
| 305 |
+
|
| 306 |
## Running Inference
|
| 307 |
|
| 308 |
Download a GGUF from the file list above — `localvqe-v1.3-4.8M-f32.gguf`
|