ONNX
audio-watermarking
wavmark
wavmark-onnx / README.md
gotham13's picture
Wavmark watermarker as ONNX, with the export script and parity results
4eb7998 verified
|
Raw History Blame Contribute Delete
2.7 kB
---
license: mit
library_name: onnx
base_model: M4869/WavMark
tags:
- audio-watermarking
- wavmark
- onnx
inference: false
---
# WavMark, ONNX
An ONNX export of [WavMark](https://huggingface.co/M4869/WavMark) (Guangyu Chen et al.), the checkpoint
`step59000_snr39.99_pesq4.35_BERP_none0.30_mean1.81_std1.81.model.pkl` from `M4869/WavMark` at revision
`0ad3c7b74f641bddb61f6b85cdf2de0d93a5bfef`, as loaded by the `wavmark` 0.0.3 package. The weights are unchanged.
This is not an official release of the WavMark authors.
WavMark hides 32 bits in each second of 16 kHz audio. The `wavmark` package uses 16 of them as a fixed pattern
(to find the mark) and 16 as the payload, marks one second in every 1.1 s, and searches for the pattern every
50 ms when decoding; that logic stays outside the graphs.
## Files
| File | Input | Output |
| --- | --- | --- |
| `wavmark_embed.onnx` | `audio`: float32 `[B, 16000]` (one second, 16 kHz); `bits`: float32 `[B, 32]` (0 or 1) | `audio_marked`: float32 `[B, 16000]` |
| `wavmark_detect.onnx` | `audio`: float32 `[B, 16000]` | `bits`: float32 `[B, 32]`, in -1..1; a bit is 1 from 0.5 |
Both graphs are opset 18 and contain the STFT (1000 points, hop 400, Hann window), the invertible network and
the inverse STFT. `torch.istft` does not export to ONNX, so the graphs use an equivalent built from matrix
products, reshapes and pads (`OnnxISTFT` in the script; at most 1.5e-6 from `torch.istft`).
## Parity with the PyTorch original
`export_watermarkers.py wavmark OUT clip.wav ...` exports the graphs and compares them with the package's
`Model.encode` and `Model.decode`, torch 2.15 on CPU, on two speech clips at 16 kHz, one second in every 1.1 s, with
random 32-bit messages:
| | 11.8 s clip (10 seconds marked) | 4.1 s clip (3 seconds marked) |
| --- | --- | --- |
| embedded audio, max abs difference (PyTorch vs ONNX) | 5.8e-5 | 6.3e-5 |
| decoder output on marked audio, max abs difference | 1.4e-3 | 2.8e-3 |
| bits right, marked by PyTorch: decoded by PyTorch / ONNX | 100 % / 100 % | 100 % / 100 % |
| bits right, marked by ONNX: decoded by PyTorch / ONNX | 100 % / 100 % | 100 % / 100 % |
| bits right on unmarked audio (chance) PyTorch / ONNX | 49.7 % / 49.7 % | 41.7 % / 41.7 % |
`parity.json` has the exact numbers.
## Licence and attribution
MIT, as declared on the original model card (`M4869/WavMark`) and in the `wavmark` package (see `LICENSE`). The
export script is MIT as well.
```bibtex
@article{chen2023wavmark,
title={WavMark: Watermarking for Audio Generation},
author={Chen, Guangyu and Wu, Yu and Liu, Shujie and Liu, Tao and Du, Xiaoyong and Wei, Furu},
journal={arXiv preprint arXiv:2308.12770},
year={2023}
}
```