File size: 2,701 Bytes
4eb7998 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | ---
license: mit
library_name: onnx
base_model: M4869/WavMark
tags:
- audio-watermarking
- wavmark
- onnx
inference: false
---
# WavMark, ONNX
An ONNX export of [WavMark](https://huggingface.co/M4869/WavMark) (Guangyu Chen et al.), the checkpoint
`step59000_snr39.99_pesq4.35_BERP_none0.30_mean1.81_std1.81.model.pkl` from `M4869/WavMark` at revision
`0ad3c7b74f641bddb61f6b85cdf2de0d93a5bfef`, as loaded by the `wavmark` 0.0.3 package. The weights are unchanged.
This is not an official release of the WavMark authors.
WavMark hides 32 bits in each second of 16 kHz audio. The `wavmark` package uses 16 of them as a fixed pattern
(to find the mark) and 16 as the payload, marks one second in every 1.1 s, and searches for the pattern every
50 ms when decoding; that logic stays outside the graphs.
## Files
| File | Input | Output |
| --- | --- | --- |
| `wavmark_embed.onnx` | `audio`: float32 `[B, 16000]` (one second, 16 kHz); `bits`: float32 `[B, 32]` (0 or 1) | `audio_marked`: float32 `[B, 16000]` |
| `wavmark_detect.onnx` | `audio`: float32 `[B, 16000]` | `bits`: float32 `[B, 32]`, in -1..1; a bit is 1 from 0.5 |
Both graphs are opset 18 and contain the STFT (1000 points, hop 400, Hann window), the invertible network and
the inverse STFT. `torch.istft` does not export to ONNX, so the graphs use an equivalent built from matrix
products, reshapes and pads (`OnnxISTFT` in the script; at most 1.5e-6 from `torch.istft`).
## Parity with the PyTorch original
`export_watermarkers.py wavmark OUT clip.wav ...` exports the graphs and compares them with the package's
`Model.encode` and `Model.decode`, torch 2.15 on CPU, on two speech clips at 16 kHz, one second in every 1.1 s, with
random 32-bit messages:
| | 11.8 s clip (10 seconds marked) | 4.1 s clip (3 seconds marked) |
| --- | --- | --- |
| embedded audio, max abs difference (PyTorch vs ONNX) | 5.8e-5 | 6.3e-5 |
| decoder output on marked audio, max abs difference | 1.4e-3 | 2.8e-3 |
| bits right, marked by PyTorch: decoded by PyTorch / ONNX | 100 % / 100 % | 100 % / 100 % |
| bits right, marked by ONNX: decoded by PyTorch / ONNX | 100 % / 100 % | 100 % / 100 % |
| bits right on unmarked audio (chance) PyTorch / ONNX | 49.7 % / 49.7 % | 41.7 % / 41.7 % |
`parity.json` has the exact numbers.
## Licence and attribution
MIT, as declared on the original model card (`M4869/WavMark`) and in the `wavmark` package (see `LICENSE`). The
export script is MIT as well.
```bibtex
@article{chen2023wavmark,
title={WavMark: Watermarking for Audio Generation},
author={Chen, Guangyu and Wu, Yu and Liu, Shujie and Liu, Tao and Du, Xiaoyong and Wei, Furu},
journal={arXiv preprint arXiv:2308.12770},
year={2023}
}
```
|