ONNX
audio-watermarking
wavmark
wavmark-onnx / README.md
gotham13's picture
Wavmark watermarker as ONNX, with the export script and parity results
4eb7998 verified
|
Raw History Blame Contribute Delete
2.7 kB
metadata
license: mit
library_name: onnx
base_model: M4869/WavMark
tags:
  - audio-watermarking
  - wavmark
  - onnx
inference: false

WavMark, ONNX

An ONNX export of WavMark (Guangyu Chen et al.), the checkpoint step59000_snr39.99_pesq4.35_BERP_none0.30_mean1.81_std1.81.model.pkl from M4869/WavMark at revision 0ad3c7b74f641bddb61f6b85cdf2de0d93a5bfef, as loaded by the wavmark 0.0.3 package. The weights are unchanged. This is not an official release of the WavMark authors.

WavMark hides 32 bits in each second of 16 kHz audio. The wavmark package uses 16 of them as a fixed pattern (to find the mark) and 16 as the payload, marks one second in every 1.1 s, and searches for the pattern every 50 ms when decoding; that logic stays outside the graphs.

Files

File Input Output
wavmark_embed.onnx audio: float32 [B, 16000] (one second, 16 kHz); bits: float32 [B, 32] (0 or 1) audio_marked: float32 [B, 16000]
wavmark_detect.onnx audio: float32 [B, 16000] bits: float32 [B, 32], in -1..1; a bit is 1 from 0.5

Both graphs are opset 18 and contain the STFT (1000 points, hop 400, Hann window), the invertible network and the inverse STFT. torch.istft does not export to ONNX, so the graphs use an equivalent built from matrix products, reshapes and pads (OnnxISTFT in the script; at most 1.5e-6 from torch.istft).

Parity with the PyTorch original

export_watermarkers.py wavmark OUT clip.wav ... exports the graphs and compares them with the package's Model.encode and Model.decode, torch 2.15 on CPU, on two speech clips at 16 kHz, one second in every 1.1 s, with random 32-bit messages:

11.8 s clip (10 seconds marked) 4.1 s clip (3 seconds marked)
embedded audio, max abs difference (PyTorch vs ONNX) 5.8e-5 6.3e-5
decoder output on marked audio, max abs difference 1.4e-3 2.8e-3
bits right, marked by PyTorch: decoded by PyTorch / ONNX 100 % / 100 % 100 % / 100 %
bits right, marked by ONNX: decoded by PyTorch / ONNX 100 % / 100 % 100 % / 100 %
bits right on unmarked audio (chance) PyTorch / ONNX 49.7 % / 49.7 % 41.7 % / 41.7 %

parity.json has the exact numbers.

Licence and attribution

MIT, as declared on the original model card (M4869/WavMark) and in the wavmark package (see LICENSE). The export script is MIT as well.

@article{chen2023wavmark,
  title={WavMark: Watermarking for Audio Generation},
  author={Chen, Guangyu and Wu, Yu and Liu, Shujie and Liu, Tao and Du, Xiaoyong and Wei, Furu},
  journal={arXiv preprint arXiv:2308.12770},
  year={2023}
}