ONNX parity
onnx/model.onnx and onnx/model_int8.onnx were exported from the PyTorch weights in this repository (opset 17,
dynamic batch and length) and checked with onnxruntime 1.30.0 against PyTorch 2.8.0 on the 160 development documents
of the SporeLabs Scrub benchmark (716 marked items), running the full scrubber (rules + model).
| graph | size | largest logit difference | findings identical to PyTorch | secrets recall | direct-identifier recall | precision |
|---|---|---|---|---|---|---|
| PyTorch | 0.986 | 0.954 | 0.878 | |||
| fp32 ONNX | 67 MB | 0.00004 | 796 of 796 | 0.986 | 0.954 | 0.878 |
| int8 ONNX | 19 MB | 3.1 | 782 of 796 | 0.986 | 0.948 | 0.883 |
fp32 gives the same findings as PyTorch and is the default. int8 is a quarter of the size and catches slightly fewer
direct identifiers; use it where size matters more (Scrub(precision="int8")).