Parity
Date 2026-10-06 14:54. CPU, fp32. The standalone cue.py was run on 500 dev rows (seed 7) against the training checkpoint; scores must match exactly.
| head | max abs diff | gate (= 0) |
|---|---|---|
| commitment | 0.00e+00 | pass |
| open_question | 0.00e+00 | pass |
| delegate | 0.00e+00 | pass |
ONNX: cue-text
Date 2026-10-06 14:54. onnx_export.py output (opset 17, dynamic batch and length) run with onnxruntime 1.30.0 on CPU, against the torch model, on AMI dev rows. Per head: up to 250 positives (AutoJev label >= 0.5) plus negatives, 500 rows, seed 7. AUROC is against the AutoJev labels; the gate is that int8 stays within 0.005 AUROC of torch.
| head | n | pos | AUROC torch | AUROC fp32 onnx | AUROC int8 | int8 - torch | max abs diff fp32 | max abs diff int8 | gate |
|---|---|---|---|---|---|---|---|---|---|
| commitment | 500 | 152 | 0.9901 | 0.9901 | 0.9894 | -0.0008 | 1.0e-06 | 0.101 | pass |
| open_question | 500 | 250 | 0.9835 | 0.9835 | 0.9828 | -0.0007 | 1.1e-06 | 0.200 | pass |
| delegate | 500 | 29 | 0.9823 | 0.9823 | 0.9823 | +0.0000 | 8.0e-07 | 0.183 | pass |
int8 moves single scores by up to about 0.15 (last-but-one column) and the ranking by under 0.005 AUROC. Thresholds were chosen on the torch model and apply to the int8 graph as well.
Sizes: fp32 128.4 MB, int8 39.6 MB.