| # Parity |
|
|
| Date 2026-10-06 14:54. CPU, fp32. The standalone `cue.py` was run on 500 dev rows (seed 7) against the training checkpoint; scores must match exactly. |
|
|
| | head | max abs diff | gate (= 0) | |
| |---|---|---| |
| | commitment | 0.00e+00 | pass | |
| | open_question | 0.00e+00 | pass | |
| | delegate | 0.00e+00 | pass | |
| |
| ## ONNX: cue-text |
| |
| Date 2026-10-06 14:54. `onnx_export.py` output (opset 17, dynamic batch and length) run with onnxruntime 1.30.0 on CPU, against the torch model, on AMI dev rows. Per head: up to 250 positives (AutoJev label >= 0.5) plus negatives, 500 rows, seed 7. AUROC is against the AutoJev labels; the gate is that int8 stays within 0.005 AUROC of torch. |
|
|
| | head | n | pos | AUROC torch | AUROC fp32 onnx | AUROC int8 | int8 - torch | max abs diff fp32 | max abs diff int8 | gate | |
| |---|---|---|---|---|---|---|---|---|---| |
| | commitment | 500 | 152 | 0.9901 | 0.9901 | 0.9894 | -0.0008 | 1.0e-06 | 0.101 | pass | |
| | open_question | 500 | 250 | 0.9835 | 0.9835 | 0.9828 | -0.0007 | 1.1e-06 | 0.200 | pass | |
| | delegate | 500 | 29 | 0.9823 | 0.9823 | 0.9823 | +0.0000 | 8.0e-07 | 0.183 | pass | |
| |
| int8 moves single scores by up to about 0.15 (last-but-one column) and the ranking by under 0.005 AUROC. Thresholds were chosen on the torch model and apply to the int8 graph as well. |
| |
| Sizes: fp32 128.4 MB, int8 39.6 MB. |
| |
