MoE_detector / aug_study /phase4 /mix_reference.md
assa8945's picture
Augmentation study: RUNBOOK v2.1 complete (Phase 2b, final models, pre-registered selection, mix_setting_v1 Phase 4, band reliance, DINOv2 references)
f06a441 verified
|
Raw History Blame Contribute Delete
3.51 kB

Phase 4 reference rows: DINOv2 WaveRep baselines (n = 9939; report only, no selection)

system n AUC F1 TPR TNR TPR1 AUC_lo AUC_hi F1_lo F1_hi
(E2, E3p) 9939 0.842 0.735 0.844 0.627 0.301 0.834 0.849 0.727 0.744
(E2, F3_s0) 9939 0.849 0.739 0.836 0.648 0.335 0.842 0.857 0.731 0.748
E1 alone 9939 0.637 0.495 0.336 0.87 0.141 0.627 0.648 0.484 0.504
(E2, E1) 9939 0.759 0.583 0.474 0.85 0.145 0.75 0.769 0.573 0.591
D2wr alone 9939 0.737 0.638 0.816 0.452 0.157 0.728 0.747 0.629 0.648
(E2, D2wr) 9939 0.795 0.696 0.878 0.492 0.159 0.787 0.803 0.686 0.705
system vs dAUC dAUC_lo dAUC_hi dF1 dF1_lo dF1_hi
(E2, E1) (E2, E3p) -0.083 -0.092 -0.074 -0.153 -0.164 -0.142
(E2, E1) (E2, F3_s0) -0.09 -0.1 -0.081 -0.157 -0.169 -0.146
(E2, D2wr) (E2, E3p) -0.046 -0.053 -0.039 -0.04 -0.05 -0.03
(E2, D2wr) (E2, F3_s0) -0.054 -0.061 -0.047 -0.044 -0.054 -0.033
(E2, D2wr) (E2, E1) (E2, E3p) (E2, F3_s0) D2wr alone E1 alone
('family', 'HunyuanVideo-1.5') 0.895 / 0.654 0.839 / 0.734 0.963 / 0.746 0.971 / 0.76 0.82 / 0.595 0.683 / 0.652
('family', 'LTX-2') 0.802 / 0.664 0.764 / 0.66 0.832 / 0.72 0.831 / 0.724 0.767 / 0.627 0.659 / 0.612
('family', 'Relight-Video') 0.646 / 0.4 0.673 / 0.556 0.641 / 0.455 0.615 / 0.459 0.543 / 0.368 0.552 / 0.518
('family', 'SkyReels-V3') 0.631 / 0.54 0.588 / 0.533 0.7 / 0.613 0.7 / 0.616 0.619 / 0.519 0.53 / 0.483
('family', 'Wan-animate/replace') 0.802 / 0.406 0.78 / 0.563 0.826 / 0.476 0.906 / 0.502 0.799 / 0.386 0.701 / 0.574
('family', 'Wan2.2') 0.833 / 0.644 0.814 / 0.706 0.874 / 0.716 0.9 / 0.732 0.721 / 0.569 0.646 / 0.615
('platforms', 'measured only') 0.87 / 0.731 0.863 / 0.762 0.914 / 0.813 0.932 / 0.838 0.873 / 0.704 0.83 / 0.716
('platforms', 'mixed') 0.76 / 0.683 0.715 / 0.479 0.79 / 0.69 0.788 / 0.684 0.664 / 0.598 0.537 / 0.356
('platforms', 'old only') 0.828 / 0.698 0.816 / 0.686 0.896 / 0.792 0.906 / 0.803 0.813 / 0.699 0.767 / 0.606
('re-encodes', np.int64(1)) 0.849 / 0.714 0.838 / 0.725 0.904 / 0.803 0.919 / 0.821 0.841 / 0.703 0.792 / 0.663
('re-encodes', np.int64(5)) 0.792 / 0.706 0.748 / 0.519 0.819 / 0.711 0.825 / 0.723 0.698 / 0.618 0.572 / 0.389
('re-encodes', np.int64(10)) 0.741 / 0.671 0.684 / 0.443 0.792 / 0.694 0.779 / 0.667 0.651 / 0.596 0.497 / 0.311
('re-encodes', np.int64(20)) 0.696 / 0.644 0.676 / 0.429 0.736 / 0.588 0.711 / 0.582 0.59 / 0.544 0.516 / 0.344