================================================================= RAPID PROTOTYPE v2: Differentiation-Centered Bank ================================================================= Device: cuda ================================================================= PHASE 0: EXTRACTION ================================================================= Captions: 20,000 Extracting: bert... Loading weights: 100%  199/199 [00:00<00:00, 4216.36it/s, Materializing param=pooler.dense.weight] BertModel LOAD REPORT from: google-bert/bert-base-uncased Key | Status | | -------------------------------------------+------------+--+- cls.predictions.bias | UNEXPECTED | | cls.predictions.transform.dense.bias | UNEXPECTED | | cls.predictions.transform.LayerNorm.weight | UNEXPECTED | | cls.predictions.transform.LayerNorm.bias | UNEXPECTED | | cls.seq_relationship.weight | UNEXPECTED | | cls.predictions.transform.dense.weight | UNEXPECTED | | cls.seq_relationship.bias | UNEXPECTED | | Notes: - UNEXPECTED :can be ignored when loading from different task/architecture; not ok if you expect identical arch. bert: 100%|██████████| 157/157 [00:23<00:00, 6.56it/s] Shape: torch.Size([20000, 768]) Extracting: modern... Loading weights: 100%  134/134 [00:00<00:00, 4047.07it/s, Materializing param=layers.21.mlp_norm.weight] ModernBertModel LOAD REPORT from: answerdotai/ModernBERT-base Key | Status | | ------------------+------------+--+- head.dense.weight | UNEXPECTED | | head.norm.weight | UNEXPECTED | | decoder.bias | UNEXPECTED | | Notes: - UNEXPECTED :can be ignored when loading from different task/architecture; not ok if you expect identical arch. modern: 100%|██████████| 157/157 [00:35<00:00, 4.39it/s] Shape: torch.Size([20000, 768]) ================================================================= PHASE 0b: GENERALIZED PROCRUSTES ALIGNMENT (no reference bias) ================================================================= GPA iter 1: delta=1.19668072 GPA iter 3: delta=0.00029225 GPA iter 6: delta=0.00006347 GPA iter 9: delta=0.00002718 bert : cos_after=0.8541 cos_to_mean=0.9865 modern : cos_after=0.8577 cos_to_mean=0.9867 cos(consensus, bert): 0.9867 cos(consensus, modern): 0.9868 Equidistance range: 0.0001 (should be near 0) Measuring consensus statistics... CV: 0.1771 Mean cos: 0.0018 Eff dim: 109.5 Spectral: [0.0343, 0.0322, 0.0275, 0.0240, 0.0222...] ================================================================= PHASE 1: TRAIN STUDENT ================================================================= Student: 11,269,632 params CV target: 0.1771 E1: 2s loss=2.9588 t_acc=0.362 t_cos=0.334 v_acc=0.494 v_cos=0.503 v_cv=0.223 E2: 2s loss=1.4268 t_acc=0.761 t_cos=0.543 v_acc=0.704 v_cos=0.588 v_cv=0.212 E3: 2s loss=0.9784 t_acc=0.887 t_cos=0.604 v_acc=0.822 v_cos=0.639 v_cv=0.182 E4: 2s loss=0.7289 t_acc=0.943 t_cos=0.641 v_acc=0.912 v_cos=0.676 v_cv=0.182 E5: 2s loss=0.5807 t_acc=0.968 t_cos=0.666 v_acc=0.920 v_cos=0.686 v_cv=0.182 Student saved. v_cos=0.686, v_cv=0.182 ================================================================= PHASE 2: TRAIN ALIGNMENT BANK (student frozen) ================================================================= Pre-encoding through frozen student... Student embeddings: torch.Size([18000, 768]) Expert 0 (bert): rotation + whitener + mean loaded, cos_after=0.8541 Expert 1 (modern): rotation + whitener + mean loaded, cos_after=0.8577 Anchors: 512 initialized from consensus embeddings Targets: CV=0.1771, mean_cos=0.0018 Bank: 2,921,088 params Bank targets: CV=0.1771, mean_cos=0.0018 Calibrated disagreement (n=2000): cross_cos: 0.0794 ± 0.0035 disagree_ratio: median=0.000000 mean=0.000000 std=0.000000 expert_cos: 1.0000 ± 0.0000 E 1: 1s loss=0.4789 v_loss=0.4172 Geometry: b_cv=0.2688 e_cv=0.1603 spread=0.03940 a_max=0.652 Experts: cos=0.794±0.006 agr=0.000092 ortho=0.000388 Disagree: x_cos=0.0740±0.0009 ratio=0.004326 preserve=0.013135 norms=0.1626 E 2: 1s loss=0.4002 v_loss=0.3818 Geometry: b_cv=0.2229 e_cv=0.1588 spread=0.02779 a_max=0.668 Experts: cos=0.807±0.006 agr=0.000007 ortho=0.000288 Disagree: x_cos=0.0805±0.0014 ratio=0.003575 preserve=0.000024 norms=0.1703 E 3: 1s loss=0.3743 v_loss=0.3625 Geometry: b_cv=0.2189 e_cv=0.1606 spread=0.02500 a_max=0.670 Experts: cos=0.835±0.005 agr=0.000005 ortho=0.000152 Disagree: x_cos=0.0774±0.0018 ratio=0.002279 preserve=0.000016 norms=0.1066 E 4: 1s loss=0.3591 v_loss=0.3615 Geometry: b_cv=0.2100 e_cv=0.1643 spread=0.02302 a_max=0.670 Experts: cos=0.822±0.005 agr=0.000003 ortho=0.000094 Disagree: x_cos=0.0781±0.0021 ratio=0.001569 preserve=0.000020 norms=0.1137 E 5: 1s loss=0.3537 v_loss=0.3665 Geometry: b_cv=0.2118 e_cv=0.1664 spread=0.02133 a_max=0.670 Experts: cos=0.815±0.006 agr=0.000002 ortho=0.000066 Disagree: x_cos=0.0765±0.0021 ratio=0.001389 preserve=0.000026 norms=0.1669 E 6: 1s loss=0.3506 v_loss=0.3527 Geometry: b_cv=0.2097 e_cv=0.1600 spread=0.02009 a_max=0.670 Experts: cos=0.829±0.005 agr=0.000003 ortho=0.000048 Disagree: x_cos=0.0846±0.0024 ratio=0.001772 preserve=0.000021 norms=0.1363 E 7: 1s loss=0.3459 v_loss=0.3502 Geometry: b_cv=0.2055 e_cv=0.1628 spread=0.01906 a_max=0.670 Experts: cos=0.759±0.007 agr=0.000004 ortho=0.000040 Disagree: x_cos=0.0774±0.0022 ratio=0.003070 preserve=0.000049 norms=0.1964 E 8: 1s loss=0.3442 v_loss=0.3479 Geometry: b_cv=0.2078 e_cv=0.1643 spread=0.01817 a_max=0.669 Experts: cos=0.745±0.007 agr=0.000003 ortho=0.000033 Disagree: x_cos=0.0782±0.0023 ratio=0.001258 preserve=0.000021 norms=0.1772 E 9: 1s loss=0.3419 v_loss=0.3451 Geometry: b_cv=0.2015 e_cv=0.1646 spread=0.01756 a_max=0.670 Experts: cos=0.767±0.006 agr=0.000007 ortho=0.000030 Disagree: x_cos=0.0823±0.0024 ratio=0.001625 preserve=0.000049 norms=0.2007 E10: 1s loss=0.3433 v_loss=0.3433 Geometry: b_cv=0.2074 e_cv=0.1594 spread=0.01746 a_max=0.669 Experts: cos=0.762±0.005 agr=0.000006 ortho=0.000026 Disagree: x_cos=0.0766±0.0018 ratio=0.001418 preserve=0.000073 norms=0.0529 E11: 1s loss=0.3392 v_loss=0.3501 Geometry: b_cv=0.2021 e_cv=0.1609 spread=0.01705 a_max=0.669 Experts: cos=0.721±0.007 agr=0.000004 ortho=0.000026 Disagree: x_cos=0.0698±0.0022 ratio=0.006405 preserve=0.000037 norms=0.1509 E12: 1s loss=0.3383 v_loss=0.3534 Geometry: b_cv=0.1983 e_cv=0.1639 spread=0.01693 a_max=0.668 Experts: cos=0.753±0.005 agr=0.000014 ortho=0.000026 Disagree: x_cos=0.0743±0.0021 ratio=0.000903 preserve=0.000076 norms=0.0763 E13: 1s loss=0.3374 v_loss=0.3398 Geometry: b_cv=0.1996 e_cv=0.1603 spread=0.01660 a_max=0.669 Experts: cos=0.714±0.006 agr=0.000004 ortho=0.000022 Disagree: x_cos=0.0791±0.0021 ratio=0.006335 preserve=0.000060 norms=0.1257 E14: 1s loss=0.3376 v_loss=0.3415 Geometry: b_cv=0.1992 e_cv=0.1657 spread=0.01647 a_max=0.669 Experts: cos=0.704±0.006 agr=0.000006 ortho=0.000022 Disagree: x_cos=0.0824±0.0021 ratio=0.006577 preserve=0.000061 norms=0.0873 E15: 1s loss=0.3372 v_loss=0.3409 Geometry: b_cv=0.2003 e_cv=0.1615 spread=0.01635 a_max=0.669 Experts: cos=0.745±0.005 agr=0.000003 ortho=0.000019 Disagree: x_cos=0.0760±0.0020 ratio=0.002660 preserve=0.000045 norms=0.0958 E16: 1s loss=0.3355 v_loss=0.3328 Geometry: b_cv=0.1990 e_cv=0.1601 spread=0.01600 a_max=0.669 Experts: cos=0.689±0.005 agr=0.000004 ortho=0.000018 Disagree: x_cos=0.0814±0.0024 ratio=0.002029 preserve=0.000042 norms=0.1414 E17: 1s loss=0.3350 v_loss=0.3432 Geometry: b_cv=0.1945 e_cv=0.1604 spread=0.01603 a_max=0.668 Experts: cos=0.751±0.003 agr=0.000028 ortho=0.000020 Disagree: x_cos=0.0825±0.0023 ratio=0.001129 preserve=0.000155 norms=0.0187 E18: 1s loss=0.3372 v_loss=0.3336 Geometry: b_cv=0.2044 e_cv=0.1605 spread=0.01590 a_max=0.668 Experts: cos=0.720±0.003 agr=0.000004 ortho=0.000022 Disagree: x_cos=0.0799±0.0020 ratio=0.002103 preserve=0.000055 norms=0.0331 E19: 1s loss=0.3326 v_loss=0.3456 Geometry: b_cv=0.1948 e_cv=0.1654 spread=0.01562 a_max=0.668 Experts: cos=0.741±0.003 agr=0.000004 ortho=0.000021 Disagree: x_cos=0.0797±0.0019 ratio=0.003153 preserve=0.000054 norms=0.0169 E20: 1s loss=0.3351 v_loss=0.3460 Geometry: b_cv=0.1992 e_cv=0.1596 spread=0.01567 a_max=0.668 Experts: cos=0.725±0.005 agr=0.000002 ortho=0.000018 Disagree: x_cos=0.0776±0.0023 ratio=0.008188 preserve=0.000053 norms=0.0326 ================================================================= PHASE 3: GEOMETRIC VERIFICATION ================================================================= Passthrough: 1.000000 (target: 1.000) Emb CV: 0.1635 (consensus: 0.1771) Geo context CV: 0.1892 Geo eff_dim: 30.7 / 128 Expert cos: 0.725 ± 0.005 Anchor max cos: 0.668 Disagreement: Cross-expert: 0.0776 ± 0.0023 Ratio: 0.008188 (target: 0.000000) Norm spread: 0.0326 ================================================================= PHASE 4: CLASSIFIER STABILITY TEST ================================================================= Mode Dim Train Val Gap -------------------------------------------------- raw_768 1536 0.498 0.357 0.141 raw+diff 3072 0.567 0.475 0.092 bank_enriched 1792 0.766 0.532 0.235 bank+diff 3584 0.722 0.670 0.052 geo_explicit 6 0.326 0.363 -0.037 ================================================================= SUMMARY ================================================================= Consensus CV: 0.1771 Consensus eff_dim:109.5 Student v_cos: 0.686 Student v_cv: 0.182 Bank params: 2,921,088 Bank geo_eff_dim: 30.7 Bank geo_cv: 0.1892 ================================================================= DONE =================================================================