================================================================= RAPID PROTOTYPE: 2-Expert Consensus + Alignment Bank ================================================================= Device: cuda ================================================================= PHASE 0: EXTRACTION ================================================================= Captions: 20,000 Extracting: bert... Loading weights: 100%  199/199 [00:00<00:00, 4115.47it/s, Materializing param=pooler.dense.weight] BertModel LOAD REPORT from: google-bert/bert-base-uncased Key | Status | | -------------------------------------------+------------+--+- cls.predictions.transform.LayerNorm.bias | UNEXPECTED | | cls.seq_relationship.weight | UNEXPECTED | | cls.predictions.transform.dense.weight | UNEXPECTED | | cls.predictions.transform.LayerNorm.weight | UNEXPECTED | | cls.seq_relationship.bias | UNEXPECTED | | cls.predictions.transform.dense.bias | UNEXPECTED | | cls.predictions.bias | UNEXPECTED | | Notes: - UNEXPECTED :can be ignored when loading from different task/architecture; not ok if you expect identical arch. bert: 100%|██████████| 157/157 [00:23<00:00, 6.55it/s] Shape: torch.Size([20000, 768]) Extracting: modern... Loading weights: 100%  134/134 [00:00<00:00, 3938.48it/s, Materializing param=layers.21.mlp_norm.weight] ModernBertModel LOAD REPORT from: answerdotai/ModernBERT-base Key | Status | | ------------------+------------+--+- head.dense.weight | UNEXPECTED | | head.norm.weight | UNEXPECTED | | decoder.bias | UNEXPECTED | | Notes: - UNEXPECTED :can be ignored when loading from different task/architecture; not ok if you expect identical arch. modern: 100%|██████████| 157/157 [00:35<00:00, 4.38it/s] Shape: torch.Size([20000, 768]) ================================================================= PHASE 0b: PROCRUSTES ALIGNMENT ================================================================= bert : cos 1.0000 → 1.0000 modern : cos -0.0025 → 0.4849 Consensus: torch.Size([20000, 768]) cos(consensus, bert): 0.9574 cos(consensus, modern): 0.9584 Consensus CV: 0.1316 ================================================================= PHASE 1: TRAIN STUDENT (2 experts, 20K captions) ================================================================= Student: 11,269,632 params E1: 2s loss=3.1731 t_acc=0.322 t_cos=0.287 v_acc=0.448 v_cos=0.437 v_cv=0.227 E2: 2s loss=1.6553 t_acc=0.713 t_cos=0.470 v_acc=0.649 v_cos=0.518 v_cv=0.197 E3: 2s loss=1.1581 t_acc=0.858 t_cos=0.531 v_acc=0.814 v_cos=0.566 v_cv=0.169 E4: 2s loss=0.8765 t_acc=0.922 t_cos=0.567 v_acc=0.867 v_cos=0.598 v_cv=0.167 E5: 2s loss=0.7007 t_acc=0.954 t_cos=0.593 v_acc=0.892 v_cos=0.612 v_cv=0.169 Student saved. v_cos=0.612, v_cv=0.169 ================================================================= PHASE 2: TRAIN ALIGNMENT BANK (student frozen) ================================================================= Pre-encoding through frozen student... Student embeddings: torch.Size([18000, 768]) Expert 0 (bert): rotation loaded, cos_after=1.0000 Expert 1 (modern): rotation loaded, cos_after=0.4849 Anchors: 128 initialized from consensus embeddings Bank: 1,305,152 params E 1: 1s loss=0.3517 v_loss=0.3123 expert_agr=0.00029 ortho=0.00018 spread=0.02575 cv=0.2469 anchor_max=0.560 expert_cos=0.657±0.061 E 2: 1s loss=0.2758 v_loss=0.2718 expert_agr=0.00017 ortho=0.00027 spread=0.02053 cv=0.2049 anchor_max=0.587 expert_cos=0.634±0.064 E 3: 1s loss=0.2505 v_loss=0.2533 expert_agr=0.00016 ortho=0.00023 spread=0.01885 cv=0.1988 anchor_max=0.590 expert_cos=0.619±0.067 E 4: 1s loss=0.2391 v_loss=0.2338 expert_agr=0.00017 ortho=0.00022 spread=0.01824 cv=0.1965 anchor_max=0.593 expert_cos=0.627±0.069 E 5: 1s loss=0.2327 v_loss=0.2376 expert_agr=0.00018 ortho=0.00023 spread=0.01752 cv=0.1933 anchor_max=0.595 expert_cos=0.613±0.070 E 6: 1s loss=0.2274 v_loss=0.2334 expert_agr=0.00017 ortho=0.00022 spread=0.01680 cv=0.1882 anchor_max=0.597 expert_cos=0.610±0.070 E 7: 1s loss=0.2249 v_loss=0.2297 expert_agr=0.00018 ortho=0.00021 spread=0.01603 cv=0.1880 anchor_max=0.597 expert_cos=0.614±0.074 E 8: 1s loss=0.2221 v_loss=0.2236 expert_agr=0.00017 ortho=0.00021 spread=0.01550 cv=0.1857 anchor_max=0.598 expert_cos=0.601±0.074 E 9: 1s loss=0.2206 v_loss=0.2351 expert_agr=0.00019 ortho=0.00021 spread=0.01500 cv=0.1850 anchor_max=0.599 expert_cos=0.588±0.077 E10: 1s loss=0.2200 v_loss=0.2142 expert_agr=0.00024 ortho=0.00022 spread=0.01470 cv=0.1831 anchor_max=0.599 expert_cos=0.605±0.075 E11: 1s loss=0.2181 v_loss=0.2254 expert_agr=0.00028 ortho=0.00021 spread=0.01445 cv=0.1782 anchor_max=0.599 expert_cos=0.629±0.072 E12: 1s loss=0.2179 v_loss=0.2212 expert_agr=0.00021 ortho=0.00023 spread=0.01419 cv=0.1812 anchor_max=0.599 expert_cos=0.634±0.070 E13: 1s loss=0.2170 v_loss=0.2259 expert_agr=0.00022 ortho=0.00023 spread=0.01387 cv=0.1792 anchor_max=0.599 expert_cos=0.614±0.072 E14: 1s loss=0.2152 v_loss=0.2106 expert_agr=0.00016 ortho=0.00020 spread=0.01400 cv=0.1750 anchor_max=0.599 expert_cos=0.647±0.069 E15: 1s loss=0.2151 v_loss=0.2301 expert_agr=0.00021 ortho=0.00021 spread=0.01368 cv=0.1766 anchor_max=0.599 expert_cos=0.639±0.073 E16: 1s loss=0.2149 v_loss=0.2134 expert_agr=0.00022 ortho=0.00023 spread=0.01338 cv=0.1754 anchor_max=0.599 expert_cos=0.631±0.070 E17: 1s loss=0.2151 v_loss=0.2110 expert_agr=0.00019 ortho=0.00022 spread=0.01341 cv=0.1778 anchor_max=0.599 expert_cos=0.642±0.073 E18: 1s loss=0.2146 v_loss=0.2114 expert_agr=0.00023 ortho=0.00022 spread=0.01306 cv=0.1734 anchor_max=0.599 expert_cos=0.630±0.069 E19: 1s loss=0.2147 v_loss=0.2127 expert_agr=0.00020 ortho=0.00023 spread=0.01300 cv=0.1768 anchor_max=0.599 expert_cos=0.610±0.072 E20: 1s loss=0.2151 v_loss=0.2211 expert_agr=0.00019 ortho=0.00020 spread=0.01300 cv=0.1779 anchor_max=0.599 expert_cos=0.626±0.072 ================================================================= PHASE 3: GEOMETRIC VERIFICATION ================================================================= Passthrough integrity: 1.000000 (should be ~1.000) Geo context CV: 0.1691 Geo context eff_dim: 21.9 Geo context shape: torch.Size([2000, 64]) ================================================================= PHASE 4: CLASSIFIER STABILITY TEST ================================================================= with_bank : train_acc=0.499 val_acc=0.390 gap=0.109 without_bank : train_acc=0.442 val_acc=0.372 gap=0.070 ================================================================= DONE ================================================================= Student: mini_student.pt Bank: alignment_bank.pt Consensus CV: 0.1316 Student v_cos: 0.612