Create rapid_prototype_output_bigger_bank.txt

rapid_prototype_output_bigger_bank.txt

@@ -0,0 +1,118 @@
+=================================================================
+RAPID PROTOTYPE: 2-Expert Consensus + Alignment Bank
+=================================================================
+  Device: cuda
+
+=================================================================
+PHASE 0: EXTRACTION
+=================================================================
+  Captions: 20,000
+
+  Extracting: bert...
+Loading weights: 100%
+ 199/199 [00:00<00:00, 4157.84it/s, Materializing param=pooler.dense.weight]
+BertModel LOAD REPORT from: google-bert/bert-base-uncased
+Key                                        | Status     |  | 
+-------------------------------------------+------------+--+-
+cls.predictions.transform.LayerNorm.bias   | UNEXPECTED |  | 
+cls.seq_relationship.weight                | UNEXPECTED |  | 
+cls.predictions.transform.dense.weight     | UNEXPECTED |  | 
+cls.predictions.transform.LayerNorm.weight | UNEXPECTED |  | 
+cls.seq_relationship.bias                  | UNEXPECTED |  | 
+cls.predictions.transform.dense.bias       | UNEXPECTED |  | 
+cls.predictions.bias                       | UNEXPECTED |  | 
+
+Notes:
+- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.
+    bert: 100%|██████████| 157/157 [00:24<00:00,  6.54it/s]
+    Shape: torch.Size([20000, 768])
+
+  Extracting: modern...
+Loading weights: 100%
+ 134/134 [00:00<00:00, 3957.73it/s, Materializing param=layers.21.mlp_norm.weight]
+ModernBertModel LOAD REPORT from: answerdotai/ModernBERT-base
+Key               | Status     |  | 
+------------------+------------+--+-
+head.dense.weight | UNEXPECTED |  | 
+head.norm.weight  | UNEXPECTED |  | 
+decoder.bias      | UNEXPECTED |  | 
+
+Notes:
+- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.
+    modern: 100%|██████████| 157/157 [00:35<00:00,  4.38it/s]
+    Shape: torch.Size([20000, 768])
+
+=================================================================
+PHASE 0b: PROCRUSTES ALIGNMENT
+=================================================================
+  bert      : cos 1.0000 → 1.0000
+  modern    : cos -0.0025 → 0.4849
+  Consensus: torch.Size([20000, 768])
+  cos(consensus, bert): 0.9574
+  cos(consensus, modern): 0.9584
+  Consensus CV: 0.1316
+
+=================================================================
+PHASE 1: TRAIN STUDENT (2 experts, 20K captions)
+=================================================================
+  Student: 11,269,632 params
+  E1: 2s  loss=3.1731  t_acc=0.322  t_cos=0.287  v_acc=0.448  v_cos=0.437  v_cv=0.227
+  E2: 2s  loss=1.6553  t_acc=0.713  t_cos=0.470  v_acc=0.649  v_cos=0.518  v_cv=0.197
+  E3: 2s  loss=1.1581  t_acc=0.858  t_cos=0.531  v_acc=0.814  v_cos=0.566  v_cv=0.169
+  E4: 2s  loss=0.8765  t_acc=0.922  t_cos=0.567  v_acc=0.867  v_cos=0.598  v_cv=0.167
+  E5: 2s  loss=0.7007  t_acc=0.954  t_cos=0.593  v_acc=0.892  v_cos=0.612  v_cv=0.169
+
+  Student saved. v_cos=0.612, v_cv=0.169
+
+=================================================================
+PHASE 2: TRAIN ALIGNMENT BANK (student frozen)
+=================================================================
+  Pre-encoding through frozen student...
+  Student embeddings: torch.Size([18000, 768])
+    Expert 0 (bert): rotation loaded, cos_after=1.0000
+    Expert 1 (modern): rotation loaded, cos_after=0.4849
+    Anchors: 128 initialized from consensus embeddings
+  Bank: 1,305,152 params
+  E 1: 1s  loss=0.3517  v_loss=0.3123  expert_agr=0.00029  ortho=0.00018  spread=0.02575  cv=0.2469  anchor_max=0.560  expert_cos=0.657±0.061
+  E 2: 1s  loss=0.2758  v_loss=0.2718  expert_agr=0.00017  ortho=0.00027  spread=0.02053  cv=0.2049  anchor_max=0.587  expert_cos=0.634±0.064
+  E 3: 1s  loss=0.2505  v_loss=0.2533  expert_agr=0.00016  ortho=0.00023  spread=0.01885  cv=0.1988  anchor_max=0.590  expert_cos=0.619±0.067
+  E 4: 1s  loss=0.2391  v_loss=0.2338  expert_agr=0.00017  ortho=0.00022  spread=0.01824  cv=0.1965  anchor_max=0.593  expert_cos=0.627±0.069
+  E 5: 1s  loss=0.2327  v_loss=0.2376  expert_agr=0.00018  ortho=0.00023  spread=0.01752  cv=0.1933  anchor_max=0.595  expert_cos=0.613±0.070
+  E 6: 1s  loss=0.2274  v_loss=0.2334  expert_agr=0.00017  ortho=0.00022  spread=0.01680  cv=0.1882  anchor_max=0.597  expert_cos=0.610±0.070
+  E 7: 1s  loss=0.2249  v_loss=0.2297  expert_agr=0.00018  ortho=0.00021  spread=0.01603  cv=0.1880  anchor_max=0.597  expert_cos=0.614±0.074
+  E 8: 1s  loss=0.2221  v_loss=0.2236  expert_agr=0.00017  ortho=0.00021  spread=0.01550  cv=0.1857  anchor_max=0.598  expert_cos=0.601±0.074
+  E 9: 1s  loss=0.2206  v_loss=0.2351  expert_agr=0.00019  ortho=0.00021  spread=0.01500  cv=0.1850  anchor_max=0.599  expert_cos=0.588±0.077
+  E10: 1s  loss=0.2200  v_loss=0.2142  expert_agr=0.00024  ortho=0.00022  spread=0.01470  cv=0.1831  anchor_max=0.599  expert_cos=0.605±0.075
+  E11: 1s  loss=0.2181  v_loss=0.2254  expert_agr=0.00028  ortho=0.00021  spread=0.01445  cv=0.1782  anchor_max=0.599  expert_cos=0.629±0.072
+  E12: 1s  loss=0.2179  v_loss=0.2212  expert_agr=0.00021  ortho=0.00023  spread=0.01419  cv=0.1812  anchor_max=0.599  expert_cos=0.634±0.070
+  E13: 1s  loss=0.2170  v_loss=0.2259  expert_agr=0.00022  ortho=0.00023  spread=0.01387  cv=0.1792  anchor_max=0.599  expert_cos=0.614±0.072
+  E14: 1s  loss=0.2152  v_loss=0.2106  expert_agr=0.00016  ortho=0.00020  spread=0.01400  cv=0.1750  anchor_max=0.599  expert_cos=0.647±0.069
+  E15: 1s  loss=0.2151  v_loss=0.2301  expert_agr=0.00021  ortho=0.00021  spread=0.01368  cv=0.1766  anchor_max=0.599  expert_cos=0.639±0.073
+  E16: 1s  loss=0.2149  v_loss=0.2134  expert_agr=0.00022  ortho=0.00023  spread=0.01338  cv=0.1754  anchor_max=0.599  expert_cos=0.631±0.070
+  E17: 1s  loss=0.2151  v_loss=0.2110  expert_agr=0.00019  ortho=0.00022  spread=0.01341  cv=0.1778  anchor_max=0.599  expert_cos=0.642±0.073
+  E18: 1s  loss=0.2146  v_loss=0.2114  expert_agr=0.00023  ortho=0.00022  spread=0.01306  cv=0.1734  anchor_max=0.599  expert_cos=0.630±0.069
+  E19: 1s  loss=0.2147  v_loss=0.2127  expert_agr=0.00020  ortho=0.00023  spread=0.01300  cv=0.1768  anchor_max=0.599  expert_cos=0.610±0.072
+  E20: 1s  loss=0.2151  v_loss=0.2211  expert_agr=0.00019  ortho=0.00020  spread=0.01300  cv=0.1779  anchor_max=0.599  expert_cos=0.626±0.072
+
+=================================================================
+PHASE 3: GEOMETRIC VERIFICATION
+=================================================================
+  Passthrough integrity: 1.000000 (should be ~1.000)
+  Geo context CV: 0.1691
+  Geo context eff_dim: 21.9
+  Geo context shape: torch.Size([2000, 64])
+
+=================================================================
+PHASE 4: CLASSIFIER STABILITY TEST
+=================================================================
+  with_bank      : train_acc=0.499  val_acc=0.390  gap=0.109
+  without_bank   : train_acc=0.442  val_acc=0.372  gap=0.070
+
+=================================================================
+DONE
+=================================================================
+
+  Student: mini_student.pt
+  Bank: alignment_bank.pt
+  Consensus CV: 0.1316
+  Student v_cos: 0.612