Create analysis_v1_results.txt

analysis_v1_results.txt

@@ -0,0 +1,243 @@
+================================================================================
+CONSTELLATION BOTTLENECK — FULL ANALYSIS
+  Params: 286,911,940
+  Bottleneck: 281,845,505
+================================================================================
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+TEST 1: Bottleneck Diagnostics
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+  Patches: 16, Anchors/patch: 16, Patch dim: 16
+  Drift: mean=0.171680 rad (9.84°)
+         std=0.053721  min=0.046172  max=0.353512
+         max degrees: 20.25°
+  Skip gate: 0.1179
+  Near 0.29154: 9.0%
+
+  Per-patch drift:
+    P 0: mean=0.1744 (10.0°) max=0.2897 (16.6°) ◄ MAX near 0.29
+    P 1: mean=0.1470 (8.4°) max=0.2480 (14.2°) ◄ MAX near 0.29
+    P 2: mean=0.1465 (8.4°) max=0.2351 (13.5°)
+    P 3: mean=0.1497 (8.6°) max=0.2218 (12.7°)
+    P 4: mean=0.1327 (7.6°) max=0.2235 (12.8°)
+    P 5: mean=0.1639 (9.4°) max=0.2024 (11.6°)
+    P 6: mean=0.1369 (7.8°) max=0.1980 (11.3°)
+    P 7: mean=0.1643 (9.4°) max=0.2249 (12.9°)
+    P 8: mean=0.1752 (10.0°) max=0.2869 (16.4°) ◄ MAX near 0.29
+    P 9: mean=0.1797 (10.3°) max=0.2899 (16.6°) ◄ MAX near 0.29
+    P10: mean=0.1726 (9.9°) max=0.2314 (13.3°)
+    P11: mean=0.2548 (14.6°) max=0.3535 (20.3°) ◄ 0.29
+    P12: mean=0.2078 (11.9°) max=0.2778 (15.9°) ◄ MAX near 0.29
+    P13: mean=0.1867 (10.7°) max=0.2696 (15.4°) ◄ MAX near 0.29
+    P14: mean=0.1440 (8.3°) max=0.1875 (10.7°)
+    P15: mean=0.2105 (12.1°) max=0.2880 (16.5°) ◄ MAX near 0.29
+
+  Anchor spread per patch:
+    P0: mean_cos=0.0030 max=0.6000 min=-0.7023
+    P1: mean_cos=0.0238 max=0.5213 min=-0.5922
+    P2: mean_cos=-0.0004 max=0.6274 min=-0.7749
+    P3: mean_cos=0.0757 max=0.7292 min=-0.5168
+    P4: mean_cos=0.0092 max=0.6820 min=-0.6667
+    P5: mean_cos=0.0548 max=0.8025 min=-0.6882
+    P6: mean_cos=-0.0070 max=0.6800 min=-0.6984
+    P7: mean_cos=0.0219 max=0.7150 min=-0.5714
+
+  Anchor effective dimensionality:
+    P0: eff_dim=11.5 / 16
+    P1: eff_dim=11.4 / 16
+    P2: eff_dim=11.6 / 16
+    P3: eff_dim=11.1 / 16
+    P4: eff_dim=11.1 / 16
+    P5: eff_dim=10.6 / 16
+    P6: eff_dim=11.1 / 16
+    P7: eff_dim=10.9 / 16
+
+  Drift distribution:
+    0.00-0.05:   1 █
+    0.05-0.10:  31 ███████████████████████████████
+    0.10-0.15:  53 █████████████████████████████████████████████████████
+    0.15-0.20: 100 ████████████████████████████████████████████████████████████████████████████████████████████████████
+    0.20-0.25:  51 ███████████████████████████████████████████████████
+    0.25-0.30:  19 ███████████████████
+    0.30-0.35:   0 
+    0.35-0.40:   1 █
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+TEST 2: Sphere Representation — CV of bottleneck embeddings
+  These live on S^15. Does CV approach 0.20?
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+       t  CV_sphere     CV_tri  eff_d_sph  eff_d_tri   sph_norm
+
+  Per-patch CV at t=0 (should be ≈0.20 if d=16):
+    Patch 0: CV=0.6609
+    Patch 1: CV=0.8585
+    Patch 2: CV=0.9021
+    Patch 3: CV=0.8482
+    Patch 4: CV=0.7948
+    Patch 5: CV=0.8473
+    Patch 6: CV=0.8640
+    Patch 7: CV=1.1050
+
+    0.00     0.4321     0.4884       57.9       35.8     4.0000
+    0.10     0.3555     0.4503       68.9       41.9     4.0000
+    0.25     0.3288     0.4592       70.5       43.8     4.0000
+    0.50     0.3436     0.4650       69.5       43.6     4.0000
+    0.75     0.3845     0.5050       59.4       37.5     4.0000
+    0.90     0.4392     0.5671       45.6       29.1     4.0000
+    1.00     0.5039     0.5428       28.9       20.2     4.0000
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━��━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+TEST 3: Per-Class Anchor Routing
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+  Patch 0 — nearest anchor per class:
+       class    0    1    2    3    4    5    6    7    8    9   10   11   12   13   14   15
+       plane 23%█   0%   0%   0%   0%   0%   0%   1% 43%█   0%  7%░ 10%░   0%   2%   0% 14%░
+        auto 19%█   0%   0%   0%   0%   0%   1%   0% 31%█   0% 14%░ 26%█   0%   1%   0%  9%░
+        bird 23%█   0%   0%   0%   0%   1%   1%   0% 36%█   0%  7%░ 20%█   0%   1%   0% 10%░
+         cat 14%░   0%   0%   0%   0%   1%   0%   0% 27%█   0% 19%█ 21%█   0%   0%   1% 16%█
+        deer 15%█   1%   1%   0%   0%   2%   1%   2% 24%█   0% 16%█ 21%█   0%   2%   1% 14%░
+         dog 14%░   0%   0%   0%   0%   0%   1%   0% 31%█   0% 13%░ 25%█   0%   0%   1% 14%░
+        frog 11%░   0%   0%   0%   0%   1%   2%   0% 23%█   0% 18%█ 27%█   0%   0%   0% 17%█
+       horse 18%█   0%   1%   0%   0%   2%   2%   1% 43%█   0% 11%░ 10%░   1%   0%   0% 11%░
+        ship 13%░   1%   0%   0%   0%   1%   0%   0% 50%█   0% 10%░  7%░   0%   1%   0% 16%█
+       truck 22%█   0%   0%   0%   0%   1%   1%   0% 47%█   0%  5%░ 14%░   0%   1%   0%  7%░
+
+  Patch 1 — nearest anchor per class:
+       class    0    1    2    3    4    5    6    7    8    9   10   11   12   13   14   15
+       plane   0%   0%   1%   0%   0%   0%  5%░   0% 93%█   0%   0%   0%   0%   0%   0%   0%
+        auto   0%   0%   2%   0%   0%   1%   4%   1% 92%█   0%   0%   0%   0%   0%   0%   0%
+        bird   0%   0%   0%   0%   0%   0%  7%░   0% 93%█   0%   0%   0%   0%   0%   0%   0%
+         cat   0%   0%   1%   0%   0%   1%   4%   0% 94%█   0%   0%   0%   0%   0%   0%   0%
+        deer   0%   0%   2%   0%   0%   0%   5%   0% 92%█   0%   0%   0%   0%   0%   0%   0%
+         dog   0%   0%   0%   0%   0%   1%  6%░   0% 93%█   0%   0%   0%   0%   0%   0%   0%
+        frog   0%   0%   0%   0%   0%   0%  5%░   0% 94%█   0%   0%   0%   0%   0%   0%   0%
+       horse   0%   0%   1%   0%   0%   2%  5%░   0% 92%█   0%   0%   0%   0%   0%   0%   0%
+        ship   0%   0%   1%   0%   0%   1%   5%   0% 93%█   0%   0%   0%   0%   0%   0%   0%
+       truck   1%   0%   1%   0%   0%   0% 12%░   0% 86%█   0%   0%   0%   0%   0%   0%   0%
+
+  Patch 2 — nearest anchor per class:
+       class    0    1    2    3    4    5    6    7    8    9   10   11   12   13   14   15
+       plane   0%   0%   0%   0%   4%   0%   0%   0%   0%   0%   0% 91%█   1%   0%   0%  5%░
+        auto   0%   0%   0%   0% 14%░   0%   0%   0%   0%   0%   0% 73%█   2%   0%   0% 10%░
+        bird   0%   0%   0%   0%  8%░   0%   0%   0%   0%   0%   0% 75%█   3%   0%   0% 12%░
+         cat   0%   2%   0%   0% 19%█   0%   0%   0%   0%   0%   0% 63%█   2%   0%   0% 14%░
+        deer   0%   1%   0%   0%  9%░   0%   0%   0%   0%   0%   0% 60%█  7%░   0%   0% 22%█
+         dog   0%   1%   0%   0% 21%█   0%   0%   0%   0%   0%   0% 63%█   1%   0%   0% 12%░
+        frog   0%   1%   0%   0% 18%█   0%   0%   0%   0%   0%   0% 64%█   4%   0%   0% 13%░
+       horse   0%   1%   0%   0%  8%░   0%   0%   0%   0%   0%   0% 72%█   4%   0%   0% 14%░
+        ship   0%   0%   0%   0%   2%   0%   0%   0%   0%   0%   0% 87%█   2%   0%   0%  9%░
+       truck   0%   0%   0%   0%   3%   0%   0%   0%   0%   0%   0% 86%█   3%   0%   0%  7%░
+
+  Patch 3 — nearest anchor per class:
+       class    0    1    2    3    4    5    6    7    8    9   10   11   12   13   14   15
+       plane   0%   0%   0%   1%   0%   0% 74%█   0%   0%   0%   0%   4%  5%░ 14%░   1%   0%
+        auto   0%   0%   0%   0%   0%   0% 50%█   1%   0%   0%   0% 26%█  8%░ 13%░   1%   0%
+        bird   0%   0%   0%   0%   0%   0% 70%█   2%   0%   0%   0% 12%░   4%  9%░   2%   0%
+         cat   0%   0%   0%   0%   0%   0% 60%█   1%   0%   1%   0% 20%█  7%░ 10%░   1%   1%
+        deer   0%   0%   0%   0%   0%   0% 67%█   1%   1%   0%   0% 16%█  5%░  8%░   1%   0%
+         dog   0%   0%   0%   0%   0%   0% 59%█   2%   0%   0%   0% 19%█  6%░ 13%░   1%   0%
+        frog   0%   0%   0%   0%   0%   0% 64%█   2%   0%   0%   0% 21%█   4%  8%░   1%   0%
+       horse   0%   0%   0%   0%   0%   0% 73%█   0%   0%   0%   0%  9%░  5%░ 11%░   1%   1%
+        ship   0%   0%   0%   1%   0%   0% 70%█   1%   0%   0%   0%   2%   2% 20%█   3%   0%
+       truck   0%   0%   0%   1%   0%   0% 58%█   1%   0%   0%   0%  5%░  7%░ 23%█   4%   0%
+
+  Anchor routing entropy per class (lower = more concentrated):
+         plane: H=1.203 / 2.773 (43.4% of max)
+          auto: H=1.284 / 2.773 (46.3% of max)
+          bird: H=1.123 / 2.773 (40.5% of max)
+           cat: H=1.121 / 2.773 (40.4% of max)
+          deer: H=1.247 / 2.773 (45.0% of max)
+           dog: H=1.189 / 2.773 (42.9% of max)
+          frog: H=1.106 / 2.773 (39.9% of max)
+         horse: H=1.217 / 2.773 (43.9% of max)
+          ship: H=1.202 / 2.773 (43.4% of max)
+         truck: H=1.212 / 2.773 (43.7% of max)
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+TEST 4: Skip Gate — how much goes through constellation vs skip?
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+  Skip gate value: 0.1179
+  Skip path:          11.8%
+  Constellation path: 88.2%
+  Skip proj params:   268,451,840
+  Patchwork params:   9,194,496
+
+  ⚠ skip_proj is Linear(16384, 16384) = 268,435,456 params
+  ⚠ This single layer is 268M params — larger than the rest of the model combined
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+TEST 5: Generation Quality
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+       class  intra_cos      std       CV     norm
+       plane     0.8908   0.0565   0.5620    33.59
+        auto     0.7927   0.0750   0.4022    27.18
+        bird     0.8655   0.0650   0.5689    28.36
+         cat     0.8152   0.0754   0.4562    27.54
+        deer     0.8810   0.0495   0.4669    27.55
+         dog     0.7997   0.0946   0.4984    27.54
+        frog     0.8504   0.0649   0.4737    25.12
+       horse     0.8360   0.0747   0.4253    28.41
+        ship     0.8957   0.0590   0.6958    32.40
+       truck     0.8352   0.0535   0.3488    29.22
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+TEST 6: Ablation — Skip-only vs Constellation-only
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+        skip_only: pixel_Δ=0.430410  cos_sim=0.597844  max_Δ=1.9629
+       const_only: pixel_Δ=0.111031  cos_sim=0.945127  max_Δ=1.0788
+  ✓ Saved (top=full, mid=skip_only, bot=constellation_only)
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+TEST 7: Velocity Field Quality
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+       t     v_norm   v·target        mse
+    0.05      52.96     0.8513     0.3386
+    0.10      55.94     0.8974     0.2402
+    0.25      58.35     0.9355     0.1554
+    0.50      59.08     0.9474     0.1284
+    0.75      58.68     0.9421     0.1423
+    0.90      57.26     0.9242     0.1869
+    0.95      56.64     0.9130     0.2168
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+TEST 8: ODE Trajectory — geometry through generation
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+    step      t     x_norm      x_std       CV
+       0   1.00      54.24     0.9787   0.0113
+       1   0.98      53.14     0.9589   0.0119
+       5   0.90      48.81     0.8807   0.0114
+      10   0.80      43.57     0.7860   0.0119
+      20   0.60      33.96     0.6124   0.0249
+      30   0.40      26.51     0.4790   0.0735
+      40   0.20      23.17     0.4245   0.1868
+      49   0.02      25.22     0.4671   0.2458
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━��━━
+TEST 9: Inter vs Intra Class Separation
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+  Intra-class cos: 0.8462 ± 0.0350
+  Inter-class cos: 0.8319 ± 0.0244
+  Separation ratio: 1.017×
+
+================================================================================
+ANALYSIS COMPLETE
+================================================================================
+
+  Files in analysis_bn/:
+    class_*.png              per-class samples
+    all_classes.png          4 per class grid
+    ablation_skip_vs_const.png  top=full, mid=skip, bot=constellation
+
+  Key questions answered:
+    1. Does per-patch CV ≈ 0.20? (Test 2)
+       → If yes, the bottleneck lives at the natural S^15 dimension
+    2. Is anchor routing class-specific? (Test 3)
+       → If entropy varies by class, constellation routes differently
+    3. Does the skip path dominate? (Tests 4 & 6)
+       → If skip_only ≈ full, the 268M skip_proj IS the model
+    4. Does constellation-only work at all? (Test 6)
+       → The real test of whether geometric encoding carries signal
+
+================================================================================