Upload analysis/findings/activation_analysis_findings.md with huggingface_hub
Browse files
analysis/findings/activation_analysis_findings.md
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Activation Energy Analysis - Key Findings
|
| 2 |
+
|
| 3 |
+
The analysis ran on the actual HPS model (hps_model.h5 from HuggingFace) with 100 random input traces.
|
| 4 |
+
|
| 5 |
+
Important correction: The previous report claimed "52.2% energy destroyed" but the actual analysis shows only 5.9% of L2 energy is destroyed by GAP. This makes sense because GAP preserves the mean — it's the VARIANCE (spatial information) that is destroyed, not the total energy.
|
| 6 |
+
|
| 7 |
+
The correct framing should be about INFORMATION LOSS (variance), not raw energy:
|
| 8 |
+
- Pre-GAP: 1008 spatial positions x 512 channels = 516,096 values
|
| 9 |
+
- Post-GAP: 1 x 512 = 512 values (1008:1 compression)
|
| 10 |
+
- Mean per-channel spatial variance: 14.40
|
| 11 |
+
- Total spatial variance destroyed: 7,373.82
|
| 12 |
+
- The spatial variance IS the discriminative information — it tells the model WHERE features are located
|
| 13 |
+
- GAP destroys 100% of this spatial variance by definition (averaging removes all position-dependent information)
|
| 14 |
+
|
| 15 |
+
The figure shows:
|
| 16 |
+
(a) Heatmap: clear spatial structure with hotspots at specific positions (especially around positions 0-10 in binned space)
|
| 17 |
+
(b) Per-channel comparison: pre-GAP energy varies dramatically across positions, post-GAP is a flat average
|
| 18 |
+
(c) Variance distribution: highly skewed, a few channels have very high spatial variance (up to 1400+), most near zero
|
| 19 |
+
(d) Single channel collapse: channel 90 has a spike at one spatial position (value ~220) but GAP reduces it to 49.1
|
| 20 |
+
(e) Pie chart: 94.1% energy retained, 5.9% destroyed (but this understates the information loss)
|