Compare importance-ranked and random pruning quality curves

Overlay the measured random 10% and 25% controls on the pruning-quality figure and update the model-card insight to foreground the advantage of HEAPr-style importance ranking.

README.md

@@ -148,16 +148,16 @@ pruned child-group work.
 
 ## Insights
 
-### Quality degradation is nonlinear
+### Importance ranking improves the pruning tradeoff
 
 <p align="center">
-  <img alt="Line chart showing a nonlinear increase in perplexity as more structured MoE parameters are pruned" src="assets/pruning_quality_curve.png" width="88%">
+  <img alt="Line chart showing lower perplexity degradation for importance-ranked structured MoE pruning than random pruning" src="assets/pruning_quality_curve.png" width="88%">
 </p>
 
-Pruning more groups consistently increases full-cache perplexity, but the tradeoff is not linear.
-The first 10% adds only `1.66%` perplexity, while moving from 20% to 40% raises the delta from
-`7.18%` to `39.99%`. The downstream GSM8K curve also worsens sharply beyond 10%, so we recommend
-10% as the initial operating point.
+Importance-ranked pruning consistently beats the measured random controls. At the recommended 10%
+point, perplexity rises by only `1.66%`, compared with `14.28%` for random pruning. The ranked curve
+is also nonlinear: moving from 20% to 40% raises its perplexity delta from `7.18%` to `39.99%`.
+Together with the GSM8K curve, this supports 10% as the initial operating point.
 
 ### Later layers have lower-importance atomic experts
 

scripts/plot_pruning_insights.py

@@ -51,27 +51,54 @@ def plot_pruning_curve(summary: dict, output_dir: Path) -> None:
     baseline = summary["native_group_loss_sweep_baseline"]["perplexity"]
     x = np.array([row["groups_pruned_ratio"] * 100 for row in rows])
     y = np.array([(row["perplexity"] / baseline - 1) * 100 for row in rows])
+    random_controls = [
+        summary["random_native_group_10pct_control"],
+        summary["random_native_group_25pct_control"],
+    ]
+    random_x = np.array([0, *[row["groups_pruned_ratio"] * 100 for row in random_controls]])
+    random_y = np.array([0, *[(row["perplexity"] / baseline - 1) * 100 for row in random_controls]])
 
     fig, ax = plt.subplots(figsize=(10, 5.6))
-    sns.lineplot(x=x, y=y, marker="o", linewidth=3, markersize=10, color="#78dce8", ax=ax)
+    sns.lineplot(
+        x=x,
+        y=y,
+        marker="o",
+        linewidth=3,
+        markersize=10,
+        color="#78dce8",
+        label="HEAPr-style importance ranking",
+        ax=ax,
+    )
+    sns.lineplot(
+        x=random_x,
+        y=random_y,
+        marker="o",
+        linewidth=2.5,
+        markersize=9,
+        linestyle="--",
+        color="#f5a65b",
+        label="Random control",
+        ax=ax,
+    )
     selected = int(np.flatnonzero(x == 10)[0])
     ax.scatter([x[selected]], [y[selected]], s=180, color="#b7d36b", zorder=4)
     ax.annotate(
         "Recommended operating point\n10% pruning, +1.66% perplexity",
         xy=(x[selected], y[selected]),
-        xytext=(28, 24),
+        xytext=(48, 34),
         textcoords="offset points",
         color="#e6fbff",
         fontsize=12,
         arrowprops={"arrowstyle": "->", "color": "#b7d36b"},
     )
     ax.set(
-        title="Pruning Quality Degrades Nonlinearly",
+        title="Importance Ranking Outperforms Random Pruning",
         xlabel="Structured MoE parameters identified for removal (%)",
         ylabel="Perplexity increase vs. BF16 baseline (%)",
     )
     ax.set_xlim(-1, 42)
     ax.set_ylim(-1, 44)
+    ax.legend(frameon=False)
     sns.despine(ax=ax)
     fig.tight_layout()
     fig.savefig(output_dir / "pruning_quality_curve.png", dpi=180)