made sure prompts are always alphabetically ordered within categories

app/src/content/chapters/3-experiments.mdx

@@ -101,7 +101,7 @@ Can we design prompts that consistently beat DCLM?
 
 ### Can New Prompts Beat DCLM?
 
-Since most existing prompts fail to beat DCLM, we designed nine novel prompt formats targeting different skills ([math](#math), [table](#table), [faq](#faq), [tutorial](#tutorial), [article](#article), [commentary](#commentary), [discussion](#discussion), [narrative](#narrative), [explanation](#explanation)), all using Gemma-3-1B on FineWeb-Edu-HQ. Four prompts ([math](#math), [table](#table), [faq](#faq), [tutorial](#tutorial)) outperform DCLM, while [article](#article), [narrative](#narrative), [explanation](#explanation), [commentary](#commentary), and [discussion](#discussion) are at or below DCLM level (see <FigRef target="new-prompts" />). The best-performing prompts all restructure the source content into pedagogically rich formats.
+Since most existing prompts fail to beat DCLM, we designed nine novel prompt formats targeting different skills ([article](#article), [commentary](#commentary), [discussion](#discussion), [explanation](#explanation), [faq](#faq), [math](#math), [narrative](#narrative), [table](#table), [tutorial](#tutorial)), all using Gemma-3-1B on FineWeb-Edu-HQ. Four prompts ([faq](#faq), [math](#math), [table](#table), [tutorial](#tutorial)) outperform DCLM, while [article](#article), [commentary](#commentary), [discussion](#discussion), [explanation](#explanation), and [narrative](#narrative) are at or below DCLM level (see <FigRef target="new-prompts" />). The best-performing prompts all restructure the source content into pedagogically rich formats.
 
 <HtmlEmbed
   id="new-prompts"
@@ -109,15 +109,15 @@ Since most existing prompts fail to beat DCLM, we designed nine novel prompt for
   desc="Nine new prompts compared against the DCLM baseline."
   config={{
     datasets: {
-      "mix-fw_edu_hq-math_1b_hq": "Math",
-      "mix-fw_edu_hq-table_1b_hq": "Table",
-      "mix-fw_edu_hq-faq_1b_hq": "FAQ",
-      "mix-fw_edu_hq-tutorial_1b_hq": "Tutorial",
-      "mix-fw_edu_hq-narrative_1b_hq": "Narrative",
       "mix-fw_edu_hq-article_1b_hq": "Article",
-      "mix-fw_edu_hq-explanation_1b_hq": "Explanation",
       "mix-fw_edu_hq-commentary_1b_hq": "Commentary",
       "mix-fw_edu_hq-discussion_1b_hq": "Discussion",
+      "mix-fw_edu_hq-explanation_1b_hq": "Explanation",
+      "mix-fw_edu_hq-faq_1b_hq": "FAQ",
+      "mix-fw_edu_hq-math_1b_hq": "Math",
+      "mix-fw_edu_hq-narrative_1b_hq": "Narrative",
+      "mix-fw_edu_hq-table_1b_hq": "Table",
+      "mix-fw_edu_hq-tutorial_1b_hq": "Tutorial",
       dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
     }
   }}
@@ -131,8 +131,8 @@ We want to know whether using a stronger model leads to better synthetic data. W
 
 #### Does the model size matter?
 
-We compare all Gemma-3 sizes (270M, 1B, 4B, 12B, 27B) on the [tutorial](#tutorial), [math](#math), and REWIRE's [guided_rewrite](#guided_rewrite_original) prompts (use the Setup dropdown in <FigRef target="model-size" /> to switch between them).
-For [tutorial](#tutorial) and [math](#math), the 270M model underperforms, but 1B through 27B show no significant difference.
+We compare all Gemma-3 sizes (270M, 1B, 4B, 12B, 27B) on the [math](#math), [tutorial](#tutorial), and REWIRE's [guided_rewrite](#guided_rewrite_original) prompts (use the Setup dropdown in <FigRef target="model-size" /> to switch between them).
+For [math](#math) and [tutorial](#tutorial), the 270M model underperforms, but 1B through 27B show no significant difference.
 SmolLM2 (135M, 360M, 1.7B) tells the same story on [tutorial](#tutorial): there is a clear performance gradient up to the 1B range.
 The one exception is [guided_rewrite](#guided_rewrite_original), where the 4B model edges ahead of the 1B, while 4B through 27B remain equivalent.
 This prompt is substantially more complex (detailed rewriting instructions, quality criteria, multi-step formatting requirements), which likely raises the minimum capability threshold.
@@ -148,16 +148,6 @@ It is possible that larger models produce richer or more nuanced rephrasings tha
   desc="Model sizes across Gemma-3 and SmolLM2. Use the Setup dropdown to compare across models and prompts."
   config={{
     setups: {
-      "Gemma-3: Tutorial": {
-        datasets: {
-          "mix-fw_edu_hq-tutorial_27b_hq": "Gemma-3 27B",
-          "mix-fw_edu_hq-tutorial_12b_hq": "Gemma-3 12B",
-          "mix-fw_edu_hq-tutorial_4b_hq": "Gemma-3 4B",
-          "mix-fw_edu_hq-tutorial_1b_hq": "Gemma-3 1B",
-          "mix-fw_edu_hq-tutorial_270m_hq": "Gemma-3 270M",
-          dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
-        }
-      },
       "Gemma-3: Math": {
         datasets: {
           "mix-fw_edu_hq-math_27b_hq": "Gemma-3 27B",
@@ -178,6 +168,16 @@ It is possible that larger models produce richer or more nuanced rephrasings tha
           dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
         }
       },
+      "Gemma-3: Tutorial": {
+        datasets: {
+          "mix-fw_edu_hq-tutorial_27b_hq": "Gemma-3 27B",
+          "mix-fw_edu_hq-tutorial_12b_hq": "Gemma-3 12B",
+          "mix-fw_edu_hq-tutorial_4b_hq": "Gemma-3 4B",
+          "mix-fw_edu_hq-tutorial_1b_hq": "Gemma-3 1B",
+          "mix-fw_edu_hq-tutorial_270m_hq": "Gemma-3 270M",
+          dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
+        }
+      },
       "SmolLM2: Tutorial": {
         datasets: {
           "mix-fw_edu_hq-tutorial_smollm2_1.7b_hq": "SmolLM2 1.7B",
@@ -194,7 +194,7 @@ On high-quality source data, we see no evidence that larger models help. But REW
 
 #### Do we need better models for rephrasing low-quality data?
 
-The REWIRE [@rewire] paper claims that upcycling low-quality data requires large models (Llama-3.3 70B in their case). We compare 1B vs 12B models on HQ vs LQ source data across four prompts ([continue](#continue), [summarize](#summarize), [tutorial](#tutorial), [faq](#faq)). Use the Setup dropdown to switch between prompts. The results are mixed: for some prompts 12B helps slightly with LQ data, but for the [FAQ](#faq) prompt the 1B model actually wins (see <FigRef target="size-quality" />). We see no consistent advantage of using larger models for low-quality data.
+The REWIRE [@rewire] paper claims that upcycling low-quality data requires large models (Llama-3.3 70B in their case). We compare 1B vs 12B models on HQ vs LQ source data across four prompts ([continue](#continue), [summarize](#summarize), [faq](#faq), [tutorial](#tutorial)). Use the Setup dropdown to switch between prompts. The results are mixed: for some prompts 12B helps slightly with LQ data, but for the [FAQ](#faq) prompt the 1B model actually wins (see <FigRef target="size-quality" />). We see no consistent advantage of using larger models for low-quality data.
 
 <HtmlEmbed
   id="size-quality"
@@ -220,15 +220,6 @@ The REWIRE [@rewire] paper claims that upcycling low-quality data requires large
           dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
         }
       },
-      "Tutorial Prompt": {
-        datasets: {
-          "mix-fw_edu_hq-tutorial_1b_hq": "1B, HQ Source",
-          "mix-fw_edu_hq-tutorial_12b_hq": "12B, HQ Source",
-          "mix-fw_edu_hq-tutorial_12b_lq": "12B, LQ Source",
-          "mix-fw_edu_hq-tutorial_1b_lq": "1B, LQ Source",
-          dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
-        }
-      },
       "FAQ Prompt": {
         datasets: {
           "mix-fw_edu_hq-faq_1b_hq": "1B, HQ Source",
@@ -237,6 +228,15 @@ The REWIRE [@rewire] paper claims that upcycling low-quality data requires large
           "mix-fw_edu_hq-faq_12b_lq": "12B, LQ Source",
           dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
         }
+      },
+      "Tutorial Prompt": {
+        datasets: {
+          "mix-fw_edu_hq-tutorial_1b_hq": "1B, HQ Source",
+          "mix-fw_edu_hq-tutorial_12b_hq": "12B, HQ Source",
+          "mix-fw_edu_hq-tutorial_12b_lq": "12B, LQ Source",
+          "mix-fw_edu_hq-tutorial_1b_lq": "1B, LQ Source",
+          dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
+        }
       }
     }
   }}
@@ -280,39 +280,28 @@ We hypothesize that SmolLM2's consistently strong rephrasing performance origina
           dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
         }
       },
-      "Tutorial Prompt": {
+      "Explanation Prompt": {
         datasets: {
-          "mix-fw_edu_hq-tutorial_smollm2_1.7b_hq": "SmolLM2",
-          "mix-fw_edu_hq-tutorial_falcon3_1b_hq": "Falcon3",
-          "mix-fw_edu_hq-tutorial_qwen3_1.7b_hq": "Qwen3",
-          "mix-fw_edu_hq-tutorial_1b_hq": "Gemma-3",
-          "mix-fw_edu_hq-tutorial_granite3_1b_hq": "Granite3",
-          "mix-fw_edu_hq-tutorial_llama3.2_1b_hq": "Llama-3.2",
+          "mix-fw_edu_hq-explanation_smollm2_1.7b_hq": "SmolLM2",
+          "mix-fw_edu_hq-explanation_falcon3_1b_hq": "Falcon3",
+          "mix-fw_edu_hq-explanation_granite3_1b_hq": "Granite3",
+          "mix-fw_edu_hq-explanation_1b_hq": "Gemma-3",
+          "mix-fw_edu_hq-explanation_llama3.2_1b_hq": "Llama-3.2",
+          "mix-fw_edu_hq-explanation_qwen3_1.7b_hq": "Qwen3",
           dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
         }
       },
       "FAQ Prompt": {
         datasets: {
           "mix-fw_edu_hq-faq_smollm2_1.7b_hq": "SmolLM2",
-          "mix-fw_edu_hq-faq_llama3.2_1b_hq": "Llama-3.2",
           "mix-fw_edu_hq-faq_falcon3_1b_hq": "Falcon3",
-          "mix-fw_edu_hq-faq_1b_hq": "Gemma-3",
           "mix-fw_edu_hq-faq_granite3_1b_hq": "Granite3",
+          "mix-fw_edu_hq-faq_1b_hq": "Gemma-3",
+          "mix-fw_edu_hq-faq_llama3.2_1b_hq": "Llama-3.2",
           "mix-fw_edu_hq-faq_qwen3_1.7b_hq": "Qwen3",
           dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
         }
       },
-      "Table Prompt": {
-        datasets: {
-          "mix-fw_edu_hq-table_smollm2_1.7b_hq": "SmolLM2",
-          "mix-fw_edu_hq-table_falcon3_1b_hq": "Falcon3",
-          "mix-fw_edu_hq-table_granite3_1b_hq": "Granite3",
-          "mix-fw_edu_hq-table_qwen3_1.7b_hq": "Qwen3",
-          "mix-fw_edu_hq-table_llama3.2_1b_hq": "Llama-3.2",
-          "mix-fw_edu_hq-table_1b_hq": "Gemma-3",
-          dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
-        }
-      },
       "Math Prompt": {
         datasets: {
           "mix-fw_edu_hq-math_smollm2_1.7b_hq": "SmolLM2",
@@ -335,14 +324,25 @@ We hypothesize that SmolLM2's consistently strong rephrasing performance origina
           dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
         }
       },
-      "Explanation Prompt": {
+      "Table Prompt": {
         datasets: {
-          "mix-fw_edu_hq-explanation_smollm2_1.7b_hq": "SmolLM2",
-          "mix-fw_edu_hq-explanation_falcon3_1b_hq": "Falcon3",
-          "mix-fw_edu_hq-explanation_granite3_1b_hq": "Granite3",
-          "mix-fw_edu_hq-explanation_1b_hq": "Gemma-3",
-          "mix-fw_edu_hq-explanation_llama3.2_1b_hq": "Llama-3.2",
-          "mix-fw_edu_hq-explanation_qwen3_1.7b_hq": "Qwen3",
+          "mix-fw_edu_hq-table_smollm2_1.7b_hq": "SmolLM2",
+          "mix-fw_edu_hq-table_falcon3_1b_hq": "Falcon3",
+          "mix-fw_edu_hq-table_granite3_1b_hq": "Granite3",
+          "mix-fw_edu_hq-table_1b_hq": "Gemma-3",
+          "mix-fw_edu_hq-table_llama3.2_1b_hq": "Llama-3.2",
+          "mix-fw_edu_hq-table_qwen3_1.7b_hq": "Qwen3",
+          dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
+        }
+      },
+      "Tutorial Prompt": {
+        datasets: {
+          "mix-fw_edu_hq-tutorial_smollm2_1.7b_hq": "SmolLM2",
+          "mix-fw_edu_hq-tutorial_falcon3_1b_hq": "Falcon3",
+          "mix-fw_edu_hq-tutorial_granite3_1b_hq": "Granite3",
+          "mix-fw_edu_hq-tutorial_1b_hq": "Gemma-3",
+          "mix-fw_edu_hq-tutorial_llama3.2_1b_hq": "Llama-3.2",
+          "mix-fw_edu_hq-tutorial_qwen3_1.7b_hq": "Qwen3",
           dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
         }
       }
@@ -386,7 +386,7 @@ So far we've always mixed synthetic data with a <Glossary term="source dataset"
 
 #### Is synthetic data enough?
 
-We compare synthetic-only training vs mixed training (synthetic + source) for [tutorial](#tutorial) and [faq](#faq) prompts on DCLM and FineWeb-Edu-HQ sources. Synthetic-only training falls short of both DCLM and mixed training (see <FigRef target="synthetic-only" />). Mixed training consistently improves over both the synthetic-only and original-data-only baselines.
+We compare synthetic-only training vs mixed training (synthetic + source) for [faq](#faq) and [tutorial](#tutorial) prompts on DCLM and FineWeb-Edu-HQ sources. Synthetic-only training falls short of both DCLM and mixed training (see <FigRef target="synthetic-only" />). Mixed training consistently improves over both the synthetic-only and original-data-only baselines.
 
 <HtmlEmbed
   id="synthetic-only"
@@ -460,7 +460,7 @@ The mix-in dataset matters enormously. But what about the source dataset we feed
 
 #### Does the source dataset matter?
 
-We rephrase four datasets (DCLM, Cosmopedia, FineWeb-Edu-HQ, FineWeb-Edu-LQ) with [tutorial](#tutorial) and [faq](#faq) prompts, testing two regimes: (a) mix-in equals source, and (b) fixed mix-in (FineWeb-Edu-HQ). When mix-in varies with source, source quality appears to matter: FineWeb-Edu-HQ and DCLM clearly outperform FineWeb-Edu-LQ and Cosmopedia (see <FigRef target="source-dataset-mixin-source" />). But when we fix the mix-in to FineWeb-Edu-HQ, the source effect nearly vanishes (see <FigRef target="source-dataset-fixed-mixin" />). Source dataset quality is secondary to mix-in dataset quality. With a strong mix-in, even low-quality sources produce competitive synthetic data.
+We rephrase four datasets (DCLM, Cosmopedia, FineWeb-Edu-HQ, FineWeb-Edu-LQ) with [faq](#faq) and [tutorial](#tutorial) prompts, testing two regimes: (a) mix-in equals source, and (b) fixed mix-in (FineWeb-Edu-HQ). When mix-in varies with source, source quality appears to matter: FineWeb-Edu-HQ and DCLM clearly outperform FineWeb-Edu-LQ and Cosmopedia (see <FigRef target="source-dataset-mixin-source" />). But when we fix the mix-in to FineWeb-Edu-HQ, the source effect nearly vanishes (see <FigRef target="source-dataset-fixed-mixin" />). Source dataset quality is secondary to mix-in dataset quality. With a strong mix-in, even low-quality sources produce competitive synthetic data.
 
 <HtmlEmbed
   id="source-dataset-mixin-source"
@@ -468,15 +468,6 @@ We rephrase four datasets (DCLM, Cosmopedia, FineWeb-Edu-HQ, FineWeb-Edu-LQ) wit
   desc="Effect of source dataset when mix-in equals source. Use the Setup dropdown to compare prompts."
   config={{
     setups: {
-      "Tutorial Prompt": {
-        datasets: {
-          "mix-fw_edu_hq-tutorial_1b_hq": "Source: FineWeb-Edu-HQ",
-          "mix-dclm-tutorial_1b_dclm": "Source: DCLM",
-          "mix-cosmopedia-tutorial_1b_cosmopedia": "Source: Cosmopedia",
-          "mix-fw_edu_lq-tutorial_1b_lq": "Source: FineWeb-Edu-LQ",
-          dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
-        }
-      },
       "FAQ Prompt": {
         datasets: {
           "mix-dclm-faq_1b_dclm": "Source: DCLM",
@@ -485,6 +476,15 @@ We rephrase four datasets (DCLM, Cosmopedia, FineWeb-Edu-HQ, FineWeb-Edu-LQ) wit
           "mix-cosmopedia-faq_1b_cosmopedia": "Source: Cosmopedia",
           dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
         }
+      },
+      "Tutorial Prompt": {
+        datasets: {
+          "mix-fw_edu_hq-tutorial_1b_hq": "Source: FineWeb-Edu-HQ",
+          "mix-dclm-tutorial_1b_dclm": "Source: DCLM",
+          "mix-cosmopedia-tutorial_1b_cosmopedia": "Source: Cosmopedia",
+          "mix-fw_edu_lq-tutorial_1b_lq": "Source: FineWeb-Edu-LQ",
+          dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
+        }
       }
     }
   }}
@@ -496,15 +496,6 @@ We rephrase four datasets (DCLM, Cosmopedia, FineWeb-Edu-HQ, FineWeb-Edu-LQ) wit
   desc="Effect of source dataset with FineWeb-Edu-HQ as fixed mix-in. Use the Setup dropdown to compare prompts."
   config={{
     setups: {
-      "Tutorial Prompt": {
-        datasets: {
-          "mix-fw_edu_hq-tutorial_1b_dclm": "Source: DCLM",
-          "mix-fw_edu_hq-tutorial_1b_hq": "Source: FineWeb-Edu-HQ",
-          "mix-fw_edu_hq-tutorial_1b_cosmopedia": "Source: Cosmopedia",
-          "mix-fw_edu_hq-tutorial_1b_lq": "Source: FineWeb-Edu-LQ",
-          dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
-        }
-      },
       "FAQ Prompt": {
         datasets: {
           "mix-fw_edu_hq-faq_1b_dclm": "Source: DCLM",
@@ -513,6 +504,15 @@ We rephrase four datasets (DCLM, Cosmopedia, FineWeb-Edu-HQ, FineWeb-Edu-LQ) wit
           "mix-fw_edu_hq-faq_1b_cosmopedia": "Source: Cosmopedia",
           dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
         }
+      },
+      "Tutorial Prompt": {
+        datasets: {
+          "mix-fw_edu_hq-tutorial_1b_dclm": "Source: DCLM",
+          "mix-fw_edu_hq-tutorial_1b_hq": "Source: FineWeb-Edu-HQ",
+          "mix-fw_edu_hq-tutorial_1b_cosmopedia": "Source: Cosmopedia",
+          "mix-fw_edu_hq-tutorial_1b_lq": "Source: FineWeb-Edu-LQ",
+          dclm: { display: "Baseline (DCLM)", color: "#8b8b8b", baseline: true }
+        }
       }
     }
   }}
@@ -608,7 +608,7 @@ Here are the key takeaways from our experiments:
 - **Q: Which individual prompts from the synthetic baselines match DCLM?**<br/>
   A: Only Diverse QA Pairs and REWIRE's Guided Rewrite.
 - **Q: Can new prompts beat DCLM?**<br/>
-  A: Yes. Math, Table, FAQ, and Tutorial all outperform DCLM. Narrative, Explanation, Article, Commentary, and Discussion do not.
+  A: Yes. FAQ, Math, Table, and Tutorial all outperform DCLM. Article, Commentary, Discussion, Explanation, and Narrative do not.
 - **Q: Does model size matter?**<br/>
   A: Not much. 1B is sufficient for simple prompts, 4B for complex ones.
 - **Q: Do we need better models for low-quality data?**<br/>
@@ -628,4 +628,4 @@ Here are the key takeaways from our experiments:
 - **Q: Do typos in the prompt hurt?**<br/>
   A: No. Typos have no negative effect on downstream performance.
 
-So what actually matters? Prompt design, above all else. Structured formats like Math, Table, FAQ, and Tutorial consistently beat curated baselines. Everything else is surprisingly forgiving. A 1B model handles simple prompts just fine, 4B covers the complex ones, and going bigger buys you nothing. Source data quality barely matters either, as long as you mix in strong original data. That last point is worth emphasizing: low-quality sources with a good mix-in match high-quality sources, which means you can draw from a much larger and more diverse data pool. The recipe we landed on is simple: pick a structured prompt, use the smallest model that handles it, blend with high-quality original data, and pour the saved compute into volume.
+So what actually matters? Prompt design, above all else. Structured formats like FAQ, Math, Table, and Tutorial consistently beat curated baselines. Everything else is surprisingly forgiving. A 1B model handles simple prompts just fine, 4B covers the complex ones, and going bigger buys you nothing. Source data quality barely matters either, as long as you mix in strong original data. That last point is worth emphasizing: low-quality sources with a good mix-in match high-quality sources, which means you can draw from a much larger and more diverse data pool. The recipe we landed on is simple: pick a structured prompt, use the smallest model that handles it, blend with high-quality original data, and pour the saved compute into volume.