Sync from GitHub via hub-sync

README.md

@@ -9,7 +9,8 @@ app_file: streamlit_app.py
 pinned: false
 ---
 
-# Electricity Utility Inefficiency & Residential Rate Analysis
+
+# Electricity Utility Fairness Residential Rate Analysis
 
 > **Research Question:** Are system-level inefficiencies — high energy losses and poor load factors — statistically correlated with higher retail rates for residential consumers?
 
@@ -21,7 +22,8 @@ This project investigates a fundamental fairness question in the U.S. electricit
 
 New York State serves as the primary case study. NY was selected through a data-driven ranking process (get_state_variance) that scores all 50 states across five analytical criteria: number of utilities, number of ownership types, residential price standard deviation, maximum system loss percentage, and industrial revenue dependency. New York ranks at or near the top on all five: it has over 100 utilities across 6 distinct ownership models, exhibits high residential price variance, and sits within one of the most actively scrutinized regulatory environments in the U.S.
 
-[![Open in HF Space](https://huggingface.co/datasets/huggingface/badges/resolve/main/open-in-hf-spaces-sm.svg)](https://huggingface.co/spaces/chalseee/utilities-equity-efficiency-gap)
+[![Open in Streamlit](https://static.streamlit.io/badges/streamlit_badge_black_white.svg)](https://your-app-url.streamlit.app)
+
 ---
 
 ## Key Metrics
@@ -56,14 +58,12 @@ New York State serves as the primary case study. NY was selected through a data-
 ├── requirements.txt
 ├── README.md
 ├── data/
+│   └── electricity.py
 │   └── app.py
 │   └── electricity.data
 │   └── electricity.data
 ├── src/
-│   └── util/
-│       ├── data_util.py
-│       └── plot_util.py
-└── images/            
+│   └── util/         
 ```
 
 The three modules work as a clean pipeline:
@@ -89,7 +89,7 @@ The three modules work as a clean pipeline:
 
 - The dataset is pre-bundled — no external downloads required for it.
 - Run `jupytr notebook utility_efficiency_fairness.ipynb.`
-- Run pip install pandas plotly scipy kaleido streamlit
+- Run `pip install pandas plotly scipy kaleido streamlit`
 
 ---
 

src/util/plot_util.py

@@ -45,7 +45,6 @@ def get_state_variance_table(df: pd.DataFrame) -> go.Figure:
 
     return fig
 
-
 def get_price_spread_strip_plot(df: pd.DataFrame):
     """
     Strip plot of Price Spread by ownership model.

streamlit_app.py

@@ -1,11 +1,14 @@
-import streamlit.src.util.plot_util as plot_util
-import streamlit.src.util.data_util as data_util
+import sys
+import os
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+import src.util.plot_util as plot_util
+import src.util.data_util as data_util
 import data.electricity as electricity
 
 import streamlit as st
 import pandas as pd
 
-
 st.set_page_config(
     page_title="Utility Efficiency & Rates",
     layout="wide"
@@ -118,4 +121,4 @@ utility_flow = data_util.get_utility_usage(utility_row, level="Utility")
 st.plotly_chart(
     plot_util.get_energy_use_sankey_plot(utility_flow),
     use_container_width=True
-)
+)

utility_efficiency_fairness.ipynb

@@ -139,14 +139,147 @@
     "---\n",
     "## 3. Ownership Model Analysis\n",
     "\n",
-    "Before testing the efficiency hypothesis, we establish baseline pricing distributions by ownership model. This matters because ownership type is a structural variable — investor-owned utilities (IOUs) answer to shareholders and profit motives, while cooperatives and municipals serve member-owners or public ratepayers. If ownership type already explains most of the price variance, efficiency metrics may be redundant."
+    "Before testing the efficiency hypothesis, we establish baseline pricing distributions by ownership model. This matters because ownership type is a structural variable — investor-owned utilities (IOUs) answer to shareholders and profit motives, while cooperatives and municipals serve member-owners or public ratepayers. If ownership type already explains most of the price variance, an efficiency metrics may be redundant."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 4,
    "id": "54f2b308",
    "metadata": {},
+   "outputs": [],
+   "source": [
+    "strip_plot = plot_util.get_price_spread_strip_plot(ny_df)\n",
+    "\n",
+    "# strip_plot.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "42f0d824",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## 4. Correlation Analysis\n",
+    "\n",
+    "Before building directional charts, we compute a Pearson correlation matrix across all key metrics. This serves two purposes: it quantifies the *strength and direction* of every pairwise relationship, and it surfaces any unexpected correlations that warrant further investigation. A positive correlation between `SystemLossPercentage` and `ResidentialUnitPrice` would support the hypothesis. A negative correlation between `LoadFactor` and `ResidentialUnitPrice` (higher efficiency → lower price) would also support it."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "6e0480c9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Key metrics: System Loss %, Load Factor', Industrial Revenue %, Price Spread\n",
+    "heatmap = plot_util.get_key_metrics_corr_matrix(ny_df)\n",
+    "\n",
+    "# heatmap.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0003dd2d",
+   "metadata": {},
+   "source": [
+    "**Interpretation:** The heatmap gives us a first look at which efficiency metrics are most predictive of residential pricing. The strong negative correlation between `LoadFactor` and both `ResidentialUnitPrice` and PriceSpread stands out immediately — it suggests infrastructure utilization is a meaningful driver of what residential customers pay. `SystemLossPercentage` shows a weaker positive relationship, which we'll test directly in the next section."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c92e34ee",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## 5. Fairness Audit — Efficiency vs. Residential Price\n",
+    "\n",
+    "This is the central test of the research question, presented as two side-by-side scatter plots sharing a y-axis (residential price). Each plot approaches inefficiency from a different angle:\n",
+    "\n",
+    "| Plot | X-axis | What it tests |\n",
+    "|---|---|---|\n",
+    "| Left | `SystemLossPercentage` | Does wasted energy in the grid cost residential customers more? |\n",
+    "| Right | `LoadFactor` | Does underutilized infrastructure translate to higher per-MWh costs? |\n",
+    "\n",
+    "> **Filtering note:** This chart excludes utilities with zero system loss or zero load factor. These edge cases typically represent pass-through entities (pure resellers) or data reporting anomalies — including them would distort the OLS fit."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "de2b920d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Keep residential utilities that are using energy (instead of ONLY reseale, etc.)\n",
+    "residential_lf_sys_loss_df = data_util.get_residential_sys_loss(ny_df)\n",
+    "residential_lf_sys_loss_df = data_util.get_residential_load_factor(\n",
+    "    residential_lf_sys_loss_df).round(2)\n",
+    "\n",
+    "dual_y_scatter = plot_util.get_fairness_dual_y_scatter_plot(\n",
+    "    residential_lf_sys_loss_df)\n",
+    "\n",
+    "# dual_y_scatter.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ccdd7bc3",
+   "metadata": {},
+   "source": [
+    "**Interpretation:** The results are asymmetric: load factor explains 41.7% of residential price variance (R² = 0.417, p < 0.001), a statistically strong relationship. System loss explains 10.8% (R² = 0.108, p = 0.016) — modest but significant. The core hypothesis holds, but the mechanism differs: it is primarily underutilized infrastructure, not transmission waste, that correlates with higher residential rates in New York's utility landscape."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c69057f2",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## 6. Rate Disparity — Residential vs. Industrial\n",
+    "\n",
+    "Even if inefficiency drives prices up overall, the burden may not fall equally. This chart examines the **top 10 utilities by Price Spread** — the gap between what residential and industrial customers pay per MWh. Industrial customers typically negotiate volume discounts — some spread is expected. But when residential customers pay 2–3× the industrial rate at the same utility, it raises questions about whether the rate structure reflects true cost-of-service differences or something else. This chart identifies the specific utilities where that premium is most extreme."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "38d3d584",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Keep utilities that offer both industrial and residential services\n",
+    "res_ind_customers_df = data_util.get_customer_utilities(ny_df).round(2)\n",
+    "\n",
+    "dumbbell = plot_util.get_rate_disparity_dumbbell_plot(res_ind_customers_df)\n",
+    "\n",
+    "# dumbbell.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "86fe4d1f",
+   "metadata": {},
+   "source": [
+    "**Interpretation:** Con Edison (Consolidated Edison) shows the largest residential–industrial spread in the dataset, charging residential customers roughly $233/MWh against a near-zero industrial rate. Eight of the top ten utilities by spread are investor-owned — consistent with the box plot finding that IOUs exhibit the widest pricing distributions. Notably, Con Edison also carries a high system loss percentage, placing it in the upper-right quadrant of the fairness audit scatter: both inefficient and inequitable in its rate structure."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8e8ae178",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## 7. Energy Flow Analysis\n",
+    "\n",
+    "The Sankey diagrams provide operational context for the efficiency metrics computed above. Rather than a single number, they show the *full picture* of where a utility's energy comes from and where it goes. A wide `Losses` band at a utility with high residential prices is the inefficiency–cost story in one image. We can compare the utility-level Sankey against the U.S. national average to understand whether NY's profile is typical or anomalous."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "0ae862c6",
+   "metadata": {},
    "outputs": [
     {
      "data": {
@@ -156,417 +289,77 @@
        },
        "data": [
         {
-         "alignmentgroup": "True",
-         "boxpoints": "all",
-         "fillcolor": "rgba(255,255,255,0)",
-         "hoveron": "points",
-         "hovertemplate": "<b>%{hovertext}</b><br><br>Type=%{x}<br>Residential Premium ($/MWh)=%{y}<extra></extra>",
-         "hovertext": [
-          "Agway Energy Services, LLC",
-          "Energy Coop of New York, Inc",
-          "Robison Energy, LLC",
-          "U.S. Energy Partners LLC",
-          "NOCO Electric",
-          "Columbia Utilities Power, LLC",
-          "Hudson Energy Services",
-          "Palmco Power NJ, LLC",
-          "Major Energy Electric Services",
-          "Palmco Power PA, LLC",
-          "Palmco Power MD, LLC",
-          "Plymouth Rock Energy, LLC",
-          "Palmco Power OH, LLC",
-          "Stream Energy New York, LLC",
-          "EnergyMark, LLC",
-          "Palmco Power IL, LLC",
-          "Respond Power LLC",
-          "Kiwi Energy NY LLC",
-          "SmartEnergy Holdings, LLC",
-          "Alpha Gas and Electric, LLC",
-          "Energy.Me Midwest LLC",
-          "Marathon Power LLC",
-          "Harborside Energy, LLC",
-          "Abest Power & Gas, LLC",
-          "MPower Energy LLC",
-          "Palmco Power MA, LLC",
-          "Agera Energy LLC",
-          "Greenlight Energy Inc.",
-          "New Wave Energy Corporation",
-          "HIKO Energy, LLC",
-          "Aequitas Energy Inc",
-          "Great Eastern Energy",
-          "East Coast Power & Gas, LLC",
-          "Spring Energy RRH LLC dba Spring Power &",
-          "MPower Energy NJ LLC",
-          "Atlantic Energy LLC",
-          "Palmco Power DC, LLC",
-          "Palmco Power DE, LLC",
-          "South Bay Energy Corp.",
-          "Flanders Energy, LLC",
-          "Pure Energy USA, LLC",
-          "First Choice Energy LLC",
-          "Agressive Energy LLC"
-         ],
-         "legendgroup": "Retail Power Marketer",
-         "line": {
-          "color": "rgba(255,255,255,0)"
-         },
-         "marker": {
-          "color": "rgb(95, 70, 144)"
-         },
-         "name": "Retail Power Marketer",
-         "offsetgroup": "Retail Power Marketer",
-         "orientation": "v",
-         "pointpos": 0,
-         "showlegend": true,
-         "type": "box",
-         "x": [
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer",
-          "Retail Power Marketer"
-         ],
-         "x0": " ",
-         "xaxis": "x",
-         "y": {
-          "bdata": "1HdJqZaIVEDlcPM237NFQJ4t+TeSfVVA9Y6hQZjxREC7n0AeSF5FQAJayxan5mdA+NB2EXoPWEDkHoKBHplrQAf1wZiCkFxAdfEDDupzYUBcKbC7K7hjQNg9bETnNGBAwv1PMJqEXEBzGFLAfrxQQMb9Fc3b/0VA4ydnH2Q/VkBOiMxbaUtiQE1aVCBa8FtAdyZPlzdMWUD/BbayJZdaQNFZ0CR2H1NAVaW6iXyDUkAWCcnsvT5kQL6sGTMR6VRAOFE8FqijYEDwxWACoKxkQLReHZWsKFhANJ/x5rw1XUAUtkGpzFhMQEG4G/seUWFA7ZpZKnJNXEC+qlbjMDJZQMjcovLsXVRAJYv3tlpiWkAv2u+X74BeQM7YHwYLZ2BAFkOHeY7iW0DXGD4bRKpZQNjxRK4pNV9AnhCBas4qZEATgY2Ufz1jQNvxsf19yVhAbl5aGi50UUA=",
-          "dtype": "f8"
-         },
-         "y0": " ",
-         "yaxis": "y"
-        },
-        {
-         "alignmentgroup": "True",
-         "boxpoints": "all",
-         "fillcolor": "rgba(255,255,255,0)",
-         "hoveron": "points",
-         "hovertemplate": "<b>%{hovertext}</b><br><br>Type=%{x}<br>Residential Premium ($/MWh)=%{y}<extra></extra>",
-         "hovertext": [
-          "Village of Arcade - (NY)",
-          "Village of Akron - (NY)",
-          "Village of Andover - (NY)",
-          "Village of Angelica - (NY)",
-          "Bath Electric Gas & Water",
-          "Village of Bergen - (NY)",
-          "Village of Boonville - (NY)",
-          "Village of Brocton - (NY)",
-          "Village of Castile - (NY)",
-          "Village of Churchville - (NY)",
-          "Village of Endicott - (NY)",
-          "Village of Fairport - (NY)",
-          "Village of Frankfort - (NY)",
-          "Village of Freeport - (NY)",
-          "Village of Green Island - (NY)",
-          "Village of Greene - (NY)",
-          "Village of Groton - (NY)",
-          "Village of Hamilton - (NY)",
-          "Village of Holley - (NY)",
-          "Village of Ilion - (NY)",
-          "Jamestown Board of Public Util",
-          "Lake Placid Village, Inc - (NY)",
-          "Village of Little Valley - (NY)",
-          "Village of Marathon - (NY)",
-          "Town of Massena - (NY)",
-          "Village of Mayville - (NY)",
-          "Mohawk Municipal Comm",
-          "Village of Penn Yan - (NY)",
-          "Village of Philadelphia - (NY)",
-          "City of Plattsburgh - (NY)",
-          "Village of Richmondville - (NY)",
-          "Village of Rockville Centre - (NY)",
-          "Village of Rouses Point - (NY)",
-          "City of Salamanca - (NY)",
-          "Village of Sherburne - (NY)",
-          "City of Sherrill - (NY)",
-          "Village of Silver Springs - (NY)",
-          "Village of Skaneateles - (NY)",
-          "Village of Solvay - (NY)",
-          "Village of Spencerport - (NY)",
-          "Village of Springville - (NY)",
-          "Village of Theresa - (NY)",
-          "Village of Tupper Lake - (NY)",
-          "Village of Watkins Glen - (NY)",
-          "Village of Wellsville - (NY)",
-          "Village of Westfield - (NY)"
-         ],
-         "legendgroup": "Municipal",
-         "line": {
-          "color": "rgba(255,255,255,0)"
-         },
-         "marker": {
-          "color": "rgb(29, 105, 150)"
-         },
-         "name": "Municipal",
-         "offsetgroup": "Municipal",
-         "orientation": "v",
-         "pointpos": 0,
-         "showlegend": true,
-         "type": "box",
-         "x": [
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal",
-          "Municipal"
-         ],
-         "x0": " ",
-         "xaxis": "x",
-         "y": {
-          "bdata": "4zx0l/glSkCrn6PKMxZKQDFIRJo2x0VAGOWbWs1/T0CCUB+yIaJNQEZpzEOSWVNAykfneWTjSUC5RSoUu+tGQD5k23fSFk1Azmj3sPXNUkDJCaumGFlQQBb2vzXcvktA7+AqSao3Q0CQqdv9nMlfQC8zhjbsPltAXfLkhENnS0DUHfrfiDZKQJdz0sw7pkhAl5Nt4wsgTkCCI8bcBMRKQNIB9qAznlRAw5X6vW53S0At0Q+nexBTQCjVNgwQnVFASdVlz85LTkAsdSyDtF5QQNhzo/V/JkVArLuSbKG6QkCXOXo4ck9OQMfHf05/EUdAK8FbUIDWUkDSX/M3ctteQF6gAPx8Ej1Aw45Qct0fSUC0jal315FLQNRIT1cjHU1AjVKkDWvMUUDTH1W7sehEQAeqFkKflkxAe68yr+vZRkBpZcd2fvFKQMx1qVre1FNApaDbJmbZSkCWccuMD2tCQNSJR5YBWU5ARDUJMx5yTEA=",
-          "dtype": "f8"
-         },
-         "y0": " ",
-         "yaxis": "y"
-        },
-        {
-         "alignmentgroup": "True",
-         "boxpoints": "all",
-         "fillcolor": "rgba(255,255,255,0)",
-         "hoveron": "points",
-         "hovertemplate": "<b>%{hovertext}</b><br><br>Type=%{x}<br>Residential Premium ($/MWh)=%{y}<extra></extra>",
-         "hovertext": [
-          "Central Hudson Gas & Elec Corp",
-          "Consolidated Edison Co-NY Inc",
-          "Fishers Island Utility Co Inc",
-          "Massachusetts Electric Co",
-          "New York State Elec & Gas Corp",
-          "Niagara Mohawk Power Corp.",
-          "Orange & Rockland Utils Inc",
-          "Pike County Light & Power Co",
-          "Rochester Gas & Electric Corp",
-          "Rockland Electric Co"
-         ],
-         "legendgroup": "Investor Owned",
-         "line": {
-          "color": "rgba(255,255,255,0)"
-         },
-         "marker": {
-          "color": "rgb(56, 166, 165)"
-         },
-         "name": "Investor Owned",
-         "offsetgroup": "Investor Owned",
-         "orientation": "v",
-         "pointpos": 0,
-         "showlegend": true,
-         "type": "box",
-         "x": [
-          "Investor Owned",
-          "Investor Owned",
-          "Investor Owned",
-          "Investor Owned",
-          "Investor Owned",
-          "Investor Owned",
-          "Investor Owned",
-          "Investor Owned",
-          "Investor Owned",
-          "Investor Owned"
-         ],
-         "x0": " ",
-         "xaxis": "x",
-         "y": {
-          "bdata": "DlfHeglgZEAky8QEPyJtQOHh4eHh4XNAiNVwftXYZkBAg+xCRm5aQLlstvUwUV1AP0jm2o/iZEC7M68wZX1eQCRaHECiYF5AzRC7X0JqY0A=",
-          "dtype": "f8"
-         },
-         "y0": " ",
-         "yaxis": "y"
-        },
-        {
-         "alignmentgroup": "True",
-         "boxpoints": "all",
-         "fillcolor": "rgba(255,255,255,0)",
-         "hoveron": "points",
-         "hovertemplate": "<b>%{hovertext}</b><br><br>Type=%{x}<br>Residential Premium ($/MWh)=%{y}<extra></extra>",
-         "hovertext": [
-          "Delaware County Elec Coop Inc",
-          "North Shore Towers Apts Inc",
-          "Otsego Electric Coop, Inc",
-          "Steuben Rural Elec Coop, Inc"
-         ],
-         "legendgroup": "Cooperative",
-         "line": {
-          "color": "rgba(255,255,255,0)"
-         },
-         "marker": {
-          "color": "rgb(15, 133, 84)"
-         },
-         "name": "Cooperative",
-         "offsetgroup": "Cooperative",
-         "orientation": "v",
-         "pointpos": 0,
-         "showlegend": true,
-         "type": "box",
-         "x": [
-          "Cooperative",
-          "Cooperative",
-          "Cooperative",
-          "Cooperative"
-         ],
-         "x0": " ",
-         "xaxis": "x",
-         "y": {
-          "bdata": "+aWVpF9ZYUDMuXx9SDRxQE3e4teNEl9Aw0BzNP3fXkA=",
-          "dtype": "f8"
-         },
-         "y0": " ",
-         "yaxis": "y"
-        },
-        {
-         "alignmentgroup": "True",
-         "boxpoints": "all",
-         "fillcolor": "rgba(255,255,255,0)",
-         "hoveron": "points",
-         "hovertemplate": "<b>%{hovertext}</b><br><br>Type=%{x}<br>Residential Premium ($/MWh)=%{y}<extra></extra>",
-         "hovertext": [
-          "Long Island Power Authority"
-         ],
-         "legendgroup": "State",
-         "line": {
-          "color": "rgba(255,255,255,0)"
-         },
-         "marker": {
-          "color": "rgb(115, 175, 72)"
-         },
-         "name": "State",
-         "offsetgroup": "State",
-         "orientation": "v",
-         "pointpos": 0,
-         "showlegend": true,
-         "type": "box",
-         "x": [
-          "State"
-         ],
-         "x0": " ",
-         "xaxis": "x",
-         "y": {
-          "bdata": "WOYOiWF7aUA=",
-          "dtype": "f8"
-         },
-         "y0": " ",
-         "yaxis": "y"
-        },
-        {
-         "alignmentgroup": "True",
-         "boxpoints": "all",
-         "fillcolor": "rgba(255,255,255,0)",
-         "hoveron": "points",
-         "hovertemplate": "<b>%{hovertext}</b><br><br>Type=%{x}<br>Residential Premium ($/MWh)=%{y}<extra></extra>",
-         "hovertext": [
-          "Greenbacker Renewable Energy Corporation"
-         ],
-         "legendgroup": "Behind the Meter",
-         "line": {
-          "color": "rgba(255,255,255,0)"
-         },
-         "marker": {
-          "color": "rgb(237, 173, 8)"
+         "link": {
+          "source": [
+           0,
+           1,
+           2,
+           3,
+           3,
+           3,
+           3,
+           3
+          ],
+          "target": [
+           3,
+           3,
+           3,
+           4,
+           5,
+           6,
+           7,
+           8
+          ],
+          "value": [
+           38.241995847617716,
+           61.36801867501161,
+           0.3899854773706723,
+           64.90663809625768,
+           31.667357390277083,
+           3.200301490366235,
+           0.17504657881402372,
+           0.05065630657952972
+          ]
          },
-         "name": "Behind the Meter",
-         "offsetgroup": "Behind the Meter",
-         "orientation": "v",
-         "pointpos": 0,
-         "showlegend": true,
-         "type": "box",
-         "x": [
-          "Behind the Meter"
-         ],
-         "x0": " ",
-         "xaxis": "x",
-         "y": {
-          "bdata": "tp977y/kYkA=",
-          "dtype": "f8"
+         "node": {
+          "color": [
+           "rgb(95, 70, 144)",
+           "rgb(29, 105, 150)",
+           "rgb(56, 166, 165)",
+           "rgb(15, 133, 84)",
+           "rgb(115, 175, 72)",
+           "rgb(237, 173, 8)",
+           "rgb(225, 124, 5)",
+           "rgb(204, 80, 62)",
+           "rgb(148, 52, 110)",
+           "rgb(111, 64, 112)",
+           "rgb(102, 102, 102)"
+          ],
+          "label": [
+           "Generated",
+           "Purchased",
+           "Other",
+           "Uses",
+           "Retail Sales",
+           "Resale",
+           "Losses",
+           "Consumed",
+           "No Charge"
+          ],
+          "line": {
+           "color": "black",
+           "width": 0.5
+          },
+          "pad": 15,
+          "thickness": 20
          },
-         "y0": " ",
-         "yaxis": "y"
+         "type": "sankey",
+         "valueformat": ".1f",
+         "valuesuffix": "%"
         }
        ],
        "layout": {
-        "boxmode": "overlay",
-        "legend": {
-         "title": {
-          "text": "Type"
-         },
-         "tracegroupgap": 0
-        },
-        "showlegend": false,
+        "hovermode": "x",
         "template": {
          "data": {
           "bar": [
@@ -579,7 +372,7 @@
             },
             "marker": {
              "line": {
-              "color": "white",
+              "color": "#E5ECF6",
               "width": 0.5
              },
              "pattern": {
@@ -595,7 +388,7 @@
            {
             "marker": {
              "line": {
-              "color": "white",
+              "color": "#E5ECF6",
               "width": 0.5
              },
              "pattern": {
@@ -611,16 +404,16 @@
            {
             "aaxis": {
              "endlinecolor": "#2a3f5f",
-             "gridcolor": "#C8D4E3",
-             "linecolor": "#C8D4E3",
-             "minorgridcolor": "#C8D4E3",
+             "gridcolor": "white",
+             "linecolor": "white",
+             "minorgridcolor": "white",
              "startlinecolor": "#2a3f5f"
             },
             "baxis": {
              "endlinecolor": "#2a3f5f",
-             "gridcolor": "#C8D4E3",
-             "linecolor": "#C8D4E3",
-             "minorgridcolor": "#C8D4E3",
+             "gridcolor": "white",
+             "linecolor": "white",
+             "minorgridcolor": "white",
              "startlinecolor": "#2a3f5f"
             },
             "type": "carpet"
@@ -1237,10 +1030,10 @@
           "geo": {
            "bgcolor": "white",
            "lakecolor": "white",
-           "landcolor": "white",
+           "landcolor": "#E5ECF6",
            "showlakes": true,
            "showland": true,
-           "subunitcolor": "#C8D4E3"
+           "subunitcolor": "white"
           },
           "hoverlabel": {
            "align": "left"
@@ -1250,47 +1043,47 @@
            "style": "light"
           },
           "paper_bgcolor": "white",
-          "plot_bgcolor": "white",
+          "plot_bgcolor": "#E5ECF6",
           "polar": {
            "angularaxis": {
-            "gridcolor": "#EBF0F8",
-            "linecolor": "#EBF0F8",
+            "gridcolor": "white",
+            "linecolor": "white",
             "ticks": ""
            },
-           "bgcolor": "white",
+           "bgcolor": "#E5ECF6",
            "radialaxis": {
-            "gridcolor": "#EBF0F8",
-            "linecolor": "#EBF0F8",
+            "gridcolor": "white",
+            "linecolor": "white",
             "ticks": ""
            }
           },
           "scene": {
            "xaxis": {
-            "backgroundcolor": "white",
-            "gridcolor": "#DFE8F3",
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
             "gridwidth": 2,
-            "linecolor": "#EBF0F8",
+            "linecolor": "white",
             "showbackground": true,
             "ticks": "",
-            "zerolinecolor": "#EBF0F8"
+            "zerolinecolor": "white"
            },
            "yaxis": {
-            "backgroundcolor": "white",
-            "gridcolor": "#DFE8F3",
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
             "gridwidth": 2,
-            "linecolor": "#EBF0F8",
+            "linecolor": "white",
             "showbackground": true,
             "ticks": "",
-            "zerolinecolor": "#EBF0F8"
+            "zerolinecolor": "white"
            },
            "zaxis": {
-            "backgroundcolor": "white",
-            "gridcolor": "#DFE8F3",
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
             "gridwidth": 2,
-            "linecolor": "#EBF0F8",
+            "linecolor": "white",
             "showbackground": true,
             "ticks": "",
-            "zerolinecolor": "#EBF0F8"
+            "zerolinecolor": "white"
            }
           },
           "shapedefaults": {
@@ -1300,19 +1093,19 @@
           },
           "ternary": {
            "aaxis": {
-            "gridcolor": "#DFE8F3",
-            "linecolor": "#A2B1C6",
+            "gridcolor": "white",
+            "linecolor": "white",
             "ticks": ""
            },
            "baxis": {
-            "gridcolor": "#DFE8F3",
-            "linecolor": "#A2B1C6",
+            "gridcolor": "white",
+            "linecolor": "white",
             "ticks": ""
            },
-           "bgcolor": "white",
+           "bgcolor": "#E5ECF6",
            "caxis": {
-            "gridcolor": "#DFE8F3",
-            "linecolor": "#A2B1C6",
+            "gridcolor": "white",
+            "linecolor": "white",
             "ticks": ""
            }
           },
@@ -1321,59 +1114,30 @@
           },
           "xaxis": {
            "automargin": true,
-           "gridcolor": "#EBF0F8",
-           "linecolor": "#EBF0F8",
+           "gridcolor": "white",
+           "linecolor": "white",
            "ticks": "",
            "title": {
             "standoff": 15
            },
-           "zerolinecolor": "#EBF0F8",
+           "zerolinecolor": "white",
            "zerolinewidth": 2
           },
           "yaxis": {
            "automargin": true,
-           "gridcolor": "#EBF0F8",
-           "linecolor": "#EBF0F8",
+           "gridcolor": "white",
+           "linecolor": "white",
            "ticks": "",
            "title": {
             "standoff": 15
            },
-           "zerolinecolor": "#EBF0F8",
+           "zerolinecolor": "white",
            "zerolinewidth": 2
           }
          }
         },
         "title": {
-         "text": "<b>Rate Equity by Ownership Model:</b> Residential Premium Over Industrial Rates"
-        },
-        "xaxis": {
-         "anchor": "y",
-         "categoryarray": [
-          "Retail Power Marketer",
-          "Municipal",
-          "Investor Owned",
-          "Cooperative",
-          "State",
-          "Behind the Meter"
-         ],
-         "categoryorder": "array",
-         "domain": [
-          0,
-          1
-         ],
-         "title": {
-          "text": "Type"
-         }
-        },
-        "yaxis": {
-         "anchor": "x",
-         "domain": [
-          0,
-          1
-         ],
-         "title": {
-          "text": "Residential Premium ($/MWh)"
-         }
+         "text": "<b>Energy Flow: </b>United States"
         }
        }
       }
@@ -1382,147 +1146,6 @@
      "output_type": "display_data"
     }
    ],
-   "source": [
-    "strip_plot = plot_util.get_price_spread_strip_plot(ny_df)\n",
-    "\n",
-    "strip_plot.show()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "99326360",
-   "metadata": {},
-   "source": [
-    "**Interpretation:** Investor-owned utilities dominate rates on the high end, but the spread within each ownership category is wide enough that ownership type alone doesn't fully explain the disparity. There's meaningful variation within ownership types worth investigating."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "42f0d824",
-   "metadata": {},
-   "source": [
-    "---\n",
-    "## 4. Correlation Analysis\n",
-    "\n",
-    "Before building directional charts, we compute a Pearson correlation matrix across all key metrics. This serves two purposes: it quantifies the *strength and direction* of every pairwise relationship, and it surfaces any unexpected correlations that warrant further investigation. A positive correlation between `SystemLossPercentage` and `ResidentialUnitPrice` would support the hypothesis. A negative correlation between `LoadFactor` and `ResidentialUnitPrice` (higher efficiency → lower price) would also support it."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "6e0480c9",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Key metrics: System Loss %, Load Factor', Industrial Revenue %, Price Spread\n",
-    "heatmap = plot_util.get_key_metrics_corr_matrix(ny_df)\n",
-    "\n",
-    "# heatmap.show()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0003dd2d",
-   "metadata": {},
-   "source": [
-    "**Interpretation:** The heatmap gives us a first look at which efficiency metrics are most predictive of residential pricing. The strong negative correlation between `LoadFactor` and both `ResidentialUnitPrice` and PriceSpread stands out immediately — it suggests infrastructure utilization is a meaningful driver of what residential customers pay. `SystemLossPercentage` shows a weaker positive relationship, which we'll test directly in the next section."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c92e34ee",
-   "metadata": {},
-   "source": [
-    "---\n",
-    "## 5. Fairness Audit — Efficiency vs. Residential Price\n",
-    "\n",
-    "This is the central test of the research question, presented as two side-by-side scatter plots sharing a y-axis (residential price). Each plot approaches inefficiency from a different angle:\n",
-    "\n",
-    "| Plot | X-axis | What it tests |\n",
-    "|---|---|---|\n",
-    "| Left | `SystemLossPercentage` | Does wasted energy in the grid cost residential customers more? |\n",
-    "| Right | `LoadFactor` | Does underutilized infrastructure translate to higher per-MWh costs? |\n",
-    "\n",
-    "> **Filtering note:** This chart excludes utilities with zero system loss or zero load factor. These edge cases typically represent pass-through entities (pure resellers) or data reporting anomalies — including them would distort the OLS fit."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "id": "de2b920d",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Keep residential utilities that are using energy (instead of ONLY reseale, etc.)\n",
-    "residential_lf_sys_loss_df = data_util.get_residential_sys_loss(ny_df)\n",
-    "residential_lf_sys_loss_df = data_util.get_residential_load_factor(\n",
-    "    residential_lf_sys_loss_df).round(2)\n",
-    "\n",
-    "dual_y_scatter = plot_util.get_fairness_dual_y_scatter_plot(\n",
-    "    residential_lf_sys_loss_df)\n",
-    "\n",
-    "# dual_y_scatter.show()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ccdd7bc3",
-   "metadata": {},
-   "source": [
-    "**Interpretation:** The results are asymmetric: load factor explains 41.7% of residential price variance (R² = 0.417, p < 0.001), a statistically strong relationship. System loss explains 10.8% (R² = 0.108, p = 0.016) — modest but significant. The core hypothesis holds, but the mechanism differs: it is primarily underutilized infrastructure, not transmission waste, that correlates with higher residential rates in New York's utility landscape."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c69057f2",
-   "metadata": {},
-   "source": [
-    "---\n",
-    "## 6. Rate Disparity — Residential vs. Industrial\n",
-    "\n",
-    "Even if inefficiency drives prices up overall, the burden may not fall equally. This chart examines the **top 10 utilities by Price Spread** — the gap between what residential and industrial customers pay per MWh. Industrial customers typically negotiate volume discounts — some spread is expected. But when residential customers pay 2–3× the industrial rate at the same utility, it raises questions about whether the rate structure reflects true cost-of-service differences or something else. This chart identifies the specific utilities where that premium is most extreme."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "38d3d584",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Keep utilities that offer both industrial and residential services\n",
-    "res_ind_customers_df = data_util.get_customer_utilities(ny_df).round(2)\n",
-    "\n",
-    "dumbbell = plot_util.get_rate_disparity_dumbbell_plot(res_ind_customers_df)\n",
-    "\n",
-    "# dumbbell.show()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "86fe4d1f",
-   "metadata": {},
-   "source": [
-    "**Interpretation:** Con Edison (Consolidated Edison) shows the largest residential–industrial spread in the dataset, charging residential customers roughly $233/MWh against a near-zero industrial rate. Eight of the top ten utilities by spread are investor-owned — consistent with the box plot finding that IOUs exhibit the widest pricing distributions. Notably, Con Edison also carries a high system loss percentage, placing it in the upper-right quadrant of the fairness audit scatter: both inefficient and inequitable in its rate structure."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8e8ae178",
-   "metadata": {},
-   "source": [
-    "---\n",
-    "## 7. Energy Flow Analysis\n",
-    "\n",
-    "The Sankey diagrams provide operational context for the efficiency metrics computed above. Rather than a single number, they show the *full picture* of where a utility's energy comes from and where it goes. A wide `Losses` band at a utility with high residential prices is the inefficiency–cost story in one image. We can compare the utility-level Sankey against the U.S. national average to understand whether NY's profile is typical or anomalous."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "id": "0ae862c6",
-   "metadata": {},
-   "outputs": [],
    "source": [
     "# Look at a specific utility's energy usage/flow\n",
     "utility_usage = data_util.get_utility_usage(ny_df.sum(), level=\"State\")\n",
@@ -1536,7 +1159,7 @@
     "\n",
     "us_sankey = plot_util.get_energy_use_sankey_plot(us_energy_flow)\n",
     "\n",
-    "# us_sankey.show()"
+    "us_sankey.show()"
    ]
   },
   {
@@ -1563,9 +1186,8 @@
    "outputs": [],
    "source": [
     "# Sankey chart with a drop down energy flow for each individual utility\n",
-    "utility_sankey_with_dropdown = plot_util.add_utility_dropdown(ny_sankey, df=ny_df)\n",
     "\n",
-    "# utility_sankey_with_dropdown.show()"
+    "# plot_util.add_utility_dropdown(ny_sankey, df=ny_df)"
    ]
   },
   {
@@ -1586,15 +1208,36 @@
    "execution_count": 10,
    "id": "cb116404",
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Defaulting to user installation because normal site-packages is not writeable\n",
+      "Requirement already satisfied: kaleido in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (1.2.0)\n",
+      "Requirement already satisfied: choreographer>=1.1.1 in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (from kaleido) (1.2.1)\n",
+      "Requirement already satisfied: logistro>=1.0.8 in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (from kaleido) (2.0.1)\n",
+      "Requirement already satisfied: orjson>=3.10.15 in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (from kaleido) (3.11.8)\n",
+      "Requirement already satisfied: packaging in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (from kaleido) (26.0)\n",
+      "Requirement already satisfied: pytest-timeout>=2.4.0 in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (from kaleido) (2.4.0)\n",
+      "Requirement already satisfied: simplejson>=3.19.3 in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (from choreographer>=1.1.1->kaleido) (4.0.1)\n",
+      "Requirement already satisfied: pytest>=7.0.0 in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (from pytest-timeout>=2.4.0->kaleido) (9.0.3)\n",
+      "Requirement already satisfied: colorama>=0.4 in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (from pytest>=7.0.0->pytest-timeout>=2.4.0->kaleido) (0.4.6)\n",
+      "Requirement already satisfied: iniconfig>=1.0.1 in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (from pytest>=7.0.0->pytest-timeout>=2.4.0->kaleido) (2.3.0)\n",
+      "Requirement already satisfied: pluggy<2,>=1.5 in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (from pytest>=7.0.0->pytest-timeout>=2.4.0->kaleido) (1.6.0)\n",
+      "Requirement already satisfied: pygments>=2.7.2 in C:\\Users\\chals\\AppData\\Roaming\\Python\\Python313\\site-packages (from pytest>=7.0.0->pytest-timeout>=2.4.0->kaleido) (2.19.2)\n",
+      "Note: you may need to restart the kernel to use updated packages.\n"
+     ]
+    }
+   ],
    "source": [
-    "# %pip install kaleido\n",
+    "%pip install kaleido\n",
     "\n",
-    "# # Gather all plots and export them as SVGs\n",
-    "# plots = [top_variance_table, strip_plot, heatmap,\n",
-    "#          dual_y_scatter, dumbbell, ny_sankey, us_sankey]\n",
+    "# Gather all plots and export them as SVGs\n",
+    "plots = [top_variance_table, strip_plot, heatmap,\n",
+    "         dual_y_scatter, dumbbell, ny_sankey, us_sankey]\n",
     "\n",
-    "# plot_util.export_plots_as_svg(plots)"
+    "plot_util.export_plots_as_svg(plots)"
    ]
   }
  ],