docs/performance/rendering-optimization-plan.md

Rendering Optimizations for Large Grid NAD Visualization

Overview

Co-Study4Grid renders pypowsybl Network Area Diagrams (NAD) for power grids with 11,000+ lines and 500+ voltage levels. At this scale, naive rendering causes multi-second tab switches, invisible line colors at zoom-out, and zoom lag/crashes. This document traces the critical rendering features, their rationale, and regression risks.

Critical CSS Properties

1. vector-effect: non-scaling-stroke — Line Visibility & Zoom Performance

Files: frontend/src/App.css, standalone_interface.html (CSS section)

.svg-container svg path,
.svg-container svg line,
.svg-container svg polyline,
.svg-container svg rect {
    vector-effect: non-scaling-stroke;
}

What it does: Keeps stroke widths at a constant screen-pixel size regardless of SVG viewBox zoom level.

Why it's critical:

• Without it at full zoom-out: Lines become sub-pixel width on large grids — native pypowsybl colors are invisible (the diagram appears as scattered dots)
• Without it when zoomed in: Strokes scale to hundreds of screen pixels, causing extremely expensive anti-aliased rendering → zoom lag and browser crashes
• With it: Strokes stay at ~1-2px screen width at any zoom level. Native pypowsybl line colors are always visible. Rendering cost is constant regardless of zoom.

Regression history: Removed in commit 6d03b24 ("Fix thick lines"), causing lines to lose visible colors and zoom to lag/crash. Restored in df20d54.

DO NOT REMOVE this CSS rule. It is the single most impactful rendering property for large grids. pypowsybl SVGs include native colors on paths — this rule ensures they remain visible. If lines appear "too thick" at some zoom level, address it by adjusting individual stroke-width values, not by removing non-scaling-stroke.

2. contain: layout style paint — CSS Containment

Files: frontend/src/App.css, standalone_interface.html

.svg-container {
    contain: layout style paint;
}

What it does: Tells the browser that layout/paint within .svg-container is independent of the rest of the page.

Why it's critical: During viewBox changes (zoom/pan) and tab switches, the browser would otherwise propagate style/layout recalculations to ancestor elements. Containment limits the scope of recalculation to the SVG subtree.

3. text-hidden Class — Text Culling on Large Grids

Files: frontend/src/App.css, standalone_interface.html

.svg-container.text-hidden foreignObject,
.svg-container.text-hidden .nad-edge-infos,
.svg-container.text-hidden .nad-text-edges {
    display: none !important;
}

What it does: Hides thousands of text labels (foreignObject, edge info) when zoomed out on large grids. Text is too small to read at full zoom-out, and rendering it is expensive.

When it activates: Controlled by usePanZoom hook — text is hidden when the viewBox covers ≥55% of the original diagram size, shown when zoomed in to ≤45% (hysteresis prevents flicker near the boundary).

Pan/Zoom Architecture (usePanZoom)

Files: frontend/src/hooks/usePanZoom.ts, standalone_interface.html (usePanZoom function)

Design Principles

1. Direct DOM manipulation during interaction — viewBox changes go directly to the SVG element via setAttribute, bypassing React's render cycle entirely. React state is only updated when interaction ends (debounced).

2. Cached SVG element reference — svgElRef.current is set once when the diagram loads (useLayoutEffect([initialViewBox])), avoiding repeated querySelector('svg') calls during the hot path (wheel/drag events).

3. Debounced React state sync — commitViewBox() fires 150ms after the last wheel event, preventing React re-renders during rapid zoom.

4. rAF-throttled drag — Mouse move events are batched to at most one DOM update per display frame via requestAnimationFrame.

5. Interaction-time paint culling — while .svg-interacting is set (the gesture window), App.css drops the two most expensive paint element classes from the render tree: the HTML <foreignObject> voltage-level labels and the .nad-edge-infos flow values/arrows. They repaint once on settle. ~1.5–1.7x cheaper frames on the 5247-VL European grid, GPU-independent — this is the default. A CSS-transform GPU-compositing path (much smoother on GPU, but a regression under software/VDI rendering where every pan frame re-rasters new layer tiles) ships as the off-by-default "Smooth pan/zoom (GPU)" toggle (utils/smoothPanZoom.ts, read by usePanZoom at gesture start). Full write-up + benchmark: history/interaction-paint-culling.md.

Critical useLayoutEffect Hooks

┌─ useLayoutEffect([initialViewBox])
│  Cache svgElRef, apply text-hidden on large grids.
│  MUST have [initialViewBox] deps — without deps it runs every render,
│  blocking paint on every tab switch.
│
├─ useLayoutEffect([active])
│  When tab becomes active, apply current viewBox to SVG DOM BEFORE paint.
│  Prevents one frame of stale/default viewBox on tab switch.
│
└─ useLayoutEffect([activeTab]) — in App.tsx / standalone
   Tab synchronization: copies viewBox from previous tab to new tab
   before the browser paints, so the new tab shows the same zoom region.

Regression risk: Changing any of these to useEffect will cause visible flicker on tab switch (one frame of wrong zoom state). Removing the [initialViewBox] dependency will cause all three usePanZoom instances to run querySelector on every React render, blocking paint for ~100-300ms on large grids.

Tab-Switch Optimization

Problem

On a France-scale grid (11,225 lines, ~500+ voltage levels), switching between N / N-1 / Action tabs was taking 1-3 seconds. The tab wouldn't appear until all decorations (highlights, voltage filter, delta visuals) finished running.

Solution: Deferred Decorations

Highlights and voltage filters are deferred to the next animation frame on tab switch:

User clicks tab → React render → useLayoutEffect (viewBox sync)
→ Browser paints tab (SVG visible immediately)
→ requestAnimationFrame → apply highlights + voltage filter

Implementation:

• The highlight effect detects tab switches via prevActiveTabRef
• On tab switch: decorations are deferred via requestAnimationFrame
• On data change (same tab): decorations apply synchronously

Stale tracking: Inactive tabs are marked as "stale" in a Set. When switching to a stale tab, decorations re-apply in the deferred rAF callback.

SVG Container Strategy

All three diagram containers (N, N-1, Action) stay mounted in the DOM with visibility: hidden / z-index: -1 when inactive. This avoids destroying and recreating the SVG on every tab switch, preserving zoom state and avoiding expensive initial parse/render.

<div style={{
    zIndex: activeTab === 'n' ? 10 : -1,
    visibility: activeTab === 'n' ? 'visible' : 'hidden',
}}>

Highlight & Decoration Optimizations

ID Map Cache (getIdMap)

Files: frontend/src/utils/svgUtils.ts, standalone_interface.html

Instead of container.querySelector([id="${svgId}"]) (O(n) per call), a Map<string, Element> is built once per SVG and cached. Subsequent lookups are O(1). The cache is invalidated when the diagram changes.

CTM Cache for Highlight Positioning

getScreenCTM() is cached per highlight pass instead of computed inside loops. The background layer's CTM is constant for all highlights in a single call, so caching it avoids redundant layout-forcing calls.

Delta Visuals Guard

The data-deltas-applied attribute on the container tracks whether delta CSS classes have been applied. On cleanup, the expensive querySelectorAll scans only run when deltas were previously applied, skipping 4 full-tree scans when switching between Flows/Impacts mode and no deltas exist.

Voltage Filter Early-Return

if (minKv <= uniqueVoltages[0] && maxKv >= uniqueVoltages[uniqueVoltages.length - 1]) return;

When the voltage range slider covers all voltages (the default state), the filter skips iterating all nodes/edges — avoiding ~33,000 style.display writes on large grids.

SVG Boost for Large Grids (boostSvgForLargeGrid)

Files: frontend/src/utils/svgUtils.ts, standalone_interface.html

For grids with ≥500 voltage levels and viewBox ratio > 3× the reference size (1250), text sizes, bus node radii, and edge info elements are scaled up proportionally so they're readable when zoomed in. The function:

1. Parses the SVG string with DOMParser
2. Scales font sizes, circle radii, and transform groups
3. Adds data-large-grid attribute (used by text-hidden CSS)
4. Serializes back to string

Boost cache: Results are cached in an LRU map (max 6 entries: N + N-1 + Action × 2 view modes) keyed by length:vlCount:first200chars to avoid redundant DOM parse/serialize on the same SVG.

SVG DOM Recycling (svgPatch)

Files: frontend/src/utils/svgPatch.ts, frontend/src/hooks/useDiagrams.ts, frontend/src/App.tsx, expert_backend/services/diagram_mixin.py, expert_backend/main.py

Problem

Before this work, switching to the N-1 tab or selecting a different action re-fetched the FULL pypowsybl NAD SVG every time. On the bare_env_20240828T0100Z reference grid (~10 k branches, ~12 MB SVG) this was:

• ~2–4 s of backend get_network_area_diagram work per click (the dominant cost),
• ~27 MB payload on the wire,
• ~250 ms of client-side JSON.parse + DOMParser.parseFromString,
• full re-layout of ~200 k DOM nodes.

The network topology is byte-identical across N, N-1, and most action variants when pypowsybl runs with fixed_positions — only a handful of elements actually change.

Solution

Two new SVG-less endpoints ship only the per-branch delta needed to transform the N-state SVG DOM into N-1 / post-action. The frontend clones the already-mounted N-state SVGSVGElement and patches the clone in-place.

Endpoint	When	Payload
POST /api/contingency-diagram-patch	N-1 tab fetch	{disconnected_edges, absolute_flows, lines_overloaded, flow_deltas, asset_deltas, lf_*, ...} — no SVG body
POST /api/action-variant-diagram-patch	Action click	Same shape, plus vl_subtrees (per-VL node subtree + affected edges) when bus counts change

Client-side pipeline in applyPatchToClone:

1. Splice per-VL subtrees — for node-merging / splitting / coupling actions, the backend ships pypowsybl-native <g id="nad-vl-*"> fragments (focused NAD at depth=1, rendered against the same fixed_positions). The client parses each fragment, rewrites the root id attribute to the main-diagram svgId (pypowsybl svgIds are positional — nad-vl-0 in a focused sub vs. nad-vl-42 in the main NAD), and splices via replaceWith. Same treatment for the affected branches' edge subtrees so their piercing geometry matches the new bus count.
2. Mark disconnected edges dashed — every branch whose connected1 AND connected2 is false on the action variant gets the .nad-disconnected class (new CSS rule with stroke-dasharray + vector-effect: non-scaling-stroke). Covers the N-1 contingency plus any disco_* target; reco_* drops the class.
3. Overwrite absolute flow labels — backend ships absolute_flows.p1/p2/q1/q2, client rewrites each edgeInfo1/2 text with the target-state value (backup in data-patched-flow distinct from the data-original-text owned by applyDeltaVisuals).

Critical performance rule — id map

The flow-label loop touches ~2 × N edges (N ≈ 11 k on the reference grid). clonedSvg.querySelector('[id=...]') inside that loop is O(n_dom_nodes) per call ⇒ billions of comparisons ⇒ the browser tab locks up. Build the id map once with a single querySelectorAll('[id]'), then do O(1) Map.get lookups:

const idMap = buildSvgIdMap(clonedSvg);  // one O(D) scan
for (const edgeId in absolute_flows.p1) {
    const el = idMap.get(baseMetaIndex.edgesByEquipmentId.get(edgeId)?.edgeInfo1?.svgId);
    if (el) patchEdgeInfoText(el, formatFlowValue(...));
}

Do not call querySelector in the flow-label / edge-splice / disconnected-edges loops. Same O(E·D) browser-lock trap that earlier highlight passes had (and why getIdMap exists).

Fresh viewBox identity per patch

usePanZoom caches the live <svg> element via svgElRef and only refreshes that ref on useLayoutEffect([initialViewBox]). If the patch path passes the same originalViewBox object reference across N → patched-N-1 transitions, the layout-effect never re-runs and svgElRef keeps pointing at the previous (now-detached) clone. Pan/zoom then writes viewBox on a detached element — no visible change, main thread saturates — "page not responding".

Fix: shallow-copy originalViewBox on every patch so each transition produces a fresh object reference. See App.tsx fetchN1 and useDiagrams.ts handleActionSelect.

Blank-flash elimination + stale-response guard

Two large-grid-only hazards:

• Blank flash. Calling setActionDiagram(null) synchronously on action click, followed by await api.getActionVariantDiagramPatch(...), broke React's automatic batching. The null commit fired on its own → innerHTML = '' → container blank for the ~200–500 ms the patch needed. Fix: keep the previous cloned DOM mounted through the patch window; only null the diagram on explicit deselect (actionId === null).
• Stale patch response. Rapid A → B clicks with A's patch still in flight used to let A's late response setActionDiagram(A) after B's had already rendered, reverting the user's selection. Fix: latestActionSelectRef tracks the latest click; every await rechecks it on resume and drops a mismatch silently. Same guard around the full-NAD fallback.

Fallback matrix

Situation	Path
Normal N-1 selection with N diagram loaded	patch (/api/contingency-diagram-patch)
N-1 selection during session reload	full (/api/contingency-diagram) — preserves save/load contract
N-1 selection before N SVG is mounted	full fallback
PST / redispatch / load-shedding / curtailment	patch
Line disconnect / reconnect (disco_* / reco_*)	patch (toggles the nad-disconnected class)
Node merging / splitting / coupling	patch with VL-subtree splice (pypowsybl-native focused NAD per affected VL)
VL-subtree extraction partial / raises	full fallback (patchable: false, reason: "vl_topology_changed")
Patch endpoint throws	full fallback

Combined-action line targets

getActionTargetLines used to evaluate isCouplingAction on the full combined action ID/description. For disco_X+coupling_Y the presence of "coupling" in the string suppressed every line-target extraction — the disco line lost its pink halo AND its clickable action-card badge. Fixed by splitting on + and evaluating the coupling flag per sub-part; topology-based bus/line extraction also limited to non-combined actions (combined topologies merge bus changes from multiple sub-actions and can't be cleanly attributed).

Measured savings

On bare_env_20240828T0100Z, contingency ARGIAL71CANTE, warm-median of 3:

Endpoint	Cold	Warm	Payload
/api/contingency-diagram (full)	3.01 s	2.39 s	27.1 MB
/api/contingency-diagram-patch (new)	0.49 s	0.50 s	5.5 MB
Δ	−83.8 %	−79.1 %	20.3 % of full

Raw numbers in profiling_patch_results.json, benchmark driver in benchmarks/bench_n1_diagram_patch.py. Historical detail in docs/performance/history/svg-dom-recycling.md.

Regression Test Coverage

File: frontend/src/utils/cssRegression.test.ts

Automated tests verify that critical CSS rules are present in both App.css and standalone_interface.html:

Test Category	What It Verifies
non-scaling-stroke	CSS rule present for path/line/polyline/rect
CSS containment	contain: layout style paint on .svg-container
text-hidden	display: none for foreignObject when class active
Highlight styles	.nad-overloaded (orange), .nad-action-target (yellow)
Delta visualization	Positive (orange) and negative (blue) delta styles
usePanZoom guards	useLayoutEffect deps correct, tab sync uses useLayoutEffect
Voltage filter	Early-return when range covers all voltages
Deferred highlights	requestAnimationFrame used on tab switch
Boost cache	_boostCache and BOOST_CACHE_MAX present

File: frontend/src/hooks/usePanZoom.test.tsx

Tests verify:

• ViewBox sync on mount, activation, and diagram changes
• ViewBox preservation across active/inactive transitions
• Text visibility toggle on large grids (hidden at zoom-out, visible at zoom-in)
• No corruption after rapid tab switching

Summary: Do's and Don'ts

Do	Don't
Use vector-effect: non-scaling-stroke on SVG elements	Remove it to fix "thick lines" — adjust stroke-width instead
Use useLayoutEffect for viewBox sync	Change to useEffect — causes visible flicker
Defer decorations via requestAnimationFrame on tab switch	Apply highlights synchronously on tab switch
Cache getScreenCTM() and ID maps	Call querySelector or getScreenCTM() in loops
Keep all SVG containers mounted (visibility toggle)	Conditionally render/destroy SVG containers on tab switch
Short-circuit voltage filter when range covers all	Iterate all elements even when no filtering needed
Run cssRegression.test.ts after CSS changes	Skip tests after modifying App.css or standalone CSS
Build an id map once per applyPatchToClone call	Call querySelector('[id=...]') inside flow-label / edge-splice loops
Shallow-copy originalViewBox on every patch so usePanZoom refreshes its cached svgElRef	Share the same viewBox object reference across N → patched-N-1 swaps
Rewrite spliced <g id="nad-vl-*"> root ids to the main-diagram svgId	Trust the focused sub-diagram's positional svgIds as-is
Keep the previous cloned DOM mounted through the patch-fetch window	setActionDiagram(null) synchronously on click before an await
Drop late patch responses via latestActionSelectRef	Let a stale response overwrite the current action selection
Evaluate isCouplingAction per +-split part on combined actions	Evaluate it on the full combined action ID (suppresses line-target extraction)