docs/data/grid-layout-coordinate-scale.md

Grid-layout coordinate scale

TL;DR: grid_layout.json MUST be in raw Mercator metres (span ≈ 1.4–1.6 M for the French grid). Pypowsybl emits VL outer circles at a fixed r = 27.5 user-space units; if the layout span gets squashed to e.g. 8 000, then 27.5 stops being negligible and every neighbour overlaps the rendered circle. This document explains the math and records the 2026-05-08 fix.

Symptom

On pypsa_eur_fr225_400 the Network (N) tab rendered as an unreadable dark blob in dense urban regions (Paris, Lyon, Lille). VL circles visibly overlapped one another even at maximum zoom-in. Branch lines, flow arrows, and labels were all crammed inside the blob. Frontend SVG post-processing (svgBoost) could mitigate but never fully clear the overlap.

Diagnosis

Pypowsybl's NAD generator (powsybl-network-area-diagram) emits each VL bus node as a <circle r="27.5"> in user-space SVG coordinates, regardless of the surrounding layout scale. Other glyph sizes are similarly fixed (line stroke 5, arrow 10×10, edge-info font 20). The viewBox→screen transform applies uniformly to all of them, so the relative size of nodes vs inter-node distance is determined entirely by the on-disk coordinate scale.

We compared two layout files for the French grid:

Layout	Source	Entries	x-span (units)	Median nearest-neighbour	Median NN ÷ r=27.5
Operator reference (RTE study format)	Internal RTE export	18 141	1 643 372	1 776	64.6 ×
Pre-fix regenerate_grid_layout.py	OSM → Mercator, ÷ 8000/x_range	1 196	8 000	26	0.95 ×

A median NN of 0.95 × r means adjacent VL circles overlap by definition — the circle's diameter is roughly the distance between its centre and the next centre over. There is no clipping, alpha, or zoom tweak that can recover whitespace from this geometry.

The operator file's 65 × ratio is comfortable: the next neighbour sits 65 r away, leaving > 100 × visual whitespace per node. That is the geometry pypowsybl was designed for.

Where the rescale came from

scripts/pypsa_eur/regenerate_grid_layout.py (originally cribbed from Step 9 of convert_pypsa_to_xiidm.py) projected raw (lon, lat) to Web Mercator, then rescaled every coordinate by 8000 / (max_x − min_x) to fit a fixed 8 000-unit target width. The constant TARGET_WIDTH = 8000 was inherited from the conversion pipeline. It made smaller test grids look "right-sized" at hand-curated viewBoxes but broke for any production-scale dataset.

D8 (2026-07-09): the source of the constant — convert_pypsa_to_xiidm.py's step_write_metadata — still carried the same TARGET_WIDTH = 8_000 rescale, so a pipeline rebuild silently re-emitted the forbidden layout. It now writes grid_layout.json in raw Mercator metres directly (re-centred, no rescale), matching regenerate_grid_layout.py's default. test_grid_layout.py::test_coordinate_spans_in_reasonable_range was corrected from the stale [5000, 20000] assertion (which encoded the forbidden scale) to the raw-metres regime.

Fix (2026-05-08)

regenerate_grid_layout.py:

• Default behaviour now writes raw Mercator metres (centred on the bounding-box midpoint). For the French grid this gives x_span ≈ 1.4 M, matching the operator reference scale.
• The --target-width N flag is preserved for backward compatibility but defaults to None. When set below 500 000 the script logs a warning explaining the readability-vs-rescale trade-off.
• The regeneration prints a median NN / r ratio at the end so you can see at a glance whether the output will read cleanly.

Both bundled layouts were regenerated:

Bundle	Old span	New span	Old median NN/r	New median NN/r
data/pypsa_eur_fr225_400/	8 000	1 401 015	0.95 ×	166.8 ×
data/pypsa_eur_fr400/	8 000	1 307 946	varied	1 090 ×

(The fr400 ratio is unusually high because that dataset only carries 400 kV transmission VLs — 190 entries spread across the same physical area — so neighbours are kilometres apart.)

The pre-fix files are kept as grid_layout.json.bak.8000width siblings so the rescale can be reverted with a single cp if a downstream consumer turns out to depend on the old span.

How to regenerate

# Default (recommended): raw Mercator metres
python scripts/pypsa_eur/regenerate_grid_layout.py --network data/pypsa_eur_fr225_400

# Legacy: rescale to 8 000-unit width (NOT recommended for dense grids)
python scripts/pypsa_eur/regenerate_grid_layout.py --network data/pypsa_eur_fr225_400 --target-width 8000

Regression guards

• scripts/pypsa_eur/test_regenerate_grid_layout.py::TestCoordinateSanity::test_x_span_matches_france_mercator asserts the on-disk span is in the operator-clean range [1.3 M, 1.7 M].
• The test docstring explains the failure mode if someone reverts the script.

Why this also retires the svgBoost band-aids

The branch's earlier work (utils/svg/svgBoost.ts) added several frontend SVG transforms — node-circle shrinking, edge-info downscale, polyline kink-drop, line-extension to the new circle edge, and flow-indicator projection. All of them were trying to compensate for the squashed coordinate scale. With the layout fixed at the source, those transforms are no-ops on the operator-clean default (median NN/r = 65–166 ×, well above the threshold the boost code triggers on). The boost code can stay as a safety net for any caller that intentionally uses a small --target-width, but the canonical path no longer needs it.

Cross-references

• scripts/pypsa_eur/regenerate_grid_layout.py — module docstring carries the same math summary.
• CHANGELOG.md — [Unreleased] "Fixed" section.
• frontend/src/utils/svg/svgBoost.ts — frontend mitigations that the layout fix retires.
• voltage-level-separation.md — the follow-on fix for substations whose several voltage levels share one location; run separate_voltage_levels.py after any regeneration here.