# Grid-layout coordinate scale

> **TL;DR**: `grid_layout.json` MUST be in raw Mercator metres (span ≈ 1.4–1.6 M for the French grid). Pypowsybl emits VL outer circles at a *fixed* `r = 27.5` user-space units; if the layout span gets squashed to e.g. 8 000, then 27.5 stops being negligible and every neighbour overlaps the rendered circle. This document explains the math and records the 2026-05-08 fix.

## Symptom

On `pypsa_eur_fr225_400` the Network (N) tab rendered as an unreadable dark blob in dense urban regions (Paris, Lyon, Lille). VL circles visibly overlapped one another even at maximum zoom-in. Branch lines, flow arrows, and labels were all crammed inside the blob. Frontend SVG post-processing (svgBoost) could mitigate but never fully clear the overlap.

## Diagnosis

Pypowsybl's NAD generator (powsybl-network-area-diagram) emits each VL bus node as a `<circle r="27.5">` in **user-space SVG coordinates**, regardless of the surrounding layout scale. Other glyph sizes are similarly fixed (line stroke 5, arrow 10×10, edge-info font 20). The viewBox→screen transform applies uniformly to all of them, so the *relative* size of nodes vs inter-node distance is determined entirely by the on-disk coordinate scale.

We compared two layout files for the French grid:

| Layout                                  | Source                       | Entries | x-span (units) | Median nearest-neighbour | Median NN ÷ r=27.5 |
|-----------------------------------------|------------------------------|---------|----------------|--------------------------|--------------------|
| Operator reference (RTE study format)   | Internal RTE export          | 18 141  | 1 643 372      | 1 776                    | **64.6 ×**         |
| Pre-fix `regenerate_grid_layout.py`     | OSM → Mercator, `÷ 8000/x_range`| 1 196 | 8 000          | 26                       | **0.95 ×**         |

A median NN of 0.95 × r means *adjacent VL circles overlap by definition* — the circle's diameter is roughly the distance between its centre and the next centre over. There is no clipping, alpha, or zoom tweak that can recover whitespace from this geometry.

The operator file's 65 × ratio is comfortable: the next neighbour sits 65 r away, leaving > 100 × visual whitespace per node. That is the geometry pypowsybl was designed for.

## Where the rescale came from

`scripts/pypsa_eur/regenerate_grid_layout.py` (originally cribbed from Step 9 of `convert_pypsa_to_xiidm.py`) projected raw `(lon, lat)` to Web Mercator, then rescaled every coordinate by `8000 / (max_x − min_x)` to fit a fixed 8 000-unit target width. The constant `TARGET_WIDTH = 8000` was inherited from the conversion pipeline. It made smaller test grids look "right-sized" at hand-curated `viewBox`es but broke for any production-scale dataset.

> **D8 (2026-07-09):** the *source* of the constant — `convert_pypsa_to_xiidm.py`'s `step_write_metadata` — still carried the same `TARGET_WIDTH = 8_000` rescale, so a pipeline rebuild silently re-emitted the forbidden layout. It now writes `grid_layout.json` in raw Mercator metres directly (re-centred, no rescale), matching `regenerate_grid_layout.py`'s default. `test_grid_layout.py::test_coordinate_spans_in_reasonable_range` was corrected from the stale `[5000, 20000]` assertion (which encoded the forbidden scale) to the raw-metres regime.

## Fix (2026-05-08)

`regenerate_grid_layout.py`:

* Default behaviour now **writes raw Mercator metres** (centred on the bounding-box midpoint). For the French grid this gives `x_span ≈ 1.4 M`, matching the operator reference scale.
* The `--target-width N` flag is preserved for backward compatibility but defaults to `None`. When set below 500 000 the script logs a warning explaining the readability-vs-rescale trade-off.
* The regeneration prints a `median NN / r` ratio at the end so you can see at a glance whether the output will read cleanly.

Both bundled layouts were regenerated:

| Bundle                         | Old span  | New span         | Old median NN/r | New median NN/r |
|--------------------------------|-----------|------------------|-----------------|-----------------|
| `data/pypsa_eur_fr225_400/`    | 8 000     | 1 401 015        | 0.95 ×          | **166.8 ×**     |
| `data/pypsa_eur_fr400/`        | 8 000     | 1 307 946        | varied          | **1 090 ×**     |

(The fr400 ratio is unusually high because that dataset only carries 400 kV transmission VLs — 190 entries spread across the same physical area — so neighbours are kilometres apart.)

The pre-fix files are kept as `grid_layout.json.bak.8000width` siblings so the rescale can be reverted with a single `cp` if a downstream consumer turns out to depend on the old span.

## How to regenerate

```bash
# Default (recommended): raw Mercator metres
python scripts/pypsa_eur/regenerate_grid_layout.py --network data/pypsa_eur_fr225_400

# Legacy: rescale to 8 000-unit width (NOT recommended for dense grids)
python scripts/pypsa_eur/regenerate_grid_layout.py --network data/pypsa_eur_fr225_400 --target-width 8000
```

## Regression guards

* `scripts/pypsa_eur/test_regenerate_grid_layout.py::TestCoordinateSanity::test_x_span_matches_france_mercator` asserts the on-disk span is in the operator-clean range [1.3 M, 1.7 M].
* The test docstring explains the failure mode if someone reverts the script.

## Why this also retires the svgBoost band-aids

The branch's earlier work (`utils/svg/svgBoost.ts`) added several frontend SVG transforms — node-circle shrinking, edge-info downscale, polyline kink-drop, line-extension to the new circle edge, and flow-indicator projection. All of them were trying to compensate for the squashed coordinate scale. With the layout fixed at the source, those transforms are no-ops on the operator-clean default (median NN/r = 65–166 ×, well above the threshold the boost code triggers on). The boost code can stay as a safety net for any caller that *intentionally* uses a small `--target-width`, but the canonical path no longer needs it.

## Cross-references

* `scripts/pypsa_eur/regenerate_grid_layout.py` — module docstring carries the same math summary.
* `CHANGELOG.md` — `[Unreleased]` "Fixed" section.
* `frontend/src/utils/svg/svgBoost.ts` — frontend mitigations that the layout fix retires.
* [`voltage-level-separation.md`](voltage-level-separation.md) — the follow-on fix for substations whose **several voltage levels** share one location; run `separate_voltage_levels.py` *after* any regeneration here.