# CLAUDE.md - Co-Study4Grid ## Project Overview Co-Study4Grid is a full-stack web application for **power grid contingency analysis and N-1 planning**. It provides an interface to the `expert_op4grid_recommender` library, allowing operators to simulate element disconnections, visualize network overflow graphs, and receive prioritized remedial action recommendations. ## Architecture **Monorepo** with two main components plus a standalone HTML mirror: ``` Co-Study4Grid/ ├── CLAUDE.md # This file — project overview + standalone parity audit ├── README.md # User-facing project description + quick start ├── CHANGELOG.md # Per-release changelog (current: 0.9.0) ├── CONTRIBUTING.md # Contributor setup, code-quality gate ├── pyproject.toml # Python project metadata + ruff config (E9/F ruleset) ├── pytest.ini # Pytest config (testpaths = expert_backend/tests) ├── expert_backend/ # Python FastAPI backend │ ├── CLAUDE.md # Backend-scoped guide (singletons, mixins, lifecycle) │ ├── main.py # FastAPI app: endpoints, CORS, gzip helpers, NDJSON streaming │ ├── test_backend.py # Ad-hoc integration script (not part of pytest) │ ├── recommenders/ # Pluggable recommendation-model registry: registry.py, │ │ │ # random_basic / random_overflow canonical examples, │ │ │ # overflow_path_filter + network_existence sampling │ │ │ # filters, synthetic_actions builders. Models are │ │ │ # registered at package import; the service integration │ │ │ # is EXPLICIT composition (RecommenderService inherits │ │ │ # ModelSelectionMixin; AnalysisMixin.run_analysis_step2 │ │ │ # consumes the registry — no import-time patching). │ │ │ # Full reference: docs/backend/recommender_models.md │ ├── services/ │ │ ├── network_service.py # pypowsybl Network singleton + metadata queries │ │ ├── recommender_service.py # Analysis orchestrator (composes the 4 mixins below) │ │ ├── diagram_mixin.py # NAD/SLD orchestrator — delegates to services/diagram/ │ │ ├── analysis_mixin.py # Two-step analysis orchestrator (model-aware step-2 │ │ │ # dispatch through the recommenders registry) — │ │ │ # delegates to services/analysis/ │ │ ├── simulation_mixin.py # Manual-action + superposition orchestrator │ │ ├── model_selection_mixin.py # Active recommender model + overflow-graph toggle │ │ ├── simulation_helpers.py # Stateless helpers extracted from simulation_mixin (PR #104) │ │ ├── overflow_overlay.py # Pin / filter overlay injector for the interactive │ │ │ # HTML overflow viewer (PR #116, 0.7.0) │ │ ├── sanitize.py # NumPy → native-Python recursive coercion │ │ ├── analysis/ # PR #104 decomposition — action_enrichment, │ │ │ # mw_start_scoring, analysis_runner, pdf_watcher, │ │ │ # overflow_geo_transform (PR #116 — geo-layout SVG │ │ │ # transform for /api/regenerate-overflow-graph) │ │ └── diagram/ # PR #104 decomposition — layout_cache, nad_params, │ │ # nad_render, sld_render, overloads, flows, deltas, │ │ # obs_prewarm (post-contingency obs cache prewarm │ │ # that lets run_analysis_step1 skip the LF), │ │ # action_patch (extracted action-variant patch │ │ # pipeline — keeps diagram_mixin under the LoC │ │ # ceiling) │ └── tests/ # pytest suite — see tests/CLAUDE.md for the mock layer ├── frontend/ # React 19 + TypeScript 5.9 + Vite 7 frontend │ ├── CLAUDE.md # Frontend-scoped guide (App.tsx hub, hooks, SVG levers) │ ├── package.json, vite.config.ts, vite.config.standalone.ts, │ │ # eslint.config.js, tsconfig*.json │ └── src/ │ ├── App.tsx # State orchestration hub (~1400 lines) │ ├── api.ts # Axios HTTP client (base URL: 127.0.0.1:8000) │ ├── types.ts # All TypeScript interfaces (one file) │ ├── styles/ # Design-token palette (PR #120, 0.7.0): │ │ # tokens.css (CSS custom properties) + │ │ # tokens.ts (typed colors / space / text / │ │ # radius / pinColors* constants). Single │ │ # source of truth for the entire UI; the │ │ # code-quality gate enforces zero hex │ │ # literals outside these two files. │ ├── hooks/ # useSettings / useActions / useAnalysis / useDiagrams / │ │ # useSession / useDetachedTabs / useTiedTabsSync / │ │ # usePanZoom / useSldOverlay / useContingencyFetch (svgPatch fast- │ │ # path + full fallback) / useDiagramHighlights (per-tab │ │ # highlight pipeline + Flow/Impacts view-mode state) / │ │ # useOverflowIframe (PR #116 — iframe lifecycle, layer │ │ # toggles, postMessage bridge, pin overlay payload) / │ │ # useSldTopologyEdit (interactive SLD switch-edit → │ │ # manual action, 0.8.0) / useTheme (light/dark theme │ │ # toggle + persistence, 0.8.0) │ ├── components/ # Header, ActionFeed, ActionCard, ActionCardPopover, │ │ # ActionSearchDropdown (editable Δ MW column │ │ # for redispatch in score table), │ │ # ActionTypeFilterChips, │ │ # ActionFilterRings (shared severity + action-type + │ │ # Max-loading ring strip, 0.8.0), ActionTypeIcon / │ │ # SeverityIcon (action-type + severity pictograms), │ │ # AdditionalLinesPicker, │ │ # ActionOverviewDiagram, AppSidebar (collapsible │ │ # shell — readability-feed PR), SidebarSummary │ │ # (sticky strip — hosts Clear button + overload │ │ # info bubble that absorbed the legacy │ │ # OverloadPanel affordances), │ │ # NotificationHost (typed toast store — │ │ # severity/dismiss/aria-live, PR D5), │ │ # VisualizationPanel, OverloadPanel │ │ # (kept on disk for unit-test backwards-compat; │ │ # no longer rendered from App.tsx), │ │ # CombinedActionsModal, ComputedPairsTable, │ │ # ExplorePairsTab, SldOverlay, SldEditPanel │ │ # (interactive maneuver list, 0.8.0), │ │ # SldInjectionPopover (SLD load/gen active- │ │ # power editor bubble), DetachableTabHost, │ │ # MemoizedSvgContainer, ErrorBoundary, NoticesPanel │ │ # (PR #122 tier system), DiagramLegend (PR #122), │ │ # InspectSearchField + DetachedPlaceholder (PR #116 │ │ # — extracted from VisualizationPanel) │ │ # + modals/ (SettingsModal, ReloadSessionModal, │ │ # ConfirmationDialog) │ ├── game/ # Timed, scored Game Mode (0.8.0; active only │ │ # with ?game=1) — GameShell / useGameSession / │ │ # gameBridge / GameConfigScreen / GameHud / │ │ # GameResults / scoring / gameLog / presets / │ │ # types. See docs/features/game-mode-codabench.md │ └── utils/ # svgUtils (barrel re-exporting utils/svg/*), │ # svgPatch (DOM-recycling patch applier), │ # overloadHighlights, sessionUtils, interactionLogger, │ # popoverPlacement, mergeAnalysisResult, actionTypes │ # (classifyActionType + DEFAULT_ACTION_OVERVIEW_FILTERS), │ # inspectables (filterInspectables — match an element │ # by its displayed name, not just its raw id), │ # ndjsonStream (single NDJSON reader, D5), │ # notifications (typed toast store, D5), │ # fileRegistry (structure regression guard) │ └── svg/ # PR #104 decomposition — idMap, metadataIndex, │ # svgBoost, fitRect, deltaVisuals, actionPinData, │ # actionPinRender, highlights, vlInteractions (VL-disk │ # hover name / click-to-inspect / dbl-click SLD), │ # overflowPinPayload │ # + overflowOverlayRender + pinGlyph (PR #116 — │ # iframe-overlay pin pipeline) ├── standalone_interface_legacy.html # DECOMMISSIONED 2026-04-20 — hand-maintained │ # single-file mirror frozen at its last version and │ # tracked here for reference only. Replaced by the │ # auto-generated `frontend/dist-standalone/standalone.html` │ # (`npm run build:standalone`). New UI changes land ONLY │ # in `frontend/src/` — do NOT edit this file further. ├── docs/ # Design docs — organized into features/, performance/ │ # (+ history/), architecture/, proposals/, data/. │ # See `docs/README.md` for the index. ├── data/ # Sample grids: bare_env_small_grid_test, pypsa_eur_fr400, │ # pypsa_eur_fr225_400 (France 225/400 kV — its large │ # network ships compressed as network.xiidm.zip, │ # auto-decompressed by network_service on load) ├── benchmarks/ # Perf scripts (bench_load_study, _bench_common) ├── Overflow_Graph/ # Generated PDFs (created at runtime) ├── overrides.txt # Pinned versions for transitive Python deps ├── requirements_py310.txt # Python 3.10-pinned requirements superset ├── scripts/ # Integration / parity / build helpers — │ # `check_standalone_parity.py`, │ # `check_session_fidelity.py`, │ # `check_gesture_sequence.py`, │ # `check_invariants.py`, `check_code_quality.py`, │ # `check_openapi_contract.py` (D2 — diffs the live │ # app.openapi() against expert_backend/openapi.snapshot.json), │ # `code_quality_report.py`, `profile_diagram_perf.py`, │ # `test_code_quality_report.py`, │ # `test_estimation_vs_simulation_small_grid.py`, │ # `pypsa_eur/` (full PyPSA-EUR → XIIDM pipeline │ # with its own pytest coverage; `build_pipeline.py` │ # writes a per-bundle provenance.json, D8), and │ # `game_mode/` (`e2e_game_session.py` — real-backend │ # Game Mode replay; `scoring_program/score.py` — │ # in-repo Codabench scorer twin of scoring.ts, │ # pinned to `scoring_golden.json` by test_score.py │ # + scoring.test.ts for cross-language parity, D8) ├── Dockerfile # Single-container HuggingFace Docker Space image — │ # same-origin SPA + FastAPI on :7860, game mode on ├── .dockerignore ├── deploy/ # HuggingFace Space README + step-by-step SETUP.md ├── config.default.json # Bundled first-run settings (fr225_400 grid + │ # per-action-type recommender minima) ├── .editorconfig # Cross-editor indent / EOL defaults ├── .env.example # Template for backend env vars (CORS, …) ├── .gitattributes # Git LFS tracking for *.zip / *.png / *.jpg(eg) │ # (HuggingFace Space git endpoint requires LFS) └── .gitignore # Excludes __pycache__/, *.pyc, *.pyo, node_modules/ ``` ### Per-subtree docs | File | Scope | |------|-------| | `CLAUDE.md` (this file) | Project overview, API table, conventions, parity-audit pointer | | `frontend/PARITY_AUDIT.md` | Full standalone-parity audit: feature inventory, mirror-status table, Layer 1–4 conformity findings, gap-priority list, deltas. Split out of this file 2026-04-20. | | `expert_backend/CLAUDE.md` | Backend internals: singletons, mixin composition, state lifecycle, NDJSON streaming, gzip helpers, layout cache invariants, NAD prefetch | | `expert_backend/tests/CLAUDE.md` | Test conventions, the `conftest.py` mock layer for `pypowsybl` / `expert_op4grid_recommender`, frontend Vitest patterns | | `frontend/CLAUDE.md` | Frontend internals: hook split, data flow, state reset, SVG performance levers, detached/tied tabs, interaction logger contract | | `docs/README.md` | Index of design/feature/perf/architecture/proposal docs. Start here for any `docs/**` lookup. | | `docs/features/save-results.md` | Save / reload session contract (JSON schema, reload flow, regression-guard matrix) | | `docs/features/adding-action-type.md` | Cross-cutting checklist for adding/upgrading a remedial-action type (lib → backend → frontend → save/log/reload triad → regression specs) | | `docs/features/interaction-logging.md` | Replay-ready event log contract | | `docs/features/sld-topology-edit.md` | Interactive SLD topology edit → manual action card | | `docs/features/sld-diagram-feeder-labels.md` | SLD feeders relabelled by far-end VL name (+ parallel index), overload-halo friendly-name↔IIDM-id bridge, and the charging-current annotation explaining the "after" loading of a line opened at one end | | `docs/features/vl-disk-interactions.md` | Interactive VL disks on the NAD (hover name / click → Inspect / double-click → SLD) + delegation performance contract | | `docs/features/game-mode-codabench.md` | Timed, scored Game Mode (`?game=1`) + Codabench benchmark bundle | | `deploy/huggingface/` | HuggingFace Docker Space deployment (Space README + `SETUP.md`) | ## Tech Stack ### Backend - **Python** with **FastAPI** + **Uvicorn** - **pypowsybl** - Power system network loading, load flow, and diagram generation - **expert_op4grid_recommender** - Domain-specific grid optimization recommendations - **grid2op** / **pandapower** / **lightsim2grid** - Grid simulation backends ### Frontend - **React 19** with **TypeScript 5.9** - **Vite 7** - Build tool and dev server - **axios** - HTTP client - **react-select** - Searchable dropdown for branch selection - **vite-plugin-singlefile** - Auto-generated single-file standalone bundle - **Vitest** + **React Testing Library** - Unit / integration tests ## Development Workflow ### Running the Backend ```bash # From the project root: python -m expert_backend.main # Or: uvicorn expert_backend.main:app --host 0.0.0.0 --port 8000 ``` The backend serves on `http://localhost:8000`. It expects `pypowsybl` and `expert_op4grid_recommender` to be available in the Python environment. ### Running the Frontend ```bash cd frontend npm install npm run dev # Start Vite dev server with HMR ``` The frontend dev server proxies API calls to `http://localhost:8000` (hardcoded in `frontend/src/api.ts`). ### Build & Lint ```bash cd frontend npm run build # TypeScript compilation (tsc -b) + Vite production build npm run lint # ESLint npm run preview # Preview production build ``` ### Running Tests Backend unit tests use `pytest` and run against the in-repo mock layer (no live pypowsybl required): ```bash pytest # Full backend suite pytest expert_backend/tests/test_foo.py # Single file ``` Ad-hoc integration scripts live in `scripts/` (and `scripts/pypsa_eur/` for the PyPSA-EUR → XIIDM pipeline). The pipeline scripts carry their own pytest coverage (`scripts/pypsa_eur/test_*.py`) alongside the backend suite; the rest require a running backend with real data: ```bash pytest scripts/pypsa_eur # Pipeline unit tests python scripts/pypsa_eur/test_pipeline.py # End-to-end smoke test python scripts/pypsa_eur/test_n1_calibration.py # N-1 flow calibration check python scripts/pypsa_eur/test_grid_layout.py # Layout loading sanity check python scripts/profile_diagram_perf.py # NAD rendering profiler ``` Frontend unit tests use Vitest: ```bash cd frontend npm run test # Run Vitest test suite ``` ### Code-Quality Checks (continuous reporting) ```bash # Generate a full JSON + Markdown report (backend + frontend metrics) python scripts/code_quality_report.py --output reports/code-quality.json \ --markdown reports/code-quality.md # Gate a pull request: non-zero exit on threshold violation python scripts/check_code_quality.py ``` Both scripts run in CI (`.github/workflows/code-quality.yml`). The gate guards the reductions documented in [`docs/architecture/code-quality-analysis.md`](docs/architecture/code-quality-analysis.md) (no new `print()` / bare except, module-size ceilings, no `any` / `@ts-ignore` in frontend source). ## API Endpoints | Method | Path | Description | |--------|------|-------------| | GET | `/api/user-config` | Read persisted user configuration (paths, recommender params) | | POST | `/api/user-config` | Persist user configuration | | GET | `/api/config-file-path` | Get the current user-config file path | | POST | `/api/config-file-path` | Set a custom user-config file path | | POST | `/api/config` | Set network path, action file path, and all recommender parameters (incl. `model` and `compute_overflow_graph`) | | GET | `/api/models` | List registered recommendation models with their `params_spec()` and capability flags | | POST | `/api/recommender-model` | Lightweight swap of the active recommender model (no network reload) — fired by the model dropdowns in Settings and above Analyze & Suggest | | GET | `/api/branches` | List disconnectable elements (lines + 2-winding transformers) | | GET | `/api/voltage-levels` | List voltage levels in the network | | GET | `/api/nominal-voltages` | Map voltage level IDs to nominal voltages (kV) | | GET | `/api/element-voltage-levels` | Resolve equipment ID to its voltage level IDs | | GET | `/api/voltage-level-substations` | Map voltage level IDs to their parent substation IDs (used by the SLD overlay and overflow pin pipeline) | | POST | `/api/run-analysis` | Run full N-1 contingency analysis (streaming NDJSON, legacy) | | POST | `/api/run-analysis-step1` | Two-step analysis Part 1: detect overloads | | POST | `/api/run-analysis-step2` | Two-step analysis Part 2: resolve with actions (streaming NDJSON) | | GET | `/api/network-diagram` | Get N-state network SVG diagram (NAD) | | POST | `/api/contingency-diagram` | Get post-contingency N-1 diagram with flow deltas | | POST | `/api/contingency-diagram-patch` | SVG-less per-branch delta for DOM-recycling fast path (PR #108) | | POST | `/api/action-variant-diagram` | Get network state after applying a remedial action | | POST | `/api/action-variant-diagram-patch` | Per-branch delta + VL-subtree splice for action DOM recycling | | POST | `/api/focused-diagram` | Generate NAD sub-diagram focused on a specific element | | POST | `/api/action-variant-focused-diagram` | Focused NAD for specific VL in post-action state | | POST | `/api/n-sld` | Single Line Diagram for voltage level in N state. Response includes `switch_states` (per-switch open/closed map), `injections` (per-load/generator active-power baseline) used by the interactive SLD-edit feature, **and** `feeder_labels` (per-branch `{name, other_vl, label}` — `label` = far-end VL name + parallel index, used to relabel feeders and bridge friendly-named overloads to the SLD cell). | | POST | `/api/contingency-sld` | Single Line Diagram in N-1 state (with flow deltas + `switch_states` + `injections` + `feeder_labels`). | | POST | `/api/action-variant-sld` | SLD in post-action state (with flow deltas, `changed_switches`, `switch_states`, `injections`, `feeder_labels`). | | POST | `/api/sld-topology-preview` | Target-topology preview SLD for the interactive SLD-edit feature: applies staged switch overrides on a throwaway variant and re-renders with topological colouring (no load flow; `stale_flows: true`). | | GET | `/api/actions` | Return all available action IDs and descriptions | | POST | `/api/regenerate-overflow-graph` | Regenerate (or serve from cache) the overflow graph in hierarchical / geo layout — drives the toggle on the Overflow Analysis tab | | POST | `/api/simulate-manual-action` | Simulate a specific action against a contingency. Accepts an optional `voltage_level_id` field used to auto-name switch-only user actions (interactive SLD-edit feature). | | POST | `/api/simulate-and-variant-diagram` | NDJSON stream: `{type:"metrics"}` then `{type:"diagram"}` so sidebar updates ahead of the SVG | | POST | `/api/compute-superposition` | Compute combined effect of two actions (superposition theorem) | | POST | `/api/save-session` | Save session folder with JSON snapshot + PDF copy | | GET | `/api/list-sessions` | List available session folders in a directory | | POST | `/api/load-session` | Load session JSON and restore PDFs | | POST | `/api/restore-analysis-context` | Restore analysis context from saved session | | GET | `/api/pick-path` | Open native OS file/directory picker (tkinter subprocess) | | GET | `/results/pdf/{filename}` | Serve generated overflow-graph files from `Overflow_Graph/` — HTML (interactive viewer, current default via `config.VISUALIZATION_FORMAT="html"`) or PDF (legacy sessions). URL path kept for backward compatibility. | ## Key Patterns & Conventions ### Backend - **Singleton services**: `network_service` and `recommender_service` are module-level singleton instances - **Streaming responses**: Analysis uses `StreamingResponse` with NDJSON (`application/x-ndjson`), yielding `{"type": "pdf", ...}` then `{"type": "result", ...}` events - **AC/DC fallback**: Analysis first tries AC load flow; falls back to DC if AC does not converge - **Threaded analysis**: `run_analysis` runs the computation in a background thread and polls for PDF generation - **JSON sanitization**: NumPy types are recursively converted to native Python types via `sanitize_for_json()` - **Unified error contract (D2, 2026-07)**: every error is `{"detail", "code"}` produced in one place (`services/api_errors.py`, `install_error_handlers(app)`). Raise a plain `HTTPException` (code derived from status: 400/404/409/422/500) or `AppHTTPException(status, detail, code)` when the frontend branches on the failure (e.g. `ACTION_RESULT_UNAVAILABLE`, `STUDY_BUSY`). Uncaught exceptions → generic logged 500 (never `str(e)` — it leaks paths). The frontend reads it via `frontend/src/utils/apiError.ts`. - **Machine-checked API contract (D2, 2026-07)**: `app.openapi()` is snapshotted to `expert_backend/openapi.snapshot.json` and diffed in CI (`scripts/check_openapi_contract.py`, `test_openapi_contract.py`). Regenerate on any deliberate endpoint / model / status change: `python scripts/check_openapi_contract.py --write`. - **Concurrency ownership (D3, 2026-07)**: a service-level re-entrant lock (`services/service_lock.py`, `@with_network_lock[_stream]`) serializes the variant-switching entry points on the shared `Network`; overlapping study mutations (config load / analysis) get **HTTP 409**. See [`docs/architecture/shared-network-concurrency.md`](docs/architecture/shared-network-concurrency.md). - **Mixin → helper-package decomposition (PR #104 / #106)**: `DiagramMixin`, `AnalysisMixin` and `SimulationMixin` are thin orchestrators. Pure numerics live in `services/diagram/`, `services/analysis/` and `services/simulation_helpers.py` respectively — dependency-injected so existing `@patch` tests keep working. - **SVG DOM recycling (PR #108)**: patch endpoints (`/api/contingency-diagram-patch`, `/api/action-variant-diagram-patch`) return per-branch deltas + optional VL-subtree splices so the frontend can clone the already-mounted N-state SVG instead of re-downloading the full NAD (~80 % faster tab switches on large grids). - **Interactive overflow viewer (PR #116, 0.7.0)**: `services/overflow_overlay.py` injects a Co-Study4Grid pin / filter overlay (`