README.md

---
license: mit
tags:
- reinforcement-learning
- multi-agent
- time-series
- diffusion-model
- energy-management
- smart-grid
---
# SolarSys: Scalable Hierarchical Coordination for Distributed Solar Energy
## Abstract

Distributed solar energy resources (DSERs) in residential power systems are rapidly increasing due to declining photovoltaic module prices. This increase has led to the development of virtual power plants (VPPs) that coordinate local generation to support reliable and efficient grid operation. However, many existing VPPs rely on centralized control, where a central operator aggregates household-level information and determines trading outcomes, preventing individual users from independently and concurrently trading surplus energy. 
As a result, researchers have explored decentralized coordination, where individual households trade surplus solar energy directly with others to improve utilization and fairness within residential networks. However, such approaches perform well only in small deployments and become inefficient at scale as the number of participating households increases.

To address these limitations, we design SolarSys, a hierarchical coordination system for peer-to-peer energy trading in distributed
solar and storage communities. SolarSys integrates local power sensing from smart meters and inverters with edge computation and distributed coordination to manage energy usage across large residential networks. The system is organized hierarchically to support decision making at both the household and cluster levels. Each household node measures its generation, demand, and battery state, and makes local energy management decisions using a policy trained with Multi-Agent Proximal Policy Optimization to improve local solar usage and maintain cluster-level energy balance.

Across clusters, SolarSys leverages a mean-field multi-agent reinforcement learning approach that updates high-level decisions using aggregated cluster conditions. This design allows clusters to adjust their decisions so the overall system remains consistent with practical grid constraints. In addition, we deploy SolarSys on a Raspberry Pi and find that the learned policies can run efficiently on low-power edge hardware.

We evaluate SolarSys using real smart meter data from seven residential communities. We find that SolarSys reduces the energy drawn from the grid by 27.48 ± 0.42\%, increases daytime solar utilization to 82.76 ± 5.11\%, and improves fairness to 0.773 (Jain’s Index). The results show that SolarSys enables efficient, fair, and scalable peer-to-peer energy trading in large-scale virtual power plant deployments.

---


## System Architecture

The core of SolarSys is a two-level decision hierarchy:

1.  **Low-Level (Intra-Cluster):** Individual households use a **MAPPO** agent to make instantaneous decisions (charge, discharge, local P2P trade, grid trade) based on local meter readings and price signals.
2.  **High-Level (Inter-Cluster):** Cluster Managers use a **Mean-Field** policy to coordinate bulk energy transfers between clusters, ensuring the overall system remains balanced against grid constraints.


![SolarSys Hierarchical Architecture](assets/solarSys.png)

---

##  Data Generation Framework

To enable large-scale simulation with realistic temporal dynamics, SolarSys includes a **Hierarchical Diffusion Model** for generating synthetic, long-duration energy profiles that maintain both long-term (seasonal/monthly) and short-term (daily/hourly) characteristics.

* **Model:** Hierarchical Diffusion U-Net
* **Input:** Location based housing dataset(eg. Ausgrid dataset, newyork dataset)
* **Output:** High-resolution time series for Grid Usage and Solar Generation (kWh).
![Hierarchical Diffusion Framework](assets/hier_diffusion.png)

To ensure all necessary libraries are available, install dependencies using the provided `requirements.txt` file:

```bash
pip install -r requirements.txt
---
## Repository Structure
The project is organized into core modules and data folders.
```tree
SolarSys_Hugging_face/
├── assets/                             
├── requirements.txt                    # Project Dependencies
├── Dataset/                            # Energy Data (Raw, Processed, and Synthetic)
│   ├── Data/                           
│   │   ├── Ausgrid_processed_data.zip  # Processed data (Australia)
│   │   ├── Ausgrid_raw_data/
│   │   └── Pecan_street_processed_data.zip # Processed data (US)
│   └── Generated_Data.zip              # Example synthetic data (2000 agents)
├── Data_generation_tool_kit/           # Hierarchical Diffusion Model Code
│   ├── Hier_diffusion_energy/
│   │   ├── hierarchial_diffusion_model.py # H-Diffusion U-Net Architecture
│   │   └── global_scaler.gz            # Global normalization scaler, please delete and rerun if had problem
│   ├── dataloader.py                   # 
│   ├── train.py                        # Diffusion model training script
│   └── generate.py                     # Script to generate long-term sequences
├── Models/                             # Trained RL Policies for trial, please retrain your model for best results
│   ├── 100agents_10size/
│   │   └── models/
│   │       ├── inter_ep10000.pth       # High-Level (Inter-Cluster) policy
│   │       └── low_clusterX_ep10000.pth # Low-Level (Intra-Cluster) policies
│   └── (Additional scaling folders)    # e.g., 5agents_5size/, 1000agents_20size/, etc.
├── SolarSys/                           # SolarSys Implementation (MAPPO + MeanField)
│   ├── Environment/
│   │   ├── cluster_env_wrapper.py      # Vectorized environment wrapper
│   │   └── solar_sys_environment.py    # Core Gym environment definition
│   ├── mappo/                          # MAPPO policy definition (Intra-Cluster)
│   │   └── trainer/
│   │       └── mappo.py                # MAPPO algorithm
│   ├── meanfield/                      # MeanField policy definition (Inter-Cluster)
│   │   └── trainer/
│   │       └── meanfield.py            # MeanField algorithm core
│   ├── cluster.py                      # Inter-Cluster Coordinator logic
│   ├── training_freezing.py            # Main SolarSys HRL training script
│   └── cluster_evaluation.py           # Evaluation script for SolarSys
└── Other_algorithms/                   # Baselines 
    ├── HC_MAPPO/                       # Hierarchical Cluster MAPPO 
    │   ├── Environment/
    │   ├── HC_MAPPO_train.py           # Training script for HC-MAPPO
    │   └── HC_MAPPO_evaluation.py      # Evaluation script for HC-MAPPO
    └── Flat_System/                    # Flat Baselines 
        ├── maddpg/                     # Multi-Agent DDPG
        │   ├── maddpg_train.py
        │   └── maddpg_evaluation.py
        ├── mappo/                      # Flat MAPPO
        │   ├── mappo_train.py
        │   └── mappo_evaluation.py
        ├── meanfield/                  # Flat MeanField Actor-Critic (MFAC)
        │   ├── meanfield_train.py
        │   └── meanfield_evaluation.py
        ├── PG/                         # Policy Gradient (PG)
        │   ├── pg_train.py
        │   └── pg_evaluation.py
        └── solar_sys_environment.py    # Copy of the flat environment for baselines