|
|
--- |
|
|
license: mit |
|
|
tags: |
|
|
- reinforcement-learning |
|
|
- multi-agent |
|
|
- time-series |
|
|
- diffusion-model |
|
|
- energy-management |
|
|
- smart-grid |
|
|
--- |
|
|
# SolarSys: Scalable Hierarchical Coordination for Distributed Solar Energy |
|
|
|
|
|
SolarSys is a novel **Hierarchical Multi-Agent Reinforcement Learning (HRL)** system designed to manage energy storage and peer-to-peer (P2P) trading across large communities of solar-equipped residences. This repository contains the full source code for the SolarSys system, including the trained policies, the custom Gym environment, and the hierarchical diffusion model used for data generation. |
|
|
|
|
|
--- |
|
|
|
|
|
|
|
|
## System Architecture |
|
|
|
|
|
The core of SolarSys is a two-level decision hierarchy: |
|
|
|
|
|
1. **Low-Level (Intra-Cluster):** Individual households use a **MAPPO** agent to make instantaneous decisions (charge, discharge, local P2P trade, grid trade) based on local meter readings and price signals. |
|
|
2. **High-Level (Inter-Cluster):** Cluster Managers use a **Mean-Field** policy to coordinate bulk energy transfers between clusters, ensuring the overall system remains balanced against grid constraints. |
|
|
|
|
|
|
|
|
 |
|
|
|
|
|
--- |
|
|
|
|
|
## Data Generation Framework |
|
|
|
|
|
To enable large-scale simulation with realistic temporal dynamics, SolarSys includes a **Hierarchical Diffusion Model** for generating synthetic, long-duration energy profiles that maintain both long-term (seasonal/monthly) and short-term (daily/hourly) characteristics. |
|
|
|
|
|
* **Model:** Hierarchical Diffusion U-Net |
|
|
* **Input:** Location based housing dataset(eg. Ausgrid dataset, newyork dataset) |
|
|
* **Output:** High-resolution time series for Grid Usage and Solar Generation (kWh). |
|
|
 |
|
|
|
|
|
To ensure all necessary libraries are available, install dependencies using the provided `requirements.txt` file: |
|
|
|
|
|
```bash |
|
|
pip install -r requirements.txt |
|
|
--- |
|
|
|
|
|
## Repository Structure |
|
|
|
|
|
The project is organized into core modules and data folders. |
|
|
|
|
|
```tree |
|
|
SolarSys_Hugging_face/ |
|
|
├── assets/ |
|
|
├── requirements.txt # Project Dependencies |
|
|
├── Dataset/ # Energy Data (Raw, Processed, and Synthetic) |
|
|
│ ├── Data/ |
|
|
│ │ ├── Ausgrid_processed_data.zip # Processed data (Australia) |
|
|
│ │ ├── Ausgrid_raw_data/ |
|
|
│ │ └── Pecan_street_processed_data.zip # Processed data (US) |
|
|
│ └── Generated_Data.zip # Example synthetic data (2000 agents) |
|
|
├── Data_generation_tool_kit/ # Hierarchical Diffusion Model Code |
|
|
│ ├── Hier_diffusion_energy/ |
|
|
│ │ ├── hierarchial_diffusion_model.py # H-Diffusion U-Net Architecture |
|
|
│ │ └── global_scaler.gz # Global normalization scaler, please delete and rerun if had problem |
|
|
│ ├── dataloader.py # |
|
|
│ ├── train.py # Diffusion model training script |
|
|
│ └── generate.py # Script to generate long-term sequences |
|
|
├── Models/ # Trained RL Policies for trial, please retrain your model for best results |
|
|
│ ├── 100agents_10size/ |
|
|
│ │ └── models/ |
|
|
│ │ ├── inter_ep10000.pth # High-Level (Inter-Cluster) policy |
|
|
│ │ └── low_clusterX_ep10000.pth # Low-Level (Intra-Cluster) policies |
|
|
│ └── (Additional scaling folders) # e.g., 5agents_5size/, 1000agents_20size/, etc. |
|
|
├── SolarSys/ # SolarSys Implementation (MAPPO + MeanField) |
|
|
│ ├── Environment/ |
|
|
│ │ ├── cluster_env_wrapper.py # Vectorized environment wrapper |
|
|
│ │ └── solar_sys_environment.py # Core Gym environment definition |
|
|
│ ├── mappo/ # MAPPO policy definition (Intra-Cluster) |
|
|
│ │ └── trainer/ |
|
|
│ │ └── mappo.py # MAPPO algorithm |
|
|
│ ├── meanfield/ # MeanField policy definition (Inter-Cluster) |
|
|
│ │ └── trainer/ |
|
|
│ │ └── meanfield.py # MeanField algorithm core |
|
|
│ ├── cluster.py # Inter-Cluster Coordinator logic |
|
|
│ ├── training_freezing.py # Main SolarSys HRL training script |
|
|
│ └── cluster_evaluation.py # Evaluation script for SolarSys |
|
|
└── Other_algorithms/ # Baselines |
|
|
├── HC_MAPPO/ # Hierarchical Cluster MAPPO |
|
|
│ ├── Environment/ |
|
|
│ ├── HC_MAPPO_train.py # Training script for HC-MAPPO |
|
|
│ └── HC_MAPPO_evaluation.py # Evaluation script for HC-MAPPO |
|
|
└── Flat_System/ # Flat Baselines |
|
|
├── maddpg/ # Multi-Agent DDPG |
|
|
│ ├── maddpg_train.py |
|
|
│ └── maddpg_evaluation.py |
|
|
├── mappo/ # Flat MAPPO |
|
|
│ ├── mappo_train.py |
|
|
│ └── mappo_evaluation.py |
|
|
├── meanfield/ # Flat MeanField Actor-Critic (MFAC) |
|
|
│ ├── meanfield_train.py |
|
|
│ └── meanfield_evaluation.py |
|
|
├── PG/ # Policy Gradient (PG) |
|
|
│ ├── pg_train.py |
|
|
│ └── pg_evaluation.py |
|
|
└── solar_sys_environment.py # Copy of the flat environment for baselines |
