---
license: mit
tags:
- reinforcement-learning
- multi-agent
- time-series
- diffusion-model
- energy-management
- smart-grid
---
# SolarSys: Scalable Hierarchical Coordination for Distributed Solar Energy

SolarSys is a novel **Hierarchical Multi-Agent Reinforcement Learning (HRL)** system designed to manage energy storage and peer-to-peer (P2P) trading across large communities of solar-equipped residences. This repository contains the full source code for the SolarSys system, including the trained policies, the custom Gym environment, and the hierarchical diffusion model used for data generation.

---


## System Architecture

The core of SolarSys is a two-level decision hierarchy:

1.  **Low-Level (Intra-Cluster):** Individual households use a **MAPPO** agent to make instantaneous decisions (charge, discharge, local P2P trade, grid trade) based on local meter readings and price signals.
2.  **High-Level (Inter-Cluster):** Cluster Managers use a **Mean-Field** policy to coordinate bulk energy transfers between clusters, ensuring the overall system remains balanced against grid constraints.


![SolarSys Hierarchical Architecture](assets/solarSys.png)

---

##  Data Generation Framework

To enable large-scale simulation with realistic temporal dynamics, SolarSys includes a **Hierarchical Diffusion Model** for generating synthetic, long-duration energy profiles that maintain both long-term (seasonal/monthly) and short-term (daily/hourly) characteristics.

* **Model:** Hierarchical Diffusion U-Net
* **Input:** Location based housing dataset(eg. Ausgrid dataset, newyork dataset)
* **Output:** High-resolution time series for Grid Usage and Solar Generation (kWh).
![Hierarchical Diffusion Framework](assets/hier_diffusion.png)

To ensure all necessary libraries are available, install dependencies using the provided `requirements.txt` file:

```bash
pip install -r requirements.txt
---

## Repository Structure

The project is organized into core modules and data folders.

```tree
SolarSys_Hugging_face/
├── assets/                             
├── requirements.txt                    # Project Dependencies
├── Dataset/                            # Energy Data (Raw, Processed, and Synthetic)
│   ├── Data/                           
│   │   ├── Ausgrid_processed_data.zip  # Processed data (Australia)
│   │   ├── Ausgrid_raw_data/
│   │   └── Pecan_street_processed_data.zip # Processed data (US)
│   └── Generated_Data.zip              # Example synthetic data (2000 agents)
├── Data_generation_tool_kit/           # Hierarchical Diffusion Model Code
│   ├── Hier_diffusion_energy/
│   │   ├── hierarchial_diffusion_model.py # H-Diffusion U-Net Architecture
│   │   └── global_scaler.gz            # Global normalization scaler, please delete and rerun if had problem
│   ├── dataloader.py                   # 
│   ├── train.py                        # Diffusion model training script
│   └── generate.py                     # Script to generate long-term sequences
├── Models/                             # Trained RL Policies for trial, please retrain your model for best results
│   ├── 100agents_10size/
│   │   └── models/
│   │       ├── inter_ep10000.pth       # High-Level (Inter-Cluster) policy
│   │       └── low_clusterX_ep10000.pth # Low-Level (Intra-Cluster) policies
│   └── (Additional scaling folders)    # e.g., 5agents_5size/, 1000agents_20size/, etc.
├── SolarSys/                           # SolarSys Implementation (MAPPO + MeanField)
│   ├── Environment/
│   │   ├── cluster_env_wrapper.py      # Vectorized environment wrapper
│   │   └── solar_sys_environment.py    # Core Gym environment definition
│   ├── mappo/                          # MAPPO policy definition (Intra-Cluster)
│   │   └── trainer/
│   │       └── mappo.py                # MAPPO algorithm
│   ├── meanfield/                      # MeanField policy definition (Inter-Cluster)
│   │   └── trainer/
│   │       └── meanfield.py            # MeanField algorithm core
│   ├── cluster.py                      # Inter-Cluster Coordinator logic
│   ├── training_freezing.py            # Main SolarSys HRL training script
│   └── cluster_evaluation.py           # Evaluation script for SolarSys
└── Other_algorithms/                   # Baselines 
    ├── HC_MAPPO/                       # Hierarchical Cluster MAPPO 
    │   ├── Environment/
    │   ├── HC_MAPPO_train.py           # Training script for HC-MAPPO
    │   └── HC_MAPPO_evaluation.py      # Evaluation script for HC-MAPPO
    └── Flat_System/                    # Flat Baselines 
        ├── maddpg/                     # Multi-Agent DDPG
        │   ├── maddpg_train.py
        │   └── maddpg_evaluation.py
        ├── mappo/                      # Flat MAPPO
        │   ├── mappo_train.py
        │   └── mappo_evaluation.py
        ├── meanfield/                  # Flat MeanField Actor-Critic (MFAC)
        │   ├── meanfield_train.py
        │   └── meanfield_evaluation.py
        ├── PG/                         # Policy Gradient (PG)
        │   ├── pg_train.py
        │   └── pg_evaluation.py
        └── solar_sys_environment.py    # Copy of the flat environment for baselines