File size: 3,534 Bytes
0a8d192 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 |
EVChargeEnv: An OpenEnv Benchmark for EV Charging Optimization
1. Motivation
As AI agents move from static prediction to acting autonomously in dynamic environments, we need richer environments than toy grids and games. One domain that naturally combines uncertainty, long-horizon planning, and multi-objective decision-making is electric vehicle (EV) charging.
EVChargeEnv is my contribution to the OpenEnv Challenge. It simulates a simplified but realistic EV charging process where an agent must decide how much to charge at each timestep while adapting to fluctuating electricity prices and grid load.
The core objective is:
Reach full battery while minimizing cost and avoiding grid overload.
This makes EVChargeEnv a clean and interpretable environment that still contains meaningful complexity.
---
2. Environment Design
EVChargeEnv exposes a continuous-control RL task with a 4-dimensional state:
- charge_level ∈ [0, 1] – battery state of charge
- price ∈ [0, 1] – dynamic energy price
- grid_load ∈ [0, 1] – current grid stress/instability
- time_step_norm ∈ [0, 1] – normalized timestep
Agents output a continuous charging rate between 0 and 1.
Scenarios
To test robustness, EVChargeEnv includes three difficulty modes:
- easy – smooth price/load curves and short episodes
- medium – balanced volatility (default scenario)
- hard – noisy price/load dynamics and slower charging
---
3. Reward Function
The reward balances several competing factors:
- progress_reward – reward for increasing charge
– cost_penalty – charging during high prices costs more
– overload_penalty – charging when grid load is high is discouraged
– time_penalty – each step costs a tiny penalty to encourage faster execution
This encourages agents to:
- charge aggressively during low-price, low-load periods
- slow or stop charging during peak price/load
- finish charging efficiently
---
4. Implementation and OpenEnv Integration
The environment is implemented using gymnasium and structured to reflect OpenEnv specifications. Key files include:
- env/ev_charge_env.py – main environment logic
- openenv.yaml – metadata describing observation/action spaces, rewards, and termination criteria
- run_evaluation.py – produces standardized JSON outputs for assessment
Example output:
{
"avg_reward": -1.23,
"avg_steps": 31.2,
"episodes": 5
}
Published on the Hugging Face Hub:
https://huggingface.co/oozan/EVChargeEnv
---
5. Baseline Agents and Training
Baselines included:
Random Baseline – ignores price/load.
Price-Aware Baseline – charges more when price is low.
PyTorch Policy-Gradient Agent – small neural model trained with REINFORCE.
The learned agent shows:
- improved reward
- sensible patterns
- adaptation to medium scenario
---
6. Running the Environment
Install:
pip install -r requirements.txt
Evaluate:
python run_evaluation.py
Heuristic baseline:
python run_price_aware_evaluation.py
Train agent:
python train_evchargeenv_pg.py
Notebook also included for Colab execution.
---
7. Future Improvements
Possible extensions:
- Day/night pricing cycles
- Renewable energy influence
- Emergency events / blackouts
- Battery degradation modeling
- PPO/SAC/LLM-based agents
- Visualization tools
---
Final Thoughts
EVChargeEnv is:
- simple
- realistic
- modular
- OpenEnv-compliant
It provides a practical environment for research on planning and resource optimization under uncertainty.
Repo:
https://huggingface.co/oozan/EVChargeEnv
|