A Caltech Library Service

Learning-based Predictive Control via Real-time Aggregate Flexibility

Li, Tongxin and Sun, Bo and Chen, Yue and Ye, Zixin and Low, Steven H. and Wierman, Adam (2021) Learning-based Predictive Control via Real-time Aggregate Flexibility. IEEE Transactions on Smart Grid . ISSN 1949-3053. doi:10.1109/TSG.2021.3094719. (In Press)

[img] PDF - Accepted Version
See Usage Policy.

[img] PDF - Submitted Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


Aggregators have emerged as crucial tools for the coordination of distributed, controllable loads. To be used effectively, an aggregator must be able to communicate the available flexibility of the loads they control, as known as the aggregate flexibility to a system operator. However, most of existing aggregate flexibility measures often are slow-timescale estimations and much less attention has been paid to real-time coordination between an aggregator and an operator. In this paper, we consider solving an online optimization in a closed-loop system and present a design of real-time aggregate flexibility feedback, termed the maximum entropy feedback (MEF). In addition to deriving analytic properties of the MEF, combining learning and control, we show that it can be approximated using reinforcement learning and used as a penalty term in a novel control algorithm – the penalized predictive control (PPC), which modifies vanilla model predictive control (MPC). The benefits of our scheme are (1). Efficient Communication. An operator running PPC does not need to know the exact states and constraints of the loads, but only the MEF. (2). Fast Computation. The PPC often has much less number of variables than an MPC formulation. (3). Lower Costs We show that under certain regularity assumptions, the PPC is optimal. We illustrate the efficacy of the PPC using a dataset from an adaptive electric vehicle charging network and show that PPC outperforms classical MPC.

Item Type:Article
Related URLs:
URLURL TypeDescription Paper
Low, Steven H.0000-0001-6476-3048
Alternate Title:Real-time Aggregate Flexibility via Reinforcement Learning
Additional Information:© 2021 IEEE. Tongxin Li and Steven Low acknowledge the support received from National Science Foundation (NSF) through grants CCF 1637598, ECCS 1931662 and CPS ECCS 1932611. Bo Sun is supported by Hong Kong Research Grant Council (RGC) General Research Fund (Project 16207318). Adam Wierman’s research is funded by NSF (AitF-1637598 and CNS-1518941), Amazon AWS and VMware.
Funding AgencyGrant Number
Hong Kong Research Grant Council16207318
Amazon Web ServicesUNSPECIFIED
Subject Keywords:Aggregate flexibility, closed-loop control systems, online optimization, model predictive control, reinforcement learning, electric vehicle charging
Record Number:CaltechAUTHORS:20210510-084600512
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:109022
Deposited By: Tony Diaz
Deposited On:10 May 2021 17:56
Last Modified:02 Aug 2021 16:45

Repository Staff Only: item control page