The electric power system is undergoing dramatic transformations due to the emergence of renewable resources and demand-side revolutions. However, in order to face the increasing level of system complexity and uncertainty, we need to come up with algorithms that are able to operate the power grid in a safe, reliable and sustainable manner. Such algorithms need to be compatible with infrastructure advancements as well as societal needs. We argue in this thesis that data-driven methods combined with physical knowledge and optimization theories provide a set of powerful tools that can achieve performance guarantees in a diverse set of energy system applications.

This dissertation covers the uncertainty modeling, optimal control, and decision-making for power systems with high penetration of renewables. Leveraging the availability of sensing, actuation and data platforms, we are able to construct specifically-design data-driven algorithms by incorporating domain knowledge such as time series characterization, function convexity, and optimization problem structures. In particular, we present three algorithmic contributions in this work:​ we present a generative model to forecast possible future scenarios of renewable generators;​ we illustrate an optimal control framework which is built upon on input convex neural network;​ we formulate a machine learning task for the optimal power flow problem, and we come up with a generalizable and robust learning-based optimization solver to enable real-time decision-making under the environment of stochastic electricity demand. The development of such algorithms is built upon the foundation of control, optimization and machine learning theories.

In addition to these algorithmic contributions, a central focus of this thesis is the application of such data-driven tools in many challenging energy system tasks, while extending the state of the art. The developed methods have enabled more robust electric grid planning and energy storage operations. We also apply the neural network controller to voltage regulation and building energy management tasks, where in both domains the underlying systems are hard to model, and our approaches demonstrate the possibility of achieving optimal control with only sensing data available.