IEEE ICAL 2009 Plenary Talk #4

Professor Warren B. Powell

Department of Operations Research and Financial Engineering

Princeton University, USA

 

Approximate Dynamic Programming

for High-Dimensional Resource Allocation Problems

Abstract

Managing fleets of trucks, locomotives, business jets, vaccines, medical resources and energy all share something in common: they are all high-dimensional stochastic resource allocation problems. They can be formulated as dynamic programs with state, information and action variables with thousands or even millions of dimensions, a characteristic we refer to as the "three curses of dimensionality." Classical techniques in approximate dynamic programming solve only part of the problem. We show how the use of the post-decision state variable allows us to break a problem into three distinct components: simulation, deterministic optimization and statistical learning. This strategy decomposes problems over time, allowing us to use commercial optimization packages to solve the problem at each point in time using value function approximations to produce good solutions over time. The challenge is designing and estimating value function approximations. I will describe a simple method that works well for a wide range of problems, and discuss some of the algorithmic challenges that we are still facing. I will describe several applications of ADP in transportation and energy policy modeling.
 

Biography

A faculty member at Princeton University since 1981, Professor Powell specializes in stochastic optimization problems arising in a variety of resource allocation problems, with applications encompassing freight transportation, military operations, energy resource management, health and finance. He founded and currently heads CASTLE Laboratory within the Department of Operations Research and Financial Engineering at Princeton where he has developed advanced optimization models and algorithms for some of the largest freight transportation companies in the country.

His research includes fundamental contributions to stochastic optimization, computational advances in the solution of large-scale stochastic optimization problems, and the formulation and solution of complex resource allocation problems arising in transportation. He has also recently begun a new line of research in optimal learning that addresses the challenges of collecting information in an efficient way. These problems arise in drug discovery and medical testing, business, R&D portfolio optimization, engineering design, optimization of expensive simulations, and biosurveillance, to name a few.

He has authored or coauthored over 140 refereed publications, and he is the author of Approximate Dynamic Programming: Solving the curses of dimensionality, published by John Wiley and Sons. This research has led to the first stochastic, multiscale model for energy policy analysis, and his techniques have been used in applications in finance and health. His work in freight transportation has spawned two consulting firms, and he was twice a finalist in the Edelman competition. A recipient of the Informs Fellows Award, Professor Powell has served in a variety of editorial and administrative positions for Informs, including Informs Board of Directors, Area Editor for Operations Research, President of the Transportation Science Section, and numerous prize and administrative committees.
 

 

Website updated on July 04, 2009. (c) IEEE ICAL 2009 conference.