Data-driven distributionally robust optimization is a recently emerging paradigm aimed at finding a solution that is driven by sample data but is protected against sampling errors. An increasingly popular approach, known as Wasserstein distributionally robust optimization (DRO), achieves this by applying the Wasserstein metric to construct a ball centred at the empirical distribution and finding a solution that performs well against the most adversarial distribution from the ball. In this paper, we present a general framework for studying different choices of a Wasserstein metric and point out the limitation of the existing choices. In particular, while choosing a Wasserstein metric of a higher order is desirable from a data-driven perspective, given its less conservative nature, such a choice comes with a high price from a robustness perspective - it is no longer applicable to many heavy-tailed distributions of practical concern. We show that this seemingly inevitable trade-off can be resolved by our framework, where a new class of Wasserstein metrics, called coherent Wasserstein metrics, is introduced. Like Wasserstein DRO, distributionally robust optimization using the coherent Wasserstein metrics, termed generalized Wasserstein distributionally robust optimization (GW-DRO), has all the desirable performance guarantees: finite-sample guarantee, asymptotic consistency, and computational tractability. The worst-case expectation problem in GW-DRO is in general a nonconvex optimization problem, yet we provide new analysis to prove its tractability without relying on the common duality scheme. Our framework, as shown in this paper, offers a fruitful opportunity to design novel Wasserstein DRO models that can be applied in various contexts such as operations management, finance, and machine learning.
This dissertation deals with the development of fundamental data-driven optimization under uncertainty, including its modeling frameworks, solution algorithms, and a wide variety of applications. Specifically, three research aims are proposed, including data-driven distributionally robust optimization for hedging against distributional uncertainties in energy systems, online learning based receding-horizon optimization that accommodates real-time uncertainty data, and an efficient solution algorithm for solving large-scale data-driven multistage robust optimization problems. There are two distinct research projects under the first research aim. In the first related project, we propose a novel data-driven Wasserstein distributionally robust mixed-integer nonlinear programming model for the optimal biomass with agricultural waste-to-energy network design under uncertainty. A data-driven uncertainty set of feedstock price distributions is devised using the Wasserstein metric. To address computational challenges, we propose a reformulation-based branch-and-refine algorithm. In the second related project, we develop a novel deep learning based distributionally robust joint chance constrained economic dispatch optimization framework for a high penetration of renewable energy. By leveraging a deep generative adversarial network (GAN), an f-divergence-based ambiguity set of wind power distributions is constructed as a ball in the probability space centered at the distribution induced by a generator neural network. To facilitate its solution process, the resulting distributionally robust chance constraints are equivalently reformulated as ambiguity-free chance constraints, which are further tackled using a scenario approach. Additionally, we derive a priori bound on the required number of synthetic wind power data generated by f-GAN to guarantee a predefined risk level. To facilitate large-scale applications, we further develop a prescreening technique to increase computational and memory efficiencies by exploiting problem structure. The second research aim addresses the online learning of real-time uncertainty data for receding-horizon optimization-based control. In the related project, data-driven stochastic model predictive control is proposed for linear time-invariant systems under additive stochastic disturbance, whose probability distribution is unknown but can be partially inferred from real-time disturbance data. The conditional value-at-risk constraints on system states are required to hold for an ambiguity set of disturbance distributions. By leveraging a Dirichlet process mixture model, the first and second-order moment information of each mixture component is incorporated into the ambiguity set. As more data are gathered during the runtime of controller, the ambiguity set is updated based on real-time data. We then develop a novel constraint tightening strategy based on an equivalent reformulation of distributionally robust constraints over the proposed ambiguity set. Additionally, we establish theoretical guarantees on recursive feasibility and closed-loop stability of the proposed model predictive control. The third research aim focuses on algorithm development for data-driven multistage adaptive robust mixed-integer linear programs. In the related project, we propose a multi-to-two transformation theory and develop a novel transformation-proximal bundle algorithm. By partitioning recourse decisions into state and control decisions, affine decision rules are applied exclusively on the state decisions. In this way, the original multistage robust optimization problem is shown to be transformed into an equivalent two-stage robust optimization problem, which is further addressed using a proximal bundle method. The finite convergence of the proposed solution algorithm is guaranteed for the multistage robust optimization problem with a generic uncertainty set. To quantitatively assess solution quality, we further develop a scenario-tree-based lower bounding technique. The effectiveness and advantages of the proposed algorithm are fully demonstrated in inventory control and process network planning.
We study distributionally robust chance-constrained programming (DRCCP) optimization problems with data-driven Wasserstein ambiguity sets. The proposed algorithmic and reformulation framework applies to distributionally robust optimization problems subjected to individual as well as joint chance constraints, with random right-hand side and technology vector, and under two types of uncertainties, called uncertain probabilities and continuum of realizations. For the uncertain probabilities case, we provide new mixed-integer linear programming reformulations for DRCCP problems and derive a set of precedence optimality cuts to strengthen the formulations. For the continuum of realizations case with random right-hand side, we propose an exact mixed-integer second-order cone programming (MISOCP) reformulation and a linear programming (LP) outer approximation. For the continuum of realizations case with random technology vector, we propose two MISOCP and LP outer approximations. We show that all proposed relaxations become exact reformulations when the decision variables are binary or bounded general integers. We evaluate the scalability and tightness of the proposed MISOCP and (MI)LP formulations on a distributionally robust chance constrained knapsack problem.
Robust optimization is still a relatively new approach to optimization problems affected by uncertainty, but it has already proved so useful in real applications that it is difficult to tackle such problems today without considering this powerful methodology. Written by the principal developers of robust optimization, and describing the main achievements of a decade of research, this is the first book to provide a comprehensive and up-to-date account of the subject. Robust optimization is designed to meet some major challenges associated with uncertainty-affected optimization problems: to operate under lack of full information on the nature of uncertainty; to model the problem in a form that can be solved efficiently; and to provide guarantees about the performance of the solution. The book starts with a relatively simple treatment of uncertain linear programming, proceeding with a deep analysis of the interconnections between the construction of appropriate uncertainty sets and the classical chance constraints (probabilistic) approach. It then develops the robust optimization theory for uncertain conic quadratic and semidefinite optimization problems and dynamic (multistage) problems. The theory is supported by numerous examples and computational illustrations. An essential book for anyone working on optimization and decision making under uncertainty, Robust Optimization also makes an ideal graduate textbook on the subject.
Provides a relatively brief introduction to conjugate duality in both finite- and infinite-dimensional problems. An emphasis is placed on the fundamental importance of the concepts of Lagrangian function, saddle-point, and saddle-value. General examples are drawn from nonlinear programming, approximation, stochastic programming, the calculus of variations, and optimal control.
Optimal transport costs include as a special case the so-called Wasserstein distance, which is popular in various statistical applications. The use of optimal transport costs is advantageous relative to the use of divergence-based formulations because the region of distributional uncertainty contains distributions which explore samples outside of the support of the empirical measure, therefore explaining why many machine learning algorithms have the ability to improve generalization. Moreover, the DRO representations that we use to unify the previously mentioned machine learning algorithms, provide a clear interpretation of the so-called regularization parameter, which is known to play a crucial role in controlling generalization error. As we establish, the regularization parameter corresponds exactly to the size of the distributional uncertainty region. Another contribution of this dissertation is the development of statistical methodology to study data-driven DRO formulations based on optimal transport costs.
The availability of big data due to computerization and automation has generated an urgent need for new techniques to analyze and convert big data into useful information and knowledge. Data mining is a promising and leading-edge technology for mining large volumes of data, looking for hidden information, and aiding knowledge discovery. It can be used for characterization, classification, discrimination, anomaly detection, association, clustering, trend or evolution prediction, and much more in fields such as science, medicine, economics, engineering, computers, and even business analytics. This book presents basic concepts, ideas, and research in data mining.
Optimization under uncertainty forms the foundation for many of the fundamental problems the operations research community seeks to solve. In this thesis, we develop and analyze algorithms that incorporate ideas from machine learning to optimize uncertain objectives directly from data. In the first chapter, we consider problems in which the decision affects the observed outcome, such as in personalized medicine and pricing. We present a framework for using observational data to learn to optimize an uncertain objective over a continuous and multi-dimensional decision space. Our approach accounts for the uncertainty in predictions, and we provide theoretical results that show this adds value. In addition, we test our approach on a Warfarin dosing example, and it outperforms the leading alternative methods. In the second chapter, we develop an approach for solving dynamic optimization problems with covariates that uses machine learning to approximate the unknown stochastic process of the uncertainty. We provide theoretical guarantees on the effectiveness of our method and validate the guarantees with computational experiments. In the third chapter, we introduce a distributionally robust approach for incorporating covariates in large-scale, data-driven dynamic optimization. We prove that it is asymptotically optimal and provide a tractable general-purpose approximation scheme that scales to problems with many temporal stages. Across examples in shipment planning, inventory management, and finance, our method achieves improvements of up to 15% over alternatives. In the final chapter, we apply the techniques developed in previous chapters to the problem of optimizing the operating room schedule at a major US hospital. Our partner institution faces significant census variability throughout the week, which limits the amount of patients it can accept due to resource constraints at peak times. We introduce a data-driven approach for this problem that combines machine learning with mixed integer optimization and demonstrate that it can reliably reduce the maximal weekly census.
Artificial intelligence, or AI, now affects the day-to-day life of almost everyone on the planet, and continues to be a perennial hot topic in the news. This book presents the proceedings of ECAI 2023, the 26th European Conference on Artificial Intelligence, and of PAIS 2023, the 12th Conference on Prestigious Applications of Intelligent Systems, held from 30 September to 4 October 2023 and on 3 October 2023 respectively in Kraków, Poland. Since 1974, ECAI has been the premier venue for presenting AI research in Europe, and this annual conference has become the place for researchers and practitioners of AI to discuss the latest trends and challenges in all subfields of AI, and to demonstrate innovative applications and uses of advanced AI technology. ECAI 2023 received 1896 submissions – a record number – of which 1691 were retained for review, ultimately resulting in an acceptance rate of 23%. The 390 papers included here, cover topics including machine learning, natural language processing, multi agent systems, and vision and knowledge representation and reasoning. PAIS 2023 received 17 submissions, of which 10 were accepted after a rigorous review process. Those 10 papers cover topics ranging from fostering better working environments, behavior modeling and citizen science to large language models and neuro-symbolic applications, and are also included here. Presenting a comprehensive overview of current research and developments in AI, the book will be of interest to all those working in the field.
