Handbook of Markov Decision Processes

Handbook of Markov Decision Processes

Author: Eugene A. Feinberg

Publisher: Springer Science & Business Media

Published: 2012-12-06

Total Pages: 560

ISBN-13: 1461508053

DOWNLOAD EBOOK

Eugene A. Feinberg Adam Shwartz This volume deals with the theory of Markov Decision Processes (MDPs) and their applications. Each chapter was written by a leading expert in the re spective area. The papers cover major research areas and methodologies, and discuss open questions and future research directions. The papers can be read independently, with the basic notation and concepts ofSection 1.2. Most chap ters should be accessible by graduate or advanced undergraduate students in fields of operations research, electrical engineering, and computer science. 1.1 AN OVERVIEW OF MARKOV DECISION PROCESSES The theory of Markov Decision Processes-also known under several other names including sequential stochastic optimization, discrete-time stochastic control, and stochastic dynamic programming-studiessequential optimization ofdiscrete time stochastic systems. The basic object is a discrete-time stochas tic system whose transition mechanism can be controlled over time. Each control policy defines the stochastic process and values of objective functions associated with this process. The goal is to select a "good" control policy. In real life, decisions that humans and computers make on all levels usually have two types ofimpacts: (i) they cost orsavetime, money, or other resources, or they bring revenues, as well as (ii) they have an impact on the future, by influencing the dynamics. In many situations, decisions with the largest immediate profit may not be good in view offuture events. MDPs model this paradigm and provide results on the structure and existence of good policies and on methods for their calculation.


Water Policy and Management

Water Policy and Management

Author: Darrell G. Fontane

Publisher:

Published: 1994

Total Pages: 936

ISBN-13:

DOWNLOAD EBOOK

This collection contains 219 papers presented at the 21st Annual Conference on Water Resources Planning and Management, held in Denver, Colorado, May 23-26, 1994.


Approximate Dynamic Programming

Approximate Dynamic Programming

Author: Warren B. Powell

Publisher: John Wiley & Sons

Published: 2007-10-05

Total Pages: 487

ISBN-13: 0470182954

DOWNLOAD EBOOK

A complete and accessible introduction to the real-world applications of approximate dynamic programming With the growing levels of sophistication in modern-day operations, it is vital for practitioners to understand how to approach, model, and solve complex industrial problems. Approximate Dynamic Programming is a result of the author's decades of experience working in large industrial settings to develop practical and high-quality solutions to problems that involve making decisions in the presence of uncertainty. This groundbreaking book uniquely integrates four distinct disciplines—Markov design processes, mathematical programming, simulation, and statistics—to demonstrate how to successfully model and solve a wide range of real-life problems using the techniques of approximate dynamic programming (ADP). The reader is introduced to the three curses of dimensionality that impact complex problems and is also shown how the post-decision state variable allows for the use of classical algorithmic strategies from operations research to treat complex stochastic optimization problems. Designed as an introduction and assuming no prior training in dynamic programming of any form, Approximate Dynamic Programming contains dozens of algorithms that are intended to serve as a starting point in the design of practical solutions for real problems. The book provides detailed coverage of implementation challenges including: modeling complex sequential decision processes under uncertainty, identifying robust policies, designing and estimating value function approximations, choosing effective stepsize rules, and resolving convergence issues. With a focus on modeling and algorithms in conjunction with the language of mainstream operations research, artificial intelligence, and control theory, Approximate Dynamic Programming: Models complex, high-dimensional problems in a natural and practical way, which draws on years of industrial projects Introduces and emphasizes the power of estimating a value function around the post-decision state, allowing solution algorithms to be broken down into three fundamental steps: classical simulation, classical optimization, and classical statistics Presents a thorough discussion of recursive estimation, including fundamental theory and a number of issues that arise in the development of practical algorithms Offers a variety of methods for approximating dynamic programs that have appeared in previous literature, but that have never been presented in the coherent format of a book Motivated by examples from modern-day operations research, Approximate Dynamic Programming is an accessible introduction to dynamic modeling and is also a valuable guide for the development of high-quality solutions to problems that exist in operations research and engineering. The clear and precise presentation of the material makes this an appropriate text for advanced undergraduate and beginning graduate courses, while also serving as a reference for researchers and practitioners. A companion Web site is available for readers, which includes additional exercises, solutions to exercises, and data sets to reinforce the book's main concepts.


Statistics in Industry

Statistics in Industry

Author: Ravindra Khattree

Publisher: Gulf Professional Publishing

Published: 2003-07-18

Total Pages: 1222

ISBN-13: 9780444506146

DOWNLOAD EBOOK

This volume presents an exposition of topics in industrial statistics. It serves as a reference for researchers in industrial statistics/industrial engineering and a source of information for practicing statisticians/industrial engineers. A variety of topics in the areas of industrial process monitoring, industrial experimentation, industrial modelling and data analysis are covered and are authored by leading researchers or practitioners in the particular specialized topic. Targeting the audiences of researchers in academia as well as practitioners and consultants in industry, the book provides comprehensive accounts of the relevant topics. In addition, whenever applicable ample data analytic illustrations are provided with the help of real world data.