Competitive Markov Decision Processes

Competitive Markov Decision Processes

Author: Jerzy Filar

Publisher: Springer Science & Business Media

Published: 2012-12-06

Total Pages: 400

ISBN-13: 1461240549

DOWNLOAD EBOOK

This book is intended as a text covering the central concepts and techniques of Competitive Markov Decision Processes. It is an attempt to present a rig orous treatment that combines two significant research topics: Stochastic Games and Markov Decision Processes, which have been studied exten sively, and at times quite independently, by mathematicians, operations researchers, engineers, and economists. Since Markov decision processes can be viewed as a special noncompeti tive case of stochastic games, we introduce the new terminology Competi tive Markov Decision Processes that emphasizes the importance of the link between these two topics and of the properties of the underlying Markov processes. The book is designed to be used either in a classroom or for self-study by a mathematically mature reader. In the Introduction (Chapter 1) we outline a number of advanced undergraduate and graduate courses for which this book could usefully serve as a text. A characteristic feature of competitive Markov decision processes - and one that inspired our long-standing interest - is that they can serve as an "orchestra" containing the "instruments" of much of modern applied (and at times even pure) mathematics. They constitute a topic where the instruments of linear algebra, applied probability, mathematical program ming, analysis, and even algebraic geometry can be "played" sometimes solo and sometimes in harmony to produce either beautifully simple or equally beautiful, but baroque, melodies, that is, theorems.


Stochastic Games and Applications

Stochastic Games and Applications

Author: Abraham Neyman

Publisher: Springer Science & Business Media

Published: 2012-12-06

Total Pages: 466

ISBN-13: 9401001898

DOWNLOAD EBOOK

This volume is based on lectures given at the NATO Advanced Study Institute on "Stochastic Games and Applications," which took place at Stony Brook, NY, USA, July 1999. It gives the editors great pleasure to present it on the occasion of L.S. Shapley's eightieth birthday, and on the fiftieth "birthday" of his seminal paper "Stochastic Games," with which this volume opens. We wish to thank NATO for the grant that made the Institute and this volume possible, and the Center for Game Theory in Economics of the State University of New York at Stony Brook for hosting this event. We also wish to thank the Hebrew University of Jerusalem, Israel, for providing continuing financial support, without which this project would never have been completed. In particular, we are grateful to our editorial assistant Mike Borns, whose work has been indispensable. We also would like to acknowledge the support of the Ecole Poly tech nique, Paris, and the Israel Science Foundation. March 2003 Abraham Neyman and Sylvain Sorin ix STOCHASTIC GAMES L.S. SHAPLEY University of California at Los Angeles Los Angeles, USA 1. Introduction In a stochastic game the play proceeds by steps from position to position, according to transition probabilities controlled jointly by the two players.


Handbook of Markov Decision Processes

Handbook of Markov Decision Processes

Author: Eugene A. Feinberg

Publisher: Springer Science & Business Media

Published: 2012-12-06

Total Pages: 560

ISBN-13: 1461508053

DOWNLOAD EBOOK

Eugene A. Feinberg Adam Shwartz This volume deals with the theory of Markov Decision Processes (MDPs) and their applications. Each chapter was written by a leading expert in the re spective area. The papers cover major research areas and methodologies, and discuss open questions and future research directions. The papers can be read independently, with the basic notation and concepts ofSection 1.2. Most chap ters should be accessible by graduate or advanced undergraduate students in fields of operations research, electrical engineering, and computer science. 1.1 AN OVERVIEW OF MARKOV DECISION PROCESSES The theory of Markov Decision Processes-also known under several other names including sequential stochastic optimization, discrete-time stochastic control, and stochastic dynamic programming-studiessequential optimization ofdiscrete time stochastic systems. The basic object is a discrete-time stochas tic system whose transition mechanism can be controlled over time. Each control policy defines the stochastic process and values of objective functions associated with this process. The goal is to select a "good" control policy. In real life, decisions that humans and computers make on all levels usually have two types ofimpacts: (i) they cost orsavetime, money, or other resources, or they bring revenues, as well as (ii) they have an impact on the future, by influencing the dynamics. In many situations, decisions with the largest immediate profit may not be good in view offuture events. MDPs model this paradigm and provide results on the structure and existence of good policies and on methods for their calculation.


Markov Decision Processes in Practice

Markov Decision Processes in Practice

Author: Richard J. Boucherie

Publisher: Springer

Published: 2017-03-10

Total Pages: 563

ISBN-13: 3319477668

DOWNLOAD EBOOK

This book presents classical Markov Decision Processes (MDP) for real-life applications and optimization. MDP allows users to develop and formally support approximate and simple decision rules, and this book showcases state-of-the-art applications in which MDP was key to the solution approach. The book is divided into six parts. Part 1 is devoted to the state-of-the-art theoretical foundation of MDP, including approximate methods such as policy improvement, successive approximation and infinite state spaces as well as an instructive chapter on Approximate Dynamic Programming. It then continues with five parts of specific and non-exhaustive application areas. Part 2 covers MDP healthcare applications, which includes different screening procedures, appointment scheduling, ambulance scheduling and blood management. Part 3 explores MDP modeling within transportation. This ranges from public to private transportation, from airports and traffic lights to car parking or charging your electric car . Part 4 contains three chapters that illustrates the structure of approximate policies for production or manufacturing structures. In Part 5, communications is highlighted as an important application area for MDP. It includes Gittins indices, down-to-earth call centers and wireless sensor networks. Finally Part 6 is dedicated to financial modeling, offering an instructive review to account for financial portfolios and derivatives under proportional transactional costs. The MDP applications in this book illustrate a variety of both standard and non-standard aspects of MDP modeling and its practical use. This book should appeal to readers for practitioning, academic research and educational purposes, with a background in, among others, operations research, mathematics, computer science, and industrial engineering.


Mathematical and Engineering Methods in Computer Science

Mathematical and Engineering Methods in Computer Science

Author: Antonin Kucera

Publisher: Springer

Published: 2013-01-17

Total Pages: 224

ISBN-13: 3642360467

DOWNLOAD EBOOK

This volume contains the post-proceedings of the 8th Doctoral Workshop on Mathematical and Engineering Methods in Computer Science, MEMICS 2012, held in Znojmo, Czech Republic, in October, 2012. The 13 thoroughly revised papers were carefully selected out of 31 submissions and are presented together with 6 invited papers. The topics covered by the papers include: computer-aided analysis and verification, applications of game theory in computer science, networks and security, modern trends of graph theory in computer science, electronic systems design and testing, and quantum information processing.


Computer Aided Verification

Computer Aided Verification

Author: Sharon Shoham

Publisher: Springer Nature

Published: 2022

Total Pages: 560

ISBN-13: 3031131886

DOWNLOAD EBOOK

This open access two-volume set LNCS 13371 and 13372 constitutes the refereed proceedings of the 34rd International Conference on Computer Aided Verification, CAV 2022, which was held in Haifa, Israel, in August 2022. The 40 full papers presented together with 9 tool papers and 2 case studies were carefully reviewed and selected from 209 submissions. The papers were organized in the following topical sections: Part I: Invited papers; formal methods for probabilistic programs; formal methods for neural networks; software Verification and model checking; hyperproperties and security; formal methods for hardware, cyber-physical, and hybrid systems. Part II: Probabilistic techniques; automata and logic; deductive verification and decision procedures; machine learning; synthesis and concurrency. This is an open access book.


Mathematical Programming And Game Theory For Decision Making

Mathematical Programming And Game Theory For Decision Making

Author: S K Neogy

Publisher: World Scientific

Published: 2008-04-28

Total Pages: 498

ISBN-13: 9814471240

DOWNLOAD EBOOK

This edited book presents recent developments and state-of-the-art review in various areas of mathematical programming and game theory. It is a peer-reviewed research monograph under the ISI Platinum Jubilee Series on Statistical Science and Interdisciplinary Research. This volume provides a panoramic view of theory and the applications of the methods of mathematical programming to problems in statistics, finance, games and electrical networks. It also provides an important as well as timely overview of research trends and focuses on the exciting areas like support vector machines, bilevel programming, interior point method for convex quadratic programming, cooperative games, non-cooperative games and stochastic games. Researchers, professionals and advanced graduates will find the book an essential resource for current work in mathematical programming, game theory and their applications.


Optimization and Related Topics

Optimization and Related Topics

Author: Alexander M. Rubinov

Publisher: Springer Science & Business Media

Published: 2013-04-17

Total Pages: 466

ISBN-13: 147576099X

DOWNLOAD EBOOK

This volume contains, in part, a selection of papers presented at the sixth Australian Optimization Day Miniconference (Ballarat, 16 July 1999), and the Special Sessions on Nonlinear Dynamics and Optimization and Operations Re search - Methods and Applications, which were held in Melbourne, July 11-15 1999 as a part of the Joint Meeting of the American Mathematical Society and Australian Mathematical Society. The editors have strived to present both con tributed papers and survey style papers as a more interesting mix for readers. Some participants from the meetings mentioned above have responded to this approach by preparing survey and 'semi-survey' papers, based on presented lectures. Contributed paper, which contain new and interesting results, are also included. The fields of the presented papers are very large as demonstrated by the following selection of key words from selected papers in this volume: • optimal control, stochastic optimal control, MATLAB, economic models, implicit constraints, Bellman principle, Markov process, decision-making under uncertainty, risk aversion, dynamic programming, optimal value function. • emergent computation, complexity, traveling salesman problem, signal estimation, neural networks, time congestion, teletraffic. • gap functions, nonsmooth variational inequalities, derivative-free algo rithm, Newton's method. • auxiliary function, generalized penalty function, modified Lagrange func tion. • convexity, quasiconvexity, abstract convexity.


Advanced Video Communications over Wireless Networks

Advanced Video Communications over Wireless Networks

Author: Ce Zhu

Publisher: CRC Press

Published: 2017-12-19

Total Pages: 512

ISBN-13: 143988000X

DOWNLOAD EBOOK

Wireless video communications encompass a broad range of issues and opportunities that serve as the catalyst for technical innovations. To disseminate the most recent advances in this challenging yet exciting field, Advanced Video Communications over Wireless Networks provides an in-depth look at the fundamentals, recent technical achievements, challenges, and emerging trends in mobile and wireless video communications. The editors have carefully selected a panel of researchers with expertise in diverse aspects of wireless video communication to cover a wide spectrum of topics, including the underlying theoretical fundamentals associated with wireless video communications, the transmission schemes tailored to mobile and wireless networks, quality metrics, the architectures of practical systems, as well as some novel directions. They address future directions, including Quality-of-Experience in wireless video communications, video communications over future networks, and 3D video communications. The book presents a collection of tutorials, surveys, and original contributions, providing an up-to-date, accessible reference for further development of research and applications in mobile and wireless video communication systems. The range of coverage and depth of expertise make this book the go-to resource for facing current and future challenges in this field.


Implementation and Applications of Automata

Implementation and Applications of Automata

Author: Oscar H. Ibarra

Publisher: Springer Science & Business Media

Published: 2008-07-10

Total Pages: 299

ISBN-13: 354070843X

DOWNLOAD EBOOK

This book constitutes the thoroughly refereed post-proceedings of the 13th International Conference on Implementation and Application of Automata, CIAA 2008, held in San Francisco, USA, in July 2008. The 26 revised full papers togehter with 4 invited papers were carefully reviewed and selected from 40 submissions and have gone through two rounds of reviewing and improvement. The papers cover various topics in the theory, implementation, and applications of automata and related structures.