Covers the optimism in the face of uncertainty principle applied to large scale optimization problems under finite numerical budget. The initial motivation for this research originated from the empirical success of the Monte-Carlo Tree Search method popularized in Computer Go and further extended to other games, optimization, and planning problems.
Faced with the challenge of solving the hard optimization problems that abound in the real world, existing methods often encounter great difficulties. Important applications in business, engineering or economics cannot be tackled by the techniques that have formed the predominant focus of academic research throughout the past three decades. Exact and heuristic approaches are dramatically changing our ability to solve problems of practical significance and are extending the frontier of problems that can be handled effectively. This monograph details state-of-the-art optimization methods, both exact and heuristic, for the LOP. The authors employ the LOP to illustrate contemporary optimization technologies as well as how to design successful implementations of exact and heuristic procedures. Therefore, they do not limit the scope of this book to the LOP, but on the contrary, provide the reader with the background and practical strategies in optimization to tackle different combinatorial problems.
This book constitutes the thoroughly refereed post-proceedings of the 5th International Conference on Computers and Games, CG 2006, co-located with the 14th World Computer-Chess Championship and the 11th Computer Olympiad. The 24 revised papers cover all aspects of artificial intelligence in computer-game playing. Topics addressed are evaluation and learning, search, combinatorial games and theory opening and endgame databases, single-agent search and planning, and computer Go.
The significantly expanded and updated new edition of a widely used text on reinforcement learning, one of the most active research areas in artificial intelligence. Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives while interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the field's key ideas and algorithms. This second edition has been significantly expanded and updated, presenting new topics and updating coverage of other topics. Like the first edition, this second edition focuses on core online learning algorithms, with the more mathematical material set off in shaded boxes. Part I covers as much of reinforcement learning as possible without going beyond the tabular case for which exact solutions can be found. Many algorithms presented in this part are new to the second edition, including UCB, Expected Sarsa, and Double Learning. Part II extends these ideas to function approximation, with new sections on such topics as artificial neural networks and the Fourier basis, and offers expanded treatment of off-policy learning and policy-gradient methods. Part III has new chapters on reinforcement learning's relationships to psychology and neuroscience, as well as an updated case-studies chapter including AlphaGo and AlphaGo Zero, Atari game playing, and IBM Watson's wagering strategy. The final chapter discusses the future societal impacts of reinforcement learning.
This book constitutes the refereed proceedings of the First Workshop on Monte Carlo Search, MCS 2020, organized in conjunction with IJCAI 2020. The event was supposed to take place in Yokohama, Japan, in July 2020, but due to the Covid-19 pandemic was held virtually on January 7, 2021. The 9 full papers of the specialized project were carefully reviewed and selected from 15 submissions. The following topics are covered in the contributions: discrete mathematics in computer science, games, optimization, search algorithms, Monte Carlo methods, neural networks, reinforcement learning, machine learning.
This book comprises selected research papers from the 2015 edition of the EVOLVE conference, which was held on June 18–June 24, 2015 in Iași, Romania. It presents the latest research on Probability, Set Oriented Numerics, and Evolutionary Computation. The aim of the EVOLVE conference was to provide a bridge between probability, set oriented numerics and evolutionary computation and to bring together experts from these disciplines. The broad focus of the EVOLVE conference made it possible to discuss the connection between these related fields of study computational science. The selected papers published in the proceedings book were peer reviewed by an international committee of reviewers (at least three reviews per paper) and were revised and enhanced by the authors after the conference. The contributions are categorized into five major parts, which are: Multicriteria and Set-Oriented Optimization; Evolution in ICT Security; Computational Game Theory; Theory on Evolutionary Computation; Applications of Evolutionary Algorithms. The 2015 edition shows a major progress in the aim to bring disciplines together and the research on a number of topics that have been discussed in previous editions of the conference matured over time and methods have found their ways in applications. In this sense the book can be considered an important milestone in bridging and thereby advancing state-of-the-art computational methods.
Research on general video game playing aims at designing agents or content generators that can perform well in multiple video games, possibly without knowing the game in advance and with little to no specific domain knowledge. The general video game AI framework and competition propose a challenge in which researchers can test their favorite AI methods with a potentially infinite number of games created using the Video Game Description Language. The open-source framework has been used since 2014 for running a challenge. Competitors around the globe submit their best approaches that aim to generalize well across games. Additionally, the framework has been used in AI modules by many higher-education institutions as assignments, or as proposed projects for final year (undergraduate and Master's) students and Ph.D. candidates. The present book, written by the developers and organizers of the framework, presents the most interesting highlights of the research performed by the authors during these years in this domain. It showcases work on methods to play the games, generators of content, and video game optimization. It also outlines potential further work in an area that offers multiple research directions for the future.
Decision-making in the face of uncertainty is a significant challenge in machine learning, and the multi-armed bandit model is a commonly used framework to address it. This comprehensive and rigorous introduction to the multi-armed bandit problem examines all the major settings, including stochastic, adversarial, and Bayesian frameworks. A focus on both mathematical intuition and carefully worked proofs makes this an excellent reference for established researchers and a helpful resource for graduate students in computer science, engineering, statistics, applied mathematics and economics. Linear bandits receive special attention as one of the most useful models in applications, while other chapters are dedicated to combinatorial bandits, ranking, non-stationary problems, Thompson sampling and pure exploration. The book ends with a peek into the world beyond bandits with an introduction to partial monitoring and learning in Markov decision processes.
Multi-armed bandits is a rich, multi-disciplinary area that has been studied since 1933, with a surge of activity in the past 10-15 years. This is the first book to provide a textbook like treatment of the subject.