Explainable and Interpretable Reinforcement Learning for Robotics
Author: Aaron M. Roth
Publisher: Springer Nature
Published:
Total Pages: 123
ISBN-13: 3031475186
DOWNLOAD EBOOKPosts
Robot Learning from Human Demonstration
Author: Sonia Dechter
Publisher: Springer Nature
Published: 2022-06-01
Total Pages: 109
ISBN-13: 3031015703
DOWNLOAD EBOOKLearning from Demonstration (LfD) explores techniques for learning a task policy from examples provided by a human teacher. The field of LfD has grown into an extensive body of literature over the past 30 years, with a wide variety of approaches for encoding human demonstrations and modeling skills and tasks. Additionally, we have recently seen a focus on gathering data from non-expert human teachers (i.e., domain experts but not robotics experts). In this book, we provide an introduction to the field with a focus on the unique technical challenges associated with designing robots that learn from naive human teachers. We begin, in the introduction, with a unification of the various terminology seen in the literature as well as an outline of the design choices one has in designing an LfD system. Chapter 2 gives a brief survey of the psychology literature that provides insights from human social learning that are relevant to designing robotic social learners. Chapter 3 walks through an LfD interaction, surveying the design choices one makes and state of the art approaches in prior work. First, is the choice of input, how the human teacher interacts with the robot to provide demonstrations. Next, is the choice of modeling technique. Currently, there is a dichotomy in the field between approaches that model low-level motor skills and those that model high-level tasks composed of primitive actions. We devote a chapter to each of these. Chapter 7 is devoted to interactive and active learning approaches that allow the robot to refine an existing task model. And finally, Chapter 8 provides best practices for evaluation of LfD systems, with a focus on how to approach experiments with human subjects in this domain.
Interpretable Machine Learning
Author: Christoph Molnar
Publisher: Lulu.com
Published: 2020
Total Pages: 320
ISBN-13: 0244768528
DOWNLOAD EBOOKThis book is about making machine learning models and their decisions interpretable. After exploring the concepts of interpretability, you will learn about simple, interpretable models such as decision trees, decision rules and linear regression. Later chapters focus on general model-agnostic methods for interpreting black box models like feature importance and accumulated local effects and explaining individual predictions with Shapley values and LIME. All interpretation methods are explained in depth and discussed critically. How do they work under the hood? What are their strengths and weaknesses? How can their outputs be interpreted? This book will enable you to select and correctly apply the interpretation method that is most suitable for your machine learning project.
Explainable and Interpretable Models in Computer Vision and Machine Learning
Author: Hugo Jair Escalante
Publisher: Springer
Published: 2018-11-29
Total Pages: 305
ISBN-13: 3319981315
DOWNLOAD EBOOKThis book compiles leading research on the development of explainable and interpretable machine learning methods in the context of computer vision and machine learning. Research progress in computer vision and pattern recognition has led to a variety of modeling techniques with almost human-like performance. Although these models have obtained astounding results, they are limited in their explainability and interpretability: what is the rationale behind the decision made? what in the model structure explains its functioning? Hence, while good performance is a critical required characteristic for learning machines, explainability and interpretability capabilities are needed to take learning machines to the next step to include them in decision support systems involving human supervision. This book, written by leading international researchers, addresses key topics of explainability and interpretability, including the following: · Evaluation and Generalization in Interpretable Machine Learning · Explanation Methods in Deep Learning · Learning Functional Causal Models with Generative Neural Networks · Learning Interpreatable Rules for Multi-Label Classification · Structuring Neural Networks for More Explainable Predictions · Generating Post Hoc Rationales of Deep Visual Classification Decisions · Ensembling Visual Explanations · Explainable Deep Driving by Visualizing Causal Attention · Interdisciplinary Perspective on Algorithmic Job Candidate Search · Multimodal Personality Trait Analysis for Explainable Modeling of Job Interview Decisions · Inherent Explainability Pattern Theory-based Video Event Interpretations
Explainable Agency in Artificial Intelligence
Author: Silvia Tulli
Publisher: CRC Press
Published: 2024-01-22
Total Pages: 121
ISBN-13: 1003802923
DOWNLOAD EBOOKThis book focuses on a subtopic of explainable AI (XAI) called explainable agency (EA), which involves producing records of decisions made during an agent’s reasoning, summarizing its behavior in human-accessible terms, and providing answers to questions about specific choices and the reasons for them. We distinguish explainable agency from interpretable machine learning (IML), another branch of XAI that focuses on providing insight (typically, for an ML expert) concerning a learned model and its decisions. In contrast, explainable agency typically involves a broader set of AI-enabled techniques, systems, and stakeholders (e.g., end users), where the explanations provided by EA agents are best evaluated in the context of human subject studies. The chapters of this book explore the concept of endowing intelligent agents with explainable agency, which is crucial for agents to be trusted by humans in critical domains such as finance, self-driving vehicles, and military operations. This book presents the work of researchers from a variety of perspectives and describes challenges, recent research results, lessons learned from applications, and recommendations for future research directions in EA. The historical perspectives of explainable agency and the importance of interactivity in explainable systems are also discussed. Ultimately, this book aims to contribute to the successful partnership between humans and AI systems. Features: • Contributes to the topic of explainable artificial intelligence (XAI) • Focuses on the XAI subtopic of explainable agency • Includes an introductory chapter, a survey, and five other original contributions
Machine Learning Under a Modern Optimization Lens
Author: Dimitris Bertsimas
Publisher:
Published: 2019
Total Pages: 589
ISBN-13: 9781733788502
DOWNLOAD EBOOKAlgorithms for Reinforcement Learning
Author: Csaba Grossi
Publisher: Springer Nature
Published: 2022-05-31
Total Pages: 89
ISBN-13: 3031015517
DOWNLOAD EBOOKReinforcement learning is a learning paradigm concerned with learning to control a system so as to maximize a numerical performance measure that expresses a long-term objective. What distinguishes reinforcement learning from supervised learning is that only partial feedback is given to the learner about the learner's predictions. Further, the predictions may have long term effects through influencing the future state of the controlled system. Thus, time plays a special role. The goal in reinforcement learning is to develop efficient learning algorithms, as well as to understand the algorithms' merits and limitations. Reinforcement learning is of great interest because of the large number of practical applications that it can be used to address, ranging from problems in artificial intelligence to operations research or control engineering. In this book, we focus on those algorithms of reinforcement learning that build on the powerful theory of dynamic programming. We give a fairly comprehensive catalog of learning problems, describe the core ideas, note a large number of state of the art algorithms, followed by the discussion of their theoretical properties and limitations. Table of Contents: Markov Decision Processes / Value Prediction Problems / Control / For Further Exploration
A Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence
Author: Nikos Kolobov
Publisher: Springer Nature
Published: 2022-06-01
Total Pages: 71
ISBN-13: 3031015436
DOWNLOAD EBOOKMultiagent systems is an expanding field that blends classical fields like game theory and decentralized control with modern fields like computer science and machine learning. This monograph provides a concise introduction to the subject, covering the theoretical foundations as well as more recent developments in a coherent and readable manner. The text is centered on the concept of an agent as decision maker. Chapter 1 is a short introduction to the field of multiagent systems. Chapter 2 covers the basic theory of singleagent decision making under uncertainty. Chapter 3 is a brief introduction to game theory, explaining classical concepts like Nash equilibrium. Chapter 4 deals with the fundamental problem of coordinating a team of collaborative agents. Chapter 5 studies the problem of multiagent reasoning and decision making under partial observability. Chapter 6 focuses on the design of protocols that are stable against manipulations by self-interested agents. Chapter 7 provides a short introduction to the rapidly expanding field of multiagent reinforcement learning. The material can be used for teaching a half-semester course on multiagent systems covering, roughly, one chapter per lecture.
Machine and Deep Learning Algorithms and Applications
Author: Uday Shankar
Publisher: Springer Nature
Published: 2022-05-31
Total Pages: 107
ISBN-13: 3031037588
DOWNLOAD EBOOKThis book introduces basic machine learning concepts and applications for a broad audience that includes students, faculty, and industry practitioners. We begin by describing how machine learning provides capabilities to computers and embedded systems to learn from data. A typical machine learning algorithm involves training, and generally the performance of a machine learning model improves with more training data. Deep learning is a sub-area of machine learning that involves extensive use of layers of artificial neural networks typically trained on massive amounts of data. Machine and deep learning methods are often used in contemporary data science tasks to address the growing data sets and detect, cluster, and classify data patterns. Although machine learning commercial interest has grown relatively recently, the roots of machine learning go back to decades ago. We note that nearly all organizations, including industry, government, defense, and health, are using machine learning to address a variety of needs and applications. The machine learning paradigms presented can be broadly divided into the following three categories: supervised learning, unsupervised learning, and semi-supervised learning. Supervised learning algorithms focus on learning a mapping function, and they are trained with supervision on labeled data. Supervised learning is further sub-divided into classification and regression algorithms. Unsupervised learning typically does not have access to ground truth, and often the goal is to learn or uncover the hidden pattern in the data. Through semi-supervised learning, one can effectively utilize a large volume of unlabeled data and a limited amount of labeled data to improve machine learning model performances. Deep learning and neural networks are also covered in this book. Deep neural networks have attracted a lot of interest during the last ten years due to the availability of graphics processing units (GPU) computational power, big data, and new software platforms. They have strong capabilities in terms of learning complex mapping functions for different types of data. We organize the book as follows. The book starts by introducing concepts in supervised, unsupervised, and semi-supervised learning. Several algorithms and their inner workings are presented within these three categories. We then continue with a brief introduction to artificial neural network algorithms and their properties. In addition, we cover an array of applications and provide extensive bibliography. The book ends with a summary of the key machine learning concepts.
Representation Analysis of Deep Reinforcement Learning Algorithms in Robotic Environments
Author: Mehran Taghian Jazi
Publisher:
Published: 2022
Total Pages: 0
ISBN-13:
DOWNLOAD EBOOKThe rise of Deep Learning (DL) and its assistance in learning complex feature representations significantly impacted Reinforcement Learning (RL). Deep Reinforcement Learning (DRL) made it possible to apply RL to complex real-world problems and even achieve human-level performance. One of these problems is related to robotics. Recently, DRL agents successfully learned optimal behavior in a range of robotic environments. The policy can provide much information from its learned representation. However, this policy is approximated using a neural network and, therefore, is a black box. Explainable Artificial Intelligence (XAI) is a new AI subfield focusing on interpreting Machine Learning models' behavior. A large part of XAI's literature has emerged on feature relevance techniques to explain a deep neural network (DNN) output processing on images. These techniques have been extended to explain Graph classification tasks using Graph Networks (GN). Nevertheless, these methods haven't been exploited to analyze the DRL agent's behavior learned to perform in a robotic environment. In this work, we proposed to analyze the representation learned by a DRL agent's policy in a robotic environment. We use graph structure to represent the robot's observation in an entity-relationship manner and graph neural networks as function approximators in DRL. For the interpretation phase, an explainability technique called Layer-wise Relevance Propagation (LRP), a feature relevance technique that had been successfully applied to explain image and graph classification tasks, is used to interpret the learned policy. We evaluate the information provided by the LRP on two simulated robotic environments on MuJoCo. The experiments and evaluation methods were delicately designed to effectively measure the value of knowledge gained by our approach to analyzing learned representations in the Deep Reinforcement Learning task.
