This work is essentially an extensive revision of my Ph.D. dissertation, [1J. It 1S primarily a research document on the application of probability theory to the parameter estimation problem. The people who will be interested in this material are physicists, economists, and engineers who have to deal with data on a daily basis; consequently, we have included a great deal of introductory and tutorial material. Any person with the equivalent of the mathematics background required for the graduate level study of physics should be able to follow the material contained in this book, though not without eIfort. From the time the dissertation was written until now (approximately one year) our understanding of the parameter estimation problem has changed extensively. We have tried to incorporate what we have learned into this book. I am indebted to a number of people who have aided me in preparing this docu ment: Dr. C. Ray Smith, Steve Finney, Juana Sunchez, Matthew Self, and Dr. Pat Gibbons who acted as readers and editors. In addition, I must extend my deepest thanks to Dr. Joseph Ackerman for his support during the time this manuscript was being prepared.
This book is primarily a research document on the application of probability theory to the parameter estimation problem. The people who will be interested in this material are physicists, chemists, economists, and engineers who have to deal with data on a daily basis; consequently, we have included a great deal of introductory and tutorial material. Any person with the equivalent of the mathematics background required for the graduate-level study of physics should be able to follow the material contained in this book, though not without effort. In this work we apply probability theory to the problem of estimating parameters in rather general models. In particular when the model consists of a single stationary sinusoid we show that the direct application of probability theory will yield frequency estimates an order of magnitude better than a discrete Fourier transform in signal-to-noise of one. Latter, we generalize the problem and show that probability theory can separate two close frequencies long after the peaks in a discrete Fourier transform have merged.
The first comprehensive development of Bayesian Bounds for parameter estimation and nonlinear filtering/tracking Bayesian estimation plays a central role in many signal processing problems encountered in radar, sonar, communications, seismology, and medical diagnosis. There are often highly nonlinear problems for which analytic evaluation of the exact performance is intractable. A widely used technique is to find bounds on the performance of any estimator and compare the performance of various estimators to these bounds. This book provides a comprehensive overview of the state of the art in Bayesian Bounds. It addresses two related problems: the estimation of multiple parameters based on noisy measurements and the estimation of random processes, either continuous or discrete, based on noisy measurements. An extensive introductory chapter provides an overview of Bayesian estimation and the interrelationship and applicability of the various Bayesian Bounds for both static parameters and random processes. It provides the context for the collection of papers that are included. This book will serve as a comprehensive reference for engineers and statisticians interested in both theory and application. It is also suitable as a text for a graduate seminar or as a supplementary reference for an estimation theory course.
This volume has its origin in the Fifth, Sixth and Seventh Workshops on and Bayesian Methods in Applied Statistics", held at "Maximum-Entropy the University of Wyoming, August 5-8, 1985, and at Seattle University, August 5-8, 1986, and August 4-7, 1987. It was anticipated that the proceedings of these workshops would be combined, so most of the papers were not collected until after the seventh workshop. Because all of the papers in this volume are on foundations, it is believed that the con tents of this volume will be of lasting interest to the Bayesian community. The workshop was organized to bring together researchers from different fields to critically examine maximum-entropy and Bayesian methods in science and engineering as well as other disciplines. Some of the papers were chosen specifically to kindle interest in new areas that may offer new tools or insight to the reader or to stimulate work on pressing problems that appear to be ideally suited to the maximum-entropy or Bayesian method. A few papers presented at the workshops are not included in these proceedings, but a number of additional papers not presented at the workshop are included. In particular, we are delighted to make available Professor E. T. Jaynes' unpublished Stanford University Microwave Laboratory Report No. 421 "How Does the Brain Do Plausible Reasoning?" (dated August 1957). This is a beautiful, detailed tutorial on the Cox-Polya-Jaynes approach to Bayesian probability theory and the maximum-entropy principle.
This volume has its origin in the third ·Workshop on Maximum-Entropy and Bayesian Methods in Applied Statistics,· held at the University of Wyoming, August 1 to 4, 1983. It was anticipated that the proceedings of this workshop could not be prepared in a timely fashion, so most of the papers were not collected until a year or so ago. Because most of the papers are in the nature of advancing theory or solving specific problems, as opposed to status reports, it is believed that the contents of this volume will be of lasting interest to the Bayesian community. The workshop was organized to bring together researchers from differ ent fields to examine critically maximum-entropy and Bayesian methods in science, engineering, medicine, economics, and other disciplines. Some of the papers were chosen specifically to kindle interest in new areas that may offer new tools or insight to the reader or to stimulate work on pressing problems that appear to be ideally suited to the maximum-entropy or Bayes ian method.
In this richly illustrated book, the tutorial style of writing, combined with a comprehensive glossary, makes this an ideal primer for the novice who wishes to become familiar with the basic principles of Bayesian analysis.
Praise for Bayes Rules!: An Introduction to Applied Bayesian Modeling “A thoughtful and entertaining book, and a great way to get started with Bayesian analysis.” Andrew Gelman, Columbia University “The examples are modern, and even many frequentist intro books ignore important topics (like the great p-value debate) that the authors address. The focus on simulation for understanding is excellent.” Amy Herring, Duke University “I sincerely believe that a generation of students will cite this book as inspiration for their use of – and love for – Bayesian statistics. The narrative holds the reader’s attention and flows naturally – almost conversationally. Put simply, this is perhaps the most engaging introductory statistics textbook I have ever read. [It] is a natural choice for an introductory undergraduate course in applied Bayesian statistics." Yue Jiang, Duke University “This is by far the best book I’ve seen on how to (and how to teach students to) do Bayesian modeling and understand the underlying mathematics and computation. The authors build intuition and scaffold ideas expertly, using interesting real case studies, insightful graphics, and clear explanations. The scope of this book is vast – from basic building blocks to hierarchical modeling, but the authors’ thoughtful organization allows the reader to navigate this journey smoothly. And impressively, by the end of the book, one can run sophisticated Bayesian models and actually understand the whys, whats, and hows.” Paul Roback, St. Olaf College “The authors provide a compelling, integrated, accessible, and non-religious introduction to statistical modeling using a Bayesian approach. They outline a principled approach that features computational implementations and model assessment with ethical implications interwoven throughout. Students and instructors will find the conceptual and computational exercises to be fresh and engaging.” Nicholas Horton, Amherst College An engaging, sophisticated, and fun introduction to the field of Bayesian statistics, Bayes Rules!: An Introduction to Applied Bayesian Modeling brings the power of modern Bayesian thinking, modeling, and computing to a broad audience. In particular, the book is an ideal resource for advanced undergraduate statistics students and practitioners with comparable experience. Bayes Rules! empowers readers to weave Bayesian approaches into their everyday practice. Discussions and applications are data driven. A natural progression from fundamental to multivariable, hierarchical models emphasizes a practical and generalizable model building process. The evaluation of these Bayesian models reflects the fact that a data analysis does not exist in a vacuum. Features • Utilizes data-driven examples and exercises. • Emphasizes the iterative model building and evaluation process. • Surveys an interconnected range of multivariable regression and classification models. • Presents fundamental Markov chain Monte Carlo simulation. • Integrates R code, including RStan modeling tools and the bayesrules package. • Encourages readers to tap into their intuition and learn by doing. • Provides a friendly and inclusive introduction to technical Bayesian concepts. • Supports Bayesian applications with foundational Bayesian theory.
Bayesian inference provides a simple and unified approach to data analysis, allowing experimenters to assign probabilities to competing hypotheses of interest, on the basis of the current state of knowledge. By incorporating relevant prior information, it can sometimes improve model parameter estimates by many orders of magnitude. This book provides a clear exposition of the underlying concepts with many worked examples and problem sets. It also discusses implementation, including an introduction to Markov chain Monte-Carlo integration and linear and nonlinear model fitting. Particularly extensive coverage of spectral analysis (detecting and measuring periodic signals) includes a self-contained introduction to Fourier and discrete Fourier methods. There is a chapter devoted to Bayesian inference with Poisson sampling, and three chapters on frequentist methods help to bridge the gap between the frequentist and Bayesian approaches. Supporting Mathematica® notebooks with solutions to selected problems, additional worked examples, and a Mathematica tutorial are available at www.cambridge.org/9780521150125.
Now in its third edition, this classic book is widely considered the leading text on Bayesian methods, lauded for its accessible, practical approach to analyzing data and solving research problems. Bayesian Data Analysis, Third Edition continues to take an applied approach to analysis using up-to-date Bayesian methods. The authors—all leaders in the statistics community—introduce basic concepts from a data-analytic perspective before presenting advanced methods. Throughout the text, numerous worked examples drawn from real applications and research emphasize the use of Bayesian inference in practice. New to the Third Edition Four new chapters on nonparametric modeling Coverage of weakly informative priors and boundary-avoiding priors Updated discussion of cross-validation and predictive information criteria Improved convergence monitoring and effective sample size calculations for iterative simulation Presentations of Hamiltonian Monte Carlo, variational Bayes, and expectation propagation New and revised software code The book can be used in three different ways. For undergraduate students, it introduces Bayesian inference starting from first principles. For graduate students, the text presents effective current approaches to Bayesian modeling and computation in statistics and related fields. For researchers, it provides an assortment of Bayesian methods in applied statistics. Additional materials, including data sets used in the examples, solutions to selected exercises, and software instructions, are available on the book’s web page.
This book is a guide to the practical application of statistics in data analysis as typically encountered in the physical sciences. It is primarily addressed at students and professionals who need to draw quantitative conclusions from experimental data. Although most of the examples are taken from particle physics, the material is presented in a sufficiently general way as to be useful to people from most branches of the physical sciences. The first part of the book describes the basic tools of data analysis: concepts of probability and random variables, Monte Carlo techniques, statistical tests, and methods of parameter estimation. The last three chapters are somewhat more specialized than those preceding, covering interval estimation, characteristic functions, and the problem of correcting distributions for the effects of measurement errors (unfolding).
