The first unified treatment of the interface between information theory and emerging topics in data science, written in a clear, tutorial style. Covering topics such as data acquisition, representation, analysis, and communication, it is ideal for graduate students and researchers in information theory, signal processing, and machine learning.
This interdisciplinary text offers theoretical and practical results of information theoretic methods used in statistical learning. It presents a comprehensive overview of the many different methods that have been developed in numerous contexts.
A unique and comprehensive text on the philosophy of model-based data analysis and strategy for the analysis of empirical data. The book introduces information theoretic approaches and focuses critical attention on a priori modeling and the selection of a good approximating model that best represents the inference supported by the data. It contains several new approaches to estimating model selection uncertainty and incorporating selection uncertainty into estimates of precision. An array of examples is given to illustrate various technical issues. The text has been written for biologists and statisticians using models for making inferences from empirical data.
This book presents the elaboration model for the multivariate analysis of observational quantitative data. This model entails the systematic introduction of "third variables" to the analysis of a focal relationship between one independent and one dependent variable to ascertain whether an inference of causality is justified. Two complementary strategies are used: an exclusionary strategy that rules out alternative explanations such as spuriousness and redundancy with competing theories, and an inclusive strategy that connects the focal relationship to a network of other relationships, including the hypothesized causal mechanisms linking the focal independent variable to the focal dependent variable. The primary emphasis is on the translation of theory into a logical analytic strategy and the interpretation of results. The elaboration model is applied with case studies drawn from newly published research that serve as prototypes for aligning theory and the data analytic plan used to test it; these studies are drawn from a wide range of substantive topics in the social sciences, such as emotion management in the workplace, subjective age identification during the transition to adulthood, and the relationship between religious and paranormal beliefs. The second application of the elaboration model is in the form of original data analysis presented in two Analysis Journals that are integrated throughout the text and implement the full elaboration model. Using real data, not contrived examples, the text provides a step-by-step guide through the process of integrating theory with data analysis in order to arrive at meaningful answers to research questions.
Modern statistics deals with large and complex data sets, and consequently with models containing a large number of parameters. This book presents a detailed account of recently developed approaches, including the Lasso and versions of it for various models, boosting methods, undirected graphical modeling, and procedures controlling false positive selections. A special characteristic of the book is that it contains comprehensive mathematical theory on high-dimensional statistics combined with methodology, algorithms and illustrations with real data examples. This in-depth approach highlights the methods’ great potential and practical applicability in a variety of settings. As such, it is a valuable resource for researchers, graduate students and experts in statistics, applied mathematics and computer science.
Information theory and inference, taught together in this exciting textbook, lie at the heart of many important areas of modern technology - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics and cryptography. The book introduces theory in tandem with applications. Information theory is taught alongside practical communication systems such as arithmetic coding for data compression and sparse-graph codes for error-correction. Inference techniques, including message-passing algorithms, Monte Carlo methods and variational approximations, are developed alongside applications to clustering, convolutional codes, independent component analysis, and neural networks. Uniquely, the book covers state-of-the-art error-correcting codes, including low-density-parity-check codes, turbo codes, and digital fountain codes - the twenty-first-century standards for satellite communications, disk drives, and data broadcast. Richly illustrated, filled with worked examples and over 400 exercises, some with detailed solutions, the book is ideal for self-learning, and for undergraduate or graduate courses. It also provides an unparalleled entry point for professionals in areas as diverse as computational biology, financial engineering and machine learning.
Theory and Methods of Statistics covers essential topics for advanced graduate students and professional research statisticians. This comprehensive resource covers many important areas in one manageable volume, including core subjects such as probability theory, mathematical statistics, and linear models, and various special topics, including nonparametrics, curve estimation, multivariate analysis, time series, and resampling. The book presents subjects such as "maximum likelihood and sufficiency," and is written with an intuitive, heuristic approach to build reader comprehension. It also includes many probability inequalities that are not only useful in the context of this text, but also as a resource for investigating convergence of statistical procedures. - Codifies foundational information in many core areas of statistics into a comprehensive and definitive resource - Serves as an excellent text for select master's and PhD programs, as well as a professional reference - Integrates numerous examples to illustrate advanced concepts - Includes many probability inequalities useful for investigating convergence of statistical procedures
Statistical Foundations of Data Science gives a thorough introduction to commonly used statistical models, contemporary statistical machine learning techniques and algorithms, along with their mathematical insights and statistical theories. It aims to serve as a graduate-level textbook and a research monograph on high-dimensional statistics, sparsity and covariance learning, machine learning, and statistical inference. It includes ample exercises that involve both theoretical studies as well as empirical applications. The book begins with an introduction to the stylized features of big data and their impacts on statistical analysis. It then introduces multiple linear regression and expands the techniques of model building via nonparametric regression and kernel tricks. It provides a comprehensive account on sparsity explorations and model selections for multiple regression, generalized linear models, quantile regression, robust regression, hazards regression, among others. High-dimensional inference is also thoroughly addressed and so is feature screening. The book also provides a comprehensive account on high-dimensional covariance estimation, learning latent factors and hidden structures, as well as their applications to statistical estimation, inference, prediction and machine learning problems. It also introduces thoroughly statistical machine learning theory and methods for classification, clustering, and prediction. These include CART, random forests, boosting, support vector machines, clustering algorithms, sparse PCA, and deep learning.
This book comprehensively covers the topic of data science. Data science is an umbrella term that encompasses data analytics, data mining, machine learning, and several other related disciplines. This book synthesizes both fundamental and advanced topics of a research area that has now reached maturity. The chapters of this book are organized into three sections: The first section is an introduction to data science. Starting from the basic concepts, the book will highlight the types of data, its use, its importance and issues that are normally faced in data analytics. Followed by discussion on wide range of applications of data science and widely used techniques in data science. The second section is devoted to the tools and techniques of data science. It consists of data pre-processing, feature selection, classification and clustering concepts as well as an introduction to text mining and opining mining. And finally, the third section of the book focuses on two programming languages commonly used for data science projects i.e. Python and R programming language. Although this book primarily serves as a textbook, it will also appeal to industrial practitioners and researchers due to its focus on applications and references. The book is suitable for both undergraduate and postgraduate students as well as those carrying out research in data science. It can be used as a textbook for undergraduate students in computer science, engineering and mathematics. It can also be accessible to undergraduate students from other areas with the adequate background. The more advanced chapters can be used by postgraduate researchers intending to gather a deeper theoretical understanding.
Originally developed by Claude Shannon in the 1940s, information theory laid the foundations for the digital revolution, and is now an essential tool in telecommunications, genetics, linguistics, brain sciences, and deep space communication. In this richly illustrated book, accessible examples are used to introduce information theory in terms of everyday games like ‘20 questions’ before more advanced topics are explored. Online MatLab and Python computer programs provide hands-on experience of information theory in action, and PowerPoint slides give support for teaching. Written in an informal style, with a comprehensive glossary and tutorial appendices, this text is an ideal primer for novices who wish to learn the essential principles and applications of information theory.
