Scientific progress often begins with the difficult task of preparing informed, conclusive reviews of existing research. Since the 1970s, the traditional "subjective" approach to research reviewing in the social sciences has been challenged by a statistical alternative known as meta-analysis. Meta-analysis provides a principled method of distilling reliable generalizations from previous studies on a single topic, thereby providing a quantitative and objective background for future research. The Future of Meta-Analysis brings together expert researchers for an in-depth examination of this new methodology—not to promote a consensus view but rather to explore from several perspectives the theories, tensions, and concerns of meta-analysis, and to illustrate through concrete examples the rationale behind meta-analytic decisions. In a meta-analysis prepared especially for this volume, a statistician and a psychologist review the existing literature on aphasia treatment. In a second study, experts analyze six still-unpublished meta-analyses sponsored by the National Institute of Education to investigate the effects of school desegregation on the academic achievement of black children. This unique case study approach provides valuable discussion of the process of meta-analysis and of the current implications of meta-analysis for policy assessment. Prepared under the auspices of the National Research Council, The Future of Meta-Analysis presents a forum for leaders in this rapidly evolving field to discuss salient conceptual and technical issues and to offer a new theoretical framework, further methodological guidance, and statistical innovations that anticipate a future in which meta-analysis will play an even more effective and valuable role in social science research.
Scientific progress often begins with the difficult task of preparing informed, conclusive reviews of existing research. Since the 1970s, the traditional "subjective" approach to research reviewing in the social sciences has been challenged by a statistical alternative known as meta-analysis. Meta-analysis provides a principled method of distilling reliable generalizations from previous studies on a single topic, thereby providing a quantitative and objective background for future research. The Future of Meta-Analysis brings together expert researchers for an in-depth examination of this new methodology—not to promote a consensus view but rather to explore from several perspectives the theories, tensions, and concerns of meta-analysis, and to illustrate through concrete examples the rationale behind meta-analytic decisions. In a meta-analysis prepared especially for this volume, a statistician and a psychologist review the existing literature on aphasia treatment. In a second study, experts analyze six still-unpublished meta-analyses sponsored by the National Institute of Education to investigate the effects of school desegregation on the academic achievement of black children. This unique case study approach provides valuable discussion of the process of meta-analysis and of the current implications of meta-analysis for policy assessment. Prepared under the auspices of the National Research Council, The Future of Meta-Analysis presents a forum for leaders in this rapidly evolving field to discuss salient conceptual and technical issues and to offer a new theoretical framework, further methodological guidance, and statistical innovations that anticipate a future in which meta-analysis will play an even more effective and valuable role in social science research.
Policymakers, medical practitioners, and the public alike face an increasingly bewildering flood of new and often contradictory scientific studies on almost every topic. Whether the issue is the the best treatment for breast cancer, the need for prenatal food programs to improve the health of poor infants and mothers, or the ability of women to succeed in scientific professions, the healthy growth of modern science has at times done more to stir up controversy than to establish reliable knowledge. But now scientists in several fields have developed a sophisticated new methodology called meta-analysis to address this problem. By numerically combining diverse research findings on a single question, meta-analysis can be used to identify their central tendency and reach conclusions far more reliable than those of any single investigation. How Science Takes Stock vividly tells the story of meta-analysis through the eyes of its architects and champions, and chronicles its history, techniques, achievements, and controversies. Noted science author Morton Hunt visits key practitioners and recounts their use of meta-analysis to resolve important scientific puzzles and longstanding debates. Does psychotherapy work, and if so what form works best? Does spending federal money on education really improve student performance? Can a single enzyme significantly decrease the risk of heart attack? Do boot camps reduce juvenile delinquency? With each account, Hunt illustrates the major components of the meta-analytic method, reveals strategies for resolving practical and theoretical problems, and discusses the impact of meta-analysis on the science and policy communities. In many cases, he demonstrates how meta-analysts have gone a step further to determine the causes of earlier discrepancies. In this way they not only identify successful approaches to the question at hand, but also clarify the conditions under which they will work best. Hunt also portrays the important but frequently controversial business of doing meta-analysis for legislators and government agencies, particularly in sensitive areas of social policy. How Science Takes Stock demonstrates how the statistical techniques of meta-analysis produce more accurate data than the standard literature review or the old-fashioned process of tallying up the results of each scientific study as if they were votes in an election to decide the truth. Hunt also addresses issues of quality control in each phase of the meta-analytic process, and answers skeptics who claim that the dissimilarities between studies are often too significant for meta-analysis to be any more than an apples and oranges approach. This volume conveys the power of meta-analysis to help social policymakers and health professionals resolve their most pressing problems. How Science Takes Stock concludes with a discussion of the future of meta-analysis that examines its potential for further refinements, its growth in the scientific literature, and exciting new possibilities for its future use. An appendix by meta-analysis expert Harris Cooper offers some finer points on the mechanics of conducting a meta-analytic investigation.
Healthcare providers, consumers, researchers and policy makers are inundated with unmanageable amounts of information, including evidence from healthcare research. It has become impossible for all to have the time and resources to find, appraise and interpret this evidence and incorporate it into healthcare decisions. Cochrane Reviews respond to this challenge by identifying, appraising and synthesizing research-based evidence and presenting it in a standardized format, published in The Cochrane Library (www.thecochranelibrary.com). The Cochrane Handbook for Systematic Reviews of Interventions contains methodological guidance for the preparation and maintenance of Cochrane intervention reviews. Written in a clear and accessible format, it is the essential manual for all those preparing, maintaining and reading Cochrane reviews. Many of the principles and methods described here are appropriate for systematic reviews applied to other types of research and to systematic reviews of interventions undertaken by others. It is hoped therefore that this book will be invaluable to all those who want to understand the role of systematic reviews, critically appraise published reviews or perform reviews themselves.
Research synthesis is the practice of systematically distilling and integrating data from many studies in order to draw more reliable conclusions about a given research issue. When the first edition of The Handbook of Research Synthesis and Meta-Analysis was published in 1994, it quickly became the definitive reference for conducting meta-analyses in both the social and behavioral sciences. In the third edition, editors Harris Cooper, Larry Hedges, and Jeff Valentine present updated versions of classic chapters and add new sections that evaluate cutting-edge developments in the field. The Handbook of Research Synthesis and Meta-Analysis draws upon groundbreaking advances that have transformed research synthesis from a narrative craft into an important scientific process in its own right. The editors and leading scholars guide the reader through every stage of the research synthesis process—problem formulation, literature search and evaluation, statistical integration, and report preparation. The Handbook incorporates state-of-the-art techniques from all quantitative synthesis traditions and distills a vast literature to explain the most effective solutions to the problems of quantitative data integration. Among the statistical issues addressed are the synthesis of non-independent data sets, fixed and random effects methods, the performance of sensitivity analyses and model assessments, the development of machine-based abstract screening, the increased use of meta-regression and the problems of missing data. The Handbook also addresses the non-statistical aspects of research synthesis, including searching the literature and developing schemes for gathering information from study reports. Those engaged in research synthesis will find useful advice on how tables, graphs, and narration can foster communication of the results of research syntheses. The third edition of the Handbook provides comprehensive instruction in the skills necessary to conduct research syntheses and represents the premier text on research synthesis. Praise for the first edition: "The Handbook is a comprehensive treatment of literature synthesis and provides practical advice for anyone deep in the throes of, just teetering on the brink of, or attempting to decipher a meta-analysis. Given the expanding application and importance of literature synthesis, understanding both its strengths and weaknesses is essential for its practitioners and consumers. This volume is a good beginning for those who wish to gain that understanding." —Chance "Meta-analysis, as the statistical analysis of a large collection of results from individual studies is called, has now achieved a status of respectability in medicine. This respectability, when combined with the slight hint of mystique that sometimes surrounds meta-analysis, ensures that results of studies that use it are treated with the respect they deserve....The Handbook of Research Synthesis is one of the most important publications in this subject both as a definitive reference book and a practical manual."—British Medical Journal When the first edition of The Handbook of Research Synthesis was published in 1994, it quickly became the definitive reference for researchers conducting meta-analyses of existing research in both the social and biological sciences. In this fully revised second edition, editors Harris Cooper, Larry Hedges, and Jeff Valentine present updated versions of the Handbook's classic chapters, as well as entirely new sections reporting on the most recent, cutting-edge developments in the field. Research synthesis is the practice of systematically distilling and integrating data from a variety of sources in order to draw more reliable conclusions about a given question or topic. The Handbook of Research Synthesis and Meta-Analysis draws upon years of groundbreaking advances that have transformed research synthesis from a narrative craft into an important scientific process in its own right. Cooper, Hedges, and Valentine have assembled leading authorities in the field to guide the reader through every stage of the research synthesis process—problem formulation, literature search and evaluation, statistical integration, and report preparation. The Handbook of Research Synthesis and Meta-Analysis incorporates state-of-the-art techniques from all quantitative synthesis traditions. Distilling a vast technical literature and many informal sources, the Handbook provides a portfolio of the most effective solutions to the problems of quantitative data integration. Among the statistical issues addressed by the authors are the synthesis of non-independent data sets, fixed and random effects methods, the performance of sensitivity analyses and model assessments, and the problem of missing data. The Handbook of Research Synthesis and Meta-Analysis also provides a rich treatment of the non-statistical aspects of research synthesis. Topics include searching the literature, and developing schemes for gathering information from study reports. Those engaged in research synthesis will also find useful advice on how tables, graphs, and narration can be used to provide the most meaningful communication of the results of research synthesis. In addition, the editors address the potentials and limitations of research synthesis, and its future directions. The past decade has been a period of enormous growth in the field of research synthesis. The second edition Handbook thoroughly revises original chapters to assure that the volume remains the most authoritative source of information for researchers undertaking meta-analysis today. In response to the increasing use of research synthesis in the formation of public policy, the second edition includes a new chapter on both the strengths and limitations of research synthesis in policy debates
This book provides a clear and thorough introduction to meta-analysis, the process of synthesizing data from a series of separate studies. Meta-analysis has become a critically important tool in fields as diverse as medicine, pharmacology, epidemiology, education, psychology, business, and ecology. Introduction to Meta-Analysis: Outlines the role of meta-analysis in the research process Shows how to compute effects sizes and treatment effects Explains the fixed-effect and random-effects models for synthesizing data Demonstrates how to assess and interpret variation in effect size across studies Clarifies concepts using text and figures, followed by formulas and examples Explains how to avoid common mistakes in meta-analysis Discusses controversies in meta-analysis Features a web site with additional material and exercises A superb combination of lucid prose and informative graphics, written by four of the world’s leading experts on all aspects of meta-analysis. Borenstein, Hedges, Higgins, and Rothstein provide a refreshing departure from cookbook approaches with their clear explanations of the what and why of meta-analysis. The book is ideal as a course textbook or for self-study. My students, who used pre-publication versions of some of the chapters, raved about the clarity of the explanations and examples. David Rindskopf, Distinguished Professor of Educational Psychology, City University of New York, Graduate School and University Center, & Editor of the Journal of Educational and Behavioral Statistics. The approach taken by Introduction to Meta-analysis is intended to be primarily conceptual, and it is amazingly successful at achieving that goal. The reader can comfortably skip the formulas and still understand their application and underlying motivation. For the more statistically sophisticated reader, the relevant formulas and worked examples provide a superb practical guide to performing a meta-analysis. The book provides an eclectic mix of examples from education, social science, biomedical studies, and even ecology. For anyone considering leading a course in meta-analysis, or pursuing self-directed study, Introduction to Meta-analysis would be a clear first choice. Jesse A. Berlin, ScD Introduction to Meta-Analysis is an excellent resource for novices and experts alike. The book provides a clear and comprehensive presentation of all basic and most advanced approaches to meta-analysis. This book will be referenced for decades. Michael A. McDaniel, Professor of Human Resources and Organizational Behavior, Virginia Commonwealth University
Meta-analysis is a powerful statistical methodology for synthesizing research evidence across independent studies. This is the first comprehensive handbook of meta-analysis written specifically for ecologists and evolutionary biologists, and it provides an invaluable introduction for beginners as well as an up-to-date guide for experienced meta-analysts. The chapters, written by renowned experts, walk readers through every step of meta-analysis, from problem formulation to the presentation of the results. The handbook identifies both the advantages of using meta-analysis for research synthesis and the potential pitfalls and limitations of meta-analysis (including when it should not be used). Different approaches to carrying out a meta-analysis are described, and include moment and least-square, maximum likelihood, and Bayesian approaches, all illustrated using worked examples based on real biological datasets. This one-of-a-kind resource is uniquely tailored to the biological sciences, and will provide an invaluable text for practitioners from graduate students and senior scientists to policymakers in conservation and environmental management. Walks you through every step of carrying out a meta-analysis in ecology and evolutionary biology, from problem formulation to result presentation Brings together experts from a broad range of fields Shows how to avoid, minimize, or resolve pitfalls such as missing data, publication bias, varying data quality, nonindependence of observations, and phylogenetic dependencies among species Helps you choose the right software Draws on numerous examples based on real biological datasets
1. Provides a comprehensive overview of meta-analysis methods and applications. 2. Divided into four major sub-topics, covering univariate meta-analysis, multivariate, applications and policy. 3. Designed to be suitable for graduate students and researchers new to the field. 4. Includes lots of real examples, with data and software code made available. 5. Chapters written by the leading researchers in the field.
Individual Participant Data Meta-Analysis: A Handbook for Healthcare Research provides a comprehensive introduction to the fundamental principles and methods that healthcare researchers need when considering, conducting or using individual participant data (IPD) meta-analysis projects. Written and edited by researchers with substantial experience in the field, the book details key concepts and practical guidance for each stage of an IPD meta-analysis project, alongside illustrated examples and summary learning points. Split into five parts, the book chapters take the reader through the journey from initiating and planning IPD projects to obtaining, checking, and meta-analysing IPD, and appraising and reporting findings. The book initially focuses on the synthesis of IPD from randomised trials to evaluate treatment effects, including the evaluation of participant-level effect modifiers (treatment-covariate interactions). Detailed extension is then made to specialist topics such as diagnostic test accuracy, prognostic factors, risk prediction models, and advanced statistical topics such as multivariate and network meta-analysis, power calculations, and missing data. Intended for a broad audience, the book will enable the reader to: Understand the advantages of the IPD approach and decide when it is needed over a conventional systematic review Recognise the scope, resources and challenges of IPD meta-analysis projects Appreciate the importance of a multi-disciplinary project team and close collaboration with the original study investigators Understand how to obtain, check, manage and harmonise IPD from multiple studies Examine risk of bias (quality) of IPD and minimise potential biases throughout the project Understand fundamental statistical methods for IPD meta-analysis, including two-stage and one-stage approaches (and their differences), and statistical software to implement them Clearly report and disseminate IPD meta-analyses to inform policy, practice and future research Critically appraise existing IPD meta-analysis projects Address specialist topics such as effect modification, multiple correlated outcomes, multiple treatment comparisons, non-linear relationships, test accuracy at multiple thresholds, multiple imputation, and developing and validating clinical prediction models Detailed examples and case studies are provided throughout.
