Informing Federal Policies on Evaluation Methodology: Building the Evidence Base for Method Choice in Government Sponsored Evaluations

Informing Federal Policies on Evaluation Methodology: Building the Evidence Base for Method Choice in Government Sponsored Evaluations

Author: George Julnes

Publisher: Jossey-Bass

Published: 2007-04-20

Total Pages: 168

ISBN-13: 9780787997342

DOWNLOAD EBOOK

One of the most divisive issues in the evaluation community has been the debate over which methodologies are to be considered adequate or commendable in addressing different evaluation questions in different settings. One form of this debate involved opposing camps of proponents of qualitative versus quantitative methods. A decade ago, there was some hope that the two sides of this debate, referred to as the paradigm war, were learning to respect each other. More recently, however, a federal agency priority for funding random assignment experimental studies has reignited the debate. This volume provides a space for a productive dialogue that, by identifying areas of agreement but also fundamental differences, will promote a more durable working consensus on the circumstances in which some methods are to be preferred over others. The chapter authors and discussants make clear that there are different types of evidence with which to inform this dialogue, including empirical findings of the impact of method choice on evaluation outcomes, the evidence contained in the wisdom of practice, and the results of critical analyses of the broader social impacts of method choice. The editors build on these contributions to suggest pragmatic policies for federal agencies, promoting both context-appropriate method choice and the importance of managing portfolios of evaluative research that maintain desired distributions of methodologies. This is the 113th volume of the Jossey-Bass quarterly report series New Directions for Evaluation, a publication of Jossey-Bass and the American Evaluation Association.


Context: A Framework for Its Influence on Evaluation Practice

Context: A Framework for Its Influence on Evaluation Practice

Author: Debra J. Rog

Publisher: John Wiley & Sons

Published: 2012-09-06

Total Pages: 140

ISBN-13: 1118465059

DOWNLOAD EBOOK

Context is a force in evaluation. It shapes our practice, influencing how we approach and design our studies, how we carry them out, and how we report our findings. Context also moderates and mediates the outcomes of the programs and policies we evaluate. This issue focuses squarely on the role that context plays in practice and illuminates its effect on the implementation and outcomes of programs. Exploring the ways in which attending to context may improve the quality of evaluation practice, the contributions span theory, methods, and practice in an effort to move to a more comprehensive conceptualization of context that can guide our work. It: Provides an historical and theoretical view of evaluators’ treatment of context Illustrates how context has influenced evaluation practice Presents a five-area framework for guiding a contextual analysis of evaluations Introduces “context assessment,” which provides a means of integrating context and its implications within the important stages of evaluation. This is the 135th volume of the Jossey-Bass quarterly report series New Directions for Evaluation, an official publication of the American Evaluation Association.


Credible and Actionable Evidence

Credible and Actionable Evidence

Author: Stewart I. Donaldson

Publisher: SAGE Publications

Published: 2014-09-10

Total Pages: 296

ISBN-13: 148335508X

DOWNLOAD EBOOK

Addressing one of the most important and contentious issues challenging applied research and evaluation practice today—what constitutes credible and actionable evidence?—this volume offers a balanced and current context in which to analyze the long-debated quantitative-qualitative paradigms. In the Second Edition, the contributors, a veritable "who’s who" in evaluation, discuss the diversity and changing nature of credible and actionable evidence; offer authoritative guidance about using credible and actionable evidence; explain how to use it to provide rigorous and influential evaluations; and include lessons from their own applied research and evaluation to suggest ways to address the key issues and challenges. Reflecting the latest developments in the field and covering both experimental and non-experimental methods, the new edition includes revised and updated chapters, summaries of strengths and weaknesses across varied approaches, and contains diverse definitions of evidence. Also included are two new chapters on assessing credibility and synthesizing evidence for policy makers. This is a valuable resource for students and others interested in how to best study and evaluate programs, policies, organizations, and other initiatives designed to improve aspects of the human condition and societal well-being.


Prevention, Policy, and Public Health

Prevention, Policy, and Public Health

Author: Amy A. Eyler

Publisher: Oxford University Press

Published: 2016

Total Pages: 409

ISBN-13: 0190224657

DOWNLOAD EBOOK

Prevention, Policy, and Public Health provides a basic foundation for students, professionals, and researchers to be more effective in the policy arena. It offers information on the dynamics of the policymaking process, theoretical frameworks, analysis, and policy applications. It also offers coverage of advocacy and communication, the two most integral aspects of shaping policies for public health.


Impact Evaluation in Practice, Second Edition

Impact Evaluation in Practice, Second Edition

Author: Paul J. Gertler

Publisher: World Bank Publications

Published: 2016-09-12

Total Pages: 444

ISBN-13: 1464807809

DOWNLOAD EBOOK

The second edition of the Impact Evaluation in Practice handbook is a comprehensive and accessible introduction to impact evaluation for policy makers and development practitioners. First published in 2011, it has been used widely across the development and academic communities. The book incorporates real-world examples to present practical guidelines for designing and implementing impact evaluations. Readers will gain an understanding of impact evaluations and the best ways to use them to design evidence-based policies and programs. The updated version covers the newest techniques for evaluating programs and includes state-of-the-art implementation advice, as well as an expanded set of examples and case studies that draw on recent development challenges. It also includes new material on research ethics and partnerships to conduct impact evaluation. The handbook is divided into four sections: Part One discusses what to evaluate and why; Part Two presents the main impact evaluation methods; Part Three addresses how to manage impact evaluations; Part Four reviews impact evaluation sampling and data collection. Case studies illustrate different applications of impact evaluations. The book links to complementary instructional material available online, including an applied case as well as questions and answers. The updated second edition will be a valuable resource for the international development community, universities, and policy makers looking to build better evidence around what works in development.


Environmental Program and Policy Evaluation: Addressing Methodological Challenges

Environmental Program and Policy Evaluation: Addressing Methodological Challenges

Author: Matthew Birnbaum

Publisher: Jossey-Bass

Published: 2009-07-14

Total Pages: 132

ISBN-13:

DOWNLOAD EBOOK

Although environmental policy and program evaluation emerged rather late compared to many other areas of public policy, an energetic evaluation community in the environmental field has emerged during the last decade. This is a community of evaluators with diverse backgrounds in environmental sciences, social sciences, and general evaluation. Evaluation in the environmental field is characterized by complex policies and programs around wicked problems. They exist within complex systems composed of interacting environmental and socioeconomic systems. In furthering the state of evaluation in the environmental field, this issue of focuses on key methodological challenges: time horizons scaling data credibility research designs and counterfactuals Contributors look at each challenge with two chapters, to enhance a pluralistic discourse for development of the theory and practice of environmental evaluation. The authors?from Australia, Europe, and North America?represent the diversity of the community with respect to their formal training, personal experiences, and institutional affiliations. The issue concludes with two commentaries reflecting on the discussions in relation to that of contemporary evaluation in general and a summary of the insights for the future of environmental evaluation. These chapters cumulatively hold promise for furthering the quality of evaluations not only in the environmental field but in other fields as well. This is the 122nd volume of the Jossey-Bass quarterly report series New Directions for Evaluation, an official publication of the American Evaluation Association.


International Perspectives on Evaluation Standards

International Perspectives on Evaluation Standards

Author: Craig Russon

Publisher: Jossey-Bass

Published: 2005-02-11

Total Pages: 112

ISBN-13: 9780787978587

DOWNLOAD EBOOK

Prior to 1995, there were fewer than half a dozen regional and national evaluation organizations around the world. Today there are more than fifty, attesting to a growing interest in the practice of program evaluation internationally. Many of these new organizations have undertaken efforts to develop their own standards or to modify existing sets--most typically, the Program Evaluation Standards of the Joint Committee on Standards for Educational Evaluation--for use in their own cultural context. Following two introductory chapters, one a conceptual overview and the second a history of the development and revisions of the Program Evaluation Standards, this issue documents standards development efforts in three different settings: Western Europe, Africa, and Australasia. In addition, because nongovernmental organizations and governments have entered the standard-setting business, other chapters describe standards development activities by the European Commission and CARE International. The content points to the challenge of formalizing standards for program evaluation given cross-cultural differences in values and to the continuing challenges related to implementing voluntary standards. This is the 104th issue of the quarterly journal New Directions for Evaluation.


Qualitative Research & Evaluation Methods

Qualitative Research & Evaluation Methods

Author: Michael Quinn Patton

Publisher: SAGE Publications

Published: 2014-10-29

Total Pages: 833

ISBN-13: 1483301451

DOWNLOAD EBOOK

Drawing on more than 40 years of experience conducting applied social science research and program evaluation, author Michael Quinn Patton has crafted the most comprehensive and systematic book on qualitative research and evaluation methods, inquiry frameworks, and analysis options available today. Now offering more balance between applied research and evaluation, this Fourth Edition of Qualitative Research & Evaluation Methods illuminates all aspects of qualitative inquiry through new examples, stories, and cartoons; more than a hundred new summarizing and synthesizing exhibits; and a wide range of new highlight sections/sidebars that elaborate on important and emergent issues. For the first time, full case studies are included to illustrate extended research and evaluation examples. In addition, each chapter features an extended "rumination," written in a voice and style more emphatic and engaging than traditional textbook style, about a core issue of persistent debate and controversy.


Program Evaluation Theory and Practice

Program Evaluation Theory and Practice

Author: Donna M. Mertens

Publisher: Guilford Press

Published: 2012-02-20

Total Pages: 642

ISBN-13: 1462503241

DOWNLOAD EBOOK

This engaging text takes an evenhanded approach to major theoretical paradigms in evaluation and builds a bridge from them to evaluation practice. Featuring helpful checklists, procedural steps, provocative questions that invite readers to explore their own theoretical assumptions, and practical exercises, the book provides concrete guidance for conducting large- and small-scale evaluations. Numerous sample studies—many with reflective commentary from the evaluators—reveal the process through which an evaluator incorporates a paradigm into an actual research project. The book shows how theory informs methodological choices (the specifics of planning, implementing, and using evaluations). It offers balanced coverage of quantitative, qualitative, and mixed methods approaches. Useful pedagogical features include: *Examples of large- and small-scale evaluations from multiple disciplines. *Beginning-of-chapter reflection questions that set the stage for the material covered. *"Extending your thinking" questions and practical activities that help readers apply particular theoretical paradigms in their own evaluation projects. *Relevant Web links, including pathways to more details about sampling, data collection, and analysis. *Boxes offering a closer look at key evaluation concepts and additional studies. *Checklists for readers to determine if they have followed recommended practice. *A companion website with resources for further learning.