The "CSE Program Evaluation Kit" is a series of nine books intended to assist people conducting program evaluations. This volume, fifth in the kit, discusses the role and importance of implementation evaluation and presents methods for designing, using, and reporting the results using assessment instruments to describe accurately how a program looks in operation. Step-by-step guides are provided for program records, observations, and self-reports. The following chapters are included: (1) "Assessing Program Implementation: An Overview"; (2) "Initial Planning: Deciding What To Measure"; (3) "Planning for Data Collection"; (4) "Methods for Assessing Program Implementation: Program Records"; (5) "Methods for Assessing Program Implementation: Questionnaires and Interviews"; (6)"Methods for Assessing Program Implementation: Observations"; and (7) "Summarizing, Analyzing, and Reporting Your Data." An appendix contains questions for an implementation evaluation. (Contains 8 tables, 15 figures, and 10 references.) (SLD)
This book confronts the widespread impression that policy or program implementation should be easy, arguing instead that implementation, even under the best of circumstances, is exceedingly difficult. Using the Oakland Project as a case study, this book discusses each stage of the process of implementation, demonstrating that completion of what might seem to be a simple sequence of events will in fact depend on a complex chain of reciprocal interactions. Each part of the chain must be built with the others in view, so the separation of policy design from implementation is fatal. The first four chapters illustrate the movement from simplicity to complexity. Chapter 5 discusses the number of decision points throughout the process, giving an indication of the magnitude of the task. Chapter 6 examines why project targets may be set even if they are unlikely to be met, considering both the position of those who set targets -- top federal officials who wish large accomplishments from small resources in a short time -- and those who must implement them -- career bureaucrats and local participants characterized by high needs and low cohesion. The last chapter discusses the relationship between the evaluation of programs and the study of their implementation, arguing that tendencies to assimilate the two should be resisted.
Quantitative Studies in Social Relations: Social Program Implementation provides information pertinent to implementation problems in social policy areas. This book discusses the greatest difficulty in devising better social programs. Organized into five parts encompassing 12 chapters, this book begins with an overview of implementation problems in federally funded programs. This text then focuses on the social agency where the many bureaucratic problems in multilayered structure clearly illustrate crucial implementation issues. Other chapters consider the structure and incentive systems of public schools as they relate to the adoption of innovations and to their implementation in the schools. This book discusses as well the idea of planned variation from the perspective of a model sponsor. The final chapter deals with crucial impediment to improving complex operating programs, policy analysis, and experimentation in social policy areas. This book is a valuable resource for policy planners, administrators, and analysts. Researchers engaged in social policy studies will also find this book useful.
Abstract curricular program implementation in the context of randomized field trials Gloria Isabel Miller This study examined three cases of commercially available curricular program implementations to determine if a unified approach to measuring the level of implementation was possible (proof of concept). Further, the study investigated whether the level of curriculum and implementation plan specificity made a difference to the strength of implementation achieved in classrooms; and described the implementation evolution in different contexts. The study sample consists of a total of 163 teachers in eight school districts across the United States. In each case teachers were randomly assigned to using the curricular innovation or their currently used materials and processes. The three cases, HS-Math, NewScience, and MathIntervention, were purposely chosen to represent three different points of curricular and implementation specificity and two different subject areas, math and science. Each case features a commercially available program that also had opportunities for teachers to use "electronic" technology to enhance their learning or to engage their students. The cases represent differing student grade levels. The cases are different enough to provide a range that exercises the measurement techniques introduced in this study so results can begin to generalize across curricular programs and grades. However, the cases are similar enough in research design, instrumentation, and data collection methods to make them comparable. A key contribution of this investigation is the creation of a framework to measure the level of implementation (the extent to which the teacher and students display the actions, behaviors, and interactions expected by using the innovation). The unified conceptual framework arrived at by using an Activity Theory perspective together with the analytical methods employed provide a way to view the rich complex interaction of implementation as a system with the larger system of the school organization. Data from the analysis revealed that variations in the level of implementation were no different regardless of the level of specificity. A strong finding of this work is that implementation evolves slowly even when the curricular program is scripted and coaching support is provided to teachers. The paper concludes with implications for policy and future research.
Program Implementation in Preventive Trials shows you how you can take a more active part in program evaluation and how you can direct existing programs toward new horizons of more effective service. In this concise, focused look at community-based psychology and its operative programs, you’ll see how and why community programs should be comprehensively evaluated. You’ll see the importance of understanding how interventions were conducted before making conclusions about a program’s impact, and you’ll discover why there’s an ever-widening gap between what is planned and what actually gets implemented in community-based programs.In short, Program Implementation in Preventive Trials helps you see the increasing need for the assessment of implementation, the “active” side of community psychology. You’ll gain instant understanding as to why there’s a need for constant monitoring of a program’s use, and you’ll find answers to the following questions that continue to plague community psychologists who are interested in implementing programs of change: Does the current personnel follow the implementation program? Will new members to the setting understand and utilize the procedures developed for that setting? Will the procedures be modified such that their utility decreases? Specifically, you’ll read about: how to ensure intervention programs are conducted as planned why implementation data should be collected what protocol compliance is and its role in treatment programs when to be flexible so modifications can be made in program procedures who can enhance program adherence by “buying in” to a multi-change agent approach where cultural sensitivity helps programs be more faithfully adopted and conductedIf you’re a scholar or a student interested in studying the fundamental issue of implementation, you’ll definitely want to see what these professionals have compiled in Program Implementation in Preventive Trials. You’ll find that your program agenda, however beneficial it is now, will only be raised and elevated to a new level of performance by the positive examples and research carefully collected here.
The Amer. Recovery and Reinvestment Act of 2009 (Recovery Act) -- initially estimated to cost $787 billion in spending and tax provisions -- aims to promote economic recovery, make investments, and minimize or avoid reductions in state and local government services. The Recovery Act provided the Dept. of Energy (DoE) more than $43.2 billion, including $36.7 billion for projects and activities and $6.5 billion in borrowing authority, in areas such as energy efficiency and renewable energy, nuclear waste clean-up, and electric grid modernization. This testimony discusses: (1) the extent to which DoE has obligated and spent its Recovery Act funds; and (2) the factors that have affected DoE's ability to select and start Recovery Act projects. Illus.
