The Independent Evaluation Group (IEG) of the World Bank has undertaken impact evaluations of the Bank's support to irrigation in Andhra Pradesh, India (under AP Irrigation II and III), and of the U.K. Department for International Development supported Rural Livelihoods Project (RLP). This is one of a series of IEG impact evaluations (see appendix H). IEG's program of impact evaluation is in part carried out under a Department for International Development-IEG partnership agreement; hence the focus on RLP. However, survey villages are also covered by the Bank supported DPIP project, so that the findings are also relevant to this project.
The report assesses the World Bank Group?s support for growth and productivity in the agriculture sector. Enhancing agricultural growth and productivity is essential to meeting the worldwide demand for food and to reducing poverty, particularly in the poorest developing countries. Between 1998 and 2008, the period covered by this evaluation, the World Bank Group (WBG) provided $23.7 billion in financing for agriculture and agribusiness in 108 countries (roughly 8 percent of total WBG financing), spanning areas from irrigation and marketing to research and extension. However, this was a time of declining focus on agricultural growth and productivity by both countries and donors. The cost of inadequate attention to agriculture, especially in agriculture-based economies, came into focus with the food crisis of 2007-08. The crisis added momentum to an emerging renewal of attention and stepped-up financing to agriculture and agribusiness at the World Bank and International Finance Corporation (IFC), as well as at several multilateral and bilateral agencies. World Bank financing rose two and a half times from 2008 to 2009, though that increase in lending seems to have been accompanied by a decline in analytical work, which this review finds valuable for results. This evaluation seeks to provide lessons from successes and failures to help improve the development impact of the renewed attention to the sector. Ratings against the World Bank?s stated objectives and IFC?s market-based benchmarks for agriculture and agribusiness projects have been equal to or above portfolio averages in East Asia, Latin America, and the transition economies in Europe, with notable successes over a long period in China and India. But performance of WBG interventions has been well below average in Sub-Saharan Africa, where IFC has had little engagement in agribusiness. Inconsistent client commitment and weak capacity have limited the effectiveness of WBG support in agriculture-based economies, particularly in Sub-Saharan Africa, and constraints on staffing and internal coordination within the WBG have also hurt outcomes. Financial sustainability has been constrained by insufficient government funding and the difficulty of maintaining agricultural services and infrastructure. The WBG has a unique opportunity to match the increases in the financing for agriculture with sharper focus on improving agricultural growth and productivity in agriculture-based economies, notably in Sub-Saharan Africa. Greater effort will be needed to connect sectoral interventions and achieve synergies from public and private sector interventions; to build capacity and knowledge exchange; to take stock of experience in rain-fed agriculture; to ensure attention to financial sustainability and to cross-cutting issues of gender, environmental and social impacts, and climate; and to better integrate WBG support at the global and regional levels with that at the country level.
This book explains how access to and use of land, water and language helped shape Andhra politics in India from 1850 down to the present day. After independence, the debate over land reform and policies on irrigation has shaped the fortunes of various governments, while the debate over the make-up of the language-based state has stimulated separatist movements like the one in support of Telangana. The book discusses how British innovations in irrigation in coastal Andhra in the mid-nineteenth century transformed the economy there from food crops to cash crops, and created new markets for local entrepreneurs. This stimulated increased education and social reform in the region, which in turn supported new politics in search of constitutional concessions. The drive for a Telugu language-based province then arose in concert, and those political resources were then used to determine local patterns down to independence. The 1930s ruse of the socialists, then the communist organisations, was an extension of land and water tax debates, which impacted the political nature of development — both before and after — independence. This is one of the first books on Andhra that recounts this story and is based on extensive archival research exploring the deep relationships between land, water, language and politics. It would be of primary interest to those studying modern nationalism in India, natural resource management, Indian politics and economic growth.
Evaluation Summary What Can We Learn from Nutrition Impact Evaluations? High levels of child malnutrition in developing countries contribute to mortality and have long-term consequences for children s cognitive development and earnings as adults. Recent impact evaluations show that many different interventions have had an impact on children s anthropometric outcomes (height, weight, and birth weight), but there is no simple answer to the question What works? to address the problem. Similar interventions have widely different results in different settings, owing to differences in local context, the causes and severity of malnutrition, and the capacity for program implementation. Impact evaluations of programs supported by the Bank, which are generally large-scale, complex inter-ventions in low-capacity settings, show equally variable results. The findings confirm that it should not be assumed that an intervention found effective in a randomized medical setting will have the same effects when implemented under field conditions. There are many robust experimental and quasi-experimental methods for assessing impact under difficult circumstances often found in field settings. The relevance and impact of nutrition impact evaluations could be enhanced by collecting data on service delivery, demand-side behavioral outcomes, and implementation processes to better understand the causal chain and what part of the chain is weak, in parallel with impact evaluations. It is also important to understand better the distribution of impacts, particularly among the poor, and to document better the costs and effectiveness of interventions. High levels of child malnutrition in developing countries are contributing to mortality and present long-term consequences for the survivors. An estimated 178 million children under age 5 in developing countries are stunted (low height for age) and 55 million are wasted (low weight for height). Malnutrition makes children more susceptible to illness and strongly affects child mortality. Beyond the mortality risk in the short run, the developmental delays caused by undernutrition affect children s cognitive outcomes and productive potential as adults. Micronutrient deficiencies vitamin A, iron, zinc, iodine, for example are also common and have significant consequences. Progress in reducing malnutrition has been slow: More than half of countries are not on track to achieve the Millennium Development Goal of halving the share of children who are malnou-rished (low weight for age) by 2015. The food price and financial crises are making achievement of this goal even more elusive. The World Bank has recently taken steps to ex-pand its support for nutrition in response to the underlying need and the increased urgency due to the crises. WHAT DO WE KNOW ABOUT REDUCING MALNUTRITION? The increased interest and resources focused on the problem of high and potentially increasing rates of undernutrition raises the question, what do we know about the causes of malnutrition and the in-terventions most likely to reduce it? The medical literature points to the need to inter-vene during gestation and the first two years of life to prevent child malnutrition and its consequences. It suggests that investments in interventions during this window of opportunity among children under 2 are likely to have the greatest benefits. Recently published meta-analyses of the impact evaluation literature point to several interventions found effective for reducing undernutrition in spe-cific settings. However, there are limitations to the generalizability of those reviews findings, particularly in the context of large-scale government programs most likely to be supported by the World Bank. The reviews tend to disproportionately draw on the findings of smaller, controlled experiments; there are few examples of evaluations of large-scale programs, over which there is less control in implementation. In reviewing a large number of studies, interventions, and outcomes, they tend to focus on average impacts. They generally do not explain the magnitude or variability of impacts across or within studies. Very few address the programmatic reasons why some interventions work or don t work, nor do they assess the cost-effectiveness of interventions. Objectives of the Review This paper reviews recent impact evaluations of interventions and programs to improve child anth-ropometric outcomes height, weight, and birth weight with an emphasis on both the findings and limitations of the literature and on understanding what might happen in a non-research setting. It further reviews in greater detail the experience and lessons from evaluations of the impact of World Bank-supported programs on nutrition outcomes. Specifically, the review addresses four questions. First, what can be said about the impact of different interventions on children s anthropometric outcomes? Second, how do these findings vary across settings and within target groups, and what accounts for this variability? Third, what is the evidence of the cost-effectiveness of these interventions? Finally, what have been the lessons from implementing impact evaluations of Bank-supported programs with anthropometric impacts? While there are different dimensions of child nutri-tion that could be explored, the report focuses on child anthropometric outcomes -- weight, height, and birth weight. These are the most common nutrition outcome indicators in the literature and the most frequently monitored by national nutrition programs supported by the World Bank. Low weight for age (underweight) is also the indicator for one of the MDGs. Methodology and Scope Forty-six nutrition impact evaluations published since 2000 were systematically reviewed. These evaluations assessed the impact of diverse interven-tions community nutrition programs, conditional and unconditional cash transfers, early child devel-opment programs, food aid, integrated health and nutrition services, and de-worming. All of the evaluations used research designs that compared the outcomes among those affected by the project to the counterfactual that is, what would have happened to a similar group of people in the absence of the intervention. About half used randomized assignment to create treatment and control groups, while the remainder used matching and various econometric techniques to construct a counterfactual. Among the 46 evaluations, twelve assessed the im-pact of World Bank-supported programs on nutri-tion outcomes in eight countries. While the broader review relies on the analysis of the published impact evaluations as the main source of data, for these twelve evaluations project documents and research outputs were reviewed and World Bank staff, country officials and the evaluators and re-searchers who conducted the studies were interviewed. Findings A wide range of interventions had a positive impact on indicators related to height, weight, wasting, and low birth weight. There were a total of 10 different outcome indica-tors for the four main anthropometric outcomes. A little more than half of the evaluations addressing a height-related indicator found program impacts on at least one group of children, and this was true for about the same share of interventions aimed at improving weight-related and wasting (low weight for height)-related indicators. About three-quarters of the 11 evaluations of interventions that aimed at improving birth weight indicators registered an impact in at least one specification, including five out of seven micronutrient interven-tions. There was no clear pattern of impacts across interventions in every intervention group there were examples of programs that did and did not have an impact on a given indicator, and with varying magnitude. Evaluations of the nutritional impact of programs supported by the World Bank, which are generally large-scale, complex, and implemented in low-capacity settings, show equally variable results. Even controlling for the specific outcome indicator, studies often targeted children of different age groups that might be more or less susceptible to the interventions. It is thus difficult to point to inter-ventions that are systematically more effective than others in reducing malnutrition across diverse set-tings and age groups. Differences in local context, variation in the age of the children studied, the length of exposure to the intervention, and differing methodologies of the studies account for much of the variability in results. Context includes factors like the level and local determinants of malnutrition, differences in the characteristics of beneficiaries (including their age), the availability of service infrastructure, and the implementation capacity of government. Outside of a research setting in the context of a large government program there are many things that can go wrong in either service delivery or the demand response that can compromise impact. Beyond this, there are social factors like the status of women or the presence of civil unrest that can affect outcomes. These findings underscore the conclusion that it should not be assumed that an intervention found effective in a randomized controlled trial in a re-search setting will have the same effects when im-plemented under field conditions in a different set-ting. They also point to the need to understand the prevailing underlying causes of malnutrition in a given setting and the age groups most likely to benefit in selecting an intervention. Further, impact evaluations need to supplement data measuring impact with data on service delivery and demand-side behavioral outcomes to demonstrate the plausibility of the findings, to understand what part of a program works, and to address weak links in the results chain to improve performance. There is scant evidence on the distribution of nutrition impacts who is benefiting and who is not or on the cost-effectiveness of interventions Just because malnutrition is more common among the poor does not mean that they will disproportio-nately benefit from an intervention, particularly if acting on new knowledge or different incentives relies on access to education or quality services. Only a third of the 46 evaluations looked at the distribution of impacts by gender, mother s education, poverty status, or availability of complementary health services. Only nine assessed the impacts on nutritional outcomes of the poor compared with the non-poor. Among the evaluations that did examine variation in results, several found that the children of more educated mothers or in better-off communities are be-nefitting the most. Bank-supported cash transfers, community nutrition, and early child development programs in six of eight countries had some impact on child anthropometric outcomes. Of the 12 impact evaluations of Bank support, all but one were of large-scale government programs with multiple interventions and a long results chain. Three-quarters found a positive impact on anthro-pometric outcomes of children in at least one age group, although the magnitude was in some cases not large or applied to a narrow age group. Most of the impact evaluations involved assessment of completely new programs and involved World Bank researchers. Most used quasi-experimental evaluation designs and two-thirds assessed impact after at most 3 years of program implementation. Only half of the evaluations documented the distribution of impacts and only a third presented information on the costs of the intervention (falling short of cost-effectiveness analysis). In two of the countries (Colombia and the Philippines) the evaluations likely had an impact on government policy or programs. Lessons A number of lessons for development practi-tioners and evaluators arose from the review of impact evaluations of World Bank nutrition support. For task managers: Impact evaluations of interventions that are clearly beyond the means of the government to sustain are of limited relevance. The complexity, costs, and fiscal sustainability of the intervention should figure into the decision as to whether an impact evaluation is warranted. Impact evaluations are often launched for the purpose of evaluating completely new pro-grams, but they may be equally or even more useful in improving the effectiveness of ongo-ing programs. There are methods for obtaining reliable impact evaluation results when randomized assignment of interventions is not possible for political, ethical, or practical reasons. For evaluators: In light of the challenges of evaluating large-scale programs with a long results chain, it is well worth the effort to assess the risks to disruption of the impact evaluation ahead of time and identify mitigation measures. The design and analysis of nutrition impact evaluations need to take into account the likely sensitivity of children of different ages to the intervention. For the purposes of correctly gauging im-pact, it is important to know exactly when delivery of an intervention took place in the field (as opposed to the official start of the program). Evaluations need to be designed to provide evidence for timely decision-making, but with sufficient elapsed time for a plausible impact to have occurred. The relevance of impact evaluations for po-licymakers would be greatly enhanced if im-pact evaluations were to document both the
The focus on results in development agencies has led to increased focus on impact evaluation to demonstrate the effectiveness of development programmes. This book illustrates the broad range of methods available for counterfactual analysis of infrastructure programmes such as establishment, rehabilitation and maintenance of roads, water supply and electrical power plants and grids. Understanding the impact of interventions requires understanding of the context in which the intervention takes place and the channels through which it is expected to occur. For infrastructure interventions it is particularly important to identify the links between the input and the outcomes and impacts because the well-being of people, the ultimate impact, does not change directly as a consequence of the intervention. Therefore impact evaluation of infrastructure programmes typically requires mixing both quantitative and qualitative approaches as illustrated in many of the contribution to this edited volume. This book was originally published as a special issue of the Journal of Development Effectiveness.
The evaluation finds that the content of the World Bank s Country Policy and Institutional Assessment (CPIA) is largely relevant for growth and poverty reduction in the sense that it maps well with the determinants of growth and poverty reduction identified in the economics literature. However, some CPIA criteria need to be revised (in particular trade and finance), and one needs to be added (assessment of disadvantaged socio-economic groups). Second, the evaluation finds that the CPIA ratings are in general reliable and correlate well with similar indicators. The World Bank s internal review process helps guard against potential biases in having Bank staff rate countries on which their work programs depend. The CPIA ratings are found to correlate better with similar indicators for middle income countries than for low income countries. This could be because there is more information available on middle income countries, which increases the likelihood of different institutions having similar assessments on them. This could also be because the CPIA rating exercise takes into account the stage of development, which is more pertinent for low income countries, and which also subject the ratings of those countries to more judgment in an exercise that is already centered on staff judgment.
This IEG evaluation, requested by the World Bank s Board of Executive Directors, represents the first independent evaluation of the PSIA experience. The evaluation finds that: The PSIA approach has appropriately emphasized the importance of assessing the distributional impact of policy actions, understanding institutional and political constraints to development, and building domestic ownership for reforms PSIAs have not always explicitly stated their operational objectives (i.e., informing country policies, informing Bank operations, and/or contributing to country capacity) PSIAs have had limited ownership by Bank staff and managers and have often not been effectively integrated into country assistance programs Quality assurance and Monitoring and Evaluation of the overall effectiveness of PSIAs have been weak The evaluation recommends that the World Bank: Ensure that Bank staff understand what the PSIA approach is and when to use it Clarify the operational objectives of each PSIA and tailor the approach and timeline to those objectives Improve integration of the PSIA into the Bank s country assistance program by requiring that all earmarked funding for PSIAs be matched by a substantial contribution from the country unit budgets Strengthen PSIA effectiveness through enhanced quality assurance
This evaluation assesses the performance of IFC s energy efficiency finance program in China aimed at stimulating energy efficiency investments through bank guarantees and technical assistance. The difference made by the program is traced along the chain of interventions: (i) at the level of banks, the program is narrowly based on one of the two partner banks, which, with the help of the program, expanded its energy efficiency lending as a new business line; (ii) at the level of energy management companies, the program s technical assistance improved the program participants access to finance; and (iii) at the end-user level, it promoted the use of energy efficiency investments that achieved reduction of greenhouse gas emissions. The utilization of IFC s program has been rapid compared with other similar programs. The energy efficiency investments supported by the program have reduced greenhouse gas emissions by 14 million CO2 tons per year, slightly in excess of the target set at the beginning of the program. However, there is only a weak differentiation in behavior surrounding energy efficiency investment between end users supported by the program and other similar companies that were not. It is important to note that the performance of the program was heavily influenced by the government s policy actions and the earlier efforts of other players: The Chinese government and other players such as the World Bank. The CHUEE program, relying mainly on commercial funding through IFC s guarantees, builds on these efforts
"In 2001 the World Bank adopted a gender equality policy as a means to help reduce poverty. This policy was outlined in Integrating Gender into the World Bank's Work: A Strategy for Action (referred as the 2001 Gender Strategy). Through this evaluation IEG finds that the World Bank made progress in gender integration between 2002 and 2008 integrating gender concerns in more than half of the relevant projects. These signs of progress are qualified by findings that implementation of this policy weakened in the latter half of the review period and that there was no built-in results framework in the strategy.
Cities now house half the world s population and produce 70 percent of its GDP. Managing them well helps development. Strengthening municipal management of planning, finance, and service provision has been at the core of World Bank support through municipal development projects (MDPs). This book reviews how, worldwide, nearly 3,000 municipalities have benefitted from 190 World Bank-supported MDPs over the past decade, three quarters of which achieved satisfactory outcomes. The finance dimension of MDPs computerized accounting, revenue generation, and municipal credit produced some of the best results, but weaker outcomes came from attempts to stimulate private finance of municipal services. City planning, used by municipalities worldwide, was not a strong priority for MDPs. But building municipal information systems, for example in Chile, were successful. Monitoring and evaluation rarely worked well, except when municipalities themselves were convinced of its usefulness, such as in Russia, Tunisia, and Colombia. Results in managing service provision were mixed. The poverty focus of MDPs was strikingly weak across the portfolio. Cost-benefit analysis rarely prioritized municipal investments. But MDPs helped municipalities strengthen their procurement function. MDPs helped municipalities manage services more effectively. Better results still can come from a stronger poverty focus, more attention to planning and prioritizating investments, and more effective operation and maintenance of such investments.
