Brookings Series - Scaling Up Development Impacthttp://www.brookings.edu/about/projects/development-assistance/scaling-up-development-impact?rssid=scaling+up+development+impactBrookings Series - Scaling Up Development ImpactenTue, 21 Feb 2012 14:15:00 -0500http://www.brookings.edu/series.aspx?feed=scaling+up+development+impactThu, 21 Jul 2016 12:53:05 -0400 no Brookings Series Feed Brookings Series Feed http://www.brookings.edu/research/opinions/2012/02/21-world-bank-evaluation-linn?rssid=scaling+up+development+impact{40D82661-AF90-4810-BA8A-2020818B0F8A}http://webfeeds.brookings.edu/~/65487797/0/brookingsrss/series/scalingupdevelopmentimpact~Evaluating-the-Evaluators-Some-Lessons-from-a-Recent-World-Bank-SelfEvaluationEvaluating the Evaluators: Some Lessons from a Recent World Bank Self-Evaluation World Bank President Zoellick attends a session at the World Economic Forum in Davos

Editor's Note: The World Bank’s Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird’s-eye view on the lessons learned.

An Overview of the World Bank’s IEG Self-Evaluation Report

In 2011 the World Bank’s Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer.

The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations.

At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG’s role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.”

The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank’s management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank’s self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG’s impact on the Bank’s policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently.

The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group’s branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank’s analytical work and that evaluations draw on comparative evidence.

Assessment

In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international financial agencies, the IEG self-evaluation is accurate and helpful. From my own experience as an operational manager in the Bank whose activities were evaluated by IEG in years past, and as a user of IEG evaluations (and of evaluations of other international aid organizations) for my research on aid effectiveness, I concur that IEG is independent and effective in meeting its mandate as defined. Moreover, the self-evaluation produces useful quantitative evidence (including survey results, budget analysis, etc.) to corroborate qualitative judgments.

However, the self-evaluation suffers from a number of limitations in approach and gaps in focus, which are broadly representative of the practices prevalent among many of the evaluation offices of international aid agencies.

Approach of the IEG self-evaluation

The core of the self-evaluation report is about the evaluation process followed by IEG, with very little said about the substance of IEG’s evaluations. The following questions could have usefully been raised, but were not: do evaluations cover the right issues with the right intensity, such as growth and poverty; environmental, governance, and gender impacts; regional dimensions versus exclusive country or project focus; effectiveness in addressing the problems of fragile and conflict states; effectiveness in dealing with global public goods; sustainability and scaling up; etc. Therefore the report does not deal with the question of whether IEG effectively responds in its evaluations to the many important strategic debates and issues with which the development community is grappling.

Related to this limitation is the fact that the report assessed the quality of IEG’s mostly in terms of (a) whether its approach and processes meet certain standards established by the Evaluation Cooperation Group; and (b) how it is judged by stakeholders in response to a survey commissioned for this evaluation. Both these approaches are useful, but they do not have any basis in professional assessments of the quality of individual products. This is equivalent to IEG evaluating the World Bank’s projects on the quality of its processes (e.g., appraisal and supervision processes) and on the basis of stakeholder surveys, without evaluating individual products and their impacts.

Gaps in the Self-Evaluation and in Evaluation Practice

Careful reading of the report reveals six important gaps in the IEG self-evaluation, in the prevailing evaluation practice in the World Bank, and more generally in the way international financial organizations evaluate their own performance. The first three gaps relate to aspects of the evaluation approach used and the second three gaps relate to lack of focus in the self-evaluation on key internal organizational issues:

1. Impact Evaluations: The report notes that IEG carries out two to three impact evaluations per year, but it sidesteps the debate in the current evaluation literature and practice as to what extent the “gold standard” of randomized impact evaluation should occupy a much more central role. Given the importance of this debate and divergence of views, it would have been appropriate for the self-evaluation to assess IEG’s current practice of very limited use of randomized evaluations.

2. Evaluation of Scaling Up: The report does not address the question of to what extent current IEG practice not only assesses the performance of individual projects in terms of their outcomes and sustainability, but also in terms of whether the Bank has systematically built on its experience in specific projects to help scale up their impact through support for expansion or replication in follow-up operations or through effective hand-off to the government or other partners. In fact, currently IEG does not explicitly and systematically consider scaling up in its project and program evaluations. For example, in a recent IEG evaluation of World Bank funded municipal development projects (MDPs) , IEG found that the Bank has supported multiple MDPs in many countries over the years, but the evaluation did not address the obvious question whether the Bank systematically planned for the project sequence or built on its experience from prior projects in subsequent operations. While most other evaluation offices like IEG do not consider scaling up, some (in particular those of the International Fund for Agricultural Development and the United Nations Development Program) have started doing so in recent years.

3. Drawing on the Experience of and Benchmarking Against Other Institutions: The self-evaluation report does a good job in benchmarking IEG performance in a number of respects against that of other multilateral institutions. In the main text of the report it states that “IEG plans to develop guidelines for approach papers to ensure greater quality, in particular in drawing on comparative information from other sources and benchmarking against other institutions.” This is a welcome intention, but it is inadequately motivated in the rest of the report and not reflected in the Executive Summary. The reality is that IEG, like most multilateral evaluation offices, so far has not systematically drawn on the evaluations and relevant experience of other aid agencies in its evaluations of World Bank performance. This has severely limited the learning impact of the evaluations.

4. Bank Internal Policies, Management Processes and Incentives: IEG evaluations traditionally do not focus on how the Bank’s internal policies, management and incentives affect the quality of Bank engagement in countries. Therefore evaluations cannot offer any insights into whether and how Bank-internal operating modalities contribute to results. Two recent exceptions are notable exceptions. First, the IEG evaluation of the Bank’s approach to harmonization with other donors and alignment with country priorities assesses the incentives for staff to support harmonization and alignment. The evaluation concludes that there are insufficient incentives, a finding disputed by management. Second, is the evaluation of the Bank’s internal matrix management arrangements, which is currently under way. The self-evaluation notes that Bank management tried to quash the matrix evaluation on the grounds that it did not fall under the mandate of IEG. This is an unfortunate argument, since an assessment of the institutional reasons for the Bank’s performance is an essential component of any meaningful evaluation of Bank-supported programs. While making a good case for the specific instance of the matrix evaluation, the self-evaluation report shies away from a more general statement in support of engaging IEG on issues of Bank-internal policies, management processes and incentives. It is notable that IFAD’s Independent Office of Evaluation appears to be more aggressive in this regard: It currently is carrying out a full evaluation of IFAD’s internal efficiency and previous evaluations (e.g., an evaluation of innovation and scaling up) did not shy away from assessing internal institutional dimensions.

5. World Bank Governance: The IEG self-evaluation is even more restrictive in how it interprets its mandate regarding the evaluation of the World Bank’s governance structures and processes (including its approach to members’ voice and vote, the functioning of its board of directors, the selection of its senior management, etc.). It considers these topics beyond IEG’s mandate. This is unfortunate, since the way the Bank’s governance evolves will substantially affect its long-term legitimacy, effectiveness and viability as an international financial institution. Since IEG reports to the Bank’s board of directors, and many of the governance issues involve questions of the board’s composition, role and functioning, there is a valid question of how effectively IEG could carry out such an evaluation. However, it is notable that the IMF’s Independent Evaluation Office, which similarly reports to the IMF board of directors, published a full evaluation of the IMF’s governance in 2008, which effectively addressed many of the right questions.

6. Synergies between World Bank, IFC and MIGA: The self-evaluation report points out that the recent internal reorganization of IEG aimed to assure more effective and consistent evaluations across the three member branches of the World Bank Group. This is welcome, but the report does not assess how past evaluations addressed the question of whether the World Bank, IFC and MIGA effectively capitalized on the potential synergies among the three organizations. The recent evaluation of the World Bank Group’s response to the global economic crisis of 2008/9 provided parallel assessments of each agency’s performance, but did not address whether they work together effectively in maximizing their synergies. The reality is that the three organizations have deeply engrained institutional cultures and generally go their own ways rather than closely coordinating their activities on the ground. Future evaluations should explicitly consider whether the three effectively cooperate or not. While the World Bank is unique in the way it has organizationally separated its private sector and guarantee operations, other aid organizations also have problems of a lack of cooperation, coordination and synergy among different units within the agency. Therefore, the same comment also applies to their evaluation approaches.

Conclusions

Self-evaluations are valuable tools for performance assessment and IEG is to be congratulated for carrying out and publishing such an evaluation of its own activities. As for all self-evaluations, it should be seen as an input to an independent external evaluation, a decision that, for now, has apparently been postponed by the Bank’s board of directors.

IEG’s self-evaluation has many strengths and provides an overall positive assessment of IEG’s work. However, it does reflect some important limitations of analysis and of certain gaps in approach and coverage, which an independent external review should consider explicitly, and which IEG’s management should address. Since many of these issues also likely apply to most of the other evaluation approaches by other evaluation offices, the lessons have relevance beyond IEG and the World Bank.

Key lessons include:

  • An evaluation of evaluations should focus not only on process, but also on the substantive issues that the institution is grappling with.
  • An evaluation of the effectiveness of evaluations should include a professional assessment of the quality of evaluation products.
  • An evaluation of evaluations should assess:
    o How effectively impact evaluations are used;
    o How scaling up of successful interventions is treated;
    o How the experience of other comparable institutions is utilized;
    o Whether and how the internal policies, management practices and incentives of the institution are effectively assessed;
    o Whether and how the governance of the institution is evaluated; and
    o Whether and how internal coordination, cooperation and synergy among units within the organizations are assessed.

Evaluations play an essential role in the accountability and learning of international aid organizations. Hence it is critical that evaluations address the right issues and use appropriate techniques. If the lessons above were reflected in the evaluation practices of the aid institutions, this would represent a significant step forward in the quality, relevance and likely impact of evaluations.

Image Source: © Christian Hartmann / Reuters
      
 
 
]]> Tue, 21 Feb 2012 14:15:00 -0500Johannes F. Linn Editor's Note: The World Bank's Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird's-eye view on the lessons learned. An Overview of the World Bank's IEG Self-Evaluation Report In 2011 the World Bank's Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer. The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations. At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG's role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.” The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank's management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank's self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG's impact on the Bank's policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently. The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group's branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank's analytical work and that evaluations draw on comparative evidence. Assessment In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international ... Editor's Note: The World Bank's Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the ... World Bank President Zoellick attends a session at the World Economic Forum in Davos

Editor's Note: The World Bank’s Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird’s-eye view on the lessons learned.

An Overview of the World Bank’s IEG Self-Evaluation Report

In 2011 the World Bank’s Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer.

The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations.

At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG’s role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.”

The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank’s management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank’s self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG’s impact on the Bank’s policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently.

The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group’s branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank’s analytical work and that evaluations draw on comparative evidence.

Assessment

In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international financial agencies, the IEG self-evaluation is accurate and helpful. From my own experience as an operational manager in the Bank whose activities were evaluated by IEG in years past, and as a user of IEG evaluations (and of evaluations of other international aid organizations) for my research on aid effectiveness, I concur that IEG is independent and effective in meeting its mandate as defined. Moreover, the self-evaluation produces useful quantitative evidence (including survey results, budget analysis, etc.) to corroborate qualitative judgments.

However, the self-evaluation suffers from a number of limitations in approach and gaps in focus, which are broadly representative of the practices prevalent among many of the evaluation offices of international aid agencies.

Approach of the IEG self-evaluation

The core of the self-evaluation report is about the evaluation process followed by IEG, with very little said about the substance of IEG’s evaluations. The following questions could have usefully been raised, but were not: do evaluations cover the right issues with the right intensity, such as growth and poverty; environmental, governance, and gender impacts; regional dimensions versus exclusive country or project focus; effectiveness in addressing the problems of fragile and conflict states; effectiveness in dealing with global public goods; sustainability and scaling up; etc. Therefore the report does not deal with the question of whether IEG effectively responds in its evaluations to the many important strategic debates and issues with which the development community is grappling.

Related to this limitation is the fact that the report assessed the quality of IEG’s mostly in terms of (a) whether its approach and processes meet certain standards established by the Evaluation Cooperation Group; and (b) how it is judged by stakeholders in response to a survey commissioned for this evaluation. Both these approaches are useful, but they do not have any basis in professional assessments of the quality of individual products. This is equivalent to IEG evaluating the World Bank’s projects on the quality of its processes (e.g., appraisal and supervision processes) and on the basis of stakeholder surveys, without evaluating individual products and their impacts.

Gaps in the Self-Evaluation and in Evaluation Practice

Careful reading of the report reveals six important gaps in the IEG self-evaluation, in the prevailing evaluation practice in the World Bank, and more generally in the way international financial organizations evaluate their own performance. The first three gaps relate to aspects of the evaluation approach used and the second three gaps relate to lack of focus in the self-evaluation on key internal organizational issues:

1. Impact Evaluations: The report notes that IEG carries out two to three impact evaluations per year, but it sidesteps the debate in the current evaluation literature and practice as to what extent the “gold standard” of randomized impact evaluation should occupy a much more central role. Given the importance of this debate and divergence of views, it would have been appropriate for the self-evaluation to assess IEG’s current practice of very limited use of randomized evaluations.

2. Evaluation of Scaling Up: The report does not address the question of to what extent current IEG practice not only assesses the performance of individual projects in terms of their outcomes and sustainability, but also in terms of whether the Bank has systematically built on its experience in specific projects to help scale up their impact through support for expansion or replication in follow-up operations or through effective hand-off to the government or other partners. In fact, currently IEG does not explicitly and systematically consider scaling up in its project and program evaluations. For example, in a recent IEG evaluation of World Bank funded municipal development projects (MDPs) , IEG found that the Bank has supported multiple MDPs in many countries over the years, but the evaluation did not address the obvious question whether the Bank systematically planned for the project sequence or built on its experience from prior projects in subsequent operations. While most other evaluation offices like IEG do not consider scaling up, some (in particular those of the International Fund for Agricultural Development and the United Nations Development Program) have started doing so in recent years.

3. Drawing on the Experience of and Benchmarking Against Other Institutions: The self-evaluation report does a good job in benchmarking IEG performance in a number of respects against that of other multilateral institutions. In the main text of the report it states that “IEG plans to develop guidelines for approach papers to ensure greater quality, in particular in drawing on comparative information from other sources and benchmarking against other institutions.” This is a welcome intention, but it is inadequately motivated in the rest of the report and not reflected in the Executive Summary. The reality is that IEG, like most multilateral evaluation offices, so far has not systematically drawn on the evaluations and relevant experience of other aid agencies in its evaluations of World Bank performance. This has severely limited the learning impact of the evaluations.

4. Bank Internal Policies, Management Processes and Incentives: IEG evaluations traditionally do not focus on how the Bank’s internal policies, management and incentives affect the quality of Bank engagement in countries. Therefore evaluations cannot offer any insights into whether and how Bank-internal operating modalities contribute to results. Two recent exceptions are notable exceptions. First, the IEG evaluation of the Bank’s approach to harmonization with other donors and alignment with country priorities assesses the incentives for staff to support harmonization and alignment. The evaluation concludes that there are insufficient incentives, a finding disputed by management. Second, is the evaluation of the Bank’s internal matrix management arrangements, which is currently under way. The self-evaluation notes that Bank management tried to quash the matrix evaluation on the grounds that it did not fall under the mandate of IEG. This is an unfortunate argument, since an assessment of the institutional reasons for the Bank’s performance is an essential component of any meaningful evaluation of Bank-supported programs. While making a good case for the specific instance of the matrix evaluation, the self-evaluation report shies away from a more general statement in support of engaging IEG on issues of Bank-internal policies, management processes and incentives. It is notable that IFAD’s Independent Office of Evaluation appears to be more aggressive in this regard: It currently is carrying out a full evaluation of IFAD’s internal efficiency and previous evaluations (e.g., an evaluation of innovation and scaling up) did not shy away from assessing internal institutional dimensions.

5. World Bank Governance: The IEG self-evaluation is even more restrictive in how it interprets its mandate regarding the evaluation of the World Bank’s governance structures and processes (including its approach to members’ voice and vote, the functioning of its board of directors, the selection of its senior management, etc.). It considers these topics beyond IEG’s mandate. This is unfortunate, since the way the Bank’s governance evolves will substantially affect its long-term legitimacy, effectiveness and viability as an international financial institution. Since IEG reports to the Bank’s board of directors, and many of the governance issues involve questions of the board’s composition, role and functioning, there is a valid question of how effectively IEG could carry out such an evaluation. However, it is notable that the IMF’s Independent Evaluation Office, which similarly reports to the IMF board of directors, published a full evaluation of the IMF’s governance in 2008, which effectively addressed many of the right questions.

6. Synergies between World Bank, IFC and MIGA: The self-evaluation report points out that the recent internal reorganization of IEG aimed to assure more effective and consistent evaluations across the three member branches of the World Bank Group. This is welcome, but the report does not assess how past evaluations addressed the question of whether the World Bank, IFC and MIGA effectively capitalized on the potential synergies among the three organizations. The recent evaluation of the World Bank Group’s response to the global economic crisis of 2008/9 provided parallel assessments of each agency’s performance, but did not address whether they work together effectively in maximizing their synergies. The reality is that the three organizations have deeply engrained institutional cultures and generally go their own ways rather than closely coordinating their activities on the ground. Future evaluations should explicitly consider whether the three effectively cooperate or not. While the World Bank is unique in the way it has organizationally separated its private sector and guarantee operations, other aid organizations also have problems of a lack of cooperation, coordination and synergy among different units within the agency. Therefore, the same comment also applies to their evaluation approaches.

Conclusions

Self-evaluations are valuable tools for performance assessment and IEG is to be congratulated for carrying out and publishing such an evaluation of its own activities. As for all self-evaluations, it should be seen as an input to an independent external evaluation, a decision that, for now, has apparently been postponed by the Bank’s board of directors.

IEG’s self-evaluation has many strengths and provides an overall positive assessment of IEG’s work. However, it does reflect some important limitations of analysis and of certain gaps in approach and coverage, which an independent external review should consider explicitly, and which IEG’s management should address. Since many of these issues also likely apply to most of the other evaluation approaches by other evaluation offices, the lessons have relevance beyond IEG and the World Bank.

Key lessons include:

  • An evaluation of evaluations should focus not only on process, but also on the substantive issues that the institution is grappling with.
  • An evaluation of the effectiveness of evaluations should include a professional assessment of the quality of evaluation products.
  • An evaluation of evaluations should assess:
    o How effectively impact evaluations are used;
    o How scaling up of successful interventions is treated;
    o How the experience of other comparable institutions is utilized;
    o Whether and how the internal policies, management practices and incentives of the institution are effectively assessed;
    o Whether and how the governance of the institution is evaluated; and
    o Whether and how internal coordination, cooperation and synergy among units within the organizations are assessed.

Evaluations play an essential role in the accountability and learning of international aid organizations. Hence it is critical that evaluations address the right issues and use appropriate techniques. If the lessons above were reflected in the evaluation practices of the aid institutions, this would represent a significant step forward in the quality, relevance and likely impact of evaluations.

Image Source: © Christian Hartmann / Reuters
      
 
 
]]> http://www.brookings.edu/research/papers/2012/02/development-interventions-linn?rssid=scaling+up+development+impact{9FD87F2D-F185-4C87-92EA-3E43DD45E98F}http://webfeeds.brookings.edu/~/65487798/0/brookingsrss/series/scalingupdevelopmentimpact~Scaling-Up-Development-Interventions-A-Review-of-UNDPs-Country-Program-in-TajikistanScaling Up Development Interventions: A Review of UNDP's Country Program in Tajikistan

A key objective of the United Nations Development Programme (UNDP) is to assist its member countries in meeting the Millennium Development Goals (MDGs). UNDP pursues this objective in various ways, including through analysis and advice to governments on the progress towards the MDGs (such as support for the preparation and monitoring Poverty Reduction Strategies, or PRSs, in poor countries), assistance for capacity building, and financial and technical support for the preparation and implementation of development programs.

The challenge of achieving the MDGs remains daunting in many countries, including Tajikistan. To do so will require that all development partners, i.e., the government, civil society, private business and donors, make every effort to scale up successful development interventions. Scaling up refers to “expanding, adapting and sustaining successful policies, programs and projects on different places and over time to reach a greater number of people.” Interventions that are successful as pilots but are not scaled up will create localized benefits for a small number of beneficiaries, but they will fail to contribute significantly to close the MDG gap.

This paper aims to assess whether and how well UNDP is supporting scaling up in its development programs in Tajikistan. While the principal purpose of this assessment was to assist the UNDP country program director and his team in Tajikistan in their scaling up efforts, it also contributes to the overall growing body of evidence on the scaling up of development interventions worldwide.

Downloads

      
 
 
]]> Tue, 21 Feb 2012 12:12:00 -0500Johannes F. Linn A key objective of the United Nations Development Programme (UNDP) is to assist its member countries in meeting the Millennium Development Goals (MDGs). UNDP pursues this objective in various ways, including through analysis and advice to governments on the progress towards the MDGs (such as support for the preparation and monitoring Poverty Reduction Strategies, or PRSs, in poor countries), assistance for capacity building, and financial and technical support for the preparation and implementation of development programs. The challenge of achieving the MDGs remains daunting in many countries, including Tajikistan. To do so will require that all development partners, i.e., the government, civil society, private business and donors, make every effort to scale up successful development interventions. Scaling up refers to “expanding, adapting and sustaining successful policies, programs and projects on different places and over time to reach a greater number of people.” Interventions that are successful as pilots but are not scaled up will create localized benefits for a small number of beneficiaries, but they will fail to contribute significantly to close the MDG gap. This paper aims to assess whether and how well UNDP is supporting scaling up in its development programs in Tajikistan. While the principal purpose of this assessment was to assist the UNDP country program director and his team in Tajikistan in their scaling up efforts, it also contributes to the overall growing body of evidence on the scaling up of development interventions worldwide. Downloads - Download Full Paper Authors - Johannes F. Linn A key objective of the United Nations Development Programme (UNDP) is to assist its member countries in meeting the Millennium Development Goals (MDGs). UNDP pursues this objective in various ways, including through analysis and advice to ...

A key objective of the United Nations Development Programme (UNDP) is to assist its member countries in meeting the Millennium Development Goals (MDGs). UNDP pursues this objective in various ways, including through analysis and advice to governments on the progress towards the MDGs (such as support for the preparation and monitoring Poverty Reduction Strategies, or PRSs, in poor countries), assistance for capacity building, and financial and technical support for the preparation and implementation of development programs.

The challenge of achieving the MDGs remains daunting in many countries, including Tajikistan. To do so will require that all development partners, i.e., the government, civil society, private business and donors, make every effort to scale up successful development interventions. Scaling up refers to “expanding, adapting and sustaining successful policies, programs and projects on different places and over time to reach a greater number of people.” Interventions that are successful as pilots but are not scaled up will create localized benefits for a small number of beneficiaries, but they will fail to contribute significantly to close the MDG gap.

This paper aims to assess whether and how well UNDP is supporting scaling up in its development programs in Tajikistan. While the principal purpose of this assessment was to assist the UNDP country program director and his team in Tajikistan in their scaling up efforts, it also contributes to the overall growing body of evidence on the scaling up of development interventions worldwide.

Downloads

      
 
 
]]> http://www.brookings.edu/research/opinions/2011/10/25-development-assistance-linn?rssid=scaling+up+development+impact{3C88EB6F-A0D3-425F-BCD0-CDD4B082D2EF}http://webfeeds.brookings.edu/~/65487799/0/brookingsrss/series/scalingupdevelopmentimpact~It%e2%80%99s-Time-to-Scale-Up-Success-in-DevelopmentIt’s Time to Scale Up Success in Development

Development ministers and experts will meet at the Fourth High Level Forum on Aid Effectiveness in Busan, Korea, in November 2011 to assess their efforts to improve the impact of aid. A recent survey by the Organisation for Economic Co-operation and Development (OECD) shows that little progress has been made since they met in Accra in 2008 for their Third High Level Forum. The many good intentions to improve coordination among donors, to enhance the alignment of aid programs with the priorities of aid recipients, and to develop effective partnerships in practice have turned out to be difficult to implement.

If anything, the challenge has become greater: the number of aid agencies keeps rising, as does the number of aid-supported projects, while average project size continues to drop. According to the OECD, more than half of the 90,000 official aid projects implemented annually are now well below $100,000 in size. With so many small interventions, most of them one-time, without links to each other, driven by the short-lived preferences of individual agencies and individuals in agencies, it is no surprise that the lofty goals of aid ministers go unrealized and that the cumulative impact of the many well-intentioned small aid projects is minimal at best.

It doesn’t have to be that way. There are examples of successful development programs that have lifted millions of people out of poverty, have greatly improved health conditions and have generated new business and employment opportunities. Examples such as the Mexican government’s national program of cash transfers to poor households (“Progresa- Oportunidades”) which conditions assistance on whether children attend school and mothers take their infants for health check-ups; the multi-donor program to eradicate the deadly river-blindness disease in West-Africa; the community based microcredit and employment programs of Grameen Bank and BRAC in Bangladesh; the Chinese government’s program for the development of the loess plateau with support of the World Bank; or the program of rural poverty reduction in the highlands of Peru supported by the International Fund for Agricultural Development (IFAD) – these are just a few examples of cases where the impact of development programs has been at a scale such that it made a real and lasting difference in the lives of millions of people. And success at scale is also possible in fragile and conflict-affected states as a recent review by the Brookings Institution for the Australian aid agency AusAID has shown.

This raises three questions that development ministers should consider as they prepare for and meet in Busan: What made these success stories possible? Why are they the exception rather than the rule? What needs to be done the make scaling up the norm? Let us take these questions in turn.

What made these success stories possible? Each case has its own ingredients of success, but three dimensions are common to them all:

  • The programs pursued a scaling up pathway towards a long-term goal: Few successful programs followed a blueprint for long-term scaling up from the start, but they all built on the recognition that if the early steps were successfully piloted, subsequent steps needed to systematically replicate and scale up what works, adapting the approach in the light of lessons learned at each of the earlier project. A key element in this connection is that the long-term objective of scale impact is part of the program concept from the beginning; that monitoring and evaluation are designed to test not only whether an idea works, but also to measure progress against the long-term goal; and that an effort is made to identify the drivers, to create the spaces and to chart a suitable scaling up pathway for programs to move from small pilot to impact at scale.

  • The programs benefitted from strong and sustained drivers for scaling up: Specifically, they had strong leadership with a clear vision of the need for large scale impact, with ideas that were suited to the challenge, ready to learn from experience, willing to stay engaged for the long haul, championing the cause, building partnerships with other like-minded actors and politically savvy in overcoming obstacles. This leadership could come from public officials, such as the President of Mexico and his Deputy Minister of Finance in the case of “Progresa”, from private individuals, such as the founders of Grameen Bank and BRAC, from a community of national experts and community leaders, as in the case of the Peru Highlands Development Program, from outside aid donors, as in the case of the West-African River- Blindness program, or be part of a wellestablished system of experimentation with replication of success as established in China’s approach to economic growth.

  • The programs created the space for sustained growth: As programs expanded successfully they managed to open up financial and fiscal space by keeping costs down and finding suitable financing mechanisms; they pursued policy reforms that created favorable legal and regulatory conditions; they created the institutional space by identifying appropriate organizational approaches and building institutions for managing the programs at scale; they adapted approaches to the specific cultural realities; and they created political coalitions and operational partnerships that made it possible to grow and sustain the initiative. In the case of “Progresa- Oportunidades” the Mexican government designed a program with a long-term goal of universal coverage of all poor, but started with carefully designed pilots, which were subjected to detailed evaluation against control groups and adapted as needed during the 5-10 year scale-up phase. It created the required fiscal and financial space by abandoning other less successful social programs and by seeking the support of international financial institutions. The government also insulated “Progresa-Oportunidades” from political controversy by carefully monitoring and documenting its positive impacts and by legally assuring that it did not get caught up in party politics. Finally, it designed an institutional approach suitable for phased nation-wide scale-up with minimal bureaucratic obstacles.

Why are these success stories the exception rather than the rule? The first explanation for the lack of systematic and effective focus on scaling up lies in the nature of governmental and bureaucratic incentives and the resulting planning and implementation mechanisms in the developing countries themselves. Typically, governmental plans set out broad targets, policies and implementation modalities, but they generally do not link specific interventions, projects and programs or individual agency budgets and investment plans with the longer-term goals set forth in national or sectoral plans. Moreover, whenever governments or heads of agencies change, the new leadership has a strong tendency to discard the programs supported by the former incumbents and instead to pursue new ideas and new programs. Finally, the practice of methodically evaluating the impact of programs is poorly understood in most countries, and in any case is not well appreciated, since politicians and agencies like to claim success, but prefer not to acknowledge failures in their programs. Contrast this with the incentives for scaling up in the private sector: In a competitive market a successful new initiative, i.e., one that makes a profit, will be replicated and scaled up either by the firm that pioneered it, or by competitors who see the opportunity to garner some of the potential profit for themselves.

The second explanation can be found in the way aid agencies work. While some donors help governments with advice and technical assistance to develop a longer-term national, sub-national and sectoral plans and improved budgeting and investment planning mechanisms, the aid agencies’ own operational modalities and incentives tend to operate just like those of governments: Their operational policies, programming, management and staffing do not encourage support for systematic scaling up. On the contrary, they tend to focus on innovative initiatives and even discourage replication of successful projects and programs. They do not reward effective monitoring and evaluation against longer-term objectives. They rotate managers and staff frequently and with little attention to ensure appropriate hand-over. And the incentives for staff are to start new projects rather than focusing on implementing and building on ongoing ones. And while partnerships, coordination and handing off programs to the clients are encouraged at the level of ministers and agency heads, in practice staff have little incentive to pursue these avenues, since they take time, effort and even budgetary resources, increase risks of delay and of loss of institutional identity and control, and since fiduciary requirements for procurement and disbursements are not harmonized among donors.

What needs to be done to make scaling up the norm? Let us start with aid agencies. Donors have an obligation to do no harm, and it can be argued that their proliferation of small, one-time, uncoordinated and unevaluated interventions do more harm than good. They certainly represent an opportunity forgone, namely the opportunity to support a systematic focus on scaling up successful development interventions. Indeed, this represents an obligation that should be reflected in the mission statements of all official aid organizations, as well as in those of the larger non-governmental organizations and foundations that provide development assistance A recent assessment of donor performance in terms of their attention to scaling up concluded that donors need to address five critical gaps in their operation approach:

  1. Institutional information gap: Aid agencies should review and develop their institutional approaches to scaling up.
  2. Evaluation gap: Evaluations of donor projects should include an assessment of the scaling up practices of donors.
  3. Incentives gap: Donors need to develop internal and external incentives (e.g., operational policies and staff incentives; replication funds, competitions) to help drive the scaling up process.
  4. Partnership gap: Donors should expand the use of programmatic approaches and instruments with joint funding of programs designed to bring donors together so they can help scale up successful interventions;
  5. Ownership gap: Ultimately, scaling up is a country’s job; donors need to help by setting an example, build capacity and hand off to agents in the country.

In their turn, the governments of developing countries need to make scaling up of successful interventions an explicit part of their national planning and programming, need to implement rigorous monitoring and evaluation as learning and accountability mechanisms for the political and agency leadership, and need to find ways to ensure that successful programs do not fall victim of the electoral cycle.

The good news is that progress is being made. There is now a well established body of evidence that scaling up can and does work, even in fragile states. There exists a framework for analyzing, planning monitoring and evaluating scaling up approaches, building on the scaling up pathway, drivers and incentives concepts as summarized above. Examples of governments focused on scaling up success show that it is possible to pursue this avenue to development, with China the outstanding case in point. And some aid agencies have begun to focus systematically on scaling up in their operational mission, strategy, policies, processes and incentives, among them IFAD, the Global Fund to Fight AIDS, Tuberculosis and Malaria, and the Bill and Melinda Gates Foundation. The United Nations Development Program has made scaling up an explicitcriterion in its evaluation of its programs. And the Gesellschaft für Internationale Zusammenarbeit (GIZ) recently issued practical guidelines for scaling up.

What is needed now, and what development ministers and agency heads should focus on in Busan and beyond, are the following three important priorities to ensure that an operational focus on scaling up becomes the rule, not the exception, in the way governments and aid agencies work:

  1. Developing country governments commit themselves to introduce the scaling up objective and practice into their own planning, implementation, evaluation and accountability mechanisms.
  2. Official donors and large private donors commit to introduce the objective and practice of scaling up into their mission statements, operational policies and evaluation practices.
  3. Donors specifically commit to assist development partners through their technical and financial assistance to implement systematic approaches and incentive mechanisms that help drive the scaling up of successful development interventions. Scaling up success is not rocket science. It is a simple, intuitively appealing concept. And yet in practice it has been an orphan in the development literature and practice. Fortunately, this is now changing. We do not need complex models and metrics, nor do governments and aid agencies need sophisticated operational instruments. What we need is for scaling up to become the accepted goal at the political and institutional level. We need a clear vision of scaling up pathways, an assessment of the needed drivers and spaces for scaling up. And we need a readiness to evaluate progress against ultimate and intermediate goals and to adjust the scaling up pathway in light of the lessons learned.

Downloads

Publication: KfW Entwicklungsbank
      
 
 
]]> Fri, 21 Oct 2011 10:04:00 -0400Johannes F. Linn Development ministers and experts will meet at the Fourth High Level Forum on Aid Effectiveness in Busan, Korea, in November 2011 to assess their efforts to improve the impact of aid. A recent survey by the Organisation for Economic Co-operation and Development (OECD) shows that little progress has been made since they met in Accra in 2008 for their Third High Level Forum. The many good intentions to improve coordination among donors, to enhance the alignment of aid programs with the priorities of aid recipients, and to develop effective partnerships in practice have turned out to be difficult to implement. If anything, the challenge has become greater: the number of aid agencies keeps rising, as does the number of aid-supported projects, while average project size continues to drop. According to the OECD, more than half of the 90,000 official aid projects implemented annually are now well below $100,000 in size. With so many small interventions, most of them one-time, without links to each other, driven by the short-lived preferences of individual agencies and individuals in agencies, it is no surprise that the lofty goals of aid ministers go unrealized and that the cumulative impact of the many well-intentioned small aid projects is minimal at best. It doesn't have to be that way. There are examples of successful development programs that have lifted millions of people out of poverty, have greatly improved health conditions and have generated new business and employment opportunities. Examples such as the Mexican government's national program of cash transfers to poor households (“Progresa- Oportunidades”) which conditions assistance on whether children attend school and mothers take their infants for health check-ups; the multi-donor program to eradicate the deadly river-blindness disease in West-Africa; the community based microcredit and employment programs of Grameen Bank and BRAC in Bangladesh; the Chinese government's program for the development of the loess plateau with support of the World Bank; or the program of rural poverty reduction in the highlands of Peru supported by the International Fund for Agricultural Development (IFAD) – these are just a few examples of cases where the impact of development programs has been at a scale such that it made a real and lasting difference in the lives of millions of people. And success at scale is also possible in fragile and conflict-affected states as a recent review by the Brookings Institution for the Australian aid agency AusAID has shown. This raises three questions that development ministers should consider as they prepare for and meet in Busan: What made these success stories possible? Why are they the exception rather than the rule? What needs to be done the make scaling up the norm? Let us take these questions in turn. What made these success stories possible? Each case has its own ingredients of success, but three dimensions are common to them all: - The programs pursued a scaling up pathway towards a long-term goal: Few successful programs followed a blueprint for long-term scaling up from the start, but they all built on the recognition that if the early steps were successfully piloted, subsequent steps needed to systematically replicate and scale up what works, adapting the approach in the light of lessons learned at each of the earlier project. A key element in this connection is that the long-term objective of scale impact is part of the program concept from the beginning; that monitoring and evaluation are designed to test not only whether an idea works, but also to measure progress against the long-term goal; and that an effort is made to identify the drivers, to create the spaces and to chart a suitable scaling up pathway for programs to move from small pilot to impact at scale. - The programs benefitted from strong and sustained drivers for scaling up: Specifically, they had strong leadership with a clear vision ... Development ministers and experts will meet at the Fourth High Level Forum on Aid Effectiveness in Busan, Korea, in November 2011 to assess their efforts to improve the impact of aid. A recent survey by the Organisation for Economic Co-operation and ...

Development ministers and experts will meet at the Fourth High Level Forum on Aid Effectiveness in Busan, Korea, in November 2011 to assess their efforts to improve the impact of aid. A recent survey by the Organisation for Economic Co-operation and Development (OECD) shows that little progress has been made since they met in Accra in 2008 for their Third High Level Forum. The many good intentions to improve coordination among donors, to enhance the alignment of aid programs with the priorities of aid recipients, and to develop effective partnerships in practice have turned out to be difficult to implement.

If anything, the challenge has become greater: the number of aid agencies keeps rising, as does the number of aid-supported projects, while average project size continues to drop. According to the OECD, more than half of the 90,000 official aid projects implemented annually are now well below $100,000 in size. With so many small interventions, most of them one-time, without links to each other, driven by the short-lived preferences of individual agencies and individuals in agencies, it is no surprise that the lofty goals of aid ministers go unrealized and that the cumulative impact of the many well-intentioned small aid projects is minimal at best.

It doesn’t have to be that way. There are examples of successful development programs that have lifted millions of people out of poverty, have greatly improved health conditions and have generated new business and employment opportunities. Examples such as the Mexican government’s national program of cash transfers to poor households (“Progresa- Oportunidades”) which conditions assistance on whether children attend school and mothers take their infants for health check-ups; the multi-donor program to eradicate the deadly river-blindness disease in West-Africa; the community based microcredit and employment programs of Grameen Bank and BRAC in Bangladesh; the Chinese government’s program for the development of the loess plateau with support of the World Bank; or the program of rural poverty reduction in the highlands of Peru supported by the International Fund for Agricultural Development (IFAD) – these are just a few examples of cases where the impact of development programs has been at a scale such that it made a real and lasting difference in the lives of millions of people. And success at scale is also possible in fragile and conflict-affected states as a recent review by the Brookings Institution for the Australian aid agency AusAID has shown.

This raises three questions that development ministers should consider as they prepare for and meet in Busan: What made these success stories possible? Why are they the exception rather than the rule? What needs to be done the make scaling up the norm? Let us take these questions in turn.

What made these success stories possible? Each case has its own ingredients of success, but three dimensions are common to them all:

  • The programs pursued a scaling up pathway towards a long-term goal: Few successful programs followed a blueprint for long-term scaling up from the start, but they all built on the recognition that if the early steps were successfully piloted, subsequent steps needed to systematically replicate and scale up what works, adapting the approach in the light of lessons learned at each of the earlier project. A key element in this connection is that the long-term objective of scale impact is part of the program concept from the beginning; that monitoring and evaluation are designed to test not only whether an idea works, but also to measure progress against the long-term goal; and that an effort is made to identify the drivers, to create the spaces and to chart a suitable scaling up pathway for programs to move from small pilot to impact at scale.

  • The programs benefitted from strong and sustained drivers for scaling up: Specifically, they had strong leadership with a clear vision of the need for large scale impact, with ideas that were suited to the challenge, ready to learn from experience, willing to stay engaged for the long haul, championing the cause, building partnerships with other like-minded actors and politically savvy in overcoming obstacles. This leadership could come from public officials, such as the President of Mexico and his Deputy Minister of Finance in the case of “Progresa”, from private individuals, such as the founders of Grameen Bank and BRAC, from a community of national experts and community leaders, as in the case of the Peru Highlands Development Program, from outside aid donors, as in the case of the West-African River- Blindness program, or be part of a wellestablished system of experimentation with replication of success as established in China’s approach to economic growth.

  • The programs created the space for sustained growth: As programs expanded successfully they managed to open up financial and fiscal space by keeping costs down and finding suitable financing mechanisms; they pursued policy reforms that created favorable legal and regulatory conditions; they created the institutional space by identifying appropriate organizational approaches and building institutions for managing the programs at scale; they adapted approaches to the specific cultural realities; and they created political coalitions and operational partnerships that made it possible to grow and sustain the initiative. In the case of “Progresa- Oportunidades” the Mexican government designed a program with a long-term goal of universal coverage of all poor, but started with carefully designed pilots, which were subjected to detailed evaluation against control groups and adapted as needed during the 5-10 year scale-up phase. It created the required fiscal and financial space by abandoning other less successful social programs and by seeking the support of international financial institutions. The government also insulated “Progresa-Oportunidades” from political controversy by carefully monitoring and documenting its positive impacts and by legally assuring that it did not get caught up in party politics. Finally, it designed an institutional approach suitable for phased nation-wide scale-up with minimal bureaucratic obstacles.

Why are these success stories the exception rather than the rule? The first explanation for the lack of systematic and effective focus on scaling up lies in the nature of governmental and bureaucratic incentives and the resulting planning and implementation mechanisms in the developing countries themselves. Typically, governmental plans set out broad targets, policies and implementation modalities, but they generally do not link specific interventions, projects and programs or individual agency budgets and investment plans with the longer-term goals set forth in national or sectoral plans. Moreover, whenever governments or heads of agencies change, the new leadership has a strong tendency to discard the programs supported by the former incumbents and instead to pursue new ideas and new programs. Finally, the practice of methodically evaluating the impact of programs is poorly understood in most countries, and in any case is not well appreciated, since politicians and agencies like to claim success, but prefer not to acknowledge failures in their programs. Contrast this with the incentives for scaling up in the private sector: In a competitive market a successful new initiative, i.e., one that makes a profit, will be replicated and scaled up either by the firm that pioneered it, or by competitors who see the opportunity to garner some of the potential profit for themselves.

The second explanation can be found in the way aid agencies work. While some donors help governments with advice and technical assistance to develop a longer-term national, sub-national and sectoral plans and improved budgeting and investment planning mechanisms, the aid agencies’ own operational modalities and incentives tend to operate just like those of governments: Their operational policies, programming, management and staffing do not encourage support for systematic scaling up. On the contrary, they tend to focus on innovative initiatives and even discourage replication of successful projects and programs. They do not reward effective monitoring and evaluation against longer-term objectives. They rotate managers and staff frequently and with little attention to ensure appropriate hand-over. And the incentives for staff are to start new projects rather than focusing on implementing and building on ongoing ones. And while partnerships, coordination and handing off programs to the clients are encouraged at the level of ministers and agency heads, in practice staff have little incentive to pursue these avenues, since they take time, effort and even budgetary resources, increase risks of delay and of loss of institutional identity and control, and since fiduciary requirements for procurement and disbursements are not harmonized among donors.

What needs to be done to make scaling up the norm? Let us start with aid agencies. Donors have an obligation to do no harm, and it can be argued that their proliferation of small, one-time, uncoordinated and unevaluated interventions do more harm than good. They certainly represent an opportunity forgone, namely the opportunity to support a systematic focus on scaling up successful development interventions. Indeed, this represents an obligation that should be reflected in the mission statements of all official aid organizations, as well as in those of the larger non-governmental organizations and foundations that provide development assistance A recent assessment of donor performance in terms of their attention to scaling up concluded that donors need to address five critical gaps in their operation approach:

  1. Institutional information gap: Aid agencies should review and develop their institutional approaches to scaling up.
  2. Evaluation gap: Evaluations of donor projects should include an assessment of the scaling up practices of donors.
  3. Incentives gap: Donors need to develop internal and external incentives (e.g., operational policies and staff incentives; replication funds, competitions) to help drive the scaling up process.
  4. Partnership gap: Donors should expand the use of programmatic approaches and instruments with joint funding of programs designed to bring donors together so they can help scale up successful interventions;
  5. Ownership gap: Ultimately, scaling up is a country’s job; donors need to help by setting an example, build capacity and hand off to agents in the country.

In their turn, the governments of developing countries need to make scaling up of successful interventions an explicit part of their national planning and programming, need to implement rigorous monitoring and evaluation as learning and accountability mechanisms for the political and agency leadership, and need to find ways to ensure that successful programs do not fall victim of the electoral cycle.

The good news is that progress is being made. There is now a well established body of evidence that scaling up can and does work, even in fragile states. There exists a framework for analyzing, planning monitoring and evaluating scaling up approaches, building on the scaling up pathway, drivers and incentives concepts as summarized above. Examples of governments focused on scaling up success show that it is possible to pursue this avenue to development, with China the outstanding case in point. And some aid agencies have begun to focus systematically on scaling up in their operational mission, strategy, policies, processes and incentives, among them IFAD, the Global Fund to Fight AIDS, Tuberculosis and Malaria, and the Bill and Melinda Gates Foundation. The United Nations Development Program has made scaling up an explicitcriterion in its evaluation of its programs. And the Gesellschaft für Internationale Zusammenarbeit (GIZ) recently issued practical guidelines for scaling up.

What is needed now, and what development ministers and agency heads should focus on in Busan and beyond, are the following three important priorities to ensure that an operational focus on scaling up becomes the rule, not the exception, in the way governments and aid agencies work:

  1. Developing country governments commit themselves to introduce the scaling up objective and practice into their own planning, implementation, evaluation and accountability mechanisms.
  2. Official donors and large private donors commit to introduce the objective and practice of scaling up into their mission statements, operational policies and evaluation practices.
  3. Donors specifically commit to assist development partners through their technical and financial assistance to implement systematic approaches and incentive mechanisms that help drive the scaling up of successful development interventions. Scaling up success is not rocket science. It is a simple, intuitively appealing concept. And yet in practice it has been an orphan in the development literature and practice. Fortunately, this is now changing. We do not need complex models and metrics, nor do governments and aid agencies need sophisticated operational instruments. What we need is for scaling up to become the accepted goal at the political and institutional level. We need a clear vision of scaling up pathways, an assessment of the needed drivers and spaces for scaling up. And we need a readiness to evaluate progress against ultimate and intermediate goals and to adjust the scaling up pathway in light of the lessons learned.

Downloads

Publication: KfW Entwicklungsbank
      
 
 
]]> http://www.brookings.edu/research/papers/2011/09/development-activities-chandy-linn?rssid=scaling+up+development+impact{2B0FBB5E-5B14-4355-AAE4-494404B98107}http://webfeeds.brookings.edu/~/65487800/0/brookingsrss/series/scalingupdevelopmentimpact~Taking-Development-Activities-to-Scale-in-Fragile-and-Low-Capacity-EnvironmentsTaking Development Activities to Scale in Fragile and Low Capacity Environments

EXECUTIVE SUMMARY

Fragile states present one of the greatest challenges to global development and poverty reduction. Despite much new learning that has emerged from within the development community in recent years, understanding of how to address fragility remains modest. There is growing recognition that donor engagement in fragile states must look beyond the confines of the traditional aid effectiveness agenda if it is to achieve its intended objectives, which include statebuilding, meeting the needs of citizens, and managing risk more effectively. Current approaches are constrained by relying heavily on small-scale interventions, are weakened by poor coordination and volatility, and struggle to promote an appropriate role for the recipient state.

Scaling up (i.e., the expansion, replication, adaption and sustaining of successful policies and programs in space and over time to reach a greater number of people) is highly relevant to fragile settings, both as an objective and as a strategic approach to development. As an objective, it reinforces the logic that the scale of the challenges in fragile states demands interventions that are commensurate in purpose and equal to the task. As a strategy, it encourages donors to identify and leverage successes, and to integrate institutional development more explicitly into projects and programs. In addition, scaling up can assist donors in addressing the priority areas of improved project design and implementation, sustainability and effective risk management.

Successful scaling up in fragile states almost certainly occurs less often than is possible and does not always involve a systematic approach. Donors should therefore look to more systematically pursue scaling up in fragile states and evaluate their performance with specific reference to this objective. This can be done by incorporating relevant elements of a scaling up framework into operational policies, from strategy development through to program design and monitoring.

Contrary to expectations, there are compelling examples of successful scaling up in fragile states. While the conditions prevailing in fragile states create serious obstacles in terms of “drivers” (the forces that push the scaling up process forward) and “spaces” (the opportunities that need to be created, or potential obstacles that need to be removed for interventions to grow), and in terms of the operational modalities of donors, these can be overcome through the careful design and delivery of programs with a clear focus on creating scaling up pathways, and through close partnership and sustained engagement of governments, communities and foreign partners.

Case study evidence suggests that the pathways taken to reach scale in fragile states demand different approaches by donors. Donors need to adopt greater selectivity in determining which areas or sectors for scaling up are justified—a strategy that has also assisted some donors in managing risk. More investment and time are required in upfront analysis and building the evidence for successful scaling up pathways. In some cases, donors require longer time horizons to achieve scale, although demand from government or beneficiaries has sometimes forced donors to move immediately to scale, allowing little or no time for piloting. Regardless of the pace of scaling up, donors that were most successful were engaged early and then remained engaged, often far beyond the replication phase of scaling up, to increase the likelihood of interventions being sustained. Other common characteristics of successful scaling up were simple project design and a focus on the institutional aspects of the scaling up pathway.

Case studies also point to the crucial role of drivers in moving the scaling up process forward in fragile states. Proven ideas and practical models have often been picked up in fragile states, contrary to the expectation that actors may be less responsive to recognizing and acting on the utility of promising results. Leaders undoubtedly have a role to play in supporting scaling up, although there are clear dangers that must be avoided, including avoiding the perception that donors are picking (political) winners by nominating leaders, and tying the survival of projects too closely to the fortunes of a leader’s political career. Incentives were found to be one of the most important drivers in fragile states, and there is a good case to be made for donors introducing new inducements, greater transparency or similar reforms to strengthen the role incentives play. Finally, and in contrast to the standard scaling up framework, community demand was found to be an important driver in many fragile states, both in demanding the expansion of small-scale projects and by facilitating the community’s own resources to support the scaling up process.

The greatest challenge to scaling up in fragile states is the limited spaces these environments provide. This is especially the case in respect to those spaces which concern aspects of governance: political, institutional and policy spaces. When working in fragile states, donors must recognize that spaces for scaling up are almost always more constrained, but look for ways to expand upon them. Some of the most successful examples of scaling up used creative approaches to build space quickly or used existing capacity to the fullest possible extent. Also relevant are the lessons of robust analysis, greater realism and cost control. The case studies confirm the importance of two additional spaces in fragile states. For example, security space often imposed horizontal obstacles to scaling up which could not realistically be overcome while ownership space served as a good indication of the perceived legitimacy of the scaling up process and the likelihood that interventions would be sustained longer term.

Case studies also affirm the importance of emphasizing robust project design and implementation, and the close linkages between the scaling up agenda and the role of risk management and sustainability in fragile states. While sustainability presented a significant problem for many of the projects and programs reviewed, a more focused approach around scaling up may assist donors in addressing sustainability concerns. This would entail adopting a longer-term perspective beyond the immediate confines of any individual project, looking for available drivers and supportive spaces, and focusing on effective implementation and consistent monitoring and evaluation (M&E). Any intervention introduced on a small scale that scores well in sustainability serves as a possible candidate for scaling up.

Similarly, many of the methods used by donors for managing risk—an emphasis on analysis, scenario planning, realism and making use of specialized aid instruments—are equally relevant for supporting scaling up in fragile countries. A persuasive argument can be made that the adoption of a more explicit scaling up approach by donors can form part of a risk management strategy in fragile states. Scaling up can enable donors to more ambitiously tackle development risks without allowing institutional and project risks to grow unchecked. Ultimately, a donor approach that combines good risk management and scaling up requires strong leadership and well-aligned incentives.

Downloads

Image Source: © Beawiharta Beawiharta / Reuters
      
 
 
]]> Thu, 01 Sep 2011 10:17:00 -0400Laurence Chandy and Johannes F. Linn EXECUTIVE SUMMARY Fragile states present one of the greatest challenges to global development and poverty reduction. Despite much new learning that has emerged from within the development community in recent years, understanding of how to address fragility remains modest. There is growing recognition that donor engagement in fragile states must look beyond the confines of the traditional aid effectiveness agenda if it is to achieve its intended objectives, which include statebuilding, meeting the needs of citizens, and managing risk more effectively. Current approaches are constrained by relying heavily on small-scale interventions, are weakened by poor coordination and volatility, and struggle to promote an appropriate role for the recipient state. Scaling up (i.e., the expansion, replication, adaption and sustaining of successful policies and programs in space and over time to reach a greater number of people) is highly relevant to fragile settings, both as an objective and as a strategic approach to development. As an objective, it reinforces the logic that the scale of the challenges in fragile states demands interventions that are commensurate in purpose and equal to the task. As a strategy, it encourages donors to identify and leverage successes, and to integrate institutional development more explicitly into projects and programs. In addition, scaling up can assist donors in addressing the priority areas of improved project design and implementation, sustainability and effective risk management. Successful scaling up in fragile states almost certainly occurs less often than is possible and does not always involve a systematic approach. Donors should therefore look to more systematically pursue scaling up in fragile states and evaluate their performance with specific reference to this objective. This can be done by incorporating relevant elements of a scaling up framework into operational policies, from strategy development through to program design and monitoring. Contrary to expectations, there are compelling examples of successful scaling up in fragile states. While the conditions prevailing in fragile states create serious obstacles in terms of “drivers” (the forces that push the scaling up process forward) and “spaces” (the opportunities that need to be created, or potential obstacles that need to be removed for interventions to grow), and in terms of the operational modalities of donors, these can be overcome through the careful design and delivery of programs with a clear focus on creating scaling up pathways, and through close partnership and sustained engagement of governments, communities and foreign partners. Case study evidence suggests that the pathways taken to reach scale in fragile states demand different approaches by donors. Donors need to adopt greater selectivity in determining which areas or sectors for scaling up are justified—a strategy that has also assisted some donors in managing risk. More investment and time are required in upfront analysis and building the evidence for successful scaling up pathways. In some cases, donors require longer time horizons to achieve scale, although demand from government or beneficiaries has sometimes forced donors to move immediately to scale, allowing little or no time for piloting. Regardless of the pace of scaling up, donors that were most successful were engaged early and then remained engaged, often far beyond the replication phase of scaling up, to increase the likelihood of interventions being sustained. Other common characteristics of successful scaling up were simple project design and a focus on the institutional aspects of the scaling up pathway. Case studies also point to the crucial role of drivers in moving the scaling up process forward in fragile states. Proven ideas and practical models have often been picked up in fragile states, contrary to the expectation that actors may be less ... EXECUTIVE SUMMARY

EXECUTIVE SUMMARY

Fragile states present one of the greatest challenges to global development and poverty reduction. Despite much new learning that has emerged from within the development community in recent years, understanding of how to address fragility remains modest. There is growing recognition that donor engagement in fragile states must look beyond the confines of the traditional aid effectiveness agenda if it is to achieve its intended objectives, which include statebuilding, meeting the needs of citizens, and managing risk more effectively. Current approaches are constrained by relying heavily on small-scale interventions, are weakened by poor coordination and volatility, and struggle to promote an appropriate role for the recipient state.

Scaling up (i.e., the expansion, replication, adaption and sustaining of successful policies and programs in space and over time to reach a greater number of people) is highly relevant to fragile settings, both as an objective and as a strategic approach to development. As an objective, it reinforces the logic that the scale of the challenges in fragile states demands interventions that are commensurate in purpose and equal to the task. As a strategy, it encourages donors to identify and leverage successes, and to integrate institutional development more explicitly into projects and programs. In addition, scaling up can assist donors in addressing the priority areas of improved project design and implementation, sustainability and effective risk management.

Successful scaling up in fragile states almost certainly occurs less often than is possible and does not always involve a systematic approach. Donors should therefore look to more systematically pursue scaling up in fragile states and evaluate their performance with specific reference to this objective. This can be done by incorporating relevant elements of a scaling up framework into operational policies, from strategy development through to program design and monitoring.

Contrary to expectations, there are compelling examples of successful scaling up in fragile states. While the conditions prevailing in fragile states create serious obstacles in terms of “drivers” (the forces that push the scaling up process forward) and “spaces” (the opportunities that need to be created, or potential obstacles that need to be removed for interventions to grow), and in terms of the operational modalities of donors, these can be overcome through the careful design and delivery of programs with a clear focus on creating scaling up pathways, and through close partnership and sustained engagement of governments, communities and foreign partners.

Case study evidence suggests that the pathways taken to reach scale in fragile states demand different approaches by donors. Donors need to adopt greater selectivity in determining which areas or sectors for scaling up are justified—a strategy that has also assisted some donors in managing risk. More investment and time are required in upfront analysis and building the evidence for successful scaling up pathways. In some cases, donors require longer time horizons to achieve scale, although demand from government or beneficiaries has sometimes forced donors to move immediately to scale, allowing little or no time for piloting. Regardless of the pace of scaling up, donors that were most successful were engaged early and then remained engaged, often far beyond the replication phase of scaling up, to increase the likelihood of interventions being sustained. Other common characteristics of successful scaling up were simple project design and a focus on the institutional aspects of the scaling up pathway.

Case studies also point to the crucial role of drivers in moving the scaling up process forward in fragile states. Proven ideas and practical models have often been picked up in fragile states, contrary to the expectation that actors may be less responsive to recognizing and acting on the utility of promising results. Leaders undoubtedly have a role to play in supporting scaling up, although there are clear dangers that must be avoided, including avoiding the perception that donors are picking (political) winners by nominating leaders, and tying the survival of projects too closely to the fortunes of a leader’s political career. Incentives were found to be one of the most important drivers in fragile states, and there is a good case to be made for donors introducing new inducements, greater transparency or similar reforms to strengthen the role incentives play. Finally, and in contrast to the standard scaling up framework, community demand was found to be an important driver in many fragile states, both in demanding the expansion of small-scale projects and by facilitating the community’s own resources to support the scaling up process.

The greatest challenge to scaling up in fragile states is the limited spaces these environments provide. This is especially the case in respect to those spaces which concern aspects of governance: political, institutional and policy spaces. When working in fragile states, donors must recognize that spaces for scaling up are almost always more constrained, but look for ways to expand upon them. Some of the most successful examples of scaling up used creative approaches to build space quickly or used existing capacity to the fullest possible extent. Also relevant are the lessons of robust analysis, greater realism and cost control. The case studies confirm the importance of two additional spaces in fragile states. For example, security space often imposed horizontal obstacles to scaling up which could not realistically be overcome while ownership space served as a good indication of the perceived legitimacy of the scaling up process and the likelihood that interventions would be sustained longer term.

Case studies also affirm the importance of emphasizing robust project design and implementation, and the close linkages between the scaling up agenda and the role of risk management and sustainability in fragile states. While sustainability presented a significant problem for many of the projects and programs reviewed, a more focused approach around scaling up may assist donors in addressing sustainability concerns. This would entail adopting a longer-term perspective beyond the immediate confines of any individual project, looking for available drivers and supportive spaces, and focusing on effective implementation and consistent monitoring and evaluation (M&E). Any intervention introduced on a small scale that scores well in sustainability serves as a possible candidate for scaling up.

Similarly, many of the methods used by donors for managing risk—an emphasis on analysis, scenario planning, realism and making use of specialized aid instruments—are equally relevant for supporting scaling up in fragile countries. A persuasive argument can be made that the adoption of a more explicit scaling up approach by donors can form part of a risk management strategy in fragile states. Scaling up can enable donors to more ambitiously tackle development risks without allowing institutional and project risks to grow unchecked. Ultimately, a donor approach that combines good risk management and scaling up requires strong leadership and well-aligned incentives.

Downloads

Image Source: © Beawiharta Beawiharta / Reuters
      
 
 
]]> http://www.brookings.edu/research/papers/2010/10/ifad-linn-kharas?rssid=scaling+up+development+impact{9198393C-9F97-4A02-BB96-5D2AB5CA5810}http://webfeeds.brookings.edu/~/65487802/0/brookingsrss/series/scalingupdevelopmentimpact~Scaling-Up-the-Fight-Against-Rural-PovertyScaling Up the Fight Against Rural Poverty

ABSTRACT—

The International Fund for Agricultural Development (IFAD) has for many years stressed innovation, knowledge and scaling up as essential ingredients of its strategy to combat rural poverty in developing countries. This institutional review of IFAD’s approach to scaling up is the fi rst of its kind: A team of development experts were funded by a small grant from IFAD to assess IFAD’s track record in scaling up successful interventions, its operational policies and processes, instruments, resources and incentives, and to provide recommendations to management for how to turn IFAD into a scaling-up institution. Beyond IFAD, this institutional scaling up review is a pilot exercise that can serve as an example for other development institutions.

Downloads

Authors

Image Source: © STRINGER Argentina / Reuters
      
 
 
]]> Tue, 26 Oct 2010 13:30:00 -0400Arntraud Hartmann, Homi Kharas, Richard Kohl, Johannes F. Linn and Barbara Massler ABSTRACT— The International Fund for Agricultural Development (IFAD) has for many years stressed innovation, knowledge and scaling up as essential ingredients of its strategy to combat rural poverty in developing countries. This institutional review of IFAD’s approach to scaling up is the fi rst of its kind: A team of development experts were funded by a small grant from IFAD to assess IFAD’s track record in scaling up successful interventions, its operational policies and processes, instruments, resources and incentives, and to provide recommendations to management for how to turn IFAD into a scaling-up institution. Beyond IFAD, this institutional scaling up review is a pilot exercise that can serve as an example for other development institutions. Downloads - Read Full Paper Authors - Arntraud Hartmann- Homi Kharas- Richard Kohl- Johannes F. Linn- Barbara Massler Image Source: © STRINGER Argentina / Reuters ABSTRACT— The International Fund for Agricultural Development (IFAD) has for many years stressed innovation, knowledge and scaling up as essential ingredients of its strategy to combat rural poverty in developing countries.

ABSTRACT—

The International Fund for Agricultural Development (IFAD) has for many years stressed innovation, knowledge and scaling up as essential ingredients of its strategy to combat rural poverty in developing countries. This institutional review of IFAD’s approach to scaling up is the fi rst of its kind: A team of development experts were funded by a small grant from IFAD to assess IFAD’s track record in scaling up successful interventions, its operational policies and processes, instruments, resources and incentives, and to provide recommendations to management for how to turn IFAD into a scaling-up institution. Beyond IFAD, this institutional scaling up review is a pilot exercise that can serve as an example for other development institutions.

Downloads

Authors

Image Source: © STRINGER Argentina / Reuters
      
 
 
]]> http://www.brookings.edu/research/papers/2008/10/scaling-up-aid-linn?rssid=scaling+up+development+impact{DD05A496-B2B4-40FF-A68B-393C8EF69CB5}http://webfeeds.brookings.edu/~/65487803/0/brookingsrss/series/scalingupdevelopmentimpact~Scaling-Up-A-Framework-and-Lessons-for-Development-Effectiveness-from-Literature-and-PracticeScaling Up: A Framework and Lessons for Development Effectiveness from Literature and Practice

Abstract

Scaling up of development interventions is much debated today as a way to improve their impact and effectiveness. Based on a review of scaling up literature and practice, this paper develops a framework for the key dynamics that allow the scaling up process to happen. The authors explore the possible approaches and paths to scaling up, the drivers of expansion and of replication, the space that has to be created for interventions to grow, and the role of evaluation and of careful planning and implementation. They draw a number of lessons for the development analyst and practitioner. More than anything else, scaling up is about political and organizational leadership, about vision, values and mindset, and about incentives and accountability—all oriented to make scaling up a central element of individual, institutional, national and international development efforts. The paper concludes by highlighting some implications for aid and aid donors.

An annotated bibliography of the literature on scaling up and development aid effectiveness was created by Oksana Pidufala to supplement this working paper. Read more »

Downloads

Authors

      
 
 
]]> Fri, 31 Oct 2008 12:00:00 -0400Arntraud Hartmann and Johannes F. Linn Abstract Scaling up of development interventions is much debated today as a way to improve their impact and effectiveness. Based on a review of scaling up literature and practice, this paper develops a framework for the key dynamics that allow the scaling up process to happen. The authors explore the possible approaches and paths to scaling up, the drivers of expansion and of replication, the space that has to be created for interventions to grow, and the role of evaluation and of careful planning and implementation. They draw a number of lessons for the development analyst and practitioner. More than anything else, scaling up is about political and organizational leadership, about vision, values and mindset, and about incentives and accountability—all oriented to make scaling up a central element of individual, institutional, national and international development efforts. The paper concludes by highlighting some implications for aid and aid donors. An annotated bibliography of the literature on scaling up and development aid effectiveness was created by Oksana Pidufala to supplement this working paper. Read more » Downloads - Download Authors - Arntraud Hartmann- Johannes F. Linn Abstract Scaling up of development interventions is much debated today as a way to improve their impact and effectiveness. Based on a review of scaling up literature and practice, this paper develops a framework for the key dynamics that allow the ...

Abstract

Scaling up of development interventions is much debated today as a way to improve their impact and effectiveness. Based on a review of scaling up literature and practice, this paper develops a framework for the key dynamics that allow the scaling up process to happen. The authors explore the possible approaches and paths to scaling up, the drivers of expansion and of replication, the space that has to be created for interventions to grow, and the role of evaluation and of careful planning and implementation. They draw a number of lessons for the development analyst and practitioner. More than anything else, scaling up is about political and organizational leadership, about vision, values and mindset, and about incentives and accountability—all oriented to make scaling up a central element of individual, institutional, national and international development efforts. The paper concludes by highlighting some implications for aid and aid donors.

An annotated bibliography of the literature on scaling up and development aid effectiveness was created by Oksana Pidufala to supplement this working paper. Read more »

Downloads

Authors

      
 
 
]]> http://www.brookings.edu/research/papers/2008/10/scaling-up-linn?rssid=scaling+up+development+impact{E61D7FAB-8E5A-4C36-9199-2F9AB22FFB4F}http://webfeeds.brookings.edu/~/65487804/0/brookingsrss/series/scalingupdevelopmentimpact~Scaling-Up-Through-Aid-The-Real-ChallengeScaling Up Through Aid: The Real Challenge

Summary

At the Gleneagles Summit in 2005, leaders of the G8 group of nations committed to increase aid to poor nations by $50 billion per year. During the same year, in a meeting in Paris, donors promised to coordinate their interventions for more effective delivery. These commitments are now often referred to as the promise of donors to “scale up aid.” Increasing aid flows and improving coordination are indeed important goals and, in fact, goals that donors seem to have trouble meeting. The international donor community met this fall in Accra and will meet in Doha in November 2008 to review progress with this aspect of scaling up aid, and it is hoped that they will recommit to meet the ambitious targets set three years ago.

Scaling up aid is only one of the challenges that donors face. A more important challenge is to “scale up through aid,” meaning that aid flows should not merely support short-lived, one-time and partial development interventions—pilot projects, short-term technical assistance, programs that only address part of the problem, but leave major bottlenecks unaddressed—but should support projects, programs and policies that scale up successful interventions in a country, region or globally to reach the entire target population. Scaling up means that programs are long-term and sustained and that external support is aligned with country needs and deals comprehensively with the development challenges—often by working in partnership with other donors and pooling resources. This is the scaling up challenge that donors should address head-on, but so far have not.

This policy brief reports on the findings of an in-depth review of the literature and practice of scaling up development interventions and focuses on the role that aid donors can play in supporting scaling up for effective development. It stresses that successful scaling up with external assistance means that donor agencies need to: work with a vision and leadership; help create the political constituencies for large-scale implementation; create linkages among project, program and policy interventions; strengthen the institutional capacity of the implementing entities; provide for effective incentives and accountabilities of their own staff and management; work together with each other; monitor and evaluate the progress of programs with special attention to the scaling up dimension; and finally make sure they focus on effective preparation and flexible implementation of the scaling up process. While this is a long-term agenda, donors can take a few practical steps right away that will provide a basis for a more ambitious effort over time.

Downloads

Authors

      
 
 
]]> Mon, 20 Oct 2008 12:00:00 -0400Arntraud Hartmann and Johannes F. Linn Summary At the Gleneagles Summit in 2005, leaders of the G8 group of nations committed to increase aid to poor nations by $50 billion per year. During the same year, in a meeting in Paris, donors promised to coordinate their interventions for more effective delivery. These commitments are now often referred to as the promise of donors to “scale up aid.” Increasing aid flows and improving coordination are indeed important goals and, in fact, goals that donors seem to have trouble meeting. The international donor community met this fall in Accra and will meet in Doha in November 2008 to review progress with this aspect of scaling up aid, and it is hoped that they will recommit to meet the ambitious targets set three years ago. Scaling up aid is only one of the challenges that donors face. A more important challenge is to “scale up through aid,” meaning that aid flows should not merely support short-lived, one-time and partial development interventions—pilot projects, short-term technical assistance, programs that only address part of the problem, but leave major bottlenecks unaddressed—but should support projects, programs and policies that scale up successful interventions in a country, region or globally to reach the entire target population. Scaling up means that programs are long-term and sustained and that external support is aligned with country needs and deals comprehensively with the development challenges—often by working in partnership with other donors and pooling resources. This is the scaling up challenge that donors should address head-on, but so far have not. This policy brief reports on the findings of an in-depth review of the literature and practice of scaling up development interventions and focuses on the role that aid donors can play in supporting scaling up for effective development. It stresses that successful scaling up with external assistance means that donor agencies need to: work with a vision and leadership; help create the political constituencies for large-scale implementation; create linkages among project, program and policy interventions; strengthen the institutional capacity of the implementing entities; provide for effective incentives and accountabilities of their own staff and management; work together with each other; monitor and evaluate the progress of programs with special attention to the scaling up dimension; and finally make sure they focus on effective preparation and flexible implementation of the scaling up process. While this is a long-term agenda, donors can take a few practical steps right away that will provide a basis for a more ambitious effort over time. Downloads - Download Authors - Arntraud Hartmann- Johannes F. Linn Summary

Summary

At the Gleneagles Summit in 2005, leaders of the G8 group of nations committed to increase aid to poor nations by $50 billion per year. During the same year, in a meeting in Paris, donors promised to coordinate their interventions for more effective delivery. These commitments are now often referred to as the promise of donors to “scale up aid.” Increasing aid flows and improving coordination are indeed important goals and, in fact, goals that donors seem to have trouble meeting. The international donor community met this fall in Accra and will meet in Doha in November 2008 to review progress with this aspect of scaling up aid, and it is hoped that they will recommit to meet the ambitious targets set three years ago.

Scaling up aid is only one of the challenges that donors face. A more important challenge is to “scale up through aid,” meaning that aid flows should not merely support short-lived, one-time and partial development interventions—pilot projects, short-term technical assistance, programs that only address part of the problem, but leave major bottlenecks unaddressed—but should support projects, programs and policies that scale up successful interventions in a country, region or globally to reach the entire target population. Scaling up means that programs are long-term and sustained and that external support is aligned with country needs and deals comprehensively with the development challenges—often by working in partnership with other donors and pooling resources. This is the scaling up challenge that donors should address head-on, but so far have not.

This policy brief reports on the findings of an in-depth review of the literature and practice of scaling up development interventions and focuses on the role that aid donors can play in supporting scaling up for effective development. It stresses that successful scaling up with external assistance means that donor agencies need to: work with a vision and leadership; help create the political constituencies for large-scale implementation; create linkages among project, program and policy interventions; strengthen the institutional capacity of the implementing entities; provide for effective incentives and accountabilities of their own staff and management; work together with each other; monitor and evaluate the progress of programs with special attention to the scaling up dimension; and finally make sure they focus on effective preparation and flexible implementation of the scaling up process. While this is a long-term agenda, donors can take a few practical steps right away that will provide a basis for a more ambitious effort over time.

Downloads

Authors

      
 
 
]]> http://www.brookings.edu/research/papers/2007/11/poverty-desai?rssid=scaling+up+development+impact{1F8AC6EA-08AE-42F3-8CB9-3EB2F46B4440}http://webfeeds.brookings.edu/~/65487805/0/brookingsrss/series/scalingupdevelopmentimpact~The-Political-Economy-of-Poverty-ReductionThe Political Economy of Poverty Reduction

Executive Summary

Large-scale antipoverty programs have achieved significant and positive results in many developing countries around the world in the past decade. This paper explores the challenges of “scaling up” small-scale antipoverty programs—taken here to mean the processes by which successful efforts to raise the incomes of the poorest citizens in developing counties are expanded in coverage over time and across geography. In particular, I advocate supplementing approaches that highlight resource and program constraints with an expanded focus on the political dynamics involved in expanding pro-poor policies. Thus, greater emphasis should be placed on understanding the political factors that limit the expansion and survivability of antipoverty programs. A broader view along these lines highlights the bargaining strength of beneficiaries, the need to secure public support, the potential for political misuse of antipoverty programs, and how institutional fragilities affect their sustainability. Antipoverty programs can be effectively scaled up if attention is paid to addressing these political and institutional challenges. An agenda for future research is also identified.

Downloads

Authors

      
 
 
]]> Fri, 30 Nov 2007 12:00:00 -0500Raj M. Desai Executive Summary Large-scale antipoverty programs have achieved significant and positive results in many developing countries around the world in the past decade. This paper explores the challenges of “scaling up” small-scale antipoverty programs—taken here to mean the processes by which successful efforts to raise the incomes of the poorest citizens in developing counties are expanded in coverage over time and across geography. In particular, I advocate supplementing approaches that highlight resource and program constraints with an expanded focus on the political dynamics involved in expanding pro-poor policies. Thus, greater emphasis should be placed on understanding the political factors that limit the expansion and survivability of antipoverty programs. A broader view along these lines highlights the bargaining strength of beneficiaries, the need to secure public support, the potential for political misuse of antipoverty programs, and how institutional fragilities affect their sustainability. Antipoverty programs can be effectively scaled up if attention is paid to addressing these political and institutional challenges. An agenda for future research is also identified. Downloads - Download Authors - Raj M. Desai Executive Summary Large-scale antipoverty programs have achieved significant and positive results in many developing countries around the world in the past decade. This paper explores the challenges of “scaling up”

Executive Summary

Large-scale antipoverty programs have achieved significant and positive results in many developing countries around the world in the past decade. This paper explores the challenges of “scaling up” small-scale antipoverty programs—taken here to mean the processes by which successful efforts to raise the incomes of the poorest citizens in developing counties are expanded in coverage over time and across geography. In particular, I advocate supplementing approaches that highlight resource and program constraints with an expanded focus on the political dynamics involved in expanding pro-poor policies. Thus, greater emphasis should be placed on understanding the political factors that limit the expansion and survivability of antipoverty programs. A broader view along these lines highlights the bargaining strength of beneficiaries, the need to secure public support, the potential for political misuse of antipoverty programs, and how institutional fragilities affect their sustainability. Antipoverty programs can be effectively scaled up if attention is paid to addressing these political and institutional challenges. An agenda for future research is also identified.

Downloads

Authors

      
 
 
]]> http://www.brookings.edu/research/papers/2007/10/scaling-up-linn?rssid=scaling+up+development+impact{4B453490-1E5A-4015-9B1B-3426AEA8854A}http://webfeeds.brookings.edu/~/65487806/0/brookingsrss/series/scalingupdevelopmentimpact~Scaling-Up-A-Path-to-Effective-DevelopmentScaling Up: A Path to Effective Development

Introduction

The global community has set itself the challenge of meeting the Millennium Development Goals (MDGs) by 2015 as a way to combat world poverty and hunger. In 2007, the halfway point, it is clear that many countries will not be able to meet the MDGs without undertaking significantly greater efforts. One constraint that needs to be overcome is that development interventions—projects, programs, policies—are all too often like small pebbles thrown into a big pond: they are limited in scale, short-lived, and therefore have little lasting impact. This may explain why so many studies have found that external aid has had weak or no development impact in the aggregate, even though many individual interventions have been successful in terms of their project- or program-specific goals.

Confronted with the challenge of meeting the MDGs, the development community has recently begun to focus on the need to scale up interventions. Scaling up means taking successful projects, programs, or policies and expanding, adapting, and sustaining them in different ways over time for greater development impact. This emphasis on scaling up has emerged from concern over how to deploy and absorb the substantially increased levels of official development assistance that were promised by the wealthy countries at recent G8 summits. A fragmented aid architecture complicates this task; multilateral, bilateral, and private aid entities have multiplied, leading to many more—but smaller— aid projects and programs and increasing transaction costs for recipient countries. In response, some aid donors have started to move from project to program support, and in the Paris Declaration, official donors committed themselves to work together for better coordinated aid delivery.

The current focus on scaling up is not entirely new, however. During the 1980s, as nongovernmental organizations (NGOs) increasingly began to engage in development activities, scaling up emerged as a challenge. NGO interventions were (and are) typically small in scale and often apply new approaches. Therefore, the question of how to replicate and scale up successful models gained prominence even then, especially in connection with participatory and community development approaches. Indeed, the current interest among philanthropic foundations and NGOs in how to scale up their interventions is an echo of these earlier concerns.

In response to this increased focus on scaling up—and its increased urgency—this policy brief takes a comprehensive look at what the literature and experience have to say about whether and how to scale up development interventions.

Downloads

Authors

Publication: International Food Policy Research Institute
      
 
 
]]> Wed, 31 Oct 2007 12:00:00 -0400Arntraud Hartmann and Johannes F. Linn Introduction The global community has set itself the challenge of meeting the Millennium Development Goals (MDGs) by 2015 as a way to combat world poverty and hunger. In 2007, the halfway point, it is clear that many countries will not be able to meet the MDGs without undertaking significantly greater efforts. One constraint that needs to be overcome is that development interventions—projects, programs, policies—are all too often like small pebbles thrown into a big pond: they are limited in scale, short-lived, and therefore have little lasting impact. This may explain why so many studies have found that external aid has had weak or no development impact in the aggregate, even though many individual interventions have been successful in terms of their project- or program-specific goals. Confronted with the challenge of meeting the MDGs, the development community has recently begun to focus on the need to scale up interventions. Scaling up means taking successful projects, programs, or policies and expanding, adapting, and sustaining them in different ways over time for greater development impact. This emphasis on scaling up has emerged from concern over how to deploy and absorb the substantially increased levels of official development assistance that were promised by the wealthy countries at recent G8 summits. A fragmented aid architecture complicates this task; multilateral, bilateral, and private aid entities have multiplied, leading to many more—but smaller— aid projects and programs and increasing transaction costs for recipient countries. In response, some aid donors have started to move from project to program support, and in the Paris Declaration, official donors committed themselves to work together for better coordinated aid delivery. The current focus on scaling up is not entirely new, however. During the 1980s, as nongovernmental organizations (NGOs) increasingly began to engage in development activities, scaling up emerged as a challenge. NGO interventions were (and are) typically small in scale and often apply new approaches. Therefore, the question of how to replicate and scale up successful models gained prominence even then, especially in connection with participatory and community development approaches. Indeed, the current interest among philanthropic foundations and NGOs in how to scale up their interventions is an echo of these earlier concerns. In response to this increased focus on scaling up—and its increased urgency—this policy brief takes a comprehensive look at what the literature and experience have to say about whether and how to scale up development interventions. Downloads - Download Authors - Arntraud Hartmann- Johannes F. Linn Publication: International Food Policy Research Institute Introduction

Introduction

The global community has set itself the challenge of meeting the Millennium Development Goals (MDGs) by 2015 as a way to combat world poverty and hunger. In 2007, the halfway point, it is clear that many countries will not be able to meet the MDGs without undertaking significantly greater efforts. One constraint that needs to be overcome is that development interventions—projects, programs, policies—are all too often like small pebbles thrown into a big pond: they are limited in scale, short-lived, and therefore have little lasting impact. This may explain why so many studies have found that external aid has had weak or no development impact in the aggregate, even though many individual interventions have been successful in terms of their project- or program-specific goals.

Confronted with the challenge of meeting the MDGs, the development community has recently begun to focus on the need to scale up interventions. Scaling up means taking successful projects, programs, or policies and expanding, adapting, and sustaining them in different ways over time for greater development impact. This emphasis on scaling up has emerged from concern over how to deploy and absorb the substantially increased levels of official development assistance that were promised by the wealthy countries at recent G8 summits. A fragmented aid architecture complicates this task; multilateral, bilateral, and private aid entities have multiplied, leading to many more—but smaller— aid projects and programs and increasing transaction costs for recipient countries. In response, some aid donors have started to move from project to program support, and in the Paris Declaration, official donors committed themselves to work together for better coordinated aid delivery.

The current focus on scaling up is not entirely new, however. During the 1980s, as nongovernmental organizations (NGOs) increasingly began to engage in development activities, scaling up emerged as a challenge. NGO interventions were (and are) typically small in scale and often apply new approaches. Therefore, the question of how to replicate and scale up successful models gained prominence even then, especially in connection with participatory and community development approaches. Indeed, the current interest among philanthropic foundations and NGOs in how to scale up their interventions is an echo of these earlier concerns.

In response to this increased focus on scaling up—and its increased urgency—this policy brief takes a comprehensive look at what the literature and experience have to say about whether and how to scale up development interventions.

Downloads

Authors

Publication: International Food Policy Research Institute
      
 
 
]]>