DFID carried out a Multilateral Aid Review (the MAR) in 2010-11, alongside reviews of bilateral aid (the BAR) and humanitarian aid (the HERR). I reviewed them all at the time, the MAR and BAR here, and the HERR here. DFIDDepartment for International Development has now published an update of the MAR, as a half-way house to a full repeat exercise in 2015.

My judgement is that the MAR Update is a competent review, but necessarily limited in its coverage of the issues, and no substitute for a policy on multilateral aid. I recommend a White Paper on the subject. DFIDDepartment for International Development is planning a full MAR exercise in 2015. My recommendation is to see first whether it cannot be merged with other donor efforts.

The update focuses on the progress 37 multilateral agencies funded by DFIDDepartment for International Development have made in implementing the individual reform agendas laid down for them in 2011. The opportunity is taken to amend the method slightly, in response to criticism the first time round, that the analysis did not take sufficient account of the different mandates of multilateral organisations. Overall, the results are relatively favourable. The update report concludes that ‘all organisations assessed have made some progress against the agreed reform agenda, and about half of them have done better than this. Importantly, all of the UN “special measures” agencies are making steady progress towards emerging from “special measures”’. On the other hand, there is work to do, especially on gender, transparency and management of costs. Some organisations have more work to do than others: the Commonwealth Secretariat, UNDPUnited Nations Development Programme and UNFPAUnited Nations Fund for Population Activities fall into this category.

The methodology is sound. We have that on the authority of the two external reviewers, Lawrence Haddad and Alison Evans. The report is also very well-written. It is a pity the authors do not seem to be credited.

However, the publication of the Update raises two larger questions. First, has DFIDDepartment for International Development responded as well as it could to the critiques of the first exercise? And, second, what are the implications for policy towards the multilateral system as a whole?

Has DFIDDepartment for International Development responded as well as it could to the critiques of the MAR?

On the first question, it is helpful to refer to reviews conducted by the National Audit Office (in 2012) and two parliamentary select committees, the Public Accounts Committee and the International Development Select Committee (both in 2013). There have also been many independent reviews. My own comments are pasted in at the end for ease of reference. Declaration of interest: I am credited as a specialist adviser in the IDCInternational Development Committee Report.

There are many technical points, to do with selection of criteria, collection of evidence and weighting of results. We can leave those aside for now. More general points include:

1. That the original method did not deal well with agencies that have a normative or standard-setting function (for example, FAO);
2. That a focus on short-term results and value-for-money may favour specialised vertical funds, like GAVI or the Global Fund, especially those that work in relatively small numbers of countries, over multi-purpose and geographically dispersed agencies, especially UN bodies and the EU;
3. That there was insufficient coordination of reviews by different donors, resulting in overload for the agencies;
4. That making decisions about resource allocation needs comparative information about other options, notably bilateral aid, that the MAR did not provide;
5. That the implications for action were not always easy to read off from the analysis, especially on the question of whether more engagement and more funding might produce faster change than restricting funding and declaring agencies to be failing or in special measures; and
6. Finally, that there was little discussion of how donor behaviour might shape multilateral performance, for example by providing too small a share of core funding, and too large a share of trust funds or other special purpose vehicles.

Dealing with agencies that have a normative mandate

The Update team claim to have solved the first problem, that of dealing with normative agencies, by revising the method to incorporate new criteria (Pgs 28-9 and Annex 2). The key idea seems to be to use a theory of change approach, in order to acquire line-of-sight between global normative activities and results on the ground.  The overall framework also asks whether an agency ‘develops norms and standards or global public goods that are critical for the achievement of the MDGsMillennium Development Goals and poverty reduction, or for the delivery of other international development or humanitarian goals’. I would have liked to see a more detailed example of how this works for the big standard-setting agencies in the UN. It must be quite hard, for example, though not impossible, to provide evidence of a line of sight for normative work on food standards or nutrition indicators or drug safety. There is not much evidence of the new approach having an impact in the individual agency ‘fiches’, for example for FAOFood and Agriculture Organisation of the United Nations or WHO. The external reviewers do not discuss this issue in their commentary on the exercise (Pgs 183-4). The IDCInternational Development Committee conclude that ‘for the next full-scale MAR in 2015, DFIDDepartment for International Development should make a much clearer distinction between different organisations’ mandates’. I agree.

            A bias to vertical funds?

The second problem is not addressed.

            More scope for cooperation with other donors?

The third criticism, that there may be too many reviews, is noted by pretty well all the commentators on the report. The IDCInternational Development Committee defend DFID, on the grounds that having its own review is the best way to test the specific contribution agencies make to DFIDDepartment for International Development objectives. In a narrow sense, that may be true, but independent observers, like the OECD/DAC, have noted that there is a great deal of overlap between the criteria used by different donors. The latest DACDevelopment Assistance Committee (of the OECD) annual Report on Multilateral Aid notes that comprehensive reviews have been undertaken by Australia, Sweden, Denmark and the Netherlands, as well as the UK, and are supplemented by the DAC’s own work and the international programme of work by MOPAN, the Multilateral Organisation Performance Assessment Network, to which all the above belong. Diplomatically, the DACDevelopment Assistance Committee (of the OECD) suggests that there is scope for much greater cooperation:

‘While there is certainly a political dimension to assessments and the demand for evidence, this chapter has sought to demonstrate that there is a strong degree of convergence in the criteria adopted by different reviews and assessments. This suggests that there is great potential for joint approaches and assessments.  Collective efforts (such as MOPAN) to assess the effectiveness of multilateral organisations, or (like the DACDevelopment Assistance Committee (of the OECD) EVALNET initiative) their evaluation functions, require collective work to define methodology, frameworks, financing, and other parameters of joint assessments. They therefore have higher initial transaction costs. On the other hand, they also offer a number of significant advantages. Enhancing existing joint assessments – for example, through a shared, regularly updated “menu” of assessment criteria – rather than promoting comprehensive bilateral assessments with new criteria stems the proliferation of assessments, thereby easing the significant administrative burden for development actors, who include partner countries and multilateral agencies themselves. Also, joint approaches can help ensure that an organisation is assessed against collective objectives and that collective efforts, undertaken to make tough decisions within its broader governance setting, incite reform.’

Does this suggest that DFIDDepartment for International Development should review its plan to repeat the MAR in 2015?

            Comparing bilateral and multilateral aid

The fourth criticism is that it is difficult to draw policy conclusions unless multilateral results can be compared with bilateral. This is a point I made in my original review, and one the IDCInternational Development Committee also examined. In fact, they asked Alison Evans about it, and quoted her reply:

‘I doubt whether the MAR has actually helped us very much in answering the question of whether a pound of UK aid is better spent in health through the multilateral or the bilateral programme... As an external reviewer on the MAR, I have had no contact at all with the Bilateral Aid Review, which of course would be appropriate, but I have no sense about how the two come together at all.’

The Select Committee concluded that:

‘In order for DFIDDepartment for International Development to make informed decisions about the level of funding it provides to multilateral organisations, it needs to be able to compare multilateral aid with bilateral aid. Despite the Minister of State's assertion that the decision to conduct a Bilateral Aid Review at the same time as the original MAR enabled such comparisons to be drawn, the fact remains that following the original MAR, DFID's funding plans for multilateral agencies were not generally informed by comparisons with the value for money of bilateral aid.DFID's decisions about the level of funding it provides to multilateral organisations should be informed by comparisons with the value for money of bilateral aid. These comparisons should form an integral part of the MAR’.

I might add that having a separate humanitarian review, when the MAR covers humanitarian agencies like ECHOEuropean Community Humanitarian Office and WFPWorld Food Programme also makes little sense. If there is to be a review in 2015, perhaps it should be unified?

            Carrots or sticks?

The fifth criticism was how about how to interpret the results. Does a poor score imply more engagement or less. The Select Committee also picked up this point, and asked Alan Duncan, the Minister of State, what he thought. This was his answer:

‘I do not quite go along with the stark which-is-better validity of your question. You could ask any teacher, "Which is better for the child, the carrot or the stick?" That is really what you are asking. I do not think there is any certain black-and-white answer.’

In fact, the MAR showed that DFIDDepartment for International Development was ruthless with some agencies, cutting core funding to UNIDO, for example, and UNISDR, but leaving open the possibility of project funding, but more engaged with those in special measures, continuing to fund and investing heavily in reform.

It would be interesting to know, actually, whether any money did continue going to UNIDO, UNISDR, Habitat and ILO, the four organisations whose core funding was to be eliminated as a result of the MAR. I’ve checked the latest edition of Statistics on International Development, published in October 2013, and can find no breakdown of funding to UN organisations. Perhaps an MP could ask a Parliamentary Question?

ICAI, the Independent Commission on Aid Impact, took note of the different approaches. In its Annual Report for 2012-13, it noted that

Having undertaken three reports on DFID’s relationships with and work though multilaterals this year, we observe that DFIDDepartment for International Development works with them in very different ways and that it needs to be clear what it wants to achieve in those different scenarios. For example, there are aspects of the UK’s engagement with the European Union and the Asian Development Bank which can only be achieved through a mixture of dialogue, influencing work and funding decisions at the headquarters level and we made recommendations for maximising the impact of these activities. DFIDDepartment for International Development also, however, co-finances multilateral projects in-country. Our reports on the Asian Development Bank and UNICEFUnited Nations Children's Fund show that the Multilateral Aid Review analysis is only a starting point for allocation decisions and that DFIDDepartment for International Development needs to understand each organisation’s strengths and capabilities at a local level before committing funding.’

            Too little core funding?

Finally, there is no discussion in the MAR Update about whether DFID’s own behaviour, alongside that of other donors, might make it more difficult to manage multilateral agencies. We will have to come back to that.

Implications for multilateral policy as a whole

It is apparent that what we learn from the MAR and the Update is how individual multilateral agencies perform, individually and in comparison with each other. That is useful information, though the evidence suggests that the UK could save money, and save time and money for the agencies, by combining its efforts with those of others. There is more to do in deciding how to assess agencies which have a normative remit, also in providing evidence to judge whether to provide aid through bilateral or multilateral channels, and in thinking about how to influence change in multilaterals.

More generally, the MAR and its Update provide contributions to, but do not in themselves shape, multilateral policy. Core questions remain: what share of UK oda should be provided through which multilaterals, for what purposes, by what means and how managed?

The DACDevelopment Assistance Committee (of the OECD) Multilateral Aid Report provides some guidance on these questions, with eight good practice lessons for ‘good multilateral donorship’ (Figure 1). These are rooted in analysis which implies that there are too many multilaterals (over 200), incoherently funded, and with very high transactions costs.

Figure 1

The DACDevelopment Assistance Committee (of the OECD) is especially worried about proliferation of agencies, building in this connection on the consensus at the High Level Forum on Aid Effectiveness at Busan in 2011. It also notes – I wanted to say ‘laments’ but the DACDevelopment Assistance Committee (of the OECD) is too cautious for that – the high share of non-core funding to the multilateral system. So-called multi-bi aid accounted for 12% of all oda in 2010, compared to 28% for ‘pure’ multilateral. Overall, then, the multilateral system accounted for 40% of all oda, a total of $US 54bn.

The non-core share is much higher for the UN, as can be seen from the Figure below. In fact, 74% of operational funding for UN development and humanitarian activities is non-core.

Figure 2

The DACDevelopment Assistance Committee (of the OECD) identifies the respective advantages of core and non-core as follows. It is apparent that the advantages of non-core are more for bilateral donors than any other actor.

Figure 3

Finally, the DACDevelopment Assistance Committee (of the OECD) suggests how to reduce proliferation, with an additional set of principles, as in the Figure below. Among other things, it emphasises the importance of harmonisation at country level.

Figure 4

 By the way, all this material is taken from the 2012 DACDevelopment Assistance Committee (of the OECD) Report. There has been no Multilateral Aid Report published in 2013, and a new format is proposed for the future, involving a series of working papers published over a two-year cycle. Interestingly, two of the proposed three priority themes are on issues discussed here: managing earmarked funding, and the country-level fit of trust funds. The third is the role of emerging donors.

Where does the UK fit in this picture? The DACDevelopment Assistance Committee (of the OECD) Report only has data for 2010. It shows that the UK is an above average user of the multilateral system, 60% compared to a DACDevelopment Assistance Committee (of the OECD) average of 40% - but this is partly because contributions to the EUEuropean Union count as multilateral. The UK falls below the average in terms of the share of its contributions made as core – 63% compared to an average of 69%. Again, this is distorted by EUEuropean Union funding. If the EUEuropean Union is excluded, nearly half of the UK’s contribution through multilateral agencies is non-core.

All this suggests to me that there is work to do on multilateral policy, combining the findings of the MAR and its update with wider considerations. This issue was last looked at in the 2009 White Paper, which made a strong commitment to multilateralism. There has been no update under the present Government. In fact, I suggested back in 2011 that the three reviews would provide a good foundation for a White Paper.

Conclusion

Since this is a paper about multilateral oda, I can’t resist finishing with my own twenty-point programme for reform of the multilateral system, published in 2012, under the title ‘Conquering the Hydra’. Here it is.

‘The following list might provide at least the basis for a conversation. To make things easy, I have used the same headings as in the DAC Conceptual Framework, and phrased the points, in communiqué style as ‘We Wills’.

A. For bilateral donors in their headquarters interaction with multilateral agencies:

1. We will, over the next decade, and in agreement with developing countries, reduce the number of multilateral agencies, funds and programmes by at least 5% each year, by closures or mergers.
2. We will, as far as the UN is concerned, channel all funding through central mechanisms, especially the Central Emergency Relief Fund on the humanitarian side, and the UNDP on the development side.
3. We will channel all climate funding through the Green Climate Fund.
4. We will ensure that innovative finance, including blending options, is fully integrated into mainstream aid mechanisms.
5. We will adopt the practice of replenishment funding for all multilateral agencies, on a 3-7 year basis.
6. We will restrict the share of funding to multilaterals directed through Trust Funds or similar arrangements to no more than 10% of total funding.
7. We will, year on year, increase the share of total oda spent through the multilateral system, aiming to reach 75% of the total by 2018.
8. We will, in the spirit of contestability, ensure that the World Bank, the UN and the Regional Development Banks all have the option to deliver the right mix of grants loans, and technical cooperation in pursuit of their objectives.
9. We will ensure that the governance of all multilateral bodies is democratic and accountable.
10. We will report annually on progress, to the DAC, the Global Partnership for Effective Development and the Development Cooperation Framework.

B. Organisational effectiveness of multilateral organisations:

1. We will focus our assessment efforts on MOPAN, eschewing individual Government reviews of multilateral agencies.
2. We will ensure that the default approach to aid evaluation is multi-donor and comparative, with strong recipient country engagement.
3. We will put in place independent evaluation, focused on results, value-for-money and development effectiveness.
4. We will guarantee that developing countries participate in all peer review exercises, including those carried out by DAC donors.
5. We will report transparently on the results of comparative evaluation and peer review, in order to ensure continuous improvement of aid programmes.

C. For the effective delivery of multilateral aid:

1. We will ensure that the chief executive and other senior management positions of all multilateral agencies are filled by open competition, and on the basis of transparent criteria.
2. We will develop jointly-owned country-level results frameworks.

D. For country-level harmonisation between bi- and multilateral donors:

1. We will devise and apply a Code of Conduct on Division of Labour, which limits the number of donors active in each sector.
2. We will make maximum use of budget support, either General Budget Support or Sector Budget Support, and ensure that all aid is on-budget.
3. We will apply fully the ownership, harmonisation and alignment principles of the Paris, Accra and Busan Declarations, and hold ourselves accountable through independent monitoring and evaluation of progress.

______________

Review of the MAR

How about the multilateral review? This was a different kind of exercise, in which 43 organisations were assessed on a range of criteria, summarised in terms of their contribution to UK development objectives and their organisational strength. There were 10 ‘components’ of the assessment and a total of 41 specific criteria, ranging from promotion of gender equality to the commitment to change of the Governing Body. All these were investigated, scored and then aggregated into a single ‘value for money’ chart, which classifies agencies into four categories: very good; good, adequate; and poor. This is a comprehensive and systematic exercise and adds to the work of other reviews, like the multi-donor MOPAN network.

Nine of the 43 organisations qualify as ‘very good’. These are the Asian Development Bank, ECHO, the European Development Fund, GAVI, the Global Fund, the ICRC, theIDA, PIDG (the Private Infrastructure Development Group), and UNICEF. Funding increases are promised to most of these (though not at this time, interestingly, the organs of theEU).

Nine organisations qualify as ‘poor’: the Commonwealth Secretariat, FAO, Habitat, ILO, IOM, ISDR, UNESCO,UNIDO, and UNIFEM. Some of these are to be placed in ‘special measures’ (including FAO and UNESCO). Some are to have funding withdrawn completely: Habitat, ILO,UNIDO and ISDR (the Secretariat for disaster reduction).

The methodology used for this exercise will no doubt spawn a small industry of commentary, tackling issues like the choice of criteria, the weighting, the aggregation procedures, and the quality of the evidence base. It is not easy to score on a single scale, agencies which have very different constitutions, functions and levels of resources. The IDA, for example, which exists to channel funding to poor countries, is a very different kind of organisation to FAO, which has important normative and technical supervisory functions, and only provides a small amount of aid, mostly in the form of technical assistance. No doubt, the exercise will be refined and extended in future years.

Methodology apart, the review throws up some practical issues.

First, it leaves open the question of whether the current multilateral share (29% if only core funding is counted, according to the MAR; closer to 50%, according to the DAC, if trust funds and other multi-bi operations are included) is about right, or not. The previous Government had pledged (in the 2009 White Paper) to increase the multilateral share year on year. This policy appears to have lapsed. It is a pity the bilateral and multilateral reviews were not integrated, so that the question could have been explored.

Second, it is notable that the highest-scoring agencies or organisations tend to be quite specialised, in several cases being vertical funds, like GAVI or the Global Fund, in other cases being specialist humanitarian organisations, likeECHO or ICRC. I do have a suspicion that vertical funds, in particular, are more popular with donors than recipients, partly because they give donors greater control.

Third, there is little discussion of why some agencies score poorly, and whether donor behaviour might not be to blame. The One-UN Panel, for example, emphasised the damage that is done to multilateral organisations, especially the UN, by over-reliance on trust funds and special purpose vehicles, at the expense of predictable core funding. The UK is certainly guilty in this area, though not alone.

Fourth, in general, the UN does not come out of this exercise as well as the multilateral development banks. It is interesting to ask how the results would have differed if universality and accountability had been more highly weighted.

Fifth, the low scores for some organisations present a real difficulty, because they are key to the international architecture and/or represent international initiatives on issues of real importance. There are few alternatives in some cases, and in other cases alternatives which present problems of universality and accountability. Both ‘ending core support’ and ‘special measures’ are unpalatable options, and have a punitive tone, of course the first more than the second. Thus

• FAO scores as ‘poor’, which may or may not be justified, but the world is facing a food crisis in 2011 which absolutely needs the kind of coherent management which FAO was set up to provide, and which only FAO can deliver, for example through the Committee on Food Security. This is presumably why the organisation was placed in ‘special measures’, rather than having its funding cut. But rather than a focus on special measures, the alternative might have been to highlight the (alleged) problems, and then reinvigorate the commitment to FAO, which is in the process of electing a new Director General and which will need more and more secure funding. ‘Tough love’ needs to be the strategy, but a bit less ‘tough’ and a bit more ‘love’ might have been appropriate.
• UNIDO also scores as poor, and is to have its core funding ended. This seems a real pity, given the importance of manufacturing to the growth prospects of the poorest countries, but also given the new impetus UNIDO has received since Kandeh Yumkellah became Director General. He has also led for the whole of the UN on energy, a key ingredient of the poverty reduction package, and a major item on the international agenda, with 2012 having been designated as the International year for Sustainable Energy for All.
• ISDR falls into the same category, with the review apparently concluding that the same impact can be achieved by channelling resources to a World Bank Trust Fund on disaster risk reduction. This moves UK funding, and, if followed, the global centre of gravity, away from the UN and into the World Bank – a strange move in the year of the Global Platform for Disaster Risk Reduction, marking five years since the adoption of the Hyogo Framework.

Overall, there is a risk with exercises like the multilateral review that the technical wizardry of the scoring might displace common sense or obscure the politics of decision-making. Andrew Mitchell will, of course, need to use the results of the analysis carefully, and make sure the messages are in the right tone. He has been more supportive of FAO than the review might have suggested, for example.

Are we better off as a result of these reviews? More focused, certainly, both geographically and in terms of results. Better informed, certainly, both as development specialists and as UK tax payers. Inquisitive also, with many questions still to be answered. That is one benefit of the exercise. The reviews have helped inform policy. They also contribute to ongoing debate about how best the UK can help to reduce poverty in the world.

___________

Source: http://www.simonmaxwell.net/blog/looking-behind-the-headlines-of-dfids-bilateral-and-multilateral-aid-reviews.html