Does ICAI need a broader remit? Lessons from the Review of the Conflict, Security and Stability Fund

The ICAI Review of the Conflict, Security and Stability Fund (CSSF) has attracted quite a bit of publicity and commentary, but the report itself and the debate it has engendered both overlook some key issues. Is there a general problem with ICAI’s remit, or is the issue something to do with the difficulty of examining cross-Government programmes, or more specifically with the difficulty of assessing a programme governed by security considerations and therefore not fully transparent? Probably all the above, but the main conclusion is that ICAI may need a wider perspective and possibly a wider formal remit.

The Report was published in March 2018. As a typical ICAI Performance Reviews, it provides background and context, and then looks at (a) relevance, (b) effectiveness, and (c) learning. Each of these is scored on a four point scale, with the three scores combined to give an overall score. In this case, the CSSF gets green-amber for relevance, amber-red for effectiveness, and amber-red for learning, giving an amber-red score overall.

The Report explains that the CSSF was established in 2015, as the latest example of cross-Government conflict prevention funds dating back to 2001. Its initial annual budget of £1 billion is set to rise to £1.3 billion by 2020-21. The CSSF exists to support and implement strategies defined by the National Security Council, and only part of its funding counts as ODA: in the first two years, 34% and 41% of programmable funds. The CSSF works in 70 countries (Figure 1) and on a range of topics (Figure 2). 52% of its programmable funding is spent through private companies and NGOs, 27% through multilaterals, and 18% through Government Departments. NB what is meant by ‘programmable’ needs unpacking: the figure ‘excludes ring-fenced activities’, , ‘including peacekeeping’. The CSSF’s own website says that the main ring-fenced items are Afghan Security, the Deployed Military Assistance Pool, and contributions to multilateral peace-keeping, mainly via the UN. Some peace-keeping costs qualify as ODA, but are excluded from the ICAI review. Should they have been?

Figure 1

Source: https://icai.independent.gov.uk/report/cssf/

Figure 2

Source: https://icai.independent.gov.uk/report/cssf/

The Report finds that the CSSF is well-aligned with overall UK aid policy, but that theories of change and results frameworks are both weak, and that the programme is not well monitored or evaluated. It is not very good at learning. There are six recommendations, mostly about results frameworks and learning.

The media coverage of the report mainly focused on the amber-red score, on inadequate safeguards for human rights, on the results frameworks, and on some egregious examples of poor spending highlighted in the report, including a prison building in Mali: see e.g. here,  here, here, and here. There has also been a debate on Twitter about the application of complexity theory to results frameworks, and the difficulty of designing such frameworks in volatile conflict situations. Craig Valters has led on this, based on his recent analysis of DFID’s approach to results, as well as his earlier synthesis of the evidence on the (limited) contribution of capacity-building to security.

There is obviously a lot to say about conflict prevention, peace-making, peace-keeping, conflict resolution, transition to sustainable peace, and long-term social and political resilience  – each of which, by the way, is a separate topic. This is not my specialist area, and the report does not really summarise best-practice, but it does point to some useful resources, including ‘Pathways for Peace’ by the World Bank and the UN, and a report by the OECDOrganisation for Economic Cooperation and Development on Evaluating Peacebuilding Activities in Settings of Conflict and Fragility. I don’t think ICAI makes much use of the OECDOrganisation for Economic Cooperation and Development State of Fragility Report, the latest edition of which, in 2016, focuses specifically on violence. Not does it cite NGONon-governmental organisation material, like the work on peacebuilding by Phil Vernon for International Alert. I was surprised not to see more on humanitarian aid, and humanitarian dilemmas, including the serious challenge of supporting state capacity in conflict zones where humanitarian aid is supposed to be neutral as between warring parties.

This was obviously a difficult review to carry out, covering work in places of which some are unsafe, and with public transparency hampered by incomplete access to documentation and analysis hidden behind the security barrier of the National Security Council.

The bigger issue, however, is one I tackled in an earlier piece, about what the key questions are in relation to cross-Government funds. Writing in November 2017, I said

‘To my mind, the exam question is not ‘Is it good to focus on big, cross-sectoral issues like peace and prosperity?’. The answer to that question is self-evidently ‘yes’ . . . Nor is the question ‘Is it good to have cross-ministerial collaboration in delivering international development?’. The answer to that question is also self-evidently ‘yes’ . . . (In that case) the debate should . . . focus on six questions:

1. Where do strategic questions get asked about the balance between bilateral and multilateral allocations?
2. Are the criteria for allocating aid pro-development?
3. Is oda spent through the FCO untied?
4. Does setting up separate funds lead to excessive earmarking of aid?
5. Does having multiple agents undermine unitary management and Balkanise the aid programme?
6. And, of course, is the FCO any good?’

My reading of the ICAI report is that it answers Q 6, and to some extent Q 2. It could have answered question 3, but did not (I discuss tying here, including the question of whether military spending should be market-tested). The ICAI Report does not deal with the other questions, viz 1, 4 and 5.

I see two problems, one of practice and one of policy.

The first problem relates to practice, which is that ICAI does not routinely look at alternatives to current practice. At least, it did not in this case. That really relates to Q 1: ‘OK, we agree that peace and security are legitimate topics for UK development, now, what is the best way to approach them?’. In November, I pointed out that the UK supports many other funds and programmes on this topic, including (and this is a non-exhaustive list):

• The UN-World Bank Fragility and Conflict Partnership Trust;
• IDA funding, including the State and Peacebuilding Fund;
• The United Nations Peacebuilding Fund;  
• The UNDP Crisis and Prevention Recovery Trust;
• The EU Africa Peace Facility; and
• The EU Emergency Trust fund for Africa.

Would it not have been interesting for the ICAI review to carry out a comparative analysis? And, if not ICAI, who? The National Audit Office? The Public Accounts Committee? The IDC? Or DFID’s own evaluation department?

A comparative perspective would have been useful also on the core competence question 6, viz is the FCOForeign and Commonwealth Office any good? The report is not complimentary about implementation by the FCO, but it is not clear that DFIDDepartment for International Development itself would have done any better on its own, with a relatively new programme in the difficult environments in which CSSF operates, and with all the problems of designing foolproof, ex ante theories of change in such circumstances. Nor is it clear that the FCOForeign and Commonwealth Office finds the job any easier than the UN or the EU. I am inclined to give the FCOForeign and Commonwealth Office the benefit of the doubt. But, in any case, surely, comparative analysis should be standard for ICAI?

The second problem relates to ICAI’s remit, specifically whether it is empowered to look at ‘policy’ questions, and what these are. The remit has been discussed over the years, not least by the International Development Select Committee, which has been rightly been concerned about possible overlap with its own mandate. For discussion, including of a previous review of ICAI by the Cabinet Office see here and here: so, No, these are not new issues.

ICAI’s current remit is formally described on its website as follows:

‘ICAI’s formal remit is to provide independent evaluation and scrutiny of the impact and value for money of all UK Government ODA. This involves:

• carrying out a small number of well-prioritised, well-evidenced, credible, thematic reviews on strategic issues faced by the UK Government’s aid spending
• informing and supporting Parliament in its role of holding the UK Government to account
• ensuring our work is made available to the public.

The remit and other aspects of ICAI’s work were reviewed by the Cabinet Office in a Tailored Review published in December 2017. The review analysed ICAI’s remit in the terms given above and concluded that ‘ICAI’s remit is clear and well understood by its stakeholders. ICAI distinguishes itself from other scrutiny bodies by focusing on assessing aid impact.’

The focus on impact is important, but there are some other important words in the remit, like ‘thematic’ and ‘strategic’. Qs  1, 4 and 5 from my list are all both thematic and strategic, so could they form part of ICAI’s mandate or not? Perhaps that needs to be tested in the next iteration of ICAI’s work programme, submitted to and approved by the IDC. I imagine that if the IDCInternational Development Committee were strongly supportive, then the boundaries of the remit could easily be tested.

If not, then either we have to wait for the next triennial review by the Cabinet Office, or another solution has to be found. Perhaps the IDCInternational Development Committee itself should pick up the baton in its own enquiries?