On Monday, 21 January 2015, I had the chance to assist in a very interesting speech given by Dr. Eva Terberger from the KfW Bank in Germany on this topic at Vienna University of Economics. Dr. Terberger is the head of the independent evaluation unit within KfW.
It was interesting to see how, influenced by the global discourse on rigorous evaluations and randomised controlled trials (especially Ester Duflo, Poverty Action lab) as the gold standard of evaluation, KfW made several attempts to design and carry out experimental evaluations for infrastructure projects. Resulting from these experiences Dr. Terberger shared some of the reflections on RCTs:
- in fragile contexts randomisation is hardly possible (for example in Mali a RCT was jeopardized by the outbreak of the civil war)
- there is only limited possibility to transfer knowledge between programmes as context matters so much
- the cost to carry out such experiments is just extremely high with a quite limited information gain, so they found the method not very cost-effective
- to measure the impact of large infrastructure projects as financed by KfW is almost impossible with this method as there are no control groups to define
In conclusion, KfW prefers to adopt a mixed methods approach with expert judgements based on data, facts and appraisals.
It will be interesting to see if other institutions in the world will adopt RCTs as THE standard in development evaluation or if RCTs in development evaluation will remain
itself an experiment or a short-term phenomenon.