Is it really a bubble? Whilst there has been rapid growth of impact evaluations of aid projects across agencies, and plenty of internet chatter, the vast majority of aid spending still does not get properly evaluated.
For example, while there has been a substantial growth in impact evaluations of the World Bank development projects, only 8.8% of World Bank investment loans in 2009/10 had an impact evaluation. In 1999/00 the proportion was 2.4%. ----- Martin Ravallion, World Bank Research Director
5 comments:
What about the supply side of RCTs? They are still inherently research-driven - we have had lots of RCTs because they do well in journals, not because well-meaning donors have really taken them seriously. Surely it's possible for a bubble on the research side, with academics moving on to another method/or types of questions that RCTs don't cover (and don't wave the Rajastan RCT in my face, that's testing very small scale rule sets, not how to come up with better functioning systems).
I think the research-driven agenda HAS had an effect on minimum standards for policy and the demand side, and even if researchers move onto the next fad the M&E game has been changed and I'm not sure it can go back. There is just a big lag between leaders like MCC and IADB who have rigorous evaluations for big chunks of their portfolios, and other donors who have started talking about "impact evaluation" but in reality mean no such thing. I think the basic concept of impact and of thinking about a counterfactual is pretty fundamental whatever methodology you use to estimate it, but also something which is still new to far too many policy people (such as David Cameron claiming 50% of TESCO work experience kids left benefits, failing to note that 50% of everyone leaves benefits in the same period).
I agree on most of these points - I think the key thing is that we shouldn't continue to use "RCT" and "impact evaluation" interchangeably......
Fair enough. Next you'll be telling me invisible children shouldn't be using "Uganda" and "Africa" interchangeably. I saw you running on St Aldgates today and shouted but you must have been in the zone.
Yeah I was in the zone - running the Teddy Hall relay. I had a baton that I needed to hand off to Bet, stat!
Post a Comment