Two very smart folks, Mark Rozenzweig and Martin Ravallion have reviews of Poor Economics in the latest Journal of Economic Literature (thanks to Abhi and Andrea for the papers). Obviously self-recommending when smart economists review smart economists. But there does seem to be a bit of a rehashing.
Martin's biggest score is the "where the hell is China?" line. Some of the other criticisms are a bit weaker.
Then
Mark scores his biggest hit in the final footnote on the last page of his article;
The other serious and neglected criticism for me is on general equilibrium, raised by Daron Acemoglu in the Journal of Economic Perspectives. What if you measure a positive impact of a program on earnings, but those are coming at the expense of others? A training program that increases earnings might just be equipping some individuals to out-compete others in the market, rather than necessarily increasing aggregate productivity, in which case scaling the program ain't gonna work.
So maybe I've missed them - but has anyone seen a convincing rebuttal to the moral hazard and general equilibrium critiques of micro aid project impact evaluation?
-----
Update: A couple of things I missed in my haste - Abhi points out that Rosenzweig makes good points on the sometimes tiny effect sizes lauded in Poor Economics (e.g. where "15% increase" translates to something like 2 weeks schooling or 50 cents), and that RCTs can focus our attention away from the big (important?) questions, but I felt this criticism is pretty well rehearsed.
Update 2: Also Ravallion loses points for his cliched title: "Fighting Poverty One Experiment at a Time". "x one y at a time" is a boring, tired, tired, catchphrase.
Update 3: Ravallion gains points for coining "regressionistas."
Martin's biggest score is the "where the hell is China?" line. Some of the other criticisms are a bit weaker.
Another likely bias in the learning process is that J-PAL’s researchers have evidently worked far more with nongovernmental organizations (NGOs) than governments.Which is a bit of a cheap shot, and a bit innacurrate. Researchers have worked with whoever will let them experiment, which yes initially was NGOs but is increasingly governments - see Peru's Quipu commission, Chile's Compass commission, the teaching assistant initiative in Ghana, working with the planning Ministry in South Africa, experimenting with police service reform in Rajasthan, even Britain's Behavioural Insights Unit.
Then
how confident can we really be that poor people all over the world will radically change their health-seeking behaviors with a modest subsidy, based on an experiment in one town in Rajasthan, which establishes that lower prices for vaccination result in higher demand?Ummmm... well thats why J-PALs policy recommendation for health pricing is based on 6 different studies....
Mark scores his biggest hit in the final footnote on the last page of his article;
Also absent is a discussion of the standard but major problem in the implementation of any programs or transfers targeted to the poor and that do not really spur development—moral hazard."Moral hazard" works at both the individual and national government level. If you get aid, you are probably less likely to work hard. The critical question is the magnitude of this effect. I think that on balance the positive value of effective aid outweighs the moral hazard, but that is more of a feeling than an evidence-based proposition. This is also one of the key points made by aid critics Bauer/Easterly/Moyo. Not necessarily that aid doesn't work, as Banerjee/Duflo would like to present their argument, but that even if aid does work, the negative moral hazard effect might outweigh the positive. I haven't seen this argument really addressed at all.
The other serious and neglected criticism for me is on general equilibrium, raised by Daron Acemoglu in the Journal of Economic Perspectives. What if you measure a positive impact of a program on earnings, but those are coming at the expense of others? A training program that increases earnings might just be equipping some individuals to out-compete others in the market, rather than necessarily increasing aggregate productivity, in which case scaling the program ain't gonna work.
So maybe I've missed them - but has anyone seen a convincing rebuttal to the moral hazard and general equilibrium critiques of micro aid project impact evaluation?
-----
Update: A couple of things I missed in my haste - Abhi points out that Rosenzweig makes good points on the sometimes tiny effect sizes lauded in Poor Economics (e.g. where "15% increase" translates to something like 2 weeks schooling or 50 cents), and that RCTs can focus our attention away from the big (important?) questions, but I felt this criticism is pretty well rehearsed.
Update 2: Also Ravallion loses points for his cliched title: "Fighting Poverty One Experiment at a Time". "x one y at a time" is a boring, tired, tired, catchphrase.
Update 3: Ravallion gains points for coining "regressionistas."
12 comments:
Is moral hazard ever considered in the same way in OECD countries? Do we concern ourselves that if people have reliable access to clean water or free vaccinations for their children, that they won't work hard? Sure in terms of some welfare state issues (workforce participation, poverty traps etc) we do, but they tend to be at the much higher quality problem of relative rather than absolute poverty.
I think its clearly a big problem with working-age benefits in the UK ---> good argument for first focusing on children, elderly and disabled. But also the wonderful success of Labour's targeting of "child poverty", with is hard for conservatives to argue against, but essentially means transfers to poor households. Moral hazard is also an issue at government level as well as individual level. If aid continues to support health services, poor country governments will continue to spend less on health budgets. As I said, I do worry about these issues but I don't think that they dominate the pure positive impact of the resource transfer. But there ain't a whole lot of evidence either way.
What are the case examples of where a country has had a decline in sector specific aid (say for health services), and made up the difference through internal reallocation from other portfolios (say from defence)? In most circumstances it might be difficult to tease apart cause and effect (lower aid=>more spend or more spend=>lower aid), not least due to the un/mal coordination in donor approaches, unless they were very disruptive circumstances (eg regime change, Zim or Fiji being kicked out of the Commonwealth, etc) which would tend to overwhelm any analysis for many other reasons. That might contribute to the lack of evidence.
Not sure that the J-Pal chart comparing 4 completely different products across 6 locations hits the external validity mark as well as as establishing a relationship between price and demand for one product in 6 locations...
Plus, is Ravallion saying that no researchers are working with governments, or just that the majority are working with NGOs? Not sure you've shown evidence that the latter statement isn't true...
On the first matter, I'll refer you back to my Proven Mean Initiative, which I know you enjoyed so much:
http://aidthoughts.org/?p=1727
GE is rarely being addressed. Only example that comes to my mind is Attanasio/Meghir/Santiago (RES, 2012), which show that Progresa led to an increase in child labor wages which mitigated the subsidy's effect on enrollment. GE effect was only small though. Studies that address GE effects are even extremely rare in the case of developed countries (examples: Lise/Seitz/Smith & Heckman/Lochner/Taber), so thats frontier stuff.
MH? Well, that's really an ambitious one. No study I know of which separates the (hopefully) positive productivity from possible negative MH effects. I guess what you would need is the counterfactual of an aid inflow / project that did not bring MH with it - so you need exogenous variation in sources of MH that would not be related with outcomes and the productivity of aid / the project. In most cases we probably can only observe at most (causal) net effects, which, however, would be fine in some cases when it comes to policy recommendations (even though a theory-minded economist would want to dig deeper). For example, subsidizing deworming medicine might lead to reductions in the individuals' own efforts to protect themselves. Everyone would agree, but what policy recommendation would follow from findings like "MH mitigation reduced the gross effect of the policy on health by 20%"? Also, certain interventions are trying to "push" (or nudge) individuals towards behaviors which are thought to improve their lot in the long-run - and thus in poor people's very self-interest - but which can not be easily attained due to behavioral issues, credit contraints, or lack of information (of course this idea is patronizing, but so are many forms of aid). My question is to what extent we should be worried about MH in such circumstances. Yes, families who send their kids to school due to a conditional subsidy might reduce other efforts to guarantee their children's education - but if the grant led to an increase in edcuation, i.e. if the net effect was positive, would we really care about any possible mitigation by MH? If a intelligently timed low fertilizer subsidy could help people to overcome their present-bias, and would thus make them change their behavior, what would be the marginal benefit of knowing that effect would have been 10% stonger if MH had not been stimulated simultaneously?
"
i.e. if the net effect was positive, would we really care about any possible mitigation by MH? "
It might tell us something about whether education should be the priority or not.
"I think its clearly a big problem with working-age benefits in the UK". Really? The vast majority of unemployment in the UK at present is cyclical. And the vast majority of those unemployed will be short term unemployed. The number of people who decide not to work but rather indulge in the luxory of unemployment benefit land is actually very very very small as a proportion of the population who could do this if moral hazard was a real issue.
And, more generally, I'd say the onus is actually on Ravillion to prove moral hazard exists here than on JPAL to prove that it doesn't.
Depends on your policy objective. If you wanna minimizing MH, yes. But if you care about bulding up human capital - I would not see much benefit from identifying MH effects (unless for external validity, of course).
My biggest problem with J-PAL is that I can'd find a frickin' RSS feed anywhere on their site. Please point it out if I've missed it.
Not that that's the worst problem an NGO could have...
Bada bing:
http://www.povertyactionlab.org/evaluations.xml
When the UK tightened access to job-seekers benefit numbers on incapacity benefit shot up in response. There are clearly massive work disincentives at the margin --> someone moving from full benefits to full-time at minimum wage is not making a massive leap in earnings.
Post a Comment