The Lancet's editors don't get evaluation (sadface)

So Matt beat me to the punch on Friday on the Lancet Millennium Village retraction. Since then I've being trying to think of a polite way of expressing my total dismay and despair at the tripe written by the Lancet editors in response to the retraction (for which, by the way, a little bit of Kudos to Pronyk et al).

The Lancet editors write:

The Millennium Villages project team has quickly and commendably corrected the record after understanding the validity of the challenge it received. But the withdrawal of this element of the paper does not detract from the larger result—namely, that after 3 years Millennium Villages saw falls in poverty, food insecurity, stunting, and malaria parasitaemia, together with increases in access to safe water and sanitation.

Which is just total nonsense. For all we know, poverty fell in the Villages at the exact same rate as everywhere else. That is not an important result to be celebrated. I challenged Lancet editor Richard Horton on twitter as to why he would continue to emphasise this non-result, and he responded with yet more nonsense;

1. 
   richardhorton1 @rovingbandit To be fair, there were falls in each of the 5 MDG-1 poverty/nutrition measures, but these were not statistically significant. 18 May 2012 from web

That isn't even true. The first of the measures - wealth - is the opposite of poverty. It is *wealth* that fell (statistically insignificantly) in the Villages relative to comparisons. I despair. And kind of question my own sanity. Despite what Tim Worstall says, I'm really not a scientist, but its pretty galling that people say economics is not a science like the physical sciences when this is the kind of guff published by the world's top medical journal.

Bill Easterly has a whole long list here of more terrible social science published in medical journals. At the bottom of the post, Ben Goldacre comments

i think journals publishing things outside of their field of expertise is risky, but i wld caution against developing a world view that economics journals are in a better shape overall than medical ones. as someone who flits into both, there are lots of things that are routine in medical journals, to a greater or lesser extent, but notably almost unheard of in economics. stuff like declarations of conflict of interest, structured write-ups, registering a protocol in advance of doing a study, etc. all of which wld be great to see more of outside medicine.

All of which is true. In particular I am struck by how easily readable a short, structured, 4 page Lancet write-up is. There are definitely lessons to be learnt across disciplines both ways. It's just an incredibly sad state of affairs that one of the lessons that journals of medicine, the discipline that gave us randomized controlled trials, needs to learn from economics, is a more careful attention to statistics and causality.