First RISE Working Papers

The first set of working papers from RISE (Research on Improving Systems of Education) are out.

Paul Glewwe and Karthik Muralidharan have an updated review of everything we know about rigorous evaluations of interventions to improve learning in developing countries (paper here, my comment on the RISE blog here).

Rukmini Banerji describes how a disruptive pedagogical innovation spread (and didn’t) in Bihar, and Kara Hanson tell us what education can learn from health systems research.

Mari Oye also has a blogpost up about the UN Myworld survey and the SDGs.

Coming soon, Lant’s consolidated explanation of what an Education System actually is, grand general theory of why some things work sometimes but not all the time, and tentative framework for diagnosing systems for constraints and prioritising action. Watch this space.

We have no idea what countries are spending on education

Listen to some international education people and you get the impression that the education problem is mostly solved if we could just spend more money. The story goes something like “Poor countries spend X on education, if they could 1.5X then all the kids could get a good education, they can’t afford 1.5X, so we should fill the gap with aid."

The reality is, even if it was the case that just filling the gap would solve the problem (which is dubious to say the least) , we don’t really even know what the gap is.

This is Silvia Montoya, Director of the UNESCO Institute for Statistics:

"governments need detailed and disaggregated data to ensure that their resources are allocated equitably and effectively within their education systems. At the same time, donors need the data to better evaluate whether the aid they provide is an incentive for governments to increase spending commitments or if they are crowding-out domestic resources.

For the moment, the availability and completeness of education finance data is unfit for these purposes, with less than one-half of countries able to regularly report key information, such as total government expenditure on education” (my emphasis)

Nevermind the purpose of accountability and transparency to the citizens of developing countries...

Good luck to the new Commission on Financing Global Education!

Does foreign aid harm political institutions?

Good news for reflective aid business -types who like agonising about what the point of it all is and sometimes wondering whether we’re even making things worse (err... talking about a friend... not me...). Also even good news for developing countries I suppose.

A new paper in the Journal of Development Economics by Sam Jones & Finn Tarp* using new data on aid (from aiddata.org) and institutions (from the Quality of Government Institute) finds no evidence that aid has undermined institutions on average, if anything there seems to be a positive relationship. I’m less confident in the positive findings than reassured that in *none* of their various different approaches is the relationship negative.

Now you’re probably thinking “what about the 2006 CGD review paper by Todd Moss, Gunilla Pettersson & Nicolas Van de Walle, described by Blattman as "the best summary I know of the evidence”, which concluded that aid could have a harmful effect on institutional development”? Well the word “could” is important there - that conclusion was somewhat speculative, and this new evidence from Jones & Tarp fills an important gap in terms of systematic quantitative evidence on this topic, and should probably shift your priors at least a little in that direction.

I wonder what Angus Deaton would say?

---

* Thanks to UNU-WIDER the paper is open-access, which is great for what it is, but obviously having public institutions pay private journal owners something greater than the cost of production isn’t an ideal long-run equilibrium, and we really need something that fundamentally shifts the whole publishing industry.

The State of the Humanitarian Aid System 2015

“ALNAP” launched today the 2015 “State of the Humanitarian Aid System” Report.

One of the key findings highlighted in their fancy infographics:

"44% of aid recipients surveyed were not consulted on their needs by aid agencies prior to the start of their programmes”.

In totally unrelated news, the DFID-ODI-CGD High Level Panel on Humanitarian Cash Transfers chaired by Owen Barder published it’s report a few weeks ago, arguing that much more use should be made of cash transfers, because most of the time they are more cost effective than giving out stuff.

In further totally unrelated news, DFID published two press releases today highlighting substantial non-cash aid in response to humanitarian crises in the Central African Republic and Malawi.

In Owen’s words: "the questions should always be asked: “Why not cash? And, if not now, when?”"

Is rescuing migrants from the Med good value for money?

Thousands of people die each year trying to cross the Mediterranean to seek asylum in Europe. Christopher and Regina Catrambone, American and Italian entrepreneurs, decided to take matters into their own hands and set up their own private rescue mission.

Naturally when I read that

"a fundraising drive by the activist organisation Avaaz reached $500,000, slightly less than a month’s costs”,

I started wondering about cost effectiveness. Elsewhere the Guardian article states

"Setting up Moas was not cheap, with monthly operating costs of up to €600,000"

and

"The Phoenix rescued 1,462 people in 10 weeks"

So let’s go with the higher figure of €600,000 per month - over 10 weeks (2.3 months) that is a total cost of €1.4m (roughly $1.6m or £1m). And to save 1,462 people, that is a cost of £700 (~ $1000) per death averted.

Is that a lot or a little? As Owen has pointed out, the UK NHS considers anything less than £100,000 per death averted to be good value for money.

At the other extreme, childhood vaccinations, "long recognized as among the most cost-effective uses of limited health resources in low-income countries” (Disease Control Priorities) cost $275 per death averted.

At face value, Catrambone’s "Migrant Offshore Aid Station" (MOAS) looks like a pretty good value for money philanthropic bet.

Effective Altruism, RCTs, NGOs, & the Government End-Game

Good Ventures just gave a $25 million unrestricted grant to Give Directly on the advice of Givewell. That’s a lot of good news in one sentence, but it’s not even the best part. Givewell buried the lede when they mention around paragraph 20 that;

"GiveDirectly plans to discuss partnerships with the following types of institutions:

- Donor aid agencies
- Developing country governments (national and local). (For example, several governors in Kenya have already approached GiveDirectly about running cash transfer programs in their counties.)"

That’s what it’s all about. To really get sustainability and scale in social policy you need government involvement - that’s why the best NGOs combine a mixture of immediate direct service delivery in places where government just doesn’t have the capacity to deliver, with support to interested governments to build that capacity for the longer-term, often at the local level where administrators struggle to actually implement well-designed central policy documents, and with innovation in new models of service delivery, that governments might later adopt, of which GiveDirectly is clearly a strong example. Similarly whilst Innovations for Poverty Action and J-PAL may have started off following that recently infamous Kremer-Miguel deworming study by working on service delivery through small NGOs, their focus is on things that can work at scale, and having built a reputation through working with NGOs have been able to transition to working with governments (for example in Ghana and Peru).

As Jessica Brass writes,

"Government and NGOs learn from each other to improve what they do. In particular, many government agencies notice the successes achieved by NGOs and, whether intentionally or not, mimic their actions"

So yes, maybe some of the effective altruists can be accused of being philosophers not development wonks, and potentially even naive about politics, but for every anecdote-backed theoretical case for how aid might undermine the process of building citizen-state accountability, I can come up with an anecdote-backed theoretical case for how aid can support improved governance through innovation in service delivery models, and until we get some quantitative evidence on the issue, I don’t see how else we’re going to resolve the debate.

Did I miss anything?

New education economics papers

A few papers caught my eye from last month's repec new education economics papers feed. All from developed countries, but such is economics, a lot of the interesting new research happens on rich countries where the researchers are more likely to know about interesting policies and institutional features to study, and where there is better data (both problems which RISE is seeking to address, by encouraging collaborations between developing country-based researchers and leading academics based at top universities in rich countries, and also by funding new data collection in developing countries).

"Quantifying the Supply Response of Private Schools to Public Policies” by Michael Dinerstein and Troy Smith looks at a reform in New York which increased the budget for some public schools, finding an increase in enrolment at these schools, and that nearby private schools lost business and were slightly more likely to shut down. In an interesting twist, whilst the reform improved quality at the public schools that received extra money, the movement of some students from higher quality private schools to lower quality public schools meant that overall outcomes from the school system were not improved. All of which reminds me of the recent story from Rwanda that some private schools seem to be going out of business by the growth of public schools. What is that shift doing to the overall quality mix?

“The Information Value of Central School Exams” by Guido Schwerdt & Ludger Woessmann compares students in Germany who graduated from states which use a centralized common school-leaving exam to those with a local school-set leaving exam. Better grades are roughly three times more valuable in the labour market when they come from centralized exams than from school-set exams. In Lagos, private school associations are currently in the process of joining together to put their students through common school leaving exams for partly this reason.

Nicola Bianchi’s Job Market Paper looks at "The General Equilibrium Effects of Educational Expansion” - when Italy expanded STEM higher education in 1961, enrolment increased by 200%. However - those students who enrolled didn’t earn any more than they would have had they not enrolled, because the massive increase in the supply of qualified students reduced the labour market premium for that qualification, as well as the quality of education suffering due to congestion and peer effects. Which of course should remind you of Lant’s classic “Where has all the education gone?"

How much does the new deworming replication matter for Effective Altruists?

It doesn’t at all, as far as I can tell. As Calum points out, what matters is the systematic review of evidence not one study. And the new Cochrane systematic review doesn’t seem to have responded to the criticism from Duflo et al to their 2012 review, that it ignores quasi-experimental and long-term evidence on positive impacts of deworming (specifically Bleakley 2004, Ozier, and Baird et al).

A replication of the famous Miguel and Kremer deworming paper that launched the whole RCT in development economics movement, is published in the Journal of International Epidemiology today (along with comment from Hicks, Kremer, and Miguel, and reply from the replication authors), with coverage in the Guardian and by Ben Goldacre for Buzzfeed.

You may remember Berk Ozler's review of the draft of the replication paper back in January - concluding

"Bottom line: Based on what I have seen in the reanalysis study by DAHH and the response by HKM, my view of the original study is more or less unchanged."

You can probably expect to see more on the replication coming from @cblatts, which I’m not going to get into, but back in 2012, Givewell were convinced that the Cochrane review shoudn’t change their recommendation to donate to the Schistosomiasis Control Initiative or Deworm the World.

The ambiguity does make me a little queasy, and pushes me more in the direction of GiveDirectly (I see basically zero risk that giving $1000 to someone on a very low income can really be totally wasted, in the way that an ineffective drug could theoretically have zero impact).

Migration fact of the day

"Today, approximately 7 million Indians work in six GCC countries, which is more than 50% of estimated 13 million foreign workers present in the GCC. The Indian workers in GCC remit about US$40 billion i.e. around 57% of the total remittances, i.e. US$70 billion India receives annually. Besides contributing significantly to the national forex reserves, the remittances received directly by the workers’ families help in poverty alleviation, support local business, promote entrepreneurship and generate employment."

That’s Zakir Hussain on the World Bank blog. Worth remembering this context next time you read a scandal about the poor treatment of Indian workers in the Gulf.

Calling Education Researchers…

I just got back from the fourth of seven events being held around the world drumming up interest in bidding for the RISE “Research on Systems of Education” project. There is £21 million of DFID money to be split between 5 country research teams (with a preference for bids from Ethiopia, Nigeria, Ghana Pakistan, India, Indonesia, Vietnam, or other DFID focus countries) to study reforms that are happening to education systems that might credibly have a big impact of student learning. EOIs due 23rd August. 

strong turnout for Abuja @riseprogramme meeting and #educationsystems reform discussion! @LeeCrawfurd pic.twitter.com/GV3gsNQQvX

— Mari Oye (@marimoye) July 13, 2015

There is plenty more information on the CGD website and the new RISE website,  but to make things really easy, here are a few key links about the project and how to bid (very helpfully put together by Mari).

• RISE Research Director Lant Pritchett and Justin Sandefur presenting the research agenda (Video)
• Slides (PDF)
• Expression of Interest scoring criteria  (PDF)
• Full RISE Terms of Reference (needed for proposal due November) (PDF) 
• Frequently Asked Questions
• Online form for submitting any additional queries
• Registration for matchmaking for looking for research partners (optional) 
• Lant Pritchett's book, "The Rebirth of Education"

New evidence on (lack of) external validity

"Site selection bias" can occur when the probability that a program is adopted or evaluated is correlated with its impacts. I test for site selection bias in the context of the Opower energy conservation programs, using 111 randomized control trials involving 8.6 million households across the United States. Predictions based on rich microdata from the first 10 replications substantially overstate efficacy in the next 101 sites. Several mechanisms caused this positive selection. For example, utilities in more environmentalist areas are more likely to adopt the program, and their customers are more responsive to the treatment. Also, because utilities initially target treatment at higher-usage consumer subpopulations, efficacy drops as the program is later expanded. The results illustrate how program evaluations can still give systematically biased out-of-sample predictions, even after many replications.

H. Allcott in the QJE

Celebrating more Brits

The population of the UK has increased by 500,000 in the last year.

Unlike what you may read elsewhere, this is great news.

- British people are great - having more of us is better
- London is the best part of Britain, and not coincidentally the most populous and densely populated part
- Population growth is concentrated in cities
- Larger cities support economies of scale, more specialisation and diversification, enabling the clusters of activity and agglomeration that drive innovation
- A larger population means a greater supply of innovators
- A larger population means a greater demand for innovators, and a bigger market for producers
- A larger population means more people to share the burden of fixed costs, including national debt

Of course there are costs to crowding, and we need to plan for more infrastructure provision (not least building more housing), but that’s just part of life and really shouldn’t be beyond our wit.

Global Organ Trade

Here’s a great idea from Al Roth, the 2012 Economics Nobel Prize winner.

Al got his prize for developing his theoretical matching ideas into a computerized kidney exchange - so if you want to donate a kidney to a family member but you aren’t the right match, you can find another pair of people in the same situation from a different city and criss-cross the pairing, so both kidney transplants can go ahead.

In his new book (reviewed here by Alex Tabarrok), Al proposes extending the kidney exchange internationally.

"Mr. Roth, however, wants to go further. The larger the database, the more lifesaving exchanges can be found. So why not open U.S. transplants to the world? Imagine that A and A´ are Nigerian while B and B´ are American. Nigeria has virtually no transplant surgery or dialysis available, so in Nigeria patient A’ will die for certain. But if we offered a free transplant to him, and received a kidney for an American patient in return, two lives would be saved.

The plan sounds noble but expensive. Yet remember, Mr. Roth says, “removing an American patient from dialysis saves Medicare a quarter of a million dollars. That’s more than enough to finance two kidney transplants.” So offering a free transplant to the Nigerian patient can save money and lives. It’s hard to think of a better example of gains from trade (or a better PR coup for the U.S. on the world stage). Better matching with computerized markets is saving lives, but more than 100,000 people are still waiting for kidneys in the United States alone."

Graduate Jobs in South Sudan

Looking for your first job in international development? Charlie Goldsmith is hiring in South Sudan;

"International development work is generally best done by people of the country in question: there is no shortage of talent in and from any of Somalia, South Sudan, DRC, or any other FCAS place you might name, only the conditions in which it might be deployed and developed. 

But there is still a role in development work for people from the Global North if they have the right skills, the humility, understanding and connection to apply them well where they are sent, and hopefully the intention to continue to apply them in this work for the medium term. That doesn’t just mean water engineers and hard-bitten Treasury hands, it can also mean the high-achieving, high-potential generalists/fast-streamers that any organisation, the world over, would be glad to have. 

But for those bright young people, getting into international development is not always straightforward: it can seem unwise to set off to a fragile state with no particular fixed plan, as many of those now working in this sector first did; getting to, and staying in, some of the places we work is expensive even if you do have systems already set up, let alone if you’re doing this the first time, straight out of College. 

This Autumn, we are therefore going to be looking to hire up to six CGA Fellows to send to South Sudan, who will be either immediate or fairly recent graduates."