New evidence on (lack of) external validity

"Site selection bias" can occur when the probability that a program is adopted or evaluated is correlated with its impacts. I test for site selection bias in the context of the Opower energy conservation programs, using 111 randomized control trials involving 8.6 million households across the United States. Predictions based on rich microdata from the first 10 replications substantially overstate efficacy in the next 101 sites. Several mechanisms caused this positive selection. For example, utilities in more environmentalist areas are more likely to adopt the program, and their customers are more responsive to the treatment. Also, because utilities initially target treatment at higher-usage consumer subpopulations, efficacy drops as the program is later expanded. The results illustrate how program evaluations can still give systematically biased out-of-sample predictions, even after many replications.

H. Allcott in the QJE

Celebrating more Brits

The population of the UK has increased by 500,000 in the last year.

Unlike what you may read elsewhere, this is great news.

- British people are great - having more of us is better
- London is the best part of Britain, and not coincidentally the most populous and densely populated part
- Population growth is concentrated in cities
- Larger cities support economies of scale, more specialisation and diversification, enabling the clusters of activity and agglomeration that drive innovation
- A larger population means a greater supply of innovators
- A larger population means a greater demand for innovators, and a bigger market for producers
- A larger population means more people to share the burden of fixed costs, including national debt

Of course there are costs to crowding, and we need to plan for more infrastructure provision (not least building more housing), but that’s just part of life and really shouldn’t be beyond our wit.

Global Organ Trade

Here’s a great idea from Al Roth, the 2012 Economics Nobel Prize winner.

Al got his prize for developing his theoretical matching ideas into a computerized kidney exchange - so if you want to donate a kidney to a family member but you aren’t the right match, you can find another pair of people in the same situation from a different city and criss-cross the pairing, so both kidney transplants can go ahead.

In his new book (reviewed here by Alex Tabarrok), Al proposes extending the kidney exchange internationally.

"Mr. Roth, however, wants to go further. The larger the database, the more lifesaving exchanges can be found. So why not open U.S. transplants to the world? Imagine that A and A´ are Nigerian while B and B´ are American. Nigeria has virtually no transplant surgery or dialysis available, so in Nigeria patient A’ will die for certain. But if we offered a free transplant to him, and received a kidney for an American patient in return, two lives would be saved.

The plan sounds noble but expensive. Yet remember, Mr. Roth says, “removing an American patient from dialysis saves Medicare a quarter of a million dollars. That’s more than enough to finance two kidney transplants.” So offering a free transplant to the Nigerian patient can save money and lives. It’s hard to think of a better example of gains from trade (or a better PR coup for the U.S. on the world stage). Better matching with computerized markets is saving lives, but more than 100,000 people are still waiting for kidneys in the United States alone."

Graduate Jobs in South Sudan

Looking for your first job in international development? Charlie Goldsmith is hiring in South Sudan;

"International development work is generally best done by people of the country in question: there is no shortage of talent in and from any of Somalia, South Sudan, DRC, or any other FCAS place you might name, only the conditions in which it might be deployed and developed. 

But there is still a role in development work for people from the Global North if they have the right skills, the humility, understanding and connection to apply them well where they are sent, and hopefully the intention to continue to apply them in this work for the medium term. That doesn’t just mean water engineers and hard-bitten Treasury hands, it can also mean the high-achieving, high-potential generalists/fast-streamers that any organisation, the world over, would be glad to have. 

But for those bright young people, getting into international development is not always straightforward: it can seem unwise to set off to a fragile state with no particular fixed plan, as many of those now working in this sector first did; getting to, and staying in, some of the places we work is expensive even if you do have systems already set up, let alone if you’re doing this the first time, straight out of College. 

This Autumn, we are therefore going to be looking to hire up to six CGA Fellows to send to South Sudan, who will be either immediate or fairly recent graduates."

Innovative education financing modalities from 1862

Apparently payment by results isn’t quite so new.

"The reference here is to England’s Payment by Results school reform of 1862. According to the Revised code of the Department of Education in Britain in 1862, capitation grants to schools were reduced and payments were made to school on the basis of students passing on-site examinations given by inspectors in reading, writing and arithmetic. There has been much debate among historians about what the payment for results reform really accomplished. Mitch (2010) looks at educational performance across British counties over the 30 years of the policy and shows that during this time, inequalities across counties declined. But in the absence of data on trends prior to the reform, it is hard to establish whether this was a consequence of the reforms. In contrast, the quote here paraphrases Matthew Arnold, a poet and school inspector who returns from a trip to France and notes: “I find in English schools…..a deadness, a slackness and a discouragement….This change is certainly to be attributed to the `Payment by Results’ school legislation of 1862.”(Great Britain Privy Council 1868, Page 290)."

From a new paper by Andrabi, Das, & Khwaja

The learning crisis in Sierra Leone

"Mohammed’s father is an illiterate petty trader. Although he never got any school himself, he has always been determined that Mohammed [13] should get a good education. When Mohammed joined us, we asked him, as we ask all our students, to complete a word reading assessment. The assessment, which we administer one-to-one in the child’s home, involves reading out a list of 90 words that increase in complexity and difficulty, and from the number and difficulty of the words read correctly an inference can be drawn about the student’s reading age based on UK norms. 

Mohammed got stuck straight away. He barely made it past the first line or two – words like ‘he’, ‘she’, ‘cup’, ‘said’. Mohammed’s father couldn’t read but he could see that his son was struggling and did his best to encourage him. “Try your best Mohammed”, he said. And Mohammed did. He kept trying. But no matter how long he took he couldn’t recognise the words, and eventually we had to call time on the test. Mohammed’s father was heartbroken. “I’ve paid all this money for school,” he said, “but his head is empty.” 

Unfortunately, Mohammed’s story is far from unique."

From an essay by Paul Skidmore, CEO of the Rising Academy Network