What should NGOs say & do about private schools?

Susannah Hares from Ark and I attended a meeting with the senior staff of an INGO last week, along with a couple of other external invitees, to help them think through their position towards private schools and fees in education. We wrote up our recommendations & takeaways on the Huffington Post.

First and foremost, it should reaffirm its principles: that education should be free; that government should be the guarantor but not necessarily the sole provider of education.

Second, it should focus its education reform efforts on setting high expectations for all children and improving accountability for all schools, whether public or private. School inspections, good data, robust assessments: these are all critical components of a good system accountability and should apply to public and private schools alike.

Third, in contexts in which substantial numbers of children already do attend private schools, the INGO could see if there are any clear market failures in the private sector that could be improved by government or NGO intervention. For example, training of teachers, or addressing information asymmetries through better community engagement.

Fourth, where public-private partnerships are planned or in place, the INGO should challenge government and private sector partners to place equity and quality at the heart of any programme. The UK academy programme started out under the Labour government with a very clear objective: to turn around the 200 worst performing secondary schools in the most deprived areas. This laser sharp focus on closing the achievement gap between rich and poor has changed the conversation about education in the UK: a school cannot be rated as “outstanding” anymore unless it can show it is delivering learning gains for children on free school meals.

Finally, and perhaps most radical of all, the INGO could set up its own network of non-profit PPP schools in a developing country. Demonstrating what high quality education for disadvantaged communities looks like would be a significant value add to any system, particularly if delivered through a PPP with the sustainability and public accountability that should bring. The experience of PEAS schools in Uganda shows that it is possible for an NGO to deliver better management practices and improve performance, at the same cost as local operators and with the inbuilt sustainability that comes from domestic government financing. And Ark’s own experience of running a high performing network of non-profit PPP schools in the UK shows how much an organisation can learn from actually running a set of schools, and how much more effective advisors to governments this can make them. Finally, establishing more non-profit providers in this space might calm some of the heated rhetoric around some of the international for-profit providers.

Why School Systems Matter, and How We Can Fix Them

This post first appeared on the CGD blog Views from the Center

Accountability in school systems is essential to deliver better learning and accelerate progress in developing countries. What is still really lacking—and what RISE is working towards (such as with Lant Pritchett’s coherence paper)—is a coherent and complete analytical framework capturing the key elements of a system of school accountability that can explain the divergent experiences we have seen in school reform.

The latest issue of the Journal of Economic Perspectives has a great symposium on schools and accountability, covering much of the research that motivated the development of RISE. Here are a few of the research highlights:

• Isaac Mbiti (part of the RISE Tanzania team) discusses “The Need for Accountability in Education in Developing Countries.”
   
• Brian Jacob & Jesse Rothstein dig into how we measure student ability in modern assessment systems (including some helpful discussion of what IRT can and can’t do).
   
• David Deming & David Figlio draw lessons from US experience with test-based school accountability systems, including caution about unintended consequences and ‘gaming,’ and noting that accountability seems to work best for low-performers.
   
• Julia Chabrier, Sarah Cohodes, and Philip Oreopoulos sum up what we can learn from charter school lotteries, re-emphasizing the point that charter schools seem to work best in neighbourhoods with poor performing schools.

Here though I’m going to focus on Ludger Woessmann’s article, "The Importance of School Systems” to help break down the possible reasons for the differences in performance across systems.

1. Why school systems matter: Your country matters for your test score


Woessmann starts by documenting the size of the gap in learning performance of 15-year-olds across countries, using combined PISA and TIMMS scores. As a rule of thumb, children learn on average between 25-30 points per year on this scale, which means that the average 15-year-old in the best performing countries (Singapore & Hong Kong) is roughly two years ahead of the average OECD country (for comparison, the UK scores roughly around this OECD mean).

Down at the other end, the average 15-year-old in Peru or Indonesia is well over 100 points below average—at least four years behind the UK. Ghana and South Africa are more like 200 points below average—6-7 years behind the UK/OECD average. This means that if a typical UK 15-year-old is in year (grade) 10, then the average Ghanaian 15-year old is just at grade 3 level.

Whilst the data for the poor countries we do have is really really bad, for most poor countries we don’t even have any data. In his blog post, “The Case for Global Standardized Testing,” Justin Sandefur highlights just what a tiny percentage of the population of low-income countries are covered by any internationally comparable standardized assessment. It’s essentially indistinguishable from the zero percent of students included in international assessments. Even the regional assessments (which provide some comparability) only cover 40 percent of kids from low-income countries, and only at primary school level. By contrast, 80 percent of children from high-income countries are covered at both primary and secondary level in international tests.

 

2. Country or school factors?

Is the difference in test scores among countries really about school quality though, or rather is it just national wealth, culture, or something else? Woessmann presents a mammoth international “education production function” exercise looking at correlates of test scores across over 200,000 students in 29 countries.
He finds that specific features of systems (such as school autonomy or centralised leaving exams) do correlate with student performance. Further, together these school system variables seem to matter more than student’s family background or school resources.
Moreover, features of the education system seem to matter 3-5 times more as additional variables in explaining outcomes than the level of school spending (adding 0.259 vs. 0.045 in explanatory power).

He also cites Abhijeet Singh’s research (part of the RISE India team), who demonstrates that gaps between students across countries are small when they enter school, and grows as they progress through the school system.

3. Correlation or causation?

The “education production function” approach that Woessmann takes here is explicitly based on correlations as it is difficult to impossible to nail causality down at the aggregate level.
But one first step is to look at changes in variables rather than levels. Take spending: whereas you might be able to find a positive correlation between GDP or the level of education spending and student learning, when you look at increases in spending, there is no correlation with increases in learning—which suggests that just looking at levels rather than changes is confounded by something else (e.g., while richer countries might spend more and have better test scores,  it’s not the spending that causes the test scores,  but insteada third factor correlated with both).

 

4. Money aside, what about other “inputs”?

Spending aside, Woessmann goes on to review what the rest of the literature says about three key inputs (for a look at other inputs, see the RISE working paper by Paul Glewwe & Karthik Muralidharan).

• Class Size: has “a limited role at best"—for example, a 2002 paper by Woessmann used quasi-random variation in small sizes to demonstrate the generally small and heterogeneous effect of class size in different countries.
   
• Instruction Time: is better, and "can increase educational opportunities.”
   
• Teacher Quality: (as measured by gains in student performance) "is related to better student achievement." (It’s worth noting that none of the “thin input” measures of teacher quality (such as qualifications or experience or pay) have substantial effects on learning.)

5. So what features of *systems* leads to more of these inputs and better performance?

• External exams: "A large literature has shown consistent positive associations between external exams and student achievement” (including cross-country analysis but also a cross-subject diff-in-diff approach within Germany).
   
• School autonomy: "School autonomy has a significant effect on student achievement, but this effect varies systematically … part of the negative effect of school autonomy stems from a lack of accountability” (including with a country fixed effects approach looking at changes over time).
   
• Private competition: "Cross-country evidence suggests a strong association of achievement levels with the share of privately operated schools” (robust to exogenous variation based on historical differences in religion).
   
• Tracking: "earlier tracking tends to raise the inequality of educational outcomes."

All of these points reiterate those made in various RISE-framing documents. School systems really matter—the differences in performance across systems are huge, and much of the gap seems to be attributable to real differences in school quality across systems, rather than other country-specific characteristics. And much of the discrepancy in school quality seems to be related to aspects of systems around accountability.
RISE is continuing to sort out and analyze these diverging experiences in school reform—and more importantly how developing countries can accelerate their progress towards global standards of learning.

Reaching universal secondary school won’t solve the learning crisis

Secondary school graduates in Jakarta, urban Ghana, and urban Kenya, have worse literacy skills than primary school graduates in rich countries. More here on the RISE blog.

What can we learn from Sociology about learning?

My summary on the RISE blog of the working paper from the RISE team main sociologist Susan Watkins (& Amy Kaler):

"One of the key things that should strike you from this particular framework and this way of disaggregating the WDR04 accountability triangle, is that the Delegation aspect of accountability relationships in education systems is probably the least studied. The WDR04 has spawned a range of literature looking at the role of better information and better incentives for performance, and there has been plenty of research looking at financing and resourcing. But there’s next to nothing on delegation - what do parents and governments actually expect from schools? Much of the research on the economics of education looks at the effects of schooling on later outcomes, for example on earnings or health, but that is not the same question as what was initially intended. 

This is where the RISE working paper by Susan Watkins and Amy Kaler comes in. Their paper focuses on the collective understandings of education by teachers, parents, and students."

In praise of universalisation (& why it is distinct from Westernisation)

This is great from Scott Alexander:

"the incorrect model of “foreign cultures being Westernized” casts Western culture as the aggressor, whereas the model of “every culture is being universalized” finds Western culture to be as much a victim as anywhere else. Coca-Cola might have replaced traditional yak’s milk in Mongolia, but it also replaced traditional apple cider in America. A Hopi Indian saddened that her children no longer know the old ritual dances differs little from a Southern Baptist incensed that her kids no longer go to church. Universal values have triumphed over both."