tag:blogger.com,1999:blog-48672303081599015472024-03-16T01:09:46.525+00:00Roving BanditProbably the best economics blog (founded) in South SudanLeehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.comBlogger1177125tag:blogger.com,1999:blog-4867230308159901547.post-163925507519370122020-01-22T11:35:00.002+00:002020-01-22T11:35:45.701+00:00CGD Blogging 2019<div dir="ltr" style="text-align: left;" trbidi="on">
<div>
Here's a roundup of the 9 blogs I wrote or co-wrote for CGD last year.<br />
<br /></div>
<div>
August </div>
<div>
<br /></div>
<div>
<a href="https://www.cgdev.org/blog/who-smokes-developing-countries-implications-tobacco-tax">Who Smokes in Developing Countries? Implications for a Tobacco Tax</a> </div>
<div>
<br />
September</div>
<div>
<br /></div>
<div>
<a href="https://www.cgdev.org/blog/how-algorithm-can-make-school-choice-fairer">How an Algorithm Can Make School Choice Fairer</a></div>
<div>
<br /></div>
<div>
<a href="https://www.cgdev.org/blog/eight-updates-global-education-unga">Eight Updates on Global Education from UNGA</a></div>
<div>
<br /></div>
<div>
<a href="https://www.cgdev.org/blog/want-more-spending-education-give-kids-vote">Want More Spending on Education? Give Kids the Vote</a></div>
<div>
<br /></div>
<div>
October </div>
<div>
<br /></div>
<div>
<a href="https://www.cgdev.org/blog/unchained-melody-role-private-school-chains-developing-countries">Unchained Melody: The Role of Private School Chains in Developing Countries</a> <br />
<br /></div>
<div>
<a href="https://www.cgdev.org/blog/low-cost-private-schools-what-have-we-learned-five-years-dfid-rigorous-review">Low-Cost Private Schools: What Have We Learned in the Five Years Since the DFID Rigorous Review</a><br />
<br /></div>
<div>
<a href="https://www.cgdev.org/blog/does-education-need-qaly-and-lays-it">Does Education Need a QALY and Is LAYS It?</a></div>
<div>
<br />
November</div>
<div>
<br /></div>
<div>
<a href="https://www.cgdev.org/blog/unnatural-selection-under-performance-elite-public-secondary-schools">Unnatural Selection: The (Under) Performance of Elite Public Secondary Schools</a> </div>
<div>
<br />
December</div>
<div>
<br /></div>
<div>
<a href="https://www.cgdev.org/blog/pisa-2018-few-reactions-new-global-education-rankings">PISA 2018: A Few Reactions to the New Global Education Rankings</a> </div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com9tag:blogger.com,1999:blog-4867230308159901547.post-4069984228394270922019-07-11T21:18:00.000+01:002019-07-11T21:17:59.934+01:00Rwanda Could Thrive Faster by Putting (Even) More Women in Charge<div dir="ltr" style="text-align: left;" trbidi="on">
<blockquote class="tr_bq">
Rwanda has the largest share of female parliamentarians of any country in the world. But things in Rwanda seldom sit still, and it is time to start going faster and further. </blockquote>
<blockquote class="tr_bq">
Rwanda’s impressive female representation in national politics is not yet replicated at the local level. Only eight of the thirty district mayors in Rwanda (twenty-seven percent) are women, and just thirteen percent of village leaders are women.</blockquote>
Me in <i><a href="https://www.kigalian.com/viewpoint/2019/07/rwanda-quotas-women-politics/">The Kigalian</a></i></div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com2tag:blogger.com,1999:blog-4867230308159901547.post-71149722466039033662019-05-21T10:45:00.001+01:002019-06-10T20:51:48.117+01:00Cassette tapes "cause" conflict<div dir="ltr" style="text-align: left;" trbidi="on">
Nathan Nunn and Nancy Qian published a very worrying paper in 2014 showing that US food aid causes conflict in recipient countries. Their research design used total US wheat production as a source of quasi-experimental variation in the amount of food aid countries received, to show causality rather than just correlation.<br /><br />A new paper by Paul Christian and Christopher Barrett apparently debunks the study, showing that the "causal" correlation is spurious. Replace "US wheat production" with "US tape cassette sales" and you can almost exactly replicate the results. <br /><br />Which reminds me of an <a href="http://freakonomics.com/2011/07/15/for-economic-growth-does-penis-size-matter-more-than-political-system/">earlier paper</a> showing that "average male organ length" is a strong predictor of GDP growth. We only have about 200 countries, which is not a lot of observations to power a robust statistical analysis, so you should take most cross-country empirical analyses with a pinch of salt. These "male organ" and "cassette sales" papers are helpfully colourful reminders.<br /><br /> HT: <a href="https://www.econthatmatters.com/2019/05/the-economics-of-violence-conflict-and-crime-in-developing-countries-middev-2019-recap/">Jeffrey Bloem</a></div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-69754186273569620552019-05-07T11:00:00.000+01:002019-05-07T11:00:01.519+01:00The Global Education Architecture is Broken<div dir="ltr" style="text-align: left;" trbidi="on">
So says Nicholas Burnett in a (sadly, gated) <a href="https://www.sciencedirect.com/science/article/pii/S0738059319303633">essay for the International Journal of Educational Development</a>. <br /><br />Nicholas has some authority on the topic, as chair of the Board of UNESCO’s International Institute for Educational Planning, and previously Director of the UNESCO Education for All Global Monitoring Report. <br /><br />In his opening paragraph he writes;<br /><blockquote class="tr_bq">
"The international architecture for education is failing the world. There is little leadership; global priorities are obscure; the major debates are increasingly irrelevant and divorced from reality on the ground; the number of children out-of-school has stagnated for a decade; little progress has been made in tackling the global learning crisis; knowledge about what works in education is surprisingly limited; global public goods are massively underfunded; huge global financing requirements show little prospect of being met; and the neediest low-income countries, mainly in sub-Saharan Africa, do not receive the external financial and technical support necessary if they are to develop their education systems."</blockquote>
Tell us what you really think Nicholas. <div>
<br /></div>
<div>
He criticises:<div>
<blockquote class="tr_bq">
"the work of the Right to Education Initiative lobby and its recent Abidjan Principles, which would have governments place severe restrictions on private education."</blockquote>
Points to a major problem being the decline of UNESCO:<br /><blockquote class="tr_bq">
"In the past, UNESCO could have been counted on to be the central voice advocating for education, including education as a human right, in international fora. UNESCO has become so weakened, however, by its internal politicization and inadequate budget, that it is no longer the respected international voice on education, rather just one of many rather weak voices. These problems preceded the withdrawal first of United States' financial support and, more recently, of US membership but these steps now mean that UNESCO cannot function effectively as it has insufficient resources. UNESCO’s total regular budget for education is now only $51 million per year."</blockquote>
With a few other choice quotes:</div>
<div>
<blockquote>
"It is a real paradox that those working in international education increasingly (and rightly) call for systems-wide approaches but fail to study their own non-functional international architecture system. </blockquote>
<blockquote>
... </blockquote>
<blockquote>
It is astonishing both how little we know about what works in education and how poorly we disseminate what we do know. </blockquote>
<blockquote>
... </blockquote>
<blockquote>
If the situation is bad regarding generating knowledge, it is even worse regarding promoting innovation in education. </blockquote>
<blockquote>
... </blockquote>
<blockquote>
There is thus no systematic regular review of how the international architecture is performing."</blockquote>
Well written and well worth reading in full.<br />
<div>
<section id="sec0035" style="background-color: #181a1b; box-sizing: border-box; color: #d7d5cf; font-family: NexusSerif, Georgia, "Times New Roman", Times, STIXGeneral, "Cambria Math", "Lucida Sans Unicode", "Microsoft Sans Serif", "Segoe UI Symbol", "Arial Unicode MS", serif; font-size: 18px; margin: 0px 0px 8px; padding: 0px;"></section></div>
</div>
</div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-79321825944106253052019-04-17T08:56:00.000+01:002019-04-17T08:56:07.744+01:00Coaching is better than training, but there is still a questionmark on scalability<div dir="ltr" style="text-align: left;" trbidi="on">
<blockquote class="tr_bq">
"So should governments switch to frequent coaching sessions? Possibly, but the next step should first be to try this type of intervention at scale. </blockquote>
<blockquote class="tr_bq">
Finding three highly skilled coaches is one thing, but you might need hundreds or thousands of them if you were to run a similar programme across an entire country. </blockquote>
<blockquote class="tr_bq">
One potential route to scale is through new uses of technology. A study in <a href="https://protect-eu.mimecast.com/s/gTflCy81ViNvooEIMDwDv?domain=sciencedirect.com">Brazil</a> found positive impacts of a virtual-coaching programme run via Skype, for example. </blockquote>
<blockquote class="tr_bq">
But perhaps the most straightforward type of technology to go for is scripts, which this paper suggests have positive effects on learning both when presented through centralised training and intensive coaching."</blockquote>
<div>
<a href="https://www.tes.com/news/improve-teaching-quality-training-cpd">My latest at TES.com</a></div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com1tag:blogger.com,1999:blog-4867230308159901547.post-76876874744627625112019-03-12T21:21:00.000+00:002019-03-12T21:21:00.055+00:00"Maybe one of the most cost-effective interventions ever studied"<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both;">
In this month's <a href="https://www.tes.com/news/parent-teacher-meetings-really-do-boost-attainment-study-finds">TES column</a> (I'm calling it a column, it sounds better than a blog), I call parent-teacher meetings in Bangladesh "maybe one of the most cost-effective interventions ever studied". Here's the maths behind that claim. </div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
First, the intervention found 0.377 standard deviation effect on Grade 5 scores and 0.141 standard deviation (not statistically significant) effect on Grade 3 scores. If we take the average of those, that is 0.259. That's equivalent to around 1.7 extra years of school (based on Evans & Yuan's estimate that 1 standard deviation ~ 6.5 years of school).</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
The cost was $3 per student over the two years. The author Asad Islam does the conversion using only the 0.377 effect size for Grade 5, writing "Thus, the cost per average 0.1 SD increase in test scores per student is $0.66 or $1.58 for the full program over
2 years."</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
J-Pal put together a list of the cost-effectiveness of different interventions on their website, now gone, but replicated by Romero, Sandholtz, & Sandefur in the Liberia Partnerships Schools paper (copied below). Islam's $1.58 per 0.1 SD increase is equivalent to 6.3 standard deviations per $100. If we use the more conservative estimate of 0.259 SD (averaging across Grade 5 and Grade 3 results) that still works out at 4.3 SD per $100 spent. That lower estimate still puts this intervention at third place in the ranking, so there you go: "maybe one of the most cost-effective interventions ever studied".</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
<a href="https://2.bp.blogspot.com/-2ITlCAtIu2E/XIgeMDN8tDI/AAAAAAABOiU/MWg-MJU5KP0317L0ENRV9ucpagYp_9ahACLcBGAs/s1600/Cost%2Bbenefit.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="750" data-original-width="794" height="377" src="https://2.bp.blogspot.com/-2ITlCAtIu2E/XIgeMDN8tDI/AAAAAAABOiU/MWg-MJU5KP0317L0ENRV9ucpagYp_9ahACLcBGAs/s400/Cost%2Bbenefit.PNG" width="400" /></a></div>
<br /></div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-53314853271900707552019-03-11T11:53:00.001+00:002019-03-11T11:53:32.038+00:00The Latest Economics Research on Global Education<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="-en-clipboard: true;">
Last week I was at the Society for Research on Educational Effectiveness (SREE) conference. <a href="https://twitter.com/alexeble">Alex Eble</a> made a big and apparently successful push to increase representation by researchers focused on developing countries. In time-honoured Dave Evans style, here's my one-sentence roundup of 22 idiosyncratically selected studies presented at the conference. You can see the <a href="https://www.sree.org/conferences/2019s/program/">full programme here</a>. </div>
<div style="-en-clipboard: true;">
<b><br /></b></div>
<div style="-en-clipboard: true;">
<b>---</b></div>
<div style="-en-clipboard: true;">
<b><br /></b></div>
<div style="-en-clipboard: true;">
<b>Public-private partnerships</b></div>
<div>
<br /></div>
<div>
A subsidy for private schools in Haiti lead to higher enrolment (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2860_intro.pdf">Adelman, Holland, and Heidelk</a>) #Haiti</div>
<div>
<br /></div>
<div>
Chile has a universal school voucher and a higher voucher targeted at low-income students. The universal voucher is better for aggregate efficiency but worse for equity (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2860_intro.pdf">Sanchez</a>) #Chile #StructuralModel
</div>
<div>
<br /></div>
<div>
Giving out vouchers to attend 5 years of low-cost private primary school in Delhi led to worse Hindi scores and no change in English or Maths (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2860_intro.pdf">Crawfurd, Patel, and Sandefur</a>) #India</div>
<div>
<br /></div>
<div>
Contracting out management of public schools to NGOs in Liberia led to a 60% increase in learning (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2860_intro.pdf">Romero, Sandholtz, and Sandefur</a>) #Liberia</div>
<div>
<br /></div>
<div>
<b>School management</b></div>
<div>
<br /></div>
<div>
A mobile-phone based support programme for school councils in Pakistan led to no improvement for students (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2888_0.pdf">Asim</a>) #Pakistan #Diff-in-Diff
</div>
<div>
<br /></div>
<div>
A major school inspection reform in Madhya Pradesh led to no improvement in schools (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2815_0.pdf">Muralidharan and Singh</a>) #India</div>
<div>
<br /></div>
<div>
Independent monitoring of teachers led to better student performance (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2819_0.pdf">Kim, Yang, Inayat</a>) #Pakistan #Diff-in-Diff
</div>
<div>
<br /></div>
<div>
<b>Mindfulness</b></div>
<div>
<br /></div>
<div>
Mindfulness interventions reduced sadness and aggression of children in Niger (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2901_intro.pdf">Kim, Brown, De Oca, Annan, Aber</a>), improved concentration and prosocial behaviour in Sierra Leone (Brown, Kim, Annan, Aber), and increased prosocial behaviour amongst Syrian refugees (Keim and Kim) #Niger #SierraLeone #Syria
</div>
<div>
<br /></div>
<div>
<b>Information for parents</b></div>
<div>
<br /></div>
<div>
Giving parents information about their child’s performance led to some temporary improvements (<a href="https://felipe-barrera-osorio.gse.harvard.edu/files/gse-felipe-barrera-osorio/files/effects_timing_and_heterogeneity_fbarrera.pdf">Barrera-Osorio, Gonzalez, Lagos, Deming</a>) #Colombia
</div>
<div>
<br /></div>
<div>
<b>Incentives for teachers </b></div>
<div>
<b><br /></b></div>
<div>
The theoretically optimal “Pay for Percentile” incentive scheme works to increase effort, which is complementary to inputs (<a href="http://ftp.iza.org/dp11516.pdf">Gilligan, Karachiwalla, Kasirye, Lucas, Neal</a>) #Uganda
</div>
<div>
<br /></div>
<div>
BUT A simpler “threshold” incentive scheme can be as effective as the theoretically optimal “Pay for Percentile” (at least in the short-run) (<a href="http://barrett.dyson.cornell.edu/NEUDC/paper_613.pdf">Mbiti, Romero, Schipper</a>) #Tanzania </div>
<div>
<br /></div>
<div>
<b>Methodology</b></div>
<div>
<b><br /></b></div>
<div>
Studies commissioned by the developer of an intervention find effect sizes 80% larger than studies commissioned independently (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2715_0.pdf">Wolf, Morrison, Slavin, Risman</a>) #USA #MetaAnalysis #EvaluatorIndependence<br />
</div>
<div>
<br /></div>
<div>
Tests designed specifically for evaluations produce effect sizes 63% larger than generic tests (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2848_0.pdf">Pellegrini, Inns, Lake, Slavin</a>) #USA #MetaAnalysis #TestDesign
</div>
<div>
<br /></div>
<div>
External validity bias (non-random selection of schools into trials) is twice as big as internal validity bias (from using observational not experimental methods) (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2941_0.pdf">White, Hansen, Lycurgus, Rowan</a>) #USA #ExternalValidity
</div>
<div>
<br /></div>
<div>
<b>Technology</b></div>
<div>
<b><br /></b></div>
<div>
The One Laptop Per Child programme in Peru had zero effect on learning (Cristia, Ibarrarán, Cueto, Santiago and Severín) #Peru</div>
<div>
<br /></div>
<div>
In addition, providing internet had no effect on student learning (Malamud, Cueto, Cristia, Beuermann) #Peru</div>
<div>
<br /></div>
<div>
<b>Peer effects</b></div>
<div>
<br /></div>
<div>
Being the weakest student in a better (selective) school can be worse than being the strongest student in a worse school (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2727_0.pdf">Fabregas</a>) #RDD #Mexico
</div>
<div>
<br /></div>
<div>
<b>Finance</b></div>
<div>
<br /></div>
<div>
Temporary subsidies can have permanent effects on enrolment (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2838_0.pdf">Nakajima</a>) #Indonesia #Diff-in-Diff
</div>
<div>
<br /></div>
<div>
Merit-based scholarships have bigger effects than need-based scholarships (<a href="https://www.sree.org/conferences/2019s/program/download/abstract/2718_0.pdf">Barrera-Osorio, de Barros, Filmer</a>) #Cambodia</div>
<div>
<br /></div>
<div>
<b>Heat</b></div>
<div>
<b><br /></b></div>
<!--?xml version="1.0" encoding="UTF-8"?-->
<br />
<div>
Each 1 degree Fahrenheit of school year temperature reduces learning by 1 percent. Air conditioning entirely offsets this. (<a href="https://scholar.harvard.edu/files/joshuagoodman/files/w24639.pdf">Goodman, Hurwitz, Park, Smith</a>) #FE #USA </div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-27151429239386221792019-02-18T17:51:00.002+00:002019-02-18T17:51:36.722+00:00Is testing good for education?<div dir="ltr" style="text-align: left;" trbidi="on">
<blockquote class="tr_bq">
<i>This post was first published on the Centre for Education Economics website.</i> </blockquote>
I <a href="http://www.cfee.org.uk/node/314">blogged recently</a> about a new <a href="https://riseprogramme.org/sites/www.riseprogramme.org/files/publications/RISE_WP-025_Woessman_1.pdf">RISE working paper</a> by Annika Bergbauer, Eric Hanushek, and Ludger Woessmann, which finds that:<br /><blockquote class="tr_bq">
“standardized external comparisons, both school-based and student-based, is associated with improvements in student achievement.”</blockquote>
William Smith <a href="https://twitter.com/william_c_smith/status/1085896108843388929">pointed me</a> to his <a href="https://gemreportunesco.wordpress.com/2018/10/24/more-testing/">rebuttal blog</a> written with Manos Antoninis, which argues that there are “multiple weaknesses in their analysis that undermine their conclusions”.<br /><br />This blog is my attempt to make sense of the disagreement. The main issue appears to me to be a misunderstanding by Antoninis & Smith (“AS” from here on) of the mechanism proposed by Bergbauer, Hanushek, and Woessmann (BHW). AS presume that the main mechanism through which testing is hypothesised to improve outcomes is through school choice (allowing parents to shift their students to schools with better test scores) or through punitive government accountability for teachers and schools. But BHW make it clear that their main focus is on the principal-agent relationship between parents as the principal and both students and teachers as their agents. Parents can’t observe the effort made by students and teachers, but standardized testing can provide them with a proxy indicator for effort. This should induce greater effort from both students and teachers. This proposed mechanism has nothing to do with school choice or accountability from government.<br /><br />First AS argue that<br /><blockquote class="tr_bq">
“Our review of the evidence found that evaluative policies promoting school choice exacerbated disparities by further advantaging more privileged children (pp. 49-52).”</blockquote>
This review of the evidence in pp 49-52 of the UNESCO <a href="https://unesdoc.unesco.org/ark:/48223/pf0000259338">Global Monitoring Report</a> focuses on policies designed to promote school choice. But that is not at all the focus of the BHW analysis, which is on policies that allow for the comparison of schools and students with the purpose of incentivising greater effort. School choice doesn’t need to have anything to do with it. As BHW write:<br /><blockquote class="tr_bq">
“That is the focus of this paper: By creating outcome information, student assessments provide a mechanism for developing better incentives to elicit increased effort by teachers and students, thereby ultimately raising student achievement levels to better approximate the desires of the parents”</blockquote>
Second, AS argue that<br /><blockquote class="tr_bq">
“punitive systems had unclear achievement effects but troublesome negative consequences, including removing low-performing students from the testing pool and explicit cheating (pp. 52-56).”</blockquote>
As mentioned above, the proposed mechanism in BHW does not at all require a punitive system. BHW write<br /><blockquote class="tr_bq">
“accountability systems that use standardized tests to compare outcomes across schools and students produce greater student outcomes. These systems tend [my emphasis] to have consequential implications and produce higher student achievement than those that simply report the results of standardized tests.”</blockquote>
Having said that, there are some flaws in the literature review cited by AS. This section first cites studies on four individual countries (US, Brazil, Chile, South Korea), without noting that there are significantly positive results from two of them. One of the two papers they cite on Brazil (<a href="https://unesdoc.unesco.org/ark:/48223/pf0000259531">IDados 2017</a>) concludes that there was “a large, continuous improvement in all those years in both absolute and relative terms when compared to other municipalities in the Northeastern region and in Brazil as a whole ” and “it is very likely that [the reform] is at least partially responsible for the changes.” On Chile, a paper not cited as it was published in 2017 just after the review was completed (<a href="https://www.nber.org/papers/w23550">Murnane et al</a>) found that “On average, student test scores increased markedly and income-based gaps in those scores declined by one-third in the five years after the passage of [the reform]”.<br /><br />Next the review cites two papers (<a href="https://www.researchgate.net/profile/Pilnam_Yi/publication/290457410_Do_school_accountability_and_autonomy_affect_pisa_achievement_Evidence_from_South_Korea/links/57026b4308ae47378f664eb7.pdf">Yi 2015</a>; <a href="https://files.eric.ed.gov/fulltext/EJ1084111.pdf">Gándara and Randall (2015)</a> that present correlational analysis with no attempt to address any potential bias from omitted variables or reverse causality. The latter study is based on a small sub-sample of the fuller data used by BHW.<br /><br />Next AS take issue with the way that BHW construct their 4 categories of test usage. For ease of reference I first reproduce below the 4 categories, along with the wording of the questions that go into constructing each category.<br /><br />---<br /><blockquote class="tr_bq">
1. Standardized External Comparison<ul style="text-align: left;">
<li>“In your school, are assessments of 15-year old students used to compare the school to district or national performance?” (PISA)</li>
</ul>
<ul style="text-align: left;">
<li>existence of national/central examinations at the lower secondary level (OECD, EAG)</li>
</ul>
<ul style="text-align: left;">
<li>National exams (primary) (Euryadice (EACEA))</li>
</ul>
<ul style="text-align: left;">
<li>Central exit exams end secondary (Leschnig, Schwerdt, and Zigova (2017))</li>
</ul>
<br />2. Standardized Monitoring<ul style="text-align: left;">
<li>“Generally, in your school, how often are 15- year-old students assessed using standardized tests?” (PISA)</li>
</ul>
<ul style="text-align: left;">
<li>“During the last year, have [tests or assessments of student achievement] been used to monitor the practice of teachers at your school?” (PISA)</li>
</ul>
<ul style="text-align: left;">
<li>“In your school, are achievement data … tracked over time by an administrative authority[?]”</li>
</ul>
<br />3. Internal testing<ul style="text-align: left;">
<li> whether assessments are used “to inform parents about their child’s progress.”</li>
</ul>
<ul style="text-align: left;">
<li>use of assessments “to monitor the school’s progress from year to year.”</li>
</ul>
<ul style="text-align: left;">
<li>“achievement data are posted publicly (e.g. in the media).” (vaguely phrased and is likely to be understood by school principals to include such practices as posting the school mean of the grade point average of a graduating cohort, derived from teacher-defined grades rather than any standardized test, at the school’s blackboard.)</li>
</ul>
<br />4. Internal teaching monitoring<ul style="text-align: left;">
<li>whether assessments are used “to make judgements about teachers’ effectiveness.”</li>
</ul>
<ul style="text-align: left;">
<li>practice of teachers is monitored through “principal or senior staff observations of lessons.”</li>
</ul>
<ul style="text-align: left;">
<li>“observation of classes by inspectors or other persons external to the school” are used to monitor the practice of teachers.</li>
</ul>
</blockquote>
<div>
---</div>
<div>
<br /></div>
First, AS argue that question 3c should really fall under category 1. The effect of this question on outcomes is primarily statistically insignificant, though for Maths and Science the direction of the coefficients in the interacted model are the same as the other variables in category 1 (positive in the base model, with a negative coefficient on the interaction with initial score). Would adding this one variable to the 4 variables already in the category make the results statistically insignificant overall? I think probably not, but can’t say for sure without looking at the raw data.<br /><br />Second, AS claim that question 4a should really fall under category 1 or 2. This claim seems debateable. The theoretical mechanism that BHW put forward is that providing credible information to parents induces greater effort from teachers. This use of testing is clearly internal to the school, and could clearly mean internal school assessments rather than necessarily standardized assessments that allow for external comparison with teachers at other schools.<br /><br />Third, AS criticise the inclusion of high stakes student assessments as indicators, as by placing the stakes on students and not schools they do not relate to accountability from government. But this is not what BHW claimed was driving the effect.<br /><br />Fourth, AS suggest the use standardized testing in grade 15 may be effectively “teaching to the test”. This seems odd to me - they clearly aren’t literally teaching to the test because it is a different test. BHW are looking at the effect of introducing high-stakes national standardized testing on student results in a totally separate, low-stakes sample-based test (PISA). AS then don’t really address the argument that “teaching to the test” can also be a positive thing if the test is well-designed and includes a good sample of the things that students are expected to have learnt.<br /><br />Finally, AS focus only on those results that are statistically significant in the baseline model (estimating the average effect across all countries). However they miss a really important conclusion from the paper which is about heterogeneity. The effects of testing are largest for the weakest performing systems. This is clear in Figure 3.<div>
<br /><br /><img height="233" src="https://www.cfee.org.uk/sites/default/files/styles/large/public/Figure%203.png?itok=2nD8EV0x" width="400" /><br /><br /></div>
<div>
Looking at the interacted model (Table A5), both of the other 2 questions in category 2 (2b and 2c) are statistically significant.<br /><br />To sum up, there are weaknesses in the interpretation by AS of BHW which undermine their criticism. BHW focus on the role that testing can play in increasing the effort of students and teachers, with or without government accountability systems. In addition, the review of government accountability systems presented in the UNESCO Global Monitoring Report also has weaknesses, and presents an unduly negative picture. My prior remains that standardized testing plays a positive role, particularly in weak systems.<br /><br /><i>Thanks to Gabriel Heller-Sahlgren, William Smith, and Manos Antoninis for comments on a draft of this post. This acknowledgement clearly does not imply that Smith and Antoninis agree with this post - they don’t!</i><br /><div class="region region-content">
<div about="/node/327" class="node node-blog" id="node-327" typeof="sioc:Post sioct:BlogPost">
<div class="content">
<div class="field field-name-field-blog-category field-type-taxonomy-term-reference field-label-above" style="background: url("https://www.cfee.org.uk/sites/all/themes/tophit/images/tag.png") left top no-repeat scroll rgb(28, 40, 50); clear: both; color: #cdcac4; font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin: 15px 0px; overflow: hidden; padding: 0px 0px 0px 35px;">
</div>
</div>
</div>
</div>
</div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-86180116007837094332019-02-06T14:52:00.000+00:002019-02-06T14:52:58.686+00:00CfEE Blogging: Giving students information on future wages improves school outcomes<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="tr_bq" dir="ltr">
As of this January and following last year's <i>Annual</i> Research Digest from the Centre for Education Economics, I'll be co-editing the <i>Month</i><i>ly</i> Digest, along with Gabriel Heller-Sahlgren.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
This is basically an excuse and commitment device to get me actually blogging again on at least a monthly basis. Each issue will include commentary on new papers, plus a selection of abstracts from recent publications (lightly edited for jargon). </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<a href="http://www.cfee.org.uk/digest">Here's the link</a>.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
My first comment is on a new paper by <span class="content" style="box-sizing: border-box; margin: 0px; padding: 0px;"><a href="https://www.sciencedirect.com/science/article/pii/S0304387818304565"><span class="text given-name" style="box-sizing: border-box; margin: 0px; padding: 0px;">Ciro </span><span class="text surname" style="box-sizing: border-box; margin: 0px; padding: 0px;">Avitabile and </span></a></span><span class="content" style="box-sizing: border-box; margin: 0px; padding: 0px;"><span class="text given-name" style="box-sizing: border-box; margin: 0px; padding: 0px;"><a href="https://www.sciencedirect.com/science/article/pii/S0304387818304565">Rafael de</a></span><span class="text surname" style="box-sizing: border-box; margin: 0px; padding: 0px;"><a href="https://www.sciencedirect.com/science/article/pii/S0304387818304565"> Hoyos</a>: </span></span></div>
<blockquote>
Did you know what career you wanted to do when you were in secondary school? I didn’t. Most pupils make critically important choices that will affect their lives throughout their educational career, often on the basis of poor information about what those choices will mean for their future. In most countries, there is little transparency on the costs and benefits of pursuing education and information on the various career paths available. </blockquote>
<blockquote>
In this paper, Ciro Avitabile and Rafael de Hoyos study whether or not providing pupils with better information about the earnings returns to education and the options available lead to greater effort and learning. Several studies have previously shown that providing information about the wage gains from schooling leads pupils to stay in school a bit longer, and affects their educational choices, but there is limited evidence that such information can affect learning per se, at least in a slightly longer-term perspective.</blockquote>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-33650119885034646312019-01-16T08:00:00.002+00:002019-01-16T08:00:29.737+00:00PubhD Kigali<div dir="ltr" style="text-align: left;" trbidi="on">
For any readers in (or visiting) Kigali (presumably a niche audience), I've started a monthly research talk event, using the PubhD format that is going in around 20 European cities now.<br />
<br />
3 speakers get 10 minutes each to present their research, followed by Q&A. It's a great way to learn a bit about some random subjects you might not have considered much before, for the speakers to practice their extended elevator pitch, and a pretty low-effort way of organising some kind of regular academic vaguely seminar-like discussion for me.<br />
<br />
The next one is this Thursday at 7.30pm, <a href="https://www.facebook.com/events/345479669566645/">see here</a> for more details, and get in touch if you'd like to speak sometime.</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-7594323286771819842019-01-16T07:56:00.000+00:002019-01-16T07:56:13.229+00:00Does temporary migration from rich to poor countries cause commitment to development?<div dir="ltr" style="text-align: left;" trbidi="on">
Nevermind that none of the journals I've sent it to so far are interested, my new working paper got picked up by <i><a href="https://marginalrevolution.com/marginalrevolution/2019/01/temporary-migration-rich-poor-countries-cause-commitment-development.html">Marginal Revolution</a></i> the king of economics blogs, which is probably way better anyway right?<br />
<blockquote class="tr_bq">
Public support in rich countries for global development is critical for sustaining effective government and individual action. But the causes of public support are not well understood. Temporary migration to developing countries might play a role in generating individual commitment to development, but finding exogenous variation in travel with which to identify causal effects is rare. In this paper I address this question using a natural experiment – the assignment of Mormon missionaries to two year missions in different world regions – and test whether the attitudes and activities of returned missionaries differ. Data comes from a unique survey gathered on Facebook. Missionaries assigned to treat regions (Africa, Asia, Latin America) are balanced with those assigned to the control region (Europe) on high school test scores and prior language and travel experience. Those assigned to the treatment region report greater interest in global development and poverty, but no difference in support for government aid or higher immigration, and no difference in personal international donations, volunteering, or other involvement.</blockquote>
Here's the <a href="https://osf.io/preprints/socarxiv/3hwga/">link to the paper</a> and the <a href="https://twitter.com/leecrawfurd/status/1082928454557986816">twitter discussion</a>. </div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-13401356329221582302019-01-15T15:32:00.000+00:002019-01-15T15:32:41.303+00:00Testing, testing: the 123's of testing<div dir="ltr" style="text-align: left;" trbidi="on">
Here's my summary of the new Annika Bergbauer, Eric Hanushek, and Ludger Woessmann working paper for CFEE.<br />
<blockquote class="tr_bq">
"teachers tend to oppose standardised tests, partly because they perceive them to narrow the curriculum and crowd out wider learning. However, it is intuitive that the effects of testing could vary dramatically by context. Indeed, the impact may very well follow a so-called “Laffer curve”. At low levels of testing, an increase may lead to better performance as it provides relevant information and incentives to actors in the education system. Yet if there are already high levels of testing, further increases may very well decrease performance, due to stress, for example, or the effects of an overly-narrowed curriculum. If so, we should expect the impact of testing to follow an inverted U-curve – or at the very least display diminishing returns. Furthermore, the impact of tests is also likely to depend on exactly how they are used in the education system. </blockquote>
<blockquote class="tr_bq">
This paper provides perhaps the first systematic evidence on these issues"</blockquote>
<div>
Read the rest <a href="http://www.cfee.org.uk/node/314">here</a>.</div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-37504727015675275372018-09-27T10:30:00.002+01:002018-09-27T10:30:51.500+01:00The best thing about cash benchmarking is it highlights just how small most aid is<div dir="ltr" style="text-align: left;" trbidi="on">
The best take on the <a href="http://blogs.worldbank.org/impactevaluations/lessons-cash-benchmarking-evaluation-authors-version?cid=SHR_BlogSiteShare_XX_EXT">new cash benchmarking study from Rwanda</a> was this one by <a href="https://twitter.com/mkevane/status/1040695361206018048">Michael Kevane</a>:<br />
<blockquote class="tr_bq">
"the main takeaway is that neither intervention (when evaluated at the low Gikuriro cost of $141 per household) improved child outcomes." Yikes. I guess though if household size = 7 that is only $20 per person.</blockquote>
Should we really be surprised that giving someone $20 doesn't improve any measurable outcomes? Maybe Kevane's maths is wrong and household size is smaller, but $20 is so small we could double it and still not expect to see anything.<br /><br />
That's one big advantage of cold hard individual cash transfers, they make explicit what the actual amount per person is, which is not so obvious when its one big community project costing loadsamoney and affecting loadsapeople. <a href="http://www.rovingbandit.com/2009/05/just-give-them-damn-money.html">John Quiggin</a> made this point years ago.<br />
<br />
Last year, DFID gave £2.6 BILLION pounds to countries in Africa. That's so much money! How are they still so poor when we give them BILLIONS of pounds every year? Well, hold on, there are 1.2 billion PEOPLE in Africa. So that works out at just over £2 each for the whole YEAR. Of course it doesn't go to everyone, let's say the money is perfectly targeted on the poorest 10% of people. So they get £20 each.<br />
<br />
Somehow there is a lot of magical thinking that by pooling money together it somehow automatically has totally outsized impacts. Of course its possible that smart investment in research or better governance can have truly outsized impact if it can nudge a country toward a slightly higher growth rate, but that isn't what most aid is even trying to do, and even when it is they stuff is wicked hard and we can expect most attempts to fail.<br />
<br />
Aid is great but less hubris please. And less ridiculous implicit expectations from what aid could plausibly achieve from the sceptics too.</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com3tag:blogger.com,1999:blog-4867230308159901547.post-45789532830914548362018-09-27T10:14:00.000+01:002018-09-27T10:14:43.500+01:00Don't Buy Local<div dir="ltr" style="text-align: left;" trbidi="on">
This summer I finally got around to doing my first Park Run, joining the now millions of people around the world who turn up on Saturday mornings for a free 5km timed run around their local park (I managed a not too shabby 27 minute time, roughly average for my age). <div>
<br /></div>
<div>
I also work on global development, so was pretty disappointed to receive an email announcing a new clothing line from the founder of Park Run that will be manufactured exclusively in Europe. Paul Sinton-Hewitt CBE was concerned with the “horrendously exploitative … factories in the Far East employing questionable practices, paying the lowest wages and exposing their workers to dangerous conditions”. </div>
<div>
<br /></div>
<div>
Paul is right to be concerned about the wellbeing of East Asian factory workers, but moving those jobs to Europe is not the solution. Moving manufacturing jobs from places where jobs are scarce, to Europe where jobs are not scarce, is not a good thing. </div>
<div>
<br /></div>
<div>
My point is not a new one; Paul Krugman, the nobel-prize winning left-wing economist wrote more than ten years ago <a href="http://www.slate.com/articles/business/the_dismal_science/1997/03/in_praise_of_cheap_labor.html">in praise of sweatshops</a>. But the point still stands, and is still apparently missed. The factories in which most of our sports gear are made have poor working conditions compared with jobs in rich countries, but they are usually preferable to the actual alternatives facing people unfortunate enough to be stuck in poor countries.</div>
<div>
<br /></div>
<div>
In 1980, before it became the workshop of the world, extreme poverty in China was close to 90 per cent. Today it is less than 1 per cent. That change would not have been possible without the manufacturing industry. It’s hard to think of anything else that has had a bigger positive impact on human wellbeing that the transformation of China’s economy. </div>
<div>
<br /></div>
<div>
And it’s not just China that has benefited. Now that wages are starting to rise in China, manufacturers are moving on to other lower-income countries, such as neighbouring Vietnam and Cambodia, but also further afield, to Ethiopia and Nigeria. </div>
<div>
<br /></div>
<div>
Researchers Rachel Heath and Mushfiq Mobarak <a href="http://faculty.som.yale.edu/mushfiqmobarak/papers/garments.pdf">looked at what happened</a> in Bangladesh when garment factories started opening. As factories rewarded basic literacy and numeracy, girls who lived in villages near to new factories chose to stay in school longer. The effect on education was bigger than a government social programme explicitly designed to increase schooling. In Ethiopia, Chris Blattman and Stefan Dercon found that people use factory jobs as a safety net. Pay and conditions may be poor in the new factory jobs, but they are always there, even when other informal means of getting by fall through. </div>
<div>
<br /></div>
<div>
So what is a concerned Park Runner to do? Ultimately pay and conditions in poor countries will only improve when workers get better outside options. Thanks to the hard work of poor Chinese in the 1980s and 1990s, China’s economy has grown, wages have risen, and Chinese workers today can afford to ask for more and turn down the worst offers. The best way to encourage that trend is to continue to buy things made in poor countries. Other ways to give poor people more options are to just give them cash directly (you can do literally that at givedirectly.org), to go on holiday to poor countries and spend money on services from poor people, or if you’re a citizen of a rich country to encourage your government to make it easier for people from poor countries to come and work in your country. </div>
<div>
<br /></div>
<div>
I have nothing but admiration for Paul Sinton-Hewitt’s founding of Park Run, and of his desire to create an inclusive and ethical line of sportswear. I just hope he reconsiders his decision not to support jobs where they are needed most.</div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-27457272547171961472018-09-20T10:41:00.001+01:002018-09-20T10:41:38.987+01:00Probably the best new research in global education economics<div dir="ltr" style="text-align: left;" trbidi="on">
A couple of months ago the Centre for Education Economics asked me to edit their Annual Research Digest - a series of essays by leading thinkers on their favourite research paper from the past year.<br />
<br />
The Digest is out today and I'm really pleased with how its come out, a fantastic set of blogs summarising a fascinating set of papers.<br />
<br />
Here's the summary, and you can download the full report <a href="http://www.cfee.org.uk/ARD2017-18">here</a>.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-bf08fSE-pRY/W6NqJGSo4II/AAAAAAABLKc/fKYp4aTmSfoABct5vztV1z86Y7vTmDjJwCLcBGAs/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="818" data-original-width="720" height="320" src="https://2.bp.blogspot.com/-bf08fSE-pRY/W6NqJGSo4II/AAAAAAABLKc/fKYp4aTmSfoABct5vztV1z86Y7vTmDjJwCLcBGAs/s320/Capture.PNG" width="281" /></a><a href="https://2.bp.blogspot.com/-lRpm43fwnkw/W6NqJnIXZ7I/AAAAAAABLKg/TbWQjXjbV6g6PERt8ryCOCwe0fZqYlUSACLcBGAs/s1600/Capture2.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="754" data-original-width="736" height="320" src="https://2.bp.blogspot.com/-lRpm43fwnkw/W6NqJnIXZ7I/AAAAAAABLKg/TbWQjXjbV6g6PERt8ryCOCwe0fZqYlUSACLcBGAs/s320/Capture2.PNG" width="312" /></a></div>
<br /></div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com1tag:blogger.com,1999:blog-4867230308159901547.post-76749476070737055002018-06-18T15:40:00.000+01:002018-06-18T15:40:54.641+01:00How smart are teachers in developing countries?<div dir="ltr" style="text-align: left;" trbidi="on">
Eric Hanushek, Marc Piopiunik, and Simon Wiederhold published some fascinating analysis in a 2014 NBER working paper (<a href="http://www.nber.org/papers/w20727">link</a>) comparing the literacy and numeracy of teachers to the overall population (with a university degree) in a range of OECD countries. In the figures below, the grey bars show the gap between the 25th and 75th percentile of skill for all university-educated adults in each country, and the red marker in the middle shows the median skill level of teachers in that country. Perhaps unsurprisingly, teachers in Finland are the highest skilled internationally, but also within Finland they are drawn from relatively high in the distribution of adults. This fits with common narratives about teaching being a particularly well-regarded, and selective, profession in Finland.<br />
<br />
<a href="https://1.bp.blogspot.com/-tVMJOrIKp6A/WyevnI1wiyI/AAAAAAABH88/e-B3vTR9ZJsGzLJ27_dvqyuErwA9rRpcACLcBGAs/s1600/Screen%2BShot%2B2018-06-18%2Bat%2B15.10.52.png" imageanchor="1" style="color: #8b5346; font-family: Georgia, Garamond, Palatino, "Times New Roman", Times, serif; font-size: 19px; font-weight: normal; margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="511" data-original-width="578" height="352" src="https://1.bp.blogspot.com/-tVMJOrIKp6A/WyevnI1wiyI/AAAAAAABH88/e-B3vTR9ZJsGzLJ27_dvqyuErwA9rRpcACLcBGAs/s400/Screen%2BShot%2B2018-06-18%2Bat%2B15.10.52.png" width="400" /></a><a href="https://1.bp.blogspot.com/-iH5pPdTsBtc/WyevnKbIYJI/AAAAAAABH84/mgRTB3Xnho8NVkqbuTiDLBgo-WTIf5gwwCLcBGAs/s1600/Screen%2BShot%2B2018-06-18%2Bat%2B15.11.01.png" imageanchor="1" style="color: #8b5346; font-family: Georgia, Garamond, Palatino, "Times New Roman", Times, serif; font-size: 19px; font-weight: normal; margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="498" data-original-width="527" height="377" src="https://1.bp.blogspot.com/-iH5pPdTsBtc/WyevnKbIYJI/AAAAAAABH84/mgRTB3Xnho8NVkqbuTiDLBgo-WTIf5gwwCLcBGAs/s400/Screen%2BShot%2B2018-06-18%2Bat%2B15.11.01.png" width="400" /></a>
<br />
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
As far as I'm aware, no such comparison exists for low- and lower-middle-income countries. Tessa Bold and coauthors present results on the tragically low absolute level of teachers in sub-Saharan Africa (<a href="http://documents.worldbank.org/curated/en/882091485440895147/pdf/WPS7956.pdf">link</a>), and similarly Justin Sandefur presents data comparing the skill level of teachers in sub-Saharan Africa to <i>students</i> in OECD countries (<a href="https://www.cgdev.org/publication/math-scores-fourteen-african-countries">link</a>). But neither compare the level of skill of teachers in Africa to teachers in high-income countries, and to the skill of other adults in Africa.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
So I made one*. The World Bank's STEP Skills surveys use the same literacy assessment as the OECD PIAAC survey that Hanushek et al used, so I replicated and extended their graph, adding on the countries from the STEP data in which it was possible to identify teachers and their literacy level - specifically Vietnam, Colombia, Armenia, Georgia, Ghana, Kenya, and Bolivia. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-KyIV6qgl1ws/WybQPQvpccI/AAAAAAABH7w/XIF3Na-AAnAi__4TS918EycQE9SSuyuSACLcBGAs/s1600/Uni.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="652" data-original-width="897" height="290" src="https://4.bp.blogspot.com/-KyIV6qgl1ws/WybQPQvpccI/AAAAAAABH7w/XIF3Na-AAnAi__4TS918EycQE9SSuyuSACLcBGAs/s400/Uni.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both;">
The first point to note is how low the overall distribution in lower income countries is. The majority of adults (in urban areas) <i>who have graduated from university</i> fall into the Level 2 category, whereas in high income countries most fall into Level 3. The PIAAC guide (copied below) explains what these levels mean: Level 2 tasks “may require low-level inferences” whereas Level 3 tasks require “navigating complex texts”.</div>
<div class="separator" style="clear: both;">
<br /></div>
The second point to note from the figure is the remarkable regularity of average (median) teacher performance in each national distribution. There is some variation, but most teachers in high-income countries are roughly in the middle of the distribution of university educated adults. Teaching in lower-income countries tends to be more selective - the median teacher is much closer to the 75th percentile of adults in Colombia, Ghana, and Kenya.<br />
<br />
These two facts, the low overall level of literacy skill amongst graduates in the lower middle income countries, and the position of teachers within the distribution, imply an upper bound on the ability of a more selective recruitment process to improve the average quality of teachers. If for example Ghana and Kenya managed to increase the selectivity of the teaching profession enough to raise average teacher skills to above the 75th percentile (something no other country has done), this level could still be well below Level 3 on the PIAAC scale.<br />
<br />
Getting better teaching is critical for improving education in developing countries. This data highlights the scale of one aspect of the challenge. Education systems are going to have to figure out how to deliver for children with teachers who may be able to make "low-level inferences" but are unable to "navigate complex texts."<br />
<br />
---<br />
<br />
*Thanks to Laura Moscoviz at the Education Partnerships Group for assistance with the graph!<br />
<br />
---<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-XAqlAy7q58k/Wye6MCmnluI/AAAAAAABH9M/clmPE1IHXpwNJpHqMNWEANg-Q_XVAiU_ACLcBGAs/s1600/Screen%2BShot%2B2018-06-18%2Bat%2B15.56.30.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="655" data-original-width="650" height="640" src="https://4.bp.blogspot.com/-XAqlAy7q58k/Wye6MCmnluI/AAAAAAABH9M/clmPE1IHXpwNJpHqMNWEANg-Q_XVAiU_ACLcBGAs/s640/Screen%2BShot%2B2018-06-18%2Bat%2B15.56.30.png" width="633" /></a></div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-81261218810172881692018-06-08T10:37:00.002+01:002018-06-08T10:38:16.258+01:00Are Rwandans really too scared to answer surveys honestly? (TLDR; Probably no)<div dir="ltr" style="text-align: left;" trbidi="on">
<div>
Hi Roving Bandit blog fans! Long time no write. I've just relocated from London to Kigali, so I'm hoping that being in a more stimulating environment is going to encourage me to start blogging more frequently. Probably the best economics blog in Rwanda!</div>
<div>
<br /></div>
<div>
I'll kick things off by just regurgitating some recent twitter chat. A couple of days ago I posted Gallup survey findings that Rwanda is the 3rd most accepting countries for migrants in the world (pretty cool Rwanda).</div>
<div>
<br /></div>
The brilliant Dina Pomeranz responded "<i>A friend who works at Afrobarometer told me that survey responses from Rwanda are problematic in general, because people tend to answer "it's very good" to everything (out of fear of repercussions)</i>."<br />
<br />
This would indeed be worrying were it the case, but I was sceptical. Afrobarometer doesn't actually operate in Rwanda, but I thought I'd take a look at some responses on some of the surveys that have been done here.<br />
<br />
First, the <a href="https://t.co/8HNaXyyJy3">2014 National Household Survey</a> asks people about satisfaction with some government services.<br />
<br />
- 45% said they were dissatisfied with water services<br />
<div>
- 23% said they were dissatisfied with roads </div>
<div>
<br /></div>
<div>
That seems to be a lot of people who were not afraid to give a negative answer to a survey. </div>
<div>
<br />
The <a href="https://dhsprogram.com/pubs/pdf/FR316/FR316.pdf">2015 DHS</a> asks people about their experience of personal violence, and a large number of people do say they have been affected.<br />
<br />
Finally, a <a href="http://rgb.rw/fileadmin/Citizen_Report_Card-all/CRC_2016_English.pdf">2016 <i>government</i> survey</a> by the Rwanda Governance Board also asked people about their satisfaction with a range of government services. Some were pretty positive, some not so much. There was</div>
<div>
<br /></div>
<div>
<div>
- 31% net dissatisfaction with social protection services </div>
</div>
<div>
- 38% net dissatisfaction with hygiene & sanitation services. </div>
<div>
- 45% net dissatisfaction with infrastructure services </div>
<div>
<br /></div>
<div>
These do not at all look to me like the kind of numbers you'd expect from people so cowed by fear that they are afraid to give negative responses to an anonymous survey. Maybe its time for the team at Afrobarometer to take another look at Rwanda?</div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-85747325085909543572018-02-08T20:42:00.005+00:002018-02-08T20:42:45.354+00:00Ark Blogging: The British government’s new plan to get children learning<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="tr_bq">
Over at the <a href="http://arkonline.org/blog/british-government%E2%80%99s-new-plan-get-children-learning">Ark blog</a>:</div>
<blockquote style="box-sizing: inherit; font-family: "New Century Schoolbook Roman"; font-size: 16px; letter-spacing: 0.16px;">
<span id="docs-internal-guid-7ea10719-70f0-0747-b544-48a8e70374a5" style="box-sizing: inherit;">"The new DFID education policy “</span><a href="https://www.gov.uk/government/publications/dfid-education-policy-2018-get-children-learning" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; box-sizing: inherit; color: #00a2ca; text-decoration-line: none;">Get Children Learning”</a> was published last week. As its name suggests, the policy is all about moving the needle on learning outcomes. It sets out a strategy to tackle the learning crisis in developing countries, which has left 90 percent of primary school leavers in low-income countries without basic literacy or numeracy. As a strategy, it’s relevant and ambitious, and it’s been widely welcomed by the education sector. </blockquote>
<blockquote style="box-sizing: inherit; font-family: "New Century Schoolbook Roman"; font-size: 16px; letter-spacing: 0.16px;">
<span id="docs-internal-guid-7ea10719-70f0-0747-b544-48a8e70374a5" style="box-sizing: inherit;">Of course, tackling a crisis this deep and complex is easier said than done. So how does DFID plan to do it?</span> </blockquote>
<blockquote style="box-sizing: inherit; font-family: "New Century Schoolbook Roman"; font-size: 16px; letter-spacing: 0.16px;">
<span style="letter-spacing: 0.16px;">Three priorities underpin DFID’s strategy to tackle the learning crisis and form the backbone of their new policy: better teaching, education system reform, and targeted support to the most marginalised kids. And permeating the strategy are three themes – more and better research; more attention paid to the political economy of education reform; and the “Best of British” – how UK expertise can be better leveraged to improving schools in developing countries. These themes are interesting because each represents a fairly fundamental shift in or crystallisation of thinking from DFID, and together they provide some insight into how the strategy will be executed. The “Best of British” theme in particular reflects a new willingness by DFID to think more strategically about how to facilitate cross-system learning.</span></blockquote>
<a href="http://arkonline.org/blog/british-government%E2%80%99s-new-plan-get-children-learning">Read the rest here</a></div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-20812815481747301342018-01-15T10:01:00.003+00:002018-01-15T10:01:35.091+00:00Why Kenya should partner with private schools to achieve Universal Secondary School <div dir="ltr" style="text-align: left;" trbidi="on">
Op-ed for the <a href="http://www.nation.co.ke/oped/opinion/Use-private-schools-to-spur-universal-education/440808-4254584-s36brtz/index.html">Daily Nation</a>.</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-77633073919941889082017-12-21T11:39:00.002+00:002017-12-21T11:42:34.578+00:00The Global Education Reform Movement (GERM) in 2017: A Year in Review<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="box-sizing: inherit;">
<h1 style="box-sizing: inherit; color: #00a2ca; font-size: 32px; font-weight: normal; margin: 15px 0px;">
<img alt="" src="http://arkonline.org/sites/default/files/styles/top_image_adjustable/public/Ark_DotDusk_00432.jpg?itok=WxYj2OZe" style="border: 0px; box-sizing: inherit; font-family: "New Century Schoolbook Italic"; font-size: 14px; height: auto; max-width: 100%;" typeof="foaf:Image" /></h1>
<h1 style="box-sizing: inherit; color: #00a2ca; font-size: 32px; font-weight: normal; margin: 15px 0px;">
<i style="font-family: "New Century Schoolbook Roman"; font-size: 16px; letter-spacing: 0.16px;"><a href="http://arkonline.org/blog/global-education-reform-movement-germ-2017-year-review">First posted at the Ark blog</a></i><span style="font-family: "new century schoolbook roman"; font-size: 16px; letter-spacing: 0.16px;">. </span></h1>
If the Adam Smith Institute can <a href="https://www.adamsmith.org/blog/coming-out-as-neoliberals">reclaim</a> the pejorative label ‘neoliberal’, maybe we should be reclaiming the pejorative ‘global education reform movement (GERM)’? Aside from the unfortunate acronym, it’s actually a pretty good description of the cluster of people and organisations trying to shake things up a bit and do things differently in global education policy.<br /><br />Here’s a round-up of some of my highlights from the GERM in 2017. There has been some solid progress in documenting the scale of the global learning crisis, and steps towards building the kind of global data infrastructure we need to measure the SDG on learning. In 2018 I’d like to see more measurement, and also more experimentation. The conversation in education still needs to shift some more away from “we know how to fix this just give us the cash” to the maybe slightly less compelling but more honest “this is a crisis that needs fixing, we know where we want to get to but we don’t yet know how to get there, so we’re going to throw everything we have at it, and measure the hell out of everything so we can learn and course correct until we get there.”<br /><br />---<br /><br /><b>January</b> - The big global education jamboree kicks off each year with the Education World Forum in London, not to be confused with the World Education Forum, the Global Education and Skills Forum, or the World Global Forum for Education and Children Who Can’t Read Good (I might have made up the last one).<br /><br />George K. Werner, the Minister of Education in Liberia, publishes the first of many <a href="https://qz.com/876708/why-liberia-is-working-with-bridge-international-brac-and-rising-academies-by-education-minister-george-werner/">articles</a>written in 2017 on the controversial Partnership Schools programme. “Less than 60% of school-aged children in Liberia are in school … Those who do attend school may not fare much better: among adult women who reached fifth grade in Liberia, less than 20% can read a single sentence.”<br /><br />In Colombia, 22 of Bogota’s ‘Concession schools’ (PPP schools, like charters in the US or academies in the UK) <a href="http://elcatolicismo.com.co/es/noticias/10464-los-colegios-en-concesion-de-bogota-siguieron-y-la-mayoria-en-manos-de-comunidades-religiosas-.html">re-open</a> for another 10 year contract, after their last contract ended and was renegotiated last year. <br /><br /><b>February</b> - The second, annual meeting of the <a href="http://www.globalschoolsforum.org/">Global Schools Forum</a> is launched. The GSF event is a gathering of non-state school operators working in low- and middle-income countries and targeting the bottom half of the income distribution, comes together, networks, and shares ideas. I blogged about my general feelings of inferiority whilst rubbing shoulders with all these inspiring people who have set up their own schools <a href="https://medium.com/leecrawfurd/introducing-the-global-schools-forum-910a06855e3b">here</a>. Watch out for GSF in 2018, as the annual meeting promises to be even bigger and better!<br /><br /><b>March</b> - The next big global education jamboree in the calendar is the Global Education and Skills Forum in Dubai. Daisy Christodoulou <a href="https://thewingtoheaven.wordpress.com/2017/03/25/the-global-education-and-skills-forum-2017/">blogged</a> about how she remarkably managed to win a debate in favour of teaching kids facts in the age of Google. She also reported on a fascinating debate about teaching global versus national values. If you don’t know Daisy, 2007 University Challenge champion, you should really read <a href="https://schoolsweek.co.uk/daisy-christodoulou-director-no-more-marking/">this superb feature</a> profile by Laura McInerney in SchoolsWeek.<br /><br />As we don’t (yet) have very good data on actual learning in many developing countries, a growing movement is spreading the idea started by Pratham in India of recruiting citizen-volunteers to administer simple learning assessments on a mass scale. The “People’s Action for Learning” (PAL) network held its 5th annual meeting in Mexico, welcomed Nepal as the newest member, reaching a total of <a href="http://palnetwork.org/our-growth/">16 countries</a>.<br /><br /><b>April</b> - The Government of the Punjab province in Pakistan contracts out the management around 1,000 failing schools to private operators, bringing the total for the first year of the Public School Support Programme (PSSP) to almost 5,000 schools - that’s about how many schools have been converted to Academies in the UK over 16 years!<br /><br /><b>June</b> - The great and the good of global education research meets in Washington DC for the Research on Improving Systems of Education (RISE) conference (<a href="http://www.riseprogramme.org/sites/www.riseprogramme.org/files/RISE%20Annual%20Conference%20Report%202017.pdf">report here</a>). Ark EPG launches its <a href="http://arkonline.org/ppp-review">Rigorous Review</a> of the evidence on public-private partnerships in education in developing countries (post 2009). The review only found 22 studies that met the quality criteria, but of these most were positive.<br /><br />In the Philippines, the senior high school (Grade 11 and 12) voucher programme enters its second year - over 600,000 students started in Grade 11 through the programme last year. <br /><br /><b>July</b> - The Western Cape government drafted <a href="https://wcedonline.westerncape.gov.za/comms/press/2017/36_4jul.html">legislation</a> for the creation of a new independent School Evaluation Authority. <br /><br />Amitav Virmani, CEO of the Education Alliance, <a href="https://medium.com/AmitavVirmani/making-government-schools-in-india-great-again-7e4287fbbfa1">blogged an update</a> on South Delhi Municipal Government’s Partnership Schools. Starting with one (Ark) school in 2015, the programme is now up to 29 schools, “While the rest of the system lost 12,000 students last year, these schools doubled their enrolment to 1600 children.”<br /><br />Education International and ActionAid publishes a <a href="https://download.ei-ie.org/Docs/WebDepot/LIBERIA18julyv7.pdf">critical research report</a> on Partnership Schools for Liberia based on qualitative research in “up to” 20 schools (how many was it?), just 2 months before the actual results would be published from the large-scale RCT which assessed the actual learning of thousands of children in 185 schools, which everyone agreed was a “really helpful” contribution to the dialogue.<br /><br /><b>August</b> - The 3rd annual “<a href="http://centralsquarefoundation.org/research/researchstate-of-the-nation-rte-section-12-1-c">State of the Nation</a>” report on India’s Right to Education Act (Section 12(1)(c)) comes out, highlighting that 5 years after the policy was confirmed, still only 1 in 3 states in India have accessed any federal funding for the programme. This year’s report dives into some of the details of the private school lotteries in states that are actually implementing the policy at scale, including Gujarat, Karnataka, Madhya Pradesh, Maharashtra, and Rajasthan.</div>
<div style="box-sizing: inherit;">
<br /><b>September</b> - In a major contribution to the all-too-sparse global data on learning, the UNESCO Institute for Statistics released a new global indicator for reading proficiency (based on PIRLS, TIMSS, and SACMEQ) - finding a headline figure of 387 million children worldwide who can’t read, and of these 2 in 3 are actually attending school, they just aren’t learning. <br /><br />Results from the 1st year of <a href="https://www.cgdev.org/publication/can-public-private-partnership-improve-liberias-schools">Partnership Schools for Liberia (PSL) evaluation</a> comes in, with both supporters and critics hailing the results as proving them right. On average PSL schools saw solid improvements in teaching and learning, great for the first year of a pilot, but with some operators spending large amounts of money they raised, and weaknesses in contracting allowing “the largest operator” to move some children and teachers out of their schools.<br /><br />The World Bank publishes its first ever annual World Development Report <a href="http://www.worldbank.org/en/publication/wdr2018">focused entirely on education</a>. The report carried many of the same themes as the RISE research programme - highlighting the global learning crisis and calling for more learning assessment and for systemic reforms to try and address the crisis in quality.<br /><br />UNESCO publishes its flagship Global Education Monitoring Report on the topic of accountability, and easily came away with the prize for <a href="https://twitter.com/leecrawfurd/status/922761622803894273">best illustrated</a> report.<br /><br />As part of the above-mentioned South Delhi Government Partnership Schools programme, <a href="http://arkonline.org/news/ark-supports-two-more-pioneering-new-schools-open-india">2 new Ark-backed schools open</a> this September as part of a plan to grow to 10 schools by 2022.<br /><br /><b>October</b> - The Uganda National Examinations Board (UNEB) <a href="http://www.monitor.co.ug/News/Education/Uneb-introduces-new-system-grade-schools/688336-4137612-smcmp4/index.html">announce</a>s that starting in 2018, a new measure of secondary school effectiveness will be published based on value-added. This is a fairer way to compare schools than just looking at the level of test scores, which are much more heavily influenced by home background. This will make Uganda the first country in sub-Saharan Africa to use value-added to judge school quality.<br /><br /><b>November</b> - The UK Parliament International Development Committee <a href="http://www.parliament.uk/business/committees/committees-a-z/commons-select/international-development-committee/news-parliament-2017/education-report-publication-17-19/">publishes its report</a>on DFID’s work on global education. The report welcomed the results from the PSL study and called for more research into non-state schooling, as well as backing the <a href="http://www.riseprogramme.org/">RISE research programme</a> focused on systems of accountability.<br /><br />In South Africa, an op-ed-war breaks on South Africa’s collaboration school project, with Equal Education calling the pilot <a href="https://www.iol.co.za/capetimes/opinion/caution-over-collaboration-schools-11921676">illegal, undemocratic, and unaccountable</a>, to which Education Minister Debbie Shafer <a href="https://www.iol.co.za/capetimes/opinion/collaboration-schools-within-the-law-and-our-best-option-12000528">replies</a> “We will not be deterred by the likes of Equal Education, who cannot come up with any feasible plan to address the inequalities that still exist as a result of our apartheid legacy,” and David Harrison from the DG Murray Trust <a href="https://dgmt.co.za/the-public-school-partnership-pilot-is-it-worth-it-let-parents-decide/">adds</a> that the programme “gives parents the right to decide whether or not they want to be part of the experiment.”<br /><br />At the World Innovation Summit for Education (WISE) in Doha, more progress was made on the nascent “global education ecosystem” project being launched by the Education Commission, R4D, Teach for All, Asia Society, and Brookings. At the exact same time, Pratham were <a href="http://palnetwork.org/teaching-at-the-right-level-tarl/">putting these ideas into action</a>, sharing their promising “Teaching at the right level” programme with participants from 12 countries, an idea which <a href="https://www.povertyactionlab.org/policy-lessons/education/increasing-test-score-performance">according to JPAL</a> is “the most consistently effective at improving learning” from all of their learning-focused RCTs.<br /><br /><b>December</b> - Just in time for Christmas, the latest international test score results are published - this time in reading from PIRLS. These tests included no low-income countries, and just a handful of lower-middle income countries: Egypt, Georgia, and Morocco. The worst performing country though was upper-middle income South Africa, where 8 out of 10 children can’t read. Nic Spaull provided extensive coverage on very depressing news for South Africa on his <a href="https://nicspaull.com/2017/12/05/the-unfolding-reading-crisis-the-new-pirls-2016-results/">must-read blog</a>.<br /><br />What did I miss?</div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-16581517160108092262017-11-30T17:51:00.001+00:002017-11-30T17:51:38.669+00:00How to achieve public policy reform by surprise and confusion <div dir="ltr" style="text-align: left;" trbidi="on">
This is a great quote from Simeon Djankov, former Finance Minister of Bulgaria (and founder of the World Bank Doing Business indicators) - pulling slightly in the opposite direction of the Tony Blair school of thought on reform (ruthless prioritisation), Djankov instead suggests go off 7 different directions at once in order to surprise and confuse the opposition. <blockquote class="tr_bq">
"Well, one thing that did certainly affect it is the tactics of how to reform, in the sense that, certainly in academia, you are basically told you need to think deeply. Then there are a lot of pressure groups, lobbies, so you need to talk to them. You need to use the media for communicating the benefits of reform, and so on. Some of the reformers, successful reformers that I spoke with, before I joined the Bulgarian government, basically I said, 'You go, and on Day 1, you surprise everybody. So, you go in every direction you can, because they will be confused what's happening and you may actually be successful in some of the reforms. So, this is what I did. I went to Bulgaria in late July 2009; the Eurozone Crisis had already started around us. Greece was just about to collapse a few months later. So, there was kind of a feeling that something is to happen. But, instead of going, 'Let's now do labor reform,' then, 'Let's do business entry reform,' in the government we literally went 6 or 7 different directions hoping that Parliament will be, you know, confused or too happy to be elected--they were just elected. And we actually succeeded in most of these reforms. When I tried to do meaningful, well-explained reforms two years after, they all got bogged down, because lobbying will essentially take over and, 'Not now; let's wait for next year's government,' and so on."</blockquote>
From the always interesting <a href="http://www.econtalk.org/archives/2017/11/simeon_djankov.html">Econ Talk</a>.</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-36569252747767378272017-11-23T17:32:00.001+00:002017-11-23T17:45:44.257+00:00Innovations in Bureaucracy<div dir="ltr" style="text-align: left;" trbidi="on">
<figure class="graf graf--figure graf-after--h3" id="0dd5"><div class="aspectRatioPlaceholder is-locked">
<div class="progressiveMedia js-progressiveMedia graf-image is-imageLoaded is-canvasLoaded" data-height="606" data-image-id="1*uj82pH5mxvJ6OK6_0Nvodw.png" data-scroll="native" data-width="600">
<img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://cdn-images-1.medium.com/max/800/1*uj82pH5mxvJ6OK6_0Nvodw.png" src="https://cdn-images-1.medium.com/max/800/1*uj82pH5mxvJ6OK6_0Nvodw.png" /></div>
</div>
</figure>
<br />
<div class="graf graf--p graf-after--figure" id="bf57">
Last week I was at the “World Innovation Summit for Education” (WISE) in Doha, and I don’t think I heard the word “bureaucrat” once. Clearly the organisers don’t read <a class="markup--anchor markup--p-anchor" data-href="https://chrisblattman.com/2016/03/07/13766/" href="https://chrisblattman.com/2016/03/07/13766/" rel="nofollow noopener" target="_blank">Blattman</a> or they would know that <em class="markup--em markup--p-em">Bureaucracy is so hot right now</em>.</div>
<div class="graf graf--p graf-after--p" id="9cea">
<br />
The World Bank might be a bit more ahead of the curve here, and held a workshop earlier this month on “<a class="markup--anchor markup--p-anchor" data-href="http://www.worldbank.org/en/events/2017/10/16/innovating-bureaucracy#2" href="http://www.worldbank.org/en/events/2017/10/16/innovating-bureaucracy#2" rel="nofollow noopener" target="_blank">Innovating Bureaucracy</a>.” I wasn’t able to attend (ahem, wasn’t invited), and so as the king of <a class="markup--anchor markup--p-anchor" data-href="https://twitter.com/tukopamoja?lang=en" href="https://twitter.com/tukopamoja?lang=en" rel="nofollow noopener" target="_blank">conference write-ups</a>doesn’t seem to have gotten around to it yet, I’ve written up my notes from skimming through the slides (<a class="markup--anchor markup--p-anchor" data-href="http://www.worldbank.org/en/events/2017/10/16/innovating-bureaucracy#2" href="http://www.worldbank.org/en/events/2017/10/16/innovating-bureaucracy#2" rel="nofollow noopener" target="_blank">you can read the full presentations here</a>).</div>
<div class="graf graf--p graf-after--p" id="94a9">
<br />
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/586511510687956381/1-Tim-Besley-Innovating-Bureaucracy.pdf" href="http://pubdocs.worldbank.org/en/586511510687956381/1-Tim-Besley-Innovating-Bureaucracy.pdf" rel="nofollow noopener" target="_blank">Tim Besley</a> — state effectiveness now lies at the heart of the study of development. Incentives, selection, and culture are key, and it is essential to study the 3 together not in isolation.</div>
<div class="graf graf--p graf-after--p" id="4596">
<br />
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/409621510687958996/Individuals-and-Organizations-as-Sources-of-State-E%EF%AC%80ectiveness-and-Consequences-for-Policy-Design-Nov-8-2017-Michael-Best" href="http://pubdocs.worldbank.org/en/409621510687958996/Individuals-and-Organizations-as-Sources-of-State-E%EF%AC%80ectiveness-and-Consequences-for-Policy-Design-Nov-8-2017-Michael-Best" rel="nofollow noopener" target="_blank">Michael Best</a> — looks at efficiency of procurement across 100,000 government agencies (each with decentralised hiring) in Russia. Wide variation in prices paid by different individuals/agencies, with big potential for improvement.<br />
<br />
<img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://cdn-images-1.medium.com/max/800/1*rDxsMcBrROS_fXMB08dmhg.png" height="235" src="https://cdn-images-1.medium.com/max/800/1*rDxsMcBrROS_fXMB08dmhg.png" width="640" /></div>
<br />
<div class="graf graf--p graf-after--figure" id="0a76">
<br />
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/764151510687961360/3-Zahid-WWBI-for-conference-final.pdf" href="http://pubdocs.worldbank.org/en/764151510687961360/3-Zahid-WWBI-for-conference-final.pdf" rel="nofollow noopener" target="_blank">Zahid Hasnain</a> — presents Worldwide Bureaucracy Indicators (WWBI) for 79 countries. Public sector employment is 60% of formal employment in Africa & South Asia, and is usually better paid than private employment.<br />
<br />
<img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://cdn-images-1.medium.com/max/800/1*6n_Q680Gls8DymaCa9mpag.png" height="276" src="https://cdn-images-1.medium.com/max/800/1*6n_Q680Gls8DymaCa9mpag.png" width="640" /></div>
<br />
<div class="graf graf--p graf-after--figure" id="3e34">
<br />
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/372991510688513638/4-Richard-Disney-The-public-sector-labour-market.pdf" href="http://pubdocs.worldbank.org/en/372991510688513638/4-Richard-Disney-The-public-sector-labour-market.pdf" rel="nofollow noopener" target="_blank">Richard Disney</a> — provides a critique of simple public-private pay gap comparisons — need to consider conditions, pensions, and vocation. Lack of well-identified causal studies.<br />
<br />
<img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://cdn-images-1.medium.com/max/800/1*5P1soBUP9QeXwAWddq2GEw.png" height="356" src="https://cdn-images-1.medium.com/max/800/1*5P1soBUP9QeXwAWddq2GEw.png" width="640" /></div>
<br />
<div class="graf graf--p graf-after--figure" id="666d">
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/694531510688516143/5-Jim-Perry-Motivating-Bureaucrats-Evidence-for-Developing-Countries4.pdf" href="http://pubdocs.worldbank.org/en/694531510688516143/5-Jim-Perry-Motivating-Bureaucrats-Evidence-for-Developing-Countries4.pdf" rel="nofollow noopener" target="_blank">James L. Perry</a> — 5 key lessons on motivating bureaucrats in developing countries.</div>
<div class="graf graf--p graf-after--p" id="e61a">
(1) select for ‘vocation’</div>
<div class="graf graf--p graf-after--p" id="2e22">
(2) work on prosocial culture</div>
<div class="graf graf--p graf-after--p" id="2eb4">
(3) leverage employee-service beneficiary ties</div>
<div class="graf graf--p graf-after--p" id="461f">
(4) teacher newcomers public service values</div>
<div class="graf graf--p graf-after--p" id="991d">
(5) develop leaders who model public service values. (<a class="markup--anchor markup--p-anchor" data-href="http://onlinelibrary.wiley.com/doi/10.1111/puar.12796/abstract" href="http://onlinelibrary.wiley.com/doi/10.1111/puar.12796/abstract" rel="nofollow noopener" target="_blank">full paper here</a>)</div>
<div class="graf graf--p graf-after--p" id="991d">
<br /></div>
<div class="graf graf--p graf-after--p" id="0830">
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/241451510688518469/6-new-Erika-Deserranno-WB-Financial-incentives-in-the-public-sector.pdf" href="http://pubdocs.worldbank.org/en/241451510688518469/6-new-Erika-Deserranno-WB-Financial-incentives-in-the-public-sector.pdf" rel="nofollow noopener" target="_blank">Erika Deserranno</a>: Summary of experimental lit on financial & non-financial incentives for workers. Both can work when well designed, or backfire when not. 3 conditions for effective performance-based incentives;</div>
<div class="graf graf--p graf-after--p" id="dbde">
(1) Simple to understand</div>
<div class="graf graf--p graf-after--p" id="8d0c">
(2) Linked to measurable targets</div>
<div class="graf graf--p graf-after--p" id="0879">
(3) Workers can realistically affect targets </div>
<br />
<br />
<br />
<figure class="graf graf--figure graf-after--p" id="0988"><div class="aspectRatioPlaceholder is-locked">
<div class="progressiveMedia js-progressiveMedia graf-image is-canvasLoaded is-imageLoaded" data-action-value="1*WXuGJMbyU3GWaKTC-n_acA.png" data-action="zoom" data-height="849" data-image-id="1*WXuGJMbyU3GWaKTC-n_acA.png" data-scroll="native" data-width="1075">
<canvas class="progressiveMedia-canvas js-progressiveMedia-canvas" height="57" width="75"></canvas><img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://cdn-images-1.medium.com/max/800/1*WXuGJMbyU3GWaKTC-n_acA.png" height="315" src="https://cdn-images-1.medium.com/max/800/1*WXuGJMbyU3GWaKTC-n_acA.png" width="400" /></div>
</div>
</figure>
<br />
<div class="graf graf--p graf-after--figure" id="22ae">
<div style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;">
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/712721510689215694/7-new-Ang-WB-bureaucracy-talk1.pdf" href="http://pubdocs.worldbank.org/en/712721510689215694/7-new-Ang-WB-bureaucracy-talk1.pdf" rel="nofollow noopener" target="_blank">Yuen Yuen Ang</a> — How has China done so well in last 40 years without democratic reform? Through <em class="markup--em markup--p-em">bureaucratic</em> reform which has provided accountability, competition, limits on power. 50 million bureaucrats: 20% managers & 80% frontline workers. Managers have performance contracts focused on outcomes, with published league tables. Frontline workers have large performance-based informal compensation. (bonus podcast edition with <a class="markup--anchor markup--p-anchor" data-href="https://soundcloud.com/user-845572280/how-china-escaped-the-poverty-trap-dr-yuen-yuen" href="https://soundcloud.com/user-845572280/how-china-escaped-the-poverty-trap-dr-yuen-yuen" rel="nofollow noopener" target="_blank">Alice Evans here</a>)<br />
<br />
<img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://cdn-images-1.medium.com/max/800/1*2G_fomMmJTtX94M08BeHQg.png" height="320" src="https://cdn-images-1.medium.com/max/800/1*2G_fomMmJTtX94M08BeHQg.png" width="238" /><a href="https://cdn-images-1.medium.com/max/800/1*OmkSmFMDkdPKX_fq1VNXcA.png" imageanchor="1"><img border="0" class="progressiveMedia-image js-progressiveMedia-image" data-src="https://cdn-images-1.medium.com/max/800/1*OmkSmFMDkdPKX_fq1VNXcA.png" height="230" src="https://cdn-images-1.medium.com/max/800/1*OmkSmFMDkdPKX_fq1VNXcA.png" width="320" /></a></div>
<br /></div>
<div class="graf graf--p graf-after--figure" id="22ae">
<br /></div>
<br />
<div style="margin: 0px;">
<br /></div>
<div style="margin: 0px;">
<br /></div>
<div style="margin: 0px;">
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/847861510689217920/8-Khemani-PoliticsEndogenousCapacity.pdf" href="http://pubdocs.worldbank.org/en/847861510689217920/8-Khemani-PoliticsEndogenousCapacity.pdf" rel="nofollow noopener" target="_blank">Stuti Khemani</a> — research & policy rightly moving from short-route accountability to long-route. Need much more evidence on how public sector workers are selected. One example suggests elected Chairpersons have higher cognitive ability, higher risk aversion, lower integrity.</div>
<div style="margin: 0px;">
<br /></div>
<div class="graf graf--p graf-after--figure" id="c1d7" style="-webkit-text-stroke-width: 0px; color: black; font-family: Times; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">
</div>
<div class="graf graf--p graf-after--figure" id="811c" style="-webkit-text-stroke-width: 0px; color: black; font-family: Times; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="margin-left: 1em; margin-right: 1em;">
<img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://cdn-images-1.medium.com/max/800/1*ahm3pZhx8Z6JccyiLd69ug.png" height="278" src="https://cdn-images-1.medium.com/max/800/1*ahm3pZhx8Z6JccyiLd69ug.png" width="400" /></div>
<br />
<div class="graf graf--p graf-after--figure" id="c1d7">
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/590721510689225110/9-2017-11-World-Bank-Fountain.pdf" href="http://pubdocs.worldbank.org/en/590721510689225110/9-2017-11-World-Bank-Fountain.pdf" rel="nofollow noopener" target="_blank">Jane Fountain</a> — government IT projects fail in part because they’re too large — should move to agile development (build small and quick, get feedback, revise)<br />
<br />
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/115621510689213187/10-5-Arianna-Legovini-Data-for-accountabiity-2017.pdf" href="http://pubdocs.worldbank.org/en/115621510689213187/10-5-Arianna-Legovini-Data-for-accountabiity-2017.pdf" rel="nofollow noopener" target="_blank">Arianna Legovini</a> — improved inspections of health facilities in Kenya seem to be improving patient safety.</div>
<div class="graf graf--p graf-after--p" id="46be">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="margin-left: 1em; margin-right: 1em;">
<img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://cdn-images-1.medium.com/max/800/1*4J2h3Z49eI7uuSOHgTwu0w.png" height="228" src="https://cdn-images-1.medium.com/max/800/1*4J2h3Z49eI7uuSOHgTwu0w.png" width="400" /></div>
<br />
<br />
<div class="graf graf--p graf-after--figure" id="09cc">
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/571121510690860881/12-Daniel-Rogger-Empirics-of-the-Public-Sector.pdf" href="http://pubdocs.worldbank.org/en/571121510690860881/12-Daniel-Rogger-Empirics-of-the-Public-Sector.pdf" rel="nofollow noopener" target="_blank">Daniel Rogger</a> — new empirics of bureaucracy — World Bank bureaucracy lab investing in substantial new descriptive work on bureaucracy and bureaucrats using both surveys & administrative data, as well as RCTs on reforms</div>
<div class="graf graf--p graf-after--p" id="2548">
<br />
<a class="markup--anchor markup--p-anchor" data-href="http://pubdocs.worldbank.org/en/330281510690869313/d2-3-Jim-Brumby-Day-Two-v1.pdf" href="http://pubdocs.worldbank.org/en/330281510690869313/d2-3-Jim-Brumby-Day-Two-v1.pdf" rel="nofollow noopener" target="_blank">Jim Brumby</a>, Raymond Muhula, Gael Rabelland — two helpful 2x2s — need to understand both capacity & incentive for reform, and then match data architecture to difficulty of measuring performance.</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="margin-left: 1em; margin-right: 1em;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="margin-left: 1em; margin-right: 1em;">
<img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://cdn-images-1.medium.com/max/800/1*DHxpX6LXPyx5uLK_tvfbrw.png" height="298" src="https://cdn-images-1.medium.com/max/800/1*DHxpX6LXPyx5uLK_tvfbrw.png" style="text-align: center;" width="400" /></div>
<br />
<div class="graf graf--p graf-after--p" id="2548">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-aSYlDH74QkY/WhcHtWj9zYI/AAAAAAABAok/5_0Dq8f7a_YJmbFqaZi9Lvy-w41tS8KowCLcBGAs/s1600/Screen%2BShot%2B2017-11-23%2Bat%2B17.38.41.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="214" data-original-width="899" height="95" src="https://3.bp.blogspot.com/-aSYlDH74QkY/WhcHtWj9zYI/AAAAAAABAok/5_0Dq8f7a_YJmbFqaZi9Lvy-w41tS8KowCLcBGAs/s400/Screen%2BShot%2B2017-11-23%2Bat%2B17.38.41.png" width="400" /></a></div>
<br />
<br />
<br />
<br />
<figure class="graf graf--figure graf-after--figure graf--trailing" id="3959"><div class="aspectRatioPlaceholder is-locked">
<div class="progressiveMedia js-progressiveMedia graf-image is-imageLoaded is-canvasLoaded" data-action-value="1*OdSSJ21hRMosSrVWngHgFg.png" data-action="zoom" data-height="220" data-image-id="1*OdSSJ21hRMosSrVWngHgFg.png" data-scroll="native" data-width="909">
</div>
</div>
</figure>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-14858987605274355382017-10-16T11:34:00.004+01:002017-10-16T11:37:01.229+01:00Open Data for Education<div dir="ltr" style="text-align: left;" trbidi="on">
<i><b>There’s a global crisis in learning, and we need to learn more about how to address it. Whilst data collection is costly, developing countries have millions of dollars worth of data about learning just sitting around unused on paper and spreadsheets in government offices. It’s time for an Open Data Revolution for Education.</b></i><br />
<i><b><br /></b></i>
The <a href="http://www.worldbank.org/en/publication/wdr2018">2018 World Development Report</a> makes clear the scale of the global learning crisis. Fewer than 1 in 5 primary school students in low income countries can pass a minimum proficiency threshold. The report concludes by listing 3 ideas on what external actors can do about it;<br />
<div>
<ol style="text-align: left;">
<li>Support the creation of objective, politically salient information</li>
<li>Encourage flexibility and support reform coalitions</li>
<li>Link financing more closely to results that lead to learning</li>
</ol>
The first of these, generating new information about learning, can be expensive. Travelling up and down countries to sit and test kids for a survey can cost a lot of money. The average RCT costs in the range of $0.5m. Statistician <a href="http://www.copenhagenconsensus.com/sites/default/files/data_assessment_-_jerven.pdf">Morten Jerven</a> added up the costs of establishing the basic census and national surveys necessary to measure the SDGs — coming to a total of $27 billion per year, far more than is currently spent on statistics.<br />
<br />
And as expensive as they can be, surveys have limited value to policymakers as they focus on a limited sample and can only provide data about trends and averages, not individual schools. As my colleague <a href="https://www.cgdev.org/blog/seeing-state-africa-data-needed">Justin Sandefur has written</a>; “International comparability be damned. Governments need disaggregated, high frequency data linked to sub-national units of administrative accountability.”<br />
<br />
Even for research, much of the cutting edge education literature in advanced countries makes use of administrative not survey data. Professor <a href="http://www.rovingbandit.com/2016/02/tom-kane-on-education-rcts.html">Tom Kane (Harvard) has argued</a> persuasively that education researchers in the US should abandon expensive and slow data collection for RCTs, and instead focus on using existing administrative testing and data infrastructure, linked to data on school inputs, for quasi-experimental analyses than can be done quickly and cheaply.<br />
<br />
<b>Can this work in developing countries?</b><br />
My first PhD chapter (published in the <a href="https://academic.oup.com/jae/article-abstract/doi/10.1093/jae/ejx021/4096501/School-Management-and-Public-Private-Partnerships">Journal of African Economies</a>) uses administrative test score data from Uganda, made available by the Uganda National Exams Board at no cost, saving data collection that would have cost hundreds of thousands of pounds and probably been prohibitively expensive. We’ve also analysed the same data to estimate the quality of all schools across the country, so policymakers can look up the effectiveness of any school they like, not just the handful that might have been in a survey (announced last week in the <a href="http://www.monitor.co.ug/News/Education/Uneb-introduces-new-system-grade-schools/688336-4137612-smcmp4/index.html">Daily Monitor</a>).<br />
<br />
Another paper I’m working on is looking at the Public School Support Programme (PSSP) in Punjab province, Pakistan. The staged roll-out of the program provides a neat quasi-experimental design that lasted only for the 2016–17 school year (the control group have since been treated). It would be impossible to go in now and collect retrospective test score data on how students would have performed at the end of the last school year. Fortunately, Punjab has a great administrative data infrastructure (though not quite as open as <a href="http://open.punjab.gov.pk/schools/">the website</a> makes out), and I’m able to look at trends in enrolment and test scores over several years, and how these trends change with treatment by the program. And all at next to no cost.<br />
<br />
For sure there are problems associated with using administrative data rather than independently collected data. As Justin Sandefur and Amanda Glassman point out in <a href="https://www.cgdev.org/publication/political-economy-bad-data-evidence-african-survey-administrative-statistics-working">their paper</a>, official data doesn’t always line up with independently collected survey data, likely because officials may have a strong incentive to report that everything is going well. Further, researchers don’t have the same level of control or even understanding about what questions are asked, and how data is generated. Our colleagues at Peas have <a href="https://chalkonboard.org/2017/10/06/assessing-the-use-of-state-administered-exam-scores-to-evaluate-impact/">tried to use</a>official test data in Uganda but found the granularity of the test is not sufficient for their needs. In India there is not one but several test boards, who end up competing with each other and <a href="http://cf.orfonline.org/wp-content/uploads/2017/07/ORF_IssueBrief_191_ExamDebacle-.pdf]">driving grade inflation</a>. But not all administrative data is that bad. To the extent that there is measurement error, this only matters for research if it is systematically associated with specific students or schools. If the low quality and poor psychometric properties of an official test are just noisy estimates of true learning, this isn’t such a huge problem.<br />
<br />
Why isn’t there more research done using official test score data? Data quality is one issue, but another big part is the limited accessibility of data. Education writer Matt Barnum wrote recently about “<a href="https://www.the74million.org/article/who-gets-access-to-school-data-a-case-study-in-how-privacy-politics-budget-pressures-can-affect-education-research/">data wars</a>” between researchers fighting to get access to public data in Louisiana and Arizona. When data is made easily available it gets used; a <a href="https://scholar.google.co.uk/scholar?hl=en&as_sdt=0%2C5&q=%22National+Pupil+Database%22&btnG=">google scholar search</a> for the UK “National Pupil Database” finds 2,040 studies.<br />
<br />
<b>How do we get more Open Data for Education?</b><br />
Open data is not a new concept. There is an <a href="https://opendatacharter.net/">Open Data Charter</a> defining what open means (Open by default, timely and comprehensive, accessible and usable, comparable and interoperable). The Web Foundation ranks countries on how open their data is across a range of domains in their <a href="http://opendatabarometer.org/4thedition/report/">Open Data Barometer</a>, and there is also an <a href="https://index.okfn.org/">Open Data Index</a> and an <a href="http://opendatawatch.com/monitoring-reporting/open-data-inventory-2016/">Open Data Inventory</a>.<br />
<br />
Developing countries are increasingly signing up to transparency initiatives such as the <a href="https://www.opengovpartnership.org/">Open Government Partnership</a>, attending the <a href="http://africaopendata.net/">Africa Open Data conference</a>, or signing up to the <a href="http://od4d.net/result/africa-data-consensus/">African data consensus</a>.<br />
<br />
But whilst the high-level political backing is often there, the technical requirements for putting together a National Pupil Database are not trivial, and there are costs associated with cleaning and labelling data, hosting data, and regulating access to ensure privacy is preserved.<br />
<br />
There is a gap here for a set of standards to be established in how governments should organise their existing test score data, and a gap for financing to help establish systems. A good example of what could be useful for education is the <a href="https://opendatacharter.net/agriculture-open-data-package/introduction/">Agriculture Open Data Package</a>: a collaboratively developed “roadmap for governments to publish data as open data to empowering farmers, optimising agricultural practice, stimulating rural finance, facilitating the agri value chain, enforcing policies, and promoting government transparency and efficiency.” The roadmap outlines what data governments should make available, how to think about organising the infrastructure of data collection and publication, and further practical considerations for implementing open data.<br />
<br />
Information wants to be free. It’s time to make it happen.<br />
<footer class="u-paddingTop10" style="background-color: white; color: rgba(0, 0, 0, 0.84); font-family: medium-content-sans-serif-font, -apple-system, system-ui, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Open Sans", "Helvetica Neue", sans-serif; font-size: 20px; padding-top: 10px !important;"><div class="container u-maxWidth740" style="box-sizing: border-box; margin-left: auto; margin-right: auto; max-width: 740px !important; padding-left: 20px; padding-right: 20px;">
<div class="row" style="box-sizing: border-box; margin-left: -20px; margin-right: -20px;">
</div>
</div>
</footer></div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-59501895562164820102017-10-11T11:32:00.001+01:002017-10-11T11:34:52.650+01:00Why don’t parents value school effectiveness? (because they think LeBron’s coach is a genius)<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
A <a href="http://papers.nber.org/tmp/33395-w23912.pdf">new NBER study</a> exploits the NYC centralised school admissions database to understand how parents choose which schools to apply for, and finds (shock!) parents choose schools based on easily observable things (test scores) rather than very difficult to observe things (actual school quality as estimated (noisily!) by value-added).<br />
<br />
Value-added models are great — they’re a much fairer way of judging schools than just looking at test scores. Whilst test scores conflate what the student’s home background does with what the school does, value-added models (attempt to) control for a student’s starting level (and therefore all the home inputs up that point), and just looking at the progress that students make whilst at a school.<br />
<br />
<a href="https://www.nytimes.com/2017/05/02/opinion/school-vouchers-charters-betsy-devos.html">David Leonhardt</a> put in well;<br />
<div>
<blockquote class="tr_bq">
“For the most part, though, identifying a good school is hard for parents. Conventional wisdom usually defines a good school as one attended by high-achieving students, which is easy to measure. But that’s akin to concluding that all of LeBron James’s coaches have been geniuses.”</blockquote>
Whilst value-added models are fairer on average, they’re typically pretty noisy for any individual school, with large and overlapping confidence intervals. Here’s the distribution of school value-added estimates for Uganda (below). There are some schools at the top and bottom that are clearly a lot better or worse than average (0), but there are also a lot of schools around the middle that are pretty hard to distinguish from each other, and that is using an econometric model to analyse hundreds of thousands of data points. A researcher or policymaker who can observe the test score of every student in the country can’t distinguish between the actual quality of many pairs of schools, and we expect parents to be able to do so on the basis of just a handful of datapoints and some kind of econometric model in their head??<br />
<br />
<img height="462" src="https://cdn-images-1.medium.com/max/800/1*o9V75Xq-KWRsaRmKF2DcUg.png" width="640" /><br />
<br />
<b><br /></b>
<b>Making school quality visible</b><br />
<br />
If parents don’t value school effectiveness when it is invisible, what happens if we make it visible by publishing data on value-added? There are now several studies looking at the effect of providing information to parents on test score levels, finding that parents do indeed change their behaviour, but there are far fewer studies directly comparing the provision of value-added information with test score levels.<br />
<br />
<a href="http://www.sciencedirect.com/science/article/pii/S0094119015000352">One study from LA</a> did do this, looking at the effect of releasing value-added data compared to just test score levels on local house prices, finding no additional effect of providing the value-added data. But this might just be because not enough of the right people accessed the online database (indeed, <a href="https://academic.oup.com/qje/article-abstract/123/4/1373/1933172/Information-School-Choice-and-Academic-Achievement?redirectedFrom=fulltext">another study</a> from North Carolina found that providing parents with a 1-page sheet with information that had already been online for ages already still caused a change in school choice).<br />
<br />
It is still possible that publishing and properly targeting information on school effectiveness might change parent behaviour.<br />
<br />
Ultimately though, we’re going to keep working on generating value-added data with our partners because even if parents don’t end up valuing the value-added data, there are two other important actors who perhaps might — the government when it is considering how to manage school performance, and the school themselves.<br />
<br />
<footer class="u-paddingTop10" style="background-color: white; color: rgba(0, 0, 0, 0.84); font-family: medium-content-sans-serif-font, -apple-system, system-ui, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Open Sans", "Helvetica Neue", sans-serif; font-size: 20px; padding-top: 10px !important;"><div class="container u-maxWidth740" style="box-sizing: border-box; margin-left: auto; margin-right: auto; max-width: 740px !important; padding-left: 20px; padding-right: 20px;">
<div class="row" style="box-sizing: border-box; margin-left: -20px; margin-right: -20px;">
</div>
</div>
</footer></div>
</div>
Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0tag:blogger.com,1999:blog-4867230308159901547.post-33596131237985009472017-09-26T18:02:00.001+01:002017-09-26T18:02:21.475+01:00JOB: Research Assistant on Global Education Policy<div>I’m hiring a full-time research assistant based in London, for more details see the <a href="https://www.arkcareers.org/e/phases_and_departments/positions/dAshRJWpjbpk_u3SEV1Ob0">Ark website here</a>.</div>
<div> </div>
<div>---</div>
<div> </div>
<div>Research and evidence are at the heart of EPG’s work. We have:</div>
<div>
<div>
<ul>
<li>Collaborated with JPAL on a large-scale field experiment on school accountability in Madhya Pradesh, India</li>
<li>Commissioned a randomized evaluation by IPA of Liberia’s public-private partnership in primary schooling</li>
<li>Led a five-year randomized trial of a school voucher programme in Delhi</li>
<li>Helped the Ugandan National Examinations Bureau create new value-added measures of school performance</li>
<li>Commissioned scoping studies of non-state education provision in Kenya and Uganda<strong> </strong></li>
</ul>
<p>Reporting to the Head of Research and Evaluation, the Research Assistant will contribute to EPG’s work through a mixture of background research, data analysis, writing, and organizational activities. S/he will support and participate in ongoing and future academic research projects and EPG project monitoring and evaluation activities.</p>
<p>The role is based in Ark’s London office with some international travel.</p>
</div>
<div>
<p><strong>The successful candidate will perform a range of research, data analysis, and coordination duties, including, but not limited to, the following:</strong> </p>
<ul>
<li>Conduct literature and data searches for ongoing research projects.</li>
<li>Organize data, provide descriptive statistics, and run other statistical analysis using Stata and preparing publication quality graphics</li>
<li>Collaborate with EPG’s project team to draft blogs, policy briefs, and notes on research findings.</li>
<li>Support EPG’s project team in the design and implementation of project monitoring and evaluation plan</li>
<li>Provide technical support and testing on the development of value-added models of school quality</li>
<li>Coordination and update of the EPG/GSF research repository</li>
<li>Organise internal research and policy seminars</li>
<li>Perform other duties as assigned. </li>
</ul>
<p><strong>The successful candidate will have the following qualifications and skills:</strong> </p>
<ul>
<li>Bachelor’s (or Master’s) degree in international development, economics, political science, public policy, or a related field.</li>
<li>Superb written and verbal communication skills.</li>
<li>Competence and experience conducting quantitative research. Experience with statistical software desired.</li>
<li>Familiarity with current issues, actors and debates in global education</li>
<li>Proven ability to be a team player and to successfully manage multiple and changing priorities in a fast-paced, dynamic environment, all while maintaining a good sense of humor.</li>
<li>Outstanding organization and time management skills, with an attention to detail.</li>
<li>Essential software skills: Microsoft Office (specifically Excel) and Stata</li>
<li>Experience working in developing country contexts or international education policy -- a plus</li>
<li>Experience designing or supporting the implementation of research evaluations and interpreting data -- a plus</li>
<li>Fluency or advanced language capabilities in French -- a plus</li>
</ul>
</div>
</div>
<div> </div>Leehttp://www.blogger.com/profile/08388633107597320203noreply@blogger.com0