No, Rwanda didn't "fiddle" its poverty stats

A couple of weeks ago, France24 ran a story featuring accusations by Belgian Professor Filip Reyntjens that the Government of Rwanda had manipulated its poverty statistics. The truth, to my relief*, is somewhat less exciting.

What seems to have actually happened, is that Rwanda quite resonably decided to update the methodology for calculating what the poverty line should be, but then found that the new methodology led to an implausibly high poverty line, and so decided to (slightly arbitrarily) “adjust” the new methodology, resulting in the final poverty line being almost exactly what you would have expected it to be had you simply updated the original poverty line for inflation.

It took me a while to figure all this out, as the original criticism and rebuttal by NISR weren’t entirely clear, and it was only in Filip’s reaction to NISR’s rebuttal that I grasped his (mistaken) point (here’s also the Rwanda EICV4 Report and EICV3 Report).

How is poverty measured?

Rwanda has followed a fairly typical process – set a poverty line by first defining a minimum quantity of calories needed, second working out how much it would cost a poor person to buy that many calories, third increasing that amount by 40% to account for some basic minimum non-food spending needs. Then to get your poverty rate, just calculate how many people spend less than the poverty line.

What was the disagreement about?

Rwanda’s poverty line was set in 2001 based on how much it cost then to purchase a basket of goods that poor people bought back then. You need to keep your methodology consistent over time to allow for fair comparisons, but its also reasonable to think that the minimum consumption basket is likely to change over 15 years of rapid growth.

The government of Rwanda decided to keep the minimum assumed number of calories (2,500 per day, which is pretty high), but change step 2 – the way of working out how much it costs to buy these calories. In a normal survey year, this cost is simply updated for inflation (even if prices and consumption habits have changed in the meantime). This year, Rwanda decided to make an update to the prices and consumption habits, but found something odd. Most poor people consume far fewer than the minimum number of calories – almost half. So how do you construct a hypothetical “ bare minimum" food consumption basket, that is twice as big as what people actually buy? Do you just double everything? Or do you assume that if people bought more food than they did, they might buy more of some items than others? This is where the big disagreement presumably came. Rather than choosing to simply double everything, the Rwandan stats agency made a few arguably arbitrary choices about which items to increase and which to decrease, that has a big effect on the overall price of the basket, and therefore the overall poverty line, and therefore the poverty rate.

Why is Filip Reyntjens wrong?

Filip argues, correctly, that Rwanda’s assumptions about how to scale up consumption patterns to reach their minimum calorie basket, affects the overall line. In fact, their adjustments reduce the line by 19%. But his next step is wrong. He argues that as this new methodology line has been reduced by 19%, we should also reduce the 2010/11 line by 19%, giving a substantially lower poverty rate in 2010/11, and therefore an increase in 2013/14. But he misses the intermediate step – the Rwandans didn’t just adjust the new food basket, they first also calculated a whole new food basket.

Yes, what should really happen is for the new methodology to applied retrospectively to all the old survey data to allow for truly comparable numbers, but the adjustment made to the new methodology leads you to a poverty line that is basically the same as it would have been with the old methodology anyway.

Implications for how we measure poverty?

One thing that this choice really highlights is the number of assumptions you sometimes need to make, and the fragility of the whole concept of poverty estimates.

Here's an example of another seemingly arbitrary choice of assumption with big consequences – the Indian stats agency used to measure poverty with surveys that asked people how much food they had bought in the last 30 days, longer than practice elsewhere which uses 7 day recall periods. To their great credit the stats agency decided to run a randomised control trial to test these two methods against each other. The result was that moving from a 30 day to a 7 day recall period increased measured consumption massively – and reducing poverty by 175 million people – close to half of all those in poverty (from Angus Deaton’s 2014 LSE lecture, via Nic Spaull).

The bottom line: measurement is hard, and it is possible for reasonable people to disagree, without there necessarily being any nefarious trickery.

---

* Relief, because I have previously worked both for OPM as a staff member, on a project with OPM for the Rwandan Stats agency, and directly on a project for the Rwandan Ministry of Finance.

Why Germany is probably doing more for Syria than the UK

How do you compare the good that the UK is doing with its whopping 0.7% aid budget, against the good that Germany is doing by accepting large numbers of refugees? A smart (German) friend asked me if there are any numbers on the size of the remittances we might expect to see from Syrian refugees in Germany to Syria. Of course, remittances are far from the most important reason for accepting refugees, but they do allow for a nice easy cash sum with which we can make a comparison to aid flows.

The UK is spending somewhere between £200 million and £400 million on Syria this year. For comparison, whilst Germany is ramping up aid spending, it is still less than 0.4% of GDP overall.

But in terms of numbers of refugees, Germany expects to take 800,000 this year (compared to just a few thousand in the UK), though fewer than that have been documented so far, and not all will be Syrian. Let’s assume for a moment that the total will be 400,000 from Syria, and they will be quickly processed so that they are able to work. If every Syrian refugee in Germany was able to send home £1,000 to family and friends, that already equal Britain’s aid budget for Syria. Is £1,000 a realistic prospect? One way to think about this is to look at remittances from existing migrants in Germany (p33) to the middle east. There are currently around 67,000 migrants from Lebanon living in Germany, who send back to Lebanon almost $1 billion a year - that’s around £9,500 each, which seems almost implausibly large, but who knows, the Chinese and Vietnamese also send home large sums, and the Nigerians send home even more. In any case, it certainly seems plausible, even likely, that Syrian refugees to Germany, once permitted to work for even low German salaries, will be able to send home at least £1,000, if not more.

Is “technical assistance” counterproductive?

Duncan Green reviews a fascinating new AidData survey on what developing country policymakers think about donors.

One of the key findings he points to is that

"Reliance upon technical assistance undermines a development partner’s ability to shape and implement host government reform efforts. The share of official development assistance (ODA) allocated to technical assistance is negatively correlated with all three indicators of development partner performance."

Obviously alarm-bells should be ringing about such firm causal conclusions being drawn from a correlation. One of the best ways of assessing these things is with some rigorous eyeball econometrics - take a look at this chart showing the relationship driving that claim.

Looks to me like that is a pretty weak relationship, and you could just as easily have drawn a totally flat line (no relationship). And indeed, deep in the weeds, Table E.11 tell us that this is a simple correlation between these two variables with a sample size of just 44 data points (countries). It might technically pass a statistical significance test, but it doesn’t really tell us that there is a reliable correlation, let alone causality. And even if you believed the estimated negative relationship - it’s really not huge - implicitly going from 0% aid on technical assistance to a massive 50% of aid spent on technical assistance would only reduce the perceived quality of your advice by 0.55 points on a 5 point scale.

Bottom line for technical assisters - don’t give up your day job quite yet.

Why are people so opposed to immigration? #142538

As the evidence piles up that migrants don’t steal jobs (one of the implications of them being human beings is that migrants also buy stuff - so they create exactly as many new jobs as they “take”), some of the more sophisticated immigration opponents turn to the negative impacts of immigration on other things such as housing or public services instead to support their case.

So what does the research evidence say about the impacts of immigration on public services? Really very little actually. The University of Oxford’s Migration Observatory says that there is “no systematic data or analysis.” In health, we know that many healthcare providers are immigrants, but it’s hard to know the impact of migrants as users of health services as (rightly) nobody records people’s migration status when they go to the doctor.

Using household survey data, Jonathan Wadsworth at Royal Holloway found that (shock!) immigrants tend to use GP services and hospitals at roughly the same rate as natives (via Ferdinando Giugliano in the FT).

Taking another approach, a new paper by Osea Giuntella from the Blavatnik School of Government at Oxford, combines household survey data with administrative data on NHS waiting times. Do you need to wait longer for a referral or in A&E in places where there are more immigrants? Come find out at the CGD Europe research seminar on Weds 18 Nov (there will be sandwiches).