Randomised control trials are not new. As Ben Goldacre likes to say, the first example is in the bible. The idea is simple.  It is normally fairly difficult to work out what the effect of some policy intervention is, because there are all sorts of different things going on. To use a medical example, you’re not sure if the medicine you gave made the patient well, or whether it was really just time/placebo effect/exercise/better diet etc etc. Regressions try to estimate the effect of all these different things: controlling for all the factors you don’t care about in order to get an estimate of the factor you do care about.

Enter RCTs. The idea is that you take a large group of people, split them in half (or some other fraction) and then treat the two groups differently. In medicine the best practise is to administer the best pre-existing alternative as the control, and the new thing to the treated group. The fuss is that in recent years economists have started to use this technique to answer big questions. To see some examples look at the leading light of the randomistas, Esther Duflo. In development this has included novel/interesting/gimmicky questions about how to improve educational outcomes. Some have been very interesting because they help us understand something, others feel like they’ve been written to be interesting to non-experts (a kind of ted talk, cool geeky research).

To academics, RCTs have been taken on board because they offer a clear identification strategy. That means you can be fairly sure that you are accurately measuring the effect of your policy intervention. This means good publications. Next in the series I’ll look at the critique of the methods. But for now, here are some good (pro) links on RCTs:

• Podcast: Banerjee on econtalk, with a skeptical Russ Roberts
• Video: Duflo on TED talks
• Article: Angrist and  Pischke‘s NBER version of their JEP paper
• Poor Economics, a book by Banerjee and Duflo

This is undoubtedly a real phenomenon: people get experience of living in developing countries, then get nice jobs, a family and comfortable. Which  means most of the people with power have experience of living in developing countries a while ago, and of visiting a few more recently. I don’t think the credential of living in a low income country makes your opinion more valid. It makes the likelihood you’ll have an interesting insight higher, but it doesn’t garantee that what you are saying right now is correct.

One such instance was his interview with Eugene Fama, famous for his efficient market hypothesis. Essentially, he says that prices reflect all information and so an investor can’t predictably beat the market. This view even predicts that some people will appear to beat the market, through pure chance.

Warren Buffet dealt with this kind of view in a lecture in 1984. In the last 27 years, he’s continued to do pretty well, and is one of the three richest men in the world. If stock picking was pure chance, I don’t see how that explains the continued success of the best (of course it could explain short term good performance). I don’t research finance and I don’t follow all the arguments in this, but it appears bizarre to me.

Perhaps anyone that can see how the market is inefficient is able to make more money exploiting through investing than through writing about it, which means academia will tend to. Michael Lewis’ book certainly seem to show that some people predicted exactly where the system was going to break before it did. It certainly doesn’t seem to be an easy time to argue that financial markets are inefficient.

It is frustrating that whatever evidence is produced Fama just explained it away chance, given the track record of success of investors.  He also dismissed behavioural finance with no real explanation (and wasn’t pressed on this by Roberts). I guess nothing is more frustrating than listening to two people agree, when you think they are avoiding evidence.

Here is WIllingham’s book, which has influenced my thinking about learning quite a lot. On top of that, it is a great read.

For all these reasons, the Chinese model is not an easy or attractive model for other countries to follow as they seek to build effective states and thereby reduce poverty. Yet the western European model is problematic, too, since up until World War II, it emerged largely out of the need to fight countless wars.

It really is fascinating trying to work out why we can work together in society, and why we are trustworthy people most of the time. Of course, it is another question how to produce good governance and good societal norms in the future, but we should at least learn from history as much as we can. That doesn’t mean we should prescribe a nice dose of war though…

I’m looking forward to this book which should drill this vein a little more. To get a preview, you could always watch the lecture version below. I’m betting that even in the book length version they don’t recommend a few decades of war.

… one major weakness of many of today’s indicators and indexes, especially those that measure policies and institutions, is that a law on paper may be miles away from the reality of enforcement on the ground (as CIPE and its partners recently found in Kenya, for example) . Conditions may also differ radically from one region to another within a single country — Lima may be full of successful small and medium enterprises, but in rural areas of Peru attitudes towards entrepreneurship are still weak.

I disagree. The problem with the indices that I’ve looked at isn’t that they are right in general but wrong in specifics, or that they lack detail. Instead I think indices are very sensitive to a host of decisions that are ‘back end’ and the uninformed reader underestimates the effects that simple decisions have. An index is deceptively simple to explain, and it implies that this lack of complexity translates into a lack of sensitivity to different choices. From what I’ve seen they appear more whimsical and sensitive than regression techniques. They might be more dangerous as their simplicity can give the reader a false confidence.

Obviously, if the indices aren’t robust to simple changes in measurement technique, then they’re not giving a lot of information. And that means that the instruments on the pilots dashboard (to use CIPEs metaphor) aren’t working very well.

This came to mind recently as I was pondering the difficulty that non-economists have in assessing the merits of different arguments. Take a typical disagreement between Paul Krugman and Greg Mankiw. Normally if I can’t verify a statement, I would rely on credentials to separate two opinions. For example, I listen to the mechanic about a car and an experienced or recommended mechanic more than an inexperienced one. But in this case they both have incredible credentials. Yes one has a Nobel prize, but they are the 18th and 21st most cited economists in the world. A political party would be able to convince an average voter that they have a real expert on their side with either economist, despite their opposing views on some important matters.

Ariel Rubinstein is very pessimistic about the ability to know things, rather than just express points of view, using economics (e.g. here and here, both pdf). There is an absolutely great article by Mike McGovern discussing ‘story time’ in two books by Paul Collier, where Collier earns the trust of a reader using stats, and then launches into wild speculation (story time) to explain some correlation or other (see also Andrew Gelman’s discussion of Mike’s article). I don’t offer a solution to distinguish between the view points of others. I suppose more transparent authors can be trusted more but this relies on someone spending time to verify claims. Authors that stay within their field of expertise can be more readily trusted. But the best weapon is a general scepticism that realises that proof almost never conclusively proves what the author wants it to. You need to think about how likely it is that the author is right. Because we’ve all got PhDs, and we disagree on some pretty important stuff.