What if all those times really were different?

Hopefully you've been following the tussle between the famed Reinhart and Rogoff and the researchers who tried to replicate their results (Hendon, Ash, and Pollin). This post isn't really about that, but here's a quick summary: RR claim that after a country's debt hits 90% of GDP  or more, it grows much more slowly. HAP take RR's data and find that RR have done some dubious things with it, including A) randomly excluding countries from the sample, B) weighting outcomes by "growth period" instead of by year, and C) making an Excel error. The Excel error is not a big deal, just a "gotcha", but the other two things are pretty important, especially the weird weighting. When the data are weighted by year and the excluded countries are allowed back in the sample, HAP find that:

1) There is no "sudden deceleration" or "structural break" in growth when debt hits 90% of GDP, i.e. there is nothing special about 90% as RR often claim, and

2) Countries with more debt may tend to grow more slowly, but the effect is considerably smaller than RR claim, and not statistically significant.

RR respond by email and defend themselves, focusing mainly on the first part of the latter result (i.e. that higher-debt countries grow more slowly), largely ignoring the other findings, and not really defending the oddities of their approach. In particular, the notion that 90% is some kind of special threshold - which Reinhart and Rogoff have repeated time and again when making the public case for austerity -  appears to lack material support, but RR's email response repeats the 90% level but doesn't really address its disappearance in HAP's results.

Hopefully a more comprehensive response to HAP's findings is forthcoming. (Update: Here is that response. Still pretty unsatisfying.)

Also, many smart people point out that even the weak result that slower-growing countries may tend to have (slightly) higher debt levels is not necessarily a causal relationship - it may be that slow growth makes countries borrow more in relation to GDP, not vice versa. Reinhart and Rogoff claim, in their email response, that they never presented this relationship as causal in their book, This Time Is Different, or in their papers. However, in their public op-eds, they clearly and unambiguously assert that debt causes slow growth rather than vice versa.

(In fact, as an aside, I don't even think that RR's usual critics go far enough in challenging the debt-growth result. RR's cross-country sample does not control for a country's level of development. So if debt/GDP levels tend to grow over time, and if countries converge a la the Solow growth model, then slower growth and higher debt could have a purely benign non-causal relationship.)

In short, RR appear to be doing everything they can to imply that correlation = causation, while never seriously addressing the possibility that it might not.

But actually, this post is not about that dispute. This post is about Reinhart and Rogoff's book, This Time Is Different.

The book basically presents a whole lot of features of financial crises across time and space, and shows how these features are similar. The authors thus try to draw some general conclusions about the "anatomy" of a crisis. The book has been hailed as an alternative to the sort of formal modeling done by most academic macroeconomists, and I agree that the naturalistic approach adds value. I just think its limitations are too rarely recognized. Naturalistic observations can be all wrong.

Reinhart and Rogoff carefully document a number of apparent similarities between financial crises. The implication is that financial crises are all inherently similar - that even if we don't know exactly what causes them, they must have a common cause because they look so similar. In other words, this time isn't different.

But that's not really valid! Similar-looking things can have a bunch of different underlying causes. Wars might be fought in similar fashion, but the causes of the wars might vary wildly - religious fervor for one war, desire for grazing land for another. Communicable diseases may all involve fever and weakness, but some are bacterial and some are viral.

Similarly, financial crises might be different animals that simply look the same. Some might be caused by domestic asset bubbles, others by currency pegs and foreign capital flows. Some might involve borrowing in external currencies, others mostly domestic borrowing. Some might be triggered by wars or political instability, others by the collapse of unsustainable growth models, others by overly complex financial systems. 

Bottom line: If you just pick out the similar features, you will bias yourself toward concluding in favor of structural underlying similarity, where no such conclusion is warranted!

And here's my second problem with the methodology of This Time Is Different: Documenting similarities automatically biases oneself (and one's readers) toward drawing the kind of inappropriate causal conclusions that many draw from RR's finding on debt and growth. If you focus mainly on the fact that all deadly communicable diseases involve fevers, you will probably try to treat them by dipping sick people's heads in buckets of ice. That's not going to work!

"Similar symptoms, disparate causes." When evaluating the history of financial crises, we should constantly keep this phrase in mind. This is the alternative hypothesis; maybe each time is different. Proving a common cause, or even a common structure, requires more than simply tabulating lists of similarities.

Now a disclaimer: I strongly suspect that RR are onto something, and that certain causal features of financial crises really do crop up again and again across time and space. But I think that books like This Time Is Different are merely jumping-off points for an investigation of that hypothesis; they do not constitute any kind of proof. Naturalism is where understanding of the world begins, but not where it ends.