Coase Against Torture
I’m taking a course on data analysis, and the book contains the following quote from economist Ronald Coase: “If you torture the data long enough, Nature will confess.” The book makes it seem like Coase approves of this process, calling it a “lofty goal.”
As someone with a casual interest in economics, I didn’t think Coase, a theoretician, would have such a cheery opinion of purely statistical techniques. A quick trip to Wikipedia suggests that my class’ book has taken this quote out of context.
Gordon Tullock, fellow-travelling economist, uses the quote in a journal article while arguing against the use of “fancy models and fancy statistical significance,” citing his personal experience with Coase as authority: “I have heard him say this several times. So far as I know he has never published it.” Here is the relevant paragraph in full:
“More important, there is data torture. As Ronald Coase says, ‘if you torture the data long enough it will confess’. The young researcher, convinced he knows the truth will make changes in his specifications and very likely produce significant results. In some cases this is correct; his original specification was wrong and his new one is right. Nevertheless, this procedure reduces the significance of the significance test.”
