- 8 The Rhetioric of Significance Tests
- University of Wisconsin Press
- book
- View Citation
- Additional Information

8 THE RHETORIC OF SIGNIFICANCE TESTS Statistical Significance Has Ruined Empirical Work in Economics Econometrics in particular has made a tragic mistake by not facing its rhetoric of importance. The tragic mistake is to turn back to statistics itself to answer the question whether the deviations from purchasing power parity are important. It makes the statistical machinery into something that takes care of the whole scientific job, from start to finish, soup to nuts. But you can see that something is wrong. We care about the statistics of purchasing power parity because we are humans with some purpose in mind, not because some number is absolutely high or low. At some point, in other words, you have to turn away from your statistical machinery and ask the common-sense question, '~ll right, people : What does it matter?" Is the gravitational effect of one galaxy on another worth taking account of? Is the effect of the meteor big enough to account for the extinction of the dinosaurs? Are the prices in the United States importantly connected to those in the rest of the world? The numbers are necessary material. But they are not sufficient to bring the matter to a scientific conclusion. Only the scientists can do that, because "conclusion" is a human idea, not Nature's. It is a property of human minds, not of the statistics. The tragic turn was taken in the 1940s by Lawrence Klein and other inventors of modern econometrics. What Klein and everyone in modern science is looking for is a mechanical, uncontroversial way of deciding whether some effect is large or small. No human judgments, please: we're scientists. Unfortunately for economic science, and some other sciences like medicine, right at Klein's elbow in the 1940s was a machine that seemed to promise an uncontroversial way of deciding whether a number is large or small, inside statistics, without messing with human judgment. Horribly for the outcome in economics the machine was already called "statistical significance," and had been so called for seventy 112 113 The Rhetoric of Significance Tests years. This was unspeakably sad. Klein picked up the machine and started using it to claim he had gotten "significant" results. No need to assess whether a number was large or small. Klein believed that the very statistics used to estimate, say, the effects of the foreign prices on American prices could be used to decide on their own whether they really mattered, whether a slope of 1.20, or 0.80, or .08, for that matter, was worth getting excited about. In Klein's very first scientific paper, published in 1943 when he had just gotten his PhD. in Economics from MIT (Paul Samuelson supervised his dissertation), he says at one point, in words that were to become formulaic in people who followed him, "The role of Y in the regression is not statistically significant. The ratio of the regression coefficient to its standard error is only 1.812. This low value of the ratio means that we cannot reject the hypothesis that the true value of the regression coefficient is zero" (1985, p. 35). Others imitated him, with much less discernment. The practice grew and grew, especially in the 1970s when the computer chip came to maturity. Pretty soon everyone in economics thought that statistical significance was the same things as scientific significance, that you could skip that last step of scientific work, the human assessment of largeness or smallness. I said Kravis and Lipsey (1978) were good economists. They draw a distinction on pages 204-205, on page 235, and again on page 242 of their paper between the statistical and the economic significance of their results. They make the point so often that it has to be counted as one of the major points in the paper. Even small differences between domestic and export prices, they say, can make a big difference to the incentive to export: "This is a case in which statistical significance [that is, a correlation of the two prices near 1.0, which one might mistakenly suppose to imply that they were insignificantly different] does not necessarily connote economic significance" (1978, p. 205). Yet they don't follow through. No wonder: without a rhetoric of economic significance, and in the face of a rhetoric of statistical significance with the prestige of alleged science behind it, they are not aware they need to: the statistics take care of themselves. The abuse of the word "significant" in connection...

**You are not currently authenticated.**