In lieu of an abstract, here is a brief excerpt of the content:

Observational Studies 4 (2018) 61-64 Submitted 12/17; Published 1/18 Statistical Criticism, Self-Criticism and the Scientific Method David Rindskopf drindskopf@gc.cuny.edu CUNY Graduate Center New York, NY 10016, U.S.A. I have long admired Bross’s article on statistical criticism, and in my mind it has much broader implications than those Bross chose to present. In fact, others have discussed much the same points in the context of all of scientific methodology. I also hope that Bross would say that his rules should be applied to self-criticism as well as criticism by others. There would be less need for criticism by others if there were better self-criticism in the first place. Cochran (1965) is cited by Rosenbaum: When summarizing the results of a study that shows an association consistent with the causal hypothesis, the investigator should always list and discuss all alternative explanations of his results (including different hypotheses and biases in the results) that occur to him. Such advice is valuable, but alas mere mortals (including me) are usually deficient in self-criticism. What sometimes helps is to put a piece away for a while after writing it, and then going back specifically to criticize it before putting it on display for others to criticize. In spite of the difficulty of self-criticism, the author is best placed to criticize, having full access to all the data of the study. As data sets more often become publicly available, this advantage will diminish. Even so, many data sets have restricted access and in those cases it will remain difficult for critics to test alternative theories. Chamberlain (1890, reprinted 1964) and Platt (1965) have similar views as Bross, but applied more broadly to scientific methods as a whole. Chamberlain discussed the affection a scientist feels for his or her ideas, and how unconscious bias (“tubular vision” in Bross’s terms) takes over: As soon as this parental affection takes possession of the mind, there is a rapid passage to the adoption of the theory. There is an unconscious selection and magnifying of the phenomena that fall into harmony with the theory and support it, and an unconscious neglect of those that fail of coincidence. The mind lingers with pleasure upon the facts that fall happily into the embrace of the theory, and feels a natural coldness toward those that seem refractory. Instinctively there is a special searching- out of phenomena that support it, for the mind is led by its desires. There springs up, also, an unconscious pressing of the theory to make it fit the facts, and a pressing of the facts to make them fit the theory. When these biasing tendencies set in, the mind rapidly degenerates into the partiality of paternalism. The search for facts, the observation of phenomena and their interpretation, are all dominated by affection for the c ⃝2018 David Rindskopf. Rindskopf favored theory until it appears to its author or its advocate to have been overwhelmingly established. The theory then rapidly rises to the ruling position, and investigation, observation, and interpretation are controlled and directed by it. From an unduly favored child, it readily becomes master, and leads its author whithersoever it will. The subsequent history of that mind in respect to that theme is but the progressive dominance of a ruling idea. Chamberlain suggests that instead researchers should develop alternative theories so that they do not maintain an allegiance to any one of them. Platt (1964) thought that the best way to make progress in science was to eliminate alternative theories with each experiment. In that context, he said: In its separate elements, strong inference is just the simple and old-fashioned method of inductive inference that goes back to Francis Bacon. The steps are familiar to every college student and are practiced, off and on, by every scientist. The difference comes in their systematic application. Strong inference consists of applying the following steps to every problem in science, formally and explicitly and regularly: 1) Devising alternative hypotheses; 2) Devising a crucial experiment (or several of them), with alternative possible outcomes, each of which will, as nearly as possible...

pdf

Additional Information

ISSN
2767-3324
Pages
pp. 61-64
Launched on MUSE
2021-06-04
Open Access
Yes
Back To Top

This website uses cookies to ensure you get the best experience on our website. Without cookies your experience may not be seamless.