Content area
Full Text
Abstract
Meehl (1967) attacked significance tests used to appraise directional theoretical predictions. With perfect statistical power, the probability approaches .5 of significance in the predicted direction -- even for meritless theories. Feeble theory corroboration results. I argue that directional predictions, not significance tests, produce the feeble corroboration. I review previous solutions to Meehl, rejecting all except the multiple corroboration solution by Lykken (1968) and Kukla (1990). I defend this solution against Meehl's criticisms (Meehl, 1990b) and extend it: The complexity of a typical research design enables a theory to make multiple directional predictions, each of which can be adequately tested with a significance test. Reasonable theory corroboration results when all or most predictions succeed and I discuss techniques to assess this.
Paul Meehl, in a classic paper (Meehl, 1967), assailed the use of significance tests to appraise theories in "soft" psychology (e.g., clinical, counseling, developmental, personality, and social psychology). Meehl repeated his attack in 1978 (Meehl, 1978), and a reply to Meehl by Serlin and Lapsley (1985) generated a great deal of work (Dar, 1987; Campbell, 1990; Chow 1990; Dar, 1990; Fiske, 1990; Humphreys, 1990; Kimble, 1990; Kitcher, 1990; Kukla, 1990; Maxwell & Howard, 1990; McMullin, 1990; Meehl, 1990a, 1990b, 1990c, 1990d; Rorer, 1990; Serlin & Lapsley, 1990). Increasingly philosophical and decreasingly accessible to the psychologist, this work departed greatly from Meehl's attack without resolving it. Indeed, many of the more philosophical papers did not even mention it. The purpose of this paper is to bring the focus back to Meehl's attack, to provide a critical review of previous solutions, and to offer my own response.
Meehl's Paradox
Meehl (1967) described a paradox which illustrates the harm that evaluating theories with significance tests can do to soft psychology. His summary:
Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and hence increases corroborability. In most psychological research, improved power of a statistical design leads to a prior probability approaching 1/2 of finding a significant difference in the theoretically predicted direction. Hence the corroboration yielded by "success" is very weak, and becomes weaker with increased precision. (p. 103)
The paradox compared psychology and physics on the consequences of increased experimental precision to theory - testing research. (Increased experimental precision...