September, 1998

Brunswik's Challenge

Kenneth R. Hammond
Professor Emeritus, Department of Psychology
University of Colorado

Now that the previous three or four decades have made it obvious that judgment and decision making can be studied, the primary methodological question that has emerged over the past decade is whether the research methods conventionally employed will permit the generalization of results beyond the laboratory conditions within which the results have (usually) been obtained. It must be recognized, however, that there are those who will dispute that statement, and will argue that generalization is not the point; they will say that establishment of empirical regularities, or laws of behavior, within the laboratory is the point. I will not address that dispute here; I believe it to be unresolvable. The argument that is critical, and I hope, is resolvable, is between those who believe that current laboratory research practices do provide generalization to conditions outside the laboratory, and those who do not believe that such generalization is (ordinarily) possible.

The argument that is critical, and I hope, is resolvable, is between those who believe that current laboratory research practices do provide generalization to conditions outside the laboratory, and those who do not believe that such generalization is (ordinarily) possible.

The current centrality of this dispute can be seen in the title of an article by Wiseman and Levin (1996). The authors make the extraordinary jump from their laboratory situation to the shootdown of the Iranian jetliner by the USS Vincennes. They acknowledge that "no laboratory task could duplicate the conditions faced by the decision makers in this case" but then go on to say: "Nevertheless (italics added), decision-making experts brought in to provide expert testimony to a U.S. House Armed Services Committee were able to draw parallels to the type of decision making task so often studied in the laboratory, and to describe plausible biasing effects in the Vincennes incident based on heuristics demonstrated in laboratory settings employing hypothetical consequences" (p. 249). Two points should be noted: (1) no justification for the generalization from the laboratory to the Vincennes situation was offered aside from the assertion that "experts were able to draw parallels" between the laboratory situation "so often studied." A footnote reference to the testimony of Paul Slovic is offered as support for this statement; Slovic's testimony itself, which was in fact quite different, was not presented, however. Note that the entire generalization hinges on the word "Nevertheless," for which no justification whatever is presented.

I include these remarks by Wiseman and Levin to indicate the superficial manner in which this important topic is currently treated, and the manner in which an unjustified generalization can be perpetuated. Other researchers will, no doubt, state that "As Wiseman and Levin have shown" to support the claim of generalization. In what follows I provide a brief history of Brunswik's efforts to address this topic and his challenges to the methodological status quo during the 1940s and 1950s. In later pieces I will take up the matter of representative design and its current confusion with the concept of ecological validity.

  Brunswik's Challenge

Laboratory research by psychologists has always been treated with some suspicion by nonpsychologists, as well as some psychologists—the general idea being that the life of the mind is simply too complicated to be examined in the restricted conditions of the laboratory which, the critics often claim, "trivialize" the topic. That question comes down to what is now treated as the generalization of results: Can the results obtained in narrow, highly controlled conditions be generalized to conditions outside the laboratory? The question was introduced in a prominent, detailed and scholarly manner by Brunswik in several articles beginning in 1943 when he called for "a fundamental, all-inclusive shift in our methodological ideology regarding psychology" (p. 261; see also Brunswik, 1952, 1956; Hammond, 1966, 1996a, 1996b, Cooksey, 1996). Note Brunswik's use of the term "ideology." That in itself would have been enough to evoke wrath from the academic establishment; it bordered on the insulting to charge that members of the establishment were following an "ideology" (which connotes a cherished belief, a dogma, or a doctrine), rather than straightforward scientific method. That came close to an insult because the experimental psychologists of the day believed that their methods were establishing psychology as a science, and that they were simply extensions of logic, not part of a doctrine or dogma of any sort.

 

Brunswik had chosen the worst possible time to challenge the use of the new research methods in psychology.

That was not all, however. Brunswik had chosen the worst possible time to challenge the use of the new research methods in psychology. For during the period immediately following World War II, psychology discovered R. A. Fisher's (1925) methods of "experimental design" as well as the methods introduced by Neyman and Pearson (1933). Both offered powerful new ways of designing experiments that were being applied with great success in the field of agriculture (For an interesting description of the development of research methods in psychology prior to World War II, see Gillis & Schneider, 1966). In short, just as psychologists were finding themselves in possession of new and impressive methods for designing experiments that would bring them new knowledge—and new status as scientists—Brunswik was trying to tell them that it was all a mistake.

  The new methods of designing experiments had been quickly taken up by researchers in agriculture in the 1930s and textbooks were soon written with specific examples of their application to problems in that field. As a post World War II graduate student in psychology, 1945-1948, I stumbled on these new principles of experimental design in textbooks in agricultural research, and soon found myself in the odd position of explaining experimental designs to my professors in terms of "fertilizer, alternating crop rows, and plant growth" because I wanted to make use of the new methods in my dissertation. The first textbook for psychology students that included the new methods did not appear until 1949 (McNemar, 1949). Several textbooks appeared soon after, however, and psychologists—students and faculty alike—took hold of these ideas with enthusiasm. It was not long before every graduate student in psychology was required to pass examinations on "experimental design" and to become sufficiently competent in the application of these methods to construct their experiments according to the (unquestioned) principles of "the systematic design of experiments." If the reader is a graduate student in psychology anywhere in the world, she or he is undoubtedly familiar with, and has been examined for, competence in these methods (often known as "analysis of variance") and modern variations thereof.

The reason for this enthusiasm was that Fisher's experimental designs not only offered psychologists much more powerful ways to carry out their research, but a much greater opportunity to claim the mantle of "scientist". The Fisherian methods, and the (very different, and opposing) methods from Neyman and Pearson, offered a form of rigor and scope that was very respectable indeed, and moreover, had already demonstrably aided agricultural research and production considerably. With new rigor came a new vigor and a new respectability, and experimental psychology rapidly grew into a thriving discipline. (For an interesting description of the growth of psychology during this period by an historian, see Herman, 1995). Unmentioned during this period of optimism was the fact that the Fisherian and the Neyman/Pearson methods are completely at odds; psychologists cheerfully used both without acknowledging the contradictions between them (see Gigerenzer et al., 1989). During this period of the expansion of knowledge and skill these research methods dominated the prestigious research journals, and the research papers that did not employ them rarely got into print. That situation is not much different today.

It was precisely in these circumstances that Brunswik put forward his criticisms of the methodological "ideology" of the day. His main charge was that it was bringing more harm than good to psychology. To say this was poor timing would be a gross understatement. But, of course, Brunswik did what he had to do, namely, give voice to methodological blasphemy and thus strike a blow at what many saw as the very foundation of psychological science. Therefore, his views received exactly the kind of response one might expect in such circumstances; they were ignored (no textbook in statistical methods or experimental design ever mentioned them), and when not ignored, they were harshly denigrated. All of this had an unfortunate consequence; Brunswik committed suicide in 1955 at age 52.

The reader should know that I am not a dispassionate observer of these events. Because I have long been an admirer of Brunswik' s work my account of these developments may be biased. Therefore I feel constrained to explain briefly what my relationship with him was.

  A Personal Note

I first met Brunswik at a distance in 1938, shortly after he arrived in Berkeley from Vienna. Although only an undergraduate at the time, I had heard of the remarkable new professor and so I sat in on a few of his classes to see what a European professor was like. He made an impression on me, despite his less-than-perfect English, and when I returned to Berkeley as a graduate student in 1945 I sought him out. Indeed, I sat at his feet at every opportunity. He was both forbidding and friendly at the same time; forbidding because of his obvious store of knowledge (you thought twice about striking up a conversation with him, or going to his office), friendly because once you did he was unfailingly courteous and then you were glad you did, and thought just a bit better of yourself for having done it. Within moments of talking with him you were aware of the fact that you were in the presence of a true scholar (I was not alone in thinking that he knew everything); yet his demeanor was so pleasant (even to graduate students) that one was disarmed, and fright (but not awe) quickly disappeared. I felt that he took me seriously and listened carefully to what I had to say, despite the naiveté of my remarks. And he had a great deal to say about whatever topic you brought to him. He was not appreciated by undergraduates, however—he was always over their heads—and the chairman of the department, Edward Tolman, had to write a strong letter to the Dean pleading for his reappointment after his first year.

 

I was dubbed the "poor man's Brunswik" by my fellow graduate students.

I became completely convinced by Brunswik's arguments, and even published one article applying his views about experimental design while still a graduate student (Hammond, 1948). And because I was able to present his point of view in simpler, and perhaps more comprehensible, terms than he used, I was dubbed the "poor man's Brunswik" by my fellow graduate students. It may be that I have always occupied that role (see Hammond, 1986; Hammond, 1990), although I would like to think that I have extended his ideas as well (see, for example, Hammond, 1997). I maintained correspondence with him from the time I left Berkeley in 1948 until his death in 1955. After his death, at the request of Edward Tolman I edited a memorial volume The Psychology of Egon Brunswik (Hammond, 1966) and published several books and articles that were intended to further his contributions (see e.g., Hammond, 1954, 1955, 1996b; Hammond, Hamm, Grassia, & Pearson, 1987; Hammond, Stewart, Brehmer, & Steinmann, 1975; Hursch, Hammond, & Hursch, 1964). His general views of psychology have dominated my thinking about psychology since I was a student. I founded a "Brunswik Society" that has held annual meetings for the past 13 years.

Having placed that caveat before the reader, in my next note on this web site I will offer a brief exposition of Brunswik's challenge to the establishment of experimental psychology that eventually appeared posthumously in his 1956 book Perception and the Representative Design of Psychological Experiments.

References Brunswik, E. (1943). Organismic achievement and environmental probability. Psychological Review, 50, 255-272.

Brunswik, E. (1952). The conceptual framework of psychology. In International encyclopedia of unified science (Vol. 1, no. 10, pp. 4-102). Chicago: University of Chicago Press.

Brunswik, E. (1956). Perception and the representative design of psychological experiments (2nd ed.). Berkeley, CA: University of California Press.

Cooksey, R. (1996). Judgment analysis:  Theory, methods, and applications. New York: Academic Press.

Fisher, R. A., Sir. (1925). Statistical methods for research workers. Edinburgh, Scotland: Oliver & Boyd.

Gigerenzer, G., Swijtink, Z., Porter, T., Daston, L., Beatty, J., & Krueger, L. (1989). The empire of chance: How probability changed science and everyday life. New York: Cambridge University Press.

Gillis, J., & Schneider, C. (1966). The historical preconditions of representative design. In K. Hammond (Ed.), The psychology of Egon Brunswik (pp. 204-236). New York: Holt, Rinehart, and Winston.

Hammond, K. R. (1948). Subject and object sampling: A note. Psychological Bulletin, 45, 530-533.

Hammond, K. R. (1954). Representative vs. systematic design in clinical psychology. Psychological Bulletin, 51(2), 150-159.

Hammond, K. R. (1955). Probabilistic functioning and the clinical method. Psychological Review, 62, 255-262.

Hammond, K. R. (Ed.). (1966). The psychology of Egon Brunswik. New York: Holt, Rinehart, and Winston.

Hammond, K. R. (1986). Generalization in operational contexts: What does it mean? Can it be done? IEEE Transactions on Systems, Man, and Cybernetics, 16, 428-433.

Hammond, K. R. (1990). Functionalism and illusionism: Can integration by usefully achieved? In R. M. Hogarth (Ed.), Insights in decision making: A tribute to Hillel J. Einhorn (pp. 227-261). Chicago, IL: University of Chicago Press.

Hammond, K. R. (1996a). Human judgment and social policy: Irreducible uncertainty, inevitable error, unavoidable injustice. New York: Oxford University Press.

Hammond, K. R. (1996b). Upon reflection. Thinking and Reasoning, 2, 239-248.

Hammond, K. R. (1997). Expansion of Egon Brunswik’s psychology. In K. Fischer & F. Stadler (Eds.), Wahrnemung und gegenstandswelt: Zum lebenswerk von Egon Brunswik (1903-1955) (pp. 79-105). Vienna: Springer-Verlag.

Hammond, K. R., Hamm, R. M., Grassia, J., & Pearson, T. (1987). Direct comparison of the efficacy of intuitive and analytical cognition in expert judgment. IEEE Transactions on Systems, Man, and Cybernetics, 17, 753-770.

Hammond, K. R., Stewart, T. R., Brehmer, B., & Steinmann, D. O. (1975). Social judgment theory. In M. F. Kaplan & S. Schwartz (Eds.), Human judgment and decision processes (pp. 271-312). New York: Academic Press.

Herman, E. (1995). The romance of American psychology: Political culture in the age of experts. Berkeley: University of California Press.

Hursch, C. J., Hammond, K. R., & Hursch, J. L. (1964). Some methodological considerations in multiple-cue probability studies. Psychological Review, 71, 42-60.

McNemar, Q. (1949). Psychological statistics. New York: Wiley.

Neyman, J., & Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, 231, 289-337.

Wiseman, D. B., & Levin, I. P. (1996). Comparing risky decision making under conditions of real and hypothetical consequences. Organizational Behavior and Human Decision Processes, 66, 241-250.