Content area
Full Text
The Critical Decision Method (CDM) is an approach to cognitive task analysis. The method involves multiple-pass event retrospection guided by probe questions. The CDM has been used in the elicitation of expert knowledge in diverse domains and for applications including system development and instructional design. The CDM research illustrates the sorts of knowledge representation products that can arise from cognitive task analysis (e.g., Situation Assessment Records, time lines, decision requirements). The research also shows how one can approach methodological issues surrounding cognitive task analysis, including questions about data quality and method reliability, efficiency, and utility. As cognitive task analysis is used more widely in the elicitation, preservation, and dissemination of expert knowledge and is used more widely as the basis for the design of complex cognitive systems, and as projects move into even more field applications and real-world settings, the issues we discuss become increasingly critical.
INTRODUCTION
The purpose of this article is to examine cognitive task analysis from a number of perspectives, including its applications and products. methodological issues involved in its use, and outstanding research needs. Rather than analyze an array of cognitive task analysis methods, our strategy is to explore some methodological issues in the context of research that has relied on a single technique, the Critical Decision Method (CDM), although other methods of cognitive task analysis will be discussed along the way.
First we present the rationale for choosing the CDM as a case study and briefly describe the research that led to the development of the CDM. We then describe the CDM in detail, followed by illustrations of the kinds of products that can stem from cognitive task analysis, with examples taken from a number of projects that have used the CDM. The article centers on discussions of methodological issues of reliability, validity, efficiency, and utility.
BACKGROUND
Since about 1980, many studies have used diverse methods that have been called cognitive task analysis (for reviews, see Cooke, 1994; Hoffman, Shadbolt, Burton, & Klein, 1995; Kim & Courtney, 1988; Klein, 1992; Olson & Reuter, 1987; Woods, 1993). The purposes of the research have likewise been diverse. Cognitive task analysis methods have been used in the study of novice/expert differences to ask fundamental questions in both cognitive science (e.g., Chi, Feltovich,...