Article #15
Newsletter list 


Previous Article 

Next Article 


Design and training problems in dynamic, high-technology and high-consequence environments

Alex Kirlik
Atlanta, Georgia

We continue our studies of design and training problems in dynamic, high-technology and high-consequence environments. Working with the U.S. Naval Training Systems Division, we have investigated the factors limiting judgment performance in target identification tasks, such as those performed by Navy AEGIS operators. We have found that high and low performing participants in such tasks are distinguished primarily by execution consistency, rather than by differences in task knowledge. Our experimental work in this domain has also demonstrated the potential of part-task training and automated feedback technology for augmenting the "over the shoulder" coaching predominantly used in operational training. These studies appear in Kirlik, Fisk, Walker and Rothrock (in press) and in Bisantz, Kirlik, Gay, Phipps, Walker and Fisk (1998). Also in this context, Ann Bisantz ( has completed a dissertation investigating how experience in making judgments in a dynamic, interactive task influences subsequent performance on evidential reasoning tasks within the same context. Also in a naval context, Richard Strauss ( and I are working with the Johns Hopkins Applied Physics Lab for ONR on situation awareness issues in submarine operations. Strauss has performed task analyses and has constructed a dynamic laboratory simulation of the task of coming to periscope depth. We will soon begin experimentation to understand how operators cope with uncertainty in this task environment, in order to develop new concepts for interface design.

We have also been working with NASA Ames on human-automation interaction issues and training problems associated with the planned transition to a "free-flight" environment. The control systems of modern "glass cockpit" aircraft have such complexity that it is often said that pilots frequently ask themselves these questions: "What is it doing?," "Why is it doing that?" and "What is it going to do?" From a Brunswikian perspective it can seem as if many high-technology systems have been structured specifically to defeat the judgment strategies and heuristics evolution adopted for meeting the demands of the natural world.

Based on the recent dissertation by Degani (, we have proposed a modeling technique that can be used to document the complex structure of mode-based control systems in order to identify any interface design features that may contribute to mode confusion and mode error (Degani, Shafto, and Kirlik, in press). It may be interesting to note that this publication in an aviation psychology journal consists solely of an environmental modeling technique.

We have recently begun work with NASA Ames on flight crew training problems likely to arise as a result of the free-flight concept currently envisioned for U.S. aviation operations. Free-flight is a new traffic control regime which would give individual flight crews much more control over their flight trajectories than is currently allowed. In the current centralized control system, air traffic control (ATC) directs aircraft along a limited set of trajectories or "highways in the sky," and any requested deviation from the planned trajectory (e.g., to fly a more direct path or a more fuel-efficient altitude) must be first approved by ATC. One interesting aspect of free-flight is that individual flight crews will be given the opportunity to detect and resolve potential conflicts without ATC intervention, using cockpit displays of traffic information and a set of negotiation rules (e.g., the aircraft on the right has the right of way). Currently we are examining how judgment analysis might provide resources for investigating training issues arising from free-flight.

Finally, Ann Bisantz and I have written a chapter providing an overview of how cognitive engineering has contributed to our understanding of cognition by examining experiential and environmental aspects of adaptation.


Bisantz, A.M., Kirlik, A., Gay, P., Phipps, D.A., Walker, N., and Fisk, A.D. (1998). Modeling and analysis of a dynamic judgment task using a lens model approach. Ms. currently in review to IEEE Systems, Man, and Cybernetics.

Bisantz, A.M. (1998). Modeling environmental uncertainty to understand and support dynamic decision making. Unpublished doctoral dissertation, School of Industrial & Systems Engineering, Georgia Institute of Technology.

Degani, A., Shafto, M., and Kirlik, A. (in press). Modes in human-machine systems: review, classification, and application. International Journal of Aviation Psychology.

Degani, A. (1997). Modeling human-machine systems: On modes, error, and patterns of interaction. Unpublished doctoral dissertation, School of Industrial & Systems Engineering, Georgia Institute of Technology.

Kirlik, A., Fisk, A.D., Walker, N. and Rothrock, L. (in press). Feedback augmentation and part-task practice in training dynamic decision making skills. In J. Cannon-Bowers and E. Salas (Eds.), Decision Making in Complex Environments. APA Press.

Kirlik, A. and Bisantz, A. (in press). Cognition in human-machine systems: Experiential and environmental aspects of adaptation. In P.A. Hancock (Ed.), Handbook of Perception & Cognition Vol. 17: Human Performance & Ergonomics. Academic Press.

Contact Alex Kirlik

Previous Article                Next Article

Newsletter list