Sandra K. Wetzel-Smith, John A. Ellis, Angelique Reynolds, and Wallace H. Wulfeck
Navy Personnel Research and Development Center
San Diego, California 92152-7250

The challenges facing Antisubmarine Warfare (ASW) training in the closing years of this decade are greater than at any time since the early days of World War II. Conditions since the end of the Cold War, and those expected throughout the next decade, impose additional complexity on maintaining our ASW superiority. Russian nuclear submarine technology continues to improve and advanced submarines continue to be built and delivered to their fleet. Concurrently, the proliferation of improved diesel submarine technology to many Third World nations requires that our ASW forces also be capable of conducting operations in the vastly different littoral regions.

The training challenge is two-fold: (1) retaining the capability to detect and prosecute nuclear submarines; and (2) expanding our current capability against diesel submarines of the Third World. When coupled with dramatic reductions in ASW training resources, including at-sea training, this historic change compels the development of training for skills learned previously on -the-job and for skills required in new environments for both sensor operators and tacticians.

The Interactive Multisensor Analysis Training (IMAT) system was developed to address post Cold War ASW training requirements. Specifically, IMAT is designed to teach the complex conceptual knowledge and cognitive and procedural skills required to reason about the interrelationships among the operating modes of target submarines, the environmental variables that affect sound transmission, and the sensor systems used for detection and localization. In addition, IMAT provides extensive training on submarine acoustics and interpretation of sensor system data. The IMAT approach to training is based on recent advances in cognitive psychology and instructional technology. It combines cognitive analytic and curriculum design technology with advanced computer-based graphics and programming technology to present state-of-the-art training.

The IMAT system integrates several areas of research on cognition and instruction, including, graphical techniques to promote visualization of invisible phenomena in science teaching, elaborated explanations, contextualized or anchored instruction, and instructional sequencing. The following sections briefly summarize portions of this work.

Scientific visualization has traditionally been used by scientists to explore phenomena and to communicate with other scientists (Bryson, 1994). When used for presentations, researchers select data sets, transform them and then turn them over to specialized graphic artists to develop images and animation. However, the end products of this process have not been designed for laymen or students. IMAT aims to bring this technology into specialized technical training.

Research support for scientific visualization as a training strategy comes from the literature on instructional media. Both static and dynamic graphic displays have been shown to facilitate teaching of scientific concepts (Wetzel, Radtke, & Stern, 1994): Levie and Lentz (1982) in a meta-analysis of illustrated text studies concluded that learning and retention is facilitated by illustrations, if the illustrations are directly related to the text. Park and Gittelman (1992) found that subjects trained with dynamic graphics performed better on electronic troubleshooting problems that those trained with static displays. White (1984) used animated computer graphics to successfully teach the basic principles of Newtonian laws of motion and force. IMAT employs a computer based graphical interface to conceptual models of real world phenomena to deliver both static and dynamic graphics in a traditional classroom environment.

Providing students with elaborated explanations, analogies, etc. about how and why systems, events, and phenomena are structured and function has been shown to facilitate learning and retention. Research on learning skills and learning from text has shown that elaborated explanations enhance the students' mental models and increase retention (Konoske & Ellis, 1991). In a series of studies of learning from scientific text, Mayer (1989) found that providing students with a conceptual model increased learning, retention and transfer. The conceptual models in his instruction used both text and diagrams to highlight major objects and actions and the causal relations among them. That is, the models focused on how and why systems work. Smith and Goodman (1982) studied the effects of providing elaborated instructions on learning and performing a procedural assembly task and found that instructions containing functional information resulted in fewer errors. Swezey, Perez, and Allen (1991), in a study on transfer of electromechanical troubleshooting skill, found that some level of generic structure and functional knowledge is required for cross domain transfer. The IMAT system uses elaborated explanations throughout the instruction to (1) clarify complex relationships such as those among water temperature, pressure and depth, and salinity, (2) provide comprehensive feedback for practice exercises and (3) describe graphically displayed examples.

Contextualized or job oriented instruction has been found to be more effective in learning, retention, and performance than topic oriented instruction (Semb & Ellis, 1994; Cognition & Technology Group at Vanderbilt, 1990). Further, within a job context, mental model development is facilitated by teaching students to reason about events and phenomena that involve several interrelated variables. Proper sequencing may play an important role in cognitive skill development. While early research on sequencing showed that with simplified or isolated tasks, different sequences of instructional events made little difference, more recent research and theory suggests that for complex tasks, sequencing strategies may have significant effects. For example, Reigeluth and Stein (1983) argue that beginning instruction with a condensed "holistic" overview of a task domain leads to better learning than more traditional sequences which teach isolated topics first and integrate them later. More recently, extreme "constructivist" approaches to instruction (e.g. Duffy & Jonassen, 1991) argue that learners should "sink-or-swim" in a fully elaborated domain. Merrill, Li, & Jones (1990) also argue for a holistic approach to teaching complex domains, but include moderate structure and sequencing recommendations in their approach. Drawing from Reigeluth and Stein (1983), IMAT begins with a simplified overview of target, environment, and sensor system relationships in the context of the jobs and tasks performed by operators and tacticians. This context is revisited throughout IMAT to reinforce the reality that students are learning to do a job not memorize a list of topically related facts.

This paper describes (1) an evaluation of the application of the IMAT system in the Sonar Technician Surface (STG) 'A' school with respect to performance on factual, comprehension, and cognitive skill test items, student motivation, and instructional design; (2) a comparison of student performance on STG 'A' IMAT lessons with student performance on comparable Aviation Systems Warfare Operator (AW) 'A' school IMAT lessons; (3) an evaluation of the application of the IMAT system in the Fleet Aviation Specialized Operations Training Group Pacific Tactical Training Course (TTC) at Barbers Point; and (4) a comparison of graduates of the IMAT TTC with qualified fleet tactical officers and pilots on tests of knowledge of acoustical oceanography and tactical problem solving.

The STG 'A' IMAT application was evaluated on (1) student performance on three types of end of unit test items (fact items, comprehension items, and cognitive skill items), (2) student motivation, and (3) quality of instructional design. Seventy one STG students trained with the IMAT system were compared with 90 STG students trained with the standard instruction used in the 'A' school prior to the introduction of IMAT, and with 46 students from the AW 'A' who had also been trained with the IMAT system.

For the TTC evaluation 59 students were administered a paper and pencil test that included both knowledge items and a scenario based cognitive skills items. They were tested upon entering the school and after completing the course in a standard pre-post design. The scores of the TTC graduates were also compared with the performance of 21 qualified fleet tacticians and pilots on the same testing instrument.

The STG IMAT students scored higher on all item types than the STG Standard Instruction students, and AW IMAT students scored higher on all item types than the other two groups. Students in both the IMAT groups scored relatively better on Cognitive Skill and Comprehension items compared to Fact items than Standard Instruction students. Table 1 presents the means for these findings.

The instructional design analysis showed that instructional quality was superior in the STG IMAT lessons. Unfortunately, the motivation data for the STG Standard Instruction students was either lost or not collected. However, motivation for the STG IMAT students was high and compared favorably to previous motivation data for AW IMAT students. The instructional design analysis for the AW IMAT instruction found that this instruction was higher in overall quality than the materials for the other two groups.

The findings for the TTC course evaluation showed that TTC students had significant gains from pre test to post test for knowledge and scenario test items. Further, in the comparison with qualified fleet personnel, TTC graduates scored higher than the qualified fleet personnel on both scenario and knowledge items. Table 2 presents the means for these findings.

Several conclusions can be drawn from this study. First, the IMAT approach to training, which is based on computer-generated dynamic displays and instructional design and delivery strategies designed to enhance cognition, produce substantial gains in performance. This is especially true for the explicit IMAT goal of teaching complex knowledge and cognitive skills. Second, the IMAT system emphasis on adhering to the principles of high quality instructional design was supported in the instructional design analyses. The differences in design quality among the STG Standard Instruction, STG IMAT and AW IMAT groups undoubtedly contributed to the observed performance differences. Finally, the TTC findings show that IMAT can bring course graduates up to the level of fleet qualified personnel on both fact and scenario tests. Overall, the results of the STG and TTC implementations show that IMAT is a highly effective training system that offers a viable solution for many of the training requirements and challenges faced in the post cold war world.

Cognition & Technology Group at Vanderbilt (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19(6), 2-10.

Duffy, T.M., & Jonassen, D. H. (1991). New implications for instructional technology. Educational Technology, 31(5), 7-12.

Levie, W. H., & Lentz, R. (1982). Effects of text illustrations: A review of research. Educational Communication and Technology Journal, 30, 195-232.

Konoske, P. J., & Ellis, J. A. (1991). Cognitive factors in learning and retention of procedural tasks. In R. F. Dillon, & J. W. Pellegrino (Eds.), Instruction: Theoretical and Applied Perspectives. New York: Praeger.

Merrill, M.D., Li, Z., & Hones, M. K. (1990) Second generation instructional design. Educational Technology, 30(2), 7-14.

Park, 0., & Gittelman, S.S. (1992). Selective use of animation and feedback in computer-based instruction. Educational Technology Research and Development,40, 27-38.

Reigeluth, C.M., & Stein, F. S. The elaboration theory of instruction. In C.M. Reigeluth (Ed.), Instructional design theories and models. Hillsdale, NJ: Erlbaum.

Semb, G. B., & Ellis, J. A. (1994). Knowledge taught in school: What is remembered? Review of Educational Research, 64, 253-286.

Smith, E., & Goodman, L. (1982). Understanding instructions: The role of explanatory material. Cambridge, MA: Bolt Beranek, & Neuman.

Swezey, R. W., Perez, R. S., & Allen, J. A. (1991). Effects of instructional strategy and motion presentation conditions on the acquisition and transfer of electro- mechanical troubleshooting skill. Human Factors, 33, 309-323.

Wetzel, C. D., Radtke, P. H., & Stern, H. W. (1994) Instructional effectiveness of video media. Hillsdale, NJ: Erlbaum.

White, B. (1984). Designing computer games to help physics students understand Newton's laws of motion. Cognition and Instruction, 1, 69-108.