The U.S. Air Force Research Laboratory has been developing cost effective methods for gathering occupational and training requirements information. This information has most often been collected at an individual level of analysis focusing on the more behavioral aspects of work. Recent interest in both team and cognitive requirements for work has prompted renewed interest in team task analysis and accurately representing knowledge/ cognitive components of work. The U.S. and Allied Military Services have pioneered the development of exemplar methods that serve as the foundation for recent advances. This paper highlights recent explorations and advances in team task analysis and cognitive task analysis methods. Implications for increasing the accuracy and efficiency of the requirements analysis process will also be discussed.
Cascio (1995) noted an organizational change away from task-based work to more of a process. Jobs are no longer defined by a limited number of tasks, requiring the employee to accomplish a range of tasks that may change over time. As a higher proportion of jobs are focusing on troubleshooting activities, cognitive task analysis may be more appropriate for identifying strategies involved in effective performance.
Another trend involves workers participating as a member of a team which requires sharing information, collaboration, and communication to produce a group outcome. Individuals may play a large or small role in the team process as well as individual responsibilities. Additionally, individuals may also serve on more than one team. Researchers must consider these issues when trying to describe the tasks of team members.
In addition to these trends, some researchers have made a move towards examination of competencies, which may differ for individuals and teams (Cannon-Bowers, Tannenbaum, Salas, & Volpe, 1995). Although competencies share some similarities with traditional knowledge, skills, and abilities (KSAs), some distinguishing characteristics exist between the two frameworks. For example, competencies include knowledge, skills, and attitudes, rather than traditional knowledge, skills, and abilities. Though the knowledge component is similar, competencies include a broader concept of skills than KSAs, by incorporating both a psychomotor and a cognitive component. The concept of abilities is replaced by attitudes in the competencies framework, which are predicted to impact performance (Cannon-Bowers et al., 1995).
Team competencies can be divided into team-specific and task-specific
competencies. Team-specific competencies may only apply to a particular
team, yet encompass all tasks the team performs. Task-specific competencies
may only apply to certain tasks. Cannon-Bowers et al. (1995) revealed 11
knowledge requirements, 8 specific skill dimensions, and 9 attitude requirements
for a team. Stevens and Campion's (1994) research appears to corroborate
these findings.
Klein (1993) stated, "Cognitive Task Analysis is directed at the psychological processes underlying the performance… and the subtle cues that may depend on context and experience" (p. 88). In general, the function of cognitive task analysis is to define the actual decision requirements of the task (Klein, 1993). Regardless of the method used, cognitive task analysis should include the following steps: (1) mapping out the task using task analysis; (2) identifying the critical decision points; (3) clustering and linking the decision points; (4) prioritizing the decision points; and (5) diagnosing and characterizing the decisions as to the strategies used, cues signaling the decision points, and the inferences made regarding cues and decision points.
There are some key differences between task analysis and cognitive task analysis (Klein, 1993). In general, task analysis focuses only on observable behavior and does not offer information on overall organization of knowledge. Cognitive task analysis, on the other hand, is directed at the psychological processes underlying the behavior. Cognitive task analysis concentrates on the critical decisions and cognitive processes that separate the expert from the novice. Further, cognitive task analysis provides a description of the overall knowledge organization by looking at the interrelationship between concepts associated with the job. Another benefit of cognitive task analysis is the identification of changes in knowledge structures and mental processes when progressing from novice to expert. Redding (1989) indicated that the following components are essential to cognitive task analysis: assessing individual abilities, assessing changes in knowledge base, identifying task components, identifying differences between novices and experts, identifying the conceptual and procedural knowledge of similar components, and specifying the conditions which best facilitate progression from one knowledge state to another.
Klein (1993) identified four broad classes of cognitive task analysis, including questionnaires and interviews, controlled observation, critical incidents, and analytical methods. Although commonalities can be found among cognitive task analysis methodologies, they all vary with respect to how they elicit expert knowledge, represent expert knowledge, and use the tasks in question to bring about expert performance. The first method, questionnaires and interviews, are standard task analysis techniques that can also be used to probe the cognitive processes underlying observable performance. These techniques can be used to identify cues, goals, options, reasons of choice, knowledge, and help needed to perform the task. These techniques are very popular, easy to implement, and can be used with other cognitive task analysis methods to produce a variety of information. However, these techniques may generate general or idealized information regarding performance rather than cutting through details and contextual nuances (Klein, 1993).
Controlled observation, the second method of cognitive task analysis, uses verbal protocol analysis when experts are instructed to think out loud while performing the task or instructed to provide a retrospective account after completing the task. Another approach to controlled observation is withholding information from the experts while they perform the task to see how they respond to the lack of information and what kinds of questions they might ask. An advantage of controlled observation is that key features of the task can be controlled and manipulated. Additionally, this technique allows for data collection by a computer in addition to an observer. However, this technique may not uncover new factors related to effective performance, since the researchers have prepared the scenarios ahead of time (Klein, 1993; Nisbett & Wilson, 1977). Thus, this method may be more useful in confirming hypotheses rather than generating new ideas.
Critical incidents is the third method of cognitive task analysis, which focuses on the nonroutine aspects of a job or task (Klein, 1993). The first critical incidents approach is the conflict resolution method. This approach entails the expert recounting critical incidents, and the analyst asking the expert to hypothesize what would happen at different points in the incident if things would have occurred differently. In the second approach, critical decision, the expert and analyst recount events in four cycles. It is important to note that during the fourth cycle the experts compare their performance to that of novices. The advantage to comparing knowledge structures includes challenging the expert to come up with exceptions and special cases of the specified incident. However, this method may not be appropriate for procedural tasks because the experts have trouble remembering critical incidents for these types of tasks.
The last method of cognitive task analysis, analytical methods (otherwise
know as psychological scaling), typically involves obtaining judgments
about concepts, converting the data into pair-wise comparisons, and then
deriving a concept structure through multivariate statistical procedures
(Redding, 1989). The analytical methods are beneficial in eliminating the
ambiguities inherent in asking people to describe their own thought processes,
as they do not rely on introspection (Klein, 1993). The limitation of the
analytical methods is their tedious and expensive nature. Because of these
limitations, analytical methods are met with the greatest resistance compared
to other approaches to cognitive task analysis.
Much of the research to date has applied individual task analysis to teams. Examples have included using critical incidents (Morgan, Glickman, Woodard, Blaiwes, & Salas, 1986; Prince & Salas, 1993) and task analysis rating scales (Stout, Salas, & Carson, 1992). Specifically, Morgan et al. (1986) used critical incident technique to identify the critical behaviors appearing more frequently in effective teams such as communication, cooperation, team spirit, giving suggestions, accepting suggestions, coordination, and adaptability. Stout et al. (1992) had raters view videotapes and rate several behavioral dimensions on a scale ranging from high to low. These ratings were then compared to objective criteria such as the number of targets destroyed during the task. The main criticism of applying task analysis methods to a team is that critical teamwork behaviors such as interdependence, coordination, and cooperation are not measured (Bowers, Baker, & Salas, 1994). However, task analysis techniques are acceptable within the contexts of teams if the task analysis for each individual establishes a link across the job tasks (Dieterly, 1988). Dieterly (1988) suggested that task analyses can be integrated with respect to team goals and not simply to a set of individual tasks that may influence team performance.
There has been a paucity of research on applying task analysis to teams. Researchers have based the measurement of team characteristics on the assessment of individual members’ perceptions of task-important indices. Campion, Medsker, & Higgs (1993) derived a questionnaire by identifying clusters of common group characteristics, which was completed by team members and their managers. Simulators have also been used to examine team processes and performance (e.g., Weaver, Bowers, Salas, & Cannon-Bowers, 1995). These simulations allow for observable outcomes as well as subjective observations of team performance.
The concepts of teamwork and taskwork have been developed in an attempt to differentiate individual and team tasks (Morgan et al., 1986). Taskwork consists of individuals performing individual tasks, whereas teamwork consists of individuals interacting or coordinating tasks that are important to the team’s goals (Baker & Salas, 1996). Bowers et al. (1994) compiled a team task inventory by identifying teamwork behaviors, which were reviewed and modified by Subject-Matter Experts (SMEs). Respondents were then asked to rate each task (behavior) on importance to train, task criticality, task frequency, task difficulty, difficulty to train, and overall task importance. Therefore, the teamwork distinction is important so that future research may include team behaviors such as interaction, coordination, relationships, cooperation, and communication (McIntyre & Salas, 1995). Dieterly (1988) identified eight dimensions along which tasks can be decomposed. These dimensions were grouped under tasks that are not dependent upon a team concept and task characteristics that specifically apply to a team context.
Researchers have several issues to consider such as how to identify
team tasks, measure team-level concepts, and integrate teamwork behaviors
into the traditional task analysis methods. Until these issues have been
addressed, researchers may not have a valid system to generate team tasks
because task analysis methods may not capture teamwork behaviors. This
has been demonstrated by Bowers et al. (1994), who found a large proportion
of unexplained variance when applying task analysis to a team.
For example, using cognitive task analysis in conjunction with task analysis reduces costs and labor. These methods complement each other by providing distinct information (Koubek, Salvendy, & Noland, 1994). Finally, the performance of teams is generally being measured by aggregating individual responses rather than capturing team-level constructs. Researchers need to develop team task analysis by identifying taskwork and teamwork behaviors.
Baker, D. P., Salas, E., & Cannon-Bowers, J. A. (1998). Team task analysis: Lost but hopefully not forgotten. The Industrial and Organizational Psychologist, 35 (3), 79-83.
Bowers, C., Baker, D. & Salas, E. (1994). Measuring the importance of teamwork: The reliability and validity of job/task analysis indices for team training design. Military Psychology, 6 (4), 205-214.
Campion, M. A., Medsker, G., & Higgs, C. (1993). Relations between work group characteristics and effectiveness: Implications for designing effective work groups. Personnel Psychology, 46, 823-847.
Cannon-Bowers, J. A., Tannenbaum, S. I., Salas, E., & Volpe, C. E. (1995). Defining competencies and establishing team training requirements. In R. A. Guzzo & E. Salas (Eds.), Team effectiveness and decision making in organizations (pp. 333-381). San Francisco: Jossey-Bass.
Cascio, W. F. (1995). Applied psychology in human resource management (5th ed.). Upper Saddle River: Prentice Hall.
Dieterly, D. L. (1988). Team performance requirements. In S. Gael (Ed.), The job analysis handbook for business, industry, and government (pp. 766-777). New York: Wiley.
Dipboye, R. L., Smith, C. S., & Howell, W. C. (1994). Understanding industrial and organizational psychology: An integrated approach. Orlando, FL: Harcourt Brace & Company.
Klein, G. A. (1993). Naturalistic decision making: Implications for design. Crew System Ergonomics Information Analysis Center.
Koubek, R. J., Salvendy, G., & Noland, S. (1994). The use of protocol analysis for determining ability requirements for personnel selection on a computer-based task. Ergonomics, 37, 1787-1800.
McIntyre, R. M., & Salas, E. (1995). Measuring and managing for team performance: Emerging principles from complex environments. In R. A. Guzzo & E. Salas (Eds.), Team effectiveness and decision making in organizations (pp. 9-45). San Francisco: Jossey-Bass.
Morgan, B. B., Glickman, A. S., Woodard, E. A., Blaiwes, A. S., & Salas, E. (1986). Measurement of team behavior in a Navy training environment (Tech. Report TR-86-014). Orlando, Fl: Naval Training Systems Center, Human Factors Division.
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we know: Verbal reports on mental processes. Psychological Review, 84, 231-259.
Prince, C., & Salas, E. (1993). Training for teamwork in the military aircrew: Military CRM programs and research. In E. Weiner, B. Kanki, & R. Helmreich (Eds.), Cockpit resource management. New York: Academic.
Redding, R. E. (1989). Perspectives on cognitive task-analysis: The state of the state of the art. In 33rd Annual Proceedings of the Human Factors Society, (pp. 1348-1352). Santa Monica, CA: Human Factors Society.
Roth, E. M., Woods, D. D., & Pople, H., E. (1992). Cognitive simulation as a tool for cognitive task analysis. Ergonomics, 35 (10), 1163-1198.
Ryder, J. M., & Redding, R. E. (1993). Integrating cognitive task analysis into instructional systems development. Educational Technology Research and Development, 41, 75-96.
Ryder, J. M., Redding, R. E., & Beckshi, P. F. (1987). Training developments for complex cognitive tasks. Proceedings of the Human Factors Society 31st Annual Meeting (pp.1261-1265). Santa Monica, CA: Human Factors Society.
Stevens, M. J., & Campion, M. A. (1994). The knowledge, skills, and ability requirements for teamwork: Implications for human resource management. Journal of Management, 20 (2), 503-530.
Stout, R., Salas, E., & Carson, R. (1994). Individual task proficiency and team process behavior: What’s important for team functioning? Military Psychology, 6 (3), 177-192.
Tesluk, P., Mathieu, J. E., Zaccaro, S. J., & Marks, M. (1997). Task and aggregation issues in the analysis and assessment of team performance. In M. T. Brannick, C. Prince, & E. Salas (Eds.), Team performance assessment and measurement (pp. 197-224). Mahweh, NJ: Lawrence Erlbaum.
Weaver, J., Bowers, C., Salas, E., & Cannon-Bowers, J. (1995). Networked simulations: New paradigms for team performance research. Behavior Research Methods, Inst., & Computers, 21 (1), 12-24.
back to table of contents