Common Enlisted Soldiers' Tasks: A "Muddy Boots" Data Base for Revising Multiple Army Skill-Level Courses

Jimmy L. Mitchell, Bob Gosc, Institute for Job & Occupational Analysis
Johnny J. Weissmuller, Donna Tartell, Metrica, Inc. San Antonio, TX
Darrell Worstine, Bill Badey, Army Research Institute, Alexandria, VA

A fifteen-month study was conducted to identify those tasks performed by United States Army enlisted soldiers that are common, regardless of the Military Occupational Specialty (MOS). This study was conducted for the U.S. Army Sergeant Majors Academy (USASMA), Ft. Bliss, TX, which is responsible for all common task training, from basic training through the Sergeant Majors’ course. During this study, a single Master Task List was constructed to include common soldier tasks that were identified for skill levels one (1) through four (4) and Battle Staff, First Sergeant, Master Sergeant Staff and Sergeant Major positions. New technologies were required to screen tasks by skill-level so that respondents were only shown tasks appropriate to their skill-level. In addition, there was a need to study task migration – were tasks taught at a given skill level being performed by groups earlier in their careers? To answer this issue, each respondent was shown tasks from the adjacent skill level, and a number of tasks were identified where the training requirement was at a lower skill level. Overall, the study was successful in capturing the "Muddy Boots" field perspective on what tasks are performed and what should be trained at each level, and the data were used in realigning the Army Common Task List. To accomplish this project, several new technologies were introduced into the computer-based survey system and were successfully implemented.

Background

In a recent Army study of U.S. Army Non-Commissioned Officer Education System (NCOES) courses, it appeared that mandated manpower and skill reductions had created situations whereby soldiers were being placed into jobs previously held by soldiers of higher rank. Consequently, these soldiers were becoming skilled in higher level tasks much sooner than soldiers of equal rank had in the past. To examine these issues, the Army Research Institute (ARI) contracted with the Human Resources Research Organization (HumRRO) and the Institute for Job & Occupational Analysis to conduct a Enlisted Common Soldiers Task (ECST) survey.

Managers at Headquarters Army Training and Doctrine Command (TRADOC) and USASMA recognized the need for a study that would ultimately lead to required revisions of NCOES courses; i.e., to bring course curricula in line with soldier’s current needs for knowledge and proficiency and ensure training preceded performance. These managers also expressed the need to solicit input from "muddy boots" soldiers; i.e., those soldiers who were actually "living" in the environment where common tasks are frequently performed. This approach was deemed imperative to learning the true performance requirements, as opposed to relying on opinions from personnel without recent field experience (such as Bosnia, etc.).

Thus, the objective of the present study was: "To validate and realign an Enlisted Common Soldier Task (ECST) List, to facilitate the development of a life-cycle training model to support the Army’s Future Leader Development."

Approach

A plan was developed to provide a comprehensive map of changed training requirements. Project timelines were as follow:

Initial Planning Meeting - September 1997

Final task list - February 1998 (Includes Digital Tasks for Battle Staff NCOs)

Begin Field Administration - 30 April 1998

Preliminary Data Needed - July 1998 (Selection of Tasks for TE Ratings)

Final Data Analysis - October 1998

School Task Selection Board - Jan 1999

The study was actually initiated in September 1997 and was completed in December 1998. The data were used in the Critical Task Selection board which met at the USASMA in January 1999. A revised Critical Task list was recently approved and implemented by Headquarters TRADOC

Implementation of this plan resulted in specific project details and objectives:

Administer a 900-item Enlisted Common Soldier Task (ECST) list to 19,600 active, guard & reserve soldiers worldwide

Selectively display only appropriate tasks to each soldier, based on skill-level, position, or pay grade

Display additional tasks from plus and minus one skill level to assess task migration

Administer topic areas in rated importance order

Capture write-in comments (missing tasks, etc.)

Development of the ECST List

A preliminary set of task lists provided by the USASMA was used as a starting point for constructing a new Enlisted Common Soldiers Task List. Various agencies involved in the project submitted suggestions for potential new tasks, particularly digital tasks performed by the Battle Staff NCO and others. In addition, several field visits were conducted (Ft. Hood, Ft. Sam Houston, Ft. Bragg, and with current students at USASMA at Ft. Bliss) to have individuals with current or recent field experience review the list. Active Component, Reserve, and National Guard personnel were included in the review process. Each group was asked to review the task list, identify groupings or topic (knowledge) areas under which tasks might be organized, suggest additional tasks where needed (new equipment, procedures, etc.), and to indicate the appropriate skill-level where each task is first performed (and thus, where it should be trained). The groups also polished the language to ensure that field incumbents would easily comprehend the tasks.

As a result of these field visits, it was concluded that a field survey was indeed needed to determine the present skill and performance levels of the tasks. Recent reduction in Army forces has apparently greatly impacted performance of the tasks. It was clear that lower skill level individuals were reported to be performing tasks previously performed by higher skill-level NCOs. Field personnel also indicated that the opposite was also occurring (some lower level tasks performed at a higher skill-level), and that Platoon Sergeants were performing 1SG tasks.

Development of the Computer-Based Surveys

The disk-based, computer-assisted ECST Survey: incorporated approved task list, showed tasks appropriate to each skill level along with tasks from 1 skill level up and 1 down (except SL1 / 5); and included special logic for additional tasks related to First Sergeant (1SG and SL4) and Battle Staff NCO positions.

Respondents rated the topic areas on the relative importance to their present job, and rated tasks in terms of the frequency of task performance. Separate Training Emphasis (TE) ratings for the tasks were collected from a separate sample of field supervisors (next higher skill-level) in terms of where the task should be trained and the relative emphasis among the tasks (1 – 5 scale). Thus a separate skill-level TE was collected for each skill-level and specialized job (1^st Sgt., Battle Staff NCO) rather than relying on a generalized rank ordering of tasks (as is more typically done). This was innovative and very appropriate since the objective was to review common training needs for each level throughout an enlisted career and corresponding SL courses.

Field Administration

Surveys were mailed to selected Army units worldwide at the end of April 1998, with instructions to administer to a specified number of personnel in each skill-level.. Units were asked to complete the survey within a month, and return the disks. Some units were delayed, particularly in the National Guard and Reserve, because of deployments, exercises, or limited opportunity (non-active components typically meet once-a-month, and some individuals skipped some months). The second wave of surveys (TE) was mailed in July with a request for similar speedy return. Because the TE disks were specifically targeted to given rater levels, the disks were color coordinated to distinguish each target rater skill-level.

A problem was encountered with running the survey on some PCs. A runtime error was encountered when individuals attempted to use Pentium II 266 megahertz or faster computers. This stemmed from a timing calibration error in an older compiler used to develop the survey software (OASurv). A workaround was developed and distributed to those organizations where survey problems were reported.

Survey administration was completed in early September, and data uploaded to computer data files for auditing and review. Returns were as follow: Of the 19,600 surveys distributed across all components, 7,689 disks were returned by September 15^th, of which 6,682 were usable cases. Most of the unusable disks were disks that were returned unused – typically from the survey focal point who could not distribute (according to the sampling plan) all the disks he or she had received. Other unusable cases resulted from disks damaged in the mail, virus checker alert, corrupted disks (due to exposure to magnetic field, etc.), or physical damage to the disk (total of 4.6%). In addition, some individuals apparently removed the disk from their computer while the file was being saved. Less than six tenths of one percent of the disks had viruses (some respondents took the survey on home computers; only one virus was detected from a government machine). For all but one of these, the virus was identified, deactivated, and the data file recovered.

Data Analysis

The percent returned of distributed ECST surveys (by component) was as follows: Active 67.27%, Guard 23.71%, and Reserves 26.76%. These were considered very acceptable rates of return, given the different work schedule and situation of the non-active components. There were sufficient numbers participating to adequately represent the relative frequency of performance of tasks, and enough supervisors to have stable averages for the TE ratings.

Recall that one goal of the study was to limit the number of tasks each survey respondent had to be deal with based on skill-level. Figure 1 illustrates this difference.

Figure 1. – Number of Tasks Per Skill Level (SL)

The difference in the SME estimates for each SL and the number shown in each survey represents additional tasks shown to assess task migration (+/- one SL). The final column for each SL is the number of tasks selected to show the final task selection board which includes tasks from the survey with sufficient frequency of performance that they should be considered for training at the target skill level. For example, for SL 1, about 150 tasks were identified by SMEs as SL1 tasks (taught in basic training). Field incumbents saw about 250 tasks (+SL2).

To prepare for the Critical Task Selection Board, a number of products were prepared to summarize and display the data for each skill-level, using Microsoft Access (data input from atCODAP runs), organized by topic area. A typical report for one topic area is shown below:

Figure 2. Skill-Level 3 Summary Report

One purpose for reviewing such data was to determine if there were different TE policies (priorities) for each component or could a single composite TE average per task be used. While there are some differences in frequency of performance, the general trend is for substantial agreement among components as to the TE ratings, thus a composite average rating was considered to be meaningful.

Since this much data is difficult for a selection board to use, the USASMA suggested other reports which would focus on a single task at a time (thus, potentially 900 reports). These reports were generated, and an example of one task, 415 "Screen subordinate soldiers’ Leave and Earning Statements (LES) for potential pay problems" is displayed below.

This is typically a 1^st Sgt. or SL3 task; note that it is not done at all by SL1 (and was excluded from their survey). However, note that over half (57%) of SL2 personnel report performing the task, which makes it a good candidate for training at an early career point. In addition, 73 of the TE raters feel it should be trained at SL2 and give it an above average (3.33 on a 5-point scale) TE rating. The final Task Selection Board, composed of a number of Command Sergeants Major and the USASMA staff, concurred, and officially made this a SL2 task.

In similar actions, the Board migrated about 20 tasks to lower skill-levels during their review, although in some cases they had very good reasons to keep tasks at the present skill-levels. Using reports such as these, the Board was able to move swiftly through the complex data collected in this study, and the entire Board process was completed quickly and efficiently.

Figure 3. Task 415 Summary Report

Subsequent to the January 1999 meeting, the Board results were forwarded to Headquarters TRADOC, where it was approved and implemented. It now becomes the basis for redevelopment of field manuals, training courses, and on-the-job training programs for each SL.
Other Survey Feedback
The ECST survey also included questions of interest to the USASMA staff and Army researchers in terms of the availability of computers, CD-ROM, Internet access, and a number of other issues. Some questions about the survey itself were also included. A summary of these "extra" questions is shown in Figure 4 below:
Easy to schedule computer for survey (86%)
Instructions easy to understand (70%)
Time to complete reasonable (53%)
Prefer computer to paper-and-pencil (61%)
Took 60 minutes or less to complete (51%)

60 – 90 minutes (29%)
90 – 120 minutes (11%)
120 – 150 minutes (6%)

Figure 4. ECST Survey Respondent Feedback

Overall, soldiers taking this survey had little trouble finding a computer to use (even the 400 in Bosnia); found the instructions easy to understand, and felt the time to complete was reasonable. About 50% completed the survey in one hour or less and 80% in an hour and a half or less. Only six percent took more than 2 hours. Unfortunately these figures were not analyzed by skill level to see how time to take the survey correlated with skill-level, which was the major variable in how many tasks they had to deal with ranging from about 250 to over 700 (see Figure 2. above).
Write In Comments
Soldiers participating in this survey were afforded the opportunity to write in any task they felt had been omitted, and any comments about survey content or process. About 1 of every 7 soldiers provided some write in comments. The automated format made it very easy for respondents to provide comments. Their comments were considered anonymously but were tracked by grade, military occupational specialty (MOS), and unit to facilitate interpretation. Some new tasks were identified, mostly involving computer use, Team/Squad Leader activities, map reading, etc. Narrative comments ranged from very constructive hopes for how the data would be used to very critical comments about Army life and leadership in general. Given the time limits of our analysis phase, no detailed thematic analysis was feasible.

Conclusions

This project was a substantial success in that a new Common Task List was developed, used to survey, and the resulting data analyzed for use by a Critical Task Selection Board. The data were used as one factor – a major one – in finalizing the new ECST list for each skill level. Several new technologies for survey administration, presentation order, and screening by skill-level were developed and successfully implemented.

Next paper