Introduction
The Air Force Research Laboratory is a recognized leader in the development and evaluation of advanced technologies for education and training. As such, the Laboratory was requested to help support the development and evaluation of training technologies and content to support the U.S. Air Force Surgeon General’s Mirror Force Training Initiative. The initial effort is aimed at providing improved training to Reserve and Guard Medical Technicians
The goal of the effort in support of Mirror Force is to develop, implement, and validate a process-oriented approach to requirements identification, instructional design and delivery, and the evaluation of training. Part of this effort is to demonstrate a variety of training delivery options that will provide faster, better, and cheaper training which can be delivered at anytime and anywhere. A key aspect of achieving the goal is demonstrating that there is a return on our investment in advanced training technology in terms of manpower and training resource savings to the Air Force operational community while, at the same time, sustaining mission performance effectiveness. The figure below illustrates the flow of information from the different parts of this process.
A Systematic Process
This process is composed of three major steps. The first step is the requirements identification, or needs assessment, step. This step includes identifying content areas for training development and delivery. A number of sources were used to gather potential content areas: Existing Stan/Eval documents and Air Force Audit Agency findings, outcomes from focus group discussions with SMEs, existing occupational survey information, career field education and training planning documents, Specialty Training Standards (STSs), Career Development Courses (CDCs), and existing Plans of Instruction (POIs). Once the content areas were identified, customer feedback, obtained with field interviews and surveys, was used to identify the most critical content areas for Medical Technician field performance in both peacetime and wartime terms. In our present effort, over 360 content areas related to Medical Technician activities were identified and built into computer-assisted surveys which were sent to the field. Two thousand surveys were sent to Reserve and Guard Medical Technicians respectively, and an additional 1,000 were sent to Active Duty personnel so that a more complete picture of field requirements and recommendations could be obtained. By including all three Service components, a direct comparison of the common and different needs of each was possible. Over 1600 surveys were returned with useable information from the field resulting in an initial response rate of 53%.
To help develop training priorities, information about the number of personnel who actually use the content as part of their current job, recommendations from field personnel for the timing of refresher training, and perceived consequences of poor performance of the content were used to help prioritize content for training development.
Findings from the needs assessment step indicated that key areas of the current CDC such as checking blood pressure, using a pulse oximeter, diagnosing respiratory symptoms, and using respiration equipment would benefit from advanced training technology. These content areas are identified in the general categories shown in the figures that follow. In addition, training related to maintaining proficiency in inspecting, storing, and donning chemical gear would be highly beneficial to field readiness. Therefore, respiration-related CDC content and readiness-related content associated with Nuclear, Biological, and Chemical (NBC) gear were the focus of our initial training development, delivery and evaluation efforts. The following figures show selected general categories of content and some of the resulting information obtained for Reserve, Guard, and Active Duty field data collection.
During the needs assessment step, existing medical training products and content from within the Air Force and across the DoD were identified and matched with training requirements. Although we have not completed this activity and we did not evaluate these products, we have identified targets of opportunity for collaboration and technology-sharing to reduce the need to develop unique training for identified training needs in any future effort.
Instructional Design and Delivery
Once the critical content areas were identified, this information was used for instructional design and delivery. In this step, the critical content areas were reduced to a series of learning objectives which, in turn, drove the development of interactive courseware (ICW). It should be noted that there is never a single solution to the development and delivery of training. It is always critical to match the objectives and the content of the training with instructional design and delivery technologies that are most appropriate. In our demonstration effort, respiration training was developed and delivered using a simulation-based system known as RIDES. The NBC gear content was developed and delivered using a multi-media-based system known as XAIDA. In addition to these systems, We also developed an Internet-based Career Development Course for the career field. Details about the design and delivery step of the process for the demonstration are described in related project brochures.
Evaluation
The third step in the process is evaluating the training in the field. This step involves developing and fielding measures which are tied to the learning objectives of the courseware. Specifically, measures of: (a) learning in training, (b) trainee attitudes (e.g., toward interactive courseware, motivation to train, perceptions of the quality of training, ease of use of the training and ease of access to the training using available computer equipment in the schoolhouse, field locations and potentially at home), (c) knowledge and skill proficiency, and (d) actual hands-on job performance, were developed and used to evaluate the training. Preliminary results indicated that trainees had a positive reaction to the training. In addition, there were positive gains in knowledge performance from the pre-test to post-test scores, although these data are based on a very small number of participants. Additional comparative studies are planned. Selected preliminary data on attitudes and instructional effectiveness are provided in the following figures.
Although the demonstration
effort concluded in February 1998, evaluations of the training will continue
throughout the remainder of the year. This will permit periodic enhancements
to the training and an examination of knowledge & skill retention and
decay as a function of real-world constraints on practice and performance
opportunities. The longer-term performance information will be used to
develop guidelines for refresher training content and training intervals
for all Medical Technicians.
Preliminary results are encouraging.
There is considerable interest from field units in participating in the
training development, delivery, and evaluation process. The Air Force Research
Laboratory is presently planning activities for the next steps in our support
of the Mirror Force initiative.
For further information, please contact Dr. Winston Bennett, Air Force Research Laboratory/AFRL/HEAA, 6001 South Power Road, Bldg 561, Mesa AZ 85206-0904; Comm (602) 988 6561, ext 297, or winston.bennett@williams.af.mil
