Training efficiency examines whether the appropriate amount of time is devoted to training tasks, or whether tasks are potentially over- or under- trained. Task-level occupational survey information, such as Task Learning Difficulty which is based on the notion of time to learn, is compared to the actual amount of time devoted to training each task in a course. The results are then graphically depicted as one of three outcomes. A training match is depicted if the estimated time to learn, for example, is relatively the same as the actual time devoted to training that task, indicating that the relative amount of time devoted to training that task seems appropriate. A potentially over-trained task is depicted if relatively more time is spent training a task, compared to the corresponding time to learn estimate, indicating that training time might be reduced for those tasks. A potentially under-trained task is depicted if the time to learn estimate is relatively greater than the corresponding actual training time, indicating that more training time be allocated to those tasks. Thus, each task taught in training is graphically displayed as one of these three outcomes.
The results of the three matching outcomes are then integrated with training effectiveness information. Training effectiveness examines whether course graduates perform tasks above or below a specified standard later on the job. A 2 X 3 matrix is then utilized to display the effectiveness and efficiency integration. Two columns represent the training effectiveness for each trained task as above or below the training standard. Three rows represent the three potential outcomes from the efficiency results: a match, a potentially over-trained task, and a potentially under-trained task. Thus, a task may fall into any of the six boxes of this 2 X 3 matrix. For example, a task performed below standard may also be under-trained, indicating that more training time might be allocated to that task to improve its effectiveness.
Finally, efficiency and effectiveness results are integrated with training transfer data. Training transfer examines whether or not learning transfers from the course to the job. Efficiency is measured in terms of the average number of times each task is performed on the job. Effectiveness is measured in terms of graduate abilities to apply what was taught. This information is integrated into the 2 X 3 matrix and is displayed with the tasks that fall in each box to allow course designers to see, in one place, all of the data that are related to each task.
Examined separately, the results of training efficiency, training effectiveness and training transfer data provide useful information for understanding and changing different aspects of a training system. Results of the matching technique suggest that the course efficiency can be improved by reducing training time for over-trained tasks and increasing training time for under-trained tasks. Training effectiveness data suggest that for tasks performed below the performance standard, course revisions may be necessary while tasks performed above standard are less likely to require course changes. Training transfer data indicate that more task experience might be required early on-the job to facilitate job performance where the opportunity to perform the trained tasks is low. However, these data are more useful when considered in combination. For example, the results illustrate how a training program can be re-designed to be more efficient by reducing training time for over-trained tasks that were performed above standard, while increasing training time for under-trained tasks that were performed below standard. Furthermore, the training transfer information is useful in determining whether the training program itself should be modified, or the transfer environment changed. For example, if a task is under-trained, the amount of practice on the job is sufficient, but it is performed below standard, the training program might be modified by increasing the training time for that task. Alternatively, if an over-trained task is performed below standard on the job, yet the amount of practice for that task is low, modifying the training program is probably not a reasonable action. A better solution might be to ensure training graduates receive more practice for that task on the job, or consider eliminating it from training, and allocate that time to other tasks in the program. Of course, other options should always be considered, such as developing improved instructional approaches.
The TEEM software consists of two distinct components; they are the Data Entry component and the Presentation component. The main purposes of the Data Entry component are to load and format Occupational Survey data into the software and to guide users through the process of allocating training time to tasks taught in a course. After the data entry has been completed and a database for their course has been created, users will primarily use the Presentation component. The Presentation component provides users with the capability to select and have presented data which can support formal training decisions. Specifically, the Presentation component displays the Efficiency Graphs and the Efficiency, Effectiveness, and Transfer Integration Charts.
TEEM Database
Before the TEEM Presentation component can be used to display graphs
and charts, a TEEM database must be created for the training course in
question. This database consists of several elements which must either
be obtained from an existing agency, generated by the user or collected
in the form of surveys from the field. These elements are Occupational
Survey (OS) data, course Plan of Instruction (POI) data, Specialty Training
Standard (STS) data, training time information, and effectiveness and transfer
data.
Data Entry Component. The Data Entry component involves five
steps:
1) Enter Course POI Elements. This function enables users to enter the Blocks, Units, Objectives, and STS references associated with a specific POI.
2) Install Entries. This function installs the POI element data entered in Function 1 into the database and formats it for use in TEEM.
3) Identify Tasks Taught in Course. This function enables users to identify those tasks included in an entire AFS task inventory that are to be taught in the course for which the POI was developed.
4) Link Tasks to Blocks, Units and Objectives. This function allows users to complete the task linkage process.
5) Identify Task Training Time (TT). This function enables SMEs to enter, for each objective, the amount of time spent training each task linked to that objective.
Presentation Component
After all of the Data Entry component steps have been completed, elements of the database may be uploaded into the Presentation component and formatted for display. (Note: There is currently no method for uploading effectiveness and transfer data into the system.) The Presentation component is comprised of five primary functions:
1) Training Efficiency Graphs.
2) Efficiency, Effectiveness, and Transfer Integration Charts.
3) Data Reports.
4) Notebooks.
5) Editing the Database.
For further information, please contact Dr. Winston Bennett, Air Force
Research Laboratory/AFRL/HEAA, 6001 South Power Road, Bldg 561, Mesa
AZ 85206-0904; Comm (602) 988 6561, ext 297, or winston.bennett@williams.af.mil
