WHAT IS GenSurv? .
A Research and Development (R&D) project to develop a Windows-based surveying utility capable of operating a variety of occupational-oriented types of surveys on the internet, via e-mail, or on disk.
Increasingly, the military is taking advantage of the technologies produced by the "network" age by collecting and sharing data much more collaborative ways. Many DoD personnel routinely use e-mail, post information to and download information from the Internet, and perorm some interactive processing. Some mainframe applications have been developed to collect Manpower, Personnel, and Training (MPT) data using electronic forms to automate what have traditionally been paper and pencil processes. Recently, some agencies have begun collecting data electronically via computer disks or from electronic forms transmitted over the Internet.
Internet technology provides three key benefits. These benefits include hardware independence, easy access to data and improved software portability.
With the recent operational implementation of automated surveying technology (AUTHOR 3.0 & OASurv 3.1) in the Air Force, a number of other possibilities become feasible: improved authoring of survey items and task lists, using electronic mail for survey administration, and interactive, platform-independent surveys on the Internet using Java or similar language. Such Internet data collection can be used for more expeditious training requirements analysis, occupational surveys, collecting job and training history information, task performance and time-to-learn assessments, gathering real-time consensus estimations of learning curves by topic or task cluster, and facilitating refinement of training resource requirements analysis by type of organization (mission) for representative units.
The developing interactivity of the Internet suggests that on-call reference files (including film clips and animated gif files), consultant services (chat room, etc.), just-in-time training, and reinforcing feedback through reports to individuals can be economically developed and provided to individuals. Web posting of results of surveys and analyses with e-mail notification to respondents is also now both feasible and practical. There is, however, much work to be accomplished in the area of enhancing data relevance and quality as well as developing and applying intelligent search engines for defining occupations and training requirements, and other interactive enhancements. Current Internet developments and usage suggest that significant reductions in time and costs can be made, with a strong probability of enhanced reliability and validity of survey responses as well as increased participant motivation and involvement.
A successful large-scale laboratory test of computer-assisted survey software (Albert, et al., 1993) was followed by a feasibility study to assess the possibility of computer-assisted occupational analysis surveying in an operational environment; this study concluded that occupational data could be collected via diskettes more quickly and efficiently than by traditional paper-and-pencil surveys and with much reduced costs ( Mitchell, et al., 1994). One of the major recommendations made in the final report of the feasibility project was the need for the development of a windows-based survey system which could operate over the internet to facilitate real-time surveying and enhance the quality of data collected (Mitchell, Weissmuller, Gosc, & Bennett, 1995).
Platform Independent Survey Authoring and Administration: The preliminary technology for AF automated surveying presently executes in a DOS environment. This approach has been shown to be feasible and ensures the maximum reliability of the administrations in the field (Mitchell, Weissmuller, Bennett, Agee, & Albert, 1995). User acceptance of the current approach remains quite high. There are a number of very prudent reasons why the DOS approach was taken for initial surveying efforts. First, field data from over 6000 administrations indicates a wide variety of platforms and varying levels of network software and installations currently operating in the operational Air Force (Weissmuller, et. Al, 1997. The continued diversity of Windows operating systems (3.1, Windows95, 97, NT, etc.) precluded reliable survey administration in field settings in the Windows-type of environment. Work is now underway to adapt both the authoring tool and executable survey engines to be platform-independent. Using Java or an equivalent approach avoids the pitfalls and software faults associated with a Windows-dependent environment. Initial investigations indicate that this new approach to occupational analysis and training evaluation is not only feasible but will probably yield quicker results and improve data quality.
Interactive Data Collection and Feedback: Among the advantages of the Internet is greater interaction between a survey respondent and a central database, although some of this type of interactivity is also possible with hard disk-based software systems (Albert, et al., 1994). This permits respondents to look up definitions, specifications, course titles, official job titles, unit and base designations, and other information that might be needed to complete a survey. In addition, respondents can be queried selectively, based on their response to an earlier question, or can be asked a series of questions based on a logic tree built into a particular survey instrument. Background information can also be selectively presented so that a survey taker sees only the questions relevant to his or her status (military, civilian, contractor, etc.). This type of interactivity is obvious with Internet technologies, although there are many details yet to be worked on in terms of what type of central database is needed in each study and the degree of development needed for complex logic trees.
Another aspect of such interactivity that needs to be defined for operational practice is the amount of feedback to be given to survey respondents. For example, it should be possible to provide a respondent with a summary of the information he or she has completed. For an occupational survey, this might take the form of an individual job description with completed calculations of relative or "actual" time spent per task and/or major grouping of tasks (duties, modules, etc.). This would permit the individual to do a final review of the information and make any changes, corrections, or modifications that are necessary. In the initial laboratory study of automated surveys (Albert, et al., 1994), such feedback was used as a final quality control step to ensure realistic responses and assessment of time allocation, using a variety of time rating scales. Such a file could also be printed, downloaded to disk, or otherwise stored for the individual's later use or disposition; this type of feedback has motivating potential and the degree of improved quality for such a procedure should be assessed. As experience with interactive surveys develops, a number of other ways such interactivity can help motivate individuals to provide realistic and reliable information will be identified.
Mitchell, J.L., Weissmuller, J.J., Gosc, R.L., & Bennett, W., Jr. (1995, September). Feasibility study of the development, implementation & evaluation of computer-based job & occupational data collection methods. Draft final report, prepared for the Technical Training Research Division of the Armstrong Laboratory, Human Resources Directorate, Brooks AFB, TX.
Mitchell, J.L., Weissmuller, J.J., Bennett, W.R. Jr., Agee, R.C., Albert, W.G., & Selander, D.M. (1994, October). A field study of automated occupational survey administration methods. In the symposium (Bennett, W.R., Chair), Training needs assessment and occupational measurement: Advances from recent research. Proceedings of the 36th Annual Conference of the International Military Testing Association. Rotterdam, The Netherlands: European Members of the IMTA.
Weissmuller, J.J., Grimes, G.R., Siem, F.M., & Kenny,
J.E., (1997).. Automated surveys or paper & pencil: A real choice.
Proceedings of the Ninth International Occupational Analyst Workshop.
San Antonio, TX: Air Force Occupational Measurement Squadron.
