Martha Reesman
National Center for Manufacturing Sciences

Distance learning occurs when instructor and learner are not in the same place. Over the past century this has evolved from correspondence courses sent by mail between the learner and the instructor, to today’s ability to bring instruction into the workplace or the home using communications technology. Technology is used to bring the instruction to the learner and to provide a vehicle for ongoing interaction between the learner and the instructor as well as among the learners. The vision of the National Center for Manufacturing Sciences (NCMS) Distance Learning Project was, and continues to be, to deliver, via modern communication technology with extensive verbal and digital interactivity, the highest quality and value education and training from the master teachers throughout North America to the workplace, computers and homes of employees.

From September, 1995-December, 1996, a group of NCMS member companies and organizations conducted a collaborative project to research and pilot interactive distance learning (IDL). These companies included:

• Cadkey
• Eastman Kodak Company
• Electronic Data Systems (EDS)
• Ford Motor Company
• General Motors Corporation
• Lockheed Martin Energy Systems
• National Institute of Standards and Technology, Manufacturing Extension Partnership
• RWD Technologies
• Texas Instruments, Inc.
• University of Rochester, Center for Optics Manufacturing
• U.S. Air Force.

Each company or organization provided resources to conduct the project. This included funds from a cooperative agreement with the U.S. Air Force ManTech program and from the National Institute of Standards and Technology, Manufacturing Extension Partnership.

Two courses, An Overview of Six Sigma and Geometric Dimensioning and Tolerancing: the New Standard (GD&T), were developed. Three broadcasts, reaching over 418 participants at more than 24 sites were conducted (see Figure 1).

In addition, Instructional Design Guidelines were developed to provide instructional designers, project managers, instructors and subject matter experts with a reference for designing IDL. The IDL Instructional Design Guide is already being used and has been adapted by the participant companies for development of future courses.

The Guidelines were designed to be used in conjunction with a two day workshop. The goal of the workshop is to teach instructional designers to develop IDL. The workshop focuses on a learner-centered approach. Two pilot sessions of the workshop were held as a part of the project.

The IDL group is moving forward, to take what they have learned and implement it by developing and broadcasting manufacturing focused IDL. The mission of such an effort will be to develop high quality, manufacturing focused IDL training and education through a multi-company program.

This project was initiated by Dr. Jerry Steele, who, before his retirement from Ford Motor Company, championed the development of a collaborative IDL project at NCMS. For this particular project IDL focused on:

• Two-way audio and or textual communication between the instructor and the learners during and between classes
• One-way video of the instructor to the learners via satellite
• Technical media used to link the instructor with students, students with students and students with course materials
• An interactive data response system, One-Touch^®, that allows for immediate response and tabulation

A graphic representation of the technical configuration is shown in Figure 2.

This technology was selected because most of the participating companies either possessed the infrastructure, were willing to acquire the required equipment, or were able to access the required equipment through another participating organization.

The goal of the companies is to use the technology to allow for participation by larger number of learners at multiple locations while maintaining or improving the quality of instruction. The key to achieving this goal was to develop instructional design guidelines that support use of the technology to benefit the learner.

During the planning process it was decided that this element should be supported by a consultant with knowledge of IDL. NIST/MEP funded the support of a contractor through a grant to NCMS.

Mei Technology, San Antonio Texas, was selected as the contractor. Mei has been actively involved with the U.S. Air Force in research and development for IDL.

The objectives of the project were:

• To pilot the use of IDL across the project companies
• To use the pilot to develop a business case for the use of IDL by individual companies as well as consortia of companies
• To use the pilot to develop a "best of the best" understanding of instructional design for IDL
• To evaluate the effectiveness of IDL for providing instruction to the employees of the participating firms
• To evaluate the effectiveness of IDL for providing training to small and medium sized companies

These objectives were met by:

• Developing IDL courses in Six Sigma and GD&T.
• Conducting three broadcasts. Transmitting the courses to 418 employees at 24 sites across ten company locations.
• Conducting evaluations on the instructional methodologies and the effectiveness of IDL using the courses delivered to the participants.
• Developing guidelines for designing IDL drawn from the expertise of the project participants and lessons learned from the project courses.
• Using the guidelines as the basis for the development of a workshop for instructional designers.
• Developing guidelines for technology interoperability that addresses transmission of interactive courses across multiple companies and technological platforms.

The approach undertaken by the participants was to utilize existing expertise in instructional design and distance learning to develop guidelines to conduct IDL. The specific approach selected by the participants was to conduct three short courses of common interest across the participant companies.

In the spring of 1995, during the planning of the project, the selection of courses to be designed and developed for the project began. The project planning group identified priority subject areas for their company. A combined list of these areas was circulated and ranked by the companies. The top three areas identified were:

1. Six Sigma
2. GD&T
3. An environmental course.

An ambitious schedule was set for the redesign and broadcast of the Six Sigma course. The broadcast was scheduled for October 26, 1995. This was considered possible as the course content was drawn from an existing classroom course. The course’s instructor, from Texas Instruments, had extensive experience teaching the course in the classroom.

The original classroom course, An Overview of Six Sigma, is eight hours in length. The IDL course was half that long, i.e., four hours. This required significant re-working of the course materials and content. In addition exercises were developed that were to take advantage of the IDL medium.

The Course team members were drawn from Texas Instruments, The Center for Optics Manufacturing and Eastman Kodak. The Broadcast was based at EDS. The course was broadcast to 148 people at 22 sites on October 26, 1995.

The IDL project Steering Group met (in November, 1995) to review evaluation results from the first course and discuss ways in which the course could be improved. These included:

• Pacing the flow of the course—the lecture, exercises and evaluation
• Improving the graphics; using fewer words, different colors, not just static visuals
• Having very complete participant materials with check lists, One-Touch^® information, and test questions
• Having students arrive 1/2 hour early to get organized
• Building in more and better interaction that addresses different learning styles
• Using the time carefully; focusing the materials, the instructor and the learner
• Building interactivity at the sites, with the instructor, or -across sites
• Being careful of the line between entertainment and education

Based upon the feedback from the group the course was revised and re-broadcast on April 29, 1996. It was attended by 131 people at 19 sites. The findings of the debriefing and evaluation materials for the revised course indicated that:

• The participant manual was greatly improved
• A lack of studio practice time was a problem. The EDS studio was being upgraded and did not come back on line until the last minute

The IDL medium isn’t tolerant and it is imperative to reach out and capture the audience quickly. One-Touch’s^® response time lag was obvious in this course. It takes planning to cover this lag and is something the course design and instructor must address. The design of the course sets the stage for questions. A bailout position is needed if the first approach doesn’t work. Course design requires a mixture of approaches lecture, questions and exercises.

The second topic selected by the project Steering Group was GD&T. A cross company team of subject matter experts (SME) and instructional designers was convened in November, 1995. The individual companies have numerous courses in GD&T, so the SMEs focused the IDL course on recent changes made to the GD&T ASME standard.

The team developed course objectives, major topics to address and a scope and sequence for each topic. The SMEs and Instructional Designers formed sub teams to address each of the major topics as well as deadlines for completion. In addition the graphics and production team generated drawings and graphics.

The team developed a script format and a modification of an audio visual script to be used for the course. The script provided the instructor, and the production staff, with a common document to follow. The left side of the script included the graphics that would be on the screen as well as production cues. The right side of the script contained the instructors notes. The script proved to be a very valuable tool.

The course instructor attended a week long distance learning instructor training course offered by the U.S. Air Force at Maxwell Air Force Base. This course was being piloted by the Air Force. Major Kim Bowling, who represented the Air Command and Staff College on the Steering Group, arranged for two NCMS instructors to attend. The course at Maxwell AFB was helpful in building confidence as well as an understanding that you can’t walk in and do it, but you can if you practice. In addition, the team was able to schedule several days of practice in the GM studio. GM does not have a practice studio, but two team members developed a mock practice studio that allowed for additional practice.

The GD&T course was broadcast on May 5, 1996. It was attended by 139 people at 22 sites. The participants responded very well to the course, and there was a great deal of interaction throughout. Elements of humor, in an otherwise very technical subject, were very well received. The evaluation data was very positive. Major elements targeted for future improvement are:

• Understanding the variation among the sites, e.g. size of viewing screen.
• Experimenting with capturing the broadcast onto CD-ROM for future use.
• Freezing the script, graphics and videos earlier.
• Standardizing an instructor guide format.

Several practices that should be continued are:

• Using questions carefully. Questions, in IDL, have very different affect and must be very focused.
• Giving the instructor a clear understanding of the audience.
• Learning more and more about how to do IDL well and codifying the results.
• Using good graphics such as the split screen.
• Using humor as an effective learning tool.
• Offering diverse activities to address different learning styles as well as for interest.

During the planning process, an environmental course was selected as the third course. A broad range of topics exists within this subject area and much discussion took place among company SMEs to identify a specific environmental topic that would be of interest to most of the companies. Lockheed took the lead role in this effort, conducting a needs assessment that resulted in the identification of a specific topic: Source Reduction.

However, at the May, 1996 Steering Group Meeting it was decided, after two courses and three broadcasts, that the goals and objectives of the group had been met and that the effort required to design and broadcast an environmental course was not needed.

From January-April, 1995, during the planning process, a survey of the participants was conducted to gather information about their communications capabilities. This survey resulted in the selection of satellites and One-Touch^® as the technology delivery platform for the project. An EDS team was tasked with the responsibility of linking the companies.

Each of the three courses was transmitted to more than 22 sites representing ten companies. The task of linking so many sites across different companies was not insignificant. Several of the large companies, Eastman Kodak, U.S. Air Force, Ford Motor Company, Texas Instruments, General Motors, EDS and Lockheed Martin, have existing satellite networks, some of which were analog, while others were digital. This required that all courses be simulcast.

Several of the companies had to acquire One-Touch^® equipment. Because of high demand for One-Touch^® equipment, there were some delays in obtaining key pads. This was solved by inter-company cooperation, e.g., EDS loaned Texas Instruments One-Touch^® equipment.

NIST/MEP and COM did not have satellite link-ups for their members. These were provided by EDS at either EDS or EDS client facilities across the country. The logistics of making these arrangements were often daunting.

Prior to each broadcast a company and site registration form was sent to each participant organization by NCMS. The completed form was then forwarded to EDS. Prior to each broadcast two tests were conducted, the first several days prior to the event, and the second in the hour preceding the course.

Two of the courses were broadcast from the EDS studio in Plano, Texas, and the third was broadcast from General Motors studio in Detroit, Michigan. This change required reconfiguring some elements of the broadcast.

Each of the broadcasts had some technology difficulties at the receive sites. These included loss of key pad functionality, visual distortion of the picture and, the worst problem, lack of connectivity. Although satellite technology is mature, it is not without its foibles. Linking across systems added a dimension of complexity.

• The importance of gathering detailed information early.
• Identification of a technical support person at the company.
• Pre-testing of the connectivity prior to the day of the event.
• Standardizing the technology used as much as possible.

Evaluations were conducted for each of the three broadcasts. A Level One evaluation was built into each of the courses, with the data collected through the One-Touch^® system. The evaluation data gathered included:

• Demographic information about students.
• Respondent reaction to the instructor, course pace and IDL medium. (Level One).
• Respondent knowledge of course content at the beginning and end of the course. (Level Two).

In addition, several of the participants held evaluative discussions with the attendees at their sites immediately following the broadcasts. These anecdotal information allowed for more in depth understanding of the responses. Given the research objectives of the project, this was important to improve understanding of IDL design and delivery.

Each of the participating companies or organizations were asked to provide Mei Technology with copies of either their ISD guides or any distance learning design guides. These were provided by Eastman Kodak, Ford Motor Co., GM, EDS, Texas Instruments and the Air Force.

The Guidelines are organized so that users can progress from a general understanding of the fundamental issues regarding IDL instructional design to more specific applications of design principles. As such, the Guidelines begin with a discussion of learning and how learning can be affected by distance. There is a comparison of distance learning or IDL with traditional classroom teaching. This gives designers a chance to shake off some old stereotypes and to make use of what they have been doing throughout most of their careers. The focus is on the IDL student and devising ways of building effective IDL learning environments for students. Specific objectives for the Guidelines are that:

• Project managers are able to tell whether the process being used by their team is adequate and will provide the excellence expected.
• Instructional designers are able to look up activities, principles and methods to apply to projects.
• Subject matter experts get better insight into what is expected of them and how their collaborations and contributions are transformed into a course through the instructional design process.
• Media specialists should recognize some principles to follow in tailoring their products for distance learning applications.
• Instructors will begin to understand why distance learning courses are structured the way they are and how they play a critical role in making each course successful.

Although traditional instructional systems development (ISD) terminology and approaches are still used, some strikingly new approaches to learning and training design are included in the Guidelines.

Several years ago, the U.S. Air Force, Armstrong Laboratory asked a committee of experts to assess the effect of automation and multimedia on instructional design and development. Their conclusions were two:

1. Conventional design and development practices, particularly for interactive instruction, require such inordinate levels of expertise and resources that they are beyond most organizations' means, and
2. Conventional design and development theory simply does not address the opportunities inherent in modern adaptive training approaches, e.g., interactive, computer-based, multimedia training. They tend, if anything, to strip away the interactive aspects of such training.

All of their conclusions apply to IDL. These same experts conceptualized new approaches to design and development based on emerging theories of instructional design. Armstrong Laboratory currently has a vigorous R&D program based on many of these theories.

The focus of the Guidelines and of learning theory in general today, is on the learner. Instructional design is about improving the learner’s ability to understand.

The Guidelines, as developed by Mei Technology, have taken the path which ushers into the future by including those aspects of current instructional theory which have been tested and work. At times, the Guide also alludes to newer approaches from instructional theory which promise success but have not yet been fully tested.

The Guidelines are designed to be used in conjunction with a two day distance learning instructional design workshop. The workshop provides the user with practical examples to apply the general principles explained in the guide. After the workshop the guide becomes a desktop reference for instructional designers, subject matter experts, program managers and other members of an IDL project team.

The workshop consisted of two days of interactive sessions in which attendees learned about IDL, new skills pertinent to design and development of IDL and how to apply these skills. Student activities included selecting IDL candidate courses, modifying existing course materials for IDL, creating media for IDL and identifying IDL presentation skills. Students brought sample course materials with them to make the workshop more relevant.

A majority of the workshop consisted of discussions, practice exercises, and demonstrations. Classroom activities entailed collaborative teamwork, individual presentations and constructive critiques. Each attendee participated in an IDL laboratory session. Attendees designed, developed and delivered a brief IDL lesson in the studio.

The learning objectives for the workshop are:

• Understand how people learn and what makes learning more effective.
• Differentiate between a traditional versus distance classroom.
• Identify factors to consider in selecting IDL courses.
• Identify student interaction versus learner involvement.
• Develop instructional strategies and IDL techniques for implementation.
• Match learning strategies with the audience.
• Understand testing: What is it? And, why do we do it?
• Use questions to enhance learning.
• Integrate group activities into an IDL course
• Train experienced teachers new to IDL.
• Identify techniques for IDL instructors.
• Prepare media for IDL.
• Prepare handouts & workbooks for IDL.
• Develop evaluations: what are you trying to prove?

Two workshops were held as part of the project. Workshop attendance was limited to 12 people per session. Each organization in the project was offered the opportunity to send two people.

Level one evaluation data was collected by NCMS for both sessions. The instructor, Bill Walsh from Mei Technology, noted that the sessions had very different groups of attendees with different backgrounds. The demographic data collected in the evaluation showed a range of instructional design experience from 0-20+ years.

As with all pilot programs, areas for improvement were suggested, as well as areas that the participants thought were successful.

The project was managed by a Steering Group from the participating companies and organizations, and an NCMS Project Manager. The intent of the team was to work by consensus.

Each of the companies or organizations participating in the project provided a representative to the Steering Group. This group was then charged with overseeing the overall project implementation and assuring that their company’s resources were provided as required.

This project was designed to learn more about the use of IDL. The intent was to use the results to increase the use of IDL at the participant companies. A major premise of the group is that the combination of economic savings and improvement of learning quality for employees makes widespread use of IDL likely.

Technology mediated instruction is becoming more and more a part of industrial training. The maturity and costs of instructional technologies make it possible for companies to increase their usage. Proprietary data has shown that from 1994 to 1995 there was a seven percent increase in the use of instructional technologies by major corporations. Companies can use IDL to increase access to training opportunities and to feel assured that people can gain higher content retention. IDL is a mechanism that enables enterprises to deal with rapid change. It allows companies to reach a larger number of people with a consistent message over a shorter period of time.

Companies have found through benchmarking that 40% of an organizations training-related costs are non-instructional time and travel expenses. IDL allows companies to achieve significant cost savings through the reduction in time and travel to attend training, reduction in need for training facilities, and in the ability to provide training to more people at one time. Ensuring that this includes high learner comprehension has been a critical need.

The project served as a pilot for broad based implementation within the companies. The deliverables provided them with the ability to design and deliver IDL instruction for their own employees, suppliers and member companies.

Courses in Six Sigma, and GD&T were designed and broadcast using cross company teams that operated at great geographic distance and from different corporate perspectives. While this approach added some barriers to the efforts, it also added value. The team members learned from one another, and from being exposed to different views and approaches.

One of the courses, GD&T, has already been re-broadcast twice by General Motors. Hopefully others will follow suit.

During the project, participants learned the benefits of leveraging across organizations and the value to be learned from other organizations and experts.

• They supported the project with joint funds and resources, focusing on using existing satellite technology and equipment.
• They used existing course content and modified it for IDL instruction.
• They developed a new course using multi-company subject-matter experts and instructional designers.

The ten organizations that worked together built an understanding of what it takes to design effective instruction for IDL, to transmit courses to multiple organizations, and to work together as a consortium. Together, they pooled their knowledge to design, develop and broadcast courses among the consortium participants.

As this project ends, the focus has turned to the creation of an alliance of organizations to develop and transmit manufacturing-focused IDL courses. The goal is to use what has been learned collectively and apply it for mutual benefit. The key objective is to address the common needs of providing high-quality training for employees and supplier firms while reducing cycle time and cost. The vision is to cost effectively supplement training at participating organizations by collaboratively developing the necessary infrastructure for technology-supported education and training.

The group is looking for partners to create a Manufacturing IDL Consortium. Organizations that:

• Have a commitment to IDL and will support it with their funds and other resources
• Are willing to share their learning needs
• Want to collaborate with others to develop courses that meet common goals
• Can take what we have learned collectively and move from a developmental mode into a implementation mode.

This approach offers a wealth of opportunities for the partners. These opportunities include:

• High quality instructional design
• Master teachers
• "Best-of-the-Best" practices for IDL
• Cost savings for course delivery
• Shared overhead costs
• Higher utilization of assets
• Reduced course development costs
• Broader scope of courses and curriculum.

Partners in the effort will be drawn from manufacturing firms and organizations focused on manufacturing that are using IDL to train their employees or suppliers. Common characteristics of partners will be that they are the purchasers of courses; dedicated to working collaboratively to share our knowledge of learning and our courseware development needs.

To ensure a consistent approach and course quality, the IDL instructional design guidelines developed in Phase 1 of the project will be used as the basis for future courses. Courses created through the consortium will be reviewed for quality and adherence to the IDL guidelines. A mock run-through will be required for each course.

The consortium will broadcast the courses across the participating companies. It could also explore the possibility of offering the courses to other companies for a fee.

Satellites are currently the least expensive and most prevalent IDL technology; however, other technology is evolving rapidly. The goal is to use the new IDL technologies to develop a series of instructional tools and populate them with courses.

The benefits of these new technology platforms are numerous. They allow companies to expand the array of options they consider when designing and delivering instruction to their employees. The key advantage of these technology platforms is that they allow students quick access to learning resources without the need to travel. The new technology also offers better quality video, real-time interaction, and evaluation of students.