THE EFFECTS OF STANDARDS ON LEARNING
IN AUTOMOTIVE REPAIR PROGRAMS

A Third-Party Summative Evaluation of the Standards Established by the National Institute for Automotive Service Excellence

Morgan V. Lewis
Lawrence Gill

Conducted for
National Automotive Technicians Education Foundation, Inc.
101 Blue Seal Drive, Suite 101
Leesburg, Virginia 20175

August 1995

Center on Education and Training for Employment
The Ohio State University
1900 Kenny Road
Columbus Ohio 43210-1090

Discussion and Conclusion

This evaluation provided a rigorous test of the effect of program standards on learning. The noncertified programs were selected to be very similar to the certified programs. The outcome measure of learning used items from a newly developed standardized test. And differences in the intelligence of the students that were associated with performance on the outcome measure were statistically controlled.

The comparison, noncertified programs were so similar to the certified ones that one of them - the Florida, noncertified post-secondary program - changed categories shortly after participating in the evaluation. This program caused the only difference between certified and noncertified program that was not in the expected direction. Even with this discrepancy, however, the statistical tests of the effect of certification were still significant at the .05 probability level.

The results of this analysis make a strong case that certification improves the learning that takes place in an automotive repair program. To provide a rigorous test of the effects of the standards, the noncertified programs were selected to be as similar to the certified programs as possible. It is very likely that if the comparison group had been selected from a more representative population of all noncertified programs, the differences between the certified and noncertified programs would have been larger than those found in this study.

Since this was a summative, not formative, evaluation, it did not attempt to identify the ways in which certification enhances learning, but the results of the site visits give some clues. The most obvious way that standards can influence learning is by ensuring that facilities, equipment, tools, and instruction are relevant to the real needs of the work place. Automotive technicians serve as members of NATEF certification teams to add their knowledge of local practices to the review of programs. Noncertified programs may not have the same level of linkage with the labor market.

The ASE standards also set forth clear objectives for the knowledge and skills students should acquire. These objectives focus instruction and may motivate students by communicating clearly the expectations for satisfactory performance. There is a large body of research that has established that expectations can influence learning either positively or negatively (Rosenthal and Jacobson 1968, Swann and Snyder 1980). It seems unlikely that noncertified programs would have the same degree of clarity in their objectives.

The goal of achieving ASE technician certification may also provide motivation. Students in certified programs know that the instruction they are receiving meets national standards. They can reasonably assume that if they satisfactorily master the skills they are studying, they will qualify for ASE certification after they have acquired the necessary on - the - job experience.

Conclusion

The overall conclusion of this third - party evaluation is that ASE standards have a positive effect on the learning that takes place in automotive repair programs. Students from programs certified by ASE scored significantly higher on a standardized test of knowledge of automotive repair than students from similar noncertified programs. It is highly likely that if the comparison programs were selected to be more representative of all noncertified programs, the differences between certified and noncertified programs would be larger than those found in this study.