Evaluation of computer-based training for professional engineers

Abstract

The purpose of this research was to develop a novel framework for the evaluation of computer-based training (CBT) programmes used by employees of industrial organisation. Whilst there is increasing demand for training within industrial organisations, appropriate models for the evaluation of CBT programmes appeared to be absent. The literature review indicated that there are many models of CBT programme evaluation that can provide data about a programme’s effectiveness. However none of these models addressed the contextual factors that could affect a programme’s operation or effectiveness. These models would indicate to the designer of the CBT programme whether or not a particular programme was effective but not why. This research aimed to rectify this situation by producing a model of evaluation that would assist designers of CBT programmes understand the influence of contextual factors on their designs. In order to achieve this it was necessary to gain an understanding of the factors that would affect the attainment of appropriate training outcomes. It was also necessary to understand those variables that would influence the use of CBT programmes within industrial organisations. The former objective was achieved through an analysis of the existing literature on learning, the latter through investigation of the criteria used by employees of industrial organisations to evaluate CBT programmes. In the case of this research, 47 employees of 9 industrial organisations were asked to evaluate a specific CBT programme. Evaluative verbal utterances about a CBT programme were elicited from participants using a methodology that involved the use of concurrent verbal protocols supplemented by a series of specific questions on particular design features of the CBT programme. The research strategy that was adopted arose in response to a need to minims the time demands placed on participants, all of whom were employees of industrial organisations. The results of this analysis allowed the development of a descriptive model that identified the contextual factors that affected the perceived effectiveness of CBT programmes. Further models of evaluation were also produced. A normative model that described the cognitive and motivational factors that were believed to be present during the usage of different CBT programmes. The descriptive and normative models were combined to provide a normative model for the evaluation of CBT programmes used in industrial organisations. The limitations of this model and the research methods employed are also discussed.