ADDIE - Evaluation Phase

Evaluation Phase

Evaluation in ISD

Background

Definition

Evaluation is the systematic determination of merit, worth, and significance of a learning or training process by using criteria against a set of standards.

The primary purpose is to ensure that the stated goals of the learning process will actually meet the required business need.

Ongoing

Evaluation is performed in an ongoing process throughout the first four phases of ISD

In Analysis, it looks at deciphering whether the performance problem is a training problem. If it is, how will a learning platform positively impact a business need or goal? Then, what must the learners be able to do in order to ensure the required change in performance?

In Design, evaluation occurs when you identify what must be learned by the learners that will enable them to fulfill the required business need or goal.

In Development, you find the activities that will best bring about the required performance.

In Implementation, you find out if the learners have now become performers.

Phase

It is taken one step farther by ensuring that the learner can actually meet the new performance standards once they have completed their training and returned to their jobs. And secondly, ensuring that the business need or goal is actually being met.

Purpose

Reich’s Gap

The gap between what the teacher teachers and what the student learns.

Evaluation helps to measure the gap by determining the value and effectiveness of a learning program.

Bramley and Newby (1984)

Feedback

Control

Research

Intervention

Power Games

ROI

Evaluation ensures that training support the business units’ needs. This is primarily because stakeholders for the most part do not view training as a profit center but rather as a strategic partner who supports their goals.

Types of Evaluations

Formative

Definition

A formative evaluation, sometimes referred to as internal, is a method for judging the worth of a program while the program activities are forming, or in progress. This part of the evaluation focuses on the process

Purpose

Its main purpose is to catch deficiencies so that the proper learning interventions can take place that allow the learners to master the required skills and knowledge.

Wally Guyot

Formative evaluation is primarily a building process which accumulates a series of components of new materials, skills, and problems into an ultimate meaningful whole

Summative

Definition

A summative evaluation, sometimes referred to as an external, is a method of judging the worth of a program at the end of the program activities, summation.

Methodology

The model used to gather the date should be a specified step-by-step procedure.

History

Scriven (1967)

Formative evaluation was intended to foster development and improvement within an ongoing activity. Summative evaluation is used to assess whether the results of the object being evaluated met the stated goals.

Saettler (1990)

Formative is used to refine goals and evolve strategies for achieving goals while summative is undertaken to test the validity of a theory or determine the impact of an educational practice so that future efforts may be improved or modified.

Today

Most organizations are demanding shorter design times. Thus the formative part is move over to the other methods, such as through the use of rapid prototyping and using testing and evaluation methods to improve as one moves on. Which, of course, is not as accurate but it is more appropriate to most organizations as they are not really that interested in accurate measurements of the content but rather the end product, namely skilled and knowledgeable workers.

Kirkpatrick’s Evaluation Model

Stages

Reaction

How the learners react to the learning process

Learning

The extent to which the learners gain knowledge and skills

Behavior (Performance)

Capability to perform the learned skills while on the job

Results

Includes such items as monetary, efficiency, morale