How to assess what people learned?

Evaluating the course – what does it mean?

Evaluation is essential for periodic updates in the course. Evaluating a course applies to the verification of its effectiveness towards achieving performance indicators for the course, or the learning achievements successfully accomplished through the course.

Evaluation is performed in two main ways: Through post-course surveys and through evaluation criteria. Your senior management and training managers are the ideal stakeholders who can define the course evaluation criteria. Consider the evaluation score as an important evidence of the effectiveness of your course. The post-survey scores will also determine the areas that need improvement and areas that need to be maintained in your course.

Why evaluation is important

Evaluation is a refined method of assessing the deeper course structure. Features like images, diagrams, interactivity, audio, video and other plugin players all need to be evaluated for relevance and clarity. Lessons need to be evaluated for conciseness and accuracy. Sources need to be evaluated for validity and reliability. Quizzes need to be evaluated for alignment with lesson objectives. Course practices and assessment (Knowledge Check) strategies depend on the learners and learning material.

These need to be evaluated to determine:

• The type of learning (e.g., knowledge, skill, and attitude)
• The level of learning (e.g., fact, rule, procedure, or discrimination learning)

Tools for evaluation

Pre and post surveys are developed by training developers and training managers to measure each level accurately and report for further course improvement. Keep surveys anonymous to retrieve unbiased opinions and suggestions. Use your learning management system’s metric tools to administer and calculate responses in the surveys. Have learners fill out the survey online and submit it upon entry and exit of the course. Learning objectives need evaluation too. Objectives present content to teach knowledge, skills, and attitudes. Assessments are written to test the acquisition of knowledge, skills, and attitudes. The Assessment (Knowledge Check questions) is written next, based on the objectives determined in the course.

The following strategies are used to evaluate different features of a course:

• Ask what (knowledge, skills, and attitude changes) you want the learner to acquire and demonstrate after participation in the course. Use the evaluation criteria for this feature to create compelling assessment items.
• Describe the expected learner’s performance or outcomes in measurable terms using action verbs. Use the evaluation scores to determine what needs to be edited to improve the transfer of learning to context.
• Specify conditions under which the performance will take place.
• Describe the minimum acceptable standard of performance (in terms of quality, quantity, completeness, or accuracy). This criterion is important to categorize the learning material into improvement groups.

Kirkpatric model for evaluation

The Kirkpatrick model of training evaluation was developed by Donald Kirkpatrick, Professor Emeritus of the University of Wisconsin in North America and a past president of the American Society for Training and Development.

It consists of four levels:

• Reaction
• Learning
• Behavior
• Results

Let’s look at each level in detail.

• Level 1: Reaction

  This level measures how your learners reacted to the training. Obviously, you want them to feel that the training was a valuable experience, and you want them to feel good about the instructor, the topic, the material, its presentation, and the venue. It’s important to measure reaction because it helps you understand how well the training was received by your audience. It also helps you improve the training for future trainees, including identifying important areas or topics that are missing from the training.

• Level 2: Learning

  At level 2, you measure what your employees have learned. How much has their knowledge increased as a result of the training? When you planned the training session, you hopefully started with a list of specific learning objectives. These should be the starting point for your measurement. Keep in mind that you can measure learning in different ways depending on these objectives, and depending on whether you’re interested in changes regarding knowledge, skills, or attitude. It’s important to measure this, because knowing what your learners are learning and what they aren’t, will help you improve your future training.

• Level 3: Behavior

  At this level, you evaluate how far your learners have changed their behavior, based on the training they received. Specifically, this looks at how learners apply the information. It’s important to realize that behavior can only change if conditions are favorable. For instance, imagine you’ve skipped measurement at the first two Kirkpatrick levels and, when looking at your group’s behavior, you determine that no behavior change has taken place. Therefore, you assume that your learners haven’t learned anything and that the training was ineffective. However, just because the behavior hasn’t changed, it doesn’t mean that learners haven’t learned anything. Perhaps their boss won’t let them apply new knowledge. Or, maybe they’ve learned everything you taught, but they have no desire to apply the knowledge themselves.

• Level 4: Results

  At this level, you analyze the final results of your training. This includes outcomes that you or your organization have determined to be good for business, good for the employees, or good for the bottom line. Bear in mind that this model isn’t practical in all situations, and that measuring the effectiveness of training with this model can be time-consuming and use a lot of resources.