Evaluate your online training with Thalheimer's evaluation model!

Evaluating your training - why would you do it?

The importance of evaluating your training during the learning process is often underestimated. We track how many lessons we attend, whether we participate actively, and then we conclude a subject with an exam or an e-learning with a quiz, and that’s that. We get a grade, wonder what we did wrong, but as long as we pass, we don't think much more about it. We also don't stop to consider whether we can actually apply this knowledge and these skills in our daily work.

This is, of course, a real shame.

If you receive valuable feedback on your online training while you are learning, the final learning outcome is much greater. You can see which thought processes or skills you still need to adjust or refine. And not just afterwards, but during the process as well, so you can change course before the course ends.

There are various feedback models used in e-learning. For example:

• Kirkpatrick's 4 levels
• The “Five Moments of Need”
• The five phases of ADDIE
• The “Nine Events of Instruction”
• The “Decisive Dozen”

To name just a few.

During the Next Learning festival, we attended an interesting session with Will Thalheimer. His message was as follows.

Research has shown that the actual learning outcome cannot simply be measured with “old-style” evaluation forms. By formulating and administering questions differently, you get a much better picture of what has actually been learned. In this regard, the “Learning-Transfer Evaluation Model” is a useful tool.

In this blog, we tell you more about Thalheimer's evaluation model and why it appeals to us. But before you continue, it is good to remind you that it is always better to make your training as good as possible in advance. And that effective knowledge transfer often goes hand-in-hand with a strong online learning platform. Don't have a platform that meets all your needs yet? Take a look at the possibilities of Pluvo!

Evaluating your online training the old way

The fact that someone participates in a learning path does not mean that this person has actually learned anything; therefore, measuring attendance (how many sessions this person attended) says nothing about the learning outcome. If students complete a course (the familiar percentage bar at the top of an online learning environment), it does not mean that this person has learned anything from the e-learning. That is why measuring the percentage of the amount of material a student has gone through is pointless.

The same applies to participation. If someone pays attention during lessons, or is interested in the material, or even indicates that they really like a learning event, it still does not necessarily mean that he or she has learned anything. Measuring interest is therefore not useful.

Furthermore, old feedback models are also ineffective at measuring knowledge. If someone can regurgitate information, it does not mean they can apply this knowledge in practice; so why would you measure the recitation of knowledge? Even if someone indicates they understand a concept or even demonstrates a certain skill during a lesson or online, it does not mean that he or she can make the translation to the work practice. Therefore, measuring knowledge of a concept or the level of skill mastery immediately after a learning event (lesson, online meeting) is not useful.

Thalheimer argues that when creating an evaluation, we should focus on: understanding, remembering, decision-making, task competence, and persistence in applying what students have learned to their work or other “performance situations.”

But how do you do that? Thalheimer explains it according to his “Learning-Transfer Evaluation Model.” We will briefly explain the model. The links in this blog refer to the comprehensive article on this subject, written by Thalheimer himself.

The Learning-Transfer Evaluation Model

1. Attendance and participation (levels 1 & 2)

Thalheimer previously argued that attendance and participation are not useful metrics, but there are still several reasons to track them. Measuring a student's attention, interest, and participation can be useful to help you identify the strengths and weaknesses in your e-learning and can also give you an indirect indication of the student's interest, motivation, and engagement - this information can be useful for improving the design of the e-learning.

2. Perception (level 3) 

When we ask about students' perception, we usually mean: would they recommend the course to others? Did they think the instructor was good? Did the course meet their expectations? Thalheimer advocates asking questions that are far more valuable for measuring the actual learning outcome, such as: did the e-learning support your understanding of the material, your remembering of the material, and did it contribute to your motivation to apply what you learned in practice?

3. Knowledge (level 4) 

If students are tested shortly after the course on what knowledge has stuck, this is only a snapshot of long-term memory. This offers no guarantee that the student has actually made the knowledge their own. That is why Thalheimer recommends testing the knowledge again at a later point to find out what the student has actually internalized. However, this is still not an ideal method, because it says nothing about how the student applies this knowledge when making decisions, for example.

Our blog on creative online knowledge testing is also a fun read! 

4. Decision-making (level 5) 

To evaluate realistic decision-making, we can present students with realistic situations so they can practice making decisions similar to the kinds of decisions they have to make in their daily work.

5. Task competence (level 6)

To measure task competence, we can have students evaluate realistic situations and then enable them to make decisions and take actions that match those decisions.

Also read our blog on digital tools, where we go deeper into using lifelike simulations and gamification in your training.

6. Transfer (level 7)

“Are students successful in using what they have learned in targeted work situations?”

7. Transfer effects (level 8)

“If students have indeed succeeded in transferring what they learned to their work situation, what effects does that have on targeted and untargeted outcomes and stakeholders?”

Training evaluation example

In a next blog, we will explain how you can actually apply this model while developing your own e-learning. Want to get started right away? Contact us in the chat!