Measuring Effectiveness

In today's economy, you can't afford to have your training time wasted. Besides the cost of classes and related travel costs, the loss of a technician for a few days can be difficult--especially with some shops running on leaner staffs.

Good training providers will provide at least two levels of assessment, and use the information gained to improve future classes. The first two levels are:

  • Level I--Survey administered immediately after class. Often called a "smile sheet," this measures how the student feels about the value of the training, what he or she liked about the class, etc.
  • Level II--Post-Test administered immediately after class. This measures the knowledge and skills transfer that the class should have provided. If there are hands-on objectives for the course, the post-test should include at least some hands-on assessments in addition to a written test.

Unfortunately, some training providers do no assessment at all, providing a certificate based on attendance only. If this is the case, you should do something similar--informally or formally--when the technician returns to work. Some simple questions to ask trainees can include:

  • Did the training class have clear objectives? Were you told what was expected of you?
  • Did most of the training cover systems and procedures that were unfamiliar to you?
  • Did the training provide knowledge and/or skills you can directly relate to job responsibilities?
  • Was the instructor knowledgeable about the subject matter? (Well-designed training courses make the instructor less of a factor, but there are still instructor differences that can affect training outcomes.)
  • (If on-vehicle skills are an expected outcome) did the class have enough hands-on activity?
  • Were you tested on what you knew or could do after the training?
  • Do you still feel you need more training on this subject?
  • What did you like best about the training? What could be improved?

These and other questions you would like to add can be done in face-to-face interviews, on paper, or on a company intranet. After gathering data, share the information with your training provider. Ask what plans they can put in place to make the suggested improvements, and/or fill in the "omissions" you discover.

You should also take time to make additional assessments approximately 90 days after training. This allows enough time for the technician/student to use the training on the job, and get further practice on any procedures learned. This is considered a Level III assessment, because it measures behavioral change from training. Some questions on the 90-day survey could include:

  • Are you getting enough opportunities to apply what you learned in training to your daily work? (Training needs to be reinforced through on-the-job practice within a reasonable amount of time)
  • What tasks that were covered in training do you still struggle with? Would further training or mentoring help?
  • What new tasks are you doing on the job now that were never covered in training? Would additional training or mentoring help?

A sure way to reduce the impact of training is to send the employee right back to doing what he was doing before the class, with no follow-up. While training providers should do their part with at least the first two levels of assessment (some even provide Level III), there is also an expectation on management to make sure the training sticks.

There is one other level of assessment we haven't covered--that is Level IV, which measures the business impact of training (customer satisfaction/retention, profitability, employee turnover rates, etc.) We'll save that one for another article.