Augmenting maintenance tech training with AR

Feb. 21, 2020
AR is on the cusp of becoming a viable solution to enhance the quality and speed up maintenance and repair in the trucking sector.

Shadowing. Interning. Mentoring. These are all great ways for one-on-one and education and training. Large-scale problems require large-scale solutions and one-on-one won’t cut it.

This is the digital age, and the need to increase quality technicians requires scalable high-tech solutions. The most approachable ones appear to harness augmented reality (AR). Unlike virtual reality, which is 100 percent immersive, an AR application lays digital images over the real world to provide additional information about the environment. Putting on smart glasses typically is the preferred method for industrial applications, as this leaves your hands free to do work, but smartphones and tablets are more than capable interfaces.

While the technology hasn’t often been used in trucking, that looks like it’s all about to change as the technology becomes more popular. And it’s already being used to upskill truck technicians to maintain increasingly complex equipment. By the end of the decade, 25 percent of the vehicle market worldwide will be electrified, according to Ken Davis, founder of Greentree Advisors LLC.

Training boost

One company, Design Interactive, developed a training platform called AugMentor that enables an OEM, fleet or shop to create custom AR training content.  This includes work instructions for brake inspections, tire analysis or a full PM. The developer can help customers build their app.

“Using AR and the Microsoft HoloLens, the content is provided at the point of need, right at the tractor,” said Matt Johnston, division head of commercial solutions at Design Interactive. “While you can use it in a classroom or on training equipment, the training or procedure content can be provided to a technician right at the tractor or trailer in the bay itself.”

The HoloLens projects digital parts and text boxes in the user’s field of view, and the user can interact with them using hand gestures. QR codes on equipment can be set up to automatically provide contextual information to assist the worker. Cameras can take pictures of various stages of the procedure to document everything was done properly, and data can be instantly sent to supervisors to track a tech’s training progress and use as paperless work documentation.

AugMentor software running on a HoloLens can provide a more immersive user experience for technicians. Screengrab: Design Interactive

“Consider a technician self-guiding through a procedure using an AR or VR application as opposed to taking that same technician off the floor, putting them in a classroom and having a supplier or dealer train them,” Johnston said. “Now multiply that technician by the number in a fleet. You see the cost.”

For now, the industrial-minded HoloLens 2 might be too steep of an investment, starting at $3,500 (which doesn’t count the $125/month cost per user for the optional Dynamics 365 Remote Assist).

“I am not suggesting a one-to-one replacement, but a supplement whereby you can focus the classroom and one-on-one mentoring on your most difficult challenges and leverage AR or VR such that you realize a return on your training investment,” Johnston said.

Virtual expertise

The $2,500 RealWear HMT-1 is a more affordable, ruggedized option that provides work instructions or video through a monocular screen while also streaming their point of view to an expert off-site or downloading the information for future reference. The headset also works like a phone. It’s completely voice-operated and often used for handsfree applications used at refineries, factories, and on power infrastructure.

The relatively new company, launched in 2017, has achieved lightning-fast success and year-over-year growth of 500 percent.

“Transportation may be our biggest area in 2020,” said RealWear CEO Andy Lowery in an effort to “scrape every penny of profit” because margins are so thin. “When they find something that truly moves that process improvement needle, they move much, much faster than other industries.”

The HMT-1, which can slash truck inspection time by half, as evidenced by user data from Schlumberger, an oil field equipment company that previously used tablets to inspect fracking fluid trucks on the job site. By not having to take off work gloves and always having their hands free, workers cut the process from 20 to 10 minutes.

For serious trucking downtime issues, the results have been even more dramatic.

“We’re seeing things like a 75 percent reduction in troubleshooting and fixing trucks that are particularly ornery,” said the former Raytheon chief engineer and retired Naval officer. He said that data comes from BMW, which agreed to use the device in all 347 BMW Centers and MINI dealers in the U.S.

The HMT-1 maximizes the productivity of a limited amount of master mechanics, who Lowery noted “are harder and harder to come by", by having lower-level techs wear the HMT-1, which allowed one expert to see through the eyes of workers at five disparate dealerships in a day without leaving their workstation.

Lowery said DAF Trucks, Renault, and Volvo Group are all testing the technology in Europe.

Volvo Group Trucks Central Europe used the HMT-1 to maximize the productivity of its limited amount of product quality engineers. Less experienced techs wearing the HMT-1 and running the LibreStream Onsight Connect remote mentor solution can provide the engineer an immediate real-time view of a diesel engine, and from a  laptop, that expert can telestrate screengrabs, load video and diagrams, or simply talk through what needs to be done to complete a task or diagnose a problem. 

Lowery could not name the trucking company due to an NDA but did have one example that proved how valuable the device could be.

“For three weeks, these guys had a dead truck not pulling any precious cargo—they couldn’t figure it out,” Lowery recalled. The company sent an HMT-1 by FedEx to the shop “and with a 15-minute video and phone call session, they had figured it out.”

It turned out a red and blue wire were swapped. Three other minor issues were noted by the virtual engineers to improve operation.

Lowery envisions a day when truckers will have these in their cab for breakdowns, as that wiring problem could have easily been fixed by a driver on the side of the road, preventing all that lost uptime.

He noted that when more trucks become electric, even many current master techs will need help diagnosing certain issues. He likened what fleets could face was the same as what early Tesla owners had to endure: waiting months for a service appointment.

“That will be the fallout if we don’t take an aggressive position on video assist and on-the-job training with virtual experts by your side,” he warned.

The economical alternative

The problem with all bleeding-edge tools is that they cut too deeply into small and mid-sized fleets’ budgets.

“People will embrace technology, but they don’t want to be on the front end of it because the way development goes, if you buy it first, you’re going to be obsolete before the check clears,” said Tyson Sontag, a master mechanic at McKee Foods who was also the TMC SuperTech champion in 2006.

It may be a few years before the aforementioned gadgets are widely available.  They should be ready by the time the last of the baby boomers reach age 65 in 2030.

Sontag, who teaches high school diesel tech in Gentry, Ark., said the digital natives he teaches will excel at using the new technology, as they have grown up with it.

“If my dad, [also a diesel mechanic], picks up that monocular device, he’ll say, ‘What the heck kind of Star Wars is this?’’’ Sontag said. “One of the kids in my classes would say, ‘Wow, let me try that.’ So that’s kind of the biggest advantage kids have nowadays: their adaptability.”

You shouldn’t wait for prices to come down. While the experienced elders are still on site, have someone record that soon-to-be retiree doing all the PMs for a week and save them on an external hard drive or load to a private YouTube channel. Anyone can watch via phone or tablet before they begin the work, and when the price of smartglasses is within your budget, they can stream the video in a handsfree way while they perform the task. The result will be faster, quality work, and you invested maybe a few hundred dollars.

Because if you don’t, your competitor will, drawing the best candidates away.

“It is inevitable,” Johnston said. “I think a lot of firms, particularly larger firms, realize this. AR, in particular, is entering the consumer markets. The next-generation technician is growing up with this technology as fairly ubiquitous. It will be expected.”    

Sponsored Recommendations

ADAS Case Study: From 10 Calibrations a Month to Over 10 A Day

Originally published by Vehicle Service Pros, March 26, 2024

Snap-on Training: Approach to Intermittent Problems

Snap-on's live training sessions can help you develop your own strategy for approaching vehicle repair.

Snap-on Training: ADAS Level 2 - Component Testing

The second video for Snap-on's comprehensive overview of Advanced Driver Assistance Systems (ADAS), covering the fundamental concepts and functionalities essential for automotive...

Snap-on Training: Intro to ADAS

Snap-on's training video provides a comprehensive overview of Advanced Driver Assistance Systems (ADAS), covering the fundamental concepts and functionalities essential for automotive...

Voice Your Opinion!

To join the conversation, and become an exclusive member of Vehicle Service Pros, create an account today!