A food-processing manufacturer in Canada this year experienced production-stopping breakdowns of a bagging machine made in Germany and an X-ray machine made in Australia. Without augmented reality (AR) technology, repairs would have taken weeks, costing hundreds of thousands of dollars in lost production time. Instead, in each case, the subject-matter experts at the OEMs—linked to the front-line workers via computer, hands-free video tech and mixed-reality glasses—were able to help on-site technicians identify the problem and bring the equipment back on line within 30 minutes, said Kognitiv Spark CEO Yan Simard.
“Before the advent of mixed-reality devices, experts had to travel to the site and it could take days or weeks,” he said. “Now with the pandemic, travel often is not even an option. Being able to troubleshoot completely remotely is absolutely critical.”
Kognitiv Spark makes RemoteSpark software, which allows remote workers wearing smart glasses to establish low-bandwidth, secure, heads-up and hands-free video and audio calls.
Since 2019, augmented reality in assembly has turned the corner from proof of concept to prime time, proponents said. A key driver is HoloLens 2, Microsoft’s second-generation mixed reality smart glasses.
“AR adoption has definitely grown in leaps and bounds, especially in the last 18 months,” said Scope AR CEO Scott Montgomerie. His company makes WorkLink, a platform that combines AR work instructions and remote AR assistance.
With improved AR tech, a worker facing equipment failure can put on a HoloLens 2 headset, call a subject matter expert anywhere in the world and start trouble shooting.
The computer software sees what the front-line worker sees. The expert can drag and drop 2D and 3D files to show the worker holograms and animation illustrating how to do necessary repairs, Simard said.
One change in about the last 18 months, he said, is that now the operations side of manufacturing is asking for the technology.
In the past, IT departments investigated and bought tools like RemoteSpark, Simard said. “IT can evaluate the technology but they have to convince the operational groups to put it on workers’ heads. In the last 12 to 18 months, the people who reach out to us are the operations groups themselves. Now, it’s not going to be used as a proof of concept but will be used on the shop floor.”
Customers have told Scope AR that they’ve been able to reduce touch labor by more than 90 percent, diagnose problems 50 percent faster and improve the metrics on fixing a problem the first time to 100 percent, Montgomerie said.
Customers have told Kognitiv Spark that they have been able to save 75 percent of their budget for travel by subject matter experts.
“As soon as manufacturers are able to use it a few times, there’s a significant ROI,” Simard said.
“We’re at the point where the C suite and managers understand the viability and the metrics that AR brings to the business,” Montgomerie said.
The first-generation HoloLens, released in 2015, immediately caught the attention of some manufacturers, said Greg Sullivan, Microsoft’s director of mixed reality.
“People loved the idea of bringing the physical and digital worlds together in 3D,” he said. “We hoped to open new windows in manufacturing.”
Although early adopters were interested in the possibilities, they also noted issues, including discomfort, an off-balance center of gravity, lack of ease of use, the need for the user to move their head and neck for tracking and a smaller-than-desired field of view, he said.
Users did not like the unbalanced heft and the ring that clamped down on their heads, Sullivan said.
“It was like a Windows PC you wore on your head,” he said. “It was a little front-heavy—it didn’t have a perfectly centered mass—and could cause discomfort.”
The team working to improve comfort set a goal of 2x improvement and achieved 3x, measurable by numerous comfort metrics, Sullivan said.
HoloLens 2 is balanced, a little lighter, and has a padded brow.
Compared with the first iteration, HoloLens 2 “feels more natural,” Montgomerie said. Wearing the first one for hours was “really awkward,” he recalled. “It’s less weight now, and they’ve balanced it. The new one feels much lighter. It feels like wearing a heavy cowboy hat.”
Another significant improvement was overall ease of use.
The first-generation HoloLens enabled interaction with holograms through the user’s head and neck movements and so-called air tapping with one hand.
HoloLens 2 enables direct manipulation of holograms via hand-tracking and also supports eye tracking and voice commands as additional input models, Sullivan said.
“HoloLens 2 can navigate with your voice,” Montgomerie said. “It has really robust hand tracking. If you stare at a button long enough, it will click it for you. HoloLens 2 went from a basic tech device to a very nice user experience.”
The field of view on HoloLens 2 also is 2.5 times larger, Sullivan said.
“Think of wearing bifocals versus contact lenses,” Montgomerie said.
The first version was self-contained for computing power, Sullivan said, which made it impossible, for example, to render a 100 million-polygon CAD file.
HoloLens 2 runs on cloud-based software.
This change significantly boosts the computing power and enables users to work with larger, higher-resolution files, he added.
The more complex or complicated the task, the better the use case.
“The nature and complexity of aerospace assembly makes it so augmented reality is the most efficient way to communicate between a mechanical engineer who understands what has to happen and the front-line technician who needs to know what to do with their hands,” Montgomerie said.
Many use cases remain in the aerospace and defense sectors, as well as in life sciences and medical devices, he said.
Applications are emerging in automotive assembly for on-boarding and training—replacing thick binders full of training materials. Using augmented reality to train new hires is more effective because workers can see what they’re supposed to be doing, as opposed to reading about it, he said. “When a worker is looking at a car engine, you can overlay a diagram of the engine on top of it.”
Because training with AR is easier and more like the way that trainees are used to learning, customers have told Scope AR that they have improved employee retention rates by over 40 percent, Montgomerie said.
Or, consider a data center with numerous columns of machines with wires extending in multiple directions that has to be installed in a consistent way, he said. Having the AR visual overlay to show what piece goes exactly where and what wire needs to be pulled is very useful.
“We’re starting to see the advent of some very high-end, cloud-powered software that can be used to solve real industrial use cases,” Montgomerie said.
The pandemic illustrated the critical need to be able to link subject matter experts and front-line technicians, Simard said.
On the other hand, the pandemic prevented sales people from demonstrating the technology face to face.
“Companies are saying, ‘We’ve always known we need to adopt more advanced ways of supporting our workers. It was always something we’ll do later,’” he said. “To be able to maintain operations, they had to look at new ways of doing things. The pandemic accelerated the change and forced them to do it. There is no going back.”
The key challenge, Montgomerie and Simard said, is getting the initial buy-in. Once a manufacturer tries out the system, they are ready to buy more AR technology, he said.
“If we can get somebody to spend a month using it, they will definitely buy more,” he said.
Another challenge is security, especially because HoloLens 2 is now connected to the cloud.
“You have a camera pointed at what you’re working on that’s capable of capturing proof-of-work data more effectively than anything out there ,” Montgomerie said. “You’re also dealing with CAD models, some of the most important intellectual property a company has. Companies want to ensure these assets and information are safe in an encrypted and fully compliant solution, such as Scope AR’s WorkLink.”
Connect With Us
