Training now involves VR and AR for humans and cameras, digital twins and a touch of AI for robots
Brian Murphy has a big job. As the operation learning and development manager at Raytheon Missiles and Defense in Tucson, Ariz., he has been tasked with redesigning and implementing a learning program for every single operations position within the company. That means he and his team must find a way to teach everyone from operations engineers to operations professionals and leaders the skills needed to do their jobs safely, effectively and, above all, securely.
This last part is vital, given that he and the other 30,000 employees of the Raytheon Technologies subsidiary spend their days designing, testing and manufacturing the military defense systems that help keep the U.S. safe. These include the weapons to engage in air, land and naval warfare, protection against missiles and unmanned aircraft and solutions for a more recent threat: hypersonic weaponry. The stakes are high, to say the least.
Murphy was given five years to complete his mission. Eighteen months into it, he has discovered that traditional teaching methods are often better than newer, more high-tech approaches. “It’s easy to get caught up in technology solutions and think that, because they’re the latest and greatest, they’re automatically the best way to go about training people,” he said. “In some cases, they can be. But first you have to evaluate the job and its various needs—and then tailor the training to those requirements, irrespective of the technology used.”
As a result, he’s taken an approach that many parents and students will recognize, thanks to the past year of COVID-restricted schooling: blended learning.
Blended learning relies on a combination of in-person classes, self-learning and interactive online tools. And it is proving to be the most efficient solution for training thousands of Raytheon employees—or any manufacturing company’s employees, for that matter.
There is, of course, plenty of room in Murphy’s curriculum for technology-driven training methods, with virtual reality (VR) and other forms of simulation chief among them.
To be effective, it must be used in a manner that addresses the job’s unique requirements. For example, VR training makes good sense for an engineer who will be assembling or troubleshooting a piece of hardware. On the other hand, a software engineer might be better served by participating in a think tank or other collaborative environment, where she can work with others trying to push the envelope on coding and programming.
Murphy sees many opportunities for VR and simulation.
“Both are great technologies,” he said. “In fact, the latter is probably the most powerful training tool available to us when teaching various soft skills. We might present an employee with a series of emails and instant messages to make them feel the chaos of the day, for instance, which helps them develop prioritization and decision-making skills. It’s about putting them in a safe, virtual environment where they can learn to make mistakes without damaging the business or the company brand.”
Addressing the skills gap issue
Raytheon Missiles and Defense is concerned about the fact that many skilled manufacturing workers are reaching retirement age. So Murphy is working on capturing as much of their knowledge as possible before they head off to experience some well-deserved R&R.
Jon Newman, Vice President of product management at Librestream Technologies, is right there with Murphy: The Canadian firm developed the Onsight Augmented Reality Platform, in part to help with this situation.
Augmented reality is an excellent way to record highly experienced workers’ knowledge ahead of their retirement, Newman said.
“Every manufacturing company has that someone with decades of experience who can walk up to a piece of equipment, put his or her hand on it, and just by feeling the temperature or vibration know whether there’s a problem or not,” he said. “Our objective is to develop ways to record such worker activities, then apply advanced technologies like AI, computer vision and natural language processing to extract the relevant information and convert it into an augmented-reality experience.”
For the moment, training experts create AR and VR sessions by constructing virtual environments and inserting text, diagrams and similar learning material where appropriate.
This has been the task of Chad Schron and his colleagues at Tooling U-SME—who recently piloted a VR-based learning program set to hit the market later this year.
“A typical manufacturing training solution uses what’s called a competency model,” said Schron, senior director at Tooling U-SME (a unit of SME, which also publishes Smart Manufacturing). “It contains everything a person needs to know to do their job, whether you’re a welder, machinist or mechatronics assembler.
“Students can learn much of this from our digital e-learning classes and then pick up the hands-on skills from on-the-job training programs, apprenticeships and so on. Our new virtual reality labs can be thought of as a bridge between the two. Students will get up to speed with our e-learning, then go into a VR world and apply what they’ve learned.”
Such an approach reduces risk to the employer while getting the novice up to speed more quickly.
Students can experience a wide range of scenarios in a short amount of time, providing them with troubleshooting skills that might otherwise take years to develop.
Rather than crashing an expensive machine tool or scrapping a batch of parts, they can make these mistakes virtually, over and over again until the lesson is learned.
This approach also cuts costs by allowing high schools and vocational-technical centers to introduce students to equipment they might not otherwise have safe access to, resulting in more fully trained students with on hands-on skills at the end of the program.
“As manufacturing grows more complex and the skills gap gets ever wider, the need for advanced training is clear,” Schron said. “The data on using AR and VR as part of an overall blended solution indicates tremendous benefits, particularly in manufacturing due to the safety implications and the need for greater comprehension. We’ve also seen that AR/VR learning is very immersive and fun to use. Considering that the next generation of workers grew up with video games and the Internet, the timing for these learning tools is quite fortunate.”
You could say this USC group is writing 'Manufacturing for Robots'
Let’s face it: Robots are dummies. Without humans around to walk them through every tedious, mind-numbing task, they would be nothing but cool-looking, albeit expensive, paperweights.
Satyandra Gupta knows this better than anyone, which is why he spends his days thinking of ways to give robots the same decision-making capabilities as their human operators, thus freeing the operators to do more interesting and important things like play the piano or spend more time with their kids.
He’s getting close.
A mechanical engineering and computer science professor at University of Southern California (USC), Gupta has been leading a group developing a technology with the obvious name of “smart robotics.”
He offers the example of a panel-sanding robot.
Rather than teaching or programming the robot to move from one end of the workpiece to the other, applying a specific amount of pressure as it goes, Gupta can—because advanced vision systems, accurate 3D models of the workpiece and an ever-increasing amount of artificial intelligence—give his robot the instructions, “Here’s a panel. Go sand it.”
The robot would then scan the panel with its own “eyes,” much as a human would, figure out the best way to complete its task, and monitor its performance as it goes.
With Gupta’s blessing, two of his students have taken the robotic ball and run with it by forming the company GrayMatter Robotics. Their first product is just as described, the Scan&Sand surface finishing robot, said to reduce human labor and health risks, increase production capacity and improve product quality—largely on its own.
This last part is important. Benefits aside, none of this means humans can take a permanent vacation.
“No matter how smart the robot, it will always need some level of guidance or support from humans,” Gupta said. “Perhaps the sanding disc isn’t working well, or the camera gets fogged up, or any of 100 other situations that a human will know how to handle but a robot won’t.
“What’s important here is that the robot is smart enough to recognize it’s in trouble and then ask for help,” he added. “It should also learn through experimentation and interact efficiently with humans in an intelligent way. That’s how we see this technology evolving.”
Maintenance also made more efficient with smart tech
Librestream Technologies is applying its Onsight AR platform to field maintenance issues—and saving manufactures a ton of time. The software lets service technicians connect remotely for expert support, collaboration and access to corporate work instructions and knowledge bases.
“Renault Trucks is using Onsight to support its network of service centers in the U.K. and Ireland,” Librestream’s Jon Newman said. “If a mechanic is servicing a vehicle and runs into a problem, he or she can very easily connect to one of Renault’s support technicians via an augmented reality (AR) headset. The mechanic can share a video of what they’re seeing, and the remote technician can then walk them through the necessary troubleshooting steps. It’s also easy to send schematics and parts lists, and document the session for training others in the future.”
Renault’s service centers have since reported that there is no longer a need to wait for an expert to travel on site, allowing their mechanics to complete procedures that once took 24 to 48 hours in as little as 20 to 30 minutes.
The system has cut the need for in-person warranty inspections five-fold, significantly decreasing travel costs and associated greenhouse gases while getting vehicle operators back behind the wheel sooner, he said.
Raytheon Technologies also uses Onsight’s platform—to securely support remote technicians servicing drones and other military hardware, Newman said.