Delta Sigma started thinking about augmented reality (AR) in 2005. By January 2008, the company, which designs and builds custom machinery to automate manufacturing and assembly processes for aircraft production, made its first installation on the F-22 vertical stabilizer. “And since that time, our augmented reality system, called ProjectionWorks, has been installed on 22 different aircraft production lines, in 31 different factories” in locations around the world, CEO Roger Richardson said during a panel talk at the recent AeroDef Manufacturing conference in California.
AR adoption is a no-brainer for many A&D manufacturers, he suggested.
Delta Sigma provided show attendees with a live simulation of work that actually happens, using augmented reality, on part of the F-35. The Air Force invested about $2.7 million in the project, which involves 78 projectors in 10 work cells at one location on the F35 center fuselage, and is now “receiving an $82 million return on that investment, in their total recurring flyaway cost,” Richardson said.
Scott Bryan, senior sustainment engineer with the US Air Force research laboratories, said his lab’s efforts to get the ProjectionWorks augmented reality system into the Air Force’s OEM manufacturing lines have resulted in “drastic improvements in throughput and quality.”
“We have a few ongoing projects to take this technology to the next level,” he added. “We are also looking at transferring it to the logistics environment for aircraft maintenance.”
AR devices like the HoloLens are now so affordable that he has seen some units buy them and start offering instructions to do aircraft maintenance, Bryan said. “From an enterprise approach, that makes us nervous because we have zero control over what the airmen are doing. But we do realize the utility. We’re trying to make sure that we can transition that across the Air Force in a smart manner.”
Lockheed Martin has developed AR pilots for maintenance on, for example, the F-35, Greg Harrison, senior systems engineer in Lockheed Martin’s rotary and mission systems division, said. AR allows for “two-handed maintenance” while pictures and other information that used to be communicated only in books is displayed before the technician’s eyes, as he needs it, on smart glasses.
Mechdyne, which specializes in large, immersive and interactive displays, has been using remote AR, which helps a junior technician troubleshooting a problem out in the field connect with a senior person back in the home office, said Kurt Hoffmeister, VP of R&D.
The senior technician can “see firsthand what the junior technician is looking at” and provide overlays of augmented reality onto the products that they’re talking about, he said. “And we tested it internationally. It works very well. It’s like a Skype for augmented reality.”
The AR studies Delta Sigma has done show an average 72% reduction in direct value-added labor, a 71% reduction in standard deviation across skill levels, and a 92% reduction in rework costs, Richardson said.
Bryan chimed in: “For instance, in the wiring harness manufacturing, I’ve heard evidence that there’s at least a 10-12% quality first time defect ratio. And the harness work system that we paid to have developed by Delta Sigma cut that error rate by 90%. And so, in generalities, I think you can expect a 30-60% improvement in through time, and about a 90% improvement in quality, first time quality.”
Bryce Notheis, systems engineer in Raytheon Missile Systems’ immersive design center operations unit, said manufacturers need to work hard to “bring in those people who are more maintenance minded and more manufacturing minded into the design phase.”
With virtual reality (VR), ”you’re able to walk around whatever part it is,” he added. “And the more stakeholders you can get to have that understanding of the design early on, the more … you can create products using these types of tools.”
Even small, simple desktop assembly operations can benefit, the moderator said. Sequenced steps are projected onto workbenches, for example, casting a green arrow toward the bin from which a worker needs to pick the next part, or a red X on the worker’s hand in the event he goes to the wrong bin.
Delta Sigma’s system allows manufacturers to project instruction out onto entire commercial airplanes, Richardson said. “There’s no size limitation to this.”
After the audience at the event heard how clear the incentives are for AR adoption, the moderator asked how small businesses can begin implementing the technology.
It is important to first realize that a complete mindset change is needed, Notheis said. “There’s a lot of affordable technology out there that you can apply once you’ve changed your culture” so that your company is approaching design, manufacturing and maintenance issues from “a virtual, augmented reality, computer-based approach.”
Everything will be done differently, Richardson said. “You’re not just going to take a paper process and directly convert it to augmented reality. A paper process has a lot of limitations in how information is conveyed, and augmented reality doesn’t have these limitations.”
The first step, he said, is to collect “the CAD model data that you have and ensure that you really understand your process – the one you really do.”
Forgetting about your former limits on how to convey the assembly information about the processes is important. With AR, it is relatively easy to convey what is in your mind into someone else’s mind.
Then it is a usually a good idea to run a pilot program—which can be done with a system leased from Delta Sigma.
“So somebody can try integrating our system into their process for, say, three months,” Richardson said. “There’s still going to be some upfront costs in addition to that, because you’re still going to have to have the data to do it. But you can run a pilot program for the low tens of thousands of dollars, almost all of which directly reduces the final implementation costs.”
Manufacturers can spend several million dollars on an AR implementation, Hoffmeister said. So his firm typically holds a discovery discussion with clients. “The return on investment is really key. If you have a bottleneck that’s costing you time, if you have a problem with rework, if you have an assembly process that you just wish was more efficient, identifying those problem areas self directs what you try first.”
Bryan suggested coming up a detailed definition of individual use cases—“whether that’s supply chain, kitting, or reach-back, field service support . Just do a detailed use case definition. And then get with the smart integrators” to find the right devices and software.
Harrison said he has been working for eight years to develop such things as optical lenses for augmented reality and head-mounted displays.
He noted it is important to make sure any company new to AR involve a digital art, or CAD, expert who knows how to take what is in a CAD library and “bring it into something like Unity, which is the major application nowadays.”
That step is necessary so that what is in the CAD library looks good in head wearables, for example. Although Delta Sigma takes the approach that any AR system used on an aircraft production line needs to use CAD model data in standard data formats used by aerospace companies.
Hoffmeister noted that Lockheed Martin has a computer-human interface lab in Denver that serves as “an example of a large-screen, VR-immersive display, as well as head-mounted displays being used together so that many participants can share the same virtual environment, observe or participate in a training or a procedure that’s being simulated.”
AR is perfect for production processes that are long and tedious and complicated, Richardson said, because AR feeds a technician, for example, a sequence of work instructions one at a time.
“On a drawing, there might be 100 steps, and you have to keep in your head which ones are done. And the technician is probably choosing the order in which he would do it,” he said. “With augmented reality, … you would only see one step at a time. In any moment, you’re only doing one thing so all the rest of that information is just noise.
“So, if you’ve got a production line where you’re rolling out an airplane say every 10 days, then everybody’s work is 10 days long, and it’s sufficiently complicated that it’s not safe to just say that you can memorize that. Particularly when you’ve got configuration changes or multiple variants. That’s where errors happen. AR virtually eliminates these errors.”
AR let Lockheed Martin’s rotary and mission systems division ditch its paper system for cabling on items on the manufacturing floor, Harrison said.
“It also provided a method to give much more information than you could get from a piece of paper. We found that in a lot of cases, paper instructions left out things. But when you had to go to an augmented reality experience, you had to provide enough to match reality. Because you’ve got the actual physical piece of hardware there, and you have to overlay it. Then you also have to be able to see it from different sides. So if you’re wearing something like the HoloLens and you go look at the device, you don’t just look at it from the one viewpoint that a picture might provide. You are able to move right in and look around and look behind it, look at the various connections, look at all the parts. So it requires much more detail, but it gives you more accuracy and a better chance of getting it right the first time.”
Cyber security is certainly a complication, Richardson said.
“We have a lot of systems that are stand alone, and they usually stand up and are working in a few hours or a day. As soon as you involve the IT department and you start going to an enterprise-wide system, just moving that data around in a way that keeps your assembly process safe becomes more complicated. And you need to account for the time that that’s going to take.
“If you’re going to put in an enterprise-wide system, don’t wait until installation day to start talking to your IT department. Start that far ahead of that process.”
Lockheed Martin has a lot of security in its systems, Harrison said.
“We found it very difficult to communicate the augmented reality information across our network. It took us many months, many meetings with security” to sort out such issues as which firewalls need to remain closed and which can be opened, he said.
“In some cases, just putting the devices on the network is not allowed because they’re not fully qualified” and require security reviews, Harrison said. “So that’s still ongoing. I don’t think anybody has really come up with a good set of head wearables that are secure yet.”
Connect With Us
