Extended reality (XR) has burst onto the scene in the last few years, offering us amazing visualization possibilities in myriad application areas along the aerospace supply chain. While there has been explosive growth in this space of late, there is actually a very rich history between XR technologies, aerospace and Boeing.
Tom Caudell and David Mizell coined the term “augmented reality” (a sub-type of XR) while experimenting in the early 1990s with better ways to manufacture wiring looms on a then-new Boeing plane called the 777.
They set out to enable mechanics to build the wiring looms on peg boards without having to look at reference drawings or use other jigs. Wearing something akin to what are now called “smart glasses,” the mechanic could at once look at the peg board and digital wiring drawings overlaid onto it. This let the mechanic simply trace the digital image to build the looms.
While the concept successfully demonstrated the capability, and laid the foundation for future work in this area, the technology needed further development to make it more usable.
One way to help clarify the various technologies in the extended reality collection is to differentiate them by how much immersion each leverages in doing what it does. (The Shades of Reality graphic on the following page describes the XR technology continuum based on seminal work by Paul Milgram called the Virtuality Continuum.)
Boeing has made great progress since those pioneering efforts, and the company continues to explore the use of XR.
Today, Boeing is exploring, and in some cases leveraging, these emerging technologies in many business units, including: commercial, defense, space and security, and global services. Applications include delivery of work instructions for assembly, repair and maintenance, training, design reviews and product “visioneering.”
As part of these efforts, Boeing collaborates with universities and industry leaders.
One such effort, a collaboration between Boeing and Iowa State University in 2013, offered Boeing significant insight on several key aspects of the potential benefits of use of XR in production.
The Iowa State team, which excels with human computer interaction systems, worked with Boeing to develop an optimal user interface for Boeing’s AR system. Additionally, the university team ran a study comparing the assembly time of a mock wing under various modes of instruction.
The study compared printed desktop model-based instructions (MBI), tablet MBI instructions, and augmented realty instructions and their impact on assembly time, errors and quality of the build.
Results showed that users of the AR system required less time to complete the assembly—and made significantly fewer assembly errors. Overall, the study witnessed an almost 90 percent improvement in first-time quality between desktop and augmented reality modes, with AR reducing time to build the wing by about 30%. Researchers also found that when instructions are presented with augmented reality, people gain a faster understanding and need less convincing of the correctness of tasks.
The bottom line is that this study shows and quantifies how complex tasks performed for the first time can benefit from augmented reality work instructions. If the task is done faster with fewer errors, the impact on productivity is highly significant.
Paul Davies, Boeing associate technical fellow, called the numbers “very exciting,” noting that the company rolls about two brand new commercial airplanes out of the factory a day. “If we could just reduce the number of errors by a small percentage, this is huge for us.”
Boeing continues to explore the use of XR throughout the company to address myriad tough and pervasive business problems in a variety of areas, including:
For training, Boeing is pursuing a number of use cases in this space, including building, repair and maintenance of commercial and military aircraft, as well as pilot training for vehicle and device operations.
In the commercial aircraft market, it is estimated that over the next 20 years there will be a need for approximately 600,000 new pilots, 600,000 new aircraft mechanics, and 800,000 air crew. Training the sheer volume of people needed to fill the 2 million-plus jobs in the aircraft industry will be expensive and time consuming, and it will require innovative new approaches to meet the demand for a market currently representing billions of dollars annually with steady, climbing growth predicted for the next two decades.
Recent advances in virtual reality technologies have made it possible to dramatically reduce overall costs in many areas of training. An example of this is the use of virtual reality flight simulation as a part of the training curriculum for certification of new pilots.
Preliminary results suggest the certification costs for training new pilots could be cut by as much as 70% by reducing full-motion simulator time and making it possible for student pilots to learn and practice fundamental piloting skills remotely.
Gate maintenance and “aircraft-on-ground” studies have shown that for commercial airlines, an “aircraft on ground” for unscheduled repair/maintenance can cost airlines a lot.
Boeing has conservatively estimated actual costs to an airline of an airplane-on-ground delay lasting one or two hours to be “at least $10,000, and as high as $150,000,” depending on the airplane model and operator, according to a Boeing Frontiers magazine article from 2011. With that kind of financial hemorrhaging, any time savings results in huge savings—in terms of money, avoiding reputation damage and further incurred expenses on behalf of airlines’ traveling customers.
By providing access to authoritative documents and files, as well as the ability to clock on and off of jobs, record operations and get remote assistance from others where needed, XR can play a huge role in mitigating the exorbitant costs of schedule disruption. XR can also assure a higher quality of operations and greater job satisfaction on behalf of those servicing the plane.
One of the most difficult and arduous jobs when constructing an airplane is that of an electrician.
The wiring of an average plane is literally many miles long, and when modifications are made to the base model configuration, it can lead to increases in complexity for engineering plans.
Additionally, mechanics traditionally printed the engineering drawings they needed for reference on plotter paper, reducing 3D drawings to a 2D representation that is tremendously more difficult to understand when installing a wiring harness in our 3D world.
Mixed reality (MR) is a perfect fit, technologically, for this circumstance, in that it enables the mechanic to see the original 3D digital engineering drawings indexed to, and over laid on, the inside of the airplane where he or she is installing the wiring. This dramatically decreases the cognitive load for the mechanics, making the wiring runs far easier to understand and complete and reducing overall task time, confusion and the need to search for the correct drawings.
To address the problem, Boeing pulled together a highly experienced, cross-discipline team of engineers, inspectors, electricians, IT architects, programmers, modelers and R&D employees to assure all aspects of the problem were well understood and accounted for. They started with 767 tanker wiring.
The team set about flow diagramming the process and performing a series of interviews and time studies. Once they understood the production goals and existing processes, they redesigned the process flows to eliminate all unnecessary steps—with the aid of XR technologies.
While XR technology is very impressive and offers us capabilities that were incapable before, the secret to success when leveraging XR technologies is their appropriate use, based on real business requirements that are co-discovered with end users by collaboratively interrogating their business processes.
The multi-disciplinary team Boeing assembled let the company “look at the entirety of the jobs in a different way, helping us to break out of older ways of thinking to become more innovative and efficient in our approach,” said Brian McCarthy, enterprise lean coach at Boeing. The company took a “system of systems” view that looked at each contributing function as co-dependent pieces of a larger puzzle—and considered systemic effects of change in any one part of the process on all other components.
Preliminary testing of the new XR-enhanced process provided overall process improvements of about 40%, and even better performance is expected with continual refinements going forward. The team was recognized as a runner up for Boeing’s annual “engineer of the year award.”
“Together we are changing the Boeing Company,” said Linda Hapgood, then the 767 aircraft systems chief engineer.
While Boeing has a rich heritage of XR innovation and development, it has only just begun to leverage the emerging capabilities of this rapidly developing technology area.
Future development underway for the XR system includes automated model decimation and creation of reusable modeling components.
The company is also working to develop a cross-platform OS and device-agnostic, integrative backbone architecture that will enable plug and play capabilities across all end-user devices, including headsets, smart phones, tablets and laptop computers.
These efforts will help mitigate “technical debt” caused by the plethora of devices and platforms in the market today—facilitating the ability to use XR devices and platforms interchangeabley based on use-case requirements.
By being able to leverage any technology, regardless of OS or form factor, Boeing will effectively future-proof its IT systems and dramatically eliminate technical debt—by streamlining the spin up new technologies and more gracefully retiring those whose utility has expired.
This backbone will also serve as the foundation for easier integrations of the Internet of Things (IoT), additive manufacturing, wearables, cloud computing and data analytics.
Through the effective use of XR technologies, and integration with other emerging technical capabilities meant to accelerate and maintain business efficiencies and capabilities, Boeing will propel itself into the next 100 years of aerospace excellence.
Connect With Us
