Illumination Works (ILW) recently conducted a live webinar featuring augmented reality (AR) and internet of things (IoT) software to perform pre-flight helicopter inspections.
Imagine you’re a new Army helicopter pilot. Wouldn’t it be great to simply point your iPad at the helicopter and leverage a robust AR application that shows you the steps to perform for your pre-flight inspection and capture the real-time inspection data? This app does just that!
We sat down with Max Fritzhand, ILW AR/IoT Designer/Developer, to dive into the interesting aspects of this project.
Tell us about the Blackhawk pre-flight inspection AR application
Our AR pre-flight procedural checklist application allows a helicopter pilot to view specific parts of a Blackhawk helicopter via highlighting items or objects. The app provides animated and meaningful procedures for understanding how to do a specific task on a pilot’s to-do list leveraging an AR experience.
In the demo, we show the different instruction content types for how information can be presented to the pilot leveraging the checklist. What’s really cool is the integration of training and performance aides such as photos, how-to videos, and technical documentation for the pilot to reference. Upon executing the checklist steps, the AR software collects the data inputs from the pilot and sends them to the ThingWorx IoT software.
What are some of the benefits of this technology?
For this particular use case, this technology can reduce the time it takes to train new pilots and reduce training costs. The app can also assist new pilots by showing them how to perform the pre-flight inspection steps correctly, such as where to stand to inspect properly, and provide quick access to electronic instructions such as manuals and how-to videos. This tool can provide consistent, streamlined safety measures for pilots.
Can you delve a little deeper into the tech?
To develop this application, we leveraged a Blackhawk helicopter CAD model and PTC AR and IoT software. Vuforia Studio is the main backbone of the demo, providing a powerful web-native AR tool built on an ionic framework utilizing Angular JavaScript. Vuforia View is the application that allows the pilot to view the 3D content of IoT data on devices such as an iPad, HoloLens 2, and Realwear. Creo Illustrate is the 3D CAD software we used to create the illustrations that were imported into Vuforia Studio. We also leveraged the ThingWorx IoT platform, which integrates with the Studio AR and dynamic graphs.
Tell us more about the capabilities of Vuforia Studio
Vuforia Studio is easy to use. Studio is a simple user interface and is scalable to deploy and get out to people very seamlessly and easily. This tool captures data using an iPad and AR software, which writes back to and pulls from an IoT software like ThingWorx.
Within Vuforia Studio, there are things called targets, which provide a recognition system for AR apps and provides context of the world it is interpreting. There is ThingMarx Target, Spatial Target, and Model Target. ThingMarx is like a QR code, where you take your device and scan a code and it provides you with meaningful data. Spatial Target is like a 3D object on a planar surface, and Model Target is essentially a digital twin overlay on the physical twin. These capabilities enable the creation of a very robust AR experience.
The architecture behind Vuforia Studio is a very powerful system. With the Blackhawk demo, we utilize Creo and Creo Illustrate to create the 3D object and the animations of propellers showing different sequences of events happening. We are able to import that into Studio to have that information within the CAD object and provide widgets and IoT data and information pulled from ThingWorx.
We utilize media, such as video and PDFs, to have meaningful, insightful information to know about a specific part of the helicopter. Once the individual wants to publish, the “publish” button is clicked within Studio and it gets pushed to the Studio Experience Service. Once it’s there, it will render and load, and be pushed to Vuforia View where the individual will be able to process and use the creation on devices like an iPad or a HoloLens.
What are next steps for this project?
Our next steps include a managerial view for the pre-flight inspection. We are looking at ways for a manager to be able to view the work that a pilot submitted, like the photos and additional logs of the checklist. In addition, this capability is currently demonstrated on an Army helicopter, but could easily be leveraged within other branches of the military, such as Air Force. It could also be used in the automobile industry as well the industrial machinery industry. Anything with complex rotary machinery that has complex parts that have to work a certain way or could face critical failure is a prime target for this technology.
About the Innovation Lab
ILW is a leading provider of technology consulting, and as such we actively strive to stay ahead of the technology curve. We have invested in an Innovation Lab on site in our Dayton, Ohio office led by our Chief Technology Officer, Jon Mitchell and supported by our Director of Research and Innovation, Dr. Rob Keefer. They have staffed the Innovation Lab with intellectually curious and technically capable team members who work on developing and testing the latest tools and technologies.
We choose which of the newest technologies to invest in based on research and client needs and use the Innovation Lab resources to prove out relevant use cases. In addition to the current staff, the Innovation Lab is available to team members who need to do training for current assignments or improve their skills while in between assignments.
About Max
Max Fritzhand is a problem-solving visual designer successful at creating a seamless user experience throughout entire product lines. He is confident, naturally curious, and extremely driven with five years’ experience in user interface design. His current areas of expertise are Vuforia, Creo, and ThingWorx to develop IoT and AR technologies.
Click here to watch the 30-minute on-demand webinar on the Blackhawk pre-flight inspection application.
Learn more about our AR and IoT offerings.
Reach out to Jon Mitchell or Jim Blough to learn how AR and IoT can benefit your business.
Join our mailing list like hundreds of other like-minded technology lovers!