Search

Digital twins have revolutionized manufacturing and maintenance, allowing us to interact with virtual yet realistic representations of the physical world in simulations to identify potential problems or opportunities for improvement. However, traditional digital twins do not have the ability to communicate with humans using natural language, which limits their potential usefulness. Although conventional natural language processing methods have proven to be effective in solving certain tasks, neuro-symbolic AI offers a new approach that leads to more robust and versatile solutions. In this paper, we propose neuro-symbolic reasoning (NSR)—a fundamental method for interacting with 3D digital twins using natural language. The method understands user requests and contexts to manipulate 3D components of digital twins and is able to read maintenance manuals and implement installations and removal procedures autonomously. A practical neuro-symbolic dataset of machine-understandable manuals, 3D models, and user queries is collected to train the neuro-symbolic reasoning interaction mechanism. The evaluation demonstrates that NSR can execute user commands accurately, achieving 96.2% accuracy on test data. The proposed method has industrial importance since it provides the technology to perform maintenance procedures, request information from manuals, and serve as a tool to interact with complex virtual machinery using natural language. [ABSTRACT FROM AUTHOR]

Corrosion identification and repair is a vital task in aircraft maintenance to ensure continued structural integrity. Regarding fuselage lap joints, typically, visual inspections are followed by non-destructive methodologies, which are time-consuming. The visual inspection of large areas suffers not only from subjectivity but also from the variable probability of corrosion detection, which is aggravated by the multiple layers used in fuselage construction. In this paper, we propose a methodology for automatic image-based corrosion detection of aircraft structures using deep neural networks. For machine learning, we use a dataset that consists of D-Sight Aircraft Inspection System (DAIS) images from different lap joints of Boeing and Airbus aircrafts. We also employ transfer learning to overcome the shortage of aircraft corrosion images. With precision of over 93%, we demonstrate that our approach detects corrosion with a precision comparable to that of trained operators, aiding to reduce the uncertainties related to operator fatigue or inadequate training. Our results indicate that our methodology can support specialists and engineers in corrosion monitoring in the aerospace industry, potentially contributing to the automation of condition-based maintenance protocols. [ABSTRACT FROM AUTHOR]

The signal-to-noise ratios (SNR) of ultrasonic imaging and non-destructive evaluation (NDE) applications can be greatly improved by driving each piezoelectric transducer (single or in array) with tuned HV capacitive-discharge drivers. These can deliver spikes with kW pulsed power at PRF ≈ 5000 spikes/s, achieving levels higher even than in CW high-power ultrasound: up to 5 kWpp. These conclusions are reached here by applying a new strategy proposed for the accurate modeling of own-design re-configurable HV capacitive drivers. To obtain such rigorous spike modeling, the real effects of very high levels of pulsed intensities (3–10 A) and voltages (300–700 V) were computed. Unexpected phenomena were found: intense brief pulses of driving power and probe emitted force, as well as nonlinearities in semiconductors, though their catalog data include only linear ranges. Fortunately, our piezoelectric and circuital devices working in such an intense regime have not shown serious heating problems, since the finally consumed "average" power is rather small. Intensity, power, and voltage, driving wideband transducers from our capacitive drivers, are researched here in order to drastically improve (∆ >> 40 dB) their ultrasonic "net dynamic range available" (NDRA), achieving emitted forces > 240 Newtonspp and receiving ultrasonic signals of up to 76–205 Vpp. These measurements of ultrasonic pulsed voltages, received in NDE and Imaging, are approximately 10,000 larger than those usual today. Thus, NDRA ranges were optimized for three laboratory capacitive drivers (with six commercial transducers), which were successfully applied in the aircraft industry for imaging landing flaps in Boeing wings, despite suffering acoustic losses > 120 dB. [ABSTRACT FROM AUTHOR]

Metaverses embedded in our lives create virtual experiences inside of the physical world. Moving towards metaverses in aircraft maintenance, mixed reality (MR) creates enormous opportunities for the interaction with virtual airplanes (digital twin) that deliver a near-real experience, keeping physical distancing during pandemics. 3D twins of modern machines exported to MR can be easily manipulated, shared, and updated, which creates colossal benefits for aviation colleges who still exploit retired models for practicing. Therefore, we propose mixed reality education and training of aircraft maintenance for Boeing 737 in smart glasses, enhanced with a deep learning speech interaction module for trainee engineers to control virtual assets and workflow using speech commands, enabling them to operate with both hands. With the use of the convolutional neural network (CNN) architecture for audio features and learning and classification parts for commands and language identification, the speech module handles intermixed requests in English and Korean languages, giving corresponding feedback. Evaluation with test data showed high accuracy of prediction, having on average 95.7% and 99.6% on the F1-Score metric for command and language prediction, respectively. The proposed speech interaction module in the aircraft maintenance metaverse further improved education and training, giving intuitive and efficient control over the operation, enhancing interaction with virtual objects in mixed reality. [ABSTRACT FROM AUTHOR]

To realize an early warning of unbalanced workload in the aircraft cockpit, it is required to monitor the pilot's real-time workload condition. For the purpose of building the mapping relationship from physiological and flight data to workload, a multi-source data fusion model is proposed based on a fuzzy neural network, mainly structured using a principal components extraction layer, fuzzification layer, fuzzy rules matching layer, and normalization layer. Aiming at the high coupling characteristic variables contributing to workload, principal component analysis reconstructs the feature data by reducing its dimension. Considering the uncertainty for a single variable to reflect overall workload, a fuzzy membership function and fuzzy control rules are defined to abstract the inference process. An error feedforward algorithm based on gradient descent is utilized for parameter learning. Convergence speed and accuracy can be adjusted by controlling the gradient descent rate and error tolerance threshold. Combined with takeoff and initial climbing tasks of a Boeing 737–800 aircraft, crucial performance indicators—including pitch angle, heading, and airspeed—as well as physiological indicators—including electrocardiogram (ECG), respiration, and eye movements—were featured. The mapping relationship between multi-source data and the comprehensive workload level synthesized using the NASA task load index was established. Experimental results revealed that the predicted workload corresponding to different flight phases and difficulty levels showed clear distinctions, thereby proving the validity of data fusion. [ABSTRACT FROM AUTHOR]