Search

This document is a summary of a special issue of the journal Sensors, titled "Emerging AI+X-Based Sensor and Networking Technologies including Selected Papers from ICGHIT 2022–2023." The special issue features selected papers from the 10th and 11th International Conferences on Green and Human Information Technology (ICGHITs), which were held in Korea and Thailand. The conferences focused on the theme of "Emerging Artificial Intelligent (AI)+X technology" and "Hyper Automation + Human AI" respectively. The selected papers cover various topics such as network security, routing protocols, signal detection, and clustering mechanisms, all incorporating AI-based methods. The issue also includes papers on topics like secure authentication, distance estimation in RFID systems, energy optimization in smart homes, blockchain technology, and radar signal detection. The authors emphasize the importance of both technology and humanity in advancing green and information technologies. [Extracted from the article]

In recent years, the wide application of deep neural network models has brought serious risks of intellectual property rights infringement. Embedding a watermark in a network model is an effective solution to protect intellectual property rights. Although researchers have proposed schemes to add watermarks to models, they cannot prevent attackers from adding and overwriting original information, and embedding rates cannot be quantified. Therefore, aiming at these problems, this paper designs a high embedding rate and tamper-proof watermarking scheme. We employ wet paper coding (WPC), in which important parameters are regarded as wet blocks and the remaining unimportant parameters are regarded as dry blocks in the model. To obtain the important parameters more easily, we propose an optimized probabilistic selection strategy (OPSS). OPSS defines the unimportant-level function and sets the importance threshold to select the important parameter positions and to ensure that the original function is not affected after the model parameters are changed. We regard important parameters as an unmodifiable part, and only modify the part that includes the unimportant parameters. We selected the MNIST, CIFAR-10, and ImageNet datasets to test the performance of the model after adding a watermark and to analyze the fidelity, robustness, embedding rate, and comparison schemes of the model. Our experiment shows that the proposed scheme has high fidelity and strong robustness along with a high embedding rate and the ability to prevent malicious tampering. [ABSTRACT FROM AUTHOR]

Contributions in the Field of Image Techniques Five papers in this Special Issue deal with image processing, including face images, microscopic images, and infrared images. Nowadays, with numerous sensors placed everywhere around us, we can obtain signals collected from a variety of environment-based sensors, including the ones placed on the ground, cased in the air or water, etc. M. Geremek and K. Szklanny in [[9]] investigated deep learning based detection of genetic diseases from face images, for 15 genetic disorders associated with facial dysmorphism. Image and video data analyzed in the presented papers include microscope images and infrared images, as well as face images. [Extracted from the article]

This study investigates the characteristics of a novel origami-based, elastomeric actuator and a soft gripper, which are controlled by hand gestures that are recognized through machine learning algorithms. The lightweight paper–elastomer structure employed in this research exhibits distinct actuation features in four key areas: (1) It requires approximately 20% less pressure for the same bending amplitude compared to pneumatic network actuators (Pneu-Net) of equivalent weight, and even less pressure compared to other actuators with non-linear bending behavior; (2) The control of the device is examined by validating the relationship between pressure and the bending angle, as well as the interaction force and pressure at a fixed bending angle; (3) A soft robotic gripper comprising three actuators is designed. Enveloping and pinch grasping experiments are conducted on various shapes, which demonstrate the gripper's potential in handling a wide range of objects for numerous applications; and (4) A gesture recognition algorithm is developed to control the gripper using electromyogram (EMG) signals from the user's muscles. [ABSTRACT FROM AUTHOR]

Sugarcane is an important raw material for sugar and chemical production. However, in recent years, various sugarcane diseases have emerged, severely impacting the national economy. To address the issue of identifying diseases in sugarcane leaf sections, this paper proposes the SE-VIT hybrid network. Unlike traditional methods that directly use models for classification, this paper compares threshold, K-means, and support vector machine (SVM) algorithms for extracting leaf lesions from images. Due to SVM's ability to accurately segment these lesions, it is ultimately selected for the task. The paper introduces the SE attention module into ResNet-18 (CNN), enhancing the learning of inter-channel weights. After the pooling layer, multi-head self-attention (MHSA) is incorporated. Finally, with the inclusion of 2D relative positional encoding, the accuracy is improved by 5.1%, precision by 3.23%, and recall by 5.17%. The SE-VIT hybrid network model achieves an accuracy of 97.26% on the PlantVillage dataset. Additionally, when compared to four existing classical neural network models, SE-VIT demonstrates significantly higher accuracy and precision, reaching 89.57% accuracy. Therefore, the method proposed in this paper can provide technical support for intelligent management of sugarcane plantations and offer insights for addressing plant diseases with limited datasets. [ABSTRACT FROM AUTHOR]

This paper discusses the problem of recognizing defective epoxy drop images for the purpose of performing vision-based die attachment inspection in integrated circuit (IC) manufacturing based on deep neural networks. Two supervised and two unsupervised recognition models are considered. The supervised models examined are an autoencoder (AE) network together with a multi-layer perceptron network (MLP) and a VGG16 network, while the unsupervised models examined are an autoencoder (AE) network together with k-means clustering and a VGG16 network together with k-means clustering. Since in practice very few defective epoxy drop images are available on an actual IC production line, the emphasis in this paper is placed on the impact of data augmentation on the recognition outcome. The data augmentation is achieved by generating synthesized defective epoxy drop images via our previously developed enhanced loss function CycleGAN generative network. The experimental results indicate that when using data augmentation, the supervised and unsupervised models of VGG16 generate perfect or near perfect accuracies for recognition of defective epoxy drop images for the dataset examined. More specifically, for the supervised models of AE+MLP and VGG16, the recognition accuracy is improved by 47% and 1%, respectively, and for the unsupervised models of AE+Kmeans and VGG+Kmeans, the recognition accuracy is improved by 37% and 15%, respectively, due to the data augmentation. [ABSTRACT FROM AUTHOR]

Fishing nets are dangerous obstacles for an underwater robot whose aim is to reach a goal in unknown underwater environments. This paper proposes how to make the robot reach its goal, while avoiding fishing nets that are detected using the robot's camera sensors. For the detection of underwater nets based on camera measurements of the robot, we can use deep neural networks. Passive camera sensors do not provide the distance information between the robot and a net. Camera sensors only provide the bearing angle of a net, with respect to the robot's camera pose. There may be trailing wires that extend from a net, and the wires can entangle the robot before the robot detects the net. Moreover, light, viewpoint, and sea floor condition can decrease the net detection probability in practice. Therefore, whenever a net is detected by the robot's camera, we make the robot avoid the detected net by moving away from the net abruptly. For moving away from the net, the robot uses the bounding box for the detected net in the camera image. After the robot moves backward for a certain distance, the robot makes a large circular turn to approach the goal, while avoiding the net. A large circular turn is used, since moving close to a net is too dangerous for the robot. As far as we know, our paper is unique in addressing reactive control laws for approaching the goal, while avoiding fishing nets detected using camera sensors. The effectiveness of the proposed net avoidance controls is verified using simulations. [ABSTRACT FROM AUTHOR]

Shadow removal for document images is an essential task for digitized document applications. Recent shadow removal models have been trained on pairs of shadow images and shadow-free images. However, obtaining a large, diverse dataset for document shadow removal takes time and effort. Thus, only small real datasets are available. Graphic renderers have been used to synthesize shadows to create relatively large datasets. However, the limited number of unique documents and the limited lighting environments adversely affect the network performance. This paper presents a large-scale, diverse dataset called the Synthetic Document with Diverse Shadows (SynDocDS) dataset. The SynDocDS comprises rendered images with diverse shadows augmented by a physics-based illumination model, which can be utilized to obtain a more robust and high-performance deep shadow removal network. In this paper, we further propose a Dual Shadow Fusion Network (DSFN). Unlike natural images, document images often have constant background colors requiring a high understanding of global color features for training a deep shadow removal network. The DSFN has a high global color comprehension and understanding of shadow regions and merges shadow attentions and features efficiently. We conduct experiments on three publicly available datasets, the OSR, Kligler's, and Jung's datasets, to validate our proposed method's effectiveness. In comparison to training on existing synthetic datasets, our model training on the SynDocDS dataset achieves an enhancement in the PSNR and SSIM, increasing them from 23.00 dB to 25.70 dB and 0.959 to 0.971 on average. In addition, the experiments demonstrated that our DSFN clearly outperformed other networks across multiple metrics, including the PSNR, the SSIM, and its impact on OCR performance. [ABSTRACT FROM AUTHOR]

Due to the complexity of real optical flow capture, the existing research still has not performed real optical flow capture of infrared (IR) images with the production of an optical flow based on IR images, which makes the research and application of deep learning-based optical flow computation limited to the field of RGB images only. Therefore, in this paper, we propose a method to produce an optical flow dataset of IR images. We utilize the RGB-IR cross-modal image transformation network to rationally transform existing RGB image optical flow datasets. The RGB-IR cross-modal image transformation is based on the improved Pix2Pix implementation, and in the experiments, the network is validated and evaluated using the RGB-IR aligned bimodal dataset M3FD. Then, RGB-IR cross-modal transformation is performed on the existing RGB optical flow dataset KITTI, and the optical flow computation network is trained using the IR images generated by the transformation. Finally, the computational results of the optical flow computation network before and after training are analyzed based on the RGB-IR aligned bimodal data. [ABSTRACT FROM AUTHOR]

Among the numerous gaze-estimation methods currently available, appearance-based methods predominantly use RGB images as input and employ convolutional neural networks (CNNs) to detect facial images to regressively obtain gaze angles or gaze points. Model-based methods require high-resolution images to obtain a clear eyeball geometric model. These methods face significant challenges in outdoor environments and practical application scenarios. This paper proposes a model-based gaze-estimation algorithm using a low-resolution 3D TOF camera. This study uses infrared images instead of RGB images as input to overcome the impact of varying illumination intensity in the environment on gaze estimation. We utilized a trained YOLOv8 neural network model to detect eye landmarks in captured facial images. Combined with the depth map from a time-of-flight (TOF) camera, we calculated the 3D coordinates of the canthus points of a single eye of the subject. Based on this, we fitted a 3D geometric model of the eyeball to determine the subject's gaze angle. Experimental validation showed that our method achieved a root mean square error of 6.03° and 4.83° in the horizontal and vertical directions, respectively, for the detection of the subject's gaze angle. We also tested the proposed method in a real car driving environment, achieving stable driver gaze detection at various locations inside the car, such as the dashboard, driver mirror, and the in-vehicle screen. [ABSTRACT FROM AUTHOR]

This paper presents a computer vision-based approach to chronic subdural hematoma segmentation that can be performed by one click. Chronic subdural hematoma is estimated to occur in 0.002–0.02% of the general population each year and the risk increases with age, with a high frequency of about 0.05–0.06% in people aged 70 years and above. In our research, we developed our own dataset, which includes 53 series of CT scans collected from 21 patients with one or two hematomas. Based on the dataset, we trained two neural network models based on U-Net architecture to automate the manual segmentation process. One of the models performed segmentation based only on the current frame, while the other additionally processed multiple adjacent images to provide context, a technique that is more similar to the behavior of a doctor. We used a 10-fold cross-validation technique to better estimate the developed models' efficiency. We used the Dice metric for segmentation accuracy estimation, which was 0.77. Also, for testing our approach, we used scans from five additional patients who did not form part of the dataset, and created a scenario in which three medical experts carried out a hematoma segmentation before we carried out segmentation using our best model. We developed the OsiriX DICOM Viewer plugin to implement our solution into the segmentation process. We compared the segmentation time, which was more than seven times faster using the one-click approach, and the experts agreed that the segmentation quality was acceptable for clinical usage. [ABSTRACT FROM AUTHOR]

With the development of industrial manufacturing intelligence, the role of rotating machinery in industrial production and life is more and more important. Aiming at the problems of the complex and changeable working environment of rolling bearings and limited computing ability, fault feature information cannot be effectively extracted, and the current deep learning model is difficult to be compatible with lightweight and high efficiency. Therefore, this paper proposes a fault detection method for power equipment based on an energy spectrum diagram and deep learning. Firstly, a novel two-dimensional time-frequency feature representation method and energy spectrum feature map based on wavelet packet transform is proposed, and an energy spectrum feature map dataset is made for subsequent diagnosis. This method can realize multi-resolution analysis, fully extract the feature information contained in the fault signal, and accelerate the convergence of the subsequent diagnosis model. Secondly, a lightweight residual dense convolutional neural network model (LR-DenseNet) is proposed. This model combines the advantages of residual learning and a dense connection, and can not only extract deep features more easily, but can also effectively use shallow features. Then, based on the lightweight residual dense convolutional neural network model, an LR-DenseSENet model is proposed. By introducing the transfer learning strategy and adding the channel domain, an attention mechanism is added to the channel feature fusion layer, with the accuracy of detection up to 99.4%, and the amount of parameter calculation greatly reduced to one-fifth of that of VGG. Finally, through an experimental analysis, it is verified that the fault detection model designed in this paper based on the combination of an energy spectrum feature map and LR-DenseSENet achieves a satisfactory detection effect. [ABSTRACT FROM AUTHOR]

In this paper, we continue the research cycle on the properties of convolutional neural network-based image recognition systems and ways to improve noise immunity and robustness. Currently, a popular research area related to artificial neural networks is adversarial attacks. The adversarial attacks on the image are not highly perceptible to the human eye, and they also drastically reduce the neural network's accuracy. Image perception by a machine is highly dependent on the propagation of high frequency distortions throughout the network. At the same time, a human efficiently ignores high-frequency distortions, perceiving the shape of objects as a whole. We propose a technique to reduce the influence of high-frequency noise on the CNNs. We show that low-pass image filtering can improve the image recognition accuracy in the presence of high-frequency distortions in particular, caused by adversarial attacks. This technique is resource efficient and easy to implement. The proposed technique makes it possible to measure up the logic of an artificial neural network to that of a human, for whom high-frequency distortions are not decisive in object recognition. [ABSTRACT FROM AUTHOR]

Machine learning with deep neural networks (DNNs) is widely used for human activity recognition (HAR) to automatically learn features, identify and analyze activities, and to produce a consequential outcome in numerous applications. However, learning robust features requires an enormous number of labeled data. Therefore, implementing a DNN either requires creating a large dataset or needs to use the pre-trained models on different datasets. Multitask learning (MTL) is a machine learning paradigm where a model is trained to perform multiple tasks simultaneously, with the idea that sharing information between tasks can lead to improved performance on each individual task. This paper presents a novel MTL approach that employs combined training for human activities with different temporal scales of atomic and composite activities. Atomic activities are basic, indivisible actions that are readily identifiable and classifiable. Composite activities are complex actions that comprise a sequence or combination of atomic activities. The proposed MTL approach can help in addressing challenges related to recognizing and predicting both atomic and composite activities. It can also help in providing a solution to the data scarcity problem by simultaneously learning multiple related tasks so that knowledge from each task can be reused by the others. The proposed approach offers advantages like improved data efficiency, reduced overfitting due to shared representations, and fast learning through the use of auxiliary information. The proposed approach exploits the similarities and differences between multiple tasks so that these tasks can share the parameter structure, which improves model performance. The paper also figures out which tasks should be learned together and which tasks should be learned separately. If the tasks are properly selected, the shared structure of each task can help it learn more from other tasks. [ABSTRACT FROM AUTHOR]

Because of its long running time, complex working environment, and for other reasons, a gear is prone to failure, and early failure is difficult to detect by direct observation; therefore, fault diagnosis of gears is very necessary. Neural network algorithms have been widely used to realize gear fault diagnosis, but the structure of the neural network model is complicated, the training time is long and the model is not easy to converge. To solve the above problems and combine the advantages of the ResNeXt50 model in the extraction of image features, this paper proposes a gearbox fault detection method that integrates the convolutional block attention module (CBAM). Firstly, the CBAM is embedded in the ResNeXt50 network to enhance the extraction of image channels and spatial features. Secondly, the different time–frequency analysis method was compared and analyzed, and the method with the better effect was selected to convert the one-dimensional vibration signal in the open data set of the gearbox into a two-dimensional image, eliminating the influence of the redundant background noise, and took it as the input of the model for training. Finally, the accuracy and the average training time of the model were obtained by entering the test set into the model, and the results were compared with four other classical convolutional neural network models. The results show that the proposed method performs well both in fault identification accuracy and average training time under two working conditions, and it also provides some references for existing gear failure diagnosis research. [ABSTRACT FROM AUTHOR]

Software defect prediction studies aim to predict defect-prone components before the testing stage of the software development process. The main benefit of these prediction models is that more testing resources can be allocated to fault-prone modules effectively. While a few software defect prediction models have been developed for mobile applications, a systematic overview of these studies is still missing. Therefore, we carried out a Systematic Literature Review (SLR) study to evaluate how machine learning has been applied to predict faults in mobile applications. This study defined nine research questions, and 47 relevant studies were selected from scientific databases to respond to these research questions. Results show that most studies focused on Android applications (i.e., 48%), supervised machine learning has been applied in most studies (i.e., 92%), and object-oriented metrics were mainly preferred. The top five most preferred machine learning algorithms are Naïve Bayes, Support Vector Machines, Logistic Regression, Artificial Neural Networks, and Decision Trees. Researchers mostly preferred Object-Oriented metrics. Only a few studies applied deep learning algorithms including Long Short-Term Memory (LSTM), Deep Belief Networks (DBN), and Deep Neural Networks (DNN). This is the first study that systematically reviews software defect prediction research focused on mobile applications. It will pave the way for further research in mobile software fault prediction and help both researchers and practitioners in this field. [ABSTRACT FROM AUTHOR]

With the acceleration of urbanisation, urban areas are subject to the combined effects of the accumulation of various natural factors, such as changes in temperature leading to the thermal expansion or contraction of surface materials (rock, soil, etc.) and changes in precipitation and humidity leading to an increase in the self-weight of soil due to the infiltration of water along the cracks or pores in the ground. Therefore, the subsidence of urban areas has now become a serious geological disaster phenomenon. However, the use of traditional neural network prediction models has limitations when examining the causal relationships between time series surface deformation data and multiple influencing factors and when applying multiple influencing factors for predictive analyses. To this end, Sentinel-1A data from March 2017 to February 2023 were used as the data source in this paper, based on time series deformation data acquired using the small baseline subset interferometric synthetic aperture radar (SBAS-InSAR) technique. A sparrow search algorithm–convolutional neural network–long short-term memory (SSA-CNN-LSTM) neural network prediction model was built. The six factors of temperature, humidity, precipitation, and ground temperature at three different depths below the surface (5 cm, 10 cm, and 15 cm) were taken as the input of the model, and the surface deformation data were taken as the output of the neural network model. The correlation between the spatial and temporal evolution characteristics of the ground subsidence in urban areas and various influencing factors was analysed using grey correlation analysis, which proved that these six factors contribute to some extent to the deformation of the urban surface. The main urban area of Hohhot City, Inner Mongolia Autonomous Region, was used as the study area. In order to verify the efficacy of this neural network prediction model, the prediction effects of the multilayer perceptron (MLP), backpropagation (BP), and SSA-CNN-LSTM models were compared and analysed, with the values of the correlation coefficients of the feature points of A1, B1, and C1 being in the range of 0.92, 0.83, and 0.93, respectively. The results show that compared with the traditional MLP and BP neural network models, the SSA-CNN-LSTM model achieves a higher performance in predicting time series surface deformation data in urban areas, which provides new ideas and methods for this area of research. [ABSTRACT FROM AUTHOR]

This document provides summaries of various research papers published in the journal Sensors. The summaries cover a wide range of topics, including personalized medicine, laser ranging, memristive interfaces, face and person detection, IoT data visualization, monitoring elderly activities, auditory illusions, brain connectivity, corona discharges, thermal data analysis, vital signal detection, and eye-blink artifact removal. The document showcases the contributions and findings of each study, offering valuable insights for researchers and library patrons interested in these areas of study. [Extracted from the article]

This study aims to enhance diagnostic capabilities for optimising the performance of the anaerobic sewage treatment lagoon at Melbourne Water's Western Treatment Plant (WTP) through a novel machine learning (ML)-based monitoring strategy. This strategy employs ML to make accurate probabilistic predictions of biogas performance by leveraging diverse real-life operational and inspection sensor and other measurement data for asset management, decision making, and structural health monitoring (SHM). The paper commences with data analysis and preprocessing of complex irregular datasets to facilitate efficient learning in an artificial neural network. Subsequently, a Bayesian mixture density neural network model incorporating an attention-based mechanism in bidirectional long short-term memory (BiLSTM) was developed. This probabilistic approach uses a distribution output layer based on the Gaussian mixture model and Monte Carlo (MC) dropout technique in estimating data and model uncertainties, respectively. Furthermore, systematic hyperparameter optimisation revealed that the optimised model achieved a negative log-likelihood (NLL) of 0.074, significantly outperforming other configurations. It achieved an accuracy approximately 9 times greater than the average model performance (NLL = 0.753) and 22 times greater than the worst performing model (NLL = 1.677). Key factors influencing the model's accuracy, such as the input window size and the number of hidden units in the BiLSTM layer, were identified, while the number of neurons in the fully connected layer was found to have no significant impact on accuracy. Moreover, model calibration using the expected calibration error was performed to correct the model's predictive uncertainty. The findings suggest that the inherent data significantly contribute to the overall uncertainty of the model, highlighting the need for more high-quality data to enhance learning. This study lays the groundwork for applying ML in transforming high-value assets into intelligent structures and has broader implications for ML in asset management, SHM applications, and renewable energy sectors. [ABSTRACT FROM AUTHOR]

In cutting processes, tool condition affects the quality of the manufactured parts. As such, an essential component to prevent unplanned downtime and to assure machining quality is having information about the state of the cutting tool. The primary function of it is to alert the operator that the tool has reached or is reaching a level of wear beyond which behaviour is unreliable. In this paper, the tool condition is being monitored by analysing the electric current on the main spindle via an artificial intelligence model utilising an LSTM neural network. In the current study, the tool is monitored while working on a cylindrical raw piece made of AA6013 aluminium alloy with a custom polycrystalline diamond tool for the purposes of monitoring the wear of these tools. Spindle current characteristics were obtained using external measuring equipment to not influence the operation of the machine included in a larger production line. As a novel approach, an artificial intelligence model based on an LSTM neural network is utilised for the analysis of the spindle current obtained during a manufacturing cycle and assessing the tool wear range in real time. The neural network was designed and trained to notice significant characteristics of the captured current signal. The conducted research serves as a proof of concept for the use of an LSTM neural network-based model as a method of monitoring the condition of cutting tools. [ABSTRACT FROM AUTHOR]

The maturity of fruits and vegetables such as tomatoes significantly impacts indicators of their quality, such as taste, nutritional value, and shelf life, making maturity determination vital in agricultural production and the food processing industry. Tomatoes mature from the inside out, leading to an uneven ripening process inside and outside, and these situations make it very challenging to judge their maturity with the help of a single modality. In this paper, we propose a deep learning-assisted multimodal data fusion technique combining color imaging, spectroscopy, and haptic sensing for the maturity assessment of tomatoes. The method uses feature fusion to integrate feature information from images, near-infrared spectra, and haptic modalities into a unified feature set and then classifies the maturity of tomatoes through deep learning. Each modality independently extracts features, capturing the tomatoes' exterior color from color images, internal and surface spectral features linked to chemical compositions in the visible and near-infrared spectra (350 nm to 1100 nm), and physical firmness using haptic sensing. By combining preprocessed and extracted features from multiple modalities, data fusion creates a comprehensive representation of information from all three modalities using an eigenvector in an eigenspace suitable for tomato maturity assessment. Then, a fully connected neural network is constructed to process these fused data. This neural network model achieves 99.4% accuracy in tomato maturity classification, surpassing single-modal methods (color imaging: 94.2%; spectroscopy: 87.8%; haptics: 87.2%). For internal and external maturity unevenness, the classification accuracy reaches 94.4%, demonstrating effective results. A comparative analysis of performance between multimodal fusion and single-modal methods validates the stability and applicability of the multimodal fusion technique. These findings demonstrate the key benefits of multimodal fusion in terms of improving the accuracy of tomato ripening classification and provide a strong theoretical and practical basis for applying multimodal fusion technology to classify the quality and maturity of other fruits and vegetables. Utilizing deep learning (a fully connected neural network) for processing multimodal data provides a new and efficient non-destructive approach for the massive classification of agricultural and food products. [ABSTRACT FROM AUTHOR]

The rapid development of deep neural networks has attracted significant attention in the infrared and visible image fusion field. However, most existing fusion models have many parameters and consume high computational and spatial resources. This paper proposes a fast and efficient recursive fusion neural network model to solve this complex problem that few people have touched. Specifically, we designed an attention module combining a traditional fusion knowledge prior with channel attention to extract modal-specific features efficiently. We used a shared attention layer to perform the early fusion of modal-shared features. Adopting parallel dilated convolution layers further reduces the network's parameter count. Our network is trained recursively, featuring minimal model parameters, and requires only a few training batches to achieve excellent fusion results. This significantly reduces the consumption of time, space, and computational resources during model training. We compared our method with nine SOTA methods on three public datasets, demonstrating our method's efficient training feature and good fusion results. [ABSTRACT FROM AUTHOR]

The use of drones has recently gained popularity in a diverse range of applications, such as aerial photography, agriculture, search and rescue operations, the entertainment industry, and more. However, misuse of drone technology can potentially lead to military threats, terrorist acts, as well as privacy and safety breaches. This emphasizes the need for effective and fast remote detection of potentially threatening drones. In this study, we propose a novel approach for automatic drone detection utilizing the usage of both radio frequency communication signals and acoustic signals derived from UAV rotor sounds. In particular, we propose the use of classical and deep machine-learning techniques and the fusion of RF and acoustic features for efficient and accurate drone classification. Distinct types of ML-based classifiers have been examined, including CNN- and RNN-based networks and the classical SVM method. The proposed approach has been evaluated with both frequency and audio features using common drone datasets, demonstrating better accuracy than existing state-of-the-art methods, especially in low SNR scenarios. The results presented in this paper show a classification accuracy of approximately 91% at an SNR ratio of − 10 dB using the LSTM network and fused features. [ABSTRACT FROM AUTHOR]

NASA's Soil Moisture Active Passive (SMAP) was originally designed to combine high-resolution active (radar) and coarse-resolution but highly sensitive passive (radiometer) L-band observations to achieve unprecedented spatial resolution and accuracy for soil moisture retrievals. However, shortly after SMAP was put into orbit, the radar component failed, and the high-resolution capability was lost. In this paper, the integration of an alternative radar sensor with the SMAP radiometer is proposed to enhance soil moisture retrieval capabilities over vegetated areas in the absence of the original high-resolution radar in the SMAP mission. ESA's Sentinel-1A C-band radar was used in this study to enhance the spatial resolution of the SMAP L-band radiometer and to improve soil moisture retrieval accuracy. To achieve this purpose, we downscaled the 9 km radiometer data of the SMAP to 1 km utilizing the Smoothing Filter-based Intensity Modulation (SFIM) method. An Artificial Neural Network (ANN) was then trained to exploit the synergy between the Sentinel-1A radar, SMAP radiometer, and the in situ-measured soil moisture. An analysis of the data obtained for a plant growing season over the Mississippi Delta showed that the VH-polarized Sentinel-1A radar data can yield a coefficient of correlation of 0.81 and serve as a complimentary source to the SMAP radiometer for more accurate and enhanced soil moisture prediction over agricultural fields. [ABSTRACT FROM AUTHOR]

The rapid advancement in AI requires efficient accelerators for training on edge devices, which often face challenges related to the high hardware costs of floating-point arithmetic operations. To tackle these problems, efficient floating-point formats inspired by block floating-point (BFP), such as Microsoft Floating Point (MSFP) and FlexBlock (FB), are emerging. However, they have limited dynamic range and precision for the smaller magnitude values within a block due to the shared exponent. This limits the BFP's ability to train deep neural networks (DNNs) with diverse datasets. This paper introduces the hybrid precision (HPFP) selection algorithms, designed to systematically reduce precision and implement hybrid precision strategies, thereby balancing layer-wise arithmetic operations and data path precision to address the shortcomings of traditional floating-point formats. Reducing the data bit width with HPFP allows more read/write operations from memory per cycle, thereby decreasing off-chip data access and the size of on-chip memories. Unlike traditional reduced precision formats that use BFP for calculating partial sums and accumulating those partial sums in 32-bit Floating Point (FP32), HPFP leads to significant hardware savings by performing all multiply and accumulate operations in reduced floating-point format. For evaluation, two training accelerators for the YOLOv2-Tiny model were developed, employing distinct mixed precision strategies, and their performance was benchmarked against an accelerator utilizing a conventional brain floating point of 16 bits (Bfloat16). The HPFP selection, employing 10 bits for the data path of all layers and for the arithmetic of layers requiring low precision, along with 12 bits for layers requiring higher precision, results in a 49.4% reduction in energy consumption and a 37.5% decrease in memory access. This is achieved with only a marginal mean Average Precision (mAP) degradation of 0.8% when compared to an accelerator based on Bfloat16. This comparison demonstrates that the proposed accelerator based on HPFP can be an efficient approach to designing compact and low-power accelerators without sacrificing accuracy. [ABSTRACT FROM AUTHOR]

The Message Queuing Telemetry Transport (MQTT) protocol stands out as one of the foremost and widely recognized messaging protocols in the field. It is often used to transfer and manage data between devices and is extensively employed for applications ranging from smart homes and industrial automation to healthcare and transportation systems. However, it lacks built-in security features, thereby making it vulnerable to many types of attacks such as man-in-the-middle (MitM), buffer overflow, pre-shared key, brute force authentication, malformed data, distributed denial-of-service (DDoS) attacks, and MQTT publish flood attacks. Traditional methods for detecting MQTT attacks, such as deep neural networks (DNNs), k-nearest neighbor (KNN), linear discriminant analysis (LDA), and fuzzy logic, may exist. The increasing prevalence of device connectivity, sensor usage, and environmental scalability become the most challenging aspects that novel detection approaches need to address. This paper presents a new solution that leverages an H2O-based distributed machine learning (ML) framework to improve the security of the MQTT protocol in networks, particularly in IoT environments. The proposed approach leverages the strengths of the H2O algorithm and architecture to enable real-time monitoring and distributed detection and classification of anomalous behavior (deviations from expected activity patterns). By harnessing H2O's algorithms, the identification and timely mitigation of potential security threats are achieved. Various H2O algorithms, including random forests, generalized linear models (GLMs), gradient boosting machine (GBM), XGBoost, and the deep learning (DL) algorithm, have been assessed to determine the most reliable algorithm in terms of detection performance. This study encompasses the development of the proposed algorithm, including implementation details and evaluation results. To assess the proposed model, various evaluation metrics such as mean squared error (MSE), root-mean-square error (RMSE), mean per class error (MCE), and log loss are employed. The results obtained indicate that the H2OXGBoost algorithm outperforms other H2O models in terms of accuracy. This research contributes to the advancement of secure IoT networks and offers a practical approach to enhancing the security of MQTT communication channels through distributed detection and classification techniques. [ABSTRACT FROM AUTHOR]

LiDAR has high accuracy and resolution and is widely used in various fields. In particular, phase-modulated continuous-wave (PhMCW) LiDAR has merits such as low power, high precision, and no need for laser frequency modulation. However, with decreasing signal-to-noise ratio (SNR), the noise on the signal waveform becomes so severe that the current methods to extract the time-of-flight are no longer feasible. In this paper, a novel method that uses deep neural networks to measure the pulse width is proposed. The effects of distance resolution and SNR on the performance are explored. Recognition accuracy reaches 81.4% at a 0.1 m distance resolution and the SNR is as low as 2. We simulate a scene that contains a vehicle, a tree, a house, and a background located up to 6 m away. The reconstructed point cloud has good fidelity, the object contours are clear, and the features are restored. More precisely, the three distances are 4.73 cm, 6.00 cm, and 7.19 cm, respectively, showing that the performance of the proposed method is excellent. To the best of our knowledge, this is the first work that employs a neural network to directly process LiDAR signals and to extract their time-of-flight. [ABSTRACT FROM AUTHOR]

Accurate short-term load forecasting (STLF) is essential for power grid systems to ensure reliability, security and cost efficiency. Thanks to advanced smart sensor technologies, time-series data related to power load can be captured for STLF. Recent research shows that deep neural networks (DNNs) are capable of achieving accurate STLP since they are effective in predicting nonlinear and complicated time-series data. To perform STLP, existing DNNs use time-varying dynamics of either past load consumption or past power correlated features such as weather, meteorology or date. However, the existing DNN approaches do not use the time-invariant features of users, such as building spaces, ages, isolation material, number of building floors or building purposes, to enhance STLF. In fact, those time-invariant features are correlated to user load consumption. Integrating time-invariant features enhances STLF. In this paper, a fuzzy clustering-based DNN is proposed by using both time-varying and time-invariant features to perform STLF. The fuzzy clustering first groups users with similar time-invariant behaviours. DNN models are then developed using past time-varying features. Since the time-invariant features have already been learned by the fuzzy clustering, the DNN model does not need to learn the time-invariant features; therefore, a simpler DNN model can be generated. In addition, the DNN model only learns the time-varying features of users in the same cluster; a more effective learning can be performed by the DNN and more accurate predictions can be achieved. The performance of the proposed fuzzy clustering-based DNN is evaluated by performing STLF, where both time-varying features and time-invariant features are included. Experimental results show that the proposed fuzzy clustering-based DNN outperforms the commonly used long short-term memory networks and convolution neural networks. [ABSTRACT FROM AUTHOR]

This paper describes the design and optimization of a smart algorithm based on artificial intelligence to increase the accuracy of an ocean water current meter. The main purpose of water current meters is to obtain the fundamental frequency of the ocean waves and currents. The limiting factor in those underwater applications is power consumption and that is the reason to use only ultra-low power microcontrollers. On the other hand, nowadays extraction algorithms assume that the processed signal is defined in a fixed bandwidth. In our approach, belonging to the edge computing research area, we use a deep neural network to determine the narrow bandwidth for filtering the fundamental frequency of the ocean waves and currents on board instruments. The proposed solution is implemented on an 8 MHz ARM Cortex-M0+ microcontroller without a floating point unit requiring only 9.54 ms in the worst case based on a deep neural network solution. Compared to a greedy algorithm in terms of computational effort, our worst-case approach is 1.81 times faster than a fast Fourier transform with a length of 32 samples. The proposed solution is 2.33 times better when an artificial neural network approach is adopted. [ABSTRACT FROM AUTHOR]

Facial emotion recognition (FER) is a computer vision process aimed at detecting and classifying human emotional expressions. FER systems are currently used in a vast range of applications from areas such as education, healthcare, or public safety; therefore, detection and recognition accuracies are very important. Similar to any computer vision task based on image analyses, FER solutions are also suitable for integration with artificial intelligence solutions represented by different neural network varieties, especially deep neural networks that have shown great potential in the last years due to their feature extraction capabilities and computational efficiency over large datasets. In this context, this paper reviews the latest developments in the FER area, with a focus on recent neural network models that implement specific facial image analysis algorithms to detect and recognize facial emotions. This paper's scope is to present from historical and conceptual perspectives the evolution of the neural network architectures that proved significant results in the FER area. This paper endorses convolutional neural network (CNN)-based architectures against other neural network architectures, such as recurrent neural networks or generative adversarial networks, highlighting the key elements and performance of each architecture, and the advantages and limitations of the proposed models in the analyzed papers. Additionally, this paper presents the available datasets that are currently used for emotion recognition from facial expressions and micro-expressions. The usage of FER systems is also highlighted in various domains such as healthcare, education, security, or social IoT. Finally, open issues and future possible developments in the FER area are identified. [ABSTRACT FROM AUTHOR]

Forecasting energy consumption models allow for improvements in building performance and reduce energy consumption. Energy efficiency has become a pressing concern in recent years due to the increasing energy demand and concerns over climate change. This paper addresses the energy consumption forecast as a crucial ingredient in the technology to optimize building system operations and identifies energy efficiency upgrades. The work proposes a modified multi-head transformer model focused on multi-variable time series through a learnable weighting feature attention matrix to combine all input variables and forecast building energy consumption properly. The proposed multivariate transformer-based model is compared with two other recurrent neural network models, showing a robust performance while exhibiting a lower mean absolute percentage error. Overall, this paper highlights the superior performance of the modified transformer-based model for the energy consumption forecast in a multivariate step, allowing it to be incorporated in future forecasting tasks, allowing for the tracing of future energy consumption scenarios according to the current building usage, playing a significant role in creating a more sustainable and energy-efficient building usage. [ABSTRACT FROM AUTHOR]

This paper presents reported machine learning approaches in the field of Brillouin distributed fiber optic sensors (DFOSs). The increasing popularity of Brillouin DFOSs stems from their capability to continuously monitor temperature and strain along kilometer-long optical fibers, rendering them attractive for industrial applications, such as the structural health monitoring of large civil infrastructures and pipelines. In recent years, machine learning has been integrated into the Brillouin DFOS signal processing, resulting in fast and enhanced temperature, strain, and humidity measurements without increasing the system's cost. Machine learning has also contributed to enhanced spatial resolution in Brillouin optical time domain analysis (BOTDA) systems and shorter measurement times in Brillouin optical frequency domain analysis (BOFDA) systems. This paper provides an overview of the applied machine learning methodologies in Brillouin DFOSs, as well as future perspectives in this area. [ABSTRACT FROM AUTHOR]

In this paper, the authors investigate the possibility of applying artificial intelligence algorithms to the outputs of a low-cost Kalman filter-based navigation solution in order to achieve performance similar to that of high-end MEMS inertial sensors. To further improve the results of the prototype and simultaneously lighten filter requirements, different AI models are compared in this paper to determine their performance in terms of complexity and accuracy. By overcoming some known limitations (e.g., sensitivity on the dimension of input data from inertial sensors) and starting from Kalman filter applications (whose raw noise parameter estimates were obtained from a simple analysis of sensor specifications), such a solution presents an intermediate behavior compared to the current state of the art. It allows the exploitation of the power of AI models. Different Neural Network models have been taken into account and compared in terms of measurement accuracy and a number of model parameters; in particular, Dense, 1-Dimension Convolutional, and Long Short Term Memory Neural networks. As can be excepted, the higher the NN complexity, the higher the measurement accuracy; the models' performance has been assessed by means of the root-mean-square error (RMSE) between the target and predicted values of all the navigation parameters. [ABSTRACT FROM AUTHOR]

Underwater target detection and identification technology are currently two of the most important research directions in the information disciplines. Traditionally, underwater target detection technology has struggled to meet the needs of current engineering. However, due to the large manifold error of the underwater sonar array and the complexity of ensuring long-term signal stability, traditional high-resolution array signal processing methods are not ideal for practical underwater applications. In conventional beamforming methods, when the signal-to-noise ratio is lower than −43.05 dB, the general direction can only be vaguely identified in the general direction. To address the above challenges, this paper proposes a beamforming method based on a deep neural network. Through preprocessing, the space-time domain of the target sound signal is converted into two-dimensional data in the angle-time domain. Subsequently, we trained the network with enough sample datasets. Finally, high-resolution recognition and prediction of two-dimensional images are realized. The results of the test dataset in this paper demonstrate the effectiveness of the proposed method, with a minimum signal-to-noise ratio of −48 dB. [ABSTRACT FROM AUTHOR]

Fingerprinting localization is a promising indoor positioning methods thanks to its advantage of using preinstalled infrastructure. For example, WiFi signal strength can be measured by pre-existing WiFi routers. In the offline phase, the fingerprinting localization method first stores of position and RSSI measurement pairs in a dataset. Second, it predicts a target's location by comparing the stored fingerprint database to the current measurement. The database size is normally huge, and data patterns are complicated; thus, an artificial neural network is used to model the relationship of fingerprints and locations. The existing fingerprinting locations, however, have been developed to predict only single locations. In practice, many users may require positioning services, and as such, the core algorithm should be capable of multiple localizations, which is the main contribution of this paper. In this paper, multiple fingerprinting localization is developed based on an artificial neural network and an analysis of the number of targets that can be estimated without loss of accuracy is conducted by experiments. [ABSTRACT FROM AUTHOR]

Ground-object classification using remote-sensing images of high resolution is widely used in land planning, ecological monitoring, and resource protection. Traditional image segmentation technology has poor effect on complex scenes in high-resolution remote-sensing images. In the field of deep learning, some deep neural networks are being applied to high-resolution remote-sensing image segmentation. The DeeplabV3+ network is a deep neural network based on encoder-decoder architecture, which is commonly used to segment images with high precision. However, the segmentation accuracy of high-resolution remote-sensing images is poor, the number of network parameters is large, and the cost of training network is high. Therefore, this paper improves the DeeplabV3+ network. Firstly, MobileNetV2 network was used as the backbone feature-extraction network, and an attention-mechanism module was added after the feature-extraction module and the ASPP module to introduce focal loss balance. Our design has the following advantages: it enhances the ability of network to extract image features; it reduces network training costs; and it achieves better semantic segmentation accuracy. Experiments on high-resolution remote-sensing image datasets show that the mIou of the proposed method on WHDLD datasets is 64.76%, 4.24% higher than traditional DeeplabV3+ network mIou, and the mIou on CCF BDCI datasets is 64.58%. This is 5.35% higher than traditional DeeplabV3+ network mIou and outperforms traditional DeeplabV3+, U-NET, PSP-NET and MACU-net networks. [ABSTRACT FROM AUTHOR]

Virtual reality, driverless cars, and robotics all make extensive use of 3D shape classification. One of the most popular ways to represent 3D data is with polygonal meshes. In particular, triangular mesh is frequently employed. A triangular mesh has more features than 3D data formats such as voxels, multi-views, and point clouds. The current challenge is to fully utilize and extract useful information from mesh data. In this paper, a 3D shape classification network based on triangular mesh and graph convolutional neural networks was suggested. The triangular face of this model was viewed as a unit. By obtaining an adjacency matrix from mesh data, graph convolutional neural networks can be utilized to process mesh data. The studies were performed on the ModelNet40 dataset with an accuracy of 91.0%, demonstrating that the classification network in this research may produce effective results. [ABSTRACT FROM AUTHOR]

Time-space four-dimensional motion target localization is a fundamental and challenging task in the field of intelligent driving, and an important part of achieving the upgrade in existing target localization technologies. In order to solve the problem of the lack of localization of moving targets in a spatio-temporal four-dimensional environment in the existing spatio-temporal data model, this paper proposes an optical imaging model in the four-dimensional time-space system and a mathematical model of the object-image point mapping relationship in the four-dimensional time-space system based on the central perspective projection model, combined with the one-dimensional "time" and three-dimensional "space". After adding the temporal dimension, the imaging system parameters are extended. In order to solve the nonlinear mapping problem of complex systems, this paper proposes to construct a time-space four-dimensional object-image mapping relationship model based on a BP artificial neural network and demonstrates the feasibility of the joint time-space four-dimensional imaging model theory. In addition, indoor time-space four-dimensional localization prediction experiments verify the performance of the model in this paper. The maximum relative error rates of the predicted motion depth values, time values, and velocity values of this localization method compared with the real values do not exceed 0.23%, 2.03%, and 1.51%, respectively [ABSTRACT FROM AUTHOR]

Organizations managing high-performance computing systems face a multitude of challenges, including overarching concerns such as overall energy consumption, microprocessor clock frequency limitations, and the escalating costs associated with chip production. Evidently, processor speeds have plateaued over the last decade, persisting within the range of 2 GHz to 5 GHz. Scholars assert that brain-inspired computing holds substantial promise for mitigating these challenges. The spiking neural network (SNN) particularly stands out for its commendable power efficiency when juxtaposed with conventional design paradigms. Nevertheless, our scrutiny has brought to light several pivotal challenges impeding the seamless implementation of large-scale neural networks (NNs) on silicon. These challenges encompass the absence of automated tools, the need for multifaceted domain expertise, and the inadequacy of existing algorithms to efficiently partition and place extensive SNN computations onto hardware infrastructure. In this paper, we posit the development of an automated tool flow capable of transmuting any NN into an SNN. This undertaking involves the creation of a novel graph-partitioning algorithm designed to strategically place SNNs on a network-on-chip (NoC), thereby paving the way for future energy-efficient and high-performance computing paradigms. The presented methodology showcases its effectiveness by successfully transforming ANN architectures into SNNs with a marginal average error penalty of merely 2.65%. The proposed graph-partitioning algorithm enables a 14.22% decrease in inter-synaptic communication and an 87.58% reduction in intra-synaptic communication, on average, underscoring the effectiveness of the proposed algorithm in optimizing NN communication pathways. Compared to a baseline graph-partitioning algorithm, the proposed approach exhibits an average decrease of 79.74% in latency and a 14.67% reduction in energy consumption. Using existing NoC tools, the energy-latency product of SNN architectures is, on average, 82.71% lower than that of the baseline architectures. [ABSTRACT FROM AUTHOR]

Ground reaction force (GRF) components can be estimated using insole pressure sensors. Principal component analysis in conjunction with machine learning (PCA-ML) methods are widely used for this task. PCA reduces dimensionality and requires pre-normalization. In this paper, we evaluated the impact of twelve pre-normalization methods using three PCA-ML methods on the accuracy of GRF component estimation. Accuracy was assessed using laboratory data from gold-standard force plate measurements. Data were collected from nine subjects during slow- and normal-speed walking activities. We tested the ANN (artificial neural network) and LS (least square) methods while also exploring support vector regression (SVR), a method not previously examined in the literature, to the best of our knowledge. In the context of our work, our results suggest that the same normalization method can produce the worst or the best accuracy results, depending on the ML method. For example, the body weight normalization method yields good results for PCA-ANN but the worst performance for PCA-SVR. For PCA-ANN and PCA-LS, the vector standardization normalization method is recommended. For PCA-SVR, the mean method is recommended. The final message is not to define a normalization method a priori independently of the ML method. [ABSTRACT FROM AUTHOR]

The ARGO-USV (Unmanned Surface Vehicle for ARchaeological GeO-application) is a technological project involving a marine drone aimed at devising an innovative methodology for marine geological and geomorphological investigations in shallow areas, usually considered critical areas to be investigated, with the help of traditional vessels. The methodological approach proposed in this paper has been implemented according to a multimodal mapping technique involving the simultaneous and integrated use of both optical and geoacoustic sensors. This approach has been enriched by tools based on artificial intelligence (AI), specifically intended to be installed onboard the ARGO-USV, aimed at the automatic recognition of submerged targets and the physical characterization of the seabed. This technological project is composed of a main command and control system and a series of dedicated sub-systems successfully tested in different operational scenarios. The ARGO drone is capable of acquiring and storing a considerable amount of georeferenced data during surveys lasting a few hours. The transmission of all acquired data in broadcasting allows the cooperation of a multidisciplinary team of specialists able to analyze specific datasets in real time. These features, together with the use of deep-learning-based modules and special attention to green-compliant construction phases, are the particular aspects that make ARGO-USV a modern and innovative project, aiming to improve the knowledge of wide coastal areas while minimizing the impact on these environments. As a proof-of-concept, we present the extensive mapping and characterization of the seabed from a geoarchaeological survey of the underwater Roman harbor of Puteoli in the Gulf of Naples (Italy), demonstrating that deep learning techniques can work synergistically with seabed mapping methods. [ABSTRACT FROM AUTHOR]

This proposed research explores a novel approach to image classification by deploying a complex-valued neural network (CVNN) on a Field-Programmable Gate Array (FPGA), specifically for classifying 2D images transformed into polar form. The aim of this research is to address the limitations of existing neural network models in terms of energy and resource efficiency, by exploring the potential of FPGA-based hardware acceleration in conjunction with advanced neural network architectures like CVNNs. The methodological innovation of this research lies in the Cartesian to polar transformation of 2D images, effectively reducing the input data volume required for neural network processing. Subsequent efforts focused on constructing a CVNN model optimized for FPGA implementation, emphasizing the enhancement of computational efficiency and overall performance. The experimental findings provide empirical evidence supporting the efficacy of the image classification system developed in this study. One of the developed models, CVNN_128, achieves an accuracy of 88.3% with an inference time of just 1.6 ms and a power consumption of 4.66 mW for the classification of the MNIST test dataset, which consists of 10,000 frames. While there is a slight concession in accuracy compared to recent FPGA implementations that achieve 94.43%, our model significantly excels in classification speed and power efficiency—surpassing existing models by more than a factor of 100. In conclusion, this paper demonstrates the substantial advantages of the FPGA implementation of CVNNs for image classification tasks, particularly in scenarios where speed, resource, and power consumption are critical. [ABSTRACT FROM AUTHOR]

This paper investigates spiking neural networks (SNN) for novel robotic controllers with the aim of improving accuracy in trajectory tracking. By emulating the operation of the human brain through the incorporation of temporal coding mechanisms, SNN offer greater adaptability and efficiency in information processing, providing significant advantages in the representation of temporal information in robotic arm control compared to conventional neural networks. Exploring specific implementations of SNN in robot control, this study analyzes neuron models and learning mechanisms inherent to SNN. Based on the principles of the Neural Engineering Framework (NEF), a novel spiking PID controller is designed and simulated for a 3-DoF robotic arm using Nengo and MATLAB R2022b. The controller demonstrated good accuracy and efficiency in following designated trajectories, showing minimal deviations, overshoots, or oscillations. A thorough quantitative assessment, utilizing performance metrics like root mean square error (RMSE) and the integral of the absolute value of the time-weighted error (ITAE), provides additional validation for the efficacy of the SNN-based controller. Competitive performance was observed, surpassing a fuzzy controller by 5% in terms of the ITAE index and a conventional PID controller by 6% in the ITAE index and 30% in RMSE performance. This work highlights the utility of NEF and SNN in developing effective robotic controllers, laying the groundwork for future research focused on SNN adaptability in dynamic environments and advanced robotic applications. [ABSTRACT FROM AUTHOR]

This paper proposes fault diagnosis methods aimed at proactively preventing potential safety issues in robot systems, particularly human coexistence robots (HCRs) used in industrial environments. The data were collected from durability tests of the driving module for HCRs, gathering time-series vibration data until the module failed. In this study, to apply classification methods in the absence of post-failure data, the initial 50% of the collected data were designated as the normal section, and the data from the 10 h immediately preceding the failure were selected as the fault section. To generate additional data for the limited fault dataset, the Wasserstein generative adversarial networks with gradient penalty (WGAN-GP) model was utilized and residual connections were added to the generator to maintain the basic structure while preventing the loss of key features of the data. Considering that the performance of image encoding techniques varies depending on the dataset type, this study applied and compared five image encoding methods and four CNN models to facilitate the selection of the most suitable algorithm. The time-series data were converted into image data using image encoding techniques including recurrence plot, Gramian angular field, Markov transition field, spectrogram, and scalogram. These images were then applied to CNN models, including VGGNet, GoogleNet, ResNet, and DenseNet, to calculate the accuracy of fault diagnosis and compare the performance of each model. The experimental results demonstrated significant improvements in diagnostic accuracy when employing the WGAN-GP model to generate fault data, and among the image encoding techniques and convolutional neural network models, spectrogram and DenseNet exhibited superior performance, respectively. [ABSTRACT FROM AUTHOR]

The presented paper investigates the problem of endoscopic bleeding detection in endoscopic videos in the form of a binary image classification task. A set of definitions of high-level visual features of endoscopic bleeding is introduced, which incorporates domain knowledge from the field. The high-level features are coupled with respective feature descriptors, enabling automatic capture of the features using image processing methods. Each of the proposed feature descriptors outputs a feature activation map in the form of a grayscale image. Acquired feature maps can be appended in a straightforward way to the original color channels of the input image and passed to the input of a convolutional neural network during the training and inference steps. An experimental evaluation is conducted to compare the classification ROC AUC of feature-extended convolutional neural network models with baseline models using regular color image inputs. The advantage of feature-extended models is demonstrated for the Resnet and VGG convolutional neural network architectures. [ABSTRACT FROM AUTHOR]

The direct current (DC) microgrid is one of the key research areas for our advancement toward carbon-free energy production. In this paper, a two-step controller is designed for the DC microgrid using a combination of the deep neural network (DNN) and exponential reaching law-based global terminal sliding mode control (ERL-GTSMC). The DC microgrid under consideration involves multiple renewable sources (wind, PV) and an energy storage unit (ESU) connected to a 700 V DC bus and a 4–12 kW residential load. The proposed control method eliminates the chattering phenomenon and offers quick reaching time by utilizing the exponential reaching law (ERL). In the two-step control configuration, first, DNNs are used to find maximum power point tracking (MPPT) reference values, and then ERL-based GTSMC is utilized to track the reference values. The real dynamics of energy sources and the DC bus are mathematically modeled, which increases the system's complexity. Through the use of Lyapunov stability criteria, the stability of the control system is examined. The effectiveness of the suggested hybrid control algorithm has been examined using MATLAB simulations. The proposed framework has been compared to traditional sliding mode control and terminal sliding mode control to showcase its superiority and robustness. Experimental tests based on the hardware-in-the-loop (HIL) setup are then conducted using 32-bit TMS320F28379D microcontrollers. Both MATLAB and HIL results show strong performance under a range of environmental circumstances and system uncertainties. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) using wearable sensors enables continuous monitoring for healthcare applications. However, the conventional centralised training of deep learning models on sensor data poses challenges related to privacy, communication costs, and on-device efficiency. This paper proposes a federated learning framework integrating spiking neural networks (SNNs) with long short-term memory (LSTM) networks for energy-efficient and privacy-preserving HAR. The hybrid spiking-LSTM (S-LSTM) model synergistically combines the event-driven efficiency of SNNs and the sequence modelling capability of LSTMs. The model is trained using surrogate gradient learning and backpropagation through time, enabling fully supervised end-to-end learning. Extensive evaluations of two public datasets demonstrate that the proposed approach outperforms LSTM, CNN, and S-CNN models in accuracy and energy efficiency. For instance, the proposed S-LSTM achieved an accuracy of 97.36% and 89.69% for indoor and outdoor scenarios, respectively. Furthermore, the results also showed a significant improvement in energy efficiency of 32.30%, compared to simple LSTM. Additionally, we highlight the significance of personalisation in HAR, where fine-tuning with local data enhances model accuracy by up to 9% for individual users. [ABSTRACT FROM AUTHOR]

Fitness is important in people's lives. Good fitness habits can improve cardiopulmonary capacity, increase concentration, prevent obesity, and effectively reduce the risk of death. Home fitness does not require large equipment but uses dumbbells, yoga mats, and horizontal bars to complete fitness exercises and can effectively avoid contact with people, so it is deeply loved by people. People who work out at home use social media to obtain fitness knowledge, but learning ability is limited. Incomplete fitness is likely to lead to injury, and a cheap, timely, and accurate fitness detection system can reduce the risk of fitness injuries and can effectively improve people's fitness awareness. In the past, many studies have engaged in the detection of fitness movements, among which the detection of fitness movements based on wearable devices, body nodes, and image deep learning has achieved better performance. However, a wearable device cannot detect a variety of fitness movements, may hinder the exercise of the fitness user, and has a high cost. Both body-node-based and image-deep-learning-based methods have lower costs, but each has some drawbacks. Therefore, this paper used a method based on deep transfer learning to establish a fitness database. After that, a deep neural network was trained to detect the type and completeness of fitness movements. We used Yolov4 and Mediapipe to instantly detect fitness movements and stored the 1D fitness signal of movement to build a database. Finally, MLP was used to classify the 1D signal waveform of fitness. In the performance of the classification of fitness movement types, the mAP was 99.71%, accuracy was 98.56%, precision was 97.9%, recall was 98.56%, and the F1-score was 98.23%, which is quite a high performance. In the performance of fitness movement completeness classification, accuracy was 92.84%, precision was 92.85, recall was 92.84%, and the F1-score was 92.83%. The average FPS in detection was 17.5. Experimental results show that our method achieves higher accuracy compared to other methods. [ABSTRACT FROM AUTHOR]

Image recognition has long been one of the research hotspots in computer vision tasks. The development of deep learning is rapid in recent years, and convolutional neural networks usually need to be designed with fixed resources. If sufficient resources are available, the model can be scaled up to achieve higher accuracy, for example, VggNet, ResNet, GoogLeNet, etc. Although the accuracy of large-scale models has been improved, the following problems will occur with the expansion of model scale: (1) There may be over-fitting; (2) increasing model parameters; (3) slow model convergence. This paper proposes a design method for a modular convolutional neural network model which solves the problem of over-fitting and large model parameters by connecting multiple modules in parallel. Moreover, each module contains several submodules (three submodules in this paper) and fuses the features extracted from the submodules. The model convergence can be accelerated by using the fused features (the fused features contain more image information). In this study, we add a gate unit based on the attention mechanism to the model, which aims to optimize the structure of the model (select the optimal number of modules), allowing the model to select an optimum network structure by learning and dynamically reducing FLOPs (floating-point operations per second) of the model. Compared to VggNet, ResNet, and GoogLeNet, the structure of the model proposed in this paper is simple and the parameters are small. The proposed model achieves good results in the Kaggle datasets Cats-vs.-Dogs (99.3%), 10-Monkey Species (99.26%), and Birds-400 (99.13%). [ABSTRACT FROM AUTHOR]

Railroads are a critical part of the United States' transportation sector. Over 40 percent (by weight) of the nation's freight is transported by rail, and according to the Bureau of Transportation statistics, railroads moved $186.5 billion of freight in 2021. A vital part of the freight network is railroad bridges, with a good number being low-clearance bridges that are prone to impacts from over-height vehicles; such impacts can cause damage to the bridge and lead to unwanted interruption in its usage. Therefore, the detection of impacts from over-height vehicles is critical for the safe operation and maintenance of railroad bridges. While some previous studies have been published regarding bridge impact detection, most approaches utilize more expensive wired sensors, as well as relying on simple threshold-based detection. The challenge is that the use of vibration thresholds may not accurately distinguish between impacts and other events, such as a common train crossing. In this paper, a machine learning approach is developed for accurate impact detection using event-triggered wireless sensors. The neural network is trained with key features which are extracted from event responses collected from two instrumented railroad bridges. The trained model classifies events as impacts, train crossings, or other events. An average classification accuracy of 98.67% is obtained from cross-validation, while the false positive rate is minimal. Finally, a framework for edge classification of events is also proposed and demonstrated using an edge device. [ABSTRACT FROM AUTHOR]