Search

In the production of paper cups in industrial factories, it is tried to print high quality cups with less waste loss with the help of sensors and heating resistances mounted on the paper cup machine. In this study, a system that detects faulty products based on image processing and removes it by controlling the machine with servo motors, asynchronous motors and programmable logic controller (PLC) is designed. For fault product detection, classification has been performed using real-time Haarcascade algorithm and You Only Look Once (YOLO) algorithm which is a deep learning methods, and real-time object detection has been carried out using the OpenCv library. With this study, an effective faulty product detection and removing hardware system was realized by adapting artificial intelligence algorithms to a machine used in industry. Based on the results, a whole system can be applied to systems that involve removing a faulty product from a band in any production, packaging etc. facility is proposed. A hardware consisting of servo motors, asynchronous motors and PLC was designed to separate faulty cups from the existing paper cup production machine in this study. Then, a data set composed of 1068 images was created with images taken from the camera for faulty and faultless paper cups. Using this dataset, the effect of different deep learning methods on performance in the real-time system has been examined and successful results have been obtained. The optimal outcome was achieved, yielding a real-time application accuracy rate of 90.8% through the utilization of the Yolov5x architecture. [ABSTRACT FROM AUTHOR]

Industrial products typically lack defects in smart manufacturing systems, which leads to an extremely imbalanced task of recognizing surface defects. With this imbalanced sample distribution, machine learning and deep learning algorithms preferentially learn features from the majority classes, potentially leading to inaccurate results. Addressing the issue of sample imbalance has thus emerged as a critical area of research within the field of industrial intelligent manufacturing. This paper discusses the imbalanced sample problem of industrial product surface defect detection algorithms, and proposes the existence of "four imbalances and two uncertainties". It also summarizes the industrial product surface dataset and innovatively adds the imbalance rate comparison to the dataset. In this study, data re-sampling, data expansion, feature extraction and identification, and re-weighting of category weights are elaborated at the level of data and algorithm respectively. Additionally, the paper explores prospective directions for future research, including supervised and unsupervised learning, transfer learning, anomaly detection, quality prediction, and future challenges. It is hoped to lay a solid foundation for the more far-reaching development of smart manufacturing and surface defect detection methods. And provide some directions for the research of sample imbalance and long-tail problems. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) is a crucial detection technique extensively employed in various contexts demanding accurate identification of human actions. Currently, mainstream HAR approaches frequently rely on sequential data generated by wearable sensors, with a primary focus on extracting feature representations that align with various human activities. Nevertheless, complex and diverse human activities encompass various kinds of features and substantial variations in the features emphasized across different activities, making it challenging for current artificial intelligence methods to achieve accurate recognition. To address this challenge, this paper introduces a network framework called the Multi-Feature Combining Attention Neural Network (MFCANN). This framework incorporates diverse feature extraction, along with both local and global feature attention mechanisms. It is composed of stacked Multi-Feature Combining Attention Blocks (MFCAB) designed by us. In contrast to traditional convolutional methods, MFCAB parallelly stacks various convolutional components, enabling the extraction of a broader range of features from human activity data. Additionally, we propose the Intra-Module Attention Block (Intra-MAB) and the Inter-Module Attention Block (Inter-MAB), which simultaneously focus on local fine-grained features within module feature maps and global distinguishing features across module feature maps. This aims to achieve more targeted feature learning, reinforcing the network's ability to distinguish between different human activities. Abundant experimental results demonstrate that the proposed MFCANN in this paper outperforms current mainstream deep learning algorithms in HAR tasks, achieving recognition accuracies of 0.9813, 0.9324, and 0.9930 on the UCI-HAR, USC-HAD, and RealWorld datasets respectively. We release our code at https://github.com/RangerHeart/MFCANN. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) is a crucial detection technique extensively employed in various contexts demanding accurate identification of human actions. Currently, mainstream HAR approaches frequently rely on sequential data generated by wearable sensors, with a primary focus on extracting feature representations that align with various human activities. Nevertheless, complex and diverse human activities encompass various kinds of features and substantial variations in the features emphasized across different activities, making it challenging for current artificial intelligence methods to achieve accurate recognition. To address this challenge, this paper introduces a network framework called the Multi-Feature Combining Attention Neural Network (MFCANN). This framework incorporates diverse feature extraction, along with both local and global feature attention mechanisms. It is composed of stacked Multi-Feature Combining Attention Blocks (MFCAB) designed by us. In contrast to traditional convolutional methods, MFCAB parallelly stacks various convolutional components, enabling the extraction of a broader range of features from human activity data. Additionally, we propose the Intra-Module Attention Block (Intra-MAB) and the Inter-Module Attention Block (Inter-MAB), which simultaneously focus on local fine-grained features within module feature maps and global distinguishing features across module feature maps. This aims to achieve more targeted feature learning, reinforcing the network's ability to distinguish between different human activities. Abundant experimental results demonstrate that the proposed MFCANN in this paper outperforms current mainstream deep learning algorithms in HAR tasks, achieving recognition accuracies of 0.9813, 0.9324, and 0.9930 on the UCI-HAR, USC-HAD, and RealWorld datasets respectively. We release our code at https://github.com/RangerHeart/MFCANN. [ABSTRACT FROM AUTHOR]

The fault management process is facilitated by equipping power distribution systems with automated devices, especially remote-controlled switches (RCSs). Although RCS plays a key role in improving the system's reliability, it imposes significant investment, installation, and maintenance costs. Hence, RCSs should be optimally located in distribution feeders for the highest profit. In previous works, reliability-oriented mathematical optimization models have been formulated to reach this goal. However, the number of test solutions exponentially grows with the problem size to find the globally optimal solution. This paper uses machine learning to propose a scalable and easy-to-implement model for optimal switch placement in real power distribution systems. At first, the features of candidate points for installing RCSs are introduced. Then, a learning model is applied to deeply explore the relationship between these features and optimal locations for installing RCSs. After training, the learning-based surrogate model directly determines the optimal RCS placement strategy in real power distribution systems by leveraging knowledge gained from past experiences. Simulation results demonstrate that the proposed surrogate model is approximately 29 times faster than the integer programming-based mathematical model, without a significant loss of accuracy, when implemented on a modified 11 kV network connected to Bus 4 of the Roy Billiton test system. This paper also employs explainable artificial intelligence (XAI) tools to select the most important features, where the Hamming loss is decreased by approximately 5%. [ABSTRACT FROM AUTHOR]

Increases in population and prosperity are linked to a worldwide rise in garbage. The "classification" and "recycling" of solid waste is a crucial tactic for dealing with the waste problem. This paper presents a new two-layer intelligent decision system for waste sorting based on fused features of Deep Learning (DL) models as well as a selection of an optimal deep Waste-Sorting Model (WSM) based on Multi-Criteria Decision Making (MCDM). A dataset comprising 1451 samples of images of waste, distributed across four classes – cardboard (403), glass (501), metal (410), and general trash (137), was used for sorting. This study proposes a Multi-Fused Decision Matrix (MFDM) based on identified fusion score level rules, evaluation criteria, and deep fused waste-sorting models. Five fusion rules used in the sorting process and the evaluation perspectives into the MFDM are sum, weighted sum, product, maximum, and minimum rules. Additionally, each of entropy and Visekriterijumska Optimizacija i Kompromisno Resenje in Serbian (VIKOR) methods was used for weighting selected criteria as well as ranking deep WSMs. The highest accuracy rate of 98% was scored by ResNet50-GoogleNet- Inception based on the minimum rule. However, under the same rule, an insufficient accuracy rate of sorting was presented by ResNet50-GoogleNet-Xception. Since Qi = 0 for Inception-Xception, the final output based on MCDM methods indicates that the fused Inception-Xception model outperforms the other fused deep WSMs, which achieved the lowest values of Qi. Thus, Inception-Xception was chosen as the best deep waste-sorting model based on images of waste, multiple evaluation criteria, and different fusion perspectives. The mean and standard deviation metrics were both used to validate the selection findings objectively. The suggested approach can aid urban decision-makers in prioritizing and choosing an Artificial Intelligence (AI)-optimized optimal sorting model. • Develop a 2-layer decision system for waste sorting using deep learning. • Construct MFDM using fusion scores, criteria, and deep waste sorting models. • Entropy and VIKOR methods rank criteria and deep learning models. [ABSTRACT FROM AUTHOR]

Human motion analysis plays a crucial role in industry 4.0 and, more recently, in industry 5.0 where human-centered applications are becoming increasingly important, demonstrating its potential for enhancing safety, ergonomics and productivity. Considering this opportunity, an increasing number of studies are proposing works on the analysis of human motion in an industrial context, taking advantage of the rise of artificial intelligence technologies and sensor technologies. The objective of this work is to provide a review of recent studies exploring these technologies in the analysis of human movement while specifically considering industrial context. First, a taxonomy of key human motion analysis applications is proposed, presenting statistical insights to reveal trends and highlighting lacks in current research. Furthermore, this work identifies benchmark datasets acquired in various industrial case studies and associated sensors. Many recommendations for selecting optimal sensors and valuable benchmarks are proposed. Then, the paper outlines the current trend of utilizing hybrid deep learning methodologies in human movement analysis while underscoring the performance and limitations of these proposed methods, considering industrial constraints such as real-time recognition and frugality. Finally, challenges and future works are highlighted, focusing on the opportunities to address problems related to the complex industrial environment in order to achieve reliable performances. [ABSTRACT FROM AUTHOR]

Conventional malware detection approaches have the overhead of feature extraction, the requirement of domain experts, and are time-consuming and resource-intensive. Learning-based approaches are the mainstay of malware detection as they overcome most of these challenges by significantly improving the detection effectiveness and providing a low false positive rate. The exponential growth of malware variants and first-time-appeared malware, which includes polymorphic and zero-day attacks, are some of the significant challenges to learning-based malware detectors. These challenges have catastrophic impacts on the detection effectiveness of these learning-based malware detectors. This paper proposes a novel deep learning-based framework to detect first-time-appeared malware effectively and efficiently by providing better performance than conventional malware detection approaches. First, it translates and visualises each Windows portable executable (PE) file into a coloured image to eliminate the overhead of feature extraction and the need for domain experts to analyse the features. In the subsequent step, a fine-tuned deep learning model is used to extract the deep features from the last fully connected layer. The step has reduced the cost of training required by the deep learning models if used for end-to-end classification. The third step selects the most important and influential features through a powerful feature selection algorithm. The most important features are then fed to a one-class classifier for final detection. With the one-class classifier, an enclosed boundary around the features of benign data is constructed. Anything outside the boundary is declared as an anomaly/malicious. It has enhanced the framework's ability to detect evolving, unseen, polymorphic, and zero-day attacks, as well as reducing the problem of overfitting. The detection effectiveness of the proposed framework is validated with state-of-the-art deep learning models and conventional approaches. The proposed framework has outperformed with an accuracy of 99.30% on the Malimg dataset. The Wilcoxon signed-rank test is used to validate the statistical significance of the proposed framework. It is evident from the results that the proposed framework is effective and can be used in the defence industry, resulting in more powerful and robust solutions against zero-day and polymorphic attacks. [Display omitted] • A novel approach of combining deep learning and machine learning is proposed. First, deep learning is used to extract deep features. The most influential and meticulous features are selected in the subsequent steps to train the machine learning classifier for final detection. The proposed framework eliminates the need for human efforts for reverse engineering tasks. • The proposed framework consists of four steps. In the first step, all PEs are transformed into coloured images. The second step used a deep learning model to extract the deep features. The subsequent step selects the most important features. Finally, the lightweight and most influential features are sent to the final machine learning classifier for final malware detection. • We demonstrate that the proposed framework is lightweight, resilient, efficient and cost-effective. An in-depth analysis is performed to validate the detection effectiveness and generalisation of the proposed framework on multiple datasets. Our results demonstrate that the proposed framework outperformed conventional and state-of-the-art malware detection approaches. [ABSTRACT FROM AUTHOR]

Deep learning has advanced the development of automated cervical cytology, yet limited studies have delved into methods for incorporating medical domain knowledge, and model interpretability has not been thoroughly investigated. To address this issue, this paper proposes a novel, explainable detection method for abnormal cervical cells, called dual-stream self-attention based feature fusion and origin grouping network (DSA-FFOGNet). To encourage the model to focus more on lesion cells and cell nuclei of diagnostic significance, the dual-stream self-attention (DSA) module is introduced to enhance the learning of lesion-specific features. In view of the complex background, cell dense distribution, cell overlap, or clumps existing in the actual cervical cytology images, multi-scale features are extracted and fused by using the path aggregation network (PAN) to enhance the feature representation ability. By integrating biomedical insights regarding cell provenance and formulating an origin grouping loss, DSA-FFOGNet adjusts the penalties for cervical cells originating from different groups, thereby enhancing the optimization of the model training process. To further improve the detection performance, the classification and localization tasks are decoupled via the use of double detection heads. Extensive experiments validate the robustness of the proposed DSA-FFOGNet. The visualization of class activation maps (CAMs) showcases the model's interpretability. The proposed approach advances the application and development of explainable artificial intelligence (XAI) models in cervical cytology and inspires further research in automated cervical cytology. [ABSTRACT FROM AUTHOR]

Object detection is one of the most important domains in computer vision tasks, which is an important branch of artificial intelligence. It aims at finding and locating the accurate position of objects in given pictures or videos. With the development of deep learning techniques, more powerful and robust algorithms have emerged to deal with multi-scale, high-level features to overcome the limitations of traditional pipeline of object detectors. The popularity of transformer framework enables larger capacity datasets by processing self-attention mechanism, and the object detection methods have evolved into a new era. This paper first reviews traditional object detection pipeline and brief history of deep learning, afterwards it focuses on the classification of deep learning-based object detection methods covering Convolution Neural Network based and transformer-based methods. Commonly used datasets and metrics are also covered in the next part. The Convolution Neural Network based methods mainly contain two-stage and one-stage detectors, Convolution Neural Network is the underlying structure of these methods convolutional stages are fundamental parts. Transformer-based models convert traditional object detection issues into end-to-end detection, which is widely used in dealing with images. Finally, the promising future of object detection areas are listed to show guidance on future work. [Display omitted] [ABSTRACT FROM AUTHOR]

The existing deep learning-based image dehazing algorithms commonly employ an encoder–decoder structure to learn a direct mapping from hazy images to haze-free images. However, these state-of-the-art methods often fail to consider the varying contents of hazy images across different scenes, resulting in unsatisfactory dehazing outcomes. To address this issue, this paper attempts to integrate a novel instance-aware subnet into the classic encoder–decoder structure in order to achieve a clear separation between figure and background, conducting the selective incorporation of instance features into the dehazing network. Specifically, we introduce a novel architecture called the hybrid residual attention network, which is capable of separately extracting full-image features and instance-level features. This architecture incorporates attention mechanisms and a multi-scale dilated convolution structure, enabling adaptive perception of haze density in different scenes. Additionally, we introduce a global feature fusion subnet that employs a pixel attention structure to fuse features from the entire image and multiple individual instances, thus being aware of instance features. Compared to existing methods, our approach offers a major advantage in accurately estimating the haze density of individual instances, reducing color distortion, and mitigating noise amplification in the output images. Experimental results demonstrate that our method outperforms existing methods across different evaluation metrics and testing benchmarks. Therefore, we believe that our method will serve as a valuable addition to the current collection of artificial intelligence models and will benefit engineering applications in video surveillance and high-level computer vision tasks. [ABSTRACT FROM AUTHOR]

Artificial intelligence employs Machine Learning (ML) and Deep Learning (DL) to analyze data. In both, the data is stored centrally. The data involved may be sensitive and leakage may incur consequences. Applications dealing with intimate data, with critical results, cannot afford this risk and are termed Data-Sensitive Applications (DSA). Some examples are healthcare, finance, etc. The data required for DSA cannot be stored centrally due to large amounts, or isolated data islands. The ML and DL techniques following a data-centralized approach have difficulties in handling the scattered data frequently associated with DSA. Federated Learning (FL) acknowledges the scattered data and provides a more secure and efficient way to analyze such data. This motivates previously reluctant entities like banks to collaborate for variety and quantity of data. Most DSA transitioned to FL, but the migration is not without concerns. These include communication costs, heterogeneity, and malicious attacks. In this paper, we deeply analyze the role of FL in DSA and provide a taxonomy for the studies and implementations of FL. Then we provide an insight into DSA covering works in healthcare and finance. A glance is provided at attempts in non-DSA with possible DSA applications. Finally, we discuss FL's open issues and challenges with their possible solutions. [ABSTRACT FROM AUTHOR]

The existing deep learning-based image dehazing algorithms commonly employ an encoder–decoder structure to learn a direct mapping from hazy images to haze-free images. However, these state-of-the-art methods often fail to consider the varying contents of hazy images across different scenes, resulting in unsatisfactory dehazing outcomes. To address this issue, this paper attempts to integrate a novel instance-aware subnet into the classic encoder–decoder structure in order to achieve a clear separation between figure and background, conducting the selective incorporation of instance features into the dehazing network. Specifically, we introduce a novel architecture called the hybrid residual attention network, which is capable of separately extracting full-image features and instance-level features. This architecture incorporates attention mechanisms and a multi-scale dilated convolution structure, enabling adaptive perception of haze density in different scenes. Additionally, we introduce a global feature fusion subnet that employs a pixel attention structure to fuse features from the entire image and multiple individual instances, thus being aware of instance features. Compared to existing methods, our approach offers a major advantage in accurately estimating the haze density of individual instances, reducing color distortion, and mitigating noise amplification in the output images. Experimental results demonstrate that our method outperforms existing methods across different evaluation metrics and testing benchmarks. Therefore, we believe that our method will serve as a valuable addition to the current collection of artificial intelligence models and will benefit engineering applications in video surveillance and high-level computer vision tasks. [ABSTRACT FROM AUTHOR]

Artificial intelligence employs Machine Learning (ML) and Deep Learning (DL) to analyze data. In both, the data is stored centrally. The data involved may be sensitive and leakage may incur consequences. Applications dealing with intimate data, with critical results, cannot afford this risk and are termed Data-Sensitive Applications (DSA). Some examples are healthcare, finance, etc. The data required for DSA cannot be stored centrally due to large amounts, or isolated data islands. The ML and DL techniques following a data-centralized approach have difficulties in handling the scattered data frequently associated with DSA. Federated Learning (FL) acknowledges the scattered data and provides a more secure and efficient way to analyze such data. This motivates previously reluctant entities like banks to collaborate for variety and quantity of data. Most DSA transitioned to FL, but the migration is not without concerns. These include communication costs, heterogeneity, and malicious attacks. In this paper, we deeply analyze the role of FL in DSA and provide a taxonomy for the studies and implementations of FL. Then we provide an insight into DSA covering works in healthcare and finance. A glance is provided at attempts in non-DSA with possible DSA applications. Finally, we discuss FL's open issues and challenges with their possible solutions. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI) plays a critical role in the prediction, planning, and control of autonomous vehicle. The original motion control methods are increasing in accuracy, but their control style markedly differs from that of human drivers. This not only degrades the passenger's ride experience but also brings unfamiliarity to other drivers, leading to safety hazards. Moreover, some of the methods are a direct application of AI techniques without any specific enhancements tailored for vehicle control. This both fails to fully utilize the potential of AI and reduces interpretability. To address these issues, in this paper, deep learning methods are integrated with driver control mechanisms to propose a human-like neural network for vehicle longitudinal dynamics estimation and control (HNN). First, the process of driver estimation and control of vehicle dynamics is proposed and described in mathematical language. Subsequently, the HNN composed of three sub-networks is presented. The three sub-networks correspond to the three sub-processes of the driver's mechanism, which makes the proposed HNN method more applicable to vehicle dynamics control and more interpretable in specific engineering domains. The effectiveness of the HNN method is verified on a real-world dataset. The results demonstrate that the HNN not only enhances vehicle human-like control but also surpasses baselines in terms of control style consistency and convergence rate. • The method is a deep integration of human control mechanisms and artificial intelligence in a specific engineering domain. • The proposed deep learning method has a unique structure derived from the driver control mechanism.The structure of the method makes it more applicable for vehicle motion control with active rule-based interpretability. • The tolerance control pattern is proposed and incorporated into the mechanism of driver control vehicle dynamics. • Compared to the baselines, the method proves to be more suitable for the engineering field of vehicle human-like control. [ABSTRACT FROM AUTHOR]

This paper introduces a novel integrated microstructure design methodology that replaces the common existing design approaches for multifunctional composites: 1) reconstruction of microstructures, 2) analyzing and quantifying material properties, and 3) inverse design of materials. The problem of microstructure reconstruction is addressed using the diffusion-based generative model (DGM), which is a state-of-the-art generative model formulated with a Markovian diffusion process. Then, the conditional formulation of DGM is introduced for guidance to the embedded desired material properties with a transformer-based attention mechanism, which enables the inverse design of multifunctional composites. Furthermore, a convolutional neural network (CNN)-based surrogate model is utilized to facilitate the prediction of linear/nonlinear material properties for building microstructure-property linkages. Combined, the proposed artificial intelligence-based design framework enables large data processing and database construction that is often not affordable with resource-intensive finite element method (FEM)-based direct numerical simulation (DNS) or iterative reconstruction methods. It is worth noting that the proposed DGM-based methodology is not susceptible to unstable training or mode collapse, which are common issues in generative models that are often difficult to address even with extensive hyperparameter tuning. An example case is presented to demonstrate the effectiveness of the proposed approach, which is designing mechanoluminescence (ML) particulate composites. The results show that the designed ML microstructure samples with the proposed generative and surrogate models meet the multiple design requirements (e.g., volume fraction, elastic constant, and light sensitivity). This assessment demonstrates that the proposed integrated methodology provides an end-to-end solution for practical material design applications. [Display omitted] • An integrated microstructure design framework utilizing the diffusion-based generative model (DGM) is proposed. • The problem of microstructure reconstruction is addressed using the DGM with forward/reverse Markovian diffusion processes. • The conditional formulation of DGM is introduced to enable the inverse design of multifunctional composites. [ABSTRACT FROM AUTHOR]

Artworks are treasures of valuable cultural and historical heritage. Artworks get damaged due to environmental and other factors. The artificial intelligence-based restoration of digitized artwork images can guide the artists in physically restoring the damaged artworks. Previous methods have not been able to restore artwork images well. This paper proposes a dual (spatial and channel) attention and channel transformer-based generative adversarial network to restore damaged artwork images digitally. The proposed generative adversarial network has spatial and channel attention layers in the encoder part of the generator and a channel transformer between skip connections from the encoder to the decoder part of the generator. Spatial and channel attention helps learn inter-spatial and inter-channel global relationships among image features. Channel transformer ensures multiscale feature fusion and reduces the semantic gap between encoder and decoder layer features. Moreover, the proposed network has been trained using a linear combination of perceptual, adversarial, and structured similarity index measure loss, which helps better train the network. Further, the proposed network has been validated on two different datasets, and the results indicate that the proposed method outperforms state-of-the-art artwork restoration methods. [Display omitted] • Spatial and channel attention guided generator architecture for artwork restoration. • Channel transformer between skip connections for multiscale feature fusion. • A combination of perceptual, adversarial, and SSIM loss as objective function. [ABSTRACT FROM AUTHOR]

The antinuclear antibody (ANA) test plays the role of screening, diagnosis, treatment and prognosis for autoimmune diseases. However, the ANA test is time-consuming, and strongly depends on the interpretation of experienced medical examiners. With the rise of artificial intelligence, a lot of convolutional neural networks (CNN) have been used to recognize the features of ANA images, such as GoogLeNet, ResNet, and MobileNet. By extracting the disease-related features from the entire ANA image, these models achieve acceptable accuracy, but none of them can replace the manual ANA test completely. This is because they ignore the key features of ANA images. Accordingly, this paper aims to design an object detection model that can directly identify the key features of ANA images, i.e., the Metaphase cells. This process is critical in the manual ANA test performed by the medical examiners. Based on the YOLO model, our design first applies various image preprocessing and data augmentation techniques to enhance the critical features in the training data. We modify the original YOLO model, in which the residual block is replaced with SENet module to improve the classification performance by emphasizing critical features and reducing the influence of low-level features. Moreover, we use the different activation function to prevent the modified model from partial judgment. The experimental results show that our design can effectively improve the performance of ANA metaphase identification. Compared to the related CNN models, our design can improve F1-score and mAP by range 5%–14% and 11%–13%, respectively. In particular, the Kappa value of our design is 0.82 that is almost perfect agreement with the ANA experts. [ABSTRACT FROM AUTHOR]

In the Industry 4.0 era, a myriad of sensors of diverse nature (temperature, pressure, etc.) is spreading throughout the entire value chain of industries, being potentially exploitable for multiple purposes, such as Predictive Maintenance (PdM): the just-in-time maintenance of industrial assets, which results in reduced operating costs, increased operator safety, etc. Nowadays, industrial processes require to be highly configurable, in order to proactively adapt their operation to diverse factors such as user needs, product updates or supply chain uncertainties. This limits current Industry 4.0-PdM solutions, typically consisting of ad-hoc developments intended for specific scenarios, i.e. they are designed to operate under certain conditions (configurations, employed sensors, etc.), being unable to manage changes in their setup. This paper presents a general Deep Learning (DL) architecture, MachNet, which deals with such heterogeneity and is able to address PdM problems of a diverse nature. The modularity of the proposed architecture enables it to deal with an arbitrary number of sensors of different types, also allowing the integration of prior information (age of assets, material type, etc.), which clearly affects performance and is often neglected. In practice, our architecture effortlessly adapts to the assets' specifications and to different PdM problems. That is, MachNet becomes an architectural template that can be instantiated for a given scenario. We tested our proposal in two different PdM-related problems: Health State (HS) and Remaining-useful-Life (RuL) estimation, achieving in both cases comparable or superior performance to other state-of-the-art approaches, with the additional advantage of the generality that MachNet offers. [ABSTRACT FROM AUTHOR]

Internet of Things is shaping the quality of living standard. With the rapid growth and expansion of adopting IoT-based approaches, their security represents a growing challenge for both manufacturers and consumers. There is a recent rising trend towards employing artificial intelligence approaches to enhance the security of IoT infrastructure. This survey paper focuses on reviewing recent developments in applying artificial intelligence to intrusion detection in the IoT domain. Selected articles are classified according to the applied AI algorithm. This study provides an in-depth survey highlighting the recent advances in artificial intelligence for improving the security of IoT. It summarizes and organizes the recent related research, then presents a comprehensive discussion on research challenges, open issues, and needed future research. [ABSTRACT FROM AUTHOR]

Artificial intelligence research in the area of computer vision teaches machines to comprehend and interpret visual data. Machines can properly recognize and classify items using digital images captured by cameras and videos, deep learning models, and then respond to what they observe. Similarly, artificial intelligence has also been able to learn complex images captured by Synthetic Aperture Radar (SAR) that are widely used for various purposes but still leave room for improvements. Researchers have proposed numerous approaches in this field, from SAR target detection to SAR target recognition. This paper presents a survey on the different techniques and architectures proposed in the literature for various SAR image applications. The paper covers a survey on target detection models and target recognition models and their respective workflow to analyze the techniques involved and the performances of these models. This paper makes novel discussions, comparisons, and observations. It highlights the advantages and disadvantages of different approaches to give researchers the idea of how each technique can influence the performance for adoption in the future. The potential future directions along with hybrid models on each processing method are also highlighted based on the study. • State-of-the-art techniques on SAR image processing comprising detection and recognition are discussed. • The different architectures and parameter settings of each technique are also examined. • The significant advantages and disadvantages of the existing techniques are also discussed to give researchers insights into each method. • Necessary observations on each processing technique are discussed based on the study. • Future approaches to improving the existing techniques for performance enhancement are also highlighted, along with a few potential models. [ABSTRACT FROM AUTHOR]

The use of artificial intelligence (AI) in medicine is beginning to alter current procedures in prevention, diagnosis, treatment, amelioration, cure of disease and other physical and mental impairments. In addition to raising concerns about public trust and ethics, advancements in this new emerging technology have also led to a lot of debate around its integration into healthcare. The objective of this work is to introduce researchers to AI and its medical applications, along with their potential pitfalls, in a comprehensive manner. This paper provides a review of current studies that have investigated how to apply AI methodologies to create a smart predictive maintenance model for the industries of the future. We begin with a brief introduction to AI and a decade's worth of its advancements across a variety of industries, including smart grids, train transportation, etc., and most recently, healthcare. In this paper, we explore the various applications of AI across various medical specialties, including radiology, dermatology, haematology, ophthalmology, etc. along with the comparative study by employing several key criteria. Finally, it highlights the challenges for large-scale integration of AI in medical systems along with a summary of the ethical, legal, trust, and future implications of AI in healthcare. [ABSTRACT FROM AUTHOR]

This paper considers the task of rice leaf disease identification using transfer-learned deep learning models. The similarity between various symptoms and the inability to distinguish diseases at glance have become a major challenge in the area. Modern deep CNN-based models have acquired state-of-the-art performance and they frequently incorporate global image as an input to learn the model. The major drawback of such a system for rice leaf disease identification is that the diseases often affect small area; CNN models trained on global images may suffer from irrelevant noisy regions. This paper proposes a Dynamic Mode Decomposition based on attention-driven preprocessing for rice leaf disease identification. The four different categories of rice leaf diseases such as bacterial blight, blast, brown spot and tungro (3416 images in total) are considered for the study. Four types of experiments are conducted in this work, Initially the effectiveness of 10 transfer-learned Deep CNN (DCNN) models for rice leaf disease identification is analyzed with an accuracy of 93.87%, DenseNet121 outperformed other transfer-learned models. Secondly, 3 machine learning models are trained on the deep features extracted from the final fully connected layers of DCNN models. The simulation results indicate that the DenseNet121 deep feature with Random Forest classifier performs better compared to other deep feature and machine learning algorithms. Furthermore, Dynamic mode decomposition (DMD) based attention-driven pre-processing mechanism to localize the infected region is investigated. The sparse component obtained from the DMD algorithm is processed to produce a hard attention map which is further multiplied with the original images to generate hard segmentation map. The influence of hard segmentation maps which localizes the infected regions for rice leaf disease identification is investigated. The performance of 10 DCNN models was evaluated using transfer learning strategy and machine learning models learned on deep features both on original images and DMD preprocessed images. The XceptionNet deep feature with SVM classifier on DMD preprocessed images outperforms other models with a test accuracy of 100%. Finally, the performance of the proposed DMD-based attention-driven pre-processing was investigated on on-field rice leaf images. XceptionNet model achieved a better classification accuracy of 94.33, compared to other transfer-learned models. The Analysis in terms of Accuracy, Precision, Recall and F1-Score revealed that the models trained on DMD pre-processed images exhibit significantly better performance. • Analysis of the transfer learned models on rice leaf disease identification using 10 state-of-the-art DCNN models. • Development of ML-based identification models on deep features extracted from the aforementioned DCNN models. • Attention-driven preprocessing of rice-leaf disease dataset using Dynamic Mode Decomposition based ROI extraction algorithm. • Analysis of the impact of DMD preprocessed images on transfer learned DCNN classifiers and ML models. • The effectiveness of DMD pre-processed images on on-field rice leaf images was evaluated using transfer-learned DCNN models. [ABSTRACT FROM AUTHOR]

Artificial intelligence based on deep learning is better at improving the representation ability of models from data. However, due to the limitation of fixed receptive field, these agents are not able to provide a correct response outside the fixed receptive field. To address this problem, this paper provides a new perspective with improving the Image Recognition tasks. This study firstly constructs two extended receptive fields using structural causal model. Then, an approximate intervention method that changes the traditional likelihood prediction to predict the result of causal intervention is proposed. Finally, this study formulates the objective function to adapt the proxy training, which makes the whole model work well. Above all of these, a new Vision Transformer variant named Causal-ViT is proposed. Furthermore, rich experimental results of different tasks are reported. These results show that the proposed perspective makes a significant improvement in Image Recognition tasks. By simply plugging Causal-ViT to different sub-tasks, all of them bring the new benchmarks of themselves field, which proves our method is flexible. • The structural causal model is introduced for removing observation bias. • The external knowledge is incorporated into causal intervention processing. • A fusion module is designed for adapting original features and causality context. • This study conducts rich experiments and achieves new SOTA in most benchmarks. [ABSTRACT FROM AUTHOR]

Deep-learning-based identification and diagnosis of faults in industrial assets has received much attention. It does not need a deep understanding of the target domain or complex signal-processing techniques for time-consuming feature identification and selection. The vast majority of approaches developed in this field rely on public data recorded in laboratory conditions with high-performance measurement equipment. Moreover, primarily vibration data is used for condition monitoring tasks, which is particularly sensitive to typical error patterns in rotating machinery. These conditions are difficult to maintain in industrial environments since using high-performance measurement systems would not be economically feasible. This paper demonstrates that modern deep learning can achieve extraordinary fault detection results with industrial standard low-cost measurement systems. We determine measurements taken from the industrial-sensory equipment that potentially capture the system's fault patterns. Hereafter, we apply traditional failure detection and identification techniques to evaluate the suitability of the taken measurements for bearing fault detection. These techniques apply envelope analysis to reveal the fault patterns in the signals and a support vector machine to separate the fault classes. We can conclude that the used low-cost sensory equipment is capable to capture meaningful, fault-describing patterns. To improve the failure detection performance we are investigating a 2D, as well as 1D, convolutional neural network approach to identify the error patterns and classify the respective errors. We compare the deep-learning-based methods with the traditional methods. Furthermore, we assess which inverter signal carries the largest fault-describing information content. Experimental results indicate that the proposed deep learning methods outperform traditional fault diagnosis methods, hence, demonstrating the effectiveness in an industrial condition monitoring application. [ABSTRACT FROM AUTHOR]

Recently, facial expression recognition (FER) methods have achieved significant progress. However, FER is still challenged by factors such as uneven illumination and low-quality expression images. Exploring the potential of facial expression features to achieve robust FER is particularly important. Inspired by the Transformer's excellent performance in modeling long-range dependencies and in vision tasks, this paper proposes a Transformer Block Enhancement Module (TBEM) for enhancing the feature representation of facial expressions. The proposed module contains a Channel Enhancement (CE) block and a Spatial Enhancement (SE) block. The CE block can adaptively enhance the expression features on the channel dimension by effectively leveraging the channel dependency information, while the SE block enhances the expression features on the spatial dimension by integrating the spatial dependency information. TBEM can output a more robust expression representation by combining CE and SE. The proposed artificial intelligence learning module greatly improves the recognition accuracy of FER engineering tasks. To further illustrate the application of TBEM in real-world FER engineering, three engineering problems are used for verification. Extensive experiments demonstrate that the proposed method improves FER performance by focusing on more accurate decisional features and can be easily embedded into regular convolutional neural network models to help them improve the accuracy on FER tasks by about 2.64%–3.03%. The results show the proposed method achieves accuracies of 90.57% on FERPlus, 89.41% on RAFDB basic and 68.43% on RAFDB compound, respectively. The proposed method also provides a meaningful reference for further research on applying Transformer to other tasks. [ABSTRACT FROM AUTHOR]

Audio commands are a preferred communication medium to keep inspectors in the loop of civil infrastructure inspection performed by a semi-autonomous drone. To understand job-specific commands from a group of heterogeneous and dynamic inspectors, a model must be developed cost-effectively for the group and easily adapted when the group changes. This paper is motivated to build a multi-tasking deep learning model that possesses a Share–Split–Collaborate architecture. This architecture allows the two classification tasks to share the feature extractor and then split subject-specific and keyword-specific features intertwined in the extracted features through feature projection and collaborative training. A base model for a group of five authorized subjects is trained and tested on the inspection keyword dataset collected by this study. The model achieved a 95.3% or higher mean accuracy in classifying the keywords of any authorized inspectors. Its mean accuracy in speaker classification is 99.2%. Due to the richer keyword representations that the model learns from the pooled training data Adapting the base model to a new inspector requires only a little training data from that inspector Like five utterances per keyword. Using the speaker classification scores for inspector verification can achieve a success rate of at least 93.9% in verifying authorized inspectors and 76.1% in detecting unauthorized ones. Further The paper demonstrates the applicability of the proposed model to larger-size groups on a public dataset. This paper provides a solution to addressing challenges facing AI-assisted human–robot interaction Including worker heterogeneity Worker dynamics And job heterogeneity. • The Share–Split–Collaborate multitask learning architecture is suitable for speaker-keyword classification. • Subject-specific and phonetic-specific features intertwined in audio data can be disentangled. • Rich keyword representations are learned from multi-subject spoken command data. • Small data of new speakers are sufficient for adding new classes to the speaker classifier. • Speaker classification scores are also effective for the speaker verification. [ABSTRACT FROM AUTHOR]

In the context of mechanical engineering design, the field of machine learning accelerated topology optimization is dominated by image-based models trained in a supervised manner on datasets with a limited diversity of boundary conditions. State-of-the-art methods show poor generalization capabilities and are strongly coupled to finite element mesh resolution, hindering scalability. In this paper, we leverage the explicit topology parameterization of the moving morphable components (MMC) framework to train a deep learning model that directly generates geometric design variables using a model architecture that is independent of the finite element mesh used for structural analysis. The developed model is trained on a large dataset of boundary conditions. Despite achieving state-of-the-art regression loss, evaluations reveal that direct-design approaches generate topologies with poor mechanical performance. Specifically, the model-generated topologies have, on average, a stiffness 11.48% lower than conventional MMC designs, as evidenced by in-distribution and out-of-distribution test samples. We demonstrate that this is due to the incompatibility between the regression loss function typically used in literature and the topology optimization objective of compliance minimization. To address this issue, we propose a novel acceleration approach that leverages the trained model to generate improved initial designs for conventional optimization. Specifically, the deep learning model is used to generate an initial design, which is then refined by conventional optimization to arrive at a final, optimal design. This approach shows a computation time-saving of 36.84% without sacrificing the final mechanical performance of the optimal topology compared to conventional optimization starting from a uniform initial layout. [ABSTRACT FROM AUTHOR]

Level Crossings are among the most critical railway assets, concerning both the risk of accidents and their maintainability, due to intersections with promiscuous traffic and difficulties in remotely monitoring their health status. Failures can be originated from several factors, including malfunctions in the bar mechanisms and warning devices, such as light signals and bells. This paper focuses on the intelligent detection of anomalies in warning bells through non-intrusive acoustic monitoring by: (1) introducing a new concept for autonomous monitoring of level crossings; (2) generating and sharing a specific dataset collecting relevant audio signals from publicly available audio recordings; (3) implementing and evaluating a solution combining deep learning and transfer learning for warning bell detection. The results show a high accuracy in detecting anomalies and suggest viability of the approach in real-world applications, especially where network cameras with on-board microphones are installed for multi-purpose level crossing surveillance. • New concept for autonomous monitoring of level crossings through audio–video analytics. • Novel approach for level crossing bell detection using deep learning and transfer learning. • Generation of a specific dataset collecting relevant audio signals. • Results show high accuracy and suggest feasibility within smart railway maintenance. [ABSTRACT FROM AUTHOR]

Home automation is seen as a potential pillar of the smart city revolution that combines smart mobility, lifestyle and ecosystem governed by intelligent sensors connected to the internet. Households can save money and be more comfortable with automated appliances. The cost of electricity and user comfort are fundamentally contradictory, so they can be presented as a dynamic multi-objective optimization problem with fluctuating priorities for the customer to use various devices at different times. For this reason, this paper proposes an advanced Intelligent Home Energy Management (IHEM) approach based on reinforcement learning to achieve home demand response (DR) efficiency. The optimal formulation of the one-hour-ahead energy consumption scheduling problem is considered a Markov Decision Process (MDP) with discrete time steps. An efficient Neural Network (NN)-based approach with a Q-learning algorithm is developed to address this problem, enabling the IHEM system to achieve better cost-effective scheduling performance. The accurate data of electricity price and energy supplied by the Photovoltaic (PV) system are analyzed in sliding periods by machine learning for uncertainty prediction. Using the newly developed approach, which has the dual objective of minimizing the electricity bill, it is possible to obtain scheduling decisions for appliances and energy storage. The results show that the proposed optimization method reduces the monthly electricity costs by 20% compared to the Integer Linear Programming (ILP)-based HEMS method. • Artificial intelligence (AI) approach to monitor household energy consumption. • An efficient deep reinforcement learning algorithm to control activity recognition in smart homes. • Design and implementation of Intelligent home energy management system. • Converge promptly and considerably reduce operating expenses adopting AI-based approach. [ABSTRACT FROM AUTHOR]

Detecting objects in hospital indoor environments is critical for scene understanding and can have various applications in healthcare. Deep learning algorithms have proven to be effective in object recognition from images or videos, but the availability of annotated datasets plays a crucial role in their successful application. However, there is a shortage of datasets for object detection in hospital settings, hindering the advancement of hospital indoor object detection algorithms. In this paper, we present the Hospital Indoor Object Detection (HIOD) dataset, consisting of 4,417 images covering 56 object categories. The HIOD dataset represents the frequently encountered objects in hospitals and comprises 51,869 annotated objects. The dataset is characterized by dense annotation, with an average of 11.7 objects and 6.8 object categories per image. An object detection benchmark was established using the HIOD dataset and eight state-of-the-art object detectors. The benchmark provides a comprehensive evaluation of the performance of the selected object detectors on a large and diverse set of images of objects commonly seen in hospital environments. The results of the benchmark can be used to compare and analyze the performance of different object detectors and identify their strengths and weaknesses for use in hospital environments. In the benchmark, one-stage detectors have shown superior performance compared to two-stage detectors of similar parameter sizes. In particular, YOLOv6-L was able to attain a mean Average Precision (mAP) of 51.7% while operating at a detection speed of 255 FPS. The benchmark and dataset can serve as a valuable resource for researchers and practitioners in the field of computer vision and robotics, helping to advance the development of more effective and efficient object detection algorithms for developing automated operations in hospitals such as robotic disinfection and patient assistance. • A new object detection dataset for artificial intelligence applications in hospitals. • A benchmark to compare and analyze the performance of different object detectors. • Improve object detection via hyperparameter optimization and transfer learning. • Evaluate the effects of image size and backbone on object detection performance. [ABSTRACT FROM AUTHOR]

When addressing product quality standards in manufacturing lines, a critical issue is the identification of the parameters that define the quality of the final product and their tracking. The problem of process control under inconsistent working condition of an automatic machinery, i.e. when some parameters are highly variable, is still quite unexplored in literature. This objective becomes even more challenging when the most important process variables are not directly measurable. This paper demonstrates that it is possible to achieve quality control by coupling a soft sensor, whose predictive model is a neural network, with an anomaly detector. The methodology has been applied to automatic machinery placed in a manufacturing line, where high variability in production rate has an important effect on the measured physical variables. This makes automated and accurate quality control difficult, due to the fact that in this test case the data collected are accelerometers signals, extremely sensible to variation in machine productivity by definition. It is shown that this approach outperforms many other classification methods (Support Vector Machines, Ensemble Bagged Tree, Discriminant Analysis, K-nearest neighbours and the direct application of a Neural Network) proposed in the past, achieving satisfactory results evaluated on the basis of four metrics (Accuracy, precision, recall and F 1 -score), even if anomalous data have been collected in a limited number of machine's working points. In particular, an accuracy over 92% has been reached also for production rates where only nominal conditions are collected. This procedure exceeds the direct training of a neural network (accuracy of 57.6% at new production rates), as well as the application of shallow methods based on the extraction of dimensionless features (around 35% in accuracy at new production rates). • Quality control is arduous when it is not possible to measure process variables. • Hard to build robust soft sensor with system working under inconsistent conditions. • A neural network can be trained at specific machine working points. • Anomaly detection is achieved by deep learning and a probabilistic model. • Accuracy higher than 92% for each production rate tested. [ABSTRACT FROM AUTHOR]

Automatic modulation classification (AMC) is an essential and indispensable topic in the development of cognitive radios. It is the cornerstone of adaptive modulation and demodulation capabilities to perceive and understand surrounding environments and make corresponding decisions. In this paper, we propose a priori regularization method in deep learning (DL-PR) for guiding loss optimization during model training process. The regularization factor designed by the combination of inter-class confrontation factor, global and dimensional divergence can help increase the inter-class distance and reduce the intra-class distance of samples. While preserving the original information of received signals as much as possible, it makes full use of the prior knowledge in the signal transmission process and ultimately helps deep learning models to be well generalized on signals with various signal-to-noise ratios (SNRs). As far as we know, this is the first attempt to regularize deep learning models based on SNR distribution of samples to improve AMC accuracy. Moreover, it can be proved that priori regularization can be interpreted as implicit data augmentation and model ensemble methods. By comparing with a series of state-of-the-art AMC methods and different regularization techniques on the public dataset RadioML 2016.10a, experimental results of multiple deep learning models illustrate the superiority of DL-PR, including CNN with accuracy of 62.6% and inference time of 0.82 ms per signal, LSTM with 61.8% and 0.87 ms, and hybrid CNN–LSTM with 64.2% and 0.94 ms. In practical applications, DL-PR can be also easily applied to complex environments due to its robustness to hyper-parameters and SNR estimation. [ABSTRACT FROM AUTHOR]

Identification of exfoliated graphene flakes and classification of the thickness are important in the nanomanufacturing of advanced materials and devices. This paper presents a deep learning method to automatically identify and classify exfoliated graphene flakes on Si/SiO 2 substrates from optical microscope images. The presented framework uses a hierarchical deep convolutional neural network that is capable of learning new images while preserving the knowledge from previous images. The deep learning model was trained and used to classify exfoliated graphene flakes into monolayer, bi-layer, tri-layer, four-to-six-layer, seven-to-ten-layer, and bulk categories. Compared with existing machine learning methods, the presented method showed high accuracy and efficiency as well as robustness to the background and resolution of images. The results indicated that the pixel-wise accuracy of the trained deep learning model was 99% in identifying and classifying exfoliated graphene flakes. This research will facilitate scaled-up manufacturing and characterization of graphene for advanced materials and devices. • A machine intelligence approach is developed to classify exfoliated graphene flakes. • A hierarchical deep convolutional neural network is applied for the classification task. • The neural network can learn from new images while preserving the learned knowledge. • The approach is robust to the background and resolution of microscopy images. • Advanced vision techniques are incorporated to handle highly imbalanced datasets. [ABSTRACT FROM AUTHOR]

Recently, developing automated video surveillance systems (VSSs) has become crucial to ensure the security and safety of the population, especially during events involving large crowds, such as sporting events. While artificial intelligence (AI) smooths the path of computers to think like humans, machine learning (ML) and deep learning (DL) pave the way more, even by adding training and learning components. DL algorithms require data labeling and high-performance computers to effectively analyze and understand surveillance data recorded from fixed or mobile cameras installed in indoor or outdoor environments. However, they might not perform as expected, take much time in training, or not have enough input data to generalize well. To that end, deep transfer learning (DTL) and deep domain adaptation (DDA) have recently been proposed as promising solutions to alleviate these issues. Typically, they can (i) ease the training process, (ii) improve the generalizability of ML and DL models, and (iii) overcome data scarcity problems by transferring knowledge from one domain to another or from one task to another. Although the increasing number of articles proposed to develop DTL- and DDA-based VSSs, a thorough review that summarizes and criticizes the state-of-the-art is still missing. To that end, this paper introduces, to the best of the authors' knowledge, the first overview of existing DTL- and DDA-based video surveillance to (i) shed light on their benefits, (ii) discuss their challenges, and (iii) highlight their future perspectives. [ABSTRACT FROM AUTHOR]

The application of Artificial Intelligence (AI) is a popular trend to make damage inspection and analysis in structural health monitoring more intelligent and automatic. However, the existing AI-based approaches, especially vision-based methods, mainly focus on damage identification and quantification from images without further analysis to obtain structural load-carrying performance. This paper proposes a Damage-T Generative Adversarial Network (Damage-T GAN) to achieve fast translation from real-world crack images to numerical damage contours. To verify its applicability and effectiveness, two datasets from different reinforced concrete beams were built, and the performance of the trained GAN model was evaluated against the metrics IS, FID, etc. After obtaining the damage contour, a purely visual approach was applied to quantify the damage. As a result, the proposed framework greatly helps field engineers to quickly judge the damage stage of beams in site scenes by simultaneous acquisition of real-world crack images and the generated damage contours of numerical model. [ABSTRACT FROM AUTHOR]

The application of test cases for detecting the faults within the software is called software testing. Manual testing is laborious and time-consuming hence automation tools to test software were introduced. Despite the use of automation tools at the User Interface (UI) level of the test pyramid, the limitations of current automation tools like automated test case generation and automated repairing of fragile tests still force us to carry out a large amount of manual testing. In this paper, we propose a novel method using AI to address the given challenges. With our proposed method test cases are automatically generated from the structure of the UI using a pipelined architecture of object detection, text detection and NLP models. We show that the test cases generated by the proposed framework can be translated into executable test scripts using a simple parser. The proposed method generates an average of 98.8% correct executable test cases for the applications under study. We also show the capability of the proposed method in generating new tests automatically when the application is modified. The proposed method generates an average of 98.605% correct executable test cases when the UI is modified for the applications under study. We also empirically prove that a GPU implementation of the proposed framework results in just an additional average runtime of 0.92 seconds per test case which is significantly low given the benefits of automated generation of test scripts and automated repairing of fragile tests. [ABSTRACT FROM AUTHOR]

Detection transformer (DETR) combines convolutional neural network (CNN) with transformer, providing a more advanced idea. In this paper, an object detection model based on DETR is proposed for fire and smoke detection. Compared with other methods that based on deep learning, the proposed one simplifies the pipeline of detection and builds an end-to-end detector. At the same time, the original DETR model usually requires long training time and large amount of computation, resulting in relatively poor performance in detection speed and accuracy, and it shows to be not friendly for small or early fire detection. Therefore, when designing the proposed network model, a normalization-based attention module is added in the feature extraction stage to highlight the effective features, being beneficial to the process of encoding. A multiscale deformable attention is also used in the encoder–decoder structure, accelerating therefore, the process of convergence during the training of the model, which includes the enhancement of the detection of small objects. Also, considering the cost of computation, the number of layers in the encoder–decoder structure is redefined to reduce the complexity of the model, and that also reduces the requirements of the application equipment. Detailed experiments are conducted on three self-built datasets and two public video sets. The results show that, the proposed method has an excellent performance on all of the datasets considered here. [ABSTRACT FROM AUTHOR]

Freezing of gait (FOG) is one of the most incapacitating symptoms in Parkinson's disease, affecting more than 50% of patients in advanced stages of the disease. The presence of FOG may lead to falls and a loss of independence with a consequent reduction in the quality of life. Wearable technology and artificial intelligence have been used for automatic FOG detection to optimize monitoring. However, differences between laboratory and daily-life conditions present challenges for the implementation of reliable detection systems. Consequently, improvement of FOG detection methods remains important to provide accurate monitoring mechanisms intended for free-living and real-time use. This paper presents advances in automatic FOG detection using a single body-worn triaxial accelerometer and a novel classification algorithm based on Transformers and convolutional networks. This study was performed with data from 21 patients who manifested FOG episodes while performing activities of daily living in a home setting. Results indicate that the proposed FOG-Transformer can bring a significant improvement in FOG detection over the reproduction of related approaches based on machine and deep learning (i.e., from 0.916 to 0.957 in the AUC metric compared with the baseline, with a corresponding sensitivity, specificity, and precision of 0.842, 0.939 and 0.617, respectively) using a leave-one-subject-out cross-validation (LOSO CV). These results present opportunities for the implementation of accurate monitoring systems for use in ambulatory or home settings. [ABSTRACT FROM AUTHOR]

The artificial intelligence (AI) revolution offers significant opportunities to capitalise on the growth of digitalisation and has the potential to enable the 'system of systems' approach required in increasingly complex infrastructure systems. This paper reviews the extent to which research in economic infrastructure sectors has engaged with fields of AI, to investigate the specific AI methods chosen and the purposes to which they have been applied both within and across sectors. Machine learning is found to dominate the research in this field, with methods such as artificial neural networks, support vector machines, and random forests among the most popular. The automated reasoning technique of fuzzy logic has also seen widespread use, due to its ability to incorporate uncertainties in input variables. Across the infrastructure sectors of energy, water and wastewater, transport, and telecommunications, the main purposes to which AI has been applied are network provision, forecasting, routing, maintenance and security, and network quality management. The data-driven nature of AI offers significant flexibility, and work has been conducted across a range of network sizes and at different temporal and geographic scales. However, there remains a lack of integration of planning and policy concerns, such as stakeholder engagement and quantitative feasibility assessment, and the majority of research focuses on a specific type of infrastructure, with an absence of work beyond individual economic sectors. To enable solutions to be implemented into real-world infrastructure systems, research will need to move away from a siloed perspective and adopt a more interdisciplinary perspective that considers the increasing interconnectedness of these systems. • Literature on the use of AI in infrastructure sectors is reviewed. • Types of AI, trends across sectors and purposes within infrastructure are identified. • Findings have been evaluated using an analysis framework. • The future of AI and the 'Survey PaperInternet of Things' in infrastructure are discussed. [ABSTRACT FROM AUTHOR]

Adding memory to artificial intelligence systems in an effective way has been addressed by researchers for many years. Recurrent neural networks and long short-term memories (LSTMs), among other neural network systems, have some inherent memory capabilities. Recently, in memory augmented neural networks, such as neural Turing machine (NTM) and its variants, a separate memory module is implemented, which can be accessed via read and write heads. Despite its capabilities in simple algorithmic tasks, such as copying and repeat copying, neural Turing machines fail when doing complex tasks with long-term dependencies due to their limited memory capacity. In this paper, we propose a new memory module in which data storing and access mechanisms are based on a graph-based neural structure rather than a matrix model. This is inspired by the human memory system, in which memories are stored via synapses (connections between neurons) and are recalled through a path passing from different neuronal networks. Differentiable mechanisms are designed for this graph-based neural memory so that it can be trained via backpropagation. The proposed structure is used to solve tasks with long-time dependencies and operations on the input sequences as well as the bAbI question answering dataset. The analysis shows that the proposed memory system can help solve these tasks better than NTM and LSTM in terms of the convergence speed and the final error [ABSTRACT FROM AUTHOR]

The building internet of things (BIoT) is quite a promising concept for curtailing energy consumption, reducing costs, and promoting building transformation. Besides, integrating artificial intelligence (AI) into the BIoT is essential for data analysis and intelligent decision-making. Thus, data-driven approaches to infer occupancy patterns usage are gaining growing interest in BIoT applications. Typically, analyzing big occupancy data gathered by BIoT networks helps significantly identify the causes of wasted energy and recommend corrective actions. Within this context, building occupancy data aids in the improvement of the efficacy of energy management systems, allowing the reduction of energy consumption while maintaining occupant comfort. Occupancy data might be collected using a variety of devices. Among those devices are optical/thermal cameras, smart meters, environmental sensors such as carbon dioxide (CO 2), and passive infrared (PIR). Even though the latter methods are less precise, they have generated considerable attention owing to their inexpensive cost and low invasive nature. This article provides an in-depth survey of the strategies used to analyze sensor data and determine occupancy. The article's primary emphasis is on reviewing deep learning (DL), and transfer learning (TL) approaches for occupancy detection. This work investigates occupancy detection methods to develop an efficient system for processing sensor data while providing accurate occupancy information. Moreover, the paper conducted a comparative study of the readily available algorithms for occupancy detection to determine the optimal method in regards to training time and testing accuracy. The main concerns affecting the current occupancy detection system in terms of privacy and precision were thoroughly discussed. For occupancy detection, several directions were provided to avoid or reduce privacy problems by employing forthcoming technologies such as edge devices, Federated learning, and Blockchain-based IoT. • The first review of deep and transfer learning for building occupancy detection. • Building occupancy detection systems based on sensor data and video analysis. • A well-defined occupancy detection taxonomy concerning different detection aspects. • Future directions include edge computing, federated learning, and blockchain. [ABSTRACT FROM AUTHOR]

Web search is widely used to find online medical advice. As such, health-related information access requires retrieval algorithms capable of promoting reliable documents and filtering out unreliable ones. To this end, different types of components, such as query-document matching features, passage relevance estimation and AI-based reliability estimators, need to be combined. In this paper, we propose an entire pipeline for misinformation detection, based on the fusion of multiple content-based features. We present experiments which study the influence of each pipeline stage for the target task. Our technological solution incorporates signals from technologies derived from diverse research fields, including search, deep learning for natural language processing, as well as advanced supervised and unsupervised learning. To combine evidence, different score fusion strategies are compared, including unsupervised rank fusion techniques and learning-to-rank methods. The reference framework for empirically validating our solution is the TREC Health Misinformation Track, which provides several challenging subtasks that foster research on the identification of reliable and correct information for health-related decision making tasks. More specifically, we address a total recall task, the goal of which is to identify all the documents conveying incorrect information for a specific set of topics, and an ad-hoc retrieval task, aiming to rank credible and correct information over incorrect information. All variants are evaluated with an assorted set of effectiveness metrics, which includes standard search measures, such as R-Precision, Average Precision or Normalised Discounted Cumulative Gain, and innovative metrics based on the compatibility between the ranked output and two reference rankings composed of helpful and harmful documents, respectively. Our experiments demonstrate the effectiveness of the proposed pipeline stages and indicate that sophisticated supervised fusion methods do not fare better than simpler fusion alternatives. Additionally, for reliability estimation, unsupervised textual similarity performs better than textual classification based on supervised learning. The results also show that the presented approach is highly competitive when compared with state-of-the-art solutions for the same problem. • A complete system for health misinformation detection is proposed. • Several content-based features are proposed for identifying health misinformation. • The performance of the different features and against external baselines is reported. • New metrics considering harmfulness and helpfulness of retrieved documents are used. • The technology developed is freely available for other researchers or stakeholders. [ABSTRACT FROM AUTHOR]

Smart traffic engineering and intelligent transportation services are in increasing demand from governmental authorities to optimize traffic performance and thus reduce energy costs, increase the drivers' safety and comfort, ensure traffic laws enforcement, and detect traffic violations. In this paper, we address this challenge, and we leverage the use of Artificial Intelligence (AI) and Unmanned Aerial Vehicles (UAVs) to develop an AI-integrated video analytics framework, called TAU (Traffic Analysis from UAVs), for automated traffic analytics and understanding. Unlike previous works on traffic video analytics, we propose an automated object detection and tracking pipeline from video processing to advanced traffic understanding using high-resolution UAV images. TAU combines six main contributions. First, it proposes a pre-processing algorithm to adapt the high-resolution UAV image as input to the object detector without lowering the resolution. This ensures an excellent detection accuracy from high-quality features, particularly the small size of detected objects from UAV images. Second, it introduces an algorithm for recalibrating the vehicle coordinates to ensure that vehicles are uniquely identified and tracked across the multiple crops of the same frame. Third, it presents a speed calculation algorithm based on accumulating information from successive frames. Fourth, TAU counts the number of vehicles per traffic zone based on the Ray Tracing algorithm. Fifth, TAU has a fully independent algorithm for crossroad arbitration based on the data gathered from the different zones surrounding it. Sixth, TAU introduces a set of algorithms for extracting twenty-four types of insights from the raw data collected. These insights facilitate the traffic understanding using curves, histograms, heatmaps, and animations. The present work presents a valuable added value for academic researchers and transportation engineers to automate the traffic video analytics process and extract useful insights to optimize traffic performance. TAU is a ready-to-use framework for any Transportation Engineer to better understand and manage daily road traffic (video demonstrations are provided here: https://youtu.be/wXJV0H7LviU and here: https://youtu.be/kGv0gmtVEbI). The source code is available at: https://github.com/bilel-bj/TAU. [ABSTRACT FROM AUTHOR]

In many real-world applications of Machine Learning it is of paramount importance not only to provide accurate predictions, but also to ensure certain levels of robustness. Adversarial Training is a training procedure aiming at providing models that are robust to worst-case perturbations around predefined points. Unfortunately, one of the main issues in adversarial training is that robustness w.r.t. gradient-based attackers is always achieved at the cost of prediction accuracy. In this paper, a new algorithm, called Wasserstein Projected Gradient Descent (WPGD), for adversarial training is proposed. WPGD provides a simple way to obtain cost-sensitive robustness, resulting in a finer control of the robustness-accuracy trade-off. Moreover, WPGD solves an optimal transport problem on the output space of the network and it can efficiently discover directions where robustness is required, allowing to control the directional trade-off between accuracy and robustness. The proposed WPGD is validated in this work on image recognition tasks with different benchmark datasets and architectures. Moreover, real world-like datasets are often imbalanced: this paper shows that when dealing with such type of datasets, the performance of adversarial training are mainly affected in term of standard accuracy. [ABSTRACT FROM AUTHOR]

The combinatorial optimization (CO) problems on the graph are the core and classic problems in artificial intelligence (AI) and operations research (OR). For example, the Vehicle Routing Problem (VRP) and Traveling Salesman Problem (TSP) are fascinating NP-hard problems and have important significance for the existing transportation system. Traditional methods such as heuristics methods, exact algorithms, and solution solvers can already find approximate solutions on small-scale graphs. However, they are helpless for large-scale graphs and other problems with similar structures. Moreover, traditional methods often require artificially designed heuristic functions to aid decision-making. In recent years, more and more work has focused on applying deep learning and reinforcement learning (RL) to learn heuristics, which allows us to learn the internal structure of the graph end-to-end and find the optimal path under the guidance of heuristic rules. However, most of these still need manual assistance, and the RL method used has the problems of low sampling efficiency and small searchable space. This paper proposes a novel framework (called OmegaZero) based on Alphago Zero, which does not prescribe expert experience or label data but is trained through self-play. We divide the learning into two stages: in the first stage, we employ graph attention network (GAT) and GRU to learn node representations and memory history trajectories. In the second stage, we employ Monte Carlo tree search (MCTS) and deep RL to search for the solution space and train the model. [ABSTRACT FROM AUTHOR]

Artificial intelligence is steadily increasing its impact on everyday life. Therefore, the societal issues of artificial intelligence have become an important concern in the AI research. The presence of data that reflects human biases towards historically discriminated groups defined by sensitive features such as race and gender, results in machine learning models which discriminate against these groups. In order to tackle the impact of bias in data, researchers developed a variety of specialized machine learning algorithms which are able to satisfy different fairness constraints imposed on the model. Group fairness constraints do not fit standard machine learning formulations easily due to their non-differentiable nature. In this paper we developed a technique for learning a fair classifier by Monte Carlo policy gradient method which naturally deals with such non-differentiable constraints. Our methodology focuses on direct optimization of both group fairness metric and predictive performance of the model. In addition, we propose two different variance reduction techniques of gradient estimation. We compare our models to seven other related and state-of-the-art models and demonstrate that they are able to achieve better trade-off between accuracy and unfairness. To the best of our knowledge, this is the first fair classification algorithm which solves the issue of non-differentiable constraints by reinforcement learning techniques. [ABSTRACT FROM AUTHOR]

The Deep Reinforcement Learning (DRL) algorithm, Proximal Policy Optimization (PPO2), is deployed on a custom spacecraft (S/C) build and loss model to determine if an Artificial Intelligence (AI) can learn to monitor satellite constellation health and determine an optimal replacement strategy. A custom environment is created to simulate how S/C are built, launched, generate revenue, and finally decay. The reinforcement learning agent successfully learned an optimal policy for two models: a Simplified Model where the financial cost of actions is ignored; and an Advanced Model where the financial cost of actions is a major element. In both models the AI monitors the constellations and takes multiple strategic and tactical actions to replace satellites to maintain constellation performance. The Simplified Model showed that the PPO2 algorithm was able to converge on an optimal solution after ∼ 200,000 simulations. The Advanced Model was much more difficult for the AI to learn, and thus, the performance drops during the early episodes, but eventually converges to an optimal policy at ∼ 25,000,000 simulations. With the Advanced Model, the AI is taking actions that are successfully providing strategies for constellation management and satellite replacements which include these actions' financial implications. Thus, the methods in this paper provide initial research developments towards a real-world tool and an AI application that can aid various Aerospace businesses in managing Low Earth Orbit (LEO) constellations. This type of AI application may become imperative for deploying and maintaining small satellite mega-constellations. [ABSTRACT FROM AUTHOR]