Search

The global burden of disease caused by cardiovascular diseases (CVDs) is increasing despite technical advancements in healthcare because of a dramatic rise in the developing nations that are experiencing rapid health transitions. The World Health Organization (WHO) estimates 17.9 million deaths worldwide in 2021 and is connected to CVDs, or 32% of all deaths. Since ancient times, people have experimented with the methods that extend their lives. The proposed technology is still a long way for attaining the aim of lessening the mortality rates. Early detection and proactive management of CVD risk factors are crucial for reducing the burden of these diseases. In recent years, researchers have been exploring the potential of deep learning methods for predicting cardiovascular disease risk depending upon data collected from IoMT devices. Deep learning (DL) methods used for cardiovascular diseases prediction have been popular in this domain. Several DL techniques are implemented to accomplish efficient prediction-based CVD. There are several steps in the CVD employing deep learning model. IoT sensors and deep learning techniques are used to process large amounts of patient-related biomedical data, enabling doctors to closely monitor their patients and make choices in real-time. An outline of the IoT, sensors, and deep learning is provided after a discussion of cardiac disease and its existing treatments. A complete analysis of the current and pertinent deep-learning techniques for heart disease prediction is reviewed. The result shows the performance metrics of the comparison of different deep learning approaches. This review is undertaken by pulling data from 44 papers published between the years 2020 and 2023, provides a thorough statistical analysis. Finally, this survey will be beneficial for CVD prediction researchers. [ABSTRACT FROM AUTHOR]

With the continuous development of computer technology, artificial intelligence has been widely applied across various industries. To address the issues of high computational cost and inefficiency in traditional numerical methods, this paper proposes a data-driven artificial intelligence approach for solving high-dimensional neutron transport equations. Based on the AFA-3G assembly model, a neutron transport equation solving model is established using deep neural networks, considering factors that influence the neutron transport process in real engineering scenarios, such as varying temperature, power, and boron concentration. Comparing the model's predicted values with reference values, the average error in the infinite multiplication factor kinf of the assembly is found to be 145.71 pcm (10−5), with a maximum error of 267.10 pcm. The maximum relative error is less than 3.5%, all within the engineering error standards of 500 pcm and 5%. This preliminary validation demonstrates the feasibility of using data-driven artificial intelligence methods to solve high-dimensional neutron transport equations, offering a new option for engineering design and practical engineering computations. [ABSTRACT FROM AUTHOR]

This article examines whether the brain is organized based on specialized mechanisms for processing specific types of stimuli or if it is organized around mechanisms that can be applied to different types of stimuli. The authors conducted a study using deep neural networks to investigate the face inversion effect, which is a difficulty in recognizing faces when they are upside-down. They found that similar inversion effects exist for other types of stimuli, but these effects do not extend to unseen stimuli. The results suggest that the face inversion effect may be due to specialized processing mechanisms rather than general processing mechanisms. The article also discusses the potential of deep learning as a computational model for understanding human vision and the need for further research on the development of processing systems in the brain. [Extracted from the article]

Wireless traffic prediction is essential to developing intelligent communication networks that facilitate efficient resource allocation. Along this line, decentralized wireless traffic prediction under the paradigm of federated learning is becoming increasingly significant. Compared to traditional centralized learning, federated learning satisfies network operators' requirements for sensitive data protection and reduces the consumption of network resources. In this paper, we propose a novel communication-efficient federated learning framework, named FedCE, by developing a gradient compression scheme and an adaptive aggregation strategy for wireless traffic prediction. FedCE achieves gradient compression through top-K sparsification and can largely relieve the communication burdens between local clients and the central server, making it communication-efficient. An adaptive aggregation strategy is designed by quantifying the different contributions of local models to the global model, making FedCE aware of spatial dependencies among various local clients. We validate the effectiveness of FedCE on two real-world datasets. The results demonstrate that FedCE can improve prediction accuracy by approximately 27% with only 20% of communications in the baseline method. [ABSTRACT FROM AUTHOR]

Physics-informed neural networks (PINNs) represent a continuous and differentiable mapping function, approximating solution curves for given differential equations. Recent studies have demonstrated the significant potential of PINNs as an alternative or complementary approach to conventional numerical methods. However, their application in structural dynamics, such as cantilever dynamics and fluid-induced excitations, poses challenges. In particular, limited accuracy and robustness in resolving high-order differential equations, including fourth-order differential equations encountered in structural dynamics, are major problems with PINNs. To address these challenges, this study explores optimal strategies for constructing PINNs in the context of cantilever dynamics: (1) performing scaling analysis for the configuration, (2) incorporating the second-order non-linear term of the input variables, and (3) utilizing a neural network architecture that reflects a series solution of decomposed bases. These proposed methods have significantly enhanced the predictive capabilities of PINNs, showing an order-of-magnitude improvement in accuracy compared to standard PINNs in resolving the dynamic oscillation of cantilevers and fluid-induced excitation driven by added mass forces. Furthermore, this study extends to the domain of fluid-induced excitation in cantilever dynamics, representing an extreme case of coupled dynamics in fluid–structure interaction. This research is expected to establish crucial baselines for the further development of PINNs in structural dynamics, with potential applicability to high-order coupled differential equations. [ABSTRACT FROM AUTHOR]

This study presents a method for classifying landslide triggers and sizes using climate and geospatial data. The landslide data were sourced from the Global Landslide Catalog (GLC), which identifies rainfall-triggered landslide events globally, regardless of size, impact, or location. Compiled from 2007 to 2018 at NASA Goddard Space Flight Center, the GLC includes various mass movements triggered by rainfall and other events. Climatic data for the 10 years preceding each landslide event, including variables such as rainfall amounts, humidity, pressure, and temperature, were integrated with the landslide data. This dataset was then used to classify landslide triggers and sizes using deep neural networks (DNNs) optimized through genetic algorithm (GA)-driven hyperparameter tuning. The optimized DNN models achieved accuracies of 0.67 and 0.82, respectively, in multiclass classification tasks. This research demonstrates the effectiveness of GA to enhance landslide disaster risk management. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) are susceptible to adversarial examples, which are similar to original examples but contain malicious perturbations. Adversarial training is a simple and effective defense method to improve the robustness of CNNs to adversarial examples. Many works explore the mechanism behind adversarial examples and adversarial training. However, mutual information is rarely present in the interpretation of these counter-intuitive phenomena. This work investigates similarities and differences between normally trained CNNs (NT-CNNs) and adversarially trained CNNs (AT-CNNs) from the mutual information perspective. We show that although mutual information trends of NT-CNNs and AT-CNNs are similar throughout training for original and adversarial examples, there exists an obvious difference. Compared with NT-CNNs, AT-CNNs achieve a lower clean accuracy and extract less information from the input. CNNs trained with different methods have different preferences for certain types of information; NT-CNNs tend to extract texture-based information from the input, while AT-CNNs prefer shape-based information. The reason why adversarial examples mislead CNNs may be that they contain more texture-based information about other classes. Furthermore, we also analyze the mutual information estimators used in this work and find that they outline the geometric properties of the middle layer's output. [ABSTRACT FROM AUTHOR]

Digital substations have adopted a high amount of information and communication technology (ICT) and cyber–physical systems (CPSs) for monitoring and control. As a result, cyber attacks on substations have been increasing and have become a major concern. An intrusion-detection system (IDS) could be a solution to detect and identify the abnormal behaviors of hackers. In this paper, a Deep Neural Network (DNN)-based IDS is proposed to detect malicious generic object-oriented substation event (GOOSE) communication over the process and station bus network, followed by the multiclassification of the cyber attacks. For training, both the abnormal and the normal substation networks are monitored, captured, and logged, and then the proposed algorithm is applied for distinguishing normal events from abnormal ones within the network communication packets. The designed system is implemented and tested with a real-time IEC 61850 GOOSE message dataset using two different approaches. The experimental results show that the proposed system can successfully detect intrusions with an accuracy of 98%. In addition, a comparison is performed in which the proposed IDS outperforms the support vector machine (SVM)-based IDS. [ABSTRACT FROM AUTHOR]

Objectives: Accurate susceptibility evaluation results can accurately prevent and control the dangers caused by landslides. Sample optimization is an important method for landslide susceptibility evaluation, which can effectively solve the problem of decision boundary offset generated by unbalanced samples and improve the accuracy of landslide susceptibility evaluation. Methods: Taking the southeast area of Wanzhou District of Chongqing, China as an example, ten influencing factors such as strata, land use and elevation were selected to construct a landslide susceptibility evaluation index system, and the relationship between landslide and the indices was quantitatively analyzed by frequency ratio method, and on this basis, deep neural network model (DNN), synthetic minority oversampling technique-DNN model (SMOTEDNN), one-class support vector machine-DNN coupling model (OS-DNN), and OS-DNN-K-means clustering coupling model (OS-DNN-K-means) were used to evaluate landslide susceptibility. Results: The results show that the distance from the road, land use and strata are the main control factors for land‐ slide development in the study area. The accuracy evaluation results show that OS-DNN-K-means (95.61%) and OS-DNN (91.16%) could improve the landslide prediction accuracy more effectively compared with SMOTE-DNN (87.97%) and DNN (81.40%). Conclusions: Sample optimization through mixed sampling and semi-supervised classification can effectively solve the problem of sample imbalance in the study area, and provide new technical support for spatial prediction of landslide disasters [ABSTRACT FROM AUTHOR]

We study the computational complexity of (deterministic or randomized) algorithms based on point samples for approximating or integrating functions that can be well approximated by neural networks. Such algorithms (most prominently stochastic gradient descent and its variants) are used extensively in the field of deep learning. One of the most important problems in this field concerns the question of whether it is possible to realize theoretically provable neural network approximation rates by such algorithms. We answer this question in the negative by proving hardness results for the problems of approximation and integration on a novel class of neural network approximation spaces. In particular, our results confirm a conjectured and empirically observed theory-to-practice gap in deep learning. We complement our hardness results by showing that error bounds of a comparable order of convergence are (at least theoretically) achievable. [ABSTRACT FROM AUTHOR]

Emotional speech has some dependency on language or within a language itself, there are certain variations due to accents. The presence of accents degrades the performance of the speech emotion recognition (SER) system. A pre-trained accent identification system (AID) could effectively capture the characteristics of accent variations in emotional speech which is an important factor to develop a more reliable SER system. In this work, we investigate the dependencies between accent identification and emotion recognition to enhance the performance of SER. This paper proposes a novel transfer learning-based approach utilizing accent identification knowledge for SER. In the proposed method, the deep neural network (DNN) is used to model the accent identification system, which uses statistical aggregation functions (mean, std, median, etc.,) of spectral subband centroid (SSC) features and Mel-frequency discrete wavelet coefficients (MFDWC). To build the SER, the deep convolutional recurrent autoencoder produces the attention-based latent representation, and the acoustic features are extracted by the openSMILE toolkit. A separate DNN model is used to learn the mapping between attention features and acoustic features for SER. In addition, the a priori knowledge of accent can lead the SER to effect the improvement which is possible through transfer learning (TL). The performance of the proposed method is assessed using the accented emotional speech utterances of the Crema-D dataset and also compared with state-of-the-art techniques. The experimental results show that transferring AID learning improves the recognition rate of the SER and results in around 8% relative improvement in accuracy as compared to the existing SER techniques. [ABSTRACT FROM AUTHOR]

Deep learning is crucial in marine logistics and container crane error detection, diagnosis, and prediction. A novel deep learning technique using Long Short-Term Memory (LSTM) detected and anticipated errors in a system with imbalanced data. The LSTM model was trained on real operational error data from container cranes. The custom algorithm employs the Synthetic Minority Oversampling TEchnique (SMOTE) to balance the imbalanced data for operational data errors (i.e., too few minority class samples). Python was used to program. Pearson, Spearman, and Kendall correlation matrices and covariance matrices are presented. The model's training and validation loss is shown, and the remaining data are predicted. The test set (30% of actual data) and forecasted data had RMSEs of 0.065. A heatmap of a confusion matrix was created using Matplotlib and Seaborn. Additionally, the error outputs for the time series for the next n seconds were projected, with the n seconds input by the user. Accuracy was 0.996, precision was 1.00, recall was 0.500, and f1 score was 0.667, according to the evaluation criteria that were produced. Experiments demonstrated that the technique is capable of identifying critical elements. Thus, future attempts will improve the model's structure to forecast industrial big data errors. However, the advantage is that it can handle imbalanced data, which is usually what most industries have. With additional data, the model can be further improved. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have achieved breakthrough progress in various fields, largely owing to the support of large-scale datasets with manually annotated labels. However, obtaining such datasets is costly and time-consuming, making high-quality annotation a challenging task. In this work, we propose an improved noisy sample selection method, termed "sample selection framework", based on a mixup loss and recalibration strategy (SMR). This framework enhances the robustness and generalization abilities of models. First, we introduce a robust mixup loss function to pre-train two models with identical structures separately. This approach avoids additional hyperparameter adjustments and reduces the need for prior knowledge of noise types. Additionally, we use a Gaussian Mixture Model (GMM) to divide the entire training set into labeled and unlabeled subsets, followed by robust training using semi-supervised learning (SSL) techniques. Furthermore, we propose a recalibration strategy based on cross-entropy (CE) loss to prevent the models from converging to local optima during the SSL process, thus further improving performance. Ablation experiments on CIFAR-10 with 50% symmetric noise and 40% asymmetric noise demonstrate that the two modules introduced in this paper improve the accuracy of the baseline (i.e., DivideMix) by 1.5% and 0.5%, respectively. Moreover, the experimental results on multiple benchmark datasets demonstrate that our proposed method effectively mitigates the impact of noisy labels and significantly enhances the performance of DNNs on noisy datasets. For instance, on the WebVision dataset, our method improves the top-1 accuracy by 0.7% and 2.4% compared to the baseline method. [ABSTRACT FROM AUTHOR]

Forecasting energy demand has been a critical process in various decision support systems regarding consumption planning, distribution strategies, and energy policies. Traditionally, forecasting energy consumption or demand methods included trend analyses, regression, and auto-regression. With advancements in machine learning methods, algorithms such as support vector machines, artificial neural networks, and random forests became prevalent. In recent times, with an unprecedented improvement in computing capabilities, deep learning algorithms are increasingly used to forecast energy consumption/demand. In this contribution, a relatively novel approach is employed to use long-term memory. Weather data was used to forecast the energy consumption from three datasets, with an additional piece of information in the deep learning architecture. This additional information carries the causal relationships between the weather indicators and energy consumption. This architecture with the causal information is termed as entangled long short term memory. The results show that the entangled long short term memory outperforms the state-of-the-art deep learning architecture (bidirectional long short term memory). The theoretical and practical implications of these results are discussed in terms of decision-making and energy management systems. [ABSTRACT FROM AUTHOR]

The design of digital filters is now mostly automated with convolutional neural networks (CNNs). State-of-the-art works in tracking methods, including the well-known correlation and deep Siamese trackers, use features from such CNNs. However, deep learning requires huge data, high computational resources, and more training time. Hence, smart and simple alternative feature extraction strategies are needed in embedded applications. In this direction, a method is proposed for obtaining enriched "intensity and color gradient features" using the "three-dimensional gradient operator" on color images. This work considered the popular first-order gradient operator ([ - 1 0 1]) and outer product operator to generate various intensity and color gradients. The generated features contain rich information, including edges, color, and mid-level segmentation-like features. This simple yet effective operator does not involve any learning parameters. Despite not having learnable parameters, the proposed method's performance is comparable to lightweight learned CNNs such as MobileNet. The efficacy of the resultant features is demonstrated for the visual tracking task using well-known datasets in tracking, namely GOT-10k, LaSOT, OTB2015, UAV123, and VOT2018. The proposed features combined with other deep learning features have boosted the performance over baseline efficient convolution operator (ECO) tracker. This work will open new frontiers in designing hybrid features for visual tracking and other visual computation tasks. The code is available "https://github.com/dasari4321/ICG_tracker.git" [ABSTRACT FROM AUTHOR]

This paper investigates challenging fully unsupervised defogging problems, i.e., how to remove fog by feeding only foggy images in deep neural networks rather than using paired or unpaired synthetic images, and how to overcome the problems of insufficient structure and detail recovery in existing unsupervised defogging methods. For this purpose, a zero-reference image defogging method (ZRDNet) is proposed to solve these two problems. Specifically, we develop an unsupervised defogging network consisting of a layer decomposition network and a perceptual fusion network, which are separately optimized by joint multiple-loss based on the stage-wise learning. The decomposition network guides the image decomposition–reconstruction process by rationally constructing loss functions. The fusion network further enhances the details and contrast of the defogged images by fusing the decomposition–reconstruction results. The joint multiple-loss optimization strategy based on the stage-wise learning guides decomposition and fusion tasks, which are completed stage-by-stage. Additionally, a non-reference loss is constructed to prevent artifacts and distortion induced by transmission value deviation. Our method is completely unsupervised, and training only relies on fog images and information derived from the fog images themselves. Experiments are conducted to demonstrate that our ZRDNet, which overcomes the problems of insufficient structure and detail recovery, and domain shift induced by using synthetic image, achieves favorable performance. [ABSTRACT FROM AUTHOR]

Recently, there has been increasing concern about the vulnerability of deep neural network (DNN)-based synthetic aperture radar (SAR) automatic target recognition (ATR) to adversarial attacks, where a DNN could be easily deceived by clean input with imperceptible but aggressive perturbations. This paper studies the synthetic-to-measured (S2M) transfer setting, where an attacker generates adversarial perturbation based solely on synthetic data and transfers it against victim models trained with measured data. Compared with the current measured-to-measured (M2M) transfer setting, our approach does not need direct access to the victim model or the measured SAR data. We also propose the transferability estimation attack (TEA) to uncover the adversarial risks in this more challenging and practical scenario. The TEA makes full use of the limited similarity between the synthetic and measured data pairs for blind estimation and optimization of S2M transferability, leading to feasible surrogate model enhancement without mastering the victim model and data. Comprehensive evaluations based on the publicly available synthetic and measured paired labeled experiment (SAMPLE) dataset demonstrate that the TEA outperforms state-of-the-art methods and can significantly enhance various attack algorithms in computer vision and remote sensing applications. Codes and data are available at https://github.com/scenarri/S2M-TEA. [ABSTRACT FROM AUTHOR]

Endobronchial ultrasound (EBUS) is used in the minimally invasive sampling of thoracic lymph nodes. In lung cancer staging, the accurate assessment of mediastinal structures is essential but challenged by variations in anatomy, image quality, and operator-dependent image interpretation. This study aimed to automatically detect and segment mediastinal lymph nodes and blood vessels employing a novel U-Net architecture-based approach in EBUS images. A total of 1161 EBUS images from 40 patients were annotated. For training and validation, 882 images from 30 patients and 145 images from 5 patients were utilized. A separate set of 134 images was reserved for testing. For lymph node and blood vessel segmentation, the mean ± standard deviation (SD) values of the Dice similarity coefficient were 0.71 ± 0.35 and 0.76 ± 0.38, those of the precision were 0.69 ± 0.36 and 0.82 ± 0.22, those of the sensitivity were 0.71 ± 0.38 and 0.80 ± 0.25, those of the specificity were 0.98 ± 0.02 and 0.99 ± 0.01, and those of the F1 score were 0.85 ± 0.16 and 0.81 ± 0.21, respectively. The average processing and segmentation run-time per image was 55 ± 1 ms (mean ± SD). The new U-Net architecture-based approach (EBUS-AI) could automatically detect and segment mediastinal lymph nodes and blood vessels in EBUS images. The method performed well and was feasible and fast, enabling real-time automatic labeling. [ABSTRACT FROM AUTHOR]

Cancer is one of the leading causes of death in the world, with radiotherapy as one of the treatment options. Radiotherapy planning starts with delineating the affected area from healthy organs, called organs at risk (OAR). A new approach to automatic OAR segmentation in the chest cavity in Computed Tomography (CT) images is presented. The proposed approach is based on the modified U‐Net architecture with the ResNet‐34 encoder, which is the baseline adopted in this work. The new two‐branch CS‐SA U‐Net architecture is proposed, which consists of two parallel U‐Net models in which self‐attention blocks with cosine similarity as query‐key similarity function (CS‐SA) blocks are inserted between the encoder and decoder, which enabled the use of consistency regularisation. The proposed solution demonstrates state‐of‐the‐art performance for the problem of OAR segmentation in CT images on the publicly available SegTHOR benchmark dataset in terms of a Dice coefficient (oesophagus—0.8714, heart—0.9516, trachea—0.9286, aorta—0.9510) and Hausdorff distance (oesophagus—0.2541, heart—0.1514, trachea—0.1722, aorta—0.1114) and significantly outperforms the baseline. The current approach is demonstrated to be viable for improving the quality of OAR segmentation for radiotherapy planning. [ABSTRACT FROM AUTHOR]

Mahjong, a complex game with hidden information and sparse rewards, poses significant challenges. Existing Mahjong AIs require substantial hardware resources and extensive datasets to enhance AI capabilities. The authors propose a transformer‐based Mahjong AI (Tjong) via hierarchical decision‐making. By utilising self‐attention mechanisms, Tjong effectively captures tile patterns and game dynamics, and it decouples the decision process into two distinct stages: action decision and tile decision. This design reduces decision complexity considerably. Additionally, a fan backward technique is proposed to address the sparse rewards by allocating reversed rewards for actions based on winning hands. Tjong consists of 15M parameters and is trained using approximately 0.5 M data over 7 days of supervised learning on a single server with 2 GPUs. The action decision achieved an accuracy of 94.63%, while the claim decision attained 98.55% and the discard decision reached 81.51%. In a tournament format, Tjong outperformed AIs (CNN, MLP, RNN, ResNet, VIT), achieving scores up to 230% higher than its opponents. Furthermore, after 3 days of reinforcement learning training, it ranked within the top 1% on the leaderboard on the Botzone platform. [ABSTRACT FROM AUTHOR]

The conceptual design of unmanned aerial vehicles (UAVs) presents significant multidisciplinary challenges requiring the optimization of aerodynamic and structural performance, stealth, and propulsion efficiency. This work addresses these challenges by integrating deep neural networks with a multiobjective genetic algorithm to optimize UAV configurations. The proposed framework enables a comprehensive evaluation of design alternatives by estimating key performance metrics required for different operational requirements. The design process resulted in a significant improvement in computational time over traditional methods by more than three orders of magnitude. The findings illustrate the framework's capability to optimize UAV designs for a variety of mission scenarios, including specialized tasks such as intelligence, surveillance, and reconnaissance (ISR), combat air patrol (CAP), and Suppression of Enemy Air Defenses (SEAD). This flexibility and adaptability was demonstrated through a case study, showcasing the method's effectiveness in tailoring UAV configurations to meet specific operational requirements while balancing trade-offs between aerodynamic efficiency, stealth, and structural weight. Additionally, these results underscore the transformative impact of integrating AI into the early stages of the design process, facilitating rapid prototyping and innovation in aerospace engineering. Consequently, the current work demonstrates the potential of AI-driven optimization to revolutionize UAV design by providing a robust and effective tool for solving complex engineering problems. [ABSTRACT FROM AUTHOR]

This study proposes novel sales forecasting approaches that merge deep learning methods in a hybrid model. Long short‐term memory (LSTM) is adopted for modeling the temporal characteristics of the data, whereas the convolutional neural network (CNN) focuses on identifying and extracting relevant exogenous information. We propose stacked (S‐CNN‐LSTM) and parallel (P‐CNN‐LSTM) hybrid architectures to understand complex time series data with varying seasonal patterns and multiple products correlations. The performance drivers of both architectures were empirically tested with a real‐world multivariate retail dataset and outperformed when compared with simple neural network architectures and standard autoregressive methods for short and long‐term forecasting horizons. When compared with traditional predictive approaches, the proposed hybrid models reduce the computational complexity while providing flexibility and robustness. [ABSTRACT FROM AUTHOR]

Imbalanced datasets pose a significant challenge in credit card fraud detection, hindering the training effectiveness of models due to the scarcity of fraudulent cases. This study addresses the critical problem of data imbalance through an in-depth exploration of techniques, including cross-entropy loss minimization, weighted optimization, and synthetic minority oversampling technique-based resampling, coupled with deep neural networks (DNNs). The urgent need to combat class imbalances in credit card fraud datasets is underscored, emphasizing the creation of reliable detection models. The research method delves into the application of DNNs, strategically optimizing and resampling the dataset to enhance model performance. The study employs a dataset from October 2018, containing 284,807 transactions, with a mere 492 classified as fraudulent. Various resampling techniques, such as undersampling and SMOTE oversampling, are evaluated alongside weighted optimization. The results showcase the effectiveness of SMOTE oversampling, achieving an accuracy of 99.83% without any false negatives. The study concludes by advocating for flexible strategies, integrating cutting-edge machine learning methods, and developing adaptive defenses to safeguard against emerging financial risks in credit card fraud detection. [ABSTRACT FROM AUTHOR]

Intelligent forecasting of economic indexes has been an important demand for sustainable management of smart cities. Existing methods for this purpose were mostly established upon the basis of economic mechanism. Econometric models are the most general technical means in this area. However, in era of digital economy, increasing amount of big data has brought great change to traditional production. It is becoming more difficult for conventional technological forecasting methods to deal with multi-dimensional economic indexes. To deal with such challenge, this paper introduces the artificial intelligence algorithms to implement automatic information processing, and proposes a deep neural network-based intelligent forecasting method for multi-dimensional economic indexes in smart cities. Specifically, a deep neural network with three-layer structure is developed as the backbone methodology. For empirical analysis, the real-world data from "Chengdu–Chongqing Economic Circle" in China from 2012 to 2022 are selected as the main simulation scenario. Four major indexes are selected as the main research object: gross product (GDP), per capita GDP, GDP growth rate and the proportion of tertiary industry in GDP. The experimental results show that the proposal can well deal with such forecasting problem from a data-driven perspective, with a proper forecasting effect on historical data. [ABSTRACT FROM AUTHOR]

Deep learning has revolutionized many sectors of industry and daily life, but as application scale increases, performing training and inference with large models on massive datasets is increasingly unsustainable on existing hardware. Highly parallelized hardware like Graphics Processing Units (GPUs) are now widely used to improve speed over conventional Central Processing Units (CPUs). However, Complementary Metal-oxide Semiconductor (CMOS) devices suffer from fundamental limitations relying on metallic interconnects which impose inherent constraints on bandwidth, latency, and energy efficiency. Indeed, by 2026, the projected global electricity consumption of data centers fueled by CMOS chips is expected to increase by an amount equivalent to the annual usage of an additional European country. Silicon Photonics (SiPh) devices are emerging as a promising energy-efficient CMOS-compatible alternative to electronic deep learning accelerators, using light to compute as well as communicate. In this review, we examine the prospects of photonic computing as an emerging solution for acceleration in deep learning applications. We present an overview of the photonic computing landscape, then focus in detail on SiPh integrated circuit (PIC) accelerators designed for different neural network models and applications deep learning. We categorize different devices based on their use cases and operating principles to assess relative strengths, present open challenges, and identify new directions for further research. [ABSTRACT FROM AUTHOR]

AbstractSaliency maps can explain how deep neural networks classify images. But are they actually useful for humans? The present systematic review of 68 user studies found that while saliency maps can enhance human performance, null effects or even costs are quite common. To investigate what modulates these effects, the empirical outcomes were organised along several factors related to the human tasks, AI performance, XAI methods, images to be classified, human participants and comparison conditions. In image-focused tasks, benefits were less common than in AI-focused tasks, but the effects depended on the specific cognitive requirements. AI accuracy strongly modulated the outcomes, while XAI-related factors had surprisingly little impact. The evidence was limited for image- and human-related factors and the effects were highly dependent on the comparisons. These findings may support the design of future user studies by focusing on the conditions under which saliency maps can potentially be useful. [ABSTRACT FROM AUTHOR]

As a second-order method, the Natural Gradient Descent (NGD) has the ability to accelerate training of neural networks. However, due to the prohibitive computational and memory costs of computing and inverting the Fisher Information Matrix (FIM), efficient approximations are necessary to make NGD scalable to Deep Neural Networks (DNNs). Many such approximations have been attempted. The most sophisticated of these is KFAC, which approximates the FIM as a block-diagonal matrix, where each block corresponds to a layer of the neural network. By doing so, KFAC ignores the interactions between different layers. In this work, we investigate the interest of restoring some low-frequency interactions between the layers by means of two-level methods. Inspired from domain decomposition, several two-level corrections to KFAC using different coarse spaces are proposed and assessed. The obtained results show that incorporating the layer interactions in this fashion does not really improve the performance of KFAC. This suggests that it is safe to discard the off-diagonal blocks of the FIM, since the block-diagonal approach is sufficiently robust, accurate and economical in computation time. [ABSTRACT FROM AUTHOR]

High-quality cameras collect a large amount of surveillance video that can be labor-consuming for security guards to browse and analyze. One way to ease the browsing burden is to condense a long surveillance video to a much shorter clip with the technology of video synopsis. This paper introduces deep neural networks to sort object tubes for video synopsis. To the best of our knowledge, this first deep learning-based approach for video synopsis. Our approach first estimates the static background and separates foreground objects from the video. The object tubes are then obtained after stacking the foreground areas of the same object. The tubes are then represented by deep features with a 3D CNN. Finally, a Transformer sorts the object tubes and gives the final locations of all tubes. Since our approach combines tube representation extraction with neural networks, we call our approach Tube Sorting with Deep Neural Networks (TSDNN). In addition, we optimize the network with unsupervised learning that utilizes activity, collision, and chronological losses. Experiments demonstrate that the proposed TSDNN produces condensed video with few collision and chronological disorder artifacts. [ABSTRACT FROM AUTHOR]

This work presents a lightweight phoneme recognition model using object detection techniques. This model is mainly proposed to run on devices with low processing power, such as tablets and mobile phones. The use of the combination of hardware network architecture research complemented by the NetAdapt algorithm has led to the use of a simpler and lighter network architecture called MobileNet. The MobileNetV3 convolutional network architecture was combined with the Single-Shot Detection. The databases used in model training were TIMIT and LibriSpeech, both have spoken audios in English. To generate a graphical representation using the audiobases, for each audio, its spectrogram was calculated on the Mel scale. To train the algorithm of phoneme location detection, the temporal position of the occurrence of each phoneme in respective spectrogram is used. Additionally, it was necessary to increase the training dataset, in order to provide improvement in the generalization of the model. Therefore, the two databases were joined and data augmentation techniques were applied to audios. The main idea was to achieve learning using a lightweight architecture that can be used on devices with low processing power, such as tablets and mobile phones. Thus, this research used the MobileNet-Large architecture, which obtained an accuracy of 0.72 mAP@0.5IOU. For comparison, the MobileNet-Small architecture was also used, which obtained an accuracy of 0.63 mAP@0.5IOU. [ABSTRACT FROM AUTHOR]

The aim of this study is to address the challenge of 12-lead ECG delineation by different encoder–decoder architectures of deep neural networks (DNNs). This study compares four concepts for encoder–decoders based on a fully convolutional architecture (CED-Net) and its modifications with a recurrent layer (CED-LSTM-Net), residual connections between symmetrical encoder and decoder feature maps (CED-U-Net), and sequential residual blocks (CED-Res-Net). All DNNs transform 12-lead representative beats to three diagnostic ECG intervals (P-wave, QRS-complex, QT-interval) used for the global delineation of the representative beat (P-onset, P-offset, QRS-onset, QRS-offset, T-offset). All DNNs were trained and optimized using the large PhysioNet ECG database (PTB-XL) under identical conditions, applying an advanced approach for machine-based supervised learning with a reference algorithm for ECG delineation (ETM, Schiller AG, Baar, Switzerland). The test results indicate that all DNN architectures are equally capable of reproducing the reference delineation algorithm's measurements in the diagnostic PTB database with an average P-wave detection accuracy (96.6%) and time and duration errors: mean values (−2.6 to 2.4 ms) and standard deviations (2.9 to 11.4 ms). The validation according to the standard-based evaluation practices of diagnostic electrocardiographs with the CSE database outlines a CED-Net model, which measures P-duration (2.6 ± 11.0 ms), PQ-interval (0.9 ± 5.8 ms), QRS-duration (−2.4 ± 5.4 ms), and QT-interval (−0.7 ± 10.3 ms), which meet all standard tolerances. Noise tests with high-frequency, low-frequency, and power-line frequency noise (50/60 Hz) confirm that CED-Net, CED-Res-Net, and CED-LSTM-Net are robust to all types of noise, mostly presenting a mean duration error < 2.5 ms when compared to measurements without noise. Reduced noise immunity is observed for the U-net architecture. Comparative analysis with other published studies scores this research within the lower range of time errors, highlighting its competitive performance. [ABSTRACT FROM AUTHOR]

We propose an artificial intelligence approach based on deep neural networks to tackle a canonical 2D scalar inverse source problem. The learned singular value decomposition (L-SVD) based on hybrid autoencoding is considered. We compare the reconstruction performance of L-SVD to the Truncated SVD (TSVD) regularized inversion, which is a canonical regularization scheme, to solve an ill-posed linear inverse problem. Numerical tests referring to far-field acquisitions show that L-SVD provides, with proper training on a well-organized dataset, superior performance in terms of reconstruction errors as compared to TSVD, allowing for the retrieval of faster spatial variations of the source. Indeed, L-SVD accommodates a priori information on the set of relevant unknown current distributions. Different from TSVD, which performs linear processing on a linear problem, L-SVD operates non-linearly on the data. A numerical analysis also underlines how the performance of the L-SVD degrades when the unknown source does not match the training dataset. [ABSTRACT FROM AUTHOR]

We investigate the potential of combining the computational power of noisy quantum computers and of classical scalable convolutional neural networks (CNNs). The goal is to accurately predict exact expectation values of parameterized quantum circuits representing the Trotter-decomposed dynamics of quantum Ising models. By incorporating (simulated) noisy expectation values alongside circuit structure information, our CNNs effectively capture the underlying relationships between circuit architecture and output behaviour, enabling, via transfer learning, also predictions for circuits with more qubits than those included in the training set. Notably, thanks to the quantum information, our CNNs succeed even when supervised learning based only on classical descriptors fails. Furthermore, they outperform a popular error mitigation scheme, namely, zero-noise extrapolation, demonstrating that the synergy between quantum and classical computational tools leads to higher accuracy compared with quantum-only or classical-only approaches. By tuning the noise strength, we explore the crossover from a computationally powerful classical CNN assisted by quantum noisy data, towards rather precise quantum computations, further error-mitigated via classical deep learning. [ABSTRACT FROM AUTHOR]

Reinforcement learning (RL) and imitation learning (IL) are quite two useful machine learning techniques that were shown to be potential in enhancing navigation performance. Basically, both of these methods try to find a policy decision function in a reinforcement learning fashion or through imitation. In this paper, we propose a novel algorithm named Reinforcement Imitation Learning (RIL) that naturally combines RL and IL together in accelerating more reliable and efficient navigation in dynamic environments. RIL is a hybrid approach that utilizes RL for policy optimization and IL as some kind of learning from expert demonstrations with the inclusion of guidance. We present the comparison of the convergence of RIL with conventional RL and IL to provide the support for our algorithm's performance in a dynamic environment with moving obstacles. The results of the testing indicate that the RIL algorithm has better collision avoidance and navigation efficiency than traditional methods. The proposed RIL algorithm has broad application prospects in many specific areas such as an autonomous driving, unmanned aerial vehicles, and robots. [ABSTRACT FROM AUTHOR]

The population loss of trained deep neural networks often follows precise power-law scaling relations with either the size of the training dataset or the number of parameters in the network. We propose a theory that explains the origins of and connects these scaling laws. We identify variance-limited and resolution-limited scaling behavior for both dataset and model size, for a total of four scaling regimes. The variance-limited scaling follows simply from the existence of a well-behaved infinite data or infinite width limit, while the resolution-limited regime can be explained by positing that models are effectively resolving a smooth data manifold. In the large width limit, this can be equivalently obtained from the spectrum of certain kernels, and we present evidence that large width and large dataset resolution-limited scaling exponents are related by a duality. We exhibit all four scaling regimes in the controlled setting of large random feature and pretrained models and test the predictions empirically on a range of standard architectures and datasets. We also observe several empirical relationships between datasets and scaling exponents under modifications of task and architecture aspect ratio. Our work provides a taxonomy for classifying different scaling regimes, underscores that there can be different mechanisms driving improvements in loss, and lends insight into the microscopic origin and relationships between scaling exponents. [ABSTRACT FROM AUTHOR]

Humans and animals excel at generalizing from limited data, a capability yet to be fully replicated in artificial intelligence. This perspective investigates generalization in biological and artificial deep neural networks (DNNs), in both in-distribution and out-of-distribution contexts. We introduce two hypotheses: First, the geometric properties of the neural manifolds associated with discrete cognitive entities, such as objects, words, and concepts, are powerful order parameters. They link the neural substrate to the generalization capabilities and provide a unified methodology bridging gaps between neuroscience, machine learning, and cognitive science. We overview recent progress in studying the geometry of neural manifolds, particularly in visual object recognition, and discuss theories connecting manifold dimension and radius to generalization capacity. Second, we suggest that the theory of learning in wide DNNs, especially in the thermodynamic limit, provides mechanistic insights into the learning processes generating desired neural representational geometries and generalization. This includes the role of weight norm regularization, network architecture, and hyper-parameters. We will explore recent advances in this theory and ongoing challenges. We also discuss the dynamics of learning and its relevance to the issue of representational drift in the brain. [ABSTRACT FROM AUTHOR]

The purpose of the article is to present a new approach to geographic data processing using deep neural networks, which provides more efficient, accurate and automated analysis. The methodology includes the use of deep neural networks for automatic identification of characteristics of geographic objects without prior manual processing. Findings include numerical data in the form of tables and graphs demonstrating the performance improvement of GeoDNN compared to traditional methods. This article presents an improved approach to geographic data processing based on the use of deep neural networks (GeoDNN). The proposed technique provides a more efficient and automated analysis of geodata. GeoDNN uses deep neural networks to automatically identify the characteristics of geographic objects without prior manual processing. This avoids human errors and increases the accuracy of the analysis, and self-learning mechanisms ensure continuous improvement using new data. GeoDNN is characterized by high performance when processing large volumes of geodata due to its optimized structure and parallel computations. The GeoDNN architecture is described in detail, including flowcharts, mathematical formulas, and algorithms. Processes of data preparation, their normalization and augmentation, as well as model training with parameters, optimization methods and loss functions are considered. Originality is the application of deep neural networks to automate the processing of geographic data, which allows to avoid human errors, increases the accuracy of the analysis and ensures constant improvement with the use of new data. The practical value of GeoDNN lies in its ability to automate the analysis of geodata, which significantly reduces processing time and reduces the need for manual labor. This makes the system efficient for use in real-world applications such as automating land parcel analysis. [ABSTRACT FROM AUTHOR]

Pneumonia is a dangerous disease that causes severe inflammation of the air sacs in the lungs. It is one of the infectious diseases with high morbidity and mortality in all age groups worldwide. Chest X-ray (CXR) is a diagnostic and imaging modality widely used in diagnosing pneumonia due to its low dose of ionizing radiation, low cost, and easy accessibility. Many deep learning methods have been proposed in various medical applications to assist clinicians in detecting and diagnosing pneumonia from CXR images. We have proposed a novel PneumoNet using a convolutional neural network (CNN) to detect pneumonia using CXR images accurately. Transformer-based deep learning methods, which have yielded high performance in natural language processing (NLP) problems, have recently attracted the attention of researchers. In this work, we have compared our results obtained using the CNN model with transformer-based architectures. These transformer architectures are vision transformer (ViT), gated multilayer perceptron (gMLP), MLP-mixer, and FNet. In this study, we have used the healthy and pneumonia CXR images from public and private databases to develop the model. Our developed PneumoNet model has yielded the highest accuracy of 96.50% and 94.29% for private and public databases, respectively, in detecting pneumonia accurately from healthy subjects. [ABSTRACT FROM AUTHOR]

We consider a singularly perturbed system of stochastic differential equations proposed by Chaudhari et al. (Res. Math. Sci. 2018) to approximate the entropic gradient descent in the optimization of deep neural networks via homogenization. We embed it in a much larger class of two-scale stochastic control problems and rely on convergence results for Hamilton--Jacobi--Bellman equations with unbounded data proved recently by ourselves (ESAIM Control Optim. Calc. Var. 2023). We show that the limit of the value functions is itself the value function of an effective control problem with extended controls and that the trajectories of the perturbed system converge in a suitable sense to the trajectories of the limiting effective control system. These rigorous results improve the understanding of the convergence of the algorithms used by Chaudhari et al., as well as of their possible extensions where some tuning parameters are modeled as dynamic controls. [ABSTRACT FROM AUTHOR]

The paper presents the analysis of modern Artificial Intelligence algorithms for the automated system supporting human beings during their conversation in Polish language. Their task is to perform Automatic Speech Recognition (ASR) and process it further, for instance fill the computer-based form or perform the Natural Language Processing (NLP) to assign the conversation to one of predefined categories. The state-of-the-art review is required to select the optimal set of tools to process speech in the difficult conditions, which degrade accuracy of ASR. The paper presents the top-level architecture of the system applicable for the task. Characteristics of Polish language are discussed. Next, existing ASR solutions and architectures with the End-To-End (E2E) Deep Neural Network (DNN) based ASR models are presented in detail. Differences between Recurrent Neural Networks (RNN), Convolutional Neural Networks (CNN) and transformers in the context of ASR technology are also discussed. [ABSTRACT FROM AUTHOR]

Detecting double compressed (DC) speech signals is an important audio forensics task since it is highly related to the integrity and the authenticity of the recording. Adaptive multi-rate (AMR) speech codec is a popular audio compression technique specifically optimized for speech signals and it is a standard audio recording format in the vast majority of the smart phones. All of the previous studies addressing the detection of DC AMR signals report their findings for the speech signals compressed using the narrowband AMR codec (AMR-NB). Meanwhile, wideband AMR codec (AMR-WB) has been used by several mobile phone manufacturers, but DC AMR-WB speech signal detection performance remains unknown. To the best of our knowledge, this is the first study focusing on detecting the DC signals compressed using the AMR-WB speech codec. To this end, we propose three different deep neural network-based DC AMR-WB signal detection systems where the spectrogram representations of the speech signals are used as the input features. Experimental results conducted on TIMIT database provide several important findings regarding the DC AMR-WB speech detection. Firstly, DC AMR-WB detection is found to be a more challenging task than detecting the AMR-NB signals. For example, convolutional neural network (CNN)-based system yields 74.83% and 99.93% detection rates on AMR-WB and AMR-NB coded signals, respectively. Secondly, capturing the temporal information using long short-term memory (LSTM) network with the DC AMR-WB signal detection accuracy of 86.25% is found to be superior to the CNN system. Thirdly, combining the deep feature representations learned by CNN and LSTM networks further improves the performance. Fourthly, the detection rates are found to deteriorate when the signals are first encoded using different audio codecs prior to AMR-WB compression. Finally, applying score level or decision level fusion to the proposed three systems improves the detection rates, in general. [ABSTRACT FROM AUTHOR]

In the scene recognition task, there are cases where heterogeneous scenes contain items with high similarity or the spatial layout of similar scenes is too different, that is, the inter-class similarity and intra-class difference of scenes.Existing methods improve the discriminant ability of classifiers by enhancing data sets or using multi-level information complementation. Although some improvements have been made, there are still limitations.In this study, the DPE(Discriminative Patch Extraction) and TSC(Two-Stage Classification) network method are proposed to overcome the inter-class similarity and intra-class difference of scenes. DPE avoids the impact of intra-class differences on scene recognition by preserving the key information regions in images, while the TSC network avoids the impact of inter-class similarities on scene recognition by the coarse-fine two-stage training.After combining the proposed method with baseline networks such as ViT(Vision Transformer), the classification accuracy of classical scene recognition data sets Scene15, MITindoor67 and SUN397 reaches 96.9%, 88.4% and 76.0%, respectively. The proposed method achieves the highest classification accuracy of 60.5% on the largest scene recognition dataset Places365. [ABSTRACT FROM AUTHOR]

This study briefly describes the concept of guided training of deep neural networks (DNNs) to learn physically reasonable solutions. The proposed method does not need the gradients of the physical equations, although the conventional physics-informed models need the gradients. DNNs are widely used to predict phenomena in physics and mechanics. One of the issues with DNNs is that their output does not always satisfy physical equations. One approach to consider with physical equations is adding a residual of the equations into the loss function; this is called physics-informed neural network (PINN). One feature of PINNs is that the physical equations and corresponding residuals must be implemented as part of a neural network model. In addition, the residual does not always converge to a small value. The proposed model is a physics-guided generative adversarial network (PG-GAN) that uses a GAN architecture, in which physical equations are used to judge whether the neural network's output is consistent with physics. The proposed method was applied to a simple problem to assess its potential usability. [ABSTRACT FROM AUTHOR]

Identifying cases of intellectual property violation in multimedia files poses significant challenges for the Internet infrastructure, especially when dealing with extensive document collections. Typically, techniques used to tackle such issues can be categorized into either of two groups: proactive and reactive approaches. This article introduces an approach combining both proactive and reactive solutions to remove illegal uploads on a platform while preventing legal uploads or modified versions of audio tracks, such as parodies, remixes or further types of edits. To achieve this, we have developed a rule-based focused crawler specifically designed to detect copyright infringement on audio files coupled with a visualization environment that maps the retrieved data on a knowledge graph to represent information extracted from audio files. Our system automatically scans multimedia files that are uploaded to a public collection when a user submits a search query, performing an audio information retrieval task only on files deemed legal. We present experimental results obtained from tests conducted by performing user queries on a large music collection, a subset of 25,000 songs and audio snippets obtained from the Free Music Archive library. The returned audio tracks have an associated Similarity Score, a metric we use to determine the quality of the adversarial searches executed by the system. We then proceed with discussing the effectiveness and efficiency of different settings of our proposed system. [ABSTRACT FROM AUTHOR]

Traditional linear mediation analysis has inherent limitations when it comes to handling high-dimensional mediators. Particularly, accurately estimating and rigorously inferring mediation effects is challenging, primarily due to the intertwined nature of the mediator selection issue. Despite recent developments, the existing methods are inadequate for addressing the complex relationships introduced by confounders. To tackle these challenges, we propose a novel approach called DP2LM (Deep neural network-based Penalized Partially Linear Mediation). This approach incorporates deep neural network techniques to account for nonlinear effects in confounders and utilizes the penalized partially linear model to accommodate high dimensionality. Unlike most existing works that concentrate on mediator selection, our method prioritizes estimation and inference on mediation effects. Specifically, we develop test procedures for testing the direct and indirect mediation effects. Theoretical analysis shows that the tests maintain the Type-I error rate. In simulation studies, DP2LM demonstrates its superior performance as a modeling tool for complex data, outperforming existing approaches in a wide range of settings and providing reliable estimation and inference in scenarios involving a considerable number of mediators. Further, we apply DP2LM to investigate the mediation effect of DNA methylation on cortisol stress reactivity in individuals who experienced childhood trauma, uncovering new insights through a comprehensive analysis. [ABSTRACT FROM AUTHOR]

Environmental sound classification (ESC) is one of the important research topics within the nonspeech audio classification field. While deep neural networks (DNNs) have achieved significant advances in ESC recently, their high computational and memory demands render them highly unsuitable for direct deployment on resource-constrained Internet of Things (IoT) devices based on microcontroller units (MCUs). To address this challenge, we propose a novel DNN compression framework specifically designed for such devices. On the one hand, we leverage pruning techniques to significantly compress the large number of model parameters in DNNs. To reduce the accuracy loss that follows pruning, we propose a knowledge distillation scheme based on feature information from multiple intermediate layers. On the other hand, we design a two-stage quantization-aware knowledge distillation scheme to mitigate the accuracy degradation of mandatory quantization required by MCU hardware. We evaluate our framework on benchmark ESC datasets (UrbanSound8K, ESC-50) using the STM32F746ZG device. The experimental results demonstrate that our framework can achieve compression rates up to 97% while maintaining competitive inference performance compared to the uncompressed baseline. [ABSTRACT FROM AUTHOR]

While collecting training data, even with the manual verification of experts from crowdsourcing platforms, eliminating incorrect annotations (noisy labels) completely is difficult and expensive. In dealing with datasets that contain noisy labels, over-parameterized deep neural networks (DNNs) tend to overfit, leading to poor generalization and classification performance. As a result, noisy label learning (NLL) has received significant attention in recent years. Existing research shows that although DNNs eventually fit all training data, they first prioritize fitting clean samples, then gradually overfit to noisy samples. Mainstream methods utilize this characteristic to divide training data but face two issues: class imbalance in the segmented data subsets and the optimization conflict between unsupervised contrastive representation learning and supervised learning. To address these issues, we propose a Balanced Partitioning and Training framework with Pseudo-Label Relaxed contrastive loss called BPT-PLR, which includes two crucial processes: a balanced partitioning process with a two-dimensional Gaussian mixture model (BP-GMM) and a semi-supervised oversampling training process with a pseudo-label relaxed contrastive loss (SSO-PLR). The former utilizes both semantic feature information and model prediction results to identify noisy labels, introducing a balancing strategy to maintain class balance in the divided subsets as much as possible. The latter adopts the latest pseudo-label relaxed contrastive loss to replace unsupervised contrastive loss, reducing optimization conflicts between semi-supervised and unsupervised contrastive losses to improve performance. We validate the effectiveness of BPT-PLR on four benchmark datasets in the NLL field: CIFAR-10/100, Animal-10N, and Clothing1M. Extensive experiments comparing with state-of-the-art methods demonstrate that BPT-PLR can achieve optimal or near-optimal performance. [ABSTRACT FROM AUTHOR]

Optimization of machine learning architectures is essential in determining the efficacy and the applicability of any neural architecture to real world problems. In this work a generalized Newton’s method (GNM) is presented as a powerful approach to learning in deep neural networks (DNN). This technique was compared to two popular approaches, namely the stochastic gradient descent (SGD) and the Adam algorithm, in two popular classification tasks. The performance of the proposed approach confirmed it as an attractive alternative to state-of-the-art first order solutions. Due to the good results presented in the case of shallow DNN, in the last part of the article an hybrid optimization method is presented. This method consists in combining two optimization algorithms, i.e. GNM and Adam or GNM and SGD, during the training phase within the layers of the neural network. This configuration aims to benefit from the strengths of both first-and second-order algorithms. In this case a convolutional neural network is considered and its parameters are updated with a different optimization algorithm. Also in this case, the hybrid approach returns the best performance with respect to the first order algorithms. [ABSTRACT FROM AUTHOR]

This study presents a novel approach to forecasting the evolution of hysteresis stress-strain response of different types of soils under repeated loading-unloading cycles. The forecasting is made solely from the knowledge of soil properties and loading parameters. Our approach combines mathematical modeling, regression analysis, and Deep Neural Networks (DNNs) to overcome the limitations of traditional DNN training. As a novelty, we propose a hysteresis loop evolution equation and design a family of DNNs to determine the parameters of this equation. Knowing the nature of the phenomenon, we can impose certain solution types and narrow the range of values, enabling the use of a very simple and efficient DNN model. The experimental data used to develop and test the model was obtained through Torsional Shear (TS) tests on soil samples. The model demonstrated high accuracy, with an average R² value of 0.9788 for testing and 0.9944 for training. [ABSTRACT FROM AUTHOR]

Research on Chinese Sign Language (CSL) provides convenience and support for individuals with hearing impairments to communicate and integrate into society. This article reviews the relevant literature on Chinese Sign Language Recognition (CSLR) in the past 20 years. Hidden Markov Models (HMM), Support Vector Machines (SVM), and Dynamic Time Warping (DTW) were found to be the most commonly employed technologies among traditional identificationmethods. Benefiting from the rapid development of computer vision and artificial intelligence technology, Convolutional Neural Networks (CNN), 3D-CNN, YOLO, Capsule Network (CapsNet) and various deep neural networks have sprung up. Deep Neural Networks (DNNs) and their derived models are integral tomodern artificial intelligence recognitionmethods. In addition, technologies thatwerewidely used in the early days have also been integrated and applied to specific hybrid models and customized identification methods. Sign language data collection includes acquiring data from data gloves, data sensors (such as Kinect, LeapMotion, etc.), and high-definition photography.Meanwhile, facial expression recognition, complex background processing, and 3D sign language recognition have also attracted research interests among scholars. Due to the uniqueness and complexity of Chinese sign language, accuracy, robustness, real-time performance, and user independence are significant challenges for future sign language recognition research. Additionally, suitable datasets and evaluation criteria are also worth pursuing. [ABSTRACT FROM AUTHOR]

Fault localization (FL) and automated program repair (APR) are two main tasks of automatic software debugging. Compared with traditional methods, deep learning-based approaches have been demonstrated to achieve better performance in FL and APR tasks. However, the existing deep learning-based FL methods ignore the deep semantic features or only consider simple code representations. And for APR tasks, existing template-based APR methods are weak in selecting the correct fix templates for more effective program repair, which are also not able to synthesize patches via the embedded end-to-end code modification knowledge obtained by training models on large-scale bug-fix code pairs. Moreover, in most of FL and APR methods, the model designs and training phases are performed separately, leading to ineffective sharing of updated parameters and extracted knowledge during the training process. This limitation hinders the further improvement in the performance of FL and APR tasks. To solve the above problems, we propose a novel approach called MTL-TRANSFER, which leverages a multi-task learning strategy to extract deep semantic features and transferred knowledge from different perspectives. First, we construct a large-scale open-source bug datasets and implement 11 multi-task learning models for bug detection and patch generation sub-tasks on 11 commonly used bug types, as well as one multi-classifier to learn the relevant semantics for the subsequent fix template selection task. Second, an MLP-based ranking model is leveraged to fuse spectrum-based, mutation-based and semantic-based features to generate a sorted list of suspicious statements. Third, we combine the patches generated by the neural patch generation sub-task from the multi-task learning strategy with the optimized fix template selecting order gained from the multi-classifier mentioned above. Finally, the more accurate FL results, the optimized fix template selecting order, and the expanded patch candidates are combined together to further enhance the overall performance of APR tasks. Our extensive experiments on widely-used benchmark Defects4J show that MTL-TRANSFER outperforms all baselines in FL and APR tasks, proving the effectiveness of our approach. Compared with our previously proposed FL method TRANSFER-FL (which is also the state-of-the-art statement-level FL method), MTL-TRANSFER increases the faults hit by 8/11/12 on Top-1/3/5 metrics (92/159/183 in total). And on APR tasks, the number of successfully repaired bugs of MTL-TRANSFER under the perfect localization setting reaches 75, which is 8 more than our previous APR method TRANSFER-PR. Furthermore, another experiment to simulate the actual repair scenarios shows that MTL-TRANSFER can successfully repair 15 and 9 more bugs (56 in total) compared with TBar and TRANSFER, which demonstrates the effectiveness of the combination of our optimized FL and APR components. [ABSTRACT FROM AUTHOR]