Search

Multi-objective immune algorithm (MOIA) is a heuristic algorithm based on artificial immune system model. Due to its characteristics of antibody clonal selection, automatic antigen recognition and immune memory in the immune system, artificial immune algorithm has become a research hotspot in the field of multi-objective optimization after the evolutionary algorithms. In this paper, most MOIAs can be classified into three main categories according to the type of problem solving, i.e., they are mostly designed to solve multi-objective optimization problems (MOPs), dynamic MOPs, and constrained MOPs. In this paper, a comprehensive survey is presented to summarize most existing MOIAs, in which their corresponding characteristics, principles and theoretical analyses are discussed in details. Moreover, the performance of MOIAs on solving various kinds of MOPs and many-objective optimization problems is also studied in our experimental comparisons. Finally, a brief conclusion is given to summarize the current drawbacks, challenges, and some future directions for MOIAs. [ABSTRACT FROM AUTHOR]

With the rapid development of society and the economy, autonomous driving techniques are widely applied in many areas, such as autonomous vehicles, autonomous drones, and robotics. As a dominating technique, deep learning has become more and more popular for 2-D and 3-D object detection. Numerous deep learning-based methods have been proposed to solve various vision issues. To further help with the development of unmanned systems, this paper presents a comprehensive survey of the recent processes from the past five years for 3-D object detection, road detection, traffic sign detection, and traffic light detection and classification. To summarize and analyze previous works in detail, this paper only focuses on deep learning-based object detection tasks in autonomous driving that take place when the input is a point cloud or image(s). It also presents comparative results for insight comparison and inspiring future researches. [ABSTRACT FROM AUTHOR]

Coronary artery disease is among one of the diseases human suffer most. Intravascular coronary arterial image analysis consists of denoising, segmentation, detection, and three-dimensional reconstruction, having a significant meaning for auxiliary diagnosis and treatment of coronary artery disease. Intravascular ultrasound (IVUS) and intravascular optical coherence tomography (IVOCT) are the two most commonly applied intravascular coronary arterial imaging techniques. Based on these fundamental imaging techniques, in recent years, many advanced technologies from traditional machine learning algorithms to deep learning methods were employed in the analysis of intravascular coronary arterial images and made huge progress in this field. In this survey, we reviewed more than one hundred papers published in top journals or conferences such as Neural Networks and MICCAI. These papers proposed approaches or schemes for the intravascular coronary arterial image analysis, including lumen border segmentation, atherosclerotic plaque characterization, media-adventitia segmentation, stent strut detection, and three-dimensional reconstruction. Our survey began with introducing coronary artery intravascular imaging techniques, essential neural networks, and deep learning and then presented an across-the-board review of methods, applications, and trends of intravascular image analysis. This survey is more comprehensive than other articles not only for its scope and reference number but also for discussing the future direction in this field. Compared to other review papers in this field, this article could assist beginners in constructing a basic knowledge frame of coronary artery intravascular image analysis methods and brought state-of-the-art progress in this field to fellow researchers. We hope this paper could benefit either the beginners for coronary arterial image analysis or experienced researchers. [ABSTRACT FROM AUTHOR]

Statistical techniques incorporated with machine-learning algorithms in unison with gene-environment interaction are giving unparalleled understanding of complex diseases. Accurate analysis and intricate capturing of common, rare, and low MAF (Minor Allele Frequency) variants alongside gene-environmental interaction is pivotal whilst concluding reliable and accurate classification of complex diseases. Various complex diseases including genres of diabetes Type 1 and Type 2 alongside the vastly under-researched Lada (Latent Autoimmune Diabetes in Adults) diabetes require further investigation alongside significant machine learning research to gain a deeper understanding of the disease complexities. Despite existing efforts, an ideal combination of statistical techniques with optimal machine-learning algorithms that can accurately capture and model the gene-environment interaction is lacking. Intentionally exploring future and simultaneously exploiting modern-day computational methods in genomic analysis, this paper profoundly investigates both the future and present interaction of statistical analysis techniques and machine-learning algorithms and Ensembles with gene-environmental factors. In this context, this paper firstly presents a conceptual understanding of genomic conventions; secondly, conducts potential future machine learning algorithms alongside an extensive analysis of a range of classification, regression and Ensemble techniques along with exhibiting their imperative relationship and roles in investigating and classifying common, rare variants and a wide array of gene-environmental factors; and thirdly, utilisation of statistical techniques in Genome Wide Association Studies is scrutinised whilst analysing common, rare and MAF variants. As an important contribution, this paper identifies efficient machine-learning algorithms alongside Ensemble models and future potential analysis techniques and exhibits their inherent characteristics that can enhance the reliability and accuracy of the gene-environment classification analysis. [ABSTRACT FROM AUTHOR]

In this paper, the gain-scheduled state estimation issue is investigated for a kind of complex networks subject to randomly occurring nonlinearities under bit-rate constraints. An array of random variables is introduced to govern the nonlinearities whose occurring probability is a time-varying but bounded value with known upper and lower bounds. A bit-rate constraint model is established and an encoding–decoding mechanism is proposed, under which an upper bound of the decoding error is acquired. The primary purpose of the issue considered in this paper is to design a gain-scheduled state estimator to obtain an estimate of the network state with an acceptable accuracy according to available output measurements. By means of the stochastic analysis and Lyapunov stability theory, a sufficient condition is provided such that the estimation error dynamics achieve the exponentially mean-square ultimate boundedness. The required estimator gain matrix is parameterized by solving a series of matrix inequalities. A numerical simulation is exploited to show the usefulness of the obtained gain-scheduled state estimator. [ABSTRACT FROM AUTHOR]

Time-variant Yang-Baxter-like matrix equation (YBLME) with the disturbances of noises is the hotspot in various scientific disciplines. The existing research, either can not handle the noise, or rely on the build-in numerical algorithm provided by MATLAB. Therefore, it is necessary to further study the digital-computer discrete solution of this problem. In this paper, the authors propose a noise-handling ten-instant discrete solution for the time-variant YBLME. By defining the matrix-form error function, Zhang neural network (ZNN) is exploited to design the continuous-time noise-handling ZNN model. For potential digital hardware (i.e., digital computer) realization, a discrete-time ZNN-based noise-handling ten-instant (DTZNH10i) algorithm, which is capable of handling three types of noises, is proposed on the basis of a Zhang time discretization (ZTD) formula. The convergence and precision of the proposed DTZNH10i algorithm are investigated and discussed. Theoretical analyses show the correctness and effectiveness of the proposed algorithm. The paper offers three examples of time-variant YBLME with disturbances of constant, linear, and bounded random noises to illustrate the efficacy and convergence of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Image recognition based on deep learning methods has gained remarkable achievements by feeding with abundant training data. Unfortunately, collecting a tremendous amount of annotated images is time-consuming and expensive, especially in grocery product recognition tasks. It is challenging to recognise grocery products accurately when the deep learning model is trained with insufficient data. This paper proposes multi-angle Generative Adversarial Networks (MAGAN), which can generate realistic training images with different angles for data augmentation. Mutual information is employed in the novel GAN to achieve the learning of angles in an unsupervised manner. This paper aims to create training images containing grocery products from different angles, thus improving grocery product recognition accuracy. We first enlarge the fruit dataset by using MAGAN and the state-of-the-art GAN variants. Then, we compare the top-1 accuracy results from CNN classifiers trained with different data augmentation methods. Finally, our experiments demonstrate that the MAGAN exceeds the existing GANs for grocery product recognition tasks, obtaining a significant increase in the accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we present an extensive review on deep learning methods for image restoration tasks. Deep learning techniques, led by convolutional neural networks, have received a great deal of attention in almost all areas of image processing, especially in image classification. However, image restoration is a fundamental and challenging topic and plays significant roles in image processing, understanding and representation. It typically addresses image deblurring, denoising, dehazing and super-resolution. There are substantial differences in the approaches and mechanisms in deep learning methods for image restoration. Discriminative learning based methods are able to deal with issues of learning a restoration mapping function effectively, while optimisation models based methods can further enhance the performance with certain learning constraints. In this paper, we offer a comparative study of deep learning techniques in image denoising, deblurring, dehazing, and super-resolution, and summarise the principles involved in these tasks from various supervised deep network architectures, residual or skip connection and receptive field to unsupervised autoencoder mechanisms. Image quality criteria are also reviewed and their roles in image restoration are assessed. Based on our analysis, we further present an efficient network for deblurring and a couple of multi-objective training functions for super-resolution restoration tasks. The proposed methods are compared extensively with the state-of-the-art methods with both quantitative and qualitative analyses. Finally, we point out potential challenges and directions for future research. [ABSTRACT FROM AUTHOR]

This paper proposes a sensor network information fusion based distributed state estimation algorithm for tracking non-cooperative maneuvering target. In order to lower the communication burden and energy consumption, an event-triggered mechanism is introduced. The distributed state estimation algorithm for maneuvering target tracking is designed by two stages, namely, local filtering stage and consensus fusion stage. The algorithm proposed in this paper can be used to obtain the high accurate state estimation of the target by introducing a multiple suboptimal fading factor even when the target makes big maneuvering. Moreover, a contribution factor is designed in the information fusion stage to improve the accuracy of maneuvering target state estimation and reduce the consensus iteration times. Besides, the stochastic boundedness of the event-triggered distributed estimation algorithm is proved by introducing a stochastic process. Finally, Monte Carlo numerical simulation example is designed to illustrate the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]

• This paper develops a noise/fault aware training objective for incremental ELM. • This paper uses two representative algorithms to develop two noise aware ELM. • The two proposed algorithms are much better than existing ones. • The multiple set concept can further enhance the performance. • We can make other non-noise tolerant algorithms to be noise tolerant. This paper investigates noise/fault tolerant incremental algorithms for the extreme learning machine (ELM) concept. Existing incremental ELM algorithms can be classified into two approaches: non-recomputation and recomputation. This paper first formulates a noise/fault aware objective function for nonlinear regression problems. Instead of developing noise/fault aware algorithms for the two computational approaches in a one-by-one manner, this paper uses two representative incremental algorithms, namely incremental ELM (I-ELM) and error minimized ELM (EM-ELM), to develop two noise/fault aware incremental algorithms. The proposed algorithms are called generalized I-ELM (GI-ELM) and generalized EM-ELM (GEM-ELM). The GI-ELM adds k hidden nodes into the existing network at each incremental step without recomputing the existing weights. To have a fair comparison, we consider a modified version of I-ELM as a comparison algorithm. The simulation demonstrates that the noise/fault tolerance of the proposed GI-ELM is better than that of the modified I-ELM. In the GEM-ELM, k hidden nodes are added into the existing network at each incremental step. Meanwhile, all output weights are recomputed based on a recursive formula. We also consider a modified version of EM-ELM as a comparison algorithm. The simulation demonstrates that the noise/fault tolerance of the proposed GEM-ELM is better than that of the modified EM-ELM. Moreover, we demonstrate that the multiple set concept can further enhance the performance of the two proposed algorithms. Following our research results, one can make some non-noise/fault tolerant incremental algorithms to be noise/fault tolerant. [ABSTRACT FROM AUTHOR]

The process of aligning one image, in the coordinate system of another is called registration. Image registration is vital in the medical domain and is used for diagnosis, therapy planning, and treatment of diseases. Due to its huge applicability and importance, medical image registration has emerged to be a separate domain of research. Beginning from the invasive landmark based registration, innumerable algorithms have been proposed to register two images of the human physiology. However, a breakthrough in the literature occurred when an information theory based measure, mutual information was used to register two images. Since then a large number of new algorithms have developed, which use mutual information for fully automatic registration of medical images. This paper is a survey of these algorithms. Beginning from its development, it discusses about some of the major works done on mutual information based image registration. Some comparative studies with other algorithms have also been discussed. The paper ends with a discussion of the developments in medical image registration algorithms post mutual information, which primarily include deep neural network (DNN) based algorithms. [ABSTRACT FROM AUTHOR]

• This paper proposes a new feature selection method called SLMEA. • The sparse transformation and pseudo label matrix are combined to optimize. • The pseudo label matrix is used to guide the learning of low-dimensional space. • The similarity matrix is constructed adaptively. • The constraint of ℓ 2 , 1 / 2 -norm is applied to the matrix. • The SLMEA can have higher clustering accuracy and normalized mutual information. Traditional feature selection algorithms usually explore the relationship between data and cluster structure in a single space, so the internal relationship obtained is not very rich, and it is not enough to select more valuable features. To solve the above problems, in order to fully mine the intrinsic correlation information in different spaces, so that the relationship between data, features and clustering structure can be explored at the same time, this paper proposes a feature selection method based on sparse and low-dimensional representation with maximum entropy adaptive graph (SLMEA). Firstly, the SLMEA combines the sparse transform representation with pseudo-label matrix learning to optimize, and uses the pseudo-label matrix to guide the learning of sparse low-dimensional space. It can not only explore the relationship between data and pseudo-labels in data space, but also mine the association between features and pseudo-labels in feature space, so as to select more discriminative features. Secondly, based on the maximum entropy theory, the similarity matrix is constructed adaptively, so that the two different manifold structures corresponding to sparse transform representation and pseudo label matrix learning can be adaptively learned and retained in the iterative process. In addition, in order to ensure the sparsity of the transformation matrix, the constraint of ℓ 2 , 1 / 2 -norm is applied to the matrix, which can better deal with the redundant features and obtain more sparse solutions. Finally, the SLMEA uses an alternate iterative update method to optimize the objective function, and carries out extensive experiments on eight mainstream datasets. Compared with seven state-of-the-art algorithms, SLMEA can have higher clustering accuracy and normalized mutual information. [ABSTRACT FROM AUTHOR]

Modern and new integrated technologies have changed the traditional systems by using more advanced machine learning, artificial intelligence methods, new generation standards, and smart and intelligent devices. The new integrated networks like the Internet of Things (IoT) and 5G standards offer various benefits and services. However, these networks have suffered from multiple object detection, localization, and classification issues. Conventional Neural Networks (CNN) and their variants have been adopted for object detection, classification, and localization in IoT networks to create autonomous devices to make decisions and perform tasks without human intervention and helpful to learn in-depth features. Motivated by these facts, this paper investigates existing object detection and recognition techniques by using CNN models used in IoT networks. This paper presents a Conventional Neural Networks for 5G-Enabled Internet of Things Network (CNN-5GIoT) model for moving and static objects in IoT networks after a detailed comparison. The proposed model is evaluated with existing models to check the accuracy of real-time tracking. The proposed model is more efficient for real-time object detection and recognition than conventional methods. [ABSTRACT FROM AUTHOR]

• Heterogeneous networks with both heterogeneous coefficients and heterogeneous nonlinear functions are investigated. • A generalized impulsive controller is designed. • Parameter uncertainties of both linear and nonlinear dynamics are considered. • The sufficient criteria of quasi-synchronization are derived for the heterogeneous networks. The target of this paper is studying the problem of quasi-synchronization of heterogeneous Lur'e networks with uncertain parameters and impulsive effect. Considering the heterogeneity of Lur'e networks, quasi-synchronization instead of synchronization is discussed. Noting that the uncertain factors exist extensively in reality, uncertain parameters are introduced to the system in this paper. By using the Schur complement lemma, stability theory and the principle of comparison, the sufficient criteria for the heterogeneous networks to achieve quasi-synchronization are derived, and the upper bound of exponential quasi-synchronization errors is calculated. In addition, two corollaries are presented in consideration of homogeneous systems and weighted undirected graphs. Finally, three numerical simulations are given to illustrate the effectiveness of the control protocol and theoretical results. [ABSTRACT FROM AUTHOR]

• This paper proposes a disentangled module to decouple the unary term and pairwise term in the one-shot video object segmentation approaches. • The pairwise term is used to capture spatio-temporal information. Meanwhile, the salient target in each frame is enhanced in the unary term. • The pairwise term tends to seek the relationship between the target pixel and its reference pixel. The unary term seeks the salience of different pixels in each frame. This paper proposes a novel self-supervised method for one-shot video object segmentation where the object annotations are only provided in the first frame. The current self-supervised video object segmentation approaches are implemented by modeling the pairwise correspondence between the target and reference frames. The pairwise correspondence only maintains spatio-temporal consistency. However, the VOS tasks not only require a spatio-temporal relationship between the two frames but also require the salient object information for each frame. In order to achieve this goal, we propose a disentangled representation strategy to disentangle the temporal correspondence into the pairwise term and unary term. The pairwise and unary terms capture inter-frame spatio-temporal and intra-frame salient object information, respectively. To demonstrate the importance of the disentangled representation, we apply the proposed approach to DAVIS-2017 and YouTube-VOS datasets. Experimental results confirm the effectiveness of the proposed solution. [ABSTRACT FROM AUTHOR]

Studies, with the increasing concern for mental health, have shown that interventions along with social support can reduce stress and depression. However, counselling centers do not have enough resources to provide counselling and social support to all the participants in their interest. This paper helps social support organizations (e.g., university counselling centers) sequentially select the participants for interventions. Meanwhile, Deep Reinforcement Learning (DRL) has shown significant success in learning an efficient policy for sequential decision-making problems in both fully observable environments and partially observable environments with small action space. In this paper, we consider emotion propagation from other neighbours of the influencees, initial uncertainties of mental states and influence in the student network. We propose a new architecture called DRLPSO (Deep Reinforcement Learning with Particle Swarm Optimization) to enhance learning performance in a partially observable environment with a large state and action space. DRLPSO consists of two stages: the Discrete Particle Swarm Optimization (DPSO) and Deep Q-learning integrated with Long Short-Term Memory (DQ-LSTM). In the first stage, we apply DPSO by initializing n particles that converge to multiple optimal actions for each belief state. In the second stage, the action with the best Q-value from the DPSO action set is executed to obtain belief and observation (history of action). We evaluated the proposed method empirically with the simulated student networks with mental state propagation compared to the state-of-the-art algorithms. The experimental results demonstrate that DRLPSO outperforms the state-of-the-art DRL methods by an average of 32%. [ABSTRACT FROM AUTHOR]

Structure from motion (SFM), a research hotspot in the intelligent transportation field (including autonomous driving, environmental perception and augmented reality (AR) for artificial intelligence terminals), can automatically recover ego-motion state estimations and 3D scene reconstructions from multiple images or video sequences. Most existing vision methods operate offline in indoor scenes, and their reconstruction accuracies greatly depend on the tracking lifetimes and accuracies of feature points. Reprojection matrix and redundant regression noise computations are exponential disasters for the calculation and reconstruction drift of large-scale scenes. This paper proposes a quadruple tripatch-wise modular architecture (QTMA) for autonomous vehicle stereo image sequences that decomposes rigid scenes into nonrigid motion-segmented pieces for reconstruction. An advanced energy function for salient image features is established by combining multiple feature types with weighted finite-element mesh. Closed quadruple annular matching and relocation are performed via multiresolution pyramid images. The proposed incremental integral-mapping calculation method and unique tree-like stacked storage containers prolong the tracking lifetimes of consecutive frames and ensure the spatiotemporal consistency and robustness of the homonymous image features in different subsequences. Experimental results verify the effectiveness of this architecture for different transportation scenes; the frame rate processing speed reaches 30 fps, the calculation accuracy regarding the path distance difference reaches 99.49%, and the estimation results regarding the maximum speeds of motion are closer to the ground truth. The translation error of the motion pose is 0.0136%, and the rotation error is 0.0035 [deg/m], which has more yaw stability than the existing state-of-the-art methods. Furthermore, in the reconstructed point cloud quality demonstration, the mean value of roughness is reduced by 40.127%, the mean value of density is improved by 27.701%, and the accuracy reaches 91.149% within a certain distance tolerance. This paper has significant theoretical research value and application potential for positioning, path tracking, and navigation in adaptive cruise control (ACC) and advanced driver assistance systems (ADAS). [ABSTRACT FROM AUTHOR]

The present paper aims to propose a new type of information-theoretic method to augment the ratio of collective potentiality to its cost for the interpretation. Collective potentiality represents how many components, such as connection weights or neurons, are collectively used to deal with relations between inputs and outputs, while individual potentiality shows how an individual component potentially contributes to the inputs or outputs. From our viewpoint, the conventional interpretation methods have focused on the individual potentiality of components, ignoring these collective behaviors. Thus, the present paper stresses that neural networks should try to deal with the collective properties of components and to examine the final results from multiple points of view. For implementing this concept of potentiality maximization, we introduce the complementary potentiality minimization, which aims to reduce the strength of larger weights as much as possible, and at the same time, it can be used for increasing the potentiality. The method was applied to two intuitively interpretable data sets, namely, the absenteeism and online shoppers data sets. With both data sets, experimental results confirmed that the method could increase collective potentiality and considerably reduce the corresponding cost. The new method could extract many groups of weights with the same small strength, all of which tried to respond to the coming inputs, while the conventional methods, including the regularization ones, tried to produce a smaller number of stronger connection weights very selectively. In particular, the present method could produce stable compressed weights similar to the original correlation coefficients of the data sets, meaning that simple, independent, and linear relations could be detected, contrary to the presupposed non-linear ones. Thus, the collective interpretation, applied to two business data sets, revealed a possibility that, behind seemingly complicated non-linear relations between inputs and outputs, neural networks with collective potentiality augmentation with a smaller cost can extract the very simple relations for easy interpretation. All complicated relations can possibly be generated, based on those simple ones. [ABSTRACT FROM AUTHOR]

This paper is concerned with the problem of dynamic event-triggered bipartite consensus control for nonlinear multi-agent systems based on signed networks. A new dynamic event-triggered control mechanism is proposed, whereby an adjustment variable is introduced to dynamically schedule the triggering frequency, enabling the minimization of the triggering times while ensuring system performance. Based on the designed event-triggered control algorithm, sufficient conditions are derived to guarantee that bipartite consensus can be reached by the considered nonlinear multi-agent system. Furthermore, it is proved that Zeno behavior will not occur. Numerical examples are provided in this paper to verify the effectiveness of the proposed control algorithm. The simulation results reveal that, compared with the static event-triggered mechanism and the traditional dynamic event-triggered mechanism, the proposed event-triggered algorithm can further reduce the triggering times and enhance the flexibility of the event-triggered mechanism. [ABSTRACT FROM AUTHOR]

Knowledge Graph (KG) stores human knowledge facts in an intuitive graphical structure but faces challenges such as incomplete construction or inability to handle new knowledge. Knowledge Graph Reasoning (KGR) can make KGs more accurate, complete, and trustworthy to support various artificial intelligence applications better. Currently, the popular KGR methods are based on graph neural networks (GNNs). Recent studies have shown that hybrid logic rules and synergized pre-trained language models (PLMs) can enhance the GNN-based KGR methods. These methods mainly focus on data sparsity, insufficient knowledge evolution patterns, multi-modal fusion, and few-shot reasoning. Although many studies have been conducted, there are still few review papers that comprehensively summarize and explore KGR methods related to GNNs, logic rules, and PLMs. Therefore, this paper provides a comprehensive review of GNNs and PLMs for KGR based on a large number of high-quality papers. To present a clear overview of KGR, we propose a general framework. Specifically, we first introduce the KG preparation. Then we provide an overview of KGR methods, in which we categorize KGR methods into GNNs-based, logic rules-enhanced, and pre-trained language models-enhanced KGR methods. Furthermore, we also compare and analyze the GNN-based KGR methods in two scenarios. Moreover, we also present the application of KGR in different fields. Finally, we discuss the current challenges and future research directions for KGR. • Proposing the general framework for knowledge graph reasoning. • Systematic review of GNNs and PLMs for knowledge graph reasoning. • Performance evaluation and in-depth analysis of knowledge graph reasoning methods. • Practical application of knowledge graph reasoning methods in various fields. • Identifying current challenges and future directions in knowledge graph reasoning. [ABSTRACT FROM AUTHOR]

Knowledge tracing presents a fundamental research challenge within personalized education, aiming to dynamically monitor students' evolving mastery of individual skills through analysis of their online answer data. Noteworthy advancements have been achieved in knowledge tracing models, particularly with the integration of deep learning techniques. These models leverage long-short-term memory (LSTM) networks to process students' answer sequences, discern latent skill states, and predict subsequent responses. Nevertheless, real-world scenarios introduce a substantial challenge, given the inherent sparsity of student interaction data, implying that each student interacts with only a limited number of skills. Deep knowledge tracing models, when confronted with sparse data, tend to exhibit a bias towards learning from students with large sample sizes, consequently resulting in compromised learning outcomes due to insufficient interaction data. Addressing these challenges, this paper introduces an innovative self-paced learning data augmentation approach designed to augment the number of student answer sequence samples. Additionally, the paper employs self-supervised contrastive learning to strike a balance between local and global information, thereby mitigating the model's inclination to overly focus on local samples. To effectively address the aforementioned challenges, the paper proposes a deep knowledge tracing framework grounded in self-paced contrastive learning. The efficacy of the self-paced contrastive learning strategy is validated using real datasets, demonstrating superior prediction accuracy compared to alternative algorithms. [Display omitted] • Self-paced data augmentation automatically discovers the difficulty of questions. • Self-paced data augmentation solves the problem of data sparseness in education. • Self-paced contrastive learning strategy improves the prediction accuracy of knowledge tracing models. [ABSTRACT FROM AUTHOR]

Point cloud classification for substation is challenging due to the occluding layout as well as the complexity of methods. Most previous methods trade inference speed for precision by building complex extractors. To alleviate the paradox of performance and complexity trade-off, this paper proposes the lightweight long-range attention classification for substation point cloud, including shuffle channel attention(SCA) and dilated channel attention(DCA). First, SCA captures local cross-channel interaction via 1D convolution and global interaction via channel shuffling, shown promising performance. Second, to further reduce the amount of computation involved in shuffling, we propose a more elegant method DCA by resizing the 1D vector after pooling into 2D feature map. Note that proposed DCA are implemented by just 2D convolution, determining the coverage of cross-channel interaction, with a significantly higher inference speed. Furthermore, we develop a heuristic algorithm to adaptively determine parameters like kernel size, shuffle group and size of 2D feature map. Besides ModelNet40 and ScanObjectNN(PB_T50_RS), this paper also selects ten of main objects in substation for training and testing. Finally, experimental results suggest that proposed methods bring notable performance gain, nearly 1%. DCA reaches 93.801% and just increases 0.0003M parameters but 15.5% faster than SCA. We also make an attempt to simplify proposed methods via reducing the coding dimensions and coding blocks. And simplified DCA achieves 93.4% performance with almost quarter parameters and 112% faster, guaranteeing both efficiency and effectiveness. A simplified version is used to achieve the highest accuracy of 85.115% and 83.304% on ScanObjectNN, with the highest improvements of 1.735% and 2.391%. This paper also conducts robustness test where different proportions of points are missing. Results show that when 93.75% missing, the accuracy only decreases by 4.093%, and can still reach 89%, which is significantly ahead of other methods. • We propose a high-fidelity point cloud style transfer module(PST), which transforms raw data into a unified Gaussian model while retaining its unique style to suit different feature extractors based on style deviation vector R d , scaling coefficient α and β. • This paper proposes a long-range attention method, called shuffle channel attention(SCA), which captures the local and the global cross-channel interaction in an efficient way. • To further improve the inference speed, we revisit SCA and propose the latest version: dilated channel attention(DCA). This method achieve a 15% advantage in speed via changing the shape of feature map, guaranteeing both efficiency and effectiveness. [ABSTRACT FROM AUTHOR]

Path planning is one of the most crucial elements in the field of robotics, such as autonomous driving, minimally invasive surgery and logistics distribution. This review begins by summarizing the limitations of conventional path planning methods and recent work on DRL-based path planning methods. Subsequently, the paper systematically reviews the construction of key elements of DRL methods in recent work, with the aim of assisting readers in comprehending the foundation of DRL research, along with the underlying logic and considerations from a practical perspective. Facing issues of sparse rewards and the exploration–exploitation balance during the practical training process, the paper reviews enhancement methods for training efficiency and optimization results in DRL path planning. In the end, the paper summarizes the current research limitations and challenges in practical path planning applications, followed by future research directions. [ABSTRACT FROM AUTHOR]

General use quadcopters have been under development for over a decade but many of their potential applications are still under evaluation and have not yet been adopted in many of the areas that could benefit from their use. While the current generation of quadcopters use a mature set of control algorithms, the next steps, especially as autonomous features are developed, should involve a more complex learning capability to be able to adapt to unknown circumstances in a safe and reliable way. This paper provides baseline quadcopter control models learnt using eight general reinforcement learning (RL) algorithms in a simulated environment, with the object of establishing a reference performance, both in terms of precision and generation cost, for a simple set of trajectories. Each algorithm uses a tailored set of hyperparameters while, additionally, the influence of random seeds is also studied. While not all algorithms converge in the allocated computing budget, the more complex ones are able to provide stable and precise control models. This paper recommends the use of the TD3 algorithm as a reference for comparison with new RL algorithms. Additional guidance for future work is provided based on the weaknesses identified in the learning process, especially regarding the strong dependence of agent performance on random seeds. • Quadcopter control models learnt using model-free reinforcement learning algorithms. • Trained and tested in a simulated environment using OpenAI gym and pybullet. • Trained using small displacements, tested on trajectories unknown to the agent. • TD3 algorithm shows the most consistent performance. • All algorithms show strong dependence on the random seed selection. [ABSTRACT FROM AUTHOR]

Time-variant linear equations (TVLEs) are fundamental problems appeared in many practical applications. There is an increasing demand for algorithms that can solve TVLEs with high accuracy. This paper proposes an inverse-free continuous-time Getz–Marsden dynamic system (CTGMDS) for solving TVLEs. Additionally, a general eleven-instant finite-difference method (FDM) is proposed to discretize the CTGMDS, resulting in a general eleven-instant discrete-time GMDS (11-DTGMDS) model. Theoretically, the proposed 11-DTGMDS has truncation error of seven-order precision that delivers higher precision than the existing FDMs. This paper also compares five DTGMDS models using other FDMs to discretize the CTGMDS. The convergence of the CTGMDS and 11-DTGMDS is proven by theoretical analysis. Numerical validations are performed by comparing the 11-DTGMDS and other DTGMDSs, showing that the 11-DTGMDS provides higher solution accuracy. Finally, a path-tracking task is completed by applying the 11-DTGMDS model to a Franka Emika Panda robot, validating the practicality of the 11-DTGMDS. [ABSTRACT FROM AUTHOR]

Visible–Infrared Person Re-Identification (VI-ReID) aims to retrieve the identities of persons from different modalities. The significant distribution discrepancy makes this task challenging. Recent methods usually generate an intermediate modality to bridge the modality gap. However, these methods may lead to spectral confusion, and ignore the negative impact of visible channel redundancy information on establishing cross-modal correspondence. In this paper, we propose a Progressive Discrepancy Elimination Model (PDEM) for VI-ReID. Specifically, we design a Single-Channel Stripe Aggregation (SCSA) module to synthesize high-quality transitional modality images by using only raw visible and infrared spectral stripes. Meanwhile, we propose a two-stage Spectral Information Filtering (SIF) strategy, consisting of our designed Semantic Consistency Loss (SCL) and Cascade Aggregation Loss (CAL). SCL and CAL are employed in different training stages to achieve visible task-related information filtering and visible–infrared multi-scale alignment, respectively. In this way, the most relevant semantics with infrared modality in visible features are retained, and spectral correspondence is subsequently learned in a small cross-modal gap. We conduct extensive experiments on two VI-ReID datasets and achieve superior performance compared to most state-of-the-art methods. The source code of this paper is available at https://github.com/wxz0530/PDEM. [ABSTRACT FROM AUTHOR]

A systematic review on machine-learning strategies for improving generalization in electroencephalography-based emotion classification was realized. In particular, cross-subject and cross-session generalization was focused. In this context, the non-stationarity of electroencephalographic (EEG) signals is a critical issue and can lead to the Dataset Shift problem. Several architectures and methods have been proposed to address this issue, mainly based on transfer learning methods. In this review, 449 papers were retrieved from the Scopus , IEEE Xplore and PubMed databases through a search query focusing on modern machine learning techniques for generalization in EEG-based emotion assessment. Among these papers, 79 were found eligible based on their relevance to the problem. Studies lacking a specific cross-subject or cross-session validation strategy, or making use of other biosignals as support were excluded. On the basis of the selected papers' analysis, a taxonomy of the studies employing Machine Learning (ML) methods was proposed, together with a brief discussion of the different ML approaches involved. The studies reporting the best results in terms of average classification accuracy were identified, supporting that transfer learning methods seem to perform better than other approaches. A discussion is proposed on the impact of (i) the emotion theoretical models and (ii) psychological screening of the experimental sample on the classifier performances. [Display omitted] • The non-stationarity of EEG signals can lead to the Dataset Shift problem. • Transfer learning methods improve generalizability in EEG-based emotion classification. • Adaptive feature extraction also in combination with transfer learning are promising for generalization. [ABSTRACT FROM AUTHOR]

At present, the exploration of Knowledge Graph Embedding (KGE) mainly focuses on static KGE models. Due to the diversity and complexity of vertical domain data, it is difficult for existing KGE models to achieve excellent performance. Specific KGE models must be constructed for research on vertical domain knowledge graphs. This paper explores the relationships between the composition of Chinese characters and their sounds, meanings, and forms by constructing the Chinese Radical Knowledge Graph (CRKG). CRKG was constructed from a unique perspective according to the characteristics of Chinese radicals and knowledge graphs. The minor components of Chinese characters are separated and constructed into the fourth dimension of the knowledge graph. In this paper, we propose a new Chinese Radical Knowledge Graph Embedding (CRKGE) model, RotAL , which introduces dual quaternions and hierarchical transformers into KGE. RotAL performs a translation-rotation operation on the head entity with the relation as a parameter, then does a translation operation on the head entity with the minor component as a parameter to be close to the tail entity. Finally, we feed the trained quaternions into hierarchical transformers to jointly learn entity-related composition and relational contextualization based on a head entity's neighborhood. We demonstrate that our method can model and infer various relational patterns of CRKG, such as symmetric/antisymmetric, reflexive, inversion, and multiple. The experimental results on four CRKG datasets demonstrate that the RotAL model achieves the highest Hit@1 score, outperforming existing KGE models in accurately identifying the target entity on the first attempt. Furthermore, the RotAL model consistently achieves superior performance across other evaluation metrics, substantiating its significant advantage over existing models. [ABSTRACT FROM AUTHOR]

This paper studies the decentralized stochastic optimization problem over an undirected network, where each agent owns its local private functions made up of two non-smooth functions and an expectation-valued function. A decentralized stochastic primal–dual algorithm is proposed, by combining the variance-reduced method and the stochastic approximation method. The local gradients are estimated by using the mean of a variable number of sample gradients and the stochastic error decreases with the number of samples in the stochastic approximation process. The highlight of this paper is the extension of the primal–dual algorithm to the stochastic optimization problems. The effectiveness of the proposed algorithm and the correctness of the theory are verified by numerical experiments. [ABSTRACT FROM AUTHOR]

Specifically focusing on the landscape of abstractive text summarization, as opposed to extractive techniques, this survey presents a comprehensive overview, delving into state-of-the-art techniques, prevailing challenges, and prospective research directions. We categorize the techniques into traditional sequence-to-sequence models, pre-trained large language models, reinforcement learning, hierarchical methods, and multi-modal summarization. Unlike prior works that did not examine complexities, scalability and comparisons of techniques in detail, this review takes a comprehensive approach encompassing state-of-the-art methods, challenges, solutions, comparisons, limitations and charts out future improvements — providing researchers an extensive overview to advance abstractive summarization research. We provide vital comparison tables across techniques categorized — offering insights into model complexity, scalability and appropriate applications. The paper highlights challenges such as inadequate meaning representation, factual consistency, controllable text summarization, cross-lingual summarization, and evaluation metrics, among others. Solutions leveraging knowledge incorporation and other innovative strategies are proposed to address these challenges. The paper concludes by highlighting emerging research areas like factual inconsistency, domain-specific, cross-lingual, multilingual, and long-document summarization, as well as handling noisy data. Our objective is to provide researchers and practitioners with a structured overview of the domain, enabling them to better understand the current landscape and identify potential areas for further research and improvement. [Display omitted] • Overview of state-of-the-art techniques in abstractive text summarization. • Comparative analysis of models in abstractive summarization. • Identification of challenges and potential improvements in the field. • Exploration of future research directions and emerging frontiers. • Holistic survey of abstractive text summarization. [ABSTRACT FROM AUTHOR]

Multi-view subspace clustering (MVSC) has attracted increasing attention because it can extract information from multiple views and explore the underlying structure. In general, most of the existing anchor strategies solve the problem of excessive complexity, but there is a loss of information in the process of affinity graph passing to spectral clustering. In addition, the noise in the original data leads to the learned anchor graph not representing the data features adequately. To solve the above problems, this paper proposes a Large-Scale Multi-View Subspace Clustering via Embedding Space and Partition Matrix(LMVSC-EPM) algorithm that preserves the distribution of the original data by embedding matrix mapping. The algorithm utilizes the centroid matrix and the clustering assignment matrix to derive the clustering results directly. Specifically, LMVSC-EPM employs embedding matrices to map the raw data into the embedding space and adaptively learns anchors using a view-sharing anchor strategy. Moreover, the non-negative orthogonal matrix is adopted to assign the results to the clustering assignment matrix, which avoids the loss of the affinity matrix in the passing process. Furthermore, an alternating minimization optimization method is designed in this paper to solve the optimization problem. Experimental results on seven underlying datasets demonstrate the efficiency and superiority of the proposed method. [ABSTRACT FROM AUTHOR]

This paper studies the problem of event-triggered impulsive tracking control (ETITC) for uncertain strict-feedback nonlinear systems (USFNSs). In contrast to existing impulsive control schemes, this paper incorporates the neural-network (NN)-based backstepping technique into impulsive control design, such that stronger nonlinearities and uncertainties are allowed to be included in the concerned systems. The proposed state-feedback ETITC scheme guarantees that all the signals of the closed-loop system are bounded and the tracking error ultimately converges to an adjustable bounded region, while also avoiding the Zeno behavior. In addition, by constructing a NN-based observer, this paper further develops an output-feedback ETITC scheme to extend its investigation to the scenario where full states are not available for impulsive control design. Finally, an illustrative example involving a chemical reactor system is presented to demonstrate the effectiveness of our control schemes. [ABSTRACT FROM AUTHOR]

In this paper, the fixed-time stability of general impulsive systems and the fixed-time synchronization issue of impulsive complex networks are investigated. First, the fixed-time stability of a class of nonlinear systems with general impulsive effects is analyzed by means of inequality method and using some special functions. Then, the developed fixed-time stability results are used to study the fixed-time synchronization of impulsive complex networks under the saturation controller. Compared with some early published works, in this paper, the estimation accuracy of settling time is improved by calculating the value of improper integral more precisely, and the saturation controller effectively suppresses chattering phenomenon caused by discontinuous signum function. Lastly, some numerical examples are given to verify the feasibility of our theoretical results. [ABSTRACT FROM AUTHOR]

Spoken Language Understanding (SLU) is a crucial component of task-oriented dialog systems. In recent years, there has been increasing attention on multi-intent SLU due to its relevance to complex real-world applications. Most existing joint models only utilize multi-intent information to guide the slot filling, with only a small number of models achieving bidirectional interaction. Additionally, unlike traditional single-intent SLU, multi-intent SLU is scope-dependent, where each intent in a sentence has its specific dependency range. However, current bidirectional joint models often employ a pipeline approach to implement interaction between the two sub-tasks, which fails to fully leverage the semantic information between them and can lead to error propagation. Moreover, attention-based multi-intent joint models do not adequately model the positional relationships between words, resulting in suboptimal overall performance. In this paper, we propose a novel multi-intent SLU model called Position-aware Interactive Attention Network (PIAN), which consists of interactive attention and rotary position embedding. The aim of PIAN is to fully exploit the correlation between the two sub-tasks and capturing the positional dependencies between words. This facilitates mutual guidance and enhancement between the two sub-tasks. Additionally, to address the issue of uncoordinated slot problem generated by traditional slot filling decoders, we employ an MCRF slot filling decoder to constrain the slot labels. We evaluate our model on two public multi-intent SLU datasets, and the experimental results demonstrate that our model achieves state-of-the-art performance on key metrics. Code for this paper is publicly available at https://github.com/puhahahahahaha/SLU_with_Co_PRoE. • To achieve sufficient interaction and bidirectional promotion between the two sub-tasks, we propose an Interactive Attention Framework to achieve explicit bidirectional interaction between multi-intent and slot information. • To establish word dependencies within sentences, we use rotary position embedding, which captures more local features on the basis of global feature attention. For slot filling, MCRF replaces the traditional decoder, resolving decoding issues with uncoordinated slots. • Experimental results show that our model performs better than the current SOTA models on key metrics in two public datasets. [ABSTRACT FROM AUTHOR]

Graph neural networks are widely employed in diverse domains; however, they confront the peril of privacy infringement. To address this concern, federated learning emerges as a privacy-preserving approach that avoids sharing data for model training, effectively resolving the issue of privacy leakage in graph neural networks. The rapid advancement of federated neural networks has spurred the demand for more potent tools to enhance model performance owing to the concealed correlation information amongst federated learning participants. However, the structural attributes of federated graph neural networks render them vulnerable to inference attacks, reconstruction attacks, inversion attacks, and the like, potentially endangering privacy. This study delves into the intricacies of privacy-preserving within federated graph neural networks. Firstly, it introduces the architecture and variants of federated graph neural networks, analyzes the privacy risks encountered by these networks from four perspectives, and elucidates three primary attack methods. In accordance with the privacy-preserving mechanism of federated graph neural networks, it summarizes the privacy-preserving techniques and synthesizes the existing strategies from four perspectives: encryption methods, perturbation methods, anonymization, and hybrid methods. Furthermore, it summarily presents the prevailing framework for preserving privacy in neural networks. Ultimately, this paper examines the challenges and outlines future research directions pertaining to federated graph neural network technology. • This study investigates the privacy leakage risks of FedGNNs, summarizing and analyzing these risks and attack methods to provide a multifaceted investigation for privacy-preserving FedGNNs. • It collates cutting-edge research results and classifies protection mechanisms into encryption-based, perturbation-based, anonymization-based, and hybrid techniques, detailing their advantages and shortcomings. • The paper proposes future research directions for building robust, interpretable, efficient, fair, inductive, and comprehensive FedGNNs. [ABSTRACT FROM AUTHOR]

This paper is concerned with the fixed-time consensus control problem for non-strict feedback multi-agent systems with asymmetric output constraints and full state constraints. Considering the feasibility of controlling execution, a novel practical virtual control signal is developed utilizing both saturation function and hyperbolic tangent function to ensure that this signal can remain within the same restricted range as the corresponding state variable throughout entire operation process. In backstepping steps, the design of ideal virtual control signal also adopts a different form of piecewise function than before, introducing high-order polynomial functions to avoid singularity problems in the derivation process. In addition, function approximation ability of radial basis function neural networks technique is applied to estimate uncertainties derived from the system functions and controller design procedure. Moreover, universal barrier Lyapunov function approach is improved for constructing an adaptive constrained synchronization control scheme. By fixed-time stability theory, it is shown that the tracking errors of the MAS converge to an adjustable region around the origin in a fixed time and the state variables always obey their constraints. And the upper bound of the settling time is merely dependent on design parameters, which is not affected by the initial states of MAS. The effectiveness of the proposed control strategy is shown by a numerical simulation example at last. Two scenarios are provided to demonstrate the advantages of the control protocol proposed in this paper. [ABSTRACT FROM AUTHOR]

Exploring more effective multimodal fusion strategies is still challenging for RGB-T salient object detection (SOD). Most RGB-T SOD methods tend to focus on the strategy of acquiring modal complementary features by utilizing foreground information while ignoring the importance of background information for salient object localization. In addition, feature fusion without information filtering may introduce more noise. To solve these problems, this paper proposes a new cross-modal interaction guidance network (CIGNet) for RGB-T saliency object detection. Specifically, we construct a transformer-based dual-stream encoder to extract multimodal features. In the decoder, we propose an attention mechanism-based modal information complementary module (MICM) for capturing cross-modal complementary information for global comparison and salient object localization. Based on the MICM features, we design a multi-scale adaptive fusion module (MAFM) to find the optimal salient region of the multi-scale fusion process and reduce redundant features. In order to enhance the completeness of salient features after multi-scale feature fusion, this paper proposes the saliency region mining module (SRMM), which corrects the features in the boundary neighborhood by exploiting the differences between foreground and background pixels and the boundary. Comparisons with other state-of-the-art methods on three RGB-T datasets and five RGB-D datasets, the experimental results demonstrate the superiority and extensiveness of the proposed CIGNet. • The significance of thermal images in salient object detection was investigated. • Using attentional mechanisms to capture multimodal complementary features. • Importance of background pixels for correcting salient object boundary features. [ABSTRACT FROM AUTHOR]

This paper presents a novel safe integral reinforcement learning (IRL)-based optimal trajectory tracking scheme for nonlinear systems with uncertain dynamics that is subject to constraints. We leverage multilayer neural networks (MNNs) for actor-critic MNNs along with an NN identifier in the backstepping process for minimizing a discounted value function. A time-varying barrier Lyapunov function (TVBLF) is utilized for handling constraints and to provide safety assurances. Online weight update laws for the actor and critic MNNs are derived that are driven by Bellman error and control input error. We introduce an online lifelong learning (LL) method in the critic NN, utilizing the Bellman error in MNNs to address catastrophic forgetting. The method's effectiveness is demonstrated through simulations on mobile robot multitask tracking. The paper concludes with a stability analysis of the closed-loop system. [ABSTRACT FROM AUTHOR]

Most learning-to-rank recommendation models assume users prefer interacted items more than non-interacted ones. All non-interacted items have the same chance to be selected as negative samples. However, the observed interaction data usually shows long-tailed distributions, making traditional recommendation models trained on such data tend to rank popular items higher than users' true preference and degrading the performance of recommender systems. Some existing debias methods focus on eliminating such popularity bias, while simply removing all the popularity bias may diminish recommendation accuracy because popularity also contains some useful information implying item attractiveness, making it pivotal to handling popularity bias in proper ways. In this paper, popularity is recognized to have both positive and negative effects on recommendation. The positive effect of popularity should be amplified while the negative effect should be mitigated. Therefore, this paper proposes two novel schemes to distinguish the positive and negative impact of popularity in learning-to-rank models. First, we propose a popular-weighted sampler to sample non-interacted popular items from users' collaborative items as negative samples with higher probabilities. We believe that if a popular item is collaboratively similar to a user but not interacted with by him, it is more likely to be a true negative sample for the user. We sample such items with higher probabilities to mitigate the harmful negative popularity bias. Second, popularity could be a significant indicator of items' intrinsic quality. Hence, we design a popularity regularization scheme to capture item attractiveness, prompting cold-start users to prefer popular items. Finally, we conduct extensive experiments to evaluate the performance of the proposed schemes. Experimental results demonstrate that our approach can improve existing learning-to-rank models' performance and outperform state-of-the-art debias methods. • We design a novel popularity-weighted sampler to sample negative items. • We propose a popularity regularization term to capture items' attractiveness. • We evaluate the performance of the schemes with extensive experiments. [ABSTRACT FROM AUTHOR]

The predefined-time cooperative output regulation of second-order nonlinear multiagent systems with an unknown exosystem is investigated in this paper. Existing similar work is almost limited to exactly known exosystems, and the upper bound of convergence time for closed-loop systems is difficult to adjust and predict, which may not meet practical needs. In this paper, we first propose a new dynamic compensator called predefined-time distributed observer, which guarantees that each agent estimates the state and parameters about the linear exosystem with unknown system matrix through the general directed communication network within the time set by an explicit parameter. Subsequently, combining the improved dynamic gain control method and the proposed predefined-time distributed observer, a novel distributed dynamic predefined-time control scheme is proposed, which solves the problem of this paper by simplifying the tedious arithmetic process caused by the traditional backstepping method. The final simulation experiments verify the effectiveness of the proposed control scheme. • We extended similar articles to more general leader and controlled objects. • The settling time of our controller can be set directly and structure is simple. • We present a new distributed observer for unknown leader and general digraph. • The designed observer can work for an arbitrarily set period of time. • We transform an unknown leader into a known one in a predefined time. [ABSTRACT FROM AUTHOR]

This paper investigates fully distributed scaled non-negative edge-consensus problems of networked systems with actuator saturation and unmeasurable internal states. By using the adaptive control method, output-feedback-based low-gain technique, and graph theory, a new adaptive algorithm is designed to obtain scaled edge-consensus conditions, under which the difficulties caused by the constraints on edge states and controllers are overcome. There are three interesting characteristics that any global information of networks is not used in this paper's algorithm and result, including the number of vertexes and edges; the feasible solutions of the scaled edge-consensus conditions exist and can be easily obtained; the convergence rate of the controller is adjustable. Furthermore, the designed algorithm is expanded in the cases without actuator saturation. Finally, three examples are given to verify the theoretical results. [ABSTRACT FROM AUTHOR]

Modern machine learning algorithms aim to extract fine-grained information from data to provide accurate predictions, which often conflicts with the goal of privacy protection. This paper addresses the practical and theoretical importance of developing privacy-preserving machine learning algorithms that ensure good performance while preserving privacy. In this paper, we focus on the privacy and utility (measured by excess risk bounds) performances of differentially private stochastic gradient descent (SGD) algorithms in the setting of stochastic convex optimization. Specifically, we examine the pointwise problem in the low-noise setting for which we derive sharper excess risk bounds for the differentially private SGD algorithm. In the pairwise learning setting, we propose a simple differentially private SGD algorithm based on gradient perturbation. Furthermore, we develop novel utility bounds for the proposed algorithm, proving that it achieves optimal excess risk rates even for non-smooth losses. Notably, we establish fast learning rates for privacy-preserving pairwise learning under the low-noise condition, which is the first of its kind. [ABSTRACT FROM AUTHOR]

This paper introduces Elegans-AI models, a class of neural networks that leverage the connectome topology of the Caenorhabditis elegans to design deep and reservoir architectures. Utilizing deep learning models inspired by the connectome, this paper leverages the evolutionary selection process to consolidate the functional arrangement of biological neurons within their networks. The initial goal involves the conversion of natural connectomes into artificial representations. The second objective centers on embedding the complex circuitry topology of artificial connectomes into both deep learning and deep reservoir networks, highlighting their neural-dynamic short-term and long-term memory and learning capabilities. Lastly, our third objective aims to establish structural explainability by examining the heterophilic/homophilic properties within the connectome and their impact on learning capabilities. In our study, the Elegans-AI models demonstrate superior performance compared to similar models that utilize either randomly rewired artificial connectomes or simulated bio-plausible ones. Notably, these Elegans-AI models achieve a top-1 accuracy of 99.99% on both Cifar10 and Cifar100, and 99.84% on MNIST Unsup. They do this with significantly fewer learning parameters, particularly when reservoir configurations of the connectome are used. Our findings indicate a clear connection between bio-plausible network patterns, the small-world characteristic, and learning outcomes, emphasizing the significant role of evolutionary optimization in shaping the topology of artificial neural networks for improved learning performance. [ABSTRACT FROM AUTHOR]

This paper studies the consensus problem of a new multi-agent system via multi-task, multi-control mechanism and multi-consensus strategy. Firstly, two multi-task algorithms are proposed, which can reasonably select some agent nodes with better performance to form the multi-agent system according to different tasks. Secondly, a multi-control protocol designed in this paper integrates continuous control mechanism, impulsive control mechanism and hybrid control mechanism, which has strong applicability and efficiency. Thirdly, a multi-consensus strategy is designed, and it can be converted into "average consensus strategy", "leader-following consensus strategy" and "hybrid consensus strategy" through different parameter settings, which greatly broadens the application field of the multi-agent system. We construct a dynamic model for the proposed multi-agent system, analyze it by using matrix measure and Lyapunov stability theory, and obtain the theorems that enable it to achieve multiple consensus. Finally, the efficiency and practicability of our theories are verified through several simulation experiments. We further carry out some comparative analysis with some relevant papers, and obtain the superiority of our theories in multiple indicators. [ABSTRACT FROM AUTHOR]

In recent years, exchangeable network structures that take datasets as input have been widely used to obtain representations of various datasets. Although they perform well, analyzing exchangeable network with a dataset as input is challenging. Given that this type of network can be viewed as a function acting on probability measures since datasets are drawn from various distributions, this paper theoretically analyzes exchangeable network structures from a probabilistic perspective. This paper proposes a probabilistic analytical framework that neural networks acting on probability measures, which is an extension of Multi-Layer Perceptrons (MLP). When only samples from each distribution are available, in this new analytical framework, neural networks acting on probability measures correspond to the traditional exchangeable structure defined on datasets. Using this new analytical framework, we can demonstrate the ability of exchangeable network structures to capture complex patterns, as it provides the universal approximation property of exchangeable network structures. Furthermore, we derive a consistency result that shows the parameter estimation of exchangeable network structures is consistent statistically under certain conditions. [ABSTRACT FROM AUTHOR]

Steganography and steganalysis are two interrelated aspects of the field of information security. Steganography seeks to conceal communications, whereas steganalysis aims to discover or, if possible, recover the data they contain. These two areas have garnered significant interest, especially among law enforcement agencies. Cybercriminals and even terrorists often employ steganography to avoid detection while in possession of incriminating evidence, even when that evidence is encrypted, since cryptography is prohibited or restricted in many countries. Therefore, a deep understanding of cutting-edge techniques for uncovering concealed information is essential in exposing illegal activities. Over the last few years, a number of strong and reliable steganography and steganalysis techniques have been introduced in the literature. This review paper provides a comprehensive overview of deep learning-based steganalysis techniques used to detect hidden information within digital media. The paper covers all types of cover in steganalysis, including image, audio, and video, and discusses the most commonly used deep learning techniques. In addition, the paper explores the use of more advanced deep learning techniques, such as deep transfer learning (DTL) and deep reinforcement learning (DRL), to enhance the performance of steganalysis systems. The paper provides a systematic review of recent research in the field, including data sets and evaluation metrics used in recent studies. It also presents a detailed analysis of DTL-based steganalysis approaches and their performance on different data sets. The review concludes with a discussion on the current state of deep learning-based steganalysis, challenges, and future research directions. • Detect covert messages through neural nets, enhancing steganalysis accuracy. • Steganalysis can benefit from auto-encoders, DNN, CNN, RNN, LSTM, GNN, DBN, and DRN. • Reinforcement learning, transfer learning, and adversarial-based methods are effective to steganalysis. • Transform domain, attention layer, and noise-based features empower steganalysis schemes. • Unsupervised learning extracts nuanced patterns from multimedia for detection. [ABSTRACT FROM AUTHOR]

Low computational cost artificial intelligence (AI) models are vital in promoting the accessibility of real-time medical services in underdeveloped areas. The recent Once-For-All (OFA) network (without retraining) can directly produce a set of sub-network designs with Progressive Shrinking (PS) algorithm; however, the training resource and time inefficiency downfalls are apparent in this method. In this paper, we propose a new OFA training algorithm, namely the Progressive Expansion (ProX) to train the medical image analysis model. It is a reversed paradigm to PS, where technically we train the OFA network from the minimum configuration and gradually expand the training to support larger configurations. Empirical results showed that the proposed paradigm could reduce training time up to 68%; while still being able to produce sub-networks that have either similar or better accuracy compared to those trained with OFA-PS on ROCT (classification), BRATS and Hippocampus (3D-segmentation) public medical datasets. The code implementation for this paper is accessible at: https://github.com/shin-wl/ProX-OFA. [ABSTRACT FROM AUTHOR]

This paper introduces a novel feature selection method, called Feature Selection based on Importance Measures (FS-IM), to enhance the forecasting of crude oil returns. FS-IM innovatively combines active learning with the application of Gaussian noise to input features and selects the most relevant features using an optimal threshold value. The paper applies a ridge regression (RR) model based on FS-IM (FS-RR) to identify the factors that have important information for crude oil return forecasting. The paper compares FS-IM with other dimension reduction methods such as Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), and Independent Component Analysis (ICA). The results show that FS-IM can significantly improve model accuracy, demonstrating its effectiveness in finding key features. Moreover, FS-IM is more stable and consistent than other dimension reduction methods in enhancing the prediction accuracy in different scenarios, indicating its superior capability in capturing complex relationships between input and output variables. Furthermore, this study compares FS-RR model with other 13 prediction models by conducting experiments using a series of evaluation metrics, different statistical tests, and different step-ahead predictions and training sets. The results confirm that the RR model based on FS-IM can consistently outperform other model in terms of predictive performance and economic value, proving its effectiveness and robustness. This study contributes to the literature on crude oil price forecasting by addressing the challenges of high-dimensional and complex data, and by providing a robust, practical tool for professionals in energy economics and finance. • A new feature selection method for crude oil forecasting using importance scores and Gaussian noise is proposed. • The method selects the most relevant features based on the prediction errors of a base forecasting model. • The method enhances the accuracy and value of ridge regression model compared to other methods. • The method handles high-dimensional and complex data without losing information or interpretability. [ABSTRACT FROM AUTHOR]