Search

In the technical battlefield of text classification, extracting key features and solving the sparsity problem play a decisive role in improving the performance of classification results. Euclidean geometric models often distort the processed vectors because they are difficult to deal with complex data structures. This exploration uses hyperbolic space with huge storage potential and hierarchical structure, and proposes an innovative hyperbolic graph-based short text classification technology - L-HGAT, aiming to improve the efficiency of processing concise information. This method combines two technologies, hyperbolic geometry and attention network, to optimize the representation of text through in-depth interaction between labels and text features. The research results significantly show that L-HGAT not only has high accuracy and excellent efficiency in many benchmark data sets, but also effectively integrates label information, significantly enhancing the model's ability to capture local features. This discussion brings an innovative perspective to processing hierarchical information and demonstrates the effectiveness of hyperbolic geometry in text classification challenges.

Graph neural network (GNN) is one of the most widely used methods that leverage relational information in the data to learn and make predictions. Fraud detection is a challenging task considering the nature of fraudulent transactions which changes drastically from one case to another as fraudsters often collude to hide their abnormal behavior/features. To this end, GNN has a fitting application because it leverages graph structure to learn relational information to distinguish malicious transactions from legitimate ones. This study implements various GNNs such as graph convolution network (GCN), graph attention network (GAT), and modified GAT to predict fraudulent Bitcoin transactions. It focuses on benchmarking the results of two versions of GAT against GCN to demonstrate the superior predictive power of the attention mechanism. The two versions include conventional GAT and modified GAT, the latter consists of a dynamic attention mechanism. The two versions of the GAT model are also compared in detail. GNN has been used to detect fraud or anomalies for various practical implementations such as financial transactions, credit cards, and customer reviews. However, a detailed study focusing on the two versions of GAT and benchmarking it against GCN has not yet been conducted. We show that GAT has an enhanced ability to predict fraudulent transactions. The excellent predictive performance of GAT gives a clear indication that it could play a vital role in detecting broader cryptocurrency fraud. Finally, this study discusses the challenges of building an explainable GNN models.

Although graph neural networks (GNNs) have achieved great success in the field of semi-supervised bearing fault diagnosis recently, most GNN-based studies still face the challenge of insufficient labeled samples. Meanwhile, the signals collected from bearings usually have a time offset due to the alternating speed in the actual working conditions, reducing the applicability of traditional GNN diagnosis methods relying on Euclidean distance. And incomplete use of unlabeled data and insufficient exploitation of neighborhood information further reduce the reliability of fault diagnosis models. To solve these problems, a semi-supervised few-shot fault diagnosis method driven by multi-head dynamic graph attention network (MHDGAT) is proposed in this study. First, dynamic time warping (DTW) similarity is introduced in the construction of K-nearest neighbor graph to solve the problem of local time offset caused by speed change. Second, some pseudo labels generated by smoothness assumption are injected into the raw training set, and more comprehensive information can be learned in the process of model training. Then, the presented MHDGAT model can mine rich fault characteristics from different scales by the multi-head attention and extract the most sensitive features due to the proposed dynamic graph mechanism adaptively. Finally, the fault categories can be obtained by a softmax classifier located at the end of the model. The proposed model has been validated in two case studies, and the experimental results indicate that MHDGAT exerts strong performance even under extremely low-label ratios. © 2024 Elsevier Inc.

Traditional two-dimensional dynamic fault detection methods describe nonlinear dynamics by constructing a two-dimensional sliding window in the batch and time directions. However, determining the shape of a two-dimensional sliding window for different phases can be challenging. Samples in the two-dimensional sliding windows are assigned equal importance before being utilized for feature engineering and statistical control. This will inevitably lead to redundancy in the input, complicating fault detection. This paper proposes a novel method named attention-based two-dimensional dynamic-scale graph autoencoder (2D-ADSGAE). Firstly, a new approach is introduced to construct a graph based on a predefined sliding window, taking into account the differences in importance and redundancy. Secondly, to address the training difficulties and adapt to the inherent heterogeneity typically present in the dynamics of a batch across both its time and batch directions, we devise a method to determine the shape of the sliding window using the Pearson correlation coefficient and a high-density gridding policy. The method is advantageous in determining the shape of the sliding windows at different phases, extracting nonlinear dynamics from batch process data, and reducing redundant information in the sliding windows. Two case studies demonstrate the superiority of 2D-ADSGAE. © 2024 by the authors.

Recently, research on multisensor fault diagnosis under noisy signals has gained significant attention. Due to various degrees of external interference and differences in sensor precision, the signal quality across different channels is inconsistent. These discrepancies are often neglected by fault diagnosis models and are also difficult to capture accurately. To solve this problem, this article introduces a multiscale channel attention-driven graph dynamic fusion network for mechanical fault diagnosis. It can mine the differences in importance among channels at multiple scales and calculate the channel attention weights to enhance the node feature representation. Additionally, a graph dynamic fusion framework for multisource features is proposed to process the subgraphs in parallel, which achieves a deep-level feature fusion and enables dynamic adjustments to the fusion process based on real-time model output. With the proposed graph dynamic reconstruction module, the reliability of the feature fusion process is further improved. In the experimental part, three noise distribution scenarios were simulated to validate the robustness of the proposed method on an axial flow pump and a gearbox. The comparative analysis with various state-of-the-art models and traditional deep learning models confirms the effectiveness and superiority of the proposed method. IEEE

There is an ever-increasing usage of mobile data with global traffic having reached 115 exabytes per month at the end of 2022 for mobile data traffic including fixed wireless access. This is projected to grow up to 453 exabytes at the end of 2028, according to Ericssons 2022 mobile data traffic outlook report. To meet the increasing demand radio access networks (RAN) used for mobile communication are continuously being improved with the current generation enabling larger virtualization of the network through the Cloud RAN (C-RAN) architecture. This facilitates the usage of commercial off-the-shelf servers (COTS) in the network replacing specialized hardware servers and making it easier to scale up or down the network capacity after traffic demand. This thesis looks at how we can efficiently identify servers needed to meet traffic demand in a network consisting of both COTS servers and specialized hardware servers while trying to reduce the energy consumption of the network. We model the problem as a network where the antennas and radio heads are connectedto the core network through a C-RAN and a specialized hardware layer. The network is then represented using a graph where the nodes represent servers in the network. Using this problem model as a base we then generate problem instances with varying topologies, server profiles, and traffic demands. To find out how the traffic should be passed through the network we test two different methods: A mixed integer linear programming (MILP) method focused on energy minimization and a graph attention network (GAT) predictor combined with the energy minimization MILP. To help evaluate the results we also create three other methods: a MILP model that tries to spread the traffic as evenly as possible, a random predictor combined with the energy minimization MILP and a greedy method. Our results show that the energy optimization MILP method can be used to create optimal solutions, but it suffer from a slow computation time compa

This paper proposes a combined approach of deep deterministic policy gradient (DDPG) and graph attention network (GAT) to the geometry optimization of latticed shells with surface shapes defined by a Bézier control net. The optimization problem is formulated to minimize the strain energy of the latticed structures with heights of the Bézier control points as design variables. The information of the latticed shells, including nodal configurations, element properties and internal forces, and the Bézier control net, consisting of control points and control net, are represented as graphs using node feature matrices, adjacency matrices, and weighted adjacency matrices. A specifically designed DDPG agent utilizes GAT and matrix manipulations to observe the state of the structure through the graphs, and decides which and how Bézier control points to move. The agent is trained to excel in the task through a reward signal computed from changes in the strain energy in each optimization step. As shown in numerical examples, the trained agent can effectively optimize structures of different sizes, control nets, configurations, and initial geometries from those used during the training. The performance of the trained agent is competitive compared to particle swarm optimization and simulated annealing despite using a lower computational cost.

Inom beräkningsbiologin är det snabbt på väg att bli allt vanligare att ta fram genetiska regleringsnätverk (GRN). På grund av storleken på de undersökta nätverken använder många forskare maskininlärning för att härleda GRN från genuttrycksdata, vanligtvis från RNA-seq. Noggrannheten hos sådana moderna metoder kan dock fortfarande förbättras, särskilt när det gäller tidsseriemodellen. I den här forskningen föreslås två tidsseriemodeller för GRN-inferens: GATv2-regressionsmodell och GATv2-länkprediktionsmodell. Den förstnämnda modellen är baserad på idén om en konventionell GRN-inferensmodell som regresserar målgenuttrycksdata med hjälp av data om kandidatkontrollgener. Den senare modellen bygger på den vanliga länkprediktionsmetoden med hjälp av grafiska neurala nätverk som utför binariseringen av om det finns kanter mellan specifika genknutpunkter. GATv2-regressionsmodellen presterar bra i en regressionsuppgift. Noggrannheten för GRN-inferensen är dock låg och var nästan densamma som en slumpmässig utgång. Å andra sidan presterar GATv2-modellen för prediktion av länkar bra om träningsdata är tillräckligt korrekta. Att härleda genreglerande relationer med hög uppskattningsnoggrannhet är dock fortfarande en utmaning inom området GRN-inferens även för partiella genreglerande relationer. Av den anledningen har modellens noggrannhet minskat betydligt på grund av behovet av större noggrannhet i träningsdata., In computational biology, inferring Gene regulatory networks (GRN) is rapidly expanding. Due to the size of the examined networks, many researchers use machine learning to infer GRN from gene expression data, typically from RNA-seq. However, the accuracy of such state-of-the-art methods still has room for improvement, especially for the time-series model. This research proposes two timeseries GRN inference models: GATv2 regression model and GATv2 link prediction model. The former model is based on the idea of conventional GRN inference model that regresses the target gene expression data by using candidate control gene data. The latter model is based on the usual link prediction method by Graph neural networks that performs the binarization task of whether edges exist between specific gene nodes. The GATv2 regression model performs well in a regression task. However, the accuracy of GRN inference is low, and it was almost the same as a random output. On the other hand, the GATv2 link prediction model performs well if the training data is correct enough. However, inferring gene regulatory relationships with high estimation accuracy is still a challenge in the field of GRN inference even for partial gene regulatory relationships. For that reason, the accuracy of the model has significantly decreased because of the need for more accuracy in the training data.

There is an ever-increasing usage of mobile data with global traffic having reached 115 exabytes per month at the end of 2022 for mobile data traffic including fixed wireless access. This is projected to grow up to 453 exabytes at the end of 2028, according to Ericssons 2022 mobile data traffic outlook report. To meet the increasing demand radio access networks (RAN) used for mobile communication are continuously being improved with the current generation enabling larger virtualization of the network through the Cloud RAN (C-RAN) architecture. This facilitates the usage of commercial off-the-shelf servers (COTS) in the network replacing specialized hardware servers and making it easier to scale up or down the network capacity after traffic demand. This thesis looks at how we can efficiently identify servers needed to meet traffic demand in a network consisting of both COTS servers and specialized hardware servers while trying to reduce the energy consumption of the network. We model the problem as a network where the antennas and radio heads are connectedto the core network through a C-RAN and a specialized hardware layer. The network is then represented using a graph where the nodes represent servers in the network. Using this problem model as a base we then generate problem instances with varying topologies, server profiles, and traffic demands. To find out how the traffic should be passed through the network we test two different methods: A mixed integer linear programming (MILP) method focused on energy minimization and a graph attention network (GAT) predictor combined with the energy minimization MILP. To help evaluate the results we also create three other methods: a MILP model that tries to spread the traffic as evenly as possible, a random predictor combined with the energy minimization MILP and a greedy method. Our results show that the energy optimization MILP method can be used to create optimal solutions, but it suffer from a slow computation time compa

Inom beräkningsbiologin är det snabbt på väg att bli allt vanligare att ta fram genetiska regleringsnätverk (GRN). På grund av storleken på de undersökta nätverken använder många forskare maskininlärning för att härleda GRN från genuttrycksdata, vanligtvis från RNA-seq. Noggrannheten hos sådana moderna metoder kan dock fortfarande förbättras, särskilt när det gäller tidsseriemodellen. I den här forskningen föreslås två tidsseriemodeller för GRN-inferens: GATv2-regressionsmodell och GATv2-länkprediktionsmodell. Den förstnämnda modellen är baserad på idén om en konventionell GRN-inferensmodell som regresserar målgenuttrycksdata med hjälp av data om kandidatkontrollgener. Den senare modellen bygger på den vanliga länkprediktionsmetoden med hjälp av grafiska neurala nätverk som utför binariseringen av om det finns kanter mellan specifika genknutpunkter. GATv2-regressionsmodellen presterar bra i en regressionsuppgift. Noggrannheten för GRN-inferensen är dock låg och var nästan densamma som en slumpmässig utgång. Å andra sidan presterar GATv2-modellen för prediktion av länkar bra om träningsdata är tillräckligt korrekta. Att härleda genreglerande relationer med hög uppskattningsnoggrannhet är dock fortfarande en utmaning inom området GRN-inferens även för partiella genreglerande relationer. Av den anledningen har modellens noggrannhet minskat betydligt på grund av behovet av större noggrannhet i träningsdata., In computational biology, inferring Gene regulatory networks (GRN) is rapidly expanding. Due to the size of the examined networks, many researchers use machine learning to infer GRN from gene expression data, typically from RNA-seq. However, the accuracy of such state-of-the-art methods still has room for improvement, especially for the time-series model. This research proposes two timeseries GRN inference models: GATv2 regression model and GATv2 link prediction model. The former model is based on the idea of conventional GRN inference model that regresses the target gene expression data by using candidate control gene data. The latter model is based on the usual link prediction method by Graph neural networks that performs the binarization task of whether edges exist between specific gene nodes. The GATv2 regression model performs well in a regression task. However, the accuracy of GRN inference is low, and it was almost the same as a random output. On the other hand, the GATv2 link prediction model performs well if the training data is correct enough. However, inferring gene regulatory relationships with high estimation accuracy is still a challenge in the field of GRN inference even for partial gene regulatory relationships. For that reason, the accuracy of the model has significantly decreased because of the need for more accuracy in the training data.

Despite the significant advances in RGB-D scene recognition, there are several major limitations that need further investigation. For example, simply extracting modal-specific features neglects the complex relationships among multiple modalities of features. Moreover, cross-modal features have not been considered in most existing methods. To address these concerns, we propose to integrate the tasks of cross-modal matching and modal-specific recognition, termed as Matching-to-Recognition Network (MRNet). Specifically, the cross-modal matching network enhances the descriptive power of the recognition network via a layer-wise semantic loss. The recognition network obtains multi-modal features from a two-stream CNN: global features are obtained by a higher-layer of a CNN to preserve the semantic content, and local layout features are learned by the graph attention network, thus better capturing the key object regions and modelling their relationships. Extensive experiments results demonstrate the MRNet achieves superior performance to state-of-the-art methods, especially for recognition solely based on single modality. © 2023 IEEE.

Learning and training processes are starting to be affected by the diffusion of Artificial Intelligence (AI) techniques and methods. AI can be variously exploited for supporting education, though especially deep learning (DL) models are normally suffering from some degree of opacity and lack of interpretability. Explainable AI (XAI) is aimed at creating a set of new AI techniques able to improve their output or decisions with more transparency and interpretability. In the educational field it could be particularly significant and challenging to understand the reasons behind models outcomes, especially when it comes to suggestions to create, manage or evaluate courses or didactic resources. Deep attentional mechanisms proved to be particularly effective for identifying relevant communities and relationships in any given input network that can be exploited with the aim of improving useful information to interpret the suggested decision process. In this paper we provide the first stages of our ongoing research project, aimed at significantly empowering the recommender system of the educational platform “WhoTeach” by means of explainability, to help teachers or experts to create and manage high-quality courses for personalized learning. The presented model is actually our first tentative to start to include explainability in the system. As shown, the model has strong potentialities to provide relevant recommendations. Moreover, it allows the possibility to implement effective techniques to completely reach explainability.

Learning and training processes are starting to be affected by the diffusion of Artificial Intelligence (AI) techniques and methods. AI can be variously exploited for supporting education, though especially deep learning (DL) models are normally suffering from some degree of opacity and lack of interpretability. Explainable AI (XAI) is aimed at creating a set of new AI techniques able to improve their output or decisions with more transparency and interpretability. Deep attentional mechanisms proved to be particularly effective for identifying relevant communities and relationships in any given input network that can be exploited with the aim of improving useful information to interpret the suggested decision process. In this paper we provide the first stages of our ongoing research project, aimed at significantly empowering the recommender system of the educational platform”WhoTeach” by means of explainability, to help teachers or experts to create and manage high-quality courses for personalized learning. The presented model is actually our first tentative to start to include explainability in the system. As shown, the model has strong potentialities to provide relevant recommendations. Moreover, it allows the possibility to implement effective techniques to completely reach explainability3