Search

In this paper, we explore a process called neural teleportation, a mathematical consequence of applying quiver representation theory to neural networks. Neural teleportation teleports a network to a new position in the weight space and preserves its function. This phenomenon comes directly from the definitions of representation theory applied to neural networks and it turns out to be a very simple operation that has remarkable properties. We shed light on the surprising and counter-intuitive consequences neural teleportation has on the loss landscape. In particular, we show that teleportation can be used to explore loss level curves, that it changes the local loss landscape, sharpens global minima and boosts back-propagated gradients at any moment during the learning process. [ABSTRACT FROM AUTHOR]

In the field of natural language processing, machine translation is a colossally developing research area that helps humans communicate more effectively by bridging the linguistic gap. In machine translation, normalization and morphological analyses are the first and perhaps the most important modules for information retrieval (IR). To build a morphological analyzer, or to complete the normalization process, it is important to extract the correct root out of different words. Stemming and lemmatization are techniques commonly used to find the correct root words in a language. However, a few studies on IR systems for the Urdu language have shown that lemmatization is more effective than stemming due to infixes found in Urdu words. This paper presents a lemmatization algorithm based on recurrent neural network models for the Urdu language. However, lemmatization techniques for resource-scarce languages such as Urdu are not very common. The proposed model is trained and tested on two datasets, namely, the Urdu Monolingual Corpus (UMC) and the Universal Dependencies Corpus of Urdu (UDU). The datasets are lemmatized with the help of recurrent neural network models. The Word2Vec model and edit trees are used to generate semantic and syntactic embedding. Bidirectional long short-term memory (BiLSTM), bidirectional gated recurrent unit (BiGRU), bidirectional gated recurrent neural network (BiGRNN), and attention-free encoder–decoder (AFED) models are trained under defined hyperparameters. Experimental results show that the attention-free encoder-decoder model achieves an accuracy, precision, recall, and F-score of 0.96, 0.95, 0.95, and 0.95, respectively, and outperforms existing models. [ABSTRACT FROM AUTHOR]

Complex network science is an interdisciplinary field of study based on graph theory, statistical mechanics, and data science. With the powerful tools now available in complex network theory for the study of network topology, it is obvious that complex network topology models can be applied to enhance artificial neural network models. In this paper, we provide an overview of the most important works published within the past 10 years on the topic of complex network theory-based optimization methods. This review of the most up-to-date optimized neural network systems reveals that the fusion of complex and neural networks improves both accuracy and robustness. By setting out our review findings here, we seek to promote a better understanding of basic concepts and offer a deeper insight into the various research efforts that have led to the use of complex network theory in the optimized neural networks of today. [ABSTRACT FROM AUTHOR]

The seasonal autoregressive integrated moving average with exogenous factors (SARIMAX) has shown promising results in modeling small and sparse observed time-series data by capturing linear features using independent and dependent variables. Long short-term memory (LSTM) is a promising neural network for learning nonlinear dependence features from data. With the increase in wildlife roadkill patterns, the SARIMAX-only and LSTM-only models would likely fail to learn the precise endogenous and/or exogenous variables driven by this wildlife roadkill data. In this paper, we design and implement an error correction mathematical framework based on LSTM-only. The framework extracts features from the residual error generated by a SARIMAX-only model. The learned residual features correct the output time-series prediction of the SARIMAX-only model. The process combines SARIMAX-only predictions and LSTM-only residual predictions to obtain a hybrid SARIMAX-LSTM. The models are evaluated using South African wildlife–vehicle collision datasets, and the experiments show that compared to single models, SARIMAX-LSTM increases the accuracy of a taxon whose linear components outweigh the nonlinear ones. In addition, the hybrid model fails to outperform LSTM-only when a taxon contains more nonlinear components rather than linear components. Our assumption of the results is that the collected exogenous and endogenous data are insufficient, which limits the hybrid model's performance since it cannot accurately detect seasonality on residuals from SARIMAX-only and minimize the SARIMAX-LSTM error. We conclude that the error correction framework should be preferred over single models in wildlife time-series modeling and predictions when a dataset contains more linear components. Adding more related data may improve the prediction performance of SARIMAX-LSTM. [ABSTRACT FROM AUTHOR]

Adversarial examples easily mislead vision systems based on deep neural networks (DNNs) trained with softmax cross entropy (SCE) loss. The vulnerability of DNN comes from the fact that SCE drives DNNs to fit on the training examples, whereas the resultant feature distributions between the training and adversarial examples are unfortunately misaligned. Several state-of-the-art methods start from improving the inter-class separability of training examples by modifying loss functions, where we argue that the adversarial examples are ignored, thus resulting in a limited robustness to adversarial attacks. In this paper, we exploited the inference region, which inspired us to apply margin-like inference information to SCE, resulting in a novel inference-softmax cross entropy (I-SCE) loss, which is intuitively appealing and interpretable. The inference information guarantees that it is difficult for neural networks to cross the decision boundary under an adversarial attack, and guarantees both the inter-class separability and the improved generalization to adversarial examples, which was further demonstrated and proved under the min-max framework. Extensive experiments show that the DNN models trained with the proposed I-SCE loss achieve a superior performance and robustness over the state-of-the-arts under different prevalent adversarial attacks; for example, the accuracy of I-SCE is 63% higher than SCE under the P G D 50 u n attack on the MNIST dataset. These experiments also show that the inference region can effectively solve the misaligned distribution. [ABSTRACT FROM AUTHOR]

The actual behavior of welded T-junctions in tubular structures depends strongly on the topology of the junction at the joint level. In finite element analysis, beam-type elements are usually employed due to their simplicity and low computational cost, even though they cannot reproduce the joints topologies and characteristics. To adjust their behavior to a more realistic situation, elastic elements can be introduced at the joint level, whose characteristics must be determined through costly validations. This paper studies the optimization and implementation of the validation data, through the creation of an optimal surrogate model based on neural networks, leading to a model that predicts the stiffness of elastic elements, introduced at the joint level based on available data. The paper focuses on how the neural network should be chosen, when training data is very limited and, more importantly, which of the available data should be used for training and which for verification. The methodology used is based on a Monte Carlo analysis that allows an exhaustive study of both the network parameters and the distribution and choice of the limited data in the training set to optimize its performance. The results obtained indicate that the use of neural networks without a careful methodology in this type of problems could lead to inaccurate results. It is also shown that a conscientious choice of training data, among the data available in the problem of choice of elastic parameters for T-junctions in finite elements, is fundamental to achieve functional surrogate models. [ABSTRACT FROM AUTHOR]

With the evolution of modern digital pathology, examining cancer cell tissues has paved the way to quantify subtle symptoms, for example, by means of image staining procedures using Eosin and Hematoxylin. Cancer tissues in the case of breast and lung cancer are quite challenging to examine by manual expert analysis of patients suffering from cancer. Merely relying on the observable characteristics by histopathologists for cell profiling may under-constrain the scale and diagnostic quality due to tedious repetition with constant concentration. Thus, automatic analysis of cancer cells has been proposed with algorithmic and soft-computing techniques to leverage speed and reliability. The paper's novelty lies in the utility of Zernike image moments to extract complex features from cancer cell images and using simple neural networks for classification, followed by explainability on the test results using the Local Interpretable Model-Agnostic Explanations (LIME) technique and Explainable Artificial Intelligence (XAI). The general workflow of the proposed high throughput strategy involves acquiring the BreakHis public dataset, which consists of microscopic images, followed by the application of image processing and machine learning techniques. The recommended technique has been mathematically substantiated and compared with the state-of-the-art to justify the empirical basis in the pursuit of our algorithmic discovery. The proposed system is able to classify malignant and benign cancer cell images of 40× resolution with 100% recognition rate. XAI interprets and reasons the test results obtained from the machine learning model, making it reliable and transparent for analysis and parameter tuning. [ABSTRACT FROM AUTHOR]

In this paper, we present an impulsive pinning control algorithm for discrete-time complex networks with different node dynamics, using a linear algebra approach and a neural network as an identifier, to synthesize a learning control law. The model of the complex network used in the analysis has unknown node self-dynamics, linear connections between nodes, where the impulsive dynamics add feedback control input only to the pinned nodes. The proposed controller consists of the linearization for the node dynamics and a reorder of the resulting quadratic Lyapunov function using the Rayleigh quotient. The learning part of the control is done with a discrete-time recurrent high order neural network used for identification of the pinned nodes, which is trained using an extended Kalman filter algorithm. A numerical simulation is included in order to illustrate the behavior of the system under the developed controller. For this simulation, a 20-node complex network with 5 different node dynamics is used. The node dynamics consists of discretized versions of well-known continuous chaotic attractors. [ABSTRACT FROM AUTHOR]

In this paper, the global asymptotical stability of Riemann-Liouville fractional-order neural networks with time-varying delays is studied. By combining the Lyapunov functional function and LMI approach, some sufficient criteria that guarantee the global asymptotical stability of such fractional-order neural networks with both discrete time-varying delay and distributed time-varying delay are derived. The stability criteria is suitable for application and easy to be verified by software. Lastly, some numerical examples are presented to check the validity of the obtained results. [ABSTRACT FROM AUTHOR]