Search

This work presents the development of a state-input affine differential neural network that approximates a class of nonlinear systems with uncertain dynamics and perturbations. The feasibility of using differential neural networks is based on the approximation capabilities of artificial neural networks, assuring that the approximation error of the right-hand side of the system with uncertain dynamics is bounded. The dynamics of the free parameters of the differential neural network is obtained using the Lyapunov's second stability method, which provides the convergence of the identification error to an invariant set around the origin. The approximation capabilities of the proposed identifier are compared with a classical differential neural network, aiming to identify the dynamics of a Stewart platform, which is used as testing example. The results show that the cumulative norm of identification's error is smaller using the state-input affine differential neural network, which demonstrates better identification outcomes with the proposed modified identifier. An additional advantage of the state-input affine differential neural network is that it has a quasi-linear structure that could help formulate a controller for several systems, which is a further research interest. • Learning laws for State-input affine differential neural network (SIADNN) identifier. • Design based on the second stability method of Lyapunov. • The novel DNN identifier has more applications than the traditional DNN identifier. • Evaluation of performing the non-parametric identification of a Stewart platform. [ABSTRACT FROM AUTHOR]

Spiking neural networks (SNNs) become popular choices for processing spatiotemporal input data and enabling low‐power event‐driven spike computation on neuromorphic processors. However, direct SNN training algorithms are not well compatible with error back‐propagation process, while indirect conversion algorithms based on artificial neural networks (ANNs) are usually accuracy–lossy due to various approximation errors. Both of them suffer from lower accuracies compared with their reference ANNs and need lots of time steps to achieve stable performance in deep architectures. In this article, a novel conversion framework is presented for deep SNNs with negative‐spike dynamics, which takes a quantization constraint and spike compensation technique into consideration during ANN‐to‐SNN conversion, and a truly lossless accuracy performance with their ANN counterparts is obtained. The converted SNNs can retain full advantages of simple leaky‐integrate‐and‐fire spiking neurons and are very suited for hardware implementation. In the experimental results, it is shown that converted spiking LeNet on MNIST/FashionMNIST and VGG‐Net on CIFAR‐10 dataset yield the state‐of‐the‐art classification accuracies with quite shortened computing time steps and much fewer synaptic operations. [ABSTRACT FROM AUTHOR]

Developing machine learning models that outperform the existing ones has been defined as the main goal of researchers and industry experts. In this context, the carbon footprint caused by conducting extended experiments on huge datasets for fine tuning the model parameters is usually ignored. Here, we present an efficient hybrid trust-aware incremental recommendation system that is based on factorization of user-item matrices. The method uses trust networks of the customers obtained from social networks. The main advantage of the method is that it is incremental and there is no need to re-train the system from scratch in presence of new customer activities and only newly added users and items are learned. To tackle the sparseness of the matrix, known as cold-start problem, we employed a deep neural network to decrease the user ratings approximation error. Moreover, we evaluated three different initialization methods for matrix factorization algorithm. The method is evaluated on two versions of popular MovieLens dataset from GroupLens research and also Extended Epinions dataset. The experimental results confirm that our method significantly requires less computational resources, while providing comparable prediction error. [ABSTRACT FROM AUTHOR]

We prove Carl's type inequalities for the error of approximation of compact sets K by deep and shallow neural networks. This in turn gives estimates from below on how well we can approximate the functions in K when requiring the approximants to come from outputs of such networks. Our results are obtained as a byproduct of the study of the recently introduced Lipschitz widths. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (DCNNs) have manifested significant performance gains for single-image super-resolution (SISR) in the past few years. Most of the existing methods are generally implemented in a fully supervised way using large-scale training samples and only learn the SR models restricted to specific data. Thus, the adaptation of these models to real low-resolution (LR) images captured under uncontrolled imaging conditions usually leads to poor SR results. This study proposes a zero-shot blind SR framework via leveraging the power of deep learning, but without the requirement of the prior training using predefined imaged samples. It is well known that there are two unknown data: the underlying target high-resolution (HR) images and the degradation operations in the imaging procedure hidden in the observed LR images. Taking these in mind, we specifically employed two deep networks for respectively modeling the priors of both the target HR image and its corresponding degradation kernel and designed a degradation block to realize the observation procedure of the LR image. Via formulating the loss function as the approximation error of the observed LR image, we established a completely blind end-to-end zero-shot learning framework for simultaneously predicting the target HR image and the degradation kernel without any external data. In particular, we adopted a multi-scale encoder–decoder subnet to serve as the image prior learning network, a simple fully connected subnet to serve as the kernel prior learning network, and a specific depthwise convolutional block to implement the degradation procedure. We conducted extensive experiments on several benchmark datasets and manifested the great superiority and high generalization of our method over both SOTA supervised and unsupervised SR methods. [ABSTRACT FROM AUTHOR]

This article presents an effective method to address the tracking control problem arising in uncertain switched high-order nonlinear system in strict-feedback form. The system under consideration contains unknown functions, which causally make the asymptotic tracking performance difficult to be achieved. By adopting the adding a power integrator approach in the framework of backstepping, a novel tracking controller is developed to guarantee an asymptotic tracking performance in the presence of the approximation error cased by neural networks (NNs) under arbitrary switching. The main contributions lie in: 1) the article for the first time embeds the backstepping technique in designing a kind of discontinuous controller with proportional integral (PI) compensation and 2) with the help of Filippov’s theory, a new defined system described by differential inclusions can be first obtained by taking some transformations, and then a novel nonsmooth Lyapunov function approach along with its upper right Dini derivative technique is applied to complete the construction of the discontinuous controller. Finally, two simulation examples are exhibited to verify the validity of the proposed design techniques. [ABSTRACT FROM AUTHOR]

Deep learning—in particular, deep neural networks (DNNs)—as a mesh-free and self-adapting method has demonstrated its great potential in the field of scientific computation. In this work, inspired by the Deep Ritz method proposed by Weinan E et al. to solve a class of variational problems that generally stem from partial differential equations, we present a coupled deep neural network (CDNN) to solve the fourth-order biharmonic equation by splitting it into two well-posed Poisson's problems, and then design a hybrid loss function for this method that can make efficiently the optimization of DNN easier and reduce the computer resources. In addition, a new activation function based on Fourier theory is introduced for our CDNN method. This activation function can reduce significantly the approximation error of the DNN. Finally, some numerical experiments are carried out to demonstrate the feasibility and efficiency of the CDNN method for the biharmonic equation in various cases. [ABSTRACT FROM AUTHOR]

Spiking neural networks (SNNs) have shown clear advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency, due to their event-driven nature and sparse communication. However, the training of deep SNNs is not straightforward. In this paper, we propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition, which is referred to as progressive tandem learning. By studying the equivalence between ANNs and SNNs in the discrete representation space, a primitive network conversion method is introduced that takes full advantage of spike count to approximate the activation value of ANN neurons. To compensate for the approximation errors arising from the primitive network conversion, we further introduce a layer-wise learning method with an adaptive training scheduler to fine-tune the network weights. The progressive tandem learning framework also allows hardware constraints, such as limited weight precision and fan-in connections, to be progressively imposed during training. The SNNs thus trained have demonstrated remarkable classification and regression capabilities on large-scale object recognition, image reconstruction, and speech separation tasks, while requiring at least an order of magnitude reduced inference time and synaptic operations than other state-of-the-art SNN implementations. It, therefore, opens up a myriad of opportunities for pervasive mobile and embedded devices with a limited power budget. [ABSTRACT FROM AUTHOR]

The sigmoid function is a widely used nonlinear activation function in neural networks. In this article, we present a modular approximation methodology for efficient fixed-point hardware implementation of the sigmoid function. Our design consists of three modules: piecewise linear (PWL) approximation as the initial solution, Taylor series approximation of the exponential function, and Newton–Raphson method-based approximation as the final solution. Its modularity enables the designer to flexibly choose the most appropriate approximation method for each module separately. Performance evaluation results indicate that our work strikes an appropriate balance among the objectives of approximation accuracy, hardware resource utilization, and performance. [ABSTRACT FROM AUTHOR]

Holographic optical tweezers have unique non-physical contact and can manipulate and control single or multiple cells in a non-invasive way. In this paper, the dynamics model of the cells captured by the optical trap is analyzed, and a control system based on a novel self-organizing fuzzy cerebellar model neural network (NSOFCMNN) is proposed and applied to the cell manipulation control of holographic optical tweezers. This control system consists of a main controller using the NSOFCMNN with a new self-organization mechanism, a robust compensation controller, and a higher order sliding mode. It can accurately move the captured cells to the expected position through the optical trap generated by the holographic optical tweezers system. Both the layers and blocks of the proposed NSOFCMNN can be adjusted online according to the new self-organization mechanism. The compensation controller is used to eliminate the approximation errors. The higher order sliding surface can enhance the performance of controllers. The distances between cells are considered in order to further realize multi-cell cooperative control. In addition, the stability and convergence of the proposed NSOFCMNN are proved by the Lyapunov function, and the learning law is updated online by the gradient descent method. The simulation results show that the control system based on the proposed NSOFCMNN can effectively complete the cell manipulation task of optical tweezers and has better control performance than other neural network controllers. [ABSTRACT FROM AUTHOR]

The stable generalized finite element method (SGFEM) is an improved version of generalized or extended FEM (GFEM/XFEM), which (i) uses simple and unfitted meshes, (ii) reaches optimal convergence orders, and (iii) is stable and robust in the sense that conditioning is of the same order as that of FEM and does not get bad as interfaces approach boundaries of elements. This paper designs the SGFEM for the discontinuous interface problem (DIP) by coupling a deep neural network (DNN). The main idea is to construct a function using the DNN, which captures the discontinuous interface condition, and transform the DIP to an (approximately) equivalent continuous interface problem (CIP) based on the DNN function such that the SGFEM for CIPs can be applied. The SGFEM for the DIP is a conforming method that maintains the features (i)–(iii) of SGFEM and is free from penalty terms. The approximation error of the proposed SGFEM is analyzed mathematically, which is split into an error of SGFEM of the CIP and a learning error of the DNN. The learning dimension of DNN is one dimension less than that of the domain and can be implemented efficiently. It is known that the DNN enjoys advantages in nonlinear approximations and high-dimensional problems. Therefore, the proposed SGFEM coupled with the DNN has great potential in the high-dimensional interface problem with interfaces of complex geometries. Numerical experiments verify the efficiency and optimal convergence of the proposed method. [ABSTRACT FROM AUTHOR]

This article considers the Nussbaum gain adaptive control issue for a type of nonlinear systems, in which some sophisticated and challenging problems, such as periodic disturbances, dead zone output, and unknown control direction are addressed. The Fourier series expansion and radial basis function neural network are incorporated into a function approximator to model time-varying-disturbed function with a known period in nonlinear systems. To deal with the problems of the dead zone output and unknown control direction, the Nussbaum-type function is recommended in the design of the control algorithm. Applying the Lyapunov stability theory and backstepping technique, the proposed control strategy ensures that the tracking error is pulled back to a small neighborhood of origin and all closed-loop signals are bounded. Finally, simulation results are presented to show the availability and validity of the analysis approach. [ABSTRACT FROM AUTHOR]

This paper studies how well generative adversarial networks (GANs) learn probability distributions from finite samples. Our main results establish the convergence rates of GANs under a collection of integral probability metrics defined through Hölder classes, including the Wasserstein distance as a special case. We also show that GANs are able to adaptively learn data distributions with low-dimensional structures or have Hölder densities, when the network architectures are chosen properly. In particular, for distributions concentrated around a low-dimensional set, we show that the learning rates of GANs do not depend on the high ambient dimension, but on the lower intrinsic dimension. Our analysis is based on a new oracle inequality decomposing the estimation error into the generator and discriminator approximation error and the statistical error, which may be of independent interest. [ABSTRACT FROM AUTHOR]

This article proposes neural network (NN) based observer schemes and a sensorless robust optimal control scheme for partially unknown permanent magnet synchronous motors with disturbances and saturating voltages. First, an NN-observer scheme is designed to estimate back-electromotive force (EMF), for which the mathematical model in rotary or stationary reference frames is relaxed. The NN weight tuning law is designed via Lyapunov theory to guarantee that EMF is ultimately uniformly bounded. Second, to compensate the inexact extraction of the estimated back-EMF at any speed conditions, disturbances, and NN approximation errors, another NN-observer scheme is designed to estimate the tracking errors of rotor position and speed, for which low-pass filters and/or phase-locked loops are not needed. Third, a sensorless saturated robust optimal control scheme dealing with general disturbances and saturating voltages is designed. The scheme includes the augmented feedforward controller to transform the speed and current tracking problem into an optimal control problem. Finally, the feedback control law and worst disturbance law are obtained without estimating unknown internal dynamics. The effectiveness of the proposed schemes is tested through simulations and comparative experiments on a load drive application with a DSP board TMS320F28379D. [ABSTRACT FROM AUTHOR]

Deep learning and neural networks are being extended to geometric design and computation. Recent studies show that deep neural networks are vulnerable to adversarial samples. However, for 3D mesh adversarial samples, the most related studies actually attack the 2D victim networks, in which they have to project 3D objects to 2D images. In this paper, we present D3AdvM (Direct 3D adversary for mesh) to directly generate adversarial samples inside mesh data. Specifically, we propose two adversary generation approaches: vertex-based and edge-based. The first one, 3DVP (3D Vertex-based Perturbation), skillfully searches the optimized vertices positions based on opposite gradient fitness. The second one, KES (Key Edge-based Selection), carefully collapses the key feature edges according to importance of edge feature. Thus, our approaches avoid 3D/2D projections and approximation errors. Also, our approach can reduce the computation overhead. In addition, the proposed D3AdvM can control the number of perturbed vertices for real-world engineering designs and applications. Extensive experiments show that the generated 3D meshes are effective to attack classification networks. Furthermore, we evaluate the transferability, in which D3AdvM can attack both mesh-based networks and point cloud-based networks as victim networks. Our findings could benefit 3D geometry design based on the new generation of artificial intelligence and big data. • This work is the first one to directly attack mesh-based neural networks with adversarial meshes in 3D space without the 2D projection. • We explore the characteristics of 3D mesh topological entities and propose both vertex-based and edge-based adversarial approaches. • D3AdvM can constrain the adversarial samples within a specific number to support high feasibility in real-world applications. • D3AdvM shows valuable transferability to attack different versions of mesh-based networks and even the point cloud-based networks. • Our findings could benefit 3D geometry design based on the new generation of artificial intelligence and big data. [ABSTRACT FROM AUTHOR]

Deep reinforcement learning has emerged as a powerful control method, especially for the complex nonlinear system such as aeroengine control system, due to its strong representation ability and capability of learning from data measurements. This paper presents a novel control strategy based on deep reinforcement learning to speed up the acceleration process of aero-engine. The actor-critic framework is adopted, where the actor neural network aims to find the optimal control policy and the critic network aims to evaluate the current control policy. The deep deterministic policy gradient algorithm is used to update the parameters of the neural networks. In addition, a complementary integrator is introduced to eliminate the steady-state error caused by the approximation error of the deep neural networks, and a momentum term is introduced to set limits for the input of the control system, thereby suppressing the overrun during the early learning and exploration processes. The numerical simulations show that the controller with this new control strategy can cope with different flight conditions, and significantly speed up the acceleration process of aero-engine. [ABSTRACT FROM AUTHOR]

• A discrete-time control scheme applied to steering-by-wire system is proposed. • Design an identification model to adjust the neural network output weight. • Disturbance observers are proposed to handle the disturbance. • Simulation and experiment are offered to validate the proposed method. This paper investigates the design and implementation of the discrete-time adaptive neural network control with disturbance observer (DO) on a steer-by-wire (SbW) system, to simultaneously realize accurate tracking and anti-interference performance. Specifically, to approximate the lumped system uncertainty including the friction torque and self-aligning torque, the neural network is employed. To improve the steering tracking performance, the discrete-time identification model is proposed so that the tracking error and modeling error can be utilized to adjust the neural network updating law. Then, the unknown compound disturbances caused by external disturbance, Euler approximation errors and neural network approximation error are restrained by two DOs. Finally, the Lyapunov stability theory shows that the system tracking error is uniformly ultimately bounded. Both numerical simulations and experiments are implemented to show the superiority of the proposed controller. [ABSTRACT FROM AUTHOR]

Combustion of energy fuels or organic waste is associated with the emission of harmful gases and aerosols into the atmosphere, which strongly affects air quality. Air quality monitoring devices are unreliable and measurement gaps appear quite often. Missing data modeling techniques can be used to complete the monitoring data. Concentrations of monitored pollutants can be approximated with regression modeling tools, such as artificial neural networks. In this study, a long-term set of data from the air monitoring station in Zabrze (Silesia, South Poland) was analyzed. Concentration prediction was tested for the main air pollutants, i.e., O3, NO, NO2, SO2, PM10, CO. Multilayer perceptrons were used to model the concentrations. The predicted concentrations were compared to the observed ones to evaluate the approximation accuracy. Prediction errors were calculated separately for the whole concentration range as well as for the specified concentration subranges. Some different measures of error were estimated. It was stated that the use of a single measure of the approximation accuracy may lead to incorrect interpretation. The application of one neural network to the entire concentration range results in different prediction accuracy in various concentration subranges. Replacing one neural network with several networks adjusted to specific concentration subranges should improve the modeling accuracy. [ABSTRACT FROM AUTHOR]