Search

Artificial neural networks (ANNs) have been widely adopted as general computational tools both in computer science as well as many other engineering fields. Stochastic gradient descent (SGD) and adaptive methods such as Adam are popular as robust optimization algorithms used to train the ANNs. However, the effectiveness of these algorithms is limited because they calculate a search direction based on a first-order gradient. Although higher-order gradient methods such as Newton's method have been proposed, they require the Hessian matrix to be semi-definite, and its inversion incurs a high computational cost. Therefore, in this paper, we propose a variable three-term conjugate gradient (VTTCG) method that approximates the Hessian matrix to enhance search direction and uses a variable step size to achieve improved convergence stability. To evaluate the performance of the VTTCG method, we train different ANNs on benchmark image classification and generation datasets. We also conduct a similar experiment in which a grasp generation and selection convolutional neural network (GGS-CNN) is trained to perform intelligent robotic grasping. After considering a simulated environment, we also test the GGS-CNN with a physical grasping robot. The experimental results show that the performance of the VTTCG method is superior to that of four conventional methods, including SGD, Adam, AMSGrad, and AdaBelief. [ABSTRACT FROM AUTHOR]

Turing (1937) introduces a model of code that is followed by other pioneers of computing machines (such as Flowers 1983, Eckert, Mauchly, Brainerd 1945 and others). One of them is John von Neumann, who defines the concept of optimal code in the context of the conception of EDVAC. He later uses it to build on in his theoretical considerations of the universal constructor (von Neumann 1966). Von Neumann (1963) further presents one of the first neural network models, in relation to the work of McCulloch and Pitts (1943), for both theoretical purposes (von Neumann probe) and practical applications (computer architecture of EDVAC). The aim of this paper is (1) to describe the differences between Turingʼs and von Neumannʼs conceptualizations of code and the mechanical computing model. Between von Neumann's abstract technical conception (von Neumann 1963 and 1966) and Turingʼs more concrete biochemical conception (Turing 1952). Furthermore, (2) we want to answer the question why these influential models of mechanisms (predominantly in computer science) have so far been ignored by philosophers of the new mechanism (Machamer, Darden, Craver 2000, Glennan 2017). We will show that these classical models of machines are not only compatible with the new mechanism, but moreover complement it, since they represent a completely separate type of model of mechanism, alongside producing, maintaining and underlying (Zámečník 2021). The final (3) and main goal of our paper will be an attempt to relate von Neumannʼs and Turingʼs notion of mechanism to Barbieriʼs notion of extended mechanism (Barbieri 2015). [ABSTRACT FROM AUTHOR]

The problem of subspace clustering which refers to segmenting a collection of data samples approximately drawn from a union of linear subspaces is considered in this paper. Among existing subspace clustering algorithms, low rank representation (LRR) based subspace clustering is a very powerful method and has demonstrated that its performance is good. Latent low rank representation (LLRR) subspace clustering algorithm is an improvement of the original LRR algorithm when the observed data samples are insufficient. The clustering accuracy of LLRR is higher than that of LRR. Recently, Frobenius norm minimization based LRR algorithm has been proposed and its clustering accuracy is higher than that of LRR demonstrating the effectiveness of Frobenius norm as another convex surrogate of the rank function. Combining LLRR and Frobenius norm, a new low rank representation subspace clustering algorithm is proposed in this paper. The nuclear norm in the LLRR algorithm is replaced by the Frobenius norm. The resulting optimization problem is solved via alternating direction method of multipliers (ADMM). Experimental results show that compared with LRR, LLRR and several other state-of-the-art subspace clustering algorithms, the proposed algorithm can get higher clustering accuracy. Compared with LLRR, the running time of the proposed algorithm is reduced significantly. [ABSTRACT FROM AUTHOR]

Artificial neural networks, inspired by the biological networks of the human brain, have become game-changing computing models in modern computer science. Inspired by their wide scope of applications, synthetic biology strives to create their biological counterparts, which we denote synthetic biological neural networks (SYNBIONNs). Their use in the fields of medicine, biosensors, biotechnology, and many more shows great potential and presents exciting possibilities. So far, many different synthetic biological networks have been successfully constructed, however, SYNBIONN implementations have been sparse. The latter are mostly based on neural networks pretrained in silico and being heavily dependent on extensive human input. In this paper, we review current implementations and models of SYNBIONNs. We briefly present the biological platforms that show potential for designing and constructing perceptrons and/or multilayer SYNBIONNs. We explore their future possibilities along with the challenges that must be overcome to successfully implement a scalable in vivo biological neural network capable of online learning. [ABSTRACT FROM AUTHOR]

This paper mainly focuses on how to construct concept lattice effectively and efficiently based on improved variable precision rough set. On the basis of preprocessing formal concept, one algorithm that can determine the value range of variable precision parameter β according to the approximate classification quality is proposed. An improved β -upper and lower distribution attribute reduction algorithm is also proposed based on the improved variable precision rough set, the algorithm can be used for attribute reduction on the original data of the concept lattice, and to eliminate the redundant knowledge or noises of the formal context. For the reduced formal context, the paper combines the concept construction algorithm with an improved rule acquisition algorithm seamlessly, and proposes a novel approach of concept lattice construction based on improved variable precision rough set. Finally, a concept lattice generation prototype system is developed, this paper also performs comprehensive experiments, and the effectiveness of the improved algorithm is proved through the experimental results. [ABSTRACT FROM AUTHOR]

In the last decade, many computer science disciplines have been manipulating large amounts of data like artificial neural networks, data recognition, data mining and big data. Recently, many programmers and application designers have relied on the matrices in their algorithms to handle large numbers of data simultaneously to enriching the parallelism for their systems. However, due to the growth of the data amount in the matrices, there is a need to reshaping these matrices according to the capabilities of the computer systems that are processing these data. Consequently, our idea is developing algebraic operations that can represent the reshaping of matrices within the mathematical equations. In this paper, we develop fully two novel compatible algebraic matrices operations to solve this problem. The first operation called "Matrix-Separation" and the second operation denoted by "Matrices-Joining". These two novel operations assist the programmers and scientists to perform their programs evaluations and developments. [ABSTRACT FROM AUTHOR]

Recommendation systems are information filtering tools that present items to users based on their preferences and behavior, for example, suggestions about scientific papers or music a user might like. Based on what we said and with the development of computer science that has started to take an interest in big data and how it is used to discover user interest, we have found a lot of research going on in the area of recommendation and there are powerful systems available. In the unsupervised learning domain, this paper introduces a novel method for creating a hybrid recommender framework that combines Collaborative Filtering with Content Based Approach and Self-Organizing Map neural network technique. By testing our system on a subset of the Movies Database, we demonstrate that our method outperforms state-of-the-art methods in terms of accuracy and precision, as well as improving the efficiency of the traditional Collaborative Filtering methodology. [ABSTRACT FROM AUTHOR]

Depth information has been discovered to improve the performance of computer vision tasks, such as semantic segmentation and object recognition. However, careful acquisition of depth data needs highly developed depth sensors which are expensive. As a classic computer vision task, depth estimation from a single image has obtained promising results based on supervised learning methods. In this paper, we investigate the extension of color images with corresponding deep-regressed depth images in boosting the performance of semantic segmentation. Furthermore, the usage of combining color channels with the estimated depth or the ground truth depth channel is compared. Specifically, there are two stages in our work. Firstly, we adopt the framework of convolutional neural networks (CNN) for the depth estimation by combing the global depth network and the depth gradient network. After refining based on these two networks, the depth image map can be estimated in a deep-regressed manner. Secondly, after augmenting the color images with the predicted depth images, fully convolutional networks (FCN) are further used to implement the pixel-level semantic labeling. In the experiments, we employ two popular RGBD datasets, i.e., SUNRGBD and NYUDv2, for 37 and 40-class semantic segmentation, respectively. By comparing with the ground truth depth images, experimental results demonstrate that the networks trained on the estimated depth images can achieve comparable performance on facilitating the accuracy of semantic segmentation task. [ABSTRACT FROM AUTHOR]

This paper proposes an efficient and effective active online sequential learning approach, named as Less Annotated Active Learning Extreme Learning Machine (LAAL-ELM). It leverages the predictions’ confidence of the new arriving data to actively select both query-annotated samples and confidence-weighted predict-annotated ones to update the classifier, which contributes to less actively query annotation, and applies WOS-ELM, a discriminant model, to significantly reduce the computation complexity for doing online updating in one step. The proposed approach firstly gives a principle to evaluate confidence of the prediction in WOS-ELM; then determines what and how to update the model with new arriving data in the online phase: the uncertain instances are annotated by query their classes, almost-certain ones are weighted on its prediction’s confidence and the certain ones are discarded directly for reducing over-fitting; at last, the weighted and query-annotated samples are used to update the classifier. The proposed approach is evaluated on five real-world benchmark classification issues. And the experimental results demonstrate that the proposed LAAL-ELM can effectively reduce the number of queried samples while maintaining high level of classification performance. [ABSTRACT FROM AUTHOR]

This paper presents a novel unsupervised multi-source domain adaptation approach, named as coupled local–global adaptation (CLGA). At the global level, in order to maximize the adaptation ability, CLGA regards multiple domains as a unity, and jointly mitigates the gaps of both marginal and conditional distributions between source and target dataset. At the local level, with the intention of maximizing the discriminative ability, CLGA investigates the relationship among distinctive domains, and exploits both class and domain manifold structures embedded in data samples. We formulate both local and global adaptation in a concise optimization problem, and further derive an analytic solution for the objective function. Extensive evaluations verify that CLGA performs better than several existing methods not only in multi-source adaptation tasks but also in single source scenarios. [ABSTRACT FROM AUTHOR]

Compressed sensing (CS) has been one of the great successes of applied mathematics in the last decade. This paper proposes a new method, combining the advantage of the Compressive Sampling Matching Pursuit (CoSaMP) algorithm and the Quasi–Newton Iteration Projection (QNIP) algorithm, for the recovery of sparse signal from underdetermined linear systems. To get the new algorithm, Quasi–Newton Projection Pursuit (QNPP), the least-squares technique in CoSaMP is used to accelerate convergence speed and QNIP is modified slightly. The convergence rate of QNPP is studied, under a certain condition on the restricted isometry constant of the measurement matrix, which is smaller than that of QNIP. The fast version of QNPP is also proposed which uses the Richardson’s iteration to reduce computation time. The numerical results show that the proposed algorithms have higher recovery rate and faster convergence speed than existing techniques. [ABSTRACT FROM AUTHOR]

Deterministic finite automata are widely used in control systems: from abstract protocols such as TCP to mechanical devices such as elevators or traffic lights. Some of these systems are quite complex and can be defined only in terms of formal language theory. In this paper, we propose new approach for synthesizing finite automata from a dictionary of some language that uses neural networks. The results show that the proposed approach works correctly and quickly for automata with up to six states and four characters in the alphabet. For larger automata, the neural network suffers from the vanishing gradient problem, which is a big topic for further research. [ABSTRACT FROM AUTHOR]

In this paper, an Arabic statistical learning-based text classification system has been developed using Polynomial Neural Networks. Polynomial Networks have been recently applied to English text classification, but they were never used for Arabic text classification. In this research, we investigate the performance of Polynomial Networks in classifying Arabic texts. Experiments are conducted on a widely used Arabic dataset in text classification: Al-Jazeera News dataset. We chose this dataset to enable direct comparisons of the performance of Polynomial Networks classifier versus other well-known classifiers on this dataset in the literature of Arabic text classification. Results of experiments show that Polynomial Networks classifier is a competitive algorithm to the state-of-the-art ones in the field of Arabic text classification. [ABSTRACT FROM AUTHOR]

Identified cluster of atmospheric discharges, sufficiently near from transmissions line, could be an important alarm to support real time decisions. Lightning are important events that affect the electrical power system operation, which are often responsible for transmission lines outages, and can trigger a sequence of events that lead to system collapse. The Brazilian lightning network detection monitors nearly 18 million events monthly and all this data must be processed and analyzed. This paper uses a hybrid model named the Quantum binary-real evolving Spiking Neural Network (QbrSNN) for clustering problem, where the features and parameters of a spiking neural network (SNN) are optimized using the Quantum-Inspired Evolutionary Algorithm with representation Binary-Real (QIEA-BR). The proposed model is applied to atmospheric discharges data, with a significantly higher clustering accuracy than traditional techniques. [ABSTRACT FROM AUTHOR]

Deep learning techniques have been successfully used in recent years to learn useful image trans- formations and features, thus contributing significantly to the advancements in neural networks. However deep nets suffer from the drawback that they require large training times and mul- tifarious parameters that need to be hand tuned for optimal performance. In this paper we investigate the use Recurrent neural network architectures to learn useful transformations of an image(object), progressively over time. Learning these latent transformations enables the recur- rent architecture to correctly predict, to a high degree of accuracy, the original representation of an object from its transformed representations. [ABSTRACT FROM AUTHOR]

This paper presents two novel methods for skin detection: HP-ELM and BD-SOM. Both SOM and ELM are fast for large data sets, but not yet suitable for Big Data. We show how they can be improved in order to fulfill the strict requirements for Big Data. Both new methods are described and their implementations are explained. A comparison on a large example is presented in the experiment section. We find that BD-SOM is more accurate but not as computationally efficient as HP-ELM. As a result, we show that both methods work well on a Big Data task. The given task deals with the classification of more than one billion samples (pixels) between Skin and Non Skin categories. [ABSTRACT FROM AUTHOR]

A concussion is an invisible and poorly understood mild traumatic brain injury (mTBI) that can alter the way the brain functions. Patients who have screened positive for mTBI are at an increased risk of depression, post-traumatic stress disorder (PTSD), headaches, sleep disorders, and other neurological and psychological problems. Early detection of psychological conditions such as PTSD following a concussion might improve the overall outcome of a patient and could potentially reduce the cost associated with intense interventions often required when conditions go untreated for a long time. Statistical and predictive models that leverage large-scale clinical repositories and use pre-existing conditions to determine the probability of a patient developing psychological conditions following a concussion have not been widely studied. This paper presents an SVM-based model that has been trained with a longitudinal dataset of over 5.3 million clinical encounters of 89,840 service members that have sustained a concussion. The model has been tested and validated with over 16,045 patients that developed PTSD and it has shown an accuracy of over 85% (AUC of 86.52%) at predicting the condition within the first year following the injury. [ABSTRACT FROM AUTHOR]

This paper presents a computational framework, the Generic Programmable Neural Network (GPNN), for efficient implementation of Back-Propagation based neural learning algorithms running on multi-core machines. GPNN has three components: parallelization of neural learning, abstraction of network components, and compile-time generalization. Together these computational components make GPNN an efficient framework for fast implementation of back-propagation based neural learning algorithms, and provide flexibility and reusability for modifying neural network topologies. The GPNN was applied to four different neural learning algorithms: classic back-propagation (BP), quick propagation (QP), resilient propagation (RP) and Levenberg-Marquardt (LM) algorithm. Experiments were conducted to evaluate the effectiveness of GPNN, and results show that the neural learning algorithms implemented in GPNN are more efficient than their respective functions provided by Matlab. [ABSTRACT FROM AUTHOR]

Domain generation algorithm (DGA) is used by botnets to build a stealthy command and control (C&C) communication channel between the C&C server and the bots. A DGA can periodically produce a large number of pseudo-random algorithmically generated domains (AGDs), a few of which direct the bots to the C&C server. AGD detection algorithms provide a lightweight, promising solution in response to the existing DGA techniques. In the constantly evolving attacker–defender game, attackers may seek more advanced DGA techniques to gain a better chance of evading detection by defenders. In this paper, we propose a new DGA, namely a neural networks-based domain name generation (NDG) architecture. NDG is based on a variational autoencoder (VAE), where the encoder and decoder networks use stacked gated convolutional neural networks (GCNNs) to learn the contextual structure hierarchically. NDG is experimentally validated using a set of state-of-the-art AGD detection algorithms. The existing DGAs of different classes following a DGA taxonomy are used to benchmark NDG. NDG shows the best overall anti-detection performance among all tested DGAs. We also demonstrate that NDG is effective in benchmarking AGD detection algorithms. [ABSTRACT FROM AUTHOR]

• This paper used a neural network based method to analyze factors influencing dataset popularity in UCI data repositories. • Sensitivity degree as a weight was used to re-rank the datasets in order to predict high-popularity datasets. • We examined whether the relationship between factors and popularity differs depending on the subject domain in the UCI. • GitHub data repository was used for evaluating the applicability of the proposed framework to other types of factor analysis. With their rapid development, data repositories usually provide abundant metadata—including data types, keywords, downloads, stars, forks, and citations—along with the data content. These rich metadata can be used as valuable resources to study the factors that facilitate data sharing. However, few previous studies have attempted to study which metadata are correlated with the popularity of data. This study overcomes these issues by extracting the major factors for each dataset from a well-known data repository, the UCI Machine Learning Repository, and a popular open-source software repository, GitHub. We trained a neural network model and measured the influence of these features on quantified popularity metrics using the weight product of connecting neurons. We grouped the UCI factors into two categories (intrinsic and extrinsic) and the GitHub factors into three categories (intrinsic, extrinsic, and web-related) to analyze their influence on popularity at each level. The quantified influence was used to predict the popularity of the data or software. We conducted a statistical analysis to explore the relationship between these factors and popularity with five different domains (life sciences, physical sciences, computer science/engineering, social sciences, and others) for the UCI repository. This study's findings contribute to understanding the factors that affect the popularity of open datasets or software for providing guidance on data sharing, reuse, and organization. [ABSTRACT FROM AUTHOR]