Search

It has been widely recognized that the efficient training of neural networks (NNs) is crucial to classification performance. While a series of gradient-based approaches have been extensively developed, they are criticized for the ease of trapping into local optima and sensitivity to hyperparameters. Due to the high robustness and wide applicability, evolutionary algorithms (EAs) have been regarded as a promising alternative for training NNs in recent years. However, EAs suffer from the curse of dimensionality and are inefficient in training deep NNs (DNNs). By inheriting the advantages of both the gradient-based approaches and EAs, this article proposes a gradient-guided evolutionary approach to train DNNs. The proposed approach suggests a novel genetic operator to optimize the weights in the search space, where the search direction is determined by the gradient of weights. Moreover, the network sparsity is considered in the proposed approach, which highly reduces the network complexity and alleviates overfitting. Experimental results on single-layer NNs, deep-layer NNs, recurrent NNs, and convolutional NNs (CNNs) demonstrate the effectiveness of the proposed approach. In short, this work not only introduces a novel approach for training DNNs but also enhances the performance of EAs in solving large-scale optimization problems.

During the last decade, deep neural networks have shown a great performance in many machine learning tasks such as classification and clustering. One of the most successful networks is the CNN (Convolutional Neural Network), which has been applied in many application domains such as pattern recognition, medical diagnosis, and signal processing. Despite the very interesting performance of CNNs, their architecture design is still so far a major challenge for researchers and practitioners. Several works have been proposed in the literature with the aim to find optimized architectures such as ResNet and VGGNet. Unfortunately, most of these architectures are either manually defined by experts or automatically designed by greedy induction algorithms. Recent works suggest the use of Evolutionary Algorithms (EAs) thanks to their ability to escape locally-optimal architectures. Despite the fact that EAs have shown interesting performance, researchers in this direction have considered the design task as a single-level optimization problem; which represents the main research gap we tackle in this paper. The main contribution behind our work consists in the fact that CNN architecture design has a hierarchical nature and thus could be seen as a Bi-Level Optimization Problem (BLOP) where: (1) the upper level minimizes the network complexity defined by the number of blocks and the number of nodes per block; and (2) the lower level optimizes the convolution block ‘graphs’ topologies by maximizing the classification accuracy. Motivated by the originality of our observation with respect to the state of the art, we frame for the first time the CNN architecture design problem as a BLOP and then solve it using an adapted version of an existing efficient bi-level EA; through the definition of the solution encoding, the fitness function, and the variation operators at each level. The adapted EA is named BLOP-CNN and is assessed on the image classification task using the commonly employed CIFAR-10 and CIFAR-100 benchmark data sets. The analysis of our experimental results show the merits of our proposed method in providing the user with optimized architectures that outperform many recent and prominent architectures coming from the three different approaches, namely: manual design, reinforcement learning-based generation, and evolutionary optimization. Moreover, to show the applicability of our approach, we have conducted a case study on the detection of the COVID-19 using a set of benchmark chest X-ray and Computed Tomography (CT) images.

It is quite difficult to quantitatively measure heat extraction in a fractured enhanced geothermal system (EGS) reservoir and its impacting factors. This study thus aims to define a characteristic parameter that is failure area percentage for evaluating and optimizing geothermal extraction. Through establishing a thermal-fluid-solid coupling model, the operation lifecycle and economy of an EGS case was evaluated by the proposed parameter. A sensitivity indicator was developed to measure the quantitatively effection of various factors on heat extraction. The results show that heat extraction increases with the three growing-up factors: fracture network complexity, pressure difference, and rock permeability. However, too large value of these factors are adverse to geothermal system smooth operation. Based on the sensitivity indicator analysis, heat extraction is the most sensitive to the variation of fracture network complexity. This study would provide a guidance for optimization of EGS designing.

The Elman neural network (ElmanNN) is well-known for its capability of processing dynamic information, which has led to successful applications in stock forecasting. In this paper, we introduce direct input-to-output connections (DIOCs) into the ElmanNN and show that the proposed Elman neural network with DIOCs (Elman-DIOCs) significantly out-performs the original ElmanNN without such DIOCs. Four different global stock indices, i.e., the Shanghai Stock Exchange (SSE) Composite Index, the Korea Stock Price Index (KOSPI), the Nikkei 225 Index (Nikkei225), and the Standard & Poor's 500 Index (SPX), are used to demonstrate the affecacy of the Elman-DIOCs in time-series prediction. We systematically evaluate 8 models, depending whether or not there are hidden layer biases, whether or not there are output layer biases, and whether or not there are DIOCs. The experimental results show that DIOCs lead to much better prediction accuracy, while requiring fewer than a half of the hidden neurons. Take the SPX index, for example - the root mean squared error (RMSE) and the mean absolute error (MAE) of the Elman-DIOCs are improved by 44.2% and 41.1%, respectively, compared to the ElmanNN, and 65.6% and 60.8%, respectively, compared to the multi-layer perceptron (MLP). We argue that (1) DIOCs can always help to improve accuracy, while reducing network complexity and computational burden, as long as the problem at hand (either regression or classification) has linear components, and (2) most real-world applications contain linear components. Therefore DIOCs will be almost always beneficial in any types of neural networks for classification or regression. We also point out that in rare cases where the problem at hand is entirely nonlinear, DIOCs should not be used. This study is funded by Joint Research Fund for Overseas Chinese Scholars and Scholars in Hong Kong and Macao (Grant No. 61828601), Natural Science Foundation of Shanxi Province (Grant No. 201801D121141), and Provincial Program on Key Research Projects of Shanxi (Social Development Area) (Grant No. 201903D321003).

Narrowband Internet of Things (NB-IoT) has been utilized in a vast range of applications. The conventional NB-IoT is subjected to the limitation of reduced coverage, limited power, and delay. The conventional NB-IoT uses orthogonal frequency-division multiplexing with Long Term Evolution (LTE) is applied, which increases the network complexity. The increase in network complexity increases power consumption and delay. To overcome those limitations, modified Reduced Fast Fourier Transmission (MRFFT) sample generation technique is used in every Uplink transmission for in CAT-0 and CAT-4 based LTE implementation technique. Multiple sample frequency transmission techniques will be required for Up Link (UL) based on control information for getting from Down Link (DL). In this paper, an effective R2MDC—MRFFT technique has been used to combine hardware and data flow complexity. The proposed MRFFT architecture uses “Feedback (R2)-Commutator (R4) MRFFT” is utilized for reducing latency and hardware complexity of the data path flow part of NB-IoT in LTE. The design of feedback-commutator architecture provides the same functionality with increasing throughput (2.4 GSamples) with 30.605 ns latency reduction, lower power consumption, and reduced coverage enhancement. The proposed feedback-commutator based frequency transmission technique is compared with the traditional frequency transmission technique used in CAT-0, CAT-4 devices. This enhanced MRFFT sample generation technique is incorporated into data flow heterogeneous architecture of NB-IoT design procedure to improve the design performances of LTE wireless communication in the perspective of coverage in low mobility.

Traditional tracking algorithms reply on manually extracted features while deep learning algorithms can automatically extract features, which is of great importance for single target tracking in the complex underwater environment that is characterized by poor visibility, low contrasts, and occlusion. The convolutional neural network SiamRPN++ is an advanced deep learning algorithm whose backbone network structure is. However, this algorithm is difficult to be implemented in an underwater platform due to its large amount of computation and high requirement for the computing power of the hardware. To solve this problem, a new backbone network structure NewNet-62 characterized by inverted residual bottleneck block and the SiamRPN++ algorithm are proposed. The depth-wise separable convolution in NewNet-62 simplifies the computation so that the SiamRPN++ algorithm can realize fast and accurate tracking of underwater single targets. The video set in VOT2016 was used to test the algorithm. The results show that the SiamRPN++ algorithm using NewNet-62 as the main network has the great performance. Compared with the algorithms using ResNet-50, the accuracy of the SiamRPN++ algorithm increased to 0.524 and its EAO increased to 0.303. And its tracking speed increased to 73.74. The network complexity reduced to 3.013 billion and the network parameters reduced to 12.538 million, which significantly reduces network complexity and number of network parameters.

In recent years, few-shot learning is proposed to solve the problem of lacking samples in deep learning. However, previous works are mainly concentrated on optimizing neural network structures or augmenting the dataset while ignoring the local relationship of the images. Considering that humans pay more attention to the foreground or prominent features of the images during image recognition, we proposed the foreground feature attention module (FFAM) based on an unsupervised saliency detector for few-shot learning. The FFAM consists of two parts: the foreground extraction module and the features attention module. More specifically, we first extract the foreground images by Robust Background Detector (RBD), one of the best unsupervised saliency detectors. Secondly, we employ the same embedding module to extract the features of both original images and foreground images. Finally, we introduce three improvements to enhance the foreground features and make our network focus on the foreground features without losing background information. Our proposed FFAM is more sensitive to the foreground features than previous approaches. Hence, it effectively recognizes those images with similar backgrounds. Extensive experiments are conducted on miniImagenet and tieredImagenet datasets. It is demonstrated that our proposed FFAM greatly improves the accuracy performance over baseline systems for both one-shot and few-shot classification tasks without increasing the network complexity.

Neural Architecture Search (NAS), i.e., the automation of neural network design, has gained much popularity in recent years with increasingly complex search algorithms being proposed. Yet, solid comparisons with simple baselines are often missing. At the same time, recent retrospective studies have found many new algorithms to be no better than random search (RS). In this work we consider the use of a simple Local Search (LS) algorithm for NAS. We particularly consider a multi-objective NAS formulation, with network accuracy and network complexity as two objectives, as understanding the trade-off between these two objectives is arguably among the most interesting aspects of NAS. The proposed LS algorithm is compared with RS and two evolutionary algorithms (EAs), as these are often heralded as being ideal for multi-objective optimization. To promote reproducibility, we create and release two benchmark datasets, named MacroNAS-C10 and -C100, containing 200K saved network evaluations for two established image classification tasks, CIFAR-10 and CIFAR-100. Our benchmarks are designed to be complementary to existing benchmarks, especially in that they are better suited for multi-objective search. We additionally consider a version of the problem with a much larger architecture space. While we find and show that the considered algorithms explore the search space in fundamentally different ways, we also find that LS substantially outperforms RS and even performs nearly as good as state-of-the-art EAs. We believe that this provides strong evidence that LS is truly a competitive baseline for NAS against which new NAS algorithms should be benchmarked.

Recently, the consolidation of Social Network and the Internet of Things (IoT) acquiring appreciation and consideration among researchers due to its immense and adaptable behavior. Social Internet of Things (SIoT) comprehends the adequacy to support the inventive applications and network services with adept and persuasive demeanors. The author’s exertion for this research provides the subsistence to the SIoT amelioration Currently, the SIoT technique proven to be competent, but the exponential growth of heterogeneous smart devices may arise a challenging scenario. Searching the right object, service, or best path to the informative nodes in such highly convoluted networks is always a challenging task. This research strengthen the SIoT paradigm by consolidating the properties of the individual’s Small World. In addition, this research also advises a Smart Social Agent (SSA) to minimize the human intervention. The absolute objective of this paper is two-fold, (i) transformation of SIoT network into Individual’s Small World SIoT network, (ii) searching the right information efficiently by exploiting SSA. To accomplish the goal we have proposed an efficient Query based Searching algorithm which exploits Service Oriented properties of Individual’s Small World SIoT network and searches information. First result shows the significant alleviation in the network complexity, reduces the average path length, and network diameter. Eventually, it improves the network scalability and navigability. While second result shows that the proposed Query based searching algorithm proves to be efficient with its mean execution time not more than 2.66 ms for complete network on two different machines. Whereas, for Individual’s small world network the mean execution time is not more than 2.47 ms for both machine.

This letter presents a sub-6 GHz cost-effective wideband dual-polarized plane wave generator (PWG) design for the fifth-generation (5G) base station over-the-air test. The proposed nonuniform PWG design is composed of 256 elements and has a dimension of 1.7 m × 1.7 m. Several novel strategies are proposed and implemented to reduce the cost and complexity. To reduce the feeding network complexity and mutual coupling among PWG elements, a novel tapered amplitude-only excitation method (i.e., with the same relative phase excitations among all PWG elements) is proposed. To reduce the number of channels in the phase and amplitude control network, a four-element subarray is applied. Furthermore, the wideband PWG element design and nonuniform configuration are applied to support for ultrawideband operations from 2 to 5 GHz. The proposed PWG design is numerically simulated and experimentally validated. A cylindrical quiet zone of a diameter of 0.9 m and a depth of 0.9 m, with 2.1 m distance to the PWG, is achieved in the measurement for the considered bandwidth, with the typical peak-to-peak E-field amplitude and phase variations less than ±1.25 dB and ±7.5°, respectively, which demonstrated its effectiveness and robustness.

Despite the wide use of LCA for environmental profiling, the approach for determining the system boundary within LCA models continues to be subjective and lacking in mathematical rigor. As a result, life cycle models are often developed in an ad hoc manner, and are difficult to compare. Significant environmental impacts may be inadvertently left out. Overcoming this shortcoming can help elicit greater confidence in life cycle models and their use for decision making. This paper describes a framework for hybrid life cycle model generation by selecting activities based on their importance, parametric uncertainty, and contribution to network complexity. The importance of activities is determined by structural path analysis—which then guides the construction of life cycle models based on uncertainty and complexity indicators. Information about uncertainty is from the available life cycle inventory; complexity is quantified by cost or granularity. The life cycle model is developed in a hierarchical manner by adding the most important activities until error requirements are satisfied or network complexity exceeds user-specified constraints. The framework is applied to an illustrative example for building a hybrid LCA model. Since this is a constructed example, the results can be compared with the actual impact, to validate the approach. This application demonstrates how the algorithm sequentially develops a life cycle model of acceptable uncertainty and network complexity. Challenges in applying this framework to practical problems are discussed. The presented algorithm designs system boundaries between scales of hybrid LCA models, includes or omits activities from the system based on path analysis of environmental impact contribution at upstream network nodes, and provides model quality indicators that permit comparison between different LCA models.

In existing works of deep learning-based resource allocation, the scalability degrades heavily with the increase of network complexity, which is due to their limited learning ability of shallow neural networks and insufficient knowledge of network. Nowadays, to address the growth of cell density, cooperative beamforming in user-centric network (UCN) is emerged, where the additional degrees of freedom of multi-antenna and cell coordination aggravate the challenges. This letter proposes a deep residual learning framework, UcnBeamNet, to enhance the ability of approximating the iterative algorithm for sum rate maximization, where multi-branch subnets are connected in parallel to extract extra information. Specifically, a weighted minimum mean square error (WMMSE)-based algorithm is derived to determine the optimal clusters and beamforming matrices; then UcnBeamNet is trained to learn the input-output mapping and provide direct insight of UCN from association matrices in addition to plural inputs. Extensive experiments demonstrate UcnBeamNet still reaches 90.38% sum-rate relative to conventional algorithm even with a large network size, and achieves more than 50, $000\times $ speed up in computational efficiency.

Software-Defined Networking (SDN) is an innovational network architecture which gives network administrators the ability to directly control the whole network by programming on a centralized controller. Due to network complexity, networks are unlikely to be bug-free. The ability to verify data plane properties will make network management easier for network administrators in SDN. In this paper, we present a novel atomic predicates based data plane properties verification method for SDN using Spark which is a big data processing framework. First, we verify packet reachability which is a fundamental data plane property. Then, we verify other data plane properties such as loop-freedom and nonexistence of black holes. In addition, the proposed method can detect a security threat existing in SDN called firewall bypass threat with packet reachability verification. By adopting atomic predicates, we achieve less computational and storage overhead. We implement the methods and study the performance. The results of experiments show that we can efficiently and accurately detect loops, black holes and firewall bypass threats.

Mobile network operators (MNOs) are in the process of overlaying their conventional macro cellular networks with shorter range cells such as outdoor pico cells. The resultant increase in network complexity creates substantial overhead in terms of operating expenses, time, and labor for their planning and management. Artificial intelligence (AI) offers the potential for MNOs to operate their networks in a more organic and cost-efficient manner. We argue that deploying AI in fifth generation (5G) and beyond will require surmounting significant technical barriers in terms of robustness, performance, and complexity. We outline future research directions, identify top five challenges, and present a possible roadmap to realize the vision of AI-enabled cellular networks for Beyond- 5G and sixth generation (6G) networks.

Overfitting is a common problem for computer vision applications It is a problem that when training convolution neural networks and is caused by lack of training data or network complexity. The novel sequence-dropout (SD) method is proposed in this paper to alleviate the problem of overfitting when training networks. The SD method works by dropping out units (channels of feature) from the network in a sequence, replacing the traditional operation of random omitting. Sophisticated aggregation strategies are used to obtain the global information of feature channels, and channel-wise weights are produced by gating mechanism. The SD method then selectively drops out the feature channels according to the channel-wise weights that represent the importance degree of each channel. The proposed SD block can be plugged into state-of-the-art backbone CNN models such as VGGNet and ResNet. The SD block is then evaluated on these models, demonstrating consistent performance gains over the baseline model on widely-used benchmark image classification datasets including MNIST, CIFAR-10, CIFAR-100, and ImageNet2012. Experimental results demonstrate that the superior performance of the SD block compared to other modern methods.

Skin Lesion detection and classification are very critical in diagnosing skin malignancy. Existing Deep learning-based Computer-aided diagnosis (CAD) methods still perform poorly on challenging skin lesions with complex features such as fuzzy boundaries, artifacts presence, low contrast with the background and, limited training datasets. They also rely heavily on a suitable turning of millions of parameters which often leads to over-fitting, poor generalization, and heavy consumption of computing resources. This study proposes a new framework that performs both segmentation and classification of skin lesions for automated detection of skin cancer. The proposed framework consists of two stages: the first stage leverages on an encoder-decoder Fully Convolutional Network (FCN) to learn the complex and inhomogeneous skin lesion features with the encoder stage learning the coarse appearance and the decoder learning the lesion borders details. Our FCN is designed with the sub-networks connected through a series of skip pathways that incorporate long skip and short-cut connections unlike, the only long skip connections commonly used in the traditional FCN, for residual learning strategy and effective training. The network also integrates the Conditional Random Field (CRF) module which employs a linear combination of Gaussian kernels for its pairwise edge potentials for contour refinement and lesion boundaries localization. The second stage proposes a novel FCN-based DenseNet framework that is composed of dense blocks that are merged and connected via the concatenation strategy and transition layer. The system also employs hyper-parameters optimization techniques to reduce network complexity and improve computing efficiency. This approach encourages feature reuse and thus requires a small number of parameters and effective with limited data. The proposed model was evaluated on publicly available HAM10000 dataset of over 10000 images consisting of 7 different categories of diseases with 98% accuracy, 98.5% recall, and 99% of AUC score respectively.

Towards autonomous driving, advanced driver assistance systems increasingly undertake basic driving tasks by replacing human assessment and interactions, when controlling the vehicle. The performance of these systems is directly related to knowledge of the vehicle’s state and influential parameters. In this respect, the road condition has a major influence on the tires’ traction and thus significantly affects the behavior of the vehicle. Therefore, a prediction of the upcoming road condition can improve the performance of the assistance systems which leads to an increased driving safety and comfort. The presented work aims to classify the road surface as well as its weather-related condition, based on images of the front camera view, using deep convolutional neural networks. In order to take computational limitations of vehicle control units into account, a pruning approach is investigated to reduce the network complexity.

Driven by the growth of cloud and virtualization technologies, telecom services are under pressure to become increasingly dynamic. Cloud services can be instantiated in a matter of minutes on computing platforms, which can rapidly schedule and allocate virtualized physical resources. A similar flexibility to adapt to dynamic service requests remains a challenge with network resources. Recently, centralized controllers have been deployed in data center networks to adapt network connectivity to computational needs. However, due to the complexity and variety of carrier networking technologies, carrier networks have been slow to move towards centralized control. More recently, carriers are evolving to address network complexity by pushing for the disaggregation of network elements and the introduction of equipment with more control flexibility. Carriers require a new control paradigm to benefit from this new flexibility, which extends far beyond today's OpenFlow network controllers. The lack of a global network controller with a consistent network view slows down the development and deployment of the more dynamic network services required by extensive cloud technologies. To address the scaling requirements of the carrier environment, the envisioned network controller would be implemented as a distributed system. In this article, we describe a global network controller designed to function as a network operating system with multi-domain, multi-layer capabilities that will transform carrier-grade telecom services. We define the requirements for a carrier-grade network operating system (OS) and describe a prototype system (NetUNIX) that can control heterogeneous networks at large scale and ensure high service reliability. We highlight the advantages of the NetUNIX platform in the context of other open source platforms and illustrate the use of the NetUNIX platform with two use cases: a metro-distributed data center and 5G next-generation fronthaul.

There are a large number of insulators on the transmission line, and insulator damage will have a major impact on power supply security. Image-based segmentation of the insulators in the power transmission lines is a premise and also a critical task for power line inspection. In this paper, a modified conditional generative adversarial network for insulator pixel-level segmentation is proposed. The generator is reconstructed by encoder-decoder layers with asymmetric convolution kernel which can simplify the network complexity and extract more kinds of feature information. The discriminator is composed of a fully convolutional network based on patchGAN and learns the loss to train the generator. It is verified in experiments that the proposed method has better performances on mIoU and computational efficiency than Pix2pix, SegNet, and other state-of-the-art networks.

Purpose The purpose of this paper is to consider one of the recent and practical extensions of the resource-constrained project scheduling problem (RCPSP) termed as the multi-skill resource-constrained project scheduling problem (MSRCPSP) for investigation. The objective is the minimization of the makespan or total project duration. Design/methodology/approach To solve this complex problem, the authors propose a teaching–learning-based optimization (TLBO) algorithm in which self-study and examination have been used as additional features to enhance its exploration and exploitation capabilities. An activity list-based encoding scheme has been modified to include the resource assignment information because of the multi-skill nature of the algorithm. In addition, a genetic algorithm (GA) is also developed in this work for the purpose of comparisons. The computational experiments are performed on 216 test instances with varying complexity and characteristics generated for the purpose. Findings The results obtained after computations show that the TLBO has performed significantly better than GA in terms of average percentage deviation from the critical path-based lower bound for different combinations of three parameters, namely, skill factor, network complexity and modified resource strength. Research limitations/implications The modified TLBO proposed in this paper can be conveniently applied to any product or service organization wherein human resources are involved in executing project activities. Practical implications The developed model can suitably handle resource allocation problems faced in real-life large-sized projects usually administered in software development companies, consultancy firms, R&D-based organizations, maintenance firms, big construction houses, etc. wherein human resources are involved. Originality/value The current work aims to propose an effective metaheuristic for a more realistic version of MSRCPSP, in which resource requirements of activities may be more than one. Moreover, to enhance the exploration and exploitation capabilities of the original TLBO, the authors use two additional concepts, namely, self-study and examination in the search process.

The human brain is an organic which whole consisting of hundreds of billions of neurons, and the neural network is the information path of the brain for processing information. Network characteristics will change along with network topology or node change, and the change may lead to brain lesions. Using EEG to construct brain network can effectively discover the internal mechanism of improvement in human brain, but the traditional method of constructing brain network is difficult to reflect the network complexity because of the small number of network nodes. In this paper, a new algorithm of brain network construction based on chaotic time series is proposed. The method estimates neuronal dynamic equations and neuronal networks topology by using only noisy time series. By analyzing the network constructed based on phase space reconstruction of Lorenz system, network characteristics that the average path length increase as embedding dimension, and the cluster coefficient on the contrary are found. The topology and statistical characteristics of the network constructed from neuronal chaotic bursting can reflect time-evolution of time series. Through analyzing the real epileptic EEG signals dataset using the proposed complex networks construction method, we found that epileptic seizure is different from seizure-free intervals in network topology and statistical characteristics. Epileptic EEG of before, during and after a tonic-clonic seizure indicate that cluster coefficients of the sliding local networks significantly increased during a seizure, so the result can provide clonic seizure prediction. The proposed method of complex network topology estimation and complex network construction of time series can depict the dynamic characteristics of brain network, and provide a new idea for the treatment of mental illness.

Smart tuning of a filter depends very much on an accurate parametric model. Here, the authors develop a parametric model for a microwave filter based on an improved extreme learning machine (ELM). First, a coupling matrix is extracted from scattering parameters in undesirable states. The automatic decomposition and modular training strategy greatly reduces the network complexity. Next, by increasing the size of the output matrix of the hidden-layer, the number of hidden layer nodes can be changed. Finally, the model parameters are optimised by combining the particle swarm optimisation (PSO) algorithm and the differential evolution (DE) algorithm. The presented parametric model benefits from the information of the fusion mechanism with a hybrid optimisation algorithm and succeeds in avoiding slow convergence and premature influence on the system. It also achieves the desired training accuracy within the target time. Compared with prediction based on back-propagation (BP) neural networks and least-squares support vector machines (LS-SVMs), the proposed method can be generalised and has a better modelling accuracy.

Improvement of mobility, especially environment-friendly green mobility, is challenging in existing megacities due to road network complexity and space constraints. Endorsing the bicycle lane network (BLN) in congested megacities is a promising option to foster green mobility. This research presents a novel bioinspired network design method that considers various constraints and preferences related to the megacity for designing an optimal BLN. The proposed method is inspired by natural Physarum polycephalum, a brainless, multi-headed single-celled organism, which is capable of developing a reticulated network of complex foraging behaviors in pursuit of food. The mathematical model of Physarum foraging behavior is adapted to maneuver various BLN constraints in megacity contexts in designing the optimal BLN. The Physarum-inspired BLN method is applied to two case studies on the megacity Dhaka for designing BLNs: the first one covers congested central city area, and the second one covers a broader area that includes major locations of the city. The obtained BLNs were evaluated comparing their available routes between different locations with the existing vehicle routes of the city in terms of distance and required travel times in different time periods, and the BLN routes were found to be suitable alternatives for avoiding congested main roads. The expected travel time using BLNs is shorter than other transport (e.g., car and public bus), additionally, at glance, the average travel speed on BLNs is almost double that of public buses in peak hours. Finally, the designed BLNs are promising for environment-friendly and healthy mobility.

Crop disease diagnosis is of great significance to crop yield and agricultural production. Deep learning methods have become the main research direction to solve the diagnosis of crop diseases. This paper proposed a deep convolutional neural network that integrates an attention mechanism, which can better adapt to the diagnosis of a variety of tomato leaf diseases. The network structure mainly includes residual blocks and attention extraction modules. The model can accurately extract complex features of various diseases. Extensive comparative experiment results show that the proposed model achieves the average identification accuracy of 96.81% on the tomato leaf diseases dataset. It proves that the model has significant advantages in terms of network complexity and real-time performance compared with other models. Moreover, through the model comparison experiment on the grape leaf diseases public dataset, the proposed model also achieves better results, and the average identification accuracy of 99.24%. It is certified that add the attention module can more accurately extract the complex features of a variety of diseases and has fewer parameters. The proposed model provides a high-performance solution for crop diagnosis under the real agricultural environment.

With increasing network complexity, intelligent mechanisms to efficiently achieve the required quality of service of wireless-enabled applications are being developed, especially for industrial environments due to the onset of the fourth industrial revolution. In this paper, the potential benefits of wireless channel quality prediction for two of the three major use cases supported by 5G viz. enhanced Mobile BroadBand (eMBB) and Ultra-Reliable Low Latency Communication (URLLC) are quantified in an industrial indoor environment through simulations. Our analysis shows that the ability to perform perfect prediction improves the 10th user throughput percentile by up to 125% for eMBB use case and decreases the 90th resource utilization percentile by up to 37% for URLLC use case. Furthermore, the maximum tolerable prediction inaccuracy is found to be up to 5 dB and 0.35 dB for eMBB and URLLC use cases, respectively.

Super resolution (SR) enables to generate a high-resolution (HR) image from one or more low-resolution (LR) images. Since a variety of CNN models have been recently studied in the areas of computer vision, these approaches have been combined with SR in order to provide higher image restoration. In this paper, we propose a lightweight CNN-based SR method, named multi-scale channel dense network (MCDN). In order to design the proposed network, we extracted the training images from the DIVerse 2K (DIV2K) dataset and investigated the trade-off between the SR accuracy and the network complexity. The experimental results show that the proposed method can significantly reduce the network complexity, such as the number of network parameters and total memory capacity, while maintaining slightly better or similar perceptual quality compared to the previous methods.

Wireless Communication Network (WSN) applications are growing drastically daily, and their extensive nature attracts researchers and development authorities. The critical issues in wireless communication have been answering by advancements in cognitive radio (CR) techniques. The dynamic spectrum analysis allows cognitive radio at the receiver section to increase the spectrum efficiency and power consumption. The next-generation wireless networks, namely 5G technologies, accept advanced cognitive radios in their functionality. In this work, teaching and Learning based optimization are proposed for future communications purpose. The main aim of the study is to increase the spectral efficiency as well as energy consumption. The earlier methods can only targeting transmission rate and network complexity. The proposed adaptive teaching and learning based WCN can improve 78% energy efficiency and 47% data rate.

This paper proposes a customized convolutional neural network for crack detection in concrete structures. The proposed method is compared to four existing deep learning methods based on training data size, data heterogeneity, network complexity, and the number of epochs. The performance of the proposed convolutional neural network (CNN) model is evaluated and compared to pretrained networks, i.e., the VGG-16, VGG-19, ResNet-50, and Inception V3 models, on eight datasets of different sizes, created from two public datasets. For each model, the evaluation considered computational time, crack localization results, and classification measures, e.g., accuracy, precision, recall, and F1-score. Experimental results demonstrated that training data size and heterogeneity among data samples significantly affect model performance. All models demonstrated promising performance on a limited number of diverse training data, however, increasing the training data size and reducing diversity reduced generalization performance, and led to overfitting. The proposed customized CNN and VGG-16 models outperformed the other methods in terms of classification, localization, and computational time on a small amount of data, and the results indicate that these two models demonstrate superior crack detection and localization for concrete structures.

Recent years have witnessed a big leap in automatic visual saliency detection attributed to advances in deep learning, especially Convolutional Neural Networks (CNNs). However, inferring the saliency of each image part separately, as was adopted by most CNNs methods, inevitably leads to an incomplete segmentation of the salient object. In this paper, we describe how to use the property of part-object relations endowed by the Capsule Network (CapsNet) to solve the problems that fundamentally hinge on relational inference for visual saliency detection. Concretely, we put in place a two-stream strategy, termed Two-Stream Part-Object RelaTional Network (TSPORTNet), to implement CapsNet, aiming to reduce both the network complexity and the possible redundancy during capsule routing. Additionally, taking into account the correlations of capsule types from the preceding training images, a correlation-aware capsule routing algorithm is developed for more accurate capsule assignments at the training stage, which also speeds up the training dramatically. By exploring part-object relationships, TSPORTNet produces a capsule wholeness map, which in turn aids multi-level features in generating the final saliency map. Experimental results on five widely-used benchmarks show that our framework consistently achieves state-of-the-art performance. The code can be found on https://github.com/liuyi1989/TSPORTNet.

Existing crowd evacuation guidance systems require the manual design of models and input parameters, incurring a significant workload and a potential for errors. This paper proposed an end-to-end intelligent evacuation guidance method based on deep reinforcement learning, and designed an interactive simulation environment based on the social force model. The agent could automatically learn a scene model and path planning strategy with only scene images as input, and directly output dynamic signage information. Aiming to solve the &ldquo, dimension disaster&rdquo, phenomenon of the deep Q network (DQN) algorithm in crowd evacuation, this paper proposed a combined action-space DQN (CA-DQN) algorithm that grouped Q network output layer nodes according to action dimensions, which significantly reduced the network complexity and improved system practicality in complex scenes. In this paper, the evacuation guidance system is defined as a reinforcement learning agent and implemented by the CA-DQN method, which provides a novel approach for the evacuation guidance problem. The experiments demonstrate that the proposed method is superior to the static guidance method, and on par with the manually designed model method.

Recently, facial priors based face super-resolution (SR) methods have obtained significant performance gains in dealing with extremely degraded facial images, and facial priors have also been proved useful in facilitating the inference of face images. Based on this, how to efficiently fuse facial priors into deep features to improve face SR performance has attracted a major attention. In this paper, we propose a learning-based face SR approach with Incremental boosting Parsing information (dubbed IPFSR) for high magnification factor. The proposed IPFSR method consists of three main parts: i) a multi-stage parsing map embedded features upsampling network, in which image recovery and prior estimation processes are performed simultaneously and progressively to improve image resolution; ii) a progressive training method and joint facial attention loss and heatmap loss to obtain better facial attributes; iii) the channel attention strategy is introduced into residual dense blocks to adaptively learn facial features. Extensive experimental results show that compared with the state-of-the-art methods in terms of quantitative and qualitative metrics, our approach can achieve an outstanding balance between SR image quality and low network complexity.

This paper introduces versatile filters to construct efficient convolutional neural networks that are widely used in various visual recognition tasks. Considering the demands of efficient deep learning techniques running on cost-effective hardware, a number of methods have been developed to learn compact neural networks. Most of these works aim to slim down filters in different ways, \eg,~investigating small, sparse or quantized filters. In contrast, we treat filters from an additive perspective. A series of secondary filters can be derived from a primary filter with the help of binary masks. These secondary filters all inherit in the primary filter without occupying more storage, but once been unfolded in computation they could significantly enhance the capability of the filter by integrating information extracted from different receptive fields. Besides spatial versatile filters, we additionally investigate versatile filters from the channel perspective. Binary masks can be further customized for different primary filters under orthogonal constraints. We conduct theoretical analysis on network complexity and an efficient convolution scheme is introduced. Experimental results on benchmark datasets and neural networks demonstrate that our versatile filters are able to achieve comparable accuracy as that of original filters, but require less memory and computation cost., Comment: Accepted by TPAMI. Extended version of NeurIPS 2018 paper

Spatial transformer networks (STNs) were designed to enable convolutional neural networks (CNNs) to learn invariance to image transformations. STNs were originally proposed to transform CNN feature maps as well as input images. This enables the use of more complex features when predicting transformation parameters. However, since STNs perform a purely spatial transformation, they do not, in the general case, have the ability to align the feature maps of a transformed image with those of its original. STNs are therefore unable to support invariance when transforming CNN feature maps. We present a simple proof for this and study the practical implications, showing that this inability is coupled with decreased classification accuracy. We therefore investigate alternative STN architectures that make use of complex features. We find that while deeper localization networks are difficult to train, localization networks that share parameters with the classification network remain stable as they grow deeper, which allows for higher classification accuracy on difficult datasets. Finally, we explore the interaction between localization network complexity and iterative image alignment., Comment: 13 pages, 7 figures, 6 tables

Community detection of complex networks has always been a hot issue. With the mixed parameters μ increase in network complexity, community detection algorithms need to be improved. Based on previous work, the paper designs a novel algorithm from the perspective of node betweenness properties and gives the detailed steps of the algorithm and simulation results. We compare the proposed algorithm with a series of typical algorithms through experiments on synthetic and actual networks. Experimental results on artificial and real networks demonstrate the effectiveness and superiority of our algorithm.

International audience; This paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts in the decoded images. In this paper, the main focus has been put on decisions defining the prediction signal in intra and inter frames. This information has been used in the training phase as well as input to help the process of learning artifacts that are specific to each coding type. Furthermore, to retain a low memory requirement for the proposed method, one model is used for all Quantization Parameters (QPs) with a Quantization Parameter (QP)-map, which is also shared between luma and chroma components. In addition to the Post Processing (PP) approach, the In-Loop Filtering (ILF) codec integration has also been considered, where the characteristics of the Group of Pictures (GoP) are taken into account to boost the performance. The proposed CNN-based Quality Enhancement (QE) framework has been implemented on top of the Versatile Video Coding (VVC) Test Model (VTM-10). Experiments show that the prediction-aware aspect of the proposed method improves the coding efficiency gain of the default CNN-based QE method by 1.52%, in terms of BD-BR, at the same network complexity compared to the default CNN-based QE filter.

To infer structures in diffusion networks, existing approaches mostly need to know not only the final infection statuses of network nodes, but also the exact times when infections occur. In contrast, in many real-world settings, such as disease propagation, monitoring exact infection times is often infeasible due to a high cost. We investigate the problem of how to learn diffusion network structures based on only the final infection statuses of nodes. Instead of utilizing sequences of timestamps to determine potential parent-child influence relationships between nodes, we propose to find influence relationships with high statistical significance. To this end, we design a probabilistic generative model of the final infection statuses to quantitatively measure the likelihood of potential structures of the objective diffusion network, taking into account network complexity. Based on this model, we can infer an appropriate number of most probable parent nodes for each node in the network. Furthermore, to reduce redundant inference computations, we are able to preclude insignificant candidate parent nodes from being considered during inferencing, if their infections have little correlation with the infections of the corresponding child nodes. Extensive experiments on both synthetic and real-world networks offer evidence that the proposed approach is effective and efficient.

With the development of mobile communication network technology, the network scale is expanding, the network complexity is also improved. Once the network fails, it will cause a series of devices to generate numerous alarms. Therefore, it is particularly important to accurately predict the alarms, which can prevent the upcoming network alarms in advance or take other measures to reduce losses. Traditional network diagnosis mainly depends on network administrators, which has many subjective factors and low efficiency, and needs to train experienced technicians, which leads to long cycle and high cost. In addition, the manual method is difficult to use a large amount of historical data. In order to solve the problem of network alarm prediction more intelligently, data mining technology is an effective solution, which can mine valuable information from a large number of historical network alarm data. In this paper, combined with the actual network alarm log data of operators and considering the time correlation of alarms, a two-class network alarm prediction (LAB-NAP) algorithm based on LSTM and association algorithm is proposed, which adopts the method of combining LSTM-based two-class model with network alarm association mode. The LAB-NAP algorithm is compared with LSTM-based two-class network alarm prediction (LB-NAP) algorithm and LM-NAP, and the performance of three network alarm prediction modeling methods on the same data set is compared. It is verified that the LAB-NAP method proposed in this paper balances the prediction effect and training time in the large-scale dataset.

Feedforward-designed convolutional neural network (FF-CNN) is an interpretable network. The parameter training of the model does not require backpropagation (BP) and optimization algorithms (SGD). The entire network is based on the statistical data output by the previous layer, and the parameters of the current layer are obtained through one-pass manner. Since the network complexity under the FF design is lower than the BP algorithm, FF-CNN has better utility than the BP training method in the directions of semi-supervised learning, ensemble learning, and continuous subspace learning. However, the FF-CNN training process or model release will cause the privacy of training data to be leaked. In this paper, we analyze and verify that the attacker can obtain the private information of the original training data after mastering the training parameters of FF-CNN and the partial output responses. Therefore, the privacy protection of training data is imperative. However, due to the particularity of the FF-CNN training method, the existing deep learning privacy protection technology is not applicable. So we proposed an algorithm called differential privacy subspace approximation with adjusted bias (DPSaab) to protect the training data in FF-CNN. According to the different contribution of the model filters to the output response, we design the privacy budget allocation according to the ratio of the eigenvalues and allocate a larger privacy budget to the filter with a large contribution, and vice versa. Extensive experiments on MNIST, Fashion-MNIST, and CIFAR-10 datasets show that DPSaab algorithm has better utility than existing privacy protection technologies.

To tackle real-world challenges, deep and complex neural networks are generally used with a massive number of parameters, which require large memory size, extensive computational operations, and high energy consumption in neuromorphic hardware systems. In this work, we propose an unsupervised online adaptive weight pruning method that dynamically removes non-critical weights from a spiking neural network (SNN) to reduce network complexity and improve energy efficiency. The adaptive pruning method explores neural dynamics and firing activity of SNNs and adapts the pruning threshold over time and neurons during training. The proposed adaptation scheme allows the network to effectively identify critical weights associated with each neuron by changing the pruning threshold dynamically over time and neurons. It balances the connection strength of neurons with the previous layer with adaptive thresholds and prevents weak neurons from failure after pruning. We also evaluated improvement in the energy efficiency of SNNs with our method by computing synaptic operations (SOPs). Simulation results and detailed analyses have revealed that applying adaptation in the pruning threshold can significantly improve network performance and reduce the number of SOPs. The pruned SNN with 800 excitatory neurons can achieve a 30% reduction in SOPs during training and a 55% reduction during inference, with only 0.44% accuracy loss on MNIST dataset. Compared with a previously reported online soft pruning method, the proposed adaptive pruning method shows 3.33% higher classification accuracy and 67% more reduction in SOPs. The effectiveness of our method was confirmed on different datasets and for different network sizes. Our evaluation showed that the implementation overhead of the adaptive method regarding speed, area, and energy is negligible in the network. Therefore, this work offers a promising solution for effective network compression and building highly energy-efficient neuromorphic systems in real-time applications.

In this paper, we propose an effective way of providing conditional features for a flow-based neural vocoder. Most conventional approaches utilize mel-spectrograms for conditioning neural vocoders, but this significantly increases the size of neural networks due to their high dimensional behavior. We show that the network size of a flow-based generative model can be reduced when we use acoustic parameters for a sinusoidal speech analysis-and-synthesis framework such as voiced/unvoiced flag, fundamental frequency, mel-cepstral coefficients, and energy of each analysis frame. We also conclude that training becomes much easier if we feed the fundamental frequency by an embedded vector representation after quantizing it with a small number of bits. Experimental results verify that the performance of the proposed algorithm is comparable to that of flow-based neural vocoders conditioned on mel-spectrograms while the required information for the feature representations and network complexity for generating speech become lower.

To enable network service providers to better allocate network resources, a variety of traffic identification algorithms have been developed, but few new features with stronger differentiation power are reported, causing unsatisfactory False Positive Rate (FPR) performance in existing methods. To deal with this issue, this paper presents a new method for recognizing streaming video sources in encrypted tunnels based on deep learning (DL). A modified Kolmogorov-Smirnov Discretization (MKSD) algorithm that produces a continuously differentiable histogram curve, is developed to train a deep neural network (DNN) model for optimal histogram bin placement. Experimental validation with simulated real network complexity (traffic combinations in encrypted VPN tunnels) demonstrates that the MKSD algorithm can improve True Positive Rate (TPR) for system identification, while reducing FPR. Compared with existing methods, the proposed method shows higher identification accuracy of video sources in single and multi-stream scenarios.

Networking pruning is widely utilized for accelerating the inference procedure of deep models in low-resource settings. In this paper, a novel Gradient-based method, Soft Taylor Pruning(STP), are proposed to reduce the network complexity in dynamic way. Given a global compression rate, all filters are categorized into two parts by the gradient-based evaluation criterion in the pruning process. Then, two types of filters are remixed and updated in the training epoch. When a pruning error occurs, the model can correct the pruning error by re-active pruned filters in the next training epoch. In this way, the capacity of the model channel space remains the same until the network structure converges. In order to reduce the impact of large-weighted filters on criterion, We take the absolute value of the product of the feature map and the gradient as the evaluation criterion. So as to reduce the model pruning time, STP allows simultaneous pruning on multiple layers by controlling the opening and closing of multiple mask layers. Moreover, STP can be applied to various advanced CNNs, such as MobileNet. The features of our method are: 1) maintain the integrity of the model channel space; 2) less time cost of model compression; 3)less dependence on the pretrained model.

In this study, we work on online learning with long short term memory (LSTM) based networks is investigated. For LSTM-based online learning, a computationally efficient training algorithm based on extended Kalman filter (EKF) is proposed. We show the considerable performance improvements achieved by the proposed algorithm through comparing with conventional LSTM methods in simulations. Results particularly show that the proposed algorithm achieves very similar performance with conventional methods in 25-40 times shorter training time depending on the network complexity.

Over the last two decades, research funding bodies have supported “Future Internet”, “New-IP”, and “Next Generation” design initiatives intended to reduce network complexity by redesigning the network protocol architecture, questioning some of its key principles [...]

Software-Defined Network (SDN) has been developed to reduce network complexity through control and manage the whole network from a centralized location. Today, SDN is widely implemented in many data center's network environments. Nevertheless, emerging technology itself can lead to many vulnerabilities and threats which are still challenging for manufacturers to address it. Therefore, deploying Intrusion Detection Systems (IDSs) to monitor malicious activities is a crucial part of the network architecture. Although the centralized view of the SDN network creates new opportunities for the implementation of IDSs, the performance of these detection techniques relies on the quality of the training datasets. Unfortunately, there are no publicly available datasets that can be used directly for anomaly detection systems applied in SDN networks. The majority of the published studies use non-compatible and outdated datasets, such as the KDD'99 dataset. This manuscript aims to generate an attack-specific SDN dataset and it is publicly available to the researchers. To the best of our knowledge, our work is one of the first solutions to produce a comprehensive SDN dataset to verify the performance of intrusion detection systems. The new dataset includes the benign and various attack categories that can occur in the different elements of the SDN platform. Further, we demonstrate the use of our proposed dataset by performing an experimental evaluation using eight popular machine-learning-based techniques for IDSs.

Persistent packet loss in the cloud-scale overlay network severely compromises tenant experiences. Cloud providers are keen to automatically and quickly determine the root cause of such problems. However, existing work is either designed for the physical network or insufficient to present the concrete reason of packet loss. In this paper, we propose to record and analyze the on-site forwarding condition of packets during packet-level tracing. The cloud-scale overlay network presents great challenges to achieve this goal with its high network complexity, multi-tenant nature, and diversity of root causes. To address these challenges, we present VTrace, an automatic diagnostic system for persistent packet loss over the cloud-scale overlay network. Utilizing the "fast path-slow path" structure of virtual forwarding devices (VFDs), e.g., vSwitches, VTrace installs several "coloring, matching and logging" rules in VFDs to selectively track the packets of interest and inspect them in depth. The detailed forwarding situation at each hop is logged and then assembled to perform analysis with an efficient path reconstruction scheme. Experiments are conducted to demonstrate VTrace's low overhead and quick responsiveness. We share experiences of how VTrace efficiently resolves persistent packet loss issues after deploying it in Alibaba Cloud for over 20 months.

The added value of Social Internet of Things (SIoT) is constantly highlighted during the recent years. The idea is to exploit the social relationships among real-world smart heterogeneous objects to the benefit of their owners, through e.g., search and finding, dedicated services, tasks augmentation. In this paper we discuss a decentralized human-centered simulator, DANOS, that enhances objects' profiles and their interaction behavior with intelligence, based on specific human aspects, i.e., personality traits. Preliminary results show that when objects travel with intelligence in the virtual space, they are able to locate faster similar objects, establishing stronger and more qualitative relationships, while at the same time minimizing the network complexity and load. Such results, increase the probability of discovering faster the information based on given intents and providing best-fit recommendations with fewer costs.

A novel mechanism to produce circularly polarized (CP) waves using low-profile single-layer stub-arrays is proposed. A line delay of 90-degree and a differential feeding network are incorporated into the single-layer antenna design in order to reduce the feeding network complexity into one radiating package. The proposed low-profile structure leads to a gain of 11 dBi within the 60 GHz ISM Band with an adjustable frequency-selective CP radiation.

We propose a neuroevolutionary speciation mechanism that is applied on NeuroEvolution of Augmenting Topologies (NEAT) that solely evolves neural networks’ topology and weights and its extension HA-NEAT that also evolves activation functions. The new speciation mechanism is defined based on the behavior of the individuals rather than their topological similarity. Focusing on classification tasks we build artificial datasets of high complexity. Performance is described by (i) median classification accuracy, (ii) computational efficiency (number of generations) and (iii) network complexity (number of nodes and connections). The performance metrics are compared using Kruskal-Wallis hypothesis tests with Bonferroni correction. It is found that the proposed behavioral speciation mechanism outperforms the original speciation solving problems that were not solvable before or improving the accuracy and reducing the network complexity.

With the emergence of e-commerce, the closed-loop supply chain composed of e-commerce direct sale channel and reverse logistics formed by return and exchange is becoming more and more complex. The lead time of each node in the supply chain increases the uncertainty of the system, making it more difficult to restrain bullwhip effect. The model predictive control (MPC) method is used to restrain the bullwhip effect. Firstly, based on the state-space model of the mixed recovery dual-channel closed-loop supply chain with lead time, an augmented state-space model is established through state reorganization to solve network complexity caused by the lead time; Secondly, a structural diagram is constructed in the SIMULINK environment. The prediction model, the state estimation, and the objective function are presented. Thirdly, the KWIK quadratic programming method is derived, and the solution steps of the method are given. Finally, by simulating with random demand and sinusoidal demand as system input, the effectiveness of the method is verified. The results show that this method can effectively restrain the bullwhip effect and make the production, ordering and inventory curves tend to be smooth.