Search

A common belief in designing deep autoencoders (AEs), a type of unsupervised neural network, is that a bottleneck is required to prevent learning the identity function. Learning the identity function renders the AEs useless for anomaly detection. In this work, we challenge this limiting belief and investigate the value of non-bottlenecked AEs. The bottleneck can be removed in two ways: (1) overparameterising the latent layer, and (2) introducing skip connections. However, limited works have reported on the use of one of the ways. For the first time, we carry out extensive experiments covering various combinations of bottleneck removal schemes and datasets using variants of Bayesian AEs. In addition, we propose the infinitely-wide AEs as an extreme example of non-bottlenecked AEs. Their improvement over the baseline implies learning the identity function is not trivial as previously assumed. Moreover, we find that non-bottlenecked architectures (highest AUROC=0.905) can outperform their bottlenecked counterparts (highest AUROC=0.714) on a recent benchmark of CIFAR (inliers) vs SVHN (anomalies), among other tasks, shedding light on the potential of developing non-bottlenecked AEs for improving anomaly detection.

The performance of a hardware accelerator is often limited by the communication bandwidth between local on-chip memories and DRAM across on-chip bus. In this paper, a system-level performance estimation algorithm is newly proposed for evaluating the communication performance of direct memory access (DMA) controlled accelerators. The proposed algorithm can estimate the communication performance accurately for both DRAM-limited and bus-limited cases. In detail, the communication performance for the DRAM-limited case is estimated using dynamic prediction of DRAM command patterns whereas the communication performance for the bus-limited case is estimated based on the maximum bus burst latency. Depending on whether the communication bandwidth is limited by the bus protocol overhead or the DRAM latency, the proposed algorithm estimates the communication bandwidth of a DMA-controlled accelerator according to the performance bottleneck. It is shown that the proposed algorithm significantly improves the estimation accuracy when it is applied to CNNs and wireless communications. Also, when the proposed algorithm together with a full-system simulator is used to explore a design space defined by a set of tile sizes and bus-related parameters, it speeds up conventional algorithms by more than a factor of 100 by filtering out a large number of unpromising design points. It is also shown that the proposed algorithm alone can approach the maximum accelerator performance with a performance degradation of less than 5%. An ablation study is applied to prove the efficacy of individual steps of the proposed algorithm.

The dock shop hoisting process is the bottleneck stage of the entire shipbuilding lifecycle, with a crucial influence on a shipyard’s competitive goals of cost and delivery time. The complexity and particularity of the hoisting process, which tends to be a project-type manufacturing process, leads to more difficulties in exploiting potentialities for a productivity improvement via computer simulation technology. In order to overcome the deficiency of the previous modelling and simulation methods in terms of the applicability to project-type manufacturing cases, the characteristics of the hoisting process are carefully investigated, followed by a discussion of the specific requirements of constructing a simulation suitable for this kind of process. Then, we apply a task-centric idea to establish an information flow and work flow integrated model (named IWI) that can properly abstract the dynamic behavior of the hoisting process, in which the interrelations between the production elements are quite different from those in the general discrete manufacturing process. According to the mechanism revealed by the IWI model, a simulation algorithm is further designed as the foundation to realize a subsequently developed prototype system. The validity of the proposed modelling and simulation method is tested with a case study on a group of actual dock shop hoisting processes, with multiple indices indicating how well the production system performs output through the simulation experiment.

Massive tremendous amount of miniaturized wireless Internet of Things (IoT) devices are widely employed in many fields such as industrial production, social life, public (and defense) security and management of human society. The limitation of node device's energy capacity is the bottleneck issue of these network systems. MAC protocol is a key communication protocol for such sensor nodes which both rationally saves energy (an alternative to energy harvesting way) and improves the performances of the wireless sensor networks. There are complex tradeoff optimization relationships between the size of contention window and energy consumption, delay and collision, in which too large or too small contention window value cannot make the network performance optimal. This paper firstly gives an optimization algorithm for the size of the contention window through theoretical analysis, which can achieve a compromise between energy consumption (i.e. alternative energy harvesting) and delay. Then, a global view based adaptive contention window (GV-ACW) MAC protocol is proposed to further reduce latency and improve alternative energy harvesting. The GV-ACW MAC protocol adopts the optimized size of contention window in the near sink area to meet the functional requirements of data forwarding, while in the far sink area, the size of contention window is larger than it required by node for data transmission so as to reduce the latency and thereby improve the network performance as a whole. The theoretical analysis and experimental results show that, comparing with previous MAC protocol, GV-ACW protocol can realize effective alternative energy harvesting which resulting increasement of the network lifetime by 6% and reduce the network delay by 15%.

With the increasing pressure from the market and the surge of “Industry 4.0,” staying competitive and relevant is becoming more and more difficult. The assembly line, which represents a long-term investment of the manufacturing industry, needs to be efficiently utilized. While assembly line balancing (ALB) problem had been studied for decades, oversights on the bottleneck resources could significantly impede its efficiency. In leveraging such information as part of the optimization problem, a contagious artificial immune network (CAIN) approach is proposed to simultaneously address ALB efficiency and bottleneck resources while achieving a truly balanced production line. A computational experiment conducted on benchmark data sets has demonstrated a proof-of-concept, where leveraging knowledge-intensive optimization approach had successfully produced high-quality solutions up to 100% improvement with statistically significant justification. Such findings may play an essential determinant in the manufacturing industry, whether being relevant or left out in the era of increasingly being information-driven.

In order to cope with the needs of fifth-generation (5G) cellular networks and beyond, phased-array antenna systems operating at millimeter-wave (mm-wave) frequencies will be required. This makes the system design very complex. In order to create insight and agility in the design process, we propose a framework that visualises the requirements and trade-offs of 5G-and-beyond systems. Our literature survey uses this framework to compare state-of-the-art papers on Silicon-based beamforming integrated circuits (BFICs) operating in the mm-wave band. Three use-cases are analyzed: Base-stations (BSs), Gateways (GtWs) and User Terminals (UTs). Based on the framework, we explore which implementation fits best with each use-cases. In UT, space and power consumption are the main constraints. For BSs, the main constraint is in output power and noise figure (NF). Finally, in GtW applications there is more flexibility as it has a larger footprint than UT but doesn’t necessarily need to cover the same link-budget constraints of BSs. One of the identified limitations throughout all the cases is the heat generation, which is seen as a major bottleneck in mm-wave phased arrays. Only a few of the references show proper modelling and simulations for heat transfer of the realized BFICs. Finally, a limitation in the BFICs is the output power. In order to realize a mm-wave link at least 13 dBm would be required at the input of each antenna element. Only few references meet this criterion, and only at saturation. Further, in order to achieve more than 13 dBm in back-off operation a higher power density would be required. This would imply a further increase of heat generation in the system.

Drug repositioning, discovering new indications for existing drugs, is known to solve the bottleneck of drug discovery and development. To support a task of drug repositioning, many in silico methods have been proposed for predicting drug-disease associations. A meta-path based approach, which extracts network-based information through paths from a drug to a disease, can produce comparable performance with less required information when compared to other approaches. However, existing meta-path based methods typically use counts of extracted paths and discard information of intermediate nodes in those paths although they are very important indicators, such as drug- and disease-associated proteins. Herein, we propose an ensemble learning method with Meta-path based Gene ontology Profiles for predicting Drug-Disease Associations (MGP-DDA). We exploit gene ontology (GO) terms to link drugs and diseases to their associated functions and act as intermediate nodes in a drug-GO-disease tripartite network. For each drug-disease pair, MGP-DDA utilizes meta-paths to generate novel profiles of GO functions, termed as meta-path based GO profiles. We train bagging and boosting classifiers with those novel features to recognize known (positive) from unknown (unlabeled) drug-disease associations. Consequently, MGP-DDA outperforms the state-of-the-art methods and yields the precision of 88.6%. By MGP-DDA, the eminent number of new drug-disease associations with supporting evidence in ClinicalTrials.gov (37.7%) ensures the practicality of our method in drug repositioning.

The spectrum low utilization and high demand conundrum created a bottleneck towards fulfilling the requirements of next-generation networks. The cognitive radio (CR) technology was advocated as a de facto technology to alleviate the scarcity and under-utilization of spectrum resources by exploiting temporarily vacant spectrum holes of the licensed spectrum bands. As a result, the CR technology became the first step towards the intelligentization of mobile and wireless networks, and in order to strengthen its intelligent operation, the cognitive engine needs to be enhanced through the exploitation of artificial intelligence (AI) strategies. Since comprehensive literature reviews covering the integration and application of deep architectures in cognitive radio networks (CRNs) are still lacking, this article aims at filling the gap by presenting a detailed review that addresses the integration of deep architectures into the intricacies of spectrum management. This is a prospective review whose primary objective is to provide an in-depth exploration of the recent trends in AI strategies employed in mobile and wireless communication networks. The existing reviews in this area have not considered the relevance of incorporating the mathematical fundamentals of each AI strategy and how to tailor them to specific mobile and wireless networking problems. Therefore, this review addresses that problem by detailing how deep architectures can be integrated into spectrum management problems. Beyond reviewing different ways in which deep architectures can be integrated into spectrum management, model selection strategies and how different deep architectures can be tailored into the CR space to achieve better performance in complex environments are then reported in the context of future research directions.

As the traffic infrastructure for collecting vehicle tolls, the capacity of the toll plaza determines the service level of the entire highway. The capacity of a toll plaza is highly correlated with its operating costs, especially in peak periods. In this paper, it is assumed that in a competitive transportation system, the residential area and the workplace are connected by a highway with a toll plaza which forms a bottleneck, parallel to a mass transit line; commuters can choose to travel by car or by public transport. By establishing an equilibrium model and two mathematical programming models, the capacity designs of the toll plaza were studied during the traffic service period to achieve three objectives, namely, the toll plaza breaking even, profit maximization, and total social cost minimization of the transportation system. The travel modal splits were analyzed under travel equilibrium in three situations, respectively. In addition, a bi-objective optimization model was developed to optimize total profit and total social cost, and a Pareto optimization scheme was analyzed. Finally, the theoretical analyses were also verified by numerical examples.

On-device DNN processing has been common interests in the field of autonomous driving research. For better accuracy, both the number of DNN models and the model-complexity have been increased. To properly respond to this, hardware platforms structured with multicore-based CPUs and DNN accelerators have been released, and the GPU is generally used as an accelerator. When multiple DNN workloads are sporadically requested, the GPU can be easily oversubscribed, thereby leading to an unexpected performance bottleneck. We propose an on-device CPU-GPU co-scheduling framework for multi-DNN execution to remove the performance barrier precluding DNN executions from being bounded by the GPU. Our framework fills up the unused CPU cycles with DNN computations to ease the computational burden of the GPU. To provide seamless computing environment for the two different core types, the framework formats each layer execution according to the computational methods supported by CPU and GPU cores. To cope with irregular arrivals of DNN workloads, and to accommodate their fluctuating demands for hardware resources, our framework dynamically selects the best fit core type after making a comparative judgement between the current availabilities of the two core types. During the core selection time, offline-trained prediction models are utilized to get precisely predicted execution time of the issued layer. Our framework mitigates the fact that even the same DNN models can have large performance deviations due to the nature of the process scheduler of the underlying OS which is GPU-agnostic. In addition, the framework minimizes the memory copy overhead inevitably occurring in the data synchronization phase between the heterogeneous cores. To do so, we further analyze GPU-to-CPU and CPU-to-GPU data transfer cases separately, and then apply the solution that best suits each case. For multi-DNN inference jobs with the NVIDIA Jetson AGX Xavier platform, our framework speeds up the execution time by up to 46.6% over the GPU-only solution.

The Transmission Control Protocol (TCP) plays a critical role in the Internet as it is the protocol used for data transport by most Internet services and applications. With rapid advances in broadband Internet and mobile/wireless networks, current TCPs are increasingly becoming the bottleneck. This work tackles this challenge by developing a novel TCP design called Fast Launch with Agile congeStion Handling (FLASH) that not only achieves improved performance for long TCP flows but also significantly raises the performance of short to medium TCP flows that are far more common in the Internet. We evaluated its long-term and short-term performance over a wide range of network environments, using two emulation platforms (Pantheon and DummyNet) as well as Internet experiments. Compared to two of the leading TCP designs deployed in the Internet, i.e., Cubic and BBR, FLASH consistently achieved higher long-term and short-term bandwidth efficiency. For example, in trace-driven emulated experiments using Poisson traffic with a mean flow size of 1 MB operating at medium link utilization of 27%, FLASH can reduce the flow completion time (FCT) by 36% (vs. Cubic) and 26% (vs. BBR), with mean packet queueing delay of 11.7 ms compared to 3.4 ms (Cubic) and 8.8 ms (BBR). It also maintained good fairness with itself and is competitive against Cubic and BBR sharing the same bottleneck. In addition, FLASH has been tested in two real-world Internet environments. In the cloud-to-cloud experiment, it reduced FCT by 52.9% (vs. Cubic) and 46.6% (vs. BBR), while in the cloud-to-client experiment, it reduced FCT by 31.3% (vs. Cubic) and 12.7% (vs. BBR). FLASH is entirely sender-based and is compatible with current TCP receivers, thereby readily deployable in current Internet servers.

The macroscopic traffic flow model is the cornerstone of traffic flow theory and can be applied to investigate manytraffic planning and management issues. However, the macroscopic model of the heterogeneous traffic flow has not been profoundly studied. This paper developed a novel macroscopic model of the heterogeneous traffic flow under a partly connected and automated environment based on the safety potential field (SPF) theory. The SPF model was applied to describe the microscopic driving behaviour of traditional human-driven vehicles (HVs) and connected and automated vehicles (CAVs). A fundamental diagram (FD) model was derived according to the correlation with the microscopic car-following model under a steady-state condition. To verify the correlation between the FD and the microscopic simulation results, a Monte-Carlo-simulation was also performed. Finally, the macroscopic model of the heterogeneous traffic flow was validated through a simulation on a road section with a bottleneck under different CAVs market penetration rates ranging from 0.0 to 1.0. The results indicate that introducing the CAVs into the pure traffic flow of HVs may slightly have a negligible impact on road capacity when the CAVs penetration rate at a lower level, but the impact increases significantly as the CAVs penetration rate increased.

The interchange diverging area is a bottleneck section of the freeway. If the capacity of a two-lane exit cannot meet the traffic demand, it will lead to a reduction in the overall operational efficiency of the freeway. To fundamentally improve the capacity of the diverging area, designers have considered expanding the two-lane exit to a three-lane exit. However, the existing Chinese design specifications do not include a design for a three-lane exit, and the method of increasing the number of ramp lanes by setting up auxiliary lanes according to traditional design concepts leads to the mainline widening section being too long, thus increasing the construction land use and project cost. Therefore, this paper proposes an innovative three-lane exit design based on a specific two-lane exit at an interchange in Xi’an. According to the different traffic organization methods, three microscopic simulation models of diverging areas were constructed by using VISSIM. The entropy method was used to objectively calculate the weights of the four selected evaluation indexes, and a comprehensive evaluation and applicability analysis of the current situation and the three-lane exit design schemes were conducted. The results show that the optimal combination of the schemes calculated by the entropy method increases the traffic volume by up to 40% and reduces delays by 50-88%.

Credit scoring has become an important risk management tool for money lending institutions. Over the years, statistical and classical machine learning models have been the most researched risk management tools in credit scoring literature, and recently the focus has turned to deep learning models. This transition is due to better performances that are shown by deep learning models in different domains. Despite deep learning models’ superior performances, there is still a need for explaining how these models make their predictions. The non-transparency nature of deep learning models has created a bottleneck for their use in credit scoring. Explanations of decisions are important for lending institutions since it is a requirement for automated decisions that are generated by non-transparent models to be explained. The other issue in using deep learning models, specifically 2D Convolutional Neural Networks (CNNs), in credit scoring is the need to have the data in image format. We propose an explainable deep learning model for credit scoring which can harness the performance benefits offered by deep learning and yet comply with the legislation requirements for the automated decision-making processes. The proposed method converts tabular datasets into images and thus allowing the application of 2D CNNs in credit scoring. Each pixel of the image corresponds to a feature bin of the tabular dataset. The predictions from the 2D CNNs were explained using state-of-the-art explanation methods. Furthermore, explanations were evaluated using a sanity check methodology and also performances of the explanation methods were compared quantitatively. The proposed explainable deep learning model outperforms the other credit scoring methods on publicly available credit scoring datasets.

Satellite communication has attracted great interests due to extensive coverage and no terrain restriction, which can make up for the shortcomings of terrestrial communication, especially in remote areas. Hence, integrated satellite-terrestrial relay network (ISTRN) is considered as a promising research direction. However, the shortage of spectrum resources is becoming the bottleneck of the future development of ISTRN because of the low utilization of spectrum. In this regard, non-orthogonal multiple access (NOMA) scheme and cognitive radio are considered as great ways to solve this problem. This paper investigates the performance of a NOMA-based ISTRN under spectrum sharing environment with primary interference. Particularly, the exact closed-form expressions of outage probability and ergodic capacity are derived to evaluate the system performance, where the interference constraint caused by the multiple primary users and the interference of primary transmitter have been considered. Furthermore, in order to reveal the impacts of key parameters on system performance at high signal-to-noise ratios efficiently, the asymptotic analysis of OP is also provided. After that, we obtain the relationship between system performance and the power allocation factor, the number of primary users, interference parameter or the channel environment, which will enlighten the research of NOMA-based cognitive ISTRN. At last, Monte Carlo results are provided to verify the effectiveness of our theoretical analysis.

Particle filtering is very reliable in modelling non-Gaussian and non-linear elements of physical systems, which makes it ideal for tracking and localization applications. However, a major drawback of particle filters is their computational complexity, which inhibits their use in real-time applications with conventional CPU or DSP based implementation schemes. The re-sampling step in the particle filters creates a computational bottleneck since it is inherently sequential and cannot be parallelized. This paper proposes a modification to the existing particle filter algorithm, which enables parallel re-sampling and reduces the effect of the re-sampling bottleneck. We then present a high-speed and dedicated hardware architecture incorporating pipe-lining and parallelization design strategies to supplement the modified algorithm and lower the execution time considerably. From an application standpoint, we propose a novel source localization model to estimate the position of a source in a noisy environment using the particle filter algorithm implemented on hardware. The design has been prototyped using Artix-7 field-programmable gate array (FPGA), and resource utilization for the proposed system is presented. Further, we show the execution time and estimation accuracy of the high-speed architecture and observe a significant reduction in computational time. Our implementation of particle filters on FPGA is scalable and modular, with a low execution time of about $5.62~\mu \text{s}$ for processing 1024 particles (compared to 64 ms on Intel Core i7-7700 CPU with eight cores clocking at 3.60 GHz) and can be deployed for real-time applications.

Traditional tracking algorithms reply on manually extracted features while deep learning algorithms can automatically extract features, which is of great importance for single target tracking in the complex underwater environment that is characterized by poor visibility, low contrasts, and occlusion. The convolutional neural network SiamRPN++ is an advanced deep learning algorithm whose backbone network structure is. However, this algorithm is difficult to be implemented in an underwater platform due to its large amount of computation and high requirement for the computing power of the hardware. To solve this problem, a new backbone network structure NewNet-62 characterized by inverted residual bottleneck block and the SiamRPN++ algorithm are proposed. The depth-wise separable convolution in NewNet-62 simplifies the computation so that the SiamRPN++ algorithm can realize fast and accurate tracking of underwater single targets. The video set in VOT2016 was used to test the algorithm. The results show that the SiamRPN++ algorithm using NewNet-62 as the main network has the great performance. Compared with the algorithms using ResNet-50, the accuracy of the SiamRPN++ algorithm increased to 0.524 and its EAO increased to 0.303. And its tracking speed increased to 73.74. The network complexity reduced to 3.013 billion and the network parameters reduced to 12.538 million, which significantly reduces network complexity and number of network parameters.

Power loss is a bottleneck in every power system and it has been in focus of majority of the researchers and industry. This paper proposes a new method for determining the power loss in wind-solar power system based on deep learning. The main idea of the proposed scheme is to freeze the feature extraction layer of the deep Boltzmann network and deploy deep learning training model as the source model. The sample data with closer distribution with the data under consideration is selected by defining the maximum mean discrepancy contribution coefficient. The power loss calculation model is developed by configuring the deep neural network through the sample data. The deep learning model is deployed to simulate the non-linear mapping relationship between the load data, power supply data, bus voltage data and the grid loss rate during power grid operation. The proposed algorithm is applied to an actual power grid to evaluate its effectiveness. Simulation results show that the proposed algorithm effectively improved the system performance in terms of accuracy, fault tolerance, nonlinear fitting and timeliness as compared with existing schemes.

The large number of visual applications in multimedia sharing websites and social networks contribute to the increasing amounts of multimedia data in cyberspace. Video data is a rich source of information and considered the most demanding in terms of storage space. With the huge development of digital video production, video management becomes a challenging task. Video content analysis (VCA) aims to provide big data solutions by automating the video management. To this end, shot boundary detection (SBD) is considered an essential step in VCA. It aims to partition the video sequence into shots by detecting shot transitions. High computational cost in transition detection is considered a bottleneck for real-time applications. Thus, in this paper, a balance between detection accuracy and speed for SBD is addressed by presenting a new method for fast video processing. The proposed SBD framework is based on the concept of candidate segment selection with frame active area and separable moments. First, for each frame, the active area is selected such that only the informative content is considered. This leads to a reduction in the computational cost and disturbance factors. Second, for each active area, the moments are computed using orthogonal polynomials. Then, an adaptive threshold and inequality criteria are used to eliminate most of the non-transition frames and preserve candidate segments. For further elimination, two rounds of bisection comparisons are applied. As a result, the computational cost is reduced in the subsequent stages. Finally, machine learning statistics based on the support vector machine is implemented to detect the cut transitions. The enhancement of the proposed fast video processing method over existing methods in terms of computational complexity and accuracy is verified. The average improvements in terms of frame percentage and transition accuracy percentage are 1.63% and 2.05%, respectively. Moreover, for the proposed SBD algorithm, a comparative study is performed with state-of-the-art algorithms. The comparison results confirm the superiority of the proposed algorithm in computation time with improvement of over 38%.

A cluster deduplication system can coordinate the work of multiple nodes, which can better alleviate the disk index bottleneck existing in the large-scale data backup system. However, there is a problem of isolated islands of information among nodes during data deduplication. When the servers use the query mode to route data, a large amount of system overhead is required to ensure a high deduplication rate and low throughput rate. At the same time, while the servers cannot obtain a higher deduplication rate if the servers adopt the stateless routing method. Data routing strategy can greatly affect the overall performance of the system. The concept of data frequency is proposed in this paper, and the classified routing strategy is designed. In the metadata server, a byte-shaped Bloom filter for recording the occurrence frequency of data blocks is maintained to record the occurrence frequency of data blocks. The values in the Bloom filter are counted. Then the frequency of the data blocks is compared with the configured threshold value to determine whether the data is “hot data”. We use stateful routing to send “clod data” to the storage nodes and use stateless routing to send the hot data to the storage nodes. Experimental results show that the classifying routing algorithm based on the frequency of data can greatly reduce the overhead of the system while guaranteeing the deduplication rate of the deduplication system as well as improve system throughput and real-time processing capabilities. Compared with the fully stateful routing scheme, our method only loses less than 2% of the deduplication rate, which reduces the communication query overhead by more than 25% and improves the real-time processing capability of the storage system.

This paper proposes a novel and efficient real time network architecture, named RT-TelSurg, for one of the most appealing tactile Internet applications, i.e., Telesurgery. In telesurgery, the patient’s vital signs and status and the required robotic commands during the surgery should be received on time. Otherwise, the life of the patient or the safety of the operation is endangered. Hence, transmitted packets should meet their respective relative deadlines. Software-defined networking is a relatively new architecture for computer and telecommunications networks in which the network control plane is separated from the data plane. One way to achieve real time telesurgery is to employ cloud and fog networks using SDN as infrastructure. By using a real time cloud controller in the SDN as the core and a fog controller on the edge of the network on the master (physician) side, one may satisfy the acceptable level of the timing constraints of the telesurgery. Accordingly, based on the presented architecture, we develop the statistical performance model of our proposed approach, RT-TelSurg. This statistical queuing theory model is designed for critical and non-critical states of the network. Based on these states, network resources are allocated to telesurgery data in both modes, and a computational bottleneck is detected. RT-TelSurg is evaluated according to real time and surgery efficiency parameters. The results show that the average deadline hit ratio is 98.2% in different conditions, which is quite acceptable for telesurgery applications.

A Concept map is a pedagogical tool to help students understand the concepts and identify their misconceptions. Grading a concept map is a time-consuming manual task causing a severe bottleneck to use concept maps in a large class effectively. This paper presents Cronus that provides useful feedback on a student concept map similar to manual assessment by comparing it with an instructor concept map. The feedback includes identifying misconceptions, finding concepts, links, and branches that are (partially) matched or missed from a student concept map, generating summary statistics based on the feedback, and suggesting a grade of the map using predefined criteria (by the instructor) on the summary stats. Cronus is evaluated on a dataset of 74 student concept maps collected as homework assignments in an undergraduate (senior-level) course on introductory computer security. The evaluation results show that Cronus can provide accurate feedback on student concept maps compared to the manual evaluation of the maps and automatically suggest their correct grades.

Large-scale distributed deep learning is of great importance in various applications. For data-parallel distributed training systems, limited hardware resources (e.g., GPU memory and interconnection bandwidth) often become a performance bottleneck, and it is necessary to consider the full utilization of multiple resources simultaneously, especially for extreme-scale deep neural networks. Although two different types of strategies, based on memory management and sparse communication, have been proposed to reduce the usage of resources, a naïve combination of these two optimizations is impractical, since they cannot successfully coexist with each other. We therefore consider the idea of collaborative optimization in terms of both system memory and bandwidth resources, and propose a layer-centric memory-efficient distributed sparse communication mechanism called LaySA. Firstly, to tackle the memory ballooning issue caused by sparse communication, the existing memory reuse strategy is refined, and the data object of the memory optimization is augmented and redefined. Secondly, a mirror weight update mechanism is proposed to address the contradiction between memory management and sparse communication optimization for weight gradients. Our scheme, which involves the deep integration and collaborative execution of these two types of strategies, can fill the gap in relation to multiple resource optimization in distributed GPU-based training systems. Our experimental results show that the proposed collaborative optimization can significantly alleviate the memory pressure on the computing nodes, and improve both the resource utilization and the throughput of distributed training systems. Compared with baseline systems using only a single strategy, LaySA can help to reduce the system memory usage by up to 80.5%, and the overall training time of the neural network models on a single GPU is reduced by about 12.25%. Furthermore, LaySA can scale up the batch size of the datasets by an extremely large factor during distributed training, and the overall throughput is increased by more than 150%, meaning that our approach outperforms current systems that use memory or communication optimization mechanisms alone.

Detecting defects, which is a branch of target detection in the field of computer vision, is widely used in factory production. To solve the problems in existing detection algorithms that relate to their insensitivity to large or medium defect targets on bearing covers, their difficulty in detecting subtle defects effectively and their lack of real-time detection, in this work, we establish a large-scale bearing-cover defect dataset and propose an improved YOLOv3 network model. The proposed model is divided into four submodels: the bottleneck attention network (BNA-Net), the attention prediction subnet model, the defect localization subnet model, and the large-size output feature branch. To test the generality, robustness and practicability of the new model, we design a comparative experiment under abnormal illumination conditions. We design an ablation experiment to verify the validity of the proposed submodules. The experimental results show that our model solves the problem of the YOLOv3 algorithm’s insensitivity to medium or large targets and satisfies real-time detection conditions. The mAP result is 69.74%, which is 16.31%, 13.4%, 13%, 10.9%, and 7.2% more than that of YOLOv3, EfficientDet-D2, YOLOv5, YOLOv4, and PP-YOLO, respectively.

The terminal control area airspace system is one of the most complicated subsystems in the entire air traffic control system. With the continuous increase of air traffic volume, the utilization of its airspace resources has become a bottleneck restricting the sustainable development of the system. Monitor and obtain its operating status information and formulating corresponding mitigation measures are essential to improve the overall performance of the system. In order to achieve that, a new system analysis method based on clustering of trajectory characteristic point is proposed. The flight trajectories from radar position data are reconstructed at first. Followed by the method of extracting flight trajectory characteristic points including turning points, altitude transition points and speed adjustment points, all characteristic points are extracted. Then, K-means is applied for clustering according to their 3D space position, while the results are analyzed by introducing evaluation indexes based on air traffic control operation strategies. Numerical experiments show significant clustering characteristics and revealing the operation status of the terminal area airspace, which can help air traffic manager to identify defects in airspace operation and providing guidance for future airspace planning and flight procedure adjustment.

This article studies the role of architecture design, i.e. choice of the number of nodes at each hidden layer, in deep neural networks (DNNs). We give a theoretical explanation that invariance and complexity of a DNN are determined by the design of its architecture. To be more specific, for DNNs with the rectified linear activation function, we prove that the variations of gradients become the largest when the bottleneck layer, the layer with the fewest nodes, changes its activation pattern and the gap of the variations of gradients of the bottleneck layer to those of the other layers is getting larger as the differences of node counts between the bottleneck layer and the other layers become larger. That is, most large variations of the function made by a DNN are concentrated on the decision boundaries corresponding to the activation patterns of the bottleneck layer. An important implication is that invariance and complexity can be improved simultaneously, which is a key to success of DNNs for many artificial intelligence domains, by designing the highest layer as the bottleneck layer. Various experiments are carried out to validate our theoretical result and its by-products.

NDN (Named Data Networking) is one of the most popular future network architecture, a “clean slate” design for replacing the traditional TCP/IP network. However, the lookup algorithm of FIB entry in NDN is the bottleneck of the current NDN. Owing to the unique identifier of content name, whose length is variable, the size of FIB entries is proliferating, and the effectiveness of lookup algorithms is low. This paper proposed an entropy-oriented name processing mechanism, compressing the content names effectively by bringing in an encoding scheme. This mechanism can be split into two parts: name compression and lookup. The first part compressed the content names and converted them into a kind of code with a smaller size by considering the information redundancies of content names; the second part built a compact structure to minimize the memory footprint of FIB entries with keeping the high lookup performance. This mechanism outperformed many traditional name lookup algorithms, had better flexibility and cost less memory footprint.

Although achieving significant improvement on pose estimation, the major drawback is that most top-performing methods tend to adopt complex architecture and spend large computational cost to achieve higher performance. Due to the edge device’s limited resources, its top-performing methods are hard to maintain fast inference speed in practice. To address this issue, we proposed the fast and lightweight human pose estimation method to maintain high performance and bear the less computational cost. Especially, the proposed method consists of two parts, i.e., the fast and lightweight pose network (FLPN) for pose estimation and a novel lightweight bottleneck block for reducing computational cost, which can integrate the simple network and lightweight bottleneck into an efficient method for accurate pose estimation. In terms of lightweight bottleneck block, we introduce the structural similarity measurement (SSIM) to refine the appropriate ratio of intrinsic feature maps and reduce the model size. Furthermore, an attention mechanism is also adopted in our lightweight bottleneck block for modeling the contextual information. We demonstrate the performance of the proposed method with extensive experiments on the two standard benchmark datasets by comparing our method with state-of-the-art methods. On the COCO keypoint detection dataset, our proposed method attains a similar accuracy with these state-of-the-art methods, but the computational cost of these top-performing methods is more than 7 times that of ours.

In recent years, reinforcement learning (RL) has been widely used to solve multi-agent navigation tasks, and a high-fidelity level for the simulator is critical to narrow the gap between simulation and real-world tasks. However, high-fidelity simulators have high sampling costs and bottleneck the training model-free RL algorithms. Hence, we propose a Multi-Fidelity Simulator framework to train Multi-Agent Reinforcement Learning (MFS-MARL), reducing the total data cost with samples generated by a low-fidelity simulator. We apply the depth-first search to obtain local feasible policies on the low-fidelity simulator as expert policies to help the original reinforcement learning algorithm explore. We built a multi-vehicle simulator with variable fidelity levels to test the proposed method and compared it with the vanilla Soft Actor-Critic (SAC) and expert actor methods. The results show that our method can effectively obtain local feasible policies and can achieve a 23% cost reduction in multi-agent navigation tasks.

Certified mail is a special postal service that provides the sender with a proof that the mail was successfully received by the other party. However, when it comes to certified electronic mail (CEM), there has not been a widely accepted protocol yet. Most of the existing solutions rely on a Trusted Third Party (TTP) in order to achieve fairness, which can become a bottleneck and lead to more charges and computations. Other solutions that do not rely on a TTP have a relatively high computation and communication overhead. Blockchains have allowed the creation of TTP-free protocols for fair exchange; however, the existing Blockchain-based protocols are either too expensive or do not meet the necessary fairness and confidentiality guarantees. In this work, we have developed two TTP-free Blockchain-based protocols for certified electronic mail that achieve the required security properties of strong fairness, non-repudiation, timeliness, and confidentiality. The first protocol uses simple bitcoin transactions for a single receiver, whereas the second one implements smart contracts for one or multiple receivers. We have analyzed and compared both protocols to existing TTP-free Blockchain solutions and have demonstrated their superior properties and cost. Furthermore, we have implemented the two protocols in a realistic testbed, demonstrating the viability of our approach.

Automatic post editing (APE) researches aim to correct errors in the machine translation results. Recently, APE research has mainly been conducted in two directions: noise-based APE and adapter-based APE. This study poses three questions based on existing APE studies and conducted a verification. The first is a question about the optimal APE research direction, and this has been figured out through a comparative analysis of the previous studies on noise-based APE and adapter-based APE. The second is about the substantial effectiveness of the bottleneck adapter layer (BAL) in adapter based APE. For the verification, various experiments on the different size of BAL has been conducted, and through these experiments, optimal approaches in adapter based APE has been proposed. For the last, this work raises a question about the reason why leveraging external knowledge is influential in APE. In this regard, we conducted several comparative experiments on the method of utilizing external data to APE training to achieve a better performance. The results revealed that the performance can be improved by applying the method of concatenating the external data with the existing data when training the APE model.Through deep analysis on these experiments, this work propose the optimal research direction in APE.

In this work a self-tuning rate and queue based proportional and integral controller called SRQ-PI is proposed to efficiently control the queue length with small overshoot and faster settling time. SRQ-PI proposes a new control tracking function that maps level of congestion to the packet drop probability dynamically. In SRQ-PI, the incoming traffic rate is estimated and used with the proportional and integral controller. The SRQ-PI tunes itself and stabilizes the system with internal feedback without requiring any external feedback. Furthermore, the stability of the SRQ-PI is analyzed using control theory and presents systematic guidelines to select the control gain parameters. NS2 is used to carry out the simulation work. The simulation result demonstrates that SRQ-PI is stable and gets faster transient response due to lower average delay jitter and robust against dynamic network parameters. The SRQ-PI outperforms proportional integral (PI), Intelligent adaptive PI (IAPI) and Random exponential marking (REM) algorithm.

A fundamental bottleneck in text-to-image synthesis is that there are rarely subjective quality evaluation metrics for a single generated image. To address this issue, this paper proposed a procedure to evaluate the single generated image, which includes a specific dataset named multiple metrics quality assessment for birds(MMQA Birds) and a learning model named blind generated image evaluator(BGIE). The motivation of our proposal is twofold. On the one hand, subjective image quality evaluation is a human perceptual task; Therefore, it tends to be a process of supervised learning. To the best of our knowledge, there are not any datasets for this study. Thus, we handle this problem via designing a specific dataset. On the other hand, we observed that the spatial content of generated image attracts more attention when humans judge its quality; According to this finding, an efficient machine-learning model that combines both pixel-level features and spatial features is proposed. Extensive experiments manifest our method can solve this problem to some extent. In the generated image dataset, BGIE surpasses the state-of-art NSS-based method by 6.3% in PLCC and SRCC. In practice, we further discuss the rationality of the MMQA Birds dataset and the application of BGIE. It proves that both in subjective and objective aspects, our method achieves convincing results.

Intelligent and Connected Vehicle (ICV) technology is considered to be a solution to improve the traffic performance. Through the information exchange in real-time among the vehicles, the roadside infrastructures, and the cloud platform, the sensing of the vehicles can be enhanced. This also enables coordinated driving decisions, which can improve traffic operations, especially at bottleneck locations. This paper addresses the problem of coordinating the vehicles near the bottleneck locations to help the vehicles passing the area quickly and smoothly. A lane advisory algorithm is designed to reduce conflicts by encouraging early lane changes. A coordinated vehicle movement planning algorithm is proposed to achieve a smooth longitudinal reference speed profiles for vehicles in the subject area. The algorithm can open enough headway for vehicles to change the lane and continue their trips. The effectiveness of the algorithm is evaluated using SUMO (Simulation of Urban MObility) as the simulation tool with no communication between vehicles as the benchmark case as well as the case where the vehicular traffic follows the so-called First-in-First-Out (FIFO) principle. The results of the evaluation summarize and indicate that the Coordinated Control Algorithm (CCA) proposed in this paper can improve traffic performance in terms of the average speed, the waiting time, the total travel time, and the traffic flow rate under different levels of service.

Wireless Sensor Networks (WSNs) have become extremely popular for sensing, collecting, and transmitting data across different environments. In particular, the AODV protocol is widely used to improve the behavior of WSNs in various applications. A bottleneck in the protocol's performance is the amount of data that need to be moved between different nodes. This bottleneck becomes evident in applications based on multimedia contents, such as images or videos, in which huge chunks of data need to be delivered over long distances. In this article, we propose a new method to enhance the performance of the AODV protocol. Simulation results show that the proposed method improves the performance of the AODV protocol for image-based applications. The technique increases the quality of the delivered images, extends the network's lifetime, and reduces the delay and the network overhead associated with providing such images.

Autoencoders implemented by artificial neural networks (ANNs) are utilized to learn the latent space representation of data in an unsupervised manner, and they have been widely used in recommender systems. For instance, several collaborative denoising autoencoder (CDAE) models have shown that their performance gains outperform that of the collaborative filtering based (CF-based) models. In this work, a near-optimal Top- $K$ forecasting solution is proposed for our advanced autoencoder recommender systems. We propose a method which utilizes CDAE model in predicting the Top- $K$ popular videos in an upcoming time period. In order to improve the prediction accuracy, we also propose an autoencoder based recommendation algorithm with the help of $K$ -means clustering that upgrades the performance of the original autoencoder model. The experimental results show that our method increases significantly the Average Precision (AP) and Recall values by nearly 30%. We then further utilize our proposed autoencoder model with clustering in predicting Top- $K$ popular videos. The applications of predicting Top- $K$ popular videos can be used in the video delivery for the Mobile Edge Computing (MEC) environment to avoid bottleneck in the constricted capacity of backhaul link. Namely, the performance gain will be upgraded if our proposed method precisely predicts and caches the Top- $K$ popular videos in advance with the help of a better forecasting model.

Pilot contamination (PC) is recognized as a bottleneck for the achievable throughput of multi-cell massive multiple-input multiple-output (MIMO) systems. In this paper, we propose a joint pilot allocation and pilot sequences optimization (JPA-PSO) scheme to mitigate the effects of PC and maximize the system spectral efficiency (SE). Specifically, we first construct a range of pilot sequence in line with the coherent interval. Then, we establish an allocation basis by defining a similarity to distinguish PC among users. An angle of arrival (AOA) positioning method is designed to obtain the user's location information. Under the location information, we can monitor and estimate the distance between users in different cells to denote the potential interference of transmit power among users. Then, we execute pilot allocation strategy in line with the similarity and distance between users in different cells, that is, assign orthogonal pilots to the users with greater similarity and shorter distance. Finally, we provide a pilot sequence optimization algorithm to maximize the system SE. Simulation results verify that combining pilot allocation and pilot sequence optimization can significantly improve the system SE.

In 5G-and-beyond wireless communication systems, Network Function Virtualization (NFV) has been widely acknowledged as an important network architecture solution to meet diverse service requirements in various scenarios. However, with the increase of network functions, the introduction of NFV may significantly increase the delay of traffic flows, which is much undesired, especially for Ultra Reliable and Low Latency Communication (URLLC) service. Network Function Parallelism (NFP) architecture has been recently proposed as an effective technique to address the bottleneck of NFV technology. NFP can potentially improve the reliability and reduce the delay of service function chains (SFCs). In this paper, we propose a learning based SFC deployment strategy under NFP architecture with aim to improve the service reliability while reducing the end-to-end service delay. Specifically, service reliability is improved by deploying back-up virtual network function (VNF) nodes, while the flow delay is reduced via parallel network function processing. We formulate the VNF deployment as an integer-programming problem with objective of minimizing the reserved computing and bandwidth resources, while guaranteeing the service reliability and end-to-end delay. Considering the hardness and properties of the problem, we transform it as a Markov Decision Process (MDP), and employ a reinforcement-learning algorithm to solve it. We conduct simulations and the numerical results demonstrate that the proposed strategy can significantly improve the service reliability and delay performance, which are crucial for URLLC service.

With the popularization and development of the IoT(Internet of Things), more and more data needs to be transmitted over the Internet, which leads to the deterioration of network quality. The CDN (Content Distribution Network) technology is an important theoretical model to ensure network QoS (Quality of Service). To improve the QoS, we extend current CDN system to the hierarchical CDN system, that traditional CDN system is just a special case of hierarchical CDN system. In the traditional CDN system and the hierarchical CDN system, by analyzing the bandwidth between subsystems, we found the inter-system bandwidth is the bottleneck that impedes CDN system expansion. To address this problem, a new kind of distributed system architecture is proposed in this paper. This new architecture uses broadcast channels to distribute broadcast type data and still using bidirection uni-cast channels for other type of data, so we call the new architecture as CHCDN (Channel Heterogeneous CDN) in this paper. The new architecture is analyzed and compared with the traditional CDN system architecture and hierarchical CDN system architecture. Moreover, the experimental simulation result has shown that the CHCDN system features better in real-time cache updating and features with higher data transmission efficiency than the hierarchical CDN system, which indicating that the new architecture has great potential for being widely used in distributed computing.

Despite the promising progress made in recent years, person re-identification (Re-ID) remains a challenging task due to the intra-class variations. Most of the current studies used the traditional Softmax loss for solutions, but its discriminative capability encounters a bottleneck. Therefore, how to improve person Re-ID performance is still a challenging task. To address this problem, we proposed a novel loss function, namely additive distance constraint with similar labels loss (ADCSLL). Specifically, we reformulated the Softmax loss by adding a distance constraint to the ground truth label, based on which similar labels were introduced to enhance the learned features to be much more stable and centralized. Experimental evaluations were conducted on two popular datasets (Market-1501 and DukeMTMC-reID) to examine the effectiveness of our proposed method. The results showed that our proposed ADCSLL was more discriminative than most of the other compared state-of-the-art methods. The rank-1 accuracy and the mAP on Market-1501 were 95.0% and 87.0%, respectively. The numbers were 88.6% and 77.2% on DukeMTMC-reID, respectively.

Most CNN-based super-resolution networks require a large number of samples for model training, which may cause overfitting when trained on a specific set, and the internal self-similarity of the test image in this way has also been discarded. To resolve this problem, this paper proposes a novel zero-shot learning (ZSL) network for super resolution. In this model, the training samples are cropped from the input image, which means no extra dataset is required for model training, the overfitting problem in this way can be well avoided. Besides, since all the training samples are cropped from the input itself, the nonlocal self-similarity attributes of the test image can be fully utilized. Moreover, the efficient spatial pyramid of dilated convolutions network with bottleneck (ESP-BNet) is applied in the model as an efficient computational structure to enhance the feature representation. Comparison experiments show that the proposed approach achieves PSNR/SSIM of 31.13dB/87.9% on Set5 dataset and PSNR/SSIM of 27.62dB/78.1% on Set14 dataset, which is around 3dB better than the traditional method and about 1dB higher than those of the state-of-art method.

The low computational capacity of mobile devices has become the main performance bottleneck for emerging computing-intensive and delay-sensitive applications. In this paper, we propose a peer-to-peer (P2P) enhanced task scheduling framework to minimize the average task duration in device-to-device (D2D) enabled mobile edge computing (MEC) network by jointly optimizing the task scheduling decision and the computational resource allocation. Our proposed framework can work in different modes in different application scenarios. In the MEC mode, an iterative algorithm is presented based on alternating optimization technology and sorting algorithm, which can derive a near-optimal solution of the original mixed-integer non-linear programming (MINLP) optimization problem. In the P2P mode, we can reformulate the original problem to a typical one-to-one assignment problem, which is solved by a task scheduling algorithm based on the Hungarian method. In more challenging hybrid mode, we make some simplifications based on reformulation linearization technology. Then, the near-optimal solution is obtained by our proposed scheduling algorithm based on the alternating direction method of multipliers (ADMM) method and difference of convex functions (D.C.) programming. Numerical results show that our proposed framework has shorter average task duration under different scheduling modes compared with the other scheduling schemes.

The development of wireless charging technology has made research in rechargeable wireless sensing networks (WRSNs) a hot topic. Charge scheduling, which involves allocating mobile charger(s) to replenish energy to the sensors of a network so that the network can remain active for a longer time, is one of the most essential issues in this field. Most previous works addressed the charge-scheduling problem by using proactive or on-demand methods, each with its own advantages and issues. In this work, we propose a novel bottleneck prediction and removal mechanism (BP&R) that seamlessly integrates the two completely different scheduling strategies. The BP&R predicts proactively the potential charging needs that may overwhelm the on-demanding scheduling and eliminate those incidents beforehand. This allows the on-demand scheduling, which usually relies on time and/or distance factors in determining charging order, to concentrate on tackling the problem considering only the distance factor. A simple distance-based near job first (NJF) algorithm is thus sufficient for charge scheduling when time-related factors are considered prior to the determination of the charging order. Use of NJF algorithm also allow us derive a shorter path length in charge scheduling, which results in a tighter charging threshold. A lower charging threshold corresponds to fewer charging requests and less pressure on charge scheduling. In general, the new scheduling strategy relies on the on-demand feature to gather the charging demands of the network, while proactively includes additional sensor nodes that could potentially cause subsequent issues in the scheduling plan. The on-demand and proactive natures of the proposed strategy ensure that it can adapt to the dynamic charging needs of sensors, while at the same time taking precautionary measures to avoid possible burst requests in the future. Extensive simulations show that the proposed strategy can achieve a better network lifetime than other state-of-art scheduling methods.

With the development of artificial intelligence and integrated sensor technologies, unmanned aerial vehicles (UAVs) are more and more applied in the air combats. A bottleneck that constrains the capability of UAVs against manned vehicles is the autonomous maneuver decision, which is a very challenging problem in the short-range air combat undergoing highly dynamic and uncertain maneuvers of enemies. In this paper, an autonomous maneuver decision model is proposed for the UAV short-range air combat based on reinforcement learning, which mainly includes the aircraft motion model, one-to-one short-range air combat evaluation model and the maneuver decision model based on deep Q network (DQN). However, such model includes a high dimensional state and action space which requires huge computation load for DQN training using traditional methods. Then, a phased training method, called “basic-confrontation”, which is based on the idea that human beings gradually learn from simple to complex is proposed to help reduce the training time while getting suboptimal but efficient results. Finally, one-to-one short-range air combats are simulated under different target maneuver policies. Simulation results show that the proposed maneuver decision model and training method can help the UAV achieve autonomous decision in the air combats and obtain an effective decision policy to defeat the opponent.

In wireless sensor networks (WSNs), target coverage is an important issue which aims at finding a set of sensors to monitor the targets for maximizing both the surveillance quality and network lifetime. However, most of them assumed that each sensor is battery powered and the Boolean Sensing Model (BSM) is applied. Sensors powered by battery have a limited lifetime while the BSM is difficult to reflect the physical features of sensing. This paper proposes target coverage mechanisms, called C-MMQT and D-MMQT , which consider the solar-powered sensors and allows the battery to be recharged for maintaining the perpetual lifetime of sensor networks. The proposed mechanisms apply the Probabilistic Sensing Model (PSM) and consider that different targets have different importance. Two challenges have been overcome in this paper. First, each sensor is well scheduled for switching between recharging and working states for maintaining its perpetual lifetime since its battery is solar powered. Second, the sensors that are able to monitor the common target are well scheduled for cooperative sensing to maximize the surveillance quality since PSM is applied. Two efficient sensor schedules are proposed to maximize the surveillance quality of the bottleneck target which has the lowest surveillance quality. Performance study shows that the proposed mechanisms outperform the existing mechanisms in terms of Quality of Monitoring, Average utility, Fairness and Efficiency index.