Search

The ever-growing parameter size and computation cost of Convolutional Neural Network (CNN) models hinder their deployment onto resource-constrained platforms. Network pruning techniques are proposed to remove the redundancy in CNN parameters and produce a sparse model. Sparse-aware accelerators are also proposed to reduce the computation cost and memory bandwidth requirements of inference by leveraging the model sparsity. The irregularity of sparse patterns, however, limits the efficiency of those designs. Researchers proposed to address this issue by creating a regular sparsity pattern through hardware-aware pruning algorithms. However, the pruning rate of these solutions is largely limited by the enforced sparsity patterns. This limitation motivates us to explore other compression methods beyond pruning. With two decoupled computation stages, we found that kernel decomposition could potentially take the processing of the sparse pattern off from the critical path of inference and achieve a high compression ratio without enforcing the sparse patterns. To exploit these advantages, we propose ESCALATE, an algorithm-hardware co-design approach based on kernel decomposition. At algorithm level, ESCALATE reorganizes the two computation stages of the decomposed convolution to enable a stream processing of the intermediate feature map. We proposed a hybrid quantization to exploit the different reuse frequency of each part of the decomposed weight. At architecture level, ESCALATE proposes a novel ‘Basis-First’ dataflow and its corresponding microarchitecture design to maximize the benefits brought by the decomposed convolution. We evaluate ESCALATE with four representative CNN models on both CIFAR-10 and ImageNet datasets and compare it against previous sparse accelerators and pruning algorithms. Results show that ESCALATE can achieve up to 325 × and 11 × compression ratio for models on CIFAR-10 and ImageNet, respectively. Comparing with previous dense and sparse accelerators, ESCALATE accelerator averagely boosts the energy efficiency by 8.3 × and 3.77 ×, and reduces the latency by 17.9 × and 2.16 ×, respectively.

Contemporary mobile devices with touchscreens capture the X/Y position of finger tips on the screen and pass these coordinates to applications as though the input were points in space. Of course, human hands are much more sophisticated, able to form rich 3D poses capable of far more complex interactions than poking at a screen. In this paper, we describe how conventional capacitive touchscreens can be used to estimate 3D hand pose, enabling richer interaction opportunities. Importantly, our software-only approach requires no special or new sensors, either internal or external. As a proof of concept, we use an off-the-shelf Samsung Tablet flashed with a custom kernel. After describing our software pipeline, we report findings from our user study, we conclude with several example applications we built to illustrate the potential of our approach.

Radar signal-based object detection has become a primary and critical issue for autonomous driving recently. Recently advanced radar object detectors indicated that the cross-model supervision-based approach presented a promising performance based on 3D hourglass convolutional networks. However, the trade-off between computational efficiency and performance of radar object detection tasks is rarely investigated. When higher performance is required in the detection tasks, the 3D convolutional backbone network rarely meets the real-time applications. This paper proposes a lightweight, computationally efficient, and effective network architecture to conquer this issue. First, Atrous convolution, as well-known as dilated convolution, is adopted in our backbone network to make a smaller convolutional kernel having a larger receptive field so as a larger convolutional kernel can be eliminated to reduce the number of parameters. Furthermore, a densely connected residual block (DCSB) is proposed to better deliver the gradient flow from the loss function to improve the feature representation ability. Finally, the hourglass network structure is made by stacking several DCSBs with Mish activation function to form our detection network, termed as DCSN. In this manner, we can keep a larger receptive field and reduce the number of parameters significantly, resulting in an efficient radar object detector. Experiments are demonstrated that the proposed DCSN achieves a significant improvement of inference time and computational complexity, with comparable performance for radar object detection. The source code can be found in https://github.com/jesse1029/RADER-DCSN.

This paper shares our experience in supporting and running the Open vSwitch (OVS) software switch, as part of the NSX product for enterprise data center virtualization used by thousands of VMware customers. Starting in 2009, the OVS design split its code between tightly coupled kernel and userspace components. This split was necessary at the time for performance, but it caused maintainability problems that persist today. In addition, in-kernel packet processing is now much slower than newer options. To solve the problems caused by the user/kernel split, OVS must adopt a new architecture. We describe two possibilities that we explored, but did not adopt, one because it gives up compatibility with drivers and tools that are important to virtual data center operators, the other because it performs poorly. Instead, we endorse a third approach, based on a new Linux socket type called AF\_XDP, which solves the maintainability problem in a compatible, performant way. The new code is already merged into the mainstream OVS repository. We include a thorough performance evaluation and a collection of lessons learned.

Modern living demands are highly dependent on science and technology, and people can apply existing materials and AI technology to make the robots present different interactive modes. The purposes of the application in our life are distinguished based on appearance, complexity, and category of robots. People use their creativity to introduce interactive modes to robots resulting in the function of human-computer coexistence and co-education. Robots are designed with various movements through the programmable kernel and JIMU interactive robots can be connected to smart phones. Therefore, this study aims at remotely controlling the robot's movements by controlling the angular displacement and velocity mechanisms of the steering engine and other mechanisms through the design of the artificial intelligent codes. Remotely controlling JIMU robots will help maintain their balance like humans and can act normally.

Extracting effective feature information is very important for deep learning methods. However, most deep learning-based CS-MRI methods use the standard convolutional layers with square kernels for feature extraction, which is difficult to further improve the accuracy of MRI reconstruction under limited computational resources. In this paper, we propose the enhanced asymmetric convolution blocks (EACBs) and the selective asymmetric kernel blocks (SAKBs) to effectively enhance the feature extraction ability of the network. Further, we propose a novel adaptive enhanced generative adversarial network, termed as AEGAN, for high-precision MRI reconstruction, where the proposed EACBs and SAKBs are embedded in the network architecture of AEGAN. In the AEGAN, the proposed EACBs are used to enhance the corresponding skeleton of the square kernel, and the SAKBs are used to efficiently capture useful feature information by adaptively adjusting the size of receptive fields. Therefore, the combination of EACBs and SAKBs can extract rich feature information more effectively and efficiently. In further experiments, it can be demonstrated that the proposed AEGAN exceeds the state-of-the-art GAN-based CS-MRI methods with fewer model parameters.

With the growth of mobile vision applications, there is a growing need to break through the current performance limitation of mobile platforms, especially for computationally intensive applications, such as object detection, action recognition, and medical diagnosis. To achieve this goal, we present our unified real-time mobile DNN inference acceleration framework, seamlessly integrating hardware-friendly, structured model compression with mobile-targeted compiler optimizations. We aim at an unprecedented, real-time performance of such large-scale neural network inference on mobile devices. A fine-grained block-based pruning scheme is proposed to be universally applicable to all types of DNN layers, such as convolutional layers with different kernel sizes and fully connected layers. Moreover, it is also successfully extended to 3D convolutions. With the assist of our compiler optimizations, the fine-grained block-based sparsity is fully utilized to achieve high model accuracy and high hardware acceleration simultaneously. To validate our framework, three representative fields of applications are implemented and demonstrated, object detection, activity detection, and medical diagnosis. All applications achieve real-time inference using an off-the-shelf smartphone, outperforming the representative mobile DNN inference acceleration frameworks by up to 6.7× in speed. The demonstrations of these applications can be found in the following link: https://bit.ly/39lWpYu.

While computer vision tasks target increasingly challenging scenarios, the need for real-time processing of images rises as well, requiring more efficient methods to accelerate convolutional neural networks. For unit stride convolutions, we use FFT-based methods and Winograd algorithms to compute matrix convolutions, which effectively lower the computing complexity by reducing the number of multiplications. For non-unit stride convolutions, we usually cannot directly apply those algorithms to accelerate the computations. In this work, we propose a novel universal approach to construct the non-unit stride convolution algorithms for any given stride and filter sizes from Winograd algorithms. Specifically, we first demonstrate the steps to decompose an arbitrary convolutional kernel and apply the Winograd algorithms separately to compute non-unit stride convolutions. We then present the derivation of this method and proof by construction to confirm the validity of this approach. Finally, we discuss the minimum number of multiplications and additions necessary for the non-unit stride convolutions and evaluate the performance of the decomposed Winograd algorithms. From our analysis of the computational complexity, the new approach can benefit from 1.5x to 3x fewer multiplications. In our experiments in real DNN layers, we have acquired around 1.3x speedup (T old / T new ) of the Winograd algorithms against the conventional convolution algorithm in various experiment settings.

As a pioneer work in the field of Image Quality Assessment (IQA), Structure Similarity (SSIM) has been widely used to evaluate the output quality of imaging systems and algorithms. However, the classical implementation of SSIM is inherent of isotropic nature and thus fail to model the anisotropic characteristic of Human Vision System (HVS). To alleviate this problem and realize as fully as possible the potential of SSIM, an anisotropic implementation is put forward in this letter in which a kernel adaptive to image local structures is integrated in SSIM computation. Extensive Numerical experiments reveal that this seemingly tiny modification (from isotropic to anisotropic) leads to significant performance improvement, and the anisotropic implementation of SSIM performs comparably with state-of-the-art on several benchmark datasets.

For an ultrasound blurry image with more dark pixels and the blurry ultrasound image is not sparse enough after the bright channel prior deblurring. In order to make more effective use of the image information and enhance the sparsity of the ultrasound images obtained after deblurring, a method of blind ultrasound image deblurring based on quadratic sparse bright channel prior. Firstly, a quadratic sparse bright channel is proposed based on the bright channel. Secondly, the prior information of the image is obtained by using the quadratic sparse bright channel. Then, the clear image and blur kernel are estimated by half-quadratic splitting. Finally, the deblurring experiments are carried out on the ultrasound image set, and compared with other deblurring algorithms. Experimental results show that this algorithm has higher feasibility and superiority.

End-to-end methods based on deep learning have gradually shown their convenience and advantages in non-uniform motion deblur. However, current deep models for deblurring still have some problems, which ignore the non-uniform deformation of motion blur kernel or global information, causing inadequately deblurring or artifacts. In this paper, we propose an enhanced network based on the four-layer multi-path hierarchy structure which divides image into multi-patches instead of down-sampling or other lossy way. Our network is combined with the self-attention module, and replace part of ordinary convolutions with deformable convolutions in the encoder-decoder backbone. For fewer artifacts and more deep feature details, perceptual loss is also applied in the training process. Compared with the results of prior frameworks, our method has improved nearly 0.4dB PSNR on the quantitative performance on GoPro testing data set, and also achieved clearer visual effect on several scenes.

With the development of artificial intelligence technology, the problems of object tracking are getting more and more attention. This article introduces the popular object tracking algorithms, from common problems in object tracking to the classification of algorithms: Early classic tracking algorithms, tracking algorithms based on kernel correlation filtering, and tracking algorithms based on deep learning. Then the original version and various improved versions of each type of tracking algorithm are introduced, analyzed, and compared. Finally, we use the OTB-2013 data set to test the above 50 object tracking algorithms. The simulation experiment obtained the accuracy figure and success figure of the overall performance comparison of these 50 object tracking algorithms and then selected 11 attributes to compare the performance of various algorithms. The 11 attributes are illumination variation, out-of-plane rotation, scale variation, occlusion, deformation, motion blur, fast movement, in-plane rotation, out of view, background clutter, and low resolution. The accuracy figures and success figures of these 50 algorithms are obtained respectively under the different conditions. The experiment mainly analyzes the results through three indicators of accuracy, success rate, and tracking speed of various algorithms, and draws the following conclusions: Compared with traditional algorithms, the tracking speed of correlation filtering algorithms can reach more than 100 frames per second, while the precision of deep learning methods can reach more than 0.7. The algorithms using convolution features and multi-features fusion algorithms have more advantages in tracking accuracy than the algorithm using a single feature, but the tracking speed will also drop rapidly. Using a powerful classifier is the basis of achieving good tracking. Tracker with scale estimation module can get better performance of success figures. The updating mechanisms of the model also affect the tracking accuracy.

Aiming at the problem that the KCF algorithm cannot adaptively track the object and even the object may be lost when the object scale changes, this paper proposes a kernel correlation filter algorithm that can perform adaptive object detection. The KCF algorithm is adopted to calculate the center position of the object, and the object is divided into sub-blocks for applying the KCF classifier idea to each sub-block at the same time. Then the center position of each sub-block is calculated for estimating the scale change ratio of the object according to the relative change of the center position of the sub-block. And the scale information of the object is finally determined for following tracking. On this basis, the HOG feature and the CN feature are combined to make full use of the image pixel information to promote accuracy of the object prediction. Finally, public dataset is used for comparing the proposed method with multiple classic algorithms. The experimental result illustrate that the proposed method can obtain better accuracy and overlapping ratio, and the processing speed can meet the real-time requirements, which verifies the effectiveness of the proposed algorithm.

Software clone detection is an active research area, which is very important for software maintenance, bug detection, etc. The two pieces of cloned code reflect some similarities or equivalents in the syntax or structure of the code representations. There are many representations of code like AST, token, PDG, etc. The PDG (Program Dependency Graph) of source code can contain both syntactic and structural information. However, most existing PDG-based tools are quite time-consuming and miss many clones because they detect code clones with exact graph matching by using subgraph isomorphism. In this paper, we propose a novel PDG-based code clone detector, CCGraph, that uses graph kernels. Firstly, we normalize the structure of PDGs and design a two-stage filtering strategy by measuring the characteristic vectors of codes. Then we detect the code clones by using an approximate graph matching algorithm based on the reforming WL (Weisfeiler-Lehman) graph kernel. Experiment results show that CCGraph retains a high accuracy, has both better recall and F1-score values, and detects more semantic clones than other two related state-of-the-art tools. Besides, CCGraph is much more efficient than the existing PDG-based tools.

While distributed application-layer tracing is widely used for performance diagnosis in microservices, its coarse granularity at the service level limits its applicability towards detecting more fine-grained system level issues. To address this problem, cross-layer stitching of tracing information has been proposed. However, all existing cross-layer stitching approaches either require modification of the kernel or need updates in the application-layer tracing library to propagate stitching information, both of which add further complex modifications to existing tracing tools. This paper introduces Deepstitch, a deep learning based approach to stitch cross-layer tracing information without requiring any changes to existing application layer tracing tools. Deepstitch leverages a global view of a distributed application composed of multiple services and learns the global system call sequences across all services involved. This knowledge is then used to stitch system call sequences with service-level traces obtained from a deployed application. Our proof of concept experiments show that the proposed approach successfully maps application-level interaction into the system call sequences and can identify thread-level interactions.

The tremendous growth in deep learning (DL) applications has created an exponential demand for computing power, which leads to the rise of AI-specific hardware. Targeted towards accelerating computation-intensive deep learning applications, AI hardware, including but not limited to GPGPU, TPU, ASICs, etc., have been adopted ubiquitously. As a result, domain-specific CAD tools play more and more important roles and have been deeply involved in both the design and compilation stages of modern AI hardware. Recently, ISPD 2020 contest introduced a special challenge targeting at the physical mapping of neural network workloads onto the largest commercial deep learning accelerator, CS-1 Wafer-Scale Engine (WSE). In this paper, we proposed CU.POKer, a high-performance engine fully-customized for WSE's DNN workload placement challenge. A provably optimal placeable kernel candidate searching scheme and a data-flow-aware placement tool are developed accordingly to ensure the state-of-the-art quality on the real industrial benchmarks. Experimental results on ISPD 2020 contest evaluation suites [1] demonstrated the superiority of our proposed framework over other contestants.

Open-source kernels have been adopted by massive downstream vendors on billions of devices. However, these vendors often omit or delay the adoption of patches released in the mainstream version. Even worse, many vendors are not publicizing the patching progress or even disclosing misleading information. However, patching status is critical for groups (e.g., governments and enterprise users) that are keen to security threats. Such a practice motivates the need for reliable patch presence testing for downstream kernels. Currently, the best means of patch presence testing is to examine the existence of a patch in the target kernel by using the code signature match. However, such an approach cannot address the key challenges in practice. Specifically, downstream vendors widely customize the mainstream code and use non-standard building configurations, which often change the code around the patching sites such that the code signatures are ineffective. In this work, we propose PDiff, a system to perform highly reliable patch presence testing with downstream kernel images. Technically speaking, PDiff generates summaries carrying the semantics related to a target patch. Based on the semantic summaries, PDiff compares the target kernel with its mainstream version before and after the adoption of the patch, preferring the closer reference version to determine the patching status. Unlike previous research on patch presence testing, our approach examines similarity based on the semantics of patches and therefore, provides high tolerance to code-level variations. Our test with 398 kernel images corresponding to 51 patches shows that PDiff can achieve high accuracy with an extremely low rate of false negatives and zero false positives. This significantly outperforms the state-of-the-art tool. More importantly, PDiff demonstrates consistently high effectiveness when code customization and non-standard building configurations occur.

The large amount of data captured by ambulatory sensing devices can afford us insights into longitudinal behavioral patterns, which can be linked to emotional, psychological, and cognitive outcomes. Yet, the sensitivity of behavioral data, which regularly involve speech signals and facial images, can cause strong privacy concerns, such as the leaking of the user identity. We examine the interplay between emotion-specific and user identity-specific information in image-based emotion recognition systems. We further study a user anonymization approach that preserves emotion-specific information, but eliminates user-dependent information from the convolutional kernel of convolutional neural networks (CNN), therefore reducing user re-identification risks. We formulate an adversarial learning problem implemented with a multitask CNN, that minimizes emotion classification and maximizes user identification loss. The proposed system is evaluated on three datasets achieving moderate to high emotion recognition and poor user identity recognition performance. The resulting image transformation obtained by the convolutional layer is visually inspected, attesting to the efficacy of the proposed system in preserving emotion-specific information. Implications from this study can inform the design of privacy-aware emotion recognition systems that preserve facets of human behavior, while concealing the identity of the user, and can be used in ambulatory monitoring applications related to health, well-being, and education.

In this paper, the problem of person-independent facial expression recognition (FER) is addressed using the 3D geometry information extracted from the 3D shape of the face. The 3D data can provide more information and more robust features under some conditions than the 2D data. State-of-the-art 3D FER methods are often based on a single descriptor which may fail to handle the large inter-class and intra-class variability of the human facial expressions. For this reason, we propose to explore the usage of covariance matrices of features, instead of directly using the local features for the 3D FER task. Since covariance matrices are elements of the non-linear manifold of Symmetric Positive Definite (SPD) matrices, we propose, in this paper, a manifold-based classification using a Graph-Matching Kernel method. The proposed Kernel is appropriate for SVM-based image classification, and thus is suitable for FER application. We evaluate the performance of the proposed method on the BU-3DFE and the Bosphorus datasets, and demonstrate its superiority compared to the state-of-the-art methods.

Face detection is widely used in various image and video processing technologies. The detection of face direction is also one of the functions of face detection, but it is not possible to obtain an effective face direction under poor data labeling conditions. This paper proposes an improved global optimal particle swarm optimization method, in which PSO completes the global optimization of SVM parameters during the training phase. The improved PSO-SVM is mainly adjusted for the regularization parameters and kernel parameters of the SVM, and the goal is the global optimal parameter combination. The experimental results show that the improved PSO-SVM has a good effect on positive sample judgment and 45-degree face judgment.

Apart from the accuracy, the size of Convolutional Neural Networks (CNN) model is another principal factor for facilitating the deployment of models on memory, power and budget constrained devices. Conventional compression techniques require human expert to setup parameters to explore the design space and iterative based pruning requires heavy training which is sub-optimal and time consuming. Given a CNN model, we propose deep reinforcement learning [8] DQN based automated compression which effectively turned off kernels on each layer by observing its significance. Observing accuracy, compression ratio and convergence rate, proposed DQN model can automatically re- activate the healthiest kernels back to train it again to regain accuracy which greatly ameliorate the model compression quality. Based on experiments on MNIST [3] dataset, our method can compress convolution layers for VGG-like [10] model up to 60% with 0.5% increase in test accuracy within less than a half the number of initial amount of training (speed-up up to 2.5×), state- of-the-art results of dropping 80% of kernels (compressed 86% parameters) with increase in accuracy by 0.14%. Further dropping 84% of kernels (compressed 94% parameters) with the loss of 0.4% accuracy. The first proposed Auto-AEC (Accuracy-Ensured Compression) model can compress the network by preserving original accuracy or increase in accuracy of the model, whereas, the second proposed Auto-CECA (Compression-Ensured Considering the Accuracy) model can compress to the maximum by preserving original accuracy or minimal drop of accuracy. We further analyze effectiveness of kernels on different layers based on how our model explores and exploits in various stages of training.

Occlusions, scale variation and numerous false positives still represent fundamental challenges in pedestrian detection. Intuitively, different sizes of receptive fields and more attention to the visible parts are required for detecting pedestrians with various scales and occlusion levels, respectively. However, these challenges have not been addressed well by existing pedestrian detectors. This paper presents a novel convolutional network, denoted as box guided convolution network (BGCNet), to tackle these challenges simultaneously in a unified framework. In particular, we proposed a box guided convolution (BGC) that can dynamically adjust the sizes of convolution kernels guided by the predicted bounding boxes. In this way, BGCNet provides position-aware receptive fields to address the challenge of large variations of scales. In addition, for the issue of heavy occlusion, the kernel parameters of BGC are spatially localized around the salient and mostly visible key points of a pedestrian, such as the head and foot, to effectively capture high-level semantic features to help detection. Furthermore, a local maximum (LM) loss is introduced to depress false positives and highlight true positives by forcing positives, rather than negatives, as local maximums, without any additional inference burden. We evaluate BGCNet on popular pedestrian detection benchmarks, and achieve the state-of-the-art results, with the significant performance improvement on heavily occluded and small-scale pedestrians.

Non-negative Matrix Factorization (NMF) is a key kernel for unsupervised dimension reduction used in a wide range of applications, including graph mining, recommender systems and natural language processing. Due to the compute-intensive nature of applications that must perform repeated NMF, several parallel implementations have been developed. However, existing parallel NMF algorithms have not addressed data locality optimizations, which are critical for high performance since data movement costs greatly exceed the cost of arithmetic/logic operations on current computer systems. In this paper, we present a novel optimization method for parallel NMF algorithm based on the HALS (Hierarchical Alternating Least Squares) scheme that incorporates algorithmic transformations to enhance data locality. Efficient realizations of the algorithm on multi-core CPUs and GPUs are developed, demonstrating a new Accelerated Locality-Optimized NMF (ALO-NMF) that obtains up to 2.29x lower data movement cost and up to 4.45x speedup over existing state-of-the-art parallel NMF algorithms.

Sparse triangular solver is an important kernel in many computational applications. However, a fast, parallel, sparse triangular solver on a manycore architecture such as GPU has been an open issue in the field for several years. In this paper, we develop a sparse triangular solver that takes advantage of the supernodal structures of the triangular matrices that come from the direct factorization of a sparse matrix. We implemented our solver using Kokkos and Kokkos Kernels such that our solver is portable to different manycore architectures. This has the additional benefit of allowing our triangular solver to use the team-level kernels and take advantage of the hierarchical parallelism available on the GPU. We compare the effects of different scheduling schemes on the performance and also investigate an algorithmic variant called the partitioned inverse. Our performance results on an NVIDIA V100 or P100 GPU demonstrate that our implementation can be 12.4 × or 19.5 × faster than the vendor optimized implementation in NVIDIA’s CuSPARSE library.

We present a valid polarization-based reflection contaminated image synthesis method, which can provide adequate, diverse and authentic training dataset. Meanwhile, we enhance the neural network by introducing the reflection information as guidance and utilizing adaptive convolution kernel size to fuse multi-scale information. We demonstrate that the proposed approach achieves convincing improvements over state of the arts.

Miniaturized fluorescent calcium imaging microscopes are widely used for monitoring the activity of a large population of neurons in freely behaving animals in vivo. Conventional calcium image analyses extract calcium traces by iterative and bulk image processing and they are hard to meet the power and latency requirements for neurofeedback devices. In this paper, we propose the calcium image processing pipeline based on a bit-sparse long short-term memory (LSTM) inference kernel (BLINK) for efficient calcium trace extraction. It largely reduces the power and latency while remaining the trace extraction accuracy. We implemented the customized pipeline on the Ultra96 platform. It can extract calcium traces from up to 1024 cells with sub-ms latency on a single FPGA device. We designed the BLINK circuits in a 28-nm technology. Evaluation shows that the proposed bit-sparse representation can reduce the circuit area by 38.7% and save the power consumption by 38.4% without accuracy loss. The BLINK circuits achieve 410 pJ/inference, which has 6293x and 52.4x gains in energy efficiency compared to the evaluation on the high performance CPU and GPU, respectively.

Neuroevolution (NE), defined as the application of evolution-based training methods to Deep Neural Networks (DNNs), has recently demonstrated encouraging results on a variety of learning tasks. NE is highly parallel and relies on DNN inference as its main computational kernel and therefore can potentially leverage large-scale distributed inference-specific hardware infrastructure in the cloud or edge. We introduce chromosome updates (CU), a novel communication-optimized method for distributing NE computation, and DarwiNN, an open-source, GPU-accelerated distributed NE toolbox based on PyTorch, which implements CU and other algorithms for distributing NE.

While systolic arrays are widely used for dense-matrix operations, they are seldom used for sparse-matrix operations. In this paper, we show how a systolic array of Multiply-and-Accumulate (MAC) units, similar to Google's Tensor Processing Unit (TPU), can be adapted to efficiently handle sparse matrices. TPU-like accelerators are built upon a 2D array of MAC units and have demonstrated high throughput and efficiency for dense matrix multiplication, which is a key kernel in machine learning algorithms and is the target of the TPU. In this work, we employ a co-designed approach of first developing a packing technique to condense a sparse matrix and then propose a systolic array based system, Sparse-TPU, abbreviated to STPU, to accommodate the matrix computations for the packed denser matrix counterparts. To demonstrate the efficacy of our co-designed approach, we evaluate sparse matrix-vector multiplication on a broad set of synthetic and real-world sparse matrices. Experimental results show that STPU delivers 16.08X higher performance while consuming 4.39X and 19.79X lower energy for integer (int8) and floating point (float32) implementations, respectively, over a TPU baseline. Meanwhile, STPU has 12.93% area overhead and an average of 4.14% increase in dynamic energy over the TPU baseline for the float32 implementation.

This study proposes a fast low-order finite element solver for crustal deformation computations by applying Tensor Core, AI-specific hardware on a Volta GPU. Tensor Core can compute large matrix-matrix multiplications rapidly in half precision. We redesign a state-of-the-art solver algorithm so that lower-precision data types can be used and memory access costs can be reduced even when we use small matrices. With the proposed solver, we solved 13 billion degrees-of-freedom two-layered problems that mimicked the Earth's crust and mantle using 36 compute nodes of Summit. In the matrix-vector kernel, we obtained a 4.1-fold speedup over a standard kernel in a single-precision format. Our proposed solver increased the FLOP count of the entire solver; however, we reduced the time-to-solution by 1.7-fold since the Tensor Core provided a high effective performance.

To improve the overall system utilization, Simultaneous Multi-Threading (SMT) has become a norm in clouds. Usually, Hardware threads are viewed and deployed directly as physical cores for attempts to improve resource utilization and system throughput. However, context switches in virtualized systems might incur severe resource waste, which further led to significant performance degradation. Worse, virtualized systems suffer from performance variations since the rescheduled vCPU may affect other hardware threads on the same physical core. In this paper, we perform an in-depth experimental study about how existing system software techniques improves the utilization of SMT Processors in Clouds. Considering the default Linux hypervisor vanilla KVM as the baseline, we evaluated two update-to-date kernel patches IdlePoll and HaltPoll through the combination of 14 real-world workloads. Our results show that mitigating they could significantly mitigate the number of context switches, which further improves the overall system throughput and decreases its latency. Based on our findings, we summarize key lessons from the previous wisdom and then discuss promising directions to be explored in the future.

Motion blur often arises in a single image because of the camera shake, the objects motion and the depth variation. The image deblurring is a challenging task due to its ill-posed nature. To remove these blurriness, the conventional energy optimization based methods always rely on the assumption such that the blur kernel is uniform across the entire image. With the development of the deep neural network, the learning based methods were proposed to tackle with the non-uniform blur cases. In this paper, we propose a U-Net network containing dense blocks for dynamic scene deblurring. By passing the kernel estimation, our model significantly reduces the inference time. The extensive experiments on both synthetic and real blurred images demonstrate that our method outperforms the state-of-the-art blind deblurring algorithms.

Convolutional neural networks (CNNs) have made great success in computer vision tasks. But the computational complexity of CNNs is huge, which makes CNNs run slowly especially when computational resources are limited. In this paper, we propose a scheme based on tensor decomposition to accelerate CNNs. Firstly, Tucker method is used to decompose the convolution kernel into a small core tensor with key information and two factor matrices reflecting the linear relationship in the third dimension and fourth dimension of the convolution kernel respectively. Then CP (CANDECOMP/PARAFAC) method is used to decompose the core tensor into several rank-1 tensors. This scheme can remove the linear redundancy in convolution kernels and greatly speed up CNNs while maintaining the high classification accuracy. The scheme is used to decompose all the convolutional layers in AlexNet, and the accelerated model is trained and tested on ImageNet. The results show that our scheme achieves a whole-model speedup of 4 x with merely a 1.9% increase in top-5 error for AlexNet.

In this paper, we revisit the question of how ML algorithms can be best integrated into existing DBMSs to not only avoid expensive data copies to external ML tools but also to comply with regulatory reasons. The key observation is that database transactions already provide an execution model that allows DBMSs to efficiently mimic the execution model of modern parallel ML algorithms. As a main contribution, this paper presents DB4ML, an in-memory database kernel that allows applications to implement user-defined ML algorithms and efficiently run them inside a DBMS. Thereby, the ML algorithms are implemented using a programming model based on the idea of so called iterative transactions. Our experimental evaluation shows that DB4ML can support user-defined ML algorithms inside a DBMS with the efficiency of modern specialized ML engines. In contrast to DB4ML, these engines not only need to transfer data out of the DBMS but also hardcode the ML algorithms and thus are not extensible.

Vulnerabilities in OS kernel are more severe than those in user space because they allow attackers to access a system with full privileges. Fuzzing is an efficient technique to detect vulnerabilities though little fuzzing efforts aim to kernels. On one hand, by hooking the kernel, passive fuzzing can satisfy the dependencies among system calls but get no feedback, and thus fails to generate test cases for a resulted crash. On the other hand, guided with run-time feedback, active fuzzing can easily reproduce the crash with generated test cases, but cannot find bugs in deeper code path due to lacking of data dependency or control dependency. In this paper, we propose a novel approach for fuzzing kernel which combines passive fuzzing and active fuzzing and therefore gain their advantages. We implement the approach in a prototype called X-AFL which currently aims to test the Android kernel. Preliminary evaluation results show that X-AFL is an effective kernel fuzzer and can indeed find kernel vulnerabilities.

Recent work in the domain of misinformation detection has leveraged rich signals in the text and user identities associated with content on social media. But text can be strategically manipulated and accounts reopened under different aliases, suggesting that these approaches are inherently brittle. In this work, we investigate an alternative modality that is naturally robust: the pattern in which information propagates. Can the veracity of an unverified rumor spreading online be discerned solely on the basis of its pattern of diffusion through the social network? Using graph kernels to extract complex topological information from Twitter cascade structures, we train accurate predictive models that are blind to language, user identities, and time, demonstrating for the first time that such "sanitized" diffusion patterns are highly informative of veracity. Our results indicate that, with proper aggregation, the collective sharing pattern of the crowd may reveal powerful signals of rumor truth or falsehood, even in the early stages of propagation., Published at The Web Conference (WWW) 2020

Sparse matrix-vector multiplication (SpMV) is an important kernel in many scientific, machine-learning, and other compute-intensive applications. Performance characteristics, however, depend on a complex combination of storage format, machine capabilities, and choices in code-generation. A deep understanding of the relative impact of these properties is important in itself, and also to better understanding the performance potential of alternative execution contexts such as web-based scientific computing, where the recent introduction ofWebAssembly offers the potential for low-level, near-native performance within a web browser. In this work we characterize the performance of SpMV operations for different sparse storage formats based on the sparse matrix structure and the machine architecture. We extract structural properties from 2000 real-life sparse matrices to understand their impact on the choice of storage format and also on the performance within those storage formats for both WebAssembly and native C. We extend this with new matrix features based on a "reuse-distance" concept to identify performance bottlenecks, and evaluate the effect of interaction between the matrix structure and hardware characteristics on SpMV performance. Our study provides valuable insights to scientific programmers and library developers to apply best practices and guide future optimization for SpMV in general, and in particular for web-based contexts with abstracted hardware and storage models.

A rootkit is a piece of code that aims to manipulate the computer behaviour without being detected. Rootkits are mainly used to disable kernel self-protection, hide malware presence, provide a covert communication channel between malware and their Command and Control server (C&C), secretly maintain a breach viable. Depending on how rootkits are injected into the target, they can either reside in user or kernel space. Kernel space rootkits are the hardest to detect and prevent because they usually take control of the kernel once compromised. In this paper, we present WpE, Write-protection Enforcement: a hypervisor-backed kernel hardening system. WpE aims to preserve the kernel self-protection active in presence of attacks. WpE is comprised of two software components, one running in hypervisor mode and one running in kernel mode. At OS instantiation, the kernel component will arrange the kernel critical data in a way that facilitate labeling such data as either writable, rarely writable, or read only. The hypervisor component will stop a compromised kernel from altering memory or registers that were flagged for read only protection. In the current stage of implementation, we managed to successfully integrate WpE with KVM and Linux kernel. Evaluation showed that WpE adds acceptable overhead.

Lightweight convolutional neural networks (LW-CNNs) such as MobileNet, ShuffleNet, SqueezeNet, etc., have emerged in the past few years for fast inference on embedded and mobile system. However, lightweight operations limit acceleration potential by GPU due to their memory bounded nature and their parallel mechanisms that are not friendly to SIMD. This calls for more specific accelerators. In this paper, we propose an FPGA-based overlay processor with a corresponding compilation flow for general LW-CNN accelerations, called Light-OPU. Software-hardware co-designed Light-OPU reformulates and decomposes lightweight operations for efficient acceleration. Moreover, our instruction architecture considers sharing of major computation engine between LW operations and conventional convolution operations. This improves the run-time resource efficiency and overall power efficiency. Finally, Light-OPU is software programmable, since loading of compiled codes and kernel weights completes switch of targeted network without FPGA reconfiguration. Our experiments on seven major LW-CNNs show that Light-OPU achieves 5.5x better latency and 3.0x higher power efficiency on average compared with edge GPU NVIDIA Jetson TX2. Furthermore, Light-OPU has 1.3x to 8.4x better power efficiency compared with previous customized FPGA accelerators. To the best of our knowledge, Light-OPU is the first in-depth study on FPGA-based general processor for LW-CNNs acceleration with high performance and power efficiency, which is evaluated using all major LW-CNNs including the newly released MobileNetV3.

Multiresolution filters, analyzing information at different scales, are crucial for many applications in digital image processing. The different space and time complexity at distinct scales in the unique pyramidal structure poses a challenge as well as an opportunity to implementations on modern accelerators such as GPUs with an increasing number of compute units. In this paper, we exploit the potential of concurrent kernel execution in multiresolution filters. As a major contribution, we present a model-based approach for performance analysis of as well single- as multi-stream implementations, combining both application- and architecture-specific knowledge. As a second contribution, the involved transformations and code generators using CUDA streams on Nvidia GPUs have been integrated into a compiler-based approach using an image processing DSL called Hipacc. We then apply our approach to evaluate and compare the achieved performance for four real-world applications on three GPUs. The results show that our method can achieve a geometric mean speedup of up to 2.5 over the original Hipacc implementation without our approach, up to 2.0 over the other state-of-the-art DSL Halide, and up to 1.3 over the recently released programming model CUDA Graph from Nvidia.

Physiological signal which consists of electrocardiogram (EEG) and peripheral signal is becoming increasingly important in affective computing, because of its intimacy with nerves. In this paper, two models are proposed on the basis of Convolution Neural Network (CNN) to process EEG and peripheral signals respectively. Taking the extraction of traditional features into account, the first model is based on two-dimensional Convolutional Neural Network (2D-CNN) using original EEG data, where its kernel is one-dimensional to extract same kinds of features for every channel. In the second model, we apply one-dimensional Convolution Neural Network (1D-CNN) to every channel of peripheral signals and then concatenate results for classification. Experiments have been done to evaluate our models on the MAHNOB-HCI database. As a result, in the three-category model, the classification accuracies in arousal dimension of the two models using CNN are 61.5%, 58.01% and 58%, 56.28% in valence. Compared with the classical methods, the accuracy using CNN is increased by 9.1% in arousal for EEG and 11.81% for peripheral signals, achieving state-of-the-art performance.

As the technology of computer vision become more advanced, it allows us to be able to classify the styles, genres and artists of artworks with the help of computer. However, never can we know how the convolutional neural networks(CNNs) extract and recognize those aesthetic features like objective or subjective elements. We apply two CNNs: VGG19 and ResNet-50 on different artworks. We compare the results from these networks to understand how these two networks work when they recognize the underlying features such as aesthetic feature. The dataset is obtained from "the best art-work on the world". We selected five subsets from this dataset which have the most paintings and belong to five different artists. Meanwhile, these five artists have five different styles. We get 86.72% accuracy on the validation set by VGG19, while we get 82.31% accuracy on the validation set by ResNet-50. We then use many approaches such as kernel visualization, grad-cam heat-map, confusion matrix and style transfer to explore how those two convolutional neural networks extract the underlying features. By analyzing the results we can conclude that the CNNs actually has the ability to extract and learn the underlying features such as aesthetic features. We discover that different CNNs have different tendencies to extract the specific underlying features. Specifically, VGG19 prefers to extract subjective feature but ResNet-50 prefers to learn objective feature.

For the purpose of developing applications for supercomputer Fugaku at an early stage, RIKEN has developed a processor simulator. This simulator is based on the general-purpose processor simulator gem5. It does not simulate the actual hardware of a Fugaku processor. However, we believe that sufficient simulation accuracy can be obtained since it simulates the instruction pipeline of out-of-order execution with cycle-level accuracy along with performing detailed parameter tuning of out-of-order resources. In order to estimate the accurate execution time of a program, it is necessary to simulate with accuracy not only the instruction execution time, but also the access time of the cache memory hierarchy. Therefore, in the RIKEN simulator, we expanded gem5 to match the performance of the cache memory hierarchy to that of a Fugaku processor. In this simulator, we aim to estimate the execution cycles of one node application on a Fugaku processor with accuracy that enables relative evaluation and application tuning. In this paper, we show the details of the implementation of this simulator and verify its accuracy compared with that of a Fugaku processor test chip. In the evaluation of the total 46 kernel benchmarks, it was confirmed that the difference is 13% or less for 85% of the kernels. In the multithreaded execution of Stream Triad benchmark, scalable performance according to the number of threads was confirmed, and achieved over 80% of memory throughput with enough accuracy.

Matrix powers kernel calculates the vectors Akv, for k = 1, 2,..., m and they are the heart of various scientific computations, including communication avoiding iterative solvers. In this paper we propose diamond matrix powers kernel - DMPK, which has the purpose to apply the "diamond tiling" stencil algorithm to general matrices. It can also be considered as an extension of the PA1 and PA2 algorithms, introduced by Demmel et al. Our approach enables us to control the balance between the amount of communication avoidance and redundant computation inherently present in communication avoiding algorithms. We present a proof of concept implementation of the algorithm using MPI routines. The experiments we performed show that the control of the amount of computation and communication is achievable, and with more thorough optimisations, DMPK is a promising alternative to existing MPK approaches.

With the development of artificial intelligence technology, more and more researchers use deep neural network to solve various difficulties in segmenting liver and liver tumors from abdominal CT images. Recently, researchers propose to use cascaded FCNs strategy to segment both liver and liver tumors automatically. The cascaded FCNs trains the first FCN to segment liver region, and uses the output of the first FCN as the input of the second FCN to train the tumor segmentation network. However, in the cascaded FCNs structure, inaccurate segmentation of liver in the first FCN will directly affect the segmentation of tumors at the edge of liver and the second FCN is not conductive to segment small tumors. To address these issues, this paper proposed a modified scheme based on cascaded FCNs: firstly, we modified the first FCN by adopting variable pooling kernel to provide a better liver region for the later segmentation of tumors. Secondly, we replaced the pooling and convolution layer of the second FCN with dilated convolution to improve global feature extraction of small tumors. We extensively evaluated our method on 3D-IRCADb-01 and 2017 LiTS dataset. The DICE coefficient of tumors reached 85.71% and 82.43% respectively, 3.71% and 3.61% higher than the original cascaded FCNs. Experimental results indicate that our method achieved very competitive performance and outperformed other state-of-the-arts on the liver tumor segmentation accuracy.

This paper presents a novel solution for approximations to some large convolution kernels by leveraging a weighted box-filtered image pyramid set. Convolution filters are widely used, but still compute-intensive for real-time rendering when the kernel size is large. Our algorithm approximates the convolution kernels, such as Gaussian and cosine filters, by two phases of down and up sampling on a GPU. The computational complexity only depends on the input image resolution and is independent of the kernel size. Therefore, our method can be applied to nonuniform blurs, irradiance probe generations, and ray-traced glossy global illuminations in real time, and runs in effective and efficient performance.

Future computer platforms will likely be built with a multitude of on-chip and off-chip processing units being potentially of different ISAs, OS-capable, and sharing memory with a form of consistency. Multiple-kernel OSes, from multikernels to single-system image OSes, have been demonstrated to mange such platforms efficiently, but they assume no shared memory between kernels as a founding principle. This position paper proposes a new multiple-kernel OS design, which leverages consistent shared memory across homogeneous and heterogeneous processing units in a machine. Among other benefits, this design enables porting commodity SMP OSes to such future platforms, capitalizing on their shared memory programming model, and extend them to multiple-kernel OSes. Herein we present such design, based on two new software primitives tackling the problem of sharing and data format differences between eventually heterogeneous computing units: typed shared memory and type-morphable executable code. We also describe an initial implementation built around Popcorn Linux for x86 and ARM.

Image segmentation is the basis of medical image processing. The proposed method can segment 3D volume data by supervoxel and kernel FCM algorithm. Firstly, the extended SLIC method is used to divide the 3D image into supervoxels. Then the supervoxels are descripted by statistical feature and segmented by kernel FCM algorithm. The experimental results on a publically available brain MR dataset show the effectiveness and superior performance of the proposed method.

Dynamic voltage and frequency scaling (DVFS) is an important solution to balance performance and energy consumption, and hardware vendors provide management libraries that allow the programmer to change both memory and core frequencies. The possibility to manually set these frequencies is a great opportunity for application tuning, which can focus on the best application-dependent setting. However, this task is not straightforward because of the large set of possible configurations and because of the multi-objective nature of the problem, which minimizes energy consumption and maximizes performance. This paper proposes a method to predict the best core and memory frequency configurations on GPUs for an input OpenCL kernel. Our modeling approach, based on machine learning, first predicts speedup and normalized energy over the default frequency configuration. Then, it combines the two models into a multi-objective one that predicts a Pareto-set of frequency configurations. The approach uses static code features, is built on a set of carefully designed micro-benchmarks, and can predict the best frequency settings of a new kernel without executing it. Test results show that our modeling approach is very accurate on predicting extrema points and Pareto set for ten out of twelve test benchmarks, and discover frequency configurations that dominate the default configuration in either energy or performance.

This article presents an implementation of a distributed autotuning engine developed as part of the Bench-testing OpenN Software Autotuning Infrastructure project. The system is geared towards performance optimization of computational kernels for graphics processing units, and allows for the deployment of vast autotuning sweeps to massively parallel machines. The software implements dynamic work scheduling to distributed-memory resources and takes advantage of multithreading for parallel compilation and dispatches kernel launches to multiple accelerators. This paper lays out the main design principles of the system and discusses the basic mechanics of the initial implementation. Preliminary performance results are presented, encountered challenges are discussed, and the future directions are outlined.

Isolation is a desirable property for applications executing in multi-tenant computing systems. On the performance side, hardware resource isolation via partitioning mechanisms is commonly applied to achieve QoS, a necessary property for many noise-sensitive parallel workloads. Conversely, on the software side, partitioning is used, usually in the form of virtual machines, to provide secure environments with smaller attack surfaces than those present in shared software stacks. In this paper, we identify a further benefit from isolation, one that is currently less appreciated in most parallel computing settings: isolation of system software stacks, including OS kernels, can lead to significant performance benefits through a reduction in variability. To highlight the existing problem in shared software stacks, we first developed a new systematic approach to measure and characterize latent sources of variability in the Linux kernel. Using this approach, we find that hardware VMs are effective substrates for limiting kernel-level interference that otherwise occurs in monolithic kernel systems. Furthermore, by enabling reductions in variability, we find that virtualized environments often have superior worst-case performance characteristics than native or containerized environments. Finally, we demonstrate that due to their isolated software contexts, most virtualized applications consistently outperform their bare-metal counterparts when executing on 64-nodes of a multi-tenant, kernel-intensive cloud system.