Search

With the development of Artificial Intelligence (AI) and Binary neural networks (BNN), the computing efficiency of the computing system is expected to be much better, however, enormous amounts of data processing have caused an intolerable ‘memory wall’ and ‘power wall’ challenge for traditional Von Neumann architectures. Therefore, more advanced Computing-in-memory (CiM) architectures are proposed. The emerging non-volatile memory Spin Transfer Torque Magnetic Random Access Memory (STT-MRAM), with its fast access speed, near-zero leakage power consumption and high density is one of the most competitive carriers for CiM architectures.Due to the inherent read-write asymmetry of STT-MRAM, the read overhead is multiples smaller than the exaggerated write overhead. To better exploit the potential of MRAM, we focus on implementing an CiM based on the reading process.This work introduces the principle of CiM and proposes four basic logic operations (XNOR, XOR, AND and OR) based on STT-MRAM. Furthermore, a delay-based XNOR-popcount (XNOR-PC) operation for BNN based on XNOR operation is proposed. Incorporating the reading characteristics of STT-MRAM and slight modifications to the peripheral circuitry, these operations achieve significant optimisation in terms of performance and energy consumption. Simulation results show the proposed scheme can reduce the latency of XOR, AND, and OR operations at least by 99.3%, 82.2% and 80.2% compared with the existing design. The proposed XNOR-PC operation achieves at least a 59.4% reduction in power consumption and has a high degree of reliability. Also, Monte Carlo simulations prove the feasibility and robustness of the schemes. [ABSTRACT FROM AUTHOR]

Real-time dense SLAM techniques aim to reconstruct the dense three-dimensional geometry of a scene in real time with an RGB or RGB-D sensor. An indoor scene is an important type of working environment for these techniques. The planar prior can be used in this scenario to improve the reconstruction quality, especially for large low-texture regions that commonly occur in an indoor scene. This article fully explores the planar prior in a dense SLAM pipeline. First, we propose a novel plane detection and segmentation method that runs at 200 Hz on a modern graphics processing unit. Our algorithm for constructing global plane constraints is very efficient; hence, we use it in the process of each input frame for the camera pose estimation while maintaining the real-time performance. Second, we propose herein a plane-based map representation that greatly reduces the memory footprint of plane regions while keeping the geometric details on planes. The experiments reveal that our system yields superior reconstruction results with planar information running at more than 30 fps. Aside from speed and storage improvements, our technique also handles the low-texture problem in plane regions. [ABSTRACT FROM AUTHOR]

Image demoireing is a multi-faceted image restoration task involving both moire pattern removal and color restoration. In this paper, we raise a general degradation model to describe an image contaminated by moire patterns, and propose a novel multi-scale bandpass convolutional neural network (MBCNN) for single image demoireing. For moire pattern removal, we propose a multi-block-size learnable bandpass filters (M-LBFs), based on a block-wise frequency domain transform, to learn the frequency domain priors of moire patterns. We also introduce a new loss function named Dilated Advanced Sobel loss (D-ASL) to better sense the frequency information. For color restoration, we propose a two-step tone mapping strategy, which first applies a global tone mapping to correct for a global color shift, and then performs local fine tuning of the color per pixel. To determine the most appropriate frequency domain transform, we investigate several transforms including DCT, DFT, DWT, learnable non-linear transform and learnable orthogonal transform. We finally adopt the DCT. Our basic model won the AIM2019 demoireing challenge. Experimental results on three public datasets show that our method outperforms state-of-the-art methods by a large margin. [ABSTRACT FROM AUTHOR]

Human brain is a complex yet economically organized system, where a small portion of critical hub regions support the majority of brain functions. The identification of common hub nodes in a population of networks is often simplified as a voting procedure on the set of identified hub nodes across individual brain networks, which ignores the intrinsic data geometry and partially lacks the reproducible findings in neuroscience. Hence, we propose a first-ever group-wise hub identification method to identify hub nodes that are common across a population of individual brain networks. Specifically, the backbone of our method is to learn common graph embedding that can represent the majority of local topological profiles. By requiring orthogonality among the graph embedding vectors, each graph embedding as a data element is residing on the Grassmannian manifold. We present a novel Grassmannian manifold optimization scheme that allows us to find the common graph embeddings, which not only identify the most reliable hub nodes in each network but also yield a population-based common hub node map. Results of the accuracy and replicability on both synthetic and real network data show that the proposed manifold learning approach outperforms all hub identification methods employed in this evaluation. [ABSTRACT FROM AUTHOR]

Conventional direction-of-arrival (DOA) estimation algorithms are sensitive to array imperfections and outliers, making it challenging to realize accurate estimates in real applications. Facing the challenge, we propose a robust variational Bayesian inference based DOA estimation algorithm using the linear sparse array in this paper, where accurate DOA estimation with increased number of degrees of freedom (DOFs) is realized. With the mixture of von Mises model, a prior-grid scheme is further proposed to alleviate the computational burden introduced by the Bayesian variational inference framework. Since there is no restriction on the prior knowledge of the number of sources, the proposed algorithm is friendly to actual scenarios. Simulation results demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

The core purpose of deep metric learning is to construct an embedding space, where objects belonging to the same class are gathered together and the ones from different classes are pushed apart. Most existing approaches typically insist to inter-class characteristics, e.g., class-level information or instance-level similarity, to obtain semantic relevance of data points and get a large margin between different classes in the embedding space. However, the intra-class characteristics, e.g., local manifold structure or relative relationship within the same class, are usually overlooked in the learning process. Hence the output embeddings have limitation in retrieving a good ranking result if existing multiple positive samples. And the local data structure of embedding space cannot be fully exploited since lack of relative ranking information. As a result, the model is prone to overfitting on a train set and get low generalization on the test set (unseen classes) when losing sight of intra-class variance. This paper presents a novel self-supervised synthesis ranking auxiliary framework, which captures intra-class characteristics as well as inter-class characteristics for better metric learning. Our method designs a synthetic samples generation of polar coordinates to generate measurable intra-class variance with different strength and diversity in the latent space, which can simulate the various local structure change of intra-class in the initial data domain. And then formulates a self-supervised learning procedure to fully exploit this property and preserve it in the embedding space. As a result, the learned embedding space not only keeps inter-class discrimination but also owns subtle intra-class diversity, leading to better global and local embedding structures. Extensive experiments on five benchmarks show that our method significantly improves and outperforms the state-of-the-art methods on the performances of both retrieval and ranking by 2%-4% (personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending an email to pubs-permissions@ieee.org). [ABSTRACT FROM AUTHOR]

Constant bit rate (CBR) videos are widely used in streaming playback applications. However, the image quality of the CBR video is often unstable, especially for scenes with large motion. To this end, we design a new model to represent the distortion of High Efficiency Video Coding (HEVC) constant bit rate video, and propose a neural network for a constant bit rate video quality enhancement (CBREN). We propose a dual-domain restoration module (DRM) to jointly learn the prior knowledge in the pixel domain and the frequency domain. To address the degradation resulting from compression, we propose a two-step quantization degradation estimation strategy. The Inverse DCT (IDCT) Translation Unit (ITU) is used to constrain the quantization table of the constant bit rate video to a suitable range, and the Dynamic Alpha Unit (DAU) is used to fine-tune the quantization table according to the content of each frame. In order to effectively reduce the block distortion of different sizes produced in the compression process, we adopt a multi-scale network. Extensive experiments show that our approach can greatly enhance the quality of CBR compressed video. Moreover, our method can also be applied to constant quantization parameter (CQP) video enhancement tasks, and is certainly superior to existing methods. [ABSTRACT FROM AUTHOR]

Unsupervised 3D model analysis has attracted tremendous attentions with the increasing growth of 3D model data and the extensive human annotations. Many effective methods have been designed to address the 3D model analysis with labeled information, while rare methods devote to unsupervised deep learning due to the difficulty of mining reliable information. In this paper, we propose a novel unsupervised deep learning method named joint local correlation and global contextual information (LCGC) for 3D model retrieval and classification, which mines the reliable triplet set and uses triplet loss to optimize the deep neural network. Our method proposes two schemes: 1) Local self-correlation information learning, which adopts the intra and inter information to construct the view-level triplet set. 2) Global neighbor contextual information learning, which employs the neighbor contextual information to explore the reliable relations among 3D models and construct the model-level triplet set. The above schemes encourage that the selected triple set can been used to improve the discrimination of learned features. Extensive evaluations on two large-scale datasets, ModelNet40 and ShapeNet55, have demonstrated the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

Cross-view geo-localization is to spot images of the same geographic target from different platforms, e.g., drone-view cameras and satellites. It is challenging in the large visual appearance changes caused by extreme viewpoint variations. Existing methods usually concentrate on mining the fine-grained feature of the geographic target in the image center, but underestimate the contextual information in neighbor areas. In this work, we argue that neighbor areas can be leveraged as auxiliary information, enriching discriminative clues for geo-localization. Specifically, we introduce a simple and effective deep neural network, called Local Pattern Network (LPN), to take advantage of contextual information in an end-to-end manner. Without using extra part estimators, LPN adopts a square-ring feature partition strategy, which provides the attention according to the distance to the image center. It eases the part matching and enables the part-wise representation learning. Owing to the square-ring partition design, the proposed LPN has good scalability to rotation variations and achieves competitive results on three prevailing benchmarks, i.e., University-1652, CVUSA and CVACT. Besides, we also show the proposed LPN can be easily embedded into other frameworks to further boost performance. [ABSTRACT FROM AUTHOR]

The optical remote sensing images (RSIs) show various spatial resolutions and cluttered background, where salient objects with different scales, types, and orientations are presented in diverse RSI scenes. Therefore, it is inappropriate to directly extend cutting-edge saliency detection methods for conventional RGB images to optical RSIs. Besides, the existing saliency models targeting RSIs often render imperfect saliency maps, where some of them are with coarse boundary details. To solve this problem, this article attempts to introduce the edge information to precisely detect salient objects in RSIs. Accordingly, we propose an edge-aware multiscale feature integration network (EMFI-Net) for salient object detection by conducting multiscale feature integration under the explicit and implicit assistance of salient edge cues. Specifically, our network contains two parts including the encoder and decoder. First, the encoder extracts multiscale deep features from three RSIs with different resolutions, where the high-level deep semantic features from three RSIs are integrated using a cascaded feature fusion module. Second, the encoder explicitly enriches the multiscale deep features by integrating the salient edge cues extracted by a salient edge extraction module. Meanwhile, we also implicitly deploy an edge-aware constraint to the supervision of the saliency map prediction by introducing a hybrid loss function. Finally, the decoder integrates the enriched multiscale deep features in a coarse-to-fine way, yielding a high-quality saliency map. The experiments conducted on two public optical RSI datasets clearly prove the effectiveness and superiority of the proposed EMFI-Net against the state-of-the-art saliency models. [ABSTRACT FROM AUTHOR]

This brief proposes a high spurious-free dynamic range (SFDR) pulse output direct digital frequency synthesizer (PDDS) with low complexity and low power consumption. Independent and uniformly distributed (IUD) high-pass-shaped dither is added to the phase accumulator output, resulting in a wideband spurious-free and low close-in noise floor. The power efficiency and speed are increased by reducing the sampling frequency of the ${m}$ -sequence generator and the high-pass filter (HPF). The SFDR improves by 29 dB as a result of the HPF, which is confirmed with a field-programmable gate array (FPGA) implementation. The application specified integrated circuit (ASIC) occupies $1168~{\mu \text {m}^{2}}$ on nangate 45-nm CMOS process and consumes 80.2 and $398~{\mu }\text{W}$ from a 1.2-V supply with the dither generator running at ${({1}/{4})f_{\text {clk}}}$ and ${f_{\text {clk}}}$ (${f_{\text {clk}}}=2$ GHz), respectively. [ABSTRACT FROM AUTHOR]

Effective 3D shape retrieval and recognition are challenging but important tasks in computer vision research field, which have attracted much attention in recent decades. Although recent progress has shown significant improvement of deep learning methods on 3D shape retrieval and recognition performance, it is still under investigated of how to jointly learn an optimal representation of 3D shapes considering their relationships. To tackle this issue, we propose a multi-scale representation learning method on hypergraph for 3D shape retrieval and recognition, called multi-scale hypergraph neural network (MHGNN). In this method, the correlation among 3D shapes is formulated in a hypergraph and a hypergraph convolution process is conducted to learn the representations. Here, multiple representations can be obtained through different convolution layers, leading to multi-scale representations of 3D shapes. A fusion module is then introduced to combine these representations for 3D shape retrieval and recognition. The main advantages of our method lie in 1) the high-order correlation among 3D shapes can be investigated in the framework and 2) the joint multi-scale representation can be more robust for comparison. Comparisons with state-of-the-art methods on the public ModelNet40 dataset demonstrate remarkable performance improvement of our proposed method on the 3D shape retrieval task. Meanwhile, experiments on recognition tasks also show better results of our proposed method, which indicate the superiority of our method on learning better representation for retrieval and recognition. [ABSTRACT FROM AUTHOR]

This brief proposes adaptive dither windows to calibrate bit weights in pipelined analog-to-digital converters (ADCs). It exploits the comparator meta-stability nature to construct dither windows, which avoids doubling the number of the comparators. The dither window size as defined by the sub-ADC comparator metastability region is tightly controlled and adapted to process-voltage-temperature (PVT) variations by adjusting the digitally controlled delay lines (DCDLs) against a capacitor-ratio reference. Besides, it cancels the voltage swing increment due to the dither windows, which relieves the design of the residue amplifier and saves the over-range margin of the stage. The proposed calibration technique is demonstrated by using a 12-bit pipelined ADC in a 40 nm CMOS process, and simulation results show that the window size is adjusted adaptively under PVT corners while the voltage swing increment is cancelled. The spurious-free-dynamic-range (SFDR) and the signal-to-noise-and-distortion-ratio (SNDR) are improved from 61.8 dB and 53 dB to 92 dB and 73.6 dB, respectively. [ABSTRACT FROM AUTHOR]

Hashing is an efficient method for nearest neighbor search in large-scale data space by embedding high-dimensional feature descriptors into a similarity preserving Hamming space with a low dimension. However, large-scale high-speed retrieval through binary code has a certain degree of reduction in retrieval accuracy compared to traditional retrieval methods. We have noticed that multi-view methods can well preserve the diverse characteristics of data. Therefore, we try to introduce the multi-view deep neural network into the hash learning field, and design an efficient and innovative retrieval model, which has achieved a significant improvement in retrieval performance. In this paper, we propose a supervised multi-view hash model which can enhance the multi-view information through neural networks. This is a completely new hash learning method that combines multi-view and deep learning methods. The proposed method utilizes an effective view stability evaluation method to actively explore the relationship among views, which will affect the optimization direction of the entire network. We have also designed a variety of multi-data fusion methods in the Hamming space to preserve the advantages of both convolution and multi-view. In order to avoid excessive computing resources on the enhancement procedure during retrieval, we set up a separate structure called memory network which participates in training together. The proposed method is systematically evaluated on the CIFAR-10, NUS-WIDE and MS-COCO datasets, and the results show that our method significantly outperforms the state-of-the-art single-view and multi-view hashing methods. [ABSTRACT FROM AUTHOR]

This brief presents a novel multistage noise–shaping (MASH) delta-sigma modulator that has low spurs for all input values by the proposed adaptive length extension technique. Based on the feedforward structure, the accumulators in the proposed MASH are fully utilized to realize an adaptive feedforward gain to compensate for the sequence length loss due to input. The proposed MASH can extend the output sequence length to the maximum under any input, thus to reduce spurs. Simulation results show that the quantization noise of the proposed MASH is distributed widely with no noticeable spurs. Synthesized with 180nm CMOS technology, the increase in hardware cost of the proposed MASH is slight. [ABSTRACT FROM AUTHOR]

3D layout is crucial for scene understanding and reconstruction, and very useful in applications like real estate and furniture design. In this paper, we propose a fully automatic solution to estimate 3D layout of an indoor scene from a single 2D image. Our technique contains two key components. Firstly, we train a neural network that directly estimates room structure lines from the input image. Secondly, we propose a novel technique to automatically identify the layout topology of an input image, followed by a nonlinear optimization with equality constraints to estimate the final 3D layout of a scene. Based on our knowledge, this is the first fully automatic technique to achieve single image-based 3D layout estimation of an indoor scene. We evaluate our method on the public datasets $LSUN$ , $Hedau$ and $3DGP$ and the results show that the proposed method achieves accurate 3D layout reconstruction on various images with different layout topologies. [ABSTRACT FROM AUTHOR]

Image captioning, a challenging task where the machine automatically describes an image with natural language, has drawn significant attention in recent years. Despite the remarkable improvements of recent approaches, however, these methods are built upon a large set of training image-sentence pairs. The expensive labor efforts hence limit the captioning model to describe the wider world. In this paper, we present a novel network structure, Cascaded Revision Network, which aims at relieving the problem by equipping the model with out-of-domain knowledge. CRN first tries its best to describe an image using the existing vocabulary from in-domain knowledge. Due to the lack of out-of-domain knowledge, the caption may be inaccurate or include ambiguous words for the image with unknown (novel) objects. We propose to re-edit the primary captioning sentence by a series of cascaded operations. We introduce a perplexity predictor to find out which words are most likely to be inaccurate given the input image. Thereafter, we utilize external knowledge from a pretrained object detection model and select more accurate words from detection results by the visual matching module. In the last step, we design a semantic matching module to ensure that the novel object is fit in the right position. By this novel cascaded captioning-revising mechanism, CRN can accurately describe images with unseen objects. We validate the proposed method with state-of-the-art performance on the held-out MSCOCO dataset as well as scale to ImageNet, demonstrating the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Deeper and wider convolutional neural networks (CNNs) achieve superior performance but bring expensive computation cost. Accelerating such overparameterized neural network has received increased attention. A typical pruning algorithm is a three-stage pipeline, i.e., training, pruning, and retraining. Prevailing approaches fix the pruned filters to zero during retraining and, thus, significantly reduce the optimization space. Besides, they directly prune a large number of filters at first, which would cause unrecoverable information loss. To solve these problems, we propose an asymptotic soft filter pruning (ASFP) method to accelerate the inference procedure of the deep neural networks. First, we update the pruned filters during the retraining stage. As a result, the optimization space of the pruned model would not be reduced but be the same as that of the original model. In this way, the model has enough capacity to learn from the training data. Second, we prune the network asymptotically. We prune few filters at first and asymptotically prune more filters during the training procedure. With asymptotic pruning, the information of the training set would be gradually concentrated in the remaining filters, so the subsequent training and pruning process would be stable. The experiments show the effectiveness of our ASFP on image classification benchmarks. Notably, on ILSVRC-2012, our ASFP reduces more than 40% FLOPs on ResNet-50 with only 0.14% top-5 accuracy degradation, which is higher than the soft filter pruning by 8%. [ABSTRACT FROM AUTHOR]

Three-dimensional multi-modal data are used to represent 3D objects in the real world in different ways. Features separately extracted from multimodality data are often poorly correlated. Recent solutions leveraging the attention mechanism to learn a joint-network for the fusion of multimodality features have weak generalization capability. In this paper, we propose a hamming embedding sensitivity network to address the problem of effectively fusing multimodality features. The proposed network called HamNet is the first end-to-end framework with the capacity to theoretically integrate data from all modalities with a unified architecture for 3D shape representation, which can be used for 3D shape retrieval and recognition. HamNet uses the feature concealment module to achieve effective deep feature fusion. The basic idea of the concealment module is to re-weight the features from each modality at an early stage with the hamming embedding of these modalities. The hamming embedding also provides an effective solution for fast retrieval tasks on a large scale dataset. We have evaluated the proposed method on the large-scale ModelNet40 dataset for the tasks of 3D shape classification, single modality and cross-modality retrieval. Comprehensive experiments and comparisons with state-of-the-art methods demonstrate that the proposed approach can achieve superior performance. [ABSTRACT FROM AUTHOR]

Correlation filter (CF) is a critical technique to improve accuracy and speed in the field of visual object tracking. Despite being studied extensively, most existing CF methods suffer from failing to make the most of the inherent spatial-temporal prior of videos. To address this limitation, as consecutive frames are eminently resemble in most videos, we investigate a novel scheme to predict targets’ future state by exploiting previous observations. Specifically, in this paper, we propose a prediction based CF tracking framework by learning the spatial-temporal similarity of consecutive frames for sample managing, template regularization, and training response pre-weighting. We model the learning problem theoretically as a novel objective and provide effective optimization algorithms to solve the learning task. In addition, we implement two CF trackers with different features. Extensive experiments are conducted on three popular benchmarks to validate our scheme. The encouraging results demonstrate that the proposed scheme can significantly boost the accuracy of CF tracking, and the two trackers achieve competitive performances against state-of-the-art trackers. We finally present a comprehensive analysis on the efficacy of our proposed method and the efficiency of our trackers to facilitate real-world visual tracking applications. [ABSTRACT FROM AUTHOR]

Most person re-identification (re-ID) approaches are based on supervised learning, which requires manually annotated data. However, it is not only resource-intensive to acquire identity annotation but also impractical for large-scale data. To relieve this problem, we propose a cross-camera unsupervised approach that makes use of unsupervised style-transferred images to jointly optimize a convolutional neural network (CNN) and the relationship among the individual samples for person re-ID. Our algorithm considers two fundamental facts in the re-ID task, i.e., variance across diverse cameras and similarity within the same identity. In this paper, we propose an iterative framework which overcomes the camera variance and achieves across-camera similarity exploration. Specifically, we apply an unsupervised style transfer model to generate style-transferred training images with different camera styles. Then we iteratively exploit the similarity within the same identity from both the original and the style-transferred data. We start with considering each training image as a different class to initialize the Convolutional Neural Network (CNN) model. Then we measure the similarity and gradually group similar samples into one class, which increases similarity within each identity. We also introduce a diversity regularization term in the clustering to balance the cluster distribution. The experimental results demonstrate that our algorithm is not only superior to state-of-the-art unsupervised re-ID approaches, but also performs favorably compared with other competing unsupervised domain adaptation methods (UDA) and semi-supervised learning methods. [ABSTRACT FROM AUTHOR]

This brief presents a two stage open loop fractional-N frequency synthesizer using proposed injection locking phase modulator for low power applications. A novel low power injection locking oscillator (ILO)-based digital phase modulator (DPM) is proposed in this brief. The maximum differential nonlinearity (DNL) of proposed DPM is smaller than 0.3 degree in the whole range with 1.6 mW power consumption. An integer-N injection locking phase locked loop (ILPLL) is designed to generate the radio frequency signals for the second stage open loop frequency synthesizer. With an injection strength adjustment unit, the open loop fractional-N frequency synthesizer employing proposed DPM achieved excellent in band phase noise and instantaneous frequency switching with low power consumption. This frequency synthesizer is fabricated on 28-nm CMOS technology and its active area is 0.08 mm2. The output frequency range is from 320 MHz to 520 MHz and the power dissipation is 2.7 mW from 0.9 V supply in fractional-N mode. The measured RMS jitter integrated phase noise from 10 KHz to 40 MHz of 440 MHz and 440.4 MHz carrier frequency are 3.8 ps and 17.6 ps, respectively. The corresponding figure of merits (FoM) are −231 dB and −211 dB, respectively. [ABSTRACT FROM AUTHOR]